# Connect LiteLLM

With LiteLLM as the [model provider](https://docs.faustine.ai/workspace/model-providers/), Faustine sends every model request to a LiteLLM gateway that you run. The gateway decides which upstream serves each model. It can also serve embeddings with the same credentials.

## Before you begin

- A LiteLLM gateway reachable on the public internet over `https`.
- An API key for the gateway. See [Virtual keys](https://docs.litellm.ai/docs/proxy/virtual_keys) in the LiteLLM documentation.
- A route in the gateway's configuration for each Claude model Faustine requires, and the name of each route. See [the LiteLLM configuration reference](https://docs.litellm.ai/docs/proxy/configs).

Faustine calls the gateway's Anthropic Messages endpoint, `/v1/messages`, for models, and `/v1/embeddings` for embeddings.

## Enter the credentials

1. In the portal, go to **Settings** > [**Model Provider**](https://auth.faustine.ai/go?to=%2Fsettings%3Ftab%3Dprovider) and select **Configure a provider**.
2. In **Provider**, select **LiteLLM**.
3. In **Base URL**, enter the gateway's address, such as `https://litellm.example.com`.
4. In **API key**, enter the gateway key.
5. Select **Continue**.

**Base URL** has to use `https` and a public hostname. An IP address, `localhost` or an internal name is refused.

## Map the models

In **Model names on your provider**:

1. For each model under **Required for analysis & code generation**, enter the route alias from your LiteLLM configuration.
2. To offer more models for chat, select **Add a chat model…** and enter its route alias.
3. Select **Test connection**. Each row shows whether the route answered.
4. Select **Continue** when every required model is reachable.

Faustine cannot derive route aliases, so every required row needs a value.

## Set embeddings and save

1. In **Embeddings**, **Use my LiteLLM gateway** is selected. Enter in **Model** an embeddings model your gateway routes. The default is `text-embedding-3-small`. To use OpenAI directly, select **Use a separate OpenAI key**.
2. Select **Continue**.
3. In **Review & save**, select **Test connection**, then **Save & activate**.

The tab shows the **LiteLLM** card with an **Active** badge and a **Base URL** row.

With **Use my LiteLLM gateway**, Faustine stores no second key. It reuses the gateway's key and calls the gateway's `/v1/embeddings`. The model has to return vectors of 1536 dimensions.

## What the validation checks

| Check | How |
| --- | --- |
| Credentials | A request to the gateway's `/v1/models` with the key. A 401 or 403 means the key was rejected, and the configuration is not saved. |
| Each model | One real request of one token to the route in its row. |
| Embeddings | One embeddings request to the model you entered. |

## Troubleshooting

| Symptom | Cause | Fix |
| --- | --- | --- |
| `litellm.baseUrl must be a public hostname` | The URL is internal. | Expose the gateway on a public hostname with `https`. |
| `the LiteLLM proxy rejected the API key` | The key is wrong or disabled. | Create a key on the gateway and enter it. |
| A model shows **Not found** with `The gateway does not route this model name` | The value is not a route in your configuration. | Enter the route alias exactly as configured. |
| `The embeddings endpoint does not serve "<model>".` | The gateway has no route for that embeddings model. | Add the route, or enter a model the gateway serves. |

## Next steps

- [Model providers](https://docs.faustine.ai/workspace/model-providers/): status, test and the other providers.
- [Choose an embeddings provider](https://docs.faustine.ai/workspace/embeddings/): the embeddings options on LiteLLM.
- [Choose the chat model](https://docs.faustine.ai/web-agents/chat-model/): pick among the models you mapped.