Connect LiteLLM | Faustine Docs

With LiteLLM as the model provider, Faustine sends every model request to a LiteLLM gateway that you run. The gateway decides which upstream serves each model. It can also serve embeddings with the same credentials.

Before you begin

A LiteLLM gateway reachable on the public internet over https.
An API key for the gateway. See Virtual keys in the LiteLLM documentation.
A route in the gateway's configuration for each Claude model Faustine requires, and the name of each route. See the LiteLLM configuration reference.

Faustine calls the gateway's Anthropic Messages endpoint, /v1/messages, for models, and /v1/embeddings for embeddings.

Enter the credentials

In the portal, go to Settings > Model Provider and select Configure a provider.
In Provider, select LiteLLM.
In Base URL, enter the gateway's address, such as https://litellm.example.com.
In API key, enter the gateway key.
Select Continue.

Base URL has to use https and a public hostname. An IP address, localhost or an internal name is refused.

Map the models

In Model names on your provider:

For each model under Required for analysis & code generation, enter the route alias from your LiteLLM configuration.
To offer more models for chat, select Add a chat model… and enter its route alias.
Select Test connection. Each row shows whether the route answered.
Select Continue when every required model is reachable.

Faustine cannot derive route aliases, so every required row needs a value.

Set embeddings and save

In Embeddings, Use my LiteLLM gateway is selected. Enter in Model an embeddings model your gateway routes. The default is text-embedding-3-small. To use OpenAI directly, select Use a separate OpenAI key.
Select Continue.
In Review & save, select Test connection, then Save & activate.

The tab shows the LiteLLM card with an Active badge and a Base URL row.

With Use my LiteLLM gateway, Faustine stores no second key. It reuses the gateway's key and calls the gateway's /v1/embeddings. The model has to return vectors of 1536 dimensions.

What the validation checks

Check	How
Credentials	A request to the gateway's `/v1/models` with the key. A 401 or 403 means the key was rejected, and the configuration is not saved.
Each model	One real request of one token to the route in its row.
Embeddings	One embeddings request to the model you entered.

Troubleshooting

Symptom	Cause	Fix
`litellm.baseUrl must be a public hostname`	The URL is internal.	Expose the gateway on a public hostname with `https`.
`the LiteLLM proxy rejected the API key`	The key is wrong or disabled.	Create a key on the gateway and enter it.
A model shows Not found with `The gateway does not route this model name`	The value is not a route in your configuration.	Enter the route alias exactly as configured.
`The embeddings endpoint does not serve "<model>".`	The gateway has no route for that embeddings model.	Add the route, or enter a model the gateway serves.

Next steps

Model providers: status, test and the other providers.
Choose an embeddings provider: the embeddings options on LiteLLM.
Choose the chat model: pick among the models you mapped.