LiteLLM
Connect your LiteLLM proxy so Ace reads model errors and spend during investigations, finds wasted model spend, and proposes fallback and budget changes for approval.
Ace reads your LiteLLM proxy during an incident for per-model calls, failures, error classes, tokens, spend and latency. It also scans the proxy for duplicate and wasted model spend.
| Category | LLM gateway |
| Ace reads | Spend logs (metadata only), daily spend, model deployments |
| Ace can change | Model fallbacks, and a virtual key's or team's budget and rate limits, each after approval |
| Opens incidents | No (evidence only) |
| Setup | Settings form |
What Ace reads
- Calls by model in the incident window: calls, failures, error classes, tokens, spend and average latency.
- The latest failed calls, with error class and a short error message.
- Daily spend across the proxy.
From spend logs Ace keeps metadata only: model, spend, tokens, times, user, key hash and alias, cache key and hit, status, and error class and message. It never holds prompts or responses.
Ace adds the model deployments behind the proxy to the System Map, with their keys masked by LiteLLM.
Cost findings
Ace reports two kinds of waste from your spend logs and shows how it calculated each one.
- Repeated identical LLM requests: many calls that repeat the same request. Ace can't fix this in your application, so turn on response caching in LiteLLM or stop the calling service from retrying completed work.
- Spend on failed LLM calls: calls the provider charged for that returned an error.
What Ace can change
Ace proposes these actions once an admin turns on Allow Ace to run approved actions for the connection. Each one needs approval by default, and earned autonomy can let the fallback action run under an Automatic policy on a workload and environment you confirm. See Actions and approvals.
| Action | What it does | Approval | Undo |
|---|---|---|---|
| LiteLLM model fallback | Sets or removes one model's fallbacks, so a failing model's requests go to other models. Ace refuses unknown models, a model as its own fallback and duplicates. It also refuses to apply if the fallbacks changed since approval. | Needs approval, can earn Automatic | Yes. Restores the fallbacks it replaced. |
| LiteLLM budget and limits | Changes a virtual key's or team's max budget and its TPM and RPM limits. Ace names keys by their hashed token, never the secret key. LiteLLM records the change as made by Ace. | Always needs approval | Yes. Restores the values it replaced. |
Both actions need the Admin role in LiteLLM. Fallback changes also need STORE_MODEL_IN_DB=True on the proxy and no router fallbacks in config.yaml, or the proxy refuses them.
What opens an incident
Nothing. LiteLLM sends Ace no alerts, and Ace doesn't poll it for failures.
To use LiteLLM as evidence, turn it on under Investigation sources for each workload that calls it.
Before you start
- The admin role in Ace. See Roles and permissions.
- Proxy Admin in LiteLLM, to create Ace's user and key.
- A proxy on HTTPS that is reachable from the public internet, because Ace doesn't connect to private or internal network addresses.
Credentials and permissions
Ace uses one virtual key, owned by a LiteLLM user you create for Ace. Keep the key type at Full Access, because AI APIs keys can't read spend logs.
| Credential | Where to create it | Role | Why Ace needs it |
|---|---|---|---|
| Admin key (needed) | Virtual Keys in the LiteLLM UI, owned by Ace's user | Ace's user has Global Proxy Role Admin (View Only) (proxy_admin_viewer) | Read spend logs, daily spend and model deployments. |
| Same key (only for actions) | Same | Ace's user has Global Proxy Role Admin (proxy_admin) | Change fallbacks, budgets and rate limits after approval. |
Set up
1. Create Ace's user and key
- As a Proxy Admin, open Internal Users > Invite User. Enter an email for Ace and set Global Proxy Role to Admin (View Only), or Admin for approved changes.
- In Ace, open Settings > Data sources and choose LiteLLM. Enter your Proxy URL, then select Create key next to Admin key.
- LiteLLM opens Virtual Keys > Create New Key with Owned By set to Another User, Key Name set to Ace and Key Type set to Full Access. Select Ace's user.
- Optionally set an expiry under Key Lifecycle. Select Create Key and copy the key.
Proxies older than June 2026 open /ui/?page=api-keys without these presets, so set the same values by hand.
2. Connect LiteLLM in Ace
- Fill in the form:
- Proxy URL: your proxy's HTTPS URL, for example
https://litellm.example.com. - Admin key: the key you copied.
- Proxy URL: your proxy's HTTPS URL, for example
- Select Connect LiteLLM. Ace saves only if the proxy is ready and the key can read spend logs.
3. Optional: allow actions
- On the connected LiteLLM page, turn on Allow Ace to run approved actions.
- For automatic fallback changes, select a workload, enter its environment (default
production), and select Confirm.
Confirm it's working
- The connection shows Connected once the readiness and spend-log checks pass. A key without the viewer role fails with "The key is missing the proxy_admin_viewer role."
- LiteLLM is on under Investigation sources for each workload that calls it.
Disconnect
- In Ace, open Settings > Data sources > LiteLLM and select Disconnect. Ace set nothing up in LiteLLM, so there is nothing else to remove there.
- In LiteLLM, delete Ace's virtual key or Ace's user.