# LiteLLM

> Connect your LiteLLM proxy so Ace reads model errors and spend during investigations, finds wasted model spend, and proposes fallback and budget changes for approval.

Source: https://tryace.dev/docs/integrations/litellm

Ace reads your LiteLLM proxy during an incident for per-model calls, failures, error classes, tokens, spend and latency. It also scans the proxy for duplicate and wasted model spend.

| | |
| --- | --- |
| Category | LLM gateway |
| Ace reads | Spend logs (metadata only), daily spend, model deployments |
| Ace can change | Model fallbacks, and a virtual key's or team's budget and rate limits, each after approval |
| Opens incidents | No (evidence only) |
| Setup | Settings form |

## What Ace reads

- Calls by model in the incident window: calls, failures, error classes, tokens, spend and average latency.
- The latest failed calls, with error class and a short error message.
- Daily spend across the proxy.

From spend logs Ace keeps metadata only: model, spend, tokens, times, user, key hash and alias, cache key and hit, status, and error class and message. It never holds prompts or responses.

Ace adds the model deployments behind the proxy to the System Map, with their keys masked by LiteLLM.

### Cost findings

Ace reports two kinds of waste from your spend logs and shows how it calculated each one.

- **Repeated identical LLM requests**: many calls that repeat the same request. Ace can't fix this in your application, so turn on response caching in LiteLLM or stop the calling service from retrying completed work.
- **Spend on failed LLM calls**: calls the provider charged for that returned an error.

## What Ace can change

Ace proposes these actions once an admin turns on **Allow Ace to run approved actions** for the connection. Each one needs approval by default, and earned autonomy can let the fallback action run under an Automatic policy on a workload and environment you confirm. See [Actions and approvals](https://tryace.dev/docs/concepts/actions-and-approvals).

| Action | What it does | Approval | Undo |
| --- | --- | --- | --- |
| LiteLLM model fallback | Sets or removes one model's fallbacks, so a failing model's requests go to other models. Ace refuses unknown models, a model as its own fallback and duplicates. It also refuses to apply if the fallbacks changed since approval. | Needs approval, can earn Automatic | Yes. Restores the fallbacks it replaced. |
| LiteLLM budget and limits | Changes a virtual key's or team's max budget and its TPM and RPM limits. Ace names keys by their hashed token, never the secret key. LiteLLM records the change as made by Ace. | Always needs approval | Yes. Restores the values it replaced. |

Both actions need the Admin role in LiteLLM. Fallback changes also need `STORE_MODEL_IN_DB=True` on the proxy and no router fallbacks in `config.yaml`, or the proxy refuses them.

## What opens an incident

Nothing. LiteLLM sends Ace no alerts, and Ace doesn't poll it for failures.

To use LiteLLM as evidence, turn it on under **Investigation sources** for each workload that calls it.

## Before you start

- The **admin** role in Ace. See [Roles and permissions](https://tryace.dev/docs/security/roles-and-permissions).
- **Proxy Admin** in LiteLLM, to create Ace's user and key.
- A proxy on HTTPS that is reachable from the public internet, because Ace doesn't connect to private or internal network addresses.

## Credentials and permissions

Ace uses one virtual key, owned by a LiteLLM user you create for Ace. Keep the key type at Full Access, because AI APIs keys can't read spend logs.

| Credential | Where to create it | Role | Why Ace needs it |
| --- | --- | --- | --- |
| Admin key (needed) | **Virtual Keys** in the LiteLLM UI, owned by Ace's user | Ace's user has Global Proxy Role **Admin (View Only)** (`proxy_admin_viewer`) | Read spend logs, daily spend and model deployments. |
| Same key (only for actions) | Same | Ace's user has Global Proxy Role **Admin** (`proxy_admin`) | Change fallbacks, budgets and rate limits after approval. |

## Set up

### 1. Create Ace's user and key

1. As a Proxy Admin, open **Internal Users** > **Invite User**. Enter an email for Ace and set **Global Proxy Role** to Admin (View Only), or Admin for approved changes.
2. In Ace, open **Settings** > **Data sources** and choose **LiteLLM**. Enter your **Proxy URL**, then select **Create key** next to **Admin key**.
3. LiteLLM opens **Virtual Keys** > **Create New Key** with Owned By set to Another User, Key Name set to Ace and Key Type set to Full Access. Select Ace's user.
4. Optionally set an expiry under **Key Lifecycle**. Select **Create Key** and copy the key.

Proxies older than June 2026 open `/ui/?page=api-keys` without these presets, so set the same values by hand.

### 2. Connect LiteLLM in Ace

1. Fill in the form:
   - **Proxy URL**: your proxy's HTTPS URL, for example `https://litellm.example.com`.
   - **Admin key**: the key you copied.
2. Select **Connect LiteLLM**. Ace saves only if the proxy is ready and the key can read spend logs.

### 3. Optional: allow actions

1. On the connected LiteLLM page, turn on **Allow Ace to run approved actions**.
2. For automatic fallback changes, select a workload, enter its environment (default `production`), and select **Confirm**.

## Confirm it's working

- The connection shows **Connected** once the readiness and spend-log checks pass. A key without the viewer role fails with "The key is missing the proxy_admin_viewer role."
- LiteLLM is on under **Investigation sources** for each workload that calls it.

## Disconnect

1. In Ace, open **Settings** > **Data sources** > **LiteLLM** and select **Disconnect**. Ace set nothing up in LiteLLM, so there is nothing else to remove there.
2. In LiteLLM, delete Ace's virtual key or Ace's user.

## Next steps

- [Actions and approvals](https://tryace.dev/docs/concepts/actions-and-approvals)
- [How Ace works](https://tryace.dev/docs/concepts/how-ace-works)
- [All integrations](https://tryace.dev/docs/integrations)