LangSmith
Connect LangSmith so Ace can read your AI workflow runs and traces during investigations.
Ace reads LangSmith to see what your AI workflows did around an incident, including which runs failed, how long they took and what their child spans did.
| Category | AI workflow traces, latency, cost and feedback |
| Ace reads | Projects, root runs, failed runs, child runs (spans) and project feedback stats |
| Ace can change | Nothing (read-only) |
| Opens incidents | No |
| Setup | Settings form |
What Ace reads
When you connect, Ace lists the projects the API key can see, with their stats. During an investigation it reads:
- Project runs in the incident window: root runs, failed root runs, runs that match the incident, and their child spans.
- The child runs (spans) of one trace, in start order.
Workspace ID limits Ace to one workspace, and Project allowlist limits it to specific projects.
What Ace can change
Nothing.
What opens an incident
Nothing. Ace uses LangSmith as evidence only.
Before you start
- The admin role in Ace.
- A LangSmith API key that can list projects.
- LangSmith cloud in the US or EU works as is. A dedicated or private base URL goes in as a custom endpoint, which must be reachable over HTTPS from Ace's cloud.
Credentials and permissions
| Credential | Where to create it | Permissions | Why Ace needs it |
|---|---|---|---|
| API key | LangSmith: Settings > API Keys > Create API Key | Read access to the projects Ace should see. A service key scoped to your workspace works best. On Enterprise, give it the Workspace Viewer role so it can only read. | List projects and read runs. |
Set up
- Create the API key in LangSmith.
- In Ace, go to Settings > Data sources > LangSmith.
- Fill in the form:
- Endpoint: US (https://api.smith.langchain.com) or EU (https://eu.api.smith.langchain.com), matching your workspace's region. Choose Custom only if LangSmith gave you a dedicated or private base URL, which must use HTTPS.
- API key: starts with
lsv2_. - Workspace ID (optional): the workspace Ace should use, if the key can access several. Find it in LangSmith under Settings > General.
- Project allowlist (optional): one project per line. Leave it blank to let Ace use every project the key can read.
- Choose Connect LangSmith. Ace validates the key first, and if a check fails it shows the error and doesn't connect.
To change the configuration, choose Update credentials, then Save and revalidate. Ace replaces the stored configuration, revalidates and refreshes discovery.
Confirm it's working
The panel shows Connected, and the discovery panel lists your projects. Two validation errors come up most often:
- "No LangSmith projects were visible to this API key. If this is an org-scoped key, add the workspace ID."
- "No LangSmith projects matched the configured project allowlist." Check the names in Project allowlist.
Disconnect
In Ace, go to Settings > Data sources > LangSmith and choose Disconnect. In LangSmith, delete the key under Settings > API Keys.
Next steps
- Logfire: add application logs and traces.
- How Ace works: how evidence feeds an investigation.
- Security and data: what Ace stores.
Sentry
Connect Sentry so issue and metric alerts open incidents in Ace, and Ace reads errors, releases, spans, logs and replays while it investigates.
LiteLLM
Connect your LiteLLM proxy so Ace reads model errors and spend during investigations, finds wasted model spend, and proposes fallback and budget changes for approval.