Trigger.dev
Connect Trigger.dev so failed production runs open incidents, deploys join the change ledger, and Ace can replay or cancel runs and pause queues after approval.
Failed Trigger.dev runs open incidents in Ace, one incident per task. Once the fix is live, Ace can replay the failed runs in stages.
| Category | Jobs |
| Ace reads | Failed runs, run details, traces and logs, error groups, queues, schedules, deployed version, compute cost of failed runs |
| Ace can change | Replay failed runs, cancel runs, pause or resume a queue, only after an admin allows actions |
| Opens incidents | Yes, for failed production runs of workloads that route Trigger.dev |
| Setup | Settings form, plus a Trigger.dev alert webhook |
What Ace reads
Ace reads Trigger.dev while it investigates a workload that lists it under Investigation sources or routes its alerts. From your production environment it reads:
- Runs that ended
FAILED,CRASHED,SYSTEM_FAILUREorTIMED_OUTin the incident window, grouped by task and status, with versions and a sample run. - One run's status, error, stack frames, attempts and version. Ace doesn't get the payload.
- Error groups, run traces and run logs.
- Read-only queries over runs.
- Queue depth and whether a queue is paused.
- Failed runs of the incident's task after a fix, plus at least one completed run, to confirm the task works again.
Ace always leaves out test runs, and a Tasks allowlist limits reads to those tasks.
For the System Map, Ace reads tasks, queues, schedules and the deployed version. An Observer key can read queues, but schedules and deployments need a broader preset and show as not permitted without one.
A scheduled cost scan reports the compute spent on failed runs, though Ace can't change that cost for you.
What Ace can change
Ace proposes Trigger.dev actions only after an admin turns on Allow Ace to run approved actions for the connection. Each one needs approval by default, and earned autonomy can let some of them run under an Automatic policy for a workload and environment you confirm. Others always need a person. See Actions and approvals.
| Action | What it does | Approval | Undo |
|---|---|---|---|
| Run replay | Replays the incident's failed runs in stages, for example 10 first and then the rest, up to 2,000 runs. It waits for the incident's fix to deploy before it starts. | Approval, or Automatic under your policy | No |
| Run cancel | Cancels a runaway task's queued and executing runs in stages, up to 2,000 runs. | Always a person | No. A cancelled run can't be resumed. |
| Queue pause | Pauses or resumes a queue. | Always a person | Yes. Restores the queue's previous state. |
Ace applies these safeguards:
- The replay list is fixed when the first stage starts, and each run replays at most once. Ace never replays its own new runs.
- The cancel list is fixed at approval.
- Ace refuses tasks outside your Tasks allowlist, and it refuses to pause or resume a queue that changed since approval.
- Ace rechecks the incident between stages.
These actions need the Operator key preset. Trigger.dev doesn't let Ace check the preset in advance, so Settings shows that the permission is checked when the action runs.
What opens an incident
Trigger.dev alerts open incidents only on workloads that route this connection (Integration alerts on, Trigger.dev selected). Ace drops alerts that no workload routes.
| Trigger.dev sends | What Ace does |
|---|---|
| A failed run alert | Opens or joins an incident for that task. SYSTEM_FAILURE and CRASHED are critical, and other failures are high. |
| A new or regressed error group | Opens or joins an incident. It carries the same task and error as the failed runs, so it joins their incident. |
| An error group that was unignored | Adds context to an open incident. It never opens one. |
| A deployment that succeeded or failed | Joins the change ledger, never an incident. Ace records the commit when Trigger.dev has git metadata for the deploy. |
Only production failures open incidents, while failures in staging, preview or development add context to an open incident. Ace ignores test runs.
Ace also polls for failed runs, so failures still open incidents before you set up the webhook or when a delivery goes missing. Polling doesn't read deployments, which arrive only through the webhook.
Before you start
- The admin role in Ace.
- A key for your Production environment, which starts with
tr_prod_. - A Trigger.dev Pro or Enterprise plan for the Observer and Operator key presets. Other plans and self-hosted Trigger.dev offer only No restrictions.
Credentials and permissions
| Credential | Where to create it | Preset | Why Ace needs it |
|---|---|---|---|
| Secret key (needed) | Project > Production environment > API keys > New API key | Observer | Read runs, errors, traces, logs and queues. |
| Same key (only for actions) | Same | Operator | Replay and cancel runs, and pause or resume queues. |
| Webhook secret (only for alerts) | Shown when you create the alert in Trigger.dev | None | Verifies that alert deliveries come from Trigger.dev. |
Set up
1. Create an API key in Trigger.dev
- Pick the project, switch to the Production environment, and open API keys > New API key.
- Name it Ace and set Expires (In 90 days or In 1 year).
- Choose Observer, or Operator if Ace should replay runs.
- Create the key and copy it, because Trigger.dev shows it only once.
2. Connect Trigger.dev in Ace
-
Open Settings > Data sources and choose Trigger.dev.
-
Fill in the form:
Field What to enter Secret key The production key ( tr_prod_…).API URL Only for self-hosted Trigger.dev. Leave blank for https://api.trigger.dev.Tasks Optional. A comma-separated allowlist of task identifiers. -
Select Connect Trigger.dev. Ace checks that the key can read runs before it saves.
A self-hosted API URL must use HTTPS and be reachable from the public internet, because Ace doesn't connect to private or internal network addresses.
3. Send alerts to Ace
- In the Alerts section, copy the webhook URL.
- In your Trigger.dev project, open Alerts, choose New alert and pick Webhook.
- Under Alert me when, select Task runs fail, Deployments fail and Deployments succeed. Paste the URL and create the alert.
- Optional: on the Errors page, choose Configure alerts… and add the same webhook. New and regressed errors then join the incident of the runs they fail.
- Copy the webhook secret Trigger.dev shows, paste it into Ace and select Save.
4. Route alerts to a workload
Open the workload in Ace, turn on Integration alerts, and select Trigger.dev as a source.
5. Optional: allow actions
Turn on Allow Ace to run approved actions. To let replays earn Automatic, confirm their workload and environment on the same panel.
Confirm it's working
- If the key can't read runs, the connection's checks report missing Observer access.
- The Alerts badge shows Not set up, Waiting for first event, Verified or Last delivery failed. It turns Verified when the first signed delivery arrives.
- A failed delivery shows the reason, such as a signature that doesn't match the saved secret.
Disconnect
- In Ace, open the Trigger.dev connection under Settings > Data sources and select Disconnect. To stop actions and keep the connection, turn off Allow Ace to run approved actions instead.
- In Trigger.dev, delete the Ace API key and remove the alert webhook. Ace creates nothing in Trigger.dev.
Next steps
Cloudflare
Connect Cloudflare so Ace tells edge errors from origin errors, records DNS, rule and Worker changes next to your deploys, opens incidents from Cloudflare notifications, and proposes edge fixes for approval.
Inngest
Connect Inngest so failed production runs open incidents, app syncs join the change ledger, and Ace can rerun, cancel or pause after approval.