# Grafana > Connect Grafana so Grafana alerts open incidents in Ace, and Ace investigates them with your alert rules, dashboards and datasources. Source: https://tryace.dev/docs/integrations/grafana Ace reads Grafana alerts, rules, annotations and dashboards during investigations and queries your datasources, and Grafana-managed alerts open incidents. | | | | --- | --- | | Category | Dashboards | | Ace reads | Firing alerts, alert rules, annotations, dashboards, datasource queries and health | | Ace can change | Annotations when an incident opens and resolves, and silences on the paging alert rule while an approved fix rolls out | | Opens incidents | Yes, from Grafana alerts | | Setup | Settings form | Ace works with Grafana Cloud, Grafana Enterprise and self-hosted open-source Grafana. A self-hosted Grafana must use HTTPS and be reachable from the public internet, because Ace doesn't connect to private or internal addresses. ## What Ace reads - Active Grafana-managed alerts and alert rule definitions. - Annotations around the incident. - Dashboard search results, and a dashboard's panels and their queries. - One PromQL, LogQL, TraceQL or SQL query at a time against a datasource, over the incident window. - Datasource health. After a fix, Ace checks whether the incident's alert is still active. It also discovers your datasources and the backend each one reads, for the System Map. ## What Ace can change | Action | What it does | Approval | Undo | | --- | --- | --- | --- | | Annotation | When an incident opens or resolves, Ace adds an annotation such as "Incident opened: ``", tagged `ace`. It goes on the alert's dashboard when the alert links to this Grafana. Otherwise it's org-wide, with the workload's tag | Runs on its own | Yes. Undo from the incident deletes the annotation | | Silence | When an approved fix starts rolling out, Ace silences each alert rule that paged, for a bounded window. The silence matches only that rule, shows "Ace" as its creator and ends on its own | Runs on its own | Yes. Undo from the incident expires the silence | Ace makes these updates only after an admin turns on **Allow Ace to update this tool during incidents** for the connection. See [Actions and approvals](https://tryace.dev/docs/concepts/actions-and-approvals). On Grafana Cloud and Enterprise, Ace reads the service account's permissions and shows which updates it can make. Open-source Grafana may not report them, so there a missing permission shows up only when Grafana refuses an update. ## What opens an incident Grafana alerts reach Ace through a webhook contact point as they fire. Ace doesn't poll Grafana, so nothing arrives until the webhook is set up. - A **firing** alert opens an incident or joins the one already open on the workload, and a **resolved** alert resolves it. - Severity comes from the `severity` label, which can be `critical` or `page`, `high` or `error`, `warning` or `medium`, and `info` or `low`. An alert without the label is high. - The route Ace creates skips alerts labeled `ace=ignore` and alerts with severity `info` or `none`. If you route alerts to Ace yourself, your notification policy decides what reaches Ace. An alert opens an incident only when Grafana is routed to a workload. To route it, open the workload's settings, turn on **Integration alerts** under **Investigation triggers** and select Grafana under **Alert sources**. ## Before you start - The **admin** role in Ace. See [Roles and permissions](https://tryace.dev/docs/security/roles-and-permissions). - The org **Admin** role in Grafana, to create a service account. - On open-source Grafana, the service account needs the Admin role to read alert rules, because Viewer and Editor can't. ## Credentials and permissions | Credential | Where to create it | Permissions | Why Ace needs it | | --- | --- | --- | --- | | Service account token (required) | **Administration** > **Users and access** > **Service accounts** | Viewer role and `alert.provisioning:read` | Reads alerts, dashboards, annotations, datasources and alert rules | | Same token (only for alerts) | | `alert.provisioning:write` | Adds Ace's contact point and notification route | | Same token (only for actions) | | `annotations:create` and `annotations:delete`, plus `alert.silences:create` and `alert.silences:write` (the `fixed:alerting.instances:writer` role, or Editor) | Adds and deletes annotations, and creates and expires silences | Ace uses one service account token. Grafana has no read-only provisioning role, so on Cloud and Enterprise, `fixed:alerting.provisioning:writer` grants `alert.provisioning:read` along with the write access that alert wiring uses. Don't use the secrets reader role, which exports decrypted secrets. ## Set up ### Create the service account token 1. In Grafana, open **Administration** > **Users and access** > **Service accounts** > **Add service account** (`https:///org/serviceaccounts/create`). 2. Set **Display name** to Ace and **Role** to Viewer, then choose **Create**. 3. On Grafana Cloud or Enterprise, add the fixed role `fixed:alerting.provisioning:writer`, and `fixed:annotations:writer` for incident annotations. On open-source Grafana, give the account the Admin role instead. 4. For silences, add `fixed:alerting.instances:writer`, or use the Editor role. 5. Choose **Add service account token** and **Set expiration date**, then **Generate token** and copy the token. ### Connect Grafana in Ace 1. In Ace, go to **Settings** > **Data sources** > **Grafana**. 2. Fill in the form: - **Grafana URL**: your Grafana's HTTPS URL, for example `https://example.grafana.net`. - **Service account token**: the token you copied. 3. Choose **Connect Grafana**. If any validation check fails, Ace doesn't save the token. ### Wire alerts If the token has `alert.provisioning:write`, Ace creates a webhook contact point and a catch-all notification route, both named `ace-`. The route continues to your other routes, so your existing routing is unchanged, and it also covers alert rules you add later. Both stay editable in Grafana, and the **Alerts** section shows what Ace set up. Ace checks it again every hour and removes it when you disconnect. To set it up yourself, choose **Show manual setup** in the **Alerts** section: 1. Copy the webhook URL Ace shows. 2. Choose **Generate token** and copy the token. Ace shows it once. 3. In Grafana, add a contact point with the **Webhook** integration and paste the URL. 4. Under **Optional Webhook settings**, leave **Authentication Header Scheme** at Bearer and paste the token into **Authentication Header Credentials**. 5. In a notification policy, route the alerts that should open incidents to this contact point. ### Allow updates (optional) On the connected Grafana page, turn on **Allow Ace to update this tool during incidents**. ## Confirm it's working - The connection shows **Connected** after Ace reads your dashboards and active alerts. - The **Alerts** section shows **Waiting for first event** until the first delivery Ace accepts, then **Verified**. ## Disconnect 1. In Ace, go to **Settings** > **Data sources** > **Grafana** and choose **Disconnect**. Ace removes its notification route and contact point, and if it can't, it tells you so you can remove `ace-` in Grafana yourself. 2. In Grafana, delete the service account or its token. ## Next steps - [Actions and approvals](https://tryace.dev/docs/concepts/actions-and-approvals): how observability updates and undo work. - [How Ace works](https://tryace.dev/docs/concepts/how-ace-works): how alerts become incidents. - [Integrations](https://tryace.dev/docs/integrations): connect more data sources.