AceDocs

How memory works

What Ace remembers from your incidents, when it learns, how memory shapes an investigation or quiets a known-noisy signal, how it fades, and how it stays inside your organization.

When a new incident opens, Ace recalls similar past incidents, what fixed them and what made things worse, and uses that to weigh the new evidence. Memory comes from your own incidents, your engineers' feedback and the verified results of Ace's actions, and it is never shared between organizations.

Memory never grants permissions. Your action policy sets what Ace may change, and your operating rules can narrow it further.

What Ace remembers

Ace keeps two kinds of memory. An incident memory summarizes one incident that reached an outcome, and a learned pattern records one observation about an alert or a fix.

Incident memory

An incident memory is a short summary for future investigations:

PartWhat it holds
Headline and workloadThe incident's headline and the workload it happened on.
What happenedAce's summary of the incident.
Root causeThe root cause Ace established and its confidence, or a note that no cause was established.
ActionsThe actions taken in the incident, and how far each one got.
Engineer notesRecent notes your engineers added to the incident.
OutcomeHow the incident ended, including how verification judged Ace's actions.

Ace writes one only when the incident has a root cause or an action that made progress. It holds summaries and notes, and raw logs and metrics stay out.

Ace searches incident memories by meaning, so a search for "duplicate summaries sent to customers" can find an incident titled "Replay re-sent completed jobs".

Engineer notes and Slack replies are redacted before they're stored, which removes credentials, tokens, keys, signed URLs, email addresses, payment card numbers, phone numbers and other personal or secret details. See Security and data.

Learned patterns

A learned pattern is a small observation, such as "this alert has been a false positive" or "this fix made things worse on this workload". Its confidence rises with supporting evidence and falls with contradicting evidence.

An incident's Memory card lists the patterns Ace used (Used) and learned (Learned), each with its scope and its count of supporting and contradicting observations.

Pattern, as shown in AceWhat it recordsWhere it comes from
Known false positiveAn alert that turned out not to be a real problem.An engineer closes an incident as Expected behavior, Not actionable or Duplicate investigation. An Ace HTTP check also records one when a single failure clears on its own.
Suppressed patternAn alert your team handles somewhere else.An engineer closes an incident as Handled elsewhere. Ace's own checks also record one when they treat a failure as known noise.
Confirmed incidentAn alert that was a real incident.An engineer closes an incident as Incident resolved. Ace also records one when an HTTP check failure turns out to be part of a real incident.
Pattern that came backAn alert Ace had quieted that turned into a real incident.Ace's own checks. Ace also weakens the pattern that quieted the alert.
HTTP error signatureHow a failing HTTP check failed: status code, error type or response.Ace HTTP checks.
Alert signatureAn alert from one of your integrations, by provider and alert name.Each alert that opens or joins an incident.
Runbook preferenceRecommendations your team rejected, and how each fix Ace applied turned out (recovered, did not recover, or made it worse).Your decisions on recommendations, and recovery verification after actions.
How your team respondsReplies, reactions and button clicks on Ace's Slack messages.Slack.

To mark an alert as noise, close the incident as Expected behavior or Not actionable, or use Handled elsewhere if another team owns it. To confirm a real incident, close it as Incident resolved.

Repeated closes can eventually quiet an alert that Ace raises itself. Alerts from your own tools always open an incident, so fix those monitors at the source.

When Ace learns

EventWhat Ace learns
An incident closes, for any reasonAce writes or refreshes its incident memory.
An engineer closes an incident in Ace and picks a reasonA pattern for that alert on that workload (see the table above). Expected behavior and Not actionable also weaken any earlier sign that the alert was a real incident.
Ace verifies the result of an actionHow that fix turned out on that workload. Ace also refreshes the incident memory with the outcome.
An engineer rejects a recommendationThat your team doesn't want that recommendation on that workload.
An alert opens or joins an incidentAn alert signature.
An Ace HTTP check fails and recoversWhether the failure was a false positive, a real incident or known noise, and how it failed.
An engineer replies, reacts or clicks in SlackHow your team responds to that alert. If the engineer edits or deletes a reply, the stored text follows.

Learning never blocks the work that produced it. If a write fails, the incident still closes and the action still completes, and Ace catches up later.

How Ace recalls memory

When Ace weighs the evidence and settles on a cause and recommendations, it pulls in:

  1. similar incidents across your organization, found by meaning from the current headline, what happened and Ace's leading hypotheses
  2. this workload's recent incidents, whether or not they read alike
  3. what Ace's past actions achieved in those incidents: verified recovered, did not recover, made the incident worse, or recovered on its own
  4. the workload's fix history, with every fix Ace applied and how often each one recovered the workload, failed or made things worse
  5. recommendations recently rejected on the workload
  6. learned patterns attached to the incident's alert

If part of memory is unavailable, Ace continues with the rest, so recall never blocks an investigation.

How memory shapes an investigation

Memory changes what Ace considers and proposes, but never the evidence Ace collects.

  • A past cause or fix applies only when current evidence points to the same service, change or failure mode. A similar headline isn't enough.
  • A fix that made an incident worse on a workload is never proposed again on that workload, however it's worded.
  • A fix that didn't recover the workload is proposed again only when the evidence shows why this time is different. The rationale explains the difference, and the fix's history appears in the proposal's evidence for reviewers.
  • A fix that worked is a candidate only when the evidence points to the same cause.
  • If your team recently rejected a similar recommendation on the workload, Ace proposes it again only with new evidence.
  • Ace treats recalled text as untrusted data, like any other incident content.

How memory quiets an alert

Learned patterns can stop Ace from raising an alert that has repeatedly been noise, but only for alerts Ace raises itself: Ace HTTP checks, metric watches and early warnings. Alerts from your own tools, such as a Datadog monitor, a CloudWatch alarm or a Sentry issue, always open or join an incident.

Ace quiets an alert only when a Known false positive or Suppressed pattern has been confirmed several times and still has high confidence. The match is exact, meaning the same alert on the same workload or the same check on that workload, and a suppression never carries over to another workload or alert.

For HTTP checks, a pattern quiets only a single, isolated failed check, because several failing together look like a real problem and Ace raises them as usual.

If a quieted alert turns out to be real, Ace weakens the pattern and records a Pattern that came back. Any pattern Ace uses appears on the incident's Memory card.

How memory fades

Memories don't expire on a schedule. Instead, a learned pattern loses confidence when nothing confirms it, and each confirmation resets the clock.

Suppressions fade fastest, so a stale one can't hide an alert for long. A suppression lapses within three weeks of its last confirmation and Ace raises the alert again, while other patterns fade more slowly.

Contradicting evidence weakens a pattern right away, for example when an engineer closes an incident as Expected behavior after it was marked a real incident, or when a quieted alert turns into a real incident.

Incident memories and fix outcomes don't fade, so Ace remembers a fix that made an incident worse for as long as the workload exists.

Scope and isolation

Every memory belongs to one organization, every read and write is limited to that organization, and nothing learned in one organization is used in another.

A learned pattern applies to your whole organization, to one workload, or to one alert or check on a workload, and the Memory card shows This organization, This workload or This workload and trigger. Suppressions and fix history apply only to the workload they were learned on.

Similar-incident search covers all your workloads, since the same failure often shows up on several services, and a result from another workload is labeled with that workload's name.

When an admin deletes a workload, Ace deletes its incidents, their incident memories and the learned patterns scoped to it.

Memory and operating rules

Operating rules and memory both shape what Ace does:

Operating rulesMemory
Who writes itYour admins, in SettingsAce, from incidents, feedback and verified outcomes
What it isInstructions such as "Prefer rollbacks over forward fixes for the payments service"Observations such as "this fix made things worse on this workload"
ScopeYour organization, by kind of work (diagnosis, recommendations, pull requests, live operations)Your organization, a workload, or one alert or check
Changes over timeOnly when an admin edits itStrengthens, weakens and fades with evidence
Can grant permissionsNoNo

Both can only narrow what Ace does. Ace ignores an operating rule that asks for more, and memory never changes your action policy. See Actions and approvals.

Using memory from your coding agent

The Ace MCP server gives coding agents and reviewers read-only access to the same memory.

  • get_lessons returns fixes that worked, fixes to avoid, and similar past incidents with the actions Ace took and how each was judged.
  • search_incidents finds incidents by meaning or by text.
Before you change anything for INS-1018-ab12, call get_lessons with that incident.
Don't propose any fix listed under "Fixes to avoid". If a fix under "Fixes that
worked" matches the root cause from get_incident_context, start from it.

Next steps

On this page