Datagras
Back to blog

AI governance24 October 2026

Why was it allowed? The question every AI agent incident ends with

Why was it allowed? The question every AI agent incident ends with

On Monday morning a compliance manager gets a one-line email from the auditors: an AI agent issued 400 refunds over the weekend, and they want to know why that was allowed. (This is an illustrative case, not a customer.)

The engineers can show what the agent did and when. The finance team can show the cost. Nobody can say which rule applied, who switched it on, or whether it would have stopped anything.

Seeing what an agent did is one record. Showing why it was allowed is a different one, and it often doesn't exist.

An AI agent, here, is software that uses an AI model to take actions on its own, such as issuing a refund or updating an account.

Four records answer "why was it allowed?": the rule and its version, who switched it on and when, what it would have done, and what it did when it fired. Each has a common gap and a savi answer.

What "why was it allowed" means

An agent's action is allowed when a rule permits it. The evidence for that is the rule history: what the rule said, which version applied, who turned it on, and when.

Most teams keep the rule itself somewhere, such as a policy document or a setting in a console. Fewer keep the history around it.

The gap, in numbers

Dataiku and The Harris Poll surveyed 685 CIOs across eight countries in July 2026 and published the results on 24 September. 81% said they lack complete oversight of agents created entirely outside approved systems or formal channels. The same survey found 83% lack standardised agent lifecycle management, and 72% can't consistently measure whether all their agents deliver business outcomes. Dataiku commissioned the survey, so read it as direction, not a census.

The report also found that 90% of those CIOs are confident they have complete tracking of their agents. A team can believe it has a full list of agents and still lack the record behind each decision. The list says what exists. The rule history says why something was allowed.

Four records that answer the question

An auditor asking "why was it allowed" is asking for four things. Here is what each one needs and what often exists today.

The auditor needsWhat it takesWhat often exists today
The rule and its versionEvery rule saved with a version number, so the one in force on Saturday can be read on MondayA current policy page, with no history
Who switched it on, and whenA log of changes with the person and the timeChat messages and memory
What the rule would have doneA replay of the rule on past activityNothing, or a guess
What it did when it firedA record of each time the rule allowed, warned about or blocked an action, with the reasonApplication logs that don't mention the rule

If one of the four is missing, the answer to the auditors has a hole in it.

Test a rule on the past before you enforce it

Writing a rule and switching it on is the easy part. Testing it first on activity that already happened is a separate step.

That replay, called policy simulation, shows how many actions a new rule would have blocked last month, and which agents they came from. It turns "we think this is safe to enforce" into a number. In savi, a simulation can look back up to 90 days.

Replay also guards against the opposite problem. A rule that blocks too much stops real work, and the team switches it off in a hurry. That change then needs its own record.

Four questions to ask your team this week

  • Can you open the version of a rule that was live on a given date?
  • Can you name who turned it on?
  • Can you show what it would have blocked in the past 30 days?
  • When it acted, can you show the action, the rule and the outcome in one place?

If the answer to any of them is "we'd have to ask around," you've found the work.

What changes with a record

An agent will do something unexpected at some point, and the auditors' email arrives either way.

With the four records in one place, the reply can be one message. Without them, someone has to pull logs from different systems and hope they agree.

Where savi fits

savi is a record of what AI agents did, under which rule, and at what cost. For this question it keeps all four records in one place:

  • The rule and its version: the policy engine saves each rule. A new rule can start in Shadow mode, where it only logs what it would have done and blocks nothing.
  • Who switched it on, and when: a signed audit log of changes.
  • What the rule would have done: policy simulation on up to 90 days of past activity.
  • What it did when it fired: each allow, warn or block is recorded with its reason.

One limit worth knowing: a rule can refuse an action only when the agent's code checks with savi first, so savi isn't a checkpoint that every request must pass through.

Facts and figures in this post were checked on 24 October 2026.

Why was it allowed? The question every AI agent incident ends with · Datagras