AI governance24 October 2026
Why was it allowed? The question every AI agent incident ends with
Why was it allowed? The question every AI agent incident ends with
On Monday morning a compliance manager gets a one-line email from the auditors: an AI agent issued 400 refunds over the weekend, and they want to know why that was allowed. (This is an illustrative case, not a customer.)
The engineers can show what the agent did and when. The finance team can show the cost. Nobody can say which rule applied, who switched it on, or whether it would have stopped anything.
Seeing what an agent did is one record. Showing why it was allowed is a different one, and it often doesn't exist.
An AI agent, here, is software that uses an AI model to take actions on its own, such as issuing a refund or updating an account.
What "why was it allowed" means
An agent's action is allowed when a rule permits it. The evidence for that is the rule history: what the rule said, which version applied, who turned it on, and when.
Most teams keep the rule itself somewhere, such as a policy document or a setting in a console. Fewer keep the history around it.
The gap, in numbers
Dataiku and The Harris Poll surveyed 685 CIOs across eight countries in July 2026 and published the results on 24 September. 81% said they lack complete oversight of agents created entirely outside approved systems or formal channels. The same survey found 83% lack standardised agent lifecycle management, and 72% can't consistently measure whether all their agents deliver business outcomes. Dataiku commissioned the survey, so read it as direction, not a census.
The report also found that 90% of those CIOs are confident they have complete tracking of their agents. A team can believe it has a full list of agents and still lack the record behind each decision. The list says what exists. The rule history says why something was allowed.
Four records that answer the question
An auditor asking "why was it allowed" is asking for four things. Here is what each one needs and what often exists today.
| The auditor needs | What it takes | What often exists today |
|---|---|---|
| The rule and its version | Every rule saved with a version number, so the one in force on Saturday can be read on Monday | A current policy page, with no history |
| Who switched it on, and when | A log of changes with the person and the time | Chat messages and memory |
| What the rule would have done | A replay of the rule on past activity | Nothing, or a guess |
| What it did when it fired | A record of each time the rule allowed, warned about or blocked an action, with the reason | Application logs that don't mention the rule |
If one of the four is missing, the answer to the auditors has a hole in it.
Test a rule on the past before you enforce it
Writing a rule and switching it on is the easy part. Testing it first on activity that already happened is a separate step.
That replay, called policy simulation, shows how many actions a new rule would have blocked last month, and which agents they came from. It turns "we think this is safe to enforce" into a number. In savi, a simulation can look back up to 90 days.
Replay also guards against the opposite problem. A rule that blocks too much stops real work, and the team switches it off in a hurry. That change then needs its own record.
Four questions to ask your team this week
- Can you open the version of a rule that was live on a given date?
- Can you name who turned it on?
- Can you show what it would have blocked in the past 30 days?
- When it acted, can you show the action, the rule and the outcome in one place?
If the answer to any of them is "we'd have to ask around," you've found the work.
What changes with a record
An agent will do something unexpected at some point, and the auditors' email arrives either way.
With the four records in one place, the reply can be one message. Without them, someone has to pull logs from different systems and hope they agree.
Where savi fits
savi is a record of what AI agents did, under which rule, and at what cost. For this question it keeps all four records in one place:
- The rule and its version: the policy engine saves each rule. A new rule can start in Shadow mode, where it only logs what it would have done and blocks nothing.
- Who switched it on, and when: a signed audit log of changes.
- What the rule would have done: policy simulation on up to 90 days of past activity.
- What it did when it fired: each allow, warn or block is recorded with its reason.
One limit worth knowing: a rule can refuse an action only when the agent's code checks with savi first, so savi isn't a checkpoint that every request must pass through.
Facts and figures in this post were checked on 24 October 2026.
