AI agent auditability is the ability to reconstruct what an agent did and to examine the evidence behind each action. A transcript keeps what was said, while an audit trail also shows which identity acted, what it was allowed to do, which tool ran and what changed.
Auditability is built at the boundary of each service, not inside the conversation. Every run and every tool call gets a stable identifier. Next to it the system records who called, the authorization decision that applied, the version of the tool, the start and finish times, the outcome and references to any artifact the call produced. The versions of the model and of the prompt stay available beside that record.
One rule matters more than the rest. The service behind a tool writes down the operation it really executed. It does not rely on the model to describe what it did.
The record also has limits that someone sets on purpose: which fields may be logged, which must be redacted, who may read the trail and how long it is kept. A trail that leaks customer data is a new incident surface.
Where it shows up in real work
MetaMCP is an open-source gateway I built. Agents reach many MCP tools through it, which makes it a natural place to keep a record. When I took it from a laptop to a cloud runtime, one question from its case study mattered most: how do we know what happened? The answer has four parts: logs, smoke tests, prompt fixtures and a ledger file that is exported as evidence. Each call is also tied to an identity, because the platform verifies the caller and the application checks a bearer token.
I measured the wider problem in a preprint, From Traceability to Justifiability. It surveys 47 delivery platforms and 30 public repositories, from public material only. No platform has a default record that emits a content-addressed identity of the deployed behavior, which means the model version, the instructions, the tool definitions and the retrieval and runtime configuration taken together.
What it is not
Auditability is not a conversation log, and observing a run is not the same as justifying it. A trace can show that a tool ran. It does not show that the user had permission, that the data supported the conclusion or that a deployment was justified.
When you need it
You need it as soon as an agent can change something: write a record, send a message, approve a step. A simple test tells you where you stand. Take a failed or disputed action and try to reconstruct it without asking the agent that performed it. If another reviewer can find the real tool arguments, the permission decision and the outcome, and can tell an attempted operation from a committed change, the trail works. Include interrupted runs, denied operations, retries and partial failures in that test.
My contribution: Authored a public preprint on accountability structures in agentic software engineering, with code and study records linked from the research index.
What this supports: Research on binding evaluated behavior to deployed behavior and checking the evidence behind promotion.
What it does not establish: A preprint with a dated public-material study; not a certification of any production system.
Source reviewed:
Common questions
What is the difference between traceability and justifiability?
Traceability shows what happened: a tool ran, a release went out. Justifiability shows that the evidence licensed the step, and it is the one level that can refuse a transition. For a release, that evidence can include the evaluated configuration, the test results, the approval and the identity of what was deployed.
What should an agent audit record contain?
A starting set is a run identifier, a tool call identifier, a reference to the actor, the operation, the tool version, the authorization decision, the start and finish times, the outcome and references to artifacts. These fields are a starting point, not a compliance standard. The record has to fit the questions the organization will need to investigate.
Should an audit trail keep everything forever?
No. Decide which fields can be logged, which need redaction, who can inspect the records and how long they stay available. A trail that exposes customer data creates a new kind of incident.
Can a delivery pipeline prove that the agent it tested is the agent it deployed?
In the public material I measured, not yet. Across 47 platforms, none emits by default a checkable identity of the deployed behavior, and on agent platforms the common answer is an immutable version number behind a pointer that can move, in 16 of 27. The findings belong to a defined sample and observation date, and the paper states what would falsify each one.