Agentic workflow integrity: connect evaluation to execution

How to preserve the identity of an agent workflow across configuration, evaluation, approval and deployment.

Agentic workflow integrity means being able to check that the workflow admitted for execution is the one that was evaluated, and that its transitions followed the required conditions. A successful run does not establish that chain by itself. The workflow's identity and the evidence authorizing its transitions have to remain connected.

Identify the behavior you are promoting

Source code is one input. Agent behavior also depends on the model version, instructions, tool definitions, retrieval configuration and runtime settings. Record the relevant versions together. Where practical, use content-addressed artifacts so a later reviewer can check whether the inputs changed.

For dynamic workflows, distinguish the version of the planner from the graph it produced. Record the executable graph and its admitted environment separately. Freezing a graph does not make its external dependencies deterministic.

Bind approval to a specific candidate

Evaluate a candidate configuration, attach the results to that candidate and make the promotion decision reference both. If a tool definition or prompt changes after evaluation, the prior result cannot silently authorize the changed behavior. Define which changes require a new evaluation and who can approve exceptions.

Illustrative promotion chain:
behavior manifest → evaluation evidence → approval → deployed manifest
        compare identity at each transition; stop on mismatch

My research on justifiability studies this binding problem in public delivery records. The Agent Graph Runtime provides a separate model for construction, qualification, admission and execution. They inform the design; neither publication certifies a specific deployment.

Preserve integrity during execution

Make state transitions explicit. A queued action, an attempted action and a completed action are different states. Use idempotency for retries, validate prerequisites before consequential steps and keep evidence of both success and refusal. Recovery should resume from durable state rather than asking a model to guess how far the previous run got.

External content can contain instructions. Treat it as data with a source, not as authority to change the workflow's permissions. An approval must remain tied to the operation approved.

Test the chain where it can break

Change a prompt after evaluation and verify promotion refuses the mismatch. Repeat a write and verify it is not duplicated. Interrupt a run between steps and check the recovery path. Remove required evidence and verify the transition remains blocked. Those negative controls establish that the controls affect execution.

When reviewing an engineer's work, ask for the candidate manifest, evaluation result and deployed identity from one real change. The sources below describe my evaluation and workflow research, including what each can and cannot establish.

Work and evidence

Evaluation gates for a multi-agent education system (opens in a new tab)

My contribution: Built the evaluation harness, calibrated LLM judging, isolated case state and environment-specific evaluation gates.

What this supports: The public account describes golden cases as code, a calibrated LLM judge and the evaluation harness around them, on a Google Cloud agent stack.

What it does not establish: The figure of 700,000+ students describes the population served by the system. It does not measure simultaneous users, requests per second or personal ownership of the whole platform. Client code and data remain private.

Source reviewed:

From Traceability to Justifiability (opens in a new tab)

My contribution: Authored a public preprint on accountability structures in agentic software engineering, with code and study records linked from the research index.

What this supports: Research on binding evaluated behavior to deployed behavior and checking the evidence behind promotion.

What it does not establish: A preprint with a dated public-material study; not a certification of any production system.

Source reviewed:

The Agent Graph Runtime (opens in a new tab)

My contribution: Authored the runtime model separating graph construction, qualification, admission, execution and evidence.

What this supports: A published conceptual model for reasoning about executable identity and workflow lifecycle.

What it does not establish: A model and reference representation, not a throughput or reliability guarantee.

Source reviewed:

Technical guides