AI Engineering

Notes on making probabilistic systems shippable: evaluation harnesses, LLM judges, context engineering, agents, and the GCP estate underneath them. Everything here comes from production systems — method extracted from work, written to be reused.

2026-08-05 · evals · testing · agents

Your AI Agent Passes Its Tests for the Wrong Reason

AI agents are built to remember, and that breaks their tests. When test cases share one user, each case inherits leftover memory from the one before it, so the pass rate measures the order the tests ran in, not the agent. Giving every test its own clean user and its own sealed world fixed that, and let the whole suite run in parallel for free.

Read →

Related: the case study on evaluation gates for a 700,000-student agent system, the GCP dossier, and the working papers on agent harnesses and structural waste.