AI Engineering

Notes on making probabilistic systems shippable: evaluation harnesses, LLM judges, context engineering, agents, and the GCP estate underneath them. Everything here comes from production systems — method extracted from work, written to be reused.

2026-08-05 · evals · harness · agents

Can You Trust an AI to Grade Your AI?

Many teams use one AI model to grade another one's answers, and then let that grade decide what ships. Almost nobody checks whether the grader itself can be trusted. This is what it took, on a real production system, to make the grader reliable enough that a failing score is allowed to stop a release.

Read →

2026-08-05 · evals · testing · agents

Your AI Agent Passes Its Tests for the Wrong Reason

AI agents are built to remember, and that breaks their tests. When test cases share one user, each case inherits leftover memory from the one before it, so the pass rate measures the order the tests ran in, not the agent. Giving every test its own clean user and its own sealed world fixed that, and let the whole suite run in parallel for free.

Read →

Related: the case study on evaluation gates for a 700,000-student agent system, the GCP dossier, and the working papers on agent harnesses and structural waste.