What is AI agent memory?

The memory of an AI agent is everything it can reach when it has to answer: what loads into its context at the start, the files it can search, the stores behind its tools and live sources. The context window is only the visible part, and the answer rarely comes from what was already sitting there.

How it works

I describe memory as a stack of tiers, in the way an operating system separates RAM, disk and network.

  • Boot-resident memory. What loads into context at startup: instruction files, tool catalogs, memory indexes. It is always visible and always paid for.
  • Searchable files. Everything on the filesystem that the agent can find with search and open on demand. Large, cheap and invisible until a query touches it.
  • Tool-gated stores. Databases and APIs behind a tool call. The agent has to know to ask.
  • The world. Live sources that no store can cache faithfully.

Policies decide what stays resident, what is read on demand, what is promoted to a faster tier and what is removed. The comparison with hardware breaks in one place. A computer copies memory faithfully. A one-line summary of a document is a lossy copy, and it drifts as the document changes. Agent memory goes stale.

Where it shows up in real work

I tested two ways of locating documents in a pre-registered study, Reading More, Finding Less. One agent had only a hand-written index of one-line digests. The other searched the corpus and read what it found. The index led the agent to the correct document on 52.0% of 102 questions, against 80.4% for grep. Downstream, search-then-read was more accurate: 72.5% against 47.1%. A public replication over 141 documents and 120 frozen questions put search ahead by 12.5 points on accuracy. Its formal verdict is revised, because one prediction about token headroom failed, and the paper reports that failure next to the passes.

The write side failed too. In the same program, of 157 eligible files promoted into a durable memory directory, 2 were read later.

The result has a scope: an authored index of one-line digests over a prose corpus, used as the only locator. It says nothing about an index used together with search.

What it is not

Memory is not the context window, and a larger window is not more memory. Saving a fact is not remembering it either: remembering needs a later read that really happens.

When you need to design it

Design memory when an agent works over more material than fits in its context, which is most real work. Before you write an index that describes your files, check whether the agent with a search tool already beats it. If the agent saves notes for later, count the reads and not the writes. Treat every summary as a cached copy that can decay, and decide what validates it.

Work and evidence

Reading More, Finding Less (opens in a new tab)

My contribution: Authored a pre-registered study and public replication of agent retrieval routing.

What this supports: Evaluation methodology with a released corpus, frozen questions, harness and adjudicator.

What it does not establish: Results concern the studied document corpus and routing policies, not all RAG or agent systems.

Source reviewed:

Common questions

Is a bigger context window the same as more memory?

No. The context window is where memory is visible, so most of the debate happens there. In practice an agent greps a repository, opens a file it has never seen or calls a tool that queries a database. The knowledge that mattered was reachable from the window, not inside it.

Should an agent use a curated index or a search tool?

In my study the index, used as the only way to locate documents, routed worse than search: 52.0% against 80.4%. Where the index did route correctly, accuracy was 86.8%, so the deficit was localization and not comprehension. The claim is limited to an authored index of one-line digests over a prose corpus.

Do agents read the memories they save?

In the system I measured, almost never. Of 157 eligible files promoted into a durable memory directory, 2 were read later, and the prediction that promotion pays back was refuted. If your agent saves facts, measure whether anything reads them again.

Is a missed document a limit of search itself?

Mostly not. The search policy still missed a fifth of the corpus, and a companion study, Finding More, Fusing Less, showed that most of that ceiling was a ranking problem: BM25 added 21.7 points on localization. The same study retired a default that fused search methods, because it failed its own frozen prediction.

Open to senior AI engineering roles and contract engagements, remote in the US, Mexico and Latin America.