Back to the research
Context EngineeringKV cacheAgent state

Why Agent Transcripts Cannot Fully Describe Cached Context

Research on direct cache control reveals a hidden requirement for reusing an agent’s previous computation.

Is an agent’s transcript enough to reuse its cache?

Under the direct cache controls proposed in Cache Engineering for Agents, matching transcripts is insufficient. The same text can correspond to different cache states because earlier cache operations changed what the model can attend to. Systems that reuse computation must account for those operations, according to the paper’s abstract.

That distinction matters when teams assess an agent’s memory. A readable conversation history does not necessarily describe all the internal state used to continue its work.

The transcript and the attention cache serve different purposes

Context engineering means arranging the instructions, evidence, history and tool results an AI system receives for a task. For a founder building a research assistant, that includes deciding which retrieved documents and earlier findings remain available during a long investigation.

The transcript records text from that investigation. A key-value cache stores intermediate model calculations used during attention, the process that lets the model use earlier context. Reusing those calculations can avoid processing the same retained text again.

The paper describes a cost of changing earlier text through trimming or summarization. Modifying the beginning of an agent’s history invalidates subsequent cached states. The system must then process retained context again to rebuild those states. The authors call this work re-prefilling. See the cache engineering abstract.

The useful distinction is between changing the text and changing the stored calculations associated with that text.

Direct cache control makes operation history matter

The authors propose giving agents operations that alter the active cache without rewriting the transcript or reprocessing retained context. Their drop operation excludes selected cache entries from future attention. Their repos operation adjusts position information in retained entries, which represents where those entries occur in the sequence. These operations are described in the paper’s abstract.

Consider an illustrative research assistant that reads several tool responses. One execution drops cached entries associated with an earlier response. Another execution retains them. Both executions can still have the same written transcript, while their active caches differ.

The researchers identify this as history dependence. The sequence of cache operations affects the resulting cache state. Matching the beginning of the text alone therefore cannot establish that a cached state is suitable for reuse. Their proposed AKV serving engine manages cache reuse, removal and restoration internally to address this challenge. See the authors’ description.

This is a research proposal with a specific evidence limit

Cache Engineering for Agents appears in the index dated October 2026. The evidence available here is the paper’s abstract. It reports benchmark improvements against threshold-triggered summarization, but does not provide enough experimental detail to assess how those results transfer to a particular application. See the research abstract.

The practical inference is conditional. If a system adopts direct cache operations, its reuse logic must distinguish states that text matching alone cannot distinguish. This does not establish that an existing hosted agent service uses these operations or has this problem.

Test cache reuse against the operations that created it

For teams considering direct cache control, the finding supports a focused evaluation.

  1. Establish what the system changes. Determine whether it rewrites history, directly changes cached entries, or combines both approaches. These mechanisms create different requirements for reuse.
  2. Test identical text with different cache operations. Construct runs whose transcripts match but whose cache operations differ. Check whether reuse decisions account for the different states.
  3. Evaluate correctness alongside efficiency. Compare task results, processing cost and response time against the existing approach. Treat the paper’s reported improvements as a reason to experiment, not an expected result for your application.
← Explore more research