Agent Native
Loading...
Back to Patterns

Updated 2026-07-26

Architecture Pattern

Context Compaction Pattern for Long-Running Agents (2026)

When to compact an agent's context, what must stay verbatim, and why observation masking beat LLM summarization in JetBrains' SWE-bench runs (52% cost cut, +2.6% solve rate). Trigger thresholds, keep-zones, and the failure modes of naive summarization.

Agent Native•Pattern: Threshold-triggered context compaction with keep-verbatim zones•2026-07-26

Problem

Long-running agent loops accumulate context - tool outputs, file dumps, failed attempts - until they hit two walls. The hard wall is the context limit ending the task mid-flight. The soft wall arrives much earlier: context rot. Chroma's 2025 study across 18 frontier models found performance degrading as input length grows even on trivially simple tasks, well before the window is full, and the classic lost-in-the-middle result (Liu et al.) measured drops of 20+ percentage points on multi-document QA when the relevant document sits in the middle of a long context. Naive fixes fail in characteristic ways: truncation amputates the system prompt's constraints or early task decisions; whole-transcript LLM summarization destroys operationally load-bearing detail; and JetBrains Research's 'The Complexity Trap' study (arXiv 2508.21433; blog.jetbrains.com/research/2025/12/efficient-context-management) measured that pure LLM summarization on long SWE-bench trajectories matched observation masking's cost savings but produced trajectories up to 15% longer because summaries obscured the signals that tell an agent it is done. The pattern's job is to reduce tokens without reducing the agent's ability to finish the task.

Use threshold-triggered compaction when tasks routinely produce trajectories that approach or exceed the effective context window - coding agents on multi-hour refactors, research agents crawling dozens of sources, operations agents running all day. Who should choose what: teams on hosted agent products largely have this decided for them (Claude Code auto-compacts as the window fills and exposes /compact for manual control; Anthropic's API now offers server-side compaction that summarizes into a dedicated compaction block when a configurable input-token trigger is reached, per platform.claude.com's compaction docs). Teams building custom loops should prefer the cheapest interventions first - Anthropic's context-engineering guidance and the JetBrains results both point to clearing/masking stale tool results before reaching for LLM summarization, and Anthropic's API ships tool-result clearing as a first-party context-editing operation. Reach for full summarization-based compaction only for genuinely long-horizon tasks, and trigger it proactively: the research consensus clusters around acting at roughly 60-70% window utilization, because summaries generated after the model is already in the rot regime are themselves degraded. Do not use aggressive compaction on short tasks (overhead exceeds benefit), in compliance contexts where the full transcript is the audit artifact (compact the working context, but persist the raw log separately), or as a substitute for sub-agent isolation when work is genuinely parallelizable - Anthropic's effective-context-engineering post names compaction, structured note-taking, and multi-agent architectures as distinct tools, not interchangeable ones.

Components

  • Token budget monitor: tracks input-token utilization per turn against the model's effective (not nominal) window, firing at a configurable threshold - the proactive zone research suggests is ~60-70% utilization, not the 90%+ panic zone
  • Keep-verbatim zones: regions never summarized - the system prompt and tool schemas (byte-stable, also to preserve prefix-cache hits), the task statement and acceptance criteria, and the most recent turns (production guides converge on roughly the last 5-7 turns kept in full)
  • Tool-result clearing / observation masking stage: the cheap first lever - replaces stale, re-fetchable tool outputs with a placeholder while keeping the record that the call happened and what it returned in one line; JetBrains' 'The Complexity Trap' (arXiv 2508.21433) measured this alone cutting cost ~52% while improving SWE-bench solve rate ~2.6% (Qwen3-Coder 480B configuration)
  • Summarizer: an LLM pass (often a cheaper model) that distills the evicted middle of the trajectory into a structured summary - decisions made, constraints discovered, files touched, dead ends ruled out - written against an explicit schema rather than freeform prose
  • Durable external memory: a scratchpad file, note store, or plan document outside the context window (Anthropic's structured note-taking; the memory tool in the Claude API) holding state that must survive any number of compactions
  • Compaction ledger: logs each compaction event - tokens before/after, what was cleared vs summarized, summary content - so post-compaction failures can be traced to specific information loss instead of being mysteries

Flow

  1. 01Each turn, compute context utilization. Below the trigger threshold, do nothing - compaction has real costs (latency, information loss, prefix-cache invalidation) and should not run eagerly on short tasks.
  2. 02At the threshold, run the cheap stage first: clear or mask stale tool results - old file reads since re-readable, superseded search results, logs already acted on - keeping a one-line record of each call. This is the intervention with the best measured cost/benefit (JetBrains, arXiv 2508.21433: ~52% cost reduction with a ~2.6% solve-rate improvement on SWE-bench trajectories) and it requires no LLM call.
  3. 03Re-check utilization. Masking alone frequently buys enough headroom; the Anthropic API's tool-result clearing exists precisely because tool outputs dominate agent-context growth.
  4. 04If still above threshold, summarize the evicted middle: everything between the keep-verbatim head (system prompt, task statement) and the keep-verbatim tail (recent turns). Summarize against a schema - decisions and their reasons, current plan state, discovered constraints, artifacts produced, approaches ruled out and why - because 'why we abandoned approach A' is exactly what freeform summaries drop and exactly what prevents the agent from retrying dead ends.
  5. 05Write durable facts to external memory rather than carrying them in-window: file paths, credentials references, the running plan. Notes survive compaction losslessly; summaries do not. This is Anthropic's structured note-taking lever, and it converts compaction from 'compress everything' to 'compress only what was never worth persisting properly'.
  6. 06Rebuild the context as: system prompt + task statement (verbatim) → compaction summary block → recent turns (verbatim) → current tool schemas. Keep the head byte-identical to preserve serving-side prefix-cache hits - a compaction design that rewrites the system prompt also silently destroys your KV-cache economics.
  7. 07Log the event in the compaction ledger and continue the loop. If the agent stumbles immediately after a compaction - re-asking answered questions, retrying ruled-out approaches - the ledger tells you what the summary dropped; tighten the schema, not just the threshold.
  8. 08For tasks that outlive even repeated compaction, escalate to architecture: split into sub-agents whose full trajectories stay out of the orchestrator's window, returning only distilled results - the third lever in Anthropic's long-horizon triad, complementary to compaction rather than competing with it.

Tradeoffs

Masking/clearing vs LLM summarization

Masking is deterministic, free, and reversible (re-fetch the file), but only shrinks tool outputs - it cannot compress dialogue or reasoning. Summarization compresses everything but is lossy in unpredictable ways and costs an LLM call at exactly the moment the context is largest. The JetBrains SWE-bench data (arXiv 2508.21433) is the clearest head-to-head: observation masking matched or beat pure summarization (~52% cost cut, ~+2.6% solve rate; summarization matched cost but produced up to 15% longer trajectories). Default to masking first; earn your way to summarization.

Trigger threshold placement

Trigger too late (90%+) and the summarizer itself operates in the rot regime - Chroma's study found degradation at every increment of context growth, with some 200K-window models degrading measurably by ~50K tokens - so the summary quality drops precisely when you need it most. Trigger too early and you pay compaction latency and information loss on tasks that would have finished fine. The 60-70% band is the defensible default; tune per task class from your ledger data, not per anecdote.

Compression ratio vs task continuity

The ACON paper (arXiv 2510.00615, 'Optimizing Context Compression for Long-horizon LLM Agents') shows the ceiling: 26-54% peak-token reduction while largely preserving task performance on AppWorld, OfficeBench, and multi-objective QA, with distilled smaller compressors preserving ~95% of the teacher's accuracy - but 'largely preserving' is doing work in that sentence. The JetBrains finding that summarization can lengthen trajectories by up to 15% shows the floor for naive approaches. The difference is schema: summaries that enumerate decisions, constraints, and dead ends preserve control flow; freeform 'summarize the above' prompts preserve vibes.

Prefix-cache economics

Compaction rewrites the context, which invalidates serving-side prefix caches for everything after the first changed byte. If your head zone (system prompt + tool schemas) stays byte-stable, you keep the most valuable cache segment; if compaction reorders or rewrites the head, every post-compaction turn pays full prefill price. On API providers with cached-input discounts, a careless compaction design can cost more in lost cache hits than it saves in tokens.

In-window compression vs external memory

Anything the agent will need again verbatim - plans, key file paths, decisions - belongs in external notes (memory tool, scratchpad file), not in a summary, because notes are lossless and re-loadable while summaries degrade with each generation. The tradeoff is tooling: external memory requires the agent to actually read its notes back in, and note-taking discipline is itself a prompt-engineering surface. Compaction without external memory forces every important fact through a lossy channel repeatedly - the agent equivalent of photocopying a photocopy.

Failure modes

  • 01Summary-induced amnesia loops: the compaction drops 'we ruled out approach A because X', and the agent burns the reclaimed tokens re-attempting A. Detectable in the ledger as repeated tool calls with identical arguments across a compaction boundary.
  • 02Lost stop signals: summaries that obscure completion criteria - the JetBrains-measured failure where summarization produced up to 15% longer trajectories because the agent could no longer tell it was done. Keep acceptance criteria in the verbatim head zone, never in the summarized middle.
  • 03Summarizing the un-summarizable: system prompts, tool schemas, and exact identifiers (file paths, IDs, error strings) that must survive byte-for-byte. A summarized error message the agent later needs to grep for is data destroyed, not compressed.
  • 04Panic-threshold compaction: triggering at 95% means the summarizer runs on rotted context and may itself overflow. Community-measured Claude Code auto-compact trigger reports range from ~77% to ~95% depending on version and configuration - the official docs specify the mechanism (threshold → summary → compaction block) but not a stable percentage, which is a hint to control the threshold yourself in custom loops rather than inherit a moving default.
  • 05Compounding summary drift: on very long tasks, summaries-of-summaries degrade generationally. Anchor each compaction against the original task statement and external notes, not solely against the previous summary.
  • 06Silent audit-trail destruction: compacting the only copy of the transcript in a domain that requires one. The working context is disposable; the raw event log must be persisted separately before any compaction runs.
  • 07Cache-hostile rebuilds: compaction implementations that re-serialize the head zone with cosmetic differences (whitespace, key order, timestamps) and silently zero the prefix-cache hit rate - visible as a prefill cost jump immediately after each compaction event.

Implementation notes

  • 01On the Claude API, use the first-party primitives before building your own: server-side compaction (configurable input-token trigger, summary delivered in a compaction block), context editing with tool-result clearing for stale tool outputs, and the memory tool for durable notes - documented at platform.claude.com (compaction docs and the tool-use cookbook). What you build yourself should be the schema and thresholds, not the plumbing.
  • 02In LangGraph-style custom graphs, implement the monitor as a pre-model hook: trim/mask messages against a max-token budget on every turn, with a summarization node that fires only past the threshold and writes its output into graph state - keeping checkpointed full history separate from the working window the model sees.
  • 03Write the summarizer prompt against an explicit schema: decisions + reasons, constraints discovered, current plan state, artifacts produced (exact paths/IDs verbatim), approaches ruled out + why, open questions. Every category maps to a failure mode above; freeform summaries are how load-bearing entities and dead-end records silently disappear.
  • 04Order the rebuild for cache stability: byte-identical system prompt and tool schemas first, then task statement, then the summary block, then verbatim recent turns. Verify with your provider's cached-token metrics that post-compaction turns still hit the prefix cache.
  • 05Log every compaction event (tokens before/after, cleared vs summarized items, the summary text) and add a regression eval: replay a sample of pre-compaction trajectories and measure task completion with and without the compaction pipeline. ACON's 26-54% peak-token reduction at ~95% preserved accuracy (arXiv 2510.00615) is the benchmark to hold your own pipeline against - if your completion rate drops more than a few points, your schema is dropping something load-bearing.
  • 06Set the trigger from measurement, not folklore: run your real task mix at different thresholds and plot completion rate vs cost. The 60-70% starting point comes from the context-rot literature (degradation onset around 50-70% fill in multiple 2026 syntheses), but your effective window depends on your model, your tool-output shapes, and how much of the window a single turn can consume.