Agent Native
Loading...
Back to Patterns

Updated 2026-07-26

Architecture Pattern

Context Offloading Pattern for Agents (2026)

Where durable working state lives between agent loop iterations: filesystem scratchpad vs vector store vs structured memory store - a decision table by state type, grounded in Anthropic's measured 39% eval lift and 84% token reduction from memory + context editing, Claude Code's memory files, LangMem, and the Manus filesystem-as-context playbook.

Agent Native•Pattern: Context offloading: externalize working state to durable substrates, keep references in the window•2026-07-26

Problem

An agent loop accumulates state faster than any context window can hold it: plans, tool outputs, intermediate artifacts, learned facts, and conversation history all compete for the same token budget, and the default failure is stuffing everything back into context every turn until quality degrades and cost explodes. The vendors have now published numbers proving externalization is not a workaround but the higher-performing architecture: Anthropic's context-management announcement (anthropic.com/news/context-management, alongside Claude Sonnet 4.5) reported that combining its file-based memory tool with context editing improved agentic-search performance by 39% over baseline - context editing alone delivered 29% - and that in a 100-turn web-search evaluation, context editing cut token consumption by 84%. The design question the pattern answers is not whether to offload but where each kind of state should live: a filesystem scratchpad, a vector store, or a structured memory store - because putting plans in a vector store or embeddings-searching your own to-do list are both real and common architecture mistakes.

Use context offloading for any agent that runs longer than a handful of turns or resumes across sessions: coding agents, research agents, long-horizon automation, and multi-agent systems where subagents must share state without sharing windows. Who should choose what: agents that already run in an environment with a real filesystem (Claude Code and Claude Agent SDK with the memory tool, OpenClaw-style operator agents, CI-based coding agents) should default to file scratchpads - it is the substrate Anthropic's own tooling uses, with CLAUDE.md project instructions and persistent memory directories as the built-in example, and the one Manus's context-engineering write-up (manus.im blog, 'Context Engineering for AI Agents: Lessons from Building Manus') calls the ultimate context: unlimited in size, persistent by nature, directly operable by the agent. Framework-based products on LangGraph should use the built-in store (BaseStore namespaces) with LangMem for extraction, because they get cross-thread persistence and background consolidation without new infrastructure (langchain-ai.github.io/langmem). Reach for a vector store only for the one state type that needs similarity search over large corpora - recall over many past episodes - not as the default memory substrate. Skip the pattern entirely for single-shot or short stateless tasks, where an offloading layer is pure latency and complexity.

Components

  • Filesystem scratchpad: a per-task workspace where the agent writes plans (todo.md), notes (NOTES.md), and intermediate artifacts as files - the substrate behind Anthropic's memory tool (file-based, public beta on the Claude Developer Platform per its context-management announcement) and Claude Code's CLAUDE.md / auto-memory files
  • Structured memory store: namespaced key-value/document storage for learned facts and preferences - LangGraph's BaseStore with LangMem's semantic (facts), episodic (past interactions), and procedural (learned behavior rules) memory types on top (langchain-ai.github.io/langmem)
  • Vector index over episodes or corpora: embedding search reserved for large-N recall ('have we seen an error like this before'), not for working state
  • Context assembly policy: the per-turn rule deciding what enters the window - active plan, distilled recent state, references (paths, IDs, URLs) to everything else - the just-in-time retrieval strategy Anthropic's context-engineering post describes as the alternative to pre-loading
  • Compaction with restorable references: summarization that replaces bulk content but preserves the pointer (file path, URL, memory key) so anything dropped can be re-fetched - Manus's 'restorable compression' rule: never compress irreversibly if the source still exists
  • Manifest/index file: a small always-in-context table of contents listing what has been offloaded where, so the agent knows its own external state exists - the piece most homegrown implementations forget

Flow

  1. 01At task start, the agent creates or loads its workspace: reads project instruction files (Claude Code loads CLAUDE.md automatically; a custom harness injects its equivalent), loads the manifest of prior offloaded state, and queries the memory store for facts relevant to the task.
  2. 02The agent writes its plan as a file before executing - Anthropic's context-engineering post (anthropic.com/engineering/effective-context-engineering-for-ai-agents) describes structured note-taking / agentic memory as notes persisted outside the window and pulled back later, with Claude Code maintaining a to-do list as the canonical example. Re-reading the plan file each iteration also recites goals into recent attention, countering long-loop drift.
  3. 03Every large tool output is offloaded on arrival: write the full artifact (log, dataset, scraped page, diff) to a file, keep only a distilled summary plus the path in context. This is the mechanical core of the pattern - the context holds handles, not payloads.
  4. 04Durable learnings extracted during work - user preferences, environment quirks, decisions with rationale - go to the structured memory store, not the scratchpad: the scratchpad dies with the task, the store outlives it. LangMem's background manager pattern does this extraction off the hot path so consolidation never blocks the loop.
  5. 05Each turn, the assembly policy rebuilds the window: system prompt + active plan + manifest + distilled recent turns + just-in-time retrieved items. Older raw turns are compacted with references preserved; Anthropic's 100-turn evaluation measured 84% token reduction from exactly this discipline (context editing), while improving task performance rather than degrading it.
  6. 06On resume (new session or post-compaction), the agent bootstraps from durable state: manifest, plan file, memory-store query - reconstructing working context from substrates instead of replaying transcripts.
  7. 07At task end, a consolidation pass promotes anything worth keeping from scratchpad to memory store, updates instruction files if behavior rules changed, and deletes the disposable workspace - preventing the scratchpad from becoming an unbounded junk drawer.

Tradeoffs

Filesystem scratchpad vs structured memory store

Files win for task-scoped working state: plans, artifacts, and drafts are naturally document-shaped, human-inspectable, git-diffable, and directly operable by the agent's existing tools - which is why Anthropic built its memory tool as file-based and Manus calls the filesystem the ultimate context. Stores win for cross-task knowledge: namespacing by user/agent/domain, structured queries, and background consolidation (LangMem's semantic/episodic/procedural split) have no clean file equivalent. Teams that put everything in files end up grepping for preferences that should have been a keyed lookup; teams that put everything in a store serialize plan documents into rows nobody can read. Map state to substrate by lifetime and shape, not by which layer you built first.

Vector store as default vs last resort

The 2023-era default - embed everything, retrieve by similarity - is the wrong architecture for working state: a plan has one correct retrieval (the current plan), not a top-k neighborhood, and similarity search over your own scratchpad returns plausible-but-stale fragments that quietly corrupt the loop. Reserve embeddings for what they are for: recall across hundreds+ of past episodes or an external corpus. Our mem0-vs-zep and pgvector comparisons on this site cover that layer; this pattern's point is that most agent working state never needs it.

Offloading overhead vs window stuffing

Every offload is an extra tool call and every retrieval is latency the stuffed-window design does not pay per-turn. The measured answer is that the overhead wins on net for long tasks: Anthropic's numbers - 39% performance lift with memory + context editing, 84% token reduction over 100 turns - show externalization improving both cost and quality simultaneously, because a curated window outperforms a polluted one. The crossover is task length: under ~10 turns with modest tool outputs, stuffing is simpler and fine; past that, per-turn token cost grows linearly with history and the offloaded design wins on both axes.

Restorable vs lossy compaction

Compaction that discards content irreversibly eventually deletes the fact the task turns out to need - the classic 50-turn failure where the agent re-fetches, re-derives, or hallucinates what it once knew. Manus's rule makes compaction safe: compress only what can be restored (keep the URL when dropping page content, keep the path when dropping file contents). The cost is that references themselves accumulate; the manifest keeps them navigable. Truly lossy summarization should be a last resort applied to the oldest, lowest-value spans - never to plans or decisions.

Framework-managed vs harness-owned state

LangGraph's store plus LangMem gives you namespacing, persistence, and background extraction for free - but couples your memory layer to the framework's runtime and schema. A harness-owned file workspace (the Claude Agent SDK / Claude Code model) is portable across frameworks and trivially debuggable, but you own retention, concurrency, and multi-agent access control yourself. Multi-agent systems tilt file-ward: a shared workspace of artifacts is the simplest cross-agent state bus that does not require every subagent to share a framework.

Failure modes

  • 01Write-only memory: the agent dutifully offloads state but never reads it back because nothing in the prompt or manifest tells it the state exists. Symptom: files pile up while the agent re-derives their contents. The manifest-in-context plus a bootstrap read step is the fix.
  • 02Stale-plan poisoning: the plan file stops being updated mid-task, and re-reading it each turn recites an obsolete goal into attention - offloading turns a drift problem into an anchoring problem. Plan updates must be part of the loop contract, not best-effort.
  • 03Scratchpad-as-landfill: without end-of-task consolidation and deletion, the workspace becomes thousands of orphaned files whose manifest no longer fits in context - reproducing the original window problem one level down.
  • 04Similarity-retrieving working state: pulling 'relevant' scratchpad fragments by embedding search returns stale near-duplicates of current state (yesterday's plan v2 when v5 is current). Working state needs deterministic retrieval by key or path, never top-k.
  • 05Irreversible compaction of load-bearing facts: summarizing away a constraint ('the API rejects batch sizes over 100') that the task needs at turn 60 - the exact failure Manus's restorable-compression rule exists to prevent.
  • 06Cross-agent write races: two subagents updating the same notes file or memory namespace without ownership rules, silently overwriting each other's learnings - file substrates need the same single-writer discipline as any shared store.
  • 07Secrets leaking into durable memory: transient credentials or user PII offloaded into files or stores that outlive the task and its access context - offloading substrates need retention and redaction policy, because unlike the context window, they do not forget on their own.

Implementation notes

  • 01Decision table to encode: plans and to-dos → scratchpad file, re-read every turn; intermediate artifacts (logs, datasets, drafts, diffs) → files, path + summary in context; learned facts and preferences → structured memory store (LangGraph BaseStore namespaces / LangMem semantic memory, or Anthropic's file-based memory tool directory); behavior rules → procedural memory or instruction files (CLAUDE.md-style); conversation history → compacted summary in window + full transcript in storage; large-N episode recall → vector index, and only that.
  • 02On Claude: the memory tool (public beta on the Claude Developer Platform, per anthropic.com/news/context-management) gives you the file-based substrate managed by the model itself, and pairing it with context editing is the configuration Anthropic measured at +39% on agentic search. Claude Code users already have the pattern built in: CLAUDE.md for standing instructions, auto-memory directories persisted across sessions.
  • 03On LangGraph: use the built-in store for cross-thread memory and add LangMem for extraction - its docs (langchain-ai.github.io/langmem) split hot-path memory ops (agent reads/writes during the turn) from a background manager that extracts and consolidates after the fact; prefer background for anything not needed this turn, because in-loop extraction taxes every response with judge-style LLM calls.
  • 04Make offloading cheap for the model: give it write_note/read_note or filesystem tools with short names and obvious semantics, and put the manifest in the system prompt region so the agent always knows what it has externalized. Adoption of the pattern by the model tracks tool ergonomics more than prompt exhortations.
  • 05Enforce restorable compression mechanically: the compaction step must emit references (paths, URLs, memory keys) for everything it drops, and a linter on the compacted summary can verify no reference-free deletions of tool outputs occurred.
  • 06Set retention by substrate: scratchpad deleted at task end after consolidation; memory store entries carry TTLs or review dates; transcripts archived per compliance needs. Offloaded state is a data asset and a liability - see our agent audit-logging checklist for the compliance half.
  • 07Measure the win on your own traffic: log tokens-per-turn and task success before and after enabling offloading. Anthropic's 84%/39% figures are the vendor's evals; your ratio depends on tool-output sizes and task length, and the per-turn token curve flattening is the signature that the pattern is working.