Harness Engineering Is the Product: What 11 Production Coding Agents Actually Share

An agent is a model plus a harness. The model gets the attention. The harness is where the engineering is.

In July 2026 four researchers published a source-code study of eleven production coding agents: Claude Code, Codex CLI, Gemini CLI, Mistral Vibe, OpenHands, Aider, Mini-SWE-Agent, Hermes, Pi, OpenCode and OpenClaw, with Databricks' Omnigent as a meta-harness contrast. Roughly four million lines of Python, TypeScript and Rust. They found seven canonical subsystems and 29 recurring design patterns. Two absences stand out more than any pattern.

No runtime imports a general-purpose agentic framework. None retrieves code with vector embeddings.

If you have been sold a framework and a vector database as the way to build agents, that is worth sitting with. The people shipping the most-used agents in the world built neither.

TL;DR

• Seven subsystems appear in every production harness: agent loop, LLM integration, tools and actions, memory and context, safety and permissions, orchestration, extensibility.
• The field runs on hand-rolled async loops and deterministic retrieval: ripgrep, tree-sitter, glob, and auto-discovered Markdown context files. Not LangChain, not embeddings.
• Skills ship in 9 of 11 systems, MCP in 8, ACP in 6. Behavioural policy is migrating from prompt prose into configuration.
• The thesis: in the first half of 2026 the coding harness completed a turn from tool to platform. Harnesses became importable SDKs, and the interesting product decisions moved into the harness.

The seven subsystems

The study's map is the most useful artifact in it, because it gives you a vocabulary for a thing most teams build by accident. Here is each subsystem, what it does, and the pattern names the paper attaches to it that I have found most useful.

1. Agent loop. The cycle that takes a task, calls the model, executes what it asked for, feeds the result back and decides whether to stop. The named patterns tell you how mature this has become: verify-on-stop guards that check the work before the loop is allowed to end, log-as-queue loops that treat the transcript as the work queue, middleware-pipeline loops that wrap each turn in interceptors. This is where loop engineering lives, and it is the subsystem I see teams underinvest in most.

2. LLM integration. Everything between the loop and the provider: model catalogs, prompt assembly, cache boundaries, retries. The patterns are all about cost and caching: cache-dialect fanout to match each provider's prompt cache, prompt-cache-sharing forks so parallel branches reuse a cached prefix, modular prompt assembly with explicit cache boundaries. If you thought prompt caching was a provider detail, the harnesses disagree; it shapes how they build prompts.

3. Tools and actions. The action space. Shell, file edits, search, browser, tests. The pattern that matters is deferred tool loading: tool catalogs stay out of the context until they are needed. The Hacker News debate over bash-only versus predefined tools is a debate about this subsystem, and the empirical study that prompted it found the answer is conditional on the model.

4. Memory and context. Retrieval, compaction, persistent context files. This is the subsystem with the most surprising finding: deterministic retrieval everywhere, embeddings nowhere. Lineage compaction, threshold compaction and pluggable condensation are the named strategies for keeping a long session inside the window. Agent-maintained memory pipelines are how the harness lets the model write to its own memory.

5. Safety and permissions. Sandboxes and the rules for what may run. Docker, Apptainer and remote sandboxes in OpenHands; native OS sandboxing in Codex and Gemini CLI; optional worktrees elsewhere. Syntax-aware command permissioning, parsing the shell command before deciding whether to allow it, and scope-based authorization are the patterns. This is the subsystem that decides whether your agent is a tool or a liability.

6. Orchestration. Sub-agents, parallel work, session trees. Session-tree version control and recursive composition are the named patterns. Note how little of this there is compared with the marketing around multi-agent systems; the harnesses are conservative here, and the research on multi-agent overhead suggests they are right to be.

7. Extensibility. Skills, MCP, ACP, plugins, hooks. The adoption numbers live here: skills in nine systems, MCP in eight, ACP in six. Capability-gated snippets and polymorphic prompts are how a harness changes its behaviour per capability without forking the prompt.

Why nobody imports a framework

This is the finding people argue about, so it is worth being precise. The study does not say frameworks are bad. It says that across roughly four million lines, no production coding runtime imports one. The runtimes are hand-rolled async loops.

I think there are three reasons, and none of them is snobbery.

First, the loop is the product. A coding harness's behaviour under failure, its compaction policy, its permission model and its cache strategy are the things users experience as "this agent is good". A general framework abstracts exactly the parts you need to own.

Second, the loop is small. A minimal harness is a few hundred lines. Mini-SWE-Agent is in the study precisely because it proves that. The framework's value proposition, saving you from writing the loop, saves you from writing the smallest part.

Third, the frameworks were built for a different problem. Chains of prompts over documents. A coding harness is a long-running process that edits a filesystem under a permission model. The abstractions do not line up.

The same logic explains the second absence. Vector retrieval over code answers "what text is similar to this query". A coding agent needs "where is this symbol defined, what calls it, what does the build say". Ripgrep, tree-sitter and a Markdown file at the repository root answer those questions exactly, and they never return a confidently wrong chunk.

Behaviour is moving out of the prompt

The study tracked the systems across a quarter and noticed two evolutions. Convergence became imitation: the harnesses copied each other's patterns, which the paper calls harness mimicry. And behavioural policy migrated from prompt prose to configuration.

That second one is the quiet big deal. A year ago, "do not edit files under vendor/" lived in a system prompt and was honoured on a good day. Now it lives in a permission configuration the harness enforces before the model sees the request. The prompt says what to do; the configuration says what may happen. That is the difference between asking and controlling, and it is why the policy layer has become its own product category.

Anti-pattern: Policy in prose

Why tempting: Writing 'never run destructive git commands' in the system prompt is one line and it usually works.

Failure mode: Usually. The model is talked out of it by a tool result, a file comment, or a long session that pushed the instruction out of the effective context. The rule had no enforcement point.

Better pattern: Put the rule where the harness can enforce it: a permission configuration with syntax-aware command parsing, or a policy proxy in front of the tools. Keep the prose as explanation, not as the control.

Guardrail: A test that runs the forbidden command through the harness with a persuasive prompt and asserts it was blocked at the enforcement point, not declined by the model.

The platform turn

The study's thesis is one sentence: in the first half of 2026 the coding harness completed a turn from tool to platform.

The evidence is structural. Harnesses became importable SDKs. Framework vendors shipped harnesses of their own. Marketplaces for skills appeared, along with switching-cost tooling and enterprise governance layers. ACP acquired a third role, harness hosting, in which one harness runs another as a backend. And Omnigent showed up as an orchestration layer above harnesses that treats them as interchangeable components behind a common API with unified policies and cross-harness sandboxing.

For builders the consequence is direct. If you are writing a product on top of an agent, you are choosing a harness the way you used to choose a web framework. The subsystem map above is your evaluation rubric. The questions to ask are the ones the map raises: How does it compact? What can it retrieve? Where is the permission enforced? Can I host it over ACP, and can it host others? What happens to my policies if I switch?

The named patterns, by subsystem

The paper's twenty-nine patterns are its most quotable content and its least read, because they are spread across a hundred pages. Here are the ones I have found myself using as vocabulary, grouped by the subsystem they belong to, with one line each on the problem they solve. The names are the paper's; the one-liners are mine.

Agent loop. Verify-on-stop guards: the loop is not allowed to end until a check, usually the tests, has run against the work. Log-as-queue loops: the transcript is the work queue, and the loop pulls the next item from what it has already written. Middleware-pipeline loops: every turn passes through a chain of interceptors that can log, redact, rate-limit or veto before the model or a tool sees it.

LLM integration. Cache-dialect fanout: the same prompt is assembled differently per provider to hit each one's prompt cache. Prompt-cache-sharing forks: parallel branches of work are arranged so they share a cached prefix rather than paying for it twice. Polymorphic prompts: the prompt changes shape by capability without duplicating the whole thing. Modular assembly with cache boundaries: the prompt is built from parts with explicit cut points where the cache can take over.

Tools and actions. Deferred tool loading: the tool catalog stays out of the context until a tool is actually needed. Syntax-aware command permissioning: a shell command is parsed before it is judged, so rm inside a quoted string is not mistaken for a deletion, and a deletion is not hidden by quoting.

Memory and context. Agent-maintained memory pipelines: the model writes to its own memory through a defined path rather than the harness inferring what to keep. Lineage compaction: compaction keeps the chain of decisions that led here and drops the rest. Threshold compaction: compaction fires at a defined fill level rather than when things break. Pluggable condensation: the strategy for shrinking context is a component you can swap.

Safety and permissions. Scope-based authorisation: permissions are granted per scope, a path, a repository, a session, rather than per tool globally. Capability-gated snippets: pieces of prompt or behaviour appear only when a capability is present.

Orchestration. Session-tree version control: sessions branch and merge like commits, so a sub-task can be forked and its result folded back. Recursive composition: a harness can run itself as a sub-agent with a narrowed scope.

Extensibility. Harness mimicry: the paper's name for convergence becoming imitation, where one harness adopts another's conventions so that users and skills transfer. Event-sourced conversation: the conversation is stored as an append-only event log from which the current state is derived.

If you recognise half of these from your own code without having had names for them, that is the point of the paper. If you recognise none, that is a different and more useful finding.

The controlled companion study

The source-code study tells you what production harnesses are. A second paper, submitted two months later, tells you what parts of a harness do, by varying them. An Empirical Study of Harness Design for Coding Agents keeps the execution loop fixed and changes three components, planning, action space and context management, across four models on SWE-bench Verified and Terminal-Bench 2.1, in 176 matched settings that span five context-management strategies and four context-window budgets.

Its findings line up with the source-code study in a way that is more convincing than either alone.

On context management: it becomes more valuable as the context-window budget tightens, and most of the benefit comes from preventing context-overflow failures. Staging rule-based elision before LLM-based summarisation gave the strongest overall efficiency. Making elided content recoverable yielded no accuracy gain. That is an empirical endorsement of exactly the compaction patterns the eleven harnesses evolved by hand: threshold-based, rules first, summarisation second.

On planning: it shifts from an accuracy scaffold for weaker models to a cost saver for stronger ones, with little change in accuracy. So planning is not a universal good; it is a component whose value depends on the model it is paired with.

On the action space: predefined tools improve performance for models with weaker bash proficiency, while bash-capable models operate effectively with a bash-only interface at substantially lower cost, especially on command-line-centric tasks. Read that next to the source-code finding that no production harness retrieves with embeddings and all of them lean on shell tools, and the picture is consistent: the harnesses that ship are the ones that let capable models use the shell.

The two papers together give you something neither does alone: a map of what exists, and evidence about which parts of it matter for which models.

Objections from the thread, answered

The Hacker News discussion of the empirical study ran to 225 points and 59 comments, and the objections in it are the ones you will hear in your own team.

The models are old. The study used Nemotron-3 and Mistral models rather than this quarter's frontier releases. True, and the defenders' reply holds: dismissing empirical evidence without counter-evidence is not an argument. The right response is to run the same matched comparison on the model you actually use. The study's contribution is the method and the finding that effects are conditional on the model; that finding gets stronger, not weaker, as models change.

Big context windows make context management moot. For a hosted frontier model with a very large window, maybe. The study is explicit that the benefit concentrates where budgets are tight, and tight budgets are the local and cost-constrained case, which is a large share of production. It also found no accuracy gain from making elided content recoverable, which suggests that what you drop matters less than that you drop it before overflow.

"Minimal" harnesses benchmark differently from each other. Yes, and that is an argument for the source-code study, not against it. SWE-agent, Pi and others are all small, and they differ in exactly the subsystem details the paper catalogues. Implementation detail is the product.

MCP was a mistake if bash is enough. The bash-only result is about the action space for a capable model on a local machine. It says nothing about a shared service that needs audit, revocation and identity, which is what the MCP defenders in the same thread pointed out. Different problems, different answers, and the source-code study's eight-of-eleven MCP adoption says the harness authors think so too.

Sandboxes: what the eleven actually do

Safety and permissions is the subsystem where the eleven diverge most, and the paper's inventory is a useful menu.

OpenHands runs the agent inside Docker or Apptainer containers or on a remote sandbox, so the blast radius is the container. Codex and Gemini CLI use native operating-system sandboxing, so the blast radius is what the OS policy allows, with no container to escape. Several harnesses offer an optional git worktree, which is not a sandbox at all but a cheap isolation of the working tree that makes a bad change easy to discard.

None of these is the policy layer. A container limits where an agent can reach; it does not decide whether a given tool call is allowed. That decision lives in the syntax-aware command permissioning and scope-based authorisation patterns inside the harness, or in a policy proxy in front of it. The teams that get this right use two of the three: an isolation boundary from the list above, plus a policy decision on every call.

Scoring your own harness

A scoring sheet, one row per subsystem, that we use when evaluating a harness to build on. Score each 0 to 3: absent, ad hoc, present, or present and measured.

Any harness scoring 0 on safety and permissions is disqualified regardless of the rest. Any harness scoring 3 on orchestration and 1 on everything else has been built for a demo.

What to do with this on Monday

Three things, in order.

Draw your harness against the seven subsystems. Most teams find one subsystem missing entirely, usually safety and permissions or orchestration, and one that is doing the job of three. The map makes the gap visible.

Move one policy from prose to enforcement. Pick the rule you most rely on, find where it lives, and give it an enforcement point. Then write the test that proves the enforcement point holds under a persuasive prompt.

Replace one embedding search with deterministic retrieval. If your agent searches code with vectors, try ripgrep plus a repository map for a week and measure. The study's population of eleven suggests you will not go back.

The model is the engine. The harness is the car. The study is the first real look under the hood of the cars people actually drive, and the most striking thing about them is how little of the marketed machinery is in there.