A Comparison of AI Agent Harnesses in 2026

I’ve spent the last few months on Helix-Org, my attempt at rebuilding an organisation as AI agents. The first version had me writing hyper-specific agents in code, each one specialising in a single job. It worked, and it became tiresome: every new job meant another small program to write, test and maintain. Eventually I tried describing one of them in Markdown instead and handing it to a harness. It did the same job. Markdown is now code.

DeepSeek shipped an agent harness in developer preview on 13 August 2026, and it collected 95,386 GitHub stars in about two days, one of the fastest adoption curves GitHub has recorded. The thing I stumbled into has a name, a plugin standard and nine products worth choosing between. We run seven of them. Which one you pick matters less than which layer you need, and no feature grid answers that.

What a harness is, and where the framework sits

An agent harness is the code around the model that turns it into something that acts: the loop that keeps calling the model, the tools it is allowed to run, the sandbox those tools run in, the memory that survives a restart, and the rules that decide what reaches the context window. LangChain puts it more bluntly. A harness is every piece of code, configuration, and execution logic that isn’t the model itself. An agentic framework is the layer above, where you define and orchestrate several agents with explicit state and branching. Frameworks compose agents. Harnesses run them.

The stack has four layers:

• the model, which reasons;
• the harness, which runs one agent;
• the framework, which composes several;
• the platform, which runs many harnesses across a team over time, with durable execution, cost attribution and governance.

MongoDB sets out the same split, which is worth having from outside this argument, though it is a database vendor arguing for a database-shaped platform.

What goes inside a harness is now reasonably settled. Sebastian Raschka’s breakdown of a coding agent names six components: live repository context gathered before the first prompt, a prompt split into a cached prefix and a variable suffix, predefined tools with validation and path-based access control, clipping and compression to keep the context small, a structured session memory held apart from the full transcript, and bounded subagents with read-only permissions and a recursion limit. That is the well-explored part. Sandboxing, persistence, connectivity and triggers sit outside it, and they are where the engineering time goes.

The harnesses worth comparing

These are the nine, checked on 19 August 2026. The fleet column marks the seven we run as sandboxed desktops inside Helix, our own agent-fleet platform, which is where the judgement in the last two columns comes from.

Two of those cells are ours rather than anyone’s documentation. Qwen Code is the best harness we have used for Qwen models and the worst for anything else. OpenCode is the one that gets open-weight models doing useful work on hardware we own. Claude Code and Codex we rate highest for day-to-day coding on their own vendors’ models, and we run them against each other because they disagree.

Aider is missing on purpose. It is a git-native pair programmer built for short supervised runs, and the table is about tools that run long and unattended.

“OpenHarness” belongs to at least four unrelated projects, including an MIT Python harness from HKUDS released in April 2026, a TypeScript guardrails library, a universal API that runs one agent’s code across other harnesses, and a deep-agents project. The open-source harness most people mean is OpenHands, which has no “harness” in its name at all.

htek.dev keeps a living comparison of more than twenty agent platforms with pros, cons and a best-for line each, last updated in July 2026. Nine rows here because these are the choices that come up, and because most of them are in our own fleet.

Changing the harness moves more than changing the model

Vercel stripped 80% of the tools out of an agent and watched its success rate go from 80% to 100% on the same model, with tokens more than halved and latency down from 724 seconds to 141. LangChain took its coding agent from 52.8% to 66.5% on Terminal Bench by changing only the harness. Princeton’s CORE-Bench recorded one model scoring 42% under one scaffold and 78% under another. Harvey more than doubled the accuracy of its legal agents through harness work alone. All four are collected by MongoDB, April 2026.

On Terminal-Bench 2.0, Letta Code scored 59.1% running Claude Opus 4.5, against Claude Code’s 41.6% on the same model, as Han Lee read the leaderboard in May 2026. A third-party harness beat the first-party one on its vendor’s own model, and Lee’s explanation is why: the benchmark rewards durable memory, Letta is built around a memory substrate, and Claude Code is thin on memory by design. Letta’s own published run is older and more modest, 42.5% with Sonnet 4 in under 200 lines of code, August 2025.

Raschka scopes the conclusion to models of similar raw capability: given that the vanilla versions of current models are much of a muchness, the harness “can often be the distinguishing factor that makes one LLM work better than another”. Without that condition the line promises more than he wrote.

What each one assumes about your stack

Feature counts are the wrong axis. What separates these tools is what each one assumes you have already decided, and the first assumption is the model. Two of the nine run one vendor’s models and nothing else, which makes the harness a downstream consequence of a contract signed months earlier. The names mislead here: Qwen Code runs anyone’s models, Codex runs only OpenAI’s, and neither is what the label suggests.

The second assumption is where the thing runs. Claude Code and Codex are managed products on a subscription. OpenCode, Qwen Code, DeepSeek Harness, Goose and OpenHands are yours to host, which means the sandbox, the credentials and the audit trail are yours to build. Zed Agent assumes you have adopted an editor. Pydantic AI Harness assumes something else again: that there is already a Python service, and the agent lives inside it.

The third is who gets to change the agent’s behaviour. A CLI steered by Markdown puts that in reach of anyone on the team who can write a document. Typed Python puts it back with the people who can deploy. An editor-native agent puts it wherever the editor already is. That is an organisational choice dressed as a technical one, and it outlives the tool.

The features, meanwhile, are converging. Zed Agent, OpenHands and DeepSeek Harness all speak the Agent Client Protocol, so they drive each other, and DeepSeek Harness will run Claude Code or Codex as child processes if you have them installed. The Linux Foundation stood up an Agentic AI Foundation in 2026 anchored by three donations: Anthropic’s Model Context Protocol for tools, OpenAI’s AGENTS.md for instructions, and Goose itself. Addy Osmani had already noticed the effect, that the harnesses “look more like each other than their underlying models do”. Compare on features and the comparison expires. Compare on assumptions and it holds.

Why this layer appeared when it did

The vocabulary settled in about five months. LangChain published its anatomy of a harness on 10 March 2026, Raschka his components on 4 April, Osmani his harness engineering on 19 April, and on 23 June Pydantic shipped a breaking v2.0.0 built round the idea. Pydantic’s own changelog calls it a “harness-first design with capabilities as a core primitive: a single, composable unit that bundles an agent’s tools, hooks, instructions, and model settings”. A framework absorbing harness features into its core is the clearest evidence the layer is real, and the Pydantic AI Harness documentation listed more than fifty capabilities when we checked it on 19 August 2026. For what Pydantic AI, LangGraph, CrewAI and DSPy are as frameworks, and what the v1-to-v2 break says about trusting a stability promise, see our comparison of open-source LLM frameworks.

Then DeepSeek made the loop itself a plugin. A category that had no agreed name in 2025 now has an MIT reference implementation, a plugin standard, and an incumbent to displace.

This layer is unfinished. Manus rebuilt its harness five times in six months, LangChain re-architected its agent three times in a year, and Vercel deleted most of its agent’s tools, all collected by Philipp Schmid in January 2026. Schmid’s diagnosis of why the leaderboards miss it: “A 1% difference on a leaderboard cannot detect the reliability if a model drifts off-track after fifty steps.”

Buyers found the subject before the answers did. In our own Search Console, over the ninety days to 19 August 2026, forty-eight distinct queries containing “harness” landed on a single one of our pages, at positions five to twenty, with zero clicks between them.

The case that harness work is throwaway

Han Lee calls the harness hidden technical debt: “The good teams have built a lot of it. The bitter part is that almost all of it is going to dissolve into the next generation of models.” Every model release retires a stack of workarounds somebody was proud of.

Lee’s own counter-example sits in the same post. Letta beat Claude Code on Anthropic’s model by investing in memory, which is the dimension the model still handles badly. So the test for any piece of harness work is whether a better model would make it unnecessary. Context compaction, probably. Retry logic and output clipping, probably. Sandboxing, permissions, spend limits and audit trails, no: those encode what your organisation will allow, not what the model can do, and no model release repeals them.

DeepSeek Harness makes the same point uncomfortable from the other side. Standardising on a developer preview that shipped four days before we looked at it is not a conservative act. It is in our fleet because it is interesting, and we would not put a client’s release process behind it yet.

Osmani’s ratchet is what keeps the rest honest. “Anytime you find an agent makes a mistake, you take the time to engineer a solution such that the agent never makes that mistake again”, with every rule in the instruction file traceable to a real failure and no rule added before one. A harness built that way is a record of what actually went wrong, which survives a model upgrade better than a harness built from predictions about what might.

When a framework earns its place

A framework earns its place on something narrower than “more than one agent”. It gives you typed contracts on what goes in and comes out, state you can inspect and resume, and branching you can reason about before it runs. If the path through the work can be drawn, drawing it is cheaper than hoping.

The same rule applies here as anywhere else in machine learning. If a rule-based engine can make the decision, use the rule-based engine, and keep the model for the decisions that are genuinely fuzzy. A framework is the rule-based engine of this stack: you write the branches, so you get the determinism. A harness is for when the path cannot be written down in advance. Our build guide puts the trade in one line: “By offloading the burden of getting it done to the agent, you’re explicitly accepting some uncertainty to deal with more diverse requests.”

That boundary may not survive long. Pydantic AI’s v2 already pulls harness features into a framework’s core, and the expectation on the framework side is that harness products absorb the abstraction layer. Choose for the next eighteen months, not the next five years.

In our own engagements we rarely reach for an agent framework any more, though we still work in whichever one a client has already committed to. Left to ourselves it is custom code or a harness, and the line between those two is a counting exercise: a harness pays for itself once there are around ten distinct jobs that can be described agentically in Markdown. Below that number you are maintaining a harness to run one or two things, which is the tedium I started with. Above it, the harness is the cheapest way to add the eleventh.

A proof of concept is the exception. Reach for an agentic framework there, or use the Claude subscription you already pay for, and save the harness for when the job list is real.

The layer above the harness

One harness per engineer works until it does not. The moment several agents run against the same systems, on machines that are not yours, and somebody senior asks what they did last week, the question stops being which harness and becomes who can see the fleet. MongoDB defines that layer as the infrastructure that runs many harnesses across teams over time, with durable execution, cost attribution, governance and data integration. Its companion piece puts the failure mode plainly: “A platform missing any of these produces harnesses that appear to work in controlled conditions and fail under production load.”

We sell into that layer. Helix is co-developed by the Winder.AI team: an agent control room where every thread boots a containerised desktop on your own infrastructure, with an IDE, a terminal, a browser and the repository cloned onto its own branch, so a team watches agents work instead of reviewing a diff blind afterwards. It is also why the fleet column exists. The frameworks comparison has the longer entry, with licence, deployment options and pricing.

What it costs is infrastructure you have to run, and one more system to operate at a point when the agents themselves are still changing monthly. One engineer, one harness, one repository is a fine place to stay for a year.

An internal platform is where the harness pattern makes most sense, because the job list is long by construction. Helix-Org is the case I know best: rebuilding an organisation as agents produces far more than ten jobs to be done, and every one of them is a Markdown file rather than a program.

Choosing a layer on a real project

Four outcomes are available, and the layer test picks between them:

• a harness alone, which is where single-agent work usually lands;
• a harness with a framework inside it, when part of the job is deterministic enough to draw;
• a framework alone, for a narrow, high-stakes task where the actions are dangerous and the path is known;
• a platform, when the fleet outgrows one person’s laptop.

Each step down that list hands the agent more freedom, and freedom is what you pay for in evaluation. The more paths an agent can take, the less a passing demo tells you. Whichever layer you pick, the thing that tells you the choice was right is an evaluation suite, and it is the piece most often missing from the agent work we are handed. A harness change that improves one demo and breaks two others looks like progress until somebody measures it. Making these things auditable and observable is the genuinely difficult part, and we set out how we approach it in our guide to evaluating an agent before you ship it. Our build guide treats evaluation as a first-class part of the build for the same reason, and we published a working agent with its eval harness and committed results so the shape is not a matter of opinion.

We make this call on live client work. AI agent development is where the harness, the sandbox and the evaluation get built, LLM consulting is where the model and vendor questions upstream of it get settled, and the position we take on the whole subject is on that first page: we say no to frameworks that fit your problem poorly, even when there is no commercial reason to. If you want the decision walked through against your own stack, get in touch.

My own day-to-day is still Claude Code and Codex, run against each other on the same work, because two agents that disagree produce better software than either alone. Qwen 3.8 under OpenCode now does my back-office work, with its own difficulties, and the hyper-specific agents I wrote in code for Helix-Org are gone: what replaced them is a directory of Markdown files and something that reads them. The last mile to running all of it on our own hardware is not capability any more. It is knowing what the agents did.