What is harness engineering: how to make an AI agent a working tool

Harness engineering in plain words

Harness engineering is the discipline of building an environment around a language model in which it reliably does real work. The word "harness" means the gear that directs a strong animal. The model is powerful, but on its own it does not know where to go, which tools to use or when to stop. The harness channels that power.

The shortest formulation is:

Agent = Model + Harness

The model is the weights, the ability to reason and to write text or code. The harness is everything else: system instructions and files such as CLAUDE.md or AGENTS.md, a set of tools (shell, file editing, MCP servers), a sandbox where the agent runs commands, state files that survive between sessions, and tests, linters and hooks that tell the agent when it made a mistake. Remove the harness and you are left with a chat that can only answer in text.

Why the agent is "a senior one day and a junior the next"

A familiar picture: in one repository the agent works like an experienced developer, in another it gets lost, forgets yesterday's decisions and reports "done" about a feature that does not work. The first reaction is to switch to a newer model. This usually does not help, because the model is the same in both cases. The difference is the environment.

This is the main shift in thinking with which harness engineering begins: the agent is not broken, what surrounds it is. The agent cannot run the tests if it has no access to them. It will not remember what it already tried if that is written nowhere. It will not learn that a feature is broken if no check tells it so. These are not the "character" of the model but defects in the design, and they can be fixed systematically.

Typical symptoms of a poor harness

• Every new session starts with fixing the environment: wrong dependencies, the database does not start, variables are missing.
• The agent repeats solutions that were already rejected, because it does not remember the previous session.
• A rule is written directly in the instructions, but the agent ignores it.
• "The tests are green", yet the feature is broken, because the tests checked the wrong thing.
• The agent takes on three tasks at once and finishes none of them.

What a harness is made of: five subsystems

A common mistake is to think of the harness as a single prompt file. In reality it is five mechanisms, and each one breaks in its own way.

Instructions

Instructions answer the question "will the agent find the rule it needs". A good entry file is a map, not a textbook: a short table of contents with check commands and hard constraints, and the details in separate documents that the agent opens when needed. Good instructions state invariants ("all SQL queries are parameterized") rather than dictating the implementation line by line.

Tools

The agent can only verify what it has access to. If it cannot run the linter, no rule saying "run the linter before committing" will work. Tools are connected through the agent's built-in abilities (reading and editing files, running commands) and through MCP, a protocol that gives the agent access to databases, browsers and task trackers.

Environment

The environment is where the agent runs commands. It should start with one command, be the same from session to session, and limit the damage of a mistake: a sandbox without access to production, a separate branch or working copy, and the ability to return everything to its original state within a minute.

State

A model has no memory between sessions: anything not written into the repository disappears together with the context window. So the goal, what has been done, the decisions made, dead ends and the next step must live in files. A simple test: open a new session, tell the agent "continue" and see whether it starts working or starts asking what to do.

Feedback

Feedback loops are the most important subsystem. Without them the agent is its own only judge and honestly considers the task finished. Tests, type checking, a linter, a hook that runs after every edit, an E2E scenario in a browser: all of these return a red signal to the agent while the mistake is still cheap to fix.

The harness is a loop, not a list

The five subsystems work as a closed loop. The model receives instructions, acts through tools in the environment, makes changes to the repository, and verification checks the result. A red result goes back to the model for fixing; a green one is recorded in the state. The state, in turn, updates the instructions for the next session.

instructions → model → tools → environment → changes
                ↑                              ↓
                └──── red ── verification ─────┘
                                     ↓ green
                                 state → (next session)

Remove any arrow and the loop breaks at that point. Without verification the agent does not learn about its mistakes. Without state the next session starts with archaeology instead of work. This is why investing in one subsystem does not compensate for a gap in another: a five-page AGENTS.md does not replace missing tests.

How harness engineering differs from prompt and context engineering

These concepts are nested inside each other rather than competing.

Prompt engineering is part of the instructions subsystem. Context engineering covers instructions and state: what the agent sees at each moment. Harness engineering adds what does not fit into the context window at all: access rights, the sandbox, tests, hooks, the work process. A well-written prompt does not help if the agent cannot physically run the tests.

Why a harness is needed: the mathematics of long tasks

It seems intuitive that a model which makes the right move in 95% of steps is almost perfect. But an agent's task consists of many steps, and errors accumulate. If the steps are independent, the probability of passing all of them without a mistake is the product of their probabilities:

0.95 ^ 10 ≈ 0.60
0.95 ^ 20 ≈ 0.36
0.95 ^ 50 ≈ 0.08

By the tenth step only about 60% of runs get through without a single error; by the fiftieth, fewer than one run in ten. Improving the model to 97% is expensive and not in your hands. But catching an error at the step where it happened and returning it to the agent is the harness's job. A feedback loop turns a chain where "one error drags the next" into a chain of short, fixable steps.

Example: the same agent in two repositories

Task: add a promo-code discount to a FastAPI payment backend.

Without a harness

The agent reads a couple of files, writes a function that calculates the discount and a unit test that checks this function in isolation. The test is green and the agent writes "done". It did not know that the final total is calculated in another module, and it could not start the database and run the whole checkout scenario. A day later it turns out that the discount is not applied in the cart.

With a harness

The root contains an AGENTS.md of a few dozen lines: how to set up the environment, which check commands are mandatory, where the payment architecture is described. The environment starts with docker compose up -d. The repository has a plan file with a completion criterion: "the E2E checkout scenario with a promo code passes". After every edit a hook runs the linter and type checker, and before finishing the agent must run the E2E test.

# AGENTS.md (excerpt)
## Verification — mandatory before saying "done"
make lint        # ruff + mypy
make test        # pytest, unit and integration
make e2e         # checkout scenario

## MUST
- Money is calculated in cents (int), never in float
- All SQL queries are parameterized
- Do not touch migrations already applied in production

The model is the same. But now the agent finds the module that computes the total through the project map, the E2E test fails on the first attempt, the agent sees that the discount did not reach the cart, fixes it and only reports after a green run. The difference in the result is entirely the work of the harness.

Where to start

1. Describe how the agent checks its own work: one to three commands it must run. Feedback gives the biggest effect, so start there.
2. Make the environment reproducible: it starts with one command and there is a quick way to roll back.
3. Create a state file: the goal, what has been done, decisions and dead ends, the next step.
4. Shorten the instructions into a map: check commands, hard constraints, links to detailed documents.
5. Keep a log of the agent's failures: for each failure, record which subsystem allowed it. The log will show what to fix next.

A useful practice is to rate each of the five subsystems from 1 to 5, spend half an hour on the weakest one, and run the same set of typical tasks before and after. This makes harness improvements measurable instead of "by feeling".

Summary

Harness engineering is the work not on the model but on everything around it: instructions, tools, environment, state and feedback loops. An agent's reliability is defined by the weakest link of this loop, so switching models rarely cures instability, while an added test or a plan file often does. All five subsystems and the assembly of the harness as a whole are covered in practice in the harness engineering course.