Harness engineering interview questions: 22 questions with answers for an AI engineer

How to prepare

Harness engineering interview questions usually go from general to specific: what a harness is, what it consists of, how you diagnose a problem, how you measure the effect. The interviewer looks for engineering thinking: the ability to tell a model problem from an environment problem and to propose a verifiable solution. Below are 22 questions with answers, grouped by topic.

Basics

1. What is an agent harness?

The harness is the whole environment in which an AI agent works, except the model itself: instructions, tools, the execution environment, state storage and feedback loops. The model makes decisions, while the harness defines what it knows, what it can do and how it learns about the result. The same model in different harnesses shows very different quality.

2. How does harness engineering differ from prompt and context engineering?

Prompt engineering works with the wording of a single request, context engineering works with what enters the model's window at each step. Harness engineering includes both and adds what cannot be solved with text: access to tools, a reproducible environment, state between sessions and machine verification of the result.

3. Name the five harness subsystems.

Instructions (AGENTS.md, CLAUDE.md, skills, docs), tools (shell, tests, linter, browser, MCP), environment (dependencies, runtime, Docker, worktree), state (PROGRESS.md, feature list, decisions) and feedback (make check, E2E, hooks, evaluator). Missing any one of them does not make things "a bit worse" but produces a systematic class of failures.

4. Why is replacing the model with a stronger one often not enough?

Because many failures are not related to the model's intelligence. A strong model will not get access to a command it was not given, will not remember yesterday's session without a state file, and will not learn about a broken migration if nobody ran it. It will not guess the conventions of a specific team either.

Diagnostics

5. The agent breaks a rule written in CLAUDE.md. What do you do?

First check whether the agent can find the rule at all: a long file with dozens of items dilutes the important one. Shorten the entry file to a table of contents and move the details into docs loaded on a condition. If the rule is critical, turn it into an executable check — a linter, a test, a grep guard or a hook — so the violation becomes visible mechanically rather than depending on the model's attention.

6. How do you tell from a symptom which subsystem is broken?

Breaking known conventions points to instructions. The agent "wants to but cannot" points to tools. Every session starting with fixing the environment points to the environment. A new session not knowing where things stopped points to state. "Done" not matching reality points to feedback.

7. Which subsystem do you start building the harness with, and why?

With feedback. Until there is a command that mechanically answers "works or not", the effect of any other change cannot be measured. Next usually come instructions, state, environment and tools, but the order is empirical and depends on what is broken most severely.

8. How do you measure the contribution of an individual harness component?

By ablation: fix the model and the set of tasks, switch off one subsystem at a time and watch the drop. But it is better to start with a failure log labeled by layer — it points to the bottleneck faster. A zero drop does not mean the component is useless: it may not be exercised by these tasks or may be redundant.

Instructions and context

9. How do you structure a good AGENTS.md?

As a router, not a textbook: what the project is, how to run it, how to check it, the Definition of Done and a short list of mandatory constraints. Details go into separate documents with links of the form "read when the task involves payments". The file states invariants ("data is validated at the boundary") rather than dictating the implementation and libraries line by line.

10. Why do you need sub-agents?

For context isolation. A side task — searching the codebase, a review, a check — runs in a separate context and returns only its output. The main thread stays uncluttered, and a verifying sub-agent does not inherit the executor's reasoning, which matters for an honest assessment.

11. What is harness debt?

The accumulated divergence of the harness from reality: outdated rules, commands that no longer work, documents about modules that no longer exist. It behaves like technical debt — it builds up unnoticed and is paid for with the agent's failures. The harness must be reviewed regularly, not configured once.

State and sessions

12. How do you deal with the agent's amnesia between sessions?

Keep the state in the repository: PROGRESS.md (status, what was done, blockers, next steps), DECISIONS.md (decision, reason, rejected options), a feature list with statuses, frequent commits. In the instructions, fix the start order: read the progress, run the environment check script, take the active feature.

13. Why the WIP=1 rule?

When an agent starts several features at once, the state gets smeared: none is finished, changes are mixed, rollback becomes difficult. One active feature gives a clear checkpoint, clean commits and simple recovery after a failure.

14. What is a cold-start test?

A check of whether a new session, without any prompts from a human, can answer basic questions: what the project is, where things stopped, which feature is active, how to check that everything is fine. If the agent answers from the files without guessing, the state subsystem works.

Verification

15. Why do green unit tests not prove that a feature works?

A unit test isolates a module and mocks its dependencies, so it does not see the seams: interface mismatches, state propagation (the schema changed, the cache is stale), resource leaks, configuration and environment problems. The defects that unit tests catch are a subset of those integration tests catch, and integration tests catch a subset of what E2E can catch.

16. What is the verification gap and how do you count it?

The share of tasks the agent declared done but independent verification did not confirm. You take 10 tasks in a row and mark both facts for each. The metric gives a baseline: after harness changes it is recalculated on a similar set of tasks.

17. How do you write a Definition of Done for an agent?

In AGENTS.md, as a list of verifiable items: the check commands finished with code 0, there is an E2E test for the scenario, migrations apply, the progress is updated. It helps to state explicitly what does not count as done: "looks correct", "unit tests pass", "works on my machine".

18. Why must the agent not set the "done" status itself?

Models tend to overrate their own work, and self-assessment happens in the same context where the decision was made. The passing status should be set by a verification script based on the results of the runs, recording the commit hash as evidence.

19. How do you set up an independent evaluator?

A separate agent or session that sees only the requirement, the diff and the results of the runs, but not the executor's reasoning. It evaluates against a rubric (correctness, architecture, tests, error handling), and every assessment refers to a file, a line or a log. Preferably use a different model, so that the blind spots do not coincide.

20. How do you make an architectural rule executable?

Three levels: a grep guard (fast, pattern-based), a linter rule that understands the import graph (import-linter, dependency-cruiser) and an architectural test in CI. All of them plug into make check, fail with a non-zero exit code and print WHY and FIX, so the agent fixes the violation itself. The trigger for a new check is the second identical comment in a review.

Autonomous work

21. What is needed to let an agent work for hours?

An unambiguous goal, a monotonic progress metric (for example, the number of features in passing), a budget per feature in time and attempts, automatic rollback on regression, and a list of actions that require a human: migrations, infrastructure, money. Plus isolation: a sandbox or a separate git worktree, so that a mistake does not touch the working environment.

22. When is one agent loop not enough, and how do you build a graph of roles?

When the executor starts checking itself and gets stuck. Then the task is split into roles: implementation, review, tests, merge. The review gets a clean context and a rubric, and the merge is performed by code when review == pass. If the review fails, the feature goes back to implementation, and after several failures it is restarted with a clean context — which means the problem is in understanding the task, not in the code.

Summary

In harness engineering interviews, the strongest answers are about things that can be verified: which files, which commands, which metric. Keep in mind the five subsystems, the verification gap and the rule "readiness is set by a script". All of these topics come together into a single harness on an end-to-end project in the harness engineering course.