Building an AI Agent Harness From Scratch: AGENTS.md, feature_list.json, verify.sh and an Autonomous Loop
Harness engineering course finale: we build an AI agent harness on an end-to-end FastAPI payments project — AGENTS.md router, feature_list.json, PROGRESS.md, verify.sh, arch-check.sh, program.md, graph.md — and compare metrics before and after.
- You will see the harness in full — not a set of scattered techniques, but one coherent system of files.
- You will be able to copy the assembled harness structure into your own working project as a starting point.
- You will learn to compare the agent's metrics before and after implementing the harness — with hard numbers, not by eye.
Awal materi
We've spent twelve lectures building the harness piece by piece. Does it actually work as one system?
- AGENTS.md, feature_list., PROGRESS.md, the sprint contract and graph.md were written in different weeks
- Each artefact was verified on its own — never checked for clashes with its neighbours
- Today brings the only test that matters: does the system work when assembled?
Here's the question that's easy never to ask until the very last day of the course: if instructions, state, verification and the feature contract were each written one component at a time, what guarantee is there that they don't contradict one another inside a single repository? The WIP=1 rule from feature_list. can quietly clash with parallel nodes in the role graph. The Definition of Done might demand a step whose script no longer exists. Today we add nothing new — we take the five subsystems of the payments backend we've already written and switch the harness on as a whole, once, to see whether the structure holds.
This is what the repository looks like once the harness is finally finished
- Top level: AGENTS.md, PROGRESS.md, DECISIONS.md, program.md, graph.md
- .agent/: feature_list., session-handoff.md, clean-state-checklist.md, quality.md
- scripts/: verify.sh, arch-check.sh, init.sh, cleanup.sh — gates, not suggestions
- Next we go through each file and see what it actually does in the assembled whole
This is what the root of the payments backend looks like if you open it today rather than in week one of the course. Eleven files and one utility directory — and not a single one of them is there to tick a box: each closes a specific layer of failure we worked through along the way. We'll go through each in turn — first the instructions and the Definition of Done, then the feature list and the verification gate, then cross-session continuity, the architectural rule, the sprint contract, the autonomous loop and the role graph. And at the end — not a story, but figures: what changed in MTTR, the verification gap and the cost of recovery on the very same project.
AGENTS.md isn't a tips file — it's an 80-line router
- The top level answers only three questions: what it is, how to run it, how to verify it
- Everything specific belongs in docs/*, not in the router itself
- Every link to a documentation section carries a 'when' condition, never just a path
- Fewer than 100 lines — otherwise the router starts degrading for the same reasons as any other instructions file
The first thing an agent sees when it opens the repository isn't a thousand lines of instructions but a short router that sends it off in the right direction. Look at what AGENTS.md for the payments backend had become by the end of the course: it contains no topic in full — only commands, hard constraints and conditional links into docs/api-patterns.md, docs/db-rules.md and docs/testing.md. A condition such as 'read before changing models' beats a bare path — the agent sees not just a file but a reason to open it.
## Router: what / how to run / how to verify
- Stack: FastAPI + PostgreSQL + Stripe webhooks.
- Run: `make setup && make dev`. Verify: `make check`.
## Details — read on condition, not everything up front
- `docs/api-patterns.md` — required before adding any endpoint
- `docs/db-rules.md` — required before changing models or migrations
- `docs/payments.md` — required before changing payment logic or webhooks
- `docs/testing.md` — when writing new tests
Materi lengkap tersedia dengan langganan
15 slide lagi di versi lengkap.
Video, semua slide, catatan, tes, tutor AI, dan wawancara terbuka setelah masuk dan berlangganan.
Masuk atau daftar BerlanggananIsi materi
- We've spent twelve lectures building the harness piece by piece. Does it actually work as one system?
- This is what the repository looks like once the harness is finally finished
- AGENTS.md isn't a tips file — it's an 80-line router
- 'Done' means the checklist has been passed — not that it merely looks done
- feature_list.: a task list that knows how to say 'no'
- Who gets to write the word 'passing'? Only this script
- A new session doesn't ask 'what were we doing yesterday?' — it reads
- One script answers the question “is everything all right?” before the agent says a word
- One architectural boundary that is now physically impossible to violate
- The feature contract is written BEFORE the code — not as an explanation afterwards
- “Looks fine” is banned here — grading runs on a rubric backed by evidence
- How an agent drives the feature queue all night and decides for itself when to stop
- A single agent can no longer cope alone — meet the four-role graph
- The graph knows how to roll itself back — here’s where a feature goes when review objects
- ❓ Checkpoint: the assembled harness hides one conflict — find it
- 💡 Breakdown: a passing result from verify.sh doesn't mean 'the full Definition of Done has been met'
- MTTR down sevenfold, verification gap from 38% to 4%: here's the full report
- The harness is something you'll still be fixing long after the course ends
Artikel terkait materi ini
- Apa itu harness engineering: cara membuat agen AI menjadi alat kerja yang nyata
- Cheat sheet harness agen: komponen, file, masalah, dan daftar periksa peluncuran
- Pertanyaan wawancara harness engineering: 22 pertanyaan dengan jawaban untuk AI engineer
- Cara membuat agen AI andal: rencana bertahap dari pengukuran sampai kerja otonom
- CLAUDE.md dan AGENTS.md: apa itu, apa yang ditulis di dalamnya, dan mengapa agen mengabaikan aturan
Semua materi kursus ini
- What Is Harness Engineering? Why AI Coding Agents Fail Because of the Environment
- Error Accumulation in AI Agents: Why 95% Per-Step Accuracy Fails by Step 10
- AI Agent Reliability Metrics: MTBF, MTTR, Blast Radius and Little's Law
- Why AI Agents Fail in Production: Five Failure Layers and a Diagnostic Loop
- What an AI Agent Harness Is Made Of: Instructions, Tools, Environment, State, Feedback Loops
- Repository Map for AI Coding Agents: Project Structure and State Without Memory
- AGENTS.md Best Practices: Why a Giant Instructions File Fails and How to Build a Router
- AI Agent Memory Between Sessions: Session Handoff, PROGRESS.md and Bootstrap Contract
- feature_list.json for AI Agents: WIP=1, Feature Status and verify.sh Verification
- Agent Says 'Done' but the Feature Is Broken: Verification Gap and Definition of Done
- All Tests Pass, Feature Is Broken: E2E Tests and Architecture Rules for AI Agents
- AI Agent Observability: Sprint Contracts, Evaluator Rubrics and Clean Session Exit
- Autonomous AI Coding Agent for Hours: Loop Engineering and Stop Conditions
- Graph Engineering for AI Agents: When a Single Loop Is Not Enough and You Need a Task Graph
- Building an AI Agent Harness From Scratch: AGENTS.md, feature_list.json, verify.sh and an Autonomous Loop