LLM context window for agents: what goes in and why agents forget rules

What the LLM context window is

The context window of an LLM is all the text the model receives as input when it generates its next response. A model has no memory in the human sense: everything it "knows" about your task at a given moment is inside this window. What is not in the window does not exist for the model.

The size of the window is measured in tokens — pieces of words. For Russian and other non-English text, a token is on average shorter than for English, so the same instruction usually costs more tokens in those languages. Modern models have windows of hundreds of thousands of tokens, and it seems there is room for everything. In practice, an agent runs out of that room faster than you expect, and it starts working worse even earlier.

What goes into the agent's context window

A coding agent has exactly three sources of information: the system prompt with the task description, the contents of the files it has read, and the output of the tools it has called. All three live in the same window and compete for space.

The key point: none of this disappears on its own. A log of failed tests that was needed twenty steps ago still sits in the window and takes attention away from the current step. A long session is a constantly growing pile in which it becomes harder and harder to find the needle.

Why the agent forgets rules in a long session

A common complaint: at the start of the session the agent follows the rules from CLAUDE.md carefully, and an hour later it starts breaking them. It has not "forgotten" them in the literal sense — the text of the rules is still in the window. The problem is how the model distributes attention over a long text.

Lost in the middle

Research by Liu and co-authors in 2023 showed that language models use information from the beginning and the end of a long context noticeably better than from the middle. The instruction file is loaded at the start of the session, but as the dialogue grows, more and more text piles up above and below it. A rule that was in plain view gradually ends up in the zone where the model "sees" worst.

The share of rules in the window shrinks

In a fresh session, the instructions are a noticeable part of the context. After a hundred tool calls they become a small fragment against a background of logs, diffs and code that has been read. The signal is the same, but the noise around it is dozens of times bigger.

Recent examples outweigh old rules

The model leans heavily on what happened recently in the dialogue. If the agent has several times in a row done something in violation of a rule and got away with it, that pattern in the history starts to weigh more than a line of instructions somewhere near the top.

Context rot: quality degradation long before the limit

Context rot is the name for the effect where the quality of the model's answers falls as the window fills up, even though the technical limit is still far away. The model formally accepts 200 thousand tokens, but it works worse at 150 thousand than at 20: it more often confuses details, repeats solutions already rejected, and loses instructions.

Signs that a session has "rotted":

• the agent proposes an approach that you both rejected half an hour ago;
• it confuses the names of files and functions that were already discussed;
• it stops running the checks that it ran itself at the start;
• the answers become longer and vaguer, and unnecessary "also fixed" remarks appear.

Adding one more rule does not help: the new rule will get lost in the same noise. You need to reduce the noise.

Context compaction

Compaction is the compression of the session history: the model writes a short summary of what has happened, and the dialogue continues with that summary instead of the full history. The window is freed, and the essence of the work is supposedly preserved.

In Claude Code, the /compact command does this, and you can pass it a hint about what to focus on when compressing. When the window gets close to its limit, Claude Code also performs compaction automatically. The /clear command wipes the history completely and starts from a clean slate, and /context shows how the window is currently used.

/context
/compact keep: the goal of the task, decisions made with reasons, the list of changed files, what is left to check
/clear

What is dangerous about compaction

A retelling always loses detail. The compressed summary may not include a rejected option and the reason for rejecting it, the exact wording of a requirement, or a nuance you agreed on in passing. After compaction the agent works from someone else's summary of its own session. So it is safer to keep important things not in the dialogue history but in files that survive any compaction.

Compaction or a new session

How to save the context window

Saving context is not about the price of tokens, but about the quality of work. The less noise in the window, the more attention goes to the task and the rules.

1. A short instruction file as a router

The entry CLAUDE.md or AGENTS.md is read in full in every session. A file of 600 lines is on the order of 10–20 thousand tokens of constant cost. Keep the verification commands, the hard constraints and the project map in it, and put the details by topic into separate documents that the agent opens only when needed.

2. Knowledge loaded on demand

Claude Code skills are built so that only their short description enters the context at first, and the full text of the instruction is loaded when the skill is needed for the task. This is a convenient way to keep rare procedures — deploys, migrations, releases — at hand without paying for them in every session.

3. Subagents for noisy work

A subagent works in its own context window and returns only the result to the main session. Searching a large codebase, parsing a long log, studying an unfamiliar module — all of this can be handed to a subagent, and you get back a few paragraphs of conclusions instead of thousands of lines of raw output.

4. Quiet tool output

Command output is the main eater of the window. Configure your verification commands so that on success they print one line, and on failure only the essentials: the failing test and its message, not the full build log.

# noisy: thousands of lines into the window on every run
pytest -v

# quiet: one line on success, only the failing tests on failure
pytest -q --tb=short

# a log not in full, but only the tail and the errors
tail -n 50 app.log | grep -i error

5. One task — one session

Do not stretch one session across three unrelated tasks. The history of the first task is useless for the third, but it takes up room and confuses the agent. When you finish, record the result, and open a new session.

6. State in files, not in the history

Write everything that must survive compaction and a new session into the repository: the goal, the decisions made with reasons, the rejected options, what has been done and how it was verified, the next step. For example, in PLAN.md and PROGRESS.md. Then a new session starts not with a retelling but with an exact document, and you can safely clear the history.

# PROGRESS.md
## Goal
Switch discount calculation to whole cents

## Done (verified)
- app/payments/discount.py: float → int, make test green
- Migration 041 added, applied locally

## Decisions
- Rounding is banker's rounding, as in the payment provider

## Rejected
- Decimal in the model: breaks webhook serialization

## Next step
Update the end-to-end checkout scenario and run make e2e

Critical rules are not only in the window

Even a perfectly organized context does not guarantee that rules are followed: text in the window is an influence on the model, not an executable prohibition. If a violation is unacceptable, the rule must be checked mechanically. In Claude Code, hooks serve this purpose — commands that run on agent events and can block an action — along with permission settings that forbid certain tools or commands. Linters, tests and CI work independently of whatever is happening in the agent's window right now.

A good harness rule: everything that can be checked automatically is checked automatically, and the context keeps only what requires judgment.

Summary

The LLM context window is the only place where the agent sees your task, and it fills up faster than it seems: instructions, history, files read and command output pile up until the end of the session. Quality falls long before the technical limit, rules from the middle of a long context get lost, and compaction saves room at the cost of detail. Keep instructions short, load rare knowledge on demand, hand noisy work to subagents, keep the state in files, and back up critical rules with hooks and checks. We practice working with agent context step by step in the harness engineering course.