Insight · September 12, 2026
The window fills.
The mind fades.
Every model gets worse as the input grows. The failure has a name, and it starts long before the window is full.
01 · The term
Context rot is quality falling as the input grows.
Context rot is the measurable drop in how well a model answers as you give it more to read. The window can hold a million tokens. That does not mean the model uses them evenly. It reads the front and the back with care and grows careless in between, and the more you load in, the wider that careless zone gets.
The term was put on solid ground in July 2025, when Chroma published a report titled Context Rot, written by Kelly Hong, Anton Troynikov, and Jeff Huber. They tested 18 current models, among them GPT 4.1, Claude 4, Gemini 2.5, and Qwen3. Every single one got worse as the input got longer, even on tasks as plain as finding a fact or copying text back word for word.
The headline is not that models fail at some hard limit. It is that they never read a long input evenly in the first place. Two identical facts, one placed early and one buried deep, do not get the same attention. The window is uniform. The reading is not.
02 · What it is not
This is not running out of room.
It is easy to confuse context rot with overflow. Overflow is simple. You hand the model more tokens than the window holds, and the extra gets cut. You can see it happen and you can plan around it.
Rot is the quiet one. It shows up well inside the limit. A model with a 200,000 token window can start slipping at a fraction of that, on inputs it accepted without complaint. Nothing errors. The answer still reads clean and sure. It is just more likely to be wrong, and you have no warning light for that.
The one line to keep
“A bigger context window is a bigger desk, not a better memory.”
03 · What the research found
The failure is not uniform, and it is not intuitive.
01
Retrieval is not a lookup
When the answer does not share the exact words of the question, the model has to reason to connect them. That reasoning gets less reliable as the input grows.
02
One distractor is enough
A single passage that looks related but is wrong lowers accuracy. Add more and it falls further, and the size of the effect differs from model to model.
03
Coherence can hurt
Models scored higher on shuffled text than on the same facts inside a logically ordered document. Well written context reads like more plausible wrong answers.
04
Focused beats full
Given a long history versus only the few hundred tokens that held the answer, every model did far better on the short version. Extra context is not free.
05
Even copying breaks
Asked to reproduce a simple sequence of repeated words, models drift as the sequence lengthens. The task needs no reasoning at all, and length alone defeats it.
04 · Lost in the middle
Models read the edges. They lose the middle.
The older half of the problem has its own name. In a 2023 study published in the journal of the Association for Computational Linguistics in 2024, Nelson Liu and colleagues at Stanford placed the right answer at different spots inside a long input and watched what happened. Accuracy was high when the answer sat near the start or the end, and fell by more than 30 percent when it sat in the middle. Plot it and you get the shape of a U.
They saw the same U across model families, on answering questions that span many documents and on plain key value lookup. It is a primacy and recency bias, the same edge effect that shapes what people remember from a list. A 2026 paper derived that curve straight from the math of attention, which means it is not a bug one release fixes. A larger window inherits it.
05 · One test that shows the whole thing
Same question. Same answer inside. Two very different scores.
Chroma ran a memory test built to look like a real assistant. The model got a long chat history and a question about something said inside it. In one condition it received the whole history. In the other it received only the few hundred tokens that actually held the answer.
The answer was present and correct in both. The only difference was how much else came with it. Every model scored far higher on the short version. The extra history did not help the model find the answer. It buried it. That gap is context rot in one clean picture, and it is the entire reason retrieval exists.
FULL one long chat history · about 113,000 tokens
answer is in there somewhere
result · every model scores lower
FOCUSED only the part that holds the answer · about 300 tokens
same answer, nothing else
result · every model scores higherChroma memory test · full input against a focused input
06 · The response
The fix is a smaller, cleaner window.
Once you accept that more context is not more memory, the work changes. The job is not to fill the window. It is to decide, for each step, what belongs in it. Retrieve the passages the task needs, leave out the stale history and the near misses, and place the important thing where the model actually reads, near the top or the end rather than lost in the pile.
This is the whole reason retrieval and context engineering are disciplines and not afterthoughts. A governed window, scoped to the task and cleared of noise, flattens the curve. The model has less to sort through, fewer plausible wrong answers to trip on, and a shorter path to the fact that matters.
In practice that is three habits. Retrieve, so only the passages the task needs enter the window. Order, so the fact that matters sits near an edge the model reads well rather than the middle it skims. Prune, so old turns and dead tool output do not crowd out the current question.
It is also why a running system needs a way to forget. An agent that keeps appending every tool result and every past turn is walking itself into the careless zone on purpose. Summarize, prune, and reset, so the input stays lean.
07 · The honest trade
Long context is a tool, not a free upgrade.
None of this means large windows are useless. A bigger window lets you truncate less and hand the model a fuller picture when the picture helps. The point is that the window is a place to put things, not a promise that the model will weigh them all.
And it is never free. Every token you add costs money and time, and as the window fills with old turns and near misses the useful signal thins out. Past a point you are paying more to make the model perform worse. The operator move is to treat window space as a budget you spend on purpose, not a bin you fill because it is there.
Closing
The model was never reading everything.
Take the longest prompt you rely on. Cut it to the part that holds the answer and run both. If the short one wins, and it usually does, you have found your rot, and you have found your fix in the same move.
Chroma · Context Rot · Hong, Troynikov, Huber · July 2025 · and Lost in the Middle · Liu et al. · Stanford · TACL 2024
Share this perspective
More insights
Adjacent perspectives.
Bttr. Field Brief
The brief Bttr. writes for senior buyers.
Monthly. One signal worth your time on Brand Operating Systems, AI search visibility, and the infrastructure buildout. No filler.