Insight · September 18, 2026
Everything has to
fit in the frame.
A context window is the fixed span of text a model can hold at once. Your prompt, its reply, and everything in between share the same frame. When it is full, something falls out.
01 · The definition
A model has no memory. It has a window.
A large language model does not remember your last conversation. Each time it runs, it reads a block of text and predicts what comes next. The context window is the size of that block. Anthropic calls it the model’s working memory: all the text the model can reference when it generates a response, including the response itself.
That is the whole idea. The window is not storage. Nothing carries over on its own. Everything the model knows in the moment is the text sitting inside the window right now, and the window has a hard edge.
02 · The unit
The window is measured in tokens, not words.
A token is a chunk of text. It can be a whole word, a piece of one, a space, or a mark. OpenAI’s tokenizer notes that in English a token ranges from a single character to a full word, and averages about four characters. Tokens are simply what the model reads, so when a model states a limit, the limit is in tokens.
The current frontier is large. Claude’s latest models hold up to one million tokens in a single request, enough for a very long book with room to spare. A few years ago the largest windows held only a few thousand tokens. The numbers keep climbing, but the shape of the thing does not change. There is always an edge, and the edge is counted in tokens.
The one line to keep
“The window is not memory. It is the desk the model works on, and the desk is only so wide.”
03 · One budget
Input and output draw from the same pool.
The window is one budget, split between what goes in and what comes out. On Claude’s million token models a single request can hold the full history as input and still generate up to 128,000 tokens of reply, but both live inside the same limit. Fill the window with input and there is no room left to answer.
This is why a long prompt can earn a short answer, or none at all. The model is not being lazy. It has run out of frame. And if the input alone is already larger than the window, the request does not run at all. It fails with a plain message: the prompt is too long.
04 · Why there is a wall
The wall is built into the math.
A bigger window sounds like a setting someone could raise. It is not. Attention, the mechanism that lets a model weigh every token against every other token, grows with the square of the length. Double the tokens and you roughly quadruple the work. Ten times the tokens is near a hundred times the cost.
That curve is why windows have edges, and why long requests cost more and run slower. Engineers have pushed the edge out year after year, with better hardware and sharper attention tricks, but the shape of the cost has not moved. The window is a budget the architecture forces, not a limit someone forgot to lift.
05 · What counts against the window
People picture it holding their question. It holds far more.
01
The system prompt
The standing instructions the model reads before your first word. It sets the role, the rules, and the voice, and it rides along on every turn.
02
The conversation so far
Every earlier message, yours and the model’s, carried forward in full. A long thread is mostly history by the end.
03
Every tool definition
The description of each tool the model can call. A rich toolset can spend thousands of tokens before the model does anything.
04
Documents and images
A page or a picture is tokens too. A single request can carry hundreds of images or pages, and each one has a cost.
05
The reply itself
The model has to leave room to answer. The output shares the same frame as everything you sent to set it up.
Add it up and the window fills faster than anyone expects. The reply is competing for space with everything you sent to set it up.
06 · Bigger is not better
A larger window is not a longer memory.
It is tempting to treat a million tokens as a place to dump everything and let the model sort it out. That is the wrong instinct. Accuracy and recall fall as the window fills, a drift Anthropic names context rot. A model reads a full window less carefully than a focused one, so what you leave out matters as much as what you put in.
The skill, then, is not filling the window. It is curating it. Send the model what the task needs and little else. A smaller, cleaner window beats a larger, cluttered one, and it costs less to run. This is the whole practice behind context engineering, and the failure it fights is context rot.
07 · How to work with it
Treat the window as a budget you spend.
Once you see the window as a fixed budget, the decisions get simpler. Know roughly how many tokens your prompt spends. Keep the standing instructions tight. Trim the history and the old tool output you no longer need. Leave clear room for the answer.
For anything that outgrows a single window, the answer is not a bigger model. It is structure: retrieval that pulls in only the passages that matter, memory that persists the few facts worth keeping, and compaction that summarizes what came before. The window stays small on purpose, and the system feeds it well.
Closing
The window holds what you put in it.
Count your tokens once. Decide what earns a place in the frame. The model does its best work when the desk in front of it is clear, and the desk is only ever as clear as you keep it.
Definitions and window sizes · Anthropic context windows documentation, 2026 · token behavior from OpenAI's tiktoken tokenizer and token counting cookbook · attention cost from the standard transformer literature
Share this perspective
More insights
Adjacent perspectives.
Bttr. Field Brief
The brief Bttr. writes for senior buyers.
Monthly. One signal worth your time on Brand Operating Systems, AI search visibility, and the infrastructure buildout. No filler.