Long context windows are a headline feature — millions of tokens. But there's a catch that undercuts the "just paste everything in" instinct: models don't attend to long inputs evenly. Information stuck in the middle of a long context often gets overlooked. Researchers call it "lost in the middle."
The phenomenon
Studies found a U-shaped pattern: models use information best when it's at the beginning or end of a long context, and worst when it's buried in the middle. Put the key fact up front or last, and the model finds it; bury it halfway through a huge document, and performance drops — sometimes dramatically. The window is big, but attention within it is uneven.
A million-token window is a huge desk. But the model reads the top and bottom of the pile carefully and skims the middle — right where your answer might be.
Why it happens
It relates to how attention and positional handling work over long sequences, and to training data patterns (important information often sits at the start or end of documents). Whatever the exact cause, the effect is robust enough to matter in practice.
The practical implications
This reshapes how you should use long context:
- Position matters — put the most important information at the start or end, not buried in the middle.
- More isn't automatically better — padding the context with everything can hurt if the key fact ends up mid-pile and irrelevant text surrounds it.
- Retrieval still wins — fetching only the relevant slice (and placing it well) beats dumping a huge document and hoping.
Why it matters
"Lost in the middle" is a concrete reason the "long context kills RAG" narrative was wrong. A big window is capacity, not comprehension — and models use that capacity unevenly. Understanding this helps you design prompts and retrieval that put the right information where the model will actually use it, rather than trusting that a large window reads everything equally. It doesn't.
