When context windows grew to a million tokens, a tidy prediction followed: just paste everything in, and retrieval is obsolete. Reality was more nuanced. RAG didn't die — it grew up, into a discipline about grounding, freshness, and cost rather than a workaround for small windows.
Why "just use long context" isn't enough
Four reasons retrieval stays essential even with huge windows:
- Cost and latency. Stuffing a massive context into every request is slow and expensive. Retrieving the relevant slice is cheaper and faster.
- Freshness. Model weights are frozen at training time. Retrieval is how you answer questions about today.
- Attribution. Grounding an answer in retrieved sources lets you cite and verify — essential for trust.
- Signal. More tokens isn't more understanding. Irrelevant context can actively distract a model and degrade the answer.
Long context and retrieval aren't rivals. Retrieval decides what the model should look at; long context decides how much it can hold once it's there.
What actually matured
Modern retrieval is less "embed and hope" and more engineering: better chunking, hybrid keyword-plus-vector search, re-ranking, and query rewriting so the system searches with a good question. The retrieval step is now something teams evaluate and tune as a first-class system, not an afterthought.
The takeaway
For anything that must be current, cited, or cost-controlled — which is most real applications — retrieval remains the backbone. The craft simply moved from "does it work at all?" to "how good is the thing you retrieve?" Long context raised the ceiling; it didn't remove the floor.
