Every large language model runs on an idea called attention. It sounds technical, but the intuition is simple — and understanding it demystifies how these models actually work.
The problem attention solves
To understand a word, you need context. In "the bank of the river" versus "money in the bank," the word bank means different things, and only the surrounding words tell you which. A model needs a way to let each word look at the other words that clarify it.
What attention does
Attention lets every word in a sentence "look at" every other word and decide which ones are relevant to it. For each word, the model asks: which other words should I pay attention to, and how much? It then blends in information from those words, weighted by relevance. Bank pulls in river or money and shifts its meaning accordingly.
Attention is the model asking, for every word: "which other words help me understand this one?" — and then listening to them in proportion.
Why it changed everything
Before attention, models processed text word by word in order, struggling to connect distant words. Attention lets a model relate any two words directly, no matter how far apart — and do it for all words in parallel, which makes training on huge data feasible. That combination, introduced by the transformer, is why AI leapt forward.
The tradeoff
Attention's power has a cost: comparing every word to every other word scales with the square of the length, which is why very long inputs get expensive (and why efficiency tricks like the ones we cover exist). But the core idea — let everything attend to everything — remains the engine under the hood of essentially every model you use.
