The transformer's attention mechanism is powerful but scales badly with sequence length. State-space models (SSMs) scale linearly but historically lagged on quality. The pragmatic answer that emerged: use both.
Two primitives, two strengths
Attention lets every token look at every other token — great for capturing relationships, but its cost grows with the square of the sequence length, and it needs a big memory cache. State-space models (like Mamba) process sequences with a fixed-size state updated step by step — cheap and constant-memory, but each token sees the past only through that compressed state.
Attention is a perfect memory that's expensive. An SSM is a cheap memory that forgets. Use each where its tradeoff wins.
The hybrid design
The insight: you don't need full attention at every layer. Hybrid models interleave attention layers (where global context matters most) with SSM or linear-attention layers (for cheap long-range processing). The result keeps much of attention's quality while dramatically cutting cost and memory on long inputs.
Why it's the future for long context
As context windows grow to millions of tokens, pure attention becomes prohibitively expensive — the KV cache alone dominates memory. Hybrids sidestep that by handling most of the sequence with cheap layers and reserving attention for where it earns its cost. Expect more frontier models to adopt hybrid designs as long context becomes standard. The lesson isn't "attention is over" — it's that we finally have a second primitive worth mixing in.
