Every headline goes to the largest frontier model. Look at what actually ships in products, though, and you'll find small models — a few hundred million to a few billion parameters — doing the day-to-day work, often on a laptop or phone, answering in milliseconds.
What changed
Small models used to be visibly worse. Three shifts closed most of the gap. Distillation lets a small student learn from a large teacher's outputs and reasoning, inheriting capability beyond what its size alone predicts. Data quality — curated, deduplicated, high-signal training sets — lifts small models more than raw scale does. And task focus: a small model tuned for one job beats a giant generalist on that job.
The right question is no longer "which model is best?" but "how small can we go before quality drops below the bar?"
The on-device turn
The clearest sign is where these models run. Phones and laptops now carry roughly three-billion-parameter models handling writing help, summarization, and smart replies entirely on-device — no round trip to a server. That means lower latency, no per-call cost, and data that never leaves the hardware.
The pattern that wins
The emerging default is a cascade: a small model handles the common cases instantly and escalates only the genuinely hard ones to a larger, slower, pricier model. For high-volume, well-scoped work — which is most work — that combination is decisive. The economics of cost, latency, and privacy all point the same way, and they point small.
