The reason small models punch above their weight has a name: distillation. A small "student" model is trained on the outputs of a large "teacher," inheriting capability its size alone wouldn't predict. It's the quiet engine behind most of the strong small models shipping today.
Beyond copying answers
Classic distillation trained the student to match the teacher's final outputs. The 2026 refinement is richer: students learn from the teacher's reasoning traces — the intermediate steps, not just the conclusion. A student that sees how a strong model works through a problem picks up transferable strategy, not just answers to memorize.
You can teach a smaller model to think like a bigger one — if you show it the working, not just the result.
Why it works so well
A large teacher has effectively done expensive search and reasoning that the student can absorb cheaply. The student never has to discover those strategies from scratch; it learns them pre-digested. That's why a well-distilled small model can approach the quality of a model many times its size on the tasks it was distilled for.
The practical shape
For teams, distillation is how you get a fast, cheap, deployable model without giving up too much quality: use a frontier model to generate high-quality training data — answers and reasoning — for your specific tasks, then train a small model on it. The result runs cheaply in production while carrying much of the teacher's competence. It's one of the highest-leverage moves in applied AI right now, and it's why "small" no longer means "weak."
