Much of the reasoning-model boom rests on a training method that's elegant precisely because of what it removes. Group Relative Policy Optimization (GRPO) simplified reinforcement learning for LLMs — and made it cheaper.
The problem it solves
Classic RL for language models (PPO-style) needs a value model — a second network that estimates how good a state is — running alongside the policy. That's expensive: extra compute, extra memory, extra things to tune.
The group-relative idea
GRPO drops the value model. Instead, for each prompt it samples a group of answers, scores them (e.g., correct or not), and judges each answer relative to its peers in the group. An answer that beats the group average gets reinforced; one below it gets discouraged. The group itself becomes the baseline.
Don't ask "how good is this answer in absolute terms?" Ask "is it better than the others we just tried?" — a question you can answer for free.
Why it mattered
By removing the value model, GRPO cut the cost and complexity of RL training substantially, making outcome-based reasoning training accessible to more labs. Paired with verifiable rewards (math and code, where correctness is checkable), it's a core ingredient behind the wave of strong open reasoning models. It's a good example of a recurring lesson: sometimes the biggest gains come from taking a component out.
