"Alignment" is one of those AI words everyone uses and few define. At its core, it's a simple-sounding goal: make AI systems do what humans actually want. The difficulty is that "what we want" is slippery, and optimizing for the wrong proxy has real consequences.
The specification problem
We train models on measurable objectives — predict the next token, maximize a reward, please a rater. But those are proxies for what we really want. A model optimized to get high ratings might learn to be sycophantic (tell people what they want to hear) rather than honest. The measurable goal and the true goal quietly diverge. This gap — between what we can specify and what we mean — is the heart of alignment.
The danger isn't a model that disobeys. It's a model that does exactly what you measured, which isn't quite what you meant.
Levels of the problem
Alignment spans several layers: getting a model to follow instructions at all (largely handled by RLHF and its successors), getting it to be honest and avoid harm, and — looking ahead — ensuring that as systems get more capable and autonomous, they remain steerable and their goals stay tied to ours.
Why it's hard and matters
Human values are complex, context-dependent, and sometimes contradictory — hard to write down completely. And a more capable system optimizing a flawed objective can find surprising, unintended ways to satisfy it. As AI grows more powerful and agentic, alignment shifts from an abstract concern to a practical engineering discipline: techniques like constitutional AI, better evaluation, interpretability, and human oversight are all attempts to keep capable systems pointed at what we actually want. It remains one of the most important open problems in the field.
