Behind every AI capability is a physical constraint most discussions skip: the chips. GPUs (and their AI-accelerator cousins) are the scarce, expensive resource that shapes what gets built, by whom, and at what cost.
Why GPUs, and why scarce
Training and running large models require massive parallel computation — exactly what GPUs excel at. But high-end AI accelerators are expensive, power-hungry, and for stretches of the boom, genuinely scarce. Access to compute became a strategic asset: labs with more chips could train bigger models faster, and cloud providers' capacity shaped who could deploy at scale.
Compute is the oil of the AI economy. Whoever controls it shapes what's possible — and what's affordable.
Training vs inference economics
There's a key split. Training is a huge one-time cost — enormous, but paid once per model. Inference — actually running the model to serve users — is an ongoing cost that scales with usage, and at scale it often dominates total spend. This is why so much engineering (quantization, speculative decoding, small models, caching) targets inference: shaving inference cost compounds across billions of requests.
The 1,000x collapse — and its limit
Efficiency gains have driven the cost of a given capability down dramatically. But demand keeps rising to meet supply — cheaper inference invites more usage, so total compute demand keeps climbing. The result: GPUs remain a central bottleneck and a central cost, influencing model architecture (sparsity to save compute), business models (who can afford to serve), and even geopolitics. Understanding AI economics means understanding that intelligence, for now, is metered in GPU-hours.
