If you build on AI APIs, you pay by the token — and misunderstanding how that works is how bills spiral. Token pricing is simple once you see it, and understanding it is the foundation of controlling AI costs.
The basics
Models process text as tokens (roughly word-pieces). APIs charge per token, and crucially, input and output tokens are priced separately — output usually costs several times more than input. So a request's cost is (input tokens × input price) + (output tokens × output price). Everything you send (system prompt, context, history, the question) is input; everything the model generates is output.
Every token in your prompt and every token in the reply is metered. Output tokens are the expensive ones — which is why "be concise" is also "be cheap."
Where costs hide
Bills balloon in predictable places:
- Long context on every call — a big system prompt or document sent each time, paid for repeatedly (fix: prompt caching).
- Verbose outputs — output tokens are pricey; unnecessarily long answers cost real money at scale.
- Conversation history — it grows each turn, and you resend it all, so long chats get expensive.
- Reasoning tokens — reasoning models generate lots of internal tokens you pay for.
How to control it
Practical levers: cache stable context (prompt caching), keep outputs as short as the task allows, trim or summarize history, route easy requests to cheaper models, and use the smallest model that clears your quality bar. Different tiers and providers vary widely in price — often several-fold for comparable quality.
Why it matters
Token pricing is the unit economics of any AI product. Understanding it turns cost from a scary surprise into a set of deliberate choices — and at scale, small per-request savings multiply enormously. The teams that run AI affordably aren't using worse models; they're managing tokens well: caching, right-sizing, and not paying for output they don't need.
