Is a smaller model always cheaper per token?
Why per-token price isn't the whole story
A smaller model may cost less per token, but it might need more tokens to answer the same question, or it might fail more often and require retries. If you have to send three prompts to get one usable answer, the effective cost per successful answer can be higher than using a larger model once.
Also, some providers charge different rates for input and output tokens. A smaller model with cheap input but relatively expensive output can be more expensive for tasks that generate long responses.
When smaller models do save money
For simple, high-volume tasks like classification, extraction, or short summaries, a smaller model is usually cheaper because it uses fewer tokens and has a lower per-token price. The savings are real when the task is well-defined and the model's accuracy is good enough.
You can often cut costs further by using a smaller model for easy requests and routing only hard requests to a larger model. This tiered approach can reduce average cost per request without hurting quality much.
- Measure cost per successful task, not just per token.
- Count retries and failed attempts in your cost math.
- Consider input vs. output token pricing separately.
- Test a smaller model on a sample before switching everything.
Common mistakes
- Assuming the cheapest per-token model is always the cheapest overall.
- Ignoring that smaller models may need longer prompts or more examples to perform well.
- Forgetting that output tokens often cost more than input tokens, so response length matters.
