What is a token, and how do you count one?
A text token is a chunk of text represented by a model’s tokenizer. It might be a word, part of a word, punctuation, or whitespace. The same sentence can have different token counts across models, so examples like “one word equals one token” are not reliable billing rules.
For English prose, roughly four characters per token is a useful first estimate. Code, numbers, non-English text, and unusual strings can differ considerably. Images, audio, and video may use other metering rules; a text estimate does not price those modalities.
Use the model’s tokenizer for pre-launch estimates and the API response’s usage fields for actual consumption. Include system instructions, conversation history, retrieved documents, tool definitions, and tool results—not just the sentence typed by the user.
How tokens turn into an API bill
Providers can meter several buckets:
| Bucket | What it represents | Budgeting check |
|---|---|---|
| Uncached input | Prompt content processed without a cache hit | Include history and tool schemas |
| Cached input | Eligible repeated content read from a cache | Verify actual cache-hit usage |
| Cache creation | Writing content into a cache where separately charged | Check write and retention rules |
| Output | Generated tokens, including billable reasoning where applicable | Read the model’s usage definition |
| Pricing data: How we verify → | ||
Uncached token cost = (input tokens × input rate + output tokens × output rate) / 1,000,000.
For 1,000 requests that each use 800 input and 200 output tokens, multiply the table’s input rate by 0.8 and output rate by 0.2, then add them. That is a workload example, not a fixed provider quote. The selected models illustrate token billing, not a ranking of today’s latest releases.
Output is often priced above input, but the ratio varies by model; do not apply a universal multiplier. Check OpenAI pricing, Anthropic pricing, or Google AI pricing for the relevant rate card and additional charges. Our API cost guide covers mixed cached and uncached workloads.
Caching is conditional, not free input
A stable repeated prefix can qualify for cheaper cache reads, but providers differ on minimum length, retention, writes, and storage. A repeated prompt does not guarantee a hit. If cached tokens are reported within total input, subtract them before billing the uncached bucket; otherwise you count the same tokens twice.
See the cached-token guide before assuming a discount applies to every request.
Context windows and practical savings
A context window limits how much information a model can process at once. Check its input limit, output limit, and whether generated or reasoning tokens share the context budget. A large advertised window is not an instruction to fill every request.
For a chatbot or coding agent, retained conversation history can increase input consumption with each turn. Summarize older context, retrieve relevant passages, and remove unnecessary tool definitions. Test that these changes preserve answer quality.
To control spend:
- Measure tokens per completed task, including retries and failed calls.
- Use a smaller model only after testing the same acceptance criteria.
- Keep useful repeated instructions stable and measure cache eligibility.
- Set appropriate output and agent-turn limits.
- Check batch pricing for asynchronous work without assuming discounts stack.
Use the full token calculator with your measured input-output mix. Its forecast still needs adjustments for storage, tools, taxes, and provider-specific tiers.
Comparing open-model APIs? Add Novita’s hosted model catalog to your shortlist when a suitable open model can handle the task. Compare its model-specific tokenizer, rate, and quality before moving traffic.
Affiliate disclosure: we may earn a commission through this link, at no extra cost to you.