AI costs

LLM token costs explained: input, output and cached tokens

Updated 6 min readMarkdown
In short

The cost of a request to a language model is the sum of four products, each a token count times a price per million tokens. The four counts are uncached input, output, tokens read from the prompt cache and tokens written to it. Output is priced higher than input because the model reads its input in one pass and writes its answer one token at a time. Input is usually the larger quantity, because every request carries the system prompt, the tool definitions and the whole conversation so far. The provider's usage field in each response gives the counts that are billed.

A request to a language model is billed in tokens, and its cost is a sum of four products: uncached input tokens, output tokens, tokens read from the prompt cache and tokens written to it, each multiplied by its own price per million. Once the four counts and the four prices are known, the cost of a request is arithmetic that anyone can check.

What a token is

A token is a piece of text: a short word, part of a longer word, a punctuation mark. A model reads and writes tokens, and providers bill by them. How a text is cut into tokens depends on the model's tokeniser, so the same prompt gives different counts on different models. Images and documents are converted into tokens as well.

As a rough guide for English, four characters make one token. That is enough to plan with. The count that is billed is the one the provider reports in the usage field of each response.

The four token types

Token typeWhat it isHow it is priced
Uncached inputEverything sent to the model that is processed from scratchThe input price
OutputEverything the model writes, including reasoning it does before answeringThe output price, higher than input
Cache readThe start of a prompt that the provider kept from an earlier requestBelow the input price
Cache writeThe start of a prompt that is stored for later requestsThe input price or more, depending on the provider

A price list gives each of these per million tokens and per model.

Why output tokens cost more

A model processes its whole input in one pass. It writes the answer one token at a time, and each new token needs another pass that takes into account everything read and written so far. A thousand output tokens keep the hardware busy far longer than a thousand input tokens, and the price list reflects that.

Two consequences follow. First, reasoning is output. When a model thinks before it answers, those tokens are billed at the output price even if the application never shows them. Second, the output limit set in the request (max_tokens or its equivalent) caps what one answer can cost.

What counts as input

A model keeps nothing between requests. Every request therefore carries all of this again:

  • the system prompt,
  • the definitions of every tool the model may call,
  • all earlier messages of the conversation, with tool calls and their results,
  • attached documents and images,
  • the new message.

That is why the input of a conversation grows with every turn. Suppose each question is 400 tokens and each answer 600. The tenth request carries nine earlier exchanges and the new question: 9 × 1,000 + 400 = 9,400 input tokens. Over ten turns the input adds up to 49,000 tokens, although the ten questions together were only 4,000. In most applications input is the larger quantity and output the higher price.

Cached tokens

Prompt caching lowers the price of input that repeats. The provider keeps the processed start of a prompt, the prefix, and a later request that begins with exactly the same prefix reads it from the cache at a lower price. Three rules apply at every provider, with different values:

  • Exact prefix. The cache matches from the first token on. A timestamp or a user name near the top of the system prompt breaks it for everything after.
  • Minimum length. Short prefixes are not cached.
  • Lifetime. An entry expires after a period without use, typically minutes.

Some providers cache automatically and charge nothing extra for writing. Others cache what the request marks as cacheable and charge more than the input price for the write. The first request then costs more than without caching, and caching pays from the second request within the lifetime. More in prompt caching and context.

How to compute the cost of a request

cost = (uncached input × input price
      + output         × output price
      + cache reads    × cache-read price
      + cache writes   × cache-write price) ÷ 1,000,000

A worked example with invented prices: 2.00 per million input tokens, 10.00 per million output tokens, 0.20 per million for cache reads and 2.50 per million for cache writes. A request sends 12,000 input tokens, of which the first 10,000 are a system prompt with tool definitions, and receives 800 output tokens.

CaseCalculationCost
No caching12,000 × 2.00 + 800 × 10.00, per million0.0320
First request, prefix written to the cache2,000 × 2.00 + 10,000 × 2.50 + 800 × 10.00, per million0.0370
Later request, prefix read from the cache2,000 × 2.00 + 10,000 × 0.20 + 800 × 10.00, per million0.0140

At 100,000 such requests a month that is 3,200.00 without caching and close to 1,400.00 with it. The same arithmetic prices a switch of model: keep the token counts, change the prices. Which requests can move to a cheaper model is the question of model routing.

Where the counts are in each API format

The usage field is named differently in each format, and it differs in one point that matters: whether the input figure includes cached tokens.

API formatInputCachedOutput
Anthropic Messagesinput_tokens, without cached tokenscache_read_input_tokens, cache_creation_input_tokensoutput_tokens
OpenAI Chat Completionsprompt_tokens, cached tokens includedprompt_tokens_details.cached_tokenscompletion_tokens
OpenAI Responsesinput_tokens, cached tokens includedinput_tokens_details.cached_tokensoutput_tokens

Where cached tokens are included, subtract them before applying the input price. Otherwise they are paid for twice in your calculation and the computed cost is too high.

What else changes the price

  • Batch processing. Requests that can wait for their result are billed below the standard rate.
  • Service tiers. Priority processing costs more than standard.
  • Long prompts. Some price lists charge a higher rate once a prompt exceeds a certain length.
  • Tools. Web search or code execution run by the provider are billed on top of tokens.

What goes wrong

  • Estimating with one blended price. A single price per token hides whether cost sits in input or output, and so hides which lever would work.
  • Forgetting the history. A chat feature is costed from one question and one answer, and the bill reflects the tenth turn.
  • Ignoring reasoning tokens. A short visible answer can have a long invisible one behind it.

How Cureman does it

Cureman prices usage with this formula and nothing else: four token counts, each multiplied by its price per million and rounded on its own, so the same tokens always give the same amount. Counts come from the providers, from their usage reports and, for requests through the gateway, from the usage field of each response, streamed ones included. Input that a provider reports with cached tokens included is split, so that input always means the uncached part. Prices come from a model price list in which every row is labelled as unverified until a person has verified it. A dated snapshot is priced as its base model, and the page says so. How this becomes a breakdown by provider, model and project is described in how to audit AI spend and on the page about the AI spend audit.

Questions and answers

How is the cost of an LLM request calculated?

Multiply each token count by its price per million tokens and add the results. There are four counts, which are uncached input, output, cache reads and cache writes. The counts come from the usage field of the provider's response, the prices from the provider's price list for the model that served the request.

Why do output tokens cost more than input tokens?

A model reads its whole input in one pass, but writes its answer one token at a time, and every new token needs another pass that takes everything before it into account. Output therefore occupies the hardware much longer per token, and providers price it higher. Tokens a model spends on reasoning before it answers are billed as output as well.

What are cached tokens?

They are input tokens at the start of a prompt that the provider has kept from an earlier request and does not process from scratch. Reading them is billed below the normal input price. Depending on the provider, writing them to the cache costs the normal input price or more, and a cache entry expires after a period without use.

Do input tokens include the conversation history?

Yes. A model keeps nothing between requests, so every request carries the system prompt, the tool definitions and all earlier messages again. The input of a conversation therefore grows with every turn, and the total input billed over a conversation grows faster than the number of turns.

How many tokens is a text?

That depends on the model's tokeniser, so the same text gives different counts on different models. As a rough guide for English, four characters make one token. For planning that estimate is enough. For billing, only the count in the provider's usage field matters.