AI costs

How to allocate AI costs to teams, projects and people

Updated 6 min readMarkdown
In short

A model provider's invoice shows an amount per model, at best per API key or project. To say which team, feature or person caused it, every request has to carry three labels when it is made. The key says which application sent it, a tag says which feature or job, and a user identifier says whom it was for. Cost is computed per request from its token counts and the price of the model that answered, then summed by any label and reconciled with the provider's usage report. What carries no label is shown as unattributed, not spread across teams.

A model provider's invoice shows an amount per model, at best per API key or project. To say which team, feature or person caused it, every request has to carry three labels at the moment it is made: the key it was sent with, a tag for the feature or job, and the user it was for. Cost is then computed per request from its tokens and summed by any of the three. What carries no label is shown as unattributed, not spread around.

What the provider's report can and cannot tell you

Usage reports break spend down by day and model, and often by API key, project or workspace. That gives you applications, provided each has a key of its own. It does not give you the features inside an application or the people behind a shared key. A chat application with 300 users on one key is one line. How to read these reports is covered in how to audit AI spend.

Three labels on every request

LabelQuestion it answersWho sets itExample
KeyWhich application, in which environment?Whoever issues keysSupport assistant, production
TagWhich feature or job inside the application?The application, on each requestticket-summary, nightly-classification
UserWhich person was the request for?The application or chat front end, from the signed-in userAn email address or a stable internal id

Each label has one rule that keeps it usable:

  • Key. One per calling service and environment, never shared between two applications.
  • Tag. From a short agreed list, in lower case. Free text turns one feature into three lines.
  • User. An identifier your directory can map to a department, written the same way every time.

Where to record the labels

  1. At the provider. One project or key per application. It costs nothing and stops at the application.
  2. In each application. Log the usage block of every response with your own labels. This is fine-grained, but every team builds it again and applies prices in its own way.
  3. In a gateway in front of all providers. Every request passes one place that reads the labels from headers or request fields, records the tokens and prices them from one price list. See what an LLM gateway is.

The large APIs also have a field for an end-user identifier in the request body. If an application already fills it, a gateway can read it and nothing has to change in the code.

From tokens to an amount per label

  1. For every request, record four token counts: uncached input, output, cache read and cache write. Record the model that answered, which can differ from the one requested when requests are routed.
  2. Price each count at that model's price per million tokens and add them up. The formula is explained in LLM token costs.
  3. Sum by key, by tag, by user and by month.
  4. Map the labels to cost centres: a key to the team that owns the application, a tag to the owner of the feature, a user to the department in the directory.
  5. Reconcile. Compare the sum of all recorded requests with the provider's usage report for the same period.

A worked example with invented amounts

The provider reports 10,000.00 for a month. The recorded requests add up as follows:

LineAmountWho carries it
Key "support assistant"4,200.00Customer service, split by tag into its features
Key "employee chat"3,100.00The departments of the users
Key "data pipeline"1,900.00The data team, split by tag into its jobs
Recorded9,200.00
Provider's usage report10,000.00
Difference800.00Nobody yet

The 800.00 is a finding, not a rounding error. Somebody calls the provider with a key that bypasses the place where requests are recorded, or a model is priced wrongly. Keep it as a line of its own until the source is found.

Show first, charge later

Show teams their figures for a few months before any internal charge depends on them. In that time the missing tags surface, the shared keys get split and the directory mapping is corrected. A charge based on labels nobody has checked starts an argument about the labels and not about the spend. Once the figures hold, the same labels carry a limit, as described in AI budgets and limits.

Figures per person are personal data

Usage per person shows how much, and for what kind of work, somebody uses a tool. Restrict who can see it, decide how long it is kept, and do not store prompt text for the sake of allocation, because token counts are enough. In Germany, introducing a technical system designed to monitor the behaviour or performance of employees is subject to co-determination by the works council under section 87(1) no. 6 of the Works Constitution Act. Involve employee representatives before per-person reports are switched on. This is no legal advice.

What goes wrong

  • One key for everything. Without separate keys, the provider's report cannot be checked against anything.
  • Pricing the requested model. When a cheaper model answered, the request cost what that model costs.
  • Pricing cached tokens as input. Cache reads are billed below the input price, so the team with the best caching is overcharged.
  • Counting twice. The provider's report and your own records describe the same traffic. Use one as the total and the other as the breakdown.
  • Identifiers nobody can map. A hashed session id attributes cost to a string, not to a department.

How Cureman does it

The gateway of Cureman records for every request the key, the tag, the user, the requested and the served model, the four token counts and the cost computed from its model price list. The tag arrives in the header x-gateway-tag. The user arrives in the header x-gateway-user, or else from the signed-in user that the workspace's chat front end passes on, or else from the end-user field of the request body. A model that has no price in the list yet is metered at zero, so prices are entered first.

Prompt insights adds a task type to each request: coding, extraction, summarisation, translation, writing, analysis, chat or other. A rule-based classifier sets it, or a small model where the routing policy uses one. The page shows the task mix and cost by tag and by person, and how many requests carry a user at all. It is built from request metadata. Prompt text is not stored unless the organisation switches on prompt samples. Figures per person are visible only to roles that may view people, and amounts only to roles that may view spend.

For the total, a connected provider usage report is the source, and gateway records fill in only for providers without one, so the same traffic is not counted twice. Where the figures support it, a finding recommends a different tool with its calculation, for example a seat for someone whose token cost on certain tasks is clearly above the seat's price. The module is described on the page about prompt insights.

Questions and answers

How do I allocate LLM API costs to teams?

Give every calling application its own key, have applications send a tag for the feature or job and an identifier for the end user, and record the token counts of every request together with those labels. Price each request at the model that answered it and sum by label. Then map keys and tags to the teams that own them and users to their departments.

What is the difference between a key, a tag and a user in cost attribution?

The key identifies the application and environment that sent a request and is set by whoever issues keys. The tag names the feature or job inside that application and is sent by the application with each request. The user is the person the request was made for. A shared chat application needs the user, a batch pipeline needs the tag, and both need their own key.

What should happen with AI costs that cannot be attributed?

Show them as a line of their own. The difference between what was recorded and what the provider reports points to a key that is used directly or a price that is missing. Spreading that amount across teams by headcount hides the finding and charges people for traffic they did not cause.

Does cost allocation need the text of the prompts?

No. The cost of a request follows from its token counts, the model and the price list, and the labels say whom it belongs to. Storing prompt text adds a data protection question and nothing to the allocation, so leave it off unless there is another reason for it.

How does Cureman attribute AI costs?

Its gateway records the key, the tag, the user, the model served, the token counts and the computed cost of every request. Tag and user arrive as request headers, and the user can also come from the request body or from the chat front end's signed-in user. Prompt insights adds a task type and shows cost by tag and by person, with per-person figures limited to roles that may view people.