What an AI feature actually costs to run

“What will it cost to run?” tends to get asked after a feature is built, which is the wrong end. The answer is knowable in advance to within a factor of two, and knowing it changes design decisions that are expensive to revisit.

The formula

Providers bill per token, priced separately for what you send and what you get back. A token is roughly four characters of English, so 750 words is about 1,000 tokens.

Cost per call is therefore:

(input tokens × input rate) + (output tokens × output rate)

Take a support answering feature. The system instructions run 300 tokens. You retrieve five passages of about 400 tokens each, so 2,000. The user’s question is 50. That is 2,350 tokens in. The answer comes back at around 250 tokens out.

At mid-range 2026 pricing for a capable model (order of €2.50 per million input tokens and €10 per million output) that is roughly €0.006 plus €0.0025, so about €0.0085 per call. Fifty thousand calls a month is around €425.

The number to notice is that retrieval accounted for 85% of the input. That is typical, and it is where the leverage is.

Where the money actually goes

Retrieved context, first and by a distance. Five passages instead of ten halves your input cost with, in most cases, no measurable accuracy loss. Teams over-retrieve because it feels safer; it is worth measuring rather than assuming.

Retries. A call that fails validation and runs again costs twice. Build a retry budget in explicitly, and log how often it triggers. An unexamined retry loop is the single most common cause of a bill that arrives three times larger than the estimate.

Multi-step chains. An agent that plans, calls two tools and summarises is four calls, each carrying the accumulated conversation. Cost grows faster than the step count because the context grows with it.

Embeddings, once. Indexing your corpus is a one-off cost, and a small one: embedding a million tokens costs cents. Re-embedding on every deploy because the pipeline has no change detection is not small, and it happens.

The four changes that reliably cut the bill

Retrieve less, better. Improving the ranking so three passages beat ten is both cheaper and more accurate: models degrade when the relevant sentence is buried in irrelevant context.

Route by difficulty. Most questions are easy. Send those to a small model and escalate only what a cheap classifier flags as hard. A 70/30 split against a model at a tenth the price cuts the bill by roughly two thirds.

Order the prompt for caching. Put stable content (instructions, shared reference material) at the front and the variable part at the end, so the provider’s prompt cache can discount the repeated prefix.

Cap output length. If you need two sentences, say so and set a maximum. Output tokens carry the highest unit price, and a model with no ceiling will fill the space it is given.

What to budget in practice

For a single grounded feature at moderate volume (a few tens of thousands of calls a month) the model bill is usually somewhere between €100 and €1,000. That is nearly always small next to the engineering cost of building it, and small next to the labour it replaces.

The costs that surprise people are elsewhere: the vector store if you reach for a managed one before you need it, the logging volume if you store every prompt and response indefinitely, and the person who has to look at the eval results every month. Budget for that last one. A feature nobody measures degrades quietly as the data around it changes.

The one number to get before you build

Cost per call, on a napkin, using your real expected context size. It takes ten minutes and it determines whether the feature is worth building, which model tier to design around, and whether you need routing from day one or can add it later.

The alternative is finding out in the second month, by which time the architecture that would have fixed it cheaply is the architecture you already shipped.