What an AI feature costs when a thousand people use it | Brain Quest
{{ post.deck }}
{{ sec.h }}
{{ para }}
- {{ li }}
{{ sec.quote }}
The market need
Pricing an AI feature requires knowing its marginal cost, and most teams discover theirs from an invoice. That is late: by then the pricing page is published, the free tier is generous, and the expensive behaviour is the one power users love.
The uncomfortable version of this problem is that cost per user is not a constant. It is a distribution with a long tail, and the tail is disproportionately made of your most engaged customers.
Work out the per-action cost before the first line of code
One action, priced end to end: system prompt plus context plus output, times the number of calls the action really makes. The number that surprises people is the multiplier — a feature that feels like one request is often four, once retries, a reranking pass, and a summarisation step are counted.
Multiply by the actions a heavy user performs in a day, then by thirty. If that number is uncomfortable next to your subscription price, the architecture needs to change now, not after launch.
Context is the dominant cost in most products, not output.
Retries are invisible in dev and material in production.
The heaviest ten percent of users routinely account for half the spend.
The three levers that actually work
Caching first: prompt caching on a stable system prompt is close to free to implement and often removes a third of the bill. Routing second: most requests do not need the largest model, and a classifier that sends the easy ones elsewhere pays for itself within days. Trimming third: retrieving four relevant chunks instead of twenty is usually better for quality as well as cost.
What rarely works is squeezing the prompt by hand. It saves a few percent and costs a week.
Cost per action is a product decision disguised as an infrastructure one.
Price the thing that varies
If cost scales with usage and revenue does not, the model breaks at exactly the moment the product succeeds. Either the plan includes a usage dimension the customer understands, or the expensive action is bounded by design — a limit, a queue, a cheaper default with an explicit upgrade.
Customers accept limits they can see. They do not accept a product that quietly gets worse because the margin ran out.
Instrument it on day one
Log tokens per request against a user id and a feature name from the first deploy. It takes an hour and it is the difference between “AI is expensive” and “this one action, used by these forty accounts, is eighty percent of the bill”.