Every few months a model provider cuts its prices, and founders breathe a little easier. Then the invoice arrives, and it is bigger than last month's.
That is not bad luck. It is the new shape of AI economics. The price of a single token keeps falling, but the number of tokens it takes to deliver a useful result is climbing faster. This week brought some of the clearest evidence yet—and some of the clearest examples of what to do about it.
The founder takeaway: stop managing AI as a monthly line item. Manage it as a cost per outcome, the same way you manage cloud spend per customer. The companies that do this well are cutting their bills by two-thirds or more without giving up quality.
The inference paradox
Gartner calls it the "inference paradox." The firm predicts that inference cost per agentic workflow will rise more than fivefold through 2028, even as token costs fall by roughly 95% by 2030 (Computerworld).
The reason is simple. A chatbot reads a question and answers it. An agent plans, calls tools, checks its own work, retries, and talks to other agents. Gartner says agents need 5 to 30 times more tokens than a basic chatbot for the same task (VentureBeat).
As Gartner analyst Will Sommer put it: "Product leaders cannot rely on more efficient token economics to rationalize AI costs" (VentureBeat).
The pressure is coming from the top, too
Your providers are feeling the squeeze as well. Anthropic's amended IPO filing this week showed compute costs rose to 58% of revenue in the third quarter, up from 48% a quarter earlier. The filing also noted that capacity bought under short-term commitments costs significantly more than multi-year capacity (The AI Insider).
Elsewhere, xAI reduced weekly token limits on its Grok bot because operating costs were too high relative to subscription revenue (This Week in NLP). And OpenAI's updated usage policies, effective November 20, require model routers and AI gateways that give third parties direct access to its models to get prior approval. Apps that simply use OpenAI models to power their own features are not affected (OpenAI).
None of this means prices are about to spike. It does mean the terms you build on today can change, and a product whose margins only work at today's list prices is fragile.
What the winners did this fall
Two detailed case studies landed in the past few weeks, at very different scales. Both are worth studying.
Cloudflare: 68% cheaper by fixing the basics
Cloudflare runs an AI system that reviews its engineers' code. Over the summer, the average cost per review crept up as the team adopted better, more token-hungry models. Then it went looking for where the tokens were going (Cloudflare).
The answer was mostly the cache. Cache writes made up 43% of the bill. The first request in each session found a usable cache only 14% to 26% of the time, because shared instructions were mixed in with details that changed on every run, like the date and working directory. Splitting them apart raised the first-request hit rate to 91%–99.5% (Azeem Hassni).
Combined with fewer wasted calls and better model and effort choices, the average review fell from $2.43 to $0.77—and the median review got faster, from 4 minutes 9 seconds to 1 minute 32 seconds (Cloudflare). About 40% of the savings came from code changes alone; model and effort changes produced the rest (Art of Saience).
Cloudflare also prices work by risk. In its earlier setup, a trivial review averaged $0.20 while a full multi-agent review averaged $1.68 (Cloudflare). Not every task deserves your most expensive path.
Revolut: 90% cheaper by owning the default
At a much larger scale, Revolut says it cut AI coding inference costs by about 90% in under a year by making self-hosted open-weight models the default for roughly 2,000 engineers. It serves them on 56 Nvidia B200 GPUs at a blended cost of about $0.30 per million tokens (The Register).
Revolut did not drop commercial models entirely. Its model is text-only, so a router sends vision tasks to other providers. "The frontier is the exception," said David Abitbol, Revolut's ops and infrastructure lead (The Register). Before the switch, he said, "We were seeing engineers spend $1,000 a month, $10,000 a month with not so much ROI" (AI Weekly).
Five levers for AI cost per outcome
Here is the order we recommend to founders. The first four apply to almost every startup. The fifth is for companies with real scale.
Pull the levers in order: the cheapest savings come from measurement and caching, not new infrastructure. Illustration: SAMO Technologies.
1. Meter it
You cannot fix what you cannot see. Tag every model call with the feature, customer, and workflow it serves. Track input, output, cache-write, and cache-read tokens separately. Then calculate the number that matters: cost per resolved ticket, per completed report, or per active customer, compared with the revenue that outcome earns.
If your AI cost per customer is rising faster than your price per customer, you have a gross-margin problem, whatever your total bill says.
2. Cache it
Prompt caching is the closest thing to free money in AI infrastructure, and most teams leave it on the table. Put stable content—system instructions, tool definitions, reference material—at the start of the prompt. Move anything that changes per request, like dates, user details, or file paths, to the end. Then verify that your first request in a session actually reads from the cache. Cloudflare's biggest single saving came from checking exactly that (Cloudflare).
3. Route it
Send each task to the cheapest model and effort level that still meets your quality bar. Classification, extraction, and formatting rarely need a frontier model. Save the expensive path for complex reasoning and high-stakes decisions. Measure quality before and after each change so savings do not quietly turn into customer complaints.
4. Trim it
Look for waste inside each workflow: tools the agent tries to call but is not allowed to use, repeated file reads, redundant reviewer steps, retry loops, and context windows stuffed with documents the model does not need. Each one adds tokens to every run. This is the same discipline we described for agent systems in Agentic AI: Build for Production, Not Demos.
5. Own it—when the math works
Self-hosting open-weight models or committing to reserved capacity can cut costs dramatically, as Revolut showed. But it only works when demand is high and predictable enough to keep hardware busy, and when you have the team to run it. Before you commit, model the full cost: hardware or lease, power, networking, on-call engineering, and the cost of falling behind frontier quality. Keep a router in place so the frontier remains an option.
A two-week AI cost sprint
- Days 1–3: Export the last 30 days of model usage. Tag spend by feature and customer. Find your top three cost drivers.
- Days 4–7: Audit caching on those three workflows. Restructure prompts and confirm hit rates on the first request.
- Week 2: Test a cheaper model or lower effort level on one workflow with a clear quality check. Remove at least one unnecessary call or tool.
- End of week 2: Report cost per outcome for each workflow and set a monthly target.
If AI coding tools are a big part of your bill, pair this with our guidance on why review, not code generation, is now the bottleneck. Spending more on code nobody can review is the most expensive token of all.
Frequently asked questions
Why are AI costs rising if token prices are falling?
Because modern AI features use far more tokens per task. Agents reason across steps, call tools, retry, and pass large context between models. Gartner predicts inference cost per agentic workflow will rise more than fivefold through 2028.
What is the fastest way to cut AI inference costs?
Measure where tokens go, fix prompt caching, and remove unnecessary model calls. Cloudflare cut its AI code review costs by 68% mostly this way, plus matching model and effort level to each task.
Should a startup self-host open-weight models?
Only when usage is high and steady and you have the team to run it. Most early-stage startups get more from caching, routing, and trimming first.
Which AI cost metric should founders track?
Cost per business outcome—such as cost per resolved ticket or per active customer—compared with the revenue that outcome earns.
Where SAMO fits
At SAMO Technologies, we help founders build AI products with margins that hold up at scale. That means usage metering, prompt and cache design, model routing, workflow trimming, and honest build-versus-buy analysis on self-hosting—backed by the same cloud cost discipline we bring to the rest of your infrastructure.
If your AI bill is growing faster than your revenue, talk with SAMO Technologies. We will find where your tokens are going and the fastest way to bring cost per outcome down.