AI & Technology

The Token Bill: Why AI Costs More Even as Tokens Get Cheaper


  • Written by
    Lokesh Goswami
  • Posted on
    Nov 14, 2026

A field guide to cutting AI token costs, for engineers and leaders

Tokens keep getting cheaper, but most teams’ AI bills continue to rise. Both trends can be true, and the gap between them is not simply a pricing problem. Effective AI cost optimization depends on how many tokens your systems consume, how efficiently they use AI models, and how well infrastructure is managed. These are engineering and management decisions that directly impact overall spending. This guide explores where AI budgets go, what engineers and business leaders should consider, and the practical strategies that reliably help reduce AI costs without compromising performance.

Cheaper tokens, bigger bills

A token is the unit a model reads and writes, roughly three quarters of an English word. Providers charge per million tokens, and on flagship models output costs about five times as much as input. Claude Opus 5, for example, lists at $5 per million tokens in and $25 out [4].

For the same capability, prices fall fast: about 10x a year by a16z’s estimate [5] and about 13x a year since 2023 by Epoch AI’s [1]. Usage grows faster. OpenRouter went from about 100 trillion to about 1.5 quadrillion tokens a year in roughly twelve months [6], and Google alone now processes 3.2 quadrillion tokens a month [7]. Gartner forecasts $2.7 trillion of AI spending in 2026, up 49.5% [2]. Economists call this the Jevons paradox: when a resource gets cheaper, total use rises enough that total spending rises too [8]. Teams also keep moving to the newest frontier models, so the average price actually paid per token fell only 6% in the first five months of 2026 [9].

The arithmetic behind every AI bill is simple, and it shows where control sits:

The market sets the price per token and is already cutting it for you. The other two terms are yours, and they are where the bill is won or lost.

Where the tokens actually go

The number that matters is tokens per task, and two changes have multiplied it. First, reasoning models think before they answer, and that thinking is billed as output. A reply that took about 7 tokens on a 2023 model can take 255 to 603 tokens with reasoning on [11]. Second, agents work in loops. Each step resends the whole conversation, so cost grows roughly with the square of the number of steps [12]. One study measured agentic coding tasks at about 1,000 times the tokens of code chat, and most of those tokens were input [13].

The loop effect is easy to underestimate, so it helps to run the numbers once. Take an agent that adds 5,000 tokens of new material at each step. If every step had to send only what was new, 20 steps would cost 100,000 input tokens. Because each step resends everything before it, the real figure is 5,000 × (1 + 2 + … + 20), or about 1.05 million tokens, ten times more. Doubling the steps roughly quadruples the bill.

Much of that resent material is the agent working out the same things again: reading the same files, retracing the same call paths, relearning the same project conventions in every session. Nothing carries over between runs, so the cost of discovery is paid again each time. The cheapest token is the one an agent never has to spend rediscovering something it already knew.

What people worry about

Engineering blogs, vendor documentation and surveys from 2025 and 2026 keep raising the same five concerns.

  • Bills nobody can predict.Paying per token turns AI into a variable cost that is hard to forecast [15]. In one widely shared case, an agent’s weekly bill went from about $127 to $18,400 within four weeks before anyone caught it [16]. In an EY survey, 98% of organizations using tokens said these costs made them reconsider their approach [17].
  • Context bloat.Tool definitions load into the context before any work starts. GitHub’s MCP server alone takes about 42,000 tokens [18] (Figure 3).
  • Loops and retries.Agent retry logic can push usage 3 to 7 times above the estimates made during development [11].
  • No visibility.Teams can’t see which agent, feature or team is spending. Without attribution metadata on each call, chargeback isn’t possible [19].
  • Fear of hurting quality.Teams worry that cutting tokens will make answers worse. Often the opposite holds: recall degrades as context grows [20], and agents given too many tools start picking the wrong ones [21].

Practices that work

Order matters. Caching and batching change how you pay without changing what the model sees, so they carry almost no quality risk, and their discounts are printed on the providers’ own price lists. Routing and context trimming change the model’s inputs, so they need an evaluation set before they ship. Teams that stack the first few levers routinely cut spend by 60 to 80% without a visible drop in quality [36].

Practice What it means Typical saving Owner
Prompt caching Put stable content (system prompt, tools, documents) first so repeat calls reuse it 50 to 90% on cached input [24, 25], up to 85% lower latency [26] Developer
Model routing Send easy requests to small models, hard ones to a frontier model 35 to 85%, keeping 95% of GPT-4 quality in RouteLLM tests [27] Platform
Batch APIs Run work that can wait (evals, backfills, classification) through batch 50% [28] Developer, platform
Lean context Fewer tools, compacted history, sub-agents that return short summaries Varies. A sub-agent may read tens of thousands of tokens and return 1,000 to 2,000 [20] Developer
Output control Cap max tokens, ask for concise structured output, lower reasoning effort Not quantified. Output costs about 5x input Developer
Loop limits Cap iterations and retries, fail fast, add circuit breakers and spend alerts Stops runaway bills like the $18,400 week Developer, platform

Where savings quietly disappear

Caching needs a byte-identical prefix. A timestamp or request ID at the top of a prompt silently breaks it, and the bill goes back to full price without any error [30]. Order prompts from most stable to most volatile, and watch the cache-hit rate the way you would watch a database’s.

Trimming context can break caching. Pruning or summarizing old turns changes the prefix, which invalidates the cache. A shorter prompt at full price can cost more than a longer one at the cached rate [37]. Compact at fixed boundaries, not on every turn.

Discounts don’t always stack. On Gemini 2.5, the cache discount replaces the batch discount instead of adding to it [29]. Model the combined price before promising a saving.

For leaders: measure outcomes, not tokens

Price per token is the wrong number to manage. Track cost per accepted task, resolved ticket or merged change, and expect that number to fall even while total spend rises [11]. IDC describes AI FinOps as tagging every cost to an owner and measuring it against a business outcome [31]. And plan for the full cost: once infrastructure, governance and change management are counted, EY estimates the enterprise cost of AI at roughly three times the token invoice [35].

  • Tag every callwith the team, feature and agent it belongs to. Start with showback, because chargeback before attribution can be trusted leads to disputes instead of savings [32].
  • Budget for growth, not today’s unit cost times today’s volume. Usage, context sizes and model upgrades grow faster than prices fall [33].
  • Know your baseline.For coding agents, Anthropic reports about $13 per developer per active day on average, and under $30 for 90% of users [34].
  • Pair every cost cut with a quality check.Routing and trimming context only pay off if the answers stay good.

Building cost-aware AI with Algosoft

Most of the levers above are cheapest to apply at design time, while architecture, model choice and evaluation are still open. Retrofitting caching, routing and spend visibility into a live product is slower and riskier. That is why Algosoft treats token cost as a product requirement rather than a post-launch surprise. As an AI product development company in Delhi NCR, Algosoft builds LLM products, retrieval-based search and agents with token cost controls, evaluation and monitoring included [38]. Here is how that maps to the practices in this guide:

  • Costs estimated before you build.Fixed-price proposals include model API and compute cost estimates up front, so inference spend is part of the budget conversation instead of a surprise after launch [38].
  • Proof on your data first.Every engagement starts with a proof-of-concept sprint on real data, with quality benchmarks reviewed before you scale. You see tokens per task early, when changing the design is cheapest [38].
  • Grounded, lean context.Retrieval-augmented generation brings in the relevant documents instead of stuffing everything into the prompt, which supports the lean-context practice above. See Algosoft’s generative AI development work [38].
  • Agents with guardrails.Multi-step agents ship with approval gates on high-risk actions, audit trails and a documented fallback path when the model fails, the same control mindset as the loop limits above [38].
  • Monitoring after launch.Dashboards track accuracy, latency and cost, and cost optimisation continues as usage scales [38].

Planning an AI product, or already watching the bill climb? Algosoft offers a free AI proposal within 48 hours: share the problem and the data you have, and the team returns a feasibility assessment, cost estimate and architecture [38]. Explore Algosoft’s AI solutions or talk to the team.

Frequently asked questions

Why do AI bills rise when tokens keep getting cheaper?

Because usage grows faster than prices fall. Price per token for the same capability drops roughly 10 to 13 times a year [1], [5], but token volume has grown even faster [6]. Teams also keep moving to the newest frontier models, so the average price actually paid fell only 6% in the first five months of 2026 [9]. This is the Jevons paradox: cheaper resources get used more, and total spend rises [8].

What is a token, and how is it priced?

A token is the unit a model reads and writes, roughly three quarters of an English word. Providers charge per million tokens, and on flagship models output costs about five times as much as input [4].

Why do AI agents cost so much more than a chat?

Reasoning models bill their thinking as output, and agents work in loops where every step resends the whole conversation, so cost grows roughly with the square of the number of steps [11], [12]. One study measured agentic coding tasks at about 1,000 times the tokens of code chat [13].

What is the fastest way to cut token costs?

Start with prompt caching and batch APIs. They change how you pay without changing what the model sees, so they carry almost no quality risk. Cached input is discounted 50 to 90% [24], [25], and batch APIs save about 50% [28]. Teams that stack the first few levers often cut spend by 60 to 80% [36].

Will using fewer tokens make answers worse?

Often the opposite. Recall degrades as context grows [20], and agents given too many tools start picking the wrong ones [21]. Routing and context trimming do change what the model sees, so test them on an evaluation set before you ship.

Why is my prompt cache not saving money?

Caching needs a byte-identical prefix, so a timestamp or request ID at the top of a prompt silently breaks it [30]. Trimming or summarizing old turns can also invalidate the cache [37]. Put stable content first, and watch your cache-hit rate.

What should leaders measure instead of price per token?

Cost per accepted task, resolved ticket or merged change [11]. Tag every call with the team, feature and agent it belongs to, start with showback before chargeback [32], and budget for growth rather than today’s unit cost times today’s volume [33].

How can Algosoft help us build AI without runaway costs?

Algosoft’s fixed-price AI proposals include model API and compute cost estimates up front, projects start with a proof of concept on your real data, and launches include monitoring for accuracy, latency and cost. You can request a free AI proposal within 48 hours [38].

Conclusion

Cheaper tokens will not shrink your bill by themselves. Prices are set by the market, but the number of requests and the tokens each task consumes are set by your own design choices. The teams that win treat tokens as an engineering budget: they make spend visible, cache what repeats, route work by difficulty, keep context lean, and measure cost per accepted task rather than price per token.

Start with the low-risk levers, caching and batching, then add routing and context trimming behind an evaluation set. Build cost awareness in at design time, and the falling price of tokens becomes real savings instead of a bigger invoice.

The bottom line

Tokens will keep getting cheaper. Whether your bill shrinks depends on how many tokens you send. Make spend visible, cache what repeats, route by difficulty, keep context lean, and judge the result by the work delivered rather than the tokens consumed. Falling prices help everyone equally. The teams that treat tokens as an engineering budget are the ones that turn those falling prices into savings.

References

[1]  https://epoch.ai/publications/the-plunging-price-of-thought

[2] https://gcom.pdo.aws.gartner.com/en/newsroom/press-releases/2026-09-16-gartner-forecasts-worldwide-ai-spending-to-grow-49-point-5-percent-in-2026

[3]  https://www.insentragroup.com/us/insights/not-geek-speak/generative-ai/token-spending-and-ai-roi-are-you-actually-getting-your-moneys-worth/

[4]  https://intuitionlabs.ai/articles/llm-api-pricing-cost-per-task-comparison

[5]  https://a16z.com/llmflation-llm-inference-cost/

[6]  https://menlovc.com/perspective/openrouter-now-processes-more-than-a-quadrillion-tokens-a-year/

[7]  https://shacknews.com/article/149205/google-3-2-quadrillion-monthly-ai-tokens

[8]  https://www.bighatgroup.com/blog/jevons-paradox-ai-cheaper-tokens-bigger-bills/

[9]  https://www.yipitdata.com/resources/cloud-llm-pricing-trends

[10]  https://substack.com/home/post/p-211786639

[11]  https://tianpan.co/blog/2026/04/14/the-inference-cost-paradox

[12]  https://www.augmentcode.com/guides/ai-agent-loop-token-cost-context-constraints

[13]  https://arxiv.org/abs/2604.22750v1

[14]  https://tianpan.co/blog/2026/07/02/who-pays-for-the-tokens

[15]  https://www.theinformation.com/articles/real-cost-ai-survey-unpredictable-token-economics

[16]  https://yusufhansacak.medium.com/the-46-001-agent-bill-what-the-viral-token-stories-get-wrong-7ee1cdd81e65

[17]  https://www.ey.com/en_us/newsroom/2026/07/ey-survey-c-suites-pivot-from-ai-adoption-to-unlocking-value-as-escalating-token-costs-trigger-fiscal-scrutiny

[18]  https://dev.to/thedailyagent/mcp-tool-overload-why-more-tools-make-your-agent-worse-5a49

[19]  https://insights.technative.eu/enterprise-ai-token-management/

[20]  https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

[21]  https://albato.com/blog/publications/embedded-mcp-context-bloat-hallucinations

[22]  https://www.apideck.com/blog/mcp-server-eating-context-window-cli-alternative

[23]  https://modelcontextprotocol.io/docs/2026-07-28/develop/clients/client-best-practices

[24]  https://intuitionlabs.ai/articles/llm-prompt-caching-cost-savings

[25]  https://openai.com/index/api-prompt-caching

[26]  https://aws.amazon.com/cn/blogs/aws/reduce-costs-and-latency-with-amazon-bedrock-intelligent-prompt-routing-and-prompt-caching-preview

[27]  https://lmsys.org/blog/2024-07-01-routellm

[28]  https://community.openai.com/t/regarding-the-issue-of-half-priced-prompt-caching/990681/1

[29]  https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/batch-inference

[30]  https://dev.to/gabrielanhaia/anthropic-prompt-caching-saves-90-heres-the-one-caveat-nobody-mentions-258k

[31]  https://www.idc.com/resource-center/definitions/what-is-ai-finops/

[32]  https://www.gmicloud.ai/en/blog/finops-for-ai-infrastructure-unit-economics-chargeback-and-cost-forecasting

[33]  https://www.techuz.com/blog/token-bill-shock-genai-unit-economics/

[34]  https://code.claude.com/docs/en/costs

[35]  https://www.ey.com/en_us/insights/ai/agentic-ai-return-on-investment

[36]  https://www.glukhov.org/llm-performance/cost-effective-llm-applications/

[37]  https://arxiv.org/html/2606.17016v2

[38]  https://www.algosoft.co/locations/ai-product-development-company-delhi-ncr