This website uses cookies to help improve your user experience
Almost every conversation about AI engineering spend starts in the wrong place. Somebody quotes a number, usually a large one, and the room divides into people who think it proves the technology is expensive and people who think it proves the opposite. Both sides are answering a question nobody asked.
The number on an API invoice is not the cost of the work. It is one component of it, sitting alongside the engineering time that went into supervising the agent, the infrastructure the agent called while it ran, and the cleanup afterward if the output was not good enough. Add those together and the picture often reverses.
There is a harder problem underneath, as even a complete cost figure tells you nothing about whether the money was well spent, because two projects with identical bills can produce wildly different outcomes. Cost and value are separate measurements, and most teams are only taking one of them.
For this Expert Talks edition we asked an Oxagile Lead to work through both, starting with the most public example anyone has.
Alexey Karankevich, AI Innovation Lead at Oxagile, builds agent-driven workflows on systems where the stakes are real and the codebases are old. He sees the consumption data most teams never look at closely: not the monthly total, but which decisions produced it.
Key takeaways:
In May 2026, the team behind Bun did something unusual. They rewrote their JavaScript runtime from Zig to Rust in eleven days, and then they published the bill1.
Roughly 535,000 lines of Zig across about 1,450 files, 6.5 thousand commits, and around $165,000 of inference at published API prices. Most companies would never let those figures out of the building.
The reaction split within a day. One camp read $165,000 and concluded that agentic engineering is an expensive toy. The other read eleven days and concluded the opposite. Neither camp did the arithmetic that would settle it, which is what a rewrite at that scale costs when people do it.
That comparison is the whole argument. A full language migration on a half-million-line codebase is measured in engineer-years, not engineer-weeks. Set $165,000 against a team of senior engineers working for the better part of the year, and the invoice stops looking like an outlier.
But the comparison is also incomplete, and this is where it gets interesting. The Bun rewrite was not unattended. A Lead Engineer monitored the output continuously, reviewed code by hand, and adjusted the workflows when they drifted. Several hours went into a porting guide before any code was generated. That time is real, it is expensive, and it is not on the invoice.
Question for Alexey
When you look at the Bun numbers, what does the $165,000 leave out? If you had to put a rough multiplier on the true all-in cost of that project, what would it be?
Alexey
You are right, the $165,000 is fundamentally an inference expenditure, not the cost of the full rewrite. You also have the engineering time spent designing the harness, workflow, preparing the porting guide, supervising agents, reviewing their output, investigating failures, running validation, and maintaining the environment around the agents.
I would be careful about putting a universal multiplier on it, but we know for sure it depends enormously on how autonomous the workflow is and how much human verification is required.
Based on my experience, for a heavily supervised engineering workflow, I would expect the real cost to be materially higher than the API bill potentially 1.5 to 3 times once engineering labor and supporting infrastructure are included. For a poorly designed autonomous workflow, it can be much higher because cleanup becomes the dominant cost.
Run the same logic in the other direction, and it holds. An agent that consumes 100,000 tokens and saves two weeks of engineering time is close to free. An agent that consumes $500 and produces code that takes another week to clean up is one of the most expensive things you can buy, and the invoice will tell you it was cheap.
Question for Alexey
Have you seen a project where a small AI bill turned out to be a bad deal once the cleanup was counted? What happened?
Alexey
Yes, and it is actually quite common case for early adopters. The dangerous cases are not the spectacular failures where the agent burns thousands of dollars, they’re the ones where it produces something that looks almost correct.
This is particularly problematic in domains where correctness is expensive to establish. An AI output that looks correct can become extremely expensive when an engineer has to spend hours proving that it isn’t.
Which points at the question worth asking. Move past “How much did the AI cost?” to “How much did it cost to reach the outcome?”.
Suppose the number is $200 for a given month. What did that buy?
The visible layer is straightforward and it is what the pricing pages describe. Input tokens, output tokens, the model chosen, the number of requests, and how much context each request carried. Anyone can reason about those.
Underneath there are three layers that most cost conversations never reach.
Question for Alexey
Of those three hidden layers, which one surprises teams most when you show them where their money went?
Alexey
Context accumulation, in my experience, is the least intuitive one. Engineers naturally think about a conversation as a sequence of messages. However, the model may be processing a large amount of (or even the same) context again and again. Once you have an agent operating over a repository, previous tool outputs, documentation, test results and system instructions, the amount of information flowing through the system can become enormous.
Agentic iteration is easier to understand once you instrument it. Context is harder because people don’t intuitively see repeated information as repeated consumption.
Then there is the layer that matters most and often gets left out of the calculation entirely. Somebody has to formulate the task, supervise the agent, review the output, debug the failures, validate correctness, deal with hallucinations, maintain the prompts, and keep the infrastructure running. All of that costs engineering time, priced accordingly, and entirely absent from the invoice.
Question for Alexey
At what point does cheap AI become expensive human supervision? Is there a signal you watch for?
Alexey
It’s hard to name a single specific signal, but if I aggregate all that class of signals, I can say I watch for a human intervention per successful outcome rate.
If an agent is cheap but constantly requires an engineer to redirect it, explain what it misunderstood, inspect intermediate results, and clean up its work, then the agent isn’t really autonomous. You’ve simply moved part of the computation from the model to a much more expensive biological processor.
There is a useful economic threshold here: once the marginal human supervision required by a cheaper model exceeds the additional inference cost of a better model (if possible), the cheaper model has stopped being cheaper.
Traditional software has predictable economics: CPU, memory, storage, bandwidth, licenses. You can model it in a spreadsheet and the spreadsheet will be roughly right.
AI introduces something that behaves less like infrastructure and more like a person having a good or bad day.
Take one well-documented bug, given to two agents with the same model behind them. The first reads three files, locates the problem, fixes it, runs the tests, and stops. Maybe four minutes of wall time.
The second reads the entire repository because nothing told it where to look. It forms three wrong assumptions about the cause, modifies several files it did not need to touch, runs the test suite repeatedly, gets stuck for a while, backs out some of its own changes, and eventually arrives at the same fix.
It’s the same ticket, same model, with the same outcome, and yet the bills are not comparable.
Question for Alexey
How wide have you seen that gap get in practice on real projects? What separated the two runs?
Alexey
Potentially an order of magnitude, and in poorly constrained agentic workflows even more.
The interesting fact is that the difference doesn’t necessarily come from the model, it comes from the trajectory. One agent can find the relevant files immediately with grep while another can explore the whole repository, follow irrelevant dependencies, make a wrong hypotheses, and spend multiple iterations recovering from it.
That’s why token consumption can’t serve as a productivity metric. We have to look at the trajectory-to-outcome relationship instead.
Good news is that modern models become smarter day after day making fewer mistakes. But we shouldn’t forget the probabilistic nature of any LLM that keeps our efforts related to managing uncertainty still valuable.
This is the part that breaks people’s mental model. Cloud infrastructure costs what it costs. An AI agent’s cost depends on how well it works, which depends on how the work was set up for it.
Question for Alexey
Is AI infrastructure becoming less like a machine and more like an employee whose productivity shows up on the bill?
Alexey
It is a useful analogy indeed, but I wouldn’t take it literally.
Traditional infrastructure has relatively deterministic economics: give a server a certain workload and you can predict its resource consumption fairly well. But an agent has something closer to behavioral variance, as its productivity depends on the task, context, tools, instructions, and even the trajectory it takes through the problem.
So economically we’re moving toward something that looks more like an employee indeed. You don’t really care how many CPU cycles an employee consumes, you care how much useful work they accomplish relative to their total cost.
The analogy breaks because AI is still fundamentally a computational system that can be instrumented much more precisely than a human, even if its reasoning remains limited compared with human thought processes.
Five things determine what a piece of agentic work costs, and they are not equally within reach.
Question for Alexey
Which of these can an engineering team actually control, and which do they just have to live with?
Alexey
Almost all of them, at least indirectly. You can’t control the provider’s price, but you can control model selection, routing, context construction, retrieval, caching, agent autonomy, iteration limits, tool access, task decomposition, and observability. That’s a lot, good news again!
“Use a cheaper model” is where most advice on this subject begins and ends. It is one of the more obvious items on a list that runs much longer.
The most powerful model does not need to handle everything. A small model handles the easy task, a larger one the difficult task, a frontier model the genuinely hard one. Most workloads contain far more easy tasks than hard ones, which is what makes routing worth building.
Question for Alexey
Should we think about models the way we think about grades of engineers? Where does that analogy break?
Alexey
It’s a useful analogy for understanding routing, but a dangerous analogy if taken too far.
You might have a small model handling routine transformations, a stronger model handling debugging, and a frontier reasoning model handling ambiguous architectural problems. That’s somewhat analogous to assigning work according to skill level indeed, but the fact is that models don’t behave like engineers.
A more expensive model isn’t simply a “senior engineer”. It can be dramatically better at one class of problems and surprisingly bad at another. And unlike humans, models can be instantiated thousands of times.
So, I’d say models are more like different computational capabilities than different employee grades.
A large share of AI waste is context waste. Information the model did not need, sent repeatedly, at full price each time. The countermeasures are unglamorous: compression, retrieval instead of wholesale inclusion, summarization of prior turns, structured state, and simply not resending what has not changed.
Question for Alexey
How much of a typical bill is information the model never needed? Do you have an example where fixing context alone moved the number?
Alexey
There is no universal percentage because it depends heavily on harness architecture and codebase specifics, but I have seen context become a majority component of consumption in poorly optimized agentic workflows.
The important point is that context waste doesn’t necessarily look like waste. A repository may be useful once, but sending the same repository information repeatedly across dozens of iterations is economically very different.
The biggest improvements usually come from changing the harness architecture instead of optimizing individual prompts: retrieval, structured state, summarization, caching, and keeping stable information outside the active context.
Context is becoming a new form of technical debt, it’s information that is useful to the system but expensive to carry around.
An agent with no limits is a machine for converting budget into logs. Iteration caps, tool restrictions, explicit budgets, stopping criteria, decomposing a large task into smaller ones, and choosing a deterministic workflow where autonomy adds nothing.
Question for Alexey
Should every agent have a budget? What happens in practice when one hits it?
Alexey
Yes, if the agent can take actions that incur meaningful cost.
The budget doesn’t necessarily have to be a hard dollar limit, it can be a combination of token budget, number of iterations, tool calls, execution time, and escalation rules.
More importantly, hitting the budget should be treated as a signal, not merely an exception. The system should stop, summarize what it knows, explain why it failed to converge, and either ask for human intervention or escalate to a more capable workflow.
If the same information gets requested repeatedly, paying for it repeatedly is a choice. Prompt caching, semantic caching, result caching, and shared context between agents working the same problem.
The Bun rewrite is instructive here. Cached input reads outnumbered uncached input tokens by more than ten to one. That ratio came from how the work was structured and had nothing to do with the provider.
Question for Alexey
What is a realistic cache hit ratio for a team that has actually thought about it, against one that has not?
Alexey
I wouldn’t publish a universal benchmark because cacheability is extremely workload-dependent.
But the Bun example demonstrates the important principle: very high ratios are possible when the workflow naturally reuses the same context. If the same repository, instructions or documents are repeatedly referenced, failing to cache them is essentially paying for the same computation repeatedly.
The right question isn’t “What cache hit ratio should we achieve?” It’s “What information in our workload is stable, and why are we processing it repeatedly?”
You cannot optimize what you do not measure, and most teams are running agents with less instrumentation than they would accept on a web service. Traditional monitoring watches latency, CPU, and memory. Agentic work needs tokens, cost, model, task, the agent’s trajectory through the problem, success rate, and how often a human had to step in.
Question for Alexey
Will AI observability become as important as traditional APM? What are teams flying blind on right now?
Alexey
I think it will become at least as important for agentic systems, but it will measure a different kind of behavior.
The traditional approach tells you whether the system is healthy: latency, errors, resource consumption, and so on. AI observability also has to tell you whether the system is behaving economically and intellectually.
Without that information, you’re effectively running production software without logs.
Here is where the standard “reduce the cloud bill” conversation stops being useful.
Model A costs $1 per task and succeeds 60% of the time. Model B costs $10 and succeeds 95% of the time. Model A looks like a 90% saving.
Work it through. To get 100 successful outcomes from Model A you run roughly 167 tasks, at $167, plus the engineering time spent identifying and rerunning 67 failures, plus whatever damage a failure causes downstream if it slips through. Model B costs $1,000 and produces 95 successes with almost no intervention. Depending on how expensive a failure is, either can win, and the per-token price tells you nothing about which one it is.
The unit that answers the question is cost per successful outcome (not dollars per million tokens). Dollars per completed task, and beyond that, total cost of ownership per successful outcome, which absorbs the inference, the infrastructure, the engineering time, the human review, the failures, and the opportunity cost of the delay.
Question for Alexey
Do you actually calculate cost per successful task on projects? How do you handle the failure cost, which is the hardest part to price?
Alexey
Well, that’s the metric I used to watch on, but in practice many organizations are still earlier in the maturity curve.
I would separate avoidable failure cost from normal operating cost. If a failed run requires ten minutes of engineering intervention, that time should be attributed to the task, but if it causes a production incident, the economics become dramatically different.
Ultimately, the metric I’d like is (total cost of ownership / successful business outcome) rather than (model cost / tokens consumed). I believe it’s a pretty good and fairly simple start.
Suppose a company spends $10,000 on AI this quarter. How would anyone know whether that produced $10,000 of value or $100,000? Three dimensions are worth separating.
You’re only four quick answers away from an AI ROI estimate.
“AI saved us 100 developer hours” is the most quoted number in this field and one of the least useful.
Ask what happened to those hours. Almost nobody went home early, and payroll stayed largely the same. The hours went into building more features, improving quality, taking on harder problems, or reducing a backlog that had been sitting there for two years.
None of that is a cost saving, as all of it is capacity.
The distinction is worth naming precisely:
AI is occasionally the first and much more often the second, and teams that report it as the first end up in an awkward conversation when the finance team asks where the savings went.
Question for Alexey
When a client asks you to justify AI spend in savings terms, how do you answer? Do you push back on the framing?
Alexey
I prefer to push back slightly on the framing. If AI allows ten engineers to accomplish the work previously requiring fifteen, that’s not necessarily a saving. If the company uses the additional capacity to build more products, improve quality, or enter a new market, it is leverage.
There are genuine cost-saving cases, of course, especially automation of repetitive processes. But in software engineering, I think capacity is often the more honest and more valuable metric.
So I’d ask the client: “What will you do with the capacity AI creates?” hoping that question will lead to a much more productive business discussion.
Everything above describes a team, yet organizations behave differently. Ten people experimenting with AI is a manageable situation. Ten thousand people using it produces a set of problems that have nothing to do with model quality: duplicated subscriptions across departments, API usage nobody is tracking, shadow AI running on personal accounts, company data going somewhere it should not, inconsistent model choices between teams solving identical problems, infrastructure built three times, no governance, and a monthly bill nobody can forecast.
The response usually involves a centralized platform, an internal gateway, an approved model catalog, per-team budgets, usage policies, and observability across all of it. Which raises a question about what AI actually is inside a company.
Question for Alexey
Should AI be treated as another IT utility, like storage or bandwidth, or as a new organizational capability that needs its own operating model?
Alexey
Both, but at different layers. At the infrastructure layer, AI should increasingly look like a utility with centralized gateways, model catalogs, observability, security, cost controls, and standardized access.
But treating AI purely as a utility is a mistake because the competitive advantage isn’t access to a model, everyone can buy access to the same models. The advantage comes from how an organization integrates intelligence into its processes, data, products, and decision-making.
I’d treat AI as a utility at the platform layer and as an organizational capability at the business layer.
The honest answer is that nobody knows, but the direction of several trends is visible.
Compute gets cheaper per unit of work, models get smaller for the same capability, hardware improves, open models keep closing the gap with frontier ones. If those hold, raw intelligence becomes abundant and very cheap.
Something else becomes scarce in that world:
Question for Alexey
If intelligence gets cheap, which of those scarce resources are teams most likely to be unprepared for?
Alexey
I think human attention and trust are the two biggest ones. If generating an answer, a piece of code, an analysis, or a design becomes almost free, the bottleneck shifts toward determining what deserves attention and whether the output can be trusted.
This creates an interesting inversion: today we’re worried about AI being too expensive to generate enough intelligence, but in the future, we may have more generated intelligence than humans can evaluate.
Organizations that prepare for that will invest in verification, provenance, evaluation systems, and high-quality context, not simply in access to more models or agents.
There is a comparison worth ending on. Nobody opening a laptop asks whether the electricity was expensive, they ask whether the thing they did with it was worth doing.
The cost of the input became small enough and predictable enough to disappear from the decision. AI may follow the same path. Today the conversation is tokens multiplied by price, however, the version that matters is intelligence multiplied by outcome.
Question for Alexey
Does the electricity comparison hold, or does it break somewhere important? What would have to be true for AI to actually get there?
Alexey
Partially, but I think the comparison breaks in one important place. Electricity is largely fungible: a kilowatt-hour from one source can perform essentially the same physical work as a kilowatt-hour from another. Intelligence isn’t like that, the value of an inference depends heavily on context, reliability, latency, integration and the consequences of being wrong.
Therefore, AI won’t become economically invisible merely because tokens become cheap.
For the electricity analogy to really hold, three things would need to happen: intelligence would need to become cheap, reliable, and sufficiently standardized, while the surrounding infrastructure would have to make it easy to consume without thinking about the underlying model.
And I suspect that even then, the conversation wouldn’t disappear, it would simply move up one level, from “How much did the artificial intelligence cost?” to “What did we accomplish with it?”
Leave an inquiry about your project or case, and Oxagile’s specialists will get back to you.
1. How we rewrote Bun in Rust — Bun
For teams using agents seriously, monthly consumption commonly runs from a few hundred to several thousand dollars per engineer. The spread depends more on the codebase and the workflow design than on the individual, and the API bill excludes supervision time, tool execution, and cleanup.
Because cost depends on how the agent behaves, not only on what it was asked. One run may read three files and finish, and another may read the whole repository, form wrong assumptions, and iterate. It’s the same model with an identical outcome, yet the consumption is very different.
Consumption rises faster than unit prices fall. As teams raise their standard for output quality, they add reasoning steps, evaluation loops, and verification passes, with each one paid for with additional inference. A maturing practice tends to consume more, not less.
No. A model at a tenth of the price with a 60% success rate can cost more per completed task than an expensive model succeeding 95% of the time, once reruns, engineering time, and the cost of failures are included.
Per successful outcome instead of per million tokens. And ideally total cost of ownership per successful outcome, which includes inference, infrastructure, engineering time, human review, failures, and delay.
Usually capacity. Hours freed up tend to go into more features, better quality, or harder problems, not reduced headcount. Efficiency means the same output for less and leverage means more output for the same (AI is more often the second).
