What Krishna said on the 21 October earnings call

On IBM’s third-quarter earnings call on 21 October, chief executive Arvind Krishna put numbers on a comparison enterprise buyers have so far made by feel. The Register’s Simon Sharwood reported the remarks on 22 October, quoting Krishna directly: open-weight models now cost “anywhere from 25 percent to 10 percent of the token cost of the frontier models. That is very attractive given that they’re now 90 percent as effective.”

Treat that as reported speech. Krishna named no benchmark, no evaluation and no source for the ratios. The Register quotes him, and Benzinga ran a headline version of the same line around 25 October, “IBM CEO Says Open AI Models Are 90% as Good as Frontier Models: ‘Why Pay 10x More?’”, with a body we could not access. Both trace to the same spoken remarks.

IBM’s written disclosure does not carry the claim. The press release attached to the call, published 21 October, says nothing about open weights, token cost or an effectiveness figure. IBM’s prepared remarks and its official webcast transcript are available only as PDF downloads, and the webcast audio was not transcribed, so whether the percentages appear in IBM’s own script remains unchecked. The percentages live in a spoken claim reported by others, not in the written record IBM published.

Krishna framed the comparison as a trend. In 2024, he said, a “broad-based, not narrow” assessment put the top model near 70 and the best open model at 20 or 30. By late 2025 the gap had narrowed to roughly 70 against 50. Now he puts open weights at 10 to 25 percent of frontier token cost while being, in his words, 90 percent as effective. The tasks he called good enough for the cheaper option were ordinary: summarising an email, examining a contract, answering a customer call, composing an email. He allowed that some tasks still need the frontier model.

Pricing the claim per workload, on public list prices

We took public list prices as of 26 October and ran the arithmetic. Claude Opus 5.5 lists at $4 per million input tokens and $20 per million output, with cache reads at $0.20 per million; Fable 5.1 lists at $10 and $50.

Among open-weight models sold as managed APIs, Z.ai’s GLM-5.2 lists at $1.40 input and $4.40 output, cached input $0.26; Moonshot’s Kimi K3 at $3.00 and $15.00, cached input $0.30; DeepSeek V4.1 Flash at $0.30 and $1.20, cached input $0.006. Every ratio below is our arithmetic on those list prices, not a measured cost.

GLM-5.2 comes out at 35 percent of Opus 5.5 on input and 22 percent on output. Against Fable 5.1, the dearest frontier price on the list, the same two ratios are 14 percent and 8.8 percent. Kimi K3 is 75 percent of Opus 5.5 on both, a quarter cheaper at best. DeepSeek V4.1 Flash reaches 7.5 percent of input and 6 percent of output, below the range Krishna gave. His band appears on one side of each comparison, never on both: 22 percent of Opus 5.5’s output price, or 14 percent of Fable 5.1’s input price.

Caching moves the comparison again, and the profile IBM and Red Hat published is unusually input-heavy. It is drawn from SemiAnalysis’s 219 production Claude Code sessions: a median main-agent request carries about 195,000 input tokens against 317 output tokens (the two counts come from different tokenizers, so the ratio mixes units), 96 percent of requests reuse at least 90 percent of the previous input, and input makes up 98.6 percent of all tokens served. That is coding-agent traffic, not a general enterprise sample. On a profile this input-heavy, cached input is the dominant cost line, and Opus 5.5’s $0.20 cache-read price sits below GLM-5.2’s $0.26 and Kimi K3’s $0.30. On the largest line of that workload the frontier model is cheaper than two of the three open models above; only DeepSeek V4.1 Flash, at $0.006 cached, lands well below it. With caching priced on both sides against a mid-priced frontier model, the open-weight advantage on this workload can disappear. Krishna’s range holds for output-heavy work, for low cache reuse, or for a comparison that leaves caching out of both sides. It fails as a general property of open weights.

The workload profile the numbers assume

Read the arithmetic together and the claim describes a particular setup: input-heavy traffic, high cache reuse, sustained high utilisation, and a cost basis that counts only GPU rental.

That last condition does the most work. The same llm-d post prices two different 64-GPU layouts at an assumed $2.71 per GPU-hour. The 3P1D layout, three prefill instances to one decoder, comes to $0.08 per million input tokens and $1.92 per million output. The 2P2D layout, two of each, comes to $0.05 per million input and $3.55 per million output. Against Z.ai’s $1.40 input list price both are a large saving, under 6 percent of list. The two output figures are about 44 percent and about 81 percent of that model’s $4.40 output list price, and the post states the 81 percent itself for 2P2D. The saving therefore sits on the input side, and the layout that streams best gives most of it back on output. These figures assume load around the clock, and the post warns that “lower utilization raises effective cost in inverse proportion”. They count GPU rental alone, leaving out storage, egress, engineering, support and tax, and compare against API prices that include managed operations, autoscaling, service commitments and provider margin. IBM and Red Hat say themselves that this is not full total cost of ownership.

A workload matching that profile, on hardware kept busy, can beat frontier API prices on input. Spiky, output-heavy or lightly cached traffic can land at or above the managed price before anyone hires the kernel engineers Krishna said most enterprises struggle to find.

What independent evaluations cover, and what none of them reports

Independent evidence does show the capability gap narrowing. None of it tests the cost claim.

Artificial Analysis scores open and proprietary models on its Intelligence Index v4.3.2, which includes agentic knowledge-work suites such as GDPval-AA and AA-Briefcase. On 23 October it reported that once Moonshot published weights for Kimi K3.1 Pro, the leading open model scored 55 and the gap to the leading proprietary model narrowed from 12 points to 3. Epoch AI, tracking capability across 94 benchmarks, put open-weight models about four months and 8 ECI points behind the frontier as of late May. The UK AI Security Institute found in July that leading open weights trail the closed cyber frontier by four to seven months on narrow cyber tasks, narrower than the six to ten months it measured through most of 2025.

The AI Security Institute is the one evaluator here that pairs effectiveness with cost, and it does so on a single narrow suite. It priced matched tasks in its cyber range at advertised first-party rates: where both models solved the task with 100 percent reliability, Opus 4.6 came to $15.17 per task against GLM-5.2’s $6.12, and Opus 4.5 to $12.50 against DeepSeek V4-Pro’s $0.28. AISI states that it covers cyber only, that inferences to other capabilities cannot be drawn, and that it did not run the open models through first-party providers, so real compute costs may differ.

The evidence that would settle the claim

The claim becomes checkable with one artifact: a broad-based enterprise evaluation on real work tasks that reports both effectiveness against a named frontier model and fully loaded cost per task, with context length, cache-hit rate and utilisation stated alongside.

“Broad-based, not narrow” is Krishna’s own hinge and the right test. Public evaluations are narrow by design: a cyber range, a set of agentic knowledge-work puzzles, a capability index. They rank capability and trace a trend. They are not a sample of enterprise traffic, and cost per task on someone else’s suite does not transfer to your invoice. Until that enterprise evaluation exists, Krishna’s sentence remains an informed but unverified assertion about a market his company sells into.

Why IBM’s interest in the answer matters

IBM sells the conclusion. It sells on-premises and sovereign AI systems, and it launched llm-d 1.0, its open inference stack, on 20 October, the day before the call. In the same remarks Krishna tied on-premises demand to sales of IBM’s Spyre accelerators and z17 machines.

The supporting figures are IBM’s. The 20 October release cites “up to five to 10 times cost reductions for context-heavy agentic workloads”, “11.5 times lower inference cost than an all-proprietary configuration”, and “5 to 10 times less per token than equivalent commercial API pricing”. Its footnotes point to an IBM Research blog, the llm-d project blog and an IBM-hosted PDF, all authored by IBM or Red Hat, and we found no independent verification of any of them. The “$0.11 per million tokens” blended serving cost comes from a quantum-error-correction study cited in a separate footnote, not from the 544-H100 GLM-5.2 demonstration behind the “5 to 10 times” figure. The two should not be merged. Krishna repeated the 5-to-10-times range in his prepared remarks, so the same vendor number turns up in the blog, the release and the CEO’s mouth. That does not make it false; it means the numbers need an outside check before a budget rests on them.

What a buyer can check this week

Choosing between frontier APIs and open weights this quarter is better done with your own traffic than with any general ratio. Log your input-to-output mix, your cache-hit rate and a realistic utilisation figure. Run those three against both sets of published prices, cache-read pricing included on both sides, then add the cost of hosting: engineering time, support, storage and the idle GPU hours your utilisation implies. That calculation uses your workload, and it will tell you more than a percentage borrowed from someone else’s.