⏱️ Lectura: 11 min
An essay published on July 20, 2026 by Ben Thompson on Stratechery reopened a discussion that seemed settled: Chinese open-weight models, like Kimi K3, aren’t free even when the model itself downloads at no cost. The central argument, which revolves around AI COGS, is that serving inference costs real money, and that cost is directly tied to how many people use the model.
📑 En este artículo
Thompson sums it up with a distinction that any team billing by the token should have clear: R&D is a fixed cost, but COGS (cost of goods sold) is variable and grows with every query served.
TL;DR
- Ben Thompson published an analysis on the economics of open Chinese models on Stratechery on July 20, 2026.
- Kimi K3 charges $3 per million input tokens and $15 per million output tokens, according to the essay.
- Sol charges $5 per million input tokens and $30 per million output tokens, more expensive but potentially more token-efficient.
- Thompson distinguishes COGS, a variable cost tied to revenue, from R&D, a fixed cost independent of usage volume.
- Jensen Huang, Nvidia’s CEO, calls its GPUs ‘token factories,’ a metric Thompson considers insufficient.
- The central argument: tokens aren’t a fungible commodity, but the correct answer is.
- Models with longer reasoning chains can wipe out per-token price advantages, according to the essay.
Introduction
The anecdote that opens Thompson’s essay comes from his first day at the Kellogg School of Management, when he confronted a professor about why the introductory MBA course didn’t include cases about technology companies. The answer was that the goal wasn’t to study specific industries, but universal principles applicable to any company. Thompson wasn’t satisfied at the time: to him, the fact that software and distribution had zero marginal cost was something qualitatively different, and much of his Aggregation Theory was born from that disagreement.
The curious thing about artificial intelligence, he writes, is that those universal principles came back with a vengeance. The discussion that sparked the essay happened over a July 2026 weekend when the tech community debated on X the implications of Kimi K3, a Chinese open-weight model that approaches state-of-the-art capabilities. Thompson’s conclusion: marginal cost matters again, both in the short-term pricing of cutting-edge free models and in the long-term structure of the industry.
What Happened
The most common misunderstanding about open-weight models is assuming they’re cheap, even free, because anyone can download the weights and skip the time and money it takes to train a model from scratch. That’s true, but that “free” refers only to research and development (R&D): a fixed cost, independent of how much revenue the product generates. If a company spends $1 million on R&D, that number doesn’t change whether it bills $100,000 or $100 million, though how much of that investment it manages to recoup does change.
What does depend on revenue is COGS. Running inference on a model, whether Kimi or Sol, costs money, and that cost is directly correlated with how many people use it. Kimi K3 charges $3 per million input tokens and $15 per million output tokens: cheaper than Sol’s $5 and $30, but that per-token price comparison, Thompson warns, might not be the right metric.
Context and History
To understand why this matters, you have to go back to the first wave of the internet. There, the zero marginal cost of software and distribution led to centralization: in a world where controlling demand mattered more than distributing supply, aggregators won. That’s the thesis behind the Aggregation Theory that Thompson has been developing for more than a decade.
Generative AI seemed, at first, to follow the same pattern: increasingly cheaper models, marginal cost trending toward zero, a winner taking almost the entire market. But the emergence of competitive open models like Kimi K3 showed that AI COGS is real, in a way traditional software stopped being twenty years ago.
Technical Details: AI COGS and the Four Factors of Per-Token Cost
Thompson breaks down AI COGS into four factors that any infrastructure team recognizes:
- Model footprint: the weights and runtime state determine how much expensive memory and how many accelerators are needed to host each serving replica.
- Inference efficiency: architectural decisions like Mixture-of-Experts (MoE) reduce the compute needed per generated token.
- Memory efficiency: optimizing the KV cache allows serving more concurrent requests with the same GPU.
- Serving efficiency: batching, scheduling, and prefix caching maximize real hardware utilization.
Nvidia, through its CEO Jensen Huang, describes what it builds as ‘token factories’: from Nvidia’s perspective, GPUs are model-agnostic, simply generating tokens as fast and efficiently as possible. Hence metrics like tokens per second, time-to-first-token, tokens per watt, or cost per token, which Huang proposes as the basis for decision-making.
💭 Key point: the mistake in reasoning only in tokens per second or cost per token is that a token from one model isn’t interchangeable with a token from another. What’s fungible, Thompson says, is the correct answer, not the individual token.
That distinction has practical consequences. In the first era of generative AI, the ChatGPT era, tokens were delivered directly to the end user, so measuring tokens per second made sense. The second era, the reasoning era, complicates that metric: reasoning implies an explosion of chain-of-thought tokens, and each model needs a different amount of those tokens to reach the correct answer. Kimi, the essay reports, uses significantly more tokens than Sol for equivalent tasks, which can wipe out its nominal per-token price advantage.
flowchart TD
A["Model weights"] --> B["Inference efficiency (MoE)"]
B --> C["Memory and KV cache"]
C --> D["Serving: batching and prefix caching"]
D --> E[("Final COGS per token")]This flow summarizes why two models with the same nominal price per million tokens can have very different real COGS once you add up the four factors.
Kimi K3 vs. Sol: Published Price per Million Tokens
| Model | Input Price (per million tokens) | Output Price (per million tokens) | Note |
|---|---|---|---|
| Kimi K3 | $3 | $15 | Open weights, published by Moonshot AI |
| Sol | $5 | $30 | Closed model, reference used by Thompson in the essay |
The table shows the nominal price, but it says nothing about how many tokens each model needs to solve the same task. That’s exactly the blind spot Thompson points out: comparing price per token without normalizing for the amount of reasoning tokens is comparing apples to oranges.
How to Test It: Measure the Real COGS of Your Own Use Case
The most honest way to compare models isn’t to look at the published price, but to measure how much it costs to reach the correct answer on your own workload. Kimi K3 exposes an OpenAI-format-compatible API, so the first step is simply making a test call.
Set the API key as an environment variable according to your operating system:
# Linux / macOS (bash or zsh)
export MOONSHOT_API_KEY="your-api-key-here"
# Windows (PowerShell)
$env:MOONSHOT_API_KEY = "your-api-key-here"
# Windows (cmd.exe)
set MOONSHOT_API_KEY=your-api-key-hereWith the variable set, a minimal first request against the Kimi K3 API looks like this:
curl https://api.moonshot.ai/v1/chat/completions \
-H "Authorization: Bearer $MOONSHOT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "kimi-k3",
"messages": [{"role": "user", "content": "Explain what AI COGS is in two sentences."}]
}'The response includes a usage object with prompt_tokens and completion_tokens: those are the numbers you need to multiply by the published price to get the real COGS of that specific query.
To compare Kimi K3 against Sol on a repeatable task, a short script automates the calculation:
const PRICING = {
"kimi-k3": { input: 3, output: 15 },
"sol": { input: 5, output: 30 },
};
function calcularCOGS(modelo, usage) {
const precio = PRICING[modelo];
const costoEntrada = (usage.prompt_tokens / 1_000_000) * precio.input;
const costoSalida = (usage.completion_tokens / 1_000_000) * precio.output;
return costoEntrada + costoSalida;
}
// Example with a real API response
const usage = { prompt_tokens: 420, completion_tokens: 1830 };
console.log(calcularCOGS("kimi-k3", usage).toFixed(6));Run that same prompt against both models, save the usage of each response, and compare the final result of calcularCOGS. If Kimi needs many more reasoning tokens to reach the same answer, the nominal per-token savings can disappear completely, exactly the phenomenon the essay describes.
⚠️ Watch out: comparing only the price per million tokens without running the same prompt against both models and measuring the real usage is the most common mistake when choosing an inference provider.
Impact and Analysis
The direct consequence of this reasoning is that competition between open and closed models isn’t defined by who gives away more weights, but by who achieves the lowest AI COGS per correct answer. That favors companies that can invest in the four factors Thompson mentions (footprint, inference efficiency, memory efficiency, and serving efficiency), not necessarily those that publish the lowest per-token price.
It also changes what product teams should be looking at. A model with a per-token price 40% cheaper but that needs twice as many reasoning tokens for the same task can end up costing more per resolved query. The right metric isn’t dollars per million tokens: it’s dollars per correct answer, and that figure depends on each model’s reasoning efficiency on each specific task, not on a published pricing table.
For Nvidia, the ‘token factories’ framing remains valid for its own business, which sells compute, not intelligence, but Thompson warns that the same approach, applied by whoever buys inference to build a product, leads to wrong decisions.
What’s Next
The logical consequence of Thompson’s argument is that model comparisons should report AI COGS per solved task, not just price per million tokens: without that data, any pricing table remains incomplete. Nvidia already publishes metrics like tokens per watt and time-to-first-token; what’s missing is for the industry to adopt an equivalent metric to express cost in dollars per correct answer, which is what a product team needs to decide between Kimi K3, Sol, or any other model.
For developers in Latin America evaluating which provider to use, the practical recommendation that follows from the essay is simple: run the same set of prompts representative of your product against each candidate model, add up the real usage of each response, and compare the resulting COGS, not the published price.
📖 Summary on Telegram: View summary
Try it yourself: run this article’s calcularCOGS script against your own prompt on the Kimi K3 API and compare the result with what your team currently pays for Sol or another closed model.
Frequently Asked Questions
Is Kimi K3 Really Free to Use?
No. The weights can be downloaded at no cost, but serving inference on those weights costs real money. Moonshot AI charges $3 per million input tokens and $15 per million output tokens on its API.
What’s the Difference Between COGS and R&D in AI?
R&D is the fixed expense of training the model: it doesn’t change based on how many people use it. COGS is the variable cost of serving each query, and it grows proportionally with usage volume.
Why Does Thompson Say Tokens Aren’t a Commodity?
Because a token from one model isn’t interchangeable with a token from another: each model needs a different amount of tokens to reach an answer. What is fungible, according to his argument, is the correct answer itself.
How Do I Measure a Model’s Real COGS for My Own Use Case?
By running the same prompt against each candidate model, taking the usage field (prompt_tokens and completion_tokens) from each response, and multiplying it by each one’s published price.
What Is a ‘Token Factory’ According to Jensen Huang?
It’s how Nvidia’s CEO describes the company’s business: GPUs that generate tokens as fast and efficiently as possible, regardless of which model they come from.
References
- Stratechery: “Who’s Afraid of Chinese Models?”: the original Ben Thompson essay that develops the AI COGS argument and the distinction between tokens and intelligence.
- Wikipedia: Marginal cost: definition of the marginal cost concept underpinning the comparison between R&D and COGS.
- Nvidia: the company whose CEO, Jensen Huang, coined the term ‘token factories’ to describe its GPU business.
- Moonshot AI: the Chinese company behind Kimi K3, the open-weight model that started the discussion.
📱 Enjoying this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
0 Comments