⏱️ Lectura: 11 min
On July 18, 2026, discussion on X about Kimi K3 (Moonshot AI’s open-weight model that comes close to the state of the art) reopened a confusion that has followed the AI industry for years: believing that an “open” model is a free model. Ben Thompson, in his Stratechery newsletter, used that debate to make a deeper argument: marginal costs, which software had banished for two decades, are back with AI, and that changes the entire structure of the business.
📑 En este artículo
The distinction Thompson proposes is simple: downloading a model’s weights costs nothing, but running it does. That inference cost, not the license price, is what determines how much margin each AI provider keeps as demand scales.
TL;DR
- Kimi K3, Moonshot AI’s open model, charges $3 per million input tokens and $15 per million output tokens.
- Sol, used as the comparison point in the analysis, charges $5 for input and $30 for output per million tokens.
- Ben Thompson published “Who’s Afraid of Chinese Models?” on Stratechery on July 20, 2026.
- The central argument: R&D is a fixed cost, but COGS (inference) scales directly with revenue.
- Jensen Huang, Nvidia’s CEO, describes AI data centers as “token factories.”
- Reasoning tokens (chain-of-thought) break the direct price-per-token comparison between models.
- Inference efficiency depends on MoE architecture, KV cache, batching, and prefix caching, not just the listed price.
What Happened
Last weekend, users on X compared Kimi K3 to closed models like Sol, arguing that an open model approaching the state of the art makes paying for access to a proprietary model irrelevant. Thompson, in his article “Who’s Afraid of Chinese Models?”, published on July 20, 2026, responded with an accounting distinction that rarely shows up in those threads: the difference between R&D and cost of goods sold (COGS).
R&D is a fixed expense. If a company spends $1 million training a model, that million doesn’t change whether the model generates $100,000 or $100 million in revenue: the investment is already made. What does vary with revenue is COGS, and in AI, COGS is real in a way traditional software hasn’t been for a long time: every token generated, whether by Kimi or any other model, costs compute money.
That’s where the mistake of reading “open weights” as “free” comes from. As Thompson cites, Kimi K3 charges $3 per million input tokens and $15 per million output tokens: cheaper than the $5 input and $30 output that Sol charges, but far from free. Someone, in some data center, pays that compute bill.
Context and Background
Thompson built his career analyzing why internet software didn’t behave like traditional industries. His best-known framework, Aggregation Theory, starts from a simple observation: when distributing and copying a product costs zero, power concentrates in whoever controls demand, not whoever controls supply. Google doesn’t need to own any website to dominate search; Netflix doesn’t need movie theaters.
Generative AI reintroduces a variable that model had declared dead: marginal cost. Every response a model generates consumes real compute, measured in tokens, and that compute isn’t free for the model provider or for whoever uses it. It’s the first time in two decades that a core internet technology has gone back to having a cost structure similar to a factory’s: the more it produces, the more it spends.
That reversal has its own timeline within AI too. The first stage, the ChatGPT era, delivered tokens directly to the end user: the cost per response was relatively predictable. The second stage, the reasoning era, broke that predictability: a model can need thousands of chain-of-thought tokens before writing the visible answer, and those tokens are billed too. Agents, the third stage, add another layer: an agent can call tools, iterate, and retry before finishing a task, and every intermediate step also consumes tokens.
Technical Details and Performance
A token from one model isn’t interchangeable with a token from another model the way a barrel of oil is interchangeable with another barrel of oil. What is fungible is the outcome: if Kimi K3 and Sol arrive at the same correct answer, that answer is worth the same to the user, even if one needed triple the reasoning tokens to get there.
Thompson identifies four factors that determine the real COGS of serving intelligence, beyond the list price:
- Model footprint: how much expensive memory and how many accelerators each active replica needs to host the weights and runtime state.
- Inference efficiency: architectural decisions, like Mixture-of-Experts (MoE), that reduce the compute needed per generated token by activating only a fraction of the total parameters.
- Memory efficiency: optimizations that reduce KV cache size, allowing more concurrent requests to be served with the same GPU.
- Serving efficiency: batching, scheduling, prefix caching, and other inference-layer techniques that maximize how many useful tokens come out per watt and per dollar of hardware.
None of these four factors shows up in the published price per million tokens. A provider can charge less per token and still have worse COGS if its model needs more reasoning tokens to reach the same result, or if its serving infrastructure wastes GPU capacity instead of saturating it with batching.
| Model | Input Price (per million tokens) | Output Price (per million tokens) | Origin |
|---|---|---|---|
| Kimi K3 (Moonshot AI) | $3 | $15 | Open weights, China |
| Sol | $5 | $30 | Closed model, Thompson’s reference |
def costo_por_tarea(tokens_entrada, tokens_salida, precio_entrada, precio_salida):
return (tokens_entrada / 1_000_000) * precio_entrada + (tokens_salida / 1_000_000) * precio_salida
# Kimi K3: cheaper per token, but may need more reasoning tokens
costo_kimi = costo_por_tarea(2000, 8000, 3, 15)
# Sol: more expensive per token, but may solve the same task with fewer tokens
costo_sol = costo_por_tarea(2000, 3000, 5, 30)
print(f"Kimi K3: ${costo_kimi:.4f}")
print(f"Sol: ${costo_sol:.4f}")This calculation illustrates Thompson’s point: if Kimi needs almost triple the output tokens to reach the same answer, its per-token price advantage can evaporate in the total cost of the task. Comparing AI providers only by list price, without measuring how many tokens each one consumes to solve the same problem, leads to wrong conclusions.
flowchart TD
A["User request"] --> B["Model generates tokens"]
B --> C["Model footprint"]
B --> D["Inference efficiency (MoE)"]
B --> E["Memory efficiency (KV cache)"]
B --> F["Serving efficiency (batching, prefix caching)"]
C --> G[("Real COGS per token")]
D --> G
E --> G
F --> GHow to Get Started / Try It
To measure the real COGS of your own use case instead of trusting the list price, the simplest approach is to run the same task against two models and compare the actual tokens each API reports, not just the published price.
On Linux and macOS, with curl:
export MOONSHOT_API_KEY="your-api-key"
curl https://api.moonshot.ai/v1/chat/completions \
-H "Authorization: Bearer $MOONSHOT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "kimi-k3",
"messages": [{"role": "user", "content": "Explain what COGS is in 3 sentences"}]
}'On Windows, with PowerShell:
$env:MOONSHOT_API_KEY = "your-api-key"
Invoke-RestMethod -Uri "https://api.moonshot.ai/v1/chat/completions" `
-Method Post `
-Headers @{ Authorization = "Bearer $env:MOONSHOT_API_KEY" } `
-ContentType "application/json" `
-Body '{"model":"kimi-k3","messages":[{"role":"user","content":"Explain what COGS is in 3 sentences"}]}'The API response includes a usage field with prompt_tokens and completion_tokens. With those two numbers and each provider’s price per million tokens, you can calculate the real cost of your task with the costo_por_tarea function from the earlier example, instead of comparing only the list price.
💡 Tip: always check the usage.total_tokens field in the response to confirm how many tokens a call actually consumed, not a prior estimate.
Impact and Analysis
The reversal of marginal cost changes who wins in the AI value chain. In classic aggregation software, the winner was whoever controlled demand (the search engine, the social network, the marketplace) because distributing the product cost nothing extra. In AI, controlling demand isn’t enough anymore: you also have to control COGS, because every additional user costs real compute.
That explains why Nvidia describes its business as a token factory: for the GPU maker, any model is interchangeable, what matters is how many tokens per second, per watt, and per dollar the hardware can produce. For AI labs, though, the question is different: how many tokens their model needs to solve a task, not just how many their hardware can generate per second.
This distinction also reorders the competition between Chinese open models and Western closed models. An open model that charges less per token doesn’t automatically win if it needs more tokens to reach the same result, and a lab with better serving efficiency can sustain higher margins even while charging the same per token as its competitors. The advantage is no longer measured only in capability benchmarks, but in how much it costs, in real compute, to produce each unit of useful intelligence.
💭 Key takeaway: the price per million tokens says nothing about the real cost of a task if you don’t know how many tokens the model needs to solve it.
What’s Next
If Thompson’s argument holds, the next competitive battle in AI won’t be fought only in capability benchmarks, but in inference efficiency: more aggressive MoE architectures, smaller KV caches, and serving layers that squeeze the most out of every GPU. Labs that post good benchmark results but need far more tokens than their competitors to get there will lose margin even if they win the headline.
It’s reasonable to expect providers to start publishing, alongside the price per token, tokens-per-task metrics on standardized benchmarks: without that data, comparing the real cost of Kimi K3, Sol, or any other model by list price alone remains incomplete.
📖 Summary on Telegram: View summary
Try it yourself: ask the same question to two models via API, compare the usage field of each response, and calculate the real cost using each provider’s price per million tokens.
Frequently Asked Questions
What does COGS mean in the context of AI?
COGS (cost of goods sold) is the expense that varies directly with usage volume. In AI, it corresponds to the compute cost needed to generate each response token, unlike R&D, which is a fixed expense of training the model.
Why isn’t an open-weight model like Kimi K3 free to use?
Because downloading the weights costs nothing, but running them does. Every query consumes GPU, memory, and energy in the data center hosting the model, and that cost gets passed on in the per-token price the provider charges.
Why isn’t price per token enough to compare models?
Because tokens aren’t a commodity: two models can need very different amounts of reasoning tokens to reach the same correct answer. You have to compare the total cost of solving a task, not just the list price per million tokens.
What is Mixture-of-Experts (MoE) and why does it reduce COGS?
It’s an architecture where the model activates only a fraction of its total parameters for each token, instead of using the full network. That reduces the compute needed per generated token without sacrificing the model’s overall capacity.
How do I measure the real cost of a task using a model’s API?
By checking the usage field in the API response, which includes prompt_tokens and completion_tokens, and multiplying each value by the price per million tokens the provider charges.
References
- Stratechery, “Who’s Afraid of Chinese Models?”: Ben Thompson’s original analysis of COGS, R&D, and the reversal of marginal cost in AI.
- Wikipedia, Cost of goods sold: general definition of the COGS accounting concept applied in the article.
- Moonshot AI on GitHub: repository of the lab behind Kimi K3.
- Official Nvidia blog: source of the idea of AI data centers as “token factories” attributed to Jensen Huang.
📱 Do you like this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
Imagen destacada: Foto de Domaintechnik en Unsplash
0 Comments