⏱️ Lectura: 12 min
Kimi K3, Moonshot AI’s open-weight model, charges $3 per million input tokens and $15 per million output tokens. That sounds cheap next to Sol’s $5 and $30, the model Stratechery compares it to, but that comparison hides a catch: open weights don’t eliminate inference cost, they only eliminate the cost of training the model.
📑 En este artículo
Ben Thompson explains it in an analysis published on July 20, 2026: the weekend debate about Kimi K3 mixed up two unrelated types of cost, R&D (fixed, sunk, independent of revenue) and COGS (variable, tied to every query the model answers). Separating these two concepts changes how you should choose an AI model for production.
TL;DR
- Kimi K3 (Moonshot AI) charges $3 per million input tokens and $15 per million output tokens.
- Sol, the model compared in the analysis, charges $5 for input and $30 for output per million tokens.
- R&D is a fixed cost that doesn’t depend on revenue; COGS (the cost of serving each query) does scale with usage.
- Open weights save the R&D of training the model, but they don’t eliminate the COGS of inference.
- In the reasoning era, Kimi may need more chain-of-thought tokens than Sol for the same answer.
- Nvidia (Jensen Huang) calls its GPUs token factories, but the token isn’t the fungible unit: the correct answer is.
- The COGS of intelligence depends on 4 factors: model footprint, inference efficiency, memory, and serving.
- The analysis is by Ben Thompson at Stratechery, published on July 20, 2026.
What Happened
Over the weekend of July 18-19, 2026, technical conversation on X centered on a single topic: Kimi K3, Moonshot AI’s new open-weight model, is closing in on state-of-the-art capabilities. For many developers that meant something simple: a model nearly as good as the best closed model, available to download and run wherever you want.
Ben Thompson, of Stratechery, responded with an argument that complicates that optimistic reading: the word free in open weights refers only to the money you don’t spend on research and development. It says nothing about the cost of serving the model to real users, which in accounting is called COGS.
The concrete example is the price of Kimi K3: $3 per million input tokens and $15 per million output tokens. It’s cheaper than Sol’s $5 and $30, but according to Thompson, comparing only price per token is the wrong metric.
Context and Background
Thompson opens his piece with an anecdote from his first day in STRT-431, the mandatory introductory course for every first-year student at the Kellogg School of Management. He was struck that there were no technology companies in the reading list. The professor explained that the course’s goal wasn’t to study specific industries, but to find universal principles applicable to any company, in any field.
That answer didn’t fully convince Thompson. For him, the nature of software, with marginal distribution costs near zero, made technology fundamentally different. That observation became the foundation of his Aggregation Theory: when marginal cost is zero, value chains centralize around whoever controls demand, not whoever distributes supply.
What makes AI interesting, Thompson says, is that those old universal principles are back. Traditional software has zero marginal cost: publishing one more copy of an app costs nothing. Running one more query on a language model does cost something: electricity, GPUs, memory. That’s the paradigm shift reopening questions that had been settled for two decades.
Technical Details: The Real Inference Cost
The most common confusion about open-weight models is treating them as free. What you actually save is the R&D: the fixed investment in research and development that doesn’t depend on how much you use the model afterward. If you spend $1 million training a model, it doesn’t matter whether you generate $100 thousand or $100 million in revenue from it, that million is already spent.
The COGS (cost of goods sold) is different: it scales with usage. Thompson gives a simple example: if it costs 50 cents to generate the tokens that produce $1 in revenue, then $100 million in revenue implies $50 million in COGS, and $100 thousand in revenue implies $50 thousand in COGS. The ratio stays the same, but the absolute cost grows with every query you serve.
| Model | Input price / million tokens | Output price / million tokens | When it makes sense |
|---|---|---|---|
| Kimi K3 (Moonshot AI) | $3 | $15 | Short-answer workloads or where you can self-host and control COGS directly |
| Sol | $5 | $30 | Reasoning tasks where fewer chain-of-thought tokens offset the higher price per token |
That’s where the second conceptual error shows up: treating the token as a commodity. A commodity is fungible, one gallon of oil equals another gallon of oil. A Kimi K3 token isn’t equal to a Sol token. What is fungible is what you build with those tokens: intelligence, meaning the correct answer. If two models arrive at the same correct answer, that answer is fungible, even if one needed three times as many tokens as the other to get there.
💭 Key point: price per token doesn’t measure the real cost of an answer. What matters is how many tokens each model needs to reach the correct answer, not the price of each individual token.
This shows up especially in the reasoning era, the second stage of AI after the original ChatGPT, which was focused on delivering tokens directly to the user. Reasoning models generate a long chain of thought before giving the final answer, and that chain varies a lot from one model to another. According to Thompson, Kimi reportedly needs considerably more tokens than Sol to reach the same answer, which can completely wipe out its price-per-token advantage.
Thompson identifies four factors that determine a model’s real inference cost:
- Model footprint: the weights and runtime state determine how much expensive memory and how many accelerators are needed to host each serving replica.
- Inference efficiency: architectural decisions like Mixture-of-Experts (MoE) reduce the compute needed per generated token.
- Memory efficiency: architectural decisions that reduce KV cache requirements allow serving more requests simultaneously and getting more out of each GPU.
- Serving efficiency: batching, scheduling, and prefix caching, among other inference optimizations, maximize the utilization of each accelerator.
flowchart TD
A["Price per token"] --> E["Real COGS of intelligence"]
B["Model footprint"] --> E
C["Inference efficiency (MoE)"] --> E
D["Memory efficiency (KV cache)"] --> E
F["Serving efficiency (batching, prefix caching)"] --> E
E --> G["Cost per correct answer"]No single model wins on all four factors at once. An MoE model may have less compute per token but a larger memory footprint because it needs to load more total parameters even though it only activates a fraction on each pass. Comparing two models purely on list price ignores that mechanic.
How to Test It: Measure Your Own COGS
Most model providers, including Moonshot AI, expose an API compatible with the OpenAI format. That means you can measure real token consumption with the same client on Windows, macOS, or Linux, without switching tools.
First, install the client on any of the three operating systems:
# Windows (PowerShell), macOS and Linux: the same command
pip install openaiNext, run a test query against the compatible endpoint and check the usage field in the response, that’s where the real breakdown of tokens consumed lives:
import openai
client = openai.OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.moonshot.ai/v1"
)
respuesta = client.chat.completions.create(
model="kimi-k3",
messages=[{"role": "user", "content": "Explain what COGS is in one sentence."}]
)
tokens_entrada = respuesta.usage.prompt_tokens
tokens_salida = respuesta.usage.completion_tokens
costo_usd = (tokens_entrada / 1_000_000 * 3) + (tokens_salida / 1_000_000 * 15)
print(f"Input tokens: {tokens_entrada}, output: {tokens_salida}, cost: ${costo_usd:.6f}")That script calculates the real cost of a single query using Kimi K3’s public price. Run the same question against Sol or another model with its own price per million tokens and you’ll get a real COGS comparison, not just a list-price one.
💡 Tip: if your use case involves long reasoning, add the chain-of-thought tokens to the calculation too. Many providers report them separately in usage.reasoning_tokens or an equivalent field; that number is usually what inflates the COGS of reasoning models beyond the published price per token.
To verify you’re measuring correctly, add up the cost of several real queries from your product, not just a hello world, and compare the total against the invoice you get from the provider at month’s end. If they don’t match, you’re probably missing system tokens, tool-call tokens, or retries.
Impact and Analysis
Jensen Huang, Nvidia’s CEO, describes what his company builds as token factories. From Nvidia’s perspective that metaphor makes sense: its GPUs are model-agnostic, they generate tokens as fast and efficiently as possible. That’s where metrics like tokens per second, time to first token, tokens per watt, and cost per token come from, metrics Huang presents as the foundation of any infrastructure decision.
That way of measuring made sense in AI’s first paradigm, the ChatGPT era, when tokens went straight from the model to the end user. The second paradigm, the reasoning era, complicates that metric: the chain of thought drives up the token count, and each model needs a different amount to reach the correct answer. Agents add another layer to the same dynamic, because some models are more efficient than others at running multi-step agentic workflows.
⚠️ Watch out: choosing the cheapest model per token without measuring how many tokens it needs for your specific task can end up costing more than a model with a higher list price but greater efficiency in reasoning tokens.
For a team evaluating which model to put into production, the practical takeaway is that price per million tokens is just the starting point. What you need to measure is the inference cost per completed task: how many input, output, and reasoning tokens each model needs to solve the same problem, multiplied by its real price. That number, not the list price, determines whether a cheap open model ends up being more economical than an expensive closed one, per token.
What’s Next
If inference cost, not training cost, is the new battleground, it’s reasonable to expect that labs releasing open weights will start competing on reasoning efficiency too, not just capability benchmarks. A model that solves a problem with half the chain-of-thought tokens of another model at the same capability level has a real COGS advantage, even if its list price per token is higher.
That also changes what to look at when a new model comes out: beyond the benchmark score, check how many reasoning tokens it consumes on tasks representative of your own use case, since that number varies by task type and isn’t always reported alongside the headline benchmark.
Try it yourself: run the script from the previous section against Kimi K3’s OpenAI-compatible API and compare the real cost of a reasoning query against the list price in this article’s table.
📖 Summary on Telegram: View summary
Frequently Asked Questions
What is Kimi K3?
It’s Moonshot AI’s most recent open-weight model, the Chinese company that splits its assistant into work, code, and claw variants. It’s closing in on state-of-the-art capabilities and can be downloaded and run on your own infrastructure.
Is Kimi K3 free?
Not in terms of total cost. Open weights eliminate the R&D, meaning what it cost to train, but running it still has COGS: every input and output token costs real money in GPU time, memory, and electricity. Its public price is $3 per million input tokens and $15 per million output tokens.
What is COGS in the context of AI?
COGS stands for cost of goods sold. In AI it’s the variable cost of serving each query: unlike R&D, which is a fixed, already-sunk expense, COGS grows proportionally with every token the model generates.
Why does Nvidia talk about token factories?
Because its GPUs are model-agnostic: they generate tokens as fast as possible, regardless of which model is involved. That metric, tokens per second and cost per token, makes sense for Nvidia, but it doesn’t capture how many tokens each model needs to reach the correct answer.
Are tokens from two different models comparable?
Not directly. A token isn’t a fungible commodity like a gallon of oil. What is fungible is intelligence, meaning the final correct answer. Two models can need very different amounts of tokens to reach the same result.
How do I choose between an open and a closed model for production?
Measure the real inference cost per completed task, not just the list price per million tokens. Run the same representative task against both models, add up the input, output, and reasoning tokens, and multiply by each one’s price.
References
- Stratechery: Who’s Afraid of Chinese Models?: Ben Thompson’s original analysis of COGS, R&D, and the economics of Kimi K3.
- Wikipedia: Cost of goods sold: the accounting definition of COGS applied here to AI model inference.
- Wikipedia: Mixture of experts: the MoE architecture that reduces compute per token in models like Kimi K3.
📱 Enjoying this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
Imagen destacada: Foto de sdl sanjaya en Unsplash
0 Comments