⏱️ Lectura: 11 min
Calvin French-Owen, co-founder of Segment, spent weeks testing gpt-5.6-luna, a lightweight and fast ai model. In one test, he asked it to review thousands of his emails looking for patterns: the final bill came to around ten cents. That number, not the model’s intelligence, is the news.
📑 En este artículo
- TL;DR
- Introduction
- What Happened
- Context and History
- Technical Details and Performance
- How to Try It
- Impact and Analysis of Small AI Models
- What’s Next
- Frequently Asked Questions
- What is gpt-5.6-luna?
- Why do small ai models matter for building apps?
- Is GLM 5.3 the same as gpt-5.6-luna?
- Do small ai models replace frontier models like Fable 5?
- What’s still needed for companies to delegate business tasks to small models without supervision?
- How much does the personalized news microsite experiment cost?
- References
For two years the conversation about ai revolved around how far the biggest, most expensive models could go. Now there’s a second, quieter story: small ai models are already good and cheap enough to power full products, not just demos.
TL;DR
- gpt-5.6-luna processes close to 100 tokens per second in real-world use, according to Calvin French-Owen, co-founder of Segment.
- A complex research task with gpt-5.6-luna, like reviewing thousands of emails, costs just a few cents.
- With Sonnet-class models (previous generation), the same task cost around $1 per run.
- GLM 5.3 joins the cost-capability Pareto frontier alongside gpt-5.6-luna, according to artificialanalysis.ai.
- A daily personalized news microsite dropped from ~$1 to ~$0.10 per run with gpt-5.6-luna.
- For complex programming, French-Owen still prefers expensive models like Fable 5 and GPT-5.6 Sol.
- Peter Reinhardt (Charm Industrial, Revoy) estimates that 95% of his daily work is repetitive tasks, not technical genius.
Introduction
The public discussion about language models has for years revolved around capability: which model reasons better, which one codes without errors, which one solves olympiad-level math. That axis still exists, but a second axis emerged that’s just as relevant: cost per task. Calvin French-Owen documented on his blog his experience using gpt-5.6-luna, a small, fast model that, according to him, solves complex research tasks for pennies.
The shift isn’t just technical, it changes what kind of ai product is viable to build. During the era of Sonnet-class models, running a complex query could cost around $1. Charging $30 a month for a consumer app with that inference cost was, in French-Owen’s words, unsustainable.
What Happened
French-Owen spent several weeks testing gpt-5.6-luna on everyday tasks: reviewing his code, his email, and his personal knowledge base. He reports speeds of around 100 tokens per second and API bills of just a few cents, even when having the model search for patterns across thousands of emails.
The benchmark he uses is a personal experiment he repeats periodically: asking a model to research his public activity, figure out what news might interest him, and put together a personalized daily microsite with the most relevant items from Hacker News, Reddit, and Twitter. With the previous generation of Sonnet-class models, that experiment cost close to $1 per run. With gpt-5.6-luna, the average cost dropped to about $0.10.
In parallel, artificialanalysis.ai placed GLM 5.3 on the cost-capability Pareto frontier, meaning among the models with the best ratio between what they cost and what they can solve. French-Owen clarifies that for programming he still prefers more expensive, more capable models like Fable 5 and GPT-5.6 Sol, but admits that preference caused him to lose sight of how far small models had advanced.
Context and History
The classic playbook for a mass-consumer company, before generative ai, was simple for two decades: build a site that’s cheap to operate, get users through some virality, raise capital to scale, and eventually build an advertising marketplace. Google, Facebook, and Snapchat followed that path, with Amazon and Netflix as the exceptions French-Owen points out.
Adding generative ai to that model breaks the first step: a cheap-to-operate site is no longer enough, because every response from a large model carries a real per-user inference cost. That explains, according to investors cited by French-Owen, why more consumer ai startups didn’t emerge despite two years of record investment in the sector: the cost per token still hadn’t dropped enough to support a reasonable subscription price.
Small ai models change that math. They don’t compete for first place on reasoning benchmarks, they compete to be good enough at a fraction of the cost, and that fraction is what determines whether a consumer ai product is viable at scale.
Technical Details and Performance
The figure French-Owen keeps repeating is speed: ~100 tokens per second in real-world use with gpt-5.6-luna, enough for latency to stop feeling like a wait and start feeling like a conversation. Speed matters as much as cost, because a slow model, even if cheap, is still unworkable for interactive products.
At the other end of the capability frontier are the models French-Owen reserves for coding: Fable 5 and GPT-5.6 Sol. These are what he uses for what his former Segment colleague Peter Reinhardt describes as IQ 180 work: a handful of problems that need a novel solution nobody has thought of before. The rest (which Reinhardt estimates at 95% of the work he does running several companies, including Charm Industrial and Revoy) is what he calls token spewer work: responding fast, coordinating people, moving forward in parallel across dozens of fronts.
💭 Key takeaway: Reinhardt raised more than $100 million for Charm Industrial and closed a Series A for Revoy, and still estimates that 95% of his daily work doesn’t require technical genius, it requires responding fast.
That distinction matters for choosing a model in a real pipeline. Not every LLM call needs the most expensive model available, and treating them all the same wastes the savings small ai models offer.
| Model or class | Typical use | Approximate cost per complex task | Limitation |
|---|---|---|---|
| gpt-5.6-luna | Daily research, email, knowledge base | A few cents (~$0.10 in French-Owen’s eval) | Not the model of choice for hard programming |
| Sonnet-class models (previous generation) | Complex research | ~$1 per task | Unsustainable for a $30/month consumer subscription |
| Fable 5 / GPT-5.6 Sol | Complex programming, IQ 180 problems | High, not the focus of the savings | Capability frontier, not cost frontier |
| GLM 5.3 | General and agentic work | New point on the cost-capability Pareto frontier | Performance data still limited to artificialanalysis.ai |
How to Try It
To replicate the personalized news microsite experiment, the minimal architecture is straightforward: a research step (search HN, Reddit, and the user’s public mentions), a synthesis step (decide what matters), and a generation step (put together the summary). The savings aren’t in the code, they’re in routing each step to the right model based on cost.
A minimal call using an API compatible with the format most small-model providers support today looks like this:
curl https://api.ejemplo-proveedor.com/v1/chat/completions \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "modelo-pequeno-rapido",
"messages": [
{"role": "system", "content": "You are a researcher who summarizes tech news."},
{"role": "user", "content": "Find today's most relevant news on HN and Reddit for a backend developer."}
],
"max_tokens": 800
}'That call returns a summary in seconds and, with a small model, at a cost of pennies per run.
To automatically route between a cheap model and an expensive one based on task complexity, a common production pattern is to decide the model before calling, not after:
function elegirModelo(tarea) {
const esCompleja = tarea.requiereRazonamientoProfundo || tarea.tokensEstimadosSalida > 4000;
return esCompleja ? "modelo-frontera-caro" : "modelo-pequeno-rapido";
}
async function ejecutarTarea(tarea) {
const modelo = elegirModelo(tarea);
const respuesta = await llamarLLM(modelo, tarea.prompt);
console.log(`Model used: ${modelo}, output tokens: ${respuesta.usage.completion_tokens}`);
return respuesta;
}The following diagram summarizes that routing:
flowchart TD
A["Incoming task"] --> B{"Is it complex?"}
B -- "No" --> C["Small, fast model"]
B -- "Yes" --> D["Frontier model, expensive"]
C --> E["Response to user"]
D --> ETo confirm the routing works as expected, the simplest approach is to log the model used and the estimated cost per call (most providers return the input and output token counts in the same response) and add it up per day. If the average cost per task gets close to the pennies French-Owen reports for gpt-5.6-luna, the pipeline is routed correctly. If it gets close to a dollar, most tasks are falling through to the expensive model by default.
💡 Tip: don’t send every task to the most expensive model out of habit; measure cost per task before optimizing prompts.
Impact and Analysis of Small AI Models
The most direct effect is economic: cutting the cost of a complex research task from ~$1 to ~$0.10 changes what subscription price is sustainable. An app that once needed to charge like a business newspaper just to avoid losing money on every query can now get closer to mass-consumer pricing.
The second effect is organizational. Reinhardt sums it up from his experience running several startups at once: most human work in a company isn’t technical genius, it’s responding fast and not dropping the ball across dozens of fronts. If small ai models can take on that load (scheduling, following up, summarizing, drafting), the bottleneck stops being IQ 180 talent and becomes the infrastructure needed to delegate business tasks to a model with reasonable safety.
That’s where the honest trade-off shows up: delegating real business work to a small model requires first solving problems that consumer ai hasn’t fully solved yet, like security against prompt injection, role-based permissions, and the harness that decides what the model can do without supervision. French-Owen admits it: engineering work remains before fast, cheap, and good enough becomes a business reality, not just a personal experiment.
What’s Next
French-Owen expects demand for frontier models, like Fable 5 and GPT-5.6 Sol, to keep growing in parallel, especially in fields that depend on genuine breakthroughs: advanced engineering, hard science, training new models. But he also expects demand for fast, cheap, good-enough models is only just taking off, driven by business use cases more than end-consumer ones.
The missing piece, by his own account, isn’t more model capability: it’s the security and permissions infrastructure needed for a company to trust giving a small model access to its email, CRM, or calendar without constant supervision.
Try it yourself: if you have API access to a recent small model, run French-Owen’s same experiment (have it research your public activity and put together a daily summary) and measure how much it charges you per run.
📖 Summary on Telegram: View summary
Frequently Asked Questions
What is gpt-5.6-luna?
It’s a small, fast language model that Calvin French-Owen, co-founder of Segment, tested for several weeks. According to his report, it processes around 100 tokens per second and solves complex research tasks for pennies.
Why do small ai models matter for building apps?
Because they determine whether a consumer ai product is economically viable. With expensive models, a complex query could cost close to $1, an unsustainable cost for a $30-a-month subscription. With small models, that cost can drop to pennies.
Is GLM 5.3 the same as gpt-5.6-luna?
No, they’re different models from different providers. What they have in common, according to data from artificialanalysis.ai, is that both appear on the cost-capability Pareto frontier: they offer a good ratio between what they cost and what they can solve.
Do small ai models replace frontier models like Fable 5?
Not for every case. French-Owen still uses the more expensive, more capable models for complex programming, where a novel solution is needed. Small models are aimed at repetitive, high-volume tasks, not the hardest problems.
What’s still needed for companies to delegate business tasks to small models without supervision?
According to French-Owen, security against prompt injection still needs to be solved, along with defining clear roles and permissions, and building the harness that decides what the model can execute autonomously.
How much does the personalized news microsite experiment cost?
With Sonnet-class models it cost close to $1 per run. With gpt-5.6-luna, French-Owen reports an average cost of about $0.10 per run.
References
- Small Models Have Arrived: original article by Calvin French-Owen documenting the cost and speed of gpt-5.6-luna.
- Artificial Analysis: chart of the cost-capability Pareto frontier placing GLM 5.3 among the most efficient models.
- Small language model (Wikipedia): general context on small language models and their use cases.
📱 Enjoy this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
Imagen destacada: Foto de BoliviaInteligente en Unsplash
0 Comments