⏱️ Lectura: 11 min
The Cerebras Inference model catalog today lists only two models on its public endpoints, GPT OSS 120B and Qwen 3.8 27B, and alongside them the company published a detailed explanation of its model compression: selective per-layer quantization, no architecture pruning. This is an unusual statement in a market where almost no inference provider says exactly what it did to the weights it serves.
📑 En este artículo
The policy matters because Cerebras sells speed as its differentiator (up to 3,000 tokens per second on GPT OSS 120B), and the technical community tends to distrust that speed when it’s unclear whether it comes from pruned layers or real hardware. The official documentation clears up that doubt model by model.
TL;DR
- Cerebras Inference currently offers only two models on public endpoints: GPT OSS 120B and Qwen 3.8 27B.
- GPT OSS 120B: 120 billion parameters, context from 65k (free) to 131k (paid), ~3,000 tokens/s.
- Qwen 3.8 27B: 27 billion parameters, context from 64k (free) to 128k (paid), ~1,500 tokens/s.
- Cerebras confirms that no public model is pruned: all are original versions with no architecture changes.
- The only compression applied is selective per-layer quantization, at 16, 8, or 4 bits, for storage only.
- Activations, attention, and the KV cache remain at full precision and unquantized during computation.
- Pruning lives separately: the REAP technique is published on Hugging Face for research, it doesn’t reach the shared API.
- Cerebras commits to naming any future pruned variant separately, without replacing an existing model.
What happened
Cerebras updated the reference page for its inference service with two concrete changes. The first is the table of models available on the free and pay-as-you-go tiers: GPT OSS 120B, from OpenAI, and Qwen 3.8 27B, from Alibaba. The second is a new model compression section that explains, FAQ included, what Cerebras does to the weights before serving them.
The company separates three things that usually remain implicit in other inference providers’ documentation: which models it hosts today, how it compresses them for storage, and how it isolates that compression from its pruning research. None of the three answers is left up to the reader’s interpretation.
To access additional models outside this short list, such as other open families, Cerebras points to its Dedicated Endpoints product, with reserved capacity and production SLAs; the public catalog that was just updated works as a free or pay-as-you-go entry point, with no capacity commitment.
Context and history
Cerebras isn’t a conventional GPU provider. Its hardware, the Wafer Scale Engine, mounts the entire model on a single wafer-sized chip and removes much of the latency of moving weights between memory and compute that a traditional GPU suffers. That architecture is why the company can promise thousands of tokens per second on open models without relying solely on batching requests together.
The problem is that speed and compression tend to go hand in hand in inference marketing: it’s easy to speed up a model by pruning it or quantizing it aggressively, and it’s hard for an outside developer to verify which of the two is actually happening. The FAQ that Cerebras added directly answers that suspicion: the explicit question is whether they’ll change a model’s architecture without notice, and the answer is no. The company distinguishes quantization from pruning so they aren’t confused as synonyms.
Pruning does exist in Cerebras’s work, but it lives elsewhere. The company is researching a technique called REAP (Router-weighted Expert Activation Pruning), designed for mixture-of-experts models, where the goal is to remove the experts the router activates least frequently without degrading quality too much. Those pruned models are published in a Hugging Face collection for research use. None of them reach the shared Cerebras Inference API.
Model compression: quantization without pruning
The difference between quantization and pruning sounds technical, but it changes the final result. Quantizing means storing the same parameters with fewer bits: dropping a weight from 16 bits to 8 or 4 bits reduces memory without touching how many neurons or layers the model has. Pruning, on the other hand, removes entire parts of the architecture (layers, experts, attention heads) and literally produces a different model.
Cerebras applies only the first on its public endpoints. According to the documentation, model compression on its platform is selective weight quantization applied only to storage: weights are stored in a mix of 16, 8, and 4 bits depending on the layer, with the most error-sensitive layers kept at full precision and dequantized on the fly before being operated on. Activations, attention, and the KV cache remain at full precision throughout inference, unquantized.
💡 Tip: If you need to reproduce a result consistently across providers, ask first whether the model is pruned, not just whether it’s quantized: quantization changes numerical precision, pruning changes the architecture.
Technical details and performance
The public Cerebras Inference table shows, beyond the catalog itself, the actual tradeoff between speed and context offered by each of the two models currently available on shared endpoints:
| Model | Parameters | Context (free / paid) | Approximate speed | When to use it |
|---|---|---|---|---|
| GPT OSS 120B | 120 billion | 65k / 131k tokens | ~3,000 tokens/s | Tasks that need more context and maximum speed |
| Qwen 3.8 27B | 27 billion | 64k / 128k tokens | ~1,500 tokens/s | Lighter workloads, lower compute cost |
GPT OSS 120B, with 120 billion parameters, is the large model on the list: 65,000 tokens of context on the free tier, 131,000 on pay-as-you-go, and a reference speed close to 3,000 tokens per second. Qwen 3.8 27B is the lightweight option, with 27 billion parameters, 64,000 tokens of free context and 128,000 on paid, running at around 1,500 tokens per second.
The following diagram summarizes what happens to the weights of either model between the time a request arrives and the response goes out:
flowchart TD
A["Client / SDK"] --> B["Cerebras Inference API"]
B --> C{"Requested model"}
C -->|"gpt-oss-120b"| D["Weights at 4/8/16-bit per layer"]
C -->|"qwen-3.8-27b"| D
D --> E["Sensitive layers at full precision"]
E --> F["Attention and KV cache unquantized"]
F --> G["Response to client"]The point most worth pausing on is the sensitive-layers-at-full-precision step. Cerebras doesn’t quantize everything equally: it identifies which layers are most sensitive to precision loss, typically normalization layers and certain attention projections, and keeps them at 16 bits or higher, while the rest of the model drops to 8 or 4 bits to save memory on the chip.
To confirm in practice that the API returns the model you requested, and not a silent variant, the simplest call is a direct curl to the chat completions endpoint:
curl https://api.cerebras.ai/v1/chat/completions \n -H "Authorization: Bearer $CEREBRAS_API_KEY" \n -H "Content-Type: application/json" \n -d '{
"model": "qwen-3.8-27b",
"messages": [{"role": "user", "content": "Explain what selective per-layer quantization is"}]
}'The response carries the model field with the same ID that was sent, gpt-oss-120b or qwen-3.8-27b. If Cerebras ever offered a pruned variant, per its own policy, that variant would have a distinct, explicit model ID, and would never replace the original under the same name.
Getting started
Trying either model takes minutes because the Cerebras Inference API is compatible with the OpenAI SDK: just swap the base URL and the API key.
Install the SDK (Windows, macOS, and Linux)
The only prerequisite is installing the Python client; the command is identical on all three platforms, whether from PowerShell on Windows or a terminal on macOS and Linux:
# Windows (PowerShell), macOS, and Linux: same command
pip install openai
With the SDK installed, just point base_url to the Cerebras domain and use the API key generated in the account dashboard:
from openai import OpenAI
client = OpenAI(
base_url="https://api.cerebras.ai/v1",
api_key="YOUR_CEREBRAS_API_KEY",
)
response = client.chat.completions.create(
model="gpt-oss-120b",
messages=[
{"role": "system", "content": "You are a concise technical assistant."},
{"role": "user", "content": "Summarize in 3 lines what Cerebras's REAP is."},
],
max_tokens=300,
)
print(response.choices[0].message.content)
print(response.model) # confirms which model respondedThat script requests a response from GPT OSS 120B, prints the text, and also the response’s model field, which serves as a quick check that the backend actually ran the requested model.
Impact and analysis
For a team evaluating inference providers, this transparency changes a common question from “I trust it isn’t pruned” to “I can verify it by reading the documentation.” That’s not a minor detail when speed is the selling point: separating what part of the performance comes from the hardware and what part would come from trimming the model allows for a fair comparison across providers.
💭 Key point: Cerebras keeps research separate from product: REAP proves that pruning works and saves memory, but it only reaches Hugging Face for experimentation, never silently replacing a model already published on the shared API.
The real limitation of this announcement is that the public catalog remains short: only two models today, compared to the much broader offering other inference providers give on their free tiers. Accessing more model families requires moving to Dedicated Endpoints, which involves reserved capacity and, presumably, a different cost than pay-as-you-go. Anyone who needs a variety of models right now, with no capacity commitment, will find a broader catalog elsewhere; anyone who prioritizes verifiable speed on the two models that are available will find in Cerebras an option with clear, documented rules.
What’s next
The documentation leaves an explicit door open: if Cerebras ever decides to offer a pruned variant in production, it would be served under an endpoint with a specific pruning name, never replacing an existing model’s ID. It’s also reasonable to expect the public catalog to grow with new open families as they appear, following the same pattern of adding a few entries at a time seen with GPT OSS 120B and Qwen 3.8 27B.
Meanwhile, REAP research continues to be published for the community on Hugging Face, separate from the commercial product; that’s the place to look if the interest is understanding how far mixture-of-experts model pruning can go without losing quality, not on Cerebras Inference’s production endpoints.
📖 Summary on Telegram: View summary
Try it yourself: generate an API key on Cerebras and run the curl example above against gpt-oss-120b to see the speed firsthand, without waiting for someone else to tell you about it.
Frequently Asked Questions
Does Cerebras prune the models it serves in production?
No. The official documentation states that no model on Cerebras Inference’s public endpoints is pruned; all are original versions with no architecture modification.
What is REAP and where can it be tried?
REAP (Router-weighted Expert Activation Pruning) is the pruning technique Cerebras is researching for mixture-of-experts models. The resulting models are published in a Hugging Face collection for research, not on the shared API.
Does the quantization Cerebras applies affect response quality?
Cerebras quantizes weights only for storage, in a mix of 16, 8, and 4 bits depending on the layer, keeps sensitive layers at full precision, and leaves activations, attention, and the KV cache unquantized during computation, which limits the impact compared to uniform, aggressive quantization.
Which models are available today on the free tier?
GPT OSS 120B, with 120 billion parameters and up to 65k tokens of free context, and Qwen 3.8 27B, with 27 billion parameters and up to 64k tokens of free context.
How do I access more models beyond these two?
Cerebras offers additional families through Dedicated Endpoints, a product with reserved capacity and production SLAs, designed for workloads that exceed what the public catalog covers.
References
- Cerebras Inference, Model Catalog: primary source with the model table and compression policy.
- Hugging Face: where Cerebras publishes its collection of REAP-pruned models for research.
- Cerebras: official site of the company behind the Wafer Scale Engine and Cerebras Inference.
- OpenAI: origin of the open-weight model GPT OSS 120B, also available on other inference providers.
📱 Enjoy this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day. @programacion
0 Comments