Qwen3.8-27B-FP8: Alibaba Releases the Model with Native FP8 and Reasoning Effort

⏱️ Lectura: 11 min

Alibaba uploaded a new checkpoint of its Qwen family to Hugging Face: Qwen3.8-27B-FP8, a 27-billion-parameter model that arrives already quantized in FP8 out of the box and lets you choose, request by request, how long it should “think” before responding.

📑 En este artículo

1. TL;DR
2. Introduction
3. What Happened
4. Context and History
5. Technical Details and Performance
   1. Reasoning Effort
   2. Native Tool Calling
   3. Vision and Video
6. How to Start Testing It
7. Impact and Analysis
8. What’s Next
9. Frequently Asked Questions
   1. What does it mean for the checkpoint to be “native FP8”?
   2. What’s the difference between reasoning_effort xhigh, medium, and low?
   3. Does Qwen3.8-27B-FP8 support images and video?
   4. How do I call tools (tool calling) with this model?
   5. What hardware do I need to run it with a real advantage?
   6. Is it compatible with vLLM?
10. References

The model card exposes the full chat template in Jinja format, and that’s where the most interesting details show up: explicit reasoning effort control, native tool calling with XML-style syntax, and vision support for images and video within the same checkpoint.

TL;DR

• Alibaba published Qwen3.8-27B-FP8 on Hugging Face, a native FP8 checkpoint with 27 billion parameters.
• The model’s chat template lets you choose the reasoning_effort level: xhigh (default), medium, or low.
• With xhigh, the system injects an instruction to validate assumptions and consider alternatives before responding.
• With low, the instruction asks the model to keep its reasoning brief and get straight to the conclusion.
• The model supports native tool calling with XML syntax: nested tool_call, function, and parameter tags.
• It also processes images and video through vision tokens within the same FP8 checkpoint.
• Native FP8 reduces disk size and VRAM usage compared to an equivalent BF16 checkpoint, but it requires a GPU with FP8 support (Hopper, Ada, Blackwell).

Introduction

Qwen3.8-27B-FP8 belongs to the family of open models that Alibaba publishes under the Qwen user on Hugging Face. What makes this release distinctive isn’t just the size (27B parameters, a middle ground between a lightweight laptop model and a frontier model that needs a cluster), but that the checkpoint comes natively quantized in FP8: it’s not a BF16 model that you quantize afterward with bitsandbytes or AWQ, but weights trained and published directly in that format.

The second relevant point is in the chat template, the Jinja file that defines how the final prompt the model receives gets built. There, Alibaba documented, right in the template itself, a reasoning effort control mechanism (reasoning_effort) that any developer can activate from transformers without touching the weights or switching models.

What Happened

The Qwen/Qwen3.8-27B-FP8 repository was published on Hugging Face with its tokenizer_config.json including an extensive chat template that handles several things at once: building multimodal messages (text, image, and video), injecting system instructions based on the chosen reasoning effort, and serializing tool calls in a tag-based format.

Unlike other models where the “thinking level” is controlled through a separate generation parameter or a manual prompt, here the template itself handles the routing: if you don’t pass anything, it defaults to xhigh, and only changes behavior if you explicitly request medium, low, or disable reasoning with enable_thinking=false.

Context and History

The Qwen family has been evolving since 2023 as one of the most active open model lines, alongside Llama, Mistral, and DeepSeek. Each generation has added capabilities that used to live in separate models: first long context, then vision, then structured tool calling, and more recently, explicit reasoning modes with visible thinking blocks (the pattern popularized by models like QwQ and DeepSeek R1).

Qwen3.8-27B-FP8 consolidates those three capabilities (vision, tool calling, and configurable reasoning) into a single checkpoint, and it also solves the size problem by publishing directly in FP8 instead of leaving quantization as a task for the end user.

Technical Details and Performance

The most useful thing for a developer evaluating Qwen3.8-27B-FP8 is understanding the three mechanisms exposed by the chat template, since they determine how it integrates into a real pipeline.

Reasoning Effort

The template defines three reasoning effort levels. With xhigh (the default value when enable_thinking is active), the system literally receives this instruction: “Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.” With low, the instruction changes to: “Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.” The medium level doesn’t add any extra instruction: it’s the neutral point.

Level	When to Use It	Behavior	Trade-off
xhigh (default)	Math, code, multi-step analysis	Validates assumptions and considers alternatives before responding	More thinking tokens, higher latency and cost
medium	General use, medium-difficulty tasks	No extra instruction in the system prompt	Middle ground between speed and depth
low	Latency-critical agents, classification, extraction	Goes straight to the conclusion, no extra elaboration	Less internal verification from the model itself

Native Tool Calling

The template builds a system block with the list of available functions under a tools tag in JSON format, and instructs the model to respond exclusively with a tool_call block that wraps a function with a name and parameters. It’s a tag-based syntax, not pure JSON like OpenAI’s, and the template itself warns that required parameters must be present and that no text should be added after the call block.

Vision and Video

The template also handles multimodal messages: when a message item carries an image, it inserts the tokens <|vision_start|><|image_pad|><|vision_end|>; for video, it uses the equivalent with video_pad. With add_vision_id active, the model even numbers images (“Video 1:”, “Video 2:”) when there are several in the same conversation.

flowchart TD
A["User request"] --> B{"reasoning_effort?"}
B -->|"xhigh"| C["Validates assumptions and alternatives"]
B -->|"medium"| D["No extra instruction"]
B -->|"low"| E["Goes straight to the conclusion"]
C --> F["Final response"]
D --> F
E --> F

⚠️ Heads up: Native FP8 doesn’t run at full speed on just any GPU. You need tensor cores with FP8 support (Hopper H100, Ada L4/L40, Blackwell); on an Ampere A100, the runtime usually upcasts to BF16 and you lose much of the memory advantage.

How to Start Testing It

To run Qwen3.8-27B-FP8 locally with transformers, the flow is the same on Windows, macOS, and Linux; only how you activate the virtual environment changes.

Windows (PowerShell):

python -m venv .venv
.venv\Scripts\Activate.ps1
pip install --upgrade transformers accelerate torch

macOS / Linux:

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade transformers accelerate torch

With the environment ready, loading the model and generating a response with low reasoning effort (useful for quick testing) looks like this:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Qwen/Qwen3.8-27B-FP8"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

mensajes = [{"role": "user", "content": "Explain what FP8 is in two sentences"}]
prompt = tokenizer.apply_chat_template(
    mensajes,
    tokenize=False,
    add_generation_prompt=True,
    reasoning_effort="low"
)
entradas = tokenizer(prompt, return_tensors="pt").to(model.device)
salida = model.generate(**entradas, max_new_tokens=200)
print(tokenizer.decode(salida[0][entradas.input_ids.shape[1]:], skip_special_tokens=True))

This first example asks for a short, direct response. For a more realistic case, with tool calling and high reasoning effort because the task requires carefully choosing which function to invoke:

tools = [
    {
        "name": "consultar_tipo_cambio",
        "description": "Returns the current dollar exchange rate for a LATAM country",
        "parameters": {
            "type": "object",
            "properties": {
                "pais": {"type": "string", "description": "Country name, e.g: Argentina, Mexico, Chile"}
            },
            "required": ["pais"]
        }
    }
]

mensajes = [{"role": "user", "content": "What's the exchange rate for the dollar in Argentina today?"}]
prompt = tokenizer.apply_chat_template(
    mensajes,
    tools=tools,
    tokenize=False,
    add_generation_prompt=True,
    reasoning_effort="xhigh"
)
# The model responds with:
# <tool_call>
# <function=consultar_tipo_cambio>
# <parameter=pais>
# Argentina
# 
# 
# </parameter=pais></function=consultar_tipo_cambio></tool_call>

To confirm that the effort level was actually applied, you don’t need to generate text: it’s enough to print the prompt returned by apply_chat_template before tokenizing it. If you passed reasoning_effort="xhigh", you’ll see the “validate key assumptions, consider plausible alternatives” instruction injected into the system message; with "low", on the other hand, the instruction to go straight to the conclusion appears. It’s the fastest way to verify that the parameter wasn’t ignored.

To serve the model in production with better performance than plain transformers, the usual route is vLLM, which supports FP8 checkpoints natively:

pip install vllm
vllm serve Qwen/Qwen3.8-27B-FP8 --port 8000

Impact and Analysis

The core contribution of Qwen3.8-27B-FP8 isn’t a benchmark number, but a design shift: moving the decision of “how much to reason” from the system prompt side to a documented chat template parameter. That simplifies the work for anyone building agents: instead of maintaining two models (a fast one for trivial tasks, a slow one for complex tasks) or writing manual prompts to simulate that behavior, a single argument in apply_chat_template is enough.

The cost of that flexibility shows up in the infrastructure. Publishing the checkpoint directly in FP8 reduces disk space and VRAM usage compared to a BF16 equivalent, but it ties the model’s real efficiency to the available hardware. A team still running Ampere GPUs (A100, RTX 30xx) won’t see the full memory advantage, because those cards don’t have native FP8 tensor cores and the runtime ends up upcasting.

💡 Tip: in agent pipelines with many trivial calls (classifying, extracting a field, deciding whether to call a tool), try reasoning_effort="low" first: you cut down on thinking tokens and latency, and save xhigh only for the steps that truly need it.

Another point worth considering: the tag-based tool calling format (tool_call/function/parameter) isn’t pure JSON like the OpenAI or Anthropic APIs. If your agent framework expects strict JSON in the tool response, you’ll need a custom parser or an adaptation layer before connecting Qwen3.8-27B-FP8 to tools already written for another format.

What’s Next

It’s likely that the configurable reasoning_effort pattern, already used by OpenAI and Anthropic models under different names, will end up standardized across open models: today each family exposes it with its own parameter name and its own instruction text, which forces developers to rewrite integration logic every time they switch models.

For teams in Latin America evaluating open models with multimodal support and tool calling in a single checkpoint, Qwen3.8-27B-FP8 is worth testing first on a rented GPU environment (RunPod, Lambda, or an instance with H100/L4) before deciding on buying dedicated hardware, precisely because of its dependency on FP8 tensor cores.

📖 Summary on Telegram: View summary

Try it yourself: run pip install transformers accelerate torch and load Qwen/Qwen3.8-27B-FP8 from the official model card on Hugging Face to see reasoning_effort in action today.

Frequently Asked Questions

What does it mean for the checkpoint to be “native FP8”?

It means the weights were trained and published directly in 8-bit floating point format, without going through a later quantization process handled by the user, unlike BF16 checkpoints that get quantized afterward with AWQ or GPTQ.

What’s the difference between reasoning_effort xhigh, medium, and low?

Each level injects a different instruction into the chat template’s system message. xhigh asks the model to validate assumptions and consider alternatives, low asks it to go straight to the conclusion, and medium doesn’t add any extra instruction.

Does Qwen3.8-27B-FP8 support images and video?

Yes, the chat template inserts specific vision tokens (vision_start, image_pad or video_pad, vision_end) for each multimodal item in the message.

How do I call tools (tool calling) with this model?

By passing the list of functions in the tools parameter of apply_chat_template. The model responds with a tool_call tag block that wraps the function name and its parameters.

What hardware do I need to run it with a real advantage?

A GPU with native FP8 tensor cores: Hopper (H100), Ada Lovelace (L4, L40), or Blackwell. On Ampere GPUs, the runtime may upcast to BF16, losing part of the memory advantage.

Is it compatible with vLLM?

Yes, vLLM supports FP8 checkpoints natively and lets you spin up the model as an OpenAI API-compatible server with vllm serve.

References

• Qwen/Qwen3.8-27B-FP8 on Hugging Face: model card and full chat template.
• Official QwenLM repository on GitHub: code and documentation for the Qwen family.
• Hugging Face Transformers chat templating documentation: how apply_chat_template works.
• vLLM on GitHub: inference engine with native support for FP8 checkpoints.

📱 Enjoy this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.

Imagen destacada: Foto de Vishnu Mohanan en Unsplash