⏱️ Lectura: 10 min
Running a 26 billion parameter model on an 8 GB laptop sounds like a contradiction. The math doesn’t add up: a model of that size usually requires 50 GB or more of memory if loaded in full. TurboFieldfare, published by developer drumih on GitHub, proposes a different approach: never load the full model at all.
📑 En este artículo
The project is called TurboFieldfare, and its stated goal is to run Gemma 4 26B-A4B, Google’s mixture-of-experts instruction-tuned variant, on any Apple Silicon Mac. That includes 8 GB RAM machines, the segment where nobody usually even attempts to run a model of this size.
TL;DR
- TurboFieldfare is a Swift and Metal runtime that runs Gemma 4 26B-A4B with just about 2 GB of RAM.
- The model weighs 14.3 GB installed on disk, but only loads a shared core of 1.35 GB into memory, plus the KV cache.
- It works on any Apple Silicon Mac, including an 8 GB MacBook Air M2.
- On the 8 GB M2 it measures between 5.1 and 6.3 tokens per second during decoding.
- On a Mac with a 24 GB M5 Pro it reaches between 31 and 35 tokens per second.
- It uses 4-bit quantization for weights and experts, and 8-bit for the mixture-of-experts router.
- It’s a runtime custom-built for this model, not a wrapper over MLX or llama.cpp.
- The GitHub repository has already racked up 1,500 stars and 47 forks.
Introduction
The TurboFieldfare project shows that mixture of experts (MoE) isn’t only useful for scaling models, it can also shrink their memory footprint in real time.
It’s a native runtime, not a generic library: it’s written specifically for Gemma 4 26B-A4B, and it doesn’t function as a wrapper over MLX or llama.cpp.
What Happened
The drumih/turbo-fieldfare repository distributes a complete runtime written in Swift and Metal, with no dependencies on MLX or llama.cpp. It’s specific to this model: it’s not a generic layer that also works with other weights.
The Swift package exposes six distinct products, each with a specific function: the runtime library, the native Mac app, a single-use decoding service, a command-line interface, an OpenAI API-compatible server, and an installer that packages the model.
They all share the same model directory, with a .gturbo extension, but only one product can own the model and Metal at a time. If the app is running, the CLI can’t load the model at the same time.
Context and Background
Gemma 4 26B-A4B is a mixture of experts (MoE) model: of its 26 billion total parameters, only about 3.88 billion get activated for each token generated. The MoE architecture splits the model into multiple experts and a router that decides which ones to use at each step.
That’s the separation TurboFieldfare takes advantage of. If only a fraction of the parameters participate in each token, there’s no need to keep all the parameters in memory at once. It’s enough to have the shared core ready (embeddings, attention, common layers) and pull from disk only the specific experts the router chooses.
This idea of streaming weights from storage instead of keeping them all in RAM isn’t unique to this project: it’s the same logic behind offloading techniques that various teams have been exploring to run large models on consumer hardware. TurboFieldfare pushes it to the extreme with a native Metal runtime instead of relying on a generic framework.
Technical Details and Performance
The project’s numbers are concrete. The installed model takes up 14.3 GB on disk, in its text-only version. In memory, TurboFieldfare keeps just about 2 GB: the 1.35 GB shared core plus an FP16 KV cache with a 4K token window.
Quantization is aggressive but segmented by role. The weights use MLX’s 4-bit affine format with a group size of 64, both for the shared core and for the routed experts. The router, on the other hand, stays at 8 bits: it’s the piece that decides which expert to activate, and an error there propagates through the entire generation.
Measured results vary by hardware. On an 8 GB MacBook Air M2, decoding measures between 5.1 and 6.3 tokens per second. On a Mac with an M5 Pro chip and 24 GB of RAM, it rises to a range of 31 to 35 tokens per second.
📌 Note: the project itself clarifies that these numbers are a reference point, not a performance ceiling. Prompt length, generation length, the state of the operating system’s page cache, and the specific hardware all affect the result.
This variability is expected in any disk-streaming scheme: the first token after loading a new expert costs more than one whose expert already lives in the system’s page cache.
flowchart TD
A["User prompt"] --> B["Shared core in RAM"]
B --> C["MoE router at 8 bits"]
C --> D["Required expert on SSD"]
D --> E["OS page cache"]
E --> F["Generated token"]
subgraph Memory
B
end
subgraph Disk
D
end| Swift Product | What It Does | When to Use It |
|---|---|---|
TurboFieldfare | Library with the runtime and Metal kernels | If you’re integrating the engine into your own app |
TurboFieldfareMac | Native Mac app for installing and generating text | Direct interactive use, no code required |
TurboFieldfareDecodeService | Single-use process that owns the model and Metal | Invoked by the app; not used manually |
TurboFieldfareCLI | Instruction chat and raw completion via terminal | Scripts, quick tests, automation |
TurboFieldfareServer | Local server compatible with OpenAI Chat Completions | If you already have code that talks to the OpenAI API |
TurboFieldfareRepack | Installer that downloads and verifies the model | First installation or to revalidate the package |
Getting Started
TurboFieldfare is Apple Silicon exclusive: the package is pure arm64 and doesn’t run on Windows, Linux, or Intel-based Macs. It requires macOS 26 with Metal 4, Xcode 26, and Swift 6.2 or later. There’s no alternative build for other platforms because the runtime depends directly on Metal.
To install and build it on an Apple Silicon Mac:
git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
swift build -c release
.build/release/TurboFieldfareMacThe first build downloads the Swift packages the tokenizer needs. It’s best to build the whole package so the app and its decoding service become available together.
When opening the app for the first time, you need to select Download so TurboFieldfare fetches the model (about 15 GB) from the Hugging Face repository the project points to. The installer never materializes the full original checkpoint: it streams the necessary byte ranges and repackages them directly into the .gturbo format as they arrive.
Once installed, Load Model loads the model into memory and Generate starts generation. The defaults are temperature 0.2, Top-K 64, and Top-P 0.95; if you need deterministic output, set the temperature to 0.
To integrate it into an existing script, the local OpenAI-compatible server is the most direct route. Here’s an example call from any language that already uses the Chat Completions API:
curl http://127.0.0.1:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gemma-4-26b-a4b",
"messages": [
{"role": "user", "content": "Explain what a mixture of experts model is in two sentences."}
],
"temperature": 0.2
}'The response comes back in the same format the OpenAI API returns, so any existing Chat Completions client works without modification, just pointing to the local port instead of the OpenAI endpoint.
To confirm the model is running with the reduced memory profile, just open macOS Activity Monitor and check the resident memory of the TurboFieldfareDecodeService process while generating text: it should hover around 2 GB and stay nowhere near the 14.3 GB of the model installed on disk.
Impact and Analysis
What matters about TurboFieldfare isn’t just the RAM number: it’s that it runs a 26B mixture-of-experts model on the cheapest Mac segment Apple sells today, the 8 GB MacBook Air M2. That segment usually gets left out of any serious experiment with large models.
The cost of that memory economy is predictable: the model depends on disk at every step where it needs an expert that isn’t already cached. On a slow SSD or with little page cache headroom, performance drops below what the project measured.
There are also scope limits. TurboFieldfare is text-only: it doesn’t process images, audio, or video, even though Gemma 4 is multimodal in other variants. The app and the CLI don’t execute tools either: only the loopback server accepts function declarations and returns the calls for the client to authorize and execute, without running them on its own.
💡 Tip: if your Mac has 8 GB of RAM and you’re already running Chrome, Slack, and an IDE at the same time, give TurboFieldfare real headroom: close what you don’t need before loading the model, because the operating system is also competing for that same page cache that speeds up expert reads.
The other honest point is the project’s own warning: the model can repeat itself or give incorrect answers, just like any LLM. Shrinking the memory footprint doesn’t change the underlying model’s quality, only where and how it runs.
What’s Next
The project keeps a curated log of experiments with 103 measured results covering kernels, caching, I/O, prefill, and decoding, meant as a reference for anyone who wants to optimize their own fork. It also publishes a guide for the community to contribute benchmark results on other Apple Silicon models, which matters because the project only officially validates against an 8 GB M2 and a 24 GB M5 Pro.
The pattern of streaming experts from SSD that TurboFieldfare uses is replicable for other MoE models, as long as someone writes the specific runtime. That’s the heavy lifting: it’s not a configuration tweak, it’s a custom-built inference engine in Swift and Metal.
📖 Summary on Telegram: View summary
Try it yourself: clone the repository with git clone https://github.com/drumih/turbo-fieldfare.git and run swift build -c release on any Apple Silicon Mac to see the model load in minutes.
Frequently Asked Questions
What do I need to run TurboFieldfare?
An Apple Silicon Mac, macOS 26 with Metal 4, Xcode 26, and Swift 6.2 or later, plus free space for the roughly 14.3 GB the installed model takes up.
Does it run on Windows or Linux?
No. The package is pure arm64 and depends on Metal, Apple’s graphics framework, so it only works on Apple Silicon Macs.
How much RAM do I actually need?
The project is validated on a MacBook Air M2 with 8 GB of total RAM, of which TurboFieldfare uses about 2 GB for the shared core and the KV cache.
Can I use it with images or audio?
No. TurboFieldfare is a text-only runtime: it doesn’t accept or generate images, audio, or video.
Is it faster than loading the full model into RAM?
The project doesn’t publish that direct comparison. What it does measure is the performance of the streaming scheme: between 5.1 and 6.3 tokens per second on an 8 GB M2, and between 31 and 35 on a 24 GB M5 Pro.
Can I use the app and the CLI at the same time?
No. All the products share the same .gturbo model directory, but only one can own the model and Metal at any given time.
References
- TurboFieldfare repository on GitHub: the project’s source code, benchmarks, and installation guide.
- Official Gemma portal: documentation for the Google model family that TurboFieldfare runs.
- Metal documentation: Apple’s graphics and compute framework that the runtime runs on.
- Swift.org: the official site for the language the entire project is written in.
📱 Enjoy this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
Imagen destacada: Foto de Nikolai Chernichenko en Unsplash
0 Comments