ggml 1.0.0 Removes Its Legacy API and Adds FP8 Support

⏱️ Reading time: 6 min

ggml, the C and C++ library that powers llama.cpp and whisper.cpp, released version 1.0.0 on GitHub on November 13, 2026. The release strips out the public API marked as deprecated, removes two legacy operations, and adds support for the FP8 floating-point format (v1.0.0 release notes).

📑 En este artículo

1. What ggml 1.0.0 breaks
2. Performance work spread across six backends
3. What we tested: cloning and installing ggml in a sandbox
4. What this means if you maintain a binding or a fork
5. ggml vs. TensorFlow and Transformers
6. Limitations and when not to use it directly
7. References

What ggml 1.0.0 breaks

The most disruptive change is internal: ggml removed the public API that had been deprecated for months and dropped the legacy operations DIAG_MASK_INF and DIAG_MASK_ZERO. Alongside that cleanup, the RPC protocol jumped to version 9 and now outright rejects any peer speaking an older version, instead of failing halfway through the handshake (v1.0.0 release).

Two new functions also landed, ggml_rope_set_truncate and ggml_rope_get_truncate, which let you configure the rounding of the YaRN correction-dim boundary, the technique that extends a model’s context beyond what it was trained on. That value used to be fixed; now any project using ggml as its engine can tune it to its own context-extension scheme.

Performance work spread across six backends

Most of the release is performance work, spread backend by backend. CUDA gained small-batch MMQ for batches of 2 to 24, OCP-standard FP8 support, and fused NVFP4 matmuls with GLU; Metal fused MoE routing into a single command buffer. Vulkan adds a FlashAttention kernel for prefill on Intel GPUs, SYCL gets a device-side MoE scheduler, and OpenCL incorporates split FlashAttention for Adreno (v1.0.0 release).

CUDA also gained a HIP branch for AMD GPUs: a fixed BLAS workspace and an MFMA indexer built for the CDNA2 architecture, plus a fast path for state scanning (SSM_SCAN) with d_state=64 and a radix Top-K that is now deterministic on ties, which matters for reproducing benchmark results (v1.0.0 release).

The other backends moved forward too: WebGPU added new kernels (GROUP_NORM, MEAN, COUNT_EQUAL, PAD_REFLECT_1D), and Hexagon widened its operation coverage with Q5_0/Q5_1 support and 3D convolution. At the central scheduler level, ggml now avoids uploading inputs to the GPU that no operation will read, and only prefetches what the previous split produces: fewer memory transfers for every compute graph that runs.

What we tested: cloning and installing ggml in a sandbox

We cloned the repository in an isolated sandbox (Python 3.12.16, Node v18.20.4, Git 2.39.5, no network isolation) to see how simple it is to get started from scratch:

git -c protocol.allow=never -c protocol.https.allow=always -c core.hooksPath=/dev/null clone --depth 1 --single-branch --no-tags -- https://github.com/ggml-org/ggml /tmp/lab-runs/lab-b2w_j88s/w/repo

The clone downloaded 32.4 MB in 1.81 seconds with a peak memory of just 33.5 MB, nothing unusual for a repo this size. The problem showed up in the next step, installing the repository’s Python dependencies:

python3 -m venv .venv
.venv/bin/python -m pip install --no-input -r requirements.txt

That install failed with exit code 2 after 38.88 seconds and a peak memory of 151.9 MB. Before pip’s output got truncated, it showed that requirements.txt tries to pull down the full TensorFlow 2.20.0 and Torch ~2.11.0, with their entire dependency tree (tensorboard, sympy, keras, rich). That finding contrasts with the repo’s own official description: “Plain C/C++ implementation without any dependencies” (ggml-org/ggml). That line describes the core C/C++ library, but the Python tooling that lives in the same repo (conversion scripts and tests) does pull in a full ML stack, and in our sandbox that stack did not even finish installing in the time we gave it.

What this means if you maintain a binding or a fork

If you maintain a fork of llama.cpp, whisper.cpp, or any Python, Node, or Rust binding that links against ggml as a submodule, the jump to 1.0.0 means recompiling: code that still called the deprecated public API or DIAG_MASK_INF/DIAG_MASK_ZERO simply stops compiling. Check your CMakeLists.txt and your calls to those two operations before bumping the submodule in CI.

If you also run ggml-rpc-server to split compute across machines, say one node with a GPU and one without, update client and server at the same time. With the RPC protocol at v9, an old peer can no longer even complete the handshake; it is not a silent failure. And if you plan to contribute to the repo’s Python scripts (weight conversion, benchmarks), assume you will need far more disk space and install time than the “no dependencies” pitch suggests: our attempt stalled midway through downloading TensorFlow and Torch.

If you instead build for the browser or for an embedded device with WebAssembly, the WebGPU additions are the ones that affect you directly: GROUP_NORM, MEAN, COUNT_EQUAL, and PAD_REFLECT_1D previously had no native kernel on that backend and likely fell back to a slower path or failed outright. If your build targets WebGPU, it is worth checking whether your compute graph uses any of those four operations and re-measuring performance after updating.

ggml vs. TensorFlow and Transformers

ggml sits at a much lower level than the alternatives with more GitHub stars. TensorFlow (tensorflow/tensorflow, about 201,596 stars as of 2026-11-14) and Hugging Face’s Transformers (huggingface/transformers, about 168,966 stars as of 2026-11-14) are full frameworks with training, serialization, and an already-integrated model catalog. ggml, by contrast, is the tensor-compute layer that runs underneath projects like llama.cpp: no autograd, no Python as the primary language, built to compile and run inference on the end device, from a phone to a GPU cluster.

Project	What it is	Stars (2026-11-14)	Core dependencies
ggml	C/C++ tensor library, no autograd	~15,765	None (pure C/C++)
TensorFlow	Full training and inference framework	~201,596	Python, CUDA, protobuf, and dozens more
Transformers	Pretrained model library on top of PyTorch/TF/JAX	~168,966	PyTorch or TensorFlow as backend

The research bundle also groups ggml with f/prompts.chat (about 177,268 stars as of 2026-11-14) because they share the GitHub topic “machine-learning”, but it is not a tensor library: it is a collection of prompts for ChatGPT. They share a tag, not a function, so the real comparison for ggml is against TensorFlow and Transformers.

Limitations and when not to use it directly

ggml has no installer or prebuilt package: the documentation itself opens with “Build from source” and a minimal matrix-multiplication example (ggml-org/ggml). If your team does not have someone comfortable compiling C++ for the target platform (x86, ARM, RISC-V, WebAssembly), you are probably better off using ggml indirectly, through llama.cpp, whisper.cpp, or an already-packaged binding, rather than directly.

Model coverage also has gaps acknowledged by the project itself: issue #59, still open, asks for native text-to-speech support and remains unresolved. And the project’s governance is in transition: ggml.ai joined Hugging Face to secure the continuity of local AI, something the Hacker News community actively discussed (HN thread). It is still too early to tell how that change of umbrella will affect the pace of releases like this one.

References

• GitHub, ggml-org/ggml release v1.0.0: release notes covering API, performance, and RPC protocol changes.
• GitHub, ggml-org/ggml: project description, stars, forks, license, and primary language.
• GitHub, issue #59: open request for native text-to-speech model support.
• Hacker News: discussion about ggml.ai joining Hugging Face.
• GitHub, tensorflow/tensorflow: reference framework for the scope and dependency comparison.
• GitHub, huggingface/transformers: pretrained model library used as the second comparison point.

Featured image: Foto de Ilya Pavlov en Unsplash

How each article is made (sources, testing and editorial criteria): How we work.