What Is ggml? The Engine Behind llama.cpp

ggml is a small open source tensor library, written in C and C++, that does the actual math when llama.cpp or whisper.cpp runs a model on your computer. It has no outside dependencies, runs on CPUs, GPUs, NPUs and in the browser, supports 2 to 8 bit quantized weights, and is MIT licensed. On November 13, 2026 the project shipped ggml v1.0.0 alongside llama.cpp v1.0.0, and from now on a major version bump signals a breaking change to its public C API.

If you only chat with local models, you never touch ggml directly. It matters when you build apps on top of llama.cpp, package it for a Linux distribution, or want to know why your GGUF file loads on a laptop and a phone alike. This guide explains what ggml is, how it differs from GGUF, what the 1.0 release changed and how to install it. Facts come from the ggml README, the v1.0.0 release notes, the llama.cpp v1.0.0 announcement and the release and versioning policy, all checked on November 14, 2026. We also built v1.0.0 ourselves; see the install section.

What ggml is, in plain words

Think of ggml as the engine and llama.cpp as the car. A language model is a long list of matrix multiplications and other tensor operations. ggml defines those operations, plans the order they run in, and executes them on whatever hardware you have. llama.cpp adds everything a user sees: model loading for more than 150 architecture variants, chat templates, sampling, the llama-server API and the web UI.

The ggml README lists the design goals:

• Plain C and C++ with no dependencies
• Runs on x86, ARM, RISC-V, LoongArch, PowerPC, s390x and WebAssembly
• SIMD optimized kernels for x86, ARM and RISC-V
• Backends for CPU, GPU, NPU and browser
• 2 to 8 bit integer quantization, plus the MXFP4 and NVFP4 microscaling formats
• Zero memory allocations while a model is running

The project site at ggml.ai says it is used by llama.cpp and whisper.cpp and is "freely available under the MIT license." The same page says ggml.ai is a company Georgi Gerganov founded in 2023 to support the library, with pre seed funding from Nat Friedman and Daniel Gross, and that Hugging Face acquired it in 2026. The repository itself dates to September 2022 and had about 15,700 GitHub stars when we checked.

Other local tools build on it too. A Hugging Face introduction to ggml from August 2024 named Ollama, Jan, LM Studio and GPT4All as projects using ggml under the hood. If you are choosing between those apps, our Ollama vs llama.cpp vs LM Studio comparison covers the user side.

ggml vs GGUF vs llama.cpp

The names are confusing because "GGML" was once also the name of a file format. Here is how the pieces fit today.

The GGUF specification lives in the ggml repository and calls GGUF "a file format for storing models for inference with GGML." It replaced three older formats called GGML, GGMF and GGJT. So when an old tutorial says "download the GGML file," it means one of the formats that came before GGUF. For the quant names inside GGUF files (Q4_K_M, Q8_0 and so on), see our GGUF quantization guide.

What changed in ggml v1.0.0

ggml had been releasing numbered versions for a few months (v0.27.0 on October 15, v0.28.0 on October 28, v0.29.0 on November 4). The v1.0.0 release landed on November 13, 2026 at about 3:34 AM ET, and a Hacker News post about it that evening reached the front page with roughly 300 points. The headline items from the release notes:

• API cleanup. Deprecated public functions were pruned and the old DIAG_MASK_INF and DIAG_MASK_ZERO operations removed. The RPC protocol moved to version 9 and now rejects incompatible peers.
• New controls. Configurable YaRN truncation for long context scaling (ggml_rope_set_truncate) and a backward pass for the TRI operation.
• FP8. CUDA gained OCP FP8 support, and llama.cpp added an FP8 KV cache in the same cycle.
• CUDA. Faster quantized matrix multiplication for small batches of 2 to 24, fused NVFP4 gate and up projections, and several mixture of experts fixes.
• Other backends. Fused MoE routing on Metal, an Intel FlashAttention prefill kernel on Vulkan, a device side MoE scheduler on SYCL, FlashAttention for Qualcomm Adreno GPUs in OpenCL, new WebGPU kernels and Q5_0 and Q5_1 support on Hexagon.
• Scheduler. It now skips uploading inputs that are never read, which saves transfers on multi device setups.

None of this changes how you run a model from the command line. The real news is the version number.

Semantic versioning: what 1.0 means for you

The llama.cpp v1.0.0 post sets one rule for every ggml-org project that uses semantic versions:

The versioning policy adds detail that matters for packagers. llama.cpp keeps its own internal copy of ggml, which can drift from the released library. Nightly llama.cpp builds use b numbers (b12377 was the latest when we checked), while tagged releases such as v1.0.0 are only cut when the internal copy matches a released ggml exactly. That means a program built from a llama.cpp release tag can safely link to a system wide ggml.

In practice:

• Homebrew ships ggml 1.0.0 and llama.cpp 1.0.0 as separate formulas, and its llama.cpp links to the shared ggml.
• Debian packages ggml and a separate ggml-cuda (split out because CUDA is not free software), and its llama.cpp and whisper.cpp link to them.
• Winget and the llama.app installer bundle their own ggml copy and publish nightly builds.
• App developers who embed libllama or libggml can now pin a major version and expect minor updates not to break their build.

If you just want llama.cpp running, our step by step llama.cpp guide covers every install route.

How ggml runs a model

You do not need this to use llama.cpp, but it explains why ggml is so portable. The Hugging Face introduction describes a handful of core objects:

1. Context (ggml_context): a container that holds tensor metadata and, optionally, the data.
2. Graph (ggml_cgraph): the order of operations, built before anything is computed.
3. Backend (ggml_backend): the interface that executes a graph on CPU, CUDA, Metal, Vulkan, RPC and others.
4. Buffers and allocators: memory reserved on a device up front, which is how ggml avoids allocating while a model runs.
5. Scheduler (ggml_backend_sched): splits one graph across several devices and sends operations a GPU backend cannot run to the CPU.

That last point is how llama.cpp runs a model larger than your VRAM: part of the graph runs on the GPU and the rest in system RAM. Our guide on whether a local LLM can use RAM instead of VRAM shows the flags that control it.

ggml is mainly an inference library, but it is not inference only. The repository ships ggml-opt.h, which the header describes as "functionality for training models using GGML," with datasets and loss functions. For real fine tuning, Python frameworks remain the normal choice; see our best ML frameworks for Python roundup.

Supported hardware

Each backend can be compiled as a separate library and loaded at runtime. The llama.cpp README lists these, among others:

Not every operation exists on every backend, a limit the Hugging Face introduction flags. The scheduler's CPU fallback covers the gaps, at a speed cost.

How to install ggml

Most people get ggml inside llama.cpp and never install it alone. If you need the library itself:

• macOS or Linux with Homebrew: brew install ggml (version 1.0.0 when we checked).
• Debian: install the ggml package, and ggml-cuda for NVIDIA GPUs.
• From source: clone the repository and build with CMake, as the README shows:

git clone https://github.com/ggml-org/ggml
cd ggml
cmake -B build
cmake --build build --config Release -j 8

We built the v1.0.0 tag on an 8 vCPU x86_64 Linux machine with the CPU backend only. Configuring and compiling the simple-ctx example took about 65 seconds, and it printed the expected 4 by 3 matrix product (first row 60, 55, 50, 110). CMake reported "ggml version: 1.0.0" with commit 417a024, and the shared libraries carry the new major version in their names (libggml.so.1.0.0, linked as libggml.so.1). One note from our run: the box had no system C++ compiler, so we pointed CMake at clang 22 and Ninja. With GCC and g++ installed, the default commands above should work as written.

Changes to the core library are developed in the llama.cpp repository, and the README asks contributors to open pull requests there.

Pros and cons of ggml

Pros

• Tiny dependency footprint; plain C and C++ that builds with GCC or Clang
• Runs the same GGUF model on laptops, phones, Macs, NVIDIA and AMD GPUs and in browsers
• Strong quantization support, from 2 bit integers to NVFP4
• MIT license, so commercial use is allowed
• Clear version rules since v1.0.0

Cons

• A low level C API; building a model by hand takes real systems knowledge
• Operation coverage differs between backends
• Training tools are basic compared with PyTorch or JAX
• Development moves fast, with 10 to 20 llama.cpp commits a day mentioned in the versioning discussion

Who should care about ggml

• Local model users: mostly no. Update llama.cpp, Ollama or LM Studio as usual and keep using GGUF files.
• App developers embedding a model: yes. Pin libllama and libggml to major version 1 and read minor release notes.
• Linux and Homebrew packagers: yes. Release tags now guarantee a matching system ggml.
• Researchers comparing frameworks: ggml is an inference runtime, not a PyTorch replacement. For that question, read PyTorch vs JAX.
• People quantizing models: the quant types live in ggml, so new formats arrive there first. Our guide on how to quantize LLM models shows the workflow.

FAQ

What does ggml stand for?

The official README and the ggml.ai site never expand the name. They write it in lowercase and describe it simply as a tensor library for machine learning. The capitalized "GGML" you see in older posts usually refers to the retired file format that GGUF replaced.

Is ggml the same as GGUF?

No. ggml is the C library that runs the math. GGUF is the file format that stores a model's weights and metadata so ggml based programs such as llama.cpp can load it in one file. The GGUF specification is published inside the ggml repository.

Is ggml free for commercial use?

Yes. ggml is released under the MIT license, which allows commercial use, modification and redistribution as long as the copyright notice is kept. Check the license of the model you run separately, because model weights have their own terms.

Do I need to install ggml to use llama.cpp?

Not usually. llama.cpp ships with its own copy of ggml, and the prebuilt downloads, Winget and the llama.app installer bundle it. On Homebrew and Debian, ggml is a separate shared library that the llama.cpp package pulls in for you as a dependency.

What is the difference between ggml v1.0.0 and llama.cpp v1.0.0?

They are two libraries released together on November 13, 2026. ggml v1.0.0 is the tensor engine (libggml), and llama.cpp v1.0.0 is the LLM inference layer (libllama) built on it. Both now follow the same semantic versioning rules, and llama.cpp release tags are only cut when its bundled ggml matches a released one.

Does Ollama use ggml?

Hugging Face's 2024 introduction to ggml listed Ollama, Jan, LM Studio and GPT4All among the projects using ggml under the hood. Each app picks its own ggml version and backends, so features land at different times than in llama.cpp.