github.com · ML Frameworks
MIT licensed C and C++ tensor library that powers llama.cpp and whisper.cpp, with CPU, GPU, NPU and WebGPU backends and 2 to 8 bit quantization.
ggml is one of 25 ML framework tools on Axiomi. See all ML framework tools or how we check facts.
// guides
What ggml is, how it relates to GGUF and llama.cpp, what changed in ggml v1.0.0 on Nov 13, 2026, how semantic versioning works now and how to install it.
// related
llama.cppgithub.comPlain C/C++ LLM inference engine that popularised GGUF quantised models running on everyday hardware.ML FrameworksOpen source★ 135kStratagithub.comMIT licensed local inference app and terminal that runs very large mixture of experts models on one consumer GPU by keeping busy experts in RAM and streaming the rest from SSD.ML FrameworksOpen source★ 44.1kMACaQgithub.comHuawei CSL's open source 4 bit quantizer that calibrates each linear layer on its own, with backends for llama.cpp GGUF, vLLM, Diffusers and PyTorch speech models.ML FrameworksOpen source★ 237
Ollamaollama.comOpen source local model runner with a one line pull and run CLI, a desktop app and a local HTTP API. Uses llama.cpp-based and MLX engines; optional paid cloud models.ML FrameworksFreemium★ 187k
vLLMvllm.aiHigh throughput open source inference and serving engine built around PagedAttention and continuous batching, with an OpenAI compatible server that scales from one GPU to clusters.ML FrameworksOpen source★ 96.9k
ComfyUIcomfy.orgNode based open source app for running image, video and audio generation models locally, with shareable JSON workflows and support for quantized checkpoints.ML FrameworksOpen source★ 146.8k