llama.cpp

github.com · ML Frameworks

Open source

Plain C/C++ LLM inference engine that popularised GGUF quantised models running on everyday hardware.

#inference#c++#quantisation#gguf

Visit llama.cpp ↗GitHub ↗

About the ML Frameworks category

The numerical foundations: tensor libraries, autodiff engines, compilers and inference runtimes that every model above them depends on.

llama.cpp is one of 14 ml frameworks tools indexed on Axiomi. Facts on this page come from the tool's official site and public APIs; prices and features change, so confirm details on the official website before you commit.

// related

Related tools

All ML Frameworks →

Ollamaollama.comOpen-source local model runner with a one-line pull-and-run CLI, a desktop app and a local HTTP API. Uses llama.cpp-based and MLX engines; optional paid cloud models.ML FrameworksFreemium★ 182.3k

Transformershuggingface.coHugging Face's library defining thousands of model architectures, the de facto reference implementation for open models.ML FrameworksOpen source★ 167k

vLLMvllm.aiHigh-throughput open-source inference and serving engine built around PagedAttention and continuous batching, with an OpenAI-compatible server that scales from one GPU to clusters.ML FrameworksOpen source★ 93.2k

SGLangsglang.ioOpen-source serving framework for language and multimodal models, known for RadixAttention prefix caching, speculative decoding and large-scale multi-GPU deployment.ML FrameworksOpen source★ 36.8k

LM Studiolmstudio.aiDesktop app for macOS, Windows and Linux to find, download and chat with local models, running GGUF on llama.cpp and MLX on Apple Silicon, with a local API server.ML FrameworksFreemiumHN 4

PyTorchpytorch.orgThe dominant deep-learning framework in research, with eager execution, torch.compile and a huge ecosystem.ML FrameworksOpen source★ 103.8k