vLLM

vllm.ai · ML Frameworks

Open source

#8 in ML Frameworks

High-throughput open-source inference and serving engine built around PagedAttention and continuous batching, with an OpenAI-compatible server that scales from one GPU to clusters.

#inference#serving#gpu#pagedattention

Visit vLLM ↗GitHub ↗

About the ML Frameworks category

The numerical foundations: tensor libraries, autodiff engines, compilers and inference runtimes that every model above them depends on.

vLLM is one of 14 ml frameworks tools indexed on Axiomi. Facts on this page come from the tool's official site and public APIs; prices and features change, so confirm details on the official website before you commit.

// related

Related tools

All ML Frameworks →

SGLangsglang.ioOpen-source serving framework for language and multimodal models, known for RadixAttention prefix caching, speculative decoding and large-scale multi-GPU deployment.ML FrameworksOpen source★ 36.8k

Transformershuggingface.coHugging Face's library defining thousands of model architectures, the de facto reference implementation for open models.ML FrameworksOpen source★ 167k

llama.cppgithub.comPlain C/C++ LLM inference engine that popularised GGUF quantised models running on everyday hardware.ML FrameworksOpen source★ 130.4k

Ollamaollama.comOpen-source local model runner with a one-line pull-and-run CLI, a desktop app and a local HTTP API. Uses llama.cpp-based and MLX engines; optional paid cloud models.ML FrameworksFreemium★ 182.3k

PyTorchpytorch.orgThe dominant deep-learning framework in research, with eager execution, torch.compile and a huge ecosystem.ML FrameworksOpen source★ 103.8k

JAXdocs.jax.devComposable transformations of NumPy-style programs: automatic differentiation, JIT compilation and parallelism on GPUs and TPUs.ML FrameworksOpen source★ 36.4k