github.com · ML Frameworks
Huawei CSL's open source 4 bit quantizer that calibrates each linear layer on its own, with backends for llama.cpp GGUF, vLLM, Diffusers and PyTorch speech models.
MACaQ is one of 24 ML framework tools on Axiomi. See all ML framework tools or how we check facts.
// guides
What MACaQ quantization is, how Huawei's new 4 bit method works across text, image, video and audio models, what MACaQ-R adds for GGUF, and who should try it.
// related
Ollamaollama.comOpen source local model runner with a one line pull and run CLI, a desktop app and a local HTTP API. Uses llama.cpp-based and MLX engines; optional paid cloud models.ML FrameworksFreemium★ 186.9kllama.cppgithub.comPlain C/C++ LLM inference engine that popularised GGUF quantised models running on everyday hardware.ML FrameworksOpen source★ 134.8kStratagithub.comMIT licensed local inference app and terminal that runs very large mixture of experts models on one consumer GPU by keeping busy experts in RAM and streaming the rest from SSD.ML FrameworksOpen source★ 43.7k
LM Studiolmstudio.aiDesktop app for macOS, Windows and Linux to find, download and chat with local models, running GGUF on llama.cpp and MLX on Apple Silicon, with a local API server.ML FrameworksFreemiumHN 3
vLLMvllm.aiHigh throughput open source inference and serving engine built around PagedAttention and continuous batching, with an OpenAI compatible server that scales from one GPU to clusters.ML FrameworksOpen source★ 96.9k
PyTorchpytorch.orgThe dominant deep learning framework in research, with eager execution, torch.compile and a huge ecosystem.ML FrameworksOpen source★ 105.5k