How to Run LFM2.5 24B Locally

To run LFM2.5-24B-A2B locally, use a recent llama.cpp or Ollama build that understands the lfm2moe architecture, then load Liquid AI's official GGUF. The practical starter is Q4_K_M at about 14.5 GB of weights — plan for roughly 16–24 GB of RAM or VRAM once you add KV cache. On Apple Silicon, use the official MLX 4-bit build with mlx-lm 0.32 or newer.

This guide covers what the model is, how much memory each quant needs, working Ollama and llama.cpp commands, the Ollama version gotcha, sampling defaults, and when a smaller LFM2.5 or a Qwen/Gemma MoE is a better fit.

What LFM2.5-24B-A2B is

LFM2.5-24B-A2B is Liquid AI's largest LFM2.5 mixture-of-experts model, released 13 October 2026. Per the official model card and Liquid docs, it has 24B total parameters with about 2.45B active per token (64 experts, top-4 routing), a 131K context window, and a 128K vocabulary. It is a pure reasoning model: every assistant turn opens with a thinking block before the final answer.

Liquid positions it for on-device agents that call tools. Vendor-reported scores on the card put it ahead of Qwen 3.6 35B-A3B and Gemma 4 26B-A4B on several agent/tool benches (for example LiveClaw 48.09 vs 38.19 / 41.25). Treat those as vendor numbers. Liquid itself says the model is weaker on knowledge-heavy tasks alone and recommends pairing it with retrieval or search.

For how open-weight licences differ from OSI open source, see open weight vs open source.

How much VRAM (or unified memory) you need

MoE active-parameter counts help speed, not footprint. All 24B weights still sit in memory. Official GGUF file sizes from LiquidAI/LFM2.5-24B-A2B-GGUF:

Add KV-cache and runtime overhead on top of the file size — see how much VRAM you need for an LLM and KV cache quantization. Liquid's edge bench used Q4_K_M with a 4K input context; longer agent traces need more headroom. If GGUF names are unfamiliar, read GGUF quantization explained.

Pick by machine:

• 16 GB GPU or 16–24 GB Mac: Q4_K_M or QAD-Q4_0; keep context modest at first.
• 24 GB GPU (3090/4090) or 32 GB Mac: Q4_K_M with room for longer context, or Q5_K_M if you want less quant loss.
• 32 GB+ unified / VRAM: Q6_K or Q8_0, or the MLX 4/5/6/8-bit builds on Apple Silicon.

Run LFM2.5-24B with llama.cpp

Liquid documents a one-liner that pulls from Hugging Face. Use a current llama.cpp build:

llama-cli -hf LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M \
  --temp 0.1 --top-k 50 --repeat-penalty 1.05

Or pin the QAD Q4_0 file explicitly:

llama-cli -hf LiquidAI/LFM2.5-24B-A2B-GGUF \
  --hf-file LFM2.5-24B-A2B-QAD-Q4_0.gguf \
  --temp 0.1 --top-k 50 --repeat-penalty 1.05

The GGUF stores recommended sampling as general.sampling.* metadata, so llama.cpp applies it by default. Frontends that override sampling can make output look rambly — reset to the values above if that happens. For a local OpenAI-compatible server:

llama-server -hf LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M

Run LFM2.5-24B with Ollama

Ollama can load the Hugging Face GGUF directly. Liquid's Ollama deployment docs warn that Ollama v0.17.0 (the stable download when they wrote the note) fails on lfm2moe with missing tensor 'output_norm.weight'. You need v0.17.1-rc0 or later for any LFM MoE (including the older LFM2-24B-A2B and LFM2-8B-A1B).

# After upgrading Ollama past 0.17.0
ollama run hf.co/LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M

For a custom Modelfile, Liquid documents temperature 0.1, top_k 50, repeat_penalty 1.05 and ChatML-style stop tokens. Serve on http://localhost:11434 and point any OpenAI-compatible client at /v1. LM Studio can also load the GGUF once its bundled llama.cpp is new enough for lfm2moe. Runner trade-offs are covered in Ollama vs llama.cpp vs LM Studio.

Apple Silicon with MLX

Install a current mlx-lm — Liquid requires mlx-lm>=0.32.0 because earlier versions route experts incorrectly:

pip install -U "mlx-lm>=0.32.0"

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler, make_logits_processors

model, tokenizer = load("LiquidAI/LFM2.5-24B-A2B-MLX-4bit")
prompt = "What is C. elegans?"
if tokenizer.chat_template is not None:
    prompt = tokenizer.apply_chat_template(
        [{"role": "user", "content": prompt}],
        add_generation_prompt=True,
    )
sampler = make_sampler(temp=0.1, top_k=50)
logits_processors = make_logits_processors(repetition_penalty=1.05)
print(generate(
    model, tokenizer, prompt=prompt, max_tokens=4096,
    sampler=sampler, logits_processors=logits_processors, verbose=True,
))

5-, 6-, and 8-bit MLX builds are also on Hugging Face. For format choice on Mac, see MLX vs GGUF on Mac. Liquid's edge bench reports about 184 tok/s decode on an Apple M5 Max at Q4_K_M (vendor-measured on their Pipette setup with a 4K input).

Transformers, vLLM and SGLang

For full-precision or fine-tuning work, load the native checkpoint with Transformers 5.0+ from LiquidAI/LFM2.5-24B-A2B (bf16; needs far more memory than Q4). For multi-user GPU serving, Liquid documents vLLM and SGLang; there is also an FP8 checkpoint (LFM2.5-24B-A2B-FP8) aimed at those stacks. Compare serving engines in vLLM vs llama.cpp.

Liquid ships Unsloth and TRL notebooks for CPT, SFT, DPO and GRPO on the base model — see fine-tune an LLM locally for the general workflow.

Settings that matter

1. Leave budget for thinking. Every reply includes a reasoning trace before the answer. Short max_tokens budgets cut off mid-thought.
2. Keep Liquid's sampling. Temperature 0.1, top_k 50, repetition_penalty 1.05 unless you have a reason to change them.
3. Raise context only as needed. Ollama and many GUIs default to a small window; agent trajectories and long docs need more — and more KV memory.
4. Pair with tools for facts. Liquid recommends retrieval or web search for knowledge-heavy work; the model's strength is tool use and instruction following on their reported benches.

Pros and cons

Pros

• Strong on-device MoE speed story (vendor: up to 1.7× vs comparable MoEs; 184 tok/s on M5 Max at Q4_K_M).
• Official GGUF, MLX, vLLM, SGLang and ONNX paths on day one.
• 131K context and always-on reasoning aimed at local agents.
• Smaller LFM2.5 siblings (230M–8B) share the same stack if 24B is too heavy.

Cons

• Q4 still needs ~14.5 GB of weights — not a phone model.
• LFM Open License v1.0 is not Apache-2.0; commercial use stops at the $10M revenue threshold without a separate deal.
• Ollama stable 0.17.0 breaks LFM MoE until you upgrade past that release.
• Knowledge-heavy tasks need RAG; vendor agent wins are not independent replications.

Who it's for

Choose LFM2.5-24B-A2B if you have about 16–32 GB of memory, want a local tool-calling / agent model that thinks before it answers, and can run a current llama.cpp, Ollama (post-0.17.0), or mlx-lm. Prefer Qwen 3.6 or Gemma 4 when you want Apache-2.0 and a broader general-chat footprint, or a tiny LFM2.5 instruct build when you only have 8 GB. For a multi-model shortlist by VRAM, see open source LLM models to run locally.

FAQ

How do I run LFM2.5-24B-A2B in Ollama?

Upgrade Ollama to v0.17.1-rc0 or later, then run ollama run hf.co/LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M. Older 0.17.0 builds fail with a missing output_norm.weight tensor on lfm2moe.

How much VRAM does LFM2.5 24B need?

The Q4_K_M GGUF is 14.5 GB on disk; QAD-Q4_0 is 13.6 GB. Plan extra for KV cache and runtime. A 16 GB card is tight; 24 GB is comfortable for short-to-medium context.

Is LFM2.5-24B-A2B open source?

Weights are downloadable under the LFM Open License v1.0. Personal and small-company commercial use is allowed under that licence; entities at or above $10M annual revenue need a different arrangement for commercial use. That is not the same as Apache-2.0 — see open weight vs open source.

Why does every answer include a long "thinking" section?

Unlike LFM2-24B-A2B, LFM2.5-24B-A2B is a reasoning-only model. The chat template opens each assistant turn with a thinking block. Leave enough output tokens for both the trace and the final answer.

LFM2.5-24B vs Qwen 3.6-35B-A3B for local use?

Both are low-active MoEs aimed at strong local machines. Liquid's own table claims wins on several agent/tool benches and losses on some knowledge and coding benches versus Qwen 3.6-35B-A3B. Pick LFM when you want Liquid's agent post-training and decode-speed claims; pick Qwen when you want Apache-2.0 and the Qwen ecosystem — guide: run Qwen 3.6 locally.

Can I fine-tune LFM2.5-24B locally?

Yes. Liquid documents Unsloth and TRL paths (CPT, SFT, DPO, GRPO) starting from the base checkpoint. Expect much higher memory than Q4 inference; see fine-tune an LLM locally.