To run Qwen3.8 Flash Next locally, pick a GGUF or MLX build that matches your total memory, leave the large n gram embedding table memory mapped on SSD when you can, and use a current llama.cpp, Unsloth Desktop, Ollama MLX tag, Strata, vLLM or SGLang. Unsloth's requirements table starts at about 32 GB of combined RAM and VRAM (or unified memory) for its smallest 1 bit quant, and about 64 GB for a solid 4 bit file. ISTA DASLab GSQ builds can keep only the transformer shard resident (about 38 to 55 GB) while the 28.8 GB n gram shard stays on disk. This is a different model from the dense 27B covered in how to run Qwen 3.8 locally.
Qwen3.8-Flash-Next is the experimental open weight preview of the architecture that will underpin Qwen4, published on Hugging Face under the Qwen Community License. On November 4, 2026 Unsloth shipped Dynamic v3.1 GGUFs for it. Community discussion on Hacker News (including the Ripple 65B thread) and r/LocalLLaMA keeps comparing it to denser local coding models, and Ollama lists an MLX tag at 105 GB. We rechecked the official card, Unsloth docs, ISTA DASLab GGUF card and Ollama library page on November 6, 2026. Nothing here was run first hand.
What Qwen3.8 Flash Next is
Per the official model card and our Qwen3.8 Flash Next tool page, Flash Next is a multimodal causal language model with:
| Fact | Value |
|---|---|
| Total MoE parameters | 125B (6B activated per token) |
| N gram embedding | 51B (20M bigram and trigram entries) |
| MTP draft head | about 4B |
| Layers | 48 |
| Experts | 512 routed, 10 routed plus 1 shared active |
| Native context | 262,144 tokens (extensible toward 1M with YaRN) |
| Licence | Qwen Community License 1.0 |
The layout interleaves Gated DeltaNet (linear attention) blocks with Qwen Sparse Attention (QSA) that attends over micro blocks rather than every token. That hybrid design is why long context stays cheaper than a dense model of similar quality, and why the 51B n gram table can sit on SSD with only sparse reads. Qwen's card reports strong agentic coding and tool use scores relative to Qwen3.8-27B and several larger models; those are lab figures, not our measurements.
Production API users should know the difference: Qwen3.8-Flash on Qwen Cloud is the managed product based on this preview, with extra production features (the card mentions 1M context by default and built in tools). This guide is about the downloadable Flash Next weights.
How much memory you need
Unsloth total memory table
Unsloth's run guide publishes combined RAM plus VRAM (or unified memory) targets. The smallest quant is larger than a typical 1 bit MoE because the n gram (PLE) table is not quantized as aggressively as matrix weights:
| Precision | Total memory (Unsloth) | Example Unsloth GGUF size |
|---|---|---|
| 1 bit | 32 GB | UD-IQ1_S, 72.5 GB on disk |
| 2 bit | 42 GB | UD-Q2_K_XL, 78.9 GB |
| 3 bit | 50 GB | UD-Q3_K_XL, 90.0 GB |
| 4 bit | 64 GB | UD-Q4_K_XL, 111.3 GB |
| 5 bit | 80 GB | UD-Q5_K_XL, 158.3 GB |
| 8 bit | 128 GB | Q8_0, 188.2 GB |
| BF16 | 355 GB | Full precision |
Disk size and resident memory are not the same. Unsloth and AtomicChat both stress that you can leave the n gram table on SSD (mmap) so fast memory holds mainly the MoE weights. Add 1 to 2 GB if you enable MTP, and about 0.9 GB for the vision projector. For how quant labels work in general, see GGUF quantization explained.
ISTA DASLab GSQ builds (smaller resident footprint)
The ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF card splits every build into two shards. Shard 1 holds the transformer weights; shard 2 is the n gram table at IQ4_NL (28.8 GB) and can stay memory mapped:
| Build | Shard 1 (resident) | Shard 2 (disk OK) | Total download |
|---|---|---|---|
| Q2_0 | 37.6 GB | 28.8 GB | 66.4 GB |
| IQ2_XS | 39.2 GB | 28.8 GB | 68.0 GB |
| IQ3_XXS | 47.0 GB | 28.8 GB | 75.8 GB |
| IQ3_S | 54.8 GB | 28.8 GB | 83.6 GB |
ISTA recommends IQ3_S when you have the headroom, and Q2_0 when throughput matters more than a small quality trade. Their card reports Q2_0 at about 368 prompt tokens/s and 94 decode tokens/s on their llama.cpp test bed; treat those as publisher numbers. A separate coder variant keeps 256 of 512 experts at about 58.4 GB total.
Which machine fits which path
- 64 GB Mac or 64 GB workstation: Unsloth 4 bit with table on SSD, or ISTA IQ3_XXS / IQ3_S with mmap.
- 128 GB Mac, DGX Spark, Strix Halo class: Unsloth 5 bit or Q8, or keep the n gram table in RAM for fewer page faults.
- 24 GB GPU plus 64 to 96 GB system RAM: ISTA or AtomicChat style offload (experts and table partly in system RAM or on SSD). Strata was built for this class of MoE.
- Under about 48 GB total with no fast SSD offload plan: prefer the dense Qwen 3.8 27B instead.
Our how much VRAM an LLM needs guide covers the general weight plus KV method. Flash Next's KV is unusually light: only 12 of 48 layers keep a growing cache (the rest are Gated DeltaNet). From the config (2 KV heads, head size 256, 12 attention layers), an FP16 cache is about 24.6 KB per token, so 32K tokens is under 1 GB of KV before overhead. That is arithmetic from the config, not a vendor bench.
Run it with Unsloth Desktop or llama.cpp
Unsloth Desktop can search, download and serve the Dynamic v3.1 GGUFs, with thinking and preserve thinking toggles and optional MTP. From a terminal with a current llama.cpp:
pip install -U "huggingface_hub[cli]"
hf download unsloth/Qwen3.8-Flash-Next-GGUF \
--local-dir Qwen3.8-Flash-Next-GGUF \
--include "*UD-Q4_K_XL*"
./llama.cpp/llama-cli \
--model Qwen3.8-Flash-Next-GGUF/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf \
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0Unsloth also documents mixing n gram table precisions (BF16, Q8_0, Q5_1, IQ4_NL) with a small set_split.py stamp so you can trade disk for table quality. For API serving, unsloth run --model unsloth/Qwen3.8-Flash-Next-GGUF:UD-Q4_K_XL wraps llama-server.
MTP for faster decode
Flash Next ships a multi token prediction draft head. Unsloth reports about 1.3 to 1.7x faster inference with MTP in Unsloth Desktop, and about 170 tokens/s versus a 100 token/s baseline on one RTX 6000 PRO in their guide. Shared MTP modules save roughly 1 to 2 GB versus a full draft file. llama.cpp needs their MTP branch or a build that already includes that path; Unsloth Desktop enables MTP by default for this model.
Run ISTA GSQ builds (and Strata)
hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF \
--include "IQ3_S/*" --local-dir .
llama-cli -m IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00001-of-00002.gguf \
-lm mmap --lazy-mode on -ngl 99 \
-p "Explain sparse attention in two sentences."-lm mmap --lazy-mode on is what keeps the 28.8 GB n gram shard on disk. For images, download mmproj-Qwen3.8-Flash-Next-BF16.gguf and pass it to llama-mtmd-cli.
Strata is the MIT licensed runner that first popularized Flash Next on gaming PCs with SSD expert streaming. The DeepSeek V4.1 Flash guide notes that Strata's later releases reused the same approach; if you already use Strata for DeepSeek, its model picker is a one click path for Flash Next GGUFs too. See how to run DeepSeek V4.1 Flash locally for Strata install commands.
Ollama, LM Studio and server engines
Ollama's library lists qwen3.8-flash-next:125b-mlx at 105 GB with a 256K context window for text and image. On Apple silicon:
ollama run qwen3.8-flash-next:125b-mlxThat tag is an MLX build; confirm your Ollama version supports it before you pull 105 GB. LM Studio can load community GGUFs (Unsloth or ISTA) from Hugging Face search. For multi GPU serving of the official safetensors, Qwen points at vLLM, SGLang and TokenSpeed cookbooks. An independent Spark comparison of the NVFP4 checkpoint under both engines appears in our SGLang vs vLLM on DGX Spark guide.
If you are choosing a desktop runner in general, start with Ollama vs llama.cpp vs LM Studio.
Thinking mode and sampling
Flash Next thinks by default. Qwen recommends:
| Mode | temperature | top_p | top_k | min_p | presence_penalty |
|---|---|---|---|---|---|
| Thinking | 1.0 | 0.95 | 20 | 0.0 | 0.0 |
| Instruct (non thinking) | 0.7 | 0.80 | 20 | 0.0 | 1.5 |
Control depth with reasoning_effort (xhigh default, also medium and low). preserve_thinking keeps prior reasoning blocks in the chat history for agents; turn it off if you want only the latest turn's trace. In llama-server or Unsloth, pass chat template kwargs such as {"reasoning_effort":"medium"}. Qwen warns that lowering effort can increase retries on hard agent tasks, so total tokens may not fall.
Pros and cons
Pros
- Strong open weight MoE quality for coding agents and tool use relative to the dense 27B sibling, per Qwen's published tables.
- Only about 6B parameters active per token, so decode can stay snappy when the working set fits.
- N gram table and hybrid attention make SSD offload and long context practical on high memory desktops.
- Broad tooling: Unsloth, llama.cpp, Ollama MLX, Strata, vLLM and SGLang.
Cons
- Qwen Community License, not Apache 2.0; check commercial terms before shipping a product.
- Comfortable quants still want roughly 64 GB class machines; a 24 GB card alone is not enough without heavy RAM and SSD help.
- Disk downloads are large (66 GB and up for competitive GSQ files, over 100 GB for many Unsloth 4 bit trees).
- Preview architecture: runners and quant recipes still move quickly.
Who this is for
Choose Flash Next if you have about 64 GB or more of fast memory (or a 24 GB GPU plus plenty of system RAM and a fast NVMe), want a local multimodal coding agent, and accept the Qwen community licence. Choose the dense Qwen 3.8 27B if you need Apache 2.0 and a 16 to 24 GB envelope. For a wider shortlist, see open source LLM models to run locally.
FAQ
How much VRAM does Qwen3.8 Flash Next need?
Plan for total memory, not VRAM alone. Unsloth lists about 32 GB combined for the smallest 1 bit quant and about 64 GB for 4 bit. With ISTA GSQ files you can keep about 38 to 55 GB resident for shard 1 and leave the 28.8 GB n gram shard on SSD. A lone 24 GB card needs system RAM offload or Strata style streaming.
Can I run it on a 64 GB Mac?
Yes, that is a common target. Use Unsloth's 3 bit or 4 bit GGUF with the n gram table on SSD, or Ollama's 105 GB MLX tag if you have enough free unified memory after macOS reserves its share. Raise the GPU memory limit on macOS if the app cannot see the full pool.
Is Qwen3.8 Flash Next the same as Qwen 3.8 27B?
No. The 27B is a dense Apache 2.0 model aimed at 16 to 24 GB machines. Flash Next is a 125B MoE preview (6B active) plus a 51B n gram table under the Qwen Community License. See how to run Qwen 3.8 locally for the 27B path.
Does Ollama support Qwen3.8 Flash Next?
Ollama lists qwen3.8-flash-next:125b-mlx (105 GB, 256K context, text and image). Community GGUFs also load through Hugging Face style pulls in some setups. Confirm your Ollama build recognizes the tag before downloading.
Should I use Unsloth or ISTA GSQ quants?
Unsloth Dynamic v3.1 builds are the default path for Unsloth Desktop and their documented llama.cpp flow, with optional MTP and mixable n gram tables. ISTA GSQ RCO builds optimize per tensor bit width and publish a clear resident versus disk split; pick IQ3_S for quality or Q2_0 for speed. Both inherit the base Qwen licence.
Is Flash Next free for commercial use?
The weights use the Qwen Community License 1.0, not Apache 2.0. Broad research and many product uses are allowed, but the licence sets conditions for commercial deployment. Read the LICENSE file on the Hugging Face repo before you ship. For licence vocabulary, see open weight vs open source.
