To run Qwen 3.6 locally, install Ollama and type ollama run qwen3.6 for the default 35B-A3B build (~23–24 GB), or ollama run qwen3.6:27b for the dense 27B (~18–19 GB). The family is Apache 2.0 on the published Hugging Face cards, multimodal (text, image and video on the open weights), and also runs in llama.cpp, LM Studio and vLLM. Plan roughly 18 GB total memory for a 4-bit 27B and about 23 GB for a 4-bit 35B-A3B.
This guide covers which Qwen 3.6 size fits your machine, the setup commands that actually work, thinking and preserve-thinking controls, MTP speedups, and when to prefer 35B-A3B over the dense 27B — or over Qwen 3.8 27B.
What Qwen 3.6 is
Qwen 3.6 is Alibaba's open-weight multimodal Qwen release that followed the Qwen 3.5 series. According to the Qwen/Qwen3.6-35B-A3B model card, the flagship open MoE has 35B total parameters with about 3B activated per token (256 experts, 8 routed + 1 shared), a native 262,144-token context (extendable toward 1M with YaRN), and a vision encoder for images and video. A dense Qwen3.6-27B sibling is also published for lighter machines. Weights are Apache 2.0 — see open weight vs open source if you need the licence distinction.
Qwen markets the line for agentic coding (frontend workflows, repository-level edits) and a new thinking preservation option that keeps prior reasoning traces across turns. Vendor tables on the card report strong coding-agent scores (for example SWE-bench Verified 73.4 and Terminal-Bench 2.0 at 51.5 for 35B-A3B). Treat those as Qwen-reported numbers, not independent replications.
| Variant | Style | Active / total | Context | Modalities (card) |
|---|---|---|---|---|
| Qwen3.6-27B | Dense | ~27B | 256K native | Text, image, video |
| Qwen3.6-35B-A3B | MoE | ~3B / 35B | 262K native | Text, image, video |
How much VRAM Qwen 3.6 needs
Unsloth's Qwen 3.6 local guide publishes total memory floors (RAM plus VRAM, or unified memory on Apple Silicon). Use them as planning numbers; long context, vision projectors and MTP heads add more. For the general method, see how much VRAM you need for an LLM.
| Variant | 3-bit | 4-bit | 6-bit | 8-bit | BF16 |
|---|---|---|---|---|---|
| 27B | ~15 GB | ~18 GB | ~24 GB | ~30 GB | ~55 GB |
| 35B-A3B | ~17 GB | ~23 GB | ~30 GB | ~38 GB | ~70 GB |
Ollama's qwen3.6 library page lists download bands that line up: default / 35b tags about 23–24 GB, and 27b about 18–19 GB. If quant names like Q4_K_M or UD-Q4_K_XL are unfamiliar, read GGUF quantization explained.
Which size for which machine
- 16 GB GPU or tight 16–24 GB Mac: Prefer
qwen3.6:27bat 4-bit, or a more aggressive Unsloth dynamic quant of 35B-A3B with short context. Do not expect full 256K context on this tier. - 24 GB GPU (RTX 3090/4090) or 32 GB Mac: 35B-A3B at 4-bit (Ollama default or Unsloth UD-Q4_K_XL) is the usual sweet spot — MoE speed with the stronger agentic-coding profile on the vendor table.
- 32 GB+ VRAM / large Mac / multi-GPU: Raise context, enable MTP speculative decoding, or move to Q8 / MXFP8 / NVFP4 (Blackwell) when you want higher fidelity or server throughput.
- Already happy with Qwen 3.8 27B for pure text: Keep that dense stack for chat and coding; try 3.6 when you want multimodal inputs or the MoE coding-agent path. Compare both on your own prompts.
For a broader shortlist that also includes Gemma 4 and gpt-oss, see open source LLM models to run locally.
How to run Qwen 3.6 locally with Ollama
Ollama is the lowest-friction path. After installing Ollama:
ollama run qwen3.6 # default ~35B-A3B, ~23–24 GB
ollama run qwen3.6:35b-a3b
ollama run qwen3.6:27b # denser 27B, ~18–19 GB
ollama run qwen3.6:35b-mlx # Apple Silicon MLX tag when available
ollama run qwen3.6:27b-mlxThe library also ships coding-oriented and MTP tags (for example 35b-a3b-mtp-q4_K_M, 27b-coding). Two settings still catch people:
- Context length. Ollama's context-length docs default to only 4K tokens on GPUs under 24 GiB of VRAM. Raise it for coding agents (
OLLAMA_CONTEXT_LENGTH=64000 ollama serveis a pattern Ollama itself documents). Qwen recommends keeping at least ~128K when you rely on thinking for hard tasks. - Thinking mode. Qwen 3.6 thinks by default. Long thought traces burn tokens and latency — prefer thinking for hard reasoning and coding, and disable it for quick chat replies (see sampling section below).
Run Qwen 3.6 with llama.cpp
For full GGUF control, Unsloth publishes Dynamic quants such as unsloth/Qwen3.6-35B-A3B-GGUF and unsloth/Qwen3.6-27B-GGUF. With a recent llama.cpp build:
llama-cli -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_XL \
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00Swap the repo for the 27B GGUF when memory is tight. For vision, download the matching mmproj file and pass --mmproj to llama-cli or llama-server. To turn thinking off in llama-server, Unsloth documents --chat-template-kwargs '{"enable_thinking":false}' (escape the quotes differently in PowerShell).
You can also fetch files with the Hugging Face Hub CLI first if you want offline copies:
hf download unsloth/Qwen3.6-35B-A3B-GGUF \
--include "*UD-Q4_K_XL*" --include "*mmproj-F16*" \
--local-dir ./qwen36-35bFaster decode with MTP
Qwen 3.6 ships Multi-Token Prediction (MTP) heads. Unsloth's MTP GGUFs (for example unsloth/Qwen3.6-35B-A3B-MTP-GGUF) plug into recent llama.cpp with --spec-type draft-mtp --spec-draft-n-max 2. Expect roughly 1.15–1.4× generation speedup depending on dense vs MoE and hardware, with about 1 GB extra memory versus the matching non-MTP quant. For the broader idea behind speculative decoding, see speculative decoding vs MTP.
Run Qwen 3.6 on Mac (LM Studio / MLX)
On Apple Silicon, LM Studio and mlx-lm / MLX vision stacks often beat generic GGUF for decode speed. Ollama's *-mlx tags and Unsloth's MLX Dynamic quants both target this path. If you are deciding between engines, our MLX vs GGUF on Mac guide explains the tradeoffs. Practical rule: use MLX when the model is always on one Mac; use GGUF when you need the same files on Windows, Linux and Mac.
Sampling, thinking and multimodal tips
Qwen's card and Unsloth recommend these defaults:
Thinking mode (general tasks)
temperature=1.0,top_p=0.95,top_k=20,presence_penalty=0.0–1.5,repetition_penalty=1.0
Thinking mode (precise coding / WebDev)
- Drop temperature to 0.6 and keep the same top-p / top-k; set
presence_penalty=0.0for coding scaffolds.
Instruct (non-thinking)
temperature=0.7,top_p=0.8,top_k=20,presence_penalty=1.5
Preserve thinking. By default only the latest turn's thought block is kept. Setting preserve_thinking=true (via chat-template kwargs or your UI toggle) retains historical reasoning. That can improve multi-step agent consistency and sometimes reduce redundant tokens, but it also grows the KV cache — pair it with KV cache quantization if VRAM is tight.
Images and video. Put images before the text instruction. Video frame sampling is framework-specific (vLLM exposes mm_processor_kwargs with an fps control). For production OpenAI-compatible serving, Qwen documents vllm>=0.19.0 and sglang>=0.5.10 with --reasoning-parser qwen3.
35B-A3B vs 27B. Prefer 35B-A3B when you have ~24 GB and want the MoE coding-agent profile; prefer 27B when you need a denser model that fits closer to 18 GB at 4-bit. That choice is different from picking Gemma 4 26B-A4B vs 31B — here both sizes are "Qwen 3.6," not a mixed family.
Who Qwen 3.6 local is for
Good fit
- Developers who want a private multimodal coding assistant on a single 24 GB GPU or 32 GB Mac
- Teams that need Apache 2.0 weights for redistribution or fine-tuning
- People comparing open MoE models for agentic coding, tool calling and vision
Weaker fit
- Sub-16 GB machines that already struggle with mid-size dense models — start with smaller Qwen tags or a lighter Gemma 4 edge build instead
- Pure text chat where a tuned Qwen 3.8 27B is already working well
- Multi-tenant serving where vLLM with NVFP4 / MTP on Blackwell is the real goal — 3.6 works there, but start with the desktop path above first
Pros and cons
| Pros | Cons |
|---|---|
| Strong Ollama library tags plus mature GGUF / MLX / MTP ecosystem | 35B-A3B at 4-bit still wants ~23 GB+ total memory |
| MoE 35B-A3B activates only ~3B params per token | Thinking traces inflate latency if left on for every chat |
| Apache 2.0 on published cards; native multimodal | Full 256K+ context needs careful KV / context tuning |
| Preserve-thinking option for agent loops | Dense 27B and MoE 35B-A3B are different feel — pick one and stick with it for a week |
FAQ
How do I run Qwen 3.6 locally on 16 GB VRAM?
Use ollama run qwen3.6:27b at 4-bit, keep context short, and avoid BF16. A more aggressive Unsloth dynamic quant of 35B-A3B can fit with SSD offload, but expect slower decode. Unsloth's table lists ~18 GB for 4-bit 27B and ~23 GB for 4-bit 35B-A3B as total-memory floors.
What is Qwen 3.6 35B A3B?
It is the mixture-of-experts open-weight model: about 35B total parameters with roughly 3B active per token, 262K native context, and text plus image/video inputs per the official model card. The "A3B" label means about 3B active parameters.
Does Ollama support Qwen 3.6 vision?
Yes on the official qwen3.6 library tags (text and image). If a third-party GGUF with a separate mmproj fails to load in Ollama, update Ollama or run that file in llama.cpp with --mmproj instead.
Qwen 3.6 or Qwen 3.8 27B for local coding?
They solve different local jobs. Qwen 3.8 27B is a strong dense text model with its own how-to at run Qwen 3.8 locally. Qwen 3.6 adds a multimodal MoE 35B-A3B path aimed at agentic coding and thinking preservation. Benchmark both on your own repository tasks before committing disk space.
Is Qwen 3.6 open source or only open weight?
The published Qwen 3.6 Hugging Face cards list Apache 2.0, which is a true open-source licence for the weights and accompanying materials as released. Always read the exact card for the tag you download.
