How to Run Nanbeige4.3 33B Locally

To run Nanbeige4.3 33B locally, download a 4 bit GGUF (the Q4_K_M file is 20.79 GB) and load it in llama.cpp, Ollama or LM Studio on a 24 GB GPU, or use a 4 bit MLX build on a Mac with 32 GB or more of memory. The model is a plain dense Llama style network, so stock runtimes load it without patches. The catch is memory: all 33 billion weights are active for every token, and its KV cache is large, so long contexts need far more room than the file size suggests.

Nanbeige4.3 33B is the new flagship from the Nanbeige LLM Lab, published on Hugging Face on October 24, 2026 under the Apache 2.0 license. The lab calls it a "teacher" model for agent work: it is meant to be strong at coding agents and office style tool use, and to be distilled later into smaller models (the card names a coming Nanbeige4.3 3B). On November 2, 2026 it was on the Hugging Face trending list with close to 1,000 likes, and the most downloaded community GGUF repository had passed 100,000 downloads in about a week.

What Nanbeige4.3 33B is

The model's config.json describes a standard LlamaForCausalLM: 52 layers, a hidden size of 7,168, 56 attention heads with 8 key and value heads (grouped query attention), a head size of 128 and a vocabulary of 166,912 tokens. Hugging Face counts 33,118,977,024 parameters stored in BF16 across 14 safetensors files, about 66 GB in total. There are no experts and no custom Python files in the repository, which is why llama.cpp, MLX and vLLM can treat it like any Llama model.

Other facts from the model card:

• Context: up to 524,288 tokens (512K).
• Thinking by default: the chat template opens a <think> block. Set enable_thinking to false for direct answers.
• Two template switches for agents: preserve_thinking keeps reasoning from earlier turns (recommended for multistep tool use and code agents, off for plain chat), and tool calls default to an XML format, with JSON also supported.
• Languages: English and Chinese are listed.
• Text only: the template tells the model to refuse image or audio input.

Nanbeige publishes its own comparison against Qwen3.8-27B, Gemma4-31B and Muse-Glimmer-30B, run at the highest thinking setting for each. In that table it scores 80.9 on Terminal-Bench 2.1 against 73.0 for Qwen3.8-27B, while Qwen3.8-27B edges it on SWE-Bench Pro (61.7 against 60.3) and Gemma4-31B leads on GPQA Diamond. These are the lab's own numbers and we have not reproduced them.

Which way to run it

If you are choosing between the first three in general, our Ollama vs llama.cpp vs LM Studio comparison covers the tradeoffs.

How much memory Nanbeige4.3 33B needs

The weights are the first cost. These file sizes come from the bartowski GGUF repository (quantized with llama.cpp b11845) and from Nanbeige's own quantized repositories, checked on November 2, 2026:

The official INT4 build is worth knowing about. Nanbeige says it was made with quantization aware training, including distillation from the BF16 model and reinforcement learning on the INT4 model itself, so it should hold up better than a plain post training 4 bit file. It targets vLLM and SGLang, not llama.cpp.

The KV cache is the hidden cost

Because this is a dense model with 52 layers that all keep a cache, long context gets expensive fast. From the config (8 key and value heads, head size 128, 52 layers), the cache at 16 bit precision works out to about 208 KiB per token. That is our arithmetic, not a vendor figure:

Compare that with Qwen 3.8 27B, which needs only about 2 GiB for 32K tokens because most of its layers use linear attention. On a 24 GB card, Nanbeige with the Q4_K_M file leaves room for roughly 8K to 16K tokens of context; switch to IQ4_XS or a quantized cache for more. Our guides on KV cache quantization and how much VRAM you need for an LLM explain both levers.

Which file for which machine

• 24 GB GPU (RTX 3090, RTX 4090): Q4_K_M with 8K to 16K context, or IQ4_XS with a q8_0 cache for 32K.
• 16 GB GPU: IQ3_XXS with a short context, or Q4_K_M with some layers in system RAM. A dense 33B model slows down a lot once many layers sit in RAM; our guide on using RAM instead of VRAM explains why.
• 32 GB GPU (RTX 5090): Q5_K_M, or Q4_K_M with long context.
• Mac with 32 GB: MLX 4 bit with modest context. Mac with 64 GB or more: MLX 4 bit with long context, or Q8_0 GGUF.
• 48 GB or more of GPU memory: the official FP8 build in vLLM.

If the quant names are new, our GGUF quantization guide decodes them.

Run Nanbeige4.3 33B with llama.cpp

Recent llama.cpp builds can download a GGUF straight from Hugging Face. Start a server with the recommended chat settings:

llama-server -hf bartowski/Nanbeige_Nanbeige4.3-33B-GGUF:Q4_K_M \
  --jinja -c 16384 -ngl 99 \
  --temp 0.6 --top-p 0.95

--jinja makes llama.cpp use the chat template stored in the file, which matters here because the template handles thinking and tool calls. The server speaks the OpenAI API on port 8080, so editors and agent tools can point at it. To save memory at longer contexts, add -ctk q8_0 -ctv q8_0. For direct answers without reasoning, pass --chat-template-kwargs '{"enable_thinking": false}'.

On a smaller GPU, lower -ngl (the number of layers on the GPU) until the model loads, and accept the speed drop.

Run it with Ollama

There is no official Nanbeige model in the Ollama library, only community uploads. The cleaner route is to pull the GGUF from Hugging Face, which Ollama supports directly:

ollama run hf.co/bartowski/Nanbeige_Nanbeige4.3-33B-GGUF:Q4_K_M

Swap the tag for IQ4_XS or Q5_K_M to pick another size. Ollama's default context is short, so raise it for agent work (for example with /set parameter num_ctx 16384 in the chat, or OLLAMA_CONTEXT_LENGTH for the server) and keep the KV cache table above in mind. If replies never close the thinking block or tool calls come back as plain text, check the template Ollama picked with ollama show --modelfile and compare it with the prompt format on the GGUF card.

Run it in LM Studio

Search for Nanbeige4.3 in LM Studio's model browser and pick a GGUF build on Windows or Linux, or an MLX build on a Mac. Set the context length before loading, and use temperature 0.6 and top p 0.95 for chat. LM Studio can then serve the model on its local OpenAI compatible endpoint, which works with coding tools in the same way as our local LLM in VS Code setup.

Run it on a Mac with MLX

Two community MLX 4 bit conversions were available on November 2, 2026, each about 18.6 GB. One of them, pratikgx/Nanbeige4.3-33B-MLX-4bit, runs with mlx-lm:

pip install mlx-lm
mlx_lm.chat --model pratikgx/Nanbeige4.3-33B-MLX-4bit

The uploader published measurements from a 64 GB Mac mini with an M5 Pro: about 14 tokens per second of generation at a 512 token prompt, falling to about 10 tokens per second at 32K, with peak memory of about 19.3 GB at short prompts and 26.4 GB at 32K. A 128K prompt peaked at about 47.8 GB and took about 16 minutes end to end. These are one person's figures on one machine. For the MLX versus GGUF choice on a Mac, see MLX vs GGUF on Mac.

Run it with vLLM or SGLang

For a shared server, Nanbeige's card gives vLLM and SGLang commands with the Qwen3 reasoning parser and the qwen3_coder tool parser. Its examples split the model across four GPUs. On a single 48 GB or 80 GB card, the FP8 build is the practical choice:

vllm serve Nanbeige/Nanbeige4.3-33B-FP8 \
  --max-model-len 65536 \
  --gpu-memory-utilization 0.9 \
  --enable-auto-tool-choice \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

The 65,536 token limit is our suggestion to leave room for the cache on one card, not a vendor setting. For the difference between these engines and desktop runners, see Ollama vs vLLM.

Settings that matter

Nanbeige recommends two profiles: temperature 1.0 and top p 0.95 for agent and tool use tasks, and temperature 0.6 and top p 0.95 for reasoning and chat. Its suggested output limits are very long (65,536 tokens for agents, 262,144 for reasoning), because the model can think at length. On a home machine, cap output to something you can wait for and leave thinking on only when the task needs it.

For multistep agent sessions, turn on preserve_thinking so earlier reasoning stays in the conversation, as the card advises. That grows the context quickly, which brings the KV cache back into play.

Pros and cons

Pros

• Apache 2.0 license, so commercial use is allowed.
• Standard Llama architecture: works in stock llama.cpp, Ollama, LM Studio, MLX and vLLM with no patches.
• Strong agent and coding results in the lab's own tests, with XML tool calls built into the template.
• An official INT4 build trained for 4 bit use.

Cons

• Dense 33B: every token touches all weights, so it is slower than mixture of experts models with similar file sizes.
• Large KV cache per token, so long context needs a lot of extra memory.
• Text only, and the benchmark claims are not yet independently confirmed.
• Thinking is on by default and can use many tokens before answering.

Who it is for

Pick Nanbeige4.3 33B if you have a 24 GB GPU or a 32 GB Mac and want a capable local model for coding agents and tool use under a permissive license. If you need long context on the same hardware, image input, or faster replies, Qwen 3.8 27B is the easier fit. If you plan to adapt it, our guide to fine tuning a thinking or instruct model covers keeping the reasoning format intact.

FAQ

Is Nanbeige4.3 33B free for commercial use?

Yes. The model and its official quantized builds are released under the Apache 2.0 license on Hugging Face. Community conversions say the original license applies, but check each repository you download from.

Can I run Nanbeige4.3 in Ollama?

Yes, by pulling a GGUF from Hugging Face with ollama run hf.co/bartowski/Nanbeige_Nanbeige4.3-33B-GGUF:Q4_K_M. There is no official Ollama library entry yet, only community uploads.

Can Nanbeige4.3 33B run on 16 GB of VRAM?

Only with compromises. The IQ3_XXS file (14.49 GB) fits with a short context, or you can run Q4_K_M with part of the model in system RAM, which is much slower for a dense model of this size.

Is Nanbeige4.3 33B a mixture of experts model?

No. Its config is a standard dense Llama model with about 33.1 billion parameters, and all of them are used for every token.

How do I turn off thinking in Nanbeige4.3?

Set enable_thinking to false in the chat template. In Transformers, pass it to apply_chat_template; in llama.cpp, use --chat-template-kwargs '{"enable_thinking": false}'.

Is there a smaller Nanbeige4.3 model?

Not yet. The model card says the 33B model will be used to distill smaller models and names a coming Nanbeige4.3 3B. Earlier small models from the lab, such as Nanbeige4.2 3B, are already on Hugging Face.