Can a Local LLM Use RAM Instead of VRAM?

Yes, a local LLM can use RAM instead of VRAM. llama.cpp, Ollama, LM Studio, vLLM and Hugging Face Transformers can all keep part or all of a model in system memory, so a model larger than your graphics card still loads and runs. The catch is speed. Generating each token means reading the model's active weights from memory, and ordinary desktop RAM is several times slower than the memory on a graphics card, so every layer you move off the GPU slows output down. Mixture of experts models suffer least, because only a small slice of their weights is read for each token.

This guide explains what happens when a model spills into RAM, how much slower to expect, and the exact settings in each runner. If you still need to work out how big your model is in the first place, start with how much VRAM you need for an LLM.

Can a local LLM use RAM instead of VRAM at full speed?

When a model writes text, it produces one token at a time. For each token, the runner reads every active weight in the model once. That makes token generation limited mainly by memory bandwidth, the number of gigabytes per second your memory can deliver, rather than by raw compute.

You can turn that into a rough ceiling with simple arithmetic: divide memory bandwidth by the size of the weights read per token. This is a theoretical upper bound, not a benchmark. Real speeds come in lower once the KV cache, software overhead and prompt processing are counted.

The Apple figures come from Apple's MacBook Pro tech specs. The DDR5 figure is the standard calculation for two 64 bit channels. Dedicated graphics cards sit well above desktop RAM, which is why a model that fits entirely in VRAM feels so much faster than one that does not.

There is a second cost. When the GPU computes some layers and the CPU computes others, the CPU does its share at CPU speed, and data has to cross the PCIe link between them. vLLM's documentation puts it plainly: its CPU offload needs a fast CPU to GPU interconnect because part of the model is copied to the GPU on every forward pass.

The three ways a model ends up in RAM

"Using RAM instead of VRAM" covers three different situations, and they behave very differently.

1. Partial offload. Some layers live in VRAM and the rest in system RAM. The GPU handles its layers, the CPU handles the others. This is the normal case when a model is a little too big for your card.
2. CPU only. The whole model sits in RAM and the CPU does all the work. It works on any machine with enough memory, just slowly.
3. Unified memory. On Apple Silicon there is no separate VRAM. The CPU and GPU share one pool, so "RAM" and "VRAM" are the same chips, and the GPU still does the work at full unified memory bandwidth.

A fourth situation is the one to avoid: the model is larger than your RAM too. llama.cpp loads models with memory mapping by default, so the operating system pages weights in from disk as needed. The model may still start, but generation slows to a crawl because the SSD becomes the bottleneck.

How to offload to RAM in llama.cpp

The llama.cpp README lists "CPU+GPU hybrid inference to partially accelerate models larger than the total VRAM capacity" as a core feature. The llama-server documentation gives you these controls:

For a dense model, lower -ngl until the model loads, then raise it one step at a time until you run out of memory. With the automatic defaults, recent builds pick a split for you, so you often only need to set a context size:

llama-server -m model-Q4_K_M.gguf -c 16384

For a mixture of experts model, offload experts rather than whole layers. Keeping attention and the shared parts on the GPU while expert weights sit in RAM is usually faster than splitting by layer, because only the experts chosen for each token are read:

llama-server -m gpt-oss-120b.gguf -ngl all --n-cpu-moe 24 -c 16384

The 24 here is only a starting point. Lower --n-cpu-moe until VRAM is nearly full. If you have plenty of RAM and want the model to stay resident instead of being paged out, use --load-mode mmap+mlock.

How to offload to RAM in Ollama

Ollama does the split for you. When a model does not fit, it puts as many layers as it can on the GPU and the rest in system memory. The Ollama FAQ explains how to check what happened with ollama ps:

NAME          ID            SIZE    PROCESSOR          UNTIL
llama3:70b    bcfb190ca3a7  42 GB   48%/52% CPU/GPU    4 minutes from now

100% GPU means everything is in VRAM, 100% CPU means everything is in system memory, and a split like 48%/52% CPU/GPU means the model is shared between them. If you see a split you did not expect, the usual cause is the context window. Ollama allocates the KV cache up front, and running parallel requests multiplies the context size, so a model that fits at 4k tokens may spill into RAM at 32k. Enabling a quantized KV cache (OLLAMA_KV_CACHE_TYPE=q8_0, which needs flash attention) roughly halves that cache; our KV cache quantization guide covers the tradeoff.

How to offload to RAM in LM Studio

LM Studio exposes GPU offload as a setting when you load a model, and the same control exists on the command line. According to the lms load documentation, --gpu takes a value from 0 to 1, off or max, and --estimate-only prints a memory estimate and exits without loading the model:

lms load qwen/qwen3-8b --estimate-only
lms load qwen/qwen3-8b --gpu 0.6 --context-length 8192

A value of 0.6 puts about 60% of the model on the GPU. LM Studio's system requirements recommend at least 16 GB of RAM and, on Windows, at least 4 GB of dedicated VRAM, which tells you the app is built with partial offload in mind. For how LM Studio compares with the other two runners, see Ollama vs llama.cpp vs LM Studio.

How to offload to RAM in vLLM and Transformers

vLLM's engine arguments include --cpu-offload-gb, the amount of space in GiB to offload to the CPU per GPU. The documentation's own example: with one 24 GB GPU and --cpu-offload-gb 10, you can treat it as a 34 GB GPU and load a 13B model in BF16, which needs at least 26 GB. vLLM also offers --kv-offloading-size to move part of the KV cache into CPU memory. Both trade speed for capacity, and vLLM is built for serving many requests, so for a single user on a small card llama.cpp is usually the better fit. Our vLLM vs llama.cpp comparison goes into that choice.

In Hugging Face Transformers, loading with device_map="auto" fills the GPU first, then CPU RAM, then disk. The Accelerate big model inference guide explains that weights offloaded to the CPU are moved to the GPU just before their forward pass, and you can cap each device with max_memory:

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    device_map="auto",
    max_memory={0: "10GiB", "cpu": "30GiB"},
)

This is handy for experiments and fine tuning scripts, but it is slow for chat compared with a GGUF model in llama.cpp.

Windows: the shared GPU memory surprise

On Windows, NVIDIA drivers can quietly use system RAM when a program runs out of VRAM. NVIDIA's support article on system memory fallback says driver 536.40 added this so applications that used to crash would keep running, "albeit at lower speeds," and driver 546.01 added a setting to turn it off. The article uses Stable Diffusion as its example, but the setting is per program: in NVIDIA Control Panel, open Manage 3D settings, add your runner's executable under Program Settings, and pick the CUDA Sysmem Fallback Policy setting and choose Prefer No Sysmem Fallback.

Why turn it off? Because a runner's own offload (llama.cpp layers, Ollama's split) is planned and predictable, while driver fallback kicks in silently near the VRAM limit. If your tokens per second suddenly drop on Windows and Task Manager shows shared GPU memory in use, this is the likely cause.

Macs: RAM is VRAM

On Apple Silicon the question mostly disappears. A Mac with 64 GB of unified memory can give the GPU most of that pool, at the bandwidth shown in the table above. macOS caps how much memory the GPU can wire by default. The mlx-lm README explains how to raise that limit with sudo sysctl iogpu.wired_limit_mb=N, where N is larger than the model in megabytes but smaller than total memory. Leave room for macOS itself. For which file format to use on a Mac, read MLX vs GGUF on Mac and see the MLX tool page.

Pros and cons of running a model from system RAM

Pros

• You can run models far bigger than your graphics card, as long as you have the RAM.
• Adding RAM is a simple upgrade, while VRAM is fixed on the graphics card you bought.
• Mixture of experts models such as gpt-oss and many Qwen releases run reasonably with experts in RAM, because only the active experts are read for each token.

Cons

• Generation slows with every layer that leaves the GPU, and CPU only runs on desktop RAM are slow for dense models above a few billion parameters.
• Prompt processing, the step before the first token appears, also slows, which hurts long documents.
• If the model exceeds RAM too, the system pages from disk and becomes close to unusable.

Which approach should you use?

• Model fits in VRAM with a modest context: keep it all on the GPU. Shrink the context or quantize the KV cache before you offload weights.
• Dense model slightly too big: use a smaller quantization first (see GGUF quantization explained), then partial offload with -ngl or LM Studio's GPU slider.
• Large mixture of experts model: keep attention on the GPU and put experts in RAM with --n-cpu-moe. Our gpt-oss local guide covers the model sizes.
• No usable GPU: run a small quantized model CPU only, or use a Mac where unified memory gives the GPU direct access.

FAQ

Can a local LLM run on RAM only, with no GPU?

Yes. llama.cpp, Ollama and LM Studio all run models entirely on the CPU from system RAM. Ollama reports this as 100% CPU in ollama ps. Expect it to be slow for dense models, since desktop RAM bandwidth sets a low ceiling on tokens per second.

How much slower is RAM than VRAM for LLMs?

It depends on your hardware, but you can estimate the ceiling by dividing memory bandwidth by the size of the weights read per token. Two channels of DDR5 at 5600 MT/s give 89.6 GB/s in theory, so an 8B model at about 4.9 GB tops out near 18 tokens per second before any overhead. Graphics card memory is several times faster.

Does system RAM count as VRAM on Windows?

Task Manager shows "shared GPU memory," and NVIDIA drivers since 536.40 can fall back to it when VRAM runs out. It is still system RAM reached over PCIe, so it is much slower than real VRAM. You can disable the fallback per program with the CUDA Sysmem Fallback Policy setting.

Is it better to offload layers or experts?

For mixture of experts models, offload experts with --cpu-moe or --n-cpu-moe in llama.cpp and keep the rest on the GPU. For dense models there are no experts, so you offload whole layers with -ngl.

Why does Ollama show a CPU/GPU split for a model that should fit?

Usually the context window. The KV cache is allocated when the model loads, and parallel requests multiply the context size. Lower the context or enable a quantized KV cache, then check ollama ps again.

How much RAM do I need to offload a model?

Enough for the part of the model that is not on the GPU, plus the KV cache if it sits in RAM, plus room for your operating system and apps. If the total goes beyond physical RAM, the model pages from disk and slows dramatically.