How much RAM do you need for local AI? (System RAM, not VRAM)

If your model fits on the GPU: 16 GB is the minimum and 32 GB is comfortable. If you plan to spill models onto the CPU, or run with no GPU, you want enough RAM to hold the model plus about 8 GB for everything else. A Mac is different, because its RAM is the GPU's memory.

Model on the GPU: RAM is a supporting act

When everything fits in VRAM, system RAM mostly helps with loading. The model file is read from disk, often through RAM, and then copied to the card. After that it sits on the GPU, and RAM speed has no effect on how fast the model writes.

So you need enough for your operating system, your browser and other apps, the inference tool itself, and some room for the file while it loads. In practice:

That's what our builds use. The RTX 3060 starter has 16 GB, the RTX 3090 desktop has 32 GB, and the used workstations for multi-GPU builds come with 64 GB, which is cheap in old DDR4 workstations. See the builds page.

Having less than the model's size in RAM usually still works, because tools like llama.cpp can stream the file from disk instead of holding it all. It can make loading slower, and it's tight if you also run a browser and Docker.

Model spilling onto the CPU: RAM is the bottleneck

If the model doesn't fit in VRAM, the rest goes in system RAM and runs on the CPU. Now RAM matters twice:

• Size. VRAM + system RAM has to hold the whole model, its context, and your OS. Rule of thumb: RAM ≥ (model file − VRAM) + 8 GB.
• Speed. Every token reads the offloaded part from RAM, so RAM bandwidth sets your speed. Fill both memory channels (two or four matching sticks), and faster RAM helps directly.

Example: gpt-oss 120B is about 65 GB. On a 24 GB card, about 41 GB has to live in RAM, plus your OS, so 64 GB of RAM is the realistic minimum. Because it's a mixture-of-experts model, this setup is usable; a dense model of the same size would crawl. See CPU offloading explained.

No GPU at all

On a CPU-only machine, RAM is your VRAM. Use the same sizing as how much VRAM you need, plus room for the OS:

Expect a few tokens per second on dense models. MoE models are the best fit for CPU-only machines. See running AI on a laptop without a GPU.

Macs: unified memory

On Apple silicon, there's one pool of memory shared by the CPU and GPU, and macOS lets the GPU use about two thirds of it on machines up to 36 GB, and about three quarters above that. So RAM is the model budget:

With a Mac, buy as much memory as you can afford. You can't add it later. See Macs for local LLMs.

Other reasons to add RAM

• Keeping several models loaded. Ollama can keep models in memory between requests. Swapping between models is quicker if the file is still cached in RAM.
• Docker and Open WebUI. A few GB for the containers. See Open WebUI setup.
• Documents and embeddings. Document search keeps an index in RAM. Small for personal use.

Quick answer by machine

• Gaming PC, 8 to 16 GB card: 16 GB is fine, 32 GB if you multitask.
• One 24 GB card: 32 GB.
• Two cards, or plans to offload big MoE models: 64 GB or more.
• Mac: as much unified memory as the budget allows.

Here's what a 24 GB card runs with everything in VRAM, so RAM size doesn't touch speed: