Gemma 3 12B
Google's mid-size open model, and the smallest one here that reads images. Strong at writing and at languages other than English.
ollama run gemma3:12bNeeds about 10 GB of graphics memory, or unified memory on a Mac. First time? Install Ollama, then paste that into a terminal.
Download8.1 GB
Memory to runAbout 10 GB
Read per token8.1 GB
LicenseGemma Terms of Use
Context128K tokens
ReleasedMarch 2025
Good for
- Describing and asking questions about images and screenshots
- Writing and translation in many languages
- Long documents, thanks to the 128K context
Not the best pick for
- Hard reasoning and math, where Qwen3 14B does better
- Projects that need a standard open-source license: Gemma has its own terms
111 GPUs and Macs
What runs Gemma 3 12B
Estimated speed on each card at the smallest memory size that fits it, from memory bandwidth. As a dense model it reads all its weights for every token, so memory bandwidth sets the speed.
Runs great 73 cards
- H200141 GB · ~330 tok/s
- H10080 GB · ~230 tok/s
- RTX 509032 GB · ~120 tok/s
- RTX Pro 6000 Blackwell96 GB · ~120 tok/s
- A10040 GB · ~110 tok/s
- RTX Pro 5000 Blackwell48 GB · ~91 tok/s
- Instinct MI10032 GB · ~83 tok/s
- Instinct MI5016 GB · ~70 tok/s
- RTX 3090 Ti24 GB · ~68 tok/s
- RTX 409024 GB · ~68 tok/s
- RTX 508016 GB · ~65 tok/s
- RTX 6000 Ada48 GB · ~65 tok/s
- RX 7900 XTX24 GB · ~65 tok/s
- RTX 309024 GB · ~64 tok/s
- A3024 GB · ~63 tok/s
- RTX 3080 Ti12 GB · ~62 tok/s
- Tesla V10016 GB · ~61 tok/s
- RTX 5070 Ti16 GB · ~61 tok/s
- RTX 5080 Laptop16 GB · ~61 tok/s
- RTX 5090 Laptop24 GB · ~61 tok/s
- RTX Pro 4500 Blackwell32 GB · ~61 tok/s
- L40S48 GB · ~59 tok/s
- Radeon Pro W790048 GB · ~59 tok/s
- Apple M3 Ultra96 GB · ~56 tok/s
- RX 7900 XT20 GB · ~54 tok/s
- Apple M1 Ultra64 GB · ~54 tok/s
- Apple M2 Ultra64 GB · ~54 tok/s
- RTX A500024 GB · ~52 tok/s
- RTX A550024 GB · ~52 tok/s
- RTX A600048 GB · ~52 tok/s
- RTX 308010 GB · ~52 tok/s
- RTX 4080 Super16 GB · ~50 tok/s
- Tesla P10012 GB · ~50 tok/s
- RTX 408016 GB · ~49 tok/s
- A4048 GB · ~47 tok/s
- RTX 4070 Ti Super16 GB · ~46 tok/s
- RTX 507012 GB · ~46 tok/s
- RTX 5070 Ti Laptop12 GB · ~46 tok/s
- Titan RTX24 GB · ~46 tok/s
- Quadro RTX 600024 GB · ~46 tok/s
- Quadro RTX 800048 GB · ~46 tok/s
- RTX Pro 4000 Blackwell24 GB · ~46 tok/s
- Titan V12 GB · ~44 tok/s
- RTX A450020 GB · ~43 tok/s
- RX 907016 GB · ~43 tok/s
- RX 9070 XT16 GB · ~43 tok/s
- Radeon AI Pro R970032 GB · ~43 tok/s
- RX 7800 XT16 GB · ~42 tok/s
- RTX 2080 Ti11 GB · ~42 tok/s
- A1024 GB · ~41 tok/s
- RTX 4090 Laptop16 GB · ~39 tok/s
- RTX 5000 Ada32 GB · ~39 tok/s
- RX 6950 XT16 GB · ~39 tok/s
- RX 7900 GRE16 GB · ~39 tok/s
- Radeon Pro W780032 GB · ~39 tok/s
- Arc A77016 GB · ~38 tok/s
- Apple M4 Max36 GB · ~37 tok/s
- RTX 3080 Ti Laptop16 GB · ~35 tok/s
- RX 680016 GB · ~35 tok/s
- RX 6800 XT16 GB · ~35 tok/s
- RX 6900 XT16 GB · ~35 tok/s
- Radeon Pro W680032 GB · ~35 tok/s
- RTX 407012 GB · ~34 tok/s
- RTX 4070 Super12 GB · ~34 tok/s
- RTX 4070 Ti12 GB · ~34 tok/s
- GTX 1080 Ti11 GB · ~33 tok/s
- Apple M5 Max36 GB · ~31 tok/s
- Arc B58012 GB · ~31 tok/s
- Arc Pro B6024 GB · ~31 tok/s
- RTX 5060 Ti16 GB · ~30 tok/s
- RTX 3080 Laptop16 GB · ~30 tok/s
- Quadro RTX 500016 GB · ~30 tok/s
- RTX A400016 GB · ~30 tok/s
Usable 30 cards
- RTX 4080 Laptop12 GB · ~29 tok/s
- RTX 4500 Ada24 GB · ~29 tok/s
- RX 7700 XT12 GB · ~29 tok/s
- Apple M1 Max32 GB · ~27 tok/s
- Apple M2 Max32 GB · ~27 tok/s
- Apple M3 Max36 GB · ~27 tok/s
- RX 6700 XT12 GB · ~26 tok/s
- Arc B57010 GB · ~26 tok/s
- RTX 306012 GB · ~24 tok/s
- RTX 4000 Ada20 GB · ~24 tok/s
- Tesla P4024 GB · ~23 tok/s
- RTX 206012 GB · ~23 tok/s
- Tesla T416 GB · ~22 tok/s
- RX 9060 XT16 GB · ~22 tok/s
- Apple M5 Pro24 GB · ~21 tok/s
- L424 GB · ~20 tok/s
- RTX 4060 Ti16 GB · ~20 tok/s
- RTX A200012 GB · ~20 tok/s
- Tesla M4012 GB · ~20 tok/s
- RX 7600 XT16 GB · ~20 tok/s
- Apple M4 Pro24 GB · ~19 tok/s
- NVIDIA DGX Spark128 GB · ~19 tok/s
- Ryzen AI Max+ 395 (Radeon 8060S)64 GB · ~17 tok/s
- Ryzen AI Max 390 (Radeon 8050S)32 GB · ~17 tok/s
- RTX 2000 Ada16 GB · ~15 tok/s
- Jetson AGX Orin32 GB · ~14 tok/s
- Apple M1 Pro16 GB · ~14 tok/s
- Apple M2 Pro16 GB · ~14 tok/s
- Apple M516 GB · ~10 tok/s
- Apple M3 Pro18 GB · ~10 tok/s
Slow 8 cards
Cheapest build from our list that runs it well: Used desktop + RTX 3060 12 GB, from $600 used, at about 24 tok/s. See the builds.
Learn
Read more about Gemma 3 12B
- Chat with your documents locally, without uploading them anywhereAsk questions about your PDFs, notes and contracts with a model that runs on your own machine. How it works, how to set it up, and how to make the answers good.
- How to increase Ollama's context length (num_ctx)Ollama's default context is short and it cuts off long prompts quietly. Three ways to raise num_ctx, and how much memory a longer context costs.
- How to read a model card: what matters for running it locallyParameters, active parameters, context, license, quant files and benchmarks. Which parts of a model card tell you if it will run on your hardware.
- LM Studio guide: download, run and serve local modelsHow to use LM Studio to find a model that fits your GPU or Mac, set GPU offload and context, and run its local OpenAI-compatible server.
- Local AI for families and kids: private, but not automatically safeWhat running AI at home means for a family. The real privacy benefits, the safety filters you lose, which models to use, and how to set it up for children.
- Local AI for Obsidian: chat with your notes privatelyHow to connect Obsidian to a model running on your own machine with Ollama, which plugins and models to use, and how to keep your vault private.
- Local RAG that actually works: chunking, embeddings and rerankingWhy local document search gives bad answers and how to fix it. Practical settings for chunking, embedding models, hybrid search and reranking on your own machine.
- Ollama using CPU instead of GPU? How to check and fix itOllama slow and your GPU idle? Check with ollama ps, then fix the usual causes - model too big, drivers, Docker, laptops, AMD and sleep - step by step.
- Qwen3 vs Gemma 3: which local model should you run?Qwen3 and Gemma 3 compared for local use. Reasoning, images, writing, licenses, memory and estimated speeds on 12 and 24 GB cards.
- Set up Open WebUI: a private ChatGPT on your own computerPut a ChatGPT-style chat window on top of Ollama in about ten minutes, with chat history, document upload, and access from your phone.
- Summarize meetings locally: transcribe and summarize without the cloudTurn a meeting recording into a transcript, summary and action items on your own computer with Whisper and a local model. Steps, models and the context-length catch.
- The best local LLM for 12 GB of VRAM (RTX 3060, 4070, B580)What to run on a 12 GB graphics card like the RTX 3060, 4070 or Arc B580. Picks for smarts, images and speed, with estimated speeds for each card.
- The best local LLM for 16 GB of VRAM (4060 Ti, 5060 Ti, 5080)What to run on a 16 GB card like the RTX 4060 Ti 16 GB, 5060 Ti, 5080, RX 9070 XT or a used Tesla P100, with estimated speeds and what doesn't fit.
- The best local LLM for 8 GB of VRAMWhich open models run well on an 8 GB graphics card like the RTX 4060, 3070 or RX 7600, how fast they go, and what to do when you outgrow it.
- The best local LLM for a 16 GB Mac (MacBook Air, Mac mini)What runs well on a Mac with 16 GB of unified memory, how fast each M-series chip is, and how to keep the model from slowing your Mac down.
- The best local LLM for writing, by how much memory you haveWhich open models write best for drafts, editing, emails and fiction on 8 GB to 48 GB+, plus the settings and prompts that make local writing sound less generic.
- The best local model for translationWhich open models translate best on your own hardware, by memory size, plus prompts that get natural results and where local translation still falls short.
- The best local vision model for reading images and screenshotsHow to run a local model that reads images, screenshots, charts and documents. Why Gemma 3 is the pick, which size to run, and what it can and can't do.
- The RTX 3060 12 GB for AI: the best cheap starter cardWhy the 12 GB RTX 3060 is the cheapest sensible GPU for local AI, what it runs and how fast, what to pay, and the 8 GB version to avoid.
- What is a system prompt, and how to set one in Ollama and other local toolsWhat a system prompt does, what to put in one, and how to set it in Ollama, a Modelfile, Open WebUI, LM Studio and the API.
- What to actually do with a local AI model: your first weekSeven practical things to try in your first week with a local model, from rewriting email to reading your own documents and scripting batch jobs.
Compare
Other models
- Qwen3 4BAbout 4 GB · Quick everyday chat and summaries
- Qwen3 8BAbout 7 GB · A solid all-rounder for chat and writing
- Qwen3 14BAbout 11 GB · Noticeably smarter at reasoning and code
- gpt-oss 20BAbout 16 GB · OpenAI's open model. Fast, strong reasoning
- Gemma 3 27BAbout 19 GB · Google's best open model for its size
- Qwen3 30B-A3BAbout 21 GB · 30B smarts at small-model speed
- Qwen3 32BAbout 22 GB · Strong coding and long answers
- Llama 3.3 70BAbout 45 GB · Close to the big cloud models
- gpt-oss 120BAbout 67 GB · OpenAI's largest open model
- Qwen3 235B-A22BAbout 144 GB · Alibaba's flagship open model, for multi-GPU rigs
- DeepSeek R1 671BAbout 406 GB · The full DeepSeek R1, for the biggest rigs