Qwen3 8B
The default first model for most people. Good enough at writing, chat and light code to be useful every day, and small enough to run fast on an 8 GB card.
ollama run qwen3:8bNeeds about 7 GB of graphics memory, or unified memory on a Mac. First time? Install Ollama, then paste that into a terminal.
Download5.2 GB
Memory to runAbout 7 GB
Read per token5.2 GB
LicenseApache 2.0
Context32K tokens, 128K with YaRN
ReleasedApril 2025
Good for
- Everyday chat and writing help
- Light coding questions
- Local assistants and home automation
- Thinking mode for step-by-step answers
Not the best pick for
- Big refactors across many files
- Anything where a wrong fact is costly
135 GPUs and Macs
What runs Qwen3 8B
Estimated speed on each card at the smallest memory size that fits it, from memory bandwidth. As a dense model it reads all its weights for every token, so memory bandwidth sets the speed.
Runs great 109 cards
- H200141 GB · ~510 tok/s
- H10080 GB · ~350 tok/s
- RTX 509032 GB · ~190 tok/s
- RTX Pro 6000 Blackwell96 GB · ~190 tok/s
- A10040 GB · ~160 tok/s
- RTX Pro 5000 Blackwell48 GB · ~140 tok/s
- Instinct MI10032 GB · ~130 tok/s
- Instinct MI5016 GB · ~110 tok/s
- RTX 3090 Ti24 GB · ~110 tok/s
- RTX 409024 GB · ~110 tok/s
- RTX 508016 GB · ~100 tok/s
- RTX 6000 Ada48 GB · ~100 tok/s
- RX 7900 XTX24 GB · ~100 tok/s
- RTX 309024 GB · ~99 tok/s
- A3024 GB · ~99 tok/s
- RTX 3080 Ti12 GB · ~96 tok/s
- Tesla V10016 GB · ~95 tok/s
- RTX 5070 Ti16 GB · ~95 tok/s
- RTX 5080 Laptop16 GB · ~95 tok/s
- RTX 5090 Laptop24 GB · ~95 tok/s
- RTX Pro 4500 Blackwell32 GB · ~95 tok/s
- L40S48 GB · ~91 tok/s
- Radeon Pro W790048 GB · ~91 tok/s
- Apple M3 Ultra96 GB · ~87 tok/s
- RX 7900 XT20 GB · ~85 tok/s
- Apple M1 Ultra64 GB · ~85 tok/s
- Apple M2 Ultra64 GB · ~85 tok/s
- RTX A500024 GB · ~81 tok/s
- RTX A550024 GB · ~81 tok/s
- RTX A600048 GB · ~81 tok/s
- RTX 308010 GB · ~80 tok/s
- RTX 4080 Super16 GB · ~78 tok/s
- Tesla P10012 GB · ~77 tok/s
- RTX 408016 GB · ~76 tok/s
- A4048 GB · ~74 tok/s
- RTX 4070 Ti Super16 GB · ~71 tok/s
- RTX 507012 GB · ~71 tok/s
- RTX 5070 Ti Laptop12 GB · ~71 tok/s
- Titan RTX24 GB · ~71 tok/s
- Quadro RTX 600024 GB · ~71 tok/s
- Quadro RTX 800048 GB · ~71 tok/s
- RTX Pro 4000 Blackwell24 GB · ~71 tok/s
- Titan V12 GB · ~69 tok/s
- RTX A450020 GB · ~68 tok/s
- RX 907016 GB · ~68 tok/s
- RX 9070 XT16 GB · ~68 tok/s
- Radeon AI Pro R970032 GB · ~68 tok/s
- RX 7800 XT16 GB · ~66 tok/s
- RTX 2080 Ti11 GB · ~65 tok/s
- RTX 3070 Ti8 GB · ~64 tok/s
- A1024 GB · ~63 tok/s
- RTX 4090 Laptop16 GB · ~61 tok/s
- RTX 5000 Ada32 GB · ~61 tok/s
- RX 6950 XT16 GB · ~61 tok/s
- RX 7900 GRE16 GB · ~61 tok/s
- Radeon Pro W780032 GB · ~61 tok/s
- Arc A7708 GB · ~59 tok/s
- Apple M4 Max36 GB · ~58 tok/s
- RTX 3080 Ti Laptop16 GB · ~54 tok/s
- RX 680016 GB · ~54 tok/s
- RX 6800 XT16 GB · ~54 tok/s
- RX 6900 XT16 GB · ~54 tok/s
- Radeon Pro W680032 GB · ~54 tok/s
- Arc A7508 GB · ~54 tok/s
- RTX 407012 GB · ~53 tok/s
- RTX 4070 Super12 GB · ~53 tok/s
- RTX 4070 Ti12 GB · ~53 tok/s
- RTX 2080 Super8 GB · ~52 tok/s
- GTX 1080 Ti11 GB · ~51 tok/s
- Apple M5 Max36 GB · ~49 tok/s
- Arc B58012 GB · ~48 tok/s
- Arc Pro B6024 GB · ~48 tok/s
- RTX 2060 Super8 GB · ~47 tok/s
- RTX 20708 GB · ~47 tok/s
- RTX 2070 Super8 GB · ~47 tok/s
- RTX 20808 GB · ~47 tok/s
- RTX 3060 Ti8 GB · ~47 tok/s
- RTX 30708 GB · ~47 tok/s
- RTX 50608 GB · ~47 tok/s
- RTX 5060 Ti8 GB · ~47 tok/s
- RTX 3070 Laptop8 GB · ~47 tok/s
- RTX 3070 Ti Laptop8 GB · ~47 tok/s
- RTX 3080 Laptop8 GB · ~47 tok/s
- Quadro RTX 500016 GB · ~47 tok/s
- RTX A400016 GB · ~47 tok/s
- RTX 4080 Laptop12 GB · ~46 tok/s
- RTX 4500 Ada24 GB · ~46 tok/s
- RX 7700 XT12 GB · ~46 tok/s
- Quadro RTX 40008 GB · ~44 tok/s
- Apple M1 Max32 GB · ~42 tok/s
- Apple M2 Max32 GB · ~42 tok/s
- Apple M3 Max36 GB · ~42 tok/s
- RTX 5070 Laptop8 GB · ~41 tok/s
- RX 6700 XT12 GB · ~41 tok/s
- Arc B57010 GB · ~40 tok/s
- RTX 30608 GB · ~38 tok/s
- RTX 4000 Ada20 GB · ~38 tok/s
- Tesla P4024 GB · ~37 tok/s
- RTX 206012 GB · ~36 tok/s
- GTX 10808 GB · ~34 tok/s
- Tesla T416 GB · ~34 tok/s
- RX 9060 XT8 GB · ~34 tok/s
- Apple M5 Pro24 GB · ~32 tok/s
- L424 GB · ~32 tok/s
- RTX 4060 Ti8 GB · ~30 tok/s
- RTX A200012 GB · ~30 tok/s
- Tesla M4012 GB · ~30 tok/s
- RX 76008 GB · ~30 tok/s
- RX 7600 XT16 GB · ~30 tok/s
Usable 24 cards
- Apple M4 Pro24 GB · ~29 tok/s
- NVIDIA DGX Spark128 GB · ~29 tok/s
- RTX 40608 GB · ~29 tok/s
- GTX 10708 GB · ~27 tok/s
- GTX 1070 Ti8 GB · ~27 tok/s
- RTX 4060 Laptop8 GB · ~27 tok/s
- RTX 4070 Laptop8 GB · ~27 tok/s
- RX 6600 XT8 GB · ~27 tok/s
- Ryzen AI Max+ 395 (Radeon 8060S)64 GB · ~27 tok/s
- Ryzen AI Max 390 (Radeon 8050S)32 GB · ~27 tok/s
- RTX 30508 GB · ~24 tok/s
- RTX 2000 Ada16 GB · ~24 tok/s
- RX 66008 GB · ~24 tok/s
- Jetson AGX Orin32 GB · ~22 tok/s
- Apple M1 Pro16 GB · ~21 tok/s
- Apple M2 Pro16 GB · ~21 tok/s
- Apple M516 GB · ~16 tok/s
- Apple M3 Pro18 GB · ~16 tok/s
- Intel Arc 140V16 GB · ~14 tok/s
- Snapdragon X Elite (Adreno X1-85)16 GB · ~14 tok/s
- Apple M416 GB · ~13 tok/s
- Radeon 890M16 GB · ~13 tok/s
- Apple M216 GB · ~11 tok/s
- Apple M316 GB · ~11 tok/s
Cheapest build from our list that runs it well: Used desktop + RTX 3060 12 GB, from $600 used, at about 38 tok/s. See the builds.
Learn
Read more about Qwen3 8B
- A local coding assistant in VS Code: Continue, Cline and OllamaSet up a private AI coding assistant in VS Code with Ollama and Continue or Cline. Which model to pick for your card, and what to expect from it.
- Batch jobs with a local model: process a folder of files with OllamaRun a local model over hundreds or thousands of files with a short Python script and Ollama's API. Summarize, tag or extract, with no per-token bill.
- Build a local voice assistant: Whisper, a local LLM and text-to-speechHow a private voice assistant works on your own hardware, which parts to use for speech, thinking and speaking, and how to make it answer fast.
- Chat with your documents locally, without uploading them anywhereAsk questions about your PDFs, notes and contracts with a model that runs on your own machine. How it works, how to set it up, and how to make the answers good.
- Does Ollama send your data anywhere? Local AI privacy, checkedWhat stays on your computer when you run a local model, what does go over the network, and how to check for yourself instead of taking anyone's word.
- Fix CUDA out of memory errors in Ollama and llama.cppWhy local models run out of GPU memory, and the fixes in order - smaller quant, shorter context, a quantized KV cache, fewer parallel slots, partial offload.
- Free up disk space in Ollama: how to update and delete modelsSee how much space your Ollama models use, delete the ones you don't need, update the rest, and clean up after LM Studio and Hugging Face too.
- GGUF vs safetensors vs MLX: which model file format do you need?What the three common model file formats are, which tools read each one, and which to download for an NVIDIA card, an older GPU, or a Mac.
- How much RAM do you need for local AI? (System RAM, not VRAM)How much system memory a local AI machine needs when the model is on the GPU, when it spills onto the CPU, on a Mac, and with no GPU at all.
- How much VRAM do you need to run an LLM locally?A simple rule for how much graphics memory a model needs, what fits in 8, 12, 16, 24 and 48 GB, and why context length adds more.
- How to check a used GPU before you buy: stress, VRAM and speed testsA step-by-step check for a used graphics card for AI. What to inspect, how to stress test it, test its memory, read temperatures and confirm its real speed.
- How to increase Ollama's context length (num_ctx)Ollama's default context is short and it cuts off long prompts quietly. Three ways to raise num_ctx, and how much memory a longer context costs.
- How to run AI locally: your first model in ten minutesInstall Ollama, run one command, and chat with an open model on the computer you already own. No account, no cloud, no monthly bill.
- How we benchmark: the RunItLocal.io test, step by stepThe exact test behind our benchmark prompt, from the two tiers and the llama-bench commands to the power numbers, what we keep when you send a result in, and what we can't check.
- Import a GGUF file into Ollama with a ModelfileRun any GGUF model in Ollama. Pull straight from Hugging Face, or write a three-line Modelfile for a local file, with templates, context and system prompts.
- Intel Arc for local AI: can the B580 and A770 run LLMs?How Intel Arc cards run local LLMs, which tools support them, estimated speeds for the B580 and A770 16 GB, and the software catches to know first.
- Is running AI locally worth it? Local vs ChatGPT, honestlyWhat you gain and give up by running open models on your own hardware instead of paying for a cloud AI: privacy, cost, quality, and speed.
- llama.cpp server guide: run llama-server with the flags that matterInstall llama.cpp, download a model, and serve it with llama-server. The handful of flags that matter for GPU layers, context and parallel users.
- LM Studio guide: download, run and serve local modelsHow to use LM Studio to find a model that fits your GPU or Mac, set GPU offload and context, and run its local OpenAI-compatible server.
- Local AI agents: tool calling, MCP and what works on small modelsHow agents and tool calling work with local models in Ollama, what MCP adds, which models handle tools reliably, and how to keep small models on track.
- Local AI for email: draft, summarize and sort mail privatelyUse a model on your own computer to draft replies, summarize threads and sort your inbox, without giving a cloud service your email. What works and what to avoid.
- Local AI for families and kids: private, but not automatically safeWhat running AI at home means for a family. The real privacy benefits, the safety filters you lose, which models to use, and how to set it up for children.
- Local AI for Home Assistant: a private voice assistant with OllamaConnect Home Assistant to a model running on your own hardware with the Ollama integration. What hardware you need, which model to pick, and how to set it up.
- Local AI for Obsidian: chat with your notes privatelyHow to connect Obsidian to a model running on your own machine with Ollama, which plugins and models to use, and how to keep your vault private.
- Local AI glossary: tokens, context, quants and GGUF in plain EnglishThe words you'll meet when running AI models locally, explained in a sentence or two each: tokens, context, VRAM, quantization, GGUF, MoE and more.
- Local embedding models: nomic-embed-text and friendsWhat embedding models do, why they power search over your documents, how to run nomic-embed-text with Ollama, and how to choose one.
- Local RAG that actually works: chunking, embeddings and rerankingWhy local document search gives bad answers and how to fix it. Practical settings for chunking, embedding models, hybrid search and reranking on your own machine.
- Local speech to text: Whisper model sizes and how to chooseRun Whisper on your own computer to transcribe audio privately. The model sizes, the speed and accuracy trade-off, and which tools to use.
- Local text to speech: natural voices without the cloudHow local text to speech works, the kinds of models and tools available, what hardware they need, and how to pair one with a local chat model.
- Mini PCs and AMD Strix Halo for local AI: unified memory on x86Can a mini PC run local LLMs? What ordinary mini PCs manage, why AMD's Strix Halo chips with up to 128 GB of shared memory are different, and the trade-offs.
- Mining cards for AI: do NVIDIA CMP cards run local LLMs?Can ex-mining NVIDIA CMP cards run local AI? What works, the limits to check before buying, and what we learned running them in our 90-GPU rack.
- Ollama API guide: curl examples and the OpenAI-compatible endpointUse Ollama from code. Working curl examples for /api/generate, /api/chat and the OpenAI-compatible /v1/chat/completions, plus the options that matter.
- Ollama commands cheat sheet: every command you'll actually useThe Ollama commands for running, downloading, listing, stopping and deleting models, plus the in-chat slash commands and the settings worth knowing.
- Ollama on a Mac: setup for Apple silicon, step by stepInstall Ollama on an M-series Mac, pick a model that fits your memory, check it's using the GPU, and change the settings that matter.
- Ollama on Linux with an NVIDIA GPU: drivers, setup and fixesInstall the NVIDIA driver, install Ollama as a service, confirm it's using the GPU, and fix the problems that send it to the CPU instead.
- Ollama using CPU instead of GPU? How to check and fix itOllama slow and your GPU idle? Check with ollama ps, then fix the usual causes - model too big, drivers, Docker, laptops, AMD and sleep - step by step.
- Ollama vs llama.cpp vs LM Studio: which should you use?The three most popular ways to run models locally, what each is best at, and when to switch. Short version, start with Ollama.
- Open model licenses explained: Apache 2.0, Llama and Gemma termsWhat the licenses on popular open models mean in practice for personal, business and product use. Apache 2.0, the Llama community license and Gemma terms.
- Quantization explained: Q4_K_M vs Q8 vs FP16What the letters and numbers in model file names mean, how much quality you lose at each size, and which one to download.
- Qwen3 vs Gemma 3: which local model should you run?Qwen3 and Gemma 3 compared for local use. Reasoning, images, writing, licenses, memory and estimated speeds on 12 and 24 GB cards.
- Run a local LLM on Windows: setup with Ollama and an NVIDIA or AMD GPUInstall Ollama on Windows 10 or 11, get it using your graphics card, change where models are stored, and fix the common problems.
- Run AI in your browser with WebGPU: local models, no installHow WebLLM and WebGPU run language models and image generation on your own GPU inside a browser tab, what works, and where it falls short of Ollama.
- Run an LLM on CPU only: local AI on a laptop without a GPUWhich models run on a laptop with no graphics card, how fast to expect them, and the settings that make CPU-only local AI usable.
- Run Ollama in Docker, with NVIDIA GPU supportRun the official ollama/ollama Docker image on CPU or NVIDIA GPU. The NVIDIA Container Toolkit setup, Docker Compose, and checking the GPU is used.
- Running LLMs on a Mac: which M-series chip and how much memoryWhy Macs with Apple silicon are good at local AI, how much unified memory you need for each model size, and how they compare to a PC with a GPU.
- Secure your local AI server: don't expose Ollama's port 11434Ollama has no passwords. How to check what's exposed, lock it to your machine or network, add auth, and reach it remotely without opening ports.
- Set up Open WebUI: a private ChatGPT on your own computerPut a ChatGPT-style chat window on top of Ollama in about ten minutes, with chat history, document upload, and access from your phone.
- Structured output with Ollama: get reliable JSON from a local modelUse Ollama's format parameter to make a local model return valid JSON, or JSON that matches your own schema. Working curl and Python examples, plus tips.
- Summarize meetings locally: transcribe and summarize without the cloudTurn a meeting recording into a transcript, summary and action items on your own computer with Whisper and a local model. Steps, models and the context-length catch.
- Temperature, top-p and min-p: the sampling settings that matterWhat temperature and the other sampling settings do, which ones are worth changing for local models, and how to set them in Ollama.
- The best GPU for local AI under $300The best graphics cards for running local LLMs on a $300 budget, what each one runs, estimated speeds, and which to pick for easy setup or raw value.
- The best laptop for local AI: what to look for and what to avoidHow to pick a laptop for running local LLMs. Why laptop GPU names mislead, how much VRAM you need, estimated speeds, and when a MacBook is the better choice.
- The best local LLM for 12 GB of VRAM (RTX 3060, 4070, B580)What to run on a 12 GB graphics card like the RTX 3060, 4070 or Arc B580. Picks for smarts, images and speed, with estimated speeds for each card.
- The best local LLM for 8 GB of VRAMWhich open models run well on an 8 GB graphics card like the RTX 4060, 3070 or RX 7600, how fast they go, and what to do when you outgrow it.
- The best local LLM for a 16 GB Mac (MacBook Air, Mac mini)What runs well on a Mac with 16 GB of unified memory, how fast each M-series chip is, and how to keep the model from slowing your Mac down.
- The best local LLM for coding, by how much memory you haveWhich open models are worth using for code on 8, 12, 16, 24 GB and up, what they're good and bad at, and how to wire them into your editor.
- The best local LLM for reasoning and mathWhich open models reason best through math, logic and multi-step problems on 8 GB to 64 GB+, how thinking mode works, and why speed matters more than you'd expect.
- The best local LLM for writing, by how much memory you haveWhich open models write best for drafts, editing, emails and fiction on 8 GB to 48 GB+, plus the settings and prompts that make local writing sound less generic.
- The best local model for translationWhich open models translate best on your own hardware, by memory size, plus prompts that get natural results and where local translation still falls short.
- The best small local model for scripts, classification and JSONWhich small open model to use when a script calls it thousands of times to classify text, extract fields or return JSON, and how to get reliable structured output.
- The RTX 3060 12 GB for AI: the best cheap starter cardWhy the 12 GB RTX 3060 is the cheapest sensible GPU for local AI, what it runs and how fast, what to pay, and the 8 GB version to avoid.
- Thinking models explained: reasoning mode, /think and effort levelsWhat reasoning mode does, when it's worth the wait, how to turn it on and off in Qwen3 and gpt-oss, and what it costs in speed and context.
- Tokens per second explained: what's a good speed for a local LLM?What tok/s means, how fast is fast enough for chat, coding and thinking models, and how to measure your own speed in Ollama.
- Uncensored local models: what abliterated means, and the tradeoffsWhy local models refuse some requests, what abliterated and uncensored fine-tunes change, what it costs in quality, and how to cut needless refusals without one.
- Use Ollama from other devices: OLLAMA_HOST, phones and other PCsMake Ollama reachable from your phone, laptop or another PC on your home network by setting OLLAMA_HOST on Mac, Windows and Linux, and keep it safe.
- vLLM vs Ollama: when to move to vLLM for a teamOllama is right for one person. vLLM is built for many users at once. When the switch is worth it, what hardware vLLM needs, and how to start it.
- What is a system prompt, and how to set one in Ollama and other local toolsWhat a system prompt does, what to put in one, and how to set it in Ollama, a Modelfile, Open WebUI, LM Studio and the API.
- What to actually do with a local AI model: your first weekSeven practical things to try in your first week with a local model, from rewriting email to reading your own documents and scripting batch jobs.
- Where Ollama stores models, and how to change the locationThe default Ollama model folder on Windows, Mac and Linux, how to move it to another drive with OLLAMA_MODELS, and how to bring your existing models along.
Compare
Other models
- Qwen3 4BAbout 4 GB · Quick everyday chat and summaries
- Gemma 3 12BAbout 10 GB · Reads images as well as text
- Qwen3 14BAbout 11 GB · Noticeably smarter at reasoning and code
- gpt-oss 20BAbout 16 GB · OpenAI's open model. Fast, strong reasoning
- Gemma 3 27BAbout 19 GB · Google's best open model for its size
- Qwen3 30B-A3BAbout 21 GB · 30B smarts at small-model speed
- Qwen3 32BAbout 22 GB · Strong coding and long answers
- Llama 3.3 70BAbout 45 GB · Close to the big cloud models
- gpt-oss 120BAbout 67 GB · OpenAI's largest open model
- Qwen3 235B-A22BAbout 144 GB · Alibaba's flagship open model, for multi-GPU rigs
- DeepSeek R1 671BAbout 406 GB · The full DeepSeek R1, for the biggest rigs