- Learn
- Get started
Is running AI locally worth it? Local vs ChatGPT, honestly
What you gain and give up by running open models on your own hardware instead of paying for a cloud AI: privacy, cost, quality, and speed.
Short answer: for a lot of everyday work, yes, and you can find out for free on the computer you already own. For the very hardest problems, the biggest cloud models are still ahead.
Here's how the trade really looks from people who run both.
What you gain
Privacy you don't have to take on trust. A local model runs on your machine. Your documents, code and questions never go to anyone's server, so there's no policy to read and no setting to opt out of. For client work, medical notes, or anything under an NDA, that alone settles it.
No metered bill. Once you own the hardware, a million tokens costs the electricity. On our rack of used data-center cards that comes to about 6 to 8 cents per million tokens. A cloud subscription is cheap until you start running scripts, agents, or a team on it.
It's always there. No rate limits, no outages, no model quietly changing under you. It works on a plane.
You can run it all day. Batch jobs like summarizing a folder of PDFs or tagging ten thousand support tickets are where local shines, because nobody is counting.
What you give up
Top-end quality. The best open models you can run at home are roughly where cloud models were a year or two ago. That's very good for writing, summarizing, coding help and chat. For the hardest reasoning, or long autonomous coding sessions, a frontier cloud model still wins.
Setup and upkeep. It's ten minutes to run your first model, but a good setup with a chat app, document search and remote access takes an afternoon.
Hardware, if you want the bigger models. A model's size is limited by memory. Bigger, smarter models need 24 GB or more; see how much VRAM you need.
Rough guide by hardware
| What you have | What runs well | Feels like |
|---|---|---|
| Laptop, 16 GB, no GPU | Qwen3 4B | A quick helper for rewriting and summaries |
| 8 GB gaming GPU | Qwen3 8B | A solid everyday assistant |
| 12 to 16 GB GPU | Qwen3 14B, gpt-oss 20B | Genuinely useful for code and reasoning |
| 24 GB GPU (RTX 3090) | Qwen3 32B, Qwen3 30B-A3B | Close to last year's cloud models |
| Mac with 128 GB, or several GPUs | gpt-oss 120B, Llama 3.3 70B | The strongest you can run at home |
The smart way to decide
- Run Qwen3 8B on what you already have. It costs nothing.
- Use it for a week on your real work, next to whatever cloud tool you use now.
- If it handles most of it, you're done. If you keep hitting its limits, look at a used 24 GB card or a build before paying for more cloud.
Most people end up using both: local for private and repetitive work, cloud for the occasional hard problem.
One email when there's a new guide, podcast episode, or result from the rack. No spam.