What is the biggest model the Tesla V100 can run?

Qwen3 32B, which needs about 22 GB, on the 32 GB version at about 25 tok/s.

What is the biggest model the Instinct MI100 can run?

Qwen3 32B, which needs about 22 GB, on the 32 GB version at about 34 tok/s.

Tesla V100 vs Instinct MI100 for local AI

Q: Which is better for local AI, the Tesla V100 or the Instinct MI100?

The Instinct MI100, for speed. They fit the same models. Both fit the same models we track, up to Qwen3 32B. The Instinct MI100 generates about 37% faster on any model both can run, because it has 1229 GB/s of memory bandwidth to the Tesla V100's 900 GB/s.

Both fit the same models we track, up to Qwen3 32B. The Instinct MI100 generates about 37% faster on any model both can run, because it has 1229 GB/s of memory bandwidth to the Tesla V100's 900 GB/s.

The numbers that matter

Memory and bandwidth

Memory decides which models fit. Bandwidth decides how fast they generate, because every token reads the model's active weights once.

	Tesla V100	Instinct MI100
Memory	16 or 32 GB VRAM	32 GB VRAM
Bandwidth	900 GB/s	1229 GB/s
Biggest model that fits	Qwen3 32B	Qwen3 32B
Best everyday pick	Qwen3 32B, ~25 tok/s	Qwen3 32B, ~34 tok/s

Model by model

Every model, side by side

Estimated generation speed at the largest memory size each was sold in (32 GB and 32 GB).

Model	Tesla V100	Instinct MI100
Qwen3 4B About 4 GB	~200 tok/s Runs great	~270 tok/s Runs great
Qwen3 8B About 7 GB	~95 tok/s Runs great	~130 tok/s Runs great
Gemma 3 12B About 10 GB	~61 tok/s Runs great	~83 tok/s Runs great
Qwen3 14B About 11 GB	~53 tok/s Runs great	~73 tok/s Runs great
gpt-oss 20B About 16 GB	~160 tok/s Runs great	~220 tok/s Runs great
Gemma 3 27B About 19 GB	~29 tok/s Usable	~40 tok/s Runs great
Qwen3 30B-A3B About 21 GB	~160 tok/s Runs great	~220 tok/s Runs great
Qwen3 32B About 22 GB	~25 tok/s Usable	~34 tok/s Runs great
Llama 3.3 70B About 45 GB	Needs 45 GB	Needs 45 GB
gpt-oss 120B About 67 GB	Needs 67 GB	Needs 67 GB
Qwen3 235B-A22B About 144 GB	Needs 144 GB	Needs 144 GB
DeepSeek R1 671B About 406 GB	Needs 406 GB	Needs 406 GB

Which should you get?

The Instinct MI100, for speed. They fit the same models. Price matters as much as either number: check used prices on the builds page, where every parts list has a measured or estimated speed.

How we estimate

Both columns use the same formula: memory bandwidth divided by the weights read per token, calibrated on our own rack, with Ollama's default 4-bit sizes. Real results are usually within about a third. Measured either one? Send us the number.

Compare

Other matchups

Every comparison