Can the A100 run gpt-oss 120B?
Yes, with 80 GB or more. gpt-oss 120B needs about 67 GB and the A100 has 80 GB of VRAM. Expect about 190 tok/s, faster than you can read. The 40 GB version doesn't have room.
ollama run gpt-oss:120bFirst time? Install Ollama, then paste that into a terminal.
gpt-oss 120B needsAbout 67 GB
A100 has40 or 80 GB VRAM
Bandwidth1555 GB/s
Estimated speed~190 tok/s
By memory size
Each A100 version
- 40 GBToo big
- 80 GBRuns great ~190 tok/s
Want it faster?
Speed comes from memory bandwidth: the card reads all 2.9 GB of active weights for every token. On the A100, Qwen3 4B runs at about 340 tok/s if you can trade some quality for speed. Cards that run gpt-oss 120B faster:
How we estimate
Speed is roughly memory bandwidth divided by the weights read per token, calibrated on our own rack. Sizes are Ollama's default 4-bit downloads plus about 1.2 GB for context. Real results are usually within about a third. Measured it yourself? Send us the number.
Keep going