Guide

Which GPUs can run DeepSeek, Qwen and other MoE models?

Total parameters set memory, active parameters set speed. VRAM math for DeepSeek R1, Qwen3.5, Kimi K2 and GPT-OSS, which nodes fit, and API vs self-hosting.

By Vi Nguyen · Published October 30, 2026 · Prices on this page are live · Editorial policy

The strongest open-weights models are now mostly mixture-of-experts (MoE) designs: DeepSeek R1 and V3, Qwen 3.5, Kimi K2, GPT-OSS. They are advertised with two parameter counts, and people often size hardware from the wrong one. The rule is simple once you see it: total parameters decide how much memory you need; active parameters decide how much compute and bandwidth each token costs. This guide applies that rule to the popular MoE models, works out the memory at FP8 and 4-bit, explains why most of them need a full multi-GPU node, and gives you a way to decide between self-hosting and paying for an API.

Total parameters versus active parameters

A dense model uses every weight for every token. An MoE model replaces most of its feed-forward layers with a set of smaller "expert" networks plus a router. For each token, the router picks a few experts per layer and the rest sit idle. A model described as 397B-A17B has 397 billion parameters in total but only about 17 billion take part in any one token.

That split creates an asymmetry. All the experts must be resident in GPU memory, because the next token might route to any of them, so memory scales with the total. But each token only reads and multiplies the active weights, so per-token compute and, at small batch sizes, per-token memory traffic scale with the active count. In effect, an MoE model has the memory bill of a giant model and the per-token speed of a much smaller one.

There is one catch at scale. With a large batch, different tokens route to different experts, so in each forward step the GPUs end up touching most of the experts anyway. The per-token savings are real, but the "only 17B is read" picture is most accurate at low concurrency.

Memory math for the popular MoE models

Using the site's rule of thumb, weights in GB ≈ total parameters in billions × bytes per parameter (1 for FP8, about 0.5 for 4-bit), before KV cache and 10 to 20 percent runtime overhead:

The model pages for Kimi K2, Qwen3.5 397B-A17B, GPT-OSS-120B and Qwen3-Next-80B-A3B carry per-model GPU recommendations.

Worked example: DeepSeek R1 at FP8 on one node

1. Weights: 671B × 1 byte = 671 GB. DeepSeek released V3 and R1 with FP8 weights, so FP8 is the natural serving precision, not an aggressive quantization.
2. Overhead at 10 to 15 percent: about 70 to 100 GB. Running total: roughly 750 to 770 GB before KV cache.
3. 8× H100 at 80 GB = 640 GB. Does not fit.
4. 8× H200 at 141 GB = 1,128 GB. Fits, with roughly 350 GB for KV cache.
5. 8× B200 or 8× MI300X at 192 GB = 1,536 GB. Fits, with roughly 750 GB for KV cache and room for large batches.

DeepSeek's multi-head latent attention compresses the KV cache far below what a conventional 671B model would need, so that headroom goes a long way. At 4-bit (about 336 GB of weights) the model fits on 8× H100 or 4× H200, at some cost in quality.

Apply the same arithmetic to Kimi K2: about 1,000 GB at FP8 barely squeezes onto 8× H200 with almost nothing left for cache, so a 192 GB-per-GPU node is the realistic target, or a 4-bit build on H200s. Qwen3.5 397B-A17B at FP8 (about 400 GB plus overhead) fits on 4× H200, 4× MI300X, 4× B200 or 8× H100. The two smaller models are different in kind: GPT-OSS-120B was published with its expert weights in a 4-bit format (MXFP4) and was designed to fit a single 80 GB GPU, and Qwen3-Next-80B-A3B at 4-bit is about 40 GB, so one 48 GB or 80 GB card handles either.

Why the big ones need multi-GPU nodes

No single GPU on the market holds 400 GB to 1 TB, so the large MoE models are always split across GPUs. The question is how well the split performs, and that is mostly about interconnect.

MoE models are typically served with a combination of tensor parallelism (splitting each layer across GPUs) and expert parallelism (placing different experts on different GPUs). Expert parallelism means every token's hidden state is sent to whichever GPU holds its chosen experts and back again, on every MoE layer. That all-to-all traffic is heavy. Inside an 8-GPU SXM node with NVLink and NVSwitch it is manageable; over PCIe or between separate machines on ordinary networking it becomes the bottleneck. That is why the practical unit for these models is one well-connected node, not "eight GPUs from anywhere".

The per-GPU memory size then decides how comfortable that node is. Here the AMD MI300X is worth a look: it has 192 GB of HBM3 and about 5.3 TB/s of bandwidth per GPU, so an 8-GPU node has the same 1.5 TB of memory as a B200 node. The trade-off is software. vLLM and SGLang run on ROCm and support DeepSeek on MI300X, but kernel maturity and the range of supported quantization formats still trail CUDA, so test your exact model and engine version before committing. The H200 is the most widely deployed option that fits DeepSeek at FP8, and the B200 offers the most bandwidth and memory, plus native FP4.

GPU	VRAM	Cheapest now	Where	Median across providers	Providers
AMD MI300X	192GB	$0.50/hr	RunPod	$0.50/hr	1
Nvidia H200	141GB	$3.58/hr	Lium	$5.64/hr	4
Nvidia B200	192GB	$4.89/hr	DeepInfra	$6.69/hr	8

Live data: median hourly price per GPU, collected in the last 24 hours. How we collect prices.

At the cheapest current H200 listing, an eight-GPU node would cost about $28.60 an hour, or roughly $20,878 a month running around the clock. Cheap single-GPU listings are not always available as full 8-GPU NVLink nodes, so the real node price may be higher.

Speed: what active parameters buy you

Because each token only touches the active weights, MoE models decode much faster than their total size suggests. A rough single-stream ceiling is tokens/sec ≈ memory bandwidth ÷ bytes of weights read per token. For DeepSeek R1 at FP8 that is about 37 GB read per token, versus 671 GB if it were dense: roughly 18 times less memory traffic per token. On one H200 (4.8 TB/s), 4,800 ÷ 37 ≈ 130 tokens/sec would be the theoretical ceiling if all active weights sat on one GPU; in a real 8-GPU deployment the communication between GPUs cuts into that, so treat it as an upper bound, not an estimate of what you will see.

The same logic explains why Qwen3-Next-80B-A3B feels fast on modest hardware. At 4-bit it reads only about 1.5 GB of weights per token, so even a 48 GB workstation card at under 1 TB/s has a theoretical ceiling in the hundreds of tokens per second. Small-active MoE models are the best fit for single-GPU renters: you get a large model's knowledge with a small model's speed, as long as the total fits in memory.

API versus self-hosting

For the giant MoE models, hosted APIs are hard to beat unless your volume is very high, because providers batch thousands of users onto the same nodes. The DeepSeek R1 model page tracks what each hosted provider charges; the current per-token prices we have are listed here.

Provider	Input $/1M tokens	Output $/1M tokens
DeepSeek API	$0.55	$2.19
OpenRouter	$0.70	$2.50

Refreshed daily from provider catalogs. All DeepSeek R1 options.

To compare, compute the break-even volume: monthly node cost ÷ blended API price per 1M tokens = millions of tokens per month at which self-hosting costs the same. A worked example with hypothetical round numbers:

1. Suppose an 8× H200 node rents at a hypothetical $25/hr. Monthly cost: $25 × 730 = $18,250.
2. Suppose the API costs a hypothetical blended $2.00 per 1M tokens (input and output weighted by your mix).
3. Break-even: $18,250 ÷ $2.00 = 9,125M tokens a month, about 9.1 billion.
4. Spread over a month (730 × 3,600 = 2.63M seconds), that is about 3,470 tokens/sec, sustained 24 hours a day.

If your traffic is below that, and most is, the API is cheaper. Self-hosting starts to make sense when you have sustained, predictable high volume; when data cannot leave your environment; when you need a fine-tuned variant no API serves; or when you need latency or rate limits the APIs will not guarantee. The self-host vs API calculator runs this with live prices, the API pricing tool compares providers, and self-host vs API break-even goes deeper on utilization.

Decision framework

• Low or bursty volume on DeepSeek, Kimi K2 or Qwen3.5 397B: use an API. A node that sits idle most of the day costs more per token than any hosted price.
• High sustained volume on DeepSeek R1/V3 at FP8: 8× H200 as the widely available choice; 8× B200 or 8× MI300X if you want batching headroom or long contexts. 8× H100 only at 4-bit.
• Kimi K2 at around 1T parameters: a 192 GB-per-GPU node (B200 or MI300X) for FP8; H200 only with a 4-bit build.
• Qwen3.5 397B-A17B: half a node is enough: 4× H200, 4× B200, 4× MI300X, or 8× H100 at FP8.
• GPT-OSS-120B: one 80 GB GPU such as an H100, or an H200 for more concurrency. See H100 vs H200 vs B200 for inference.
• Qwen3-Next-80B-A3B: one 48 GB card at 4-bit, or one 80 GB+ card at FP8. Our 48 GB GPU comparison covers the options.

Whatever you pick, check the full node before you rent: that all eight GPUs are on NVLink or Infinity Fabric within one machine, that local disk is large enough for a 700 GB to 1 TB checkpoint, and that network speed will not turn the model download into hours of paid idle time. The hidden costs of renting GPUs covers those checks, and the run-it-yourself page for DeepSeek R1 lists hardware that fits.

FAQ