估算模型在每款可租 GPU 上的生成速度和每百万 token 成本，并与最便宜的托管 API 对比。
在能装下模型的最少 GPU 上、8 个并发请求时的估算总吞吐量。
| GPU | GPU 数 | $/小时（全部 GPU） | token/秒 | $ / 百万 token | 对比最便宜 API |
|---|---|---|---|---|---|
| NVIDIA Tesla T4 | 6 | $0.11/hr | 154 | $0.204 | 便宜 36% |
| Nvidia GeForce RTX 3060 | 8 | $0.31/hr | 230 | $0.368 | 贵 1.2 倍 |
| Nvidia GeForce RTX 4070 | 8 | $0.61/hr | 323 | $0.526 | 贵 1.6 倍 |
| Nvidia GeForce RTX 3080Ti | 8 | $1.14/hr | 584 | $0.541 | 贵 1.7 倍 |
| Nvidia GeForce RTX 3090 | 4 | $0.61/hr | 300 | $0.566 | 贵 1.8 倍 |
| Nvidia GeForce RTX 5080 | 6 | $0.95/hr | 461 | $0.575 | 贵 1.8 倍 |
| Nvidia GeForce RTX 4080 | 6 | $0.76/hr | 353 | $0.601 | 贵 1.9 倍 |
| Nvidia RTX 5070 | 8 | $0.96/hr | 430 | $0.618 | 贵 1.9 倍 |
| Nvidia Q RTX 5000 | 6 | $0.53/hr | 215 | $0.689 | 贵 2.2 倍 |
| Nvidia RTX 4070 SUPER | 8 | $0.86/hr | 323 | $0.745 | 贵 2.3 倍 |
| Nvidia RTX A4000 | 6 | $0.61/hr | 215 | $0.790 | 贵 2.5 倍 |
| Nvidia A30 | 4 | $0.88/hr | 299 | $0.819 | 贵 2.6 倍 |
| Nvidia H100 NVL | 1 | $0.94/hr | 312 | $0.835 | 贵 2.6 倍 |
| Nvidia RTX 5070 Ti | 6 | $1.29/hr | 430 | $0.836 | 贵 2.6 倍 |
| Nvidia GeForce RTX 5090 | 3 | $1.36/hr | 430 | $0.881 | 贵 2.8 倍 |
| Nvidia GeForce RTX 3090 Ti | 4 | $1.08/hr | 323 | $0.930 | 贵 2.9 倍 |
| Nvidia RTX 5060 Ti | 6 | $0.74/hr | 215 | $0.956 | 贵 3.0 倍 |
| Nvidia RTX 4070 Ti SUPER | 6 | $1.13/hr | 323 | $0.976 | 贵 3.1 倍 |
| Nvidia RTX 4070 Ti | 8 | $1.13/hr | 323 | $0.976 | 贵 3.1 倍 |
| Nvidia RTX 4090 Laptop GPU | 6 | $0.99/hr | 276 | $0.992 | 贵 3.1 倍 |
| Nvidia GeForce RTX 4090 | 4 | $1.20/hr | 323 | $1.03 | 贵 3.2 倍 |
| Nvidia Titan RTX | 4 | $0.81/hr | 215 | $1.04 | 贵 3.3 倍 |
| Nvidia A100 PCIe | 3 | $1.46/hr | 373 | $1.08 | 贵 3.4 倍 |
| Nvidia RTX PRO 4500 Blackwell | 3 | $0.84/hr | 215 | $1.09 | 贵 3.4 倍 |
| Nvidia A100 SXM4 | 2 | $1.43/hr | 326 | $1.22 | 贵 3.8 倍 |
| Nvidia RTX 4090 D | 4 | $1.48/hr | 323 | $1.27 | 贵 4.0 倍 |
| Nvidia Q RTX 8000 | 2 | $0.51/hr | 108 | $1.32 | 贵 4.1 倍 |
| Nvidia GeForce RTX 4080 Super | 6 | $1.68/hr | 353 | $1.32 | 贵 4.1 倍 |
| Nvidia A10 | 4 | $0.97/hr | 192 | $1.40 | 贵 4.4 倍 |
| Nvidia RTX PRO 4500 | 3 | $1.10/hr | 215 | $1.42 | 贵 4.4 倍 |
| Nvidia RTX A4500 | 5 | $0.92/hr | 179 | $1.43 | 贵 4.5 倍 |
| Nvidia RTX A6000 | 2 | $0.64/hr | 123 | $1.44 | 贵 4.5 倍 |
| Nvidia RTX PRO 4000 | 4 | $1.13/hr | 215 | $1.46 | 贵 4.6 倍 |
| Intel Gaudi2 | 1 | $1.08/hr | 196 | $1.53 | 贵 4.8 倍 |
| AMD MI300X | 1 | $2.39/hr | 424 | $1.57 | 贵 4.9 倍 |
| NVIDIA A40 | 2 | $0.64/hr | 111 | $1.59 | 贵 5.0 倍 |
| Nvidia A100 40GB | 3 | $2.37/hr | 373 | $1.76 | 贵 5.5 倍 |
| AMD MI325 | 1 | $3.18/hr | 480 | $1.84 | 贵 5.8 倍 |
| Nvidia RTX PRO 6000 Blackwell Max-Q Workstation | 1 | $0.96/hr | 143 | $1.86 | 贵 5.8 倍 |
| Nvidia RTX PRO 5000 | 2 | $1.47/hr | 215 | $1.90 | 贵 5.9 倍 |
| Nvidia RTX 4000 Ada | 5 | $1.00/hr | 144 | $1.93 | 贵 6.0 倍 |
| Nvidia A100 80GB PCIe | 2 | $2.18/hr | 310 | $1.96 | 贵 6.1 倍 |
| Nvidia RTX 6000 Ada | 2 | $1.11/hr | 154 | $2.01 | 贵 6.3 倍 |
| Nvidia RTX PRO 6000 Blackwell Workstation | 1 | $1.08/hr | 143 | $2.10 | 贵 6.6 倍 |
| Nvidia RTX PRO 5000 Blackwell | 2 | $1.65/hr | 215 | $2.13 | 贵 6.7 倍 |
| Nvidia H100 SXM | 2 | $4.18/hr | 536 | $2.17 | 贵 6.8 倍 |
| Nvidia B200 | 1 | $4.89/hr | 616 | $2.21 | 贵 6.9 倍 |
| Nvidia RTX 2000 Ada Generation | 6 | $0.86/hr | 108 | $2.22 | 贵 6.9 倍 |
| Nvidia A100 SXM 40GB | 3 | $3.00/hr | 373 | $2.23 | 贵 7.0 倍 |
| Nvidia GH200 | 1 | $2.59/hr | 320 | $2.25 | 贵 7.0 倍 |
| Nvidia RTX 5000 Ada | 3 | $1.29/hr | 138 | $2.58 | 贵 8.1 倍 |
| Nvidia H200 SXM | 1 | $3.59/hr | 384 | $2.60 | 贵 8.1 倍 |
| Nvidia RTX PRO 6000 MaxQ | 1 | $1.39/hr | 143 | $2.69 | 贵 8.4 倍 |
| Nvidia H200 NVL | 1 | $3.79/hr | 384 | $2.74 | 贵 8.6 倍 |
| Nvidia L40 | 2 | $1.38/hr | 138 | $2.77 | 贵 8.7 倍 |
| Nvidia RTX PRO 6000 WS | 1 | $1.44/hr | 143 | $2.79 | 贵 8.7 倍 |
| Nvidia B300 | 1 | $6.73/hr | 616 | $3.03 | 贵 9.5 倍 |
| Nvidia H200 | 1 | $4.20/hr | 384 | $3.04 | 贵 9.5 倍 |
| Nvidia RTX 5880 Ada | 2 | $1.69/hr | 154 | $3.05 | 贵 9.5 倍 |
| Nvidia RTX PRO 4500 SE | 3 | $2.16/hr | 192 | $3.13 | 贵 9.8 倍 |
| Nvidia L40S | 2 | $1.58/hr | 138 | $3.17 | 贵 9.9 倍 |
| Nvidia RTX PRO 6000 | 1 | $1.69/hr | 143 | $3.27 | 贵 10.2 倍 |
| Nvidia RTX PRO 6000 WK | 1 | $1.69/hr | 143 | $3.27 | 贵 10.2 倍 |
| Nvidia RTX 6000 Ada Generation | 2 | $1.82/hr | 154 | $3.29 | 贵 10.3 倍 |
| Nvidia H100 | 2 | $6.40/hr | 536 | $3.32 | 贵 10.4 倍 |
| Nvidia RTX PRO 6000 Blackwell Server Edition | 1 | $1.58/hr | 128 | $3.44 | 贵 10.7 倍 |
| Nvidia RTX PRO 6000 Blackwell | 1 | $1.79/hr | 143 | $3.47 | 贵 10.8 倍 |
| Nvidia RTX PRO 6000 S | 1 | $1.69/hr | 128 | $3.67 | 贵 11.5 倍 |
| Nvidia RTX 2000 Ada | 6 | $1.44/hr | 108 | $3.72 | 贵 11.6 倍 |
| Nvidia L4 | 4 | $1.40/hr | 96 | $4.05 | 贵 12.6 倍 |
| Nvidia B300 SXM6 PC | 1 | $12.67/hr | 616 | $5.71 | 贵 17.9 倍 |
基于规格参数和公开公式估算，按今日各 GPU 最便宜的在售报价计价。实际结果因软件、设置和服务商而异。
生成一个 token 需要从显存中把每个活跃权重读一次,所以单请求速度由显存带宽决定:每秒 token ≈ 带宽 ÷ 活跃权重字节数,我们假设实际内核能达到标称值的约 70%。同时服务多个请求时,每次权重读取可供整批复用,所以吞吐量随并发增长 — 直到 GPU 算力耗尽,我们把上限设为其稠密 BF16 峰值的 40%(每个活跃参数每 token 2 FLOP)。
内存则不同:混合专家(MoE)模型必须加载全部专家,所以按总参数量计算,再加上你的上下文长度和并发所需的 KV 缓存。一张卡不够时,我们使用能放下的最少 GPU 数(最多一个 8 卡节点),并按整个节点计价。
每百万 token 成本 = 小时价 ÷ (每秒 token × 3,600) × 1,000,000。对比列使用同一模型最便宜托管 API 的输出 token 价格。自部署只有在 GPU 保持忙碌时才能达到这些数字:利用率 20% 时,成本要乘以五。自部署还是 API 工具和盈亏平衡指南详细讨论了利用率。
