Nvidia Q RTX 5000 mieten.

Turing 16GB VRAM 265W

AI models

Models that run on this GPU.

Open-Weights-Modelle, die für Inferenz auf diese Karte passen, mit der kleinsten Kartenzahl und der Genauigkeit, mit der jedes geladen wird. Täglich aus der Modellgröße und dem Kartenspeicher berechnet. Die 60 größten von 107 passenden Modellen.

Renting for inference?

Pair these models with the cheapest provider on the rental table. See rental rates on the Overview tab →

FAQ

AI models on this GPU.

What AI models can I run on a Nvidia Q RTX 5000?

The list above shows open-weights models whose memory need fits this card: on one card when the model uses at least a quarter of its 16 GB, or on 2, 4 or 8 of them when it is too large for one. Each row gives the fewest cards and the precision (FP16, FP8 or INT4) that load it.

What's the VRAM minimum to run a model on the Nvidia Q RTX 5000?

We count the weights (parameters × bytes per weight: FP16 = 2, FP8 = 1, INT4 = 0.5), add the KV cache for an 8K-token context, and add 15% for the runtime. A model fits when that is within 90% of the cards' memory. A 70B model needs about 164 GB at FP16 and about 43 GB at INT4. A longer context needs more.

How does quantization (FP16 vs FP8 vs INT4) affect what fits?

Lower-precision quantization shrinks the memory footprint nearly linearly with the bit count. The trade-off is output quality: FP16 is the reference, FP8 is usually indistinguishable for most prompts, INT8 introduces small quality losses, INT4 is noticeably degraded on reasoning-heavy tasks but fine for chat. The badge on each row tells you which level the recommendation assumes.

Can I fine-tune on the Nvidia Q RTX 5000 or only do inference?

The rows here are for inference. Full fine-tuning needs several times more memory than inference at the same model size, because gradients, optimizer state and activations all live in memory. LoRA and QLoRA cut that a lot. The fine-tune cost tool sizes a run for you.

How are these GPU counts worked out?

We compute them every day from the model's parameter count and the card's memory, with the formula above. They are estimates, not tests we ran. Tokens per second, where shown, is an estimate from the card's memory bandwidth on one card.

Nvidia Q RTX 5000 mieten.

Daily median across providers.

All providers carrying this GPU.

Run these on this GPU.

Models that run on this GPU.

WizardLM-2 8x22B

OpenAI: gpt-oss-safeguard-20b

Baidu: ERNIE 4.5 21B A3B

Baidu: ERNIE 4.5 21B A3B Thinking

Reka Flash 3

GPT-OSS 20B

InternLM 2.5 20B

DeepSeek Coder V2 Lite

Qwen2.5 14B

Qwen3 14B Base

Cogito V1 Preview Qwen 14B

Qwen 3 14B

Qwen 2.5 14B

DeepSeek R1 Distill Qwen 14B

Phi-4

Deepcoder 14B Preview

Mistral: Ministral 3 14B 2512

Phi-3 Medium

Code Llama 13B

MythoMax 13B

LLaVA 13B

Baichuan2-13B

ReMM SLERP 13B

Mistral Nemo 12B

FLUX.1 Dev

FLUX.1 Schnell

TheDrummer: Rocinante 12B

Meta: Llama Guard 4 12B

TheDrummer: UnslopNemo 12B

Gemma 4 12B It

Google: Gemma 3 12B

Llama 3.2 11B Vision

Qwen: Qwen3.5-9B

Moondream: Moondream 3.1

Moondream: Moondream 3 Preview

Gemma 2 9B

NVIDIA: Nemotron Nano 9B V2

Mistral: Ministral 3 8B 2512

granite-4.2-8b

IBM: Granite 4.1 8B

Qwen: Qwen3 VL 8B Instruct

Qwen: Qwen3 VL 8B Thinking

LFM2.5-8B-A1B

EssentialAI: Rnj 1 Instruct

ByteDance: UI-TARS 7B

Qwen 3 8B

Stable Diffusion 3.5 Large

Tencent: Hy-MT2-7B

Molmo 7B D 0924

Meta: Llama 3 8B Instruct

Meta Llama 3.1 8B

Llama 3.1 8B

IBM Granite Code 8B

Cogito V1 Preview Llama 8B

Qwen3 8B Base

NousResearch: Hermes 2 Pro - Llama-3 8B

AionLabs: Aion-RP 1.0 (8B)

Hermes 3 8B

Llama Guard 3 8B

Sao10K: Llama 3 8B Lunaris

AI models on this GPU.