Comparison

48 GB GPUs compared: RTX A6000 vs L40S vs RTX 6000 Ada.

RTX A6000, A40, L40, L40S and RTX 6000 Ada compared for 70B at 4-bit and 30B at FP8: Ampere vs Ada, FP8 support, speed estimates and live prices.

By Vi Nguyen · Published October 16, 2026 · Prices on this page are live · Editorial policy

48 GB is the most useful memory size in the rental market that most people overlook. It is the smallest single card that holds a 70B model at 4-bit with real room for context, and it runs 30B-class models at FP8 without any quantization gymnastics. Five cards share that capacity: the RTX A6000, the A40, the L40, the L40S and the RTX 6000 Ada. They look interchangeable on a price list, but they span two architectures and two market segments, and the differences matter for both speed and cost per token. This guide explains those differences, works through the memory math, and tells you which one to rent for which job.

Two generations, two market segments

The five cards split cleanly along two axes. The first is architecture: the RTX A6000 and A40 are Ampere (the same generation as the A100 and RTX 3090), while the L40, L40S and RTX 6000 Ada are Ada Lovelace (the RTX 4090's generation). The second is positioning: the RTX A6000 and RTX 6000 Ada are workstation cards with their own fans, built to sit in a tower under a desk, while the A40, L40 and L40S are passively cooled datacenter cards that rely on server chassis airflow.

For renters, the workstation-versus-datacenter split matters less than it would for a buyer. What it tells you is where the card is likely hosted. Datacenter parts like the L40S tend to appear on managed clouds in multi-GPU servers; workstation parts like the RTX A6000 show up more often on host-supplied marketplaces, sometimes in machines built from individual towers. That affects the CPU, disk, networking and reliability you get alongside the GPU more than the GPU itself.

The L40 and L40S share a memory system. The L40S is the version tuned for AI, with a higher power limit and considerably more tensor throughput, which is why it is the one you will see marketed for inference and fine-tuning. The plain L40 was aimed more at graphics and virtual workstations.

What Ada adds: FP8 and more compute

Ada's fourth-generation tensor cores add native FP8, which Ampere lacks. That matters in two ways. First, FP8 weights are half the size of FP16, so a 30B model drops from about 60 GB, which does not fit, to about 30 GB, which fits with room to spare, and FP8 usually costs much less quality than 4-bit. On Ampere you can still store weights in INT8 or 4-bit, but the kernels are more fragmented and FP8 checkpoints need converting. Second, Ada has substantially more tensor throughput per card, which shows up in prompt processing, batched serving, fine-tuning and image generation.

What Ada does not add is much bandwidth. The RTX 6000 Ada has 25 percent more memory bandwidth than the RTX A6000, and the L40S about 12 percent more. For single-user token generation, which is bandwidth-bound, the generational gap is therefore modest. For anything compute-bound it is large. Ampere's one structural advantage is the NVLink bridge: two A6000s or two A40s can be linked for faster tensor parallelism, while Ada cards talk over PCIe only.

The memory math: why 48 GB is the sweet spot

The site's rule of thumb: weights in GB ≈ parameters in billions × bytes per parameter (2 for FP16, 1 for FP8 or INT8, about 0.5 for 4-bit), plus KV cache, plus 10 to 20 percent runtime overhead. The VRAM fit calculator does it per model.

Worked example: Llama 3.3 70B at 4-bit on one 48 GB card

1. Weights: 70B × about 0.55 bytes (real 4-bit formats carry scales) ≈ 38.5 GB.
2. Runtime overhead: about 2 to 3 GB.
3. Left for KV cache: 48 − 38.5 − 2.5 ≈ 7 GB.
4. KV cache per token for Llama 3 70B (80 layers, 8 KV heads, 128-dimension heads) at FP16: 2 × 80 × 8 × 128 × 2 bytes ≈ 0.33 MB.
5. Context capacity: 7 GB ÷ 0.33 MB ≈ 21,000 tokens in total across all concurrent requests. Quantize the KV cache to 8-bit and that roughly doubles to about 42,000.

That is enough for one user with long documents, or a handful of users with normal chat lengths. It is the cheapest single-GPU way to serve a 70B model at quality most people accept. On a 24 GB card the same model does not fit at all, and on an 80 GB card you pay datacenter prices for headroom you may not need.

The other half of the sweet spot is 30B-class models at 8-bit. A 32B model at FP8 is about 32 GB of weights; on a 48 GB card that leaves roughly 12 GB for KV cache after overhead. With the Qwen-style 32B cache footprint of about 0.26 MB per token, that is around 45,000 tokens of context. Note that a 30B model at full FP16 is about 60 GB and does not fit in 48 GB; for FP16 the practical limit is models around 20B.

Estimated speed and cost per token

For single-stream generation, a useful upper bound is tokens/sec ≈ memory bandwidth ÷ bytes of weights read per token. For the 70B at 4-bit (about 38.5 GB read per token), the estimated ceilings are:

• A40: 696 ÷ 38.5 ≈ 18 tokens/sec
• RTX A6000: 768 ÷ 38.5 ≈ 20 tokens/sec
• L40 and L40S: 864 ÷ 38.5 ≈ 22 tokens/sec
• RTX 6000 Ada: 960 ÷ 38.5 ≈ 25 tokens/sec

These are derived from spec-sheet bandwidth, not measured; real numbers land below them. The point is the spread: for single-user decode on a 70B, the fastest 48 GB card is only about 40 percent quicker than the slowest. If you are serving one user, the cheapest card per hour is probably the cheapest per token. The picture changes with batching and long prompts, where Ada's extra compute lets it process many requests per step and ingest long prompts much faster.

To compare on cost, use $ per 1M tokens = hourly price ÷ (tokens/sec × 3,600) × 1,000,000. With hypothetical numbers: an A6000 at $0.50/hr delivering 16 tokens/sec produces 57,600 tokens an hour, so $0.50 ÷ 0.0576 = $8.68 per 1M tokens for a single stream. An L40S at a hypothetical $0.90/hr that batches eight users at an aggregate 100 tokens/sec produces 360,000 tokens an hour, for $0.90 ÷ 0.36 = $2.50 per 1M tokens. Utilization, not the card, is the biggest lever on cost per token; a single-user endpoint is expensive per token on any GPU, which is why the self-host vs API calculator is worth a look before you commit.

Live prices for 48 GB cards

Prices for these cards vary a lot by provider and change daily. The table below comes from current listings:

GPU	VRAM	Cheapest now	Where	Median across providers	Providers
Nvidia RTX A6000	48GB	$0.32/hr	Nosana	$0.54/hr	11
Nvidia L40S	48GB	$0.55/hr	Novita AI	$0.97/hr	11
Nvidia L40	48GB	$0.36/hr	Vast.ai	$0.94/hr	8
Nvidia RTX 6000 Ada	48GB	$0.56/hr	Nosana	$0.78/hr	8
NVIDIA A40	48GB	$0.32/hr	Nosana	$2.05/hr	5

Live data: median hourly price per GPU, collected in the last 24 hours. How we collect prices.

At the cheapest current A6000 listing, running one card around the clock for a month (730 hours) would cost about $232. That is the number to hold against a hosted 70B API bill.

A6000s and A40s show up frequently on marketplaces like Vast.ai, where reliability varies by host. L40S capacity is more common on managed clouds; RunPod carries several 48 GB cards across its Community and Secure clouds. For a wider view of trade-offs between those two kinds of provider, see GPU marketplaces vs managed clouds.

Decision framework

• One user, 70B at 4-bit, cost matters most: take the cheapest of the RTX A6000 or A40. Decode is bandwidth-bound and the Ada cards are at most about a quarter faster.
• You want FP8 (30B-class models at near-full quality, or FP8 checkpoints as released): L40S or RTX 6000 Ada. Ampere cards do not run FP8 natively.
• Serving several users, or long-prompt workloads like RAG: L40S. Its tensor throughput makes batched prefill and decode far faster, which usually wins on cost per token even at a higher hourly rate.
• Fine-tuning with LoRA or QLoRA: L40S or RTX 6000 Ada for speed. See what it costs to fine-tune an LLM with QLoRA.
• Image and video generation: Ada, for the compute. Our Stable Diffusion and Flux GPU guide covers it.
• You need two cards linked for a larger model: a bridged pair of A6000s or A40s, if the host confirms the NVLink bridge is installed. Many are not.

The step up: 96 GB RTX PRO 6000 Blackwell

If 48 GB is not enough, the next single-card size is the RTX PRO 6000 Blackwell, with 96 GB of GDDR7 and roughly 1.8 TB/s of bandwidth, nearly double the RTX 6000 Ada. It also adds native FP4. With 96 GB a 70B model runs at FP8 on one card with about 20 GB left for cache, or at 4-bit with enough headroom to batch seriously, and 100B-class MoE models become single-GPU jobs. Using the same bandwidth estimate, its single-stream ceiling for the 70B at 4-bit is about 1,800 ÷ 38.5 ≈ 47 tokens/sec, nearly twice the 48 GB cards.

It is listed under several names depending on the variant (workstation, Max-Q and server edition), so compare variants carefully. If it rents for less than twice a 48 GB Ada card, it usually beats running two of them over PCIe, since one card with twice the memory avoids the communication overhead of splitting a model. For how it compares with datacenter parts, see H100 vs H200 vs B200 for inference, and for sizing other models, how much VRAM you need to run LLMs.

FAQ