Guide

Hidden costs of renting GPUs beyond the hourly price.

Storage, egress, idle time, billing increments, slow hosts and model downloads can multiply a GPU bill. A worked fine-tuning example and a checklist to cut it.

Par Vi Nguyen · Publié le 23 octobre 2026 · Les prix de cette page sont en direct · Politique éditoriale

The hourly price is the number everyone compares, and it is usually the smallest surprise on a GPU rental bill. What actually inflates the invoice is everything around the compute: disks that keep billing after the GPU is stopped, data that costs money to move out, machines left running overnight, billing that rounds up, and the hour you spend waiting for a slow host to download a model you have already downloaded three times this week. None of these items is large on its own. Together they routinely turn a job you budgeted at $10 into one that costs $40 or $50. This guide goes through each hidden cost, shows how to size it, adds them up for a realistic fine-tuning job, and ends with a checklist for keeping the bill close to the sticker price.

Storage: the meter that keeps running

Most providers bill storage separately from compute, and storage keeps billing when the GPU does not. The details differ, but the patterns repeat:

• Persistent volumes. Network volumes that survive instance termination are billed per GB per month whether or not anything is attached. That is exactly what they are for, and exactly why they are easy to forget. A 500 GB volume you created for one experiment in March is still billing in September.
• Stopped instances. On many platforms, stopping (as opposed to terminating) an instance releases the GPU but keeps its disk, and the disk keeps charging. On some marketplaces the stopped-disk rate is set by the host and can be noticeably higher than a hyperscaler's block storage price. Check it before you stop a box "for later".
• Oversized root disks. Some templates default to large disks. If the disk is billed by provisioned size rather than used size, 200 GB provisioned for a 30 GB environment costs you the full 200 GB.
• Snapshots and images. Custom images and disk snapshots are storage too, and they accumulate quietly.

The sizing math is simple: GB × price per GB-month × months. At a hypothetical $0.10 per GB-month, a 1 TB volume holding a few model checkpoints costs $100 a month, which on a mid-range card can equal dozens of GPU-hours. Model files are the usual culprit. A 70B model in FP16 is about 70 × 2 = 140 GB of weights, so keeping a couple of variants and a few fine-tuned checkpoints of it on persistent storage adds up fast.

Egress and data transfer

Moving data into a cloud is almost always free. Moving it out often is not. The hyperscalers (AWS, Google Cloud, Azure) charge per GB for internet egress beyond small free allowances, and cross-region transfer is billed too. Many GPU-focused clouds and marketplaces advertise free or cheap egress, but it is worth confirming on the provider's current pricing page rather than assuming, and on a marketplace the practical limit is often the host's upload bandwidth rather than a fee.

Egress bites in three situations. The first is pulling large artifacts back: a merged fine-tuned model, full-precision checkpoints or a generated dataset. The second is a split architecture, where your data lives with one provider and your GPUs with another, so every epoch or every batch job reads across the internet. The third is serving: if an inference endpoint returns images or long outputs to users, the response bytes are egress.

The fix is mostly architectural. Keep training data in the same region or provider as the GPUs. Download LoRA adapters (megabytes) rather than merged models (gigabytes) when you can merge locally. And if you are serving from a hyperscaler, estimate response volume × request count before you are surprised by it.

Idle time, forgotten instances and billing increments

The biggest single hidden cost for individuals and small teams is simply paying for a GPU that is doing nothing. It happens in three ways:

• Idle during work. You spin up a box, then spend 40 minutes reading logs, editing a config or in a meeting while it sits at 0% utilization. On interactive work, utilization below 50% is common.
• Waiting between steps. A training run finishes at 3 a.m.; you evaluate it at 9 a.m. The box idled for six hours.
• Forgotten instances. The classic: the job ended Friday, the instance did not. A weekend is about 60 hours.

To put a live number on the last one: at the cheapest provider's current median RTX 4090 price, a forgotten 60-hour weekend costs about $18 per card, and a forgotten month (730 hours) about $219. On an 8-GPU H100 node the same mistake is an order of magnitude larger.

Minimum billing increments compound the problem for short, frequent jobs. Some providers bill per second, some per minute, and some per started hour; some have a minimum charge per launch. The hyperscalers bill many Linux instances per second with a short minimum, but it varies by product and operating system. If you run a 10-minute job twenty times a day on a provider that bills per started hour, you pay for 20 hours to get 3.3 hours of work. Per-second billing makes short jobs cheap; per-hour billing means you should batch them.

Setup time, slow hosts and model downloads

Every instance you launch spends some time before it does anything useful, and you pay for all of it.

Boot and environment setup

Container pull, driver checks, pip install of a large dependency tree, compiling a CUDA extension: on a fresh machine this can take anywhere from two minutes to half an hour. If you launch ten instances a week and each spends 20 minutes on setup, that is more than three GPU-hours a week of pure overhead. Baking dependencies into a custom image turns most of it into a single image pull.

Model and dataset downloads

Download time is size ÷ bandwidth. A 16 GB model (an 8B-parameter model in FP16 is about 8 × 2 = 16 GB) at an effective 100 MB/s takes under three minutes. The same file at 10 MB/s, not unusual on a residential marketplace host or a rate-limited model hub, takes about 27 minutes. A 140 GB 70B checkpoint at 10 MB/s takes nearly four hours. On a multi-GPU box, every one of those minutes is billed for every GPU. Keep weights on a persistent volume in the same region, or cache them in object storage close to the compute.

Failed or slow hosts

On marketplaces in particular, some fraction of machines will not work as advertised: a GPU that throttles under load, a PCIe link running at reduced width, a disk far slower than listed, or a machine that simply drops offline. You pay for the time it takes to discover this. A short throughput test right after launch (a few hundred steps compared against a known-good baseline) catches most of these in minutes instead of hours. Our guide to marketplaces vs managed clouds covers host vetting in detail.

Multi-GPU interconnect surprises

Renting four or eight GPUs does not mean you get four or eight times the throughput. How the GPUs talk to each other decides that, and listings do not always make it obvious.

• PCIe vs NVLink. SXM datacenter parts connect over NVLink: roughly 600 GB/s per GPU on the A100 SXM and 900 GB/s on the H100 SXM. PCIe 4.0 x16 is about 32 GB/s in each direction. Consumer cards like the RTX 4090 have no NVLink at all, so multi-GPU communication goes over PCIe.
• Lane width. Some multi-GPU chassis, especially ones built from repurposed mining rigs, give each card fewer PCIe lanes than x16, sometimes far fewer. That barely matters for independent single-GPU jobs and matters a lot for tensor parallelism.
• PCIe vs SXM variants of the same chip. An A100 80GB PCIe and an A100 SXM share a name but not an interconnect. Check which one a listing actually offers.

The hidden cost here is scaling efficiency. If an 8-GPU job only runs 4.5 times faster than one GPU because it is communication-bound, you are paying for 8 GPUs and getting 4.5 GPUs of work: an effective 78% premium on every hour. For jobs that fit on one card, run one card. For jobs that do not, pay for the interconnect.

Worked example: the true cost of a fine-tuning job

Here is a realistic accounting for a LoRA fine-tune of an 8B model on a single GPU. All prices are round hypothetical numbers chosen to make the arithmetic easy, not current market rates, and every overhead is a plausible amount rather than a measured one. The GPU costs $1.00/hr, the training itself takes 10 hours, persistent storage is $0.10 per GB-month and egress is $0.09 per GB (a hyperscaler-style rate; many GPU clouds charge less or nothing).

Step by step: the compute you budgeted for is $10.00. The launch overheads (setup, download, bad host) add $1.75. The failed run and the idle debugging add $3.50. The overnight mistake adds $12.00. Storage and egress, which never touched the GPU, add $24.14. The total is $10.00 + $1.75 + $3.50 + $12.00 + $24.14 = $51.39, or 5.1 times the sticker price.

Remove only the two avoidable mistakes (the overnight instance and the forgotten volume, deleted after a week instead: 200 × $0.10 × 0.25 ≈ $5.00) and the total falls to $51.39 − $12.00 − $15.00 = $24.39. Download the LoRA adapter instead of the merged model and checkpoints, and egress drops to pennies, bringing the job to about $20. That is still twice the headline, and it is a fairly disciplined run. Budget for it: a reasonable planning figure for small jobs is 1.5 to 2 times the pure compute cost. For sizing the compute line itself, see our QLoRA fine-tuning cost guide.

Checklist for keeping the bill down

1. Automate shutdown. End every training script with a command that stops or terminates the instance, and set an idle timeout if the provider offers one. This single habit removes the most expensive line in the example above.
2. Terminate, don't stop, unless you will resume within days. Stopped instances often keep billing for disk.
3. Audit storage monthly. List every volume, snapshot and image. Delete what you cannot name a reason for.
4. Bake a custom image with your dependencies so setup is a single pull instead of half an hour of installs.
5. Cache weights close to the compute, on a volume or bucket in the same region, instead of downloading from the internet on every launch.
6. Test the host in the first five minutes. Check nvidia-smi, run a short throughput benchmark against a known baseline, and kill the box immediately if it is slow.
7. Fit before you rent. Use the VRAM fit calculator so you do not burn a run on an out-of-memory error, and so you do not rent a bigger card than you need.
8. Move adapters, not models. Pull LoRA weights out and merge locally when egress is billed.
9. Match billing granularity to job length. Batch short jobs together on per-hour billing; use per-second billing for bursty work.
10. Use one GPU when one GPU is enough. Multi-GPU only pays off when the interconnect keeps scaling efficiency high.
11. Consider interruptible capacity for checkpointed batch work. Our spot vs on-demand guide has the math for when it pays.

For a quick sense of what the compute line itself costs on common cards right now, here are live prices across providers:

GPU	VRAM	Le moins cher maintenant	Où	Médiane des fournisseurs	Fournisseurs
Nvidia GeForce RTX 4090	24GB	$0.30/hr	io.net	$0.46/hr	13
Nvidia A100 80GB PCIe	80GB	$1.09/hr	Thunder Compute	$1.65/hr	10
Nvidia H100	80GB	$3.20/hr	Thunder Compute	$3.37/hr	3

Données en direct : prix horaire médian par GPU, relevé sur les dernières 24 heures. Comment nous relevons les prix.

Compare individual providers on the GPU price index, and if you are deciding whether to rent at all, the self-host vs API calculator shows when a hosted API is cheaper than any GPU once these overheads are included.

Questions fréquentes