Guide

Spot vs on-demand GPU rental: when interruptible is worth it.

How spot, preemptible and bid-tier GPU pricing works, which AI workloads survive interruption, and the checkpoint math that shows when spot really saves.

By Vi Nguyen · Published October 13, 2026 · Prices on this page are live · Editorial policy

Almost every GPU provider sells the same hardware at two prices. On-demand means the machine is yours until you stop it. Spot, interruptible, preemptible or bid pricing means the machine is yours until someone else wants it more, at which point your job is stopped, sometimes with a short warning and sometimes with none. The discount is usually large enough to be tempting, and the question is never "is spot cheaper per hour" (it is) but "is spot cheaper per finished job once I account for the work I lose". This guide walks through how the different interruptible models work, which workloads survive preemption, the checkpointing arithmetic that decides whether you come out ahead, and a framework you can apply to your own jobs.

How interruptible pricing actually works

"Spot" is used loosely, but there are really two mechanisms behind it, and they fail in different ways.

Hyperscaler spot: spare capacity, reclaimed by the provider

On AWS, Google Cloud and Azure, spot instances run on capacity the provider has not sold at full price. The price floats with supply and demand in that zone, but you are not really bidding against another customer. You are borrowing idle capacity, and when the provider needs it back for an on-demand or reserved customer, your instance is reclaimed. The warning is short: AWS documents a two-minute interruption notice, and Google Cloud and Azure give roughly thirty seconds. That is enough time to flush a small checkpoint or drain a request queue, not enough to save a large model's full optimizer state from scratch.

The practical consequence is that interruption rates depend heavily on which GPU you ask for and where. Older or less popular accelerators in a quiet region can run for days. The newest high-demand parts are often the first to be reclaimed, and sometimes spot capacity for them is simply not offered.

Marketplace bid tiers: someone else outbids you

Marketplaces such as Vast.ai and Clore.ai run something closer to a real auction. The host sets an on-demand price and a lower floor for interruptible use. You place a bid at or above the floor, and your instance runs as long as nobody bids higher or rents the machine on-demand. When someone does, your container is stopped. The details (whether the disk survives, whether you get your slot back automatically when the higher bidder leaves, how much notice you get) vary by platform and can change, so read the current terms before you depend on them. The important difference from hyperscaler spot is that your risk is set by your bid relative to everyone else's, so you can buy a lower interruption rate by bidding closer to the on-demand price.

RunPod also sells interruptible pods alongside its on-demand ones, and the general shape is the same: cheaper per hour, with the pod stopped if the capacity is needed for a full-price customer.

What the spread looks like right now

The tables below show live per-provider prices for two very different cards. They mix on-demand and interruptible listings as each provider reports them, so treat the low end of each range as the "spot-like" floor and the median as closer to what a stable on-demand box costs. The cheapest RTX 4090 listing at the moment is $0.30/hr on io.net, and the cheapest H100 is $2.29/hr on Theta EdgeCloud.

Provider	Median $/hr	Range	Offers
io.net	$0.30/hr	$0.30/hr–$0.30/hr	792
Novita AI	$0.33/hr	$0.33/hr–$0.33/hr	11
RunPod	$0.34/hr	$0.34/hr–$0.34/hr	1
Nosana	$0.36/hr	$0.29/hr–$0.50/hr	6
Theta EdgeCloud	$0.38/hr	$0.38/hr–$0.38/hr	1
Vast.ai	$0.42/hr	$0.42/hr–$0.42/hr	1

Live data, last 24 hours. Full Nvidia GeForce RTX 4090 page.

Provider	Median $/hr	Range	Offers
Theta EdgeCloud	$2.29/hr	$2.29/hr–$2.29/hr	1
Fluence	$2.81/hr	$2.71/hr–$3.04/hr	3
Thunder Compute	$3.20/hr	$3.20/hr–$3.20/hr	0

Live data, last 24 hours. Full Nvidia H100 page.

The width of each range matters more than any single number. A wide range on a marketplace means there is a real discount available if you accept worse hosts or interruptions. A narrow range means the market has already priced reliability in and spot will not save you much.

Which workloads tolerate preemption

The test is simple: if the machine disappears right now, how much work do you lose, and how long until you are running again? Workloads split cleanly into three groups.

Good fits

• Batch inference. Embedding a corpus, captioning a dataset, generating synthetic data, scoring evaluations. The work is naturally chunked into items. If you record which shards are done (a file per shard in object storage is enough), an interruption costs you only the shard in flight. This is the single best use of spot capacity.
• Hyperparameter sweeps. Each trial is independent and usually short. Losing one trial to preemption means re-queuing it, and a sweep scheduler will do that for you. Many sweeps also kill most trials early, so the expected loss per interruption is small.
• Checkpointed training and fine-tuning. If you save state every N steps and your training script can resume from the latest checkpoint without human help, interruptions become a small tax rather than a disaster. LoRA and QLoRA runs are especially easy because the trainable state is small; see our QLoRA fine-tuning cost guide for sizing.
• Rendering and image generation queues. Stable Diffusion or Flux batch jobs behave like batch inference: each image is an independent unit of work.

Poor fits

• Serving live traffic. An inference endpoint that answers user requests cannot vanish for ten minutes while a replacement boots and downloads 40 GB of weights. You can put spot replicas behind a load balancer with on-demand replicas as the floor, but a single spot box serving production traffic is an outage waiting to happen.
• Long uncheckpointed runs. A 30-hour job with no resume logic on a box that averages one interruption every 10 hours will, on average, never finish. The cost is not a percentage; it is unbounded.
• Multi-node distributed training. If eight nodes each have a small chance of interruption per hour, the chance that at least one of them is interrupted is roughly eight times higher, and a single lost node usually stalls the whole job. Spot multi-node training only works with elastic frameworks and frequent checkpoints, and even then the restart overhead multiplies.
• Interactive development. Losing your notebook kernel mid-debug does not cost much compute but it costs your attention, which is usually worth more than the hourly saving on a single card.

The checkpointing cost math

The cleanest way to compare spot and on-demand is to convert both to cost per unit of completed work. For on-demand, that is just the hourly price, because every hour you pay for is an hour of progress (ignoring host failures). For spot, some of the hours you pay for are wasted redoing work that was lost at an interruption. So:

Effective spot cost per productive hour = spot price × (1 + lost-work fraction)

The lost-work fraction has three parts:

1. Rework: on average you lose half a checkpoint interval of progress per interruption (the interruption lands at a random point between two checkpoints).
2. Restart overhead: the time to get a new machine, pull your container, download weights and data, and reload the checkpoint.
3. Checkpoint overhead: the time the job spends writing checkpoints instead of training, which you pay whether or not an interruption ever happens.

Worked example

Every price here is a round hypothetical number chosen to make the arithmetic easy to follow, not a current market rate. Suppose a fine-tuning job needs 100 GPU-hours of actual training on one card. On-demand costs a hypothetical $2.00/hr. The spot or interruptible tier for the same card costs $0.80/hr, and you observe roughly one interruption every 10 hours.

1. Checkpoint every hour. Average rework per interruption is half an interval: 0.5 hours.
2. Restart overhead. Finding a new box, pulling the image and reloading takes about 15 minutes: 0.25 hours.
3. Loss per interruption = 0.5 + 0.25 = 0.75 hours. At one interruption per 10 hours of work, that is 0.75 ÷ 10 = 7.5% of the job.
4. Checkpoint overhead. Each save takes about 3 minutes (0.05 hours) once an hour: 5% of runtime.
5. Lost-work fraction = 7.5% + 5% = 12.5%.
6. Effective spot rate = $0.80 × 1.125 = $0.90 per productive hour.
7. Job cost: spot 100 × $0.90 = $90; on-demand 100 × $2.00 = $200. Spot saves $110, or 55%.

Now make it worse. Checkpoint only every 4 hours and assume restarts take 30 minutes because the new host has a slow network. Rework becomes 2 hours and restart 0.5 hours, so each interruption costs 2.5 hours, which is 25% of the job. Checkpoint overhead drops to about 1.25%. The lost-work fraction is roughly 26%, and the effective rate is $0.80 × 1.26 ≈ $1.01/hr. Still about half the on-demand price.

The breakeven condition falls straight out of the formula. Spot wins whenever spot price × (1 + f) < on-demand price, which means f < (on-demand ÷ spot) − 1. At a 60% discount (0.4 of the on-demand price), you can lose up to 150% of your work before spot stops paying. In practice, the per-dollar case for spot almost always holds. What kills it is the cases the formula does not capture: jobs that never checkpoint, deadlines that cannot absorb the extra wall-clock time, and the engineering time spent writing resume logic.

Choosing the checkpoint interval

A classic rule of thumb from high-performance computing (Young's approximation) says the checkpoint interval that minimizes total waste is about √(2 × checkpoint cost × mean time between interruptions). With a 3-minute checkpoint (0.05 hours) and 10 hours between interruptions, that gives √(2 × 0.05 × 10) = √1 = 1 hour, which is why the first scenario above used hourly saves. If your checkpoints are cheap, save more often; if interruptions are rare, save less often.

Checkpoint cost depends on what you save. A LoRA adapter is tens to a few hundred megabytes and takes seconds. A full fine-tune that saves FP32 master weights plus Adam's two moment buffers is on the order of 12 bytes per parameter, so a 7B model's resumable state is roughly 7 × 12 ≈ 84 GB before counting anything else. At an assumed 1 GB/s write speed, that is about a minute and a half per save, and much longer if you are pushing it over a slow uplink to object storage. That is also far too much to write inside a 30-second hyperscaler warning, which is why periodic saves matter more than trying to react to the interruption signal.

Wall-clock time is the hidden variable

Cost per finished job is only half the picture. Interruptions also stretch the calendar. In the first example, 100 hours of work become about 112.5 billed hours, plus however long you wait for capacity after each interruption. On a marketplace with plenty of listings that wait may be minutes. On hyperscaler spot for a scarce GPU it can be hours, because the same demand spike that reclaimed your instance also means there is nothing free to replace it.

This is why teams with a hard deadline often run a hybrid: start the job on spot, and if it has fallen behind schedule by some threshold, move the latest checkpoint to an on-demand box to finish. You pay the premium only for the tail. The same logic applies to inference serving: keep a floor of on-demand replicas sized for your minimum traffic and add spot replicas for peaks, so an interruption degrades latency rather than availability.

Also watch what stays billed while you are interrupted. On some platforms a stopped interruptible instance keeps its disk and continues charging for storage; on others the disk is released and you lose anything not saved elsewhere. Both are fine if you plan for them and expensive if you do not. Our guide to the hidden costs of renting GPUs covers storage billing in more detail.

Making a job spot-safe

If you decide spot is worth it, a short checklist turns interruption from an emergency into a routine event:

• Write checkpoints somewhere that outlives the machine. Object storage or a network volume, not the instance's local disk. On a marketplace, assume the host's disk may not be available to you again.
• Make resume automatic. Your entrypoint should look for the latest checkpoint and continue from it with no flags. If resuming needs a human, you will lose hours to the gap between the interruption and someone noticing.
• Save the data loader position and RNG state, not just weights and optimizer. Otherwise you silently retrain on the same batches after every restart.
• Bake dependencies into the image. Every minute of pip install and model download is restart overhead paid once per interruption.
• Handle the termination signal. Catch SIGTERM, finish the current step, write a small emergency checkpoint if it fits in the warning window, and exit cleanly.
• Log interruptions. After a week you will know your real mean time between interruptions for that GPU and platform, which is the one number the whole calculation depends on.

A decision framework

Work through these in order and stop at the first one that applies:

1. Is it serving user traffic? Use on-demand for the baseline capacity. Add spot replicas only behind a load balancer, and only if cold start (image pull plus weight download) is fast enough to replace a lost replica within your latency budget.
2. Is it a single run longer than your expected time between interruptions with no resume logic? Use on-demand, or add checkpointing first. Do not gamble.
3. Is it multi-node training? Default to on-demand or reserved capacity with a good interconnect. Consider spot only if your framework supports elastic restarts and you have measured the restart cost.
4. Is the work divisible into small independent items (batch inference, sweeps, image generation)? Use spot or the lowest interruptible tier. This is where the discount is nearly free.
5. Is it checkpointed training with automatic resume? Use spot, set the checkpoint interval with the square-root rule above, and compute your effective rate with price × (1 + lost-work fraction). If you have a deadline, plan to finish the tail on on-demand.
6. Is it interactive work? Use on-demand on the cheapest card that fits. The hourly saving is rarely worth the lost focus.

For marketplace bid tiers specifically, there is one more lever: bid higher. Bidding at 80% of on-demand rather than at the floor may cut your interruption rate substantially while still saving 20%. Try it on a real job, count interruptions, and put the numbers into the formula. If you are still choosing which GPU to run on, check what fits first with the VRAM fit calculator, then compare live prices on the RTX 4090 and H100 pages. Our comparison of GPU marketplaces and managed clouds covers the host-quality side of the same trade-off.

FAQ