The cheapest GPU cloud for fine-tuning is not the one advertising the lowest hourly rate. It is the cheapest card that physically fits your run, multiplied by the hours your run actually takes, plus three costs that never appear in the headline price: setup time, storage and interruption. This page works through all four, so you can price a run instead of comparing rate cards.
The rates below are market medians used by the estimate model behind the calculator. They are assumptions, not quotes: providers differ, and spot prices move hourly. Check the number on the provider’s page before you commit. The structure of the comparison is what transfers.
Price per GPU-hour by card
| Card | VRAM | Median $/hour | $ per 1M training tokens |
|---|---|---|---|
| RTX 3090 | 24 GB | 0.22 | 0.138 |
| RTX 4090 | 24 GB | 0.44 | 0.119 |
| L4 | 24 GB | 0.44 | 0.162 |
| RTX 6000 Ada | 48 GB | 0.79 | 0.195 |
| L40S | 48 GB | 1.19 | 0.147 |
| A100 40GB | 40 GB | 1.19 | 0.170 |
| A100 80GB | 80 GB | 1.59 | 0.227 |
| H100 80GB | 80 GB | 2.69 | 0.121 |
| H200 141GB | 141 GB | 3.39 | 0.153 |
Sort that by hourly rate and the RTX 3090 wins by a mile. Sort it by cost per token and the 3090 drops to third, behind the 4090 and the H100, because it is slow: 442 tokens per second against 1,027 and 6,158. The A100 80GB is the most expensive per token in the entire list, which is worth repeating to anyone who assumes the datacentre card is the efficient one. It is not efficient. It is large.
The first filter is memory, and it is a hard one. An 8B QLoRA run needs 6.2GB and so fits anything. A 32B QLoRA run needs 21.3GB, which still fits 24GB cards with a little headroom. A 70B QLoRA run needs 44.3GB and therefore cannot run on a 3090 or a 4090 at all — your cheapest options jump from $0.22/hour to $0.79/hour, a 3.6× floor increase, before any speed argument.
Here is what that does to a real run of 22.5M tokens (5,000 examples, 1,500 tokens, 3 epochs):
| Run | Cheapest card that fits | Hours | Cost |
|---|---|---|---|
| 8B QLoRA | RTX 4090 | 6.08 | $2.68 |
| 8B QLoRA | RTX 3090 | 14.14 | $3.11 |
| 32B QLoRA | RTX 4090 | 24.9 | $10.94 |
| 32B QLoRA | A100 40GB | 13.1 | $15.60 |
| 70B QLoRA (8M tokens) | L40S 48GB | 8.67 | $10.31 |
| 70B QLoRA (8M tokens) | H100 80GB | 3.17 | $8.53 |
Two lessons, and they are the whole buying guide. The cheapest card per hour is rarely the cheapest card per run: the 3090 costs more than the 4090 for the same 8B job. And on 70B, the most expensive card per hour produces the cheapest run, because it is three times faster.
Spot vs on-demand vs reserved
These are three different products, not three discount levels.
On-demand is a fixed rate with no interruption. You pay for the convenience and it is the right default for a single run you want finished today.
Spot is idle capacity sold cheap, and it can be reclaimed mid-run with little notice. The price is real, but it is only a saving if your workflow survives eviction: checkpoints saved every N steps, a resume path that loads them, and a script that restarts without a human. Every framework worth using gives you save_steps and resume_from_checkpoint; without them, a preempted 10-hour run is a 10-hour loss, not a 30% discount.
Reserved is a commitment over weeks or months in exchange for a lower rate. It only makes sense at high, predictable utilisation. If you train two days a week, you are paying for five days of nothing.
The decision rules that actually hold up:
- One run, a few hours, a deadline: on-demand. The interruption engineering costs more than the premium.
- Recurring training, several runs a week, same config: reserved if utilisation is genuinely high, spot if not.
- Exploration where you expect to throw half the runs away: spot, because a killed run was probably a bad run anyway.
- Anything with a long single run and no checkpointing: on-demand until you add checkpointing.
I am deliberately not quoting a spot discount percentage. It varies by provider, card and hour, and a number from this page would be wrong by the time you read it. Compare your own: hours from the calculator, multiplied by the two rates on offer.
Hidden costs: storage and egress
The hourly rate covers the card. It does not cover the four costs below, and on a small run they can exceed it.
Setup time is billed. Pulling a large container image and uploading a dataset happens while the meter runs. Twenty minutes on an A100 80GB at $1.59/hour is $0.53, which is 10% of an entire 8B run. Bake the dataset into a snapshot, or use a provider with cached volumes, and that line goes away.
Storage is rent, not a one-off. Datasets, checkpoints and merged weights sit somewhere and are billed monthly per GB. A 70B model in bf16 is 141GB of weights: if you keep merged copies, the storage bill is a permanent tax on a single training job. Keep adapters (megabytes) and delete the rest.
Egress is asymmetric. Downloading a LoRA adapter costs nothing worth counting. Downloading a merged 70B model is over a hundred gigabytes of transfer, and providers price that per GB. Train where you serve, or move adapters only.
Failed runs are the most expensive line. An OOM at step 900 burns the whole run. A 50-step smoke test costs about 1% of the run and tells you whether the config fits and the loss moves. Do it every time you change rank, batch or sequence length — that is exactly the parameter set that changes VRAM.
Billing granularity is a quiet multiplier. Per-second billing charges 3.22 hours for a 3.22-hour job. Per-hour billing charges 4. On a $1.59/hour card that is $1.24 of rounding, and it is invisible on the rate card.
Picks by budget
- Around $2.70 per 8B run, and the wall clock does not matter: RTX 4090. Cheapest per run in the whole table at $2.68, and it needs no volume or snapshot trickery.
- Around $2.70 per 8B run, and you want it today: H100 at $2.73 and one hour. Same money, five fewer hours. Pick this when you are iterating.
- Under $1 per run: only by splitting the run — fewer epochs, a shorter sequence, or a smaller model. A 1B QLoRA run fits an 8GB card and costs cents.
- 32B QLoRA: 24GB cards still work at 21.3GB, but with 5% headroom you are one sequence-length change away from an OOM. An A100 40GB at $15.60 per run is the cautious choice; a 4090 at $10.94 is the cheap one.
- 70B QLoRA: you need 48GB or more. L40S at $10.31, H100 at $8.53. The RTX 6000 Ada 48GB at $0.79/hour looks attractive until you price the run: 128 tokens per second turns it into 17.3 hours and $13.69.
- Anything that needs more than 141GB: no single card. That is multi-GPU with sharding, and this model does not price it. Budget for at least two 80GB cards and expect the interconnect to matter as much as the FLOPs.
The pattern across all of them: price the run, not the rate. Get your GPU-hours from the cost calculator, then compare three candidates. If you are deciding whether to own hardware instead, the GPU buying guide covers the break-even, and A100 vs H100 is the worked example of why the faster card can be the cheaper one.