OpenAI fine-tuning is priced per million training tokens, the same unit the self-hosted estimate uses, so the comparison is one line of arithmetic. Rented GPU time for an 8B QLoRA run comes to $0.119 to $0.227 per million training tokens depending on the card. If the managed rate is higher than your card’s number, training the same data yourself is cheaper on the training line — and the rest of the decision is about everything else.
No vendor price appears on this page. Rates move, and a figure copied here would be wrong within months; the arithmetic takes whatever is on the pricing page today and turns it into a per-run number. The self-hosted figures come from the estimate model behind the cost calculator.
Supported models and pricing
Managed fine-tuning never covers the whole catalogue: the trainable set is the small-to-mid tier of each family, while the flagship models stay closed to it. To change a frontier model’s behaviour the answer is the fine-tuning vs RAG comparison, not training.
Check the documentation for the current list; it changes without notice. The shape of the pricing is stable, and has four parts.
Training is billed per million training tokens, per epoch. Two epochs cost twice one epoch. This is the line that surprises people who count examples instead of tokens: the cost is a function of your file’s token count multiplied by the number of passes over it.
Every token in every example is billed. Prompt, system message and completion all count. If your data repeats a 400-token instruction on every row, 400 tokens of every row are billed on every epoch, needed or not.
Inference is billed separately, per million input and output tokens, at a premium over the base model. That premium is charged on every request for as long as the model serves traffic, so the serving bill is the large number and training is the small one.
Hosting may be billed while the model exists. Some services charge for keeping a fine-tuned model addressable, independent of traffic. A model you use occasionally can cost more in idle hosting than in training, so delete the ones you no longer serve.
Put together, the only number you need from the pricing page is the rate per million training tokens. The self-hosted equivalent:
| Card | Median $/hour | 8B QLoRA $ per 1M training tokens | 32B QLoRA $ per 1M training tokens |
|---|---|---|---|
| RTX 4090 24GB | 0.44 | 0.119 | 0.486 |
| RTX 3090 24GB | 0.22 | 0.138 | 0.565 |
| L4 24GB | 0.44 | 0.162 | 0.663 |
| A100 40GB | 1.19 | 0.170 | 0.695 |
| A100 80GB | 1.59 | 0.227 | 0.929 |
| H100 80GB | 2.69 | 0.121 | 0.496 |
Those figures assume QLoRA at rank 16, 2,048-token sequences, batch 1 and gradient checkpointing, with a card busy for the whole run. Idle time is charged too, so a real run costs more: add cold start and setup, priced on the RunPod vs Lambda page.
Data format and limits
OpenAI fine-tuning takes JSONL, one JSON object per line, with a messages array of role-tagged turns: typically a system message, a user message and an assistant reply. The completion is the assistant turn; the rest is context the model reads but is barely trained on.
{"messages":[{"role":"system","content":"You answer support tickets about a billing API."},{"role":"user","content":"Charge failed with code 402, card is valid."},{"role":"assistant","content":"402 means the issuer declined the authorisation. Ask the customer to retry with a different card, then check the decline reason in the dashboard."}]}The constraints that bite, in the order they bite:
- A minimum number of examples. There is a floor — the docs state the current number, historically ten — below which the job is rejected or returns a model that has memorised your file. A few dozen examples is a demo; a few hundred to a few thousand is a fine-tune.
- A per-example token ceiling. Long examples are truncated or rejected, and truncation is silent enough to look like a data problem later. Check the longest row before you upload.
- A file size cap. Upload limits push people into splitting a dataset or stripping repeated boilerplate, which also makes the run cheaper.
- Training and validation files. Point the job at both, or let it split automatically, and read the validation curve. A loss that falls on training and rises on validation says the epochs are too many.
The exposed hyperparameters are the usual set: epochs, batch size, learning-rate multiplier and seed. The multiplier is not a learning rate; it scales the vendor’s internal rate, which is why 0.1 or 2 is normal rather than absurd. If the loss curve is flat, change the data.
What you do not get is the weights. Fine-tuning returns a model identifier you can call, not a checkpoint you can download, quantise, run locally or merge. That single fact decides most of the cases below.
Cost per 1k examples
Cost per 1,000 examples is a bad unit unless you also state tokens per example and epochs, because those two are the whole cost: tokens per 1,000 examples = 1,000 × tokens per example × epochs.
| Tokens per example | Epochs | Tokens per 1,000 examples | Self-hosted, 8B QLoRA on a 4090 | Self-hosted, 8B QLoRA on an A100 80GB |
|---|---|---|---|---|
| 500 | 3 | 1.5M | $0.18 | $0.34 |
| 1,500 | 3 | 4.5M | $0.54 | $1.02 |
| 4,000 | 3 | 12M | $1.43 | $2.72 |
| 8,000 | 3 | 24M | $2.86 | $5.45 |
Multiply column three by the managed rate per million and you have the managed cost of the same thousand examples. Do it in that direction, tokens first, because teams routinely compare a managed run of 1,500-token examples against a self-hosted estimate for 500-token ones and conclude the wrong thing by a factor of three.
Two consequences follow. Truncating a long example to the part that carries the answer is the largest free saving available. And epochs are a multiplier to justify: three epochs on 4.5M tokens is 13.5M tokens, and if validation loss flattened at epoch two, the third was billed in full for nothing.
Iteration is where totals get away from you. A sweep of 30 runs at 22.5M tokens each is 675M training tokens — $80 of 4090 time, about 183 GPU-hours, a week of overnight runs. On a managed service the same sweep is 675M tokens at the managed rate, and a difference that looked like cents per run is the whole budget.
When to self-host instead
Self-host when you need the weights, when inference volume is high, or when the method is not offered. Those three cover almost every case where a rented GPU beats a managed endpoint.
You need the weights. On-premise deployment, an edge device, 4-bit quantisation onto a 24GB card, merging an adapter into a base model you already run, or keeping the artefact after the catalogue changes — none of that is possible with a managed fine-tune. If the model has to leave the vendor, managed fine-tuning is not an option at any price.
Inference volume is high. Training is a one-off; serving is rent. Measure your monthly token volume — input and output separately, since they are priced separately — multiply by the fine-tuned premium, and compare with the cost of the GPU-hours that would serve the same traffic. Estimate capacity from your own measurements: run the quantised model on the card you are considering and record tokens per second at your concurrency. Below a few million tokens a month, the managed premium is usually the cheaper line.
The method is not offered. Full fine-tuning of a 70B, DPO on preference pairs, RLHF, a custom loss, distillation from your own teacher, a family outside the catalogue, or a model you keep training for months. That is QLoRA-and-PyTorch territory; the QLoRA page covers the configuration.
You are iterating. Thirty experiments on a managed service are thirty managed jobs; on rented hardware they are thirty runs at $0.12 per million tokens. The fastest card in your budget is usually cheaper per run, because 6× the speed at 6× the hourly rate is the same price per token — the A100 vs H100 comparison works that through.
Managed fine-tuning wins, decisively, in the cases it was built for: a few hundred to a few thousand examples, a task the mid-tier models already understand, no GPU operations, no serving infrastructure, and no reason to want the weights. Price both from the numbers above, and notice which line dominates — for small projects it is inference, for research work it is iteration.
Build the dataset before you price either option: the dataset guide covers JSONL, deduplication and the token counts that feed every number here, and the fine-tuning guide covers the process if you go self-hosted.