So wird gerechnet
Memory depends on the method. A full fine-tune updates every weight, so it needs the same ~16 bytes per parameter as pre-training (weights, gradients, FP32 master copy and Adam's two moments) — about 1.1 TB of GPU memory for a 70B model, which means a multi-GPU node. LoRA freezes the base model at 16-bit (2 bytes per parameter) and trains small adapter matrices, typically around 1% of the parameters. QLoRA goes further and stores the frozen base in 4-bit, which is how a 70B model can be fine-tuned on a single 80 GB card.
Time comes from the same compute rule as training: about 6 FLOPs per parameter per token for a full fine-tune, and roughly 4 for LoRA-style methods because the base weights need no gradients. QLoRA adds overhead for de-quantizing the 4-bit weights on every pass (we assume +30%). We divide by the GPU's spec-sheet BF16 throughput at 35% utilization — a realistic figure for single-node fine-tuning with a modern stack.
Dataset size is counted in tokens, not examples: 10,000 instruction pairs averaging 500 tokens each is 5M tokens. Most LoRA fine-tunes converge in 1–3 epochs. Beyond the compute shown here, budget for data preparation, evaluation runs and a few failed attempts — the fine-tuning cost guide walks through a full example.