Fine-tuning
Pretraining teaches a model language. Fine-tuning teaches it your task, for a fraction of the cost.
Continue training on your data
A pretrained model already knows a lot. Fine-tuning keeps training it on a small dataset of examples for your task: classifying support tickets, following instructions, answering in a house style. The loss on that data falls quickly because the model is not starting from scratch.
The price: forgetting
The same updates that help your task can damage what the model knew. Push too far and loss on general text climbs. Parameter-efficient methods such as LoRA freeze the base weights and train a small add-on, which limits the damage and cuts memory.
The lab uses made-up numbers with the right shape. Drag the sliders and watch the two losses trade off.
A worked example
A 7-billion-parameter model has about 4,096 × 4,096 = 16.8 million numbers in each attention projection. Full fine-tuning updates all of them, plus the optimiser keeps extra state for each. LoRA with rank 8 adds two matrices of 4,096 × 8 and 8 × 4,096, 65,536 numbers, and trains only those. The update B·A still has the full 4,096 × 4,096 shape, but it can only change the weights along 8 directions.
In practice
Supervised fine-tuning on instruction and answer pairs is what turns a base model that continues text into a chat model that follows requests. A few thousand high-quality examples can change behaviour noticeably. Keeping some general data in the mix, using a small learning rate and stopping early all limit forgetting.
Easy to confuse
- Fine-tuning mostly changes behaviour and format; adding large amounts of new knowledge is better done with retrieval.
- LoRA weights can be merged into W after training, so serving a LoRA model costs nothing extra.
What does fine-tuning start from?
Fine-tuning continues training from a pretrained model on a smaller, task-specific dataset, so it needs far less data and compute than pretraining.
After many steps of fine-tuning at a high learning rate, the model gets worse at general tasks. What is this called?
Updating many weights toward a narrow dataset can overwrite what the model knew before. Lower learning rates, fewer steps and mixing in general data all reduce it.
What does LoRA train?
LoRA freezes the base model and learns a small low-rank update for selected layers. That is far fewer trainable parameters, less memory and usually less forgetting.
Read more in the books
- Build a Large Language Model (From Scratch), Sebastian Raschka: Ch. 6, “Finetuning for Text Classification” and Ch. 7, “Finetuning to Follow Instructions”.
- AI Engineering, Chip Huyen: Ch. 7, “Finetuning”.
- Build a Large Language Model (From Scratch), Sebastian Raschka: Appendix E, “Parameter-efficient Finetuning with LoRA”.