Overfitting
A model that matches its training data perfectly can still be useless. What counts is how it does on data it has never seen.
Two kinds of error
The dark points are training data; the orange ones are held out. Both come from the same smooth curve plus noise. The model is a polynomial, and its degree sets how flexible it is.
Degree 1 is too stiff: it misses the curve on both sets (underfitting). As the degree rises, training error always drops. Past some point the curve starts bending to chase individual noisy points, and held-out error goes up (overfitting). At degree 7 the curve passes through all eight training points exactly, with zero training error, and is useless between them.
Why it matters for LLMs
Large models can memorise training text. That is why evaluations use held-out test sets, and why a benchmark leaking into training data (contamination) makes scores look better than they are.
A worked example
A degree-1 polynomial (a straight line) is too simple for curved data: both errors are high, which is underfitting. Around degree 3 both errors are low. At degree 7 the curve passes almost exactly through every training point, so training error is close to zero, but it swings wildly between them and the held-out error climbs: overfitting.
What helps
More data is the strongest cure, because a wiggly curve cannot fit many points by chance. Regularisation penalises large weights, dropout randomly switches units off during training, and early stopping halts training when the held-out error starts to rise. Large language models are trained on so much text that they rarely see the same example twice, which keeps overfitting low during pretraining; fine-tuning on a small dataset is where it returns.
Easy to confuse
- Overfitting is judged by the gap between training and held-out error, not by the training error alone.
- A model can memorise specific facts and still generalise well overall; memorising the test set, however, makes the evaluation meaningless.
Training error keeps falling as the degree rises, but test error starts rising. What is happening?
A flexible enough model can pass through every training point, noise included, and then generalises badly to new points.
Which model would you pick?
Held-out (validation) error estimates performance on new data; training error is optimistic.
Which of these reduces overfitting?
More data makes noise harder to memorise; regularisation penalises overly complex fits.
Read more in the books
- Deep Learning, Goodfellow, Bengio, Courville: Ch. 5, section 5.2 (capacity, overfitting and underfitting) and Ch. 7, “Regularization for Deep Learning”.
- Pattern Recognition and Machine Learning, Christopher Bishop: Ch. 1, section 1.1 (polynomial curve fitting).
- Dive into Deep Learning, Zhang, Lipton, Li, Smola: Ch. 3, the section on generalization.