What phenomenon describes a model's test error decreasing, then increasing, and then decreasing again as model capacity grows?
Show answer
Double Descent
This phenomenon challenges the classical bias-variance tradeoff. It suggests that once a model has enough parameters to interpolate the training data perfectly (the interpolation threshold), increasing capacity further can act as a form of implicit regularization that improves generalization.
Also asked as:
- What term describes the non-monotonic behavior where test error drops, rises, and then drops again as model capacity increases?