Bias-Variance Tradeoff

/ˈbaɪəs ˌvɛriəns ˈtreɪdˌɔːf/bias-var-i-ance-trade-offnoun

Statistical Learning CoreFoundational

Definition

1.[in machine learning] the fundamental tension between the complexity of a model and its ability to generalize to unseen data.

Understanding the Bias-Variance Tradeoff is crucial for selecting the appropriate regularization strength.

Formal statement

Error = Bias² + Variance + Irreducible Error

The total expected error is the sum of squared bias, variance, and irreducible error.

Etymology

From bias, Old French biais 'a slant', by way of the statistical sense of a systematic offset from the truth, plus variance, Latin variantia 'difference'. The decomposition that names the tradeoff was set out by Stuart Geman, Elie Bienenstock and René Doursat in 1992.

Synonyms

bias-variance dilemmasense 1 · Absolute

References

Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning, 2nd ed., ch. 6.Springer.