Loss Function

/lɔs ˈfʌŋkʃən/loss func·tionnoun

Optimization CoreFoundational

Definition

1.[in machine learning] a measure of the difference between predicted and actual values, quantifying the error of a model's output.

We must select an appropriate loss function, such as Mean Squared Error, to guide the model's training process.

Formal statement

L(y, ŷ) = (y - ŷ)^2

This example shows the Mean Squared Error (MSE) loss function.

Etymology

From loss, Old English los 'destruction, ruin', plus function, Latin functio 'performance'. The technical sense — a function assigning a penalty to a decision — comes from Abraham Wald's statistical decision theory of the 1940s, which borrowed the economic sense of loss.

Synonyms

cost functionsense 1 · Absolute

References

Goodfellow et al. (2016). Deep Learning, ch. 6.MIT Press.