Principal Component Analysis

/ˈprɪnsɪpəl kəmˈpoʊnənt əˈnæləsɪs/prin·ci·pal com·po·nent a·nal·y·sisnoun

Dimensionality Reduction Core

Definition

1.[in statistics] a transformation that rewrites correlated measurements as a smaller set of uncorrelated axes, ordered so the first captures as much of the spread as any single direction can.

Principal component analysis reduced the 200 sensor channels to six axes that retained 94% of the variance.

Formal statement

C = (1/(n-1)) X^T X, C v_i = lambda_i v_i

The axes are the eigenvectors of the covariance matrix, ranked by their eigenvalues.

Etymology

From principal, Latin principalis 'first, chief', plus component, Latin componere 'to put together'. Introduced by Karl Pearson in 1901 as a line of closest fit, and named and developed independently by Harold Hotelling in 1933.

Synonyms

Karhunen-Loève transformsense 1 · Near

References

Pearson, K. (1901). On Lines and Planes of Closest Fit to Systems of Points in Space.Philosophical Magazine, 2(11), 559–572.
Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components.Journal of Educational Psychology, 24, 417–441.