Expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix
The pseudo-Fisher linear classifier is considered as the diagonal Fisher linear classifier applied to the principal components corresponding to non-zero eigenvalues of the sample covariance matrix. An asymptotic formula for the expected (generalization) error of the Fisher classifier with the pseudo-inversion is derived which explains the peaking behaviour: with an increasing number of learning observations from one up to the number of features, the generalization error first decreases, and then starts to increase.
pseudo-inversion generalization error statistical classification fisher linear discriminant scissors effect sample size dimensionality