Question: why eigenvectors of covariance matrix?
Answer: If you have features, you can find eigenvectors of the covariance matrix of the features. This allows you to represent the data with uncorrelated features. ... The largest eigenvector of a covariance matrix points into the direction of the largest variance. All other eigenvectors are orthogonal to the largest one.