Stepwise estimation of common principal components
The standard common principal components (CPCs) may not always be useful for simultaneous dimensionality reduction in k k groups. Moreover, the original FG  algorithm finds the CPCs in arbitrary order, which does not reflect their importance with respect to the explained variance. A possible alternative is to find an approximate common subspace for all k k groups. A new stepwise estimation procedure for obtaining CPCs is proposed, which imitates standard PCA. The stepwise CPCs facilitate simultaneous dimensionality reduction, as their variances are decreasing at least approximately in all k k groups. Thus, they can be a better alternative for dimensionality reduction than the standard CPCs. The stepwise CPCs are found sequentially by a very simple algorithm, based on the well-known power method for a single covariance/correlation matrix. Numerical illustrations on well-known data are considered.
dimensionality reduction simultaneous diagonalization power iterations for k k symmetric positive definite matrices k k k k k