Variables that are correlated with one another which are also largely independent of other subsets of variables are combined into factors. Factors which are generated are thought to be representative of the underlying processes that have created the correlations among variables. PCA and FA can be exploratory in nature, FA is used as a tool in attempts to reduce a large set of variables to a more meaningful, smaller set of variables. As both FA and PCA are sensitive to the magnitude of correlations robust comparisons must be made to ensure the quality of the analysis.
Correlation coefficients tend to be less reliable when estimated from small sample sizes.
In general it is a minimum to have at least five cases for each observed variable. Missing data need be dealt with to provide for the best possible relationships between variables. Fitting missing data through regression techniques are likely to over fit the data and result in correlations to be unrealistically high and may as a result manufacture factors. Normality provides for an enhanced solution, but some inference may still be derived from nonnormal data. Multivariate normality also implies that the relationships between variables are linear.