  Low-rank matrices are pervasive throughout statistics, machine learning,
signal processing, optimization, and applied mathematics. In this paper, we
propose a novel and user-friendly Euclidean representation framework for
low-rank matrices. Correspondingly, we establish a collection of technical and
theoretical tools for analyzing the intrinsic perturbation of low-rank matrices
in which the underlying referential matrix and the perturbed matrix both live
on the same low-rank matrix manifold. Our analyses show that, locally around
the referential matrix, the sine-theta distance between subspaces is equivalent
to the Euclidean distance between two appropriately selected orthonormal basis,
circumventing the orthogonal Procrustes analysis. We also establish the
regularity of the proposed Euclidean representation function, which has a
profound statistical impact and a meaningful geometric interpretation. These
technical devices are applicable to a broad range of statistical problems.
Specific applications considered in detail include Bayesian sparse spiked
covariance model with non-intrinsic loss, efficient estimation in stochastic
block models where the block probability matrix may be degenerate, and
least-squares estimation in biclustering problems. Both the intrinsic
perturbation analysis of low-rank matrices and the regularity theorem may be of
independent interest.
