# Robust Bilinear Probabilistic Principal Component Analysis
## 1 Introduction  
High-dimensional data are increasingly collected for a variety of applications in the real world. However, high-dimensional data are not often distributed uniformly in their ambient space, instead of that the interesting structure inside the data often lies in a low-dimensional space [1]. One of the fundamental challenges is how to find the low-dimensional data representation for high-dimensional observed data in pattern recognition, machine learning and statistics [2; 3]. Principal component analysis (PCA) [4] is arguably the most well-known dimension reduction method for high-dimensional data analysis, and it aims to find the first few principal eigenvectors corresponding to the first few largest eigenvalues of the covariance matrix, and then projects the high-dimensional data onto the low-dimensional subspace spanned by these principal eigenvectors to achieve the purpose of dimensionality reduction. The traditional PCA is concerned with vectorial data, i.e., 1-D data. For 2-D image trained sample matrices, it is usual to first convert 2-D image matrices into 1-D image vectors. This transformation leads to higher dimensional image sample vectors and a larger covariance matrix, and thus suffers from the difficulty of accurately evaluating the principal eigenvectors of the large scale covariance matrix. Furthermore, such vectorizing of 2-D data destroys the natural matrix structure, and ignores potentially valuable information about the spatial relationships among 2-D data. Therefore, two-dimensional PCA (2 DPCA) type algorithms [5; 6; 7] are proposed to compute principal component weight matrices directly based on 2-D image training with sample matrices instead of using vectorization. These conventional PCA and 2 DPCA algorithms are both derived and interpreted in the standard algebraic framework, thus they lack capability in handling issues of statistical inference or missing data. To remedy these drawbacks, a probabilistic PCA model (PPCA) has been proposed by [PERSON] and [PERSON] in [8], which is processed by assuming someGaussian distributions on observations with introduced extra latent variables, and it has been successfully applied in many machine learning tasks [9]. Following PPCA, a probabilistic second-order PCA, called PSOPCA, is developed in [10] to directly model 2-D image matrices based on the so-called matrix variate Gaussian distributions. Throughout this paper, \(\mathbb{R}^{n\times m}\) is the set of all \(n\times m\) real matrices, \(I_{n}\) and \(0_{m\times n}\) are the \(n\times n\) identity matrix and \(m\times n\) zero matrix, respectively. The superscript \".\" means transpose only, \(\|\cdot\|_{2}\) and \(\|\cdot\|_{\text{F}}\) denote the \(\ell_{2}\)-norm and Frobenius norm of a matrix, respectively. Denoted by \(\mathcal{N}_{d_{c},d_{r}}(M,\Omega_{c},\Omega_{r})\) is the matrix variate Gaussian distribution [11] with the mean matrix \(M\in\mathbb{R}^{d_{c}\times d_{r}}\), column covariance \(\Omega_{c}\in\mathbb{R}^{d_{c}\times d_{c}}\) and row covariance \(\Omega_{r}\in\mathbb{R}^{d_{r}\times d_{r}}\). A random matrix \(X\in\mathbb{R}^{d_{c}\times d_{r}}\) is said to follow the matrix variate Gaussian distribution \(\mathcal{N}_{d_{c},d_{r}}(M,\Omega_{c},\Omega_{r})\), i.e.,  
\[X\sim\mathcal{N}_{d_{c},d_{r}}(M,\Omega_{c},\Omega_{r}),\text{ if }\operatorname{vec}(X)\sim \mathcal{N}(\operatorname{vec}(M),\Omega_{r}\otimes\Omega_{c}), \tag{1}\]  
where \(\operatorname{vec}(\cdot)\) is the vectorization of a matrix obtained by stacking the columns of the matrix on top of one another.