# Robust Bilinear Probabilistic Principal Component Analysis
## 1 Introduction
That means that the probability density function (pdf) of \(X\) is  
\[p(X) =(2\pi)^{-\frac{1}{2}d_{c}d_{r}}|\Omega_{r}\otimes\Omega_{c}|^{- \frac{1}{2}}\exp\biggl{(}-\frac{1}{2}[\operatorname{vec}(X-M)]^{\mathsf{T}}( \Omega_{r}^{-1}\otimes\Omega_{c}^{-1})\operatorname{vec}(X-M)\biggr{)}\] \[=(2\pi)^{-\frac{1}{2}d_{c}d_{r}}|\Omega_{r}|^{-\frac{1}{2}d_{c}}| \Omega_{c}|^{-\frac{1}{2}d_{r}}\operatorname{etr}\biggl{(}-\frac{1}{2}\Omega _{c}^{-1}(X-M)\Omega_{r}^{-1}(X-M)^{\mathsf{T}}\biggr{)}, \tag{2}\]  
where \"\(\otimes\)\" is the Kronecker product of two matrices, \(\operatorname{etr}(\cdot)=\exp(\operatorname{tr}(\cdot))\) and \(\operatorname{tr}(\cdot)\) denotes the trace of a matrix. The last equality of (2) holds because of \(|\Omega_{r}\otimes\Omega_{c}|^{-\frac{1}{2}}=|\Omega_{r}|^{-\frac{1}{2}d_{c}}| \Omega_{c}|^{-\frac{1}{2}d_{r}}\) and  
\[[\operatorname{vec}(X-M)]^{\mathsf{T}}(\Omega_{r}^{-1}\otimes\Omega_{c}^{-1 })\operatorname{vec}(X-M)=\operatorname{tr}\big{(}\Omega_{c}^{-1}(X-M)\Omega _{r}^{-1}(X-M)^{\mathsf{T}}\big{)}.\]  
See ([11], Theorem 1.2.21) and ([11], Theorem 1.2.22) for more details. PSOPCA in [10] considers the following two-sided latent matrix variable model  
\[ \begin{cases}X=CZR^{\mathsf{T}}+W+Y,\\ Z\sim\mathcal{N}_{q_{c},\mathcal{H}}(0_{q_{c}\times q_{r}},I_{q_{c}},I_{q_{r}}),\ Y\sim\mathcal{N}_{d_{c},d_{r}}(0_{d_{c}\times d_{r}},\sigma^{2}I_{d_{c}}, \sigma^{2}I_{d_{r}}),\end{cases}  \tag{3}\]  
where \(C\in\mathbb{R}^{d_{c}\times q_{c}}\) and \(R\in\mathbb{R}^{d_{r}\times q_{r}}\) are the column and row factor loading matrices, respectively, \(W\in\mathbb{R}^{d_{c}\times d_{r}}\) and \(Y\in\mathbb{R}^{d_{c}\times d_{r}}\) are the mean and error matrices, respectively, and \(Z\in\mathbb{R}^{d_{c}\times q_{r}}\) is the latent core variable of \(X\). The PSOPCA model is further extended to the bilinear probabilistic principal component analysis (BPPCA) model in [12] for better establishing the relationship with the 2 DPCA algorithm [6], which is defined as  
\[ \begin{cases}X=CZR^{\mathsf{T}}+W+C\mathcal{E}_{r}+\mathcal{E}_{c}R^{\mathsf{ T}}+\mathcal{E},\\ \mathcal{E}_{c}\sim\mathcal{N}_{d_{c},d_{r}}(0_{d_{c}\times q_{r}},\sigma_{c}^ {2}I_{d_{r}},I_{q_{r}}),&\mathcal{E}_{r}\sim\mathcal{N}_{q_{c},d_{r}}(0_{q_{c} \times d_{r}},I_{q_{c}},\sigma_{r}^{2}I_{d_{r}}),\\ \mathcal{E}\sim\mathcal{N}_{d_{c},d_{r}}(0_{d_{c}\times d_{r}},\sigma_{c}^{2} I_{d_{r}},\sigma_{r}^{2}I_{d_{r}}),&Z\sim\mathcal{N}_{q_{c},d_{r}}(0_{q_{c} \times q_{r}},I_{q_{c}},I_{q_{r}}).\end{cases}  \tag{4}\]  
In contrast to the PSOPCA model (3), the column and row noise matrices \(\mathcal{E}_{c}\in\mathbb{R}^{d_{c}\times q_{r}}\) and \(\mathcal{E}_{r}\in\mathbb{R}^{d_{c}\times d_{r}}\) with different noise variances \(\sigma_{c}^{2}\) and \(\sigma_{r}^{2}\), respectively, are included in the BPPCA model, and \(\mathcal{E}\in\mathbb{R}^{d_{c}\times d_{r}}\) is represented as the common noise matrix. The model (4) improves the flexibility in capturing data uncertainty, and makes the marginal distribution \(p(X)\) to be the matrix variable Gaussian. In particular, we can see that if \(\mathcal{E}_{r}\) and \(\mathcal{E}_{c}\) are removed and \(\sigma_{c}=\sigma_{r}\), then (4) reduces to the PSOPCA model. All of the above mentioned probabilistic models assume that the noise terms follow Gaussian distributions. It is a well-known issue that Gaussian noises will lead to a serious drawback while dealing with anomalous observations. Thus, the probabilistic PCA models based on Gaussian distributions are not robust to outliers. To make probabilistic models which are insensitive to outliers, one prefers heavy-tailed distributions, such as the Student \(t\) distribution or centered Laplacian distribution with \(\ell_{1}\)-norm. Using the \(t\) distribution or centered Laplacian distribution instead of the Gaussian distribution in the PPCA model [8] results in tPPCA [13; 14] and probabilistic L1-PPCA [15] algorithms, respectively.