## 3 Methodology  
### Probabilistic Principal Component Analysis (PPCA)  
The PPCA theory essentially augments the linear projection from the principal space to the observed data space by assuming that the observed data are corrupted by isotropic Gaussian noise. When the observation noise variance is zero, PPCA is equivalent to the standard principal component analysis (PCA) [38]. Indeed, PPCA explains the standard PCA from the viewpoint of probability, which can be deduced from the Gaussian latent variable generative model. The Gaussian latent variable model of given observed signals can be written as:  
\[\mathbf{y}=\mathbf{W}\mathbf{z}+\boldsymbol{\mu}+\boldsymbol{\varepsilon} \tag{10}\]  
where \(\mathbf{y}\in\mathbb{R}^{N\times 1}\) represents an observation vector, the matrix \(\mathbf{W}\) is a \(N\times q\) dimensional loading matrix, \(\mathbf{z}\in\mathbb{R}^{q\times 1}\) denotes the vector of latent variables, \(\boldsymbol{\mu}\) is the mean vector of the signals, and \(\boldsymbol{\varepsilon}\) denotes the noise component which is independent of \(\mathbf{y}\). Generally, \(q<N\) is set in this model so that latent variables can provide a briefer explanation of the dependences between observations. The Gaussian latent variable model assumes that both \(\mathbf{z}\) and \(\boldsymbol{\varepsilon}\) have Gaussian distributions with zero mean, i.e., \(\mathbf{z}\sim\mathcal{N}\bigl{(}0,\mathbf{I}_{q}\bigr{)}\) and \(\boldsymbol{\varepsilon}\sim(0,\Psi)\), where \(\mathbf{I}_{q}\) is a \(q\times q\) identity matrix and \(\Psi\) is a diagonal matrix. According to the Gaussian linearity property, \(\mathbf{y}\) should also follow the Gaussian distribution: \(\mathbf{y}\sim\mathcal{N}(\boldsymbol{\mu},\mathbf{C})\), where \(\mathbf{C}=\mathbf{W}\mathbf{W}^{T}+\Psi\).  
Further supposing that \(\boldsymbol{\varepsilon}\) is isotropic (i.e., \(\boldsymbol{\varepsilon}\sim(0,\mathbf{I}_{N})\)), the probability density function of \(\mathbf{y}\) can be deduced as follows:  
\[\begin{split} f(\mathbf{y})&=\int f(\mathbf{y}| \mathbf{z})f(\mathbf{z})d\mathbf{z}\\ &=(2\pi)^{-N/2}|\mathbf{C}|^{-1/2}\exp\Bigl{\{}-\frac{1}{2}( \mathbf{y}-\boldsymbol{\mu})^{T}\mathbf{C}^{-1}(\mathbf{y}-\boldsymbol{\mu}) \Bigr{\}}\end{split} \tag{11}\]  
where \(\mathbf{C}=\mathbf{W}\mathbf{W}^{T}+\sigma^{2}\mathrm{I}_{N}\). It follows a Gaussian distribution as well. According to this, the logarithmic likelihood function for a set of observation vectors \(\{\mathbf{y}_{m}\}_{m=1}^{M}\) can be expressed as:  
\[\mathcal{L}(\Theta)=-\frac{M}{2}\Bigl{\{}N\ln(2\pi)+\ln|\mathbf{C}|+\text{ trace}\Bigl{(}\mathbf{C}^{-1}\mathbf{S}\Bigr{)}\Bigr{\}} \tag{12}\]  
where \(\Theta=\{\boldsymbol{\mu},\mathbf{W},\sigma^{2}\}\) represents the unknown parameters, \(\mathbf{S}\) is the covariance matrix of \(\{\mathbf{y}_{m}\}_{m=1}^{M}\), and \(\text{tr}(\cdot)\) stands for the trace operator. In [39], it is deduced that the likelihood is maximized when  
\[\mathbf{W}=\mathbf{U}_{q}\Bigl{(}\Lambda_{q}-\sigma^{2}\mathbf{I}\Bigr{)}^{1/ 2}\mathbf{R} \tag{13}\]  
where the \(q\) column vectors in the \(N\times q\) matrix \(\mathbf{U}_{q}\) are the principal eigenvectors of \(\mathbf{S}\), with corresponding eigen values \(\text{diag}\bigl{(}\lambda_{1},\lambda_{2},...,\lambda_{q}\bigr{)}\) in the \(q\times q\) diagonal matrix \(\Lambda_{q}\), and \(\mathbf{R}\) is an arbitrary \(q\times q\) orthogonal rotation matrix. Consequently, the global maximum of the likelihood can be represented by principal eigenvectors. That is to say, the Gaussian latent variable generative model with \(\Psi=\sigma^{2}\mathbf{I}_{N}\) is actually a probabilistic model of PCA. In addition, due to the introduction of latent variables, the computationally efficient expectation maximization (EM) algorithm can be utilized to estimate the model parameters.