# Low-Rank Approximation of Difference between Correlation Matrices Using Inner Product
## 2 Methods
### Algorithm for Estimating Low-Rank Correlation Matrix Based on MM Algorithm  
This subsection provides a detailed description of the MM algorithm for estimating \(\mathbf{X}\). First, we derive the majorizing function of Equation (8) in the same manner as in [24; 25]. The updated formula for \(\mathbf{x}_{i}\) is also derived based on the majorizing function. Based on the majorizing function, the problem is then converted into a linear problem. Therefore, the updated formula can be derived via the Lagrange multipliers method with the constraint Equation (9). However, first, before the derivation the majorizing function is presented, the principle of the MM algorithm is explained. For more details on the MM algorithm, see [24; 25]. Let \(f(\theta)\) be an objective function with the parameter \(\theta\), and \(g(\theta|\ \theta^{(t)})\) be a real valued function, where \(\theta^{(t)}\) is the fixed value of \(\theta\). When \(g(\theta|\ \theta^{(t)})\) satisfies  
\[g(\theta|\ \theta^{(t)})\geq f(\theta)\quad\text{for all}\quad\theta,\] \[g(\theta^{(t)}|\ \theta^{(t)})= f(\theta^{(t)}),\]  
\(g(\theta|\ \theta^{(t)})\) is said to be a majorizing function. In general, deriving the updated formula for \(g(\theta|\ \theta^{(t)})\) is expected to be easy compared to that for \(f(\theta)\). Therefore, in this proposed approach, the majorizing function of (8) is derived in the same manner as in [24; 25]. The objective function Equation (8) can now be defined as follows:  
\[\frac{1}{2}L(\mathbf{X}|\ \mathbf{\Delta})= \sum_{i<j}\left(\delta_{ij}-\|\mathbf{d}_{i}\|\ \|\mathbf{d}_{j}\|\ \mathbf{x}_{i}^{T}\mathbf{x}_{j}\right)^{2}\] \[= C-2\sum_{i<j}\delta_{ij}\ \|\mathbf{d}_{i}\|\ \|\mathbf{d}_{j}\|\ \mathbf{x}_{i}^{T}\mathbf{x}_{j}+ \sum_{i<j}\|\mathbf{d}_{i}\|^{2}\ \|\mathbf{d}_{j}\|^{2}\ \mathbf{x}_{i}^{T}\mathbf{x}_{j}\mathbf{x}_{i}^{T}\mathbf{x}_{j}. \tag{10}\]  
The third term of Equation (10) corresponding to \(i\) is expressed as follows:  
\[\mathbf{x}_{i}^{T}\Big{(}\|\mathbf{d}_{i}\|^{2}\ \sum_{j\
eq i}\|\mathbf{d}_{j}\|^{2}\ \mathbf{x}_{j}\mathbf{x}_{j}^{T}\Big{)}\mathbf{x}_{i}=\mathbf{x}_{i}^{T}\mathbf{B}\mathbf{x}_{i}, \tag{11}\]  
where \(\mathbf{B}_{i}=\|\mathbf{d}_{i}\|^{2}\ \sum_{j\
eq i}\|\mathbf{d}_{j}\|^{2}\ \mathbf{x}_{j}\mathbf{x}_{j}^{T}\). Let \(\lambda_{i}\) be the maximum eigenvalue of \(\mathbf{B}_{i}\). For any \(\mathbf{x}\in\mathbb{R}^{d}\) with the constraint \(\|\mathbf{x}\|=1\), the following inequality is satisfied:  
\[\mathbf{x}^{T}(\mathbf{B}_{i}-\lambda_{i})\mathbf{x}\leq 0, \tag{12}\]  
where \(\mathbf{I}_{d}\) is an identity matrix of size \(d\). That is, \(\mathbf{B}_{i}-\lambda_{i}\) is negative semi-definite. Let \(\mathbf{x}_{i}\in\mathbb{R}^{d}\) and \(\mathbf{z}_{i}\in\mathbb{R}^{d}\) be coordinate vectors of subject \(i\) corresponding to the current step and the previous step, respectively. Based on Equation (12), the following inequality is satisfied:\[(\mathbf{x}_{i}-\mathbf{z}_{i})^{T}(\mathbf{B}_{i}-\lambda_{i}\mathbf{I}_{d})(\mathbf{x}_{i}-\mathbf{z}_{ i})\leq 0\] \[\Longleftrightarrow \mathbf{x}_{i}^{T}\mathbf{B}_{i}\mathbf{x}_{i}\leq 2\lambda_{i}+2\mathbf{x}_{i}^{T}(\bm {B}_{i}-\lambda_{i}\mathbf{I}_{d})\mathbf{z}_{i}-\mathbf{z}_{i}^{T}\mathbf{B}_{i}\mathbf{z}_{i}. \tag{13}\]  
If \(\mathbf{x}_{i}=\mathbf{z}_{i}\), Equation (13) becomes an equality equation. Equation (13) is substituted into the third term of Equation (10), resulting in  
\[\frac{1}{2}L(\mathbf{X}\mid\mathbf{\Delta})\leq C-2\sum_{i<j}\delta_{ij}\parallel\mathbf{d}_{i}\parallel\parallel\mathbf{d}_{j} \parallel\mathbf{x}_{i}^{T}\mathbf{x}_{j}\] \[+\sum_{i<j}\Big{(}2\lambda_{i}+2\mathbf{x}_{i}^{T}(\mathbf{B}_{i}-\lambda _{i}\mathbf{I}_{d})\mathbf{z}_{i}-\mathbf{z}_{i}^{T}\mathbf{B}_{i}\mathbf{z}_{i}\Big{)}. \tag{14}\]  
From (14), the updated formula of \(\mathbf{x}_{i}\) is derived as follows:  
\[g(\theta^{(t)}|\ \theta^{(t)})= f(\theta^{(t)}),\]0  
For the algorithm based on Equation (15), see Algorithm 1.