## 3 Methodology
### Incremental Canonical Correlation Analysis (ICCA)
\(S_{2}^{-2}\) can be obtained the same as \(S_{1}^{-2}\). So, the matrix inverse is avoided. It is obvious that the overhead of vanilla SVD is not ignored as well, if the matrix is still relatively large. Towards this end, a sequential Karhunen-Loeve algorithm (SKL) [44] can be employed here. Without loss of generality, suppose that \(n\) centralized samples \(X\) and the m new centered incoming samples \(X^{\prime}\). Assume that \(X=U_{1}S_{1}V_{1}^{T}\). Obviously, the SVD of the concatenation \(X\) and \(X^{\prime}\) is \([XX^{\prime}]=U_{1}^{T}\Sigma_{1}^{\prime}V_{1}^{T}\). Let \(X^{\prime}\) be the component of \(X^{\prime}\) orthogonal to \(U_{1}\), \(U^{\prime}=[U_{1},X^{\prime}]\)and \(V^{\prime}=\left[ \begin{array}{cc}V_{1}^{\prime}&0\\ 0&I_{m}\end{array} \right]\), where \(I_{m}\) is the identity matrix with the size of \(m\). Then, we derive a small matrix \(X_{n}=U^{\prime}[X,X^{\prime}]V^{\prime T}\). The SVD of \(X_{n}\) is \(X_{n}=\widetilde{U}\widetilde{D}\widetilde{V}^{T}\), then \([X,X^{\prime}]=U^{\prime}\widetilde{U}\widetilde{D}\widetilde{V}^{T}V^{ \prime T}\). Obviously, we need the latest left singular matrix \(U_{1}^{\prime}=U^{\prime}\widetilde{U}\) and the corresponding singular value matrix \(\Sigma_{1}^{\prime}=\widetilde{D}\). Now we can obtain the inverse of the covariance matrix of \(X\). Adopting the SKL algorithm [44] can ensure its space and time complexity to be constant in \(O(n+m)\). Each update only adopts the first top \(k-\) truncated singular value and basis vectors in the former step and the \(m\) new incoming samples, thus the space complexity is reduced to \(O(d(k+m))\) instead of the previous \(O(d(n+m)^{2})\) as in [PERSON] and [PERSON] [44]. **Updating Subspaces. ** Of course, the SKL algorithm is designed for sequential SVD methods. Thus, we follow the section above to find the SVD of \(\Sigma_{xy}\). Since merging the small-sized matrices into the SVD forms of \(\Sigma_{xx}^{-1}\), \(\Sigma_{yy}^{-1}\), and \(\Sigma_{xy}\), we can efficiently obtain the SVD of \(P=\Sigma_{xx}^{-1/2}\Sigma_{xy}\Sigma_{yy}^{-1/2}\). In fact, we do not need the concrete form. This is because the projection matrix only requires the left and right singular matrices. The update formula is  
\[\left\{ \begin{array}{l}a=\Sigma_{xx}^{-1/2}u\\ b=\Sigma_{yy}^{-1/2}v\end{array} \right. \tag{9}\]  
where \(u\) and \(v\) are the left and right singular matrices of \(P\), respectively. Here, the subspace \(a\) and \(b\) can be updated. The specific optimization algorithm is shown in Algorithm 1. ```
Input:\(n\) samples \(X\) and \(Y\), \(m\) new samples \(X^{\prime}\) and \(Y^{\prime}\). Output: the updated subspace \(New\_U_{1}\), \(New\_U_{2}\) the updated mean and covariance \(\overline{X}_{n+m}\),\(\overline{\Sigma}_{xy}\);
1. Update the mean \(\overline{X}\), \(\overline{Y}\) and covariance \(\Sigma_{xy}\) and \(\Sigma_{yx}\) of \(n\) samples, according to Equation (5);
2. Update the mean \(\overline{X}_{n+m}\), \(\overline{Y}_{n+m}\) of \(n+m\) current samples according to Equation (6);
3.\(\overline{X}\leftarrow\overline{X}_{n+m}\), \(\overline{Y}\leftarrow\overline{Y}_{n+m}\), \(\Sigma_{xy}\leftarrow\Sigma_{xy}\) according to Equations (4) and (5);
4. Calculate \(U_{1}\), \(S_{1}\), \(U_{2}\), and \(S_{2}\) according to Equation (7);
5. Calculate \(\Sigma_{xx}^{-1}\), \(\Sigma_{yy}^{-1}\) according to Equation (8);
6. Calculate \(a\) and \(b\) to yield new subspace \(New\_U_{1}\), \(New\_U_{2}\) according to Equation (9). ```  
**Algorithm 1**Incremental Subspace Algorithm