## 3 Methodology
### Incremental Canonical Correlation Analysis (ICCA)  
**Mean and Covariance Update. ** In CCA, calculating the mean and covariance is a necessary step for its solution. For the mean and covariance update, it is relatively easy to keep the storage constant. For completeness, we introduce the concrete strategy, where the implementation details could not be slightly different from other methods. Suppose that there are \(n\) old samples and \(m\) new incoming samples. Let \(\overline{X}\) and \(\overline{Y}\) denote the mean of \(n\) old samples from \(X\) and \(Y\), respectively. Correspondingly,the covariance of \(n\) old samples and \(m\) new incoming samples between \(X\) and \(Y\) sample are denoted by \(\Sigma_{xy}\) and \(\Sigma^{\prime}_{xy}\), respectively. \(\Sigma^{\prime}_{xy}\) denotes the covariance of \(n^{t}\) samples. \[\Sigma^{\prime\prime}_{xy}=\frac{m-1}{n^{t}}\Sigma_{xy}+\frac{m-1}{n^{t}-1} \Sigma^{\prime}_{xy}+\frac{m(m+n^{t})}{(n^{t}-1)n^{t}}(\overline{X^{\prime}}- X)(\overline{Y^{\prime}}-Y)^{T} \tag{4}\]  
where  
\[ \begin{array}{c}\Sigma_{xy}=\sum\limits_{i=1}^{n}{(X_{i}-\overline{X})(Y_{i }-\overline{Y})}^{T},\\ \Sigma^{\prime}_{xy}=\sum\limits_{i=n+1}^{n+m}{(X_{i}-\overline{X})(Y_{i}- \overline{Y})}^{T},\\ \overline{X}=\sum\limits_{i=1}^{n}{X_{i}},\overline{Y}=\sum\limits_{i=1}^{n}{ Y_{i}},\\ \overline{X^{\prime}}=\sum\limits_{i=n+1}^{n+m}{X_{i}},\overline{Y^{\prime}}= \sum\limits_{i=n+1}^{n+m}{Y_{i}},\\ n^{t}=fn+m\end{array}  \tag{5}\]  
and \(f\) denotes a forgotten factor to eliminate old samples, \(f\in[0,1]\). The parameter \(f=1\) expresses that more emphasis is put on all the samples, while \(f=0\) indicates that a full focus is placed on the new samples. The \(n\) previous history matrices including \(\Sigma_{xy}\), \(\overline{X}\) and \(\overline{Y}\) need to be saved before \(m\) new samples arrive. The matrix \(\Sigma^{\prime}_{xy}\), \(\overline{X^{\prime}}\) and \(\overline{Y^{\prime}}\) can be obtained with the new \(m\) samples and then the covariance of matrix \(\Sigma^{\prime\prime}_{xy}\) can be calculated, based on these \(n\) old samples and \(m\) new incoming samples. Thus, all the samples do not take up much memory. Of them, the mean of all the samples at this time can be updated as follows:  
\[ \begin{array}{c}\overline{X}_{n+m}=\frac{n}{n^{t}}.\overline{X}+\frac{n}{n^ {t}}\overline{X^{\prime}}\\ \overline{Y}_{n+m}=\frac{n}{n^{t}}.\overline{Y}+\frac{n}{n^{t}}\overline{Y^{ \prime}}\end{array}  \tag{6}\]  
when the next \(m\) samples emerge again, once the matrix \(\Sigma^{\prime\prime}_{xy}\), \(\overline{X}\), and \(\overline{Y}\) are computed according to Equation (5), the saved value of \(n^{t}\) and the historical matrix \(\Sigma_{xy}\) directly take part in the new round of calculation and storage. The complete operation would not increase the storage space with the rapid of the number of samples. **Avoiding Matrix Inversion. ** To solve CCA, it is necessary to solve the inversion of matrix \(\Sigma_{xx}\) and \(\Sigma_{yy}\) in Equation (3). To avoid the matrix inversion, we resort to the singular value decomposition (SVD), which efficiently solves the thinner matrix. Given any matrix \(A\in R^{b\times n}\), its SVD form is as follows:  
\[A\stackrel{{ SVD}}{{=}}USV^{T} \tag{7}\]  
where \(U\in R^{b\times n}\), \(S\in R^{b\times n}\), \(V\in R^{n\times m}\); then \(AA^{T}=USV^{T}VSU^{T}\). Owing to the orthogonality \(VV^{T}=I\), and thus \(AA^{T}=USSU^{T}\). When \(X-\overline{X}=U_{1}S_{1}V_{1}^{T}\), \(\Sigma_{xx}\) is a real symmetric matrix and thus  
\[\Sigma_{xx}^{-1}=\left(\Sigma_{xx}^{T}\right)^{-1}=\left(U_{1}S_{1}^{2}U_{1}^{ T}\right)^{-1}=U_{1}S_{1}^{-2}U_{1}^{T} \tag{8}\]  
In Equation (8), there is an inverse operation \(S_{1}^{-2}\). Since \(S_{1}\) is a diagonal matrix whose elements are only located on the diagonal, its inversion can be calculated by the reciprocal of each element. Likewise, \(\Sigma_{yy}^{-1}=\left(\Sigma_{yy}^{T}\right)^{-1}=\left(U_{2}S_{2}^{2}U_{2}^{T }\right)^{-1}=U_{2}S_{2}^{-2}U_{2}^{T}\).