# A Jacobi-Davidson Method for Large Scale Canonical Correlation Analysis
## 3 The Main Algorithm
### Correction Equation
Figure 2: Cost in computing the first 10 canonical weight vectors of ORL (**top**), FERET (**middle**) and Yale (**bottom**) datasets with MINRES steps for the correction equation varying from 5 to 40. ## 5 Conclusions

To analyze the correlations between two data sets, several numerical algorithms have been available to find the canonical correlations and the associated canonical weight vectors; however, there is very little discussion of the large scale sparse and structured matrix cases in the literature. In this paper, a Jacobi-Davidson type method, i.e., Algorithm 1, is presented for large scale canonical correlation analysis by computing a small portion of eigenpairs of the canonical correlation generalized eigenvalue problem (4). The theoretical result is established in Theorem 3 to demonstrate that the cubic convergence of the approximate eigenvector if the correction equation is solved exactly and the approximate eigenvector of the previous step is close enough to the exact one. The corresponding numerical results are presented to confirm the effectiveness of asymptotic convergence rate provided by Theorem 3, and to demonstrate that Algorithm 1 performs far superior to the JDQZ method for the large scale symmetric positive definite pencil \(\{K,M\}\). Notice that the main computational tasks in every iteration of Algorithm 1 consist of solving the correction Equation (22). In our numerical example, we only focus on the plain version of MINRES, i.e., without considering any preconditioner. However, it is not hard to notice that incorporating a preconditioner presents no difficulty and can promote the numerical performance if

Figure 3: Convergence behavior of JDCCA and JDQZ for computation of the first 10 canonical weight vectors of ORL (**top**), FERET (**middle**) and Yale (**bottom**) datasets with MINRES step \(n_{g}=20\). the preconditioner is available. In addition, from the point of view that multi-set canonical correlation analysis (MCCA) [42] proposed to analyze linear relationships among more than two data sets can be equivalently transformed to the following generalized eigenvalue problem

\[ \begin{bmatrix}0&C_{12}&\dots&C_{1t}\\ C_{21}&0&\dots&C_{2t}\\ \vdots&\vdots&\vdots&\vdots\\ C_{t1}&C_{t2}&\dots&0\end{bmatrix}\begin{bmatrix}x_{1}\\ x_{2}\\ \vdots\\ x_{t}\end{bmatrix} =\lambda \begin{bmatrix}C_{11}&0&\dots&0\\ 0&C_{22}&\dots&0\\ \vdots&\vdots&\vdots&\vdots\\ 0&0&\dots&C_{t}\end{bmatrix}\begin{bmatrix}x_{1}\\ x_{2}\\ \vdots\\ x_{t}\end{bmatrix} ,\]

where \(C_{ij}=S_{i}S_{j}^{\mathrm{T}}\) and \(S_{i}\) is the data matrix, the development of efficient Jacobi-Davidson methods for treating such large scale MCCA will be a subject of our future study. Writing--original draft, Z.T. ; Writing--review and editing, X.Z. and Z.T. All authors have read and agreed to the published version of the manuscript. This work is supported in part by National Natural Science Foundation of China NSFC-11601081 and the research fund for distinguished young scholars of Fujian Agriculture and Forestry University No. xjq201727. The authors declare no conflict of interest. ## Appendix A

**Proof of Theorem 1. ** To prove (12), for any \(U\in\mathbb{R}^{m\times k}\) and \(V\in\mathbb{R}^{n\times\ell}\) satisfying \(U^{\mathrm{T}}AU=I_{k}\) and \(V^{\mathrm{T}}BV=I_{\ell}\), respectively, we first consider the augmented matrices of \(\widetilde{C}\) and \(U^{\mathrm{T}}CV\), i.e.,

\[ \begin{bmatrix}0&\widetilde{C}\\ \widetilde{C}^{\mathrm{T}}&0\end{bmatrix} \quad\text{and}\quad \begin{bmatrix} 0&U^{\mathrm{T}}CV\\ V^{\mathrm{T}}C^{\mathrm{T}}U&0\end{bmatrix} ,\]

whose eigenvalues \(\pm\lambda_{i}\) for \(i=1,\dots,n\) plus \(m-n\) zeros and \(\sigma_{i}(U^{\mathrm{T}}CV)\) for \(i=1,\dots,\min\{k,\ell\}\) plus \(k+\ell-2\min\{k,\ell\}\), respectively. Notice that

\[ \begin{bmatrix}0&U^{\mathrm{T}}CV\\ V^{\mathrm{T}}C^{\mathrm{T}}U&0\end{bmatrix} =