# A Jacobi-Davidson Method for Large Scale Canonical Correlation Analysis
## 3 The Main Algorithm
### Correction Equation
\begin{cases}\widetilde{C}^{\mathsf{T}}\widetilde{C}\tilde{w}_{2}= \lambda\widetilde{C}^{\mathsf{T}}\tilde{w}_{1}+\gamma_{1}\widetilde{C}^{ \mathsf{T}}p=\lambda^{2}\tilde{w}_{2}+\lambda\gamma_{2}q+\gamma_{1}\lambda q, \\ \widetilde{C}\widetilde{C}^{\mathsf{T}}\tilde{w}_{1}=\lambda\widetilde{C} \tilde{w}_{2}+\gamma_{2}\widetilde{C}q=\lambda^{2}\tilde{w}_{1}+\lambda\gamma _{1}p+\gamma_{1}\lambda p,\end{cases} \] \[\Rightarrow \begin{cases}(\widetilde{C}^{\mathsf{T}}\widetilde{C}-\lambda^{2 }I_{n})\tilde{w}_{2}&=(\lambda\gamma_{2}+\lambda\gamma_{1})q,\\ ((\widetilde{C}\widetilde{C}^{\mathsf{T}}-\lambda^{2}I_{m})\tilde{w}_{1}=( \lambda\gamma_{1}+\lambda\gamma_{1})p.\end{cases} \]

Therefore, both \(\tilde{w}_{1}\) and \(p\) belong to the kernel of \((\widetilde{C}\widetilde{C}^{\mathsf{T}}-\lambda^{2}I_{m})^{2}\), and both \(\tilde{w}_{2}\) and \(q\) belong to the kernel of \((\widetilde{C}^{\mathsf{T}}\widetilde{C}-\lambda^{2}I_{n})^{2}\). The simplicity of \(\lambda\) implies \(\tilde{w}_{1}\) and \(\tilde{w}_{2}\) are multiples of \(p\) and \(q\), respectively. Since \(w_{1}\in\operatorname{span}(x)^{\perp_{\lambda}}\) and \(w_{2}\in\operatorname{span}(y)^{\perp_{s}}\), we have \(\tilde{w}_{1}\in\operatorname{span}(p)^{\perp}\) and \(\tilde{w}_{2}\in\operatorname{span}(q)^{\perp}\), which means \(\tilde{w}_{1}=\tilde{w}_{2}=0\). Therefore, \(w_{1}=w_{2}=0\). The bijectivity follow from comparing dimensions. Proof of Theorem 3.: Let

\[F:= \begin{bmatrix}I_{m}-A\tilde{x}\tilde{x}^{\mathsf{T}}&0\\ 0&I_{n}-B\tilde{y}\tilde{y}^{\mathsf{T}}\end{bmatrix} .\]

Then the correction equation is

\[F \begin{bmatrix}-\theta A&C\\ C^{\mathsf{T}}&-\theta B\end{bmatrix} F^{\mathsf{T}} \begin{bmatrix}s\\ t\end{bmatrix} =- \begin{bmatrix}r_{*}\\ r_{*}\end{bmatrix} . \tag{15}\]



By Lemma 1, when \(\tilde{x}\) and \(\tilde{y}\) are close enough to \(x\) and \(y\), respectively, we see that \(F \begin{bmatrix}-\theta A&C\\ C^{\mathrm{T}}&-\theta B\end{bmatrix} F^{\mathrm{T}}\) is invertible. It follows by (A9) that

\[ \begin{bmatrix}\tilde{f}-s\\ \tilde{g}-t\end{bmatrix} =\left(F \begin{bmatrix}-\theta A&C\\ C^{\mathrm{T}}&-\theta B\end{bmatrix} F^{\mathrm{T}}\right)^{-1} \begin{bmatrix} \omega_{1}\alpha A\tilde{f}\\ \omega_{2}B\tilde{g}\end{bmatrix} \quad\Rightarrow\quad\left\| \begin{bmatrix} \tilde{f}-s\\ \tilde{g}-t\end{bmatrix} \right\|_{{}_{M}}=\mathcal{O}\left(\varepsilon^{3} \right).\]

The last equality holds because of \(\|\tilde{f}\|_{{}_{A}}=\mathcal{O}(\varepsilon)\), \(\|\tilde{g}\|_{s}=\mathcal{O}(\varepsilon)\) and (A10), which means \(\|\tilde{f}-s\|_{{}_{A}}=\mathcal{O}\left(\varepsilon^{3}\right)\) and \(\|\tilde{g}-t\|_{s}=\mathcal{O}\left(\varepsilon^{3}\right)\). Therefore,

\[|\sin\,\angle_{{}_{A}}(x,\tilde{x}+s)| =|\sin\,\angle_{{}_{A}}(x,\frac{x}{\alpha}+s-\tilde{f})|\] \[=\frac{\|X^{\perp}A(\frac{x}{\alpha}+s-\tilde{f})\|_{2}}{\|\tilde {x}+s\|_{{}_{A}}}=\frac{\|X^{\perp}A(\tilde{f}-s)\|_{2}}{\|\tilde{x}+s\|_{{}_{ A}}}\] \[\leq\frac{\|\tilde{f}-s\|_{{}_{A}}}{\|\tilde{x}+s\|_{{}_{A}}}= \mathcal{O}\left(\varepsilon^{3}\right),\]

where \(X^{\perp}=[x_{2},\dots,x_{m}]\). Similarly, we have \(|\sin\,\angle_{{}_{s}}(y,\tilde{y}+t)|=\mathcal{O}\left(\varepsilon^{3}\right)\). ## References

* (1) [PERSON]; [PERSON]; [PERSON] [PERSON] Canonical correlation analysis: An overview with application to learning methods. _Neural Comput._**2004**, _16_, 2639-2664. [CrossRef] [PubMed]
* (2) [PERSON] Relations between two sets of variates. _Biometrika_**1936**, _28_, 321-377. * (3) [PERSON]; [PERSON]; [PERSON]; [PERSON] Orthogonal canonical correlation analysis and applications. _Opt. Methods Softw._**2020**, _35_, 787-807. [CrossRef]
* (4) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] A tutorial on canonical correlation methods. _ACM Comput. Surv._**2017**, _50_, 1-33. [CrossRef]
* (5) [PERSON]; [PERSON]; [PERSON]; [PERSON] A self-consistent-field iteration for orthogonal CCA. _IEEE Trans. Pattern Anal. Mach. Intell._**2020**, 1-15.