# A Jacobi-Davidson Method for Large Scale Canonical Correlation Analysis
## 3 The Main Algorithm
### Correction Equation
A computed approximate eigenpair \((\theta_{i},\tilde{z}_{i})\) is considered converged when its relative residual norm

\[\eta(\theta_{i},\tilde{z}_{i})=\frac{\|r_{s}^{\mu}\|_{1}+\|r_{s}^{\mu}\|_{1}} {(\|C\|_{1}+\theta_{i}\|A\|_{1})\|\tilde{x}_{i}\|_{1}+(\|C\|_{1}+\theta_{i}\| B\|_{1})\|\tilde{y}_{i}\|_{1}}\leq 10^{-8}. \tag{28}\]

All the experiments in this paper are executed on a Ubuntu 12.04 (64 bit) Desktop-Intel(R) Core(TM) i7-6700 [EMAIL_ADDRESS] GHz, 32 GB of RAM using MATLAB 2010a with machine epsilon \(2.22\times 10^{-16}\) in double-precision floating point arithmetic. **Example 1**. : _We first examine Theorem 3 by using two pairs of data matrices \(S_{a}\) and \(S_{b}\) which come from a publicly available handwritten numerals dataset ([[https://archive.ics.uci.edu/ml/datasets/Multiple+Features](https://archive.ics.uci.edu/ml/datasets/Multiple+Features)]([https://archive.ics.uci.edu/ml/datasets/Multiple+Features](https://archive.ics.uci.edu/ml/datasets/Multiple+Features))). It consists of features handwritten numerals ('0'-'9') and each digit has 200 patterns. Each pattern is represented by six different feature sets, i.e., Fou, Fac, Kar, Pix, Zer and Mor. Two pairs of feature sets Fou-Zer and Pix-Fou are chosen for \(S_{a}\) and \(S_{b}\), respectively, such that \(S_{a}\in\mathbb{R}^{76\times d}\) and \(S_{b}\in\mathbb{R}^{47\times d}\) in Fou-Zer, and \(S_{a}\in\mathbb{R}^{240\times d}\) and \(S_{b}\in\mathbb{R}^{76\times d}\) in Pix-Fou with \(d=2000\). To make the numerical example repeatable, the initial vectors are set to be_

\[u_{0}=x_{1}+10^{-3}\times\mathsf{ones}(m,1)\quad\text{and}\quad v_{0}=y_{1}+1 0^{-3}\times\mathsf{ones}(n,1)\]

_where \(m\) and \(n\) are the dimension of \(S_{a}\) and \(S_{b}\), respectively, \(\mathsf{ones}\) is MATLAB built-in function, and \([x_{1}^{\mathrm{T}},y_{1}^{\mathrm{T}}]^{\mathrm{T}}\) is computed by MATLAB's function eig on (4) and considered to be the \"exact\" eigenvector for testing purposes. The corrected Equation (22) in Algorithm 1 is solved by direct methods, such as Gaussian elimination, and the solution \([s^{\mathrm{T}},t^{\mathrm{T}}]^{\mathrm{T}}\) by such methods is regarded as \"exactly\" in this example. Figure 1 plots \(\sin\bigtriangleup_{\iota}(x_{1},\tilde{x}_{1})\) and \(\sin\bigtriangleup_{\iota}(y_{1},\tilde{y}_{1})\) in the first three iterations of Algorithm 1 for computing first canonical weight vector of Fou-Zer and Pix-Fou. It is clearly shown by Figure 1 that the convergence of Algorithm 1 is very fast when the initial vectors are enough close to the exact vectors, and the cubical convergence of Algorithm 1 appears in the third iteration._

**Example 2**. : _As stated in Algorithm 1, the implementation of JDCCA involves solving the correction Equation (22) in every step. Direct solvers mentioned in Example 1 referring to \(\mathcal{O}\left((m+n)^{3}\right)\) operations are prohibitively expensive in solving large-scale sparse linear systems. In such a case, iterative methods, such as MINRES method which is simply GMRES [39] applied to symmetric linear systems, are usually preferred. In this example, we report the effect of the number of steps in the solution of the correction equation,

Figure 1: Convergence behavior of Algorithm 1 for computing the first canonical weight vector of Fou-Zer and Pix-Fou.