# A Jacobi-Davidson Method for Large Scale Canonical Correlation Analysis
## 3 The Main Algorithm
### Correction Equation
denoted by \(n_{g}\), on the total number of matrix-vector products (denoted by \"#nvp\"), outer iteration number (denoted by \"#iter\"), and CPU time in seconds for Algorithm 1 to compute the first 10 canonical weight vectors of the test problems appeared in Table 1. Table 1 presents three face datasets, i.e., ORL ([[https://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html](https://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html)]([https://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html](https://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html)),) FERET ([[http://www.nist.gov/itl/iad/ig/colorferet.cfn](http://www.nist.gov/itl/iad/ig/colorferet.cfn)]([http://www.nist.gov/itl/iad/ig/colorferet.cfn](http://www.nist.gov/itl/iad/ig/colorferet.cfn))) and Yale ([[https://computervisiononline.com/dataset/1105138686](https://computervisiononline.com/dataset/1105138686)]([https://computervisiononline.com/dataset/1105138686](https://computervisiononline.com/dataset/1105138686))) datasets. The ORL database contains 400 face images of 40 distinct persons. For each individual, there are 10 different gray scale images with \(92\times 112\) pixels. These images are collected by volunteers at different times, different lighting and different facial expressions (blinking or closed eyes, smiling or no-smiling, wearing glasses or no-glasses). In order to apply CCA, as numerical experiments in [40], the ORL dataset is partitioned into two groups. We select the first five images per individual as the first view to generate the data matrix \(S_{a}\), while the remaining for \(S_{b}\). Similarly, we get data matrices \(S_{a}\) and \(S_{b}\) for the FERET and Yale datasets. The numbers of row and column of \(S_{a}\) and \(S_{b}\) are detailed in Table 1._

_In this example, we set the initial vectors \(u_{0}=\mathsf{ones}(m,1)\) and \(v_{0}=\mathsf{ones}(n,1)\) with \(s_{\max}=30\) and \(s_{\min}=10\) for restarting and simply take regularization parameter \(\kappa_{a}=\kappa_{b}=10^{-4}\). We let MINRES steps \(n_{g}\) vary from to 5 to 40, and collect the numerical results in Figure 2. As expected, the number of total outer iterations decreases as \(n_{g}\) increases, while the total number of matrix-vector products does not change monotonically with \(n_{g}\). It depends on the degree of reduction of outer iterations by the increasing of \(n_{g}\). In addition, it is shown by Figure 2 that the total #nvp is not a unique deciding factor on the total CPU time. When \(n_{g}\) is larger, the significantly reduced #iter leads to smaller total CPU time. For these three test examples, the MINRES steps \(n_{g}\) around 15 to 25 appear to be cost-effective, further increasing \(n_{g}\) over 40 usually does not have significant effect. The least efficient case is when \(n_{g}\) is too small._

**Example 3**. : _In this example, we compare Algorithm 1, i.e., JDCCA, with Jacobi-Davidson QZ type method [41] (JDQZ) for the large scale symmetric positive definite pencil \(\{K,M\}\) defined in (4) to compute the first 10 canonical weight vectors of the test problems appeared in Table 1 with MINRES steps \(n_{g}=20\). We take \(u_{0}=\mathsf{ones}(m,1)\) and \(v_{0}=\mathsf{ones}(n,1)\) in Algorithm 1 and the initial vector \(\mathsf{ones}(m+n,1)\) for the JDQZ algorithm, and compute the same relative residual norms \(\eta(\theta_{i},\tilde{z}_{i})\). The corresponding numerical results are plotted in Figure 3. For these three test problems, it is suggested by Figure 3 that Algorithm 1 always outperforms the JDQZ algorithm. Other experiments that we tested with different test problems and MINRES steps not reported here also illustrate our points._ \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Problems** & **ORL** & **FERET** & **Yale** \\ \hline \(m\) & 10,304 & 6400 & 10,000 \\ \(n\) & 10,304 & 6400 & 10,000 \\ \(d\) & 200 & 600 & 75 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Test problems.