## 4 Orthogonalized Maximal Projection Algorithm  
As seen from Equation (38), we must make \(\|\mathbf{A}\mathbf{u}\|^{2}\) as large as possible. \(\mathbf{A}\mathbf{u}=\mathbf{A}\mathbf{V}\mathbf{y}\), mentioned in Section 1, is the projection of \(\mathbf{r}_{0}\) on \(\mathcal{L}_{m}=\mathbf{A}\mathcal{K}_{m}\). Therefore we consider  
\[\max_{\mathbf{u}}\biggl{\{}f=\frac{(\mathbf{r}_{0}\cdot\mathbf{v})^{2}}{\| \mathbf{v}\|^{2}}\biggr{\}} \tag{67}\]  
to be the maximal projection of \(\mathbf{r}_{0}\) on the direction \(\mathbf{v}/\|\mathbf{v}\|\), where \(\mathbf{v}=\mathbf{A}\mathbf{u}\). In view of Equation (66), the maximization of \(f\) is equivalent to the minimization of \(a_{0}\). We can expand \(\mathbf{u}\in\mathbf{r}_{0}+\mathcal{K}_{m}\) via  
\[\mathbf{u}=\mathbf{r}_{0}+\mathbf{V}\mathbf{y}, \tag{68}\]  
which is slightly different from the descent vector \(\mathbf{u}=\mathbf{V}\mathbf{y}\in\mathcal{K}_{m}\) used in GMRES. Theorem 4.1: _For \(\mathbf{u}\in\mathbf{r}_{0}+\mathcal{K}_{m}(\mathbf{A},\mathbf{r}_{0})\), \(\mathbf{r}_{0}\
eq\mathbf{0}\), the optimal \(\mathbf{u}\) approximately satisfying Equation (2) and subjecting to Equation (67) is given in Equation (68), where_  
\[\mathbf{W}=\mathbf{A}\mathbf{V},\ \mathbf{C}=\mathbf{W}^{\mathsf{T}} \mathbf{W}, \tag{69}\] \[\mathbf{C}\mathbf{y}=\mathbf{W}^{\mathsf{T}}(\mathbf{r}_{0}- \mathbf{A}\mathbf{r}_{0}). \tag{70}\]  
Proof: Due to \(\mathbf{v}=\mathbf{A}\mathbf{u}\) and Equation (68),  
\[\mathbf{v}=\mathbf{v}_{0}+\mathbf{W}\mathbf{y}, \tag{71}\]  
where \(\mathbf{W}=\mathbf{A}\mathbf{V}\) and  
\[\mathbf{v}_{0}=\mathbf{A}\mathbf{r}_{0}. \tag{72}\]  
By Equation (71), one has  
\[\mathbf{r}_{0}\cdot\mathbf{v}=\mathbf{r}_{0}\cdot\mathbf{v}_{0}+ \mathbf{r}_{0}^{\mathsf{T}}\mathbf{W}\mathbf{y}, \tag{73}\] \[\|\mathbf{v}\|^{2}=\|\mathbf{v}_{0}\|^{2}+2\mathbf{v}_{0}^{ \mathsf{T}}\mathbf{W}\mathbf{y}+\mathbf{y}^{\mathsf{T}}\mathbf{C}\mathbf{y}, \tag{74}\]  
where  
\[\mathbf{C}:=\mathbf{W}^{\mathsf{T}}\mathbf{W}>\mathbf{0}\in\mathbb{R}^{m\times m}. \tag{75}\]  
Resorting to the maximality condition for \(f\) in Equation (67), we can obtain  
\[\mathbf{r}_{0}\cdot\mathbf{v}\mathbf{y}_{2}-\|\mathbf{v}\|^{2}\mathbf{y}_{1} =\mathbf{0}, \tag{76}\]where  
\[\mathbf{u}=\mathbf{r}_{0}+\mathbf{V}\mathbf{y}, \tag{68}\]0 \[\mathbf{u}=\mathbf{r}_{0}+\mathbf{V}\mathbf{y}, \tag{68}\]1  
From Equation (76), \(\mathbf{y}_{2}\) is equal to \(\mathbf{y}_{1}\):  
\[\mathbf{u}=\mathbf{r}_{0}+\mathbf{V}\mathbf{y}, \tag{68}\]2  
because of  
\[\mathbf{u}=\mathbf{r}_{0}+\mathbf{V}\mathbf{y}, \tag{68}\]3  
according to Corollary 1. Then, it follows from Equations (77)-(79) that  
\[\mathbf{u}=\mathbf{r}_{0}+\mathbf{V}\mathbf{y}, \tag{68}\]4  
which can be arranged to Equation (70). We pull back the minimization problem used in the GMRES method to the following one:  
\[\mathbf{u}=\mathbf{r}_{0}+\mathbf{V}\mathbf{y}, \tag{68}\]5  
which is derived from Equations (6) and (7). **Theorem 5**. : _For \(\mathbf{u}\in\mathbf{r}_{0}+\mathcal{K}_{m}(\mathbf{A},\mathbf{r}_{0})\), derived from the minimization problem in Equation (82), \(\mathbf{u}\) in Equation (2) is optimized to be that in Theorem 4._  
Proof. : From  
\[\mathbf{u}=\mathbf{r}_{0}+\mathbf{V}\mathbf{y}, \tag{68}\]6  
and Equations (78) and (77) it follows that  
\[\mathbf{u}=\mathbf{r}_{0}+\mathbf{V}\mathbf{y}, \tag{68}\]7  
which is just Equation (70). According to Theorems 4 and 5, the maximal projection is equivalent to the minimization of residual. Therefore, the maximal projection algorithm (MPA) is an extension of GMRES to a larger space of the affine Krylov subspace, not merely in the Krylov subspace.