## 3 Maximally Decreasing Residual and Automatically Preserving Orthogonality
Inserting Equation (49) into Equation (50) yields  
\[\mathbf{r}_{k+1}=\mathbf{r}_{k}-\eta\mathbf{A}\mathbf{u}_{k}. \tag{20}\]4  
Inserting \(\gamma\) into Equation (49) and using \(\mathbf{v}=\mathbf{A}\mathbf{u}\), we can derive  
\[\mathbf{r}_{k+1}=\mathbf{r}_{k}-\eta\mathbf{A}\mathbf{u}_{k}. \tag{20}\]5  
where  
\[\mathbf{r}_{k+1}=\mathbf{r}_{k}-\eta\mathbf{A}\mathbf{u}_{k}. \tag{20}\]6  
By applying the forward Euler scheme to Equation (52), we have  
\[\mathbf{r}_{k+1}=\mathbf{r}_{k}-\eta\mathbf{A}\mathbf{u}_{k}. \tag{20}\]7  
where  
\[\mathbf{r}_{k+1}=\mathbf{r}_{k}-\eta\mathbf{A}\mathbf{u}_{k}. \tag{20}\]8An iterative form of Equation (54) is  
\[\mathbf{r}_{k+1}=\mathbf{r}_{k}-\eta\mathbf{A}\mathbf{u}_{k}. \tag{20}\]9  
It generates the residual form:  
\[\|\mathbf{r}_{k+1}\|^{2}=\|\mathbf{r}_{k}\|^{2}-(2\eta\mathbf{r}_{k}^{\mathrm{T}} \mathbf{A}\mathbf{u}_{k}-\eta^{2}\|\mathbf{A}\mathbf{u}_{k}\|^{2})=:\|\mathbf{r }_{k}\|^{2}-h. \tag{21}\]0  
Taking the squared norms of both sides yields  
\[\|\mathbf{r}_{k+1}\|^{2}=\|\mathbf{r}_{k}\|^{2}-(2\eta\mathbf{r}_{k}^{\mathrm{T}} \mathbf{A}\mathbf{u}_{k}-\eta^{2}\|\mathbf{A}\mathbf{u}_{k}\|^{2})=:\|\mathbf{r }_{k}\|^{2}-h. \tag{21}\]1  
Dividing both sides by \(\|\mathbf{r}_{k}\|^{2}\) renders  
\[\|\mathbf{r}_{k+1}\|^{2}=\|\mathbf{r}_{k}\|^{2}-(2\eta\mathbf{r}_{k}^{\mathrm{T}} \mathbf{A}\mathbf{u}_{k}-\eta^{2}\|\mathbf{A}\mathbf{u}_{k}\|^{2})=:\|\mathbf{r }_{k}\|^{2}-h. \tag{21}\]2  
where  
\[\|\mathbf{r}_{k+1}\|^{2}=\|\mathbf{r}_{k}\|^{2}-(2\eta\mathbf{r}_{k}^{\mathrm{T}} \mathbf{A}\mathbf{u}_{k}-\eta^{2}\|\mathbf{A}\mathbf{u}_{k}\|^{2})=:\|\mathbf{r }_{k}\|^{2}-h. \tag{21}\]3  
By means of Equation (47), Equation (59) changes to  
\[\|\mathbf{r}_{k+1}\|^{2}=\|\mathbf{r}_{k}\|^{2}-(2\eta\mathbf{r}_{k}^{\mathrm{T}} \mathbf{A}\mathbf{u}_{k}-\eta^{2}\|\mathbf{A}\mathbf{u}_{k}\|^{2})=:\|\mathbf{r }_{k}\|^{2}-h. \tag{21}\]4  
If we take  
\[\|\mathbf{r}_{k+1}\|^{2}=\|\mathbf{r}_{k}\|^{2}-(2\eta\mathbf{r}_{k}^{\mathrm{T}} \mathbf{A}\mathbf{u}_{k}-\eta^{2}\|\mathbf{A}\mathbf{u}_{k}\|^{2})=:\|\mathbf{r }_{k}\|^{2}-h. \tag{21}\]5  
Equation (61) is satisfied. In terms of \(\beta_{k}\), Equation (56) is recast to  
\[\|\mathbf{r}_{k+1}\|^{2}=\|\mathbf{r}_{k}\|^{2}-(2\eta\mathbf{r}_{k}^{\mathrm{T}} \mathbf{A}\mathbf{u}_{k}-\eta^{2}\|\mathbf{A}\mathbf{u}_{k}\|^{2})=:\|\mathbf{r }_{k}\|^{2}-h. \tag{21}\]6  
Noticing \(\mathbf{v}_{k}=\mathbf{A}\mathbf{u}_{k}\), Equation (63) is just the iterative Equation (18) for ROGMRES. **Remark 3**. : _For the factor defined in Equation (23), \(\eta_{k}\) plays a vital role in Equation (34) to dominate the orthogonal property of GMRES, and also in Equation (35) to control the residual decreasing property of GMRES. When GMRES blows up for some problems, by introducing \(\eta_{k}\) in GMRES, we can stabilize the iterations to avoid blowing up. In this viewpoint, \(\eta_{k}\) is a stabilization factor for GMRES._  
Inspired by Remark 3, two simple methods can be developed for GMRES and GMRES(m): we can compute \(\eta_{k}\) at each step, and consider \(|\eta_{k}-1|>10^{-10}\) or \(\eta_{k}<1/2\) to be the stopping criteria of the iterations; they are labeled as GMRES(\(\eta\)) and GMRES(m,\(\eta\)), respectively. **Remark 4**.