## 4 A Numerical Algorithm
\(\mathbf{J}_{k}\) requires \(mnq\) operations; \(\mathbf{C}_{k}\) requires \(m^{2}q\) operations; \(\mathbf{D}_{k}\) requires \(m^{2}q\) operations; \(\mathbf{V}_{k}\) requires \(nm^{2}q\) operations; and \(\mathbf{P}_{k}\) requires \(nq^{2}\) operations. In each iteration, there are \(mn+mnq+2m^{2}q+nm^{2}q+nq^{2}\) operations. Denote the number of iterations by NI. The computational complexity is in total NI\(\times[mn+mnq+2m^{2}q+nm^{2}q+nq^{2}]\). It is known that \(m\) is a key parameter in the Krylov subspace. A proper choice of \(m\) can significantly enhance the convergence speed and accuracy. For ill-posed linear least-squares problems, there exists the best value of \(m\), but for well-posed linear least-squares problems, small values of \(m\) may slow down convergence. When \(m\) is increased, both the convergence speed and accuracy are increased. However, when \(m\) is increased, more computational power is required in the construction of the projection operator \(\mathbf{P}\) and the inversion matrix \(\mathbf{C}^{-1}\). We make some comments about the initial value of \(\mathbf{x}_{0}\). For under-determined systems, there are many solutions; hence, different choices of \(\mathbf{x}_{0}\) would generate different solutions. In general, as required in Equation (1), the minimal-norm solution can be obtained by setting \(\mathbf{x}_{0}=\mathbf{0}\), whose norm is zero \(\|\mathbf{x}_{0}\|=0\). For over-determined systems, we take \(\mathbf{x}_{0}=\mathbf{0}\) such that the initial residual is a non zero vector \(\mathbf{r}_{0}=\mathbf{b}\) and the DOA based on the residual \(\mathbf{r}_{k}\) is solvable. Indeed, the DOA is not sensitive to the initial value of \(\mathbf{x}_{0}\); we will take \(\mathbf{x}_{0}=\mathbf{0}\) for most problems unless otherwise specified. The following corollaries help the clarification of the DOA and are based on the DOS used in the residual Equation (111). Corollary 3: \(\mathbf{r}_{k+1}\) _is orthogonal to \(\mathbf{A}\mathbf{z}_{k}\) in Equation (111) and \(\mathbf{r}_{k+1}-\mathbf{r}_{k}\), i.e.,_  
\[\mathbf{A}\mathbf{z}=\mathbf{r}_{0}, \tag{111}\]0 \[\mathbf{A}\mathbf{z}=\mathbf{r}_{0}, \tag{111}\]1  
By Equation (111), the next \(\mathbf{r}_{k+1}\) is given by  
\[\mathbf{A}\mathbf{z}=\mathbf{r}_{0}, \tag{111}\]2where \(\mathbf{y}_{k}=\mathbf{A}\mathbf{z}_{k}\). By Equation (116), we can also derive Equation (119) as follows:  
\[\mathbf{A}\mathbf{z}=\mathbf{r}_{0}, \tag{111}\]3  
For Equation (111), Equation (105) is written as  
\[\mathbf{A}\mathbf{z}=\mathbf{r}_{0}, \tag{111}\]4  
and taking the \(\mathbf{y}_{k}\) inner product to Equation (119), we have  
\[\mathbf{A}\mathbf{z}=\mathbf{r}_{0}, \tag{111}\]5  
Inserting \(\mathbf{A}\mathbf{z}_{k}=\mathbf{r}_{k}-\mathbf{r}_{k+1}\) into Equation (117), we can prove Equation (118). The DOA is a good approximation of Equation (2) with a better descent direction \(\mathbf{z}_{k}\) in the VAKS. Corollary 4: _For the DOA, the convergence rate is_  
\[\mathbf{A}\mathbf{z}=\mathbf{r}_{0}, \tag{111}\]6  
Proof. : It follows from Equations (119) and (120) that  
\[\mathbf{A}\mathbf{z}=\mathbf{r}_{0}, \tag{111}\]7  
where \(\theta\) is the intersection angle between \(\mathbf{r}_{k}\) and \(\mathbf{y}_{k}=\mathbf{A}\mathbf{z}_{k}\). With help from Equation (120), we have  
\[\mathbf{A}\mathbf{z}=\mathbf{r}_{0}, \tag{111}\]8  
Obviously, \(\theta\
eq\pi/2\) because \(\|\mathbf{y}_{k}\|>0\). Then, Equation (123) can be further reduced to  
\[\mathbf{A}\mathbf{z}=\mathbf{r}_{0}, \tag{111}\]9  
Dividing both sides by \(\|\mathbf{r}_{k+1}\|^{2}\) and taking the square-roots of both sides, we can obtain Equation (122). Corollary 5: _The residual is decreased step by step,_  
\[\mathbf{A}\mathbf{z}=\mathbf{A}\mathbf{x}-\mathbf{A}\mathbf{x}_{0}=\mathbf{b}- \mathbf{A}\mathbf{x}_{0}=\mathbf{r}_{0}.\]0  
Proof.