## 7 Conclusions  
Least-squares problems arise in a lot of applications, and many iterative algorithms are already available to seek their solutions. In this paper, a new concept of a varying affine Krylov subspace was introduced, which is different to the fixed-type affine Krylov subspace used in the CG-type numerical solution of over-determined least-squares problems. In the \(m+1\)-dimensional varying affine Krylov subspace, a closed-form double optimal solution in a simple form in Equation (42) was derived and further reduced to Equation (80), which was obtained by two minimizations in Equations (40) and (41). We analyzed a key equation (72) to link these two optimizations together. The double optimal solution is an excellent approximation, as verified for least-squares problems. The iterative DOA was developed, which leads to step-by-step absolute convergence. In each iterative step, by merely inverting an \(m\times m\) matrix with a small value of \(m\), the computational cost of the DOA is very low, lower than other methods. The Moore-Penrose inverses of non-square matrices were also derived by using the varying affine Krylov subspace method. For over-determined least-squares problems, we proposed a variant of the Karush-Kuhn-Tucker equation, which uses the partial pivoting Gaussian elimination method and is better than the original Karush-Kuhn-Tucker equation and CG-type numerical methods of CGNR (CGLS) and CGNE ([PERSON]'s method). It is very important that the DOA can be applied to both rectangular systems with \(q>n\) or \(q<n\). The novelties involved in this paper are as follows:  
* A double-optimization-based iterative algorithm for least-squares problems was developed in a varying affine Krylov subspace. * For dense large-size least-squares problems, the proposed iterative DOA is efficient and accurate. * The DOA can be applied to both rectangular systems with extremal cases of \(q\ll n\) or \(q\gg n\). * A variant of the Karush-Kuhn-Tucker equation was presented, which is an improved version of the original Karush-Kuhn-Tucker equation. \begin{table}
\begin{tabular}{c c c c c c} \hline \(\boldsymbol{(n,q)}\) & **(5,6)** & **(5,8)** & **(5,10)** & **(7,25)** & **(8,25)** \\ \hline GEM1 & \(1.80\times 10^{-6}\) & \(5.18\times 10^{-7}\) & \(4.57\times 10^{-7}\) & \(2.20\times 10^{-3}\) & \(1.55\times 10^{-3}\) \\ \hline GEM2 & \(3.15\times 10^{-12}\) & \(2.27\times 10^{-13}\) & \(1.02\times 10^{-12}\) & \(1.29\times 10^{-10}\) & \(4.70\times 10^{-9}\) \\ \hline CGNR & \(4.98\times 10^{-8}\) & \(1.62\times 10^{-7}\) & \(3.05\times 10^{-7}\) & \(4.83\times 10^{-8}\) & \(7.64\times 10^{-8}\) \\ \hline CGNE & \(3.52\times 10^{-12}\) & \(1.64\times 10^{-12}\) & \(1.88\times 10^{-12}\) & \(4.83\times 10^{-8}\) & \(7.64\times 10^{-8}\) \\ \hline DOA & \(1.88\times 10^{-13}\) & \(2.55\times 10^{-12}\) & \(7.81\times 10^{-13}\) & \(2.72\times 10^{-9}\) & \(1.48\times 10^{-9}\) \\ \hline SVD & \(4.85\times 10^{-12}\) & \(1.52\times 10^{-12}\) & \(9.09\times 10^{-13}\) & \(3.08\times 10^{-10}\) & \(1.52\times 10^{-8}\) \\ \hline \(\|\mathbf{Q}^{\mathrm{T}}\mathbf{Q}-\mathbf{I}_{n}\|\) & \(8.65\times 10^{-12}\) & \(4.57\times 10^{-12}\) & \(5.24\times 10^{-12}\) & \(5.66\times 10^{-10}\) & \(1.89\times 10^{-8}\) \\ \hline QR & \(1.05\times 10^{-6}\) & \(3.50\times 10^{-7}\) & \(2.27\times 10^{-7}\) & \(2.46\times 10^{-3}\) & \(5.68\times 10^{-1}\) \\ \hline \end{tabular}
\end{table} 
Table 12: Example 10: ME with different \((n,q)\) obtained by GEM1, GEM2, CGNR, CGNE, the DOA and QR. The idea of varying the affine Krylov subspace is useful to find a better solution of linear algebraic equations based on the mechanism of double optimization. In the future, we may extend the new methods of DOSs and the DOA to other linear matrix equations. The limitations are that there are several matrix multiplications needed to construct the projection operator \(\mathbf{P}\) in the Krylov subspace and the high computational cost for inversion of the matrix \(\mathbf{C}^{-1}\) with dimension \(m\).