## 3 A Double Optimal Solution
### Estimation of Residual Error  
Equation (80) is written as  
\[\mathbf{x}=\alpha_{0}(\mathbf{u}_{0}-\mathbf{V}\mathbf{A}\mathbf{u}_{0})+ \mathbf{V}\mathbf{b}, \tag{91}\]  
where  
\[\alpha_{0}=\frac{1}{\lambda_{0}}=\frac{\mathbf{b}^{\mathsf{T}}\mathbf{A} \mathbf{u}_{0}-\mathbf{b}^{\mathsf{T}}\mathbf{P}\mathbf{A}\mathbf{u}_{0}}{\| \mathbf{A}\mathbf{u}_{0}\|^{2}-\mathbf{u}_{0}^{\mathsf{T}}\mathbf{A}^{\mathsf{ T}}\mathbf{P}\mathbf{A}\mathbf{u}_{0}}, \tag{92}\]  
by using Equations (65) and (72). Let us investigate the residual square:  
\[\|\mathbf{b}-\mathbf{A}\mathbf{x}\|^{2}=\|\mathbf{y}-\mathbf{b}\|^{2}=\| \mathbf{y}\|^{2}-2\mathbf{b}^{\mathsf{T}}\mathbf{y}+\|\mathbf{b}\|^{2}, \tag{93}\]where  
\[\mathbf{y}=\mathbf{A}\mathbf{x}=\alpha_{0}\mathbf{A}\mathbf{u}_{0}-\alpha_{0} \mathbf{P}\mathbf{A}\mathbf{u}_{0}+\mathbf{P}\mathbf{b}. \tag{94}\]  
is obtained from Equation (91) by using Equation (70). From Equation (94), it follows that  
\[\|\mathbf{y}\|^{2}=\alpha_{0}^{2}(\mathbf{u}_{0}^{\mathrm{T}} \mathbf{A}^{\mathrm{T}}\mathbf{A}\mathbf{u}_{0}-\mathbf{u}_{0}^{\mathrm{T}} \mathbf{A}^{\mathrm{T}}\mathbf{P}\mathbf{A}\mathbf{u}_{0})+\mathbf{b}^{\mathrm{ T}}\mathbf{P}\mathbf{b}, \tag{95}\] \[\mathbf{b}^{\mathrm{T}}\mathbf{y}=\alpha_{0}\mathbf{b}^{\mathrm{ T}}\mathbf{A}\mathbf{u}_{0}-\alpha_{0}\mathbf{b}^{\mathrm{T}}\mathbf{P} \mathbf{A}\mathbf{u}_{0}+\mathbf{b}^{\mathrm{T}}\mathbf{P}\mathbf{b}. \tag{96}\]  
In the first equation, we take Equation (73) into account. Inserting them into Equation (93) leads to  
\[\|\mathbf{b}-\mathbf{A}\mathbf{x}\|^{2}=\alpha_{0}^{2}(\mathbf{u}_{0}^{ \mathrm{T}}\mathbf{A}^{\mathrm{T}}\mathbf{A}\mathbf{u}_{0}-\mathbf{u}_{0}^{ \mathrm{T}}\mathbf{A}^{\mathrm{T}}\mathbf{P}\mathbf{A}\mathbf{u}_{0})+2\alpha _{0}(\mathbf{b}^{\mathrm{T}}\mathbf{P}\mathbf{A}\mathbf{u}_{0}-\mathbf{b}^{ \mathrm{T}}\mathbf{A}\mathbf{u}_{0})+\|\mathbf{b}\|^{2}-\mathbf{b}^{\mathrm{ T}}\mathbf{P}\mathbf{b}. \tag{97}\]  
Finally, inserting Equation (92) for \(\alpha_{0}\) into the above equation yields  
\[\|\mathbf{b}-\mathbf{A}\mathbf{x}\|^{2}=\|\mathbf{b}\|^{2}-\mathbf{b}^{ \mathrm{T}}\mathbf{P}\mathbf{b}-\frac{(\mathbf{b}^{\mathrm{T}}\mathbf{A} \mathbf{u}_{0}-\mathbf{b}^{\mathrm{T}}\mathbf{P}\mathbf{A}\mathbf{u}_{0})^{2} }{\mathbf{u}_{0}^{\mathrm{T}}\mathbf{A}^{\mathrm{T}}\mathbf{A}\mathbf{u}_{0}- \mathbf{u}_{0}^{\mathrm{T}}\mathbf{A}^{\mathrm{T}}\mathbf{P}\mathbf{A}\mathbf{ u}_{0}}. \tag{98}\]  
If \(\mathbf{P}\approx\mathbf{I}_{q}\), the value of \(\|\mathbf{b}-\mathbf{A}\mathbf{x}\|^{2}\) in Equation (98) is made as small as possible:  
\[\|\mathbf{b}-\mathbf{A}\mathbf{x}\|^{2}\approx 0. \tag{99}\]  
**Theorem 3**. : _For the double optimal solution \(\mathbf{x}\in\mathcal{K}_{m}^{\prime}\) derived in Equation (42) of the least-squares problem (2), the squared-norm of the error vector \(\mathbf{e}\):_  
\[\|\mathbf{e}\|^{2}=\|\mathbf{b}\|^{2}-\frac{(\mathbf{b}^{\mathrm{T}}\mathbf{ A}\mathbf{x})^{2}}{\|\mathbf{A}\mathbf{x}\|^{2}}, \tag{100}\]  
_and the squared-norm of the residual vector \(\mathbf{r}=\mathbf{b}-\mathbf{A}\mathbf{x}\) are equal, i.e.,_  
\[\alpha_{0}=\frac{1}{\lambda_{0}}=\frac{\mathbf{b}^{\mathsf{T}}\mathbf{A} \mathbf{u}_{0}-\mathbf{b}^{\mathsf{T}}\mathbf{P}\mathbf{A}\mathbf{u}_{0}}{\| \mathbf{A}\mathbf{u}_{0}\|^{2}-\mathbf{u}_{0}^{\mathsf{T}}\mathbf{A}^{\mathsf{ T}}\mathbf{P}\mathbf{A}\mathbf{u}_{0}}, \tag{92}\]0  
_Moreover_,  
\[\alpha_{0}=\frac{1}{\lambda_{0}}=\frac{\mathbf{b}^{\mathsf{T}}\mathbf{A} \mathbf{u}_{0}-\mathbf{b}^{\mathsf{T}}\mathbf{P}\mathbf{A}\mathbf{u}_{0}}{\| \mathbf{A}\mathbf{u}_{0}\|^{2}-\mathbf{u}_{0}^{\mathsf{T}}\mathbf{A}^{\mathsf{ T}}\mathbf{P}\mathbf{A}\mathbf{u}_{0}}, \tag{92}\]1  
Proof.