## 2 Inverse Solution Algorithms
### Overdetermined or Square Systems  
#### 2.1.1 TSVD and Least-Squares Problems  
The most basic approach [16; 17; 18] to solve linear inverse problems is the least-squares method in which the following cost function is minimized:\[J=\|\mathbf{e}\|_{2}^{2}=\mathbf{e}^{H}\mathbf{e}, \tag{2}\]  
where \(\mathbf{e}=\mathbf{p}-\mathbf{A}\mathbf{s}\) denotes the error vector and \"\(\|\cdot\|_{2}\)\" denotes the vector 2-norm. If matrix \(\mathbf{A}\) is of full-column rank, the least-squares solution can be written as  
\[\mathbf{\hat{s}}=\left[\mathbf{A}^{H}\mathbf{A}\right]^{-1}\mathbf{A}^{H} \mathbf{p}. \tag{3}\]  
More generally, by the TSVD of  
\[\mathbf{A}=\mathbf{U}\Sigma\mathbf{V}^{H}\] \[\mathbf{\hat{s}}=\mathbf{A}^{+}\mathbf{p}, \tag{4}\]  
where \(\Sigma\) represents a diagonal matrix with singular values at its diagonal entries, \(\mathbf{U}\) and \(\mathbf{V}\) represent unitary matrices, and \(\mathbf{A}^{+}\) represents the pseudoinverse of the matrix \(\mathbf{A}\) defined as  
\[\mathbf{A}^{+}=\mathbf{\
abla}\mathbf{\Sigma}^{+}\mathbf{U}^{H}, \tag{5}\]  
where \(\mathbf{\Sigma}^{+}=\mathbf{diag}\big{[}\sigma_{1}^{-1},\dots,\sigma_{r}^{-1}, \mathbf{0},\dots,\mathbf{0}\big{]}\in\mathbb{C}^{N\times M}\), \(\mathbf{r}=\text{rank}(\mathbf{A})\)[30; 31]. Note that the expression of Equation (4) is sufficiently general that it always provides the minimum-norm least-squares solution, even when the matrix \(\mathbf{A}\) is rank-deficient. The square of the residual error becomes  
\[\mathbf{e}_{LS}^{2}=\|\mathbf{p}-\mathbf{A}\mathbf{\hat{s}}\|_{2}^{2}=\|( \mathbf{I}-\mathbf{A}\mathbf{A}^{+})\mathbf{p}\|_{2}^{2}=\sum_{i=r+1}^{M} \Big{|}\mathbf{u}_{i}^{H}\mathbf{p}\Big{|}^{2} \tag{6}\]  
with \(\mathbf{u}_{i}\) being the \(i\)th column of \(\mathbf{U}\) and \(\mathbf{I}\) being the identity matrix. In practice, the matrix \(\mathbf{A}\) can contain small singular values and can be very ill-conditioned, which leads to numerical instability during the inversion of \(\mathbf{A}\). Two common methods to cope with the ill-posedness of the problem are TSVD and TIKR. Briefly, the TSVD method is simply to discard small singular values of the matrix \(\mathbf{A}\), whereas TIKR involves attempts to minimize the following cost function [16]:  
\[J=\|\mathbf{A}\mathbf{\hat{s}}-\mathbf{p}\|_{2}^{2}+\beta^{2}\|\mathbf{s}\|_{ 2}^{2}, \tag{7}\]  
where the regularization parameter \(\beta\) weights the residual norm and the solution norm. After some manipulation, we derive with the following optimal solution:  
\[\mathbf{\hat{s}}=\left(\mathbf{A}^{H}\mathbf{A}+\beta^{2}\mathbf{I}\right)^{ -1}\mathbf{A}^{H}\mathbf{p} \tag{8}\]  
This result can be rewritten in terms of the TSVD of \(\mathbf{A}\) as follows:  
\[\mathbf{\hat{s}}=\left(\mathbf{A}^{H}\mathbf{A}+\beta^{2}\mathbf{I}\right)^{ -1}\mathbf{A}^{H}\mathbf{p}=\sum_{i=1}^{N}f_{i}\sigma_{i}^{-1}\alpha_{i} \mathbf{v}_{i}, \tag{9}\]  
where \(\mathbf{A}=\mathbf{U}\mathbf{S}\mathbf{V}^{H}=\sum\limits_{i=1}^{N}\sigma_{i} \mathbf{u}_{i}\mathbf{v}_{i}^{H},\) with \(\mathbf{u}_{i}\) and \(\mathbf{v}_{i}\) being the \(i\)th column partition of the matrices \(\mathbf{U}\) and \(\mathbf{V}\), \(\alpha_{i}=\mathbf{u}_{i}^{H}\mathbf{p}\), where  
\[f_{i}(\beta)=\frac{\sigma_{i}^{2}}{\sigma_{i}^{2}+\beta^{2}}=\frac{1}{1+\left( \beta/\sigma_{i}\right)^{2}} \tag{10}\]  
denotes the filter function. It can also be shown that the minimum residual vector can be written as  
\[\mathbf{\hat{s}}=\left[\mathbf{A}^{H}\mathbf{A}\right]^{-1}\mathbf{A}^{H} \mathbf{p}. \tag{3}\]0where \(\mathbf{r}_{\perp}=(\mathbf{I}-\mathbf{A}\mathbf{A}^{+})\mathbf{p}=\sum\limits_{i=M+1 }^{M}\alpha_{i}\mathbf{u}_{i}\) is the residual vector of the components of \(\mathbf{p}\) orthogonal to \(\{\mathbf{u}_{1}\cdots\mathbf{u}_{n}\}\).