# The Eigenspace Spectral Regularization Method for Solving Discrete Ill-Posed Systems

Authors: Fredrick Asenso Wireko; Benedict Barnes; Charles Sebil; Joseph Ackora-Prah
Journal: Journal of Applied Mathematics
 (2021)

---

## Abstract

This paper shows that discrete linear equations with Hilbert matrix operator, circulant matrix operator, conference matrix operator, banded matrix operator, TST matrix operator, and sparse matrix operator are ill-posed in the sense of Hadamard. Gauss least square method (GLSM), QR factorization method (QRFM), Cholesky decomposition method (CDM), and singular value decomposition (SVDM) failed to regularize these ill-posed problems. This paper introduces the eigenspace spectral regularization method (ESRM), which solves ill-posed discrete equations with Hilbert matrix operator, circulant matrix operator, conference matrix operator, and banded and sparse matrix operator. Unlike GLSM, QRFM, CDM, and SVDM, the ESRM regularizes such a system. In addition, the ESRM has a unique property, the norm of the eigenspace spectral matrix operatorκK=K−1K=1. Thus, the condition number of ESRM is bounded by unity, unlike the other regularization methods such as SVDM, GLSM, CDM, and QRFM.


---

## Body

## 1. Introduction

Many mathematical problems are classified as ill-posed problems. For instance, the discretization of linear ill-posed problems like the Fredholm integral equations of the first kind with a smooth kernel is ill-posed and hence referred to as a linear discrete ill-posed problem [1]. Moreover, image processing problems are mostly ill-posed when a blurred text or image is obtained. As a result, the image needs to be reconstructed to make a clear image. This study is concerned with the regularization of such linear discrete systems.Given that a system of linear equationsA:X⟶Y defined by
(1)Ax=b,where A is a matrix operator in Rm×n from a domain set X into a codomain set Y, x⊆X is a column vector in Rn, and b∈Y is also a column vector in Rm, is well-posed in the sense of Hadamard if
(1)
A is an surjective operator. Thus, the solution to the discrete equation (1) exists(2)
A is an injective operator. Thus, there is a unique solution to the discrete equation (1)(3)
the inverse,A−1:Y⟶X, is continuous. Thus, the solution to equation (1) is stable (see authors in [2])Although Hadamard provided the criteria for detecting an ill-posed problem, he did not provide any method for solving such a problem. The restoration of the well-posedness of a discrete system of the linear equation has received much attention since the beginning of the 20th century. Lavrentiev observed well-posed discrete equations as a system whose solution is sought on a compact subspace of a topological space which maps to another compact topological space. This type of well-posedness is generally referred to as conditionally well-posed discrete equations, [3].Based on the varying definitions of well-posedness of discrete equations, Tikhonov defines the solution spaceM as a compact set that is a subset of the Euclidean space. Still, there is no restriction on the data function, the vector that appears on the right-hand side of equation (1). The Tikhonov and Lavrentiev observations of well-posedness are generic definitions of Hadamard definitions. On the other hand, if a discrete equation violates one or more of the conditions stated above, then the equation is said to be ill-posed [4]. Equation (1) may have either no solution, or infinitely many solutions, or a unique solution.Ill-posed discrete systems of linear equations are numerically unstable as they come with a large condition number. The instability of these matrix operators woefully compel the solution vector of the system of linear equations to be highly sensitive to perturbations in either the matrix operator or the right hand side of the system. This observation has been made by several researchers in the scientific community (for example, see [1, 5, 6]).Ill-posed discrete equations can be regularized either by a direct approach or an iterative approach. Some direct regularization methods like Gauss least square method (GLSM) [7], QR factorization method (QRFM) [8], singular value decomposition method (SVDM) [9], and Cholesky decomposition method (CDM) [10]. In 1944, Tikhonov first constructed an iterative method that involves a regularization parameter for regularizing ill-posed discrete equations. There is no unique choice of the value of this parameter of even the same set of problems. In addition, the Tikhonov regularization method cannot restore the well-posedness of a discrete equation that does not have a solution.Theorem 1 (Gauss least square method).
IfA is an m×n matrix with rankA=n, then ATA is nonsingular and the discrete equation (1) has a unique least squares solution given by
(2)x^=ATA−1ATb.
See authors in [11].The GLSM fails to solve the problem of stability and also performs poorly when solving underdetermined discrete equations. Though the GLSM may regularize the system, the operator maintains to be unstable, which fails to satisfy the stability condition for a well-posed system. In the work of [12, 13], the GLSM has been extended to meet the Lipchitz condition to satisfy the boundedness condition.Theorem 2 (QR factorization method).
LetA be a m×n matrix with entries in F and linearly independent columns. Then, there exist a n×n matrix R and m×n matrix Q, both having entries in F, such that
(1)
A=QR(2)
QTQ=In, where T is the transpose(3)
R is a nonsingular upper triangular matrix with rkk>0(4)
The columns ofQ are an orthonormal basis for RA(5)
Q and R are unique (see author in [14])In the work of [15], the QR decomposition has been merged with the Cholesky decomposition method to provide a faster algorithm in computations. The matrix operator R, the upper triangular matrix, is closed in Rm. We can see that Theorem 2 dwells on the existence of the solution to a discrete equation. QRFM restores only the existence and the uniqueness of the discrete system of linear equations. Also, the QRFM requires the matrix operator of the discrete equation given to be a linearly independent matrix operator (see author in [16]).Theorem 3 (singular value decomposition method).
LetA be an m×n matrix. There exist an integer r,σ1≥σ2≥⋯≥σr>0, an m×m unitary matrix U, an n×n unitary matrix V, and an m×n matrix S, all of whose entries are 0 except Sii=σi,i=1,2,⋯,r, such that
(3)A=USVT.
See authors in [17].In the work of [18], the SVDM has been applied in a correspondence analysis using R. On the other hand, the SVDM does not produce stable results dealing with discrete equations with RankA<n or infinitely many solutions or no solution. The SVDM mostly has problems with the loss of small singular values, thereby giving undesirable answers in a functional space (see authors in [8, 19]).Theorem 4 (Cholesky factorization method).
Given a symmetric positive definite matrixA, there exists a lower triangular matrix L such that A=LLT, where L is the lower triangular matrix (Cholesky factor) and LT is the Cholesky factorization of A (see author in [20]).The Cholesky decomposition does not have a unique Cholesky decomposition for matrices that are not Hermitian and positive definite. Also, the CDM requires the matrix operator to be a symmetric positive definite matrix operator [10, 21]. This, therefore, suffices that the Cholesky method is unable to regularize some matrix operators and hence may still be ill-posed in the sense of Hadamard.All these regularization methods in restoring the well-posedness of the ill-posed discrete equation (1) fail to restore the existence, uniqueness, and stability conditions of well-posedness of a discrete system of linear equation [(7)]. In a worse case, these methods struggle to resolve the well-posedness of the ill-posed discrete system of linear equations with a sparse matrix operator. In numerical analysis, matrix operators with large condition numbers are prone to significant numerical errors and also very sensitive to variations in either the right-hand side or even the matrix operator of equation (1) (see authors in [5]).In addition, the GLSM, QRM, SVDM, and CDM fail to resolve the stability of the ill-posed discrete system of linear equation with Hilbert matrix, circulant matrix, conferences matrix, and the banded matrix. Since, after the applications of these methods to the discrete equation, the regularized matrix operator is mostly nearly singular and generally attributed to large condition numbers, the unstable solutions make these existing methods not reliable for approximating the solution of discrete equations.In the area of application, the GLSM, QRFM, SVDM, and CDM have not been much consistently used to regularize an ill-posed discrete equation with perturbed right-hand sideb in equation (1) (see authors in [4]). Notwithstanding this, the populous matrix operators like the Hilbert matrix operator, conference matrix operator, circulant matrix operator, and banded matrix operator occurring in a discrete equation have not been looked at and talked about their regularization process vis-a-vis regularization method. The sparse matrix operator arising in a discrete equation commonly studied by many authors across the globe, when solved with SVDM, GLSM, CDM, and QRFM, does not yield the desired solution in functional space.To overcome the myriad of ill-posed discrete equations raised in regularization, which solves these problems that are paramount, a method devoid of computational errors or minor computational errors is exciting and must be the focal point of regularizing discrete equations. This paper seeks to introduce a new regularization method, “the eigenspace spectral regularization method (ESRM),” for solving ill-posed discrete equations and check how efficient the ESRM is compared to the existing regularization methods.In this paper, Section1 contains a background to ill-posed problems. Also, in Section 2, we present the drawbacks of GLSM, QRFM, SVDM, and CDM when applied in regularizing discrete equations. Section 3 introduced the eigenspace spectral regularization method (ESRM) and used it to normalize discrete equations in which the GLSM, QRFM, SVDM, and CDM cannot solve. In addition, we also compare ESRM with GLSM, QRFM, SVDM, and CDM. Finally, Section 4 contains the conclusion of this paper.


## 2. The Ill-Posed Discrete System of Linear Equations and Drawbacks of Some Existing Methods

This section highlights definitions relevant to establishing ill-posed discrete equations and the regularization of such ill-posed discrete equations. Here, rigorous proofs for the ill-posed discrete equation are provided.Several discrete systems of linear equations are discussed here. We give examples of ill-posedness of the discrete equations in the Hadamard sense.Example 1.
Consider the discrete equations below:(4)x+y=1,x−y=3,−x+2y=−2.To determine if there is a solution to equation (4), we compute its augmented matrix as
(5)A∣b=111−1−1213−2,and the corresponding reduced row-echelon form is given also as
(6)A∣b=1001002−12.By the Rouche-Capelli theorem, the discrete equation (4) does not have a solution in R3 since the rank of the matrix A=2, which is less than the rank of the extended matrix, κ=3. Hence, the discrete equation (2) is ill-posed in the sense of Hadamard.Example 2.
Also, consider the discrete equation(7)x+3y+2z=1,2x+8y+6z=6,x+2y+z=1.Similarly, the discrete equation in (7) does not have a solution in R3 because the rank of the matrix is r=2, which is less than the rank of the extended matrix. Hence, the discrete equation (7) is ill-posed in the sense of Hadamard.Example 3.
Consider the discrete equation below:(8)x+2y+z+w=8,x+2y+2z−w=12,2x+4y+6w=4.It is also confirmed that the discrete equation in (8) has no solution in R3, thus, it is classified as an ill-posed discrete system of linear equations.Example 4.
Moreover, the discrete system of linear equation (9) below
(9)−3x−5y+36z=10,−x+7z=5,x+y−10z=−4has its reduced row-echelon computed as
(10)A∣b=10−701−3000−510.It was observed that the unknown variablez is arbitrary. This implies that the discrete equation in (5) has an infinitely many solution. Hence, the discrete equation in (9) is also classified as ill-posed.Example 5.
Similar observations were made considering the discrete equation in (11)
(11)x1+x4=4,2x3+x6=6,2x4=2,which has no unique solution in R6. Thus, it is classified as an ill-posed discrete system of linear equation.Example 6.
Again, we consider the discrete system of linear equation with a Toeplitz, Symmetric, and Tridiagonal (TST) matrix operator:(12)3x+2y=6,2x+3y+2z=7,2y+3z+2w=3,2z+3w+2u=2,2w+3u=4.The augmented matrix for equation (12) is given as
(13)A∣b=330002320002320002320002367324,and the corresponding reduced row-echelon form is given also as(14)A∣b=3200001.67200000.620000−3.67200004.0963−0.646.18.It was observed that the system of linear equation (12) with a TST matrix operator is injective since both the rank of the matrix operator and the rank of the extended matrix is RankA=5. Notwithstanding this, the inverse TST matrix operator is not continuous since condA=13.9282. Therefore, the system of linear equation with a TST operator is classified as an ill-posed discrete system of linear equation.Example 7.
On the other hand, it was observed that the discrete equation with Hilbert matrix operator in equation (15)
(15)x+12y+13z=3,12x+13y+14z=2,13x+14y+15z=5yields a unique solution but not stable since it has condA=504. Hence, the discrete equation is ill-posed in the sense of Hadamard.Example 8.
As seen in the discrete equation (15), similar observations were made given the discrete equation in (16) which has a banded matrix operator:
(16)x−8y+0z+0w=3,2x−2y−7z+0w=−5,0x+7y+3z−6w=1,0x+0y+8z−7w=6.The discrete equation (15) yields an unstable solution since it has condA=12.6139. Hence, the discrete equation is ill-posed in the sense of Hadamard.Example 9.
Also, the discrete equation with a conference matrix operator(17)0x+y+z+w=4,x+0y+z−w=5,x+y+0z+w=3,x−y+z+0w=2produces an unstable solution vector since it has condA=2.2361. This therefore makes the discrete system of linear equation in (17) classified as an ill-posed problem.Example 10.
Similarly, the discrete system of linear equation in (18) which has a sparse matrix operator
(18)1.1x1+0.5x7=3,1.9x2+0.5x7=2,2.6x3+0.5x7=5,7.8x3+0.6x4=−1,1.5x4+2.7x5=6,1.6x1+0.4x5=−4,0.9x6+1.7x7=9produced an unstable solution as condA=44.4164. Thus, it is classified as an ill-posed problem.Example 11.
Finally, the discrete system of linear equation with a circulant matrix operator(19)x+2y+3z+4w=−1,2x+y+4z+3w=1,3x+4y+2z+w=3,4x+3y+z+2w=5resulted in similar conclusions as observed in the discrete system of linear equations (15), (15), (17), and (18).


### 2.1. Drawbacks of Existing Methods of Regularizing Ill-Posed Discrete Equations

In this subsection, the shortcomings of GLSM, QRFM, SVDM, and CDM are highlighted. Though the abovementioned regularization methods are used in solving ill-posed discrete system of linear equations, they pose serious limitations.


#### 2.1.1. Gauss Least Square Method

The GLSM is applied to the ill-posed discrete system of linear equations discussed in Section2. Applying the GLSM to the discrete equation (4), it was observed that the GLSM operator
(20)ATA=3−2−26yields a unique solution, that is, x=1.7143y=−0.4286, on the grounds that ∣ATA∣=14, thus, nonsingular. On the other hand, condATA=3.5, which is far from one. This implies that the solution of the discrete equation in (2) is not stable when solved using the GLSM. Hence, the GLSM fails to restore the well-posedness of the discrete equation in (4).Also applying the GLSM to the discrete equation (7), the existence and uniqueness conditions of well-posedness are restored. Notwithstanding this,
(21)condATA=227037379566481184,and also,
(22)∣ATA∣=−0.000000000000011657341758565503671.It was observed that the GLSM operator’s determinant is nearly degenerate. This implies instability of the solution of the discrete equation in (7) when solved by GLSM. Hence, the GLSM failed to regularize the discrete system of linear equation to be homomorphic.Similarly, the discrete system of linear equation in (8) yields
(23)condATA=76509541981751072,and also,
(24)∣ATA∣=0.Thus, the GLSM could not restore the second and third conditions of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (8).The GLSM as seen in the others fails to regularize the ill-posed discrete system of linear equation (9) since it yields
(25)condATA=1822.2651.Thus, an unstable solution which is sensitive to perturbations since the condition number is far from unity.Similar results were observed as the GLSM was applied to the discrete system of linear equation (11) as
(26)condATA=∞,and also,
(27)∣ATA∣=0.Thus, it fails to restore the unique condition of well-posedness, and again, the condition number is huge and hence fails to meet the third requirement of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (11).It was further observed that the GLSM performed poorly in regularizing the ill-posed system of linear equation as the GLSM matrix operator yielded(28)condATA=193.9948.Thus, the GLSM fails to regularize the ill-posed system of linear equation(12).Also, applying the GLSM to the discrete system of linear equation (15), it yielded
(29)condATA=274635.5061.Thus, a significant condition number fails to meet the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (15).Again, the GLSM when applied to the discrete system of linear equation produced(30)condATA=93955.72,which is far from unity and, thus, fails to meet the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (16).Considering also the discrete system of linear equation (17), it was observed that
(31)condATA=5,which is, thus, very sensitive to perturbations and fails to satisfy the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (17).Again, in equation (18), the
(32)condATA=1972.8151,is very large and thus fails to satisfy the third condition of well-posedness. Thus, the GLSM fails to regularize the discrete equation in (18).Finally, given the discrete equation (19), it is also observed that
(33)condATA=50,thus failing to satisfy the stability condition of well-posedness. Thus, the GLSM could not regularize the discrete equation (19).


#### 2.1.2. QR Factorization Method

In this subsection, the concentration is on the regularization of discrete equations discussed in the previous section using the QRFM.Applying the QRFM to the discrete system of linear equation, we observed that(34)R=−1.73211.154702.160200,and therefore not feasible to proceed with this method. Hence, the QRFM fails to regularize the discrete equation in (4). Similar observations were made when applying the QRFM to the discrete equations in (7), (8), and (11).On the other hand, the QRFM when applied to the discrete system of linear equation (9) yields
(35)R=3.31664.8242−37.68890−1.65144.954300−0.0000,which produces the unique solution x=−4,y=2, and z=0. However, the solution is unstable since
(36)condR=231004359338148256.Hence, QRFM could not regularize the discrete equation (9) in the sense of Hadamard.Again, the QRFM matrix operator when applied to the ill-posed system of linear equation (12) given as
(37)R=−3.6056−3.3282−1.1094000−2.4337−3.4136−1.6436000−2.0290−3.1491−1.97140002.09332.766900001.2074was observed to be unstable as it reported a condition number of
(38)condR=13.9282.Hence, the QRFM fails to regularize the ill-posed discrete system of linear equation (12) to be homeomorphic.Similar trends are observed on the stabilities of solutions of discrete equations in (15), (16), (17), (18), and (19) when solved by the QRFM. The QRFM unequivocally fails to regularize the discrete system of linear equations in (4) to (18) from the analysis above.


#### 2.1.3. Cholesky Decomposition Method

Also, the CDM is applied to the ill-posed discrete system of linear equations in order to make them homomorphic. Unfortunately, the discrete system of linear equation (4) could not be regularized by the CDM since equation (4) fails to meet the positive definite condition of CDM. Similar limitations were encountered when applying the CDM to the discrete equations (7), (8), (9), (11), (12), (16), (17), (18), and (19).However, applying the CDM to the discrete system of linear equation (15) yields the matrix operator
(39)R=1.00000.50000.333300.28870.2887000.0745.It was observed that the CDM matrix operator yields a unique solution but very unstable since(40)condR=22.8923.Therefore, the discrete system of linear equation (15) could not be regularized using the CDM.


#### 2.1.4. Singular Value Decomposition

The SVDM presents similar results; thus, it resolves the existence and uniqueness conditions of well-posedness in the discrete equations (4)–(19) but fails to satisfy the stability condition of well-posedness.So far, these preliminary results signify that all of them, GLSM, SVDM, QRFM, and CDM, fail to restore the existence, uniqueness, and stability conditions of well-posedness in the discrete system of linear equations discussed in Section2. It was also observed that the GLSM, SVDM, QRFM, and CDM all perform poorly in regularizing discrete system of linear equations with Hilbert, conference, circulant, banded, and sparse matrix operators.


## 2.1. Drawbacks of Existing Methods of Regularizing Ill-Posed Discrete Equations

In this subsection, the shortcomings of GLSM, QRFM, SVDM, and CDM are highlighted. Though the abovementioned regularization methods are used in solving ill-posed discrete system of linear equations, they pose serious limitations.


### 2.1.1. Gauss Least Square Method

The GLSM is applied to the ill-posed discrete system of linear equations discussed in Section2. Applying the GLSM to the discrete equation (4), it was observed that the GLSM operator
(20)ATA=3−2−26yields a unique solution, that is, x=1.7143y=−0.4286, on the grounds that ∣ATA∣=14, thus, nonsingular. On the other hand, condATA=3.5, which is far from one. This implies that the solution of the discrete equation in (2) is not stable when solved using the GLSM. Hence, the GLSM fails to restore the well-posedness of the discrete equation in (4).Also applying the GLSM to the discrete equation (7), the existence and uniqueness conditions of well-posedness are restored. Notwithstanding this,
(21)condATA=227037379566481184,and also,
(22)∣ATA∣=−0.000000000000011657341758565503671.It was observed that the GLSM operator’s determinant is nearly degenerate. This implies instability of the solution of the discrete equation in (7) when solved by GLSM. Hence, the GLSM failed to regularize the discrete system of linear equation to be homomorphic.Similarly, the discrete system of linear equation in (8) yields
(23)condATA=76509541981751072,and also,
(24)∣ATA∣=0.Thus, the GLSM could not restore the second and third conditions of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (8).The GLSM as seen in the others fails to regularize the ill-posed discrete system of linear equation (9) since it yields
(25)condATA=1822.2651.Thus, an unstable solution which is sensitive to perturbations since the condition number is far from unity.Similar results were observed as the GLSM was applied to the discrete system of linear equation (11) as
(26)condATA=∞,and also,
(27)∣ATA∣=0.Thus, it fails to restore the unique condition of well-posedness, and again, the condition number is huge and hence fails to meet the third requirement of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (11).It was further observed that the GLSM performed poorly in regularizing the ill-posed system of linear equation as the GLSM matrix operator yielded(28)condATA=193.9948.Thus, the GLSM fails to regularize the ill-posed system of linear equation(12).Also, applying the GLSM to the discrete system of linear equation (15), it yielded
(29)condATA=274635.5061.Thus, a significant condition number fails to meet the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (15).Again, the GLSM when applied to the discrete system of linear equation produced(30)condATA=93955.72,which is far from unity and, thus, fails to meet the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (16).Considering also the discrete system of linear equation (17), it was observed that
(31)condATA=5,which is, thus, very sensitive to perturbations and fails to satisfy the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (17).Again, in equation (18), the
(32)condATA=1972.8151,is very large and thus fails to satisfy the third condition of well-posedness. Thus, the GLSM fails to regularize the discrete equation in (18).Finally, given the discrete equation (19), it is also observed that
(33)condATA=50,thus failing to satisfy the stability condition of well-posedness. Thus, the GLSM could not regularize the discrete equation (19).


### 2.1.2. QR Factorization Method

In this subsection, the concentration is on the regularization of discrete equations discussed in the previous section using the QRFM.Applying the QRFM to the discrete system of linear equation, we observed that(34)R=−1.73211.154702.160200,and therefore not feasible to proceed with this method. Hence, the QRFM fails to regularize the discrete equation in (4). Similar observations were made when applying the QRFM to the discrete equations in (7), (8), and (11).On the other hand, the QRFM when applied to the discrete system of linear equation (9) yields
(35)R=3.31664.8242−37.68890−1.65144.954300−0.0000,which produces the unique solution x=−4,y=2, and z=0. However, the solution is unstable since
(36)condR=231004359338148256.Hence, QRFM could not regularize the discrete equation (9) in the sense of Hadamard.Again, the QRFM matrix operator when applied to the ill-posed system of linear equation (12) given as
(37)R=−3.6056−3.3282−1.1094000−2.4337−3.4136−1.6436000−2.0290−3.1491−1.97140002.09332.766900001.2074was observed to be unstable as it reported a condition number of
(38)condR=13.9282.Hence, the QRFM fails to regularize the ill-posed discrete system of linear equation (12) to be homeomorphic.Similar trends are observed on the stabilities of solutions of discrete equations in (15), (16), (17), (18), and (19) when solved by the QRFM. The QRFM unequivocally fails to regularize the discrete system of linear equations in (4) to (18) from the analysis above.


### 2.1.3. Cholesky Decomposition Method

Also, the CDM is applied to the ill-posed discrete system of linear equations in order to make them homomorphic. Unfortunately, the discrete system of linear equation (4) could not be regularized by the CDM since equation (4) fails to meet the positive definite condition of CDM. Similar limitations were encountered when applying the CDM to the discrete equations (7), (8), (9), (11), (12), (16), (17), (18), and (19).However, applying the CDM to the discrete system of linear equation (15) yields the matrix operator
(39)R=1.00000.50000.333300.28870.2887000.0745.It was observed that the CDM matrix operator yields a unique solution but very unstable since(40)condR=22.8923.Therefore, the discrete system of linear equation (15) could not be regularized using the CDM.


### 2.1.4. Singular Value Decomposition

The SVDM presents similar results; thus, it resolves the existence and uniqueness conditions of well-posedness in the discrete equations (4)–(19) but fails to satisfy the stability condition of well-posedness.So far, these preliminary results signify that all of them, GLSM, SVDM, QRFM, and CDM, fail to restore the existence, uniqueness, and stability conditions of well-posedness in the discrete system of linear equations discussed in Section2. It was also observed that the GLSM, SVDM, QRFM, and CDM all perform poorly in regularizing discrete system of linear equations with Hilbert, conference, circulant, banded, and sparse matrix operators.


## 2.1.1. Gauss Least Square Method

The GLSM is applied to the ill-posed discrete system of linear equations discussed in Section2. Applying the GLSM to the discrete equation (4), it was observed that the GLSM operator
(20)ATA=3−2−26yields a unique solution, that is, x=1.7143y=−0.4286, on the grounds that ∣ATA∣=14, thus, nonsingular. On the other hand, condATA=3.5, which is far from one. This implies that the solution of the discrete equation in (2) is not stable when solved using the GLSM. Hence, the GLSM fails to restore the well-posedness of the discrete equation in (4).Also applying the GLSM to the discrete equation (7), the existence and uniqueness conditions of well-posedness are restored. Notwithstanding this,
(21)condATA=227037379566481184,and also,
(22)∣ATA∣=−0.000000000000011657341758565503671.It was observed that the GLSM operator’s determinant is nearly degenerate. This implies instability of the solution of the discrete equation in (7) when solved by GLSM. Hence, the GLSM failed to regularize the discrete system of linear equation to be homomorphic.Similarly, the discrete system of linear equation in (8) yields
(23)condATA=76509541981751072,and also,
(24)∣ATA∣=0.Thus, the GLSM could not restore the second and third conditions of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (8).The GLSM as seen in the others fails to regularize the ill-posed discrete system of linear equation (9) since it yields
(25)condATA=1822.2651.Thus, an unstable solution which is sensitive to perturbations since the condition number is far from unity.Similar results were observed as the GLSM was applied to the discrete system of linear equation (11) as
(26)condATA=∞,and also,
(27)∣ATA∣=0.Thus, it fails to restore the unique condition of well-posedness, and again, the condition number is huge and hence fails to meet the third requirement of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (11).It was further observed that the GLSM performed poorly in regularizing the ill-posed system of linear equation as the GLSM matrix operator yielded(28)condATA=193.9948.Thus, the GLSM fails to regularize the ill-posed system of linear equation(12).Also, applying the GLSM to the discrete system of linear equation (15), it yielded
(29)condATA=274635.5061.Thus, a significant condition number fails to meet the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (15).Again, the GLSM when applied to the discrete system of linear equation produced(30)condATA=93955.72,which is far from unity and, thus, fails to meet the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (16).Considering also the discrete system of linear equation (17), it was observed that
(31)condATA=5,which is, thus, very sensitive to perturbations and fails to satisfy the third condition of well-posedness. Hence, the GLSM fails to regularize the discrete equation in (17).Again, in equation (18), the
(32)condATA=1972.8151,is very large and thus fails to satisfy the third condition of well-posedness. Thus, the GLSM fails to regularize the discrete equation in (18).Finally, given the discrete equation (19), it is also observed that
(33)condATA=50,thus failing to satisfy the stability condition of well-posedness. Thus, the GLSM could not regularize the discrete equation (19).


## 2.1.2. QR Factorization Method

In this subsection, the concentration is on the regularization of discrete equations discussed in the previous section using the QRFM.Applying the QRFM to the discrete system of linear equation, we observed that(34)R=−1.73211.154702.160200,and therefore not feasible to proceed with this method. Hence, the QRFM fails to regularize the discrete equation in (4). Similar observations were made when applying the QRFM to the discrete equations in (7), (8), and (11).On the other hand, the QRFM when applied to the discrete system of linear equation (9) yields
(35)R=3.31664.8242−37.68890−1.65144.954300−0.0000,which produces the unique solution x=−4,y=2, and z=0. However, the solution is unstable since
(36)condR=231004359338148256.Hence, QRFM could not regularize the discrete equation (9) in the sense of Hadamard.Again, the QRFM matrix operator when applied to the ill-posed system of linear equation (12) given as
(37)R=−3.6056−3.3282−1.1094000−2.4337−3.4136−1.6436000−2.0290−3.1491−1.97140002.09332.766900001.2074was observed to be unstable as it reported a condition number of
(38)condR=13.9282.Hence, the QRFM fails to regularize the ill-posed discrete system of linear equation (12) to be homeomorphic.Similar trends are observed on the stabilities of solutions of discrete equations in (15), (16), (17), (18), and (19) when solved by the QRFM. The QRFM unequivocally fails to regularize the discrete system of linear equations in (4) to (18) from the analysis above.


## 2.1.3. Cholesky Decomposition Method

Also, the CDM is applied to the ill-posed discrete system of linear equations in order to make them homomorphic. Unfortunately, the discrete system of linear equation (4) could not be regularized by the CDM since equation (4) fails to meet the positive definite condition of CDM. Similar limitations were encountered when applying the CDM to the discrete equations (7), (8), (9), (11), (12), (16), (17), (18), and (19).However, applying the CDM to the discrete system of linear equation (15) yields the matrix operator
(39)R=1.00000.50000.333300.28870.2887000.0745.It was observed that the CDM matrix operator yields a unique solution but very unstable since(40)condR=22.8923.Therefore, the discrete system of linear equation (15) could not be regularized using the CDM.


## 2.1.4. Singular Value Decomposition

The SVDM presents similar results; thus, it resolves the existence and uniqueness conditions of well-posedness in the discrete equations (4)–(19) but fails to satisfy the stability condition of well-posedness.So far, these preliminary results signify that all of them, GLSM, SVDM, QRFM, and CDM, fail to restore the existence, uniqueness, and stability conditions of well-posedness in the discrete system of linear equations discussed in Section2. It was also observed that the GLSM, SVDM, QRFM, and CDM all perform poorly in regularizing discrete system of linear equations with Hilbert, conference, circulant, banded, and sparse matrix operators.


## 3. Main Results

In this section, the Eigenspace Spectral Regularization Method (ESRM) is introduced and also proved. The ESRM is then applied to regularize the discrete equations in (4)–(18). Detailed discussions on how the ESRM restores the existence, uniqueness, and stability conditions of well-posedness in the ill-posed discrete system of linear equations are illustrated.Theorem 5 (the eigenspace spectral regularization method).
Suppose the discrete equation (1) where Am×n is the matrix operator, x is a column vector in Rn and b is also a column vector in Rm. The eigenspace spectral matrix operator K= eigenspace of A1, where A1=A∗A. Then, a discrete equation in (1) has a stable unique solution given by
(41)x=K∗K−1K∗∧,where ∧ is the spectrum of A1.Proof.
Given a discrete equation (1), set eigenspace of K=eigenspaceofA1 and ∧ the corresponding spectrumof A1, where A1=A∗A. Replacing A and b in equation (1) by K and ∧, respectively, we have
(42)Kx=∧.
Multiplying both sides of the above equation by its adjoint operator ofK∗ yields
(43)K∗Kx=K∗∧.
Since the columns ofK are linearly independent, thus, K≠0, it implies that K−1 exists. This suffices that
(44)K∗K−1KTKx=K∗K−1K∗∧implies
(45)X=K∗K−1K∗∧.
This is a stable unique solution. In matrix theoryA∗=AT, AT is the transpose of the matrix operator A.As indicated in the proof above, the ESRM is able to restore the existence, uniqueness, and stability conditions of well-posedness in the ill-posed discrete system of linear equations. We thus apply the ESRM to the ill-posed discrete system of equations discussed in Section2 above. Note that all calculations were done by using MATLAB.Firstly, applying the ESRM to the discrete equation (4), we compute
(46)A1=ATA=3−2−26,which yields the ESRM matrix operator as
(47)K=−0.8944−0.4472−0.44720.8944and also the right hand side as
(48)∧=2.00007.0000.We thus compute(49)KTK−1=1.0000001.0000,KT∧=−4.91935.3666,which yields the unique solution vector
(50)x=−4.91935.3666,since the determinant of the ESRM matrix operator is nonzero.Also, the condition number of the ESRM matrix operator is computed as(51)condK=1,which is, thus, a stable solution which is not sensitive to perturbations. Hence, all three conditions of well-posedness are satisfied by the ESRM. Therefore, by applying the ESRM to the discrete system of linear equation (4) yields a homomorphic system.Also, the ESRM performed better when applied to the discrete equation (7). The ESRM matrix operator was computed as
(52)K=−0.5774−0.78780.21450.5774−0.20810.7895−0.57740.57970.5750,and also the measured data was evaluated as
(53)∧=−0.00000.5102123.4898.The ESRM again yields the unique solution(54)x=−71.002371.480071.4092,since∣K∣=1, which is, thus, a nondegenerate system. Also, the condition number of the ESRM matrix operator was computed as
(55)condK=1.This implies that the above solution is stable. Hence, the ESRM regularizes a discrete equation in (7). Thus, all three conditions of well-posedness are met. Therefore, the solution of the discrete equation in (7) is well-posed using ESRM.The ESRM also regularizes the discrete system of linear equation (8) when applied to it. The matrix operator was calculated as
(56)K=0.23010.8944−0.24660.29370.4602−0.4472−0.49320.5874−0.7670−0.0000−0.63840.0648−0.383500.53700.7514,and the measured data also computed as
(57)∧=−0.0000−0.000011.635461.3646.It was observed firstly that the ESRM satisfies the solvability condition of well-posedness as it yields the solution vector(58)x=−32.4562−0.000025.524846.8624.In addition, the ESRM yields a unique solution as it produces a nondegenerate matrix operator with∣K∣=1. Also, the condition number of the ESRM matrix operator is computed as
(59)condK=1.Thus, the ESRM produces a stable solution to the discrete system of linear equation (8). Hence, the discrete equation (8) is a well-posed system in the sense of Hadamard by the ESRM.Similarly, applying the ESRM to the discrete equation (9) with the ESRM matrix operator
(60)K=−0.9113−0.4027−0.0854−0.39060.9114−0.1300−0.13020.08510.9878yields a nondegenerate system, on the grounds that ∣K∣=−1.Also,(61)condK=1,which is, thus, a stable solution. Therefore, by the ESRM, the discrete equation (9) is a homeomorphic system of linear equation in the Hadamard sense.Moreover, the ESRM matrix operator derived from the discrete system of linear equation (11) is computed as
(62)ATA=A1=100000020101000010100100002001000100=100100000000004002100200000000002001,KeigenspaceofA1=0000.85070.5257001.00000000000.4472000.8944000−0.52570.850701.00000000000−0.8944000.4472,which yields ∣K∣=1; thus, the discrete equation (11) has a unique solution. The condition number for the ESRM matrix operator was also calculated as
(63)condK=1,which indicates that the unique solution is very stable. Hence, the discrete system of linear equation (11) is a homomorphic system.Subsequently, the ESRM performed better than all other regularization methods discussed in this paper, as the ESRM matrix operator(64)KeigenspaceofA1=−0.28870.50000.5774−0.50000.28870.5000−0.5000−0.0000−0.50000.5000−0.5774−0.0000−0.57740.00000.57740.50000.5000−0.00000.50000.5000−0.2887−0.50000.55740.50000.2887was observed to be continuous as ∣K∣=1. Also, the ESRM inverse operator yielded
(65)condK=1,which is, thus, a continuous mapping, that is, a stable solution.It was further observed that, by applying the ESRM to the discrete system of linear equation (12) with the ESRM matrix operator
(66)K=−0.12770.54740.82700.7137−0.52830.4599−0.6887−0.64900.3233also yields a unique and stable solution since
(67)∣K∣=−1,and
(68)condK=1.Hence, the discrete equation (12) results in a homeomorphic discrete equation after applying the ESRM.Again, applying the ESRM to the discrete equation (15) produces the ESRM matrix operator
(69)K=−0.93050.3564−0.04010.0739−0.1198−0.11340.8504−0.4996−0.2295−0.5170−0.4907−0.6628−0.2590−0.76990.18570.5528.It was again observed that∣K∣=−1; thus the discrete equation (15) has a unique solution. In addition,
(70)condK=1.Thus, the discrete equation (15) has a stable solution. Therefore, the discrete equation (15) is a well-posed discrete equation in the Hadamard sense.In addition, applying the ESRM to the discrete equation (16), we obtain the ESRM matrix operator as
(71)K=−0.707100−0.707100.70710.707100.707100−0.70710−0.70710.70710.We further observed that the ESRM yields a unique and stable solution, thus, restoring all three conditions of well-posedness in the discrete system of linear equation (16).The ESRM when applied to the discrete system of linear equation (17) which has a sparse matrix operator yields a nondegenerate ESRM matrix operator
(72)K=0.0582−0.08660.34570.87040.32100.09430.0002−0.0366−0.10910.7128−0.47660.50160.00010.0003−0.06010.01030.01250.0085−0.0124−0.03830.99720.8299−0.2611−0.0353−0.1027−0.04370.47370.0717−0.45820.1510−0.0174−0.0378−0.01190.87480.0048−0.2720−0.8517−0.3541−0.02660.27290.00010.00050.13790.4051−0.4953−0.05040.75410.00040.0202.It is again observed that∣K∣=1, implying that the discrete equation (17) has a unique solution. Furthermore, the condition number was computed as
(73)condK=1.Thus, the ESRM restores the existence, uniqueness, and stability conditions of well-posedness in the discrete system of linear equation (17).Last but not least, applying the ESRM to the discrete equation (18), we obtain the ESRM matrix operator
(74)K=−0.5478−0.4471−0.50000.50000.54780.4471−0.50000.50000.4471−0.54780.50000.5000−0.44710.54780.50000.5000,which yields a unique solution since
(75)∣K∣=1.It was again observed that(76)condK=1,which is, thus, an inversely bounded solution. Hence, the discrete equation (18) has been regularized by the ESRM to be a homeomorphic system of linear equation in the sense of Hadamard.


### 3.1. Main Findings in This Paper

This section concentrates on the performance of the ESRM as against the GLSM, QRFM, SVDM, and CDM in regularizing the above-discussed discrete system of linear equations. The results are shown in “Tables1(a)–1(c).”Table 1
The comparison of ESRM and GLSM, QRFM, SVDM, and CDM for regularizing ill-posed discrete equations.
(a)MatrixESRMGLSMQRFMSVDMCDMx+y=1x−y=3−x+2y=−2It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularizex+3y+2z=12x+8y+6z=6x+2y+z=1It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularizex+2y+z+w=8x+2y+2z−w=122x+4y+6w=4It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize−3x−5y+36z=10−x+7z=5x+y−10z=−4It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize(b)MatrixESRMGLSMQRFMSVDMCDMx1+x4=42x3+x6=62x4=2It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularizex+12y+13z=312x+13y+14z=213x+14y+15z=5It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize3x+2y=6x+3y+2z=7y+3z+2w=3z+3w+2u=2w+3u=4It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularizex−8y+0z+0w=3x−2y−7z+0w=−5x+7y+3z−6w=1x+0y+8z−7w=6It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize(c)MatrixESRMGLSMQRFMSVDMCDM0x+y+z+w=4x+0y+z−w=5x+y+0z+w=3x−y+z+0w=2It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize1.1x1+0.5x7=31.9x2+0.5x7=22.6x3+0.5x7=57.8x3+0.6x4=−11.5x4+2.7x5=61.6x1+0.4x5=−40.9x6+1.7x7=9It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularizex+2y+3z+4w=−1x+y+4z+3w=1x+4y+2z+w=3x+3y+z+2w=5It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize


## 3.1. Main Findings in This Paper

This section concentrates on the performance of the ESRM as against the GLSM, QRFM, SVDM, and CDM in regularizing the above-discussed discrete system of linear equations. The results are shown in “Tables1(a)–1(c).”Table 1
The comparison of ESRM and GLSM, QRFM, SVDM, and CDM for regularizing ill-posed discrete equations.
(a)MatrixESRMGLSMQRFMSVDMCDMx+y=1x−y=3−x+2y=−2It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularizex+3y+2z=12x+8y+6z=6x+2y+z=1It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularizex+2y+z+w=8x+2y+2z−w=122x+4y+6w=4It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize−3x−5y+36z=10−x+7z=5x+y−10z=−4It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize(b)MatrixESRMGLSMQRFMSVDMCDMx1+x4=42x3+x6=62x4=2It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularizex+12y+13z=312x+13y+14z=213x+14y+15z=5It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize3x+2y=6x+3y+2z=7y+3z+2w=3z+3w+2u=2w+3u=4It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularizex−8y+0z+0w=3x−2y−7z+0w=−5x+7y+3z−6w=1x+0y+8z−7w=6It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize(c)MatrixESRMGLSMQRFMSVDMCDM0x+y+z+w=4x+0y+z−w=5x+y+0z+w=3x−y+z+0w=2It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize1.1x1+0.5x7=31.9x2+0.5x7=22.6x3+0.5x7=57.8x3+0.6x4=−11.5x4+2.7x5=61.6x1+0.4x5=−40.9x6+1.7x7=9It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularizex+2y+3z+4w=−1x+y+4z+3w=1x+4y+2z+w=3x+3y+z+2w=5It regularizesFail to regularizeFail to regularizeFail to regularizeFail to regularize


## 4. Conclusion

We have shown that only the ESRM could regularize the discrete system of linear equations (4), (7), (8), (9), (11), (12), (15), (16), (17), (18), and (19), respectively. Thus, the ESRM restores the existence, uniqueness, and stability conditions of well-posedness in the sense of Hadamard. However, the existing methods of regularization: GLSM, QRFM, SVDM, and CDM, failed to regularize the discrete system of linear equations (4) to (19). In addition, we observed that ESRM has a unique property; thus, the norm of the ESRM matrix operator κK=K−1K=1. Also, the condition number of ESRM is bounded by unity, unlike the other regularization methods such as SVDM, GLSM, CDM, and QRFM.



---

*Source: 4373290-2021-11-26.xml*