# Least Squares Based Iterative Algorithm for the Coupled Sylvester Matrix Equations

Authors: Hongcai Yin; Huamin Zhang
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

By analyzing the eigenvalues of the related matrices, the convergence analysis of the least squares based iteration is given for solving the coupled Sylvester equationsAX+YB=C and DX+YE=F in this paper. The analysis shows that the optimal convergence factor of this iterative algorithm is 1. In addition, the proposed iterative algorithm can solve the generalized Sylvester equation AXB+CXD=F. The analysis demonstrates that if the matrix equation has a unique solution then the least squares based iterative solution converges to the exact solution for any initial values. A numerical example illustrates the effectiveness of the proposed algorithm.


---

## Body

## 1. Introduction

Matrix equations arise in systems and control, such as Lyapunov matrix equation and Riccati equation. How to solve these matrix equations becomes a major field of the matrix computations [1–4]. Main points of this field are the decompositions and transformations of the matrices, eigenvalues and eigenvectors, and Krylov subspace [5–8]. Other points contain the algorithms and the convergence analysis. The algorithms provided a set of operation steps, and according to these steps it can find a solution of a matrix equation within finite steps in given error bounds [9–11]. The convergence analysis offered more details of an algorithm, and in general, these details indicated some new research areas [12–15].The direct method and the indirect method are the two main approaches of solving the matrix equations [16–18]. With the development of the requirement of the computation and the matrix theory, the indirect method or iterative method becomes the main approach of solving the matrix equations [19, 20].Iterative method is active in studying other engineering problems. Particularly, the iterative method can be used to identify systems [21–23] and estimate parameter of systems [24–28]. For example, iterative method is useful in the identification [29–32] and parameter estimation [33–36] of linear and nonlinear systems [37–46].The Jacobi and Gauss-Seidel iterative methods were discussed in the literature. By extending the Jacobi and Gauss-Seidel iterative methods, Ding and his coworkers recently presented a large family of the least squares based iterative methods for solving the matrix equationsAx=b and AXB=F [47–49]. It has been proved that these least squares based iterative solutions always converge fast to the exact ones as long as the unique solutions exist. But the range of the convergence factor is still open. Motivated by the importance of this algorithm, we develop a new way to prove the convergence of the least squares based iterative algorithm for the coupled Sylvester matrix equations AX+YB=C and DX+YE=F. According to the new proof, we obtain the optimal convergence factor of this iterative algorithm and extend the iterative algorithm to solve the generalized Sylvester matrix equation AXB+CXD=F. The algorithm in this paper can be extended to other more general and complex matrix equations [50–52].The paper is organized as follows. Section2 gives some preliminaries. Section 3 presents a new proof to the least squares iterative algorithm for solving the coupled Sylvester matrix equations AX+YB=C and DX+YE=F. Section 4 revises this algorithm to solve the equation AXB+CXD=F. Section 5 gives an example to illustrate the effectiveness of the proposed results. Finally, we offer some concluding remarks in Section 6.


## 2. Basic Preliminaries

Some symbols and lemmas are introduced first.In is the identity matrix of size n×n. I is an identity matrix with an appropriate size. λ[M] denotes the eigenvalue set of matrix M. |X|=det⁡[X] denotes the matrix determinant. ∥X∥ denotes the Frobenius norm of X and is defined as ∥X∥2∶=tr⁡[XXT]. For an m×n matrix
(1)X=[x1,x2,…,xn]∈Rm×n,xi∈Rm;col[X] denotes an mn-dimensional vector formed by
(2)(col[X])T=[x1T,x2T,…,xnT]∈Rmn.A⊗B represents the Kronecker product of matrices A and B. A formula involved the vec-operator col and the Kronecker product is col[AXB]=(BT⊗A)col[X] [53].If matrixATA is invertible, then A(ATA)-1AT is symmetric and idempotent. To be more specific, we have the following lemma.Lemma 1.
For the symmetric matrixA(ATA)-1AT, there exists an orthogonal matrix Q such that
(3)QT[A(ATA)-1AT]Q=diag⁡[1,…,1,0,…,0]=:Λ.
Moreover, rank⁡[A(ATA)-1AT]=rank⁡[Λ]=n.


## 3. The Coupled Sylvester Matrix Equations

In this section, we will prove the convergence of the least squares iterative algorithm for solving the following coupled Sylvester matrix equations:(4)AX+YB=C,DX+YE=F,
where A,D∈Rm×m, B,E∈Rn×n, and C,F∈Rm×n are known and X,Y∈Rm×n are to be determined. By using the hierarchical identification principle, the least squares based iterative algorithm for solving (4) is given by [47]
(5)X(k)=X(k-1)+μ(ATA+DTD)-1[AD]T×[C-Y(k-1)B-AX(k-1)F-Y(k-1)E-DX(k-1)],Y(k)=Y(k-1)+μ[C-AX(k-1)-Y(k-1)B,Fmmmmmmm-DX(k-1)-Y(k-1)E]×[B,E]T(BBT+EET)-1,0<μ<2λmax⁡[N-1M].
Here, N=diag⁡[In⊗(ATA+DTD),(BBT+EET)⊗Im] and
(6)M=[In⊗(ATA+DTD)BT⊗AT+ET⊗DTB⊗A+E⊗D(BBT+EET)⊗Im].
To initialize the algorithm, we take X(0) and Y(0) as some small real matrices, for example, X(0)=Y(0)=10-61m×n with 1m×n being an m×n matrix whose elements are all 1.Theorem 2.
If (4) has unique solutions X and Y, then for any initial values X(0) and Y(0) the iterative solutions X(k) and Y(k) given by iteration in (5) converge to X and Y; that is,
(7)lim⁡k→∞X(k)=X,lim⁡k→∞Y(k)=Y;
or the error matrices X(k)-X and Y(k)-Y converge to zero; in this case, the optimal convergence factor is μ=1.Proof.
Define two error matrices(8)X~(k)∶=X(k)-X,Y~(k)∶=Y(k)-Y.
Using (4) and (5) gives
(9)X~(k)=X~(k-1)-μ(ATA+DTD)-1[AD]T×[Y~(k-1)B+AX~(k-1)Y~(k-1)E+DX~(k-1)],Y~(k)=Y~(k-1)-μ[Y~(k-1)B+AX~(k-1),mmmmmmmmY~(k-1)E+DX~(k-1)]×[B,E]T(BBT+EET)-1.
Expanding (9) gives
(10)X~(k)=X~(k-1)-μ(ATA+DTD)-1×[ATY~(k-1)B+ATAX~(k-1)mm+DTY~(k-1)E+DTDX~(k-1)],Y~(k)=Y~(k-1)-μ[Y~(k-1)BBT+AX~(k-1)BTmm+Y~(k-1)EET+DX~(k-1)ET]×(BBT+EET)-1.
Simplifying (10) gives
(11)X~(k)=(1-μ)X~(k-1)-μ(ATA+DTD)-1×[ATY~(k-1)B+DTY~(k-1)E],Y~(k)=(1-μ)Y~(k-1)-μ[AX~(k-1)BT+DX~(k-1)ET]×(BBT+EET)-1.
Using the formula col[AXB]=(BT⊗A)col[X] gives
(12)col[X~(k)]=(1-μ)col[X~(k-1)]-μ{[In⊗(ATA+DTD)-1]mm×[BT⊗AT+ET⊗DT][In⊗(ATA+DTD)-1]}×col[Y~(k-1)],col[Y~(k)]=(1-μ)col[Y~(k-1)]-μ{[(BBT+EET)-1⊗Im]mm×[B⊗A+E⊗D][(BBT+EET)-1⊗Im]}col[X~(k-1)].
Set
(13)J∶=[B⊗Im,E⊗Im]∈R(mn)×(2mn),K∶=[In⊗AIn⊗D]∈R(2mn)×(mn),L∶=[Imn(KTK)-1(JK)T(JJT)-1(JK)Imn].
Using these symbols, a compact form of (12) is
(14)col[X~(k),Y~(k)]={[Imn00Imn]-μL}×col[X~(k-1),Y~(k-1)].
Set
(15)M∶=[Imn00Imn]-μL.
The eigenvalues of M inside the unit circle will complete the proof. Let α be an eigenvalue of M; we will show that |α|<1, or, if β∈λ[L], then 0<β<2.
Consider the following characteristic polynomial ofL:
(16)p(z)=|zI2mn-L|=|[Imn(KTK)-1(JK)T(JJT)-1(JK)-Imn][zImn00zImn]m-[Imn(KTK)-1(JK)T(JJT)-1(JK)Imn]|=|[(z-1)Imn-(KTK)-1(JK)T-(JJT)-1(JK)(z-1)Imn]|.
It is not hard to show that p(2)≠0 by calculation. Since
(17)p(2)=|Imn-(KTK)-1(JK)T-(JJT)-1(JK)Imn|=|[Imn-(KTK)-1(JK)T-(JJT)-1(JK)Imn]|=|[(KTK)-100(JJT)-1][KTK-(JK)T-(JK)JJT]|=|[(KTK)-100(JJT)-1][KT-J][K,-JT]|,
it follows from the formula
(18)|ABCD|=|A||D-CA-1B|
that
(19)|[K,-JT]|=|In⊗A-BT⊗ImIn⊗D-ET⊗Im|=|In⊗A|×|-ET⊗Im+(In⊗D)(In⊗A)-1(BT⊗Im)|=|In⊗A||-ET⊗Im+BT⊗(DA-1)|=|-ET⊗Im+BT⊗(DA-1)||In⊗A|=|[-ET⊗Im+BT⊗(DA-1)](In⊗A)|=|-ET⊗A+BT⊗D|.
On the other hand, since (4) has a unique solution, it follows that the matrix
(20)[In⊗ABT⊗ImIn⊗DET⊗Im]
is invertible. A determinant expansion shows that
(21)|In⊗ABT⊗ImIn⊗DET⊗Im|=|In⊗A|×|ET⊗Im-(In⊗D)(In⊗A)-1(BT⊗Im)|=|In⊗A||ET⊗Im-BT⊗(DA-1)|=|ET⊗Im-BT⊗(DA-1)||In⊗A|=|[ET⊗Im-BT⊗(DA-1)](In⊗A)|=|ET⊗A-BT⊗D|=(-1)mn|-ET⊗A+BT⊗D|=(-1)mn|[K,-JT]|≠0.
So it gives p(2)≠0. Since
(22)p(z)=|(z-1)Imn-(KTK)-1(JK)T-(JJT)-1(JK)(z-1)Imn|,
expanding it gives
(23)p(z)=|(z-1)Imn||(z-1)Imn-(JJT)-1(JK)mmmmmmm×(z-1)-1Imn(KTK)-1(JK)T|=|(z-1)2Imn-(JJT)-1(JK)(KTK)-1(JK)T|.
Setting P∶=(JJT)-1(JK)(KTK)-1(JK)T∈R(mn)×(mn), it can be rewritten as
(24)P=(JJT)-1J[K(KTK)-1KT]JT.
According to Lemma 1, there exists an orthogonal matrix Q∶=[q1,q2,…,q2mn] such that
(25)QT[K(KTK)-1KT]Q=diag⁡[1,1,…,1,0,0,…,0],rank⁡{diag⁡[1,1,…,1,0,0,…,0]}=rank⁡{QT[K(KTK)-1KT]Q}=rank⁡[K(KTK)-1KT]=rank⁡[K]=mn.
Using (25) can be manipulated to get
(26)[K(KTK)-1KT][q1,q2,…,q2mn]=[q1,q2,…,q2mn]diag⁡[1,1,…,1,0,0,…,0]=[q1,q2,…,qmn,0,0,…,0].
If ρ∈λ[P], then there exist the nonzero vectors y∈Rmn that satisfy Py=ρy, which can be written as
(27)(JJT)-1JK(KTK)-1KTJTy=ρy,JK(KTK)-1KTJTy=ρ(JJT)y.
Since JTy∈R2mn and Q∈R(2mn)×(2mn) are orthogonal matrices, it follows that
(28)JTy=k1q1+k2q2+⋯+k2mnq2mn,
where ki∈R,i=1,2,…,2mn and k12+k22+⋯+k2mn2≠0. Then from (28), it gives
(29)(JTy)T(JTy)=(k1q1+k2q2+⋯+k2mnq2mn)T×(k1q1+k2q2+⋯+k2mnq2mn)=k12+k22+⋯+k2mn2.
Using (26) gives
(30)K(KTK)-1KTJTy=K(KTK)-1KT(k1q1+k2q2+⋯+k2mnq2mn)=k1q1+k2q2+⋯+kmnqmn.
According to (29) and (30), (27) can be manipulated to get
(31)yTJK(KTK)-1KTJTy=yTρ(JJT)y,yTJ(k1q1+k2q2+⋯+kmnqmn)=ρ(k12+k22+⋯+k2mn2),(k12+k22+⋯+kmn2)=ρ(k12+k22+⋯+k2mn2).
So it gives
(32)0⩽ρ=k12+k22+⋯+kmn2k12+k22+⋯+k2mn2⩽1.
Suppose that λ[P]={ρ12,ρ22,…,ρmn2}, where 0⩽ρi⩽1, i=1,2,…,mn. From Schur decomposition theorem, there exists a decomposition UTPU=D+F, where U is an orthogonal matrix, D=diag⁡(ρ12,ρ22,…,ρmn2), and F is a strictly upper triangular matrix. Then from (23), it gives
(33)p(z)=|(z-1)2Imn-(JJT)-1(JK)(KTK)-1(JK)T|=|(z-1)2Imn-P|=|UT||(z-1)2Imn-P||U|=|UT(z-1)2ImnU-UTPU|=|(z-1)2Imn-D-F|=[(z-1)2-ρ12][(z-1)2-ρ22]⋯[(z-1)2-ρmn2]=(z-1-ρ1)(z-1+ρ1)(z-1-ρ2)(z-1+ρ2)⋯(z-1-ρmn)(z-1+ρmn).
It follows that λ[L]={1+ρ1,1-ρ1,1+ρ2,1-ρ2,…,1+ρmn,1-ρmn}. Since p(2)≠0, we obtain ρi<1, i=1,2,…,mn. Thus, 0<β<2.
Next, we determine the optimal convergence factor. Fromλ[L]={1+ρ1,1-ρ1,1+ρ2,1-ρ2,…,1+ρmn,1-ρmn} and
(34)M=[Imn00Imn]-μL,
one gets λ[M]={1-μ(1+ρ1),1-μ(1-ρ1),…,1-μ(1+ρmn),1-μ(1-ρmn)}. Set ρ1∶=max⁡{ρ1,ρ2,…,ρmn}. Taking absolute values of these eigenvalues, the optimal convergence factor satisfies
(35)min⁡⁡max⁡{|1-μ(1+ρ1)|,|1-μ(1-ρ1)|,…,mmmmm|1-μ(1+ρmn)|,|1-μ(1-ρmn)|}=min⁡⁡max⁡{|1-μ(1+ρ1)|,|1-μ(1-ρ1)|}.
Equation (35) is equivalent to |1-μ(1+ρ1)|=|1-μ(1-ρ1)|. Solving it gives μ=1. The proof is completed.


## 4. The Generalized Sylvester Matrix Equation AXB + CXD = F

In this section, we use iteration in (5) to solve the generalized Sylvester matrix equation. Consider the following equation:
(36)AXB+CXD=F,
where A,C∈Rm×m, B,D∈Rn×n, and F∈Rm×n are given constant matrices and X∈Rm×n is the unknown matrix to be solved. The following conclusion is obvious.Equation (36) has a unique solution if and only if
(37)rank⁡{[BT⊗A+DT⊗C,col[F]]}=rank⁡[BT⊗A+DT⊗C]=mn.
In this case, the unique solution is given by col[X]=(BT⊗A+DT⊗C)-1col[F].SettingV∶=XB∈Rm×n and W∶=CX∈Rm×n, (36) can be equivalently expressed as
(38)AV+WD=F,CV-WB=0.
If
(39)rank⁡{[In⊗ADT⊗ImIn⊗C-BT⊗Im]}=2mn,
then (38) has a unique solution. It is easy to show that if |C|≠0 or |B|≠0, then (38) is equivalent to (36). Next, we show that if |C|≠0 or |B|≠0, then (39) is equivalent to (37). That is, we have the following determinant result:
(40)|In⊗C-BT⊗ImIn⊗ADT⊗Im|=|In⊗C||DT⊗Im+(In⊗A)(In⊗C)-1(BT⊗Im)|=|In⊗C||DT⊗Im+BT⊗(AC-1)|=|DT⊗Im+BT⊗(AC-1)||In⊗C|=|DT⊗C+BT⊗A|.
According to Theorem 2, (38) can be solved by iteration in (5), and from XB=V or CX=W, (36) can be solved.


## 5. Example

In this section, an example is offered to illustrate the convergence of the proposed iterative algorithm.Example 1.
Consider the coupled Sylvester matrix equations in the form of (4) with
(41)A=[7-5-19-8],B=[-112-8-14],C=[-1213-259-215],D=[-102116],E=[-10-3-1-12],F=[-104-9041-20].
The unique solution is found to be
(42)X=[x11x12x21x22]=[4594],Y=[y11y12y21y22]=[8257].
TakingX(0)=Y(0)=10-612×2 as the initial iterative values and using iteration (5) to compute X(k) and Y(k), the iterative values of X(k) and Y(k) are shown in Table 1 with the relative error
(43)δ∶=∥X(k)-X∥2+∥Y(k)-Y∥2∥X∥2+∥Y∥2.
The effect of changing the convergence factor μ is illustrated in Figure 1.Table 1
The least squares iterative solutions (μ = 1.0).
k
x
11
x
12
x
21
x
22
y
11
y
12
y
21
y
22
δ (%)
1
2.41386
5.12921
15.19476
−3.29213
8.34835
1.59243
4.18906
7.37550
58.29997
2
3.78243
4.89986
8.50024
3.36107
7.11113
1.69190
4.40964
7.01546
8.34339
3
3.76982
4.86436
9.81323
3.24974
8.02454
1.93498
4.94575
6.99503
6.82277
4
3.98646
4.99976
8.94082
3.91276
7.87581
1.98403
4.92193
7.02174
1.09451
5
3.96846
4.97726
9.13074
3.92885
8.00448
1.99063
4.99536
6.99758
0.92202
6
3.99906
5.00066
8.99011
3.98870
7.98094
2.00140
4.99013
7.00441
0.15912
7
3.99573
4.99642
9.02215
3.99419
8.00093
1.99865
4.99957
6.99955
0.14124
8
3.99996
5.00016
8.99824
3.99851
7.99694
2.00071
4.99874
7.00081
0.02497
9
3.99940
4.99943
9.00381
3.99973
8.00018
1.99980
4.99996
6.99992
0.02344
10
4.00000
5.00003
8.99969
3.99980
7.99949
2.00018
4.99983
7.00015
0.00413
11
3.99991
4.99991
9.00066
4.00004
8.00003
1.99997
5.00000
6.99999
0.00405
12
4.00000
5.00001
8.99994
3.99997
7.99991
2.00004
4.99998
7.00003
0.00070
Solution
4.00000
5.00000
9.00000
4.00000
8.00000
2.00000
5.00000
7.00000Figure 1
The relative errorδ versus iteration k.From Table1 and Figure 1, we find that the relative error goes to zero with the increasing of the iterative times. This shows that the proposed iterative algorithm is effective. In addition, Figure 1 shows that the optimal convergence factor μ=1.0. This indicates that the result of the optimal convergence suggested in this paper is correct.


## 6. Conclusions

This paper proved the convergence of the least squares based iterative algorithm of the coupled Sylvester matrix equationsAX+YB=C and DX+YE=F, and the proof determined the range of the convergence factor and the optimal convergence factor. The suggested algorithm can also be used to solve the generalized Sylvester equation AXB+CXD=F. An example indicated that the iterative solution given by the least squares based iterative algorithm converges fast to its exact solution under proper conditions.



---

*Source: 831321-2014-06-18.xml*