# H 2 / H ∞ Control for MJLS with Infinite Markov Chain

Authors: Ting Hou; Jie Wang; Yueying Liu; Xiao Shen
Journal: Mathematical Problems in Engineering
 (2017)

---

## Abstract

With the help of a stochastic bounded real lemma, we deal with finite horizonH2/H∞ control problem for discrete-time MJLS, whose Markov chain takes values in an infinite set. Besides, a unified control design for H2, H∞, and H2/H∞ is given.


---

## Body

## 1. Introduction

As we know,H∞ control is one of the most important robust control designs, usually used to eliminate the effect of disturbance vt. In particular, lots of results have been contributed to the stochastic H∞ theory; see [1–7], among many others. ForItô systems and discrete-time systems with multiplicative noise, stochastic H∞-type control problems have been considered in [2, 5], respectively. Reference [3] has designed a state feedback H∞ controller for nonlinear stochastic systems. References [4, 7] have dealt with H∞ control in the presence of stochastic uncertainty.On the other hand, after Kalman presented the question “When is a linear control system optimal?” which preserves the quadratic form in the performance index, [8] has discussed H2 or Linear Quadratic Gaussian (LQG) control, while [9] has retained the original weights and sought out a producer which can also achieve the desired degree of stability, and others have contributed to the stochastic LQ controller design [10–12]. Because of the popularity of the H2 performance for engineering, more and more researchers have been attracted by the mixed H2/H∞ control topic; see [13–16]. For example, [14, 15] have investigated the finite horizon and infinite horizon H2/H∞ control problem for discrete-time Markov jump systems.It should be pointed out that mostly researches on Markov jump systems assume that Markov chain takes values in a finite set, such as [6, 13–15, 17–20]. However, infinite Markov jump systems, where Markov process has an infinite state space, can be used to describe more plants in real world. Recently, infinite Markov jump systems have aroused more and more concern [1, 21, 22]. Specially, for discrete-time case, [21] has explored exponential stability and l2 input-state stability which is strongly detectable; [1] has established the finite horizon stochastic bounded real lemma to attain a prescribed disturbance attenuation level. Based on [1], this note seeks to set up H2/H∞ result for infinite Markov jump systems, unlike [15] which considered the finite jump case.This paper aims to handle the mixedH2/H∞ control problem through the solvability of four coupled difference matrix-valued recursions (CDMRs) for discrete-time infinite Markov jump systems. The rest of the paper is organized as follows: Section 2 provides some useful definitions and lemmas. In Section 3, we get the necessary and sufficient condition for the finite horizon H2/H∞ control problem based on the stochastic bounded real lemma. A unified control design for H2, H∞, and H2/H∞ control is given in Section 4. And Section 5 concludes the paper.For convenience, we adopt the following notations.R is the set of all real numbers; Rn is n-dimensional real vector space; Rm×n is the vector space of all m×n matrices with entries in R; A′ is the transpose of a matrix A; A≥0 (A>0): A is a positive semidefinite (positive definite) symmetric matrix; I is the identity matrix; · is the operator norm of Rm×n or the Euclidean norm of Rn; Z+ is the set of all nonnegative integers; Z1+ is the set of all positive integers; Sn is the set of all n×n symmetric matrices.


## 2. Preliminaries

Consider the following discrete-time infinite Markov jump systems defined on a complete probability space(Ω,F,P):(1)xt+1=A0ηtxt+B0ηtvt+∑k=1dAkηtxt+Bkηtvtwtk,zt=Cηtxt,ηt∈Z1+,where xt∈Rn,vt∈Rnv, and zt∈Rnz represent the system state, disturbance signal, and measurement output, respectively. Markov chain {ηt}t∈Z+ takes values in Z1+ with switching governed by a stationary transposition probability matrix P=[pij], where pij=P(ηt+1=j∣ηt=i). Let wt={wt=(wt1,…,wtd)} be a sequence of real random variables which satisfies E(wt)=0 and E(wtws′)=δt,s (Kronecker function). Make Ft be the σ-algebra generated by {ηk,ws∣0≤k≤t,0≤s≤t-1}. With the case t=0, set F0=σ{η0}. For any given t∈Z+, the σ-algebras σ{w0,…,wt} and σ{η0,…,ηt} are independent of each other. l2(0,T;Rm) denotes the set of Rm–valued processes {y(t,w):Z+×Ω→Rm}, which is Ft-measurable and ∑t=0TEyt2<∞. Apparently, l2(0,T;Rm) is a real Hilbert space with the norm being yl2(0,T;Rm)=(∑t=0TEyt2)1/2<∞.MakeH1m×n be the set {M∣M=(M(1),M(2),…),M(i)∈Rm×n} which satisfies ∑i=1∞M(i)<∞. We can easily verify that H1m×n is a Banach space with Mi1=∑i=1∞M(i). Define another Banach space H∞m×n with Mi∞=supi∈Z1+M(i). H1m×n will be written as H1n with the case m=n, so does H∞m×n. For i∈Z1+, when M(i)∈Sn and M(i)≥0, H1n(H∞n) will be written as H1n+(H∞n+).ForM,N∈H1n, if M(i)≤N(i) for all i∈Z1+, we say M≤N. And it is easy to know that M1≤N1. What is more, for a given real Banach space X, let B(X) represent a Banach space including all bounded linear operators mapping X to X. For Γ∈B(X), the induced norm is denoted by ΓX.Next, we introduce the linear perturbation operator as follows.Definition 1.
Define a linear perturbation operatorLT:l2(0,T;Rnv)→l2(0,T;Rnz) of system (1) as follows: (2)LTv=Cηtxt;0,v,where x(t;0,v) satisfies system (1) corresponding to x0=0 and v(·). When v≠0, the H∞ norm of LT is determined by (3)LT∞≔supv∈l20,T;Rnv,v≠0,η0∈Z1+,x0=0LTvl20,T;Rnzvl20,T;Rnv.When v=0, which means the system is unperturbed, the problem is trivial (in this case, LT∞=0).
Define the functional:(4)J1Tx0,η0,ut,vt=∑t=0TEzt2-γ2vt2,which is connected with the H∞ performance.To make the formulae more brief, we adopt the following notations. DenoteP as a series of symmetric matrices derived by the time t and the mode i; that is P={P(ηt)∈Sn:ηt∈Z1+,t∈Z+},P(t)=(P(η1),P(η2),P(η3),…). In subsequent analysis, we will use the notations as follows with U∈H∞n: (5)ϵiU=∑j=1∞pijtUj,Φi1U=∑k=0dAki′ϵiUAki+Ci′Ci,Φi2U=∑k=0dAki′ϵiUBki,Φi3U=∑k=0dBki′ϵiUBki-γ2I.Lemma 2.
Givenx0∈Rn,v∈l2(0,T;Rnν), η0∈Z1+, and xt≔x(t;x0,η0,v) being the solution of system (1), for any given T∈Z+, we have (6)J1Tx0,η0,ut,vt=∑t=0TExtvt′∏ηtt,Pxtvt+Ex0′P0,η0x0-ExT+1′PT+1,ηT+1xT+1,where (7)∏ηtt,P=∏it,P=Φi1Pt+1-Pt,iΦi2Pt+1Φi2Pt+1′Φi3Pt+1for ηt=i.Proof.
Due to the assumption, we know thatwtk is independent of the Markov chain {ηt} and is uncorrelated with vt, so A0(ηt)xt+B0(ηt)vt and Ak(ηt)xt+Bk(ηt)vt are uncorrelated with wtk,k=1,…,d. Besides, A0(ηt)xt+B0(ηt)vt and Ak(ηt)xt+Bk(ηt)vt are Ft-measurable, k=1,…,d; thus (8)EA0ηtxt+B0ηtvt′Pt+1,ηt+1Akηtxt+Bkηtvt∣Ft=0.When ηt=i, we have (9)Ext+1′Pt+1,ηt+1xt+1-xt′Pt,ηtxt∣Ft,ηt=i=Extvt′∏ηt∗t,Pxtvt,where (10)∏ηt∗t,P=∏i∗t,P=Φi1Pt+1-Pt,i-Ci′CiΦi2Pt+1Φi2Pt+1′Φi3Pt+1+γ2I.Taking t=0,…,T above and summarizing together, we get that (11)ExT+1′PT+1,ηT+1xT+1-x0′P0,η0x0∣Ft,ηt=i=∑t=0TExtvt′∏ηt∗t,Pxtvt.According to the definition of J1T(x0,η0,ut,vt), we obtain (12)J1Tx0,η0,ut,vt=∑t=0TEzt2-γ2vt2+∑t=0TExtvt′∏ηt∗t,Pxtvt+Ex0′P0,η0x0-ExT+1′PT+1,ηT+1xT+1. By simple computation, we can get the desired result.Lemma 3 (see [1] (stochastic bounded real lemma)).
For system (1), given γ>0 and T∈Z1+,LT∞<γ if and only if the following difference Riccati recursions are solvable with P(t)∈H∞n+: (13)Pt,i=Φ1iPt+1-Φi2Pt+1Φi3Pt+1-1Φi2Pt+1′,PT+1,i=0,i∈Z1+,t∈0,T.And P(t) satisfies (14)Φi3Pt+1<-ϵ0Inv,∀ε0∈0,γ2-LT∞2.


## 3.H2/H∞ Control

In this subsection, we attempt to discuss the finite horizonH2/H∞ control problem. Consider the following linear systems with infinite Markov jump:(15)xt+1=A0ηtxt+B¯0ηtut+B0ηtvt+∑k=1dAkηtxt+B¯kηtut+Bkηtvtwtk,zt=CηtxtDηtut,Dηt′Dηt=I,ηt∈Z1+.GivenT∈Z1+ and γ>0, our objective is to find u∗(·)∈l2(0,T;Rnu) such that(i)
LT∞<γ, for all v(·)∈l2(0,T;Rnv) and η0∈Z1+;(ii)
when the worst case disturbancevt∗ is enforced on system (15), u∗(·) will minimize the energy functional J2T(x0,η0,ut,vt∗)≔∑t=0TEzt2,∀x0∈Rn, η0∈Z1+.If there exists(ut∗,vt∗) such that (i) and (ii) hold simultaneously, we say that the mixed H2/H∞ control problem is solvable. Before discussing, we provide the following coupled matrix recursions which is defined on i∈Z1+:(16)X1t,i=∑k=0dAki+B¯kiKi2X2′εiX1t+1Aki+B¯kKi2X2+Ci′Ci-Ki3X1Hi1X1-1Ki3X1′+Ki2X2′Ki2X2,X1t≥0,X1T+1=0,Hi1X1<0,(17)Ki1X1=-Hi1X1-1Ki3X1′,(18)X2t,i=∑k=0dAki+BkiKi1X1′εiX2t+1Aki+BkiKi1X1+Ci′Ci-Ki4X2Hi2X2-1Ki4X2′,X2t,i≥0,X2T+1=0,Hi2X2>0,(19)Ki2X2=-Hi2X2-1Ki4X2′,where (20)Hi1X1=∑k=0dBki′εiX1t+1Bki-γ2I,Hi2X2=I+∑k=0dB¯ki′εiX2t+1Gki,Ki3X1=∑k=0dAki+B¯kiKi2X2′εiX1t+1Bki,Ki4X2=∑k=0dAki+BkiKi1X1′εiX2t+1B¯ki.Theorem 4.
The finite horizonH2/H∞ control problem is solvable with the solution ut∗=Kηt2(X2)xt,vt∗=Kηt1(X1)xt if and only if CDMRs (16)–(19) have a pair of solutions (X1(t,i),Ki1(X1));(X2(t,i),Ki2(X2)) for ∀(t,i)∈[0,T]×Z1+.Proof.
⇐ : Assume CDMRs (16)–(19) have a pair of solutions (X1(t,i),Ki1(X1));(X2(t,i),Ki2(X2)), ∀(t,i)∈[0,T]×Z1+. Constructing ut∗=Kηt2(X2)xt and putting u into system (15), we get LT∞<γ for all v∈l2(0,T;Rnv) and η0∈Z1+. By Lemma 3, together with the completing squares technique, it yields from (16) that (21)J1Tx0,η0,ut∗,vt=∑t=0TEEvt-vt∗′Hi1X1vt-vt∗∣ηt=i+x0′X10,η0x0≥J1Tx0,η0,ut∗,vt=x0′X10,η0x0,where vt∗=Kηt1(X1)xt and Kηt1(X1) is defined by (17). As shown above, J1T(x0,η0,ut∗,v) is minimized by vt∗=Kηt1(X1)xt, ∀x0∈Rn, and vt∗ is the worst case disturbance. By Lemma 2 and the technique of completing squares, we obtain (22)J2Tx0,η0,ut,vt∗=∑t=0TEEut-ut∗′Hi2X2ut-ut∗∣ηt=i+x0′X20,η0x0≥J2Tx0,η0,ut∗,vt∗=x0′X20,η0x0,where ut∗=Kηt2(X2)xt, and Kηt2(X2) is defined by (19). By reasoning as above, it is revealed that ut∗ is the controller to minimize the J2T(x0,η0,ut,vt∗). Thus, we conclude that (ut∗,vt∗) is a pair of solutions to the finite horizon H2/H∞ control problem.
⇒ : Suppose that system (15) has a pair of solutions to the finite horizon H2/H∞ control problem with ut∗=Kηt2(X2)xt,vt∗=Kηt1(X1)xt. Putting ut∗ into (15), we come to the following equations:(23)xt+1=A0ηt+B¯0ηtKηt2X2xt+B0ηtvt+∑k=1dAkηt+B¯kηtKηt2X2xt+Bkηtvtwtk,zt=CηtDηtKηt2X2xt,Dηt′Dηt=I,ηt∈Z1+,t∈Z+.Applying Lemma 3 to (23), we justify that X1(t,i) satisfies (16) on [0, T], also X1(t,i)≥0. From the proof of the sufficiency, we confirm that the worst case disturbance vt∗=Kηt1(X1)xt with Kηt1(X1) given by (17). Enforcing vt∗ on system (15), we obtain(24)xt+1=A0ηt+B0ηtKηt1X1xt+B¯0ηtut+∑k=1dAkηt+BkηtKηt1X1xt+B¯kηtutwtk,zt=CηtxtDηtut,Dηt′Dηt=I,ηt∈Z1+,t∈Z+.According to the assumption, we deduce that u∗ is the optimal solution of the following problem: (25)minut∈l20,T;RnuJ2Tx0,η0,ut,vt∗=∑t=0TEut′ut+xt′Cηt′Cηtxt,subject  to24.This is a standard LQ control for Markov jump systems defined on a finite horizon. Similar to the proof of Theorem 1 in [7], it is not difficult to prove that (18) is solvable with X2(t,i)≥0. The proof is ended.Remark 5.
Compared to finite horizonH2/H∞ control problem considered in [15], whose Markov chain takes values in a finite set, the dynamical model taken into account in this note is more general.Example 6.
Consider the following one-dimensional discrete-time infinite Markov jump system:(26)xt+1=12ηt+1xt+ut+vt+1ηt+1xt+ut+vtwt,ηt∈Z1+,zt=310xtut.In (26), the transition probability is defined by p(i,i)=3/4,p(i,i+1)=1/4,p(i,j)=0,j≠i,i+1,i,j∈Z1+. Set T=2,γ=0.5, the solutions of the four coupled matrix recursions (16)–(19) are given by X1(0,i)=0.14/(i+1)2+0.09,X1(0,i)=6.66/(i+1)2+0.09,Ki1(X1)=5.36/(i+1),Ki2(X2)=-0.17/(i+1).


## 4.H2, H∞, and H2/H∞ Control

In this section, we will develop a unified control design ofH2, H∞, and H2/H∞ control for system (15). Give the following indices:(27)J1Tx0,η0,ut,vt=∑t=0TEγ2vt2-zt2,J2Tx0,η0,ut,vt=∑t=0TEzt2-ρ2vt2,where γ≥0 and ρ≥0 are defined on R.Definition 7.
We call an admissible set(u∗(·),v∗(·))∈l2(0,T;Rnu)×l2(0,T;Rnv) a Nash equilibrium, if the following inequalities hold simultaneously for all admissible (u(·),v(·))∈l2(0,T;Rnu)×l2(0,T;Rnv),(28)J1Tx0,η0,ut∗,vt∗≤J1Tx0,η0,ut∗,vt,J2Tx0,η0,ut∗,vt∗≤J1Tx0,η0,ut,vt∗.For convenient, we give the following coupled matrix recursions on (t,i)∈Z+×Z1+:(29)X1t,i=∑k=0dAki+B¯kiKi2X2′εiX1t+1Aki+B¯kiKi2X2-Ci′Ci-Ki3X1Hi1X1+Ki3X1′-Ki2X2′Ki2X2,X1T+1=0,Hi1X1≥0,(30)Ki3X1=Ki3X1Hi1X1+Hi1X1,(31)X2t,i=∑k=0dAki+BkiKi1X1′εiX2t+1Aki+BkiKi1X1+Ci′Ci-Ki4X2Hi2X2+Ki4X2′-ρ2Ki1X1′Ki1X1,Hi2X2≥0,X2T+1=0,(32)Ki4X1=Ki4X2Hi2X2+Hi2X2,where (33)Hi1X1=∑k=0dBki′εiX1t+1Bki+γ2I,Hi2X2=I+∑k=0dB¯ki′εiX2t+1B¯ki,Ki3X1=∑k=0dAki+B¯kiKi2X2′εiX1t+1Bki,Ki4X2=∑k=0dAki+BkiKi1X1′εiX2t+1B¯ki,Ki1X1=-Hi1X1+Ki3X1′,Ki2X2=-Hi2X2+Ki4X2′.
(I) H 2 Control. Letting ρ=0 and γ→+∞ in (27), it is easy to get that the performance index J1T(x0,η0,ut,vt) holds naturally, and (34)J2Tx0,η0,ut,vt≔∑t=0TEzt2=∑t=0TExt′Ci′Cixt+ut′utwhich turns into a stochastic LQ optimal control problem. Because of ρ=0 in J2T(x0,η0,ut,vt), taking Bk(i)=0 in (15), we get (35)xt+1=A0ηtxt+B¯0ηtut+∑k=1dAkηtxt+B¯kηtutwtk,zt=CηtxtDηtut,Dηt′Dηt=I,ηt∈Z1+,t∈Z+.By Theorem 4, we know minut∈l2(0,T;Rnu)J2T(x0,η0,ut,vt)=J2T(x0,η0,ut∗,vt), where ut∗=Kηt2(X2)xt with Ki2(X2)=-Hi2(X2)-1∑k=1dAki′εiX2t+1B¯ki′ and X2 is the solution of the following equations:(36)X2t,i=∑k=0dAki′εiX2t+1Aki+Ci′Ci-∑k=0dAki′εiX1t+1·B¯kiHi2X2-1∑k=0dAki′εiX1t+1B¯ki′,X2T+1=0,Hi2X2=I+∑k=0dB¯ki′εiX1t+1B¯ki>0,i∈Z1+.Also, the optimal value is obtained by (37)minut∈l20,T;RnuJ2Tx0,η0,ut,0=x0′X20,η0x0.Remark 8.
Via a series of calculation, we can rewrite the first equality of (36) under the case of positiveness of Hi2(X2) as (38)X2t,i=∑k=0dAki+B¯kiKi2X2′εiX2t+1·Aki+B¯kiKi2X2+Ci′Ci+Ki2X2′Ki2X2.Obviously, we have X2(t,i)≥0; thus Hi2(X2)>0.
(II) H ∞ Control. Letting ρ=γ in (27), it follows that (39)J1Tx0,η0,ut,vt+J2Tx0,η0,ut,vt=0.In this case, the Nash game (28) turns into a zero-sum game. We know from Theorem 4 that there exists a Nash equilibrium if and only if (29)–(32) is solvable. By computing, we obtain (40)X1t,i+X2t,i=∑k=0dAki+BkiKi1X1+B¯kiKi2X2′εiX1t+1+X2t+1Aki+BkiKi1X1+B¯kiKi2X2.Noticing that X1(T+1,i)+X2(T+1,i)=0, we conclude the above equation has a unique solution such that X1(t,i)+X2(t,i)=0. Showing (31) in another way,(41)X2t,i=∑k=0dAki+BkiKi1X1′εiX2t+1Aki+BkiKi1X1+Ci′Ci-Ki4X2Hi2X2+Ki4X2′-γ2Ki1X1′Ki1X1=∑k=0dAki+B¯kiKi2X2′εiX2t+1Aki+B¯kiKi2X2+Ki1X1′-γ2I+∑k=0dBki′εiX2t+1BkiKi1X1+Ki1X1′∑k=0dBki′εiX2t+1Aki+B¯kiKi2X2+∑k=0dAki+B¯kiKi2X2′εiX2t+1BkiKi1X1+Ki2X2′Ki2X2+Ci′Ci.Substituting X2(t,i)=-X1(t,i) into (41), it is easy to recognize that(42)-X1t,i=-∑k=0dAki+B¯kiKi2X2′εiX1t+1Aki+B¯kiKi2X2-Ki1X1′Hi1X1Ki1X1-Ki1X1′Ki3X1′-Ki3X1Ki1X1+Ki2X2′Ki2X2+Ci′Ci.Similar as Theorem 1 in [23], we can prove that LT<γ, which concludes that Hi1(X)>0. Since Hi1(X1)+=Hi1(X1)-1,Hi1(X1)>0 indicates that (42) is consistent with (29). Thus, when ρ=γ, the solution X of (42) satisfies (43)Xi=∑k=0dAki+B¯kiKi2X′εiXt+1Aki+B¯kiKi2X-Ci′Ci-Ki3XHi1X-1Ki3X′-Ki2X′Ki2X,Xi≤0,XT+1=0,Hi1X≥0.Besides, according to Theorem 4, (ut∗=Kηt2(X)xt,vt∗=Kηt1(X)xt) follows that (44)J1Tx0,η0,ut,vt∗≤J1Tx0,η0,ut∗,vt∗≤J1Tx0,η0,ut∗,vt,which shows that ut∗ is the optimal control design and vt∗ is the related worst case disturbance for γ>0.
(III) Mixed H 2 / H ∞ Control. Letting ρ=0 in J2T(x0,η0,ut,vt). For system (15), the indices (27) turn into (45)J1Tx0,η0,ut,vt≔∑t=0TEγ2vt2-zt2,J2Tx0,η0,ut,vt≔∑t=0TEzt2.When x0=0, because of the linearity of system (15), we have ut∗=0, vt∗=0, and (46)J1T0,η0,ut∗,vt≥J1T0,η0,ut∗,vt∗=0,J2T0,η0,ut∗,vt∗≤J2Tx0,η0,ut,vt∗;that is, (ut∗,vt∗) is the optimal solution to the mixed H2/H∞ control problem.Remark 9.
The discussion here expands the version to the infinite Markov jump case compared to [23].


## 5. Conclusion

This note has supplied a finite horizon state feedbackH2/H∞ controller based on the solution of four CDMRs for discrete-time infinite Markov jump systems with multiplicative noise. And, a numerical example has been given to illustrate its efficiency. How to handle the infinite horizon case for the concerned systems is our next work in the future.



---

*Source: 9038469-2017-09-11.xml*