# Exponential Stability and RobustH∞ Control for Discrete-Time Time-Delay Infinite Markov Jump Systems

Authors: Yueying Liu; Ting Hou
Journal: Discrete Dynamics in Nature and Society
 (2018)

---

## Abstract

In this paper, exponential stability and robustH∞ control problem are investigated for a class of discrete-time time-delay stochastic systems with infinite Markov jump and multiplicative noises. The jumping parameters are modeled as an infinite-state Markov chain. By using a novel Lyapunov-Krasovskii functional, a new sufficient condition in terms of matrix inequalities is derived to guarantee the mean square exponential stability of the equilibrium point. Then some sufficient conditions for the existence of feedback controller are presented to guarantee that the resulting closed-loop system has mean square exponential stability for the zero exogenous disturbance and satisfies a prescribed H∞ performance level. Numerical simulations are exploited to validate the applicability of developed theoretical results.


---

## Body

## 1. Introduction

During the past decades, Markov jump systems have been the subject of a great deal of research since they have been used extensively both in theory and in applications. Markov jump systems are hybrid dynamical systems composed of subsystems with the transitions determined by a Markov chain. A number of results that focused on Markov jump systems have been published ranging from filtering, stability, observability, and control to engineering application; see, for example, [1–15] and the references therein.Note that most of the theoretical works related to Markov jump systems in the literatures concentrated on the case where the state space of the Markov chain is finite. However, it may be more appropriate to characterize abrupt changes in many real plants via an infinite-state Markov chain. As far as applications are concerned, infinite Markov jump systems are critical in some physics plants, such as solar thermal receiver, aircraft, and robotic manipulator systems. Theoretically, finite Markov jump systems are fundamentally different from those governed by infinite-state space. The work in [14] studied exponential almost sure stability of random jump systems. The work in [16] considered the definition and computation of an H2-type norm for stochastic systems with infinite Markov jump and periodic coefficients. LQ-optimal control problem has been dealt with for discrete-time infinite Markov jump systems in [17]. The work in [18] demonstrated the inequivalence between stochastic stability and mean square exponential stability in discrete-time case. With this motivation, infinite Markov jump systems have stirred widespread research interests.Time-delay is one of the inherent features of many practical systems and also is the big source of instability and poor performances in systems [19]. Moreover, stochastic modeling has had extensive applications. Hence, dynamical time-delay stochastic systems deserve our consideration. Stability analysis and controller design of time-delay Markov jump systems have been investigated by many authors [15, 20, 21]. Unfortunately, the literature about these issues for infinite Markov jump case is less developed. And, to the best of our knowledge, only a few results have been presented so far [18, 22, 23], let alone the problem involving time-delay. Actually, [18, 23] investigated the exponential stability and infinite horizon H2/H∞ control problem for discrete-time infinite Markov jump systems with multiplicative noises, respectively, but they neglected the effects of time-delay. Meanwhile, the authors in [22] considered time-delay, when discussing the stabilization problem for linear stochastic delay differential equations with infinite Markovian switching, but it was hard for the obtained stability results to deal with control problem. As mentioned above, stability and control for time-delay stochastic systems with infinite Markov jump and multiplicative noises have not received enough attention despite their importance in practical applications, which motivates us for the present research.We aim to address the exponential stability andH∞ control problem for a class of discrete-time time-delay stochastic systems with infinite Markov jumps and multiplicative noises in this paper. The main contributions of this paper are as follows: First of all, we investigate exponential stability of the equilibrium point for the considered systems by employing a novel Lyapunov-Krasovskii functional. Further, a sufficient condition is established to ensure exponential stability with a given H∞ performance index of the closed-loop system. And we introduce the slack matrix to decouple the Lyapunov matrices, which makes the H∞ controller design feasible. Moreover, some numerical examples are provided to show the effectiveness of the proposed design approaches.The remaining part of this paper is constructed as follows. In Section2, we formulate the system model and recall some definitions and lemmas. In Section 3, we present our main results, where we derive some sufficient conditions for exponential stability with a given H∞ performance index. Two numerical examples and their simulations are given to illustrate the effectiveness of the obtained results in Section 4. Conclusions are made in Section 5.For convenience, we fix some notations that will be used throughout this paper. Then-dimensional real Euclidean space is denoted by Rn. Rm×n stands for the linear space of all m by n real matrices. Let · be the Euclidean norm of Rn or the operator norm of Rm×n. By Sn and I(0) we denote the set of all n×n symmetric matrices and the identity (zero) matrix, respectively. A′ denotes the transpose of a matrix (or vector) A. We say that A is positive (semipositive) definite if A>0(≥0). λmax(A)(λmin(A)) represent the maximum (minimum) eigenvalue of A. δ(·) is called the Kronecker function. Z+≔{0,1,…}. D≔{1,2,…}. l2(Z+;Rm)≔{ς∈Rm∣ς is Ft-measurable, and (∑t=0∞Ey(t)2)1/2<∞}.


## 2. Preliminaries

Consider the following discrete-time time-delay stochastic system with infinite Markov jump parameter and multiplicative noises:(1)xt+1=C0stxt+D0stxt-d+R0stut+H0stvt+∑k=1rCkstxt+Dkstxt-d+Rkstut+Hkstvtwkt,yt=Lstxt+L0stxt-d+Nstut+Estvt,xt0=ϕt0,t0=-d~,-d~+1,…,-1,0,s0=s0∈D,t∈Z+,where x(t)∈Rn represents the system state, u(t)∈Rnu is the control input, v(t)∈Rnv denotes the disturbance, and y(t)∈Rnz is the system output. w(t)={wt∣w(t)=w1t,w2t,…,wrt′,t∈Z+} is a sequence of independent random vectors defined on a given complete probability space (Ω,F,P), which satisfies E(w(t))=0 and E(wtws′)=Irδ(t-s). ϕ(t0) is a vector-valued initial condition. d is the bounded constant delay with 0≤d≤d~. Markov chain {st}t∈Z+ takes values in a countably infinite set D with transition probability matrix P=[p(i,j)], where p(i,j)=P(st+1=j∣st=i), and P is nondegenerate, P(s0=i)>0 for all i∈D. Assume {wt}t∈Z+ and {st}t∈Z+ are mutually independent, and Ft={sk,ws∣0≤k≤t,0≤s≤t-1}, F0=σ(s0). Assume v(t) belongs to l2(Z+;Rnv).We introduce the Banach spacesA1m×n={A∣A=(A(1),A(2),…),A(i)∈Rm×n,A1=∑i=1∞A(i)<∞} and A∞m×n={A∣A=(A(1),A(2),…),A(i)∈Rm×n,A∞=supi∈DA(i)<∞}. The notations A1m×n(A∞m×n) will be written as A1n (resp., A∞n) and A1n+ (resp., A∞n+) if and only if m=n and A(i)∈Sn, A(i)≥0, i∈D, respectively. When Y,Z∈A1n+, Y≤Z means that Y(i)≤Z(i), i∈D. Therefore, we have Y1≤Z1. For all coefficients of the considered systems, we suppose they have a finite norm ·∞.Definition 1 (see [10, 18]).
System (1) with u(t)=0 and v(t)=0 is called mean square exponential stability if there exist λ≥1 and τ∈(0,1) such that(2)Ext2≤λτtsup-d~≤l≤0Eϕl2for all t∈Z+, i∈D and x0∈Rn. Further, system (1) with v(t)=0 is called exponential stabilizable if there exists a sequence {K(st)}t∈Z+∈A∞nu×n such that the closed-loop system(3)xt+1=C0st+R0stKstxt+D0stxt-d+H0stvt+∑k=1rCkst+RkstKstxt+Dkstxt-d+Hkstvtwkt,yt=Lst+NstKstxt+L0stxt-d+Estvt,xt0=ϕt0,t0=-d~,-d~+1,…,-1,0,s0=s0∈D,t∈Z+,with v(t)=0 has mean square exponential stability, where u(t)=K(st)x(t) is called exponentially stabilizing feedback.Definition 2.
Closed-loop system (3) is said to have an H∞ noise disturbance attenuation level γ>0, if under zero initial value the following condition is satisfied:(4)∑t=0∞Eyt2<γ2∑t=0∞Evt2for any v(t)∈l2(Z+;Rnv).Lemma 3 (see [22]).
We denoteA~∞n+={A∣A∈A∞n+,theirexistsε>0notdepengdinguponisuchthatA(i)≥εInforalli∈D}. Let(5)B=B11iB12iB12i′B22ii∈D.Assume that B22(i)≥εIn2>0 for all i∈D for some ε>0. Then, B∈A~∞n+ if and only if B∣B22∈A~∞n1+, where n=n1+n2 and B∣B22={B11(i)-B12(i)B22(i)-1B12i′}i∈D is called the Schur complement of B22 in B.Remark 4.
Lemma3 is the infinite-dimensional version of Schur complements (see [24]).


## 3. Main Results

Firstly, stability will be analyzed, and a sufficient condition is obtained for system (1) with u(t)=0 and v(t)=0 to have mean square exponential stability.Theorem 5.
System (1) with u(t)=0 and v(t)=0 is exponentially mean square stable, if we can find matrices P∈A~∞n+, Q∈A~∞n+ such that the following matrix inequality holds:(6)-PCxiDdi∗-Pi+d~+1Qi0∗∗-Qq<0uniformly with respect to (i,q)∈D×D, where(7)Cxi=C0i′,C1i′,…,Cri′′,Ddi=D0i′,D1i′,…,Dri′′,P=diagEiP-1,…,EiP-1︸r+1,EiP=∑j=1∞pi,jPj.Proof.
Construct the following Lyapunov-Krasovskii functional:(8)Vxt,st=xt′Pstxt+∑m=t-dt-1xm′Qsmxm+∑α=-d~+10∑β=t-1+αt-1xβ′Qsβxβ=V1xt,st+V2xt,st+V3xt,st.By the assumption that w(t) is independent of the Markov chain {st}t∈Z+ and E[w(t)]=0, besides Ft-d~⊂Ft, we have(9)
E
V
1
x
t
+
1
,
s
t
+
1
-
V
1
x
t
,
s
t
∣
F
t
,
s
t
=
i
=
x
t
x
t
-
d
′
∑
k
=
0
r
C
k
i
′
E
i
P
C
k
i
-
P
i
∑
k
=
0
r
C
k
i
′
E
i
P
D
k
i
∑
k
=
0
r
D
k
i
′
E
i
P
C
k
i
∑
k
=
0
r
D
k
i
′
E
i
P
D
k
i
x
t
x
t
-
d
,and(10)EV2xt+1,st+1-V2xt,st∣Ft,st=i=xt′Qixt-xt-d′Qst-dxt-d+∑α=t-d~t-1xα′Qsαxα-∑α=t-d~t-1xα′Qsαxα≤xt′Qixt-xt-d′Qst-dxt-d+∑α=t-d~t-1xα′Qsαxα,and(11)EV3xt+1,st+1-V3xt,st∣Ft,st=i=d~xt′Qixt-∑α=t-d~t-1xα′Qsαxα.Thus, combining (8) with (9)-(11), we get(12)EVxt+1,st+1-Vxt,st∣Ft,st=i≤at′RiqPat,where(13)RiqP=∑k=0rCki′EiPCki-Pi+d~+1Qi∑k=0rCki′EiPDki∑k=0rDki′EiPCki∑k=0rDki′EiPDki-Qqwith q=st-d and a(t) is defined as a(t)=xt′xt-d′′.
Applying Lemma3 to (6) leads to(14)diag-Pi+d~+1Qi,-Qq+Mi′P-1Mi<0,where M(i)=Cx(i)Dd(i). Further, we have Riq(P)<0. It is clear from Riq(P)<0 that there exists a sufficiently small scalar ε>0 such that Riq(P)<-εIn. Therefore, it follows that(15)EVxt+1,st+1-Vxt,st<-εExt2.On the other hand, by using (8), we deduce that(16)EVxt,st≤θ1Ext2+d~+1θ2∑β=t-d~t-1Exβ2,where(17)θ1=maxl∈DλmaxPl,θ2=maxp∈DλmaxQp.Noting (15) and (16), for any constant κ>1, we obtain that(18)κt+1EVxt+1,st+1-κtEVxt,st=κt+1EVxt+1,st+1-Vxt,st+κtκ-1EVxt,st≤-κε+κ-1θ1κtExt2+κ-1θ3∑β=t-d~t-1κtExβ2,where θ3=(d~+1)θ2. By taking summation from 0 to T-1 on both sides of (18), for T≥d~+1, it implies that(19)
κ
T
E
V
x
T
,
s
T
-
E
V
x
0
,
s
0
≤
-
κ
ε
+
κ
-
1
θ
1
∑
t
=
0
T
-
1
κ
t
E
x
t
2
+
κ
-
1
θ
3
∑
t
=
0
T
-
1
∑
β
=
t
-
d
~
t
-
1
κ
t
E
x
β
2
≤
-
κ
ε
+
κ
-
1
θ
1
∑
t
=
0
T
-
1
κ
t
E
x
t
2
+
κ
-
1
θ
3
·
d
~
∑
β
=
-
d
~
-
1
κ
β
+
d
~
E
x
β
2
+
d
~
∑
β
=
0
T
-
1
-
d
~
κ
β
+
d
~
E
x
β
2
+
d
~
∑
β
=
T
-
1
-
d
~
T
-
1
κ
β
+
d
~
E
x
β
2
≤
-
κ
ε
+
κ
-
1
θ
1
∑
t
=
0
T
-
1
κ
t
E
x
t
2
+
κ
-
1
θ
3
·
d
~
κ
d
~
m
a
x
-
d
~
≤
β
≤
0
E
ϕ
β
2
+
d
~
κ
d
~
∑
β
=
0
T
-
1
κ
β
E
x
β
2
=
-
κ
ε
+
κ
-
1
θ
1
+
κ
-
1
θ
3
d
~
κ
d
~
∑
t
=
0
T
-
1
κ
t
E
x
t
2
+
κ
-
1
θ
3
d
~
κ
d
~
m
a
x
-
d
~
≤
β
≤
0
E
ϕ
β
2
.Recalling (8) and (16), denoting θ0=minl∈Dλmin(P(l)) and θ=max{θ1,(d~+1)θ2}, we have(20)EVxT,sT≥θ0ExT2,and(21)EVx0,s0≤θmax-d~≤β≤0Eϕβ2,respectively. Furthermore, it suffices to show that there exists a constant κ0>1 such that(22)-κ0ε+κ0-1θ1+κ0-1θ3d~κ0d~=0.Actually, letting f(κ)=[-κε+(κ-1)θ1]+(κ-1)θ3d~κd~, then we have f′(κ)>0 and f(1)<0. Therefore, (22) has a unique solution κ0>1. By substituting (20)-(22) into (19), we obtain(23)ExT2≤λ01κ0Tmax-d~≤β≤0Eϕβ2,where λ0=θ+(κ0-1)θ3d~κ0d~/θ0. This indicates that system (1) with u(t)=0 and v(t)=0 has mean square exponential stability. The proof is completed.Remark 6.
Due to the consideration of an infinite-state Markov chain, the infinite dimension Banach spaces have been introduced. Furthermore, it should be pointed out that a novel Lyapunov-Krasovskii functional (8) has been constructed to analyze the mean square exponential stability for system (1) with u(t)=0 and v(t)=0.Next, we prove that system (1) with u(t)=0 verifies the H∞ performance disturbance attenuation γ.Theorem 7.
System (1) has mean square exponential stability for u(t)=0 and v(t)=0 with a prescribed H∞ performance γ for u(t)=0, if we can find matrices P∈A~∞n+, Q∈A~∞n+ such that the following matrix inequality holds:(24)-PCxiDdiHvi0∗-Pi+d~+1Qi00Li′∗∗-Qq0L0i′∗∗∗-γ2IEi′∗∗∗∗-I<0,uniformly with respect to (i,q)∈D×D, where(25)Hvi=H0i′,H1i′,…,Hri′′.Proof.
It is well established that (24) implies (6). Applying Theorem 5 one obtains that system (1) has mean square exponential stability for u(t)=0 and v(t)=0.
Let us now show that system (1) with u(t)=0 satisfies a prescribed H∞ performance level. To this end, constructing the same Lyapunov-Krasovskii functional V(x(t),st) as in Theorem 5 and under the zero initial condition, the following index is introduced:(26)JT=∑t=0TEyt2-γ2vt2=∑t=0TEyt2-γ2vt2+Vxt+1,st+1-Vxt,st-VxT+1,sT+1≤∑t=0TEyt2-γ2vt2+Vxt+1,st+1-Vxt,st≤Ebt′Ast,st-d-Bst′Cst-1Bstbt,where(27)Ast,st-d=diag-Pst+d~+1Qst,-Qst-d,-γ2I,Bi=CxstDdstHvstLstL0stEst,Cst=diag-P,-I,and b(t) is defined as b(t)=xt′xt-d′vt′′. The last ‘≤’ in (26) holds as a result of the similar line with (12). Then, by using Lemma 3 in (24), we obtain that A(st,st-d)-Bst′C(st)-1B(st)<0. Thus, JT<0. Taking the limit T→∞ in (26), we have(28)∑t=0∞Eyt2<γ2∑t=0∞Evt2.This ends the proof.Combining Theorem5 with Theorem 7, the following corollary can be easily derived for closed-loop system (3).Corollary 8.
Let the feedback control gainK(i), i∈D, be given. Then closed-loop system (3) has mean square exponential stability for v(t)=0 with a prescribed H∞ performance γ if there exist two matrices P∈A~∞n+ and Q∈A~∞n+, such that(29)-PC¯xiDdiHvi0∗-Pi+d~+1Qi00L¯i′∗∗-Qq0L0i′∗∗∗-γ2IEi′∗∗∗∗-I<0,uniformly with respect to (i,q)∈D×D, where(30)C¯xi=C0i+R0iKi′,C1i+R1iKi′,…,Cri+RriKi′′,L¯i=Li+NiKi.Below, based on Corollary8, we are ready to present the H∞ controller design for system (1).Theorem 9.
For system (1), a state feedback controller can be designed such that closed-loop system (3) has mean square exponential stability for v(t)=0 and a given H∞ performance γ can be ensured if there exist matrices P^∈A~∞n+, Q^∈A~∞n+, K^∈A∞nu×n, and F∈Rn×n such that(31)ΦiΓiΥi∗-I0∗∗-P^<0,uniformly with respect to (i,q)∈D×D, where(32)Φi=P^i+d~+1Q^i-F′-F00∗-Q^q0∗∗-γ2I,Γi=LiF+NiK^i,L0iF,Ei′,Υi=CxiF+RuiK^i,DdiF,Hvi′,Rui=R0i′,R1i′,…,Rri′′,P^=diagEiP^-1,…,EiP^-1︸r+1,EiP^=∑j=1∞pi,jP^j-1.Moreover, if matrix inequalities (31) are feasible, then an exponentially stabilizing feedback gain can be given by(33)Ki=K^iF-1.Proof.
Via Lemma3, we conclude that (29) is equivalent to the following matrix inequality:(34)-Pi+d~+1Qi00L¯i′C¯xi′∗-Qq0L0i′Ddi′∗∗-γ2IEi′Hvi′∗∗∗-I0∗∗∗∗-P<0.Premultiply diagF′,F′,I,I,I and postmultiply diag{F,F,I,I,I} with (34), and let(35)P^i=Pi-1,Q^i=F′QiF,K^i=KiF.By a tedious calculation, one can rewrite (34) as(36)Φ^iΓiΥi∗-I0∗∗-P^<0,where(37)Φ^i=-F′P^i-1F+d~+1Q^i00∗-Q^q0∗∗-γ2I.According to Corollary 8 and the fact that(38)P^i-F′P^i-1P^i-F≥0,namely,(39)-F′P^i-1F≤P^i-F-F′,the desired result is derived.Remark 10.
The work in [20] presented a necessary and sufficient condition for the existence of the mixed H2/H∞ control by four coupled matrix Riccati equations (CMREs). Note that CMREs are hardly solved in practice, and this motivates us to find a new sufficient condition in terms of matrix inequalities that can be easily solved to guarantee that the resulting closed-loop system has mean square exponential stability for the zero exogenous disturbance and satisfies a prescribed H∞ performance level.Remark 11.
With the introduction of a slack matrixF, a sufficient condition is obtained in Theorem 9, in which the Lyapunov matrices are not involved in any product with system matrices. This makes the H∞ controller design feasible and can be easily carried out by solving corresponding matrix inequalities.Remark 12.
It is worth noting that the obtained results can be extended to discrete-time time-delay infinite Markov jump stochastic systems with time-varying delays. Assume that the time-varying delayd(t) satisfies dm≤d(t)≤dM; then by similar procedures to the above and choosing the following Lyapunov-Krasovskii function(40)Vxt,st=xt′Pstxt+∑m=t-dtt-1xm′Qsmxm+∑α=-dM+2-dm+1∑β=t-1+αt-1xβ′Qsβxβ,the corresponding results can be derived.


## 4. Illustrative Example

In this section, some illustrative examples are presented to demonstrate the effectiveness of the developed method.Example 1.
Consider the following one-dimensional discrete-time time-delay stochastic system with infinite Markov jumps:(41)xt+1=c0stxt+d0stxt-d+∑k=1rckstxt+dkstxt-dwkt,where the transition probability is defined by p(i,i)=1/4, p(i,i+1)=3/4, p(i,j)=0, j≠i,i+1, i,j∈D. Now take(42)c0i=ii+1,c1i=1i+1,cki=0,k=2,3,…,r,i∈D,d0i=ii+1,d1i=1i+1,dki=0,k=2,3,…,r,i∈D.Let P(i)=4(i+1)/3i, Q(i)=1/9i(i+1), and time-delay d=2. By direct computation, (6) holds. According to Theorem 5, we deduce that system (41) has mean square exponential stability, and Figure 1 presents the state response of system (41) with initial conditions ϕ(t0)=0.5 for t0=-2,-1,0.Figure 1
System state response in Example1.Example 2.
Consider the following one-dimensional discrete-time time-delay stochastic system with infinite Markov jumps:(43)xt+1=c0st+r0stKstxt+d0stxt-d+h0stvt+∑k=1rckst+rkstKstxt+dkstxt-d+hkstvtwkt,yt=lst+nstKstxt+l0stxt-d+estvt,xt0=ϕt0,t0=-d~,-d~+1,…,-1,0,s0=s0∈D,t∈Z+,where the transition probability is defined by p(i,i)=1/2, p(i,i+1)=1/2, p(i,j)=0, j≠i,i+1, i,j∈D. The coefficients of system (43) are reset to be(44)c0i=-ii+12,c1i=1i+1,cki=0,k=2,3,…,r,i∈D,d0i=-i2i+1,d1i=1i+1,dki=0,k=2,3,…,r,i∈D,r0i=-ii+1,r1i=1,rki=0,k=2,3,…,r,i∈D,h0i=1,h1i=1,hki=0,k=2,3,…,r,i∈D,li=1i+1,l0i=1,ni=1,ei=0.1.The purpose here is to design an H∞ controller such that the closed-loop system has mean square exponential stability and with a given H∞ norm bound γ=0.5. Applying Theorem 9, the H∞ controller can be designed as(45)Ki=-1i+1.
With the initial conditionsϕ(t0)=0.1 for t0=-2,-1,0 and the exogenous disturbance v(t)=2e-tsin⁡t, Figures 2 and 3 show the state and output responses, respectively.Figure 2
System state response in Example2.Figure 3
System output response in Example2.


## 5. Conclusions

In this paper, the issue of exponential stability and robustH∞ control for a class of discrete-time time-delay stochastic systems with infinite Markov jumps and multiplicative noises has been studied. Time-delay and infinite Markov jump are taken into consideration simultaneously. By using Lyapunov-Krasovskii functional and introducing slack matrix, an matrix inequality approach has been adopted to ensure the mean square exponential stability and satisfy a prescribed H∞ performance level. Finally, some illustrative examples are given to demonstrate the usefulness of the proposed design methods. Further research directions would include the investigation on H2/H∞ control problem and asynchronous control problem for discrete-time time-delay stochastic systems with infinite Markov jumps.



---

*Source: 3676083-2018-10-22.xml*