# Further Results on Stability Analysis of Discrete-Time Markov Jump Linear Systems with Time-Varying Delay and Partly Known Transition Probabilities

Authors: Xingang Zhao; Jianda Han; Yiwen Zhao
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

This paper is concerned with the stability analysis of discrete-time Markov jump linear systems (MJLSs) with time-varying delay and partly known transition probabilities. The time delay is varying between lower and upper bounds, and the partly known transition probabilities cover the cases of known, uncertain with known lower and upper bounds, and completely unknown, which is more general than the
existing result. Via constructing an appropriate Lyapunov function and employing a new technique to separate Lyapunov variables from unknown transition probabilities, a novel stability criterion is obtained in the framework of linear matrix inequality. A numerical example is given to show the effectiveness of the proposed approach.


---

## Body

## 1. Introduction

Markov jump linear systems (MJLSs), a class of stochastic systems, have been widely applied in manufacturing systems, power systems, aerospace systems, and networked control systems. This is due to its ability to model many practical systems with abrupt random changes in their structures and parameters. Regarding the details on the basic theory of MJLSs, for example, controllability, observability, stability, stabilisability, and optimal control, and so forth, please refer to [1–8] and the references therein.Time delay, unavoidable in many practical systems, can degrade the closed loop system performance or even lead the resulting system to be unstable. In recent years, therefore, the time delay problem has been widely studied and a great number of research results concerning this point exist in the references for linear system; see, for example, [9–13] and so on. Regarding the study of MJLSs with time delay, mean square stochastic stability conditions are presented in [7] and exponential stability conditions are proposed in [8]. However, these results are delay independent. To make full use of the available information on the size of delays, delay-dependent results are more attractive. Employing a model transformation, delay-dependent stability conditions for MJLSs with constant delay are proposed in [14]. To derive less conservative results, this idea was further improved by [15, 16]. Instead of using model transformation or bounding for cross terms, zero equations are utilized by [17]. Considering the fact that time delay for practical system is time varying, based on the free weighting matrix technique, [18] proposes a less conservative stability criterion for MJLSs with time-varying delays. By introducing some improved integral equalities, a new stability condition for MJLSs with time-varying delay is proposed in [19]. Motivated by above results, [20] proposes improved stochastic stability conditions for MJLSs with interval time delay. Regarding the stability analysis of Markovian jump neural systems with mixed delay, sample data control, impulse control, and exponential stability are investigated in [21–27]. With resorting to new technique to deal with the time delay, based on a Lyapunov-Krasovskii functional and the stochastic analysis theory, some novel sufficient conditions are established in the framework of linear matrix inequalities [22–27]. It deserves to note that all above results are based on transition probabilities which are known. Practically, it is hard or costly to obtain the complete knowledge on the transition probabilities [28, 29]. A typical example can be found in networked control systems (NCSs). Due to the complexity of the network environment, the variation of delays could be vague and random, which leads to the fact that all or part of the elements in the expected transition probabilities matrix are probably hard or expensive to obtain [28–32]. To be consistent with the actual situation, transition probabilities in [28–32] are assumed to be known or unknown. With this assumption, stability analysis of MJLSs with time-varying delay is investigated in [32]. However, in these results, when transition probabilities are unknown with known lower and upper bounds, they are treated as completely unknown, which may cause conservativeness. Moreover, the technique to separate Lyapunov variables and unknown transition probabilities in [32] still leaves some room for further improvement.Motivated by the above observations, this paper is devoted to the stability analysis of discrete-time MJLSs with time-varying delay and partly known transition probabilities. The boundary information of time-varying delay is made full use of and the partly known transition probabilities include the cases of known, uncertain with known lower and upper bounds, and completely unknown, which is more general than the existing result. Via constructing an appropriate Lyapunov-Krasovskii functional, combining the property of transition probabilities and free weighted matrix technique, a novel stability criterion is obtained in the framework of linear matrix inequality. A numerical example is utilized to show the effectiveness of the proposed approach.The rest of the paper is organized as follows. Section2 gives the problem description, in Section 3, a novel theorem about delay range dependent stability for systems with more general transition probabilities is derived by using Lyapunov function method, Section 4 illustrates the efficiency of the proposed method, and Section 5 concludes this paper.Notation.
R
n denotes the n dimensional Euclidean space and the notation P>0  (P≥0) means that P is real symmetric and positive definite (semidefinite). N+ represents the set of positive integers. In symmetric block matrices or complex matrix expressions, we use an asterisk (*) to represent a term that is induced by symmetry and diag⋯ stands for a block-diagonal matrix. I and 0 represent, respectively, the identity matrix and zero matrix.


## 2. System Description and Preliminaries

Consider the following class of discrete-time Markov jump linear systems:(1)xk+1=Arkxk+Adrkxk-dk,xk=ϕk,k=-dM,-dM+1,…,0,
where x(k)∈Rn is the state vector. The time delay is considered to be time-varying with known lower and upper bounds, namely, 0≤dm≤dk≤dM. rk (k≥0) is a discrete-time homogeneous Markov chain, which takes values in a finite set I={1,2,…,N} with the following mode transition probabilities:
(2)Pr⁡rk+1=j∣rk=i=πij,
where πij≥0 and ∑j=1Nπij=1.Considering the fact that transition probabilities are hard to be measured exactly, they are assumed to be known, unknown with known lower and upper bounds, and completely unknown in this context [31]. To present the considered transition probabilities clearly, an example of partly known transition probability matrix is given in the following:
(3)Π=π11?π13⋯π1N??α⋯π2Nβ??⋯π3N⋮⋯⋯⋱⋮πN1πN2?⋯πNN,
where “?” denotes that the corresponding element is completely unknown, α and β(α_<α<α¯,  β_<β<β¯) represent unknown elements with known bounds, and πij represents the exactly known elements. Subsequently, all transition probabilities in the above matrix can be classified by the following two sets:
(4)Iki≜{j:π_ij≤πij≤π¯ij},Iuki≜{j:j∉Iki}.Remark 1.
With the fact that known transition probabilities can be treated as equal lower and upper bounds, that is to say,π_ij=π¯ij=πij, the first set, Iki, includes not only unknown transition probabilities with known bounds but also completely known case.Before proceeding further, let us introduce the following stability definition.Definition 2.
System (1) is said to be stochastically stable if for every initial condition ϕ(k)∈Rn,  k=-dM,-dM+1,…,0 and r0∈I, the following holds:
(5)E∑k=0∞‍xk2∣ϕ·,r0<∞.


## 3. Main Results

In this part, a new delay-dependent stochastic stability condition for MJLSs with partly known transition probabilities is presented in the following theorem.Theorem 3.
Consider the MJLSs (1) with partly known transition probabilities (3), if there exist symmetric matrices Pi>0,  Q>0,  R>0,  Z1>0,  Z2>0,  Miv,  Niv,  Siv,  v=1,2,3,  i∈I,  l∈Iuki such that
(6)E1+E2+E3E4*E5<0,
where(7)P¯ki=∑j∈Iki‍π¯ijPj,π_ki=∑j∈Iki‍π_ij,E1=AiTP¯ki+1-π_kiPlAi-Pi+dM-dm+1Q+RAiTP¯ki+1-π_kiPlAdi0*AdiTP¯ki+1-π_kiPlAdi-Q0**-R,E2=Mi+NiSi-Mi-Ni-Si+Mi+NiSi-Mi-Ni-SiT,E3=dMAi-IAdi0TZ1+Z2Ai-IAdi0,E4=dMMidrSidMNi,E5=-Z1000-Z1000-Z2.Proof.
Consider system (1) and construct a stochastic Lyapunov functional as
(8)Vxk,rk=Vxk,i=∑s=15‍Vsxk,i,i∈I,
where
(9)V1xk,i=xkTPixk,V2xk,i=∑l=k-dkk-1‍xTlQxl,V3xk,i=∑θ=-dM+1-dm‍∑l=k+θk-1‍xTlQxl,V4xk,i=∑l=k-dMk-1‍xTlRxl,V5xk,i=∑θ=-dM-1‍∑l=k+θk-1‍yTlZ1+Z2yl,oyl=xl+1-xl,dr=dM-dm.
DefiningΔV(xk,i)=∑s=15‍ΔVs, it follows that(10)ΔV1=E[V1xk+1,rk+1=j∣xk,rk=i-V1(xk,rk=i)]=xk+1T∑j∈I‍πijPjxk+1-xkTPixk=xkT(AiTPiAi-Pi)xk+2xkTAiTPiAdixk-d(k)+xk-dkTAdiTPiAdixk-dk,ΔV2=E[V2xk+1,rk+1=j∣xk,rk=i-V2(xk,rk=i)]=∑l=k+1-dk+1k‍-∑l=k-dkk-1‍xT(l)Qx(l)=xkTQxk-xk-dkTQxk-dk+∑l=k+1-dk+1k-dk‍xT(l)Qx(l)≤xkTQxk-xk-dkTQxk-dk+∑l=k+1-dMk-dm‍xTlQxl,ΔV3=E[V3xk+1,rk+1=j∣xk,rk=i-V3(xk,rk=i)]=∑θ=-dM+1-dm‍(∑l=k+1+θk‍-∑l=k+θk-1‍)xT(l)Qx(l)=dM-dmxkTQxk-∑l=k+1-dMk-dm‍xTlQxl,ΔV4=E[V4xk+1,rk+1=j∣xk,rk=i-V4(xk,rk=i)]=∑l=k+1-dMk‍-∑l=k-dMk-1‍xT(l)Rx(l)=xkTRxk-xk-dMTRxk-dM,ΔV5=EV5xk+1,rk+1=j∣xk,rk=i-V5xk,rk=i=∑θ=-dM-1‍∑l=k+θ+1k‍yTlZ1+Z2yliiiiiiiiiiiii-∑l=k+θk-1‍yT(l)(Z1+Z2)y(l)=∑θ=-dM-1‍∑l=k+θ+1k‍-∑l=k+θk-1‍yT(l)(Z1+Z2)y(l)=∑θ=-dM-1‍ykTZ1+Z2yk-yk+θTZ1+Z2yk+θ=dMykT(Z1+Z2)yk-∑l=k-dkk-1‍yT(l)Z1y(l)-∑l=k-dMk-dk-1‍yT(l)Z1y(l)-∑l=k-dMk-1‍yT(l)Z2y(l)=dMykT(Z1+Z2)yk-∑l=k-dkk-1‍yT(l)Z1y(l)-∑l=k-dMk-dk-1‍yT(l)Z1y(l)-∑k-dkk-1‍yT(l)Z2y(l)-∑k-dMk-dk-1‍yTlZ2yl.
Summing up (10), one has
(11)ΔVxk,rk≤xkT(AiTPiAi-Pi)xk+2xkTAiTPiAdixk-dk+xk-dkTAdiTPiAdixk-dk-xk-dkTQxk-dk+(dM-dm+1)xkTQxk+xkTRxk-xk-dMTQxk-dM+dMAi-Ixk+Adixk-dkT×(Z1+Z2)[(Ai-I)xk+Adixk-dk]-∑l=k-dkk-1‍yT(l)Z1y(l)-∑l=k-dMk-dk-1‍yT(l)Z1y(l)-∑l=k-dkk-1‍yT(l)Z2y(l)+2ξkTMi[xk-xk-dk-∑l=k-dkk-1‍y(l)]+2ξkTSi[xk-dk-xk-dM-∑l=k-dMk-dk-1‍y(l)]+2ξkTNixk-xk-dM-∑l=k-dMk-1‍yl.
Therefore, we have(12)ΔVxk,rk≤ξT(k)E^1+E2+E3+dMMiZ1-1MiTooooooooo+(dM-dm)×SiZ1-1SiTooooooooo+dMNiZ2-1NiTξk,
where
(13)ξk=xkTxk-dkTxk-dMTT,E^1=AiT(Pi)Ai-Pi+(dM-dm+1)Q+RAiTPiAdi0*AdiTPiAdi-Q0**-R,E2=Mi+NiSi-Mi-Ni-Si+Mi+NiSi-Mi-Ni-SiT,E3=dMAi-IAdi0TZ1+Z2Ai-IAdi0,E4=dMMidrSidMNi,E5=-Z1000-Z1000-Z2,Mi=Mi1TMi2TMi3TT,Ni=Ni1TNi2TNi3TT,Si=Si1TMi2TSi3TT.
On the other hand, the following formula holds:(14)Pi=∑j∈I‍πijPj=∑j∈Iki‍πijPj+∑j∈Iuki‍πijPj.
Considering the fact that∑j=1Nπij=1, it yields
(15)∑j∈Iki‍πij+∑j∈Iuki‍πij=1.
Since transition probabilities are partly known, we have πki=∑j∈Ikiπij<1 and ∑j∈Iukiπij/1-πki=1. Therefore, it follows that
(16)AiTPiAi=AiT∑j∈Iki‍πijPj+∑j∈Iuki‍πijPjAi=AiTPki+∑j∈Iuki‍πijPjAi=AiTPki+1-πki∑j∈Iuki‍πij1-πkiPjAi=∑j∈Iuki‍πij1-πkiAiTPki+1-πkiPjAi.
According to (6), we can get ΔV(xk,rk)<0.Remark 4.
According to Theorem3, when calculating the forward difference of ΔV5, -∑l=k-dMk-1‍yT(l)Z1y(l) is divided as -∑-k-dkk-1‍yT(l)Z1y(l) and -∑-k-dMk-dk-1‍yT(l)Z1y(l).Remark 5.
Different from the existing result [32], to separate Lyapunov variables from unknown transition probabilities, the property of transition probabilities is made full use of, namely, ∑j∈Iukiπij=1-∑j∈Ikiπij. Thanks to this separation, the method proposed in Theorem 3 is less conservative than that of [32].


## 4. Illustrative Example

To show the effectiveness of the proposed method, a numerical example is given below.Example 1.
Consider the MJLSs (1) with four operation modes and the following system matrices:
(17)A1=20.560.251,A2=70.640-0.24,A3=-1.230.650.44-0.87,A4=40.640.265,Ad1=-0.030.1100.13,Ad2=0.1300.020.03,Ad3=0.02-0.120.07-0.14,Ad4=0.90.070.04-0.05.
Our purpose is to check the stability of (1) with different cases of transition probabilities.Case 1.
Transition probabilities are known(18)0.40.6000.20.60.190.010.20.30.40.10.30.30.30.1.
In this case, given dM=0 and dm=0, system (1) is unstable even if all the transition probabilities are known. It implies that the underlying system will be unstable for any time-varying delay starting from dm=0.Case 2.
Transition probabilities are partly known(19)0.40.6000.20.60.190.01?0.3?0.10.3?0.3?.
In this case, we assume that the range of the time-varying delay belongs to [110]. By solving the conditions given in Theorem 1 of [32] and Theorem 3 in this paper, respectively, no feasible solution can be found for the former. Therefore, the method proposed in this paper is less conservative than the existing result.Case 3.
We consider the case that there are unknown elements in the transition probability matrix varying an interval(20)0.40.6000.2α0.19??0.3?0.10.3?0.3?,
where 0.1≤α≤0.4.
By solving the conditions given in Theorem3, Table 1 is obtained.
According to Table1, it can be seen that the more transition probabilities knowledge we have, the larger the delay range can be obtained.Table 1
TPs
Case1
Case2
Case3
d
(
k
)
1
≤
d
(
k
)
≤
20
1
≤
d
(
k
)
≤
10
1
≤
d
(
k
)
≤
5
Theorem3
Feasible
Feasible
Feasible


## 5. Conclusions

The stability analysis for a class of discrete-time MJLSs with time-varying delay and partly known transition probabilities is investigated in this paper. To separate Lyapunov variables from unknown transition probabilities, the property of transition probabilities is made full use of. Via constructing an appropriate Lyapunov function, a novel stability criterion is obtained in the framework of linear matrix inequality. The effectiveness of the proposed approach is demonstrated by a numerical example. In the future, the obtained results are considered to be extended to deal with the fault tolerant control of Markov jump systems with partly known transition probabilities and mixed time-varying delay.



---

*Source: 524584-2015-03-02.xml*