# Stability Analysis for a Class of Discrete-Time Nonhomogeneous Markov Jump Systems with Multiplicative Noises

Authors: Shaowei Zhou; Xiaoping Liu; Bing Chen; Hongxia Liu
Journal: Complexity
 (2018)

---

## Abstract

This paper is concerned with a class of discrete-time nonhomogeneous Markov jump systems with multiplicative noises and time-varying transition probability matrices which are valued on a convex polytope. The stochastic stability and finite-time stability are considered. Some stability criteria including infinite matrix inequalities are obtained by parameter-dependent Lyapunov function. Furthermore, infinite matrix inequalities are converted into finite linear matrix inequalities (LMIs) via a set of slack matrices. Finally, two numerical examples are given to demonstrate the validity of the proposed theoretical methods.


---

## Body

## 1. Introduction

There are a lot of dynamic systems which are subjected to abrupt variations in parameters caused by external surroundings or internal structure in practical engineering field. Markov jump system models have attracted more and more attention because they effectively described this kind of systems and have become a hot topic in control theory. Many mature and systematic results have been obtained [1–5].It should be pointed out that most of the current research is done in the framework of homogeneous Markov process (or Markov chain), which is under the assumption that the transition rate (or transition probability) matrix of the system at any time is the same. In fact, due to the impact of the objective environment, this assumption is difficult to satisfy. For example, in a Markov jump networked control system, the transition probability is time-variant because the packet dropouts and network delays are different at different periods. Another typical example can be found in the fault-prone systems, where Markov process is used to describe the failure rate that is influenced by the factors of the age and the usage rate. Obviously, it is not a homogeneous process. Driven by these practical problems, people turn to the nonhomogeneous Markov jump systems [6–21].In order to describe the nonhomogeneous property, several assumptions are put forward. In [6], the time-varying transition probabilities of discrete-time Markov jump systems are considered to be finite piecewise homogeneous with two types of variations in the finite set: arbitrary variation and stochastic variation, which implies that the transition probabilities are varying but invariant within an interval. The H∞ estimation problem is investigated. This assumption is generalized to the continuous-time Markov jump system [7], Markov jump neural networks [8, 9], complex networks [10], and singular Markov jump systems [11], and the robust stability, stochastic stability, passivity analysis, and synchronization are studied, respectively. Another way of describing time-varying characteristics is in a polytopic sense. It is proposed for the first time in [12] and a sufficient condition for stochastic stability is derived by using a parameter-dependent stochastic Lyapunov function. The main idea is that the transition probability matrix is assumed to be valued in a polytope convex set with some given vertices when the exact transition probability is not known. The polytopic model is more general and includes the piecewise homogeneous Markov jump system model with arbitrary switch as a special case. Subsequently, many new results are obtained on this model. In [13–15], mode-dependent, variation-dependent, and observer-based H∞ controllers are designed to satisfy the stochastic stability and prescribed H∞ performance index. The other control problems, such as l2–l∞ control [16] and fault detection [17], are also considered. An application to DC motors can be found in [18]. The model is used not only in discrete-time systems but also in continuous-time systems; see [19]. However, up to now, the model has not been applied to Markov jump systems with multiplicative noises by the authors’ knowledge. This kind of systems with multiplicative noises is often used in engineering practice. For example, a practical model with the control input dependent noise in CDMA systems can be found in [22]. This note aims to make an attempt to investigate the stability of such systems.Our purpose is to address the stability analysis and stabilization controller design for discrete-time nonhomogeneous Markov jump systems with multiplicative noises and with polytopic transition probability matrices. It is well known that stability analysis is the basis for the design and synthesis in all control systems and fruitful results have been obtained; see [23–30] and the references therein. In this article, two types of stability, stochastic stability (SS) in mean square sense and finite-time stability (FTS), will be considered.The organization of this paper is as follows. In Section2, we provide model formulation and give some definitions. Section 3 is devoted to stochastic stability and stabilization. In Section 4, the finite-time stability and stabilization will be discussed. Some numerical examples are presented in Section 5. Finally, a brief concluding remark is given in Section 6.Notations 1.
R n is n-dimensional Euclidean space; AT is the transpose of matrix A; A is the determinant of matrix A; A>0: A is a positive definite matrix; I is the identity matrix; x is the Euclidean norm of vector x; E(·) is the expectation operator; δ(·) is Kronecker function; diag(A1,…,An) is a block diagonal matrix with the block diagonal elements A1,…,An; λmin(A) (λmax(A)) is the minimum (maximum) eigenvalue of matrix A; Cond(A) is the condition number of matrix A; ∗ is the symmetric part of a symmetric matrix; N0=0,1,2,…, N=1,2,…, and NT=1,2,…,T.


## 2. Problem Formulation

Consider the following discrete-time stochastic Markov jump system with multiplicative noises:(1)xk+1=Arkxk+Crkxkwk,x0=x0.And the corresponding controlled system is as follows:(2)xk+1=Arkxk+Brkuk+Crkxk+Drkukwk,x0=x0,where xk∈Rn and uk∈Rm are the system state and control input, respectively. x0∈Rn is the initial state and A(·), B(·), C(·), and D(·) are real coefficient matrices with the appropriate dimension. wk,k∈N0 is a sequence of real random variables defined on a complete probability space (Ω,F,P) with E(wk)=0 and E(wswt)=δ(s-t). The jump process {rk,k∈N0} is described by a discrete-time nonhomogeneous Markov chain which takes values in a finite set S={1,2,…,s}. The transition probability is πij(k)=P(rk+1=j∣rk=i) which means the probability from mode i at time k to mode j at time k+1 and satisfies πij(k)≥0, ∑j=1sπij(k)=1. The transition probability matrix is Γ(k)=πij(k). It is assumed that Γ(k) is time-varying and takes values in a polytope with its vertices Γl,l=1,2,…,m; that is,(3)Γk=∑l=1mαlkΓl,αlk≥0,∑l=1mαlk=1,where Γl(l=1,2,…,m) are given matrices and the entries of Γl are written by πijl. wk,k∈N0 is independent of {rk,k∈N0}.To be convenient, we denote the coefficient matrices associated withrk=i as Mi=M(rk).In this paper, we mainly formulate some conditions of SS and FTS. They are two different and independent stability concepts: SS describes the asymptotic behavior of the systems in infinite time domain, while FTS reflects transient performance of the systems in finite-time interval. Now, let us recall the definitions.Definition 2 (see [1]).
System (1) is called stochastically stable if for any initial state x0 and initial mode r0,(4)∑k=0∞Exk2<∞.System (2) is called stochastically stabilizable if there exist a sequence of feedback controls uk=K(rk)xk such that for any initial state x0 and initial mode r0, the closed-loop system(5)xk+1=Ark+BrkKrkxk+Crk+DrkKrkxkwk,x0=x0is stochastically stable.Definition 3 (see [23]).
System (1) is called finite-time stable with respect to (c1,c2,R,T), if(6)Ex0TRx0≤c1⟹ExkTRxk≤c2,∀k∈NT,where R>0 is a given matrix, c2>c1>0, and T∈N are the given real numbers. System (2) is said to be finite-time stabilizable if there exist a sequence of feedback controls uk=K(rk)xk such that the closed-loop system (5) is finite-time stable.


## 3. Stochastic Stability and Stabilization

In this section, some sufficient conditions will be provided for the stochastic stability and stabilization of systems (1) and (2).Theorem 4.
If there exist a set of matricesPil>0,∀i∈S,l=1,2,…,m, such that for every αl(k) and βn(k), l,n=1,2,…,m,(7)AiT∑j=1s∑l=1m∑n=1mαlkβnkπijlPjnAi+CiT∑j=1s∑l=1m∑n=1mαlkβnkπijlPjnCi-∑l=1mαlkPil<0,where 0≤αl(k), βn(k)≤1, ∑l=1mαl(k)=1, and ∑n=1mβn(k)=1, then system (1) is stochastically stable.Proof.
LetV(xk,rk=i)=xkT∑l=1mαl(k)Pilxk and Fk be the σ-algebra generated by {(rt,xt),t=0,1,…,k}. Then, (8)△Vxk,rk=i=EVxk+1,rk+1=j-Vxk,rk=i=EExk+1T∑l=1mαlk+1Pjlxk+1∣Fk=Exk+1T∑j=1s∑l=1mαlk+1PjlPrk+1=j∣rk=ixk+1-xkT∑l=1mαlkPilxk=Exk+1T∑j=1sπijk∑l=1mαlk+1Pjlxk+1-xkT∑l=1mαlkPilxk=Exk+1T∑j=1s∑l=1mαlkπijl∑n=1mβnkPjlxk+1-xkT∑l=1mαlkPilxk=Exk+1T∑j=1s∑l=1m∑n=1mαlkβnkπijlPjlxk+1-xkT∑l=1mαlkPilxk=xkTAiT∑j=1s∑l=1m∑n=1mαlkβnkπijlPjlAi+CiT∑j=1s∑l=1m∑n=1mαlkβnkπijlPjlCi-∑l=1mαlkPilxk≜xkTΨikxk.
So we have(9)△Vxk,rk=i≤-mini∈S⁡mink∈N0⁡λmin-ΨikxkTxk≜-ρxkTxk,obviously, ρ>0; thus (10)∑k=0MEΔVxk,rk=EVxM+1,rM+1-Vx0,r0≤-ρ∑k=0MExk2.Hence, (11)∑k=0MExk2≤1ρVx0,r0-EVxM+1,rM+1≤1ρVx0,r0,reducing to (12)∑k=0∞Exk2=limM→∞⁡∑k=0MExk2≤1ρVx0,r0<∞.Therefore, system (1) is stochastically stable from Definition 2. The proof is completed.Remark 5.
Theorem4 develops a sufficient condition for SS of system (1) when transition probability matrix is arbitrarily valued in a polytope with given vertices. For finite Markov jump switching systems, the stochastic stability is equivalent to asymptotic mean square stability (AMSS) [1]. So Theorem 4 is also a sufficient condition for AMSS.Remark 6.
A convex parameter space method was used in control design of uncertain linear system without jumping. State feedback conditions can be easily derived by exploiting the change of matrix variable due to Geromel and coworkers [31].It is hard to verify the conditions in Theorem4 because there are infinite matrix inequalities. Hence it has more theoretical significance than its practical significance. To be solvable, we introduce a set of slack matrices and convert the infinite matrix inequalities into finite LMIs in the following theorem.Theorem 7.
The following two statements are equivalent:
(1) There exist a set of matricesPil>0, ∀i∈S, and l=1,2,…,m, such that (7) holds for every αl(k) and βn(k), l, n=1,2,…,m, satisfying 0≤αl(k), βn(k)≤1, ∑l=1mαl(k)=1, and ∑n=1mβn(k)=1.
(2) There exist a set of matricesQil>0 and matrices Gil, such that(13)Gil+GilT-QilGilTAiTΓilGilTCiTΓil∗Q¯n0∗∗Q¯n>0,∀i∈S,l,n=1,2,…,m,where Γil=πi1lI,…,πislI and Q¯n=diag(Q1n,…,Qsn).
The proof is similar to Proposition2 in [12], so it is omitted.From Theorem7, if Ai,Ci are replaced with Ai+BiKi,Ci+DiKi, respectively, it is easy to get the following stabilization condition and controller design method.Corollary 8.
If there exist a set of matricesQil>0 and matrices Gi, Ri such that the LMIs(14)Gi+GiT-QilGiTAiT+RiTBiTΓilGiTCiT+RiTDiTΓil∗Q¯n0∗∗Q¯n>0,∀i∈S,l,n=1,2,…,m,hold, then system (2) is stochastically stabilized by uk=K(rk)xk with the feedback gains Ki=RiGi-1.


## 4. Finite-Time Stability and Control

In this section, we will consider the finite-time stability. Firstly a theoretical criterion is given.Theorem 9.
If there exist a scalarγ≥1 and matrices Pil>0, ∀i∈S, such that(15)AiT∑j=1s∑l=1m∑n=1mαlkβnkπijlPjnAi+CiT∑j=1s∑l=1m∑n=1mαlkβnkπijlPjnCi-γ∑l=1mαlkPil<0,(16)c1γTCondP¯i<c2,P¯i=∑l=1mαlkR-1/2PilR-1/2,where 0≤αl(k), βn(k)≤1, ∑l=1mαl(k)=1, and ∑n=1mβn(k)=1, then system (1) is finite-time stable with respect to (c1,c2,R,T).Proof.
LetV(xk,rk=i)=xkT∑l=1mαl(k)Pilxk. Applying the same procedure as the proof of Theorem 4, we can get (17)EVxk+1,rk+1=j=xkTAiT∑j=1s∑l=1m∑n=1mαlkβnkπijlPilAi+CiT∑j=1s∑l=1m∑n=1mαlkβnkπijlPilCixk.It follows from (15) that E[V(xk+1,rk+1)]<γE[V(xk,rk)]; that is, (18)EVxk,rk<γkEVx0,r0≤γTEVx0,r0.While (19)EVx0,r0=x0T∑l=1mαlkPilx0=x0TR1/2P¯iR1/2x0≤λmaxP¯ix0TRx0≤λmaxP¯ic1,EVxk,rk=ExkT∑l=1mαlkPilxk=ExkTR1/2P¯iR1/2xk≥λminP¯iExkTRxk,so(20)ExkTRxk≤EVxk,rkλminP¯i≤c1γTλmaxP¯iλminP¯i=c1γTCondP¯i<c2.Hence, from Definition 3, system (1) is finite-time stable with respect to (c1,c2,R,T). The proof is completed.Next, we convert the infinite matrix inequalities (15) and (16) into finite matrix inequalities by use of slack matrices and properties of the condition number and design a state feedback controller to make system (2) finite-time stabilizable.Theorem 10.
(1) If there exist a scalarγ≥1, matrices Qil>0, and matrices Gil, such that(21)γGil+GilT-QilGilTAiTΓilGilTCiTΓil∗Q¯n0∗∗Q¯n>0,∀i∈S,l,n=1,2,…,m,(22)c1γTmaxl=1,…,m⁡CondR1/2QilR1/2<c2,∀i∈S,where Γil=πi1lI,…,πislI and Q¯n=diagQ1n,…,Qsn, then system (1) is finite-time stable.
(2) If there exist a scalarγ≥1, matrices Qil>0, and matrices Gi, Ri such that(23)γGi+GiT-QilGiTAiT+RiTBiTΓilGiTCiT+RiTDiTΓil∗Q¯n0∗∗Q¯n>0,∀i∈S,l,n=1,2,…,m,and (22) hold, then system (2) is finite-time stabilized by uk=K(rk)xk with the feedback gains Ki=RiGi-1.Proof.
(1) The proof can be divided into two parts. Firstly, we prove that (15) holds if (21) holds. We adopt the method used in Proposition 2 of [12].
From (21), Gil+(Gil)T-Qil>0. As Qil is positive definite, Gil+(Gil)T must be positive definite. While (1/2)(Gil+(Gil)T)≤Gil, it follows that Gil is invertible. On the other hand, (24)γQil-GilTQil-1Qil-Gil=γQil-1/2Qil-GilTQil-1/2Qil-Gil≥0,which means that γ(Gil)T(Qil)-1Gil≥γGil+(Gil)T-Qil. Then,(25)γGilTQil-1GilGilTAiTΓilGilTCiTΓil∗Q¯n0∗∗Q¯n≥γGil+GilT-QilGilTAiTΓilGilTCiTΓil∗Q¯n0∗∗Q¯n>0,which is equivalent to (26)GilT00∗Q¯n0∗∗Q¯nγQil-1AiTΓilQ¯n-1CiTΓilQ¯n-1∗Q¯n-10∗∗Q¯n-1Gil00∗Q¯n0∗∗Q¯n>0.Because of the invertibility ofGil and Q¯n, we have(27)γQil-1AiTΓilQ¯n-1CiTΓilQ¯n-1∗Q¯n-10∗∗Q¯n-1>0.LetPil=(Qil)-1; then (27) can be rewritten as(28)γPilAiTΓilP¯nCiTΓilP¯n∗P¯n0∗∗P¯n>0,where P¯n=diag(P1n,P2n,…,Psn). By Schur complement, (28) is equivalent to(29)γPil-AiT∑j=1sπijlPjnAi-CiT∑j=1sπijlPjnCi>0.Obviously, (15) holds after multiplying (29) by the corresponding coefficients and adding them.Next, we prove that (16) holds if (22) is true. Before proving this, we need to recall two properties of the condition number:(a)
Cond(cA)=Cond(A)=Cond(A-1), c>0;(b)
Cond(A1+A2)≤max{Cond(A1),Cond(A2)}.It follows from (a) and (b) that(30)Condα1A1+α2A2+⋯+αnAn≤maxi=1,2,…,n⁡CondAi,αi>0.When (22) holds and Pil=(Qil)-1, one has(31)P¯i=∑l=1mαlkR-1/2PilR-1/2=∑l=1mαlkR-1/2Qil-1R-1/2,c1γTCondP¯i≤c1γTmaxl=1,…,m⁡CondR-1/2Qil-1R-1/2=c1γTmaxl=1,…,m⁡CondR1/2QilR1/2<c2.(2) This can be easily proved by the same procedure as(1).Remark 11.
We should point out that unlike the equivalence of (7) and (13), (21) is just a sufficient condition of (15) due to the existence of γ.Remark 12.
In fact, (22) can be guaranteed by the following LMIs:(32)c1γTI<R1/2QilR1/2<c2I,l=1,2,…,m.So we can obtain a set of feasible feedback controls by solving (23) and (32) or by solving (23) and checking if they satisfy (22).


## 5. Numerical Examples

In this section, two illustrative examples are presented to show the effectiveness of the proposed main results.Example 1.
Consider system (2) with coefficient matrices as follows:(33)A1=0.221.52-1.101.37,A2=0.17-0.640.731.05,A3=0.470.110.140.30,B1=0.731.13-0.471.37,B2=-0.021.180.37-0.46,B3=-0.470.110.14-1.30,C1=1.10-0.461.861.23,C2=1.45-0.10-0.490.17,C3=-1.010.200.041.20,D1=1.05-0.240.891.01,D2=1.05-0.110.550.41,D3=-0.510.251.230.21.
The initial state is given asx0=(-0.5,0.5)T. The vertices of the polytope transition probability matrices are given as follows: (34)Γ1=0.30.50.20.50.500.10.40.5,Γ2=0100.10.10.80.30.40.3,Γ3=0.40.20.40.20.10.70.10.30.6.
By solving LMIs (14), it can be concluded that system (2) is stochastically stabilized via feedback gains (35)K1=-1.1058-0.03650.1262-1.2281,K2=-1.2668-0.00510.07470.6577,K3=-0.3565-0.47930.0552-0.0928.
The system modes and the state trajectories of the open-loop and closed-loop systems are, respectively, shown in Figures1–3.Figure 1
The system modes for Example1.Figure 2
The states of open-loop system for Example1.Figure 3
The states of closed-loop system for Example1.Example 2.
Consider system (2) with coefficient matrices as follows: (36)A1=0.22-0.780.560.01,A2=0.27-0.850.230.58,B1=0.07-0.93,B2=0.91-0.18,C1=0.060.45-0.44-0.52,C2=-1.070.06-0.890.46,D1=-0.040.27,D2=0.15-0.11.
The initial state is given asx0=(-2,1)T and the vertices of the polytope transition probability matrices are given as follows:(37)Γ1=0.10.90.50.5,Γ2=0.40.610,Γ3=0.20.80.30.7.
Givenc1=5, c2=70, R=I, and T=15 and choosing γ=1.05, by solving LMIs (23) and checking (22), it can be concluded that system (2) is finite-time stabilized via feedback gains(38)K1=0.55370.6242,K2=-0.10281.1531.
The system modes and the state trajectories of the open-loop and closed-loop systems are, respectively, shown in Figures4–6.Figure 4
The system modes for Example2.Figure 5
E x k T R x k of the open-loop system for Example 2.Figure 6
E x k T R x k of the closed-loop system for Example 2.


## 6. Conclusions

In this paper, we have investigated the stochastic stability and finite-time stability of a kind of discrete-time nonhomogeneous Markov jump systems with multiplicative noises and polytopic transition probability matrices. Some sufficient conditions for stability are proposed by parameter-dependent Lyapunov function and stabilization controllers are designed by using LMI toolbox. The simulation results show the effectiveness of the developed techniques. This research will motivate us to study the continuous-time version in future.



---

*Source: 1586846-2018-02-25.xml*