# Stability Analysis of Networked Control Systems with Random Time Delays and Packet Dropouts Modeled by Markov Chains

Authors: Li Qiu; Fengqi Yao; Xiaopin Zhong
Journal: Journal of Applied Mathematics
 (2013)

---

## Abstract

This paper investigates the stability analysis problem for a class of discrete-time networked control systems (NCSs) with random time delays and packet dropouts based on
unified Markov jump model. The random time delays and packet dropouts existed in
feedback communication link are modeled by two independent Markov chains; the resulting closed-loop system is described by a new Markovian jump linear system (MJLS)
with Markov delays. Sufficient conditions of the stochastic stability for NCSs is obtained
by constructing a novel Lyapunov functional, and the mode-dependent output feedback
controller design method is presented based on linear matrix inequality (LMI) technique.
A numerical example is given to illustrate the effectiveness of the proposed method.


---

## Body

## 1. Introduction

Networked control systems (NCSs) are a type of closed-loop systems, in which the control loops are closed through communication networks. Compared with the traditional control systems, the use of the communication networks bring many advantages such as low cost, reduced weight, and simple installation and maintenance as well as high efficiency, flexibility, and reliability. Consequently, NCSs are applied in a broad range such as manufacturing plants, vehicles, aircrafts, spacecrafts, and remote surgery [1]. However, the communication networks in control loops also present some constraints such as time delays and packet dropouts due to limited bandwidth; quantization errors caused by hybrid nature of NCSs; variable sampling or transmission intervals due to multiple nodes; clock asynchronization among local and remote nodes; network security and safety and network security due to shared communication networks [2, 3]. It is generally known that any of these networked-induced communication imperfections and constraints can degrade closed-loop performance or, even worse, can harm closed-loop stability of NCSs. Therefore, it is important to know how these effects influence the stability properties. Recently, some important results of NCSs have been reported in the existing literature for instance, the discussions of packet dropouts [4–13], time delays [14–24], quantization [25], distributed synchronization [26], communication constraints [27], stability and controller design [28–33], both data quantizations and packet losses [34], both time delays and packet dropouts [35–44], and output feedback control problem [19, 45, 46].In NCSs, time delays and packet dropouts are two important issues. To study these issues, many efforts have been made for NCSs with time delays [14–24] and packet dropouts [4–13]; for more details review, please refer to the literature therein. However, both time delays and packet dropouts exist in NCSs by the insertion of communication network in the feedback control loop. Xie and Xia [35] studied the robust fault tolerant controller design for NCSs with fast varying delay and packet dropout. Zhang and Yu [36] presented a switched system model to describe the NCSs with both delay and packet dropout, and the state feedback stabilizing controllers are designed by augmenting technique. Yu and Shi [37] addressed the two-mode-dependent state feedback controller design in NCSs with time delays and packet dropouts by augmenting the state variable approach. Dong et al. [38] studied the robust H∞ filtering problem for a class of uncertain nonlinear networked systems with both stochastic time-varying communication delays and packet dropouts. Jiang et al. [39] introduced the design of observer-based controller for NCSs with network induced delay and packet dropout. Li et al. [40] considered the observer-based fault detection problem for NCSs with long time delays and packet dropout by modeling the observers system as an uncertain switched system. Li et al. [41] studied the guaranteed const control of NCSs with the S-C packet dropouts and time delays. Wang and Yang [42] considered the problem of H∞ controller design for NCSs with time delay and packet dropout by applying the linear estimation-based time delay and packet dropout compensation method. Liu et al. [43] investigated the receding horizon H∞ control problem for a class NCSs with random delay and packet disordering by using the receding optimization principle. Qiu et al. [44] considered the state feedback control problem of NCS with both time delays and packet dropouts. In [19, 45, 46], the output feedback control problem of NCSs were investigated. To the best of the authors’ knowledge, up to now, little attention has been paid to the study of NCSs with random time delays and packet dropouts based on Markov jump unified model, which motivates our investigation.In this paper, we address the unified model and stability analysis problem of NCSs with the random time delays and packet dropouts under a Markovian jump linear system (MJLS) framework. The feedback communication link random time delays and packet dropouts are modeled by two independent Markov chains, the resulting closed-loop system is modeled as a new MJLS with Markov delays. Then, we give stability analysis and output feedback controller design which are for discrete-time NCSs with both time delays and packet dropouts by the Lyapunov stability theory and linear matrix inequality method.Notations. In the sequel, if not explicit, matrices are assumed to have appropriate dimensions. ℝn and ℝn×m denote, respectively, the n dimensional Euclidean space and the set of all n×m real matrices. The notations A>0 and A<0 are used to denote the positive and negative definite matrix, respectively. diag(A1,…,An) refers to a n×n diagonal matrix with Ai as its ith diagonal entry. I and 0 denote the identity matrix and zero matrix with appropriate dimensions, respectively. The superscript T denotes the transpose for vectors or matrices. 𝔼[·] denotes the mathematical expectation operator. The symbol * denotes blocks that are readily inferred by symmetry.


## 2. Problem Description

The framework of the system over a network medium is depicted in Figure1. Considering the same assumption in [14], the sensor, the controller, and the actuator are time-driven and are connected over a network medium. Under the assumption, it is known that the controller updates at the instant k will always use the most recent data; otherwise, it will maintain the old data. In the NCS as Figure 1, network induced time delays and packet dropouts exist in the feedback communication link.Figure 1
The structure of the NCS with random delays and/or packet dropouts.The discrete-time plant with a time-varying controller is described as(1)xp(k+1)=Axp(k)+Bu(k),y(k)=Cxp(k),
where xp(k)∈ℝn is the system state, u(k)∈ℝm is the control input, and y(k)∈ℝp is measurable output. A, B, and C are known real constant matrices with appropriate dimensions. Random time delays and packet dropouts exist in link from sensor-to-controller (S-C), as shown in Figure 1. Here, τ(k) represents the bounded random S-C time delays. One way to model the delays τ(k) is using the finite state Markov chain as shown in [17–19]. The main advantage of the Markov model is that the dependencies between the delays are taken into account since the current time delays in real networks are usually related with the previous delays [17]. In this paper τ(k) is modeled as a homogeneous Markov chain that take values in S-2={0,1,…,s2}. S denotes the network switches between the S-C. α(k)(α(k)=0,1) denotes the states of S. When S is in state α(k)=0, the packet is received successfully and the y-(k)=y(k-τ(k)). Whereas when S is in state α(k)=1, the packet is lost and the switch output is held at the previous value y-(k)=y-(k-1). The behavior of the S-C time delays and packet dropouts can be modeled as
(2)y-(k)=(1-α(k))y(k-τ(k))+α(k)y-(k-1),
where
(3)α(k)={0,ifSisclosedandthepacketisreceived,1,ifSisopenandthepacketislost.Considering the mode-dependent output feedback controller:(4)u(k)=K(α(k),τ(k))y-(k),
where K(α(k),τ(k)) is the output feedback controller gain.Letx(k)=[xp(k)Ty-(k-1)T]T be the augmented state vector. Under the control (4), the closed-loop system of (1) is
(5)x(k+1)=A-(α(k))x(k)+B-(α(k))Hx(k-τ(k)),x(k)=φ(k),k=-τmax,-τmax+1,…,0,
where A-(α(k))=[Aα(k)BK(α(k),τ(k))0α(k)I], B-(α(k))=[(1-α(k))BK(α(k),τ(k))C(1-α(k))C], H=[I0], τmax=max{τ(k)}, and φ(k) is the initial condition of x(k).In system (5), {α(k),k∈ℤ} and {τ(k),k∈ℤ} are two independent discrete-time homogeneous Markov chains taking value in a finite set S-1={0,1} and S-2={0,1,…,s2} with transition probabilities:
(6)Pr{α(k+1)=j∣α(k)=i}=πij,πi=Pr(π0=i),Pr{τ(k+1)=n∣τ(k)=m}=λmn,λm=Pr(λ0=m),
where πij≥0 and λmn≥0 for all i,j∈S-1, m,n∈S-2 and
(7)∑j=01‍πij=1,∑n=0s2‍λmn=1.
For α(k)=i, i∈S-1, when α(k) in mode i=0 and i=1, the α(k) in (5) take value α(k)=0 and α(k)=1, respectively. A-(α(k)) and B-(α(k)) are known constant matrices of appropriate dimensions.Remark 1.
The closed-loop system (5) is a MJLS with two Markov chains, which describe the behavior of the S-C time delays and packet dropouts, respectively. This enables us to analyze and synthesize such NCSs by applying MJLS theory. Note that modeling the S-C time delays and packet dropouts simultaneously in NCSs based on unified Markov jump model has not been done in the literature.Definition 2 (see [19]).
The system in (5) is stochastically stable if for every finite x0=x(0), initial mode α0=α(0)∈S-1, and τ0=τ(0)∈S-2, there exists a finite 𝒲>0 such that the following holds:
(8)𝔼{∑k=0∞‍∥x(k)∥2∣x0,α0,τ0}<x0T𝒲x0.


## 3. Main Results

By applying a new Lyapunov functional, sufficient conditions for the stochastic stability and synthesis of the mode-dependent output feedback controller design for system (5) will be established in this section.Theorem 3.
For system (5), given random but bounded scalar τ(k)∈[τminτmax], if for each mode i∈S-1, m∈S-2, there exist matrices Pi,m>0, Q1>0, Q2>0, Q3>0, R1>0, and R2>0 such that the following matrix inequalities:
(9)[Ξ1Ξ2Ξ3Ξ4*-P-i,m00**-R10***-R2]<0,
where
(10)Ξ1=[Πi,mHTR100R1H-Q3-2R1-2R2R2R1+R20R2-Q2-R200R1+R20-Q1-R1-R2],Ξ2=[P-i,mA-iP-i,mB-i00]T,Ξ3=[τmaxR1H(A-i-I)τmaxR1HB-i00]T,Ξ4=[τ~R2H(A-i-I)τ~R2HB-i00]T,τ~=τmax-τmin,P-i,m=∑j=01‍∑n=0s2‍πijλmnPj,n,Πi,m=-Pi,m+HT(Q1+Q2)H+(τmax-τmin+1)HTQ3H-HTR1H,A-i=[AiBK(i,m)0iI],B-i=[(1-i)BK(i,m)C(1-i)C],
and H is defined in (5).
Hold for alli,j∈S-1 and m,n∈S-2; then system (5) is stochastically stable.Proof.
For the closed-loop system (5), stochastic Lyapunov functional is constructed as follows:
(11)V(x(k),α(k),τ(k))=∑ρ=14‍Vρ(x(k),α(k),τ(k))=∑ρ=14‍Vρ,
where
(12)V1=x(k)TP(α(k),τ(k))x(k),V2=∑l=k-τmaxk-1‍x(l)THTQ1Hx(l)+∑l=k-τmink-1‍x(l)THTQ2Hx(l),V3=∑θ=-τmax+2-τmin+1‍∑l=k+θ-1k-1‍x(l)THTQ3Hx(l)+∑l=k-τ(k)k-1‍x(l)THTQ3Hx(l),V4=∑θ=-τmax+10‍∑l=k+θ-1k-1‍τmaxδ(l)THTR1Hδ(l)+∑θ=-τmax+1-τmin‍∑l=k+θ-1k-1‍(τmax-τmin)×δ(l)THTR2Hδ(l),
and δ(l)=x(l+1)-x(l). In the following when α(k)=i and τ(k)=m, we will write P(α(k),τ(k)), K(α(k),τ(k)), A-(α(k)) and B-(α(k)) as Pi,m, Ki,m, A-i, and B-i, respectively. We denote:
(13)ΔV(x(k),α(k),τ(k))=∑ρ=14‍ΔVρ=∑ρ=14‍[(x(k),α(k),τ(k))Vρ(α(k),τ(k))x(k+1),α(k+1),τ(k+1)∣x(k),α(k),τ(k))-Vρ(x(k),α(k),τ(k))].
Let ξ(k)=[x(k)T(Hx(k-m))T(Hx(k-τmin))T(Hx(k-τmax))T]T. Then, along the solution of system (5) we have
(14)𝔼[ΔV1]=x(k+1)T[∑j=01‍∑n=0s2‍πijλmnPj,n]×x(k+1)-xT(k)Pi,mx(k)=ξT(k)[A-iTB-iT00]P-i,m[A-iB-i00]ξ(k)-xT(k)Pi,mx(k),
where P-i,m is defined in Theorem 3.
We have(15)𝔼[ΔV2]=xT(k)HT(Q1+Q2)Hx(k)-xT(k-τmax)HTQ1Hx(k-τmax)-xT(k-τmin)HTQ2Hx(k-τmin),(16)𝔼[ΔV3]=(τmax-τmin+1)x(k)THTQ3Hx(k)-∑θ=k-τmax+1k-τmin‍x(l)THTQ3Hx(l)+(∑l=k-n+1k-1‍-∑l=k-m+1k-1‍)x(l)THTQ3Hx(l)-x(k-m)THTQ3Hx(k-m).
Note that
(17)∑l=k-n+1k-1‍xT(l)HTQ3Hx(l)=[∑l=k-τmin+1k-1‍+∑l=k-n+1k-τmin‍]xT(l)HTQ3Hx(l)≤[∑l=k-m+1k-1‍+∑l=k-τmax+1k-τmin‍]xT(l)HTQ3Hx(l).
By combining (16) and (17), we have
(18)𝔼[ΔV3]≤(τmax-τmin+1)x(k)THTQ3Hx(k)-x(k-m)THTQ3Hx(k-m),(19)𝔼[ΔV4]=τmax2δT(k)HTR1Hδ(k)-∑l=k-τmaxk-1‍τmaxδT(l)HTR1Hδ(l)+(τmax-τmin)2δT(k)HTR2Hδ(k)-∑l=k-τmaxk-τmin-1‍(τmax-τmin)δT(l)HTR2Hδ(l).
By Jensen’s inequality, we can get
(20)∑l=k-τmaxk-1‍τmaxδT(l)HTR1Hδ(l)=(∑l=k-τmaxk-τk-1‍+∑l=k-τkk-1‍)(τmax-τk+τk)×δT(l)HTR1Hδ(l)≥(τmax-τk)∑l=k-τmaxk-τk-1‍δT(l)HTR1Hδ(l)+τk∑l=k-τkk-1‍δT(l)HTR1Hδ(l)≥(∑l=k-τmaxk-τk-1‍δ(l))THTR1H(∑l=k-τmaxk-τk-1‍δ(l))+(∑l=k-τkk-1‍δ(l))THTR1H(∑l=k-τkk-1‍δ(l))≥(x(k-τmax)-x(k-τk))THTR1H×(x(k-τmax)-x(k-τk))+(x(k-τk)-x(k))THTR1H(x(k-τk)-x(k))≥ξ(k)T{[HTR1H-HTR100-R1H2R10-R100000-R10R1]}ξ(k).
Similarly, we have
(21)∑l=k-τmaxk-τmin-1‍(τmax-τmin)δT(l)HTR2Hδ(l)=(∑l=k-τmaxk-τk-1‍+∑l=k-τkk-τmin-1‍)×(τmax-τk+τk-τmin)δT(l)HTR2Hδ(l)≥ξ(k)T{[000002R2-R2-R20-R2R200-R20R2]}ξ(k).
By combining (19), (20), and (21), we have
(22)𝔼[ΔV4]≤ξ(k)T{[τmax(H(A-i-I))Tτmax(HB-i)T00]×R1[τmaxH(A-i-I)τmaxHB-i00]+[τ~(H(A-i-I))Tτ~(HB-i)T00]×R2[τ~H(A-i-I)τ~HB-i00]+[-HTR1HHTR100R1H-2R10R100000R10-R1]+[00000-2R2R2R20R2-R200R20-R2][τmax(H(A-i-I))Tτmax(HB-i)T00]}ξ(k),
where τ~ is defined in Theorem 3.
By combining (14), (15), (18), and (22), we have(23)𝔼[ΔV]≤ξT(k){+[τ~(H(A-i-I))Tτ~(HB-i)T00][Πi,mHTR100R1H-Q3-2R1-2R2R2R1+R20R2-Q2-R200R1+R20-Q1-R1-R2]+[A-iTB-iT00]P-i,m[A-iB-i00]+[τmax(H(A-i-I))Tτmax(HB-i)T00]R1[τmaxH(A-i-I)τmaxHB-i00]+[τ~(H(A-i-I))Tτ~(HB-i)T00]R2[τ~H(A-i-I)τ~HB-i00][τmax(H(A-i-I))Tτmax(HB-i)T00]}ξ(k)=ξT(k)Θi,mξ(k),where Πi,m and τ~ are defined in Theorem 3.
By Schur complement and from (9), we have Θi,m<0. Therefore,
(24)𝔼[ΔV]≤-λmin(-Θi,m)ξ(k)Tξ(k)≤-ηx(k)Tx(k),
where λmin(-Θi,m) denotes the minimal eigenvalue of -Θi,m and η=inf{λmin(-Θi,m)}. From (24), it is seen that for any t>0(25)𝔼[V(x(k+1),α(k+1),τ(k+1))]-𝔼[V(φ,α(0),τ(0))]≤-η∑k=0t‍E[x(k)Tx(k)].
Furthermore
(26)∑k=0t‍𝔼[x(k)Tx(k)]≤1η𝔼[V(φ,α(0),τ(0))].
By taking limit as t→∞, we have
(27)∑k=0∞‍𝔼[x(k)Tx(k)]≤1η𝔼[V(φ,α(0),τ(0))]<∞.
According to Definition 2, the closed-loop system (5) is stochastically stable. This completes the proof.Theorem3 gives the sufficient conditions for the stochastic stability of system (5). However, it should be noted that the conditions (9) are no more LMI conditions. To handle this, the equivalent LMI conditions are given in Theorem 4 by Cone Complementarity Linearization (CCL) algorithm.Theorem 4.
Consider system (5) with random but bounded scalar τ(k)∈[τminτmax]. There exists an output feedback controller (4) such the resulting closed-loop system is stochastically stable if for each mode i∈S-1, m∈S-2, there exist matrices Pi,m>0, Xi,m>0, Q1>0, Q2>0, Q3>0, R1>0, R2>0, U1>0, U2>0, and Ki,m such that
(28)[Ξ1Ξ^2Ξ^3Ξ^4*-Xj,n00**-U10***-U2]<0,(29)Pi,mXi,m=I,R1U1=I,R2U2=I,
where(30)Ξ^2=[ℒi,mA-iℒi,mB-i00]T,Ξ^3=[τmaxH(A-i-I)τmaxHB-i00]T,Ξ^4=[τ~H(A-i-I)τ~HB-i00]T,Xj,n=diag{X0,0,X0,1,…,X0,s2,X1,0,X1,1,…,X1,s2},(ℒi,mA-i)T=[πi0λm0A-iTπi0λm1A-iT⋯πi0λms2A-iTπi1λm0A-iT⋯πi1λms2A-iT],(ℒi,mB-i)T=[πi0λm0B-iTπi0λm1B-iT⋯πi0λms2B-iTπi1λm0B-iT⋯πi1λms2B-iT],A-i=[A00iI]+[iB0]Ki,m[0I],A-i-I=[A-I00(i-1)I]+[iB0]Ki,m[0I],B-i=[0(1-i)C]+[(1-i)B0]Ki,mC,H(A-i-I)=[A-I0]+iBKi,m[0I],HB-i=(1-i)BKi,mC.Ξ1 and τ~ are defined in Theorem 3. Moreover, if (28) (29) have solutions, the controller gain is given by Ki,m.Proof.
By Schur complement, (28) is equivalent to
(31)[Ξ1Ξ^2Ξ^3Ξ^4*-Pj,n-100**-R1-10***-R2-1]<0.
Let Pj,n-1=Xj,n, R1-1=U1, and R2-1=U2; we can obtain (28), (29). This completes the proof.The conditions state in Theorem4 are a set of LMIs with some matrix inverse constraints. Although they are nonconvex, which prevents us from solving them using the existing convex optimization tool, we can use the con complementary linearization to algorithm transform this problem into the nonlinear minimization problem with LMI constraints as follows:
(32)minTrace(∑s=12‍RsUs+∑i=01‍∑m=0s2‍Pi,mXi,m)s·t{(i)LMI(28)(ii)[RsIIUs]>0,s∈{1,2},(iii)[Pi,mIIXi,m]>0,i∈S-1,m∈S-2.The above nonlinear minimization problem can be solved by an iterative algorithm presented in the following.Algorithm 5.
Step
1. Find a feasible solution satisfying LMIs (i), (ii), and (iii) in (32); set as (R10,U10,R20,U20,Pi,m0,Xi,m0,K0) and k=0.
Step
2. Solve the following LMI optimization problem for variables (R1,U1,R2,U2,Pi,m,Xi,m,K). Minimize trace {∑s=12(RskUs+RsUsk)+∑i=01∑m=0s2(Pi,mkXi,m+Pi,mXi,mk)}, subject to LMIs (32). Set R1k+1=R1, U1k+1=U1, R2k+1=R2, U2k+1=U2, Pi,mk+1=Pi,m, Xi,mk+1=Xi,m, and Kk+1=K.
Step
3. If (31) is satisfied, then exit the iteration. If (31) is not satisfied, let k=k+1, and then return to Step 2.


## 4. Numerical Example

To illustrate the effectiveness of the proposed method, we apply the results in Section3 to a classical angular positioning system [43] in Figure 2, where θ is the angular position of the antenna, θr is the angular position of the moving object, and the angular velocity of the antenna θ˙ is measurable. The control problem is to use the input voltage to the motor to rotate the antenna so that it always points in the direction of a moving object in the plant. The output feedback controller is designed for the following values of the matrices A, B, and C:
(33)A=[10.099500.99],B=[0.00390.0783],C=[1.40.8-0.20.4].Figure 2
The angular positioning system.The stochastic jumping parameterα(k)∈{0,1} and the random delays involved in system (5) are τ(k)∈{0,1,2}; the transition probability matrices π and λ are taken by
(34)π=[0.40.60.550.45],λ=[0.360.540.10.260.520.220.180.620.2].Figures3 and 4 show part of the simulation of the stochastic jumping parameter α(k) and S-C delay τ(k) governed by their corresponding transition probability matrices, respectively.Figure 3
Values ofα(k).Figure 4
S-C random delaysτ(k).The initial valuex(0)=[-0.40.6]T. By Theorem 4, we can obtain the gain matrices Ki,m of controller (4) which are constructed as
(35)K0,0=[-1.1356-1.6672],K0,1=[-0.2870-0.4926],K0,2=[-0.2898-0.4909],K1,0=[-0.2886-0.4900],K1,1=[-0.2857-0.4909],K1,2=[-0.2829-0.4805].
The state trajectories and the delay output trajectories are shown in Figures 5 and 6, where four curves represent state trajectories and the delay output trajectories under the controller gains Ki,m. Figures 5 and 6 indicate that system (5) is stochastically stable. In contrast with the proposed method, the controller gain Kdlqr of a standard linear-quadratic regulator for nominal discrete-time systems designed by MATLAB command dlqr is
(36)Kdlqr=[0.93321.6804].
The eigenvalues of A+BKlqr are 1.1686 and 0.9567. Hence, Klqr cannot stabilize the system in this case. The proposed controller works much better for networked control system than the contrastive dlqr method.Figure 5
State trajectories underKi,m.Figure 6
The trajectories of delay outputy-(k).


## 5. Conclusion

The stability analysis problem for NCSs with random time delays and packet dropouts is investigated in this paper. The random time delays and packet dropouts existed in feedback communication link are modeled by two independent Markov chains. Then the resulting closed-loop system is modeled as a MJLS with Markov delays. Sufficient conditions on stochastic stability and stabilization are obtained by the Lyapunov stability theory and LMI method. The CCL algorithm is employed to obtain the mode-dependent output feedback controller. Finally, an example is presented to illustrate the effectiveness of the approach. Although the NCSs with random time delays and packet dropouts on only sensor to controller link are considered in this paper, the method of unified modeling and the Lyapunov functional constructing can be extended to the NCSs with the random time delays and packet dropouts existing in both the sensor to controller and controller to actuator.



---

*Source: 715072-2013-09-16.xml*