# Neural Network Based Finite-Time Stabilization for Discrete-Time Markov Jump Nonlinear Systems with Time Delays

Authors: Fei Chen; Fei Liu; Hamid Reza Karimi
Journal: Abstract and Applied Analysis
 (2013)

---

## Abstract

This paper deals with the finite-time stabilization problem for discrete-time Markov jump nonlinear systems with time delays and norm-bounded exogenous disturbance. The nonlinearities in different jump modes are parameterized by neural networks. Subsequently, a linear difference inclusion state space representation for a class of neural networks is established. Based on this, sufficient conditions are derived in terms of linear matrix inequalities to guarantee stochastic finite-time boundedness and stochastic finite-time stabilization of the closed-loop system. A numerical example is illustrated to verify the efficiency of the proposed technique.


---

## Body

## 1. Introduction

Markov jump systems (MJSs) are an important class of stochastic dynamic systems, which are popular when modeling an abrupt change in the system structure and parameters, such as component failures or repairs, changing subsystem interconnections and environmental disturbance. This family of systems has great practical potential in a variety of fields, such as solar thermal central receivers systems, economic systems, communication systems, manufacturing systems, and networked control systems [1–4]. MJSs have been extensively studied since the pioneering work on quadratic control of MJSs [5], and many achievements have been made on Lyapunov stochastic stability and stabilization in the last three decades [6–18].However, it is worth noting that the Lyapunov stochastically stable systems may not possess good or expected transient characteristics over a finite-time horizon. In many practical problems, it is of interest to investigate the stability of a system over a finite interval of time. For example, referring to aircraft control, it requests that, during the execution of a certain task, the state variables should not exceed some threshold under all admissible pilot inputs and in the presence of wind disturbances. Classical control theory does not directly address this requirement, because it focuses mainly on the asymptotic behavior of the system (over an infinite-time interval) and does not usually specify bounds on the trajectories. Therefore, it is necessary to limit the state in an acceptable region and consider finite-time stability (FTS) given by Dorato [19].The concept of FTS has been further extended into finite-time boundness (FTB) [20, 21], when system possesses bounded exogenous disturbance. A linear matrix inequality (LMI) framework has been established to distinguish FTS and Lyapunov asymptotical stability [22–24]. Compared with Lyapunov stochastically stable condition, FTS relaxes the condition by allowing that the Lyapunov-like function can increase at every sampling time instant. That is why FTS is so attractive and widely used in practical engineering.As MJSs are considered, a number of results on stochastic FTS or stochastic FTB have been developed [25–28], and recently, the obtained results have been extended to continuous-time MJSs with nonlinearities via fuzzy or neural network approach [24, 29, 30]. In order to make the stochastic systems more manageable and satisfy the requirements for finite-time behavior of a system in engineering fields, it motivates us to investigate the finite-time stability and stabilization problems for a class of MJSs. Furthermore, time delay is a common phenomenon and is inevitable in practice systems [31–33]. Due to the interaction among system dynamics, stochastic jumps, and time delays, the dynamics of MJSs with time delay become more complex than MJSs without time delay and time delay systems without jumps. So far, in comparison with the literatures available for continuous-time nonlinear MJSs with time delays, the corresponding FTS or FTB results for discrete-time nonlinear systems have been relatively few.It is, therefore, the main purpose of this paper to shorten such a gap by investigating the finite-time stabilization problem for discrete-time nonlinear MJSs with time delays. With neural networks, the nonlinearities of MJSs are approximated firstly by linear difference inclusion under state-space representation. Then, a mode-dependent finite-time controller is developed to make the nonlinear MJSs stochastic finite-time stabilizable for all admissible approximation errors of the neural networks and the norm-bounded external disturbances. The controller gains could be derived by solving a set of LMIs. An attractive feature of the proposed scheme is that the coupling relationship between time delay and given finite-time horizon is explored by obtaining delay-independent conditions.Notations in this paper are fairly standard.Rn and Rn×m denote n-dimensional Euclidean space and the set of all the n×m real matrices, respectively; AT (or xT) and A-1 denote the transpose of the matrix A (or the vector x) and the inverse of the matrix A, respectively. λmax(A) and λmin(A) denote, respectively, the maximal and minimal eigenvalues of a real matrix A, ∥A∥ denotes the Euclidean norm of matrix A, E{·} denotes the mathematics statistical expectation of the stochastic process or vector, l2[0N) is the space of summable infinite sequence over [0N), P>0 stands for a positive-definite matrix, I is the unit matrix with appropriate dimensions, and “*” means the symmetric terms in a symmetric matrix.


## 2. System Description and Problem Formulation

We consider a nonlinear discrete-time MJS, which can be described by the following mathematical model:(1)xk+1=A(rk)xk+Ad(rk)xk-d+B(rk)uk+Bw(rk)wk+C(rk)f(xk,rk),xf=φf,f∈{-d,…,0},r(0)=r0,
where xk∈Rn is the vector of state variables, uk∈Rm is the controlled input, f(·) is a discrete nonlinear mapping with f(0)=0 but not assumed to be known a prior, and wk∈l2q[0+∞) is the exogenous disturbances satisfying
(2)∥w∥22=E[∑k=0NwkTwk]<δ2.
For each possible value of rk=i, we denote
(3)A(rk)=Ai,Ad(rk)=Adi,B(rk)=Bi,Bw(rk)=Bwi,C(rk)=Ci,f(xk,rk)=fi(xk),
where rk is a discrete-state Markov chain taking values in M={1,2,…,s} with transition probabilities
(4)Prob{rk+1=j∣rk=i}=πij,
where πij is the transition probabilities from mode i to mode j that satisfies
(5)πij≥0,∑j=1mπij=1,∀i,j∈M.
For each mode i, nonlinear function fi(xk) is to be parameterized by neural networks. Such parameterization makes sense because any nonlinear function can be approximated arbitrarily well on a compact interval by a neural network. Let the L-layered perceptrons Ni(xk,Wr1,Wr2,…,WrL) be suitably trained to approximate the nonlinear term fi(xk), which is described in matrix-vector notation as
(6)Ni(xk,Wi1,Wi2,…,WiL)=ψiL[WiL⋯ψi2[Wi2[ψi1[Wi1x(t)]]]],
where all the weight matrices Wir∈Rnir×ni(r-1), r=1,…,L, from the (r-1)th layer to the (r-L)th layer will be determined via back propagation (BP) procedure [24]; the activation function vector of rth layer is defined as ψir[ζ]=[ϕi1(ςi1),ϕi2(ςi2),…,ϕinr(ςinr)]T, where nr indicates the neurons of rth layer and let
(7)ϕil(ςil)=λil(1-e-ςil/qil1+e-ςil/qil),qil,λil>0,l=1,2,…,nr.
The maximum and minimum derivatives of activation function ϕil are defined as follows:
(8)sil(m,ϕil)={minζil∂ϕil(ζil)∂ζil,m=0,maxζil∂ϕil(ζil)∂ζil,m=1.
For rth layer of neural network, activation function ϕil can be rewritten as the following min-max form:
(9)ϕil=hil(0)sil(0,ϕil)+hil(1)sil(1,ϕil),
where hil(m), m=0,1, are a set of positive real numbers associated with ϕil satisfying hil(m)>0 and hil(0)+hil(1)=1.According to the approximation theorem, for given accuracyρi>0, there exist ideal constant weight matrices Wir* defined as
(10)(Wi1*,Wi2*,…,WiL*)=argmin(Wi1*,Wi2*,…,WiL*){maxxk∈D∥fi(xk)-Ni(xk,Wi1,Wi2,…,WiL)∥},
where D is a compact set D∈Rm, such that
(11)maxxk∈D∥fi(xk)-Ni(xk,Wi1*,Wi2*,…,WiL*)∥≤ρi∥xk∥.
For each mode i, denote a set of nr dimensional index vectors of the rth layer as
(12)γnr=γnr(σi)={σi∈Rnr∣σil∈{0,1},l=1,…,nr},
where σi is used as a binary indicator. Obviously, the rth layer with nr neurons has 2nr combinations of binary indicator with m=0,1, and the elements of index vectors for all L layers neural network have 2nL×⋯×2n2×2n1 combinations in the set
(13)Θ=γnL⊕⋯⊕γn2⊕γn1.
By using (8) and adopting the compact representation [34], the multilayer neural network (6) can be expressed as follows:(14)N(xx,Wi1*,Wi2*,…,WiL*)=ψiL[WiL*⋯ψi2[Wi2*[∑m=01hi11(k)si11(m,ϕi11)×(Wi1*x)i1⋮∑m=01hi1n1(k)si1n1(m,ϕi1n1)×(Wi1*x)in1]]⋯]=∑σi∈ΘμσiAσi(σi,ψi,Wi*)xk,where(15)Aσi=diag[sLil(σLil,ϕLil)]WL*⋯diag[s2il(σ2il,ϕ2il)]×W2*diag[s1il(σ1il,ϕ1il)]W1*,∑σi∈Θμσi=∑miLnL=0⋮miL1=01⋯∑mi1n1=0⋮mi11=01hiLnL(miLnL)⋯hiL1(miL1)⋯hi1n1(mi1n1)⋯hi11(mr11)=1.
Thus by means of multilayer neural network, the nonlinear MJS (1) is translated into a group of LDIs with error bounds, in which the different inclusion is powered by stochastic Markov process; that is,
(16)xk+1=[∑σi∈ΘμσiAσi+Ai]xk+Adixk-d+Biuk+Bwiwk+CiΔfi(xk),xf=φf,f∈{-d,…,0},r(0)=r0,
where
(17)Δfi(xk)=maxxk∈D∥fi(xk)-Ni(xk,Wi1*,Wi2*,…,WiL*)∥≤ρi∥xk∥
denotes the approximation errors of networks.Remark 1.
The detailed structure and quantitative size of error dynamicsΔfi(xk) are not needed, but only norm-bounded assumption is required. This condition is easily satisfied in practical cases, such as bioinformatics system, medical diagnosis, fault diagnosis, and image and pattern recognition. Actually, the approximation error between the target function and the closest neural network function of a given network family can be made as small as desired by increasing the number of nodes [35]. Also the bounds of norm may vary according to different nonlinearities in different modes.


## 3. Main Results

Based on the LDI model (16) of networks, we consider the following discrete-time state feedback control law for nonlinear stochastic MJS (1):
(18)uk=Kixk.
The resulting closed-loop system can be obtained as follows:
(19)xk+1=A-ixk+Ad(rk)xk-d+Bw(rk)wk+Δfi(xk),xf=φf,f∈{-d,…,0},r(0)=r0,
where
(20)A-i=∑σi∈ΘμσiAσi+Ai+BiKi.
The aim of this paper is to find some sufficient conditions which guarantee stochastic finite-time boundness and stochastic finite-time stabilization of the closed-loop system (19). The general idea of finite-time control can be formalized through the following definitions over a finite-time interval for some given initial conditions.Definition 2 (stochastic finite-time stability).
A discrete-time nonlinear MJS (1) (setting uk=0 and wk=0) is said to be, stochastic finite-time stability (FTS) with respect to given (c1,c2,G,N), where c2>c1 and G>0, if E{xkTGxk}<c22, k∈{1,2,…,N} whenever maxk0-d≤k≤k0E{x0TGx0}≤c12.Definition 3 (stochastic finite-time boundness).
A discrete-time nonlinear MJS (1) (setting uk=0) is said to be of stochastic finite-time boundness (FTB) with respect to (c1,c2,G,N,δ) with c2>c1 and G>0, if E{xkTGxk}<c22,k∈{1,2,…,N}, whenever maxk0-d≤k≤k0E{x0TGx0}≤c12.Before proceeding further, we introduce the following lemmas which will be needed for the derivation of our main results.Lemma 4.
The closed-loop system (19) is stochastic FTB with respect to the given (c1,c2,G,N,δ) and scalar α≥0, if there exist mode-dependent symmetric positive-definite matrix Pi and symmetric positive-definite matrices Q and S such that
(21)[A-iTP-jA-i-(1+α)Pi+QA-iTP-jAdiA-iTP-jBwiA-iTP-jCi*AdiTP-jAdi-QAdiTP-jBwiAdiTP-jCi**BwiTP-jBwi-(1+α)SBwiTP-jCi***CiTP-jCi-(1+α)R]<0,(22)c12maxi∈M{λmax(P~i)}+c12dλmax(Q~)+δ2λmax(S)+c22ρi2λmax(R~)<c22mini∈M{λmin(P~i)}(1+α)N,
where P~i=G-1/2PiG-1/2, Q~=G-1/2QG-1/2, R~=G-1/2RG-1/2, and λmax(·), λmin(·) indicate the maximal and minimal eigenvalues of the augment, respectively.Proof.
For the closed-loop system (19), choose a stochastic Lyapunov function candidate as
(23)Vi(k)=xkTPixk+∑f=k-dk-1xfTQxf.
Simple calculation shows that
(24)E{Vi(k+1)}-Vi(k)=xkT(A-iTP-jA-i-Pi+Q)xk+2xkTA-iTP-jAdixk-d+2xkTA-iTP-jBwiwk+2xkTA-iTP-jCiΔfi(xk)+xk-dT(AdiTP-jAdi-Q)xk-d+2xk-dTAdiTP-jBwiwk+2xk-dTAdiTP-jCiΔfi(xk)+wkΤBwiTP-jBwiwk+2wkTBwiTP-jCiΔfi(xk)+ΔfiT(xk)CiTP-jCiΔfi(xk)=ζkTΩiζk,
where
(25)P-j≜∑j=1sπijPj,ζk=[xkTxk-dTwkTΔfiT(xk)]T,Ωi=[A-iTP-jA-i-Pi+Q***AdiTP-jA-iAdiTP-jAdi-Q**BwiTP-jA-iBwiTP-jAdiBwiTP-jBwi*CiTP-jA-iCiTP-jAdiCiTP-jBwiCiTP-jCi].
Conditions (21) and (24) imply that
(26)E{Vi(k+1)}≤(1+α)xkTPixk+(1+α)wkTSwk+(1+α)ΔfiT(xk)RΔfi(xk)+(1+α)∑f=k-dk-1xfTQxf=(1+α)Vi(k)+(1+α)wkTSwk+(1+α)ΔfiT(xk)RΔfi(xk).
Noting that α≥0, we can obtain from (26) that
(27)Vi(k)≤(1+α)kVi(0)+∑f=1k(1+α)k-f+1wf-1TSwf-1+∑f=1k(1+α)k-f+1c22ρi2λmax(R~)=(1+α)k[x0TPix0+∑f=-d-1xfTQxf+∑f=1k(1+α)1-fwf-1TSwf-1+∑f=1k(1+α)1-fc22ρi2λmax(R~)]≤(1+α)N[c12maxi∈M{λmax(P~i)}+c12dλmax(Q~)[c12maxi∈M{λmax(P~i)}+c12dλmax(Q~)+δ2λmax(S)+c22ρi2λmax(R~)].
Note that
(28)Vi(k)=xkTPixk+∑f=k-dk-1xfTQxf≥xkTPixk≥mini∈M{λmin(P~i)}xkTGxk.
According to (27)-(28), one has
(29)xkTGxk≤((1+α)N(c12maxi∈M{λmax(P~i)}+c12dλmax(Q~)+δ2λmax(S)+c22ρi2λmax(R~)c12maxi∈M)(1+α)N(c12maxi∈M{λmax(P~i)}+c12dλmax(Q~))×(mini∈M{λmin(P~i)})-1.
Condition (19) implies that, for k∈{1,2,…,N}, E{xkTGxk}<c22. This completes the proof.Now, we direct our attention to present a solution to the problem of finite-time stabilizing controller design. Such controller is provided by the following theorem.Theorem 5.
The closed-loop system (19) is stochastic finite-time stabilizable via state feedback with respect to the given (c1,c2,G,N,δ) and scalar α≥0, if there exist matrices Xi=XiT>0, Yi,H=HT>0, S=ST>0, and R=RT>0 and scalars λ1,λ2,λ3,λ4>0 such that
(30)[-(1+α)XiN1iT00XiN1i-M5i+N5iM3iM4i00M3iT-(1+α)S000M4iT0-(1+α)R0Xi000-H]<0,(31)λ1G-1<Xi<G-1,λ2G-1<H,S<λ3I,R<λ4G,(32)[-c22(1+α)N+δ2λ3+c22ρi2λ4c1dc1c1-λ10dc10-λ2]<0.Proof.
By using Schur complement, from condition (21) in Lemma 4, it follows that
(33)[-(1+α)Pi+Q****0-Q***00-(1+α)S**000-(1+α)R*M1iM2iM3iM4i-M5i]≤0,
where
(34)M1i=[πi1A-iT,…,πisA-iT]T,M2i=[πi1AdiT,…,πisAdiT]T,M3i=[πi1BwiT,…,πisBwiT]T,M4i=[πi1CiT,…,πisCiT]T,M5i=diag{P1-1,…,Ps-1}.
Performing matrix elementary transformation to the above inequality, we have
(35)[-(1+α)Pi+QM1iT000M1i-M5iM3iM4iM2i0M3iT-(1+α)S000M4iT0-(1+α)R00M2iT00-Q]≤0.
Performing a congruence to the above condition by diag{Pi-1IIII}, using Schur complement, and letting Xi=Pi-1 and Yi=KiXi, we get
(36)[-(1+α)Xi+XiQXiN1iT00N1i-M5i+N5iM3iM4i0M3iT-(1+α)S00M4iT0-(1+α)R]≤0,
where
(37)N1i=[πi1(A~iXi+BiYi)T,…,πis(A~iXi+BiYi)T]T,N5i=[πi1AdiHAdiTπi1πi2AdiHAdiT⋯πi1πisAdiHAdiTπi2πi1AdiHAdiTπi2AdiHAdiT⋯πi2πisAdiHAdiT⋮⋮⋱⋮πisπi1AdiHAdiTπisπi2AdiHAdiTπisAdiHAdiT],A~i=∑σi∈ΘμσiAσi+Ai.
By using Schur complement to (36) and letting H=Q-1, we obtain the linear matrix inequality (30) in Theorem 5.
On the other hand, we consider(38)λmax(X~i)=1λmin(P~i),X~i=P~i-1=G1/2XiG1/2.
Condition (22) follows that
(39)c12mini∈M{λmin(X~i)}+c12dλmax(Q~)+δ2λmax(S)<c22maxi∈M{λmax(X~i)}(1+α)N.
It is easy to check that the above inequality is guaranteed by imposing the following conditions
(40)λmax(X~i)<1,λ1<λmin(X~i),λmax(Q~)<λ2,λmax(S)<λ3,c12λ1+c12dλ2+δ2λ3<c22(1+α)N
which are equivalent to conditions (31)-(32). This completes the proof.Remark 6.
It is worth pointing out that Theorem5 is not a delay-dependent sufficient criterion, which is conservative when the delay is small. Delay-dependent result can be developed in the same way by choosing a Lyapunov functional that includes more entries, as was done in [36], or delay fractioning approach that can be employed as was done in [31–33].Remark 7.
The coupling relationship between time delay and given finite-time horizon of the underlying system is obtained through a finite-time stable constraint (32) in Theorem 5. From condition (32), it can be seen that, in given finite-time horizon, if the time delay d is larger, constraint (32) is more difficult to be satisfied, which means that the existence of time delay increases the instability of system.


## 4. Numerical Example

Consider discrete-time Markov jump nonlinear system (1) with three operation modes and the following data:
(41)A1=[0.88-0.050.40-0.72],Ad1=[-0.20.10.20.15],B1=[21],Bw1=[0.40.5],C1=[00.1],A2=[20.240.800.32],Ad2=[-0.60.40.20.6],B2=[1-1],Bw2=[0.20.6],C2=[00.3],A3=[-0.80.160.800.64],Ad3=[-0.30.10.20.5],B3=[11],Bw2=[0.10.3],C3=[00.5],f1(xk)=f2(xk)=f3(xk)=sin(x1k)cos(x2k).
Now, a single hidden layer neural network with 2 hidden neurons was chosen to approximate the nonlinear functions fi(xk). All parameters of activation functions associated with the hidden layer were chosen to be qil=0.5 and λil=1. For these activation functions, we have sil(0,ϕil)=0 and sil(1,ϕil)=1. The connection weights are trained offline by using BP algorithm. The initial weights and state vector are placed by uniformly distributed random numbers in [-11]. After 1000 training steps, the optimal approximation weights are as follows:
(42)W1*=[-0.86017-0.81881-0.950250.96405],W2*=[-0.57752-0.58342].
The upper bound of approximation error is estimated as ρi=0.022. Obviously, in this case, we have Θ=22×21. According to (15), Aσi can be obtained as follows:
(43)Ai1=Ai2=Ai3=Ai4=Ai5=A1⊕[0,0,0]T=A0⊕[i,j,k]T[0000],(i,j,k∈{0,1}),Ai6=A1⊕[1,0]T=[000.496770.47288],Ai7=A1⊕[0,1]T=[000.55439-0.56245],Ai8=A1⊕[1,1]T=[001.0512-0.089567].
The initial state and initial mode are taken as x0=[-0.30.4]T and r0=1, respectively. The iterative step is taken as N=7. The mode path from time step 0 to time step 7 is generated randomly and it is shown in Figure 1. Let c1=0.5, c2=2, N=7, G=I, d=0.5, α=0.5, and δ2=1. By solving the matrix inequalities in Theorem 5, we have the following controller gains:
(44)K1=[-0.9304-0.0683],K2=[-1.72310.3654],K3=[1.1486-0.1588].
The state trajectories of the free and controlled MJLS (16) are drawn in Figures 2 and 3, respectively. It could be seen that the free MJLS (16) is not stochastic FTB because the trajectory exceeds the given bound c22. However, the trajectory is limited between the two ellipsoids regions by employing the proposed control move which satisfactorily justify that the closed-loop MJLS (16) is stochastic FTB.Figure 1
Jump modes.Figure 2
State trajectory of free MJS.Figure 3
State trajectory under finite-time control.It should be pointed out that, in the simulation example, as long as the choice of initial condition is satisfied with∥x0TRrx0∥≤c1, then the system is robustly finite-time stabilizable; that is, system trajectories stay within a given bound.


## 5. Conclusions

The finite-time stabilization problem for discrete-time Markovian jump nonlinear system with time delay and norm-bounded exogenous disturbance is investigated in this paper. The nonlinearities are parameterized by multilayer neural network and the relationship between time delay and given finite-time horizon is explored with delay-independent conditions. The proposed framework is versatile and can accommodate a number of challenging design problems including finite-time control and filtering of discrete-time or continuous-time nonlinear MJS with parameter uncertainties, time delays, and so on. The future work can consider some delay-dependent approaches or delay fractioning approaches to reduce the conservativeness introduced by time delay.



---

*Source: 359265-2013-09-30.xml*