# H ∞ Control for Nonlinear Infinite Markov Jump Systems

Authors: Yueying Liu; Ting Hou
Journal: Mathematical Problems in Engineering
 (2018)

---

## Abstract

In this paper, we discuss the infinite horizonH∞ control problem for a class of nonlinear stochastic systems with state, control, and disturbance dependent noise. The jumping parameters are modelled as an infinite-state Markov chain. Based on the solvability of a set of coupled Hamilton-Jacobi inequalities (HJIs), the exponential mean square H∞ controller for the considered nonlinear stochastic systems is obtained. A numerical example is given to show the effectiveness of the proposed design method.


---

## Body

## 1. Introduction

During the past decades, as one of the most important robust control design,H∞ control has been extensively studied in both theory and practical applications [1]. From the time-domain viewpoint, H∞ control is to find a control law to eliminate the effect of external disturbance below a given level [2]. Due to the ability to model many real plants in practice, stochastic systems has gained much attention. In particular, stochastic H∞ control was firstly investigated in [3] for Ito^ systems, where a stochastic bounded real lemma was established in the form of linear matrix inequalities. References [4, 5], respectively, studied H∞ filtering and control for nonlinear stochastic systems via solving second-order nonlinear HJIs.Stochastic systems with Markov jumps are powerful tool to describe physical systems which may encounter abrupt changes in their dynamics. In the theoretical study of stochastic Markov jump systems, stability and observability [6–11] and robust control [12–15] have been widely investigated. Recently, stable and control problems for nonlinear systems have become a hot research topic [16–22]. It should be pointed out that most of the aforementioned researches on Markov jump systems assume that Markov chain takes values in a finite set. However, Markov jump systems with infinite-state chains can be used to describe more plants in many real scenarios [23, 24]. Therefore, infinite Markov jump systems deserve our consideration. Recently, some papers on stability [25–27] and control problems [28, 29] of linear infinite Markov jump systems have appeared. To be specific, infinite horizon H2/H∞ controller has been obtained by four coupled algebraic Riccati equations in [30]. Nevertheless, to the best of our knowledge, H∞ control problem for a class of nonlinear stochastic systems with infinite Markov jumps is still unsolved, let alone the case of (x,u,v)-dependent noise. This situation motivates us to carry out the present research.This paper is concerned with the infinite horizonH∞ control problem for a class of nonlinear stochastic systems with infinite Markov jumps and (x,u,v)-dependent noise. The rest of the paper is organized as follows. Section 2 provides some useful definitions and lemmas. In Section 3, based on the generalized Ito^-type formula and the technique of squares completion, an exponential mean square stable H∞ controller is designed in terms of a set of coupled HJIs. And a numerical example is provided to illustrate the applicability of the proposed design approach. Conclusions are made in Section 4.Next, we adopt the following notations.R denotes the set of all real numbers and R+ is the set of all nonnegative real numbers. Rn and Rm×n stand for n-dimensional real vector space and the vector space of all m×n matrices, respectively. For a matrix A, A′ represents the transpose and we denote A≥0 (A>0) the positive semidefinite (definite) symmetric matrix. Also, we make use of the notation of Sn and In for the set of all n×n symmetric and identity matrices, respectively. The operator norm of Rm×n or the Euclidean norm of Rn is ·. By l2(R+;Rm) we define the space of Rm-valued, square integrable, and Ft-measurable processes ζ={ζ(t,w):R+×Ω→Rm} satisfying ζ2=E∫0∞ζt2dt<∞. The class of functions V(x) which are twice continuously differential with respect to x∈U, except possibly at the point x=0, will be denoted by C2(U). D:={1,2,…}.


## 2. Preliminaries

Consider the following stochastic nonlinear system with infinite Markov jumps:(1)dxt=f1xt,ηt+g1xt,ηtut+h1xt,ηtvtdt+f2xt,ηt+g2xt,ηtut+h2xt,ηtvtdwt,zt=mxt,ηtut,x0=x0∈Rn,t∈R+,where x(t)∈Rn, v(t)∈Rnv, u(t)∈Rnu, and z(t)∈Rnz stand for the system state, exogenous disturbance, control input, and measurement output, respectively. w(t) is a standard one-dimensional Brownian motion on a probability space (Ω,F,P). Assume that Ft:=σ(w(s),0≤s≤t)∨σ(η(s),0≤s≤t)∨N, where N denotes the totality of P-null sets and the σ-algebras σ(w(s),0≤s≤t) and σ(η(s),0≤s≤t) are mutually independent. We denote {ηt}t∈R+ the right continuous, homogeneous Markov process on Ω taking values in the countably infinite state space D with generator Q=(qij)i,j∈D given by (2)Pηt+s=j∣ηt=i=qijs+os,i≠j,1+qiis+os,i=j,where s>0, lims→0(o(s)/s)=0, qij≥0(i,j∈D,i≠j) is the transition rate from mode i at time t to mode j at time t+s and qii=-∑j∈D,j≠iqij<∞ for all i∈D. Suppose that f1,g1,h1,f2,g2,h2, and m satisfy the local Lipschitz condition and the linear growth condition for any i∈D, which guarantee that system (1) has a unique strong solution [13, 31]. Moreover, assume f1(0,i)=0,f2(0,i)=0,i∈D.DenoteE1m×n the Banach space of all sequences {E|E=(E(1),E(2),⋯),E(i)∈Rm×n} with the norm E1=∑i=1∞E(i)<∞. Likewise, define another Banach space E∞m×n with the norm E∞=supi∈DE(i). Assume all coefficients of considered systems have a finite norm ·∞. If m=n, E1m×n will be simplified as E1n and so does E∞m×n. When E(i)∈Sn and E(i)≥0,i∈D, E1n(E∞n) is written as E1n+(resp., E∞n+). For L,M∈E1n+, L≤M implies that L(i)≤M(i),i∈D. Therefore, we have L1≤M1.For eachV∈C2(Rn×D;R), an infinitesimal operator LV:Rn×D⟶R associated with system (1) is defined as follows [28, 31]:(3)LVxt,i=∂Vxt,i′∂xf1xt,i+g1xt,iut+h1xt,ivt+∑j=1∞qijVxt,j+12tracef2xt,i+g2xt,iut+h2xt,ivt′∂2Vxt,i∂x2·f2xt,i+g2xt,iut+h2xt,ivt,i∈D.To study the infinite horizon nonlinear stochasticH∞ control, the internal stability requirement is needed; thus we introduce the following definition.Definition 1 (see [13]).
The unforced stochastic system with infinite Markov jumps,(4)dxt=f1xt,ηtdt+f2xt,ηtdwt,t∈R+, is called exponentially mean square stable (EMSS) if there exist α>0 and β≥1 such that (5)Ext2≤βe-αtx02for all t∈R+,i∈D and x0∈Rn.Definition 2.
For a givenγ>0, the control u∗(t)∈l2(R+;Rnu) is said to be an infinite horizon H∞ control of system (1), if (i)
u∗(t) stabilizes system (1) internally; i.e. when v(t)=0, u(t)=u∗(t), the trajectory of system (1) with any initial value x(0)=x0 is EMSS(ii)
For any nonzerov∗(t)∈l2(R+;Rnv) and zero initial state x0=0, we always have(6)ztl2R+;Rnu≤γvtl2R+;Rnv.Remark 3.
If (6) holds, it is easy to verify that (6) is equivalent to L∞u∗≤γ, where the perturbation operator L∞u∗ is defined by L∞u∗:l2(R+;Rnv)→l2(R+;Rnz) subject to system (1) with (7)L∞u∗≔supv·∈l2R+;Rnv,v≠0η0∈D,x0=0⁡ztl2R+;Rnzvtl2R+;Rnv.We provide some lemmas which are absolutely necessary to derive our main results.Lemma 4 (see [32]).
Forx,b∈Rn, A∈Sn, A-1 exists, we have(8)x′Ax+x′b+b′x=x+A-1b′Ax+A-1b-b′A-1b.The following lemma generalizes Theorem 5.8 [31] and Corollary 3.2.3 [13] to the infinite Markov jump and nonlinear systems, respectively. Its proof can be easily shown by analogous arguments.Lemma 5.
Assume that there are a set of positive functionsV(x,ηt)∈C2(Rn×D;R+) and positive constants c1,c2,c3 such that(9)c1x2≤Vx,i≤c2x2and(10)LVx,i≤-c3x2for all x∈Rn and i∈D. Then system (4) is EMSS.


## 3. Infinite Horizon Nonlinear StochasticH∞ Control

In this subsection, we attempt to obtain the sufficient condition for the infinite horizon nonlinear stochasticH∞ control problem of system (1).Theorem 6.
For a given disturbance attenuation levelγ>0, if there exist a set of positive functions V(x,i)∈C2(Rn×D;R+), V(0,i)=0, and ∂2V(x,i)/∂x2≥0 for all nonzero x∈Rn, i∈D with the properties of(11)c1x2≤Vx,i≤c2x2,-mx,i2≤-c3x2,for some positive constants c1,c2,c3 such that V(x,i) solves the coupled HJIs,(12)
Υ
i
=
∂
V
x
,
i
′
∂
x
f
1
x
,
i
+
1
2
f
2
x
,
i
′
∂
2
V
x
,
i
∂
x
2
f
2
x
,
i
+
m
x
,
i
′
m
x
,
i
+
∑
j
=
1
∞
q
i
j
V
x
,
j
-
1
4
f
2
x
,
i
′
∂
2
V
x
,
i
∂
x
2
h
2
x
,
i
+
∂
V
x
,
i
′
∂
x
h
1
x
,
i
-
γ
2
I
+
h
2
x
,
i
′
∂
2
V
x
,
i
∂
x
2
h
2
x
,
i
-
1
·
h
2
x
,
i
′
∂
2
V
x
,
i
∂
x
2
f
2
x
,
i
+
h
1
x
,
i
′
∂
V
x
,
i
∂
x
-
1
4
f
2
x
,
i
′
∂
2
V
x
,
i
∂
x
2
g
2
x
,
i
+
∂
V
x
,
i
′
∂
x
g
1
x
,
i
I
+
g
2
x
,
i
′
∂
2
V
x
,
i
∂
x
2
g
2
x
,
i
-
1
·
g
2
x
,
i
′
∂
2
V
x
,
i
∂
x
2
f
2
x
,
i
+
g
1
x
,
i
′
∂
V
x
,
i
∂
x
≤
0
,
-
γ
2
I
+
h
2
x
,
i
′
∂
2
V
x
,
i
∂
x
2
h
2
x
,
i
<
0
,
i
∈
D
,then(13)u∗x,i=-12I+g2x,i′∂2Vx,i∂x2g2x,i-1g2x,i′∂2Vx,i∂x2f2x,i+g1x,i′∂Vx,i∂x is an infinite horizon H∞ control of system (1).Proof.
We first verify that (6) holds. For any T>0 and initial state x0=0, η0=i, note that the generalized Ito^-type formula [28] and (3) yield(14)EVxT,ηT-Vx0,η0∣η0=i=E∫0TLVx,ηtdt∣η0=i=E∫0T∂Vx,ηt′∂xf1x,ηt+g1x,ηtu+h1x,ηtv+∑j=1∞qijVx,j+12f2x,ηt+g2x,ηtu+h2x,ηtv′∂2Vx,ηt∂x2f2x,ηt+g2x,ηtu+h2x,ηtv+mx,ηt2+u2-z2-γ2v2+γ2v2dt∣η0=i=E∫0TΛ1v,x,ηt+Λ2x,ηt+Λ3u,x,ηt+∑j=1∞qijVx,j-z2+γ2v2+12u′g2x,ηt′∂2Vx,ηt∂x2h2x,ηtv+v′h2x,ηt′∂2Vx,ηt∂x2g2x,ηtudt∣η0=i, where (15)Λ1v,x,ηt=v′-γ2I+12h2x,ηt′∂2Vx,ηt∂x2h2x,ηtv+12f2x,ηt′∂2Vx,ηt∂x2h2x,ηt+∂Vx,ηt′∂xh1x,ηtv+12v′h2x,ηt′∂2Vx,ηt∂x2f2x,ηt+h1x,ηt′∂Vx,ηt∂x,Λ2x,ηt=∂Vx,ηt′∂xf1x,ηt+12f2x,ηt′∂2Vx,ηt∂x2f2x,ηt+mx,ηt′mx,ηt,Λ3u,x,ηt=u′I+12g2x,ηt′∂2Vx,ηt∂x2g2x,ηtu+12f2x,ηt′∂2Vx,ηt∂x2g2x,ηt+∂Vx,ηt′∂xg1x,ηtu+12u′g2x,ηt′∂2Vx,ηt∂x2f2x,ηt+g1x,ηt′∂Vx,ηt∂x.Invoking ∂2V(x,i)/∂x2≥0, i∈D, we deduce that (16)12-u′g2x,ηt′+v′h2x,ηt′∂2Vx,ηt∂x2-g2x,ηtu+h2x,ηtv≥0,which shows that (17)12u′g2x,ηt′∂2Vx,ηt∂x2h2x,ηtv+v′h2x,ηt′∂2Vx,ηt∂x2g2x,ηtu≤12u′g2x,ηt′∂2Vx,ηt∂x2g2x,ηtu+v′h2x,ηt′∂2Vx,ηt∂x2h2x,ηtv.Hence,(18)EVxT,ηT-Vx0,η0∣η0=i≤E∫0TΛ1v,x,ηt+Λ2x,ηt+Λ3u,x,ηt+∑j=1∞qijVx,j-z2+γ2v2+12u′g2x,ηt′∂2Vx,ηt∂x2g2x,ηtu+v′h2x,ηt′∂2Vx,ηt∂x2h2x,ηtvdt∣η0=i=E∫0TΛ¯1v,x,ηt+Λ2x,ηt+Λ¯3u,x,ηt+∑j=1∞qijVx,j-z2+γ2v2dt∣η0=i, where (19)Λ¯1v,x,ηt=v′-γ2I+h2x,ηt′∂2Vx,ηt∂x2h2x,ηtv+12f2x,ηt′∂2Vx,ηt∂x2h2x,ηt+∂Vx,ηt′∂xh1x,ηtv+12v′h2x,ηt′∂2Vx,ηt∂x2f2x,ηt+h1x,ηt′∂Vx,ηt∂x,Λ¯3u,x,ηt=u′I+g2x,ηt′∂2Vx,ηt∂x2g2x,ηtu+12f2x,ηt′∂2Vx,ηt∂x2g2x,ηt+∂Vx,ηt′∂xg1x,ηtu+12u′g2x,ηt′∂2Vx,ηt∂x2f2x,ηt+g1x,ηt′∂Vx,ηt∂x.Applying Lemma 4 to Λ¯1(v,x,ηt) and Λ¯3(u,x,ηt), we conclude that(20)Λ¯1v,x,ηt=v+F1′-γ2I+h2x,ηt′∂2Vx,ηt∂x2h2x,ηtv+F1,-14f2x,ηt′∂2Vx,ηt∂x2h2x,ηt+∂Vx,ηt′∂xh1x,ηt·-γ2I+h2x,ηt′∂2Vx,ηt∂x2h2x,ηt-1·h2x,ηt′∂2Vx,ηt∂x2f2x,ηt+h1x,ηt′∂Vx,ηt∂x,(21)Λ¯3u,x,ηt=u+F2′I+g2x,ηt′∂2Vx,ηt∂x2g2x,ηtu+F2-14f2x,ηt′∂2Vx,ηt∂x2g2x,ηt+∂Vx,ηt′∂xg1x,ηt·I+g2x,ηt′∂2Vx,ηt∂x2g2x,ηt-1·g2x,ηt′∂2Vx,ηt∂x2f2x,ηt+g1x,ηt′∂Vx,ηt∂x,where (22)F1=12-γ2I+h2x,ηt′∂2Vx,ηt∂x2h2x,ηt-1h2x,ηt′∂2Vx,ηt∂x2f2x,ηt+h1x,ηt′∂Vx,ηt∂x,(23)F2=12I+g2x,ηt′∂2Vx,ηt∂x2g2x,ηt-1g2x,ηt′∂2Vx,ηt∂x2f2x,ηt+g1x,ηt′∂Vx,ηt∂x.Recalling (12) and substituting (20) and (21) into (18) yield that(24)EVxT,ηT-Vx0,η0∣η0=i≤E∫0Tv+F1′-γ2I+h2x,ηt′∂2Vx,ηt∂x2h2x,ηtv+F1+u+F2′I+g2x,ηt′∂2Vx,ηt∂x2g2x,ηtu+F2-z2+γ2v2dt∣η0=i. In view of (12), for x0=0, if we choose u=u∗=-F2, then it follows from (24) that (25)E∫0Tz2dt∣η0=i<γ2E∫0Tv2dt∣η0=i.Taking the limit for T⟶∞ in the above, it is easy to show (6) by Definition 2.
Next, we remain to show that whenv=0, u=u∗, the trajectory of system (1) with any initial value x(0)=x0 is EMSS. To this end, for i∈D, let Lu∗ be the infinitesimal operator of system (1) with v=0, u=u∗; then(26)Lu∗Vx,iv=0=∂Vx,i′∂xf1x,i+g1x,iu∗+∑j=1∞qijVx,j+12f2x,i+g2x,iu∗′∂2Vx,i∂x2f2x,i+g2x,iu∗=∂Vx,i′∂xf1x,i+12f2x,i′∂2Vx,i∂x2f2x,i+∑j=1∞qijVx,j+∂Vx,i′∂xg1x,iu∗+12f2x,i′∂2Vx,i∂x2g2x,iu∗+12g2x,iu∗′∂2Vx,i∂x2f2x,i+12g2x,iu∗′∂2Vx,i∂x2g2x,iu∗,=∂Vx,i′∂xf1x,i+12f2x,i′∂2Vx,i∂x2f2x,i+∑j=1∞qijVx,j+Ψ1i+Ψ2i,where(27)Ψ1i=∂Vx,i′∂xg1x,iu∗+12f2x,i′∂2Vx,i∂x2g2x,iu∗+12g2x,iu∗′∂2Vx,i∂x2f2x,i=-12∂Vx,i′∂xg1x,iI+g2x,i′∂2Vx,i∂x2g2x,i-1g2x,i′∂2Vx,i∂x2f2x,i+g1x,i′∂Vx,i∂x-14f2x,i′∂2Vx,i∂x2g2x,iI+g2x,i′∂2Vx,i∂x2g2x,i-1·g2x,i′∂2Vx,i∂x2f2x,i+g1x,i′∂Vx,i∂x-14f2x,i′∂2Vx,i∂x2g2x,i+∂Vx,i′∂xg1x,iI+g2x,i′∂2Vx,i∂x2g2x,i-1g2x,i′∂2Vx,i∂x2f2x,i,and (28)Ψ2i=12g2x,iu∗′∂2Vx,i∂x2g2x,iu∗=18f2x,i′∂2Vx,i∂x2g2x,i+∂Vx,i′∂xg1x,iI+g2x,i′∂2Vx,i∂x2g2x,i-1·g2x,i′∂2Vx,i∂x2g2x,iI+g2x,i′∂2Vx,i∂x2g2x,i-1·g2x,i′∂2Vx,i∂x2f2x,i+g1x,i′∂Vx,i∂x.By direct calculations, one obtains that(29)Ψ1i=-12f2x,i′∂2Vx,i∂x2g2x,i+∂Vx,i′∂xg1x,iI+g2x,i′∂2Vx,i∂x2g2x,i-1·g2x,i′∂2Vx,i∂x2f2x,i+g1x,i′∂Vx,i∂x,and(30)Ψ2i≤18f2x,i′∂2Vx,i∂x2g2x,i+∂Vx,i′∂xg1x,iI+g2x,i′∂2Vx,i∂x2g2x,i-1·g2x,i′∂2Vx,i∂x2f2x,i+g1x,i′∂Vx,i∂x.Implementing (29) and (30) into (16) and taking into account (11) and (12), we have(31)Lu∗Vx,iv=0≤∂Vx,i′∂xf1x,i+12f2x,i′∂2Vx,i∂x2f2x,i+∑j=1∞qijVx,j-38f2x,i′∂2Vx,i∂x2g2x,i+∂Vx,i′∂xg1x,i·I+g2x,i′∂2Vx,i∂x2g2x,i-1·g2x,i′∂2Vx,i∂x2f2x,i+g1x,i′∂Vx,i∂x≤∂Vx,i′∂xf1x,i+12f2x,i′∂2Vx,i∂x2f2x,i+∑j=1∞qijVx,j-14f2x,i′∂2Vx,i∂x2g2x,i+∂Vx,i′∂xg1x,i·I+g2x,i′∂2Vx,i∂x2g2x,i-1·g2x,i′∂2Vx,i∂x2f2x,i+g1x,i′∂Vx,i∂x≤-mx,i′mx,i-14f2x,i′∂2Vx,i∂x2h2x,i+∂Vx,i′∂xh1x,i·γ2I-h2x,i′∂2Vx,i∂x2h2x,i-1·h2x,i′∂2Vx,i∂x2f2x,i+h1x,i′∂Vx,i∂x≤-mx,i2≤-c3x2. Based on Lemma 5, it results that system (1) with v=0,u=u∗ is EMSS. The proof is complete.Remark 7.
It should be pointed that, in Theorem6, if we take Υi<0 in (12), system (1) is internally stable (globally asymptotically stable in probability) even without condition (11). Then the controller defined in (13) is still an H∞ controller (globally asymptotically stable in probability).Below, we will give an example to show the effectiveness of our above developedH∞ design method.Example 8.
Consider a one-dimensional stochastic nonlinear system with infinite Markov jumps and the parameters as follows:(32)f1x,i=-2ixi+1,g1x,i=1i+1,h1x,i=14,f2x,i=xi+1,g2x,i=1,h2x,i=1,mx,i=ix2i+1.Let {ηt}t∈R+ be a Poisson process with parameter λ>0. It is obvious that {ηt}t∈R+ is a homogeneous Markov process with the countably infinite state space, and its infinitesimal matrix Q=(qij) is given by -qii=qi,i+1=λ and qij=0,i∈D,j∈D/{i,i+1}.Assume the disturbance attenuation levelγ=2 and λ=1. Then setting V(x,i)=ix2/2(i+1), we solve the coupled HJIs (12), it is easy to verify that the conditions of Theorem 6 are satisfied; thus, via Theorem 6, we have (33)Υi=-15.5i5-53.5i4-30i3-2.75i2+8i+24i+12i+22i+1<0.So the H∞ controller is (34)u∗x,i=-ixi+1i+2.With the initial conditionsx0=0.5 and the exogenous disturbance v(t)=e-tsint, Figure 1 shows the state response.Figure 1
System state response in Example8.


## 4. Conclusion

For a class of nonlinear stochastic systems with infinite Markov jumps and (x,u,v)-dependent noise, a sufficient condition for infinite horizon H∞ control problem has been obtained in terms of coupled HJIs, and the effectiveness of the proposed design method is demonstrated by a numerical example. There are some further research directions including the investigation on H2/H∞ control and H∞ filter problems for nonlinear infinite Markov jump systems.



---

*Source: 2904521-2018-10-21.xml*