# Robust Control for Nonlinear Markov Jump Systems with Partially Unknown Transition Probabilities

Authors: Lifan Kang; Yue Wang; Ting Hou
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

This note focuses on the finite horizonH2/H∞ control for stochastic nonlinear jump systems with partially unknown transition probabilities. We derive the nonlinear stochastic bounded real lemma and the nonlinear optimal regular result for the considered system at first. A sufficient condition and a necessary condition for the solution of H2/H∞ control are, respectively, offered by four cross-coupled Hamilton–Jacobi equations (HJEs). Besides, numerical examples show the effectiveness of the obtained results.


---

## Body

## 1. Introduction

Control synthesis is one of the important parts of control theory [1–5]. The H∞ synthesis aims to seek for a suitable controller to suppress the effect of the exogenous disturbance on the dynamic system below a given level [6–9]. However, it is aware that H∞ control can guarantee the good robustness of the designed system, but cannot optimize the closed-loop to achieve perfect performance. Because of this, the linear quadratic control (H2 control) is selected to make up the lack in optimization. Combining the two control methods becomes a natural idea to reach the balance. H2/H∞ control not only ensures to repress the influence of the disturbance, but also minimizes the energy cost under the disturbance input [10–12]. So far, H∞ control and H2/H∞ control have been paid continuous attention [13–17]. In particular, for stochastic systems, bounded real lemmas in finite and infinite horizon have been derived for linear models by the coupled Riccati equations method [14, 15], and the coupled Hamilton–Jacobi equations method has been performed for nonlinear models [16, 17]. It can be seen that a local solution to the primal nonlinear H2/H∞ control exists if its linearized H2/H∞ control problem is solvable. However, most of the existing works on stochastic H2/H∞ control are concerned with jump linear systems, while little attention is paid on nonlinear systems with Markov jump.As we all know, Markov jump systems have been used widely both in theory and in engineering over the past decades [18–24]. The main motivation of research studies is that such models have numerous applications in mechanics, traffic, power, and many other fields in industry and finance. When encountering system failures, sudden environmental changes, and external noise, the structure and parameter factors of dynamics are mutated. The process of state hopping from one mode to another can be marked as Markov jumps. The transition probabilities of a jump process are crucial factors which determine the behavior of a system exactly [25, 26]. Normally, the elements of the transition probability matrix are assumed to be fully known [14, 15]. However, in some practical cases, the transition probabilities may not be fully known, which inspired scholars to study Markov jump systems with partial probability [27–37]. For instance, Zhang and Boukas considered stability and stabilization of Markovian jump systems with partially unknown transition probabilities [27]. In addition, sliding-mode control, H2, H∞, and filtering control subject to partially unknown transition probabilities have gained considerable research interest [29–35]. Nevertheless, to the best of our knowledge, there are no literatures to deal with H2/H∞ control for nonlinear jump systems with partially unknown transition probabilities up to now, which is the theoretical significance of this note. More importantly, to get the robust controller, a nonlinear stochastic bounded real lemma is derived.The remainder of this paper is arranged as follows. The second part provides some useful definitions and lemmas. In the third part, for nonlinear jump systems with partially unknown transition probabilities, a sufficient condition and a necessary condition for the finite horizonH2/H∞ control are obtained, respectively. The forth part gives numerical illustrative examples. Conclusions are drawn in the fifth part.Notations used in this study are as follows.ℛn is the n-dimensional Euclidean space; A>0A≥0: A is a positive definite (positive semidefinite) matrix; E· is the mathematical expectation; AT is the transpose of a matrix A; A−1 is the inverse of a nonsingular matrix A; ℛn×m is the set of all n×m real matrices; · is the Euclidean vector norm; Ω,ℱ,ℱtt≥0,P is the completed filtration space with the filtration ℱtt≥0 satisfying the usual conditions, i.e., it is right continuous and ℱ0 contains all P-null sets; ℒF20,T,ℛl is the space of all nonanticipative stochastic processes yt∈ℛl with respect to an increasing σ-algebra Ftt⩾0, which satisfies E∫0Tyt2dt<∞; C2,1U,0,T: the class of all functions Vx,t are twice continuously differentiable with respect to x∈U and once continuously differentiable with respect to t∈0,T, except possibly at the point x=0; In×n is the identity matrix; colx1,x2,…,xn≔x1,x2,…,xn′.


## 2. Preliminaries

Let us think about the following stochastic nonlinear jump systems described by the Itô-type equation:(1)dxt=fx,t,θt+gx,t,θtut+hx,t,θtvtdt+lx,t,θt+qx,t,θtut+sx,t,θtvtdwt,x0∈ℛn,θ0∈T,t∈0,T,zt=colmx,t,θt,ut≔mx,t,θtut,where xt∈ℛn, zt∈ℛz, ut∈ℒF20,T,ℛnu, and vt∈ℒF20,T,ℛnv stand for the system state, penalty output, control input, and exogenous disturbance signal, respectively. ωt is an one-dimensional standard wiener process defined on the filtered probability space Ω,ℱ,ℱt,P with ℱt=σωs:0≤s≤t. The stochastic mode jump process θt is a continuous-time discrete-state Markov process with values in a finite space T=1,2,…,N and is assumed to be independent with ωt. The process of transition probabilities are denoted by(2)Pθt+τ=h|θt=r=πrhτ+oτ,r≠h,1+πrrτ+oτ,r=h,where τ>0, limτ⟶0oτ/τ=0, and πrh≥0r,h∈T,r≠h represent the transition rate from mode r at time t to mode h at time t+τ and πrr=−∑h=1,r≠hNπrh for all r∈T. So, the transition probabilities matrix is defined by(3)∏=π11π12⋯π1Nπ21π22⋯π2N⋮⋮⋱⋮πN1πN2⋯πNN.In this paper, we suppose that the transition probabilities are partly unknown. For instance, forN=4, the transition rate matrix ∏ is given by(4)∏=π11?π13????π24π31?π33?π41??π44.In above, we use“?” to sign the unknown element. Furthermore, ∀r∈T, we set T=TKr+TUKr, where(5)TKr≜h:πrhis known,TUKr≜h:πrhis unknown.In addition, ifTKr≠∅, it can be described as TKr=K1r,K2r,…,Kmr, ∀1≤m≤N, where Kmr is the mth known element and the exponent Kmr is the rth row of matrix ∏. Then, set πKr≜∑j∈TKrπrh.Remark 1.
WhenTKr=0 and TUKr=T, the transition rates in the stochastic process are fully unknown. TKr=T and TUKr=0 mean that the transition rates are fully known.
For convenience, we signify·θt≔·x,t,θt throughout the paper, and all coefficients in (1) are thought to be Borel measurable. Meanwhile, assume that f0,t,r≡0, g0,t,r≡0 for ∀t,r∈0,T×T.
Now, we introduce the following definitions.Definition 1.
Givenγ>0, a feedback control law u=uT∗∈ℒℱ20,T,ℛnu is called the finite horizon robust H2/H∞ control of system (1), if the following conditions are satisfied:(i)
For any nonzero,vt∈ℒℱ20,T,ℛnu, and the trajectory of the resulted closed-loop by (1) starting from x0=0 and θ0=0, we always have(6)E∫0Tmx,t,θt2+uT∗2dt|θ0=r≤γ2E∫0Tv2dt|θ0=r,r∈T;(ii)
When the worst disturbancevT∗ is executed in (1), uT∗ can minimize the quadratic performance E∫0Tmx,t,θt2+u2dtθ0=r, r∈T.Definition 2.
(see [18]). For each Vx,t,r∈C2,1ℛn×0,T×T,ℛ, we have an operator ℒV:ℛn×0,T×T⟶ℛ associated with (1) given by(7)ℒVx,t,r=∂Vx,t,r∂t+∂V′x,t,r∂xfx,t,r+gx,t,rut+hx,t,rvt+∑h=1NπrhVh+12lx,t,r+qx,t,rut+sx,t,rvtT×∂2Vx,t,r∂x2lx,t,r+qx,t,rut+sx,t,rvt.Definition 3.
(see [16]). We set two extreme value functions V˜T1x,s,θt∈ℛn×0,T×T⟼ℛ− and V˜T2x,s,θt∈ℛn×0,T×T⟼ℛ+ related with (1) as follows:(8)V˜T1x,s,θt=infv∈ℒℱ2s,T,ℛnvxs=x,u=uT∗J1Tu,v,V˜T2x,s,θt=infu∈ℒℱ20,T,ℛnuxs=x,v=vT∗J2Tu,v,in which the performances are as follows:(9)J1Tu,v;x0,θ0≔γ2v0,T2−z0,T2=E∫0Tγ2v2−z2dtθ0=r,r∈T,J2Tu,v;x0,θ0≔z0,T2=E∫0Tz2dtθ0=r,r∈T.Remark 2.
The following results are obvious:(10)V˜T1≤0,V˜T1x,t,θt=0forallx∈ℛn;V˜T2≥0,V˜T2x,t,θt=0forallx∈ℛn.Remark 3.
Let the perturbation operator be denoted byLT:ℒℱ20,T,ℛnv⟶ℒℱ20,T,ℛnz with the norm(11)LT=supv∈ℒℱ20,T,ℛnvv≠0,θ0=r,x0=0z·0,Tv·0,T=supv∈ℒℱ20,T,ℛnvv≠0,θ0=r,x0=0E∫0Tmx,t,θt2dt|θ0=r1/2E∫0Tv2dt|θ0=r1/2,r∈T.
It can be checked that (6) is equivalent to LT≤γ.
Next, we state the following lemmas, which will be used later.Lemma 1.
For a given levelγ>0, with an initial state x0=0∈ℛn and θ0∈T, think about the following stochastic perturbed system with Markov jumps:(12)dxt=fx,t,θt+hx,t,θtvtdt+lx,t,θt+sx,t,θtvtdwt,zt=mx,t,θt.
If there existsVx,t,θt∈C2,1ℛn×0,T×T,ℛ with system (12) and satisfies the following HJE:(13)Δ1Vθt≔∂Vθt∂t+∂Vθt′∂xfθt−mθt′mθt+12lθt′∂2Vθt∂x2lθt+∑h∈TKrπrhVh−Tr−14∂Vθt′∂xhθt+lθt′∂2Vθt∂x2sθtγ2I+12sθt′∂2Vθt∂x2sθt−1hθt′∂Vθt∂x+sθt′∂2Vθt∂x2lθt=0,γ2I+12sθt′∂2Vθt∂x2sθt>0,Vh−Tr≥0,h∈TUKr,r≠h,Vh−Tr≥0,h∈TUKr,r=h,Vx,T,θt=0.Then, LT≤γ holds.Proof.
Noticing that for any symmetric matrixTr (r∈T), we have ∑h=1NπrhTr=0. Applying the generalized Itô’s formula, one gets(14)ℒVx,t,θt=E∫0T∂Vθt∂t+∂Vθt′∂xfθt+hθtv+12lθt+sθtv′∂2Vθt∂x2lθt+sθtv+∑h=1NπθthVhdtθ0=r=E∫0T∂Vθt∂t+∂Vθt′∂xfθt+hθtv+12lθt+sθtv′∂2Vθt∂x2lθt+sθtv+∑h=1NπrhVh−∑h=1NπrhTrdtθ0=r=E∫0T∂Vθt∂t+∂Vθt′∂xfθt+hθtv+12lθt+sθtv′∂2Vθt∂x2lθt+sθtv+∑h∈TKrπrhVh−Tr+∑h∈TUKrπrhVh−Trdtθ0=r.
Then, it can deduced that(15)EVxT,T,θT−Vx0,0,θ0θ0=r=E∫0TℒTVx,t,θt=E∫0T∂Vθt∂t+∂Vθt′∂xfθt+hθtv+12lθt+sθtv′∂2Vθt∂x2lθt+sθtv+∑h=1NπθthVx,t,h+z2−γ2v2−z2+γ2v2dtθ0=r=E∫0T∂Vθt∂t+∂Vθt′∂xfθt+hθtv+12lθt+sθtv′∂2Vθt∂x2lθt+sθtv+z2−γ2v2−z2+γ2v2+∑h∈TKrπrhVh−Tr+∑h∈TUKrπrhVh−Trdtθ0=r=E∫0Tz2−γ2v2dt+v−vT∗γ,s,Vθt2+Δ1Vθt+∑h∈TUKrπrhVh−Trθ0=r,r∈T,where(16)zγ,s,Vθt2=z′γ2I+12sθt′∂2Vθt∂x2sθtz,v^T∗=−12γ2I+12sθt′∂2Vθt∂x2sθt−1hθt′∂Vθt∂x+sθt′∂2Vθt∂x2lθt,πrh≥0∀r,h∈T,r≠h,πrr=−∑h=1,r≠hNπrh<0r∈T,∑h∈TUKrπrhVh−Tr≥0.
Inferring from (14)–(16), it reveals that v^T∗ is the corresponding worst disturbance, which yields that(17)E∫0Tz2dt|θ0=r=E∫0Tγ2v2−v−vT∗γ,s,Vθt2−Δ1Vθt−∑h∈TUKrπrhVh−Tr+EVxT,T,θT−Vx0,0,θ0dt|θ0=r≤E∫0Tγ2v2dt|θ0=r.
That is,(18)0≤J1TuT∗,vT∗;x0,θ0≤J1TuT∗,v;x0,θ0.
This lemma is proved.Lemma 2.
For a given levelγ>0, with an initial state x0=0∈ℛn, θ0∈T, think about the following nonlinear stochastic controlled system with Markov jumps:(19)dxt=fx,t,θt+gx,t,θtutdt+lx,t,θt+qx,t,θtutdwt,zt=mx,t,θt.
If the following HJE(20)Δ2V2θt≔∂V2θt∂t+∂V2θt′∂xfθt−mθt′mθt+12lθt′∂2V2θt∂x2lθt+∑h∈TKrπrhVh−Tr−14∂V2θt′∂xgθt+lθt′∂2V2θt∂x2qθtI+12qθt′∂2V2θt∂x2qθt−1hθt′∂V2θt∂x+qθt′∂2V2θt∂x2lθt=0,I+12qθt′∂2V2θt∂x2qθt>0,Vh−Tr≥0,h∈TUKr,r≠h,Vh−Tr≤0,h∈TUKr,r=h,V2x,T,θt=0.admits a nonnegative solution V2x,t,θt∈C2,1ℛn×0,T×T,ℛ, then we have(21)J2TuT∗,vT∗;x0,θ0≤J2TuT,vT∗;x0,θ0=V2x0,0,r,r∈T,with the optimal control(22)u^T∗=−12I+12qθt′∂2V2θt∂x2qθt−1gθt′∂V2θt∂x+qθt′∂2V2θt∂x2lθt,where(23)J2TuT,vT∗;x0,θ0≔infu∈ℒF20,T,ℛnuE∫0Tz2dt|θ0=r=infu∈ℒF20,T,ℛnuE∫0Tmθt2+u2dt|θ0=r.Proof.
Taking integration and expectation indV2x,t,θt, for any T>0, we get(24)E∫0TdV2x,t,θt=E∫0TLuV2dt+E∫0Tlθt+qθtu′∂V2θt∂xdwt|θ0=r=E∫0T∂V2θt∂t+∂V2θt′∂xfθt+gθtudt+12lθt+qθtu′∂2V2θt∂x2lθt+qθtudt+∑h=1NπrhVx,t,h|θ0=r=E∫0T∂V2θt∂t+∂V2θt′∂xfθt+gθtu+12lθt+qθtu∂2V2θt∂x2lθt+qθtudt+∑h=1NπrhVh−∑h=1NπrhTr|θ0=r=E∫0T∂V2θt∂t+∂V2θt′∂tfθt+gθtudt+12lθt+qθtu′∂2V2θt∂x2lθt+qθtudt+∑h∈TKrπrhVh−Tr+∑h∈TUKrπrhVh−Tr|θ0=r,r∈T.
By Itô’s formula and the completing square technique combined with (24), we obtain(25)J2Tu,vT∗;x0,θ0=E∫0Tz2dtθ0=r=E∫0Tmθt2+u2dt|θ0=r=E∫0Tmθt′mθt+u′udt+ℒV2x,t,θt+V2x0,0,θ0−EV2x,t,θt|θ0=r=E∫0T∂V2θt∂t+lθt′∂V2θt′∂xfθt+gθtu+12lθt+qθtu′∂2V2θt∂x2lθt+qθtudt+∑h=1NπrhVx,t,h+mθt′mθt+u′udt+V2x0,0,θ0−EVx,t,θt|θ0=r=E∫0T∂V2θt∂t+lθt′∂V2θt′∂xfθt+12lθtT∂2V2θt∂x2lθt+mθt′mθt+∑h∈TKrπrhVh−Tr+∑h∈TUKrπrhVh−Tr−14∂2V2θt′∂x2gθt+lθt′∂2V2θt∂x2qθtI+12qθt′∂2V2θt∂x2qθt−1+gθt′∂V2θt∂x+qθt′∂2V2θt∂x2lθt+u−u^T∗′I+12qθt′∂2V2θt∂x2qθtu−u^T∗dt+V2x0,0,θ0−EVx,t,θt|θ0=r=E∫0Tu−u^T∗2dt+△2V2θt+∑h∈TUKrπrhVh−Tr+V2x0,0,r−EVx,t,θt,r∈T,in which the inequality J2TuT∗,vT∗;x0,θ0≤J2TuT,vT∗;x0,θ0=V2x0,0,r holds since that V2x,T,θt=0, △2V2θt=0, and ∑h∈TUKrπrhVh−Tr≥0r=horr≠h. So, from (25), this lemma is obvious with the optimal control (22).


## 3. Main Results

In this part, a sufficient condition comes down to Theorem1 for the solvability of the H2/H∞ control with nonlinear jump systems (1).Theorem 1.
For a givenγ>0, think about the following four cross-coupled HJEs:(26)△3V1θt≔∂V1θt∂t+∂V1θt′∂xfθt−m˜θt′m˜θt+12l˜θt′∂2Vθt∂x2l˜θt+∑h∈TKrπrhVh−Tr−14∂V1θt′∂xhθt+l˜θt′∂2V1θt∂x2sθtγ2I+12sθt′∂2Vθt∂x2sθt−1hθt′∂Vθt∂x+sθt′∂2Vθt∂x2l˜θt=0,γ2I+12sθt′∂2V1θt∂x2sθt>0,Vh−Tr≥0,h∈TUKr,r≠h,Vh−Tr≤0,h∈TUKr,r=h,V1x,T,θt=0,(27)K1=−12γ2I+12sθt′∂2V1θt∂x2sθt−1hθt′∂V1θt∂x+sθt′∂2V1θt∂x2l˜θt,(28)△4V2θt≔∂V2θt∂t+∂V2θt′∂xf˜θt−mθt′mθt+12l˜1θt′∂2Vθt∂x2l˜1θt+∑h∈TKrπrhVh−Tr−14∂V2θt′∂xgθt+l˜1θt′∂2V2θt∂x2sθtI+12qθt′∂2V2θt∂x2qθt−1hθt′∂V2θt∂x+qθt′∂2V2θt∂x2l˜1θt=0,I+12qθt′∂2V2θt∂x2qθt>0,Vh−Tr≥0,h∈TUKr,r≠h,Vh−Tr⩽0,h∈TUKr,r=h,V1x,T,θt=0,(29)K2=−12I+12qθt′∂2V2θt∂x2qθt−1gθt′∂V2θt∂x+qθt′∂2V2θt∂x2l˜1θt.If there exist solutionsV1θt,V2θt with V1θt≤0 and V2θt≥0 for (26)–(29), then the finite horizon H2/H∞ control of nonlinear Markov jumps has a pair of solutions uT∗,vT∗ with uT∗=K2x, vT∗=K1x, and J2TuT∗,vT∗;x0,θ0=V2x0,0,r.Proof.
Notice the following transformations in (26)–(29):(30)f˜θt=fθt+gθtK2,l˜θt=lθt+qθtK2,m˜θt=colmθt,K2,f˜1θt=fθt+hθtK1,l˜1θt=lθt+sθtK1.
Substitutingu=K2x with K2 defined by (29) into (1), we have(31)dxt=f˜x,t,θt+hx,t,θtvtdt+l˜x,t,θt+sx,t,θtvtdwt,zt=mθt,K2.
Applying Lemma1 to system (31), we conclude(32)E∫0Tmθt2+uT∗2dt|θ0=r≤γ2E∫0Tv2dt|θ0=r,r∈T,and v=K1x is the worst case disturbance. In the meantime, implementing v=vT∗=K1x into (1), one yields that(33)dxt=f˜1x,t,θt+gx,t,θtutdt+l˜1x,t,θt+qx,t,θtutdwt,x0∈ℛn,θ0∈T,t∈0,T.
MinimizingJ2Tu,vT∗;x0,θ0 under the constraint of (33) is a standard nonlinear quadratic optimal problem. By applying the Lemma 2, J2Tu,vT∗;x0,θ0 achieves its minimum at uT∗=K2x, and J2TuT∗,vT∗;x0,θ0=V2x0,0,r. By Definition 1, this theorem is proved.
Theorem2 offers a necessary condition for the H2/H∞ control of system (1).Theorem 2.
For a givenγ>0, V˜Tθt1,V˜Tθt2∈C2,1∈ℛn×0,T×T, think about system (1). If the finite horizon H2/H∞ of nonlinear stochastic jump systems has solutions uT∗=K2x,vT∗=K1x∈ℒF20,T,ℛnu×0,T,ℛnu satisfying the following terms:(34)γ2I+12sθt′∂2V˜Tθt1∂x2sθt>0,VT1x,t,θt=0,I+12qθt′∂2V˜Tθt2∂x2qθt>0,VT2x,t,θt=0.Then, V˜Tθt1≤0 and V˜Tθt2≥0 are the solutions of the four cross-coupled HJEs (26)–(29).Proof.
Substitutingu=uT∗=K2x into (1), one reads (31). Hence, (6) holds because uT∗,vT∗ solves the H2/H∞ control. By Definition 3, Lemma 1, and Lemma 4.1 of [17], we assert that V˜Tθt1≤0 solves the following HJE:(35)ℒu=uT∗,v=0V˜T1−mϑt′mθt−K22−∑h∈TKrπrhVh−Tr−14∂V˜Tθt1′∂xhθt+l˜θt′∂2V˜Tθt1∂x2sθtγ2I+12sθt′∂2V˜Tθt1∂x2sθt−1hθt′∂V˜Tθt1∂x+sθt′∂2V˜Tθt1∂x2l˜θt=0,γ2I+12sθt′∂2V˜Tθt1∂x2sθt>0,Vh−Tr⩾0,h∈TUKr,r≠h,Vh−Tr⩽0,h∈TUKr,r=h,V˜T1x,T,θt=0.
Notice the inequality (15), apparently,(36)E∫0Tγ2v2−z2dt|θ0=r=V0,0,θ0−EVx,t,θt+E∫0Tv−vT∗γ,s,V˜Tθt12dt+Δ3V1θt+∑h∈TUKrπrhVh−Tr.
This, together with (35) and Definition 3, for each T>0(37)J1TuT∗,v;x0,θ0=V˜T1x0,0,θ0+E∫0Tv−v˜T∗γ,s,V˜Tθt12dt+∑h∈TUKrπrhVh−Tr.
From (37), we see that J1TuT∗,vT∗;x0,θ0=minJ1TuT∗,v;x0,θ0, and(38)v˜T∗=−12γ2I+12sθt′∂2V˜Tθt1∂x2sθt−1hθt′∂V˜Tθt1∂x+sθt′∂2V˜Tθt1∂x2l˜θt,is the worst perturbance. So, vT∗=v˜T∗=K1x. Then, substituting v=vT∗=K1x into (1), (33) can be obtained. Owing to minimizing V˜T2x,s,θt=infu∈ℒℱ20,T,ℛnuxs=x,v=vT∗J2Tu,v under the constraint of (33), we can infer that uT∗=K2x is the optimal solution. Next, considering the stochastic dynamic programming principle, we can certify that V˜Tθt2,uT∗ solves the following HJE:(39)−∂V˜Tθt2∂t+maxu∈ℒF20,T,ℛnuHx,t,θt,u,−∂V˜Tθt2∂x,−∂2V˜Tθt2∂x2=0,that is,(40)−∂V˜Tθt2∂t+Hx,t,θt,uT∗,−∂V˜Tθt2∂x,−∂2V˜Tθt2∂x2=0.
In this step, the generalized Hamiltonian function is prescribed as follows:(41)Hx,t,θt,u,−∂V˜Tθt2∂x,−∂2V˜Tθt2∂x2≔−u2−mθt2−∂V˜Tθt2′∂xf˜1θt+gθtu−∑h=1NπrhVx,t,h−12l˜1θt+qθtu′∂2V˜Tθt2∂x2l˜1θt+qθtu=−Δ4V˜Tθt2+∂V˜Tθt2∂t−u−uT∗1,q,V˜Tθt22−∑h∈TUKrπrhVh−Tr,with(42)u˜T∗=−12I+12qθt′∂2V˜Tθt2∂x2qθt−1gθt′∂V˜Tθt2∂x+qθt′∂2V˜Tθt2∂x2l˜1θt.
From (41), we get uT∗=u˜T∗=K2x. Substituting the above uT∗ into (40) and considering Definition 3, it tests that V˜Tθt2≥0 solves the following HJE:(43)∂V˜Tθt2∂t+∂V˜Tθt2′∂xf˜1θt+mθt′mθt+12l˜1θt∂2V˜Tθt2∂x2l˜1θt+∑h∈TKrπrhVh−Tr−14∂V˜Tθt2′∂xgθt+l˜1θt′∂2V˜Tθt2∂x2qθtI+12qθt′∂2V˜Tθt2∂x2qθt−1gθt′∂V˜Tθt2∂x+qθt′∂2V˜Tθt2∂x2l˜1θt=0,I+12qθt′∂2V˜Tθt2∂x2qθt>0,Vh−Tr≥0,h∈TUKr,r≠h,Vh−Tr≤0,h∈TUKr,r=h,V˜T2x,T,θt=0.
Combining (35) and (43), the desired result therefore is obtained.Remark 4.
It should be noted that HJEs (26)–(29) are hard to solve in general. To get the analytic expression of the controller, Takagi–Sugeno fuzzy model is constantly used, which can approximate nonlinear system effectively. The method to solve HJEs (26)–(29) is worth to make further studies.


## 4. Examples

In this part, we will give two examples to illustrate the usefulness of above results.Example 1.
Think about the following one-dimensional Markov jump systems with three modes:(44)dxt=−32x3+xut+2xvtdt+x2dwt,zt=col12x,ut,θ0∈T,(45)dxt=−12x3−2x2−8x+2ut+4vtdt+x2+2xdwt,zt=col52x,ut,θ0∈T,(46)dxt=−32x3−6x2−252x+2ut+27vtdt+3x2+2xdwt,zt=col52x,ut,θ0∈T.
We assume that the elements of the transition probabilities matrix∏1 are fully known:(47)∏1=−12141412−1121212−1.
SetVrhx=prhx2r=1,2,h=1,2,3. For given γ=1, the corresponding HJEs have solutions with p11=−1, p12=−1/2, p13=−1/2, p21=2, p22=3/2, and p23=3/2. According to Theorem 1, the H2/H∞ controller of (44)–(46) can be chosen as vT∗x,t,1=2x2, vT∗x,t,2=2x, vT∗x,t,3=7xuT∗x,t,1=−2x2, uT∗x,t,2=−3/2x, and uT∗x,t,3=−3x.Example 2.
Think about the following one-dimensional Markov jump systems with three modes:(48)dxt=−32x3+14ut+12vtdt+3x2dwt,zt=col12x,ut,θ0∈T,(49)dxt=−12x3−2x2−8x+193ut+503vtdt+x2+2xdwt,zt=col52x,ut,θ0∈T,(50)dxt=−32x3−6x2−252x+373ut+863vtdt+3x2+2xdwt,zt=col52x,ut,θ0∈T.
The elements of the transition probability matrix∏2 are supposed to be partially unknown:(51)∏2=−12??12−1?12?−1,where “?” represents the unaccessible element. Set Vrhx=prhx2r=1,2,h=1,2,3 and γ=1. It can be found that p11=−1, p12=−1/2, p13=−1/2, p21=2, p22=3/2, and p23=3/2 solving the corresponding HJEs. According to Theorem 1, the H2/H∞ controller of (48)–(50) can be given by vT∗x,t,1=1/2x, vT∗x,t,2=5/6x, vT∗x,t,3=43/6x, uT∗x,t,1=−1/2x, uT∗x,t,2=−19/2x, and uT∗x,t,3=−37/2x.


## 5. Conclusions

This note dealt with the finite horizonH2/H∞ control for stochastic nonlinear Markov jump systems with partially unknown transition probabilities. Based on the four cross-coupled Hamilton–Jacobi equations, a sufficient condition and a necessary condition for the existence of H2/H∞ control are respectively drawn, which can be regarded as the generalization of [16] to nonlinear jump models. The validity of the results has been demonstrated by two numerical examples.



---

*Source: 1940676-2020-05-11.xml*