# Optimal Vibration Control of a Class of Nonlinear Stochastic Systems with Markovian Jump

Authors: R. H. Huan; R. C. Hu; D. Pu; W. Q. Zhu
Journal: Shock and Vibration
 (2016)

---

## Abstract

The semi-infinite time optimal control for a class of stochastically excited Markovian jump nonlinear system is investigated. Using stochastic averaging, each form of the system is reduced to a one-dimensional partially averaged Itô equation of total energy. A finite set of coupled dynamical programming equations is then set up based on the stochastic dynamical programming principle and Markovian jump rules, from which the optimal control force is obtained. The stationary response of the optimally controlled system is predicted by solving the Fokker-Planck-Kolmogorov (FPK) equation associated with the fully averaged Itô equation. Two examples are worked out in detail to illustrate the application and effectiveness of the proposed control strategy.


---

## Body

## 1. Introduction

Markovian jump system (MJS) is a class of hybrid systems, which has different operation modes or forms governed by a Markov process. This class of systems can be used to represent complex real systems, which may experience abrupt variation in their structures and parameters. It is well known that random form switching can give rise to unexpected phenomena of the whole MJSs, even if the dynamics in each form is simple and well understood. Development of methodology for analysis of such systems is thus much deserving.MJSs have been studied with growing interest and activity in recent years since they are first introduced by Kats and Krasovskii [1]. For dynamics analysis, several important criteria for stochastic stability of MJSs have been established [2, 3]. The stationary response of the stochastically excited nonlinear MJSs is studied by Huan et al. [4]. For the problem of optimal control, Krasosvkii and Lidskii studied the LQR control of Markovian jump linear systems firstly [5]. As a consequence, considerable attention has been paid to the optimal control problem of MJSs [6, 7]. Sworder solved the jump linear Gaussian problem for finite time horizon [8]. The infinite time horizon problem of linear MJSs was studied by Wonham [9] and Fragoso and Hemerly [10]. The optimal control solution for linear MJSs subject to Gaussian input and measurement noise is given by Ji and Chizeck [11]. Ghosh et al. investigated the ergodic control problem of switching diffusion [12]. However, most of the published results are applicable to linear MJSs. Far less is known about the optimal control of nonlinear MJSs, particularly for stochastically excited nonlinear MJSs.Recently, a nonlinear optimal control strategy of stochastic excited nonlinear systems has been proposed by Zhu based on the stochastic averaging method [13] and stochastic dynamical programming principle. It is proved that this strategy is quite effective for response alleviation of nonlinear stochastic systems. The goal of this paper is to extend the strategy for nonlinear stochastic system to the one for nonlinear stochastic MJSs.An infinite time optimal control problem for single degree-of-freedom (SDOF), stochastically excited, nonlinear, Markovian jump system is investigated in this paper. The organization of this paper is as follows. In Section2, the equation of SDOF nonlinear Markovian jump system is examined. Stochastic averaging method is applied to this system in Section 3. The partially averaged equation of total energy for each form of the original system is obtained. A finite set of coupled dynamical programming equations is then set up in Section 4, from which the optimal control force is determined. The Fokker-Planck-Kolmogorov (FPK) equation associated with the fully averaged Itô equation is also derived and solved. In Section 5, two examples are worked out in detail to illustrate the application and effectiveness of the proposed control strategy.


## 2. Formulation of Control Problem

Consider a SDOF controlled, stochastically driven nonlinear system with Markovian jump governed by(1)x¨+gx=εfx,x˙,st+εux,x˙,st+ε1/2hkx,stWkt,xt0=x0,st0=s0,where g(x) is nonlinear stiffness; ε is a small parameter; εf(x,x˙,s(t)) denotes light damping with Markovian jump; εu(x,x˙,s(t)) is the feedback control force; ε1/2hk(x,s(t))(k=1,2,…,m) represent the Markovian jump coefficients of weakly external and (or) parametric stochastic excitations; Wk(t) are independent Gaussian white noises with zero means and intensities 2Dk.The parameters(t) is a continuous-time Markovian jump process with finite discrete state-space S={1,2,…,l} and transition matrix Λ={λij}(i,j∈S), where λij are real numbers such that, for i≠j, λij>0, and for all i∈S, λii=-∑j≠iλij. The transition probability is given by [14, 15](2)Pst+Δt=j∣st=i=λijΔt+oΔt,i≠j,1+λiiΔt+oΔt,i=j,where o(Δt) is such that limΔt→0o(Δt)/Δt=0.Our primary goal here is to design an optimal controller which will result in the minimum of a specified performance. For infinite-horizon control problem, the performance index is specified by(3)Ju=limtf→∞1tfE∫0tfLxτ,x˙τ,uτ,sτdτ,where E[·] denotes an expectation operation; tf is the terminal time of control; L(x(τ),x˙(τ),u(τ),s(τ)) is cost function. If random excitations Wk(t) and Markovian jump process st are ergodic, then the response of controlled system approaches to stationary and ergodic as tf→∞. In this case, the performance index (3) is rewritten as(4)Ju=limtf→∞1tf∫0tfLxτ,x˙τ,uτ,sτdτ.Equations (1) and (4) constitute the mathematical formulation of the infinite-horizon optimal control problem for a SDOF nonlinear system undergoing Markovian jump. In this paper, we consider the perfect observation case, which means that the system state (x(t),x˙(t)) and values of s(t) are all available at each time t.


## 3. Partially Averaged Equation

Letq=x and p=x˙; (1) can be converted to the following Itô stochastic differential equations [11, 16]:(5)dq=pdt,dp=-gq+εfq,p,st+εuq,p,st+ε1/2σkq,stdBkt,where Bk(t) are independent standard Wiener processes; q and p are the generalized displacement and generalized momentum, respectively; σk(q,s(t))=2Dkhk(q,s(t)). The Hamiltonian function H (total energy) of the Hamiltonian system associated with system (5) is(6)Hq,p=p22+∫0qgudu.Suppose that s(t) is a slow jump process and independent of system’s state. That means that, for original system (1) or (5), the system jumps among the l forms or modes slowly and independently. We consider one of the forms by letting the Markovian jump process be arbitrarily fixed at s(t)=i(i∈S). When s(t)=i, denote f(i)(q,p)=f(q,p,s(t)), u(i)(q,p)=u(q,p,s(t)), and σk(i)(q)=σk(q,s(t)). Since H is a function of q and p, the Itô equation for H can be derived from (5) by using Itô differential rule as follows [17]:(7)dH=εfiq,p∂H∂p+12σkiqσkiq∂2H∂p2+εuiq,p∂H∂p+ε1/2σkiq∂H∂pdBkt.In the case of light damping and weak excitations, the HamiltonianH in system (7) is a slowly varying process while the generalized displacement q in (5) is a rapidly varying process. Since the slowly varying process is essential for describing the long-term behavior of the system, the stochastic averaging method is used to average out the rapidly varying process and to yield the following partially averaged Itô equation for H [13, 18, 19]:(8)dH=εmiH+muiHdt+ε1/2σ-iHdBt,where (9)miH=1TH∫Ωfiq,p∂H∂p+12σkiqσkiq∂2H∂p2∂H∂p-1dq,muiH=uiq,p∂H∂p,σ-iq,p2=1TH∫Ωσkiqσkiq∂H∂p2∂H∂p-1dq,·=1TH∫Ω·∂H∂p-1dq.The region of integration is Ω={q∣H(q,0)≤H}, and the parameter(10)TH=∫Ω∂H∂p-1dq.Note that the second term mu(i)(H) on the right-hand of (8) is unknown since u(i) are unknown function of q, p, and s at this stage.Equation (8) is only valid when s(t)=i. As Markovian jumps are allowed so that s(t) takes values from S={1,2,…,l}, (8) can be extended so that(11)dH=εmH,s+muH,sdt+ε1/2σ-H,sdBt,where m(H,s), mu(H,s), and σ-(H,s) change as s(t) jumps, so that m(H,s(t)=i)=m(i)(H), mu(H,s(t)=i)=mu(i)(H), and σ-(H,s(t)=i)=σ-(i)(H).To be consistent with the partially averaged equation (11), performance index (4) is also partially averaged: that is,(12)Ju=limtf→∞1tf∫0tfLHτ,uτ,sτdτ.The original optimal control problem (1) and (4) has been converted to the partially averaged optimal control problem (11) and (12). The partially averaged optimal control problem is much simpler than the original one since (11) is only one-dimensional. It has been proved [20] that the optimal control for the partially averaged control problem is quasi-optimal control for the original one.


## 4. Optimal Control Law

For the proposed control problem, we introduce the value function:(13)VH,s=minuJu.According to the stochastic dynamical programming principle and Markovian jump rules, the following dynamical programming equation can be established [16]:(14)minu12σ-2H∂2VH,s∂H2+mH,s+muH,s∂VH,s∂H+LH,u,s+∑r=1lλrsVH,r=0,s∈S,where λrs are the transition rates described in (2). Equation (14) is a finite set of second-order partial differential equations indexed by s. These equations are coupled through the zero-order terms ∑r=1lλrsV(H,r).The necessary condition for minimizing the left-hand side of (14) is(15)∂∂umuH,s∂V∂H+LH,uq,p,s,s=0.Let the cost functionL be of the form(16)LH,u,s=fH,s+Ru2,where f(H,s)≥0, and R is a positive constant. By substituting (16) into (15) and exchanging the order of the derivative with respect to u and averaging in (15), the expression of the optimal control force is determined:(17)u∗q,p,s=-12R∂VH,s∂H∂H∂p,s∈S.Note that V(H,s) is generally a nonlinear functional of H. Hence, u∗(q,p,s) is a nonlinear function of q and p; that is, the optimal feedback control is nonlinear. To ensure that the control force u∗(q,p,s) is indeed optimal, the following sufficient condition should be satisfied:(18)∂2∂u2LH,u,s≥0.Substituting the expression foru∗(q,p,s) in (17) into (14) yields the final dynamical programming equation:(19)12σ-2H∂2VH,s∂H2+mH,s+mu∗H,s∂VH,s∂H+LH,u∗,s+∑r=1lλrsVH,r=0,where (20)mu∗H,s=u∗q,p,s∂H∂p=-12R∂VH,s∂H∂H∂p2.The value functionV(H,s) can be obtained by solving (19). The optimal control force u∗(q,p,s) is then determined by substituting V(H,s) into expression (17). By inserting the optimal control force u∗(q,p,s) into (11) and averaging the term u∗(q,p,s)∂H/∂p, the following completely averaged Itô equation is obtained:(21)dH=εmH,s+mu∗H,sdt+ε1/2σ-H,sdBt.The FPK equation associated with the fully averaged Itô equation (21) is [4, 21, 22](22)∂∂tpH,s,t=-∂∂HmH,s+mu∗H,spH,s,t+12∂2∂H2σ-H,s2pH,s,t-∑r=1r≠slλsrpH,s,t-λrspH,r,t,s∈S,where p(H,s,t) is the probability density function of total energy H for optimally controlled system (1) while the system operating in sth form (p(H,s,t) for uncontrolled system can be obtained by letting u∗(q,p,s)=0). The initial condition is(23)pH,s,0=pH0,sand boundary conditions are(24)p0,s,t=finite,pH,s,tH→∞⟶0,∂pH,s,t∂HH→∞⟶0.The FPK equation (22) is a finite set of second-order partially differential equations, which are coupled through the zero-order terms. Equation (22) does not admit easy solution, analytically or numerically. However, in practical application we are more interested in the stationary solution of FPK equation (22). In this case, FPK equation (22) is reduced by letting ∂p/∂t=0. Then, the stationary probability density p(H,s) is obtained readily from solving (22) numerically. The stationary probability density p(H) can be obtained from p(H,s) as follows:(25)pH=c∑s=1lpH,s,where c is a normalization constant. The stationary probability density p(q) of the generalized displacement and the mean value E[H] of the total energy are then obtained as(26)pq=∫-∞∞pq,pdp,EH=∫0∞HpHdH,where(27)pq,p=pHTHH=Hq,pis the stationary joint probability density of the generalized displacement and generalized momentum.


## 5. Numerical Example

To demonstrate the application and to assess the effectiveness of the proposed control strategy, numerical results are obtained for two examples.


### 5.1. Example  1

Consider a controlled van der Pol-Rayleigh oscillator subjected to both external and parametric random excitations, which is capable of independent Markovian jump and governed by(28)x¨+x=αst-βstx2+γstx˙2x˙+ux,x˙,st+h1stW1t+h2stxW2t,where α(s(t)), βst, and γ(s(t)) are Markovian jump coefficients of linear and quasi-linear damping, respectively; h1(s(t)) and h2(s(t))x denote Markovian jump coefficients of external and parametric random excitations, respectively; u(x,x˙,s(t)) is the feedback control force; Wk(t)(k=1,2) are independent Gaussian white noises with zero mean and intensities 2Dk. s(t) is a finite-state continuous-time Markov process pointing to the system’s form with the transition probability defined in (2).We consider the 3-form case here that meansl=3 and S={1,2,3}. Prescribe the transition rate λij(i,j=1,2,3) between the form i and the form j by a transition matrix Λ=[λij].Letq=x and p=x˙. The Hamiltonian associated with system (28) is(29)H=12p2+12q2.As a Hamiltonian system, (28) can be expressed as(30)q˙p˙=p-q+αst-βstq2+γstp2p+uq,p,st+h1stW1t+h2stqW2t.Upon stochastic averaging, (11) for system (28) is obtained, for which Ω={q∣H(q,0)≤H}, and(31)mH,st=i=-βi2+32γiH2+h1i2D1+αi+h2i2D2H,muH,st=i=uiq,pp,σ-2H,st=i=2h1i2D1H+h2i2D2H2,i=1,2,3.For the proposed control strategy, the partially averaged cost functionL is of the form of (16) with(32)fH,s=c0s+c1sH+c2sH2+c3sH3.Following (17), the expression of the optimal control force is(33)u∗q,p,s=-12R∂VH,s∂Hp.Substitutingu∗(q,p,s) in (33) into the dynamical programming equation (14), completing the averaging yields the following final coupled dynamical programming equations:(34)12σ-2H∂2VH,s∂H2+mH,s-14R∂VH,s∂H∂H∂p2∂VH,sdH+fH,s+∑r=13λrsVH,r=0,s=1,2,3.To solve (34), the value function V(H,s) is supposed to be of the following form:(35)VH,s=b0s+b1sH+b2sH2.Since the optimal control forceu∗ is function of ∂V/∂H, only the coefficients b1(s) and b2(s) are determined by inserting (32) and (35) into (34): (36)b1s=i=b1i=2c2+∑k=13λkib2k+2b2iαi+2D2h2i2b2iR2b2i+βiR+3γiR,b2s=i=b2i=-12βi-32γiR+12βi2R2+6βiR2γi+9γi2R2+4c3R,i=1,2,3.By substituting (35) into (17) with the coefficients determined by (36), the optimal control force is determined. Then, the following completely averaged FPK equation is obtained:(37)∂∂tpH,s,t=-∂∂HmH,s+mu∗H,spH,s,t+12∂2∂H2σ-H,s2pH,s,t-∑r=1r≠s3λsrpH,s,t-λrspH,r,t,s=1,2,3,where(38)mu∗H,s=i=-12Rb1i+2b2iHH.Solving (37) numerically with ∂p/∂t=0, the stationary probability density p(H,s) of optimal controlled system can be obtained. The stationary probability densities p(H) of the total energy, p(q) of the generalized displacement, and the mean value E[H] of the total energy are then obtained using (25) and (26).Some numerical results are obtained as shown in Figures1–4 for system parameters D1=0.03; D2=0.03; R=1.0; s1=s2=s3=1.0; Λ=-λ11λ11/2λ11/2λ22/2-λ22λ22/2λ33/2λ33/2-λ33; λ11=λ22=λ33=3.0 and(39)α1=0.01;β1=0.01;γ1=0.01;h11=2.0;h21=2.0;α2=0.01;β2=0.02;γ2=0.02;h12=2.0;h22=1.0;α3=0.01;β3=0.04;γ3=0.04;h13=1.0;h23=1.0.Figure 1
Stationary probability densityp(H) of total energy of uncontrolled and optimally controlled system (28). The lines are obtained from numerical solution of (37) while the dots are obtained from direct simulation of original system (28).Figure 2
Stationary probability densityp(q) of displacement of uncontrolled and optimally controlled system (28). The lines are obtained from numerical solution of (37) while the dots are obtained from direct simulation of original system (28).Figure 3
Mean valueE[H] of total energy of uncontrolled and optimally controlled system (28) as functions of the ratio λ11/λ22(λ22=λ33). The lines are obtained from numerical solution of (37) while the dots are obtained from direct simulation of original system (28).Figure 4
The sample time history of displacement of uncontrolled and optimally controlled system (28).The stationary probability densityp(H) of the total energy for uncontrolled and optimally controlled system (28) is shown in Figure 1. Obviously, as H increases, p(H) for optimally controlled system (28) decreases much faster than that for uncontrolled system. The stationary probability density p(q) of the displacement is plotted in Figure 2 with the same parameters as in Figure 1. For optimally controlled system, there is a much higher probability that q will be located near their equilibrium. Hence, p(q) has much larger mode and smaller dispersion around q=0 for optimally controlled system than those for uncontrolled system. This implies that the proposed control strategy has high effectiveness for attenuating the response of system (28).Figure3 shows the mean value E[H] of the total energy of uncontrolled and optimally controlled system as functions of ratio λ11/λ22(λ22=λ33). Obviously, higher value of ratio λ11/λ22 implies lower probability for the system operating in form s(t)=1 and higher probability for the system operating in forms s(t)=2 and s(t)=3. When the system operates in form s(t)=1, the system has the smallest damping coefficients and the largest amplitudes of stochastic excitations than those when it operates in forms s(t)=2 and s(t)=3. As a consequence, E[H] for uncontrolled system decreases monotonously while the ratio λ11/λ22 increases, as shown in Figure 3. However, E[H] for optimally controlled system always keeps a very low value despite changing of the ratio λ11/λ22, which indicates that the proposed control strategy is very effective and quite robust with respect to the ratio λ11/λ22. In Figures 1–3, lines are obtained by numerical solution of (37), while dots are obtained by direct simulation of system (28). Observe that the dots match closely with the corresponding lines, demonstrating the validity and accuracy of the proposed method. The displacements of the uncontrolled and optimally controlled system (28) are shown in Figure 4, from which the effect of the optimal control on the displacement can be visualized intuitively.


### 5.2. Example  2

Consider a controlled stochastically excited Duffing oscillator with Markovian jump which is governed by(40)x¨+ω2x+αx3=βstx˙+ux,x˙,st+hstξt,where β(s(t)) is Markovian jump coefficient of linear damping; h(s(t)) is Markovian jump coefficient of external random excitation; u(x,x˙,s(t)) is the feedback control force; ξ(t) is Gaussian white noise with zero mean and intensity 2D. Here, we consider the 2-form case, l=2 and S={1,2}.Upon the stochastic averaging method, the original jump system (40) can be approximately substituted by the partially averaged system (11) with(41)mH,s=i=hi2D-βiGH,muH,s=i=4TH∫0AHuiq,ppdq,σ-H,s=i2=2hi2DGH,GH=4TH∫0AH2H-ω2q2-αq421/2dq,TH=4∫0AH2H-ω2q2-αq42-1/2dq,AH=ω+4αH1/2-ω2α1/2,i=1,2.Following (17), the expression of the optimal control force is(42)u∗q,p,s=-12R∂VH,s∂Hp.Then, the following final dynamical programming equation is obtained:(43)12σ-2H∂2VH,s∂H2+mH,s-14R∂VH,s∂HGH∂VH,sdH+fH,s+∑r=12λrsVH,r=0,s=1,2.∂
V
/
∂
H can be obtained from solving (43) numerically. Then the optimal control force is determined by substituting ∂V/∂H into (17). Substituting the optimal control force (17) into the partially averaged equation (11) and completing the averaging yield the following completely averaged FPK equation:(44)∂∂tpH,s,t=-∂∂HmH,s-12R∂VH,s∂HGHpH,s,t+12∂2∂H2σ-2H,spH,s,t-∑r=1r≠s2λsrpH,s,t-λrspH,r,t,s=1,2.Solving (44) numerically with ∂p/∂t=0, the statistics p(H), p(q), and E[H] of the stationary response are then obtained from (25) and (26).The numerical results shown in Figures5–8 are for system with parameters ω=1.0; α=0.5; D=002; Λ=-λ11λ11λ22-λ122; λ11=λ22=2.0, and (45)β1=0.01;h1=2.0;β2=0.02;h2=1.0.Figure 5
Stationary probability densityp(H) of total energy of uncontrolled and optimally controlled system (40). The lines are obtained from numerical solution of (44) while the dots are obtained from direct simulation of original system (40).Figure 6
Stationary probability densityp(q) of displacement of uncontrolled and optimally controlled system (40). The lines are obtained from numerical solution of (44) while the dots are obtained from direct simulation of original system (40).Figure 7
Mean valueE[H] of total energy of uncontrolled and optimally controlled system (40) as functions of the ratio λ11/λ22. The lines are obtained from numerical solution of (44) while the dots are obtained from direct simulation of original system (40).Figure 8
The sample time history of displacements of uncontrolled and optimally controlled system (40).The stationary probability densitiesp(H) of total energy and p(q) of the displacement of uncontrolled and optimally controlled system (40) are evaluated and plotted in Figures 5 and 6, respectively. The mean value E[H] of total energy of uncontrolled and optimally controlled system (40) as functions of ratio λ11/λ22 is shown in Figure 7. It is seen that the optimal control force alleviates the response of system (40) significantly. This effect can be seen more apparently from Figure 8. Again, the analytical results obtained from solving (44) match closely with those from digital simulation of original system (40).


## 5.1. Example  1

Consider a controlled van der Pol-Rayleigh oscillator subjected to both external and parametric random excitations, which is capable of independent Markovian jump and governed by(28)x¨+x=αst-βstx2+γstx˙2x˙+ux,x˙,st+h1stW1t+h2stxW2t,where α(s(t)), βst, and γ(s(t)) are Markovian jump coefficients of linear and quasi-linear damping, respectively; h1(s(t)) and h2(s(t))x denote Markovian jump coefficients of external and parametric random excitations, respectively; u(x,x˙,s(t)) is the feedback control force; Wk(t)(k=1,2) are independent Gaussian white noises with zero mean and intensities 2Dk. s(t) is a finite-state continuous-time Markov process pointing to the system’s form with the transition probability defined in (2).We consider the 3-form case here that meansl=3 and S={1,2,3}. Prescribe the transition rate λij(i,j=1,2,3) between the form i and the form j by a transition matrix Λ=[λij].Letq=x and p=x˙. The Hamiltonian associated with system (28) is(29)H=12p2+12q2.As a Hamiltonian system, (28) can be expressed as(30)q˙p˙=p-q+αst-βstq2+γstp2p+uq,p,st+h1stW1t+h2stqW2t.Upon stochastic averaging, (11) for system (28) is obtained, for which Ω={q∣H(q,0)≤H}, and(31)mH,st=i=-βi2+32γiH2+h1i2D1+αi+h2i2D2H,muH,st=i=uiq,pp,σ-2H,st=i=2h1i2D1H+h2i2D2H2,i=1,2,3.For the proposed control strategy, the partially averaged cost functionL is of the form of (16) with(32)fH,s=c0s+c1sH+c2sH2+c3sH3.Following (17), the expression of the optimal control force is(33)u∗q,p,s=-12R∂VH,s∂Hp.Substitutingu∗(q,p,s) in (33) into the dynamical programming equation (14), completing the averaging yields the following final coupled dynamical programming equations:(34)12σ-2H∂2VH,s∂H2+mH,s-14R∂VH,s∂H∂H∂p2∂VH,sdH+fH,s+∑r=13λrsVH,r=0,s=1,2,3.To solve (34), the value function V(H,s) is supposed to be of the following form:(35)VH,s=b0s+b1sH+b2sH2.Since the optimal control forceu∗ is function of ∂V/∂H, only the coefficients b1(s) and b2(s) are determined by inserting (32) and (35) into (34): (36)b1s=i=b1i=2c2+∑k=13λkib2k+2b2iαi+2D2h2i2b2iR2b2i+βiR+3γiR,b2s=i=b2i=-12βi-32γiR+12βi2R2+6βiR2γi+9γi2R2+4c3R,i=1,2,3.By substituting (35) into (17) with the coefficients determined by (36), the optimal control force is determined. Then, the following completely averaged FPK equation is obtained:(37)∂∂tpH,s,t=-∂∂HmH,s+mu∗H,spH,s,t+12∂2∂H2σ-H,s2pH,s,t-∑r=1r≠s3λsrpH,s,t-λrspH,r,t,s=1,2,3,where(38)mu∗H,s=i=-12Rb1i+2b2iHH.Solving (37) numerically with ∂p/∂t=0, the stationary probability density p(H,s) of optimal controlled system can be obtained. The stationary probability densities p(H) of the total energy, p(q) of the generalized displacement, and the mean value E[H] of the total energy are then obtained using (25) and (26).Some numerical results are obtained as shown in Figures1–4 for system parameters D1=0.03; D2=0.03; R=1.0; s1=s2=s3=1.0; Λ=-λ11λ11/2λ11/2λ22/2-λ22λ22/2λ33/2λ33/2-λ33; λ11=λ22=λ33=3.0 and(39)α1=0.01;β1=0.01;γ1=0.01;h11=2.0;h21=2.0;α2=0.01;β2=0.02;γ2=0.02;h12=2.0;h22=1.0;α3=0.01;β3=0.04;γ3=0.04;h13=1.0;h23=1.0.Figure 1
Stationary probability densityp(H) of total energy of uncontrolled and optimally controlled system (28). The lines are obtained from numerical solution of (37) while the dots are obtained from direct simulation of original system (28).Figure 2
Stationary probability densityp(q) of displacement of uncontrolled and optimally controlled system (28). The lines are obtained from numerical solution of (37) while the dots are obtained from direct simulation of original system (28).Figure 3
Mean valueE[H] of total energy of uncontrolled and optimally controlled system (28) as functions of the ratio λ11/λ22(λ22=λ33). The lines are obtained from numerical solution of (37) while the dots are obtained from direct simulation of original system (28).Figure 4
The sample time history of displacement of uncontrolled and optimally controlled system (28).The stationary probability densityp(H) of the total energy for uncontrolled and optimally controlled system (28) is shown in Figure 1. Obviously, as H increases, p(H) for optimally controlled system (28) decreases much faster than that for uncontrolled system. The stationary probability density p(q) of the displacement is plotted in Figure 2 with the same parameters as in Figure 1. For optimally controlled system, there is a much higher probability that q will be located near their equilibrium. Hence, p(q) has much larger mode and smaller dispersion around q=0 for optimally controlled system than those for uncontrolled system. This implies that the proposed control strategy has high effectiveness for attenuating the response of system (28).Figure3 shows the mean value E[H] of the total energy of uncontrolled and optimally controlled system as functions of ratio λ11/λ22(λ22=λ33). Obviously, higher value of ratio λ11/λ22 implies lower probability for the system operating in form s(t)=1 and higher probability for the system operating in forms s(t)=2 and s(t)=3. When the system operates in form s(t)=1, the system has the smallest damping coefficients and the largest amplitudes of stochastic excitations than those when it operates in forms s(t)=2 and s(t)=3. As a consequence, E[H] for uncontrolled system decreases monotonously while the ratio λ11/λ22 increases, as shown in Figure 3. However, E[H] for optimally controlled system always keeps a very low value despite changing of the ratio λ11/λ22, which indicates that the proposed control strategy is very effective and quite robust with respect to the ratio λ11/λ22. In Figures 1–3, lines are obtained by numerical solution of (37), while dots are obtained by direct simulation of system (28). Observe that the dots match closely with the corresponding lines, demonstrating the validity and accuracy of the proposed method. The displacements of the uncontrolled and optimally controlled system (28) are shown in Figure 4, from which the effect of the optimal control on the displacement can be visualized intuitively.


## 5.2. Example  2

Consider a controlled stochastically excited Duffing oscillator with Markovian jump which is governed by(40)x¨+ω2x+αx3=βstx˙+ux,x˙,st+hstξt,where β(s(t)) is Markovian jump coefficient of linear damping; h(s(t)) is Markovian jump coefficient of external random excitation; u(x,x˙,s(t)) is the feedback control force; ξ(t) is Gaussian white noise with zero mean and intensity 2D. Here, we consider the 2-form case, l=2 and S={1,2}.Upon the stochastic averaging method, the original jump system (40) can be approximately substituted by the partially averaged system (11) with(41)mH,s=i=hi2D-βiGH,muH,s=i=4TH∫0AHuiq,ppdq,σ-H,s=i2=2hi2DGH,GH=4TH∫0AH2H-ω2q2-αq421/2dq,TH=4∫0AH2H-ω2q2-αq42-1/2dq,AH=ω+4αH1/2-ω2α1/2,i=1,2.Following (17), the expression of the optimal control force is(42)u∗q,p,s=-12R∂VH,s∂Hp.Then, the following final dynamical programming equation is obtained:(43)12σ-2H∂2VH,s∂H2+mH,s-14R∂VH,s∂HGH∂VH,sdH+fH,s+∑r=12λrsVH,r=0,s=1,2.∂
V
/
∂
H can be obtained from solving (43) numerically. Then the optimal control force is determined by substituting ∂V/∂H into (17). Substituting the optimal control force (17) into the partially averaged equation (11) and completing the averaging yield the following completely averaged FPK equation:(44)∂∂tpH,s,t=-∂∂HmH,s-12R∂VH,s∂HGHpH,s,t+12∂2∂H2σ-2H,spH,s,t-∑r=1r≠s2λsrpH,s,t-λrspH,r,t,s=1,2.Solving (44) numerically with ∂p/∂t=0, the statistics p(H), p(q), and E[H] of the stationary response are then obtained from (25) and (26).The numerical results shown in Figures5–8 are for system with parameters ω=1.0; α=0.5; D=002; Λ=-λ11λ11λ22-λ122; λ11=λ22=2.0, and (45)β1=0.01;h1=2.0;β2=0.02;h2=1.0.Figure 5
Stationary probability densityp(H) of total energy of uncontrolled and optimally controlled system (40). The lines are obtained from numerical solution of (44) while the dots are obtained from direct simulation of original system (40).Figure 6
Stationary probability densityp(q) of displacement of uncontrolled and optimally controlled system (40). The lines are obtained from numerical solution of (44) while the dots are obtained from direct simulation of original system (40).Figure 7
Mean valueE[H] of total energy of uncontrolled and optimally controlled system (40) as functions of the ratio λ11/λ22. The lines are obtained from numerical solution of (44) while the dots are obtained from direct simulation of original system (40).Figure 8
The sample time history of displacements of uncontrolled and optimally controlled system (40).The stationary probability densitiesp(H) of total energy and p(q) of the displacement of uncontrolled and optimally controlled system (40) are evaluated and plotted in Figures 5 and 6, respectively. The mean value E[H] of total energy of uncontrolled and optimally controlled system (40) as functions of ratio λ11/λ22 is shown in Figure 7. It is seen that the optimal control force alleviates the response of system (40) significantly. This effect can be seen more apparently from Figure 8. Again, the analytical results obtained from solving (44) match closely with those from digital simulation of original system (40).


## 6. Conclusions

In this paper, we proposed a nonlinear stochastic optimal control strategy for SDOF stochastically excited Markovian jump systems where the jump parameter takes on values in a finite discrete set. In the slow jump case, the original system was first reduced to one governed by a finite set of one-dimensional partially averaged Itô equation for total energy upon stochastic averaging. Using the stochastic dynamical programming principle and Markovian jump rules, a finite set of coupled dynamical programming equations was derived, from which the optimal control force was obtained. The obtained optimal control force has been proved to be the quasi-optimal control for the original system. The strategy was applied to two nonlinear oscillators that were capable of independent Markovian jumps. Numerical results showed that the strategy is fairly robust and effective in reduction of stationary response of the controlled system. Thus, it is potentially promising for practical control applications after further research.The proposed method has the potential to be extended to the optimal control for Multi-DOF Markovian jump systems. However, to obtain the concrete conclusion much work has to be done. Mathematical tools should be developed to solve the dynamical programming equation (14) for Multi-DOF case.



---

*Source: 9641075-2015-12-27.xml*