# Probability-Weighted Optimal Control for Nonlinear Stochastic Vibrating Systems with Random Time Delay

Authors: R. C. Hu; Q. F. Lü; X. F. Wang; Z. G. Ying; R. H. Huan
Journal: Shock and Vibration
 (2018)

---

## Abstract

A probability-weighted optimal control strategy for nonlinear stochastic vibrating systems with random time delay is proposed. First, by modeling the random delay as a finite state Markov process, the optimal control problem is converted into the one of Markov jump systems with finite mode. Then, upon limiting averaging principle, the optimal control force is approximately expressed as probability-weighted summation of the control force associated with different modes of the system. Then, by using the stochastic averaging method and the dynamical programming principle, the control force for each mode can be readily obtained. To illustrate the effectiveness of the proposed control, the stochastic optimal control of a two degree-of-freedom nonlinear stochastic system with random time delay is worked out as an example.


---

## Body

## 1. Introduction

In many complicated control systems, such as manufacturing plants, vehicles, aircraft, and spacecraft, a communication network is used to gather sensor data and send control signals. Time delay occurs while exchanging data among these devices, which is usually random [1–3]. Time delay, particularly random time delay, can dramatically degrade the performance of the control system and even destabilize it. Therefore, time delay should be taken into account when designing control strategy.The control problem of linear or nonlinear systems with constant time delay has been examined in the literatures [4–7]. Far less known is about the systems with time-varying delay [8, 9], especially random time delay. A key difficulty is how to model the random time delay. A simple method was to regard the delay time as a constant [10]. However, the delay time is much longer than necessary, which usually results in poor performance. Randomness of the delay time was taken into account by Nilsson et al. [11]. To keep the model simple for analyzing, the current delay is assumed to be independent of previous delay times. However, in real system, the current delay is usually correlated with the last one. Thus, a more reasonable way is to model the random delay time as a Markov jump process [12]. Based on this model, a finite horizon optimal control of linear systems with randomly varying time delay is studied [13]. Zhang et al. [14] investigated feedback stabilization of random-time-delayed linear system. Wu et al. [15] analyzed the stability of the stochastic linear system with time-varying delay. Wang et al. [16] investigated the problem of robust fault detection for networked Markov jump systems with random time delay. The problem of robust H∞ control for a class of uncertain stochastic system with random delay against actuator failures is studied by Sakthivel et al. [17]. Huan et al. [18] investigated the dynamics of nonlinear stochastic systems with random time delay. The previous works focused mainly on the linear systems. Far less known is about the nonlinear systems.In this paper, we present a probability-weighted optimal control strategy for nonlinear stochastic systems with random time delay. The organization of this paper is as follows: in Section2, such a system is converted into the Markovian jump one by modeling the random delay as a Markov process. Section 3 converted the system into the one with time delay as a parameter. Probability-weighted optimal control law is determined in Section 4. In Section 5, the application and effectiveness of the proposed procedure are demonstrated by an example. A summary of findings is given in Section 6.


## 2. Formulation of Problem

### 2.1. Modeling of Random Delay

For many complex systems, i.e., networked control systems, the random delay timeτt usually has finite number of levels due to different network traffic conditions [11]. τt may switch randomly from one level to another with time. Usually the switching of different delay levels follows the rules of Markov chain. Then, the random time delay can be modeled as the following Markov jump form:(1)τt=τst,where st is a finite state Markov process, which points to the mode of time delay. st takes discrete values from a finite set S=1,2,…,l with the following mode transition probability [18, 19](2)Pst+Δt=jst=i=λijΔt+oΔt,i≠j,1+λiiΔt+oΔt,i=j,where Δt>0 is a sufficiently small time interval; limΔt⟶0oΔt/Δt=0; Pst+Δt=jst=i denotes the transition probability from mode i to mode j; λij≥0 is the transition rate; and λii=−∑j=1j≠ilλij.


### 2.2. Nonlinear Stochastic System with Random Delay

Consider a multi-degree-of-freedom (MDOF) nonlinear stochastic system with random time delay. The motion equation of the nonlinear stochastic system is of the following form:(3)X¨i+giXi=εfijX,X˙+εU˜iXτ,X˙τ+ε1/2hikX,X˙ξkt,i,j=1,2,…,n,k=1,2,…,m,where giXi is the nonlinear restoring force; ε is a small parameter; fijX,X˙ denotes coupled nonlinear damping; hikX,X˙ represent the coefficients of random excitations; and ξkt are Gaussian white noises in the sense of Stratonovich with zero mean and correlation functions Eξktξlt+T=2DklδT. U˜iXτ,X˙τ=U˜iXt−τt,X˙t−τt are feedback control forces with random delay time. By using Equation (1), system (3) can be rewritten as the following Markov jump form:(4)X¨i+giXi=εfijX,X˙+εU˜iXt−τst,X˙t−τst+ε1/2hikX,X˙ξkt.Note that by modeling the random delay time as a finite state Markov process, system (3) has been formulated as a Markov jump system with finite mode. The optimal feedback control problem of system (3) then can be studied in the framework of the Markov jump system.The purpose of the present study is to derive an optimal feedback control law to minimize the response of system (3) or (4). For semiinfinite time interval control, the performance index can be expressed as follows:(5)J=limtf⟶∞1tf∫0tfLX,X˙,U˜Xt−τst,X˙t−τstdt,where L is called cost function and tf is terminal time, U˜=U˜1,U˜2,⋯,U˜nT.


## 2.1. Modeling of Random Delay

For many complex systems, i.e., networked control systems, the random delay timeτt usually has finite number of levels due to different network traffic conditions [11]. τt may switch randomly from one level to another with time. Usually the switching of different delay levels follows the rules of Markov chain. Then, the random time delay can be modeled as the following Markov jump form:(1)τt=τst,where st is a finite state Markov process, which points to the mode of time delay. st takes discrete values from a finite set S=1,2,…,l with the following mode transition probability [18, 19](2)Pst+Δt=jst=i=λijΔt+oΔt,i≠j,1+λiiΔt+oΔt,i=j,where Δt>0 is a sufficiently small time interval; limΔt⟶0oΔt/Δt=0; Pst+Δt=jst=i denotes the transition probability from mode i to mode j; λij≥0 is the transition rate; and λii=−∑j=1j≠ilλij.


## 2.2. Nonlinear Stochastic System with Random Delay

Consider a multi-degree-of-freedom (MDOF) nonlinear stochastic system with random time delay. The motion equation of the nonlinear stochastic system is of the following form:(3)X¨i+giXi=εfijX,X˙+εU˜iXτ,X˙τ+ε1/2hikX,X˙ξkt,i,j=1,2,…,n,k=1,2,…,m,where giXi is the nonlinear restoring force; ε is a small parameter; fijX,X˙ denotes coupled nonlinear damping; hikX,X˙ represent the coefficients of random excitations; and ξkt are Gaussian white noises in the sense of Stratonovich with zero mean and correlation functions Eξktξlt+T=2DklδT. U˜iXτ,X˙τ=U˜iXt−τt,X˙t−τt are feedback control forces with random delay time. By using Equation (1), system (3) can be rewritten as the following Markov jump form:(4)X¨i+giXi=εfijX,X˙+εU˜iXt−τst,X˙t−τst+ε1/2hikX,X˙ξkt.Note that by modeling the random delay time as a finite state Markov process, system (3) has been formulated as a Markov jump system with finite mode. The optimal feedback control problem of system (3) then can be studied in the framework of the Markov jump system.The purpose of the present study is to derive an optimal feedback control law to minimize the response of system (3) or (4). For semiinfinite time interval control, the performance index can be expressed as follows:(5)J=limtf⟶∞1tf∫0tfLX,X˙,U˜Xt−τst,X˙t−τstdt,where L is called cost function and tf is terminal time, U˜=U˜1,U˜2,⋯,U˜nT.


## 3. Converted System with Time Delay as a Parameter

System (4) has the following randomly periodic solutions [5]:(6)Xit=AicosΦi,X˙it=−AiVisinΦi,Φit=Θit+Γit,Vi=dΘitdt,where Vi denotes the instantaneous frequency and Φi, Ai, Θi, Vi, and Γi are all random processes.For the case of small time delay, i.e.,0≤τu≪1 (u∈S), it is proved that the following approximate expressions hold [5]:(7)Xit−τst=cosωiτstXit−sinωiτstωiX˙it,X˙it−τst=ωisinωiτstXit+cosωiτstX˙it,where ωi=ωiAi is the averaging frequency which is the average of the instantaneous frequency Vi, i.e.,(8)ωiAi=dΘiAi,ΦidtΦi=2GiAi−GiAicosΦiAi2sin2ΦiΦi,GiAi=∫0Aigiuidui,and Φi represents the averaging with respect to Φi in 0,2π.According to expression (7), the time-delayed control forces can be approximately expressed in terms of the state variables with time delay as a parameter, i.e., U˜iXt−τst,X˙t−τst=U˜iXt,X˙t;τst.Then, Equation (4) becomes(9)X¨i+giXi=εfijX,X˙+εU˜iX,X˙;τst+ε1/2hikX,X˙ξkt.


## 4. Probability-Weighted Optimal Control Law

According to limiting averaging principle [20, 21], the solution of Equation (9) converges in probability to the solution of the following probability-weighted averaged equations as ε⟶0:(10)X¨i+giXi=εfijX,X˙+εUiX,X˙;T+ε1/2hikX,X˙ξkt,i=1,2,…,n,where T=τ1,τ2,⋯,τlT and UiX,X˙;T is probability-weighted control force, given by(11)UiX,X˙;T=∑u∈SU˜′iX,X˙;τu⋅μu,where U˜′iX,X˙;τu is the control force when st is fixed at st=u and μu is the stationary probability distribution of st for st=u, which satisfies [18].(12)∑u=1lμuλui=0,i∈S,where λui is transition rate given in Equation (2). Combining the normalizing condition ∑u=1lμu=1, the stationary probability distribution μu can be calculated.Since probability-weighted averaged equation (10) is an approximation of the original system (3), the optimal control force Ui∗ in Equation (11) is near optimal for the original control problem.Own to the relationship in Equation (11), the optimal control force U˜′i∗X,X˙;τu for a fixed mode should be determined first. Let the Markov jump process be arbitrarily fixed at st=u, Equation (9) can be rewritten in the fix-mode quasi-Hamiltonian form:(13)Q˙i=Pi,P˙i=−giQi+εfijQ,P+εU˜′iQ,P;τu+ε1/2hikQ,Pξkt,i,j=1,2,…,n,k=1,2,…,m,where Xi=Qi, X˙i=Pi. The Hamiltonian system associated with Equation (13) is fully integrable. The Hamiltonian H is(14)H=∑i=1nHiQi,Pi,where Hi=1/2Pi2+∫0Qigividvi are called first integrals.According to the stochastic averaging method for quasi-integrable Hamiltonian systems [22], the partially averaged Itô equations associated with system (13) are obtained:(15)dHr=mrH+U˜′iQ,P;τu∂Hr∂Pidt+σrkHdBktr,i=1,2,…,n,k=1,2,…,m,where H=H1,H2,⋯,HnT and mrH and σrkH are the averaged drifts and diffusion coefficients respectively, given by(16)arH=mrH=1T∮−fij∂H∂Pj∂Hr∂Pi+12hikhjk∂2Hr∂Pi∂Pj∏l=1n1∂Hr/∂PldQl,brsH=σrkHσrsH=1T∮∏l=1n12hikhjk∂Hr/∂Pj∂Hr/∂Pi∂Hr/∂PldQl,T=TH=∏l=1n∮1∂Hr/∂PldQl,⋅=1T∏l=1n∮⋅∂Hr/∂PldQl.Accordingly, the performance index in Equation (5) becomes(17)J=limtf⟶∞1tf∫0tfLH,U˜′Q,P;τudt.Based on the stochastic dynamical programming principle, the following dynamical programming equation (DPE) can be established for averaged system (15) and performance index (17):(18)γ=minu˜′LH,U˜′Q,P;τu+mrH+U˜′iQ,P;τu∂Hr∂Pi∂V∂Hr+brs2∂2V∂Hr∂Hs,where V=VH is value function and γ=limtf⟶∞1/tf∫0tfLH,U˜′∗ds is optimal average cost. The optimal control force U˜′i∗ then can be obtained by minimizing the right-hand side of Equation (18), i.e.,(19)∂∂u˜′iLH,U˜′+U˜i∂Hr∂PidVdHr=0,i=1,2,⋯,n.Suppose thatLH,U˜′ has the form(20)LH,U˜′=L1H+U˜′TRU˜′,where L1H≥0 and R is a positive-definite symmetric matrix. Substituting Equation (20) into Equation (19) and exchanging the order of the deriving with respect to U˜′i and averaging, the optimal control force for a fixed mode of system (13) can be obtained:(21)U˜′i∗Q,P;τu=−12R−1ij∂V∂HrPj,where V is undetermined. By inserting U˜′i∗ into Equation (18) for replacing U˜′i, the final DPE can be obtained. V will be then determined by solving this equation.Note that the optimal control force in Equation (21) is a function of current system’s states Pjt. However, in system (4), the delayed states are observed. Thus, the optimal control force U˜′i∗ should be expressed in functions of delayed states Pjt−τu by using approximation relationship in Equation (7):(22)U˜′i∗=−12R−1ij∂V∂Hr−ωjsinωjτuQjt−τu+cosωjτuPjt−τu.Substituting Equation (22) into Equation (11), the optimal control force of the original system (4) is then obtained:(23)Ui∗=∑u∈SU˜′i∗Q,P;τu⋅μu=−12R−1ij∂V∂Hr∑u∈S−ωjsinωjτuPjt−τu+cosωjτuPjt−τu⋅μu.It is seen that the optimal control forceUi∗ is probability-weighted summation of the control force associated with different modes of the system, weighted by the stationary probabilities μu. The obtained optimal control force is continuous and independent of the Markov jump parameter, which can be easily executed by actuators.InsertingUi∗ in Equation (23) into system (15) for replacing U˜′i, the optimal controlled system is obtained. The response of the optimal controlled system will be predicted by solving the associated Fokker–Planck–Kolmogorov (FPK) equation or by using Monte Carlo simulation directly from the original system (4). The control effectiveness and control efficiency can then be evaluated (see the following example, i.e., Equations (32) and (33)).


## 5. Numerical Example

Consider two oscillators coupled by linear and polynomial type nonlinear dampings subject to external random excitations and random time-delayed feedback control. The motion equations of the system are of the following form:(24)X¨1+α11X˙1+α12X˙2+β1X˙1X˙12+X˙22+ω12X˙1=U1τ+k11W1t+k12W2t,X¨2+α21X˙1+α22X˙2+β2X˙2X˙12+X˙22+ω22X˙2=U2τ+k21W1t+k22W2t,where Wit (i=1,2) are uncorrelated Gaussian white noises with intensities 2Di and Uiτ are control forces with random time delay.Three-level time delay is considered here, which meansst takes values in set S=1,2,3: τ1 for low network load, τ2 for medium network load, and τ3 for high network load [10]. Then, system (24) is converted into a Markov jump system (4) with three modes. Prescribe the transition matrix of st by Λ=λij (i,j=1,2,3). The stationary probabilities μu (u∈S) of st can be calculated from Equation (12).LetXi=Qi, X˙i=Pi. By using the approximation in Equation (7) and applying the stochastic averaging method, the partially averaged Itô stochastic differential equations for a fixed mode can be obtained as the form of Equation (15):(25)dHi=miH+U′iQ,P;τu∂Hi∂Pidt+σiHdBit,i=1,2,where H=H1,H2T, Hi=1/2Pi2+1/2ωi2Qi2 and(26)m1H=k112D1+k122D2−α11H1−β1H1H12ω12+H22ω22,m2H=k212D1+k222D2−α22H2−β2H2H12ω12+H22ω22,σ12H=2k112D1+k122D2H1,σ22H=2k212D1+k222D2H2.Based on the dynamical programming principle, the DPE is derived as the form of Equation (18), from which the optimal control force for a fixed mode can be obtained as the form of Equation (21), i.e.,(27)U′i∗Q,P;τu=−12R−1ij∂V∂HrPj.To obtain the value of function∂V/∂Hr, the value function V is assumed to have the following polynomial form:(28)VH=c1H1+c2H2+c3H12+c4H1H2+c5H22.L
1
H in Equation (20) is specified by(29)L1H=s00+s10H1+s01H2+s20H12+s11H1H2+s02H22+s30H13+s21H12H2+s21H1H23+s03H23.Inserting Equations (27)–(29) into the DPE (18), the coefficients of value function V can be determined.(30)c1=2α112+R−1114D1c3k12+s111/2−α11R−111,c2=2α222+R−1224D2c5k22+s121/2−α22R−122,c3=β12+4s31k12R−1111/2−β12k1R−111,c4=0,c5=β22+4s34k22R−1221/2−β22k2R−122,R11−1=R22R11R22−R122,R22−1=R11R11R22−R122.Substituting Equation (27) into Equation (23), the probability-weighted optimal control force of system (24) is then obtained(31)Ui∗=−12R−1ij∂V∂Hj∑u∈S−ωjsinωjτuQjt−τu+cosωjτuPjt−τu⋅μu.To evaluate the proposed control strategy, the control effectiveness and control efficiency are proposed:(32)Ki=EQi2u−EQi2cEQi2u×100%,(33)μi=KiEui∗2/2Dii.They are the relative reduction in root-mean-square displacement and the ratio of the control effectiveness to the normalized root-mean-square control force, respectively. Obviously, higherKi and μi values indicate a better control strategy.Some numerical results have been calculated as shown in Figures1–4 for system’s parameters: α11=0.1, α12=0.0, α21=0.0, α22=0.1, β1=0.1, β2=0.2, ω1=1.0, ω1=1.414, k11=1.0, k12=0.0, k21=0.0, k22=1.0, D1=0.05, and D2=0.05. The three-level time delay is specified by(34)τ1=0,τ2=0.2,τ3=0.4.Figure 1
The stationary joint probability densitypuQ1,P1 of the first oscillator of uncontrolled system and pcQ1,P1 of optimal controlled system: (a, c) results by the proposed method; (b, d) results by Monte Carlo simulation of the original system (24).
(a)
(b)
(c)
(d)Figure 2
The stationary probability densitypQ1 of displacement of the first oscillator of optimal controlled system compared with uncontrolled system. The lines are obtained by proposed method, while the dots are obtained from Monte Carlo simulation of the original system (24).Figure 3
The stationary probability densitypQ2 of displacement of the second oscillator of optimal controlled system compared with uncontrolled system. The lines are obtained by proposed method, while the dots are obtained from Monte Carlo simulation of the original system (24).Figure 4
The control effectivenessK1 and control efficiency μ1 versus stochastic excitations intensity 2D1 for different transition rules.
(a)
(b)Three special Markov jump rules are considered, with(35)Λ1=−10.50.52−4222−4,Λ2=−4220.5−10.522−4,Λ3=−4222−420.50.5−1.Observe thatλ11<λ22=λ33 if Λ=Λ1 so that the system is more likely to take the low time delay mode st=1. The system favors the medium time delay mode st=2 if Λ=Λ2, while it favors the large time delay mode st=3 if Λ=Λ3.The stationary joint probability densitiespuQ1,P1 of the first oscillator of the uncontrolled system and pcQ1,P1 of the optimal controlled system are, respectively, shown in Figures 1(a) and 1(c). Obviously, pcQ1,P1 has much larger mode and smaller dispersion around their equilibrium for optimally controlled system than those for the uncontrolled system. This implies that the proposed control law has high effectiveness for attenuating the system’s response. The results of direct Monte Carlo simulation of system (24) are also obtained and shown in Figures 1(b) and 1(d). Favorable agreements between these two results can be seen, which demonstrates the validity of the proposed method.Figure2 shows the stationary probability density pQ1 of displacement of the first oscillator of the optimal controlled system for different transition rules. It can be seen that pQ1 has the largest mode around Q1=0 if Λ=Λ1, and the mode will decrease as the system cycles through Λ=Λ1, Λ=Λ2, and Λ=Λ3. This implies that the higher probability of the system working in large time delay mode will lead to worse performance of the control force. In Figure 2, the lines are obtained by the proposed method, while the dots are obtained by direct simulation of system (24). Observe that the dots match closely with the corresponding lines. The same observation can be made about pQ2 in Figure 3.In Figure4, the variation of the control effectiveness K1 and control efficiency μ1 with excitation intensity 2D1 is displayed for different transition rules. It is seen that the proposed control strategy keeps high control effectiveness and control efficiency for varying excitation intensities. It is also seen from Figure 3 that as Λ takes values through Λ=Λ1, Λ=Λ2, and Λ=Λ3 and K1 and μ1 decrease monotonously. The observation can be explained in a similar fashion as in Figure 2. Figure 5 shows the control effectiveness K2 and control efficiency μ2 for varying excitation intensity 2D2.Figure 5
The control effectivenessK2 and control efficiency μ2 versus stochastic excitations intensity 2D2 for different transition rules.
(a)
(b)Finally, sample time histories of the displacementQ1 for the controlled system compared with uncontrolled system are displayed in Figure 6.Figure 6
The sample time history of displacementQ1 for controlled system compared with uncontrolled system.


## 6. Conclusions

In this paper, a probability-weighted optimal control strategy for nonlinear systems with random time delay is proposed. Based on the random switch model of random time delay, the optimal control problem of such a system was converted into the one in framework of Markov jump systems. Upon limiting averaging principle, the optimal control force was approximately expressed as probability-weighted summation of the control force associated with different modes of the system. The proposed optimal control force is continuous and independent of the Markov jump parameter, which can be easily executed by actuators. The feasibility and effectiveness of the proposed optimal control strategy were demonstrated by dealing with an example of a two-DOF nonlinear stochastic system with random time delay.



---

*Source: 5636541-2018-09-25.xml*