# Optimal Preview Control for a Class of Linear Continuous Stochastic Control Systems in the Infinite Horizon

Authors: Jiang Wu; Fucheng Liao; Jiamei Deng
Journal: Mathematical Problems in Engineering
 (2016)

---

## Abstract

This paper discusses the optimal preview control problem for a class of linear continuous stochastic control systems in the infinite horizon, based on the augmented error system method. Firstly, an assistant system is designed and the state equation is translated to the assistant system. Then, an integrator is introduced to construct a stochastic augmented error system. As a result, the tracking problem is converted to a regulation problem. Secondly, the optimal regulator is solved based on dynamic programming principle for the stochastic system, and the optimal preview controller of the original system is obtained. Compared with the finite horizon, we simplify the performance index. We also study the stability of the stochastic augmented error system and design the observer for the original stochastic system. Finally, the simulation example shows the effectiveness of the conclusion in this paper.


---

## Body

## 1. Introduction

Future reference signals or disturbance signals are known in certain circumstances. All the known future information can be utilized by preview control theory to improve the performance of the dynamic system. The original idea of preview control theory is that in order to minimize the error between the reference signals and the controlled terms, not only the past and present information but also the future information should be concentrated on [1–5]. An augmented error system is constructed while designing the optimal preview controller for the discrete system. And also a group of identical equations of future reference signals is added to the augmented error system [6–9]. Since the continuous system is of infinite dimensions, the method in dealing with the discrete condition is no longer useful. In [10], the augmented error system was constructed by differentiating the state equation on both sides, combining with the error equation. According to the maximum principle, the optimal preview controller was obtained by solving a differential equation on reference signals backward in time. This method was extended to systems with previewable disturbance signals in [11] and to singular continuous systems in [12]. The application of preview control method to continuous systems with time delay is studied in [13].According to automatic control system theory, the controlled systems can be regarded as falling into two categories: deterministic systems and stochastic systems. Stochastic systems are a collection of dynamic systems that contain internal stochastic parameters, external stochastic disturbances, or observation noises [14]. A typical stochastic system is the stochastic differential equation. It is a class of differential equations driven by one or more stochastic processes. Therefore, the solution of a stochastic differential equation is also a stochastic process. Up to the present, a phenomenon such as stock market volatility or thermal motion in a physical system is usually described by a stochastic differential equation. Typically, a white noise stochastic variable represented by the differential form of Brownian motion or Weiner process is contained in a stochastic differential equation.In this paper, preview control theory is applied to a class of linear continuous stochastic control systems in the infinite horizon. In the finite horizon [15], an assistant system is designed, and the state equation is translated to the assistant system. Then an integrator is introduced to construct a stochastic augmented error system. As a result, the tracking problem is converted to a regulation problem. The optimal regulator is obtained based on the dynamic programming principle for stochastic systems, which means the optimal preview controller of the original stochastic system is gained. For such a system, when the integrator approaches zero at infinity, the error also approaches zero. Therefore, this property can be utilized to simplify the performance index in the infinite horizon. Due to the fact that the relative terms of reference signals are included in the stochastic augmented error system, the conclusion in the infinite horizon cannot be directly employed when solving the optimal regulation problem in this paper. Firstly, the corresponding optimal regulation problem is solved in the finite horizon with the new performance index and then the time was set to approach infinity. The stability of the stochastic augmented error system is studied and the sufficient and necessary criteria which guarantee that there exists a unique semipositive definite solution to the corresponding Riccati equation have been met. Also an observer for the original stochastic system is designed. The introduction of the integrator can eliminate the static error and the simulation example shows the effectiveness of the conclusion in this paper [16].Notation. The notations are standard. (Ω,F,P) is a complete probability space. Adaptive procedure Ft is σ algebra generated by {Bs:s≤t}, and Bs denotes Brownian motion of m′ dimensions. A∈Rm×n denotes m×n matrix. P>0(P≥0) denotes a positive definite (semidefinite) matrix. I denotes the unit matrix. The symbol ∗ denotes the symmetric terms in a symmetric matrix. Et0,x0 denotes the expectation of process (t,x) with initial time t0 and state x0. tr(·) denotes the trace of a matrix.


## 2. Problem Statement

Consider the following stochastic control system:(1)dxt=Mxt+Nutdt+ωtdBt,yt=Cxt,xt0=x0,t∈t0,∞,where x(t) is the state vector of m dimensions, y(t) is the output vector of p dimensions, u(t) is the Ft adaptive input of n dimensions, and Bt is the Brownian motion of Ft adaptive of m′ dimensions. M∈Rm×m, N∈Rm×n, and C∈Rp×m are constant matrices. ω(t)∈Rm×m′.First of all, the following assumptions are introduced.Assumption 1.
Suppose the matrices pair(M,N) is stabilizable.Assumption 2.
Suppose the matrices pair(C,M) is detectable and the matrix C is of full row rank.By denoting the reference signals ofp dimensions with r(t), another assumption about r(t) is needed.Assumption 3.
Suppose the reference signalr(t) is a piecewise continuously differentiable function defined on (t≥t0). Moreover, r(ρ) (t≤ρ≤t+lr) is available at any moment t and when ρ>t+lr, r(ρ)=r(t+lr); that is, r˙(ρ)=0. lr is the preview length of the reference signal.Remark 4.
Since there are few effects on the system when the reference signals exceed the previewable range, it is always assumed that the signals are usually constant [11].
The tracking error signale(t) is defined as the difference between the output vector y(t) and the reference signal r(t); that is, (2)et=yt-rt.In this paper, an optimal controller will be designed to make the outputy(t) in (1) tracking the reference signal r(t) as accurately as possible without static error.Then, an assistant system is defined. The following assumption is needed.Assumption 5.
Suppose(3)rankMNC0=m+p,fullrowrank.With Assumption5, the following lemma holds.Lemma 6 (see [15]).
There exist matrices pairs(Γ,γ) which solve the following:(4)MΓ+Nγ=0,CΓ=I,if and only if Assumption 5 holds, where Γ∈Rm×p and γ∈Rn×p.Definition 7 (see [17]).
If matrixA∈Rm×n and the rank of A is equal to n, then A has a left inverse: B∈Rn×m such that BA=I.Remark 8.
Ifp=n, there exists unique solution of matrices pair (Γ,γ), while if p<n, according to Definition 7, any left inverse of matrix MNC0 can be applied to calculate the solution of (4) and the solutions are infinite. In this paper only the square full-rank case is used.
Let the state vector and input vector be denoted byx∗(t) and u∗(t), respectively. Let x∗(t)=Γr(t+lr) and u∗(t)=γr(t+lr). According to Lemma 6, at the moment s, for any time t(s≤t≤tτ), x∗(t) meets(5)dx∗t=Γr˙t+lrdt,Cx∗t=CΓrt+lr.Since MΓ+Nγr(t+lr)=0 and CΓr(t+lr)=r(t+lr) according to (4), substituting these two equations into (5) yields(6)dx∗t=MΓ+Nγrt+lr+Γr˙t+lrdt,rt+lr=Cx∗t;that is,(7)dx∗t=Mx∗t+Nu∗t+Γr˙t+lrdt,rt+lr=Cx∗t.
Define(8)x~t=xt-x∗t,u~t=ut-u∗t,y~t=yt-Cx∗t,r~t=rt-rt+lr.Subtracting (7) by (1) from both sides, the following could be obtained:(9)dx~t=Mx~t+Nu~t+-Γr˙t+lrdt+ωtdBt,y~t=Cx~t,x~t0=x0-x∗t0,t∈t0,∞.
Now an integrator is introduced. Integrating the tracking error signale(t) in the interval of t0 to t yields(10)qt=∫t0tesds=∫t0tys-rsds.Obviously q(t0)=0. Let q∗(t)=∫t0tCx∗(s)-r(s+lr)ds, and then q∗(t)=0 holds. Define q~(t)=q(t)-q∗(t) and the differential form of q~(t) becomes(11)dq~t=etdt=Cx~t-r~tdt.Remark 9.
According to Assumption3, the term r~(t) in (11) is also previewable while r~(ρ)(t≤ρ≤t+lr). In particular, while ρ≥t+lr, r~(ρ)=0.
In order to make the outputy(t) tracking the reference signal r(t) as accurately as possible, the performance index J is designed as(12)Jt0,x0;u~·=Et0,x012∫t0∞q~TtQq~q~t+u~TtRu~tdt,where Qq~=Qq~T>0 and R=RT>0 are matrices of proper dimensions.Remark 10.
Compared with the finite horizon,limt→∞e(t)=0 can also be obtained in the infinite horizon while limt→∞q(t)=0. Therefore, the term eT(t)Qee(t) in the performance index of [15] can be replaced by q~T(t)Qq~q~(t) in (12). Furthermore, since there exists unique semipositive definite solution of the corresponding Riccati equation when Qq~>0, no other terms need to be included in the performance index, which simplifies the calculation.
To solve the optimal control problem of (9) with the performance index (12), the augmented error system method can be employed.


## 3. Construction of the Stochastic Augmented Error System

Combining (9) and (11), the following holds:(13)dzt=M~zt+N~u~t+D~r~t+Γ~r˙t+lrdt+ωtdBt,where(14)zt=q~tx~t,M~=0C0M,N~=0N,D~=-I0,Γ~=0-Γ.Referring to the output equation of (9) and the reference signal, the output of (13) can be defined as(15)q~t=C~zt,where C~=I0. Joining (13) and (15) together, the following hold:(16)dzt=M~zt+N~u~t+D~r~t+Γ~r˙t+lrdt+ωtdBt,q~t=C~zt,zt0=q~t0x~0t0=z0.Equation (16) is the needed stochastic augmented error system.


## 4. Design of the Optimal Controller for the Stochastic System

Denoting the performance index (12) with the related variables in (16) yields(17)J~t0,z0;u~·=Et0,z012∫t0∞zTtC~TQq~C~zt+u~TtRu~tdt.Due to the existence of the termD~r~(t)+Γ~r˙(t+lr) in (16), the conclusion of the optimal regulation problem in dealing with the infinite horizon cannot be directly employed. Therefore, similar to the condition in deterministic systems [11], the optimal regulator of the stochastic augmented error system (16) can be obtained through the following three steps.Firstly, (17) is revised to(18)J~=limtτ→∞Et0,z012∫t0tτzTtC~TQq~C~zt+u~TtRu~tdt.Secondly, the finite-time horizon optimal regulation problem with the performance index(19)J~~=Et0,z012∫t0tτzTtC~TQq~C~zt+u~TtRu~tdtis solved, where tτ is the terminal time.Compared with the performance index in [15], the two terms eT(t)Qee(t) and eT(tτ)Qe(tτ)e(tτ) are removed in (19). Therefore, according to the dynamic programming principle for stochastic systems [16, 18], the following lemma about the finite horizon optimal control problem with the terminal time tτ can be obtained by letting Qe=Qe(tτ)=0 in Theorem  1 in [15].Lemma 11 (see [15]).
If Assumptions3 and 5 hold, the optimal controller of (16), with the performance index (19), is(20)u~t=-R-1N~1TPtzt+θt,where P(t)≥0∈R(m+p)×(m+p) satisfies the Riccati differential equation(21)-P˙t=M~TPt+PtM~-PtN~R-1N~TPTt+C~TQq~C~.With the terminal condition P(tτ)=0 and θ(t) the differential equation is obtained:(22)θ˙t+M~T-PtN~R-1N~Tθt+PtD~r~t+PtΓ~r˙t+lr=0,with the terminal condition θ(tτ)=0.Thirdly, Theorem12 is received by letting tτ→∞.Theorem 12.
If Assumptions1 to 5 hold, the optimal controller u~(t) of (16) with the performance index (17) can be expressed as(23)u~t=-R-1N~TPzt-R-1N~T∫tt+lrexp⁡t0-tM~-N~R-1N~TPTPD~r~s+PΓ~r˙s+lrds,where P∈Rm~×m~ is the unique semipositive definite solution of Riccati differential equation(24)M~TP+PM~-PN~R-1N~TP+C~TQq~C~=0.Proof.
In terms of the performance index (17), if (M~,N~) is stabilizable and (Qq~1/2C~,M~) is detectable, according to the known conclusion, there exists a unique semipositive definite solution P of Riccati algebra equation (25)M~TP+PM~-PN~R-1N~TP+C~TQq~C~=0.And when t→∞, (26)limt→∞Pt=P.
As a result, the solving ofθ(t) in (22) is also simplified.
DenoteMc=M~-N~R-1N~TP. Equation (22) and its terminal condition can be expressed as(27)θ˙t+McTθt+PD~r~t+PΓ~r˙t+lr=0,θtτ=0.Based on linear system theory [19], the fundamental solution matrix of the homogeneous system corresponding to (27) is (28)Φ~t,t0=exp⁡-∫t0tMcTdt=exp⁡t0-tMcT,with the property(29)Φ~-1t0,t=Φ~t,t0,Φ~t,t0=Φ~t,tsΦ~ts,t0,t0≤s≤t.Therefore, the solution of (27) is(30)θt=Φ~t,t0θt0-∫t0tΦ~t,sPD~r~t+PΓ~r˙t+lrds,and the backward form is(31)θtτ=Φ~tτ,tθt-∫ttτΦ~tτ,sPD~r~s+PΓ~r˙s+lrds.With the terminal condition θ(tτ)=0, the following will be obtained:(32)θt=Φ~-1tτ,t∫ttτΦ~tτ,sPD~r~s+PΓ~r˙s+lrds.According to the property of the reference signals in Assumption 3, θ(t) can be expressed as(33)θt=Φ~-1tτ,t∫ttτ∧t+lrΦ~tτ,sPD~r~s+PΓ~r˙s+lrds=Φ~t,tτ∫ttτ∧t+lrΦ~tτ,sPD~r~s+PΓ~r˙s+lrds=∫ttτ∧t+lrΦ~t,sPD~r~s+PΓ~r˙s+lrds.Substituting (33) into (20) yields(34)u~t=-R-1N~TPzt-R-1N~1T∫ttτ∧t+lrΦ~t,sPD~r~s+PΓ~r˙s+lrds,where tτ∧(t+lr)=min{tτ,t+lr}.
The conclusion is that whentτ→∞, the following theorem can be obtained.Theorem 13.
If Assumptions1 to 5 hold, the optimal controller u~(t) of (16) with the performance index (17) can be expressed as(35)u~t=-R-1N~TPzt-R-1N~T∫tt+lrΦ~t,sPD~r~s+PΓ~r˙s+lrds,where P∈Rm~×m~ is the unique semipositive definite solution of Riccati differential equation(36)M~TP+PM~-PN~R-1N~TP+C~TQq~C~=0.
Then, let the matrixP be separated into(37)P=P1P2,where P1 and P2 are matrices of m~×p and m~×m. According to (8), (35), z(t)=q~(t)x~(t), and ∫tt+lrΦ~(t,s)PΓ~r˙(s+lr)ds=0, the following will be obtained:(38)ut=-R-1N~TP1qt-R-1N~TP2xt-R-1N~T∫tt+lrΦ~t,sPD~r~sds+u∗t+R-1N~TP2q∗t+R-1N~TP2x∗t,where P≥0∈Rm~×m~ satisfies Riccati differential equation (24).Due to the fact thatx∗(t)=Γr∗(t+lr), u∗(t)=γr∗(t+lr), and q∗(t)=0, the following corollary is obtained.Corollary 14.
If Assumptions1 to 5 hold, the optimal preview controller u(t) of (1) with the performance index (12) can be expressed as(39)ut=-R-1N~TP1qt-R-1N~TP2xt-R-1N~T∫tt+lrΦ~t,sPD~r~sds+γ+R-1N~TP2Γrt+lr.From (39) it can be seen that there are four parts contained in the preview controller of (1): the first part is the integral of tracking error term -R-1N~TP1q(t), stemmed from the inducing of the integrator. The second part is the state feedback -R-1N~TP2x(t). The third one -R-1N~T∫tt+lrΦ~(t,s)PD~r~(s)ds is the reference preview compensation. The last one γ+R-1N~TP2Γr(t+lr) is the previewable complement of the assistant system.


## 5. Stability of Closed-Loop System

Withω(t)dBt being an external disturbance, the stability and detectability of system (16) are studied only when ω(t)=0. Here, (16) becomes the deterministic system(40)dzt=M~zt+N~u~t+D~r~t+Γ~r˙t+lrdt.As a result, the sufficient and necessary criteria which guarantee that there exists a unique semipositive definite solution to the Riccati equation (24) can be gained by the known conclusion, as follows.Lemma 15 (see [11]).
Suppose the following conditions are satisfied:(1)
The matricesQq~ and R are both positive definite.(2)
The matrix(41)Π=MNC0is of full row rank.(3)
The matrices pair(M,N) is stabilizable.(4)
The matrices pair(C,M) is detectable.
Then there exists unique semipositive definite solution in the Riccati equation (24), and there exists the asymptotically stable coefficient matrix Mc in the closed-loop system (27).Therefore, the following theorem can be received.Theorem 16.
IfQq~ and R are both positive definite matrices and Assumptions 1 to 5 hold, the full regulation is achieved in the closed-loop system:(42)dzt=M~zt+N~u~t+D~r~t+Γ~r˙t+lrdt+ωtdBt,u~t=-R-1N~TPzt-R-1N~T∫tt+lrΦ~t,sPD~r~s+PΓ~r˙s+lrds,q~t=C~ztzt0=q~t0x~0t0=z0;namely,(43)limt→∞q~t=0.
Furthermore, the following will be obtained:(44)limt→∞et=0.


## 6. State Observer

If the state vectorx(t) in (1) cannot be measured directly, the optimal preview controller (39) cannot be realized. In order to solve this problem, the state observer (45)dx^t=Mx^t+Nut+Lyt-Cx^tdt+ωtdBt,xt0=x0,t∈t0,∞could be designed.Subtracting (45) from (1) on both sides yields(46)dx-t=M-LCx-tdt,where x-(t)=x(t)-x^(t).So, if(M-LC) is stable, the following will hold:(47)limt→∞x-t=0,which means that the state vector x^(t) in the observer equation (45) approximates the state vector x(t) in (1) when t→∞. Based on linear system theory [19], it is known that if (C,A) is detectable, (M-LC) is stable. Then the following will be obtained.Theorem 17.
If the conditions in Theorem16 hold, the closed-loop system can be received:(48)dxt=Mxt+Nutdt+ωtdBt,dx^t=Mx^t+Nut+Lyt-Cx^tdt+ωtdBt,yt=Cxt,ut=-R-1N~TP1∫t0tCx^s-rsds-R-1N~TP2x^t-R-1N~T∫tt+lrΦ~t,sPD~r~sds+γ+R-1N~TP2Γrt+lr,where (M-LC) is stable and (48) achieve the complete regulation.


## 7. Numerical Simulation

Example 1.
Consider the stochastic control system(49)dx1tx2t=01-1-1x1tx2t+01utdt+0.1dBt,yt=10x1tx2t,where the coefficient matrices are(50)M=01-1-1,N=01,C=10,ωt=0.10.1,respectively. Therefore, the coefficient matrices in (16) are(51)M~=0100010-1-1,N~=001,D~=-100-100and Brownian motion Bt satisfies(52)Bt~N0,0.01.Let the initial state of (1) be(53)x0=00,where x1 and x2 are mutually independent. The preview lengths of the reference signals are lir=0,0.5,0.75s(i=1,2,3), respectively. The weight matrices of the performance index are (54)Qq~=1,R=1,and the solution of Riccati equation(55)M~TP+PM~-PN~R-1N~TP+C~TQq~C~=0is(56)P=2.15331.81841.0001.81841.91571.1531.0001.1530.818,which is a positive definite matrix. The coefficient matrix Mc in (25) is(57)Mc=M~-N~R-1N~TP=010001-1-2.153-1.818.
When the reference signal is(58)rt=0,0≤t≤5,0.2t-5,5<t≤10,1,t>10,Figures 1–3 can be obtained.
Figure1 shows the tracking controller with nonpreview. Figure 2 shows the tracking controller with lr=0.5s preview. Figure 3 shows the tracking controller with lr=0.75s preview. Comparing the above three figures, it can be seen that based on the preview controller, the output signals can track the reference signals much faster and with less tracking error.
When the reference signal is(59)rt=0,0<t≤5,0.1sin⁡0.5t-5,5<t≤5+5π,0,t>5+5π,Figures 4–6 can be obtained.
Figure4 shows the tracking controller with nonpreview. Figure 5 shows the tracking controller with lr=0.5s preview. Figure 6 shows the tracking controller with lr=0.75s preview. Comparing the above three figures, it can be seen that based on the preview controller, the output signals can track the reference signals much faster and with less tracking error.Figure 1
Closed-loop output response with nonpreview and reference signalr(t).Figure 2
Closed-loop output response withlr=0.5s preview and reference signal.Figure 3
Closed-loop output response withlr=0.75s preview and reference signal r(t).Figure 4
Closed-loop output response with nonpreview and reference signalr(t).Figure 5
Closed-loop output response withlr=0.5s preview and reference signal.Figure 6
Closed-loop output response withlr=0.75s preview and reference signal r(t).


## 8. Conclusion

This paper has studied the optimal preview control problem for a class of continuous stochastic system in the infinite horizon. By introducing the integrator and the assistant system, the stochastic augmented error system is constructed. Compared with the finite horizon, the performance index is simplified. The stability of the stochastic augmented error system is also studied and the observer for the original stochastic system is designed. Finally, the simulation example shows the effectiveness of the conclusion in this paper.



---

*Source: 7679165-2016-10-20.xml*