# Delayed Stochastic Linear-Quadratic Control Problem and Related Applications

Authors: Li Chen; Zhen Wu; Zhiyong Yu
Journal: Journal of Applied Mathematics
 (2012)

---

## Abstract

We discuss a quadratic criterion optimal control problem for stochastic linear system with delay in both state and control variables. This problem will lead to a kind of generalized forward-backward
stochastic differential equations (FBSDEs) with Itô’s stochastic delay equations as forward equations and anticipated backward stochastic differential equations as backward equations. Especially, we present the optimal feedback regulator for the time delay system via a new type of Riccati equations and also apply to a population optimal control problem.


---

## Body

## 1. Introduction

The problem of optimal control for delayed stochastic system has received a lot of attention recently. One of the reasons is that there are many phenomena which have the nature of pastdependence that is, their behavior at timet not only depends on the current situation, but also on their past history. Such kinds of mathematical models described by stochastic delay differential equations (SDDEs) are ubiquitous and have wide range of applications in physics, biology, engineering, economics, and finance (see Arriojas et al. [1], Mohammed [2, 3], and the references therein).In control problem, a delay term may arise when there is a time lag between observation and regulation or the aftereffect of control; that is, there may be delay in state or control variables. Although many papers came out to discuss the delayed control problem, the analysis of systems with delay is fraught with many difficulties, not only for the infinite dimensional problem, but also for the absence of Itô's formula to deal with the delayed part of the trajectory. In order to surmount these difficulties, one can consider specific classes of systems with aftereffect, such as  Øksendal and Sulem [4].This paper is concerned with optimal control of a linear system with delay under a quadratic cost criteria, namely, the quadratic problem for stochastic linear control system with delay. It is well known that linear-quadratic (LQ) control is one of the most important classes of optimal control and the solution of this problem has many real-world applications. Deterministic LQ control problem with delay has been discussed in Alekal et al. [5], Basin et al. [6], and so forth. However, there is little results on stochastic LQ problem with delay since the difficulties in exploring the optimal feedback regulator, such as the Riccati equation, are very different from the case without delay.In Peng and Yang [7], a new type of backward stochastic differential equations (BSDEs) was introduced, which is named anticipated BSDEs. The anticipated BSDE provides a new method to deal with optimal control problem with delay (see Chen and Wu [8]). In our paper, we study the delayed stochastic LQ problem by virtue of the anticipated BSDEs combined with the SDDEs.In the next section, we introduce a kind of generalized FBSDE and give the existence and uniqueness result for its solution. With the help of the FBSDE, we find the optimal control of the stochastic LQ problem with delay in both state and control variables and the quadratic cost also involves delay terms.It is very important to design an optimal feedback regulator for LQ problem in practice. Traditionally, a fundamental tool to obtain the state feedback is the Riccati equation. In Section3, we introduced a new type of generalized Riccati equations and give the feedback regulator of delayed stochastic LQ problem. To the best of our knowledge, it is the first result on the optimal feedback control for the delayed stochastic LQ problem, where the state and control variables with delay are involved not only in the system, but also in the cost functional. In the last section, we apply our theoretical result to a population optimal control problem. Some technical proofs are put in Appendix.


## 2. Stochastic LQ Problem with Delay

We first introduce some notations. For any Euclidean spaceH, we denote by 〈·,·〉 (resp. |·|) the scalar product (resp. norm) of H. Let ℝn×m  be the Hilbert space consisting of all n×m matrices with the inner product:
(2.1)〈A,B〉:=tr{ABT},∀A,B∈ℝn×m.
Here, the superscript T denotes the transpose of vectors or matrices. Particularly, we denote by 𝒮n  the set of all n×n symmetric matrices.LetW(·) be a standard 1-dimensional Brownian motion on a complete probability space (Ω,ℱ,P). The information structure is given by a filtration 𝔽={ℱt}t≥0, which is generated by W(·) and augmented by all the P-null sets. We assume the dimension of Brownian motion d=1 just for the simplicity of notations. In fact, all the conclusions in this paper still hold true for the case that the dimension of Brownian motion d>1. For any K, the Euclidean spaces or sets of matrices, the following notations will be used throughout the paper:
(2.2)L2(Ω,ℱt,P;K)≔{ξis aK-valuedℱt-measurable  random  variables.t.𝔼[ξ2]<+∞};L𝔽2(t1,t2;K)≔{∫t1t2‍|φ(t)|2φ(t),0t1≤t≤t2,is aK-valued𝔽-adapted stochastic process s.t.𝔼∫t1t2‍|φ(t)|2dt<+∞};L𝔽∞(t1,t2;K)≔{φ(t),t1≤t≤t2,is aK-valued𝔽-adapted bounded process}.In this section, we consider the following linear controlled system involving delays in both state variable and control variable:(2.3)x(t)=a+∫0t‍[A1(s)x(s)+A2(s)x(s-δ)+B1(s)v(s)+B2(s)v(s-δ)]ds+∫0t‍[C1(s)x(s)+C2(s)x(s-δ)+D1(s)v(s)+D2(s)v(s-δ)]dW(s),t∈[0,T],x(t)=x0(t),t∈[-δ,0),v(t)=v0(t),t∈[-δ,0),
where δ>0 is a constant time delay, x0(·)∈L𝔽2(-δ,0;ℝn), v0(·)∈L𝔽2(-δ,0;ℝk) are deterministic functions, and a∈ℝn. A1(·),A2(·),C1(·),C2(·)∈L𝔽∞(0,T;ℝn×n), B1(·),B2(·),D1(·),D2(·)∈L𝔽∞(0,T;ℝn×k), {v(t),t∈[0,T]} is an 𝔽-adapted square-integrable process taking values in ℝk. Let 𝒰ad denote the set of stochastic processes v(·) of the form:
(2.4)v(t)={v0(t),-δ≤t<0,v(t)∈L𝔽2(0,T;ℝk),0≤t≤T.
An element of 𝒰ad is called an admissible control. By the classical results of SDDEs, we know that the system (2.3) has a unique solution for any admissible control v(·). The solution xv(·) of SDDE (2.3) is called the state trajectory corresponding to the control v(·)∈𝒰ad and (xv(·),v(·)) is called an admissible pair.The cost functional is given by(2.5)J(v(·))=12𝔼[∫0T‍(〈R1(t)x(t),x(t)〉+〈R2(t)x(t-δ),x(t-δ)〉+〈N1(t)v(t),v(t)〉+〈N2(t)v(t-δ),v(t-δ)〉)dt+〈Qx(T),x(T)〉∫0T‍].
Here, R1(·),R2(·)∈L𝔽∞(0,T;𝒮n), N1(·),N2(·)∈L𝔽∞(0,T;𝒮k), and Q is an ℱT-measurable nonnegative symmetric bounded matrix. Moreover, we assume that, for any (ω,t)∈Ω×[0,T],  R1(t)+R2(t+δ) and Q are nonnegative definite, N1(t)+N2(t+δ) is positive definite, and the inverse (N1(t)+𝔼ℱt[N2(t+δ)])-1 is also bounded.The controller hopes to minimize the above cost functionalJ by selecting an appropriate admissible control v(·); that is, the problem is to find u(·)∈𝒰ad such that
(2.6)J(u(·))=infv(·)∈𝒰adJ(v(·)).
We call the problem above the linear-quadratic optimal control problem of delayed system, and we denote it by Problem (LQD). An admissible pair (xu(·),u(·)) is called optimal for Problem (LQD) if u(·) achieves the infimum of J.The Problem (LQD) we introduced above is a general type of LQ problem for stochastic system with delay. Not only the state variable and control variable are involved with delays, but also the cost functional contains delay terms.By introducing a new type of BSDEs-anticipated BSDEs, we desire to solve Problem (LQD).Theorem 2.1.
The control(2.7)u(t)=-(N1(t)+𝔼ℱt[N2(t+δ)])-1×(B1T(t)y(t)+D1T(t)z(t)+𝔼ℱt[B2T(t+δ)y(t+δ)+D2T(t+δ)z(t+δ)]),t∈[0,T]
is the unique optimal control of Problem (LQD). Here, (x(·),y(·),z(·)) is the solution of the following generalized FBSDE:
(2.8)x(t)=a+∫0t‍{A1(s)x(s)+A2(s)x(s-δ)+B1(s)u(s)+B2(s)u(s-δ)}ds+∫0t‍{C1(s)x(s)+C2(s)x(s-δ)+D1(s)u(s)+D2(s)u(s-δ)}dW(s),t∈[0,T],y(t)=Qx(T)+∫tT‍{A1T(s)y(s)+C1T(s)z(s)+R1(s)x(s)======+𝔼ℱs[A2T(s+δ)y(s+δ)+C2T(s+δ)z(s+δ)+R2(s+δ)x(s)]}ds-∫tT‍z(s)dW(s),t∈[0,T],x(t)=x0(t),t∈[-δ,0),u(t)=v0(t),t∈[-δ,0),y(t)=0,z(t)=0,t∈(T,T+δ].Remark 2.2.
If we set(2.9)f(s,x(s),x(s-δ),y(s),z(s),y(s+δ),z(s+δ))=A1T(s)y(s)+C1T(s)z(s)+R1(s)x(s)+𝔼ℱs[A2T(s+δ)y(s+δ)+C2T(s+δ)z(s+δ)+R2(s+δ)x(s)],
then the second equation in (2.8) can be rewritten as
(2.10)y(t)=Qx(T)+∫tT‍f(s,x(s),x(s-δ),y(s),z(s),y(s+δ),z(s+δ))ds-∫tT‍z(s)dW(s),t∈[0,T].Proof.
Suppose(xv(·),v(·)) is an arbitrary admissible pair of system (2.3), then
(2.11)J(v(·))-J(u(·))=12𝔼∫0T‍[〈R1(t)xv(t),xv(t)〉-〈R1(t)x(t),x(t)〉+〈R2(t)xv(t-δ),xv(t-δ)〉-〈R2(t)x(t-δ),x(t-δ)〉+〈N1(t)v(t),v(t)〉-〈N1(t)u(t),u(t)〉+〈N2(t)v(t-δ),v(t-δ)〉-〈N2(t)u(t-δ),u(t-δ)〉]dt+12𝔼[〈Qxv(T),xv(T)〉-〈Qx(T),x(T)〉]=12𝔼∫0T‍[〈R1(t)(xv(t)-x(t)),xv(t)-x(t)〉+〈R2(t)(xv(t-δ)-x(t-δ)),xv(t-δ)-x(t-δ)〉+〈N1(t)(v(t)-u(t)),v(t)-u(t)〉+〈N2(t)(v(t-δ)-u(t-δ)),v(t-δ)-u(t-δ)〉+2〈R1(t)x(t),xv(t)-x(t)〉+2〈R2(t)x(t-δ),xv(t-δ)-x(t-δ)〉+2〈N1(t)u(t),v(t)-u(t)〉+2〈N2(t)u(t-δ),v(t-δ)-u(t-δ)〉]dt+12𝔼[〈Q(xv(T)-x(T)),xv(T)-x(T)〉+2〈Qx(T),xv(T)-x(T)〉].
Since R2(t)≡0,N2(t)≡0,t∈(T,T+δ], by the time-shifting transformation and the initial conditions in (2.8), we derive that
(2.12)J(v(·))-J(u(·))=12𝔼∫0T‍{〈(R1(t)+𝔼ℱt[R2(t+δ)])(xv(t)-x(t)),xv(t)-x(t)〉+〈(N1(t)+𝔼ℱt[N2(t+δ)])(v(t)-u(t)),v(t)-u(t)〉+2〈(R1(t)+𝔼ℱt[R2(t+δ)])x(t),xv(t)-x(t)〉+2〈(N1(t)+𝔼ℱt[N2(t+δ)])u(t),v(t)-u(t)〉}dt+12𝔼[〈Q(xv(T)-x(T)),xv(T)-x(T)〉+2〈Qx(T),xv(T)-x(T)〉].
Applying Itô's formula to 〈xv(t)-x(t),y(t)〉, we have
(2.13)𝔼〈xv(T)-x(T),y(T)〉=𝔼∫0T‍[∫0T‍-〈R1(t)x(t),xv(t)-x(t)〉-〈𝔼ℱt[R2(t+δ)]x(t),xv(t)-x(t)〉+〈B1(t)(v(t)-u(t)),y(t)〉+〈D1(t)(v(t)-u(t)),z(t)〉+〈B2(t)(v(t-δ)-u(t-δ)),y(t)〉+〈D2(t)(v(t-δ)-u(t-δ)),z(t)〉(R1)∫0T‍]dt.
Moreover, because v(t)=u(t),t∈[-δ,0) and y(t)=z(t)=0,t∈(T,T+δ], we can rewrite (2.13) as follows:
(2.14)𝔼〈xv(T)-x(T),y(T)〉=𝔼∫0T‍{-〈(R1(t)+𝔼ℱt[R2(t+δ)])+x(t),xv(t)-x(t)〉+〈B1(t)(v(t)-u(t)),y(t)〉+〈D1(t)(v(t)-u(t)),z(t)〉+𝔼ℱt[〈B2(t+δ)(v(t)-u(t)),y(t+δ)〉+〈D2(t+δ)(v(t)-u(t)),z(t+δ)〉]}dt.
Consequently, by the fact that R1(·)+R2(·+δ) and Q are nonnegative and N1(·)+N2(·+δ) is positive, we have
(2.15)J(v(·))-J(u(·))≥𝔼∫0T‍{〈(N1(t)+𝔼ℱt[N2(t+δ)])u(t),v(t)-u(t)〉+〈B1T(t)y(t)+D1T(t)z(t),v(t)-u(t)〉+〈𝔼ℱt[B2T(t+δ)y(t+δ)+D2T(t+δ)z(t+δ)],v(t)-u(t)〉}dt.
So letting
(2.16)u(t)=-(N1(t)+𝔼ℱt[N2(t+δ)])-1×(B1T(t)y(t)+D1T(t)z(t)+𝔼ℱt(B2T(t+δ)y(t+δ)+D2T(t+δ)z(t+δ))),t∈[0,T]
we have
(2.17)J(v(·))-J(u(·))≥0,∀v(·)∈𝒰ad,
that is, u(·) defined in (2.7) is the optimal control of Problem (LQD).
We will use the parallelogram rule to prove the uniqueness of the optimal control, and this method can also be seen in Wu [9]. We assume that u1(·) and u2(·) are both optimal controls, and the corresponding trajectories are x1(·) and x2(·). It is easy to know the trajectories corresponding to (u1(·)+u2(·))/2 are (x1(·)+x2(·))/2. Since N1(·)+N2(·+δ) is positive, R1(·)+R2(·+δ) and Q are nonnegative, we know that J(u1(·))=J(u2(·))=λ≥0, and
(2.18)2λ=J(u1(·))+J(u2(·))=2J(u1(·)+u2(·)2)+𝔼∫0T‍{〈(R1(t)+𝔼ℱt[R2(t+δ)])x1(t)-x2(t)2,x1(t)-x2(t)2〉+〈(N1(t)+𝔼ℱt[N2(t+δ)])u1(t)-u2(t)2,u1(t)-u2(t)2〉}dt+𝔼[〈Qx1(T)-x2(T)2,x1(T)-x2(T)2〉]≥2λ+𝔼∫0T‍〈(N1(t)+𝔼ℱt[N2(t+δ)])u1(t)-u2(t)2,u1(t)-u2(t)2〉dt.
Because of N1(·)+N2(·+δ) being positive, we have u1(·)=u2(·). We complete our proof.Remark 2.3.
The existence of the optimal control is equivalent to the existence of solution for (2.8), which is a kind of complex generalized FBSDEs. The proof of the existence and uniqueness of the solution for this kind of FBSDE are put in Appendix.


## 3. Feedback Regulator of Delayed System

It is well known that the feedback representation of optimal control is very useful in applications. And in the classical case, the optimal feedback control can be represented via the Riccati equation. But, for stochastic systems with delay, it is not easy to find the feedback control because of the dependence of the history. What should then be an appropriate “Riccati equation” corresponding to our LQ problem with delay?In this section, we will pay attention to the feedback regulator of the general delayed LQ problem discussed in Section2 and try to get the appropriate Riccati equation associated with Problem (LQD). We remark that all given coefficients of the problem are assumed to be deterministic from now on.Let us start with the following results about anticipated BSDE.Assume that for alls∈[0,t], f(s,w,y,z,ξ,η):[0,T]×Ω×ℝn×ℝn×d×L2(Ω,ℱr,P;ℝn)×L2(Ω,ℱr,P;ℝn×d)→L2(Ω,ℱr,P;ℝn), where r∈[s,T+δ]. And f satisfies the following conditions.(H3.1)
There exists a constantC>0, such that for all s∈[0,T], y,y'∈ℝn, z,z'∈ℝn×d, ξ·,ξ·′∈L𝔽2(s,T+δ;ℝn), η·,η·′∈L𝔽2(s,T+δ;ℝn×d), we have
(3.1)|f(s,y,z,ξr,ηr)-f(s,y′,z′,ξr′,ηr′)|≤C(|y-y′|+|z-z′|+𝔼ℱs[|ξr-ξr′|+|ηr-ηr′|]).(H3.2)
(𝔼[∫0T‍|f(s,0,0,0,0)|2ds]<∞).Let(yi(·),zi(·)), i=1,2, be, respectively, solutions of the following two anticipated BSDEs:
(3.2)yi(t)=ξ(T)+∫tT‍fi(s,yi(s),zi(s),yi(s+δ),zi(s+δ))dt-zi(s)dW(s),t∈[0,T],yi(t)=ξ(t),zi(t)=η(t),t∈(T,T+δ].
We have the following lemma, which is a direct corollary of Theorem 2.2 in Yu [10].Lemma 3.1.
Assume thatf1,f2 satisfy (H3.1) and (H3.2), ξ(·)∈L𝔽2(T,T+δ;ℝn), η(·)∈L𝔽2(T,T+δ;ℝn×k), and
(3.3)𝔼∫0T‍|f1(t,y2(t),z2(t),y2(t+δ),z2(t+δ))-f2(t,y2(t),z2(t),y2(t+δ),z2(t+δ))|2dt=0,
then
(3.4)y1(t)=y2(t),z1(t)=z2(t),a.e.,a.s.In order to get the feedback regulator, we introduce the following generalizedn×n matrix-valued Riccati equation system:
(3.5)-K˙1(t)=A1T(t)K1(t)+K1(t)A1(t)-K1(t)B1(t)[N1(t)+N2(t+δ)]-1[B1T(t)K1(t)+D1T(t)H1(t)]-K1(t)B2(t)[N1(t-δ)+N2(t)]-1[B2T(t)K1(t)+D2T(t)H1(t)]-K1(t+δ)B2(t+δ)[N1(t)+N2(t+δ)]-1[B1T(t)K1(t)+D1T(t)H1(t)]-K1(t-δ)B1(t-δ)[N1(t-δ)+N2(t)]-1[B2T(t)K1(t)+D2T(t)H1(t)]+A2T(t)K1(t)+K1(t+δ)A2(t+δ)-e-λδK2(t+δ)+K2(t)+[C1T(t)+C2T(t)]H1(t)+R1(t)+R2(t+δ),t∈[0,T],(3.6)-K˙2(t)=-λK2(t)-K1(t)B1(t)[N1(t)+N2(t+δ)]-1[B1T(t)K2(t)+D1T(t)H2(t)]-K1(t)B2(t)[N1(t-δ)+N2(t)]-1[B2T(t)K1(t)+D2T(t)H1(t)]-K1(t-δ)B1(t-δ)[N1(t-δ)+N2(t)]-1[B2T(t)K2(t)+D2T(t)H2(t)]-K1(t+δ)B1(t+δ)[N1(t)+N2(t+δ)]-1[B1T(t)K2(t)+D1T(t)H2(t)]+[A1T(t)+A2T(t)]K2(t)+[C1T(t)+C2T(t)]H2(t),t∈[δ,T],(3.7)H1(t)=K1(t)C1(t)-K1(t)D1(t)[N1(t)+N2(t+δ)]-1[B1T(t)K1(t)+D1T(t)H1(t)]-K1(t)D2(t)[N1(t-δ)+N2(t)]-1[B2T(t)K1(t)+D2T(t)H1(t)]-K1(t-δ)D1(t-δ)[N1(t-δ)+N2(t)]-1[B2T(t)K1(t)+D2T(t)H1(t)]-K1(t+δ)D2(t+δ)[N1(t)+N2(t+δ)]-1[B1T(t)K1(t)+D1T(t)H1(t)]+K1(t+δ)C2(t+δ),t∈[0,T],(3.8)H2(t)=-K1(t)D1(t)[N1(t)+N2(t+δ)]-1[B1T(t)K2(t)+D1T(t)H2(t)]-K1(t)D2(t)[N1(t-δ)+N2(t)]-1[B2T(t)K2(t)+D2T(t)H2(t)]-K1(t-δ)D1(t-δ)[N1(t-δ)+N2(t)]-1[B2T(t)K2(t)+D2T(t)H2(t)]-K1(t+δ)D2(t+δ)[N1(t)+N2(t+δ)]-1[B1T(t)K2(t)+D1T(t)H2(t)],t∈[0,T].
With the boundary conditions
(3.9)K1(T)=Q,K1(t)=0,t∈(T,T+δ],K2(t)=0,t∈[T,T+δ]∪(0,δ),H1(t)=H2(t)=0,t∈(T,T+δ].
Here,  λ is some constant.Then we have the following main result.Theorem 3.2.
Suppose that there exist matrix-valued deterministic processes(K1(·),K2(·),H1(·),H2(·)) satisfying the generalized Riccati equation system (3.5)–(3.8) with corresponding boundary conditions, and for system (2.3), A2(t)=B2(t)=C2(t)=D2(t)=0, t∈[0,δ). Then the optimal feedback regulator for the delayed linear quadratic optimal Problem (LQD) is
(3.10)u(t)=-[N1(t)+N2(t+δ)]-1(B1T(t)y(t)+D1T(t)z(t)=========+𝔼ℱt[B2T(t+δ)y(t+δ)+D2T(t+δ)z(t+δ)]),t∈[0,T],
with
(3.11)y(t)=K1(t)x(t)+K2(t)∫-δ0‍eλsx(t+s)ds,(3.12)z(t)=H1(t)x(t)+H2(t)∫-δ0‍eλsx(t+s)ds.
And the optimal value function is
(3.13)J(u(·))=12〈K1(0)a,a〉.Proof.
Let(K1(·),K2(·),H1(·),H2(·)) be the solution of Riccati equation system (3.5)–(3.8) and we set
(3.14)y1(t)=K1(t)x(t)+K2(t)∫-δ0‍eλsx(t+s)ds,z1(t)=H1(t)x(t)+H2(t)∫-δ0‍eλsx(t+s)ds.
Applying Itô's formula to y1(·), we have
(3.15)y1(t)=Qx(T)+∫tT‍-{∫-δ0‍K˙1(s)x(s)===========+K1(s)[A1(s)x(s)+A2(s)x(s-δ)+B1(s)u(s)+B2(s)u(s-δ)]===========+K˙2(s)∫-δ0‍eλrx(t+r)dr+K2(s)===========×[x(s)-e-λδx(s-δ)-λ∫-δ0‍eλrx(t+r)dr]}ds-∫tT‍{K1(s)[C1(s)x(s)+C2(s)x(s-δ)+D1(s)u(s)+D2(s)u(s-δ)]}dW(s),t∈[0,T],
with
(3.16)u(t)=-[N1(t)+N2(t+δ)]-1(B1T(t)y1(t)+D1T(t)z1(t)+𝔼ℱt[B2T(t+δ)y1(t+δ)+D2T(t+δ)z1(t+δ)]),t∈[0,T].
Substituting (3.16) into (3.15), we have
(3.17)y1(t)=Qx(T)+∫tT‍f1(s,x(s),x(s-δ),y1(s),z1(s),y1(s+δ),z1(s+δ))ds-∫tT‍z2(s)dW(s),t∈[0,T],
where z2(s)=K1(s)[C1(s)x(s)+C2(s)x(s-δ)+D1(s)u(s)+D2(s)u(s-δ)] and
(3.18)f1(s,x(s),x(s-δ),y1(s),z1(s),y1(s+δ),z1(s+δ))=-{∫-δ0‍K˙1(s)x(s)+K1(s)[A1(s)x(s)+A2(s)x(s-δ)+B1(s)u(s)+B2(s)u(s-δ)]+K˙2(s)∫-δ0‍eλrx(t+r)dr+K2(s)[x(s)-e-λδx(s-δ)-λ∫-δ0‍eλrx(t+r)dr]}.
The following analysis shows that z1(·)=z2(·)a.e., a.s. with the help of (3.7) and (3.8). In fact, taking (3.16) into the presentation of z2(t), we will have
(3.19)z2(t)={K1(t)C1(t)-K1(t)D1(t)[N1(t)+N2(t+δ)]-1[B1T(t)K1(t)+D1T(t)H1(t)]-K1(t)D2(t)[N1(t-δ)+N2(t)]-1[B2T(t)K1(t)+D2T(t)H1(t)]}x(t)+{K1(t)C2(t)-K1(t)D2(t)[N1(t-δ)+N2(t)]-1×[B1T(t-δ)K1(t-δ)+D1T(t-δ)H1(t-δ)]}x(t-δ)-{K1(t)D1(t)[N1(t)+N2(t+δ)]-1[B2T(t+δ)K1(t+δ)+D2T(t+δ)H1(t+δ)]}×𝔼ℱt[x(t+δ)]-{K1(t)D1(t)[N1(t)+N2(t+δ)]-1[B1T(t)K2(t)+D1T(t)H2(t)]+K1(t)D2(t)[N1(t-δ)+N2(t)]-1[B2T(t)K2(t)+D2T(t)H2(t)]}∫-δ0‍eλsx(t+s)ds-{K1(t)D1(t)[N1(t)+N2(t+δ)]-1[B2T(t+δ)K2(t+δ)+D2T(t+δ)H2(t+δ)]}×∫-δ0‍eλsx(t+δ+s)ds-{K1(t)D2(t)[N1(t-δ)+N2(t)]-1[B1T(t-δ)K2(t-δ)+D1T(t-δ)H2(t-δ)]}×∫-δ0‍eλsx(t-δ+s)ds.
By the conditions K1(t)=H1(t)=0, t∈(T,T+δ], K2(t)=H2(t)=0, t∈[T,T+δ] and B2(t)=C2(t)=D2(t)=0, t∈[0,δ), we derive
(3.20)𝔼∫0T‍|z1(s)-z2(s)|2ds=𝔼∫0T‍[∫0T‍{D2T(s)H1(s)-K1(s)C1(s)+K1(s)D1(s)×[N1(s)+N2(s+δ)]-1[B1T(s)K1(s)+D1T(s)H1(s)]+K1(s)D2(s)[N1(s-δ)+N2(s)]-1[B2T(s)K1(s)+D2T(s)H1(s)]-K1(s+δ)C2(s+δ)+K1(s+δ)D2(s+δ)×[N1(s)+N2(s+δ)]-1[B1T(s)K1(s)+D1T(s)H1(s)]+K1(s-δ)D1(s-δ)[N1(s-δ)+N2(s)]-1[B2T(s)K1(s)+D2T(s)H1(s)]}x(t)+{∫-δ0‍H2(s)+K1(s)D1(s)[N1(s)+N2(s+δ)]-1[B1T(s)K2(s)+D1T(s)H2(s)]+K1(s)D2(s)[N1(s-δ)+N2(s)]-1[B2T(s)K2(s)+D2T(s)H2(s)]+K1(s-δ)D1(s-δ)[N1(s-δ)+N2(s)]-1[B2T(s)K2(s)+D2T(s)H2(s)]+K1(t+δ)D2(t+δ)[N1(t)+N2(t+δ)]-1[B1T(t)K2(t)+D1T(t)H2(t)]∫-δ0‍}×∫-δ0‍eλrx(t+r)dr∫-δ0‍]2ds.
We can see that (3.7) and (3.8) ensure that
(3.21)𝔼∫0T‍|z1(s)-z2(s)|2ds=0.
In other words, z1(·)=z2(·)a.e., a.s.
So we use the following equation instead of (3.17):
(3.22)y1(t)=Qx(T)+∫tT‍f1(s,x(s),x(s-δ),y1(s),z1(s),y1(s+δ),z1(s+δ))ds-∫tT‍z1(s)dW(s),t∈[0,T],y1(t)=z1(t)=0,t∈(T,T+δ].
Namely, (y1(·),z1(·)) introduced in (3.14) satisfies the above anticipated BSDE. From now on, we pay attention to the anticipated BSDEs (2.8) and (3.22).
Applying the similar method we used above, we derive that (3.5) and (3.6) will lead to the following:
(3.23)𝔼∫0T‍|f(s,x(s),x(s-δ),y1(s),z1(s),y1(s+δ),z1(s+δ))-f1(s,x(s),x(s-δ),y1(s),z1(s),y1(s+δ),z1(s+δ))|2ds=0.
Thus, we get y(t)=y1(t),z(t)=z1(t) by Lemma 3.1 immediately, and feedback representation of optimal control in Theorem 3.2 is proved.Applying Itô's formula to〈x(T),y(T)〉 in J(u(·)), it is easy to get (3.13).Remark 3.3.
(i) The conditionA2(t)=B2(t)=C2(t)=D2(t)=0,t∈[0,δ) is reasonable in practice: the condition implies that there is no delay on the time interval [0,δ). That is to say there is no history of the system on [-δ,0), that is, x(t)=0,t∈[-δ,0).
(ii) The matrix-value differential equation (3.5)–(3.8) is a generalized type of Riccati equations which is different from the classical one. When there are no delays in the system, that is,  A2(t)=B2(t)=C2(t)=D2(t)=N2(t)=R2(t)=0, K2(t)=H2(t)=0, our generalized anticipated Riccati equation will degenerate to the Riccati equation in Wu [9].


## 4. Application

The solvability of Riccati equation (3.5)–(3.8) is not easy to obtain in general. In this section, we derive the unique solvability of the problem for some special cases. As an application of our above results, we will consider a kind of stochastic LQ problem with delay arising from the population control model in this section. The optimal feedback control is given by the new type of Riccati equation and we can obtain its existence and uniqueness.The following population growth model comes from Mohammed [3]:
(4.1)x(t)=a0+∫0t‍[-a1x(s)+a2x(s-δ)]ds+∫0t‍σdW(s),t∈[0,T],x(t)=x0(t),t∈[-δ,0),
where x(t) is the population at time t, a1 and a2>0 are constant death rate and birth rate per capita, respectively. δ>0 denotes the development period of each individual, and the diffusion part describes the migration of the population; that is, there is migration whose overall rate is distributed like σW˙.Based on the population growth model, we consider the following population control problem:(4.2)x(t)=a0+∫0t‍[-a1(s)x(s)+a2(s)x(s-δ)+b1(s)v(s-δ)]ds+∫0t‍σ(s)x(s)dW(s),t∈[0,T],x(t)=x0(t),t∈[-δ,0),v(t)=v0(t),t∈[-δ,0).
Here, we change the original constant death and birth rate to be time-varying a1(·),a2(·),b1(·),σ(·) are deterministic bounded functions on [0,T] and a2(·)=b1(·)=0 on [0,δ). x0(·),v0(·)∈L𝔽2(-δ,0;ℝ). And the control v(·) models the intensity of the spending on controlling the population. We take into account that there will be a time lag between control expenditure and the corresponding effect on the population level, so the system is involved with control delay. W(·) is a 1-dimensional B.M.The objection is to minimize the following cost functional:(4.3)J(v(·))=12𝔼[∫0T‍(r1(t)x2(t)+n1(t)v2(t))dt+qx2(T)],
over the admissible control set 𝒰ad={v(t)∣v(t)∈L𝔽2(0,T;ℝ)}. r1(·) and n1(·) are nonnegative- and positive-valued bounded functions on [0,T], respectively, and q≥0 is a given constant. And all the coefficients in our model are equal to zero outside of the interval [0,T]. About the cost functional, one can consider that we are going to control some pests' quantity. We want to pay less expenditure and make the quantity of pests on lower level.The above population control problem is a special case of the Problem (LQD) we discussed in Sections2 and 3. From Theorems 2.1 and 3.2, we have the following results.Proposition 4.1.
The control(4.4)u(t)=-n1-1(t)[b1(t+δ)y(t+δ)],t∈[0,T]
is the unique optimal control. Here, (x(·),y(·),z(·)) is the solution of the following generalized FBSDE:
(4.5)x(t)=a+∫0t‍{a1(s)x(s)+a2(s)x(s-δ)+b1(s)u(s-δ)}ds+∫0t‍σ(s)x(s)dW(s),t∈[0,T],y(t)=qx(T)+∫tT‍{a1(s)y(s)+σ(s)z(s)+r1(s)x(s)+𝔼ℱs[a2(s+δ)y(s+δ)]}ds-∫tT‍z(s)dW(s),t∈[0,T],x(t)=0,t∈[-δ,0),u(t)=0,t∈[-δ,0),y(t)=0,z(t)=0,t∈(T,T+δ].Proposition 4.2.
The feedback control regulator of the population control problem is given by(4.6)u(t)=-n1-1(t)b1(t+δ)[k1(t+δ)x(t+δ)+k2(t+δ)∫-δ0‍eλsx(t+s)ds],t∈[0,T],
where (k1(·),k2(·)) satisfies
(4.7)-k˙1(t)=(2a1(t)+a2(t))k1(t)+a2(t+δ)k1(t+δ)-n1-1(t-δ)b22(t)k12(t)+σ2(t)k1(t)+r1(t)+k2(t)-e-λδk2(t+δ),t∈[0,T],k˙2(t)=[λ-a1(t)-a2(t)]k2(t)+n1-1(t-δ)b22(t)k22(t),t∈[δ,T],k1(T)=qx(T),k1(t)=0,t∈(T,T+δ],k2(t)=0,t∈[0,δ)∪[T,T+δ].
Moreover, the above Riccati (4.7) admits a unique solution, so the presentation form of (4.6) exists and it is unique. Here,  λ is a parameter satisfying λ>a1(t)+a2(t), for all t∈[0,T].Proof.
Obviously, the existence and uniqueness of presentation (4.6) are equivalent to Riccati (4.7) which has a unique solution, so we pay attention to (4.7).
Let us start with the second equation in (4.7). It is a classical Riccati-type differential equation with bounded coefficients, thence, by Theorem 7.7 of Chapter 6 in Yong and Zhou [11], we get that it admits a unique solution k2(t) on [0,T] and also we have 0≤k2(t)≤M with M which  is big enough positive constant.
Now, let us turn to the first equation in (4.7). It is anticipated. In order to derive its solvability, we discuss the equation backward step by step on the time interval [T-δ,T), [T-2δ,T-δ), ….
For example, whent∈[T-δ,T), the equation is equivalent to the following:
(4.8)-k˙1(t)=2[a1(t)+12a2(t)]k1(t)+σ2(t)k1(t)-n1-1(t)b22(t)k12(t)+r1(t)+k2(t),t∈[T-δ,T),k1(T)=qx(T),k1(t)=0,t∈(T,T+δ].
Becausek2(t) can be solved in advance, we get that there exists a unique solution of (4.8) and k1(t)≥0 over [T-δ,T) also by Theorem 7.7 of Chapter 6 in Yong and Zhou [11].
The discussions on the other time intervals are similar; we omit them. All in all, the Riccati equation (4.7) is solved backward in time from [T,T+δ] as an initial value problem and the solution is unique. This will lead to the existence and uniqueness of our feedback control immediately. The proof is completed.



---

*Source: 835319-2012-09-26.xml*