# The Delayed Doubly Stochastic Linear Quadratic Optimal Control Problem

Authors: Yan Chen; Jie Xu
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

In this paper, the delayed doubly stochastic linear quadratic optimal control problem is discussed. It deduces the expression of the optimal control for the general delayed doubly stochastic control system which contained time delay both in the state variable and in the control variable at the same time and proves its uniqueness by using the classical parallelogram rule. The paper is concerned with the generalized matrix value Riccati equation for a special delayed doubly stochastic linear quadratic control system and aims to give the expression of optimal control and value function by the solution of the Riccati equation.


---

## Body

## 1. Introduction

As is known to all, the stochastic differential equation and stochastic analysis have developed rapidly. The theory of the stochastic differential equation is widely used in economy, biology, physics, financial mathematics, and other fields. The latest research on the insurance model was given in [1–3]. The social optimal mean field control problem was discussed in [4]. In order to provide a probabilistic interpretation for the solution of a kind of partial differential equations, Pardoux and Peng [5] first introduced the backward doubly stochastic differential equations and proved the existence and uniqueness of this kind of differential. Then, people began to study doubly stochastic differential equations. Zhu and Shi [6, 7] were concerned with a class of partial information control problems for backward doubly stochastic systems and gave the maximum principle and its applications for the system. Recently, Shi and Zhu [8] studied a type of forward-backward doubly stochastic differential equations driven by Brownian motions and the Poisson process and applied the result to backward doubly stochastic linear quadratic nonzero sum differential games with random jumps to get the explicit form of the open-loop Nash equilibrium point by the solution of this kind of equation. With the deepening of research, people gradually realized that many problems are not only affected by the current situation, but also by their past history. This kind of problem is called the delay problem. The equation describing this kind of problem is called the delay equation. Due to the fact that time delay widely exists in the practical systems, it will cause the change in system performance. Therefore, it can increase the control difficulty of the system. Delayed problems have become the focus of scholar’s research studies. Chen and Wu [9] considered the delayed backward system and obtained the maximum principle for these problems. Wu and Wang [10] studied the optimal control problem of the backward stochastic differential delay equation under partial information. Lv et al. [11] considered the maximum principle for optimal control of anticipated forward-backward stochastic delayed systems with regime switching. Wang and Wu [12] were concerned with the optimal control problems of forward-backward delay systems involving impulse controls and established the stochastic maximum principle for this kind of systems. Yu [13] investigated the maximum principle for stochastic optimal control problems of delay systems with random coefficients involving both continuous and impulse controls.Linear quadratic (LQ) optimal control problem is the theoretical basis for many problems. When delay variables exist in the doubly stochastic control system, the LQ problem becomes more complex and interesting. Chen and Wu [14] considered the LQ problem with delay in which the state depended on the past time but not the control in the system. Tang and Wu [15] were concerned with the linear stochastic system with Lévy processes. Huang et al. [16] were concerned with one kind of delayed forward-backward linear quadratic stochastic control problems and derived the explicit form of the optimal control. However, to our best knowledge, there is little work on the doubly stochastic LQ problem with delay. Based on the abundant literature, we want to discuss the delayed doubly stochastic LQ problem. When the LQ control system contains time delay, some important characteristic changes have taken place in research. The system contains a delayed doubly stochastic differential equation and a new kind of equation called the anticipated backward doubly stochastic differential equation which was discussed in [17, 18]. Inspired by the idea of the maximum principle for the delayed doubly stochastic control system [19, 20], we studied the general LQ system in which both the state variable and the control variable contain time delay at the same time. As is known to all, it is the key to find out the feedback control of the LQ problem. We deduce the explicit expression of the optimal control for the delayed doubly stochastic LQ problem. We consider the matrix Riccati equation for a class of the LQ problem. We deduced the solution of the LQ system by the solution corresponding to the Riccati equation, which was introduced originally by Peng [21]. We hope that our research can better describe the optimal feedback control of the delayed doubly stochastic LQ problem.The rest of our paper is organized as follows. First, we introduce preliminary results and some necessary notations. In Section3, we give the explicit expression of optimal control and prove its uniqueness by using the classical parallelogram rule. And then, we discuss a special kind of the control system, in which the time delay is contained only in the control variables. We try to introduce the generalized matrix value Riccati equation corresponding to the system. And then, we use the solution of the Riccati equation to show the optimal control for the delayed doubly stochastic LQ problem. At the same time, we indicate the objective function by the solution of the Riccati equation and the initial value of the state variable.


## 2. Preliminaries

First, let us introduce the common notations in this paper. LetΩ,ℱ,P be a probability space. Assume Wt:0≤t≤T and Bt:0≤t≤T be two mutually independent standard Brownian motions defined on Ω,ℱ,P, with values, respectively, in Rm and in Rd. Note the integral with respect to Wt as the forward Itô’s integral and Bt as the backward Itô’s integral. Let N denote the class of P null sets of ℱ. We define ℱtw=N∨σWr−W0:0≤r≤t, ℱt,TB=N∨σBr−Bt:t≤r≤T, and ℱt=ℱtw∨ℱt,TB for each t∈0,T. We denote M20,T;Rn the set of all classes of (dt×dPa.e. equal) ℱt measurable stochastic process φt satisfying E∫0Tφt2dt<+∞. Similarly, S20,T;Rn denotes the set of continuous n-dimensional ℱt measurable stochastic process φt satisfying Esupt∈0,Tφt2<+∞. Eℱt⋅=Eℱt denotes the conditional expectation under filtration ℱt. ⋅,⋅ denotes the scalar product, and ⊤ in the superscripts means the transpose of the matrix.In this paper, we mainly investigate the delayed doubly stochastic linear quadratic control system:(1)dxt=A1txt+B1txδt+C1tyt+D1tyδt+E1tut+F1tuδtdt+A2txt+B2txδt+C2tyt+D2tyδt+E2tut+F2tuδt,dWt⟶−ytdBt←,t∈0,T,xt=φt,t∈−δ,0,yt=ψt,t∈−δ,0,ut=0,t∈−δ,0,where the notation xδt=xt−δ, yδt=yt−δ, and uδt=ut−δ.Remark 1.
In this delayed doubly stochastic control system, the state and control variables contain time delay at the same time. Time delay exists all the time in the system. But, we do nothing before the initial time. So, we give the assumption thatut=0 when time t belongs to the interval before the control intervenes.The cost functional is written as(2)Ju⋅=12E∫0TKtxt,xt+Rtyt,yt+Stut,utdt+QxT,xT.For a convex subsetU⊂Rk, we define U0,T as follows:(3)U0,T≔u:0,T×Ω⟶U,E∫0Tut2dt<+∞.Our optimal control problem can be stated as minimizing the cost functional overU0,T. For optimal control u∗⋅ satisfying(4)Ju∗⋅=infu⋅∈U0,TJu⋅,the corresponding x∗⋅,y∗⋅,u∗⋅ is called an optimal triple.The corresponding adjoint equation becomes(5)−dpt=A1⊤tpt+EℱtB1⊤t+δpt+δ+A2⊤tqt−Ktxt+EℱtB2⊤t+δqt+δdt+Rtyt−C1⊤tpt−C2⊤tqt−EℱtD1⊤t+δpt+δ−EℱtD2⊤t+δqt+δdBt←,−qtdWt⟶,t∈0,T,pT=−QxT,pt=0,t∈T,T+δ,qt=0,t∈T,T+δ.We assume that the following conditions hold:(A1) Assume that the coefficient matricesAi, Bi, Ci, Di, Ei, and Fii=1,2 are the matrix process with a proper dimension(A2) The random matrixQ: Ω⟶Rn×n is the nonnegatively bounded symmetric ℱt adapt matrix(A3) All the functions oft are bounded, Kt,Rt,andQ are symmetric nonnegative definite, and St is symmetric uniformly positive definite


## 3. Main Results

Theorem 1.
The function(6)u∗t=S−1tE1⊤tpt+E2⊤tqt+EℱtF1⊤t+δpt+δ+F2⊤t+δqt+δ,t∈0,T, is the unique optimal control for the delayed doubly stochastic linear quadratic optimal control problem, where x∗⋅,y∗⋅,p⋅,q⋅ is the solution of the following system:(7)dxt=A1txt+B1txδt+C1tyt+D1tyδt+E1tut+F1tuδtdt+A2txt+B2txδt+C2tyt+D2tyδt+E2tut+F2tuδtdWt⟶−ytdBt←,t∈0,T,−dpt=A1⊤tpt+EℱtB1⊤t+δpt+δ+A2⊤tqt−Ktxt+EℱtB2⊤t+δqt+δdt+Rtyt−C1⊤tpt−C2⊤tqt−EℱtD1⊤t+δpt+δ−EℱtD2⊤t+δqt+δdBt←−qtdWt⟶,t∈0,T,xt=φt,t∈−δ,0,yt=ψt,t∈−δ,0,ut=0,t∈−δ,0,pT=−QxT,pt=0,t∈T,T+δ,qt=0,t∈T,T+δ.Proof.
The existence and uniqueness of the solution for equation (1) can be guaranteed by Theorem 3.1 in [20] under the assumptions (A1)–(A3). Equation (5) is an anticipated backward doubly stochastic differential equation. Its existence and uniqueness can be deduced by Theorem 3.2 in [18]. Now, we prove that u∗t is the optimal control. For all v⋅∈U0,T, let x∗⋅,y∗⋅ and xv⋅,yv⋅ be the trajectory of the system corresponding to u∗t and vt, respectively.
Then,(8)Jv⋅−Ju∗⋅=12E∫0TKtxvt,xvt−Ktx∗t,x∗t+Rtyvt,yvt−Rty∗t,y∗+Stvt,vt−Stu∗t,u∗tdt+QxvT,xvT−Qx∗T,x∗T,=12E∫0TKtxvt−x∗t,xvt−x∗t+Stvt−u∗t,vt−u∗t+Rtyvt−y∗t,yvt−y∗t+2Ktx∗t,xvt−x∗t+2Rty∗t,yvt−y∗t+2Stu∗t,vt−u∗tdt+QxvT−x∗T,xvT−x∗T+2Qx∗T,xvT−x∗T.
From the definitions ofKt, Rt, St, and Q, we know Kt, Rt, and Q are symmetric nonnegative definite and St is symmetric uniformly positive definite. So, we have(9)Jv⋅−Ju∗⋅≥E∫0TKtx∗t,xvt−x∗t+Stu∗t,vt−u∗t+Rty∗t,yvt−y∗tdt+Qx∗T,xvT−x∗T.
Applying the Itô–Doeblin formula to(10)Qx∗T,xvT−x∗T=−pT,xvT−x∗T,and paying attention to the initial condition and the terminal condition, we have(11)EpT,xvT−x∗T=E∫0Tpt,A1txvt−x∗t+B1txδvt−xδ∗t+C1tyvt−y∗t+D1tyδvt−yδ∗t+E1tvt−u∗t+F1tvδt−uδ∗tdt−∫0TA1⊤tpt+A2⊤tqt−Ktx∗t+EℱtB1⊤t+δpt+δ+EℱtB2⊤t+δqt+δ,xvt−x∗tdt+∫0Tqt,A2txvt−x∗t+B2txδvt−xδ∗t+C2tyvt−y∗t+D2tyδvt−yδ∗t+E2tvt−u∗t+F2tvδt−uδ∗tdt+∫0TRty∗t−C1⊤tpt−C2⊤tqt−EℱtD1⊤t+δpt+δ−EℱtD2⊤t+δqt+δ,yvt−y∗tdt.
In fact, we have(12)E∫0Tpt,B1txδvt−xδ∗t−EℱtB1⊤t+δpt+δ,xvt−x∗tdt=E∫0Tpt,B1txδvt−xδ∗tdt−∫δT+δB1⊤tpt,xδvt−xδ∗tdt=E∫0δpt,B1txδvt−xδ∗tdt−∫TT+δB1⊤tpt,xδvt−xδ∗tdt=0.
So, we have(13)E−pT,xvT−x∗T=E∫0T−Ktx∗t,xvt−x∗t+−Rty∗t,yvt−y∗t+−pt,E1tvt−u∗t+F1tvδt−uδ∗t+−qt,E2tvt−u∗t+F2tvδt−uδ∗tdt.
Then,(14)Jv⋅−Ju∗⋅≥E∫0TStu∗t,vt−u∗t+−pt,E1tvt−u∗t+F1tvδt−uδ∗t+−qt,E2tvt−u∗t+F2tvδt−uδ∗tdt.
So from the definition ofu∗t, we have(15)Jv⋅−Ju∗⋅≥0,for any v⋅∈U0,T. This shows that u∗t is the optimal control.
Next, we will prove the uniqueness. Assume thatu1⋅ and u2⋅ are both optimal controls. x1⋅,y1⋅ and x2⋅,y2⋅ are the trajectories corresponding to u1⋅ and u2⋅, respectively. Equation (5) is a new type of the anticipated backward doubly stochastic differential equation. The existence and uniqueness of the solution for the equation can be guaranteed by Theorem 3.2 in [18]. By the uniqueness of the solution of the equation, we know that x1⋅+x2⋅/2,y1⋅+y2⋅/2 is the trajectory corresponding to u1⋅+u2⋅/2. From the definition of Kt, Rt, St, and Q, we know Ju1⋅=Ju2⋅=α≥0.
Then,(16)2α=Ju1⋅+Ju2⋅=Ju1⋅+u2⋅2+E∫0TStu1t−u2t2,u1t−u2t2+Ktx1t−x2t2,x1t−x2t2+Rty1t−y2t2,y1t−y2t2dt+Qx1T−x2T2,x1T−x2T2≥2α+E∫0TStu1t−u2t2,u1t−u2t2dt.
From the definition ofSt, we have u1⋅=u2⋅.
We complete the proof of Theorem1.Next, we study a special class of the delayed doubly stochastic LQ problem. We discuss the case that only the control contained the delayed variable, and the initial value of the state variableη is deterministic. The delayed system can be written as(17)dxt=A1txt+C1tyt+F1tuδtdt+A2txt+C2tyt+F2tuδtdWt⟶−ytdBt←,t∈0,T,−dpt=A1⊤tpt+A2⊤tqt−Ktxtdt+Rtyt−C1⊤tpt−C2⊤tqtdBt←−qtdWt⟶,t∈0,T,x0=η,ut=0,t∈−δ,0,pT=−QxT,pt=0,t∈T,T+δ,qt=0,t∈T,T+δ.We also consider the optimal control problem (4) under the classical quadratic index (4).From Theorem1, we can deduce the optimal control directly:(18)u∗t=S−1tEℱtF1⊤t+δpt+δ+F2⊤t+δqt+δ,t∈0,T.Peng [21] gave the solution of a kind of stochastic Hamiltonian systems by the Riccati equation corresponding to the system. Following this way, we want to use the solution of the Riccati equation to introduce the optimal control for the delayed doubly stochastic LQ problem. First, we give the generalized matrix Riccati equation:(19)−M˙t=MtA1t+A1⊤tMt+MtC1tGt+A2⊤tNt−Kt+MtF1tS−1t−δF1⊤tMt+F2⊤tNt,MtGt=RtGt−C1⊤tMt−C2⊤tNt,Nt=MtA2t+MtF2tS−1t−δF1⊤tMt+F2⊤tNt+MtC2tGt.Theorem 2.
Let the assumptions (A1)–(A3) be satisfied. If the Riccati equation has a solutionG⋅,M⋅,N⋅, then system (17) has a unique solution:(20)xt,yt,pt,qt=xt,Gtxt,Mtxt,Ntxt,where xt is solved by(21)dxt=A1t+C1tGt+F1tS−1t−δEℱt−δF1⊤tpt+F2⊤tqtxtdt+A2t+C2tGt+F2tS−1t−δEℱt−δF1⊤tpt+F2⊤tqtxtdWt⟶−ytdBt←,t∈0,T,x0=η,ut=0,t∈−δ,0.Proof.
We apply Itô’s formula toMtxt and compare the coefficient with pt, then we can deduce the conclusion directly. Next, we will prove the uniqueness.
Assume thatxt,yt,pt,qt is a solution of system (17), with the conditions x0=η and PT=−QxT.
We sety¯t=Gtxt, p¯t=Mtxt, and q¯t=Ntxt. By differentiating p¯t, we have(22)dp¯t=M˙txtdt+Mtdxt=M˙txtdt+MtA1txt+C1tyt+F1tuδtdt+MtA2txt+C2tyt+F2tuδtdWt⟶−MtytdBt←.
Substituting the first and the second equality of the Riccati equation (19) into (22), we have(23)dp¯t=−A1⊤tp¯t+MtC1ty¯t−yt+A2⊤tq¯t−Ktxtdt+q¯t−MtC2ty¯t−ytdWt⟶−MtytdBt←.
On the other hand, from the Riccati equation (19), we have(24)Mtyt=Mty¯t−y¯t−yt=MtGtxt−Mty¯t−yt=RtGt−C1⊤tMt−C2⊤tNtxt−Mty¯t−yt=Rty¯t−C1⊤tp¯t−C2⊤tq¯t−Mty¯t−yt.
Then, equality (23) can be written as(25)dp¯t=−A1⊤tp¯t+MtC1ty¯t−yt+A2⊤tq¯t−Ktxtdt+q¯t−MtC2ty¯t−ytdWt⟶−Rty¯t−C1⊤tp¯t−C2⊤tq¯t−Mty¯t−ytdBt←.
We denotey^t,p^t,q^t=y¯t−yt,p¯t−pt,q¯t−qt, then we have(26)−dp^t=−dp¯t−−dpt=A1⊤tp^t+A2⊤tq^t+MtC1ty^tdt+Rty^t−C1⊤tp^t−C2⊤tq^t−Mty^tdBt←−q^t−MtC2ty^tdWt⟶.
Now, we setq^t−MtC2ty^t=q′t, that is, q^t=MtC2ty^t+q′t, then equation (26) can be written as(27)−dp^t=A1⊤tp^t+MtC1t+A2⊤tMtC2ty^t+A2⊤tq′tdt+Rt−C2⊤tMtC2t−Mty^t−C1⊤tp^t−C2⊤tq′tdBt←−q′tdWt⟶.
This is a linear backward doubly stochastic differential equation. This is a unique solutiony^t,p^t,q′t≡0,0,0, that is, y^t,p^t,q^t≡0,0,0. So, we have yt=Gtxt, pt=Mtxt, and qt=Ntxt.
We complete the proof of Theorem2.Theorem 3.
Let the assumptions (A1)–(A3) be satisfied. Assume thatG⋅,M⋅,N⋅ satisfies the generalized Riccati equation, then the optimal control of the delayed doubly stochastic linear quadratic optimal control problem has the following form:(28)u∗t=S−1tEℱtF1⊤t+δMt+δ+F2⊤t+δNt+δxt+δ,(29)Ju∗=−12M0η,η.Proof.
From the assumption thatG⋅,M⋅,N⋅ is the solution of the Riccati equation (19) set yt=Gtxt, pt=Mtxt, and qt=Ntxt. Applying Itô’s formula to pt, we have(30)dpt=M˙txt+MtA1txt+MtC1tyt+MtF1tuδtdt+MtA2txt+MtC2tyt+MtF2tuδtdWt⟶−MtytdBt←,t∈0,T.
From the definition of the matrix Riccati equation (19), we have(31)dpt=−A1⊤tMt+MtA1t+MtC1tGt+A2⊤tNt−Kt+MtF1tS−1t−δF1⊤tMt+F2⊤tNtxt+MtA1txt+MtC1tyt+MtF1tuδtdt+Nt−MtC2tGt−MtF2tS−1t−δF1⊤tMt+F2⊤tNtxt+MtC2tyt+MtF2tuδtdWt⟶−RtGt−C1⊤tMt−C2⊤tNtxtdBt←,t∈0,T.
That is,(32)−dpt=A1⊤tpt+A2⊤tqt−Ktxtdt+Rtyt−C1⊤tpt−C2⊤tqtdBt←−qtdWt⟶,t∈0,T.
So, we know that the solution of system (17) satisfied the formula yt=Gtxt, pt=Mtxt, and qt=Ntxt. Then, the optimal control can be written as (28).
Next, we deduce the objective functionJu∗⋅ by the solution of the Riccati equation and the initial value of the state variable.
Applying Itô’s formula toxt,pt and taking expectation, we have(33)ExT,pT−x0,p0=ExT,−QxT−η,M0η,(34)ExT,pT−x0,p0=E∫0TKtxt,xt+Rtyt,yt+F1tuδt,pt+F2tuδt,qtdt=E∫0TEℱtF1t+δut,pt+δ+EℱtF2t+δut,qt+δ+Ktxt,xt+Rtyt,ytdt.
Substituting equalities (33) and (34) into the function Ju⋅, we can deduce(35)Ju⋅=12E∫0TKtxt,xt+Rtyt,yt+Stut,utdt+QxT,xT=−12M0η,η.
We complete the proof of Theorem3.


## 4. Applications

Case 1.
For the general delayed doubly stochastic LQ system (1), when the coefficients are(36)Bit=Dit=Fit=0,i=1,2,this is the system without time delay. The optimal control deduced by Theorem 1 is consistent with the results in [22].Case 2.
WhenFit=0i=1,2 in system (1), this is the case that the system does not contain time delay variables in control variables. The optimal control deduced by Theorem 1 is(37)u∗t=S−1tE1⊤tpt+E2⊤tqt.
This is consistent with the result in [20]. From the expression of the optimal control, the factors affecting the optimal control are the same as that in [22].Case 3.
For the delayed doubly stochastic LQ system (17), when the optimal control deduced from Theorem 3 is(38)u∗t=S−1tF1⊤tMt+F2⊤tNtxt,the optimal control is derived from the solution of the matrix Riccati equation. This is consistent with the result in [21].
In this paper, we are mainly concerned about the delayed doubly stochastic LQ system in which all the variables contained time delay. We deduced the explicit form of the optimal control for the general case. The results of this paper extend our previous work and enrich the content of the delayed doubly stochastic LQ problem. But, from the conclusion of the article, there are still some contents that need further study. We only discuss the matrix-valued Riccati equation corresponding to the special cases. For a more general Riccati equation of the delayed doubly stochastic LQ system, we will work hard to increase the research on this part in our future work.



---

*Source: 2759580-2020-06-06.xml*