# Maximum Principle for Stochastic Recursive Optimal Control Problems Involving Impulse Controls

Authors: Zhen Wu; Feng Zhang
Journal: Abstract and Applied Analysis
 (2012)

---

## Abstract

We consider a stochastic recursive optimal control problem in which the control variable has two components: the regular control and the impulse control. The control variable does not enter the diffusion coefficient, and the domain of the regular controls is not necessarily convex. We establish necessary optimality conditions, of the Pontryagin maximum principle type, for this stochastic optimal control problem. Sufficient optimality conditions are also given. The optimal control is obtained for an example of linear quadratic optimization problem to illustrate the applications of the theoretical results.


---

## Body

## 1. Introduction

The nonlinear backward stochastic differential equations (BSDEs for short) were first introduced by Pardoux and Peng [1]. Independently, Duffie and Epstein [2] introduced BSDEs under economic background. In [2], they presented a stochastic recursive utility which is an extension of the standard additive utility with the instantaneous utility depending not only on the instantaneous consumption rate but also on the future utility. Actually, it corresponds to the solution of a particular BSDE whose generator does not depend on the variable z. And then, El Karoui et al. [3] gave the formulation of recursive utilities from the BSDE point of view. The problem that the cost function of the control system is described by the solution of BSDE is called the stochastic recursive optimal control problem. In this case, the control systems become forward-backward stochastic differential equations (FBSDEs).One fundamental research direction for optimal control problem is to establish the necessary optimality conditions—Pontryagin maximum principle. Stochastic maximum principle for forward, backward, and forward-backward systems has been studied by many authors, including Peng [4, 5], Tang and Li [6], Wang and Yu [7], Wu [8], and Xu [9] for full information and Huang et al. [10], Wang and Wu [11], Wang and Yu [12], and Wu [13] for partial information case. However, in these papers, there are only regular controls in the control systems and impulse controls are not included.Stochastic impulse control problems have received considerable research attention in recent years due to wide applicability in a number of different areas, especially in mathematical finance; see, for example, [14–17]. In most cases, the optimal impulse control problem was studied through dynamic programming principle. It was shown in particular that the value function is a solution of some quasi-variational inequalities.The first result in stochastic maximum principle for singular control problem was obtained by Cadenillas and Haussmann [18], in which linear dynamics, convex cost criterion, and convex state constraint are assumed. Bahlali and Chala [19] generalized [18] to the nonlinear dynamics case with a convex state constraint. Bahlali and Mezerdi [20] considered a stochastic singular control problem in which the control system is governed by a stochastic differential equation where the regular control enters the diffusion coefficient and the control domain is not necessarily convex. The stochastic maximum principle was obtained with the approach developed by Peng [4]. Dufour and Miller [21] studied a stochastic singular control problem in which the admissible control is of bounded variation. It is worth pointing out that the control systems in these works are stochastic differential equations with singular control, and few examples are given to illustrate the theoretical results. Wu and Zhang [22] were the first to study stochastic optimal control problems of forward-backward systems involving impulse controls, and they obtained both the maximum principle and sufficient optimality conditions for the optimal control problem.In this paper, we continue to study stochastic optimal control problem involving impulse controls, in which the control system is described by a forward-backward stochastic differential equation and the control variable consists of regular control and impulse control. Different from [22], it is assumed in this paper that the domain of the regular controls is not necessarily convex and the control variable does not enter the diffusion coefficient. Thus the result of this paper and that of [22] do not contain each other. We obtain the stochastic maximum principle by using a spike variation on the regular part of the control and a convex perturbation on the impulsive one. Sufficient optimality conditions are also obtained which can help to find the optimal control in applications.The rest of this paper is organized as follows. In Section2 we give some preliminary results and the formulation of our stochastic optimal control problem. In Section 3 we obtain the maximum principle for our stochastic optimal control problem. Sufficient optimality conditions for the optimal control problem is established in Section 4, and an example of linear quadratic optimization problem is also given to illustrate the applications of our theoretical results.


## 2. Formulation of the Stochastic Optimal Control Problem

Firstly we introduce some notations. Let (Ω,ℱ,ℙ) be a probability space and 𝔼 the expectation with respect to ℙ. Let T be a finite time horizon and ℱt the natural filtration of a d-dimensional standard Brownian motion {Bt,0≤t≤T} augmented by the ℙ-null sets of ℱ. For n∈ℕ and p>1, denote by Sp(ℝn) the set of n-dimensional adapted processes {φt,0≤t≤T} such that 𝔼[sup0≤t≤T|φt|p]<∞, and denote by Hp(ℝn) the set of n-dimensional adapted processes {ψt,0≤t≤T} such that 𝔼[(∫0T|ψt|2dt)p/2]<∞.LetU be a nonempty subset of ℝk and K a nonempty convex subset of ℝn. Let {τi} be a given sequence of increasing ℱt-stopping times such that τi↑+∞ as i→∞. We denote by ℐ the class of right continuous processes η(·)=∑i≥1ηi1[τi,T](·) such that each ηi is an ℱτi-measurable random variable. It is worth noting that the assumption τi↑+∞ implies that at most finitely many impulses may occur on [0,T]. Denote by 𝒰 the class of adapted processes v:[0,T]×Ω→U such that 𝔼[sup0≤t≤T|vt|3]<∞, and denote by 𝒦 the class of K-valued impulse processes η(·)∈ℐ such that 𝔼[(∑i≥1|ηi|)3]<∞. We call 𝒜:=𝒰×𝒦 the admissible control set. In what follows, for a continuous function l(·), the integration ∫0Tl(t)dηt is understood as follows: (2.1)∫0Tl(t)dηt=∑0≤τi≤Tl(τi)ηi.Givenη(·)∈ℐ and x∈ℝn, we consider the following SDE with impulses: (2.2)dXt=b(t,Xt)dt+σ(t,Xt)dBt+Ctdηt,X0=x,
where b:[0,T]×Ω×ℝn→ℝn, σ:[0,T]×Ω×ℝn→ℝn×d, and C:[0,T]→ℝn×n are measurable mappings. Similar to [22, Proposition 2.1], we have the following.Proposition 2.1.
LetC be continuous and b, σ uniformly Lipschitz in x. Assume that b(·,0)∈Hp(ℝn), σ(·,0)∈Hp(ℝn×d), and 𝔼[(∑i≥1|ηi|)p]<∞ for some p≥2. Then SDE (2.2) admits a unique solution X(·)∈Sp(ℝn).Forη(·)∈ℐ, let us consider the following BSDE with impulses: (2.3)dYt=-f(t,Yt,Zt)dt+ZtdBt-Dtdηt,YT=ζ,
where ζ∈ℱT, f:[0,T]×Ω×ℝm×ℝm×d→ℝm and D:[0,T]→ℝm×n are measurable mappings. Similar to [22, Proposition 2.2], we have the following.Proposition 2.2.
LetD be continuous and f Lipschitz in (y,z). Assume that 𝔼|ζ|p<∞, 𝔼[(∑i≥1|ηi|)p]<∞, and f(·,0,0)∈Hp(ℝm) for some p≥2. Then BSDE (2.3) admits a unique solution (Y(·),Z(·))∈Sp(ℝm)×Hp(ℝm×d).The control system of our stochastic optimal control problem is subject to the following FBSDE:(2.4)dxtv,η=b(t,xtv,η,vt)dt+σ(t,xtv,η)dBt+Ctdηt,dytv,η=-f(t,xtv,η,ytv,η,ztv,η,vt)dt+ztv,ηdBt-Dtdηt,x0v,η=a∈Rn,yTv,η=g(xTv,η),
where b:[0,T]×ℝn×U→ℝn, σ:[0,T]×ℝn→ℝn×d, f:[0,T]×ℝn×ℝm×ℝm×d×U→ℝm, g:ℝn→ℝm are measurable mappings, and C:[0,T]→ℝn×n, D:[0,T]→ℝm×n are continuous functions. The objective is to minimize the following cost functional over the class 𝒜: (2.5)J(v(⋅),η(⋅))=E[ϕ(xTv,η)+γ(y0v,η)+∫0Th(t,xtv,η,ytv,η,vt)dt+∑i≥1l(τi,ηi)],
where ϕ:ℝn→ℝ, γ:ℝm→ℝ, h:[0,T]×ℝn×ℝm×U→ℝ, and l:[0,T]×ℝn→ℝ are measurable mappings.In what follows we assume the following.(H1)
b, σ, f, g are continuous, and they are continuously differentiable in (x,y,z), with derivatives continuous and uniformly bounded. Moreover, assume that b and f have linear growth in (x,y,z,v).(H2)
ϕ, γ, h, l are continuous, and they are continuously differentiable in (x,y,η), with derivatives continuous and bounded by c(1+|x|), c(1+|y|), c(1+|x|+|y|+|v|), and c(1+|η|), respectively. Moreover, we assume |h(t,0,0,v)|≤c(1+|v|3) for any (t,v).From Propositions2.1 and 2.2, it follows that FBSDE (2.4) admits a unique solution (xv,η(·),yv,η(·),zv,η(·))∈S3(ℝn)×S3(ℝm)×H3(ℝm×d) for any (v(·),η(·))∈𝒜, and the functional J is well defined.


## 3. Stochastic Maximum Principle for the Optimal Control Problem

Let(u(·),ξ(·)=∑i≥1ξi1[τi,T](·))∈𝒜 be an optimal control and (xu,ξ(·),yu,ξ(·),zu,ξ(·)) the corresponding trajectory. We introduce the spike variation with respect to u(·) as follows: (3.1)utε={v,ifτ≤t≤τ+ε,ut,otherwise,
where τ∈[0,T) is an arbitrarily fixed time, ε>0 is a sufficiently small constant, and v is an arbitrary U-valued ℱτ-measurable random variable such that 𝔼|v|3<∞. Let η(·)∈ℐ be such that ξ(·)+η(·)∈𝒦. Then it is easy to check that ξε(·):=ξ(·)+εη(·), 0≤ε≤1 is also an element of 𝒦. Let us denote by (xε(·),yε(·),zε(·)) the trajectory associated with (uε(·),ξε(·)). For convenience, denote φ(t)=φ(t,xtu,ξ,ytu,ξ,ztu,ξ,ut), φ(utε)=φ(t,xtu,ξ,ytu,ξ,ztu,ξ,utε) for φ=b,σ,f,h,bx,σx,fx,fy,fz,hx,hy. In what follows, we use c to denote a positive constant which can be different from line to line.Let us introduce the following FBSDE (called the variational equation):(3.2)dxt1=[bx(t)xt1+b(utε)-b(t)]dt+σx(t)xt1dBt+εCtdηt,dyt1=-[fx(t)xt1+fy(t)yt1+fz(t)zt1+f(utε)-f(t)]dt+zt1dBt-εDtdηt,x01=0,yT1=gx(xTu,ξ)xT1.
By Propositions 2.1 and 2.2, FBSDE (3.2) admits a unique solution (x1(·),y1(·),z1(·))∈S3(ℝn)×S3(ℝm)×H3(ℝm×d).Similar to [9, Lemma 1], we can easily obtain the following.Lemma 3.1.
We have(3.3)sup⁡0≤t≤TE|xt1|3+sup⁡0≤t≤TE|yt1|3+E[(∫0T|zt1|2dt)3/2]≤cε3.We proceed to give the following lemma.Lemma 3.2.
The following estimations hold:(3.4)sup⁡0≤t≤TE[|xtε-xtu,ξ-xt1|2]≤Cεε2,(3.5)sup⁡0≤t≤TE[|ytε-ytu,ξ-yt1|2]≤Cεε2,(3.6)E[∫0T|ztε-ztu,ξ-zt1|2dt]≤Cεε2,
where Cε→0 as ε→0.Proof.
It is easy to check that(3.7)xtε-xtu,ξ-xt1=∫0t[Csε(xsε-xsu,ξ-xs1)+Asε]ds+∫0t[Dsε(xsε-xsu,ξ-xs1)+Bsε]dBs,
where
(3.8)Asε=∫01[bx(s,xsu,ξ+λxs1,usε)-bx(s)]dλxs1,Bsε=∫01[σx(s,xsu,ξ+λxs1)-σx(s)]dλxs1,Csε=∫01bx(s,xsu,ξ+xs1+λ(xsε-xsu,ξ-xs1),usε)dλ,Dsε=∫01σx(s,xsu,ξ+xs1+λ(xsε-xsu,ξ-xs1))dλ.
Since bx, σx are uniformly bounded, we have sup0≤s≤T(|Csε|+|Dsε|)≤c. Hence, if we can obtain
(3.9)sup⁡0≤t≤TE[∫0tAsεds+∫0tBsεdBs]2≤Cεε2,
then the estimation (3.4) can be obtained from Gronwall's lemma and (3.7). Let us take the Aε term for example. By the definition of uε and Hölder's inequality, we have
(3.10)sup⁡0≤t≤TE[∫0tAsεds]2≤2E[∫0T|∫01[bx(s,xsu,ξ+λxs1,us)-bx(s)]dλxs1|ds]2+2E[∫ττ+ε|∫01[bx(s,xsu,ξ+λxs1,v)-bx(s)]dλxs1|ds]2=:2I+2II.
From Hölder's inequality, Lemma 3.1, and the dominated convergence theorem, it follows that
(3.11)I≤TE{∫0T|∫01[bx(s,xsu,ξ+λxs1,us)-bx(s)]dλxs1|2ds}≤T∫0T{E|xs1|3}2/3{E[|∫01[bx(s,xsu,ξ+λxs1,us)-bx(s)]dλ|6]}1/3ds≤T5/3{sup⁡0≤s≤TE|xs1|3}2/3{∫0TE[|∫01[bx(s,xsu,ξ+λxs1,us)-bx(s)]dλ|6]ds}1/3≤Cεε2.
Since bx is uniformly bounded, by Lemma 3.1 we get
(3.12)II≤ε∫ττ+εE[|∫01[bx(s,xsu,ξ+λxs1,v)-bx(s)]dλxs1|2]ds≤cε2sup⁡0≤s≤TE|xs1|2≤cε4.
Thus we obtain sup0≤t≤T𝔼[∫0tAsεds]2≤Cεε2. In the same way we can get
(3.13)sup⁡0≤t≤TE[∫0tBsεdBs]2≤Cεε2.
Hence, the estimation (3.4) is proved.
Now we prove (3.5) and (3.6). Set
(3.14)Xsε=xsε-xsu,ξ-xs1,Ysε=ysε-ysu,ξ-ys1,Zsε=zsε-zsu,ξ-zs1,Πsε=(s,xsu,ξ+xs1+λXsε,ysu,ξ+ys1+λYsε,zsu,ξ+zs1+λZsε,usε),Λsε=(s,xsu,ξ+λxs1,ysu,ξ+λys1,zsu,ξ+λzs1,usε).
It is easy to obtain
(3.15)Ytε=g(xTε)-g(xTu,ξ)-gx(xTu,ξ)xT1-∫tTZsεdBs+∫tT(Es1,εxs1+Es2,εys1+Es3,εzs1)ds+∫tT(Fs1,εXsε+Fs2,εYsε+Fs3,εZsε)ds,
where
(3.16)Es1,ε=∫01[fx(Λsε)-fx(s)]dλ,Es2,ε=∫01[fy(Λsε)-fy(s)]dλ,Es3,ε=∫01[fz(Λsε)-fz(s)]dλ,Fs1,ε=∫01fx(Πsε)dλ,Fs2,ε=∫01fy(Πsε)dλ,Fs3,ε=∫01fz(Πsε)dλ.
We have
(3.17)g(xTε)-g(xTu,ξ)-gx(xTu,ξ)xT1=[g(xTε)-g(xTu,ξ+xT1)]+[g(xTu,ξ+xT1)-g(xTu,ξ)-gx(xTu,ξ)xT1]=∫01gx(xTu,ξ+xT1+λXTε)dλXTε+∫01[gx(xTu,ξ+λxT1)-gx(xTu,ξ)]dλxT1=:I+II.
Since gx is uniformly bounded, it follows from (3.4) that 𝔼|I|2≤c𝔼|XTε|2≤Cεε2. Since gx is continuous and uniformly bounded, from Lemma 3.1 and the dominated convergence theorem it follows that
(3.18)E|II|2≤{sup⁡0≤t≤TE|xt1|3}2/3{E[|∫01(gx(xTu,ξ+λxT1)-gx(xTu,ξ))dλ|6]}1/3≤Cεε2.
Consequently,
(3.19)E[|g(xTε)-g(xTu,ξ)-gx(xTu,ξ)xT1|2]≤2E|I|2+2E|II|2≤Cεε2.
From Lemma 3.1 and the dominated convergence theorem, it follows that
(3.20)sup⁡0≤t≤TE[∫tT(Es1,εxs1+Es2,εys1+Es3,εzs1)ds]2≤Cεε2.
Since fx, fy, and fz are uniformly bounded, we have
(3.21)sup⁡0≤t≤T[|Fs1,ε|+|Fs2,ε|+|Fs3,ε|]≤c.
Similar to the proof of Lemma 1 in [9] for the BSDE part, we can obtain (3.5) and (3.6) with the iterative method.We are now ready to state the variational inequality.Lemma 3.3.
The following variational inequality holds:(3.22)E[ϕx(xTu,ξ)xT1+γy(y0u,ξ)y01+ε∑i≥1lξ(τi,ξi)ηi]+E{∫0T[hx(t)xt1+hy(t)yt1+h(utε)-h(t)]dt}≥o(ε).Proof.
From the optimality of(u(·),ξ(·)), we have
(3.23)J(uε(⋅),ξε(⋅))-J(u(⋅),ξ(⋅))≥0.
From Lemmas 3.1 and 3.2, it follows that
(3.24)E[ϕ(xTε)-ϕ(xTu,ξ+xT1)]=o(ε),E[ϕ(xTu,ξ+xT1)-ϕ(xTu,ξ)]=E[ϕx(xTu,ξ)xT1]+o(ε).
Hence,
(3.25)E[ϕ(xTε)-ϕ(xTu,ξ)]=E[ϕx(xTu,ξ)xT1]+o(ε).
Similarly we get
(3.26)E[γ(y0ε)-γ(y0u,ξ)]=E[γy(y0u,ξ)y01]+o(ε),E[∑i≥1l(τi,ξi+εηi)-∑i≥1l(τi,ξi)]=εE[∑i≥1lξ(τi,ξi)ηi]+o(ε),
while
(3.27)E{∫0T[h(t,xtε,ytε,utε)-h(t)]dt}=E{∫0T[h(t,xtε,ytε,utε)-h(t,xtu,ξ+xt1,ytu,ξ+yt1,utε)]dt}+E{∫0T[h(t,xtu,ξ+xt1,ytu,ξ+yt1,utε)-h(utε)]dt}+E{∫0T[h(utε)-h(t)]dt}:=I+II+E{∫0T[h(utε)-h(t)]dt}.
Since hx,hy,hz have linear growth, it follows from Lemma 3.2 and Hölder's inequality that
(3.28)I=E{∫0T∫01[hx(Πtε)Xtε+hy(Πtε)Ytε]dλdt}=o(ε).
By Lemma 3.1 and the dominated convergence theorem, we have
(3.29)II=E{∫0T∫01[hx(Λtε)xt1+hy(Λtε)yt1]dλdt}=E{∫0T[hx(utε)xt1+hy(utε)yt1]dt}+o(ε)=E{∫0T[(hx(utε)-hx(t))xt1+(hy(utε)-hy(t))yt1]dt}+E{∫0T[hx(t)xt1+hy(t)yt1]dt}+o(ε)=E{∫ττ+ε[(hx(t,v)-hx(t))xt1+(hy(t,v)-hy(t))yt1]dt}+E{∫0T[hx(t)x1+hy(t)y1]dt}+o(ε),
where φ(t,v)=φ(t,xtu,ξ,ytu,ξ,v), φ=hx,hy. It follows from Hölder's inequality that
(3.30)II≤{E∫ττ+ε|hx(t,v)-hx(t)|2dt}1/2{E∫0T|xt1|2dt}1/2+{E∫ττ+ε|hy(t,v)-hy(t)|2dt}1/2{E∫0T|yt1|2dt}1/2+E{∫0T[hx(t)xt1+hy(t)yt1]dt}+o(ε).
Using Lemma 3.1 again, we get
(3.31)II≤E{∫0T[hx(t)xt1+hy(t)yt1]dt}+o(ε).
Consequently,
(3.32)E{∫0T[h(t,xtε,ytε,utε)-h(t)]dt}=E{∫0T[hx(t)xt1+hy(t)yt1+h(utε)-h(t)]dt}+o(ε).
The variational inequality follows from (3.25)–(3.32).Now we introduce the following FBSDE (called the adjoint equation):(3.33)dpt=[fy*(t)pt-hy*(t)]dt+fz*(t)ptdBt,dqt=[fx*(t)pt-bx*(t)qt-σx*(t)kt-hx*(t)]dt+ktdBt,p0=-γy*(y0u,ξ),qT=-gx*(xTu,ξ)pT+ϕx*(xTu,ξ).It is easy to check that the adjoint equation admits a unique solution(p(·),q(·),k(·))∈S3(ℝm)×S3(ℝn)×H3(ℝm×d).We are now in a position to state the stochastic maximum principle.Theorem 3.4.
Let(u(·),ξ(·)) be an optimal control, (xu,ξ(·),yu,ξ(·),zu,ξ(·)) the corresponding trajectory, and (p(·),q(·),k(·)) the solution of the adjoint equation. Then for any v∈U and η(·)∈𝒦 it holds that
(3.34)H(t,xtu,ξ,ytu,ξ,ztu,ξ,v,pt,qt,kt)-H(t,xtu,ξ,ytu,ξ,ztu,ξ,ut,pt,qt,kt)≥0,a.e., a.s.,(3.35)E{∑i≥1[(lξ(τi,ξi)+qτi*Cτi-pτi*Dτi)(ηi-ξi)]}≥0,
where H:[0,T]×ℝn×ℝm×ℝm×d×U×ℝm×ℝn×ℝn×d→ℝ is defined by
(3.36)H(t,x,y,z,v,p,q,k)=-〈p,f(t,x,y,z,v)〉+〈q,b(t,x,v)〉+〈k,σ(t,x)〉+h(t,x,y,v).Proof.
Applying Itô's formula to〈pt,yt1〉+〈qt,xt1〉, by Lemma 3.3 we derive
(3.37)E{∫0T[H(t,xtu,ξ,ytu,ξ,ztu,ξ,utε,pt,qt,kt)-H(t,xtu,ξ,ytu,ξ,ztu,ξ,ut,pt,qt,kt)]dt}+εE{∑i≥1[(lξ(τi,ξi)+qτi*Cτi-pτi*Dτi)ηi]}≥o(ε),
where η(·)∈ℐ satisfies ξ(·)+η(·)∈𝒦. Dividing (3.37) by ε and letting ε go to 0, we obtain
(3.38)E[H(τ,xτu,ξ,yτu,ξ,zτu,ξ,v,pτ,qτ,kτ)-H(τ,xτu,ξ,yτu,ξ,zτu,ξ,uτ,pτ,qτ,kτ)]+E{∑i≥1[(lξ(τi,ξi)+qτi*Cτi-pτi*Dτi)ηi]}≥0,a.e.τ∈[0,T].
By choosing v=uτ in (3.38) we obtain the conclusion (3.35). If we choose η(·)≡0, then for v∈ℱτ satisfying 𝔼|v|3<∞ we have
(3.39)E[H(τ,xτu,ξ,yτu,ξ,zτu,ξ,v,pτ,qτ,kτ)-H(τ,xτu,ξ,yτu,ξ,zτu,ξ,uτ,pτ,qτ,kτ)]≥0.
Now let us set vτ=v1A+uτ1A̅ for any v∈U and A∈ℱτ. Then it is obvious that vτ∈ℱτ and 𝔼|vτ|3<∞. So from (3.39) it follows that, for any A∈ℱτ,
(3.40)E{1A[H(τ,xτu,ξ,yτu,ξ,zτu,ξ,v,pτ,qτ,kτ)-H(τ,xτu,ξ,yτu,ξ,zτu,ξ,uτ,pτ,qτ,kτ)]}≥0.
Hence,
(3.41)E{[H(τ,xτu,ξ,yτu,ξ,zτu,ξ,v,pτ,qτ,kτ)-H(τ,xτu,ξ,yτu,ξ,zτu,ξ,uτ,pτ,qτ,kτ)]∣Fτ}≥0,∀v∈U.
Since the quantity inside the conditional expectation is ℱτ-measurable, the conclusion (3.34) can be obtained easily.Similar to [22, Corollary 3.1], by Theorem 3.4 we can easily obtain the followingCorollary 3.5.
AssumeK=ℝn. Then for the optimal control (u(·),ξ(·)) it holds that
(3.42)H(t,xtu,ξ,ytu,ξ,ztu,ξ,v,pt,qt,kt)-H(t,xtu,ξ,ytu,ξ,ztu,ξ,ut,pt,qt,kt)≥0,∀v∈U,a.e., a.s.,lξ(τi,ξi)+qτi*Cτi-pτi*Dτi=0,i≥1,a.s..Remark 3.6.
We can still obtain the stochastic maximum principle if the assumptions are relaxed in the following way.(i)
The regular control processv(·) and the impulse control process η(·) are assumed to satisfy 𝔼[sup0≤t≤T|vt|p]<∞ and 𝔼[∑i≥1|ηi|p]<∞ for some p∈(2,3).(ii)
The assumption|h(t,0,0,v)|≤c(1+|v|3) in Hypothesis (H2) can be weakened as |h(t,0,0,v)|≤c(1+|v|p).(iii)
In the spike variation setting, the random variablev is assumed to satisfy 𝔼|v|p<∞.
In fact, under these new assumptions both the solutions of the control system (2.4) and the variational equation (3.2) belong to Sp(ℝn)×Sp(ℝm)×Hp(ℝm×d). The conclusion of Lemma 3.1 becomes
(3.43)sup⁡0≤t≤TE|xt1|p+sup⁡0≤t≤TE|yt1|p+E[(∫0T|zt1|2dt)p/2]≤cεp.
And Lemmas 3.2 and 3.3 still hold true.


## 4. Sufficient Optimality Conditions for Optimal Controls

We still denote by(xv,η(·),yv,η(·),zv,η(·)) the trajectory corresponding to (v(·),η(·))∈𝒜. Let us first introduce an additional assumption.(H3) The control domain U is a convex body in ℝk. The maps b, f, and h are locally Lipschitz in the regular control variable v.Theorem 4.1.
Let (H1)–(H3) hold. Assume that the functionsϕ, γ, η→l(t,η) and (x,y,z,v)→H(t,x,y,z,v,p,q,k) are convex. Moreover, yTv,η has the following particular form: yTv,η=KxTv,η+ζ for K∈ℝm×n and ζ∈L3(Ω,ℱT,ℙ;ℝm). Let (pu,ξ,qu,ξ,ku,ξ) be the solution of the adjoint equation associated with (u,ξ)∈𝒜. Then (u,ξ) is an optimal control of the stochastic optimal control problem if it satisfies (3.34) and (3.35).Proof.
SetJ̃=J(v(·),η(·))-J(u(·),ξ(·)). Since ϕ, γ, η→l(t,η) are convex, we have
(4.1)ϕ(xTv,η)-ϕ(xTu,ξ)≥ϕx(xTu,ξ)(xTv,η-xTu,ξ),γ(y0v,η)-γ(y0u,ξ)≥γy(y0u,ξ)(y0v,η-y0u,ξ),∑i≥1l(τi,ηi)-∑i≥1l(τi,ξi)≥∑i≥1[lξ(τi,ξi)(ηi-ξi)].
Thus,
(4.2)J̃≥E{ϕx(xTu,ξ)(xTv,η-xTu,ξ)+γy(y0u,ξ)(y0v,η-y0u,ξ)}+E{∫0T[h(t,xtv,η,ytv,η,vt)-h(t,xtu,ξ,ytu,ξ,ut)]dt}+E{∑i≥1[lξ(τi,ξi)(ηi-ξi)]}.
Set ℋv,η(t):=H(t,xtv,η,ytv,η,ztv,η,vt,ptu,ξ,qtu,ξ,ktu,ξ). Then by Itô's formula applied to 〈qtu,ξ,(xtv,η-xtu,ξ)〉+〈ptu,ξ,(ytv,η-ytu,ξ)〉, we get J̃≥Ξ+Θ, where
(4.3)Ξ=E{∑i≥1[(lξ(τi,ξi)+qτi*Cτi-pτi*Dτi)(ηi-ξi)]},Θ=E{∫0T[Hv,η(t)-Hu,ξ(t)-Hxu,ξ(t)(xtv,η-xtu,ξ)-Hyu,ξ(t)(ytv,η-ytu,ξ)-Hzu,ξ(t)(ztv,η-ztu,ξ)]dt{∫0T[Hv,η(t)-Hu,ξ(t)-Hxu,ξ(t)(xtv,η-xtu,ξ)}}.
From (3.35) we have Ξ≥0. By (3.34) and [23, Lemma 2.3-(iii); Chapter 3], we have 0∈∂uℋu,ξ(t). By [23, Lemma 2.4; Chapter 3], we further conclude that
(4.4)(Hxu,ξ(t),Hyu,ξ(t),Hzu,ξ(t),0)∈∂x,y,z,uHu,ξ(t).
Then, by [23, Lemma 2.3-(v); Chapter 3] and the convexity of H(t,.,.,.,.,p,q,k), we obtain
(4.5)Hv,η(t)-Hu,ξ(t)≥Hxu,ξ(t)(xtv,η-xtu,ξ)+Hyu,ξ(t)(ytv,η-ytu,ξ)+Hzu,ξ(t)(ztv,η-ztu,ξ),
from which it follows immediately that Θ≥0. Thus we obtain J̃≥0 and the proof is complete.We now give an example of linear quadratic optimal control problem involving impulse controls to illustrate the application of our theoretical results.Example 4.2.
For simplicity, assume that the variables and coefficients are scalar-valued. Let us takeU={-1,1} and K=ℝ. There are only two values −1 and 1 in U which is a usual case in practice and represents only two control states: “on” and “off”. For (v(·),η(·))∈𝒜, the controlled system is subject to the following linear FBSDE:(4.6)dxt=(Axt+Bvt)dt+CxtdBt+Hdηt,dyt=-(Dxt+Eyt+Fzt+Gvt)dt+ztdBt-Rdηt,x0=a,yT=gxT,
and the cost functional is given by(4.7)J(v(⋅),η(⋅))=12E[WxT2+γy02+∫0T(Mxt2+Nyt2+Qvt2)dt+L∑i≥1ηi2].
The coefficients are deterministic constants such that W,γ,M,N≥0 and Q,L>0. By Propositions 2.1 and 2.2 we know that the control system admits a unique solution (x(·),y(·),z(·))∈S3(ℝ)×S3(ℝ)×H3(ℝ) for any (v,η)∈𝒜. And the functional J is well defined from 𝒜 into ℝ.
Let(u(·),ξ(·)=∑i≥1ξi1[τi,T](·))∈𝒜 be an optimal control and (x(·),y(·),z(·)) the corresponding trajectory. Then the following adjoint equation
(4.8)dpt=(Ept-Nyt)dt+FptdBt,dqt=(Dpt-Aqt-Ckt-Mxt)dt+ktdBt,p0=-γy0,qT=-gpT+WxT
admits a unique solution (p(·),q(·),k(·))∈S3(ℝ)×S3(ℝ)×H3(ℝ). The Hamiltonian H is given by
(4.9)H(t,x,y,z,v,p,q,k)=-p(Dx+Ey+Fz+Gv)+q(Ax+Bv)+kCx+12(Mx2+Ny2+Qv2).
Then by Corollary 3.5 we obtain
(4.10)(-Gpt+Bqt)v+12Qv2≥(-Gpt+Bqt)ut+12Qut2,∀v∈U,a.e., a.s.,(4.11)Lξi+Hqτi-Rpτi=0,i≥1,a.s..
From (4.10) we get
(4.12)ut={1,ifGpt-Bqt≥0,-1,otherwise.
From (4.11) we obtain that
(4.13)ξi=L-1(Rpτi-Hqτi),i≥1,a.s..
Hence, if (u,ξ)∈𝒜 is an optimal control of this linear quadratic control problem, then it satisfies (4.12) and (4.13).
We can prove that(u(·),ξ(·)) obtained in (4.12) and (4.13) is indeed an optimal control of this linear quadratic optimization problem. Note that Theorem 4.1 does not hold now since U is not convex in this example. In what follows, we use the same notations as those in the proof of Theorem 4.1. In fact, as in the proof of Theorem 4.1, we can still derive J(v(·),η(·))-J(u(·),ξ(·))≥Ξ+Θ. On the one hand, it follows from (4.13) that Ξ=0. On the other hand, we have
(4.14)Θ=E{∫0T[Hv,η(t)-H(t,xtv,η,ytv,η,ztv,η,ut,ptu,ξ,qtu,ξ,ktu,ξ)+Φt]dt},
where
(4.15)Φt=H(t,xtv,η,ytv,η,ztv,η,ut,ptu,ξ,qtu,ξ,ktu,ξ)-Hu,ξ(t)-Hxu,ξ(t)(xtv,η-xtu,ξ)-Hyu,ξ(t)(ytv,η-ytu,ξ)-Hzu,ξ(t)(ztv,η-ztu,ξ).
From (4.12) and the definition of H, it is easy to get
(4.16)Hv,η(t)-H(t,xtv,η,ytv,η,ztv,η,ut,ptu,ξ,qtu,ξ,ktu,ξ)=[(-Gpt+Bqt)vt+12Qvt2]-[(-Gpt+Bqt)ut+12Qut2]≥0.
Since M, N≥0, H is convex in (x,y,z), and thus Φt≥0, so we obtain Θ≥0. Consequently, it follows that J(v(·),η(·))-J(u(·),ξ(·))≥0 and the optimality of (u(·),ξ(·)) is proved.Remark 4.3.
For the classical linear quadratic optimal control problem, one can usually obtain an optimal control in a linear state feedback form by virtue of the so-called Riccati equation, and along this line the solvability of the Riccati equation leads to that of the linear quadratic problem. However, it is difficult to obtain a state feedback optimal control in terms of the Riccati equation in Example4.2 mainly due to the particular form of the regular control domain and the appearance of the impulse control in the control system.



---

*Source: 709682-2012-06-20.xml*