# The Relationship between the Stochastic Maximum Principle and the Dynamic Programming in Singular Control of Jump Diffusions

Authors: Farid Chighoub; Brahim Mezerdi
Journal: International Journal of Stochastic Analysis
 (2014)

---

## Abstract

The main objective of this paper is to explore the relationship between the stochastic maximum principle (SMP in short) and dynamic programming principle (DPP in short), for singular control problems of jump diffusions. First, we establish necessary as well as sufficient conditions for optimality by using
the stochastic calculus of jump diffusions and some properties of singular controls. Then, we give, under smoothness conditions, a useful verification theorem and we show that the solution of the adjoint equation coincides with the spatial gradient of the value function, evaluated along the optimal trajectory of the state equation. Finally, using these theoretical results, we solve explicitly an example, on optimal harvesting strategy, for a geometric Brownian motion with jumps.


---

## Body

## 1. Introduction

In this paper, we consider a mixed classical-singular control problem, in which the state evolves according to a stochastic differential equation, driven by a Poisson random measure and an independent multidimensional Brownian motion, of the following form:(1)
d
x
t
=
b
(
t
,
x
t
,
u
t
)
d
t
+
σ
(
t
,
x
t
,
u
t
)
d
B
t
+
∫
E
‍
γ
(
t
,
x
t
-
,
u
t
,
e
)
N
~
(
d
t
,
d
e
)
+
G
t
d
ξ
t
,
x
0
=
x
,
where b, σ, γ, and G are given deterministic functions and x is the initial state. The control variable is a suitable process (
u
,
ξ
), where u
:
[
0
,
T
]
×
Ω
→
A
1
⊂
ℝ
d is the usual classical absolutely continuous control and ξ
:
[
0
,
T
]
×
Ω
→
A
2
=
(
[
0
,
∞
)
)
m is the singular control, which is an increasing process, continuous on the right with limits on the left, with ξ
0
-
=
0. The performance functional has the form
(2)
J
(
u
,
ξ
)
=
E
[
∫
0
T
f
(
t
,
x
t
,
u
t
)
d
t
+
∫
0
T
k
(
t
)
d
ξ
t
+
g
(
x
T
)
]
.The objective of the controller is to choose a couple(
u
⋆
,
ξ
⋆
) of adapted processes, in order to maximize the performance functional.In the first part of our present work, we investigate the question of necessary as well as sufficient optimality conditions, in the form of a Pontryagin stochastic maximum principle. In the second part, we give under regularity assumptions, a useful verification theorem. Then, we show that the adjoint process coincides with the spatial gradient of the value function, evaluated along the optimal trajectory of the state equation. Finally, using these theoretical results, we solve explicitly an example, on optimal harvesting strategy for a geometric Brownian motion, with jumps. Note that our results improve those in [1, 2] to the jump diffusion setting. Moreover we generalize results in [3, 4], by allowing both classical and singular controls, at least in the complete information setting. Note that in our control problem, there are two types of jumps for the state process, the inaccessible ones which come from the Poisson martingale part and the predictable ones which come from the singular control part. The inclusion of these jump terms introduces a major difference with respect to the case without singular control.Stochastic control problems of singular type have received considerable attention, due to their wide applicability in a number of different areas; see [4–8]. In most cases, the optimal singular control problem was studied through dynamic programming principle; see [9], where it was shown in particular that the value function is continuous and is the unique viscosity solution of the HJB variational inequality.The one-dimensional problems of the singular type, without the classical control, have been studied by many authors. It was shown that the value function satisfies a variational inequality, which gives rise to a free boundary problem, and the optimal state process is a diffusion reflected at the free boundary. Bather and Chernoff [10] were the first to formulate such a problem. Beneš et al. [11] explicitly solved a one-dimensional example by observing that the value function in their example is twice continuously differentiable. This regularity property is called the principle of smooth fit. The optimal control can be constructed by using the reflected Brownian motion; see Lions and Sznitman [12] for more details. Applications to irreversible investment, industry equilibrium, and portfolio optimization under transaction costs can be found in [13]. A problem of optimal harvesting from a population in a stochastic crowded environment is proposed in [14] to represent the size of the population at time t as the solution of the stochastic logistic differential equation. The two-dimensional problem that arises in portfolio selection models, under proportional transaction costs, is of singular type and has been considered by Davis and Norman [15]. The case of diffusions with jumps is studied by Øksendal and Sulem [8]. For further contributions on singular control problems and their relationship with optimal stopping problems, the reader is referred to [4, 5, 7, 16, 17].The stochastic maximum principle is another powerful tool for solving stochastic control problems. The first result that covers singular control problems was obtained by Cadenillas and Haussmann [18], in which they consider linear dynamics, convex cost criterion, and convex state constraints. A first-order weak stochastic maximum principle was developed via convex perturbations method for both absolutely continuous and singular components by Bahlali and Chala [1]. The second-order stochastic maximum principle for nonlinear SDEs with a controlled diffusion matrix was obtained by Bahlali and Mezerdi [19], extending the Peng maximum principle [20] to singular control problems. A similar approach has been used by Bahlali et al. in [21], to study the stochastic maximum principle in relaxed-singular optimal control in the case of uncontrolled diffusion. Bahlali et al. in [22] discuss the stochastic maximum principle in singular optimal control in the case where the coefficients are Lipschitz continuous in x, provided that the classical derivatives are replaced by the generalized ones. See also the recent paper by Øksendal and Sulem [4], where Malliavin calculus techniques have been used to define the adjoint process.Stochastic control problems in which the system is governed by a stochastic differential equation with jumps, without the singular part, have been also studied, both by the dynamic programming approach and by the Pontryagin maximum principle. The HJB equation associated with this problems is a nonlinear second-order parabolic integro-differential equation. Pham [23] studied a mixed optimal stopping and stochastic control of jump diffusion processes by using the viscosity solutions approach. Some verification theorems of various types of problems for systems governed by this kind of SDEs are discussed by Øksendal and Sulem [8]. Some results that cover the stochastic maximum principle for controlled jump diffusion processes are discussed in [3, 24, 25]. In [3] the sufficient maximum principle and the link with the dynamic programming principle are given by assuming the smoothness of the value function. Let us mention that in [24] the verification theorem is established in the framework of viscosity solutions and the relationship between the adjoint processes and some generalized gradients of the value function are obtained. Note that Shi and Wu [24] extend the results of [26] to jump diffusions. See also [27] for systematic study of the continuous case. The second-order stochastic maximum principle for optimal controls of nonlinear dynamics, with jumps and convex state constraints, was developed via spike variation method, by Tang and Li [25]. These conditions are described in terms of two adjoint processes, which are linear backward SDEs. Such equations have important applications in hedging problems [28]. Existence and uniqueness for solutions to BSDEs with jumps and nonlinear coefficients have been treated by Tang and Li [25] and Barles et al. [29]. The link with integral-partial differential equations is studied in [29].The plan of the paper is as follows. In Section2, we give some preliminary results and notations. The purpose of Section 3 is to derive necessary as well as sufficient optimality conditions. In Section 4, we give, under-regularity assumptions, a verification theorem for the value function. Then, we prove that the adjoint process is equal to the derivative of the value function evaluated at the optimal trajectory, extending in particular [2, 3]. An example has been solved explicitly, by using the theoretical results.


## 2. Assumptions and Problem Formulation

The purpose of this section is to introduce some notations, which will be needed in the subsequent sections. In all what follows, we are given a probability space(
Ω
,
ℱ
,
(
ℱ
t
)
t
≤
T
,
ℙ
), such that ℱ
0 contains the ℙ-null sets, ℱ
T
=
ℱ for an arbitrarily fixed time horizon T, and (
ℱ
t
)
t
≤
T satisfies the usual conditions. We assume that (
ℱ
t
)
t
≤
T is generated by a d-dimensional standard Brownian motion B and an independent jump measure N of a Lévy process η, on [
0
,
T
]
×
E, where E
⊂
ℝ
m
∖
{
0
} for some m
≥
1. We denote by (
ℱ
t
B
)
t
≤
T (resp., (
ℱ
t
N
)
t
≤
T) the ℙ-augmentation of the natural filtration of B (resp., N
). We assume that the compensator of N has the form μ
(
d
t
,
d
e
)
=
ν
(
d
e
)
d
t, for some σ-finite Lévy measure ν on E, endowed with its Borel σ-field ℬ
(
E
). We suppose that ∫
E
‍
1
∧
|
e
|
2
ν
(
d
e
)
<
∞ and set N
~
(
d
t
,
d
e
)
=
N
(
d
t
,
d
e
)
-
ν
(
d
e
)
d
t, for the compensated jump martingale random measure of N.Obviously, we have(3)
ℱ
t
=
σ
[
∫
‍
∫
A
×
(
0
,
s
]
‍
N
(
d
r
,
d
e
)
;
s
≤
t
,
A
∈
ℬ
(
E
)
]
∨
σ
[
B
s
;
s
≤
t
]
∨
𝒩
,
where 𝒩 denotes the totality of ν-null sets and σ
1
∨
σ
2 denotes the σ-field generated by σ
1
∪
σ
2.Notation. Any element x
∈
ℝ
n will be identified with a column vector with n components, and its norm is |
x
|
=
|
x
1
|
+
⋯
+
|
x
n
|. The scalar product of any two vectors x and y on ℝ
n is denoted by x
y or ∑
i
=
1
n
‍
x
i
y
i. For a function h, we denote by h
x (resp., h
x
x
) the gradient or Jacobian (resp., the Hessian) of h with respect to the variable x.Givens
<
t, let us introduce the following spaces.(i)
𝕃
ν
,
(
E
;
ℝ
n
)
2 or 𝕃
ν
2 is the set of square integrable functions l
(
·
)
:
E
→
ℝ
n such that
(4)
∥
l
(
e
)
∥
𝕃
ν
,
(
E
;
ℝ
n
)
2
2
≔
∫
E
‍
|
l
(
e
)
|
2
ν
(
d
e
)
<
∞
.
(ii)
𝒮
(
[
s
,
t
]
;
ℝ
n
)
2  is the set of ℝ
n-valued adapted cadlag processes P such that
(5)
∥
P
∥
𝒮
(
[
s
,
t
]
;
ℝ
n
)
2
≔
𝔼
[
sup
r
∈
[
s
,
t
]
|
P
r
|
2
]
1
/
2
<
∞
.
(iii)
ℳ
(
[
s
,
t
]
;
ℝ
n
)
2 is the set of progressively measurable ℝ
n-valued processes Q such that
(6)
∥
Q
∥
ℳ
(
[
s
,
t
]
;
ℝ
n
)
2
≔
𝔼
[
∫
s
t
‍
|
Q
r
|
2
d
r
]
1
/
2
<
∞
.
(iv)
ℒ
ν
,
(
[
s
,
t
]
;
ℝ
n
)
2 is the set of ℬ
(
[
0
,
T
]
×
Ω
)
⊗
ℬ
(
E
) measurable maps R
:
[
0
,
T
]
×
Ω
×
E
→
ℝ
n such that
(7)
∥
R
∥
ℒ
ν
,
(
[
s
,
t
]
;
ℝ
n
)
2
≔
𝔼
[
∫
s
t
‍
∫
E
‍
|
R
r
(
e
)
|
2
ν
(
d
e
)
d
r
]
1
/
2
<
∞
.To avoid heavy notations, we omit the subscript(
[
s
,
t
]
;
ℝ
n
) in these notations when (
s
,
t
)
=
(
0
,
T
).LetT be a fixed strictly positive real number; A
1 is a closed convex subset of ℝ
n and A
2
=
(
[
0
,
∞
)
m
). Let us define the class of admissible control processes (
u
,
ξ
).Definition 1.
An admissible control is a pair of measurable, adapted processesu
:
[
0
,
T
]
×
Ω
→
A
1, and ξ
:
[
0
,
T
]
×
Ω
→
A
2, such that(1)
u is a predictable process, ξ is of bounded variation, nondecreasing, right continuous with left-hand limits, and ξ
0
-
=
0,
(2)
𝔼
[
sup
t
∈
[
0
,
T
]
|
u
t
|
2
+
|
ξ
T
|
2
]
<
∞.
We denote by𝒰
=
𝒰
1
×
𝒰
2 the set of all admissible controls. Here 𝒰
1 (resp., 𝒰
2) represents the set of the admissible controls u (resp., ξ).Assume that, for(
u
,
ξ
)
∈
𝒰, t
∈
[
0
,
T
], the state x
t of our system is given by
(8)
d
x
t
=
b
(
t
,
x
t
,
u
t
)
d
t
+
σ
(
t
,
x
t
,
u
t
)
d
B
t
+
∫
E
‍
γ
(
t
,
x
t
-
,
u
t
,
e
)
N
~
(
d
t
,
d
e
)
+
G
t
d
ξ
t
,
x
0
=
x
,
where x
∈
ℝ
n is given, representing the initial state.Let(9)
b
:
[
0
,
T
]
×
ℝ
n
×
A
1
⟶
ℝ
n
,
σ
:
[
0
,
T
]
×
ℝ
n
×
A
1
⟶
ℝ
n
×
d
,
γ
:
[
0
,
T
]
×
ℝ
n
×
A
1
×
E
⟶
ℝ
n
,
G
:
[
0
,
T
]
⟶
ℝ
n
×
m
be measurable functions.Notice that the jump of a singular controlξ
∈
𝒰
2 at any jumping time τ is defined by Δ
ξ
τ
=
ξ
τ
-
ξ
τ
-, and we let
(10)
ξ
t
c
=
ξ
t
-
∑
0
<
τ
≤
t
‍
Δ
ξ
τ
,
be the continuous part of ξ.We distinguish between the jumps ofx
τ caused by the jump of N
(
τ
,
e
), defined by
(11)
Δ
N
x
τ
≔
∫
E
‍
γ
(
τ
,
x
τ
-
,
u
τ
,
e
)
N
(
{
τ
}
,
d
e
)
≔
{
γ
(
τ
,
x
τ
-
,
u
τ
,
e
)
if
η
has
a
jump
of
size
e
at
τ
,
0
otherwise
,
and the jump of x
τ caused by the singular control ξ, denoted by Δ
ξ
x
τ
≔
G
τ
Δ
ξ
τ. In the above, N
(
{
τ
}
,
·
) represents the jump in the Poisson random measure, occurring at time τ. In particular, the general jump of the state process at τ is given by Δ
x
τ
=
x
τ
-
x
τ
-
=
Δ
ξ
x
τ
+
Δ
N
x
τ.Ifφ is a continuous real function, we let
(12)
Δ
ξ
φ
(
x
τ
)
≔
φ
(
x
τ
)
-
φ
(
x
τ
-
+
Δ
N
x
τ
)
.The expression (12) defines the jump in the value of φ
(
x
τ
) caused by the jump of x at τ. We emphasize that the possible jumps in x
τ coming from the Poisson measure are not included in Δ
ξ
φ
(
x
τ
).Suppose that the performance functional has the form(13)
J
(
u
,
ξ
)
=
𝔼
[
∫
0
T
‍
f
(
t
,
x
t
,
u
t
)
d
t
+
g
(
x
T
)
+
∫
s
T
‍
k
t
d
ξ
t
]
,
for
(
u
,
ξ
)
∈
𝒰
,
where f
:
[
0
,
T
]
×
ℝ
n
×
A
1
→
ℝ, g
:
ℝ
n
→
ℝ, and k
:
[
0
,
T
]
→
(
[
0
,
∞
)
)
m, with k
t
d
ξ
t
=
∑
l
=
1
m
k
t
l
d
ξ
t
l.An admissible control(
u
⋆
,
ξ
⋆
) is optimal if
(14)
J
(
u
⋆
,
ξ
⋆
)
=
sup
(
u
,
ξ
)
∈
𝒰
J
(
u
,
ξ
)
.Let us assume the following.(H1)
The mapsb, σ, γ, and f are continuously differentiable with respect to (
x
,
u
) and g is continuously differentiable in x.
(H2)
The derivativesb
x, b
u, σ
x, σ
u, γ
x, γ
u, f
x, f
u, and g
x are continuous in (
x
,
u
) and uniformly bounded.
(H3)
b, σ, γ, and f are bounded by K
1
(
1
+
|
x
|
+
|
u
|
), and g is bounded by K
1
(
1
+
|
x
|
), for some K
1
>
0.
(H4)
For all(
u
,
e
)
∈
A
1
×
E, the map
(15)
(
x
,
ζ
)
∈
ℝ
n
×
ℝ
n
⟶
a
(
t
,
x
,
u
,
ζ
;
e
)
≔
ζ
T
(
γ
x
(
t
,
x
,
u
,
e
)
+
I
d
)
ζ
satisfies uniformly in(
x
,
ζ
)
∈
ℝ
n
×
ℝ
n,
(16)
a
(
t
,
x
,
u
,
ζ
;
e
)
≥
|
ζ
|
2
K
2
-
1
,
for
some
K
2
>
0
.
(H5)
G
,
k are continuous and bounded.


## 3. The Stochastic Maximum Principle

Let us first define the usual Hamiltonian associated to the control problem by(17)
H
(
t
,
x
,
u
,
p
,
q
,
X
(
·
)
)
=
f
(
t
,
x
,
u
)
+
p
b
(
t
,
x
,
u
)
+
∑
j
=
1
n
‍
q
j
σ
j
(
t
,
x
,
u
)
+
∫
E
‍
X
(
e
)
γ
(
t
,
x
,
u
,
e
)
ν
(
d
e
)
,
where (
t
,
x
,
u
,
p
,
q
,
X
(
·
)
)
∈
[
0
,
T
]
×
ℝ
n
×
A
1
×
ℝ
n
×
ℝ
n
×
n
×
𝕃
ν
2. q
j and σ
j for j
=
1
,
…
,
n, denote the jth column of the matrices q and σ, respectively.Let(
u
⋆
,
ξ
⋆
) be an optimal control and let x
⋆ be the corresponding optimal trajectory. Then, we consider a triple (
p
,
q
,
r
(
·
)
) of square integrable adapted processes associated with (
u
⋆
,
x
⋆
), with values in ℝ
n
×
ℝ
n
×
d
×
ℝ
n such that
(18)
d
p
t
=
-
H
x
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
d
t
+
q
t
d
B
t
+
∫
E
‍
r
t
(
e
)
N
~
(
d
t
,
d
e
)
,
p
T
=
g
x
(
x
T
⋆
)
.


### 3.1. Necessary Conditions of Optimality

The purpose of this section is to derive optimality necessary conditions, satisfied by an optimal control, assuming that the solution exists. The proof is based on convex perturbations for both absolutely continuous and singular components of the optimal control and on some estimates of the state processes. Note that our results generalize [1, 2, 21] for systems with jumps.Theorem 2 (necessary conditions of optimality).
Let(
u
⋆
,
ξ
⋆
) be an optimal control maximizing the functional J over 𝒰, and let x
⋆ be the corresponding optimal trajectory. Then there exists an adapted process (
p
,
q
,
r
(
·
)
)
∈
𝒮
2
×
ℳ
2
×
ℒ
ν
2, which is the unique solution of the BSDE (18), such that the following conditions hold.(i)
For allv
∈
A
1
(19)
H
u
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
≤
0
,
d
t
—
a
.
e
.
,
ℙ
—
a
.
s
.
(ii)
For allt
∈
[
0
,
T
], with probability 1
(20)
k
t
i
+
G
t
i
p
t
≤
0
,
f
o
r
i
=
1
,
…
,
m
,
(21)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i
=
0
,
(22)
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≤
0
,
f
o
r
i
=
1
,
…
,
m
,
(23)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≤
0
}
Δ
ξ
t
⋆
i
=
0
,where Δ
N
p
t
=
∫
E
‍
r
t
(
e
)
N
(
{
t
}
,
d
e
).In order to prove Theorem2, we present some auxiliary results.


#### 3.1.1. Variational Equation

Let(
v
,
ξ
)
∈
𝒰 be such that (
u
⋆
+
v
,
ξ
⋆
+
ξ
)
∈
𝒰. The convexity condition of the control domain ensures that for ɛ
∈
(
0,1
) the control (
u
⋆
+
ɛ
v
,
ξ
⋆
+
ɛ
ξ
) is also in 𝒰. We denote by x
ɛ the solution of the SDE (8) corresponding to the control (
u
⋆
+
ɛ
v
,
ξ
⋆
+
ɛ
ξ
). Then by standard arguments from stochastic calculus, it is easy to check the following estimate.Lemma 3.
Under assumptions (H1)–(H5), one has
(24)
lim
ɛ
→
0
𝔼
[
sup
t
∈
[
0
,
T
]
|
x
t
ɛ
-
x
t
⋆
|
2
]
=
0
.Proof.
From assumptions (H1)–(H5), we get by using the Burkholder-Davis-Gundy inequality
(25)
𝔼
[
sup
t
∈
[
0
,
T
]
|
x
t
ɛ
-
x
t
⋆
|
2
]
≤
K
∫
0
T
‍
𝔼
[
sup
τ
∈
[
0
,
s
]
|
x
τ
ɛ
-
x
τ
⋆
|
2
]
d
s
+
K
ɛ
2
(
∫
0
T
‍
𝔼
[
sup
τ
∈
[
0
,
s
]
|
v
τ
|
2
]
d
s
+
𝔼
|
ξ
T
|
2
)
From Definition1 and Gronwall’s lemma, the result follows immediately by letting ɛ go to zero.We define the processz
t
=
z
t
u
⋆
,
v
,
ξ by
(26)
d
z
t
=
{
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
}
d
t
+
∑
j
=
1
d
‍
{
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
}
d
B
t
j
+
∫
E
‍
{
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
z
t
-
+
γ
u
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
v
t
}
×
N
~
(
d
t
,
d
e
)
+
G
t
d
ξ
t
,
z
0
=
0
.From (H2) and Definition 1, one can find a unique solution z which solves the variational equation (26), and the following estimate holds.Lemma 4.
Under assumptions (H1)–(H5), it holds that
(27)
lim
ɛ
→
0
𝔼
|
x
t
ɛ
-
x
t
⋆
ɛ
-
z
t
|
2
=
0
.Proof.
Let(28)
Γ
t
ɛ
=
x
t
ɛ
-
x
t
⋆
ɛ
-
z
t
.
We denotex
t
μ
,
ɛ
=
x
t
⋆
+
μ
ɛ
(
Γ
t
ɛ
+
z
t
) and u
t
μ
,
ɛ
=
u
t
⋆
+
μ
ɛ
v
t, for notational convenience. Then we have immediately that Γ
0
ɛ
=
0 and Γ
t
ɛ satisfies the following SDE:
(29)
d
Γ
t
ɛ
=
{
1
ɛ
(
b
(
t
,
x
t
μ
,
ɛ
,
u
t
μ
,
ɛ
)
-
b
(
t
,
x
t
⋆
,
u
t
⋆
)
)
-
(
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
)
1
ɛ
(
b
(
t
,
x
t
μ
,
ɛ
,
u
t
μ
,
ɛ
)
-
b
(
t
,
x
t
⋆
,
u
t
⋆
)
)
}
d
t
+
{
1
ɛ
(
σ
(
t
,
x
t
μ
,
ɛ
,
u
t
μ
,
ɛ
)
-
σ
(
t
,
x
t
⋆
,
u
t
⋆
)
)
-
(
σ
x
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
σ
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
)
1
ɛ
}
d
B
t
+
∫
E
{
1
ɛ
(
γ
(
t
,
x
t
-
μ
,
ɛ
,
u
t
μ
,
ɛ
,
e
)
-
γ
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
)
‍
-
(
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
z
t
-
+
γ
u
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
v
t
)
1
ɛ
(
γ
(
t
,
x
t
-
μ
,
ɛ
,
u
t
μ
,
ɛ
,
e
)
-
γ
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
)
}
×
N
~
(
d
t
,
d
e
)
.
Since the derivatives of the coefficients are bounded, and from Definition1, it is easy to verify by Gronwall’s inequality that Γ
ɛ
∈
𝒮
2 and
(30)
𝔼
|
Γ
t
ɛ
|
2
≤
K
𝔼
∫
0
t
‍
|
∫
0
1
‍
b
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
Γ
s
ɛ
d
μ
|
2
d
s
+
K
𝔼
∫
0
t
‍
|
∫
0
1
‍
σ
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
Γ
s
ɛ
d
μ
|
2
d
s
+
K
𝔼
∫
0
t
‍
∫
E
‍
|
∫
0
1
‍
γ
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
,
e
)
Γ
s
ɛ
d
μ
|
2
ν
(
d
e
)
d
s
+
K
𝔼
|
ρ
t
ɛ
|
2
,
where ρ
t
ɛ is given by
(31)
ρ
t
ɛ
=
-
∫
0
t
‍
b
x
(
s
,
x
s
⋆
,
u
s
⋆
)
z
s
d
s
-
∫
0
t
‍
σ
x
(
s
,
x
s
⋆
,
u
s
⋆
)
z
s
d
B
s
-
∫
0
t
‍
∫
E
‍
γ
x
(
s
,
x
s
-
⋆
,
u
s
⋆
,
e
)
z
s
-
N
~
(
d
s
,
d
e
)
-
∫
0
t
‍
b
v
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
d
s
-
∫
0
t
‍
σ
v
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
d
B
s
-
∫
0
t
‍
∫
E
‍
γ
v
(
s
,
x
s
-
⋆
,
u
s
⋆
,
e
)
v
s
N
~
(
d
s
,
d
e
)
+
∫
0
t
‍
∫
0
1
‍
b
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
z
s
d
μ
d
s
+
∫
0
t
‍
∫
0
1
‍
σ
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
z
s
d
μ
d
B
s
+
∫
0
t
‍
∫
E
‍
∫
0
1
‍
γ
x
(
s
,
x
s
-
μ
,
ɛ
,
u
s
μ
,
ɛ
,
e
)
z
s
-
d
μ
N
~
(
d
s
,
d
e
)
+
∫
0
t
‍
∫
0
1
‍
b
v
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
v
s
d
μ
d
s
+
∫
0
t
‍
∫
0
1
‍
σ
v
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
v
s
d
μ
d
B
s
+
∫
0
t
‍
∫
E
‍
∫
0
1
‍
γ
v
(
s
,
x
s
-
μ
,
ɛ
,
u
s
μ
,
ɛ
,
e
)
v
s
d
μ
N
~
(
d
s
,
d
e
)
.
Sinceb
x, σ
x, and γ
x are bounded, then
(32)
𝔼
|
Γ
t
ɛ
|
2
≤
M
𝔼
∫
0
t
‍
|
Γ
s
ɛ
|
2
d
s
+
M
𝔼
|
ρ
t
ɛ
|
2
,
where M is a generic constant depending on the constants K, ν
(
E
), and T. We conclude from Lemma 3 and the dominated convergence theorem, that lim
ɛ
→
0
ρ
t
ɛ
=
0. Hence (27) follows from Gronwall’s lemma and by letting ɛ go to 0. This completes the proof.


#### 3.1.2. Variational Inequality

LetΦ be the solution of the linear matrix equation, for 0
≤
s
<
t
≤
T
(33)
d
Φ
s
,
t
=
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
Φ
s
,
t
d
t
+
∑
j
=
1
d
‍
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
Φ
s
,
t
d
B
t
j
+
∫
E
‍
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
Φ
s
,
t
-
N
~
(
d
t
,
d
e
)
,
Φ
s
,
s
=
I
d
,
where I
d is the n
×
n identity matrix. This equation is linear, with bounded coefficients, then it admits a unique strong solution. Moreover, the condition (H4) ensures that the tangent process Φ is invertible, with an inverse Ψ satisfying suitable integrability conditions.From Itô’s formula, we can easily check thatd
(
Φ
s
,
t
Ψ
s
,
t
)
=
0, and Φ
s
,
s
Ψ
s
,
s
=
I
d, where Ψ is the solution of the following equation
(34)
d
Ψ
s
,
t
=
-
Ψ
s
,
t
{
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
-
∑
j
=
1
d
‍
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
-
∫
E
‍
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
ν
(
d
e
)
∑
j
=
1
d
‍
}
d
t
-
∑
j
=
1
d
‍
Ψ
s
,
t
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
d
B
t
j
-
Ψ
s
,
t
-
∫
E
‍
(
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
×
N
(
d
t
,
d
e
)
,
Ψ
s
,
s
=
I
d
,
so Ψ
=
Φ
-
1. If s
=
0 we simply write Φ
0
,
t
=
Φ
t and Ψ
0
,
t
=
Ψ
t. By the integration by parts formula ([8, Lemma 3.6]), we can see that the solution of (26) is given by z
t
=
Φ
t
η
t, where η
t is the solution of the stochastic differential equation
(35)
d
η
t
=
Ψ
t
{
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
-
∑
j
=
1
d
‍
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
-
∫
E
‍
γ
u
(
t
,
x
t
⋆
,
u
t
⋆
,
z
)
v
t
ν
(
d
e
)
∑
j
=
1
d
‍
}
d
t
+
∑
j
=
1
d
‍
Ψ
t
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
d
B
t
j
+
Ψ
t
-
∫
E
‍
(
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
×
γ
u
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
v
t
N
(
d
t
,
d
e
)
+
Ψ
t
G
t
d
ξ
t
-
Ψ
t
∫
E
‍
(
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
×
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
N
(
{
t
}
,
d
e
)
G
t
Δ
ξ
t
,
η
0
=
0
.Let us introduce the following convex perturbation of the optimal control(
u
⋆
,
ξ
⋆
) defined by
(36)
(
u
⋆
,
ɛ
,
ξ
⋆
,
ɛ
)
=
(
u
⋆
+
ɛ
v
,
ξ
⋆
+
ɛ
ξ
)
,
for some (
v
,
ξ
)
∈
𝒰 and ɛ
∈
(
0,1
). Since (
u
⋆
,
ξ
⋆
) is an optimal control, then ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
≤
0. Thus a necessary condition for optimality is that
(37)
lim
ɛ
→
0
ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
≤
0
.The rest of this subsection is devoted to the computation of the above limit. We will see that the expression (37) leads to a precise description of the optimal control (
u
⋆
,
ξ
⋆
) in terms of the adjoint process. First, it is easy to prove the following lemma.Lemma 5.
Under assumptions (H1)–(H5), one has
(38)
I
=
lim
ɛ
→
0
ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
=
𝔼
[
∫
0
T
‍
{
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
z
s
+
f
u
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
}
d
s
+
g
x
(
x
T
⋆
)
z
T
+
∫
0
T
‍
k
t
d
ξ
t
]
.Proof.
We use the same notations as in the proof of Lemma4. First, we have
(39)
ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
=
𝔼
[
∫
0
T
‍
∫
0
1
‍
{
f
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
z
s
+
f
u
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
v
s
}
d
μ
d
s
+
∫
0
1
‍
g
x
(
x
T
μ
,
ɛ
)
z
T
d
μ
+
∫
0
T
‍
k
t
d
ξ
t
]
+
β
t
ɛ
,
where
(40)
β
t
ɛ
=
𝔼
[
∫
0
T
‍
∫
0
1
‍
f
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
Γ
s
ɛ
d
μ
d
s
+
∫
0
1
‍
g
x
(
x
T
μ
,
ɛ
)
Γ
T
ɛ
d
μ
]
.
By using Lemma4, and since the derivatives f
x
,
f
u, and g
x are bounded, we have lim
ɛ
→
0
β
t
ɛ
=
0. Then, the result follows by letting ɛ go to 0 in the above equality.Substituting byz
t
=
Φ
t
η
t in (38) leads to
(41)
I
=
𝔼
[
∫
0
T
‍
{
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
η
s
+
f
u
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
}
d
s
+
g
x
(
x
T
⋆
)
Φ
T
η
T
+
∫
0
T
‍
k
t
d
ξ
t
]
.Consider the right continuous version of the square integrable martingale(42)
M
t
≔
𝔼
[
∫
0
T
‍
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
d
s
+
g
x
(
x
T
⋆
)
Φ
T
∣
ℱ
t
]
.By the Itô representation theorem [30], there exist two processes Q
=
(
Q
1
,
…
,
Q
d
) where Q
j
∈
ℳ
2, for j
=
1
,
…
,
d, and U
(
·
)
∈
ℒ
ν
2, satisfying
(43)
M
t
=
𝔼
[
∫
0
T
‍
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
d
s
+
g
x
(
x
T
⋆
)
Φ
T
]
+
∑
j
=
1
d
‍
∫
0
t
‍
Q
s
j
d
B
s
j
+
∫
0
t
‍
∫
E
‍
U
s
(
e
)
N
~
(
d
s
,
d
e
)
.Let us denotey
t
⋆
=
M
t
-
∫
0
t
‍
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
d
s. The adjoint variable is the process defined by
(44)
p
t
=
y
t
⋆
Ψ
t
,
q
t
j
=
Q
t
j
Ψ
t
-
p
t
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
,
for
j
=
1
,
…
,
d
,
r
t
(
e
)
=
U
t
(
e
)
Ψ
t
(
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
+
p
t
(
(
γ
x
(
s
,
x
t
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
-
I
d
)
.Theorem 6.
Under assumptions (H1)–(H5), one has
(45)
I
=
𝔼
[
∑
i
=
1
m
‍
∑
‍
0
<
s
≤
T
∫
0
T
‍
{
∫
E
‍
f
u
(
s
,
x
s
⋆
,
u
s
⋆
)
+
p
s
b
u
(
s
,
x
s
⋆
,
u
s
⋆
)
+
∑
j
=
1
d
‍
q
s
j
σ
u
j
(
s
,
x
s
⋆
,
u
s
⋆
)
+
∫
E
‍
r
s
(
z
)
γ
u
(
s
,
x
s
⋆
,
u
s
⋆
,
e
)
ν
(
d
e
)
}
v
s
d
s
+
∑
i
=
1
m
‍
∫
0
T
‍
{
k
s
i
+
G
s
i
p
s
}
d
ξ
s
c
i
+
∑
i
=
1
m
‍
∑
‍
0
<
s
≤
T
{
k
s
i
+
G
s
i
(
p
s
-
+
Δ
N
p
s
)
}
Δ
ξ
s
i
]
.Proof.
From the integration by parts formula ([8, Lemma 3.5]), and by using the definition of p
t
,
q
t
j for j
=
1
,
…
,
d, and r
t
(
·
), we can easily check that
(46)
E
[
y
T
η
T
]
=
𝔼
[
∑
i
=
1
m
‍
∫
0
T
‍
{
p
t
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
+
∑
j
=
1
d
‍
q
s
j
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
+
∫
E
‍
r
t
(
e
)
γ
u
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
ν
(
d
e
)
∑
j
=
1
d
‍
}
v
t
d
t
-
∫
0
T
‍
f
x
(
t
,
x
t
⋆
,
u
t
⋆
)
η
t
Φ
t
d
t
+

∑
i
=
1
m
‍
(
∫
0
T
‍
G
t
i
p
t
d
ξ
t
c
i
+
∑
‍
0
<
t
≤
T
G
t
i
(
p
t
-
+
Δ
N
p
t
)
Δ
ξ
t
i
)

∫
0
T
‍
{
p
t
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
+
∑
j
=
1
d
‍
q
s
j
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
]
.
Also we have(47)
I
=
𝔼
[
y
T
η
T
+
∫
0
T
‍
f
x
(
t
,
x
t
⋆
,
u
t
⋆
)
Φ
t
η
t
d
t
+
∫
0
T
‍
f
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
d
t
+
∫
0
T
‍
k
t
d
ξ
t
]
,
substituting (46) in (47), the result follows.


#### 3.1.3.  Adjoint Equation and Maximum Principle

Since (37) is true for all (
v
,
ξ
)
∈
𝒰 and I
≤
0, we can easily deduce the following result.Theorem 7.
Let(
u
⋆
,
ξ
⋆
) be the optimal control of the problem (14) and denote by x
⋆ the corresponding optimal trajectory, then the following inequality holds:
(48)
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
d
t
+
∫
0
T
‍
{
k
t
+
G
t
p
t
}
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
(
ξ
-
ξ
⋆
)
t
]
≤
0
,
where the Hamiltonian H is defined by (17), and the adjoint variable (
p
,
q
j
,
r
(
·
)
) for j
=
1
,
…
,
d, is given by (44).Now, we are ready to give the proof of Theorem2.Proof of Theorem2.
(i) Let us assume that(
u
⋆
,
ξ
⋆
) is an optimal control for the problem (14), so that inequality (48) is valid for every (
v
,
ξ
). If we choose ξ
=
ξ
⋆ in inequality (48), we see that for every measurable, ℱ
t-adapted process v
:
[
0
,
T
]
×
Ω
→
A
1
(49)
𝔼
[
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
d
t
]
≤
0
.
Forv
∈
𝒰
1 define
(50)
A
v
=
{
H
v
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
(
t
,
ω
)
∈
[
0
,
T
]
×
Ω
such
that
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
>
0
}
.
ObviouslyA
t
v
∈
ℱ
t, for each t
∈
[
0
,
T
]. Let us define v
~
∈
𝒰
1 by
(51)
v
~
t
(
ω
)
=
{
v
,
if
(
t
,
ω
)
∈
A
t
v
,
u
t
⋆
,
otherwise
.
Ifλ
⊗
ℙ
(
A
v
)
>
0, where λ denotes the Lebesgue measure, then
(52)
𝔼
[
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
~
t
-
u
t
⋆
)
d
t
]
>
0
,
which contradicts (49), unless λ
⊗
ℙ
(
A
v
)
=
0. Hence the conclusion follows.
(ii) If instead we choosev
=
u
⋆ in inequality (48), we obtain that for every measurable, ℱ
t-adapted process ξ
:
[
0
,
T
]
×
Ω
→
A
2, the following inequality holds:
(53)
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
‍
{
k
t
+
G
t
p
t
}
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
(
ξ
-
ξ
⋆
)
t
]
≤
0
.
In particular, fori
=
1
,
…
,
m, we put ξ
t
i
=
ξ
t
⋆
i
+
1
{
k
t
i
+
G
t
i
p
t
>
0
}
λ
(
t
). Since the Lebesgue measure is regular then the purely discontinuous part (
ξ
t
i
-
ξ
t
⋆
i
)
d
=
0. Obviously, the relation (53) can be written as
(54)
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
d
(
ξ
i
-
ξ
⋆
i
)
t
c
+
∫
0
T
‍
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
d
(
ξ
i
-
ξ
⋆
i
)
t
d
]
=
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
1
{
k
t
i
+
G
t
i
p
t
>
0
}
d
λ
(
t
)
]
>
0
.
This contradicts (53) unless for every i
∈
{
1
,
…
,
m
}
,
λ
⊗
ℙ
{
k
t
i
+
G
t
i
p
t
>
0
}
=
0. This proves (20).
Let us prove (21). Define d
ξ
t
i
=
1
{
k
t
i
+
G
t
i
p
t
-
>
0
}
d
ξ
t
⋆
i
+
1
{
k
t
i
+
G
t
i
p
t
-
≤
0
}
d
ξ
t
⋆
d
i, for i
=
1
,
…
,
m, then we have d
(
ξ
i
-
ξ
⋆
i
)
t
c
=
-
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i, and d
ξ
t
d
i
=
d
ξ
t
⋆
d
i. Hence, we can rewrite (53) as follows:
(55)
-
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i
]
>
0
.
By comparing with (53) we get
(56)
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i
]
=
0
,
then we conclude that
(57)
∑
i
=
1
m
‍
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
c
i
=
0
.
Expressions (22) and (23) are proved by using the same techniques. First, for each i
∈
{
1
,
…
,
m
} and t
∈
[
0
,
T
] fixed, we define ξ
s
i
=
ξ
s
i
+
δ
t
(
s
)
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}, where δ
t denotes the Dirac unit mass at t
.
δ
t is a discrete measure, then (
ξ
s
i
-
ξ
s
i
)
c
=
0 and (
ξ
s
i
-
ξ
s
i
)
d
=
δ
t
(
s
)
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}. Hence
(58)
𝔼
[
∑
i
=
1
m
‍
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}
]
>
0
which contradicts (53), unless for every i
∈
{
1
,
…
,
m
} and t
∈
[
0
,
T
], we have
(59)
ℙ
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}
=
0
.
Next, letξ be defined by
(60)
d
ξ
t
i
=
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≥
0
}
d
ξ
t
⋆
i
+
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
d
ξ
t
⋆
c
i
.
Then, the relation (53) can be written as
(61)
∑
i
=
1
m
‍
𝔼
[
∑
‍
-
0
<
t
≤
T
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
×
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
Δ
ξ
t
⋆
i
∑
‍
-
0
<
t
≤
T
]
>
0
,
which implies that
(62)
𝔼
[
∑
i
=
1
m
‍
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
×
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
Δ
ξ
t
⋆
i
∑
i
=
1
m
‍
]
=
0
.
By the fact that k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0, and Δ
ξ
t
i
≥
0, we get
(63)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
Δ
ξ
t
⋆
i
=
0
.
Thus (23) holds. The proof is complete.Now, by applying Itô’s formula toy
t
⋆
Ψ
t, it is easy to check that the processes defined by relation (44) satisfy BSDE (18) called the adjoint equation.


### 3.2.  Sufficient Conditions of Optimality

It is well known that in the classical cases (without the singular part of the control), the sufficient condition of optimality is of significant importance in the stochastic maximum principle, in the sense that it allows to compute optimal controls. This result states that, under some concavity conditions, maximizing the Hamiltonian leads to an optimal control.In this section, we focus on proving the sufficient maximum principle for mixed classical-singular stochastic control problems, where the state of the system is governed by a stochastic differential equation with jumps, allowing both classical control and singular control.Theorem 8 (sufficient condition of optimality in integral form).
Let(
u
⋆
,
ξ
⋆
) be an admissible control and denote x
⋆ the associated controlled state process. Let (
p
,
q
,
r
(
·
)
) be the unique solution of B
S
D
E (18). Let one assume that (
x
,
u
)
→
H
(
t
,
x
,
u
,
p
t
,
q
t
,
r
t
(
·
)
) and x
→
g
(
x
) are concave functions. Moreover suppose that for all t
∈
[
0
,
T
], v
∈
A
1, and ξ
∈
𝒰
2
(64)
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
d
t
+
∫
0
T
‍
{
k
t
+
G
t
p
t
}
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
(
ξ
-
ξ
⋆
)
t
]
≤
0
.
Then(
u
⋆
,
ξ
⋆
) is an optimal control.Proof.
For convenience, we will use the following notations throughout the proof:(65)
Θ
⋆
(
t
)
=
Θ
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
,
Θ
(
t
)
=
Θ
(
t
,
x
t
,
u
t
,
p
t
,
q
t
,
r
t
(
·
)
)
,
for
Θ
=
H
,
H
x
,
H
u
δ
ϕ
(
t
)
=
ϕ
(
t
,
x
t
⋆
,
u
t
⋆
)
-
ϕ
(
t
,
x
t
,
u
t
)
,
for
ϕ
=
b
,
σ
,
σ
j
,
j
=
1
,
…
,
n
,
f
δ
γ
(
t
,
e
)
=
γ
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
-
γ
(
t
,
x
t
,
u
t
,
e
)
,
δ
γ
-
(
t
,
e
)
=
γ
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
-
γ
(
t
,
x
t
-
,
u
t
,
e
)
.
Let(
u
,
ξ
) be an arbitrary admissible pair, and consider the difference
(66)
J
(
u
⋆
,
ξ
⋆
)
-
J
(
u
,
ξ
)
=
𝔼
[
∫
0
T
‍
δ
f
(
t
)
d
t
+
∫
0
T
‍
k
t
d
(
ξ
⋆
-
ξ
)
t
]
+
𝔼
[
g
(
x
T
⋆
)
-
g
(
x
T
)
]
.
We first note that, by concavity ofg, we conclude  that
(67)
𝔼
[
g
(
x
T
⋆
)
-
g
(
x
T
)
]
≥
𝔼
[
(
x
T
⋆
-
x
T
)
g
x
(
x
T
⋆
)
]
=
𝔼
[
(
x
T
⋆
-
x
T
)
p
T
]
=
𝔼
[
∫
0
T
‍
(
x
t
-
⋆
-
x
t
-
)
d
p
t
+
∫
0
T
‍
p
t
-
d
(
x
t
⋆
-
x
t
)
]
+
𝔼
[
∫
0
T
‍
∑
j
=
1
n
‍
(
δ
σ
j
(
t
)
)
q
t
j
d
t
+
∫
0
T
‍
∫
E
‍
(
δ
γ
-
(
t
,
e
)
)
r
t
(
e
)
N
(
d
t
,
d
e
)
∫
0
T
‍
∑
j
=
1
n
‍
]
+
𝔼
[
∑
‍
0
<
t
≤
T
G
t
(
Δ
N
p
t
)
Δ
(
ξ
-
ξ
⋆
)
t
]
,
which implies that
(68)
E
[
g
(
x
T
⋆
)
-
g
(
x
T
)
]
≥
𝔼
[
∫
0
T
‍
(
x
t
⋆
-
x
t
)
(
-
H
x
⋆
(
t
)
)
d
t
]
+
𝔼
[
∫
0
T
‍
{
p
t
(
δ
b
(
t
)
)
+
∑
j
=
1
n
‍
(
δ
σ
j
(
t
)
)
q
t
j
}
d
t
]
+
𝔼
[
∫
0
T
‍
∫
E
‍
(
δ
γ
-
(
t
,
e
)
)
r
t
(
e
)
N
(
d
t
,
d
e
)
]
+
𝔼
[
∫
0
T
‍
{
(
x
t
⋆
-
x
t
)
q
t
+
(
δ
σ
(
t
)
)
p
t
}
d
B
t
]
+
𝔼
[
∫
0
T
‍
∫
E
‍
{
(
x
t
-
⋆
-
x
t
-
)
r
t
(
e
)
+
p
t
-
(
δ
γ
-
(
t
,
e
)
)
}
×
N
~
(
d
t
,
d
e
)
∫
0
T
‍
∫
E
‍
]
+
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
‍
G
t
p
t
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
G
t
(
p
t
-
+
Δ
N
p
t
)
Δ
(
ξ
-
ξ
⋆
)
t
]
.
By the fact that(
p
,
q
j
,
r
(
·
)
)
∈
𝒮
2
×
ℳ
2
×
ℒ
ν
2 for j
=
1
,
…
,
n, we deduce that the stochastic integrals with respect to the local martingales have zero expectation. Due to the concavity of the Hamiltonian H, the following holds
(69)
𝔼
[
g
(
x
T
⋆
)
-
g
(
x
T
)
]
≥
𝔼
[
∫
0
T
‍
{
-
(
H
⋆
(
t
)
-
H
(
t
)
)
+
H
u
⋆
(
t
)
(
u
t
⋆
-
u
t
)
}
d
t
]
+
𝔼
[
∫
0
T
‍
{
p
t
(
δ
b
(
t
)
)
+
∑
j
=
1
n
‍
(
δ
σ
j
(
t
)
)
q
t
j
+
∫
E
‍
(
δ
γ
(
t
,
e
)
)
r
t
(
e
)
ν
(
d
e
)
∑
j
=
1
n
‍
}
d
t
]
+
𝔼
[
∑
0
<
t
≤
T
‍
∫
0
T
‍
G
t
T
p
t
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
0
<
t
≤
T
‍
G
t
T
(
p
t
-
+
Δ
N
p
t
)
Δ
(
ξ
-
ξ
⋆
)
t
]
.
The definition of the HamiltonianH and (64) leads to J
(
u
⋆
,
ξ
⋆
)
-
J
(
u
,
ξ
)
≥
0, which means that (
u
⋆
,
ξ
⋆
) is an optimal control for the problem (14).The expression (64) is a sufficient condition of optimality in integral form. We want to rewrite this inequality in a suitable form for applications. This is the objective of the following theorem which could be seen as a natural extension of [2, Theorem 2.2] to the jump setting and [3, Theorem 2.1] to mixed regular-singular control problems.Theorem 9 (sufficient conditions of optimality).
Let(
u
⋆
,
ξ
⋆
) be an admissible control and x
⋆ the associated controlled state process. Let (
p
,
q
,
r
(
·
)
) be the unique solution of B
S
D
E (18). Let one assume that (
x
,
u
)
→
H
(
t
,
x
,
u
,
p
t
,
q
t
,
r
t
(
·
)
) and x
→
g
(
x
) are concave functions. If in addition one assumes that(i)
for allt
∈
[
0
,
T
]
,
v
∈
A
1
(70)
H
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
=
sup
v
∈
A
1
H
(
t
,
x
t
⋆
,
v
,
p
t
,
q
t
,
r
t
(
·
)
)
,
d
t
—
a
.
e
.
,
ℙ
—
a
.
s
;
(ii)
for allt
∈
[
0
,
T
], with probability 1
(71)
k
t
i
+
G
t
i
p
t
≤
0
,
f
o
r
i
=
1
,
…
,
m
,
(72)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i
=
0
,
(73)
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≤
0
,
f
o
r
i
=
1
,
…
,
m
,
(74)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≤
0
}
Δ
ξ
t
⋆
i
=
0
.
Then(
u
⋆
,
ξ
⋆
) is an optimal control.Proof.
Using (71) and (72) yields
(75)
𝔼
[
∫
0
T
‍
{
k
t
+
G
t
p
t
}
d
ξ
t
⋆
c
]
=
𝔼
[
∑
i
=
1
m
‍
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
d
ξ
t
⋆
c
i
]
=
0
.
The same computations applied to (73) and (74) imply
(76)
𝔼
[
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
ξ
t
⋆
]
=
0
.
Hence, from Definition1, we have the following inequality:
(77)
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
{
k
t
+
G
t
p
t
}
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
(
ξ
-
ξ
⋆
)
t
]
≤
0
.
The desired result follows from Theorem8.


## 3.1. Necessary Conditions of Optimality

The purpose of this section is to derive optimality necessary conditions, satisfied by an optimal control, assuming that the solution exists. The proof is based on convex perturbations for both absolutely continuous and singular components of the optimal control and on some estimates of the state processes. Note that our results generalize [1, 2, 21] for systems with jumps.Theorem 2 (necessary conditions of optimality).
Let(
u
⋆
,
ξ
⋆
) be an optimal control maximizing the functional J over 𝒰, and let x
⋆ be the corresponding optimal trajectory. Then there exists an adapted process (
p
,
q
,
r
(
·
)
)
∈
𝒮
2
×
ℳ
2
×
ℒ
ν
2, which is the unique solution of the BSDE (18), such that the following conditions hold.(i)
For allv
∈
A
1
(19)
H
u
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
≤
0
,
d
t
—
a
.
e
.
,
ℙ
—
a
.
s
.
(ii)
For allt
∈
[
0
,
T
], with probability 1
(20)
k
t
i
+
G
t
i
p
t
≤
0
,
f
o
r
i
=
1
,
…
,
m
,
(21)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i
=
0
,
(22)
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≤
0
,
f
o
r
i
=
1
,
…
,
m
,
(23)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≤
0
}
Δ
ξ
t
⋆
i
=
0
,where Δ
N
p
t
=
∫
E
‍
r
t
(
e
)
N
(
{
t
}
,
d
e
).In order to prove Theorem2, we present some auxiliary results.


### 3.1.1. Variational Equation

Let(
v
,
ξ
)
∈
𝒰 be such that (
u
⋆
+
v
,
ξ
⋆
+
ξ
)
∈
𝒰. The convexity condition of the control domain ensures that for ɛ
∈
(
0,1
) the control (
u
⋆
+
ɛ
v
,
ξ
⋆
+
ɛ
ξ
) is also in 𝒰. We denote by x
ɛ the solution of the SDE (8) corresponding to the control (
u
⋆
+
ɛ
v
,
ξ
⋆
+
ɛ
ξ
). Then by standard arguments from stochastic calculus, it is easy to check the following estimate.Lemma 3.
Under assumptions (H1)–(H5), one has
(24)
lim
ɛ
→
0
𝔼
[
sup
t
∈
[
0
,
T
]
|
x
t
ɛ
-
x
t
⋆
|
2
]
=
0
.Proof.
From assumptions (H1)–(H5), we get by using the Burkholder-Davis-Gundy inequality
(25)
𝔼
[
sup
t
∈
[
0
,
T
]
|
x
t
ɛ
-
x
t
⋆
|
2
]
≤
K
∫
0
T
‍
𝔼
[
sup
τ
∈
[
0
,
s
]
|
x
τ
ɛ
-
x
τ
⋆
|
2
]
d
s
+
K
ɛ
2
(
∫
0
T
‍
𝔼
[
sup
τ
∈
[
0
,
s
]
|
v
τ
|
2
]
d
s
+
𝔼
|
ξ
T
|
2
)
From Definition1 and Gronwall’s lemma, the result follows immediately by letting ɛ go to zero.We define the processz
t
=
z
t
u
⋆
,
v
,
ξ by
(26)
d
z
t
=
{
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
}
d
t
+
∑
j
=
1
d
‍
{
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
}
d
B
t
j
+
∫
E
‍
{
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
z
t
-
+
γ
u
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
v
t
}
×
N
~
(
d
t
,
d
e
)
+
G
t
d
ξ
t
,
z
0
=
0
.From (H2) and Definition 1, one can find a unique solution z which solves the variational equation (26), and the following estimate holds.Lemma 4.
Under assumptions (H1)–(H5), it holds that
(27)
lim
ɛ
→
0
𝔼
|
x
t
ɛ
-
x
t
⋆
ɛ
-
z
t
|
2
=
0
.Proof.
Let(28)
Γ
t
ɛ
=
x
t
ɛ
-
x
t
⋆
ɛ
-
z
t
.
We denotex
t
μ
,
ɛ
=
x
t
⋆
+
μ
ɛ
(
Γ
t
ɛ
+
z
t
) and u
t
μ
,
ɛ
=
u
t
⋆
+
μ
ɛ
v
t, for notational convenience. Then we have immediately that Γ
0
ɛ
=
0 and Γ
t
ɛ satisfies the following SDE:
(29)
d
Γ
t
ɛ
=
{
1
ɛ
(
b
(
t
,
x
t
μ
,
ɛ
,
u
t
μ
,
ɛ
)
-
b
(
t
,
x
t
⋆
,
u
t
⋆
)
)
-
(
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
)
1
ɛ
(
b
(
t
,
x
t
μ
,
ɛ
,
u
t
μ
,
ɛ
)
-
b
(
t
,
x
t
⋆
,
u
t
⋆
)
)
}
d
t
+
{
1
ɛ
(
σ
(
t
,
x
t
μ
,
ɛ
,
u
t
μ
,
ɛ
)
-
σ
(
t
,
x
t
⋆
,
u
t
⋆
)
)
-
(
σ
x
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
σ
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
)
1
ɛ
}
d
B
t
+
∫
E
{
1
ɛ
(
γ
(
t
,
x
t
-
μ
,
ɛ
,
u
t
μ
,
ɛ
,
e
)
-
γ
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
)
‍
-
(
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
z
t
-
+
γ
u
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
v
t
)
1
ɛ
(
γ
(
t
,
x
t
-
μ
,
ɛ
,
u
t
μ
,
ɛ
,
e
)
-
γ
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
)
}
×
N
~
(
d
t
,
d
e
)
.
Since the derivatives of the coefficients are bounded, and from Definition1, it is easy to verify by Gronwall’s inequality that Γ
ɛ
∈
𝒮
2 and
(30)
𝔼
|
Γ
t
ɛ
|
2
≤
K
𝔼
∫
0
t
‍
|
∫
0
1
‍
b
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
Γ
s
ɛ
d
μ
|
2
d
s
+
K
𝔼
∫
0
t
‍
|
∫
0
1
‍
σ
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
Γ
s
ɛ
d
μ
|
2
d
s
+
K
𝔼
∫
0
t
‍
∫
E
‍
|
∫
0
1
‍
γ
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
,
e
)
Γ
s
ɛ
d
μ
|
2
ν
(
d
e
)
d
s
+
K
𝔼
|
ρ
t
ɛ
|
2
,
where ρ
t
ɛ is given by
(31)
ρ
t
ɛ
=
-
∫
0
t
‍
b
x
(
s
,
x
s
⋆
,
u
s
⋆
)
z
s
d
s
-
∫
0
t
‍
σ
x
(
s
,
x
s
⋆
,
u
s
⋆
)
z
s
d
B
s
-
∫
0
t
‍
∫
E
‍
γ
x
(
s
,
x
s
-
⋆
,
u
s
⋆
,
e
)
z
s
-
N
~
(
d
s
,
d
e
)
-
∫
0
t
‍
b
v
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
d
s
-
∫
0
t
‍
σ
v
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
d
B
s
-
∫
0
t
‍
∫
E
‍
γ
v
(
s
,
x
s
-
⋆
,
u
s
⋆
,
e
)
v
s
N
~
(
d
s
,
d
e
)
+
∫
0
t
‍
∫
0
1
‍
b
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
z
s
d
μ
d
s
+
∫
0
t
‍
∫
0
1
‍
σ
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
z
s
d
μ
d
B
s
+
∫
0
t
‍
∫
E
‍
∫
0
1
‍
γ
x
(
s
,
x
s
-
μ
,
ɛ
,
u
s
μ
,
ɛ
,
e
)
z
s
-
d
μ
N
~
(
d
s
,
d
e
)
+
∫
0
t
‍
∫
0
1
‍
b
v
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
v
s
d
μ
d
s
+
∫
0
t
‍
∫
0
1
‍
σ
v
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
v
s
d
μ
d
B
s
+
∫
0
t
‍
∫
E
‍
∫
0
1
‍
γ
v
(
s
,
x
s
-
μ
,
ɛ
,
u
s
μ
,
ɛ
,
e
)
v
s
d
μ
N
~
(
d
s
,
d
e
)
.
Sinceb
x, σ
x, and γ
x are bounded, then
(32)
𝔼
|
Γ
t
ɛ
|
2
≤
M
𝔼
∫
0
t
‍
|
Γ
s
ɛ
|
2
d
s
+
M
𝔼
|
ρ
t
ɛ
|
2
,
where M is a generic constant depending on the constants K, ν
(
E
), and T. We conclude from Lemma 3 and the dominated convergence theorem, that lim
ɛ
→
0
ρ
t
ɛ
=
0. Hence (27) follows from Gronwall’s lemma and by letting ɛ go to 0. This completes the proof.


### 3.1.2. Variational Inequality

LetΦ be the solution of the linear matrix equation, for 0
≤
s
<
t
≤
T
(33)
d
Φ
s
,
t
=
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
Φ
s
,
t
d
t
+
∑
j
=
1
d
‍
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
Φ
s
,
t
d
B
t
j
+
∫
E
‍
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
Φ
s
,
t
-
N
~
(
d
t
,
d
e
)
,
Φ
s
,
s
=
I
d
,
where I
d is the n
×
n identity matrix. This equation is linear, with bounded coefficients, then it admits a unique strong solution. Moreover, the condition (H4) ensures that the tangent process Φ is invertible, with an inverse Ψ satisfying suitable integrability conditions.From Itô’s formula, we can easily check thatd
(
Φ
s
,
t
Ψ
s
,
t
)
=
0, and Φ
s
,
s
Ψ
s
,
s
=
I
d, where Ψ is the solution of the following equation
(34)
d
Ψ
s
,
t
=
-
Ψ
s
,
t
{
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
-
∑
j
=
1
d
‍
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
-
∫
E
‍
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
ν
(
d
e
)
∑
j
=
1
d
‍
}
d
t
-
∑
j
=
1
d
‍
Ψ
s
,
t
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
d
B
t
j
-
Ψ
s
,
t
-
∫
E
‍
(
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
×
N
(
d
t
,
d
e
)
,
Ψ
s
,
s
=
I
d
,
so Ψ
=
Φ
-
1. If s
=
0 we simply write Φ
0
,
t
=
Φ
t and Ψ
0
,
t
=
Ψ
t. By the integration by parts formula ([8, Lemma 3.6]), we can see that the solution of (26) is given by z
t
=
Φ
t
η
t, where η
t is the solution of the stochastic differential equation
(35)
d
η
t
=
Ψ
t
{
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
-
∑
j
=
1
d
‍
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
-
∫
E
‍
γ
u
(
t
,
x
t
⋆
,
u
t
⋆
,
z
)
v
t
ν
(
d
e
)
∑
j
=
1
d
‍
}
d
t
+
∑
j
=
1
d
‍
Ψ
t
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
d
B
t
j
+
Ψ
t
-
∫
E
‍
(
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
×
γ
u
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
v
t
N
(
d
t
,
d
e
)
+
Ψ
t
G
t
d
ξ
t
-
Ψ
t
∫
E
‍
(
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
×
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
N
(
{
t
}
,
d
e
)
G
t
Δ
ξ
t
,
η
0
=
0
.Let us introduce the following convex perturbation of the optimal control(
u
⋆
,
ξ
⋆
) defined by
(36)
(
u
⋆
,
ɛ
,
ξ
⋆
,
ɛ
)
=
(
u
⋆
+
ɛ
v
,
ξ
⋆
+
ɛ
ξ
)
,
for some (
v
,
ξ
)
∈
𝒰 and ɛ
∈
(
0,1
). Since (
u
⋆
,
ξ
⋆
) is an optimal control, then ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
≤
0. Thus a necessary condition for optimality is that
(37)
lim
ɛ
→
0
ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
≤
0
.The rest of this subsection is devoted to the computation of the above limit. We will see that the expression (37) leads to a precise description of the optimal control (
u
⋆
,
ξ
⋆
) in terms of the adjoint process. First, it is easy to prove the following lemma.Lemma 5.
Under assumptions (H1)–(H5), one has
(38)
I
=
lim
ɛ
→
0
ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
=
𝔼
[
∫
0
T
‍
{
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
z
s
+
f
u
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
}
d
s
+
g
x
(
x
T
⋆
)
z
T
+
∫
0
T
‍
k
t
d
ξ
t
]
.Proof.
We use the same notations as in the proof of Lemma4. First, we have
(39)
ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
=
𝔼
[
∫
0
T
‍
∫
0
1
‍
{
f
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
z
s
+
f
u
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
v
s
}
d
μ
d
s
+
∫
0
1
‍
g
x
(
x
T
μ
,
ɛ
)
z
T
d
μ
+
∫
0
T
‍
k
t
d
ξ
t
]
+
β
t
ɛ
,
where
(40)
β
t
ɛ
=
𝔼
[
∫
0
T
‍
∫
0
1
‍
f
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
Γ
s
ɛ
d
μ
d
s
+
∫
0
1
‍
g
x
(
x
T
μ
,
ɛ
)
Γ
T
ɛ
d
μ
]
.
By using Lemma4, and since the derivatives f
x
,
f
u, and g
x are bounded, we have lim
ɛ
→
0
β
t
ɛ
=
0. Then, the result follows by letting ɛ go to 0 in the above equality.Substituting byz
t
=
Φ
t
η
t in (38) leads to
(41)
I
=
𝔼
[
∫
0
T
‍
{
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
η
s
+
f
u
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
}
d
s
+
g
x
(
x
T
⋆
)
Φ
T
η
T
+
∫
0
T
‍
k
t
d
ξ
t
]
.Consider the right continuous version of the square integrable martingale(42)
M
t
≔
𝔼
[
∫
0
T
‍
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
d
s
+
g
x
(
x
T
⋆
)
Φ
T
∣
ℱ
t
]
.By the Itô representation theorem [30], there exist two processes Q
=
(
Q
1
,
…
,
Q
d
) where Q
j
∈
ℳ
2, for j
=
1
,
…
,
d, and U
(
·
)
∈
ℒ
ν
2, satisfying
(43)
M
t
=
𝔼
[
∫
0
T
‍
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
d
s
+
g
x
(
x
T
⋆
)
Φ
T
]
+
∑
j
=
1
d
‍
∫
0
t
‍
Q
s
j
d
B
s
j
+
∫
0
t
‍
∫
E
‍
U
s
(
e
)
N
~
(
d
s
,
d
e
)
.Let us denotey
t
⋆
=
M
t
-
∫
0
t
‍
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
d
s. The adjoint variable is the process defined by
(44)
p
t
=
y
t
⋆
Ψ
t
,
q
t
j
=
Q
t
j
Ψ
t
-
p
t
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
,
for
j
=
1
,
…
,
d
,
r
t
(
e
)
=
U
t
(
e
)
Ψ
t
(
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
+
p
t
(
(
γ
x
(
s
,
x
t
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
-
I
d
)
.Theorem 6.
Under assumptions (H1)–(H5), one has
(45)
I
=
𝔼
[
∑
i
=
1
m
‍
∑
‍
0
<
s
≤
T
∫
0
T
‍
{
∫
E
‍
f
u
(
s
,
x
s
⋆
,
u
s
⋆
)
+
p
s
b
u
(
s
,
x
s
⋆
,
u
s
⋆
)
+
∑
j
=
1
d
‍
q
s
j
σ
u
j
(
s
,
x
s
⋆
,
u
s
⋆
)
+
∫
E
‍
r
s
(
z
)
γ
u
(
s
,
x
s
⋆
,
u
s
⋆
,
e
)
ν
(
d
e
)
}
v
s
d
s
+
∑
i
=
1
m
‍
∫
0
T
‍
{
k
s
i
+
G
s
i
p
s
}
d
ξ
s
c
i
+
∑
i
=
1
m
‍
∑
‍
0
<
s
≤
T
{
k
s
i
+
G
s
i
(
p
s
-
+
Δ
N
p
s
)
}
Δ
ξ
s
i
]
.Proof.
From the integration by parts formula ([8, Lemma 3.5]), and by using the definition of p
t
,
q
t
j for j
=
1
,
…
,
d, and r
t
(
·
), we can easily check that
(46)
E
[
y
T
η
T
]
=
𝔼
[
∑
i
=
1
m
‍
∫
0
T
‍
{
p
t
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
+
∑
j
=
1
d
‍
q
s
j
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
+
∫
E
‍
r
t
(
e
)
γ
u
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
ν
(
d
e
)
∑
j
=
1
d
‍
}
v
t
d
t
-
∫
0
T
‍
f
x
(
t
,
x
t
⋆
,
u
t
⋆
)
η
t
Φ
t
d
t
+

∑
i
=
1
m
‍
(
∫
0
T
‍
G
t
i
p
t
d
ξ
t
c
i
+
∑
‍
0
<
t
≤
T
G
t
i
(
p
t
-
+
Δ
N
p
t
)
Δ
ξ
t
i
)

∫
0
T
‍
{
p
t
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
+
∑
j
=
1
d
‍
q
s
j
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
]
.
Also we have(47)
I
=
𝔼
[
y
T
η
T
+
∫
0
T
‍
f
x
(
t
,
x
t
⋆
,
u
t
⋆
)
Φ
t
η
t
d
t
+
∫
0
T
‍
f
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
d
t
+
∫
0
T
‍
k
t
d
ξ
t
]
,
substituting (46) in (47), the result follows.


### 3.1.3.  Adjoint Equation and Maximum Principle

Since (37) is true for all (
v
,
ξ
)
∈
𝒰 and I
≤
0, we can easily deduce the following result.Theorem 7.
Let(
u
⋆
,
ξ
⋆
) be the optimal control of the problem (14) and denote by x
⋆ the corresponding optimal trajectory, then the following inequality holds:
(48)
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
d
t
+
∫
0
T
‍
{
k
t
+
G
t
p
t
}
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
(
ξ
-
ξ
⋆
)
t
]
≤
0
,
where the Hamiltonian H is defined by (17), and the adjoint variable (
p
,
q
j
,
r
(
·
)
) for j
=
1
,
…
,
d, is given by (44).Now, we are ready to give the proof of Theorem2.Proof of Theorem2.
(i) Let us assume that(
u
⋆
,
ξ
⋆
) is an optimal control for the problem (14), so that inequality (48) is valid for every (
v
,
ξ
). If we choose ξ
=
ξ
⋆ in inequality (48), we see that for every measurable, ℱ
t-adapted process v
:
[
0
,
T
]
×
Ω
→
A
1
(49)
𝔼
[
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
d
t
]
≤
0
.
Forv
∈
𝒰
1 define
(50)
A
v
=
{
H
v
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
(
t
,
ω
)
∈
[
0
,
T
]
×
Ω
such
that
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
>
0
}
.
ObviouslyA
t
v
∈
ℱ
t, for each t
∈
[
0
,
T
]. Let us define v
~
∈
𝒰
1 by
(51)
v
~
t
(
ω
)
=
{
v
,
if
(
t
,
ω
)
∈
A
t
v
,
u
t
⋆
,
otherwise
.
Ifλ
⊗
ℙ
(
A
v
)
>
0, where λ denotes the Lebesgue measure, then
(52)
𝔼
[
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
~
t
-
u
t
⋆
)
d
t
]
>
0
,
which contradicts (49), unless λ
⊗
ℙ
(
A
v
)
=
0. Hence the conclusion follows.
(ii) If instead we choosev
=
u
⋆ in inequality (48), we obtain that for every measurable, ℱ
t-adapted process ξ
:
[
0
,
T
]
×
Ω
→
A
2, the following inequality holds:
(53)
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
‍
{
k
t
+
G
t
p
t
}
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
(
ξ
-
ξ
⋆
)
t
]
≤
0
.
In particular, fori
=
1
,
…
,
m, we put ξ
t
i
=
ξ
t
⋆
i
+
1
{
k
t
i
+
G
t
i
p
t
>
0
}
λ
(
t
). Since the Lebesgue measure is regular then the purely discontinuous part (
ξ
t
i
-
ξ
t
⋆
i
)
d
=
0. Obviously, the relation (53) can be written as
(54)
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
d
(
ξ
i
-
ξ
⋆
i
)
t
c
+
∫
0
T
‍
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
d
(
ξ
i
-
ξ
⋆
i
)
t
d
]
=
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
1
{
k
t
i
+
G
t
i
p
t
>
0
}
d
λ
(
t
)
]
>
0
.
This contradicts (53) unless for every i
∈
{
1
,
…
,
m
}
,
λ
⊗
ℙ
{
k
t
i
+
G
t
i
p
t
>
0
}
=
0. This proves (20).
Let us prove (21). Define d
ξ
t
i
=
1
{
k
t
i
+
G
t
i
p
t
-
>
0
}
d
ξ
t
⋆
i
+
1
{
k
t
i
+
G
t
i
p
t
-
≤
0
}
d
ξ
t
⋆
d
i, for i
=
1
,
…
,
m, then we have d
(
ξ
i
-
ξ
⋆
i
)
t
c
=
-
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i, and d
ξ
t
d
i
=
d
ξ
t
⋆
d
i. Hence, we can rewrite (53) as follows:
(55)
-
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i
]
>
0
.
By comparing with (53) we get
(56)
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i
]
=
0
,
then we conclude that
(57)
∑
i
=
1
m
‍
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
c
i
=
0
.
Expressions (22) and (23) are proved by using the same techniques. First, for each i
∈
{
1
,
…
,
m
} and t
∈
[
0
,
T
] fixed, we define ξ
s
i
=
ξ
s
i
+
δ
t
(
s
)
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}, where δ
t denotes the Dirac unit mass at t
.
δ
t is a discrete measure, then (
ξ
s
i
-
ξ
s
i
)
c
=
0 and (
ξ
s
i
-
ξ
s
i
)
d
=
δ
t
(
s
)
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}. Hence
(58)
𝔼
[
∑
i
=
1
m
‍
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}
]
>
0
which contradicts (53), unless for every i
∈
{
1
,
…
,
m
} and t
∈
[
0
,
T
], we have
(59)
ℙ
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}
=
0
.
Next, letξ be defined by
(60)
d
ξ
t
i
=
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≥
0
}
d
ξ
t
⋆
i
+
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
d
ξ
t
⋆
c
i
.
Then, the relation (53) can be written as
(61)
∑
i
=
1
m
‍
𝔼
[
∑
‍
-
0
<
t
≤
T
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
×
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
Δ
ξ
t
⋆
i
∑
‍
-
0
<
t
≤
T
]
>
0
,
which implies that
(62)
𝔼
[
∑
i
=
1
m
‍
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
×
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
Δ
ξ
t
⋆
i
∑
i
=
1
m
‍
]
=
0
.
By the fact that k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0, and Δ
ξ
t
i
≥
0, we get
(63)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
Δ
ξ
t
⋆
i
=
0
.
Thus (23) holds. The proof is complete.Now, by applying Itô’s formula toy
t
⋆
Ψ
t, it is easy to check that the processes defined by relation (44) satisfy BSDE (18) called the adjoint equation.


## 3.1.1. Variational Equation

Let(
v
,
ξ
)
∈
𝒰 be such that (
u
⋆
+
v
,
ξ
⋆
+
ξ
)
∈
𝒰. The convexity condition of the control domain ensures that for ɛ
∈
(
0,1
) the control (
u
⋆
+
ɛ
v
,
ξ
⋆
+
ɛ
ξ
) is also in 𝒰. We denote by x
ɛ the solution of the SDE (8) corresponding to the control (
u
⋆
+
ɛ
v
,
ξ
⋆
+
ɛ
ξ
). Then by standard arguments from stochastic calculus, it is easy to check the following estimate.Lemma 3.
Under assumptions (H1)–(H5), one has
(24)
lim
ɛ
→
0
𝔼
[
sup
t
∈
[
0
,
T
]
|
x
t
ɛ
-
x
t
⋆
|
2
]
=
0
.Proof.
From assumptions (H1)–(H5), we get by using the Burkholder-Davis-Gundy inequality
(25)
𝔼
[
sup
t
∈
[
0
,
T
]
|
x
t
ɛ
-
x
t
⋆
|
2
]
≤
K
∫
0
T
‍
𝔼
[
sup
τ
∈
[
0
,
s
]
|
x
τ
ɛ
-
x
τ
⋆
|
2
]
d
s
+
K
ɛ
2
(
∫
0
T
‍
𝔼
[
sup
τ
∈
[
0
,
s
]
|
v
τ
|
2
]
d
s
+
𝔼
|
ξ
T
|
2
)
From Definition1 and Gronwall’s lemma, the result follows immediately by letting ɛ go to zero.We define the processz
t
=
z
t
u
⋆
,
v
,
ξ by
(26)
d
z
t
=
{
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
}
d
t
+
∑
j
=
1
d
‍
{
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
}
d
B
t
j
+
∫
E
‍
{
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
z
t
-
+
γ
u
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
v
t
}
×
N
~
(
d
t
,
d
e
)
+
G
t
d
ξ
t
,
z
0
=
0
.From (H2) and Definition 1, one can find a unique solution z which solves the variational equation (26), and the following estimate holds.Lemma 4.
Under assumptions (H1)–(H5), it holds that
(27)
lim
ɛ
→
0
𝔼
|
x
t
ɛ
-
x
t
⋆
ɛ
-
z
t
|
2
=
0
.Proof.
Let(28)
Γ
t
ɛ
=
x
t
ɛ
-
x
t
⋆
ɛ
-
z
t
.
We denotex
t
μ
,
ɛ
=
x
t
⋆
+
μ
ɛ
(
Γ
t
ɛ
+
z
t
) and u
t
μ
,
ɛ
=
u
t
⋆
+
μ
ɛ
v
t, for notational convenience. Then we have immediately that Γ
0
ɛ
=
0 and Γ
t
ɛ satisfies the following SDE:
(29)
d
Γ
t
ɛ
=
{
1
ɛ
(
b
(
t
,
x
t
μ
,
ɛ
,
u
t
μ
,
ɛ
)
-
b
(
t
,
x
t
⋆
,
u
t
⋆
)
)
-
(
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
)
1
ɛ
(
b
(
t
,
x
t
μ
,
ɛ
,
u
t
μ
,
ɛ
)
-
b
(
t
,
x
t
⋆
,
u
t
⋆
)
)
}
d
t
+
{
1
ɛ
(
σ
(
t
,
x
t
μ
,
ɛ
,
u
t
μ
,
ɛ
)
-
σ
(
t
,
x
t
⋆
,
u
t
⋆
)
)
-
(
σ
x
(
t
,
x
t
⋆
,
u
t
⋆
)
z
t
+
σ
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
)
1
ɛ
}
d
B
t
+
∫
E
{
1
ɛ
(
γ
(
t
,
x
t
-
μ
,
ɛ
,
u
t
μ
,
ɛ
,
e
)
-
γ
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
)
‍
-
(
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
z
t
-
+
γ
u
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
v
t
)
1
ɛ
(
γ
(
t
,
x
t
-
μ
,
ɛ
,
u
t
μ
,
ɛ
,
e
)
-
γ
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
)
}
×
N
~
(
d
t
,
d
e
)
.
Since the derivatives of the coefficients are bounded, and from Definition1, it is easy to verify by Gronwall’s inequality that Γ
ɛ
∈
𝒮
2 and
(30)
𝔼
|
Γ
t
ɛ
|
2
≤
K
𝔼
∫
0
t
‍
|
∫
0
1
‍
b
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
Γ
s
ɛ
d
μ
|
2
d
s
+
K
𝔼
∫
0
t
‍
|
∫
0
1
‍
σ
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
Γ
s
ɛ
d
μ
|
2
d
s
+
K
𝔼
∫
0
t
‍
∫
E
‍
|
∫
0
1
‍
γ
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
,
e
)
Γ
s
ɛ
d
μ
|
2
ν
(
d
e
)
d
s
+
K
𝔼
|
ρ
t
ɛ
|
2
,
where ρ
t
ɛ is given by
(31)
ρ
t
ɛ
=
-
∫
0
t
‍
b
x
(
s
,
x
s
⋆
,
u
s
⋆
)
z
s
d
s
-
∫
0
t
‍
σ
x
(
s
,
x
s
⋆
,
u
s
⋆
)
z
s
d
B
s
-
∫
0
t
‍
∫
E
‍
γ
x
(
s
,
x
s
-
⋆
,
u
s
⋆
,
e
)
z
s
-
N
~
(
d
s
,
d
e
)
-
∫
0
t
‍
b
v
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
d
s
-
∫
0
t
‍
σ
v
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
d
B
s
-
∫
0
t
‍
∫
E
‍
γ
v
(
s
,
x
s
-
⋆
,
u
s
⋆
,
e
)
v
s
N
~
(
d
s
,
d
e
)
+
∫
0
t
‍
∫
0
1
‍
b
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
z
s
d
μ
d
s
+
∫
0
t
‍
∫
0
1
‍
σ
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
z
s
d
μ
d
B
s
+
∫
0
t
‍
∫
E
‍
∫
0
1
‍
γ
x
(
s
,
x
s
-
μ
,
ɛ
,
u
s
μ
,
ɛ
,
e
)
z
s
-
d
μ
N
~
(
d
s
,
d
e
)
+
∫
0
t
‍
∫
0
1
‍
b
v
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
v
s
d
μ
d
s
+
∫
0
t
‍
∫
0
1
‍
σ
v
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
v
s
d
μ
d
B
s
+
∫
0
t
‍
∫
E
‍
∫
0
1
‍
γ
v
(
s
,
x
s
-
μ
,
ɛ
,
u
s
μ
,
ɛ
,
e
)
v
s
d
μ
N
~
(
d
s
,
d
e
)
.
Sinceb
x, σ
x, and γ
x are bounded, then
(32)
𝔼
|
Γ
t
ɛ
|
2
≤
M
𝔼
∫
0
t
‍
|
Γ
s
ɛ
|
2
d
s
+
M
𝔼
|
ρ
t
ɛ
|
2
,
where M is a generic constant depending on the constants K, ν
(
E
), and T. We conclude from Lemma 3 and the dominated convergence theorem, that lim
ɛ
→
0
ρ
t
ɛ
=
0. Hence (27) follows from Gronwall’s lemma and by letting ɛ go to 0. This completes the proof.


## 3.1.2. Variational Inequality

LetΦ be the solution of the linear matrix equation, for 0
≤
s
<
t
≤
T
(33)
d
Φ
s
,
t
=
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
Φ
s
,
t
d
t
+
∑
j
=
1
d
‍
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
Φ
s
,
t
d
B
t
j
+
∫
E
‍
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
Φ
s
,
t
-
N
~
(
d
t
,
d
e
)
,
Φ
s
,
s
=
I
d
,
where I
d is the n
×
n identity matrix. This equation is linear, with bounded coefficients, then it admits a unique strong solution. Moreover, the condition (H4) ensures that the tangent process Φ is invertible, with an inverse Ψ satisfying suitable integrability conditions.From Itô’s formula, we can easily check thatd
(
Φ
s
,
t
Ψ
s
,
t
)
=
0, and Φ
s
,
s
Ψ
s
,
s
=
I
d, where Ψ is the solution of the following equation
(34)
d
Ψ
s
,
t
=
-
Ψ
s
,
t
{
b
x
(
t
,
x
t
⋆
,
u
t
⋆
)
-
∑
j
=
1
d
‍
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
-
∫
E
‍
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
ν
(
d
e
)
∑
j
=
1
d
‍
}
d
t
-
∑
j
=
1
d
‍
Ψ
s
,
t
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
d
B
t
j
-
Ψ
s
,
t
-
∫
E
‍
(
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
×
N
(
d
t
,
d
e
)
,
Ψ
s
,
s
=
I
d
,
so Ψ
=
Φ
-
1. If s
=
0 we simply write Φ
0
,
t
=
Φ
t and Ψ
0
,
t
=
Ψ
t. By the integration by parts formula ([8, Lemma 3.6]), we can see that the solution of (26) is given by z
t
=
Φ
t
η
t, where η
t is the solution of the stochastic differential equation
(35)
d
η
t
=
Ψ
t
{
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
-
∑
j
=
1
d
‍
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
-
∫
E
‍
γ
u
(
t
,
x
t
⋆
,
u
t
⋆
,
z
)
v
t
ν
(
d
e
)
∑
j
=
1
d
‍
}
d
t
+
∑
j
=
1
d
‍
Ψ
t
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
d
B
t
j
+
Ψ
t
-
∫
E
‍
(
γ
x
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
×
γ
u
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
v
t
N
(
d
t
,
d
e
)
+
Ψ
t
G
t
d
ξ
t
-
Ψ
t
∫
E
‍
(
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
×
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
N
(
{
t
}
,
d
e
)
G
t
Δ
ξ
t
,
η
0
=
0
.Let us introduce the following convex perturbation of the optimal control(
u
⋆
,
ξ
⋆
) defined by
(36)
(
u
⋆
,
ɛ
,
ξ
⋆
,
ɛ
)
=
(
u
⋆
+
ɛ
v
,
ξ
⋆
+
ɛ
ξ
)
,
for some (
v
,
ξ
)
∈
𝒰 and ɛ
∈
(
0,1
). Since (
u
⋆
,
ξ
⋆
) is an optimal control, then ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
≤
0. Thus a necessary condition for optimality is that
(37)
lim
ɛ
→
0
ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
≤
0
.The rest of this subsection is devoted to the computation of the above limit. We will see that the expression (37) leads to a precise description of the optimal control (
u
⋆
,
ξ
⋆
) in terms of the adjoint process. First, it is easy to prove the following lemma.Lemma 5.
Under assumptions (H1)–(H5), one has
(38)
I
=
lim
ɛ
→
0
ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
=
𝔼
[
∫
0
T
‍
{
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
z
s
+
f
u
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
}
d
s
+
g
x
(
x
T
⋆
)
z
T
+
∫
0
T
‍
k
t
d
ξ
t
]
.Proof.
We use the same notations as in the proof of Lemma4. First, we have
(39)
ɛ
-
1
(
J
(
u
ɛ
,
ξ
ɛ
)
-
J
(
u
⋆
,
ξ
⋆
)
)
=
𝔼
[
∫
0
T
‍
∫
0
1
‍
{
f
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
z
s
+
f
u
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
v
s
}
d
μ
d
s
+
∫
0
1
‍
g
x
(
x
T
μ
,
ɛ
)
z
T
d
μ
+
∫
0
T
‍
k
t
d
ξ
t
]
+
β
t
ɛ
,
where
(40)
β
t
ɛ
=
𝔼
[
∫
0
T
‍
∫
0
1
‍
f
x
(
s
,
x
s
μ
,
ɛ
,
u
s
μ
,
ɛ
)
Γ
s
ɛ
d
μ
d
s
+
∫
0
1
‍
g
x
(
x
T
μ
,
ɛ
)
Γ
T
ɛ
d
μ
]
.
By using Lemma4, and since the derivatives f
x
,
f
u, and g
x are bounded, we have lim
ɛ
→
0
β
t
ɛ
=
0. Then, the result follows by letting ɛ go to 0 in the above equality.Substituting byz
t
=
Φ
t
η
t in (38) leads to
(41)
I
=
𝔼
[
∫
0
T
‍
{
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
η
s
+
f
u
(
s
,
x
s
⋆
,
u
s
⋆
)
v
s
}
d
s
+
g
x
(
x
T
⋆
)
Φ
T
η
T
+
∫
0
T
‍
k
t
d
ξ
t
]
.Consider the right continuous version of the square integrable martingale(42)
M
t
≔
𝔼
[
∫
0
T
‍
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
d
s
+
g
x
(
x
T
⋆
)
Φ
T
∣
ℱ
t
]
.By the Itô representation theorem [30], there exist two processes Q
=
(
Q
1
,
…
,
Q
d
) where Q
j
∈
ℳ
2, for j
=
1
,
…
,
d, and U
(
·
)
∈
ℒ
ν
2, satisfying
(43)
M
t
=
𝔼
[
∫
0
T
‍
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
d
s
+
g
x
(
x
T
⋆
)
Φ
T
]
+
∑
j
=
1
d
‍
∫
0
t
‍
Q
s
j
d
B
s
j
+
∫
0
t
‍
∫
E
‍
U
s
(
e
)
N
~
(
d
s
,
d
e
)
.Let us denotey
t
⋆
=
M
t
-
∫
0
t
‍
f
x
(
s
,
x
s
⋆
,
u
s
⋆
)
Φ
s
d
s. The adjoint variable is the process defined by
(44)
p
t
=
y
t
⋆
Ψ
t
,
q
t
j
=
Q
t
j
Ψ
t
-
p
t
σ
x
j
(
t
,
x
t
⋆
,
u
t
⋆
)
,
for
j
=
1
,
…
,
d
,
r
t
(
e
)
=
U
t
(
e
)
Ψ
t
(
γ
x
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
+
p
t
(
(
γ
x
(
s
,
x
t
⋆
,
u
t
⋆
,
e
)
+
I
d
)
-
1
-
I
d
)
.Theorem 6.
Under assumptions (H1)–(H5), one has
(45)
I
=
𝔼
[
∑
i
=
1
m
‍
∑
‍
0
<
s
≤
T
∫
0
T
‍
{
∫
E
‍
f
u
(
s
,
x
s
⋆
,
u
s
⋆
)
+
p
s
b
u
(
s
,
x
s
⋆
,
u
s
⋆
)
+
∑
j
=
1
d
‍
q
s
j
σ
u
j
(
s
,
x
s
⋆
,
u
s
⋆
)
+
∫
E
‍
r
s
(
z
)
γ
u
(
s
,
x
s
⋆
,
u
s
⋆
,
e
)
ν
(
d
e
)
}
v
s
d
s
+
∑
i
=
1
m
‍
∫
0
T
‍
{
k
s
i
+
G
s
i
p
s
}
d
ξ
s
c
i
+
∑
i
=
1
m
‍
∑
‍
0
<
s
≤
T
{
k
s
i
+
G
s
i
(
p
s
-
+
Δ
N
p
s
)
}
Δ
ξ
s
i
]
.Proof.
From the integration by parts formula ([8, Lemma 3.5]), and by using the definition of p
t
,
q
t
j for j
=
1
,
…
,
d, and r
t
(
·
), we can easily check that
(46)
E
[
y
T
η
T
]
=
𝔼
[
∑
i
=
1
m
‍
∫
0
T
‍
{
p
t
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
+
∑
j
=
1
d
‍
q
s
j
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
+
∫
E
‍
r
t
(
e
)
γ
u
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
ν
(
d
e
)
∑
j
=
1
d
‍
}
v
t
d
t
-
∫
0
T
‍
f
x
(
t
,
x
t
⋆
,
u
t
⋆
)
η
t
Φ
t
d
t
+

∑
i
=
1
m
‍
(
∫
0
T
‍
G
t
i
p
t
d
ξ
t
c
i
+
∑
‍
0
<
t
≤
T
G
t
i
(
p
t
-
+
Δ
N
p
t
)
Δ
ξ
t
i
)

∫
0
T
‍
{
p
t
b
u
(
t
,
x
t
⋆
,
u
t
⋆
)
+
∑
j
=
1
d
‍
q
s
j
σ
u
j
(
t
,
x
t
⋆
,
u
t
⋆
)
]
.
Also we have(47)
I
=
𝔼
[
y
T
η
T
+
∫
0
T
‍
f
x
(
t
,
x
t
⋆
,
u
t
⋆
)
Φ
t
η
t
d
t
+
∫
0
T
‍
f
u
(
t
,
x
t
⋆
,
u
t
⋆
)
v
t
d
t
+
∫
0
T
‍
k
t
d
ξ
t
]
,
substituting (46) in (47), the result follows.


## 3.1.3.  Adjoint Equation and Maximum Principle

Since (37) is true for all (
v
,
ξ
)
∈
𝒰 and I
≤
0, we can easily deduce the following result.Theorem 7.
Let(
u
⋆
,
ξ
⋆
) be the optimal control of the problem (14) and denote by x
⋆ the corresponding optimal trajectory, then the following inequality holds:
(48)
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
d
t
+
∫
0
T
‍
{
k
t
+
G
t
p
t
}
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
(
ξ
-
ξ
⋆
)
t
]
≤
0
,
where the Hamiltonian H is defined by (17), and the adjoint variable (
p
,
q
j
,
r
(
·
)
) for j
=
1
,
…
,
d, is given by (44).Now, we are ready to give the proof of Theorem2.Proof of Theorem2.
(i) Let us assume that(
u
⋆
,
ξ
⋆
) is an optimal control for the problem (14), so that inequality (48) is valid for every (
v
,
ξ
). If we choose ξ
=
ξ
⋆ in inequality (48), we see that for every measurable, ℱ
t-adapted process v
:
[
0
,
T
]
×
Ω
→
A
1
(49)
𝔼
[
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
d
t
]
≤
0
.
Forv
∈
𝒰
1 define
(50)
A
v
=
{
H
v
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
(
t
,
ω
)
∈
[
0
,
T
]
×
Ω
such
that
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
>
0
}
.
ObviouslyA
t
v
∈
ℱ
t, for each t
∈
[
0
,
T
]. Let us define v
~
∈
𝒰
1 by
(51)
v
~
t
(
ω
)
=
{
v
,
if
(
t
,
ω
)
∈
A
t
v
,
u
t
⋆
,
otherwise
.
Ifλ
⊗
ℙ
(
A
v
)
>
0, where λ denotes the Lebesgue measure, then
(52)
𝔼
[
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
~
t
-
u
t
⋆
)
d
t
]
>
0
,
which contradicts (49), unless λ
⊗
ℙ
(
A
v
)
=
0. Hence the conclusion follows.
(ii) If instead we choosev
=
u
⋆ in inequality (48), we obtain that for every measurable, ℱ
t-adapted process ξ
:
[
0
,
T
]
×
Ω
→
A
2, the following inequality holds:
(53)
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
‍
{
k
t
+
G
t
p
t
}
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
(
ξ
-
ξ
⋆
)
t
]
≤
0
.
In particular, fori
=
1
,
…
,
m, we put ξ
t
i
=
ξ
t
⋆
i
+
1
{
k
t
i
+
G
t
i
p
t
>
0
}
λ
(
t
). Since the Lebesgue measure is regular then the purely discontinuous part (
ξ
t
i
-
ξ
t
⋆
i
)
d
=
0. Obviously, the relation (53) can be written as
(54)
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
d
(
ξ
i
-
ξ
⋆
i
)
t
c
+
∫
0
T
‍
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
d
(
ξ
i
-
ξ
⋆
i
)
t
d
]
=
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
1
{
k
t
i
+
G
t
i
p
t
>
0
}
d
λ
(
t
)
]
>
0
.
This contradicts (53) unless for every i
∈
{
1
,
…
,
m
}
,
λ
⊗
ℙ
{
k
t
i
+
G
t
i
p
t
>
0
}
=
0. This proves (20).
Let us prove (21). Define d
ξ
t
i
=
1
{
k
t
i
+
G
t
i
p
t
-
>
0
}
d
ξ
t
⋆
i
+
1
{
k
t
i
+
G
t
i
p
t
-
≤
0
}
d
ξ
t
⋆
d
i, for i
=
1
,
…
,
m, then we have d
(
ξ
i
-
ξ
⋆
i
)
t
c
=
-
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i, and d
ξ
t
d
i
=
d
ξ
t
⋆
d
i. Hence, we can rewrite (53) as follows:
(55)
-
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i
]
>
0
.
By comparing with (53) we get
(56)
∑
i
=
1
m
‍
𝔼
[
∫
0
T
‍
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i
]
=
0
,
then we conclude that
(57)
∑
i
=
1
m
‍
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
c
i
=
0
.
Expressions (22) and (23) are proved by using the same techniques. First, for each i
∈
{
1
,
…
,
m
} and t
∈
[
0
,
T
] fixed, we define ξ
s
i
=
ξ
s
i
+
δ
t
(
s
)
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}, where δ
t denotes the Dirac unit mass at t
.
δ
t is a discrete measure, then (
ξ
s
i
-
ξ
s
i
)
c
=
0 and (
ξ
s
i
-
ξ
s
i
)
d
=
δ
t
(
s
)
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}. Hence
(58)
𝔼
[
∑
i
=
1
m
‍
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}
]
>
0
which contradicts (53), unless for every i
∈
{
1
,
…
,
m
} and t
∈
[
0
,
T
], we have
(59)
ℙ
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
>
0
}
=
0
.
Next, letξ be defined by
(60)
d
ξ
t
i
=
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≥
0
}
d
ξ
t
⋆
i
+
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
d
ξ
t
⋆
c
i
.
Then, the relation (53) can be written as
(61)
∑
i
=
1
m
‍
𝔼
[
∑
‍
-
0
<
t
≤
T
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
×
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
Δ
ξ
t
⋆
i
∑
‍
-
0
<
t
≤
T
]
>
0
,
which implies that
(62)
𝔼
[
∑
i
=
1
m
‍
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
}
×
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
Δ
ξ
t
⋆
i
∑
i
=
1
m
‍
]
=
0
.
By the fact that k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0, and Δ
ξ
t
i
≥
0, we get
(63)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
<
0
}
Δ
ξ
t
⋆
i
=
0
.
Thus (23) holds. The proof is complete.Now, by applying Itô’s formula toy
t
⋆
Ψ
t, it is easy to check that the processes defined by relation (44) satisfy BSDE (18) called the adjoint equation.


## 3.2.  Sufficient Conditions of Optimality

It is well known that in the classical cases (without the singular part of the control), the sufficient condition of optimality is of significant importance in the stochastic maximum principle, in the sense that it allows to compute optimal controls. This result states that, under some concavity conditions, maximizing the Hamiltonian leads to an optimal control.In this section, we focus on proving the sufficient maximum principle for mixed classical-singular stochastic control problems, where the state of the system is governed by a stochastic differential equation with jumps, allowing both classical control and singular control.Theorem 8 (sufficient condition of optimality in integral form).
Let(
u
⋆
,
ξ
⋆
) be an admissible control and denote x
⋆ the associated controlled state process. Let (
p
,
q
,
r
(
·
)
) be the unique solution of B
S
D
E (18). Let one assume that (
x
,
u
)
→
H
(
t
,
x
,
u
,
p
t
,
q
t
,
r
t
(
·
)
) and x
→
g
(
x
) are concave functions. Moreover suppose that for all t
∈
[
0
,
T
], v
∈
A
1, and ξ
∈
𝒰
2
(64)
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
‍
H
v
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
(
v
t
-
u
t
⋆
)
d
t
+
∫
0
T
‍
{
k
t
+
G
t
p
t
}
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
(
ξ
-
ξ
⋆
)
t
]
≤
0
.
Then(
u
⋆
,
ξ
⋆
) is an optimal control.Proof.
For convenience, we will use the following notations throughout the proof:(65)
Θ
⋆
(
t
)
=
Θ
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
,
Θ
(
t
)
=
Θ
(
t
,
x
t
,
u
t
,
p
t
,
q
t
,
r
t
(
·
)
)
,
for
Θ
=
H
,
H
x
,
H
u
δ
ϕ
(
t
)
=
ϕ
(
t
,
x
t
⋆
,
u
t
⋆
)
-
ϕ
(
t
,
x
t
,
u
t
)
,
for
ϕ
=
b
,
σ
,
σ
j
,
j
=
1
,
…
,
n
,
f
δ
γ
(
t
,
e
)
=
γ
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
-
γ
(
t
,
x
t
,
u
t
,
e
)
,
δ
γ
-
(
t
,
e
)
=
γ
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
-
γ
(
t
,
x
t
-
,
u
t
,
e
)
.
Let(
u
,
ξ
) be an arbitrary admissible pair, and consider the difference
(66)
J
(
u
⋆
,
ξ
⋆
)
-
J
(
u
,
ξ
)
=
𝔼
[
∫
0
T
‍
δ
f
(
t
)
d
t
+
∫
0
T
‍
k
t
d
(
ξ
⋆
-
ξ
)
t
]
+
𝔼
[
g
(
x
T
⋆
)
-
g
(
x
T
)
]
.
We first note that, by concavity ofg, we conclude  that
(67)
𝔼
[
g
(
x
T
⋆
)
-
g
(
x
T
)
]
≥
𝔼
[
(
x
T
⋆
-
x
T
)
g
x
(
x
T
⋆
)
]
=
𝔼
[
(
x
T
⋆
-
x
T
)
p
T
]
=
𝔼
[
∫
0
T
‍
(
x
t
-
⋆
-
x
t
-
)
d
p
t
+
∫
0
T
‍
p
t
-
d
(
x
t
⋆
-
x
t
)
]
+
𝔼
[
∫
0
T
‍
∑
j
=
1
n
‍
(
δ
σ
j
(
t
)
)
q
t
j
d
t
+
∫
0
T
‍
∫
E
‍
(
δ
γ
-
(
t
,
e
)
)
r
t
(
e
)
N
(
d
t
,
d
e
)
∫
0
T
‍
∑
j
=
1
n
‍
]
+
𝔼
[
∑
‍
0
<
t
≤
T
G
t
(
Δ
N
p
t
)
Δ
(
ξ
-
ξ
⋆
)
t
]
,
which implies that
(68)
E
[
g
(
x
T
⋆
)
-
g
(
x
T
)
]
≥
𝔼
[
∫
0
T
‍
(
x
t
⋆
-
x
t
)
(
-
H
x
⋆
(
t
)
)
d
t
]
+
𝔼
[
∫
0
T
‍
{
p
t
(
δ
b
(
t
)
)
+
∑
j
=
1
n
‍
(
δ
σ
j
(
t
)
)
q
t
j
}
d
t
]
+
𝔼
[
∫
0
T
‍
∫
E
‍
(
δ
γ
-
(
t
,
e
)
)
r
t
(
e
)
N
(
d
t
,
d
e
)
]
+
𝔼
[
∫
0
T
‍
{
(
x
t
⋆
-
x
t
)
q
t
+
(
δ
σ
(
t
)
)
p
t
}
d
B
t
]
+
𝔼
[
∫
0
T
‍
∫
E
‍
{
(
x
t
-
⋆
-
x
t
-
)
r
t
(
e
)
+
p
t
-
(
δ
γ
-
(
t
,
e
)
)
}
×
N
~
(
d
t
,
d
e
)
∫
0
T
‍
∫
E
‍
]
+
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
‍
G
t
p
t
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
G
t
(
p
t
-
+
Δ
N
p
t
)
Δ
(
ξ
-
ξ
⋆
)
t
]
.
By the fact that(
p
,
q
j
,
r
(
·
)
)
∈
𝒮
2
×
ℳ
2
×
ℒ
ν
2 for j
=
1
,
…
,
n, we deduce that the stochastic integrals with respect to the local martingales have zero expectation. Due to the concavity of the Hamiltonian H, the following holds
(69)
𝔼
[
g
(
x
T
⋆
)
-
g
(
x
T
)
]
≥
𝔼
[
∫
0
T
‍
{
-
(
H
⋆
(
t
)
-
H
(
t
)
)
+
H
u
⋆
(
t
)
(
u
t
⋆
-
u
t
)
}
d
t
]
+
𝔼
[
∫
0
T
‍
{
p
t
(
δ
b
(
t
)
)
+
∑
j
=
1
n
‍
(
δ
σ
j
(
t
)
)
q
t
j
+
∫
E
‍
(
δ
γ
(
t
,
e
)
)
r
t
(
e
)
ν
(
d
e
)
∑
j
=
1
n
‍
}
d
t
]
+
𝔼
[
∑
0
<
t
≤
T
‍
∫
0
T
‍
G
t
T
p
t
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
0
<
t
≤
T
‍
G
t
T
(
p
t
-
+
Δ
N
p
t
)
Δ
(
ξ
-
ξ
⋆
)
t
]
.
The definition of the HamiltonianH and (64) leads to J
(
u
⋆
,
ξ
⋆
)
-
J
(
u
,
ξ
)
≥
0, which means that (
u
⋆
,
ξ
⋆
) is an optimal control for the problem (14).The expression (64) is a sufficient condition of optimality in integral form. We want to rewrite this inequality in a suitable form for applications. This is the objective of the following theorem which could be seen as a natural extension of [2, Theorem 2.2] to the jump setting and [3, Theorem 2.1] to mixed regular-singular control problems.Theorem 9 (sufficient conditions of optimality).
Let(
u
⋆
,
ξ
⋆
) be an admissible control and x
⋆ the associated controlled state process. Let (
p
,
q
,
r
(
·
)
) be the unique solution of B
S
D
E (18). Let one assume that (
x
,
u
)
→
H
(
t
,
x
,
u
,
p
t
,
q
t
,
r
t
(
·
)
) and x
→
g
(
x
) are concave functions. If in addition one assumes that(i)
for allt
∈
[
0
,
T
]
,
v
∈
A
1
(70)
H
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
=
sup
v
∈
A
1
H
(
t
,
x
t
⋆
,
v
,
p
t
,
q
t
,
r
t
(
·
)
)
,
d
t
—
a
.
e
.
,
ℙ
—
a
.
s
;
(ii)
for allt
∈
[
0
,
T
], with probability 1
(71)
k
t
i
+
G
t
i
p
t
≤
0
,
f
o
r
i
=
1
,
…
,
m
,
(72)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
p
t
≤
0
}
d
ξ
t
⋆
c
i
=
0
,
(73)
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≤
0
,
f
o
r
i
=
1
,
…
,
m
,
(74)
∑
i
=
1
m
‍
1
{
k
t
i
+
G
t
i
(
p
t
-
+
Δ
N
p
t
)
≤
0
}
Δ
ξ
t
⋆
i
=
0
.
Then(
u
⋆
,
ξ
⋆
) is an optimal control.Proof.
Using (71) and (72) yields
(75)
𝔼
[
∫
0
T
‍
{
k
t
+
G
t
p
t
}
d
ξ
t
⋆
c
]
=
𝔼
[
∑
i
=
1
m
‍
∫
0
T
‍
{
k
t
i
+
G
t
i
p
t
}
d
ξ
t
⋆
c
i
]
=
0
.
The same computations applied to (73) and (74) imply
(76)
𝔼
[
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
ξ
t
⋆
]
=
0
.
Hence, from Definition1, we have the following inequality:
(77)
𝔼
[
∑
‍
0
<
t
≤
T
∫
0
T
{
k
t
+
G
t
p
t
}
d
(
ξ
-
ξ
⋆
)
t
c
+
∑
‍
0
<
t
≤
T
{
k
t
+
G
t
(
p
t
-
+
Δ
N
p
t
)
}
Δ
(
ξ
-
ξ
⋆
)
t
]
≤
0
.
The desired result follows from Theorem8.


## 4. Relation to Dynamic Programming

In this section, we come back to the control problem studied in the previous section. We recall a verification theorem, which is useful to compute optimal controls. Then we show that the adjoint process defined in Section3, as the unique solution to the BSDE (18), can be expressed as the gradient of the value function, which solves the HJB variational inequality.


### 4.1. A Verification Theorem

Letx
s
t
,
x be the solution of the controlled SDE (8), for s
≥
t, with initial value x
t
=
x. To put the problem in a Markovian framework, so that we can apply dynamic programming, we define the performance criterion
(78)
J
(
u
,
ξ
)
(
t
,
x
)
=
𝔼
[
∫
t
T
‍
f
(
s
,
x
s
,
u
s
)
d
s
+
∫
t
T
‍
k
s
d
ξ
s
+
g
(
x
T
)
∣
x
t
=
x
]
.Since our objective is to maximize this functional, the value function of the singular control problem becomes(79)
V
(
t
,
x
)
=
sup
(
u
,
ξ
)
∈
𝒰
J
(
u
,
ξ
)
(
t
,
x
)
.
If we do not apply any singular control, then the infinitesimal generator 𝒜
u, associated with (8), acting on functions φ, coincides on C
b
2
(
ℝ
n
;
ℝ
) with the parabolic integro-differential operator 𝒜
u given by
(80)
𝒜
u
φ
(
t
,
x
)
=
∑
i
=
1
n
‍
b
i
(
t
,
x
,
u
)
∂
φ
∂
x
i
(
t
,
x
)
+
1
2
∑
i
,
j
=
1
n
‍
a
i
j
(
t
,
x
,
u
)
∂
2
φ
∂
x
i
∂
x
j
(
t
,
x
)
+
∫
E
‍
{
∂
φ
∂
x
i
∑
i
=
1
n
‍
φ
(
t
,
x
+
γ
(
t
,
x
,
u
,
e
)
)
-
φ
(
t
,
x
)
-
∑
i
=
1
n
‍
γ
i
(
t
,
x
,
u
,
e
)
∂
φ
∂
x
i
(
t
,
x
)
}
ν
(
d
e
)
,
where a
i
j
=
∑
h
=
1
d
(
σ
i
h
σ
j
h
) denotes the generic term of the symmetric matrix σ
σ
T. The variational inequality associated to the singular control problem is
(81)
max
{
sup
u
H
1
(
t
,
x
,
(
W
,
∂
t
W
,
W
x
,
W
x
x
)
(
t
,
x
)
,
u
)
,
H
2
l
(
t
,
x
,
W
x
(
t
,
x
)
)
,
l
=
1
,
…
,
m
sup
u
H
1
}
=
0
,
for (
t
,
x
)
∈
[
0
,
T
]
×
O,
(82)
W
(
T
,
x
)
=
g
(
x
)
,
∀
x
∈
O
.
H
1 and H
2
l, for l
=
1
,
…
,
m, are given by
(83)
H
1
(
t
,
x
,
(
W
,
∂
t
W
,
W
x
,
W
x
x
)
(
t
,
x
)
,
u
)
=
∂
W
∂
t
(
t
,
x
)
+
𝒜
u
W
(
t
,
x
)
+
f
(
t
,
x
,
u
)
,
H
2
l
(
t
,
x
,
W
x
(
t
,
x
)
)
=
∑
i
=
1
n
‍
∂
W
∂
x
i
(
t
,
x
)
G
t
i
l
+
k
t
l
.We start with the definition of classical solutions of the variational inequality (81).Definition 10.
Let one consider a functionW
∈
C
1,2
(
[
0
,
T
]
×
O
), and define the nonintervention region by
(84)
C
(
W
)
=
{
∑
i
=
1
n
‍
(
t
,
x
)
∈
[
0
,
T
]
×
O
,
max
1
≤
l
≤
m
∑
i
=
1
n
‍
{
∂
W
∂
x
i
(
t
,
x
)
G
t
i
l
+
k
t
l
}
<
0
}
.
We say thatW is a classical solution of (81) if
(85)
∂
W
∂
t
(
t
,
x
)
+
sup
u
{
𝒜
u
W
(
t
,
x
)
+
f
(
t
,
x
,
u
)
}
=
0
,
∀
(
t
,
x
)
∈
C
(
W
)
,
(86)
∑
i
=
1
n
‍
∂
W
∂
x
i
(
t
,
x
)
G
t
i
l
+
k
t
l
≤
0
,
∀
(
t
,
x
)
∈
[
0
,
T
]
×
O
,
for
l
=
1
,
…
,
m
,
(87)
∂
W
∂
t
(
t
,
x
)
+
𝒜
u
W
(
t
,
x
)
+
f
(
t
,
x
,
u
)
≤
0
,
for
every
(
t
,
x
,
u
)
∈
[
0
,
T
]
×
O
×
A
1
.The following verification theorem is very useful to compute explicitly the value function and the optimal control, at least in the case where the value function is sufficiently smooth.Theorem 11.
LetW be a classical solution of (81) with the terminal condition (82), such that for some constants c
1
≥
1
,
c
2
∈
(
0
,
∞
)
,
|
W
(
t
,
x
)
|
≤
c
2
(
1
+
|
x
|
c
1
). Then, for all (
t
,
x
)
∈
[
0
,
T
]
×
O, and (
u
,
ξ
)
∈
𝒰
(88)
W
(
t
,
x
)
≥
J
(
u
,
ξ
)
(
t
,
x
)
.
Furthermore, if there exists(
u
⋆
,
ξ
⋆
)
∈
𝒰 such that with probability 1
(89)
(
t
,
x
t
⋆
)
∈
C
(
W
)
,
L
e
b
e
s
g
u
a
l
m
o
s
t
e
v
e
r
y
t
≤
T
,
(90)
u
t
⋆
∈
arg
max
u
{
𝒜
u
W
(
t
,
x
t
⋆
)
+
f
(
t
,
x
t
⋆
,
u
)
}
,
(91)
∑
l
=
1
m
{
∑
i
-
1
n
∂
W
∂
x
i
(
t
,
x
t
⋆
)
G
t
i
l
=
k
t
l
}
d
ξ
t
⋆
c
l
=
0
,
(92)
Δ
ξ
W
(
t
,
x
t
⋆
)
+
∑
l
=
1
m
k
t
l
Δ
ξ
t
⋆
l
=
0
,
for all jumping times t of ξ
t
⋆, then it follows that W
(
t
,
x
)
=
J
(
u
⋆
,
ξ
⋆
)
(
t
,
x
).Proof.
See [8, Theorem 5.2].In the following, we present an example on optimal harvesting from a geometric Brownian motion with jumps see, for example, [5, 8].Example 12.
Consider a population having a sizeX
=
{
X
t
:
t
≥
0
} which evolves according to the geometric Lévy process; that is
(93)
d
X
t
=
μ
X
t
d
t
+
σ
X
t
d
B
t
+
θ
X
t
-
∫
ℝ
+
‍
e
N
~
(
d
t
,
d
e
)
-
d
ξ
t
,
for
t
∈
[
0
,
T
]
,
X
0
-
=
x
>
0
.
Hereξ
t is the total number of individuals harvested up to time t. If we define the price per unit harvested at time t by k
(
t
)
=
e
-
ϰ
t and the utility rate obtained when the size of the population at t is X
t by e
-
ϰ
t
X
t
γ. Then the objective is to maximize the expected total time-discounted value of the harvested individuals starting with a population of size x; that is,
(94)
J
(
ξ
)
=
𝔼
[
∫
0
T
‍
e
-
ϰ
t
X
t
γ
d
t
+
∫
[
0
,
T
)
‍
e
-
ϰ
t
d
ξ
t
]
,
where T
≔
inf
{
t
≥
0
:
X
t
=
0
} is the time of complete depletion, γ
∈
(
0,1
) and μ
,
σ
,
ϰ
,
θ are positive constants with σ
2
/
2
+
θ
∫
ℝ
+
‍
e
ν
(
d
e
)
≤
μ
<
ϰ. The harvesting admissible strategy ξ
t is assumed to be nonnegative, nondecreasing continuous on the right, satisfying E
|
ξ
T
|
2
<
∞ with ξ
0
-
=
0, and such that X
t
>
0. We denote by Π
(
x
) the class of such strategies. For any ξ define
(95)
ϕ
(
t
,
x
)
=
sup
ξ
∈
Π
(
t
,
x
)
J
ξ
(
t
,
x
)
.
Note that the definition of Π
(
t
,
x
) is similar to Π
(
x
), except that the starting time is t, and the state at t is x.
If we guess the nonintervention regionC has the form C
=
{
(
t
,
x
)
:
0
<
x
<
b
} for some barrier point b
>
0, then (84) gets the form,
(96)
0
=
∂
Φ
∂
t
(
t
,
x
)
+
μ
x
∂
Φ
∂
x
(
t
,
x
)
+
1
2
σ
2
x
2
∂
2
Φ
∂
x
2
(
t
,
x
)
+
∫
ℝ
+
‍
{
Φ
(
t
,
x
(
1
+
θ
e
)
)
-
Φ
(
t
,
x
)
-
θ
x
e
∂
Φ
∂
x
(
t
,
x
)
}
ν
(
d
e
)
+
x
γ
exp
(
-
ϰ
t
)
,
for 0
<
x
<
b. We try a solution Φ of the form
(97)
Φ
(
t
,
x
)
=
Ψ
(
x
)
exp
(
-
ϰ
t
)
;
hence
(98)
𝒜
Φ
(
t
,
x
)
=
exp
(
-
ϰ
t
)
𝒜
0
Ψ
(
x
)
,
where Ψ is the fundamental solution of the ordinary integro-differential equation
(99)
-
ϰ
Ψ
(
x
)
+
μ
x
Ψ
′
(
x
)
+
1
2
σ
2
x
2
Ψ
′′
(
x
)
+
∫
ℝ
+
{
Ψ
(
x
(
1
+
θ
e
)
)
-
Ψ
(
x
)
-
θ
x
e
Ψ
′
(
x
)
}
‍
ν
(
d
e
)
+
x
γ
=
0
.
We notice that Ψ
(
x
)
=
A
x
ρ
+
K
x
γ, for some arbitrary constant A; we get
(100)
𝒜
Φ
(
t
,
x
)
=
x
γ
(
A
h
1
(
ρ
)
+
h
2
(
γ
)
)
exp
(
-
ϰ
t
)
,
where
(101)
h
1
(
ρ
)
=
1
2
σ
2
ρ
2
+
(
μ
-
1
2
σ
2
)
ρ
+
∫
ℝ
+
‍
{
(
1
+
θ
e
)
ρ
-
1
-
θ
e
ρ
}
ν
(
d
e
)
-
ϰ
,
h
2
(
γ
)
=
K
(
∫
ℝ
+
‍
1
2
σ
2
γ
2
+
(
μ
-
1
2
σ
2
)
γ
+
∫
ℝ
+
‍

{
(
1
+
θ
e
)
γ
-
1
-
θ
e
γ
}
ν
(
d
e
)
-
ϰ
)
+

1
.
Note thath
1
(
1
)
=
μ
-
ϰ
<
0 and lim
r
→
∞
h
1
(
ρ
)
=
∞; then there exists ρ
>
1 such that h
1
(
ρ
)
=
0. The constant K is given by
(102)
K
=
-
(
∫
ℝ
+
‍
1
2
σ
2
γ
2
+
(
μ
-
1
2
σ
2
)
γ
+
∫
ℝ
+
‍
{
(
1
+
θ
e
)
γ
-
1
-
θ
e
γ
}
ν
(
d
e
)
-
ϰ
)
-
1
.
OutsideC we require that Ψ
(
x
)
=
x
+
B, where B is a constant to be determined. This suggests that the value must be of the form
(103)
Φ
(
t
,
x
)
=
{
(
A
x
ρ
+
K
x
γ
)
exp
(
-
ϰ
t
)
for
0
<
x
<
b
,
(
x
+
B
)
exp
(
-
ϰ
t
)
for
x
≥
b
.
Assumingsmooth fit principle at point b, then the reflection threshold is
(104)
b
=
(
K
γ
(
1
-
γ
)
A
ρ
(
ρ
-
1
)
)
1
/
(
ρ
-
γ
)
,
where
(105)
A
=
1
-
K
γ
b
γ
-
1
ρ
b
ρ
-
1
,
B
=
A
b
ρ
+
K
b
γ
-
b
.
Sinceγ
<
1 and ρ
>
1, we deduce that b
>
0.
To construct the optimal controlξ
⋆, we consider the stochastic differential equation
(106)
d
X
t
⋆
=
μ
X
t
⋆
d
t
+
σ
X
t
⋆
d
B
t
+
∫
ℝ
+
‍
θ
X
t
⋆
e
N
~
(
d
t
,
d
e
)
-
d
ξ
t
⋆
,
(107)
X
t
⋆
≤
b
,
t
≥
0
,
(108)
1
{
X
t
⋆
<
b
}
d
ξ
t
⋆
c
=
0
,
(109)
1
{
X
t
-
⋆
+
Δ
N
X
t
⋆
≤
b
}
Δ
ξ
t
⋆
=
0
,
and if this is the case, then
(110)
Δ
ξ
t
⋆
=
min
{
l
>
0
:
X
t
-
⋆
+
Δ
N
X
t
⋆
-
l
=
b
}
.
Arguing as in [7], we can adapt Theorem  15 in [16] to obtain an identification of the optimal harvesting strategy as a local time of a reflected jump diffusion process. Then, the system (106)–(109) defines the so-called Skorokhod problem, whose solution is a pair (
X
t
⋆
,
ξ
t
⋆
), where X
t
⋆ is a jump diffusion process reflected at b.
The conditions (89)–(92) ensure the existence of an increasing process ξ
t
⋆ such that X
t
⋆ stays in C
¯ for all times t. If the initial size x
≤
b, then ξ
t
⋆ is nondecreasing and his continuous part ξ
t
⋆
c increases only when X
t
⋆
=
b so as to ensure that X
t
⋆
≤
b.
On the other hand, we only haveΔ
ξ
t
⋆
>
0 if the initial size x
>
b then ξ
0
-
⋆
=
x
-
b, or if X
t
⋆ jumps out of the nonintervention region by the random measure N; that is, X
t
-
⋆
+
Δ
N
X
t
⋆
>
b. In these cases we get Δ
ξ
t
⋆
>
0 immediately to bring X
t
⋆ to b.
It is easy to verify that, if(
X
⋆
,
ξ
⋆
) is a solution of the Skorokhod problem (106)–(109), then (
X
⋆
,
ξ
⋆
) is an optimal solution of the problem (93) and (94).
By the construction ofξ
⋆ and Φ, all the conditions of the verification Theorem 11 are satisfied. More precisely, the value function along the optimal state reads as
(111)
Φ
(
t
,
X
t
⋆
)
=
(
A
X
t
⋆
ρ
+
K
X
t
⋆
γ
)
exp
(
-
ϰ
t
)
,
for
all
t
∈
[
0
,
T
]
.


### 4.2. Link between the SMP and DPP

Compared with the stochastic maximum principle, one would expect that the solution(
p
,
q
,
r
(
·
)
) of BSDE (18) to correspond to the derivatives of the classical solution of the variational inequalities (81)-(82). This is given by the following theorem, which extends [3, Theorem 3.1] to control problems with a singular component and [2, Theorem 3.3] to diffusions with jumps.Theorem 13.
LetW be a classical solution of (81), with the terminal condition (82). Assume that W
∈
C
1,3
(
[
0
,
T
]
×
O
), with O
=
ℝ
n, and there exists (
u
⋆
,
ξ
⋆
)
∈
𝒰 such that the conditions (89)–(92) are satisfied. Then the solution of the BSDE (18) is given by
(112)
p
t
=
W
x
(
t
,
x
t
⋆
)
,
q
t
=
W
x
x
(
t
,
x
t
⋆
)
σ
(
t
,
x
t
⋆
,
u
t
⋆
)
,
r
t
(
·
)
=
W
x
(
t
,
x
t
⋆
+
γ
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
)
-
W
x
(
t
,
x
t
⋆
)
.Proof.
Throughout the proof, we will use the following abbreviations: fori
,
j
=
1
,
…
,
n, and h
=
1
,
…
,
d,
(113)
ϕ
1
(
t
)
=
ϕ
1
(
t
,
x
t
⋆
,
u
t
⋆
)
,
for
ϕ
1
=
b
i
,
σ
i
,
σ
i
h
,
σ
,
a
i
j
,
∂
b
i
∂
x
k
,
∂
b
∂
x
k
,
∂
a
i
j
∂
x
k
,
∂
σ
i
h
∂
x
k
,
∂
f
∂
x
k
,
ϕ
2
(
t
,
e
)
=
ϕ
2
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
,
for
ϕ
2
=
γ
,
γ
i
,
∂
γ
i
∂
x
k
,
∂
γ
∂
x
k
,
γ
-
(
t
,
e
)
=
γ
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
,
γ
-
i
(
t
,
e
)
=
γ
i
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
.
From Itô’s rule applied to the semimartingale(
∂
W
/
∂
x
k
)
(
t
,
x
t
⋆
), one has
(114)
∂
W
∂
x
k
(
τ
R
⋆
,
x
τ
R
⋆
⋆
)
=
∂
W
∂
x
k
(
t
,
x
t
⋆
)
+
∫
t
τ
R
⋆
‍
∂
2
W
∂
s
∂
x
k
(
s
,
x
s
⋆
)
d
s
+
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
-
⋆
)
d
x
s
⋆
i
+
1
2
∫
t
τ
R
⋆
‍
∑
i
,
j
=
1
n
‍
a
i
j
(
s
)
∂
3
W
∂
x
k
∂
x
i
∂
x
j
(
s
,
x
s
⋆
)
d
s
+
∫
t
τ
R
⋆
‍
∫
E
‍
{
∑
i
=
1
n
‍
∂
W
∂
x
k
(
s
,
x
s
-
⋆
+
γ
-
(
s
,
e
)
)
-
∂
W
∂
x
k
(
t
,
x
s
-
⋆
)
-
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
-
⋆
)
γ
-
i
(
s
,
e
)
}
N
(
d
s
,
d
e
)
+
∑
t
<
s
≤
τ
R
⋆
‍
{
∑
i
=
1
n
‍
∑
i
=
1
n
‍
Δ
ξ
∂
W
∂
x
k
(
s
,
x
s
⋆
)
-
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
-
⋆
)
Δ
ξ
x
s
⋆
i
}
,
where τ
⋆ is defined as in Theorem 11, and the sum is taken over all jumping times s of ξ
⋆. Note that
(115)
Δ
ξ
x
s
⋆
i
=
x
s
⋆
i
-
(
x
s
-
⋆
i
+
Δ
N
x
s
⋆
i
)
=
∑
l
=
1
m
‍
G
s
i
l
Δ
ξ
s
⋆
l
,
for
i
=
1
,
…
,
n
,
where Δ
ξ
s
⋆
l
=
ξ
s
⋆
l
-
ξ
s
-
⋆
l is a pure jump process. Then, we can rewrite (114) as follows:
(116)
∂
W
∂
x
k
(
τ
R
⋆
,
x
τ
R
⋆
⋆
)
=
∂
W
∂
x
k
(
t
,
x
t
⋆
)
+
∫
t
τ
R
⋆
‍
{
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
γ
i
(
s
,
e
)
)
ν
(
d
e
)
∂
2
W
∂
s
∂
x
k
(
s
,
x
s
⋆
)
+
∑
i
=
1
n
‍
b
i
(
s
)
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
+
1
2
∑
i
,
j
=
1
n
‍
a
i
j
(
s
)
∂
3
W
∂
x
k
∂
x
i
∂
x
j
(
s
,
x
s
⋆
)
+
∫
E
‍
(
∑
i
=
1
n
‍
∂
W
∂
x
k
(
s
,
x
s
⋆
+
γ
(
s
,
e
)
)
-
∂
W
∂
x
k
(
s
,
x
s
-
⋆
)
-
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
γ
i
(
s
,
e
)
)
ν
(
d
e
)
}
d
s
+
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
σ
i
(
s
)
d
B
s
+
∫
t
τ
R
⋆
‍
∫
E
‍
{
∂
W
∂
x
k
(
s
,
x
s
-
⋆
+
γ
-
(
s
,
e
)
)
-
∂
W
∂
x
k
(
s
,
x
s
-
⋆
)
}
N
~
(
d
s
,
d
e
)
+
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
∑
l
=
1
m
‍
G
s
i
l
d
ξ
s
⋆
c
l
+
∑
t
<
s
≤
τ
R
⋆
‍
Δ
ξ
∂
W
∂
x
k
(
s
,
x
s
⋆
)
.
Let ξ
s
⋆
c denotes the continuous part of ξ
s
⋆; that is, ξ
s
⋆
c
=
ξ
s
⋆
-
∑
t
<
s
≤
τ
R
⋆
Δ
ξ
s
⋆
l. Then, we can easily show that
(117)
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
G
s
i
l
d
ξ
s
⋆
c
l
=
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
G
s
i
l
1
{
(
s
,
x
s
⋆
)
∈
D
l
}
d
ξ
s
⋆
c
l
+
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
G
s
i
l
1
{
(
s
,
x
s
⋆
)
∈
C
l
}
d
ξ
s
⋆
c
l
.
For every(
t
,
x
)
∈
D
l, using (88) we have
(118)
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
t
,
x
)
G
t
i
l
=
∂
∂
x
k
{
∑
i
=
1
n
‍
∂
W
∂
x
i
(
t
,
x
)
G
t
i
l
+
k
s
l
}
=
0
,
for
l
=
1
,
…
,
m
.
This proves
(119)
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∑
l
=
1
m
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
G
s
i
l
1
{
(
s
,
x
s
⋆
)
∈
D
l
}
d
ξ
s
⋆
c
l
=
0
.
Furthermore, for every (
t
,
x
)
∈
C
l and l
=
1
,
…
,
m, we have ∑
i
=
1
n
(
∂
W
/
∂
x
k
∂
x
i
)
(
t
,
x
)
G
t
i
l
<
0.
But (91) implies that ∑
l
=
1
m
1
{
(
s
,
x
s
⋆
)
∈
C
l
}
d
ξ
s
⋆
c
l
=
0; thus
(120)
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∑
l
=
1
m
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
G
s
i
l
1
{
(
s
,
x
s
⋆
)
∈
C
l
}
d
ξ
s
⋆
c
l
=
0
.
The mean value theorem yields(121)
Δ
ξ
∂
W
∂
x
k
(
s
,
x
s
⋆
)
=
(
∂
W
∂
x
k
)
x
(
s
,
y
(
s
)
)
Δ
ξ
x
s
⋆
,
where y
(
s
) is some point on the straight line between x
s
-
⋆
+
Δ
N
x
s
⋆ and x
s
⋆, and (
∂
W
/
∂
x
k
)
x represents the gradient matrix of ∂
W
/
∂
x
k. To prove that the right-hand side of the above equality vanishes, it is enough to check that if Δ
ξ
s
⋆
l
>
0 then ∑
i
=
1
n
(
∂
2
W
/
∂
x
k
∂
x
i
)
(
s
,
y
(
s
)
)
G
s
i
l
=
0, for l
=
1
,
…
,
m. It is clear by (92) that
(122)
0
=
Δ
ξ
W
(
s
,
x
s
⋆
)
+
∑
l
=
1
m
‍
k
s
l
Δ
ξ
s
⋆
l
=
∑
l
=
1
m
‍
{
∑
i
=
1
n
‍
∂
W
∂
x
i
(
s
,
y
(
s
)
)
G
s
i
l
+
k
s
l
}
Δ
ξ
s
⋆
l
.
SinceΔ
ξ
s
⋆
l
>
0, then (
s
,
y
(
s
)
)
∈
D
l, for l
=
1
,
…
,
m. According to (88), we obtain
(123)
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
y
(
s
)
)
G
s
i
l
=
∂
∂
x
k
{
∑
i
=
1
n
‍
∂
W
∂
x
i
(
s
,
y
(
s
)
)
G
s
i
l
+
k
s
l
}
=
0
.
This shows that
(124)
∑
t
<
s
≤
τ
R
⋆
‍
Δ
ξ
∂
W
∂
x
k
(
s
,
x
s
⋆
)
=
0
.
On the other hand, define(125)
A
(
t
,
x
,
u
)
=
∂
W
∂
t
(
t
,
x
)
+
∑
i
=
1
n
‍
b
i
(
t
,
x
,
u
)
∂
W
∂
x
i
(
t
,
x
)
+
1
2
∑
i
,
j
=
1
n
‍
a
i
j
(
t
,
x
,
u
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
)
+
f
(
t
,
x
,
u
)
+
∫
E
{
∑
i
=
1
n
‍
W
(
t
,
x
+
γ
(
t
,
x
,
u
,
e
)
)
-
W
(
t
,
x
)
‍
-
∑
i
=
1
n
‍
γ
i
(
t
,
x
,
u
,
e
)
∂
W
∂
x
i
(
t
,
x
)
}
ν
(
d
e
)
.
If we differentiateA
(
t
,
x
,
u
) with respect to x
k and evaluate the result at (
x
,
u
)
=
(
x
t
⋆
,
u
t
⋆
), we deduce easily from (84), (89), and (90) that
(126)
∂
2
W
∂
t
∂
x
k
(
t
,
x
t
⋆
)
+
∑
i
=
1
n
‍
b
i
(
t
)
∂
2
W
∂
x
k
∂
x
i
(
t
,
x
t
⋆
)
+
1
2
∑
i
,
j
=
1
n
‍
a
i
j
(
t
)
∂
3
W
∂
x
k
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
+
∫
E
{
∑
i
=
1
n
‍
∂
W
∂
x
k
(
t
,
x
t
⋆
+
γ
(
t
,
e
)
)
-
∂
W
∂
x
k
(
t
,
x
t
⋆
)
‍
-
∑
i
=
1
n
‍
γ
i
(
s
,
e
)
∂
2
W
∂
x
k
∂
x
i
(
t
,
x
t
⋆
)
}
ν
(
d
e
)
=
-
∑
i
=
1
n
‍
∂
b
i
∂
x
k
(
t
)
∂
W
∂
x
i
(
t
,
x
t
⋆
)
-
1
2
∑
i
,
j
=
1
n
‍
∂
a
i
j
∂
x
k
(
t
,
x
t
⋆
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
-
∂
f
∂
x
k
(
t
,
x
t
⋆
,
u
t
⋆
)
-
∫
E
‍
∑
i
=
1
n
‍
∂
γ
i
∂
x
k
(
t
,
e
)
×
{
∂
W
∂
x
i
(
t
,
x
t
⋆
+
γ
(
t
,
e
)
)
-
∂
W
∂
x
i
(
t
,
x
t
⋆
)
}
ν
(
d
e
)
.
Finally, substituting (119), (120), (124), and (126) into (116) yields
(127)
d
(
∂
W
∂
x
k
(
t
,
x
t
⋆
)
)
=
-
{
∑
i
=
1
n
‍
∂
b
i
∂
x
k
(
t
)
∂
W
∂
x
i
(
t
,
x
t
⋆
)
+
1
2
∑
i
,
j
=
1
n
‍
∂
a
i
j
∂
x
k
(
t
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
+
∂
f
∂
x
k
(
t
)
+
∫
E
‍
∑
i
=
1
n
‍
∂
γ
i
∂
x
k
(
t
,
e
)
×

(
∂
W
∂
x
i
(
t
,
x
t
⋆
+
γ
(
t
,
e
)
)
-
∂
W
∂
x
i
(
t
,
x
t
⋆
)

)

ν
(
d
e
)

∑
i
=
1
n
‍
}

d
t
+
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
t
,
x
t
⋆
)
σ
i
(
t
)
d
B
t
+
∫
E
‍
{
∂
W
∂
x
k
(
t
,
x
t
-
⋆
+
γ
-
(
t
,
e
)
)

-
∂
W
∂
x
k
(
t
,
x
t
-
⋆
)
}
N
~
(
d
t
,
d
e
)

.
The continuity of∂
W
/
∂
x
k leads to
(128)
lim
R
→
∞
∂
W
∂
x
k
(
τ
R
⋆
,
x
τ
R
⋆
⋆
)
=
∂
W
∂
x
k
(
T
,
x
T
⋆
)
=
∂
g
∂
x
k
(
x
T
⋆
)
,
for
each
k
=
1
,
…
,
n
.
Clearly,(129)
1
2
∑
i
,
j
=
1
n
‍
∂
a
i
j
∂
x
k
(
t
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
=
1
2
∑
i
,
j
=
1
n
‍
∂
∂
x
k
(
∑
h
=
1
d
‍
σ
i
h
(
t
)
σ
j
h
(
t
)
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
=
∑
j
=
1
n
‍
∑
h
=
1
d
‍
(
∑
i
=
1
n
‍
σ
i
h
(
t
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
)
∂
σ
i
h
∂
x
k
(
t
)
.
Now, from (17) we have
(130)
∂
H
∂
x
k
(
t
,
x
,
u
,
p
,
q
,
r
(
·
)
)
=
∑
i
=
1
n
‍
∂
b
i
∂
x
k
(
t
,
x
,
u
)
p
i
+
∑
h
=
1
d
‍
∑
i
=
1
n
‍
∂
σ
i
h
∂
x
k
(
t
,
x
,
u
)
q
i
h
+
∂
f
∂
x
k
(
t
,
x
,
u
)
+
∫
E
‍
∑
i
=
1
n
‍
∂
γ
i
∂
x
k
(
t
,
x
,
u
,
e
)
r
i
(
e
)
ν
(
d
e
)
.
Thekth coordinate p
t
k of the adjoint process p
t satisfies
(131)
d
p
t
k
=
-
∂
H
∂
x
k
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
d
t
+
q
t
k
d
B
t
+
∫
E
‍
r
t
-
k
(
e
)
N
~
(
d
t
,
d
e
)
,
for
t
∈
[
0
,
T
]
,
p
T
k
=
∂
g
∂
x
k
(
x
T
⋆
)
,
with q
t
k
d
B
t
=
∑
h
=
1
d
‍
q
t
k
h
d
B
t
h. Hence, the uniqueness of the solution of (131) and relation (128) allows us to get
(132)
p
t
k
=
∂
W
∂
x
k
(
t
,
x
t
⋆
)
,
q
t
k
h
=
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
t
,
x
t
⋆
)
σ
i
h
(
t
)
,
r
t
-
k
(
·
)
=
∂
W
∂
x
k
(
t
,
x
t
-
⋆
+
γ
(
t
,
e
)
)
-
∂
W
∂
x
k
(
t
,
x
t
-
⋆
)
,
where q
t
k
h is the generic element of the matrix q
t and x
t
⋆ is the optimal solution of the controlled SDE (8).Example 14.
We return to the same example in the previous section.
Now, we illustrate a verification result for the maximum principle. We suppose thatT is a fixed time. In this case the Hamiltonian gets the form
(133)
H
(
t
,
X
t
,
p
t
,
q
t
,
r
t
(
·
)
)
=
μ
X
t
p
t
+
σ
X
t
q
t
+
X
t
γ
(
-
ϰ
t
)
+
θ
X
t
-
∫
ℝ
+
e
r
t
(
e
)
ν
(
d
e
)
.
Letξ
⋆ be a candidate for an optimal control, and let X
⋆ be the corresponding state process with corresponding solution (
p
⋆
,
q
⋆
,
r
⋆
(
·
)
) of the following adjoint equation, for all t
∈
[
0
,
T
)
(134)
d
p
t
⋆
=
-
(
μ
p
t
⋆
+
σ
q
t
⋆
+
θ
∫
ℝ
+
‍
e
r
t
⋆
(
e
)
ν
(
d
e
)
+
γ
X
t
⋆
γ
-
1
exp
(
-
ϰ
s
)
∫
ℝ
+
‍
)
d
t
+
q
t
⋆
d
B
t
+
∫
ℝ
+
‍
r
t
-
⋆
(
e
)
N
~
(
d
t
,
d
e
)
,
(135)
-
p
t
⋆
+
exp
(
-
ϰ
t
)
≤
0
,
∀
t
,
(136)
1
{
-
p
t
⋆
+
exp
(
-
ϰ
t
)
<
0
}
d
ξ
t
⋆
c
=
0
,
(137)
-
(
p
t
-
⋆
+
Δ
N
p
t
⋆
)
+
exp
(
-
ϰ
t
)
≤
0
,
(138)
1
{
-
(
p
t
-
⋆
+
Δ
N
p
t
⋆
)
+
exp
(
-
ϰ
t
)
<
0
}
Δ
ξ
t
⋆
=
0
.
Sinceg
=
0, we assume the transversality condition
(139)
𝔼
[
p
T
⋆
(
X
T
⋆
-
X
T
)
]
≤
0
.
We remark thatΔ
ξ
p
t
⋆
=
0; then p
t
-
⋆
+
Δ
N
p
t
⋆
=
p
t
⋆, and the condition (138) reduces to
(140)
1
{
-
p
t
⋆
+
exp
(
-
ϰ
t
)
<
0
}
Δ
ξ
t
⋆
=
0
.
We use the relation between the value function and the solution(
p
⋆
,
q
⋆
,
r
⋆
(
e
)
) of the adjoint equation along the optimal state. We prove that the solution of the adjoint equation is represented as
(141)
p
t
⋆
=
(
A
ρ
X
t
⋆
ρ
-
1
+
K
γ
X
t
⋆
γ
-
1
)
exp
(
-
ϰ
t
)
,
q
t
⋆
=
σ
(
A
ρ
(
ρ
-
1
)
X
t
⋆
ρ
-
1
+
K
γ
(
γ
-
1
)
X
t
⋆
γ
-
1
)
exp
(
-
ϰ
t
)
,
r
t
⋆
(
e
)
=
(
A
ρ
(
(
1
+
θ
e
)
ρ
-
1
-
1
)
X
t
⋆
ρ
-
1
+
K
γ
(
(
1
+
θ
e
)
γ
-
1
-
1
)
X
t
⋆
γ
-
1
)
exp
(
-
ϰ
t
)
for all t
∈
[
0
,
T
).
To see this, we differentiate the process(
A
ρ
X
t
⋆
ρ
-
1
+
K
γ
X
t
⋆
γ
-
1
)
exp
(
-
ϰ
t
) using Itô’s rule for semimartingales and by using the same procedure as in the proof of Theorem 13. Then, the conclusion follows readily from the verification of (135), (136), and (139). First, an explicit formula for X
t is given in [4] by
(142)
X
t
=
e
μ
t
M
t
{
∑
0
<
s
≤
t
‍
x
-
(
∑
0
<
s
≤
t
‍
∫
[
0
,
t
)
‍
M
s
-
1
exp
(
-
μ
s
)
d
ξ
s
+
∑
0
<
s
≤
t
‍
M
s
-
1
β
s
exp
(
-
μ
s
)
Δ
ξ
s
)
}
,
for
t
∈
[
0
,
T
]
,
where β
t
=
(
∫
ℝ
+
‍
θ
e
N
(
{
t
}
,
d
e
)
)
(
1
+
∫
ℝ
+
‍
θ
e
N
(
{
t
}
,
d
e
)
)
-
1, and M
t is a geometric Lévy process defined by
(143)
M
t
=
exp
{
(
-
1
2
σ
2
+
∫
ℝ
+
‍
{
ln
(
1
+
θ
e
)
-
θ
e
}
ν
(
d
e
)
)
t
+
σ
B
t
+
∫
0
t
‍
∫
ℝ
+
‍
ln
(
1
+
θ
e
)
N
~
(
d
t
,
d
e
)
(
-
1
2
σ
2
+
∫
ℝ
+
‍
{
ln
(
1
+
θ
e
)
-
θ
e
}
ν
(
d
e
)
)
}
.
From the representation (142) and by the fact that X
T
∧
t
⋆
≤
x
M
T
∧
t
exp
(
μ
(
T
∧
t
)
), we get
(144)
1
-
X
T
∧
t
X
T
∧
t
⋆
≤
1
x
(
∑
0
<
s
≤
t
‍
∫
[
0
,
T
∧
t
)
‍
M
s
-
1
exp
(
-
μ
s
)
d
ξ
s
+
∑
0
<
s
≤
t
‍
M
s
-
1
β
s
exp
(
-
μ
s
)
Δ
ξ
s
)
<
∞
,
hence
(145)
𝔼
[
p
T
∧
t
⋆
(
X
T
∧
t
⋆
-
X
T
∧
t
)
]
≤
𝔼
[
(
(
A
ρ
X
T
∧
t
⋆
ρ
+
K
γ
X
T
∧
t
⋆
γ
)
exp
(
-
ϰ
(
T
∧
t
)
)
)
2
]
1
/
2
×
𝔼
[
∑
0
<
s
≤
T
∧
t
‍
M
s
-
1
β
s
exp
(
-
μ
s
)
Δ
ξ
s
)
2
(
1
x
∫
[
0
,
T
∧
t
)
‍
M
s
-
1
exp
(
-
μ
s
)
d
ξ
s
+
∑
0
<
s
≤
T
∧
t
‍
M
s
-
1
β
s
exp
(
-
μ
s
)
Δ
ξ
s
)
2
]
1
/
2
.
By the dominated convergence theorem, we obtain (139) by sending t to infinity in (145).
A simple computation shows that the conditions (135)–(138) are consequences of (107)–(109). This shows in particular that the pair (
X
t
⋆
,
ξ
t
⋆
) satisfies the optimality sufficient conditions and then it is optimal. This completes the proof of the following result.Theorem 15.
One supposes thatσ
2
/
2
+
θ
∫
ℝ
+
‍
e
ν
(
d
e
)
≤
μ
<
ϰ, and e
≥
0
d
ν
-
a
.
e. If the strategy ξ
⋆ is chosen such that the corresponding solution of the adjoint process is given by (141), then this choice is optimal.Remark 16.
In this example, it is shown in particular that the relationship between the stochastic maximum principle and dynamic programming could be very useful to solve explicitly constrained backward stochastic differential equations with transversality condition.


## 4.1. A Verification Theorem

Letx
s
t
,
x be the solution of the controlled SDE (8), for s
≥
t, with initial value x
t
=
x. To put the problem in a Markovian framework, so that we can apply dynamic programming, we define the performance criterion
(78)
J
(
u
,
ξ
)
(
t
,
x
)
=
𝔼
[
∫
t
T
‍
f
(
s
,
x
s
,
u
s
)
d
s
+
∫
t
T
‍
k
s
d
ξ
s
+
g
(
x
T
)
∣
x
t
=
x
]
.Since our objective is to maximize this functional, the value function of the singular control problem becomes(79)
V
(
t
,
x
)
=
sup
(
u
,
ξ
)
∈
𝒰
J
(
u
,
ξ
)
(
t
,
x
)
.
If we do not apply any singular control, then the infinitesimal generator 𝒜
u, associated with (8), acting on functions φ, coincides on C
b
2
(
ℝ
n
;
ℝ
) with the parabolic integro-differential operator 𝒜
u given by
(80)
𝒜
u
φ
(
t
,
x
)
=
∑
i
=
1
n
‍
b
i
(
t
,
x
,
u
)
∂
φ
∂
x
i
(
t
,
x
)
+
1
2
∑
i
,
j
=
1
n
‍
a
i
j
(
t
,
x
,
u
)
∂
2
φ
∂
x
i
∂
x
j
(
t
,
x
)
+
∫
E
‍
{
∂
φ
∂
x
i
∑
i
=
1
n
‍
φ
(
t
,
x
+
γ
(
t
,
x
,
u
,
e
)
)
-
φ
(
t
,
x
)
-
∑
i
=
1
n
‍
γ
i
(
t
,
x
,
u
,
e
)
∂
φ
∂
x
i
(
t
,
x
)
}
ν
(
d
e
)
,
where a
i
j
=
∑
h
=
1
d
(
σ
i
h
σ
j
h
) denotes the generic term of the symmetric matrix σ
σ
T. The variational inequality associated to the singular control problem is
(81)
max
{
sup
u
H
1
(
t
,
x
,
(
W
,
∂
t
W
,
W
x
,
W
x
x
)
(
t
,
x
)
,
u
)
,
H
2
l
(
t
,
x
,
W
x
(
t
,
x
)
)
,
l
=
1
,
…
,
m
sup
u
H
1
}
=
0
,
for (
t
,
x
)
∈
[
0
,
T
]
×
O,
(82)
W
(
T
,
x
)
=
g
(
x
)
,
∀
x
∈
O
.
H
1 and H
2
l, for l
=
1
,
…
,
m, are given by
(83)
H
1
(
t
,
x
,
(
W
,
∂
t
W
,
W
x
,
W
x
x
)
(
t
,
x
)
,
u
)
=
∂
W
∂
t
(
t
,
x
)
+
𝒜
u
W
(
t
,
x
)
+
f
(
t
,
x
,
u
)
,
H
2
l
(
t
,
x
,
W
x
(
t
,
x
)
)
=
∑
i
=
1
n
‍
∂
W
∂
x
i
(
t
,
x
)
G
t
i
l
+
k
t
l
.We start with the definition of classical solutions of the variational inequality (81).Definition 10.
Let one consider a functionW
∈
C
1,2
(
[
0
,
T
]
×
O
), and define the nonintervention region by
(84)
C
(
W
)
=
{
∑
i
=
1
n
‍
(
t
,
x
)
∈
[
0
,
T
]
×
O
,
max
1
≤
l
≤
m
∑
i
=
1
n
‍
{
∂
W
∂
x
i
(
t
,
x
)
G
t
i
l
+
k
t
l
}
<
0
}
.
We say thatW is a classical solution of (81) if
(85)
∂
W
∂
t
(
t
,
x
)
+
sup
u
{
𝒜
u
W
(
t
,
x
)
+
f
(
t
,
x
,
u
)
}
=
0
,
∀
(
t
,
x
)
∈
C
(
W
)
,
(86)
∑
i
=
1
n
‍
∂
W
∂
x
i
(
t
,
x
)
G
t
i
l
+
k
t
l
≤
0
,
∀
(
t
,
x
)
∈
[
0
,
T
]
×
O
,
for
l
=
1
,
…
,
m
,
(87)
∂
W
∂
t
(
t
,
x
)
+
𝒜
u
W
(
t
,
x
)
+
f
(
t
,
x
,
u
)
≤
0
,
for
every
(
t
,
x
,
u
)
∈
[
0
,
T
]
×
O
×
A
1
.The following verification theorem is very useful to compute explicitly the value function and the optimal control, at least in the case where the value function is sufficiently smooth.Theorem 11.
LetW be a classical solution of (81) with the terminal condition (82), such that for some constants c
1
≥
1
,
c
2
∈
(
0
,
∞
)
,
|
W
(
t
,
x
)
|
≤
c
2
(
1
+
|
x
|
c
1
). Then, for all (
t
,
x
)
∈
[
0
,
T
]
×
O, and (
u
,
ξ
)
∈
𝒰
(88)
W
(
t
,
x
)
≥
J
(
u
,
ξ
)
(
t
,
x
)
.
Furthermore, if there exists(
u
⋆
,
ξ
⋆
)
∈
𝒰 such that with probability 1
(89)
(
t
,
x
t
⋆
)
∈
C
(
W
)
,
L
e
b
e
s
g
u
a
l
m
o
s
t
e
v
e
r
y
t
≤
T
,
(90)
u
t
⋆
∈
arg
max
u
{
𝒜
u
W
(
t
,
x
t
⋆
)
+
f
(
t
,
x
t
⋆
,
u
)
}
,
(91)
∑
l
=
1
m
{
∑
i
-
1
n
∂
W
∂
x
i
(
t
,
x
t
⋆
)
G
t
i
l
=
k
t
l
}
d
ξ
t
⋆
c
l
=
0
,
(92)
Δ
ξ
W
(
t
,
x
t
⋆
)
+
∑
l
=
1
m
k
t
l
Δ
ξ
t
⋆
l
=
0
,
for all jumping times t of ξ
t
⋆, then it follows that W
(
t
,
x
)
=
J
(
u
⋆
,
ξ
⋆
)
(
t
,
x
).Proof.
See [8, Theorem 5.2].In the following, we present an example on optimal harvesting from a geometric Brownian motion with jumps see, for example, [5, 8].Example 12.
Consider a population having a sizeX
=
{
X
t
:
t
≥
0
} which evolves according to the geometric Lévy process; that is
(93)
d
X
t
=
μ
X
t
d
t
+
σ
X
t
d
B
t
+
θ
X
t
-
∫
ℝ
+
‍
e
N
~
(
d
t
,
d
e
)
-
d
ξ
t
,
for
t
∈
[
0
,
T
]
,
X
0
-
=
x
>
0
.
Hereξ
t is the total number of individuals harvested up to time t. If we define the price per unit harvested at time t by k
(
t
)
=
e
-
ϰ
t and the utility rate obtained when the size of the population at t is X
t by e
-
ϰ
t
X
t
γ. Then the objective is to maximize the expected total time-discounted value of the harvested individuals starting with a population of size x; that is,
(94)
J
(
ξ
)
=
𝔼
[
∫
0
T
‍
e
-
ϰ
t
X
t
γ
d
t
+
∫
[
0
,
T
)
‍
e
-
ϰ
t
d
ξ
t
]
,
where T
≔
inf
{
t
≥
0
:
X
t
=
0
} is the time of complete depletion, γ
∈
(
0,1
) and μ
,
σ
,
ϰ
,
θ are positive constants with σ
2
/
2
+
θ
∫
ℝ
+
‍
e
ν
(
d
e
)
≤
μ
<
ϰ. The harvesting admissible strategy ξ
t is assumed to be nonnegative, nondecreasing continuous on the right, satisfying E
|
ξ
T
|
2
<
∞ with ξ
0
-
=
0, and such that X
t
>
0. We denote by Π
(
x
) the class of such strategies. For any ξ define
(95)
ϕ
(
t
,
x
)
=
sup
ξ
∈
Π
(
t
,
x
)
J
ξ
(
t
,
x
)
.
Note that the definition of Π
(
t
,
x
) is similar to Π
(
x
), except that the starting time is t, and the state at t is x.
If we guess the nonintervention regionC has the form C
=
{
(
t
,
x
)
:
0
<
x
<
b
} for some barrier point b
>
0, then (84) gets the form,
(96)
0
=
∂
Φ
∂
t
(
t
,
x
)
+
μ
x
∂
Φ
∂
x
(
t
,
x
)
+
1
2
σ
2
x
2
∂
2
Φ
∂
x
2
(
t
,
x
)
+
∫
ℝ
+
‍
{
Φ
(
t
,
x
(
1
+
θ
e
)
)
-
Φ
(
t
,
x
)
-
θ
x
e
∂
Φ
∂
x
(
t
,
x
)
}
ν
(
d
e
)
+
x
γ
exp
(
-
ϰ
t
)
,
for 0
<
x
<
b. We try a solution Φ of the form
(97)
Φ
(
t
,
x
)
=
Ψ
(
x
)
exp
(
-
ϰ
t
)
;
hence
(98)
𝒜
Φ
(
t
,
x
)
=
exp
(
-
ϰ
t
)
𝒜
0
Ψ
(
x
)
,
where Ψ is the fundamental solution of the ordinary integro-differential equation
(99)
-
ϰ
Ψ
(
x
)
+
μ
x
Ψ
′
(
x
)
+
1
2
σ
2
x
2
Ψ
′′
(
x
)
+
∫
ℝ
+
{
Ψ
(
x
(
1
+
θ
e
)
)
-
Ψ
(
x
)
-
θ
x
e
Ψ
′
(
x
)
}
‍
ν
(
d
e
)
+
x
γ
=
0
.
We notice that Ψ
(
x
)
=
A
x
ρ
+
K
x
γ, for some arbitrary constant A; we get
(100)
𝒜
Φ
(
t
,
x
)
=
x
γ
(
A
h
1
(
ρ
)
+
h
2
(
γ
)
)
exp
(
-
ϰ
t
)
,
where
(101)
h
1
(
ρ
)
=
1
2
σ
2
ρ
2
+
(
μ
-
1
2
σ
2
)
ρ
+
∫
ℝ
+
‍
{
(
1
+
θ
e
)
ρ
-
1
-
θ
e
ρ
}
ν
(
d
e
)
-
ϰ
,
h
2
(
γ
)
=
K
(
∫
ℝ
+
‍
1
2
σ
2
γ
2
+
(
μ
-
1
2
σ
2
)
γ
+
∫
ℝ
+
‍

{
(
1
+
θ
e
)
γ
-
1
-
θ
e
γ
}
ν
(
d
e
)
-
ϰ
)
+

1
.
Note thath
1
(
1
)
=
μ
-
ϰ
<
0 and lim
r
→
∞
h
1
(
ρ
)
=
∞; then there exists ρ
>
1 such that h
1
(
ρ
)
=
0. The constant K is given by
(102)
K
=
-
(
∫
ℝ
+
‍
1
2
σ
2
γ
2
+
(
μ
-
1
2
σ
2
)
γ
+
∫
ℝ
+
‍
{
(
1
+
θ
e
)
γ
-
1
-
θ
e
γ
}
ν
(
d
e
)
-
ϰ
)
-
1
.
OutsideC we require that Ψ
(
x
)
=
x
+
B, where B is a constant to be determined. This suggests that the value must be of the form
(103)
Φ
(
t
,
x
)
=
{
(
A
x
ρ
+
K
x
γ
)
exp
(
-
ϰ
t
)
for
0
<
x
<
b
,
(
x
+
B
)
exp
(
-
ϰ
t
)
for
x
≥
b
.
Assumingsmooth fit principle at point b, then the reflection threshold is
(104)
b
=
(
K
γ
(
1
-
γ
)
A
ρ
(
ρ
-
1
)
)
1
/
(
ρ
-
γ
)
,
where
(105)
A
=
1
-
K
γ
b
γ
-
1
ρ
b
ρ
-
1
,
B
=
A
b
ρ
+
K
b
γ
-
b
.
Sinceγ
<
1 and ρ
>
1, we deduce that b
>
0.
To construct the optimal controlξ
⋆, we consider the stochastic differential equation
(106)
d
X
t
⋆
=
μ
X
t
⋆
d
t
+
σ
X
t
⋆
d
B
t
+
∫
ℝ
+
‍
θ
X
t
⋆
e
N
~
(
d
t
,
d
e
)
-
d
ξ
t
⋆
,
(107)
X
t
⋆
≤
b
,
t
≥
0
,
(108)
1
{
X
t
⋆
<
b
}
d
ξ
t
⋆
c
=
0
,
(109)
1
{
X
t
-
⋆
+
Δ
N
X
t
⋆
≤
b
}
Δ
ξ
t
⋆
=
0
,
and if this is the case, then
(110)
Δ
ξ
t
⋆
=
min
{
l
>
0
:
X
t
-
⋆
+
Δ
N
X
t
⋆
-
l
=
b
}
.
Arguing as in [7], we can adapt Theorem  15 in [16] to obtain an identification of the optimal harvesting strategy as a local time of a reflected jump diffusion process. Then, the system (106)–(109) defines the so-called Skorokhod problem, whose solution is a pair (
X
t
⋆
,
ξ
t
⋆
), where X
t
⋆ is a jump diffusion process reflected at b.
The conditions (89)–(92) ensure the existence of an increasing process ξ
t
⋆ such that X
t
⋆ stays in C
¯ for all times t. If the initial size x
≤
b, then ξ
t
⋆ is nondecreasing and his continuous part ξ
t
⋆
c increases only when X
t
⋆
=
b so as to ensure that X
t
⋆
≤
b.
On the other hand, we only haveΔ
ξ
t
⋆
>
0 if the initial size x
>
b then ξ
0
-
⋆
=
x
-
b, or if X
t
⋆ jumps out of the nonintervention region by the random measure N; that is, X
t
-
⋆
+
Δ
N
X
t
⋆
>
b. In these cases we get Δ
ξ
t
⋆
>
0 immediately to bring X
t
⋆ to b.
It is easy to verify that, if(
X
⋆
,
ξ
⋆
) is a solution of the Skorokhod problem (106)–(109), then (
X
⋆
,
ξ
⋆
) is an optimal solution of the problem (93) and (94).
By the construction ofξ
⋆ and Φ, all the conditions of the verification Theorem 11 are satisfied. More precisely, the value function along the optimal state reads as
(111)
Φ
(
t
,
X
t
⋆
)
=
(
A
X
t
⋆
ρ
+
K
X
t
⋆
γ
)
exp
(
-
ϰ
t
)
,
for
all
t
∈
[
0
,
T
]
.


## 4.2. Link between the SMP and DPP

Compared with the stochastic maximum principle, one would expect that the solution(
p
,
q
,
r
(
·
)
) of BSDE (18) to correspond to the derivatives of the classical solution of the variational inequalities (81)-(82). This is given by the following theorem, which extends [3, Theorem 3.1] to control problems with a singular component and [2, Theorem 3.3] to diffusions with jumps.Theorem 13.
LetW be a classical solution of (81), with the terminal condition (82). Assume that W
∈
C
1,3
(
[
0
,
T
]
×
O
), with O
=
ℝ
n, and there exists (
u
⋆
,
ξ
⋆
)
∈
𝒰 such that the conditions (89)–(92) are satisfied. Then the solution of the BSDE (18) is given by
(112)
p
t
=
W
x
(
t
,
x
t
⋆
)
,
q
t
=
W
x
x
(
t
,
x
t
⋆
)
σ
(
t
,
x
t
⋆
,
u
t
⋆
)
,
r
t
(
·
)
=
W
x
(
t
,
x
t
⋆
+
γ
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
)
-
W
x
(
t
,
x
t
⋆
)
.Proof.
Throughout the proof, we will use the following abbreviations: fori
,
j
=
1
,
…
,
n, and h
=
1
,
…
,
d,
(113)
ϕ
1
(
t
)
=
ϕ
1
(
t
,
x
t
⋆
,
u
t
⋆
)
,
for
ϕ
1
=
b
i
,
σ
i
,
σ
i
h
,
σ
,
a
i
j
,
∂
b
i
∂
x
k
,
∂
b
∂
x
k
,
∂
a
i
j
∂
x
k
,
∂
σ
i
h
∂
x
k
,
∂
f
∂
x
k
,
ϕ
2
(
t
,
e
)
=
ϕ
2
(
t
,
x
t
⋆
,
u
t
⋆
,
e
)
,
for
ϕ
2
=
γ
,
γ
i
,
∂
γ
i
∂
x
k
,
∂
γ
∂
x
k
,
γ
-
(
t
,
e
)
=
γ
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
,
γ
-
i
(
t
,
e
)
=
γ
i
(
t
,
x
t
-
⋆
,
u
t
⋆
,
e
)
.
From Itô’s rule applied to the semimartingale(
∂
W
/
∂
x
k
)
(
t
,
x
t
⋆
), one has
(114)
∂
W
∂
x
k
(
τ
R
⋆
,
x
τ
R
⋆
⋆
)
=
∂
W
∂
x
k
(
t
,
x
t
⋆
)
+
∫
t
τ
R
⋆
‍
∂
2
W
∂
s
∂
x
k
(
s
,
x
s
⋆
)
d
s
+
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
-
⋆
)
d
x
s
⋆
i
+
1
2
∫
t
τ
R
⋆
‍
∑
i
,
j
=
1
n
‍
a
i
j
(
s
)
∂
3
W
∂
x
k
∂
x
i
∂
x
j
(
s
,
x
s
⋆
)
d
s
+
∫
t
τ
R
⋆
‍
∫
E
‍
{
∑
i
=
1
n
‍
∂
W
∂
x
k
(
s
,
x
s
-
⋆
+
γ
-
(
s
,
e
)
)
-
∂
W
∂
x
k
(
t
,
x
s
-
⋆
)
-
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
-
⋆
)
γ
-
i
(
s
,
e
)
}
N
(
d
s
,
d
e
)
+
∑
t
<
s
≤
τ
R
⋆
‍
{
∑
i
=
1
n
‍
∑
i
=
1
n
‍
Δ
ξ
∂
W
∂
x
k
(
s
,
x
s
⋆
)
-
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
-
⋆
)
Δ
ξ
x
s
⋆
i
}
,
where τ
⋆ is defined as in Theorem 11, and the sum is taken over all jumping times s of ξ
⋆. Note that
(115)
Δ
ξ
x
s
⋆
i
=
x
s
⋆
i
-
(
x
s
-
⋆
i
+
Δ
N
x
s
⋆
i
)
=
∑
l
=
1
m
‍
G
s
i
l
Δ
ξ
s
⋆
l
,
for
i
=
1
,
…
,
n
,
where Δ
ξ
s
⋆
l
=
ξ
s
⋆
l
-
ξ
s
-
⋆
l is a pure jump process. Then, we can rewrite (114) as follows:
(116)
∂
W
∂
x
k
(
τ
R
⋆
,
x
τ
R
⋆
⋆
)
=
∂
W
∂
x
k
(
t
,
x
t
⋆
)
+
∫
t
τ
R
⋆
‍
{
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
γ
i
(
s
,
e
)
)
ν
(
d
e
)
∂
2
W
∂
s
∂
x
k
(
s
,
x
s
⋆
)
+
∑
i
=
1
n
‍
b
i
(
s
)
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
+
1
2
∑
i
,
j
=
1
n
‍
a
i
j
(
s
)
∂
3
W
∂
x
k
∂
x
i
∂
x
j
(
s
,
x
s
⋆
)
+
∫
E
‍
(
∑
i
=
1
n
‍
∂
W
∂
x
k
(
s
,
x
s
⋆
+
γ
(
s
,
e
)
)
-
∂
W
∂
x
k
(
s
,
x
s
-
⋆
)
-
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
γ
i
(
s
,
e
)
)
ν
(
d
e
)
}
d
s
+
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
σ
i
(
s
)
d
B
s
+
∫
t
τ
R
⋆
‍
∫
E
‍
{
∂
W
∂
x
k
(
s
,
x
s
-
⋆
+
γ
-
(
s
,
e
)
)
-
∂
W
∂
x
k
(
s
,
x
s
-
⋆
)
}
N
~
(
d
s
,
d
e
)
+
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
∑
l
=
1
m
‍
G
s
i
l
d
ξ
s
⋆
c
l
+
∑
t
<
s
≤
τ
R
⋆
‍
Δ
ξ
∂
W
∂
x
k
(
s
,
x
s
⋆
)
.
Let ξ
s
⋆
c denotes the continuous part of ξ
s
⋆; that is, ξ
s
⋆
c
=
ξ
s
⋆
-
∑
t
<
s
≤
τ
R
⋆
Δ
ξ
s
⋆
l. Then, we can easily show that
(117)
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
G
s
i
l
d
ξ
s
⋆
c
l
=
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
G
s
i
l
1
{
(
s
,
x
s
⋆
)
∈
D
l
}
d
ξ
s
⋆
c
l
+
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
G
s
i
l
1
{
(
s
,
x
s
⋆
)
∈
C
l
}
d
ξ
s
⋆
c
l
.
For every(
t
,
x
)
∈
D
l, using (88) we have
(118)
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
t
,
x
)
G
t
i
l
=
∂
∂
x
k
{
∑
i
=
1
n
‍
∂
W
∂
x
i
(
t
,
x
)
G
t
i
l
+
k
s
l
}
=
0
,
for
l
=
1
,
…
,
m
.
This proves
(119)
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∑
l
=
1
m
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
G
s
i
l
1
{
(
s
,
x
s
⋆
)
∈
D
l
}
d
ξ
s
⋆
c
l
=
0
.
Furthermore, for every (
t
,
x
)
∈
C
l and l
=
1
,
…
,
m, we have ∑
i
=
1
n
(
∂
W
/
∂
x
k
∂
x
i
)
(
t
,
x
)
G
t
i
l
<
0.
But (91) implies that ∑
l
=
1
m
1
{
(
s
,
x
s
⋆
)
∈
C
l
}
d
ξ
s
⋆
c
l
=
0; thus
(120)
∫
t
τ
R
⋆
‍
∑
i
=
1
n
‍
∑
l
=
1
m
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
x
s
⋆
)
G
s
i
l
1
{
(
s
,
x
s
⋆
)
∈
C
l
}
d
ξ
s
⋆
c
l
=
0
.
The mean value theorem yields(121)
Δ
ξ
∂
W
∂
x
k
(
s
,
x
s
⋆
)
=
(
∂
W
∂
x
k
)
x
(
s
,
y
(
s
)
)
Δ
ξ
x
s
⋆
,
where y
(
s
) is some point on the straight line between x
s
-
⋆
+
Δ
N
x
s
⋆ and x
s
⋆, and (
∂
W
/
∂
x
k
)
x represents the gradient matrix of ∂
W
/
∂
x
k. To prove that the right-hand side of the above equality vanishes, it is enough to check that if Δ
ξ
s
⋆
l
>
0 then ∑
i
=
1
n
(
∂
2
W
/
∂
x
k
∂
x
i
)
(
s
,
y
(
s
)
)
G
s
i
l
=
0, for l
=
1
,
…
,
m. It is clear by (92) that
(122)
0
=
Δ
ξ
W
(
s
,
x
s
⋆
)
+
∑
l
=
1
m
‍
k
s
l
Δ
ξ
s
⋆
l
=
∑
l
=
1
m
‍
{
∑
i
=
1
n
‍
∂
W
∂
x
i
(
s
,
y
(
s
)
)
G
s
i
l
+
k
s
l
}
Δ
ξ
s
⋆
l
.
SinceΔ
ξ
s
⋆
l
>
0, then (
s
,
y
(
s
)
)
∈
D
l, for l
=
1
,
…
,
m. According to (88), we obtain
(123)
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
s
,
y
(
s
)
)
G
s
i
l
=
∂
∂
x
k
{
∑
i
=
1
n
‍
∂
W
∂
x
i
(
s
,
y
(
s
)
)
G
s
i
l
+
k
s
l
}
=
0
.
This shows that
(124)
∑
t
<
s
≤
τ
R
⋆
‍
Δ
ξ
∂
W
∂
x
k
(
s
,
x
s
⋆
)
=
0
.
On the other hand, define(125)
A
(
t
,
x
,
u
)
=
∂
W
∂
t
(
t
,
x
)
+
∑
i
=
1
n
‍
b
i
(
t
,
x
,
u
)
∂
W
∂
x
i
(
t
,
x
)
+
1
2
∑
i
,
j
=
1
n
‍
a
i
j
(
t
,
x
,
u
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
)
+
f
(
t
,
x
,
u
)
+
∫
E
{
∑
i
=
1
n
‍
W
(
t
,
x
+
γ
(
t
,
x
,
u
,
e
)
)
-
W
(
t
,
x
)
‍
-
∑
i
=
1
n
‍
γ
i
(
t
,
x
,
u
,
e
)
∂
W
∂
x
i
(
t
,
x
)
}
ν
(
d
e
)
.
If we differentiateA
(
t
,
x
,
u
) with respect to x
k and evaluate the result at (
x
,
u
)
=
(
x
t
⋆
,
u
t
⋆
), we deduce easily from (84), (89), and (90) that
(126)
∂
2
W
∂
t
∂
x
k
(
t
,
x
t
⋆
)
+
∑
i
=
1
n
‍
b
i
(
t
)
∂
2
W
∂
x
k
∂
x
i
(
t
,
x
t
⋆
)
+
1
2
∑
i
,
j
=
1
n
‍
a
i
j
(
t
)
∂
3
W
∂
x
k
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
+
∫
E
{
∑
i
=
1
n
‍
∂
W
∂
x
k
(
t
,
x
t
⋆
+
γ
(
t
,
e
)
)
-
∂
W
∂
x
k
(
t
,
x
t
⋆
)
‍
-
∑
i
=
1
n
‍
γ
i
(
s
,
e
)
∂
2
W
∂
x
k
∂
x
i
(
t
,
x
t
⋆
)
}
ν
(
d
e
)
=
-
∑
i
=
1
n
‍
∂
b
i
∂
x
k
(
t
)
∂
W
∂
x
i
(
t
,
x
t
⋆
)
-
1
2
∑
i
,
j
=
1
n
‍
∂
a
i
j
∂
x
k
(
t
,
x
t
⋆
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
-
∂
f
∂
x
k
(
t
,
x
t
⋆
,
u
t
⋆
)
-
∫
E
‍
∑
i
=
1
n
‍
∂
γ
i
∂
x
k
(
t
,
e
)
×
{
∂
W
∂
x
i
(
t
,
x
t
⋆
+
γ
(
t
,
e
)
)
-
∂
W
∂
x
i
(
t
,
x
t
⋆
)
}
ν
(
d
e
)
.
Finally, substituting (119), (120), (124), and (126) into (116) yields
(127)
d
(
∂
W
∂
x
k
(
t
,
x
t
⋆
)
)
=
-
{
∑
i
=
1
n
‍
∂
b
i
∂
x
k
(
t
)
∂
W
∂
x
i
(
t
,
x
t
⋆
)
+
1
2
∑
i
,
j
=
1
n
‍
∂
a
i
j
∂
x
k
(
t
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
+
∂
f
∂
x
k
(
t
)
+
∫
E
‍
∑
i
=
1
n
‍
∂
γ
i
∂
x
k
(
t
,
e
)
×

(
∂
W
∂
x
i
(
t
,
x
t
⋆
+
γ
(
t
,
e
)
)
-
∂
W
∂
x
i
(
t
,
x
t
⋆
)

)

ν
(
d
e
)

∑
i
=
1
n
‍
}

d
t
+
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
t
,
x
t
⋆
)
σ
i
(
t
)
d
B
t
+
∫
E
‍
{
∂
W
∂
x
k
(
t
,
x
t
-
⋆
+
γ
-
(
t
,
e
)
)

-
∂
W
∂
x
k
(
t
,
x
t
-
⋆
)
}
N
~
(
d
t
,
d
e
)

.
The continuity of∂
W
/
∂
x
k leads to
(128)
lim
R
→
∞
∂
W
∂
x
k
(
τ
R
⋆
,
x
τ
R
⋆
⋆
)
=
∂
W
∂
x
k
(
T
,
x
T
⋆
)
=
∂
g
∂
x
k
(
x
T
⋆
)
,
for
each
k
=
1
,
…
,
n
.
Clearly,(129)
1
2
∑
i
,
j
=
1
n
‍
∂
a
i
j
∂
x
k
(
t
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
=
1
2
∑
i
,
j
=
1
n
‍
∂
∂
x
k
(
∑
h
=
1
d
‍
σ
i
h
(
t
)
σ
j
h
(
t
)
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
=
∑
j
=
1
n
‍
∑
h
=
1
d
‍
(
∑
i
=
1
n
‍
σ
i
h
(
t
)
∂
2
W
∂
x
i
∂
x
j
(
t
,
x
t
⋆
)
)
∂
σ
i
h
∂
x
k
(
t
)
.
Now, from (17) we have
(130)
∂
H
∂
x
k
(
t
,
x
,
u
,
p
,
q
,
r
(
·
)
)
=
∑
i
=
1
n
‍
∂
b
i
∂
x
k
(
t
,
x
,
u
)
p
i
+
∑
h
=
1
d
‍
∑
i
=
1
n
‍
∂
σ
i
h
∂
x
k
(
t
,
x
,
u
)
q
i
h
+
∂
f
∂
x
k
(
t
,
x
,
u
)
+
∫
E
‍
∑
i
=
1
n
‍
∂
γ
i
∂
x
k
(
t
,
x
,
u
,
e
)
r
i
(
e
)
ν
(
d
e
)
.
Thekth coordinate p
t
k of the adjoint process p
t satisfies
(131)
d
p
t
k
=
-
∂
H
∂
x
k
(
t
,
x
t
⋆
,
u
t
⋆
,
p
t
,
q
t
,
r
t
(
·
)
)
d
t
+
q
t
k
d
B
t
+
∫
E
‍
r
t
-
k
(
e
)
N
~
(
d
t
,
d
e
)
,
for
t
∈
[
0
,
T
]
,
p
T
k
=
∂
g
∂
x
k
(
x
T
⋆
)
,
with q
t
k
d
B
t
=
∑
h
=
1
d
‍
q
t
k
h
d
B
t
h. Hence, the uniqueness of the solution of (131) and relation (128) allows us to get
(132)
p
t
k
=
∂
W
∂
x
k
(
t
,
x
t
⋆
)
,
q
t
k
h
=
∑
i
=
1
n
‍
∂
2
W
∂
x
k
∂
x
i
(
t
,
x
t
⋆
)
σ
i
h
(
t
)
,
r
t
-
k
(
·
)
=
∂
W
∂
x
k
(
t
,
x
t
-
⋆
+
γ
(
t
,
e
)
)
-
∂
W
∂
x
k
(
t
,
x
t
-
⋆
)
,
where q
t
k
h is the generic element of the matrix q
t and x
t
⋆ is the optimal solution of the controlled SDE (8).Example 14.
We return to the same example in the previous section.
Now, we illustrate a verification result for the maximum principle. We suppose thatT is a fixed time. In this case the Hamiltonian gets the form
(133)
H
(
t
,
X
t
,
p
t
,
q
t
,
r
t
(
·
)
)
=
μ
X
t
p
t
+
σ
X
t
q
t
+
X
t
γ
(
-
ϰ
t
)
+
θ
X
t
-
∫
ℝ
+
e
r
t
(
e
)
ν
(
d
e
)
.
Letξ
⋆ be a candidate for an optimal control, and let X
⋆ be the corresponding state process with corresponding solution (
p
⋆
,
q
⋆
,
r
⋆
(
·
)
) of the following adjoint equation, for all t
∈
[
0
,
T
)
(134)
d
p
t
⋆
=
-
(
μ
p
t
⋆
+
σ
q
t
⋆
+
θ
∫
ℝ
+
‍
e
r
t
⋆
(
e
)
ν
(
d
e
)
+
γ
X
t
⋆
γ
-
1
exp
(
-
ϰ
s
)
∫
ℝ
+
‍
)
d
t
+
q
t
⋆
d
B
t
+
∫
ℝ
+
‍
r
t
-
⋆
(
e
)
N
~
(
d
t
,
d
e
)
,
(135)
-
p
t
⋆
+
exp
(
-
ϰ
t
)
≤
0
,
∀
t
,
(136)
1
{
-
p
t
⋆
+
exp
(
-
ϰ
t
)
<
0
}
d
ξ
t
⋆
c
=
0
,
(137)
-
(
p
t
-
⋆
+
Δ
N
p
t
⋆
)
+
exp
(
-
ϰ
t
)
≤
0
,
(138)
1
{
-
(
p
t
-
⋆
+
Δ
N
p
t
⋆
)
+
exp
(
-
ϰ
t
)
<
0
}
Δ
ξ
t
⋆
=
0
.
Sinceg
=
0, we assume the transversality condition
(139)
𝔼
[
p
T
⋆
(
X
T
⋆
-
X
T
)
]
≤
0
.
We remark thatΔ
ξ
p
t
⋆
=
0; then p
t
-
⋆
+
Δ
N
p
t
⋆
=
p
t
⋆, and the condition (138) reduces to
(140)
1
{
-
p
t
⋆
+
exp
(
-
ϰ
t
)
<
0
}
Δ
ξ
t
⋆
=
0
.
We use the relation between the value function and the solution(
p
⋆
,
q
⋆
,
r
⋆
(
e
)
) of the adjoint equation along the optimal state. We prove that the solution of the adjoint equation is represented as
(141)
p
t
⋆
=
(
A
ρ
X
t
⋆
ρ
-
1
+
K
γ
X
t
⋆
γ
-
1
)
exp
(
-
ϰ
t
)
,
q
t
⋆
=
σ
(
A
ρ
(
ρ
-
1
)
X
t
⋆
ρ
-
1
+
K
γ
(
γ
-
1
)
X
t
⋆
γ
-
1
)
exp
(
-
ϰ
t
)
,
r
t
⋆
(
e
)
=
(
A
ρ
(
(
1
+
θ
e
)
ρ
-
1
-
1
)
X
t
⋆
ρ
-
1
+
K
γ
(
(
1
+
θ
e
)
γ
-
1
-
1
)
X
t
⋆
γ
-
1
)
exp
(
-
ϰ
t
)
for all t
∈
[
0
,
T
).
To see this, we differentiate the process(
A
ρ
X
t
⋆
ρ
-
1
+
K
γ
X
t
⋆
γ
-
1
)
exp
(
-
ϰ
t
) using Itô’s rule for semimartingales and by using the same procedure as in the proof of Theorem 13. Then, the conclusion follows readily from the verification of (135), (136), and (139). First, an explicit formula for X
t is given in [4] by
(142)
X
t
=
e
μ
t
M
t
{
∑
0
<
s
≤
t
‍
x
-
(
∑
0
<
s
≤
t
‍
∫
[
0
,
t
)
‍
M
s
-
1
exp
(
-
μ
s
)
d
ξ
s
+
∑
0
<
s
≤
t
‍
M
s
-
1
β
s
exp
(
-
μ
s
)
Δ
ξ
s
)
}
,
for
t
∈
[
0
,
T
]
,
where β
t
=
(
∫
ℝ
+
‍
θ
e
N
(
{
t
}
,
d
e
)
)
(
1
+
∫
ℝ
+
‍
θ
e
N
(
{
t
}
,
d
e
)
)
-
1, and M
t is a geometric Lévy process defined by
(143)
M
t
=
exp
{
(
-
1
2
σ
2
+
∫
ℝ
+
‍
{
ln
(
1
+
θ
e
)
-
θ
e
}
ν
(
d
e
)
)
t
+
σ
B
t
+
∫
0
t
‍
∫
ℝ
+
‍
ln
(
1
+
θ
e
)
N
~
(
d
t
,
d
e
)
(
-
1
2
σ
2
+
∫
ℝ
+
‍
{
ln
(
1
+
θ
e
)
-
θ
e
}
ν
(
d
e
)
)
}
.
From the representation (142) and by the fact that X
T
∧
t
⋆
≤
x
M
T
∧
t
exp
(
μ
(
T
∧
t
)
), we get
(144)
1
-
X
T
∧
t
X
T
∧
t
⋆
≤
1
x
(
∑
0
<
s
≤
t
‍
∫
[
0
,
T
∧
t
)
‍
M
s
-
1
exp
(
-
μ
s
)
d
ξ
s
+
∑
0
<
s
≤
t
‍
M
s
-
1
β
s
exp
(
-
μ
s
)
Δ
ξ
s
)
<
∞
,
hence
(145)
𝔼
[
p
T
∧
t
⋆
(
X
T
∧
t
⋆
-
X
T
∧
t
)
]
≤
𝔼
[
(
(
A
ρ
X
T
∧
t
⋆
ρ
+
K
γ
X
T
∧
t
⋆
γ
)
exp
(
-
ϰ
(
T
∧
t
)
)
)
2
]
1
/
2
×
𝔼
[
∑
0
<
s
≤
T
∧
t
‍
M
s
-
1
β
s
exp
(
-
μ
s
)
Δ
ξ
s
)
2
(
1
x
∫
[
0
,
T
∧
t
)
‍
M
s
-
1
exp
(
-
μ
s
)
d
ξ
s
+
∑
0
<
s
≤
T
∧
t
‍
M
s
-
1
β
s
exp
(
-
μ
s
)
Δ
ξ
s
)
2
]
1
/
2
.
By the dominated convergence theorem, we obtain (139) by sending t to infinity in (145).
A simple computation shows that the conditions (135)–(138) are consequences of (107)–(109). This shows in particular that the pair (
X
t
⋆
,
ξ
t
⋆
) satisfies the optimality sufficient conditions and then it is optimal. This completes the proof of the following result.Theorem 15.
One supposes thatσ
2
/
2
+
θ
∫
ℝ
+
‍
e
ν
(
d
e
)
≤
μ
<
ϰ, and e
≥
0
d
ν
-
a
.
e. If the strategy ξ
⋆ is chosen such that the corresponding solution of the adjoint process is given by (141), then this choice is optimal.Remark 16.
In this example, it is shown in particular that the relationship between the stochastic maximum principle and dynamic programming could be very useful to solve explicitly constrained backward stochastic differential equations with transversality condition.



---

*Source: 201491-2014-01-09.xml*