# Optimal Control with Partial Information for Stochastic Volterra Equations

Authors: Bernt øksendal; Tusheng Zhang
Journal: International Journal of Stochastic Analysis
 (2010)

---

## Abstract

In the first part of the paper we obtain existence and characterizations of an optimal control for a linear quadratic control problem of linear stochastic Volterra equations. In the second part, using the Malliavin calculus approach, we deduce a general maximum principle for optimal control of general stochastic Volterra equations. The result is applied to solve some stochastic control problem for some stochastic delay equations.


---

## Body

## 1. Introduction

Let(Ω,ℱ,ℱt,P) be a filtered probability space and B(t),t≥0 a ℱt-real valued Brownian motion. Let R0=R∖{0} and ν(dz) a σ-finite measure on (R0,ℬ(R0)). Let N(dt,dz) denote a stationary Poisson random measure on R+×R0 with intensity measure dtν(dz). Denote by Ñ(dt,dz)=N(dt,dz)-dtν(dz) the compensated Poisson measure. Suppose that we have a cash flow where the amount X(t) at time t is modelled by a stochastic delay equation of the form:
(1.1)dX(t)={A1(t)X(t)+A2(t)X(t-h)+∫t-htA0(t,s)X(s)ds}dt+C1(t)dB(t)+∫R0C2(t,z)Ñ(dt,dz);t≥0,X(t)=η(t);t∈[-h,0].
Here h>0 is a fixed delay and A1(t),A2(t),A0(t,s), C1(t),C2(t,z),andη are given bounded deterministic functions.Suppose that we consume at the rateu(t) at time t from this wealth X(t), and that this consumption rate influences the growth rate of X(t) both through its value u(t) at time t and through its former value u(t-h), because of some delay mechanisms in the system determining the dynamics of X(t).With such a consumption rateu(t) the dynamics of the corresponding cash flow Xu(t) is given by
(1.2)dXu(t)={A1(t)Xu(t)+A2(t)Xu(t-h)+∫t-htA0(t,s)Xu(s)ds+B1(t)u(t)+B2(t)u(t-h)∫t-htA0}dt+C1(t)dB(t)+∫R0C2(t,z)Ñ(dt,dz);t∈[-h,0],Xu(t)=η(t);t≤0,
where B1(t)andB2(t) are deterministic bounded functions.Suppose that the consumer wants to maximize the combined utility of the consumption up to the terminal timeT and the terminal wealth. Then the problem is to find u(·) such that(1.3)J(u):=E[∫0TU1(t,u(t))dt+U2(Xu(T))]
is maximal. Here U(t,·) and U2(·) are given utility functions, possibly stochastic. See Section 4.This is an example of a stochastic control problem with delay. Such problems have been studied by many authors. See, for example, [1–5] and the references therein. The methods used in these papers, however, do not apply to the cases studied here. Moreover, these papers do not consider partial information control (see below).It was shown in [6] that the system (1.2) is equivalent to the following controlled stochastic Volterra equation:(1.4)Xu(t)=∫0tK(t,s)u(s)ds+∫0tΦ(t,s)C(s)dB(s)+∫0t∫R0Φ(t,s)C2(s,z)Ñ(ds,dz)+Φ(t,0)η(0)+∫-h0Φ(t,s+h)A2(s+h)η(s)ds+∫-h0(∫0hΦ(t,τ)A0(τ,s)dτ)η(s)ds,
where (1.5)K(t,s)=Φ(t,s)B1(s)+Φ(t,s+h)B2(s+h),andΦ is the transition function satisfying(1.6)∂Φ∂t=A1(t)Φ(t,s)+A2(t)Φ(t-h,s)+∫t-htA0(t,τ)Φ(τ,s)dτ,Φ(s,s)=I;Φ(t,s)=0fort<s.So the control of the system (1.2) reduces to the control of the system (1.4). For more information about stochastic control of delay equations we refer to [6] and the references therein.Stochastic Volterra equations are interesting on their own right, also for applications, for example, to economics or population dynamics. See, for example, Example1.1 in [7] and the references therein.In the first part of this paper, we study a linear quadratic control problem for the following controlled stochastic Volterra equation:(1.7)Xu(t)=ξ(t)+∫0t[K1(t,s)Xu(s)+D1(t,s)u(s)+K2(t,s)]dB(s)+∫0t∫R0K4(t,s,z)Xu(s)Ñ(ds,dz)+∫0tD2(t,s)Xu(s)ds+∫0t∫R0D3(t,s,z)u(s)Ñ(ds,dz)+∫0t∫R0K5(t,s,z)Ñ(ds,dz)+∫0tK3(t,s)u(s)ds,
where u(t) is our control process and ξ(t) is a given predictable process with E[ξ2(t)]<∞ for all t≥0, while Ki,Di are bounded deterministic functions. In reality one often does not have the complete information when performing a control to a system. This means that the control processes is required to be predictable with respect to a subfiltration {𝒢t} with 𝒢t⊂ℱt. So the space of controls will be(1.8)U={u(s);u(s)is𝒢t-predictableandsuchthatE[∫0T|u(s)|2ds]<∞}.U is a Hilbert space equipped with the inner product (1.9)〈u1,u2〉=E[∫0Tu1(s)u2(s)ds].||·|| will denote the norm in U. Let 𝒜𝒢 be a closed, convex subset of U, which will be the space of admissible controls. Consider the linear quadratic cost functional
(1.10)J(u)=E[∫0TQ1(s)u2(s)ds+∫0TQ2(s)Xu(s)2ds+∫0TQ3(s)u(s)ds+∫0TQ4(s)Xu(s)ds+a1Xu(T)2+a2Xu(T)]
and the value function
(1.11)J=infu∈𝒜𝒢J(u).
In Section 2, we prove the existence of an optimal control and provide some characterizations for the control.In the second part of the paper (from Section3), we consider the following general controlled stochastic Volterra equation:(1.12)Xu(t)=ξ(t)+∫0tb(t,s,Xu(s),u(s),ω)ds+∫0tσ(t,s,Xu(s),u(s),ω)dB(s)+∫0t∫R0θ(t,s,Xu(s),u(s),z,ω)Ñ(ds,dz),
where ξ(t) is a given predictable process with E[ξ2(t)]<∞ for all t≥0. The performance functional is of the following form:
(1.13)J(u)=E[∫0Tf(t,Xu(t),u(t),ω)dt+g(Xu(T),ω)],
where b:[0,T]×[0,T]×R×R×Ω→R, σ:[0,T]×[0,T]×R×R×Ω→R, θ:[0,T]×[0,T]×R×R×R0×Ω→R and f:[0,T]×R×R×Ω→R are ℱt-predictable and g:R×Ω→R is ℱT measurable and such that
(1.14)E[∫0T|f(t,Xu(t),u(t))|dt+|g(Xu(T))|]<∞,
for any u∈𝒜𝒢, the space of admissible controls. The problem is to find û∈𝒜𝒢 such that(1.15)Φ:=supu∈𝒜𝒢J(u)=J(û).Using the Malliavin calculus, inspired by the method in [8], we will deduce a general maximum principle for the above control problem.Remark 1.1.
Note that we are off the Markovian setting because the solution of the Volterra equation is not Markovian. Therefore the classical method of dynamic programming and the Hamilton-Jacobi-Bellman equation cannot be used here.Remark 1.2.
We emphasize that partial information is different from partial observation, where the control is based on noisy observations of the (current) state. For example, our discussion includes the case𝒢t=ℱt-δ (δ>0 constant), which corresponds to delayed information flow. This case is not covered by partial observation models. For a comprehensive presentation of the linear quadratic control problem in the classical case with partial observation, see [9], with partial information see [10].


## 2. Linear Quadratic Control

Consider the controlled stochastic Volterra equation (1.7) and the control problem (1.10), (1.11). We have the following Theorem.Theorem 2.1.
Suppose that∫R0K42(t,s,z)ν(dz) is bounded and Q2(s)≥0, a1≥0 and Q1(s)≥δ for some δ>0. Then there exists a unique element u∈𝒜𝒢 such that
(2.1)J=J(u)=infv∈𝒜𝒢J(v).Proof.
For simplicity, we assumeD3(t,s,z)=0 and K5(t,s,z)=0 in this proof because these terms can be similarly estimated as the corresponding terms for Brownian motion B(·). By (1.7) we have
(2.2)E[Xu(t)2]≤7E[ξ(t)2]+7E[(∫0tK1(t,s)Xu(s)dB(s))2]+7E[(∫0tD1(t,s)u(s)dB(s))2]+7E[(∫0tK2(t,s)dB(s))2]+7E[(∫0tK3(t,s)u(s)ds)2]+7E[(∫0tD2(t,s)Xu(s)ds)2]+7E[(∫0t∫R0K4(t,s,z)Xu(s)Ñ(ds,dz))2]≤7E[ξ(t)2]+7E[∫0tK12(t,s)Xu(s)2ds]+7E[∫0tD12(t,s)u(s)2ds]+7∫0tK22(t,s)ds+7∫0tK32(t,s)dsE[∫0tu2(s)ds]+7tE[∫0tD22(t,s)Xu(s)2ds]+7E[∫0t(∫R0K42(t,s,z)ν(dz))Xu(s)2ds].
Applying Gronwall's inequality, there exists a constant C1 such that
(2.3)E[Xu(t)2]≤(C1E[∫0tu2(s)ds]+C1)eC1T.
Similar arguments also lead to
(2.4)E[(Xu1(t)-Xu2(t))2]≤C2eC2T(E[(∫0tK3(t,s)(u2(s)-u1(s))ds)2]+E[∫0tD1(t,s)2(u2(s)-u1(s))2ds])
for some constant C2. Now, let un∈𝒜𝒢 be a minimizing sequence for the value function, that is, limn→∞J(un)=J. From the estimate (2.3) we see that there exists a constant c such that
(2.5)E[∫0TQ3(s)u(s)ds+∫0TQ4(s)Xu(s)ds+a2Xu(T)]≤c∥u∥+c.
Thus, by virtue of the assumption on Q1, we have, for some constant M,
(2.6)M≥J(un)≥δ∥un∥2-c∥un∥-c.
This implies that {un} is bounded in U, hence weakly compact. Let unk,k≥1 be a subsequence that converges weakly to some element u0 in U. Since 𝒜𝒢 is closed and convex, the Banach-Sack Theorem implies u0∈𝒜𝒢. From (2.4) we see that un→u in U implies that Xun(t)→Xu(t) in L2(Ω) for every t≥0 and Xun(·)→Xu(·) in U. The same conclusion holds also for Zu(t):=Xu(t)-X0(t). Since Zu is linear in u, we conclude that equipped with the weak topology both on U and L2(Ω), Zu(t):U→L2(Ω) is continuous for every t≥0 and Zu(·):U→U is continuous. Thus,
(2.7)Xu(t):U→L2(Ω),Xu(·):U→U
are continuous with respect to the weak topology of U and L2(Ω). Since the functionals of Xu involved in the definition of J(u) in (1.10) are lower semicontinuous with respect to the weak topology, it follows that
(2.8)limk→∞J(unk)=limk→∞E[∫0TQ1(s)unk2(s)ds+∫0TQ2(s)Xunk(s)2ds+∫0TQ3(s)unk(s)ds+∫0TQ4(s)Xunk(s)ds+a1Xunk(T)2+a2Xunk(T)]≥E[∫0TQ1(s)u02(s)ds+∫0TQ2(s)Xu0(s)2ds+∫0TQ3(s)u0(s)ds+∫0TQ4(s)Xu0(s)ds+a1Xu0(T)2+a2Xu0(T)]=J(u0),
which implies that u0 is an optimal control.
The uniqueness is a consequence of the fact thatJ(u) is strictly convex in u which is due to the fact that Xu is affine in u and x2 is a strictly convex function. The proof is complete.To characterize the optimal control, we assumeD1(t,s)=0 and D3(t,s,z)=0; that is, consider the controlled system:(2.9)Xu(t)=ξ(t)+∫0t[K1(t,s)Xu(s)+K2(t,s)]dB(s)+∫0tK3(t,s)u(s)ds+∫0t∫R0K4(t,s,z)Xu(s)Ñ(ds,dz)+∫0tD2(t,s)Xu(s)ds+∫0t∫R0K5(t,s,z)Ñ(ds,dz)
Set(2.10)dF(t,s):=dsF(t,s)=K1(t,s)dB(s)+∫R0K4(t,s,z)Ñ(ds,dz)+D2(t,s)ds.
For a predictable process h(s), we have(2.11)∫0th(s)dF(t,s):=∫0tK1(t,s)h(s)dB(s)+∫0t∫R0K4(t,s,z)h(s)Ñ(ds,dz)+∫0tD2(t,s)h(s)ds.Introduce(2.12)M1(t)=ξ(t)+∑n=1∞∫0tdF(t,s1)∫0s1dF(s1,s2)⋯∫0sn-1ξ(sn)dF(sn-1,sn),M2(t)=∫0tK2(t,s1)dB(s1)+∑n=1∞∫0tdF(t,s1)∫0s1dF(s1,s2)⋯∫0sn-2dF(sn-2,sn-1)∫0sn-1K2(sn-1,sn)dB(sn),M3(t)=∫0t∫R0K5(t,s1,z)dÑ(ds1,dz)+∑n=1∞∫0tdF(t,s1)∫0s1dF(s1,s2)⋯∫0sn-2dF(sn-2,sn-1)∫0sn-1K5(sn-1,sn,z)dÑ(dsn,dz),L(t,s)=K3(t,s)+∑n=1∞∫stdF(t,s1)∫ss1dF(s1,s2)⋯∫ssn-1K3(sn,s)dF(sn-1,sn).Lemma 2.2.
Under our assumptions, the above series converges at least inL1(Ω). Thus Mi,i=1,2,3 and L are well-defined.Proof.
We first note that(2.13)E[(∫0th(s)dF(t,s))2]=E[∫0tK12(t,s)h2(s)ds]+E[∫0t∫R0K42(t,s,z)h2(s)ν(dz)ds]+E[(∫0tD2(t,s)h(s)ds)2]≤CTE[∫0tg(t,s)h2(s)ds]
for t≤T, where
(2.14)g(t,s)=K12(t,s)+∫R0K42(t,s,z)ν(dz)+D22(t,s)
is a bounded deterministic function. Because of the similarity, let us prove only that M1 is well-defined. Repeatedly using (2.13), we have
(2.15)E[(∫0tdF(t,s1)∫0s1dF(s1,s2)⋯∫0sn-1ξ(sn)dF(sn-1,sn))2]≤CT∫0tds1g(t,s1)E[(∫0s1dF(s1,s2)⋯∫0sn-1ξ(sn)dF(sn-1,sn))2]≤⋯≤CTn-1∫0tds1g(t,s1)∫0s1ds2g(s1,s2)⋯∫0sn-1dsng(sn-1,sn)E[ξ2(sn)]≤RTn-1E[∫0Tξ2(s)ds]tn-1(n-1)!
for some constant RT. This implies that
(2.16)E[|∫0tdF(t,s1)∫0s1dF(s1,s2)⋯∫0sn-1ξ(sn)dF(sn-1,sn)|]≤RT(n-1)/2(E[∫0Tξ2(s)ds])1/2t(n-1)/2(n-1)!.
Thus, we have
(2.17)E[|M1(t)|]≤E[|ξ(t)|]+∑n=1∞RT(n-1)/2(E[∫0Tξ2(s)ds])1/2t(n-1)/2(n-1)!<∞.The following theorem is a characterization of the optimal control.Theorem 2.3.
Assume that∫R0K42(t,s,z)ν(dz) and ∫R0K52(t,s,z)ν(dz) are bounded and E[∫0Tξ2(s)ds]<∞. Suppose 𝒜𝒢=U. Let u be the unique optimal control given in Theorem 2.1. Then u is determined by the following equation:
(2.18)2Q1(s)u(s)+2E[∫0Tu(t)(∫s∨tTQ2(l)L(l,t)L(l,s)dl)dt∣𝒢s]+2a1E[∫0Tu(t)L(T,t)L(T,s)dt∣𝒢s]+Q3(s)+E[∫sTQ4(l)L(l,s)dl∣𝒢s]+2E[∫sTQ2(l)(M1(l)+M2(l)+M3(l))L(l,s)dl∣𝒢s]+a2E[L(T,s)∣𝒢s]+2a1E[(M1(T)+M2(T)+M3(T))L(T,s)∣𝒢s]=0,
almost everywhere with respect to m(ds,dω):ds×P(dω).Proof.
For anyw∈U, since u is the optimal control, we have
(2.19)J′(u)(w)=ddεJ(u+εw)|ε=0=0.
This leads to
(2.20)E[2∫0TQ1(s)u(s)w(s)ds+2∫0TQ2(s)Xu(s)ddεXu+εw(s)|ε=0ds+∫0TQ3(s)w(s)ds+∫0TQ4(s)ddεXu+εw(s)|ε=0ds+2a1Xu(T)ddεXu+εw(T)|ε=0+a2ddεXu+εw(T)|ε=0∫0TQ2(s)Xu(s)ddεXu+εw(s)]=0
for all w∈U. By virtue of (2.9), it is easy to see that
(2.21)Yw(t):=ddεXu+εw(t)|ε=0
satisfies the following equation:
(2.22)Yw(t)=∫0tK1(t,s)Yw(s)dB(s)+∫0tK3(t,s)w(s)ds+∫0t∫EK4(t,s,z)Yw(s)Ñ(ds,dz)+∫0tD2(t,s)Yw(s)ds.
Remark that Yw is independent of u. Next we will find an explicit expression for Xu. Let dF(t,s) be defined as in (2.10). Repeatedly using (2.9) we have
(2.23)Xu(t)=ξ(t)+∫0t[K1(t,s1)Xu(s1)+K2(t,s1)]dB(s1)+∫0tK3(t,s1)u(s1)ds1+∫0t∫R0K4(t,s1,z)Xu(s1)Ñ(ds1,dz)+∫0tD2(t,s1)Xu(s1)ds+∫0t∫R0K5(t,s1,z)Ñ(ds1,dz)=ξ(t)+∫0tK1(t,s1)[∫0s1‍∫R0K5(s1,s2,z)Ñ(ds2,dz)ξ(s1)+∫0s1[K1(s1,s2)Xu(s2)+K2(s1,s2)]dB(s2)+∫0s1‍∫R0K4(s1,s2,z)Xu(s2)Ñ(ds2,dz)+∫0s1K3(s1,s2)u(s2)ds2+∫0s1D2(s1,s2)Xu(s2)ds2+∫0s1‍∫R0K5(s1,s2,z)Ñ(ds2,dz)]dB(s1)+∫0t∫R0K4(t,s1,z)[∫0s1‍∫R0K5(s1,s2,z)Ñ(ds2,dz)ξ(s1)+∫0s1[K1(s1,s2)Xu(s2)+K2(s1,s2)]dB(s2)+∫0s1∫R0K4(s1,s2,z)Xu(s2)Ñ(ds2,dz)+∫0s1K3(s1,s2)u(s2)ds2+∫0s1D2(s1,s2)Xu(s2)ds2+∫0s1‍∫R0K5(s1,s2,z)Ñ(ds2,dz)]Ñ(ds1,dz)+∫0t‍∫R0D2(t,s1,z)[∫0s1‍∫R0K5(s1,s2,z)Ñ(ds2,dz)ξ(s1)+∫0s1[K1(s1,s2)Xu(s2)+K2(s1,s2)]dB(s2)+∫0s1∫R0K4(s1,s2,z)Xu(s2)Ñ(ds2,dz)+∫0s1K3(s1,s2)u(s2)ds2+∫0s1D2(s1,s2)Xu(s2)ds2+∫0s1∫R0K5(s1,s2,z)Ñ(ds2,dz)]ds1+∫0tK2(t,s1)dB(s1)+∫0tK3(t,s1)u(s1)ds1+∫0t‍∫R0K5(t,s1,z)Ñ(ds1,dz)=⋯=ξ(t)+∑n=1∞∫0tdF(t,s1)∫0s1dF(s1,s2)⋯∫0sn-1ξ(sn)dF(sn-1,sn)+∑n=1∞∫0tdF(t,s1)∫0s1dF(s1,s2)⋯∫0sn-2dF(sn-2,sn-1)∫0sn-1K2(sn-1,sn)dB(sn)+∑n=1∞∫0tdF(t,s1)∫0s1dF(s1,s2)⋯∫0sn-2dF(sn-2,sn-1)∫0sn-1K3(sn-1,sn)u(sn)dsn+∑n=1∞∫0tdF(t,s1)∫0s1dF(s1,s2)⋯∫0sn-2dF(sn-2,sn-1)∫0sn-1∫R0K5(sn-1,sn,z)Ñ(dsn,dz)+∫0tK2(t,s1)dB(s1)+∫0tK3(t,s1)u(s1)ds1+∫0t∫R0K5(t,s1,z)Ñ(ds1,dz).
Similarly, we have the following expansion for Yw:
(2.24)Yw(t)=∫0tK3(t,s)w(s)ds+∑n=1∞∫0tdF(t,s1)∫0s1dF(s1,s2)⋯∫0sn-2dF(sn-2,sn-1)∫0sn-1K3(sn-1,sn)w(sn)dsn.
Interchanging the order of integration,
(2.25)Yw(t)=∫0tw(s)[K3(t,s)+∑n=1∞∫stdF(t,s1)∫ss1dF(s1,s2)⋯∫ssn-1K3(sn,s)dF(sn-1,sn)]ds=∫0tL(t,s)w(s)ds.
Now substituting Yw into (2.20) we obtain that
(2.26)E[2∫0TQ1(s)u(s)w(s)ds+2∫0TQ2(s)Xu(s)(∫0sL(s,l)w(l)dl)ds]+E[∫0TQ3(s)w(s)ds+∫0TQ4(s)(∫0sL(s,l)w(l)dl)ds]+2a1E[∫0TXu(T)L(T,s)w(s)ds+a2∫0TL(T,s)w(s)ds]=0
for all w∈U. Interchanging the order of integration and conditioning on 𝒢s we see that (2.26) is equivalent to
(2.27)E[2∫0TQ1(s)u(s)w(s)ds+2∫0Tw(s)E[∫sTQ2(l)Xu(l)L(l,s)dl∣𝒢s]ds]+E[∫0TQ3(s)w(s)ds+∫0Tw(s)E[∫sTQ4(l)L(l,s)dl∣𝒢s]ds]+2a1E[∫0TE[Xu(T)L(T,s)∣𝒢s]w(s)ds]+a2E[∫0TE[L(T,s)∣𝒢s]w(s)ds]=0.
Since this holds for all w∈U, we conclude that
(2.28)2Q1(s)u(s)+2E[∫sTQ2(l)Xu(l)L(l,s)dl∣𝒢s]+Q3(s)+E[∫sTQ4(l)L(l,s)dl∣𝒢s]+2a1E[Xu(T)L(T,s)∣𝒢s]+a2E[L(T,s)∣𝒢s]=0,m-a.e. Note that Xu(t) can be written as
(2.29)Xu(t)=M1(t)+M2(t)+M3(t)+∫0tu(s)L(t,s)ds.
Substituting Xu(t) into (2.28), we get (2.18), completing the proof.Example 2.4.
Consider the controlled system(2.30)Xu(t)=ξ(t)+∫0tK2(t,s)dB(s)+∫0tK3(t,s)u(s)ds
and the performance functional
(2.31)J(u)=E[∫0TQ1(s)u2(s)ds+∫0TQ3(s)u(s)ds+∫0TQ4(s)Xu(s)ds+a1Xu(T)2+a2Xu(T)].
Suppose 𝒢t={Ω,∅}, meaning that the control is deterministic. In this case, we can find the unique optimal control explicitly. Noting that the conditional expectation reduces to expectation, the (2.18) for the optimal control u becomes
(2.32)2Q1(s)u(s)+2a1(∫0Tu(t)K3(T,t)dt)K3(T,s)+Q3(s)+∫sTQ4(l)K3(l,s)dl+a2K3(T,s)+2a1g(T)K3(T,s)=0ds-a.e.,
where we have used the fact that E[M2(t)]=0, M1(t)=ξ(t),L(t,s)=K3(t,s) in this special case. Put
(2.33)b=∫0Tu(t)K3(T,t)dt.
Then (2.33) yields
(2.34)u(s)=-a1bK3(T,s)Q1(s)+h(s),ds-a.e.,
where
(2.35)h(s)=-Q3(s)+∫sTQ4(l)K3(l,s)dl2Q1(s)-a2K3(T,s)+2a1g(T)K3(T,s)2Q1(s).
Substitute the expression of u into (2.34) to get
(2.36)-a1b∫0TK3(T,t)2Q1(t)dt+∫0Th(t)K3(T,t)dt=b.
Consequently,
(2.37)b=11+a1∫0T(K3(T,t)2/Q1(t))dt∫0Th(t)K3(T,t)dt.
Together with (2.35) we arrive at
(2.38)u(s)=-a1(11+a1∫0T(K3(T,t)2/Q1(t))dt∫0Th(t)K3(T,t)dt)K3(T,s)Q1(s)+h(s),ds-a.e.


## 3. A General Maximum Principle

In this section, we consider the following general controlled stochastic Volterra equation:(3.1)Xu(t)=ξ(t)+∫0tb(t,s,Xu(s),u(s),ω)ds+∫0tσ(t,s,Xu(s),u(s),ω)dB(s)+∫0t∫R0θ(t,s,Xu(s),u(s),z,ω)Ñ(ds,dz),
where u(t) is our control process taking values in R and ξ(t) is as in (1.7). More precisely, u∈𝒜𝒢, where 𝒜𝒢 is a family of 𝒢t-predictable controls. Here 𝒢t⊂ℱt is a given subfiltration and b:[0,T]×[0,T]×R×R×Ω→R, σ:[0,T]×[0,T]×R×R×Ω→R and θ:[0,T]×[0,T]×R×R×R0×Ω→R are given measurable, ℱt-predictable functions. Consider a performance functional of the following form:
(3.2)J(u)=E[∫0Tf(t,Xu(t),u(t),ω)dt+g(Xu(T),ω)],
where f:[0,T]×R×D×Ω→R is ℱt predictable and g:R×Ω→R is ℱT measurable and such that
(3.3)E[∫0T|f(t,Xu(t),u(t),ω)|dt+|g(Xu(T),ω)|]<∞,∀u∈𝒜𝒢.
The purpose of this section is to give a characterization for the critical point of J(u). First, in the following two subsections we recall briefly some basic properties of Malliavin calculus for B(·) and Ñ(·,·) which will be used in the sequel. For more information we refer to [11] and [12].


### 3.1. Integration by Parts Formula forB(·)

In this subsection,ℱT=σ(B(s),0≤s≤T). Recall that the Wiener-Ito chaos expansion theorem states that any F∈L2(ℱT,P) admits the representation
(3.4)F=∑n=0∞In(fn)
for a unique sequence of symmetric deterministic function fn∈L2([0,T]×n) and
(3.5)In(fn)=n!∫0T∫0tn⋯∫0t2fn(t1,…,tn)dB(t1)dB(t2)⋯dB(tn).
Moreover, the following isometry holds:
(3.6)E[F2]=∑n=0∞n!∥fn∥L2([0,T]×n)2.
Let D1,2 be the space of all F∈L2(ℱT,P) such that its chaos expansion (3.4) satisfies
(3.7)∥F∥D1,22:=∑n=0∞nn!∥fn∥L2([0,T]×n)2<∞.
For F∈D1,2 and t∈[0,T], the Malliavin derivative of F, DtF, is defined by
(3.8)DtF=∑n=0∞nIn-1(fn(·,t)),
where In-1(fn(·,t)) is the n-1 times iterated integral to the first n-1 variables of fn keeping the last variable tn=t as a parameter. We need the following result.Theorem A (Integration by parts formula (duality formula) forB(·)).
Suppose thath(t) is ℱt-adapted with E[∫0Th2(t)dt]<∞ and let F∈D1,2. Then
(3.9)E[F∫0Th(t)dB(t)]=E[∫0Th(t)DtFdt].


### 3.2. Integration by Parts Formula forÑ

In this sectionℱT=σ(η(s),0≤s≤T), where η(s)=∫0s∫R0zÑ(dr,dz). Recall that the Wiener-Ito chaos expansion theorem states that any F∈L2(ℱT,P) admits the representation
(3.10)F=∑n=0∞In(fn)
for a unique sequence of functions fn∈L̂2((dt×ν)n), where L̂2((dt×ν)n) is the space of functions fn(t1,z1,…,tn,zn);ti∈[0,T], zi∈R0 such that fn∈L2((dt×ν)n) and fn is symmetric with respect to the pairs of variables (t1,z1),(t2,z2),…,(tn,zn). Here In(fn) is the iterated integral:
(3.11)In(fn)=n!∫0T∫R0∫0tn∫R0⋯∫0t2∫R0fn(t1,z1,…,tn,zn)Ñ(dt1,dz1)⋯Ñ(dtn,dzn).
Moreover, the following isometry holds:
(3.12)E[F2]=∑n=0∞n!∥fn∥L2((dt×ν)n)2.
Let D̃1,2 be the space of all F∈L2(ℱT,P) such that its chaos expansion (3.18) satisfies
(3.13)∥F∥D̃1,22:=∑n=0∞nn!∥fn∥L2((dt×ν)n)2<∞.
For F∈D̃1,2 and t∈[0,T], the Malliavin derivative of F, Dt,zF, is defined by
(3.14)Dt,zF=∑n=0∞nIn-1(fn(·,t,z)),
where In-1(fn(·,t,z)) is the n-1 times iterated integral with respect to the first n-1 pairs of variables of fn keeping the last pair (tn,zn)=(t,z) as a parameter. We need the following resultTheorem B (Integration by parts formula (duality formula) forÑ).
Supposeh(t,z) is ℱt-predictable with E[∫0T∫R0h2(t,z)dtν(dz)]<∞ and let F∈D̃1,2. Then
(3.15)E[F∫0T∫R0h(t,z)Ñ(dt,dz)]=E[∫0T∫R0h(t,z)Dt,zFdtν(dz)].


### 3.3. Maximum Principles

Consider (3.1). We will make the following assumptions throughout this subsection.(H.1) The functionsb:[0,T]×[0,T]×R×R×Ω→R, σ:[0,T]×[0,T]×R×R×Ω→R, θ:[0,T]×[0,T]×R×R×R0×Ω→R, f:[0,T]×R×R×Ω→R, and g:R×Ω→R are continuously differentiable with respect to x∈R and u∈R.(H.2) For allt∈(0,T) and all bounded 𝒢t-measurable random variables α the control(3.16)βα(s)=αχ[t,T](s)belongs to𝒜𝒢.(H.3) For allu,β∈𝒜𝒢 with β bounded, there exists δ>0 such that(3.17)u+yβ∈𝒜𝒢∀y∈(-δ,δ).(H.4) For allu,β∈𝒜𝒢 with β bounded, the process Yβ(t)=(d/dy)X(u+yβ)(t)|y=0 exists and satisfies the following equation:
(3.18)Yβ(t)=∫0t∂b∂x(t,s,Xu(s),u(s))Yβ(s)ds+∫0t∂b∂u(t,s,Xu(s),u(s))β(s)ds+∫0t∂σ∂x(t,s,Xu(s),u(s))Yβ(s)dB(s)+∫0t∂σ∂u(t,s,Xu(s),u(s))β(s)dB(s)+∫0t∫R0∂θ∂x(t,s,Xu(s),u(s),z)Yβ(s)Ñ(ds,dz)+∫0t∫R0∂θ∂u(t,s,Xu(s),u(s),z)β(s)Ñ(ds,dz).(H.5) For allu∈𝒜𝒢, the Malliavin derivatives Dt(g′(Xu(T))) and Dt,z(g′(Xu(T))) exist.In the sequel, we omit the random parameterω for simplicity. Let J(u) be defined as in (3.2).(H.6) The functions(∂b/∂u)(t,s,x,u)2, (∂b/∂x)(t,s,x,u)2, (∂σ/∂u)(t,s,x,u)2, (∂σ/∂x)(t,s,x,u)2, and ∫R0(∂θ/∂u)‍(t,s,x,u,z)2ν(dz), ∫R0(∂θ/∂x)(t,s,x,u,z)2ν(dz) are bounded on [0,T]×[0,T]×R×R×Ω.Theorem 3.1 (Maximum principle I for optimal control of stochastic Volterra equations).
(1) Suppose thatû is a critical point for J(u) in the sense that (d/dy)J(û+yβ)|y=0=0 for all bounded β∈𝒜𝒢. Then(3.19)E[{∫tT∂f∂x(s,X̂(s),û(s))Λ(s,t)ds+∫tT∂f∂u(s,X̂(s),û(s))ds+∫tT∂b∂x(T,s,X̂(s),û(s))Λ(s,t)g′(X̂(T))ds+∫tT∂b∂u(T,s,X̂(s),û(s))g′(X̂(T))ds+∫tT∂σ∂x(T,s,X̂(s),û(s))Λ(s,t)Ds(g′(X̂(T)))ds+∫tT∂σ∂u(T,s,X̂(s),û(s))Ds(g′(X̂(T)))ds+∫tT(∫R0∂θ∂x(T,s,X̂(s),û(s),z)Λ(s,t)Ds,z(g′(X̂(T)))ν(dz))ds+∫tT(∫R0∂θ∂u(T,s,X̂(s),û(s),z)Ds,z(g′(X̂(T)))ν(dz))ds}∣𝒢t]=0,
where Λ(s,t) is defined in (3.29) below and X̂=Xû.
(2) Conversely, supposeû∈𝒜𝒢 such that (3.19) holds. Then û is a critical point for J(·).Proof.
(1) Suppose thatû is a critical point for J(u). Let β∈𝒜𝒢 be bounded. Write X̂=Xû. Then
(3.20)0=ddyJ(û+yβ)|y=0=E[∫0T{∂f∂x(t,X̂(t),û(t))Yβ(t)+∂f∂u(t,X̂(t),û(t))β(t)}dt+g′(X̂(T))Yβ(T)∫0T],
where
(3.21)Yβ(t)=ddyX(û+yβ)(t)|y=0=∫0t∂b∂x(t,s,X̂(s),û(s))Yβ(s)ds+∫0t∂b∂u(t,s,X̂(s),û(s))β(s)ds+∫0t∂σ∂x(t,s,X̂(s),û(s))Yβ(s)dB(s)+∫0t∂σ∂u(t,s,X̂(s),û(s))β(s)dB(s)+∫0t∫R0∂θ∂x(t,s,X̂(s),û(s),z)Yβ(s)Ñ(ds,dz)+∫0t∫R0∂θ∂u(t,s,X̂(s),û(s),z)β(s)Ñ(ds,dz).
By the duality formulae (3.9) and (3.15), we have
(3.22)E[g′(X̂(T))Yβ(T)]=E[∫0T∂b∂x(T,s,X̂(s),û(s))Yβ(s)g′(X̂(T))ds]+E[∫0T∂b∂u(T,s,X̂(s),û(s))β(s)g′(X̂(T))ds]+E[(∫0T∂σ∂x(T,s,X̂(s),û(s))Yβ(s)dB(s))g′(X̂(T))]+E[(∫0T∂σ∂u(T,s,X̂(s),û(s))β(s)dB(s))g′(X̂(T))]+E[(∫0T∫R0∂θ∂x(T,s,X̂(s),û(s),z)Yβ(s)Ñ(ds,dz))g′(X̂(T))]+E[(∫0T∫R0∂θ∂u(T,s,X̂(s),û(s),z)β(s)Ñ(ds,dz))g′(X̂(T))]=E[∫0T∂b∂x(T,s,X̂(s),û(s))Yβ(s)g′(X̂(T))ds]+E[∫0T∂b∂u(T,s,X̂(s),û(s))β(s)g′(X̂(T))ds]+E[∫0T∂σ∂x(T,s,X̂(s),û(s))Yβ(s)Ds(g′(X̂(T)))ds]+E[∫0T∂σ∂u(T,s,X̂(s),û(s))β(s)Ds(g′(X̂(T)))ds]+E[(∫0T∫R0∂θ∂x(T,s,X̂(s),û(s),z)Yβ(s)Ds,z(g′(X̂(T)))ν(dz)ds)]+E[(∫0T∫R0∂θ∂u(T,s,X̂(s),û(s),z)β(s)Ds,z(g′(X̂(T)))ν(dz)ds)].
Let α be bounded, 𝒢t measurable. Choose βα(s)=αχ[t,T](s) and substitute (3.22) into (3.20) to obtain
(3.23)E[∫tT{∂f∂x(s,X̂(s),û(s))Yβα(s)ds+α∫tT∂f∂u(s,X̂(s),û(s))ds}]+E[∫tT∂b∂x(T,s,X̂(s),û(s))Yβα(s)g′(X̂(T))ds]+E[α∫tT∂b∂u(T,s,X̂(s),û(s))g′(X̂(T))ds]+E[∫tT∂σ∂x(T,s,X̂(s),û(s))Yβα(s)Ds(g′(X̂(T)))ds]+E[α∫tT∂σ∂u(T,s,X̂(s),û(s))Ds(g′(X̂(T)))ds]+E[(∫tT∫R0∂θ∂x(T,s,X̂(s),û(s),z)Yβ(s)Ds,z(g′(X̂(T)))ν(dz)ds)]+E[α∫tT∫R0∂θ∂u(T,s,X̂(s),û(s),z)Ds,z(g′(X̂(T)))ν(dz)ds]=0,
where Yβα(l)=0 for l≤t, and for l≥t,
(3.24)Yβα(l)=∫tl∂b∂x(l,s,X̂(s),û(s))Yβα(s)ds+α∫tl∂b∂u(l,s,X̂(s),û(s))ds+∫tl∂σ∂x(l,s,X̂(s),û(s))Yβα(s)dB(s)+α∫tl∂σ∂u(l,s,X̂(s),û(s))dB(s)+∫tl∫R0∂θ∂x(l,s,X̂(s),û(s),z)Yβ(s)Ñ(ds,dz)+α∫tl∫R0∂θ∂u(l,s,X̂(s),û(s),z)Ñ(ds,dz).
For l≥s, put
(3.25)dΓ(l,s):=dsΓ(l,s)=∂b∂x(l,s,X̂(s),û(s))ds+∂σ∂x(l,s,X̂(s),û(s))dB(s)+∫R0∂θ∂x(l,s,X̂(s),û(s),z)Ñ(ds,dz).
This means that for a predictable process h(s), we have
(3.26)∫tlh(s)dΓ(l,s)=∫tl∂b∂x(l,s,X̂(s),û(s))h(s)ds+∫tl∂σ∂x(l,s,X̂(s),û(s))h(s)dB(s)+∫tl∫R0∂θ∂x(l,s,X̂(s),û(s),z)h(s)Ñ(ds,dz).
Set
(3.27)D(l,t)=∫tl∂b∂u(l,s,X̂(s),û(s))ds+∫tl∂σ∂u(l,s,X̂(s),û(s))dB(s)+∫tl∫R0∂θ∂u(l,s,X̂(s),û(s),z)Ñ(ds,dz).
Repeatedly using the linear equation (3.24), as in the proof of (2.23), we obtain
(3.28)Yβα(l)=αΛ(l,t),
where
(3.29)Λ(l,t)=D(l,t)+∑k=1∞∫tldΓ(l,s1)∫ts1dΓ(s1,s2)⋯∫tsk-1D(sk,t)dΓ(sk-1,sk).
As in the proof of Lemma 2.2, we can check that the above series converges in L1(Ω) under the assumption (H.6). We substitute (3.28) into (3.23) to get
(3.30)E[α{∫tT∂f∂x(s,X̂(s),û(s))Λ(s,t)ds+∫tT∂f∂u(s,X̂(s),û(s))ds+∫tT∂b∂x(T,s,X̂(s),û(s))Λ(s,t)g′(X̂(T))ds+∫tT∂b∂u(T,s,X̂(s),û(s))g′(X̂(T))ds+∫tT∂σ∂x(T,s,X̂(s),û(s))Λ(s,t)Ds(g′(X̂(T)))ds+∫tT∂σ∂u(T,s,X̂(s),û(s))Ds(g′(X̂(T)))ds+∫tT∫R0∂θ∂x(T,s,X̂(s),û(s),z)Λ(s,t)Ds,z(g′(X̂(T)))ν(dz)ds+∫tT∫R0∂θ∂u(T,s,X̂(s),û(s),z)Ds,z(g′(X̂(T)))ν(dz)ds}]=0.
Since α is arbitrary, it follows that
(3.31)E[∫tT∂f∂x(s,X̂(s),û(s))Λ(s,t)ds+∫tT∂f∂u(s,X̂(s),û(s))ds+∫tT∂b∂x(T,s,X̂(s),û(s))Λ(s,t)g′(X̂(T))ds+∫tT∂b∂u(T,s,X̂(s),û(s))g′(X̂(T))ds+∫tT∂σ∂x(T,s,X̂(s),û(s))Λ(s,t)Ds(g′(X̂(T)))ds+∫tT∂σ∂u(T,s,X̂(s),û(s))Ds(g′(X̂(T)))ds+∫tT∫R0∂θ∂x(T,s,X̂(s),û(s),z)Λ(s,t)Ds,z(g′(X̂(T)))ν(dz)ds+∫tT∫R0∂θ∂u(T,s,X̂(s),û(s),z)Ds,z(g′(X̂(T)))ν(dz)ds∣𝒢t]=0,
completing the proof of (1).
(2) Suppose that (3.19) holds for some û∈𝒜𝒢. Running the arguments in the proof of (1) backwards, we see that (3.20) holds for all bounded β∈𝒜𝒢 of the form β(s)=αχ[t,T](s). This is sufficient because the set of linear combinations of such β is dense in 𝒜𝒢.Next we consider the case where the coefficients are independent ofx. The maximum principle will be simplified significantly. Fix a control û∈𝒜𝒢 with corresponding state process X̂(t). Define the associated Hamiltonian process H(t,u) by
(3.32)H(t,u)=f(t,u)+b(T,t,u)g′(X̂(T))+σ(T,t,u)Dt(g′(X̂(T)))+∫R0θ(T,t,u,z)Dt,z(g′(X̂(T)))ν(dz);t∈[0,T],u∈R.Theorem 3.2 (Maximum principle II for optimal control of stochastic Volterra equations).
Suppose thatf,b,σ,θ are all independent of x. Then the followings are equivalent.(i)
û is a critical point for J(u).(ii)
For eacht∈[0,T], u=û(t) is a critical point for u→E[H(t,u)∣𝒢t], in the sense that
(3.33)∂∂uE[H(t,u)∣𝒢t]u=û(t)=0.Proof.
Suppose thatf,b,σ,θ are all independent of x. Then (3.19) reduces to
(3.34)E[∫vT∂f∂u(s,û(s))ds+∫vT∂b∂u(T,s,û(s))g′(X̂(T))ds+∫vT∂σ∂u(T,s,û(s))Ds(g′(X̂(T)))ds+∫vT∫R0∂θ∂u(T,s,û(s),z)Ds,z(g′(X̂(T)))ν(dz)ds∣𝒢v]=0∀v∈[0,T].
By inserting 𝒢t we deduce that for all v≥t,
(3.35)E[∫vT∂f∂u(s,û(s))ds+∫vT∂b∂u(T,s,û(s))g′(X̂(T))ds+∫vT∂σ∂u(T,s,û(s))Ds(g′(X̂(T)))ds+∫vT∫R0∂θ∂u(T,s,û(s),z)Ds,z(g′(X̂(T)))ν(dz)ds∣𝒢t]=0.
Taking the right derivative with respect to v at the point t we obtain (3.33).


## 3.1. Integration by Parts Formula forB(·)

In this subsection,ℱT=σ(B(s),0≤s≤T). Recall that the Wiener-Ito chaos expansion theorem states that any F∈L2(ℱT,P) admits the representation
(3.4)F=∑n=0∞In(fn)
for a unique sequence of symmetric deterministic function fn∈L2([0,T]×n) and
(3.5)In(fn)=n!∫0T∫0tn⋯∫0t2fn(t1,…,tn)dB(t1)dB(t2)⋯dB(tn).
Moreover, the following isometry holds:
(3.6)E[F2]=∑n=0∞n!∥fn∥L2([0,T]×n)2.
Let D1,2 be the space of all F∈L2(ℱT,P) such that its chaos expansion (3.4) satisfies
(3.7)∥F∥D1,22:=∑n=0∞nn!∥fn∥L2([0,T]×n)2<∞.
For F∈D1,2 and t∈[0,T], the Malliavin derivative of F, DtF, is defined by
(3.8)DtF=∑n=0∞nIn-1(fn(·,t)),
where In-1(fn(·,t)) is the n-1 times iterated integral to the first n-1 variables of fn keeping the last variable tn=t as a parameter. We need the following result.Theorem A (Integration by parts formula (duality formula) forB(·)).
Suppose thath(t) is ℱt-adapted with E[∫0Th2(t)dt]<∞ and let F∈D1,2. Then
(3.9)E[F∫0Th(t)dB(t)]=E[∫0Th(t)DtFdt].


## 3.2. Integration by Parts Formula forÑ

In this sectionℱT=σ(η(s),0≤s≤T), where η(s)=∫0s∫R0zÑ(dr,dz). Recall that the Wiener-Ito chaos expansion theorem states that any F∈L2(ℱT,P) admits the representation
(3.10)F=∑n=0∞In(fn)
for a unique sequence of functions fn∈L̂2((dt×ν)n), where L̂2((dt×ν)n) is the space of functions fn(t1,z1,…,tn,zn);ti∈[0,T], zi∈R0 such that fn∈L2((dt×ν)n) and fn is symmetric with respect to the pairs of variables (t1,z1),(t2,z2),…,(tn,zn). Here In(fn) is the iterated integral:
(3.11)In(fn)=n!∫0T∫R0∫0tn∫R0⋯∫0t2∫R0fn(t1,z1,…,tn,zn)Ñ(dt1,dz1)⋯Ñ(dtn,dzn).
Moreover, the following isometry holds:
(3.12)E[F2]=∑n=0∞n!∥fn∥L2((dt×ν)n)2.
Let D̃1,2 be the space of all F∈L2(ℱT,P) such that its chaos expansion (3.18) satisfies
(3.13)∥F∥D̃1,22:=∑n=0∞nn!∥fn∥L2((dt×ν)n)2<∞.
For F∈D̃1,2 and t∈[0,T], the Malliavin derivative of F, Dt,zF, is defined by
(3.14)Dt,zF=∑n=0∞nIn-1(fn(·,t,z)),
where In-1(fn(·,t,z)) is the n-1 times iterated integral with respect to the first n-1 pairs of variables of fn keeping the last pair (tn,zn)=(t,z) as a parameter. We need the following resultTheorem B (Integration by parts formula (duality formula) forÑ).
Supposeh(t,z) is ℱt-predictable with E[∫0T∫R0h2(t,z)dtν(dz)]<∞ and let F∈D̃1,2. Then
(3.15)E[F∫0T∫R0h(t,z)Ñ(dt,dz)]=E[∫0T∫R0h(t,z)Dt,zFdtν(dz)].


## 3.3. Maximum Principles

Consider (3.1). We will make the following assumptions throughout this subsection.(H.1) The functionsb:[0,T]×[0,T]×R×R×Ω→R, σ:[0,T]×[0,T]×R×R×Ω→R, θ:[0,T]×[0,T]×R×R×R0×Ω→R, f:[0,T]×R×R×Ω→R, and g:R×Ω→R are continuously differentiable with respect to x∈R and u∈R.(H.2) For allt∈(0,T) and all bounded 𝒢t-measurable random variables α the control(3.16)βα(s)=αχ[t,T](s)belongs to𝒜𝒢.(H.3) For allu,β∈𝒜𝒢 with β bounded, there exists δ>0 such that(3.17)u+yβ∈𝒜𝒢∀y∈(-δ,δ).(H.4) For allu,β∈𝒜𝒢 with β bounded, the process Yβ(t)=(d/dy)X(u+yβ)(t)|y=0 exists and satisfies the following equation:
(3.18)Yβ(t)=∫0t∂b∂x(t,s,Xu(s),u(s))Yβ(s)ds+∫0t∂b∂u(t,s,Xu(s),u(s))β(s)ds+∫0t∂σ∂x(t,s,Xu(s),u(s))Yβ(s)dB(s)+∫0t∂σ∂u(t,s,Xu(s),u(s))β(s)dB(s)+∫0t∫R0∂θ∂x(t,s,Xu(s),u(s),z)Yβ(s)Ñ(ds,dz)+∫0t∫R0∂θ∂u(t,s,Xu(s),u(s),z)β(s)Ñ(ds,dz).(H.5) For allu∈𝒜𝒢, the Malliavin derivatives Dt(g′(Xu(T))) and Dt,z(g′(Xu(T))) exist.In the sequel, we omit the random parameterω for simplicity. Let J(u) be defined as in (3.2).(H.6) The functions(∂b/∂u)(t,s,x,u)2, (∂b/∂x)(t,s,x,u)2, (∂σ/∂u)(t,s,x,u)2, (∂σ/∂x)(t,s,x,u)2, and ∫R0(∂θ/∂u)‍(t,s,x,u,z)2ν(dz), ∫R0(∂θ/∂x)(t,s,x,u,z)2ν(dz) are bounded on [0,T]×[0,T]×R×R×Ω.Theorem 3.1 (Maximum principle I for optimal control of stochastic Volterra equations).
(1) Suppose thatû is a critical point for J(u) in the sense that (d/dy)J(û+yβ)|y=0=0 for all bounded β∈𝒜𝒢. Then(3.19)E[{∫tT∂f∂x(s,X̂(s),û(s))Λ(s,t)ds+∫tT∂f∂u(s,X̂(s),û(s))ds+∫tT∂b∂x(T,s,X̂(s),û(s))Λ(s,t)g′(X̂(T))ds+∫tT∂b∂u(T,s,X̂(s),û(s))g′(X̂(T))ds+∫tT∂σ∂x(T,s,X̂(s),û(s))Λ(s,t)Ds(g′(X̂(T)))ds+∫tT∂σ∂u(T,s,X̂(s),û(s))Ds(g′(X̂(T)))ds+∫tT(∫R0∂θ∂x(T,s,X̂(s),û(s),z)Λ(s,t)Ds,z(g′(X̂(T)))ν(dz))ds+∫tT(∫R0∂θ∂u(T,s,X̂(s),û(s),z)Ds,z(g′(X̂(T)))ν(dz))ds}∣𝒢t]=0,
where Λ(s,t) is defined in (3.29) below and X̂=Xû.
(2) Conversely, supposeû∈𝒜𝒢 such that (3.19) holds. Then û is a critical point for J(·).Proof.
(1) Suppose thatû is a critical point for J(u). Let β∈𝒜𝒢 be bounded. Write X̂=Xû. Then
(3.20)0=ddyJ(û+yβ)|y=0=E[∫0T{∂f∂x(t,X̂(t),û(t))Yβ(t)+∂f∂u(t,X̂(t),û(t))β(t)}dt+g′(X̂(T))Yβ(T)∫0T],
where
(3.21)Yβ(t)=ddyX(û+yβ)(t)|y=0=∫0t∂b∂x(t,s,X̂(s),û(s))Yβ(s)ds+∫0t∂b∂u(t,s,X̂(s),û(s))β(s)ds+∫0t∂σ∂x(t,s,X̂(s),û(s))Yβ(s)dB(s)+∫0t∂σ∂u(t,s,X̂(s),û(s))β(s)dB(s)+∫0t∫R0∂θ∂x(t,s,X̂(s),û(s),z)Yβ(s)Ñ(ds,dz)+∫0t∫R0∂θ∂u(t,s,X̂(s),û(s),z)β(s)Ñ(ds,dz).
By the duality formulae (3.9) and (3.15), we have
(3.22)E[g′(X̂(T))Yβ(T)]=E[∫0T∂b∂x(T,s,X̂(s),û(s))Yβ(s)g′(X̂(T))ds]+E[∫0T∂b∂u(T,s,X̂(s),û(s))β(s)g′(X̂(T))ds]+E[(∫0T∂σ∂x(T,s,X̂(s),û(s))Yβ(s)dB(s))g′(X̂(T))]+E[(∫0T∂σ∂u(T,s,X̂(s),û(s))β(s)dB(s))g′(X̂(T))]+E[(∫0T∫R0∂θ∂x(T,s,X̂(s),û(s),z)Yβ(s)Ñ(ds,dz))g′(X̂(T))]+E[(∫0T∫R0∂θ∂u(T,s,X̂(s),û(s),z)β(s)Ñ(ds,dz))g′(X̂(T))]=E[∫0T∂b∂x(T,s,X̂(s),û(s))Yβ(s)g′(X̂(T))ds]+E[∫0T∂b∂u(T,s,X̂(s),û(s))β(s)g′(X̂(T))ds]+E[∫0T∂σ∂x(T,s,X̂(s),û(s))Yβ(s)Ds(g′(X̂(T)))ds]+E[∫0T∂σ∂u(T,s,X̂(s),û(s))β(s)Ds(g′(X̂(T)))ds]+E[(∫0T∫R0∂θ∂x(T,s,X̂(s),û(s),z)Yβ(s)Ds,z(g′(X̂(T)))ν(dz)ds)]+E[(∫0T∫R0∂θ∂u(T,s,X̂(s),û(s),z)β(s)Ds,z(g′(X̂(T)))ν(dz)ds)].
Let α be bounded, 𝒢t measurable. Choose βα(s)=αχ[t,T](s) and substitute (3.22) into (3.20) to obtain
(3.23)E[∫tT{∂f∂x(s,X̂(s),û(s))Yβα(s)ds+α∫tT∂f∂u(s,X̂(s),û(s))ds}]+E[∫tT∂b∂x(T,s,X̂(s),û(s))Yβα(s)g′(X̂(T))ds]+E[α∫tT∂b∂u(T,s,X̂(s),û(s))g′(X̂(T))ds]+E[∫tT∂σ∂x(T,s,X̂(s),û(s))Yβα(s)Ds(g′(X̂(T)))ds]+E[α∫tT∂σ∂u(T,s,X̂(s),û(s))Ds(g′(X̂(T)))ds]+E[(∫tT∫R0∂θ∂x(T,s,X̂(s),û(s),z)Yβ(s)Ds,z(g′(X̂(T)))ν(dz)ds)]+E[α∫tT∫R0∂θ∂u(T,s,X̂(s),û(s),z)Ds,z(g′(X̂(T)))ν(dz)ds]=0,
where Yβα(l)=0 for l≤t, and for l≥t,
(3.24)Yβα(l)=∫tl∂b∂x(l,s,X̂(s),û(s))Yβα(s)ds+α∫tl∂b∂u(l,s,X̂(s),û(s))ds+∫tl∂σ∂x(l,s,X̂(s),û(s))Yβα(s)dB(s)+α∫tl∂σ∂u(l,s,X̂(s),û(s))dB(s)+∫tl∫R0∂θ∂x(l,s,X̂(s),û(s),z)Yβ(s)Ñ(ds,dz)+α∫tl∫R0∂θ∂u(l,s,X̂(s),û(s),z)Ñ(ds,dz).
For l≥s, put
(3.25)dΓ(l,s):=dsΓ(l,s)=∂b∂x(l,s,X̂(s),û(s))ds+∂σ∂x(l,s,X̂(s),û(s))dB(s)+∫R0∂θ∂x(l,s,X̂(s),û(s),z)Ñ(ds,dz).
This means that for a predictable process h(s), we have
(3.26)∫tlh(s)dΓ(l,s)=∫tl∂b∂x(l,s,X̂(s),û(s))h(s)ds+∫tl∂σ∂x(l,s,X̂(s),û(s))h(s)dB(s)+∫tl∫R0∂θ∂x(l,s,X̂(s),û(s),z)h(s)Ñ(ds,dz).
Set
(3.27)D(l,t)=∫tl∂b∂u(l,s,X̂(s),û(s))ds+∫tl∂σ∂u(l,s,X̂(s),û(s))dB(s)+∫tl∫R0∂θ∂u(l,s,X̂(s),û(s),z)Ñ(ds,dz).
Repeatedly using the linear equation (3.24), as in the proof of (2.23), we obtain
(3.28)Yβα(l)=αΛ(l,t),
where
(3.29)Λ(l,t)=D(l,t)+∑k=1∞∫tldΓ(l,s1)∫ts1dΓ(s1,s2)⋯∫tsk-1D(sk,t)dΓ(sk-1,sk).
As in the proof of Lemma 2.2, we can check that the above series converges in L1(Ω) under the assumption (H.6). We substitute (3.28) into (3.23) to get
(3.30)E[α{∫tT∂f∂x(s,X̂(s),û(s))Λ(s,t)ds+∫tT∂f∂u(s,X̂(s),û(s))ds+∫tT∂b∂x(T,s,X̂(s),û(s))Λ(s,t)g′(X̂(T))ds+∫tT∂b∂u(T,s,X̂(s),û(s))g′(X̂(T))ds+∫tT∂σ∂x(T,s,X̂(s),û(s))Λ(s,t)Ds(g′(X̂(T)))ds+∫tT∂σ∂u(T,s,X̂(s),û(s))Ds(g′(X̂(T)))ds+∫tT∫R0∂θ∂x(T,s,X̂(s),û(s),z)Λ(s,t)Ds,z(g′(X̂(T)))ν(dz)ds+∫tT∫R0∂θ∂u(T,s,X̂(s),û(s),z)Ds,z(g′(X̂(T)))ν(dz)ds}]=0.
Since α is arbitrary, it follows that
(3.31)E[∫tT∂f∂x(s,X̂(s),û(s))Λ(s,t)ds+∫tT∂f∂u(s,X̂(s),û(s))ds+∫tT∂b∂x(T,s,X̂(s),û(s))Λ(s,t)g′(X̂(T))ds+∫tT∂b∂u(T,s,X̂(s),û(s))g′(X̂(T))ds+∫tT∂σ∂x(T,s,X̂(s),û(s))Λ(s,t)Ds(g′(X̂(T)))ds+∫tT∂σ∂u(T,s,X̂(s),û(s))Ds(g′(X̂(T)))ds+∫tT∫R0∂θ∂x(T,s,X̂(s),û(s),z)Λ(s,t)Ds,z(g′(X̂(T)))ν(dz)ds+∫tT∫R0∂θ∂u(T,s,X̂(s),û(s),z)Ds,z(g′(X̂(T)))ν(dz)ds∣𝒢t]=0,
completing the proof of (1).
(2) Suppose that (3.19) holds for some û∈𝒜𝒢. Running the arguments in the proof of (1) backwards, we see that (3.20) holds for all bounded β∈𝒜𝒢 of the form β(s)=αχ[t,T](s). This is sufficient because the set of linear combinations of such β is dense in 𝒜𝒢.Next we consider the case where the coefficients are independent ofx. The maximum principle will be simplified significantly. Fix a control û∈𝒜𝒢 with corresponding state process X̂(t). Define the associated Hamiltonian process H(t,u) by
(3.32)H(t,u)=f(t,u)+b(T,t,u)g′(X̂(T))+σ(T,t,u)Dt(g′(X̂(T)))+∫R0θ(T,t,u,z)Dt,z(g′(X̂(T)))ν(dz);t∈[0,T],u∈R.Theorem 3.2 (Maximum principle II for optimal control of stochastic Volterra equations).
Suppose thatf,b,σ,θ are all independent of x. Then the followings are equivalent.(i)
û is a critical point for J(u).(ii)
For eacht∈[0,T], u=û(t) is a critical point for u→E[H(t,u)∣𝒢t], in the sense that
(3.33)∂∂uE[H(t,u)∣𝒢t]u=û(t)=0.Proof.
Suppose thatf,b,σ,θ are all independent of x. Then (3.19) reduces to
(3.34)E[∫vT∂f∂u(s,û(s))ds+∫vT∂b∂u(T,s,û(s))g′(X̂(T))ds+∫vT∂σ∂u(T,s,û(s))Ds(g′(X̂(T)))ds+∫vT∫R0∂θ∂u(T,s,û(s),z)Ds,z(g′(X̂(T)))ν(dz)ds∣𝒢v]=0∀v∈[0,T].
By inserting 𝒢t we deduce that for all v≥t,
(3.35)E[∫vT∂f∂u(s,û(s))ds+∫vT∂b∂u(T,s,û(s))g′(X̂(T))ds+∫vT∂σ∂u(T,s,û(s))Ds(g′(X̂(T)))ds+∫vT∫R0∂θ∂u(T,s,û(s),z)Ds,z(g′(X̂(T)))ν(dz)ds∣𝒢t]=0.
Taking the right derivative with respect to v at the point t we obtain (3.33).


## 4. Applications to Stochastic Delay Control

We now apply the general maximum principle for optimal control of Volterra equations to the stochastic delay problem (1.2)-(1.3) in the Introduction, by using the equivalence between (1.2) and (1.4). Note that in this case we have (see (3.1), (3.2) and compare with (1.2), (1.3))
(4.1)f(t,x,u)=U1(t,u),g(x)=U2(x),b(t,s,x,u)=K(t,s)u,σ(t,s,x,u)=Φ(t,s)C(s),Θ(t,s,x,u,z)=Φ(t,s)C2(s,z),ξ(t)=Φ(t,0)η(0)+∫-h0Φ(t,s+h)A2(s+h)η(s)ds+∫-h0(∫0hΦ(t,τ)A0(τ,s)dτ)η(s)ds.Hence the system (1.4) satisfies the conditions of Theorem 3.2. By (3.32) we get the Hamiltonian (4.2)H(t,u)=U1(t,u)+K(T,t)uU2′(X̂(T))+Φ(T,t)C(t)Dt(g′(X̂(T)))+∫R0Φ(T,t)C2(t,z)Dt,z(U2′(X̂(T)))ν(dz).Therefore by Theorem3.2 we get the following condition for an optimal harvesting rate û(t):
(4.3)E[U1′(t,û(t),ω)+K(T,t)U2′(X̂(T),ω)∣𝒢t]=0,
where X̂(T)=Xû(T) and Ui′=(∂/∂x)Ui;i=1,2.Now suppose thatU1 and U2 are stochastic utilities of the form
(4.4)U1(t,u,ω)=γt(ω)Ũ1(t,u),ω∈Ω,(4.5)U2(x,ω)=ζ(ω)Ũ2(x),ω∈Ω,
where γt(ω)>0 is ℱt-adapted, ζ(ω) is ℱT-measurable, and Ũ1,Ũ2 are concave, C1-functions on (0,∞) and R, respectively. The (4.3) simplifies to
(4.6)Ũ1′(t,û(t))E[γt𝒢t]=-K(T,t)E[ζŨ2′(X̂(T))𝒢t].
This gives a relation between the optimal control û(t) and the corresponding optimal terminal wealth X̂(T). In particular, if
(4.7)Ũ2(x)=x,
we get
(4.8)Ũ1′(t,û(t))=-K(T,t)E[ζ∣𝒢t]E[γt∣𝒢t].
We have proved the following.Corollary 4.1.
The optimal consumption rateû(t) for the stochastic delay system (1.2), (4.4), (4.5), (4.7) and the performance functional
(4.9)J(u)=E[∫0Tγt(ω)Ũ1(t,u(t))dt+ζ(ω)Xu(T)]
with partial information 𝒢t is given by (4.8).



---

*Source: 329185-2010-06-29.xml*