# Maximum Principle of Discrete Stochastic Control System Driven by Both Fractional Noise and White Noise

Authors: Yuecai Han; Zheng Li
Journal: Discrete Dynamics in Nature and Society
 (2020)

---

## Abstract

In this paper, we investigate the necessary optimality conditions of the discrete stochastic optimal control problems driven by both fractional noise and white noise. Here, the admissible control region is not necessarily convex. The corresponding variational inequalities are obtained by applying the classical variation method and Malliavin calculus. We also apply the stochastic maximum principle to a linear-quadratic optimal control problem to illustrate the main result.


---

## Body

## 1. Introduction

We consider a stochastic control problem for state process driven by both general white noise and fractional noise with Hurst parameterH∈1/2,1. More precisely, the state of the system is described as the following stochastic difference equation:(1)xtk+1=fxtk,vtk+bxtk,vtkgtk+σxtk,vtkωtk,k=0,1,…,N−1,xt0=x0∈ℝn,where the functions f,b, and σ and control variables utk are introduced in Section 2. The cost functional is defined as follows:(2)Jv=E∑k=0N−1lxtk,vtk+hxtN,where the functions l and h are also introduced in Section 2.Optimal control problems have a variety of applications in fields such as engineering, financial mathematics, and physics. The maximum principle is one of the main contents of modern control theory. As a necessary condition of the deterministic optimal control, it was formulated by Pontryagin and his group [1]. It states that any optimal control along with the optimal state trajectory must solve a Hamiltonian system, which is a two-point boundary value problem, plus a maximum condition of Hamiltonian. The theory was then developed extensively, and different versions of the maximum principle were derived.With the development of the optimal control theory, some researchers began to work on the discrete case by following the Pontryagin maximum principle for continuous optimal control problems. However, the fact has been verified that the discrete case was unlike the continuous case. By imposing convexity requirement, some researchers [2, 3] gave a derivation of the discrete maximum principle. A discrete optimization problem without assumptions of convexity and smoothness was shown by Mardanov et al. [4]. Taking into account the specific character of the discrete system, they obtained a necessary optimality condition which is not formulated in terms of the Hamilton–Pontryagin function.Naturally, with the emergence of stochastic problems, more and more researchers extend the maximum principle to the stochastic case. Kushner [5] employed the spike variation and Neustadt’s variational principle [6] to derive a stochastic maximum principle. Based on Girsanov transformations, Haussmann [7] extensively investigated the necessary conditions of stochastic optimal state feedback controls for systems with nondegenerate diffusion coefficients. Bismut [8] derived the adjoint equation via the martingale representation theorem. Peng [9] first considered the second-order term in the “Taylor expansion” of the variation and obtained a stochastic maximum principle for systems that are possibly degenerate, with control-dependent diffusions and not necessarily convex regions. The form of his maximum principle is quite different from the deterministic ones and reflects the stochastic nature of the problem. With the development of the fractional calculus, Han et al. [10] obtained a maximum principle for the stochastic control problem of general controlled stochastic differential systems driven by fractional Brownian motions (of Hurst parameter H>1/2), and the maximum principle involves Malliavin derivatives.However, as far as the discrete stochastic system, some results for the maximum principle are analogous to the deterministic systems, which are based on the Lagrange multiplier method [11]. Recently, Lin and Zhang [12] developed a maximum principle for optimal control of discrete-time stochastic systems, and the admissible control region was nonconvex. It can be found that, up to now, the existing results appearing on the stochastic discrete version mostly study the systems with general multiplicative noise. Inspiring from this, we study the maximum principle of discrete stochastic systems driven by both fractional noise and white noise by using the classical variational method and Malliavin calculus. The admissible control region is nonconvex.The rest of this paper is organized as follows. In Section2, we introduce some preliminaries and main assumptions needed to study the discrete stochastic control problem driven by both fractional noise and white noise. In Section 3, we derive the necessary conditions that the optimal control should satisfy. In Section 4, an example is given to illustrate the main results. In Section 5, we summarize the methods used and the results obtained.


## 2. Preliminaries

LetWt=W1t,W2t,…,WdtT,0≤t≤T, be a d-dimensional standard Brownian motion. Let(3)ZHt,s=κHtsH−1/2t−sH−1/2−H−12s1/2−H∫stuH−3/2u−sH−1/2du,with(4)κH=2H3/2−HΓH+1/2Γ2−2H,and define(5)BjHt=∫0tZHt,sdWjs,0≤t≤∞.Then,BHt=B1Ht,…,BdHtT, 0≤t≤T, is a d-dimensional fractional Brownian motion.Recall the operatorsBH,T∗ (see Eq. (5.35) of Hu [13]):(6)BH,T∗ft=−2Hκ1κHt1/2−HuH−1/2fudu,0≤t≤T,where(7)κ1=12HΓH−1/2Γ3/2−H.Letξ1,ξ2,…,ξk,… be an orthonormal basis of L20,T such that ξk, k=1,2,…, are smooth functions on 0,T. Let p be the set of all polynomials of the standard Brownian motions W over interval 0,T, namely, p contains all elements of the form(8)Fω=f∫0Tξ1tdBt,…,∫0TξntdBt,where f is a polynomial of n variables. If F is of the above form, then its Malliavin derivative DsF is defined as(9)DsF=∑i=1n∂f∂xi∫0Tξ1tdBt,…,∫0TξntdBtξis,0≤s≤T.For anyF∈p, we denote the following norm:(10)F1,p≔Fp+∑l=1dE∫0TDtlF2dtp/21/p.LetD1,p denote the Banach space obtained by completing p under the norm ⋅1,p.Letη1,η2,…,ηk,… be an orthonormal basis of L20,T such that ηk, k=1,2,…, are smooth functions on 0,T. Let pH be the set of all polynomials of the fractional Brownian motions W over interval 0,T, namely, pH contains all elements of the form(11)Gω=g∫0Tη1tdBtH,…,∫0TηntdBtH,where g is a polynomial of n variables. We define its Malliavin derivative by(12)DsHF=∑j=1n∂g∂xi∫0Tη1tdBtH,…,∫0TηntdBtHηjs,0≤s≤T.Similarly, we define⋅H,1,p and DH,1,p.The following duality formula will be used later to solve stochastic optimal control problems (15), (16), and (19).Lemma 1.
(Theorem 6.23 of [13]). Let f:0,T⊗Ω,ℱ,PH⟶ℝ be jointly measurable, and let G∈DH,1,2. Then,(13)E∫0TftdBtHG=∫0TEftDtHGdt.LetΩ,ℱ,P be a given complete probability space. Let ℱkk=0N⊂ℱ be the σ-field generated by Wt0,Wt1,…,WtN−1 and ℱ−1=∅,Ω, where Wtkk=0,1,2,…,N−1 is a sequence of the d-dimension Brownian motion.Similarly, letBtkHk=0,1,2,…,N−1 be a sequence of the d-dimension fractional Brownian motion that satisfies the following conditions, where 1/2<H<1:(i)
BtkHk=0,1,2,…,N is ℱk−1-measurable(ii)
The increments ofBtkH are stationary and need not be independent(iii)
For everyBtkH=BtkH,1,BtkH,2,…,BtkH,d⊤,BtkH,1,BtkH,2,…,BtkH,d are independent ℝ-valued Gaussian random variables(iv)
EBtkH=0, EBtiH,lBtjH,l=1/2ti2H+tj2H−ti−tj2H, where l=1,2,…,dLetωtk=Wtk+1−Wtk be the white noise and gtk=Btk+1H−BtkH be the fractional noise.LetD be a bounded domain, and the space of admissible controls is defined as(14)Uad=Ukk=0N−1≜vtk∈Uk∈ℝn:Ω⟶Dvtkisℱk−1−measurable andEvtk⊤vtk<+∞,k=0,1,…,N−1.For the arbitrary bounded random variablev and sufficient small ε>0, we define v¯=utm∗+εv for some m∈1,2,…,N−1. Let the admissible control utkε=1−δkmutk∗+δkmv¯, and we can rewrite utkε as utkε=utk∗+δkmεv, where δkm is the Kronecker delta, i.e., δkm=1 when k=m and δkm=0 when k≠m.The controlled stochastic system is described as the following discrete stochastic difference equation:(15)xtk+1=fxtk,vtk+bxtk,vtkgtk+σxtk,vtkωtk,k=0,1,…,N−1,xt0=x0∈ℝn.The cost functional is(16)Jv=E∑k=0N−1lxtk,vtk+hxtN.Assume thatf,b,σ,l, and h satisfy the following conditions:(17)supx∈ℝn,u∈ℝνfxxtk,utk+fuxtk,utk+∑j=1sbxjxtk,utk+∑j=1sbujxtk,utk+∑j=1dσxjxtk,utk+∑j=1dσujxtk,utk≤C.(H1) The functionf:ℝn×ℝn⟶ℝn,b=b1,b2,…,bd:ℝn×ℝn⟶ℝd and σ=σ1,σ2,…,σd:ℝn×ℝn⟶ℝd are continuous and differentiable with respect to variables and u.(H2) Assume that there exists a constantC>0 such that(H3) The functionl:ℝn×ℝn⟶ℝ and h:ℝn⟶ℝ are continuous and differentiable with bounded derivatives.According to equation (15) and the definition of admissible controls, u and x∈ℝn. Here, all derivations are for vectors. We have(18)fx=∂fx,u∂x,fu=∂fx,u∂u,bx=∂bx,u∂x,bu=∂bx,u∂u,σx=∂σx,u∂x,σu=∂σx,u∂u,where fx,fu∈Rn×n and bx,σx,bu,σu∈ℝn×d.Our stochastic optimal control problem is to minimize the cost functionalJv over, namely, to find the optimal u∗∈Uad satisfying(19)Ju∗=infuε∈UadJuε.


## 3. The Maximum Principle

We have the following theorem as the main result of this paper.Theorem 1.
Let the assumptions (H1–H3) hold. Ifxtk∗,utk∗ is a solution to optimal control problems (15), (16), and (19) and satisfies the following equation,(20)ϕtk+1=ϕtkfxxtk∗,utk∗+ϕtkbxxtk∗,utk∗gtk+ϕtkσxxtk∗,utk∗ωtk,ϕt0=In×n,where fx,bx, and σx are defined above, then there exists the following general maximum principle:(21)lxxtm∗,utm∗ϕtm∑n=0m−1ϕtn+1−1fuxtn∗,utn∗+DtmHlxxtm∗,utm∗ϕtm∑n=0m−1ϕtn+1−1buxtn∗,utn∗+Dtmlxxtm∗,utm∗ϕtm∑n=0m−1ϕtn+1−1σuxtn∗,utn∗+luxtm∗,utm∗+hxxtN∗ϕtNϕtm+1−1fuxtm∗,utm∗+DtNHhxxtN∗ϕtNϕtm+1−1buxtm∗,utm∗+DtNhxxtN∗ϕtNϕtm+1−1σuxtm∗,utm∗=0.In order to prove Theorem1, we begin with the estimation of the first-order variational equation of the state variables.Lemma 2.
Let the assumptions (H1–H3) hold. Then,(22)sup0≤k≤N−1Extkε−xtk∗−ytk∗2≤Cε2v2,where ytk∗ is the solution of(23)ytk+1∗=fxxtk∗,utk∗ytk∗+δkmfuxtk∗,utk∗v+bxxtk∗,utk∗ytk∗+δkmbuxtk∗,utk∗vgtk+σxxtk∗,utk∗ytk∗+δkmσuxtk∗,utk∗vωtk,yt0∗=0.Proof.
Let(24)ytk+1ε=xtk+1ε−xtk+1∗=fxtkε,utkε−fxtk∗,utk∗+bxtkε,utkε−bxtk∗,utk∗gtk+σxtkε,utkε−σxtk∗,utk∗ωtk=fxtkε,utkε−fxtk∗,utkε+fxtk∗,utkε−fxtk∗,utk∗+bxtkε,utkε−bxtk∗,utkε+bxtk∗,utkε−bxtk∗,utk∗gtk+σxtkε,utkε−σxtk∗,utkε+σxtk∗,utkε−σxtk∗,utk∗ωtk=fxxtk∗,utkεytkε+δkmfuxtk∗,utk∗εv+bxxtk∗,utkεytkε+δkmbuxtk∗,utk∗εvgtk+σxxtk∗,utkεytkε+δkmσuxtk∗,utk∗εvωtk.
Under assumption (H2), we have the following moment inequality:(25)sup0≤k≤N−1Eytkε2≤Cε2v2.
In fact,(26)Eytk+1ε2≤3Efxxtk∗,utkεytkε+δkmfuxtk∗,utk∗εv2+bxxtk∗,utkεytkε+δkmbuxtk∗,utk∗εv2Btk+1−Btk2d+σxxtk∗,utkεytkε+δkmσuxtk∗,utk∗εv2tk+1−tkd=3Efx2xtk∗,utkε+bx2xtk∗,utkεtk+1−tk2Hd+σx2xtk∗,utkεtk+1−tkdytkε2+2δkm2fxxtk∗,utkεfuxtk∗,utk∗+2bxxtk∗,utkεbuxtk∗,utk∗tk+1−tk2Hd+2σxxtk∗,utkεσuxtk∗,utk∗tk+1−tkdεvytkε+2δkmfu2xtk∗,utk∗+bu2xtk∗,utk∗tk+1−tk2Hd+σu2xtk∗,utk∗tk+1−tkdε2v2≤Cytkε2+Cytkεεv+Cε2v2.
We also haveEyt1ε2≤Cε2v2. So, by repeating this step, moment inequality (25) is verified. Let ytk∗=limε⟶0ytkε/ε; we have(27)ytk+1∗=fxxtk∗,utk∗ytk∗+δkmfuxtk∗,utk∗v+bxxtk∗,utk∗ytk∗+δkmbuxtk∗,utk∗vgtk+σxxtk∗,utk∗ytk∗+δkmσuxtk∗,utk∗vωtk,yt0∗=0.
In order to prove inequality (22), we consider(28)fxtk∗+ytk∗,utkε+bxtk∗+ytk∗,utkεgtk+σxtk∗+ytk∗,utkεωtk=fxtk∗,utkε+fxxtk∗,utkεytk∗+bxtk∗,utkε+bxxtk∗,utkεytk∗gtk+σxtk∗,utkε+σxxtk∗,utkεytk∗ωtk=fxtk∗,utk∗+bxtk∗,utk∗gtk+σxtk∗,utk∗ωtk+fxxtk∗,utk∗ytk∗+bxxtk∗,utk∗ytk∗gtk+σxxtk∗,utk∗ytk∗ωtk+δkmfuxtk∗,utk∗v+δkmbuxtk∗,utk∗vgtk+δkmσuxtk∗,utk∗vωtk+fxxtk∗,utkε+fxxtk∗,utk∗−bxxtk∗,utkε−bxxtk∗,utk∗gtk+σxxtk∗,utkε−σxxtk∗,utk∗ωtk=xtk+1∗+ytk+1∗+fxxtk∗,utkε−fxxtk∗,utk∗+bxxtk∗,utkε−bxxtk∗,utk∗gtk+σxxtk∗,utkε−σxxtk∗,utk∗ωtk.
According to (H2), we have that(29)sup0≤k≤N−1Efxxtk∗,utkε−fxxtk∗,utk∗2+bxxtk∗,utkε−bxxtk∗,utk∗ωtk2+σxxtk∗,utkε−σxxtk∗,utk∗gtk2≤Cε2v2.
Notice that(30)xtk+1∗+ytk+1∗=fxtk∗+ytk∗,utkε+bxtk∗+ytk∗,utkεgtk+σxtk∗+ytk∗,utkεωtk−fxxtk∗,utkε−fxxtk∗,utk∗−bxxtk∗,utkε−bxxtk∗,utk∗gtk−σxxtk∗,utkε−σxxtk∗,utk∗ωtk,xtk+1ε=fxtkε,utkε+bxtkε,utkεgtk+σxtkε,utkεωtk.
It is easy to obtain(31)xtk+1ε−xtk+1∗−ytk+1∗=Aεxtkε−xtk∗−ytk∗+Bεxtkε−xtk∗−ytk∗ωtk+Cεxtkε−xtk∗−ytk∗gtk+fxxtk∗,utkε−fxxtk∗,utk∗+bxxtk∗,utkε−bxxtk∗,utk∗gtk+σxxtk∗,utkε−σxxtk∗,utk∗ωtk,with(32)Aε=fxxtkε−θxtk∗+ytk∗,utkε,θ∈0,1,Bε=bxxtkε−λxtk∗+ytk∗,utkε,λ∈0,1,Cε=σxxtkε−ρxtk∗+ytk∗,utkε,ρ∈0,1,which are uniformly bounded according to our assumptions. Through the above derivation, Lemma 2 is proved.The estimation of the variational equation of the state equation is a critical point to obtain the maximum principle, and we need to solve the above stochastic difference equation.Lemma 3.
There exists a unique bounded solution to the following linear matrix-valued stochastic difference equations:(33)ϕtk+1=ϕtkfxxtk∗,utk∗+ϕtkbxxtk∗,utk∗gtk+ϕtkσxxtk∗,utk∗ωtk,ϕt0=In×n.Proof.
For the uniqueness of the solution to (33), assume that there exists another solution ϕ′. We have(34)Eϕtk−ϕtk′2=Eϕtk−1−ϕtk−1′2fx2xtk−1∗,utk−1∗+bx2xtk−1∗,utk−1∗tk−tk−12Hd+σx2xtk−1∗,utk−1∗tk−tk−1d≤CEϕtk−1−ϕtk−1′2.
Whenk=1, we have Eϕt1−ϕt1′2=0. By the inductive method, the uniqueness of the solution to the equations is obtained. For the boundedness of the solution, it is easy to get(35)Eϕtk2=Eϕtkϕt0−12=Eϕtkϕtk−1−1⋯ϕt1ϕt0−12=Efx2xtk−1∗,utk−1∗+bx2xtk−1∗,utk−1∗tk−tk−12Hd+σx2xtk−1∗,utk−1∗tk−tk−1d⋯fx2xt0∗,ut0∗+bx2xt0∗,ut0∗t1−t02Hd+σx2xt0∗,ut0∗t1−t0d≤C.According to Lemma3, we express y∗ in an implicit form of v as the following lemma.Lemma 4.
The solution of equations (23) has the following form:(36)ytk+1∗=ϕtk+1∑n=0kϕtn+1−1δkmvfuxtn∗,utn∗+δkmvbuxtn∗,utn∗gtn+δkmvσuxtn∗,utn∗ωtn.Proof.
According to equations (23) and equations (33), we have(37)ytk+1∗−ϕtk+1ϕtk−1ytk∗=δkmvfuxtk∗,utk∗+δkmvbuxtk∗,utk∗gtk+δkmvσuxtk∗,utk∗ωtk.
Then, we multiply both sides of the equation byϕtk+1−1, and we obtain(38)ϕtk+1−1ytk+1∗−ϕtk−1ytk∗=ϕtk+1−1δkmvfuxtk∗,utk∗+δkmvbuxtk∗,utk∗gtkδkmvσuxtk∗,utk∗ωtk.
By the iterative method, we have(39)ϕtk+1−1ytk+1∗=∑n=0kϕtn+1−1δkmvfuxtn∗,utn∗+δkmvbuxtn∗,utn∗gtn+δkmvσuxtn∗,utn∗ωtn.
Equation (36) is obtained by multiplying both sides of the equality by ϕtk+1.Lemma 5.
We expand the cost functional as(40)limε⟶0Juε−Ju∗ε=E∑k=0N−1lxxtk∗,utk∗ytk∗+δkmluxtk∗,utk∗v+hxxtN∗ytN∗≥0.Proof.
Sincextk∗,utk∗ is optimal, it is natural that(41)0≤limε⟶01εE∑k=0N−1lxtkε,utkε+hxtNε−E∑k=0N−1lxtk∗,utk∗+hxtN∗=limε⟶01εE∑k=0N−1lxtk∗+ytkε,utkε+hxtN∗+ytNε−E∑k=0N−1lxtk∗,utk∗+hxtN∗=limε⟶01εE∑k=0N−1lxtk∗+ytkε,utk∗+hxtN∗+ytNε−E∑k=0N−1lxtk∗,utk∗+hxtN∗+E∑k=0N−1lxtk∗+ytkε,utkε−lxtk∗+ytkε,utk∗.
By Lemma2, we have(42)0≤limε⟶01εE∑k=0N−1lxxtk∗,utk∗ytkε+δkmluxtk∗,utk∗εv+E∑k=0N−1lxxtk∗,utkε−lxxtk∗,utk∗ytkε+EhxxtN∗ytNε=E∑k=0N−1lxxtk∗,utk∗ytk∗+δkmluxtk∗,utk∗v+hxxtN∗ytN∗.
Then, variational inequality (40) is derived.According to equation (36), we substitute the solution of (23) in the functional Ju.(43)limε⟶0Juε−Ju∗ε=E∑k=0N−1lxxtk∗,utk∗ytk∗+δkmluxtk∗,utk∗v+hxxtN∗ytN∗=E∑k=0N−1lxxtk∗,utk∗ϕtk∑n=0k−1ϕtn+1−1δkmvfuxtn∗,utn∗+E∑k=0N−1lxxtk∗,utk∗ϕtk∑n=0k−1ϕtn+1−1δkmvbuxtn∗,utn∗gtn+E∑k=0N−1lxxtk∗,utk∗ϕtk∑n=0k−1ϕtn+1−1δkmvσuxtn∗,utn∗ωtn+E∑k=0N−1δkmluxtk∗,utk∗v+EhxxtN∗ϕtN∑n=0N−1ϕtn+1−1δkmvfuxtn∗,utn∗+EhxxtN∗ϕtN∑n=0N−1ϕtn+1−1δkmvbuxtn∗,utn∗gtn+EhxxtN∗ϕtN∑n=0N−1ϕtn+1−1δkmvσuxtn∗,utn∗ωtn.Lemma 6.
To deal with the terms with fractional noise, we have the following duality formula of Malliavin calculus:(44)EFti∑k=0i−1ztkgtk=EDtiHFti∑k=0i−1ztk.Proof.
By Lemma1, we have(45)EFti∫0tiztdBtH=E∫0tiDtHFtiztdt.
Letzt=∑k=0N−1ztk1tk,tk+1t; then, the left hand of (45) is derived as(46)EFti∫0tiztdBtH=EFti∑k=0i−1∫tktk+1ztk1tk,tk+1tdBtH=EFti∑k=0i−1ztkBtk+1H−BtkH=EFti∑k=0i−1ztkgtk.
The right hand of equality (45) is(47)E∫0tiDtHFtiztdt=E∑k=0i−1∫tktk+1DtFtit∑k=0N−1ztk1tk,tk+1tdt=E∑k=0i−1ztk∫tktk+1DtFtidt=EDtiFti∑k=0i−1ztk.
This completes the proof of Lemma6.By Lemma6, we rewrite equality (43) as(48)limε⟶0Juε−Ju∗ε=E∑k=0N−1lxxtk∗,utk∗ϕtk∑n=0k−1ϕtn+1−1δkmfuxtn∗,utn∗v+E∑k=0N−1DtkHlxxtk∗,utk∗ϕtk∑n=0k−1ϕtn+1−1δkmvbuxtn∗,utn∗+E∑k=0N−1Dtklxxtk∗,utk∗ϕtk∑n=0k−1ϕtn+1−1δkmvσuxtn∗,utn∗+E∑k=0N−1δkmluxtk∗,utk∗v+E∑k=0N−1ϕtk+1−1fuxtk∗,utk∗hxxtN∗ϕtNδkmv+E∑k=0N−1ϕtk+1−1buxtk∗,utk∗DtNHhxxtN∗ϕtNδkmv+E∑k=0N−1ϕtk+1−1σuxtk∗,utk∗DtNhxxtN∗ϕtNδkmv.For arbitraryv that is not equal to zero, we have(49)limε⟶0Juε−Ju∗ε≥0.Then, we obtain the following general maximum principle:(50)lxxtm∗,utm∗ϕtm∑n=0m−1ϕtn+1−1fuxtn∗,utn∗+DtmHlxxtm∗,utm∗ϕtm∑n=0m−1ϕtn+1−1buxtn∗,utn∗+Dtmlxxtm∗,utm∗ϕtm∑n=0m−1ϕtn+1−1σuxtn∗,utn∗+luxtm∗,utm∗+hxxtN∗ϕtNϕtm+1−1fuxtm∗,utm∗+DtNHhxxtN∗ϕtNϕtm+1−1buxtm∗,utm∗+DtNhxxtN∗ϕtNϕtm+1−1σuxtm∗,utm∗=0.This completes Theorem1.


## 4. Applications to the Linear-Quadratic Problem

In this section, we apply Theorem1 to a stochastic linear-quadratic optimal control problem.Consider the controlled system as follows:(51)xtk+1=Atkxtk+Btkvtkgtk+Ctkvtkωtk,k=0,1,…,N−1,xt0=x0,with the cost functional(52)Jv=E∑k=0N−1xtk⊤Mtkxtk+vtk⊤Qtkvtk+xtN⊤MtNxtN.Here,Atk∈ℝn and Btk∈ℝn and, Mtk,k=1,2,…,N and Qtk,k=1,2,…,N−1, are positive matrices. According to (23), the variational equation of the state equations is(53)ytk+1∗=Atkytk∗+Btkδkmvgtk+Ctkδkmvωtk,yt0∗=0.In this case, we getϕtk=∏i=0k−1Ati. We express y∗ in an implicit form of v as(54)ytk∗=ϕtk∑n=0k−1ϕtn+1−1Btnδkmvgtn+Ctnδkmvωtn.Applying Theorem1, we obtain the maximum principle as follows:(55)DtmH2xtm∗Mtmϕtm∑n=0m−1ϕtn+1−1Btn+Dtm2xtm∗Mtmϕtm∑n=0m−1ϕtn+1−1Ctn+2utm∗Qtm+DtNH2xtN∗MtNϕtNϕtm+1−1Btm+DtN2xtN∗MtNϕtNϕtm+1−1Ctm=0.In this special case, we get the optimal control directly as(56)utm∗=−12QtmDtmH2xtm∗Mtmϕtm∑n=0m−1ϕtn+1−1Btn+Dtm2xtm∗Mtmϕtm∑n=0m−1ϕtn+1−1Ctn+DtNH2xtN∗MtNϕtNϕtm+1−1Btm+DtN2xtN∗MtNϕtNϕtm+1−1Ctm.


## 5. Conclusion

In this paper, the necessary optimality conditions of the discrete stochastic optimal control problems driven by both fractional noise and white noise are derived. The admissible control region is not necessarily convex. The stochastic variational inequalities are obtained by applying the classical variation method and the iterative method. The Malliavin calculus is used to derive the maximum principle for our problems. In fact, we obtain a more general maximum principle. We also apply our maximum principle to a discrete linear-quadratic optimal control problem, and the optimal control is obtained.



---

*Source: 1959050-2020-06-05.xml*