# Optimal Control of Diffusions with Hard Terminal State Restrictions

Authors: Atle Seierstad
Journal: ISRN Applied Mathematics
 (2012)

---

## Abstract

A maximum principle is proved for certain problems of optimal control of diffusions where hard end constraints occur. The results apply to several dimensional problems, where some of the state equations involve Brownian motions, but not the equations corresponding to states being hard restricted at the terminal time.


---

## Body

## 1. Introduction

Various types of maximum principles have been proved for problems of control of diffusions in case of no or soft terminal state restrictions; see for example, Kushner [1], Haussmann [2], Peng [3], and Yong and Zhou [4]. Maximum principles for problem with hard terminal restrictions are proved for certain types of continuous time piecewise deterministic problems in Seierstad [5, 6]. Singular controls are sometimes introduced in various problems with certain types of hard restrictions, but below we merely consider problems where the controls appearing may be said to be absolutely continuous with respect to Lebesgue measure. The restriction to such controls makes it harder to operate with hard terminal state restrictions; in fact we can only work with such restrictions on states governed by differential equations not containing a Brownian motion. Brownian motion will only appear in differential equations of states unconstrained at the terminal time. So the problem we consider is a problem of control of diffusions where hard terminal restrictions are placed on states not governed by differential equations containing a Brownian motion; these states, however, can be influenced by other states directly influenced by Brownian motions. Below, a maximum principle is stated and proved for such problems. To the authors knowledge, maximum principles have not been stated for such problems before. Because the states are stochastic, the state space is infinite dimensional; so to obtain a maximum principle, one must impose a condition amounting to demand sufficient variability of the first order variations in the problem.


## 2. The Control Problem and the Statement of the Necessary Condition

LetT>0 and let x0 be a given point in the Euclidean space X:=ℝn*, let π be a projection from X into Y∶=ℝm*, m*<n*, such that πx=(x1,…,xm*) (x=(x1,…,xn*)) and let U be a Borel subset of a Euclidean space. Furnish the interval J∶=[0,T] with the Lebesgue measure. Let (Ω,Φ,Φt,P) be a filtered probability space, (i.e., for t∈[0,T], the Φt’s are sub-σ-algebras of the given σ-algebra Φ of subsets of Ω, Φs⊂Φt if s<t, and P is a probability measure on Φ), and assume that (Ω,Φ,P) is complete, that Φ0 contains all the P-null sets in Φ and that Φt is right continuous. Let L2(J×Ω,ℝn) be the set of Lebesgue ×Φ-measurable functions z(t,ω) for which E∫Jz(t,ω)2dt<∞. Related to (Ω,Φ,Φt,P), let Bt be a vector the n′ components of which (denoted Btj) are independent one-dimensional Brownian motions all adapted to {Φt}t, such that B(t)-B(s) is independent of Φs for all s, t, 0≤s<t, and is normally distributed with mean 0 and covariances (t-s)I,  with  I  being  the identity matrix. In applications where the Btj’s are the entities that can be observed, it is natural to take {Φt}t as the natural filtration generated by the Btj’s. There are given functions f(t,x,u) and σ-j(t,x), j=1,…,n′, from J×X×U into X (σ-j independent of u). The following conditions are called the basic assumptions.(A1) The functions f(t,x,u) and σ-j(t,x) have continuous derivatives fx and σ-xj with respect to x∈X.(A2) The functions f and σ-j have one-sided limits with respect to t, and f and fx are, separately, continuous in x and in u.Writeσ- for the n*×n′-matrix whose columns are σ-j; let σ-xj be the matrix with entries σ-xij, and write ∑jσ-jdBtj=σ-dBt. Also, write 1C for the indicator function of the set C. Let U′ be a set of functions u(t,ω) taking values in U, such that u(·,·), for each t, when restricted to [0,t]×Ω, is Lebesgue ×Φt-measurable (i.e., progressively measurable), and such that when ui(·,·)∈U′, i=1,…,i*, i* arbitrary, and {Ci}i, i=1,…,i*, is a measurable partition of J, then ∑i=1i*1Ci(t)ui(t,ω) belongs to U′ (so-called switching closedness). We will also assume that U′ is σˇ-closed, which means that if un(·,·)∈U′ and u(t,ω) is progressively measurable and takes values in U and meas {t:u(t,ω) is not equal to un(t,ω) a.s.} converges to zero when n→∞, then u(·,·) belongs to U′. Let |·|2 be the L2-norm on L2(Ω,Φ,ℝk).The following assumptions are called the global assumptions.(B1) πfx is uniformly continuous in x,  uniformly  in t, u.(B2) For some constant M, for all u(·,·)∈U′,
(2.1)essupt|f(t,0,u(t,ω))|2≤M,supj,t|σ-j(t,0)|≤M,(B3) A constant M+ exists such that for all (t,x,u)∈J×X×U,
(2.2)|fx(t,x,u)|≤M+,|σ-xj(t,x)|≤M+.
(The symbol |·| is used for the Euclidean norm in any Euclidean space ℝk, including ℝ1, and, applied to matrices, it is the linear operator norm.)(B4) One has
(2.3)πσ-=0.Letu(·,·)∈U′. The strong (unique) solution, continuous in  t, of  the equation
(2.4)x(t,ω)=x0+∫0t‍f(s,x(s,ω),u(s,ω))dt+∫0t‍σ-(s,x(s,ω))dBt
is denoted xu(·,·)(t,ω)=xu(t,ω) and is called a system solution.Leta∈X (a fixed, ≠0) such that πa=0; let 〈·,·〉 denote scalar product, and consider the problem
(2.5)maxu(·,·)∈U′E〈xu(·,·)(T,·),a〉,
subject to
(2.6)πxu(·,·)(T,ω)=y~a.s.,wherey~isfixedinY.Letu*(·,·)∈U′ be an optimal control in the problem and write xu*(·,·)(·,·)=x*(·,·). Let C(t,s,ω) be the resolvent of the equation
(2.7)q(t)=q0+∫0t‍fx(s,x*(s,ω),u*(s,ω))q(s,ω)dt+∑j‍∫0t‍σ-xj(s,x*(s))dBsj,
(so C(s,s,ω)=I,  with  I  being  the identity map).In the subsequent necessary conditions, the following local linear controllability condition (2.10) is needed. Let L2prog(J×Ω,Y) be the set of progressively measurable functions in L2(J×Ω,Y), and for α∈(0,∞], let
(2.8)Bα∶={z(·,·)∈L2prog(J×Ω,Y):essupt|z(t,·)|2<α},(2.9)Bα={∫0T‍z(s,·)ds:z(·,·)∈Bα},
and let co denote convex hull. There exists a number α>0 and a progressively measurable function zˇ(t,ω):J×Ω→Y, with essupt|zˇ(t,·)|2<∞, and a number c∈[0,T) such that
(2.10)1[c,T][zˇ(·,·)+Bα]⊂co{1[c,T]π[f(·,x*(·,·),u^(·,·))-f(·,x*(·,·),u*(·,·))]:u^(·,·)∈U′}.Theorem 2.1.
Assume thatu*(·,·) is optimal in problems (2.5) and (2.6), that assumptions A and B hold (the basic and global assumptions), and that (2.10) is satisfied. Then there exists a number Λ0≥0 and a linear functional ν on B∞, bounded on B1, such that, for all u(·,·)∈U′,
(2.11)〈∫0T‍πC(T,t,·)[f(t,x*(t,·),u(t,·))-f(t,x*(t,·),u*(t,·))]dt,ν〉+E〈∫0T‍C(T,t,·)[f(t,x*(t,·),u(t,·))-f(t,x*(t,·),u*(t,·))]dt,Λ0a〉≤0.
Finally,(Λ0,ν)≠0.Remark 2.2.
If (2.10) holds for zˇ(·,·)=0, then Λ0≠0 and ν is a continuous linear functional on L2(Ω,Φ,Y).Remark 2.3.
Letν*:=ϕ→〈ϕ,ν)+Λ0E〈ϕ,a〉 and let C(T,t,·)* be the transposed of C(T,t,·). Note that for t<T, C(T,t,·)*ν*|L2(Φt,Ω) is continuous in |·|2-norm and hence can be represented by an L2-function p-(t,·)∈L2(Ω,Φt,ℝn*) (p-(t,ω) progressively measurable and continuous in t). Provided U′ has the property that if u,u′∈U′ and C⊂Ωis Φt-measurable then (u1C+u′(1-1C))1[t,T]+u1[0,t)∈U′, we have that, for any u(·)∈U′, for a.e. t in (0,T), a.s. 							(2.12)〈f(t,x*(t,ω),u(t,ω))-f(t,x*(t,ω),u*(t,ω)),p-(t,ω)〉≤0
(a consequence of (2.11)).
Whenν is continuous on L2(Ω,Φ,Y), then
(2.13)limt→Tp-(t,ω)j=Λ0aj,j>m*,
(the limit being an L2-limit), in fact, when t→T, p-(t,·)→v~*(·) in L2, where v~*(ω) is the L2-function representing ν*.
Assume thatΦt is the natural filtration generated by Bt. Then the progressively measurable function p-(t,ω) satisfies the following condition: on [0,T), there exist ℝn*-valued, progressively measurable functions p(t,ω), qj(t,ω),  j=1,…,n′,  p(t,ω) continuous in t, such that E∫0tp(t,·)2dt<∞, E∫0tqj(t,·)2dt<∞ for all t<T, such that
(2.14)dp(t,ω)=-p(t,ω)fx(t,x*(t,ω),u*(t,ω))dt-∑j‍σ-xj(t,x*(t,ω))qj(t)dt+∑j‍qj(t)dBtj
and such that, for all t<T, p-(t,w)=p(t,ω)P-a.s. In this case, if ν is continuous on L2(Ω,Φ,Y), then the following additional properties hold: E∫0Tp(t,·)2dt<∞, E∫0Tqj(t,·)2dt<∞, the L2-limit limt→Tp(t,·) exists and equals v~*(·), and Pr[(Λ0,limt→Tp(t,ω))≠0]>0.


## 3. Proof of Theorem2.1

The proof consists of three lemmas and the five proof steps A–E and relies on an “abstract” maximum principle, Corollary I in the appendix.LetX′∶=ℝn** and let J′∶=[0,T′], T′∈(0,1].Lemma 3.1.
Letg∈L2(J′×Ω,X′) be progressively measurable. For any ε>0 there exists a function b(t,ω)∶=∑j=0j*g(sj,ω)1[sj,tj+1)(t)∈L2(J′×Ω,X′), with tj≤sj≤tj+1, g(sj,·)∈L2(Ω,Φsj,X′), such that ∫J|g(t,ω)-b(t,ω)|2dt<ε.Proof.
Using Dunford and Schwartz [7, III.11.16 Lemma] yields that g(t,·)∈L2(J′,L2(Ω,Φ,X′)) a.e. For each ε′>0 there exists a function
(3.1)a(t,ω)=∑j=0j*aj(ω)1[tj,tj+1)(t),t0=0,tj*+1=T′,tj<tj+1,
- a(t,ω)  piecewise constant in  t -,  aj(·)∈L2(Ω,Φ,X′), such that
(3.2)∫J′|g(t,·)-a(t,·)|2dt<ε′2.
Thus, there exists an open setA⊂J′, such that meas(A)<ε′, and A⊃A0∶={t:|g(t,·)-a(t,·)|2>ε′} (note that meas(A0)<ε′, otherwise the inequality involving ε′2 is contradicted). Let B=∁A, and let sj∶=minB∩[tj,tj+1) if j∈Γ:={j:B∩[tj,tj+1)≠0}. For j∈Γ,
(3.3)|aj(·)-g(sj,·)|2≤ε′,
so for j∈Γ,t∈B∩[tj,tj+1), we have
(3.4)|g(t,·)-g(sj,·)|2≤|g(t,·)-aj(·)|2+|aj(·)-g(sj,·)|2≤2ε′.
Define(3.5)b(t,·)∶=∑j∈Γ‍g(sj,·)1[sj,tj+1).
Ift∈B and t<T′, then for some j, t∈[tj,tj+1), so for this j, j∈Γ, t∈[sj,tj+1) and (3.4) yields
(3.6)|g(t,·)-b(t,·)|2=|g(t,·)-g(sj,·)|2≤2ε′.
Letε>0 be arbitrarily given. Assume now that ε′ is so small that
(3.7)meas(C)<ε′⇒∫C|g|2dt<ε2
(meas(C) = Lebesgue measure of C) and
(3.8)3ε′+2ε′2<ε2.
Then, using successively, (3.6), (3.2), (3.3), (3.2), (3.7), and (3.8) yields
(3.9)∫J′|g(t,·)-b(t,·)|2dt=∫B|g(t,·)-b(t,·)|2dt+∫A|g(t,·)-b(t,·)|2dt≤2ε′+∫A|g(t,·)-a(t,·)|2dt+∫A|a(t,·)-b(t,·)|2dt≤2ε′+ε′2+∫A∑j∈Γ|aj(·)-g(sj,·)|21[sj,tj+1)(t)dt+∫A∑j∈Γ|aj(·)|21[tj,sj)(t)dt+∫A∑j∉Γ|aj(·)|21[tj,tj+1)(t)dt≤2ε′+ε′2+ε′+∫A|a(t,·)|2dt≤3ε′+ε′2+∫A|a(t,·)-g(t,·)|2dt+∫A|g(t,·)|2dt≤3ε′+ε′2+ε′2+ε2≤ε.Lemma 3.2.
Letg∈L2(J′×Ω,X′) be progressively measurable and let k~∈(0,1). Then for each ε>0 there exists a set C⊂J′ such that for all s(3.10)|k~∫0sg(t,ω)dt-∫0sg(t,ω)1C(t)dt|2<ε
and such that meas(C)=k~T′ (C measurable).Proof.
Apply Lemma3.1 to obtain
(3.11)∫J′|g(t,ω)-h(t,ω)|2dt<ε4forh(t,ω)=∑k=0k*ak(ω)1[tk,tk+1)(t),
where ak(ω)∈L2(Ω,Φtk,X′),  tk<tk+1,t0=0,  tk*+1=T′. Evidently, we can assume of the tk’s that they satisfy the additional property
(3.12)|∫tktk+1|g|ds|2<ε4.
Define(3.13)ϕ(t,ω)∶=∑k=1k*1[tk,(1-k~)tk+k~tk+1)(t).
Now,(3.14)∫tktk+1h(t,ω)ϕ(t,ω)dt=∫tktk+1ak(ω)1[tk,(1-k~)tk+k~tk+1)(t)dt=k~ak(ω)(tk+1-tk)=k~∫tktk+1ak(ω)dt=k~∫tktk+1h(t,ω)dt.
Hence, for any givenk*,
(3.15)∫0tk*h(t,ω)ϕ(t,ω)dt=∑k<k*∫tktk+1h(t,ω)ϕ(t,ω)dt=∑k<k*k~∫tktk+1h(t,ω)dt=k~∫0tk*h(t,ω)dt.
Moreover, by (3.15) and (3.11),
(3.16)|∫0tk*g(t,ω)ϕ(t,ω)dt-k~∫0tk*g(t,ω)dt|2≤|∫0tk*h(t,ω)ϕ(t,ω)dt-k~∫0tk*h(t,ω)dt|2+|∫0tk*(g(t,ω)-h(t,ω))ϕ(t,ω)dt|2+k~|∫0tk*(h(t,ω)-g(t,ω))dt|2<2ε4=ε2.
Finally, for any givent, if k*=k is the largest k such that tk≤t, then, by (3.12), a.s.,
(3.17)|∫tk*tg(s,ω)ϕ(s,ω)ds|2≤ε4,|k~∫tk*tg(s,ω)ds|2≤ε4.
The conclusion of Lemma3.2 then follows from (3.16) and (3.17) and the fact that ∫0T′ϕ(s)ds=∑kk~(tk+1-tk)=k~T′.Lemma 3.3.
Letz(t,ω)∈B∞, and let z(t,ω) be continuous in t, and assume that |supt|z(t,ω)||2<∞. Let u(t,ω)∈U′. When δ↓0, then
(3.18)supθ∈[0,δ]|π{fx(t,x*(t,ω)+θz(t,ω),u(t,ω))-πfx(t,x*(t,ω),u(t,ω))}[z(t,ω)]|2→0uniformlyint.Proof.
Let an error functione(d) be a nonnegative function on [0,∞) such that e(d)↓e(0)=0 when d↓0. By uniform continuity of πfx in x, uniformly in t,u, there exists an increasing error function e(d) such that |fxji(t,x+z,u)-fxji(t,x,u)|≤e(|z|) for all z∈X, i≤m*. Suppose, by contradiction, that some ε>0 exists, such that, for each k=1,2,…, there exist θk,tk such that
(3.19)|(πfx(tk,x*(tk,ω)+θkz(tk,ω),u(tk,ω))-πfx(tk,x*(tk),u(tk,ω)))[z(tk,ω)]|2≥ε,
and θk<1/k. Then
(3.20)ζk(ω)∶=|(πfx(tk,x*(tk,ω)+θkz(tk,ω),u(tk,ω))-πfx(tk,x*(tk),u(tk,ω)))[z(tk,ω)]|≤n*2e(θk|z(tk,ω)|)|z(tk,ω)|≤n*2e(θksupt|z(t,ω)|)supt|z(t,ω)|.
Now,supt|z(t,ω)|<∞ a.s. by the L2-assumption on z(·,·) in the Lemma. So ζk(ω) converges a.s. to zero. Moreover, by (2.2), |ζk(ω)|≤2M+θκ|z(tk,ω)|≤2M+supt|z(t,ω)|, the last function being an L2-function. By dominated convergence, |ζk(·)|2→0 when k→∞, and a contradiction of (3.19) is obtained.(A) Growth Properties
Without loss of generality, from now on, letx0=0, T=1. Let πi:=x→xi, the ith component of x∈X. For x(·,·)∈L2(J×Ω,ℝn), let
(3.21)∥x(·,·)∥2∶=essupt|x(t,·)|2,∥x(·,·)∥2*=|(supt|x(t,·)|)|2
(where of course ∥x(·,·)∥2≤∥x(·,·)∥2*). For any u(·,·), u′(·,)∈U′, let qu′,u(t,ω) be the solution of
(3.22)qu′,u(t,ω)=∫0tfx(s,xu(s,ω),u(s,ω))qu′,uds+∑j∫0tσ-xj(s,xu(s,ω))qu′,udBsj+∫0t(f(s,xu(s,ω),u′(s,ω))-f(s,xu(s,ω),u(s,ω)))ds.
(By general existence results and (2.1) and (2.2), qu′,u, as well as xu, see (2.1), do exist,  with  both  being  unique (strong) solutions.)
By (2.1) ∫0tf(s,0,u(s,ω))2ds≤M2 and ∑j∫0tσ-j(s,0)2ds≤n′M2. By (A.3) in the appendix, (a consequence of Gronwall’s inequality), and (2.2), with zˇ(t,ω)=0, hˇ=f(t,x,u(t,ω)), σ*=σ-, yˇ=xu(t,ω), α(t,ω)=-f(s,0,u(s,ω)), α*(t,ω)=-σ-(t,0), and β*(t,ω)=β(t,ω)=0, we have that, for some constant Dˇ independent of u,
(3.23)∥xu(·,·)∥2*≤Dˇ(1+n′)M=:D~.
DefineX^={x(t,ω)∈L2(J×Ω,X):∥x(·,·)∥2≤D~}. Note that by (2.2), |f(s,x(s,ω),u(s,ω))|2≤|f(s,0,u(s,ω))|2+M+|x(s,ω)|2. Then, when x(·,·) belong to X^, by (2.1) and (3.23), we get the following inequality: for all s,
(3.24)|f(s,x(s,ω),u(s,ω))|2≤D˘∶=M+M+D~
(D˘ independent of u and x(·,·)∈X^). For two Φ-measurable functions ϕ(ω) and ψ(ω), let ϕ(·)≠ψ(·) mean that Pr[ω:ϕ(ω)≠ψ(ω)]>0. Define Hu′,u={(t,ω):u′(t,ω)≠u(t,ω)} and define Hu′,u={t:u′(t,·)≠u(t,·)}. Then a.s. 1Hu′,u≥1Hu′,u.
Letu′′,u∈U′. Using (2.4), (2.2), and (A.3) in the appendix, (with zˇ=xu, yˇ=xu′′, β*=β=0, hˇ=f(t,x,u′′(t,ω)), σ*=σ-, yˇ0=zˇ0=0, α=f(s,xu(s,ω),u(s,ω))-f(s,xu(s,ω),u′′(s,ω)), and α*=0), for some constant D independent of u′′ and u, we get
(3.25)|(supt|xu′′(·,·)-xu(·,·)|)|2≤D∫01|1Hu′′,u(f(sˇ,xu(sˇ,ω),u′′(sˇ,ω))-f(sˇ,xu(sˇ,ω),u(sˇ,ω)))|2dsˇ≤2DD˘∫011Hu′′,udt.
(the last inequality by (3.24).)
Letu′, u∈U′, k∈(0,1]. As explained below, we have
(3.26)∥qu′,u(·,·)∥2*≤D|∫01|f(t,xu(t,ω),u′(t,ω))-f(t,xu(t,ω),u(t,ω))|dt|2≤2DD˘∫011Hu′,udt,(3.27)sups≤t|qu′′,u(s,·)-kqu′,u(s,·)|2≤Dsups≤t|∫0s(f(sˇ,xu(sˇ,ω),u′′(sˇ,ω))-(1-k)f(sˇ,xu(sˇ,ω),u(sˇ,ω))-kf(sˇ,xu(sˇ,ω),u′(sˇ,ω)))dsˇ∫0s|2
(D independent of u′′,u′,u,k.) The inequalities (3.27) and (3.26) follow from (A.2) and (A.3), respectively, in the appendix, (together with (2.2) and (3.22)), for β*=β=0, zˇ=qu′′,u-kqu′,u, yˇ=0, hˇ(t,x,ω)=fx(t,xu(t,ω),uu(t,ω))[x], σ*j(t,x,ω)=σ-xj(t,xu(t,ω))[x], α*=0, and
(3.28)α(t,ω)∶=f(t,xu(t,ω),u′′(t,ω))-f(t,xu(t,ω),u(t,ω))-k[f(t,xu(t,ω),u′(t,ω))-f(t,xu(t,ω),u(t,ω))]
(and, for (3.26) for u′′=u, k=1).
Similarly, foru~, u′′, u′, u∈U′,
(3.29)sups≤t|qu~,u(s,·)-(kqu′,u(s,·)+(1-k)qu′′,u(s,·))|2≤Dsups≤t|∫0s[f(sˇ,xu(sˇ,ω),u~(sˇ,ω))-{kf(sˇ,xu(sˇ,ω),u′(sˇ,ω))+(1-k)f(sˇ,xu(sˇ,ω),u′′(sˇ,ω))}]dsˇ.
Define(3.30)σˇ(u,u′)∶=∫J1Hu,u′(t)dt.
Define also(3.31)Ii∶=(1-12i,1-12i+1],i=0,1,…,(3.32)σ*(u′,u)∶=supi2i+1∫Ii1Hu′,u(t)dt.
We need to prove that(3.33)supi2i|∫Iiπx˙u′(t,·)-πx˙u(t,·)dt|2→0whenu′→uinσ*-metric.
This follows from (3.24), (3.25), (2.2), and (3.32), because, in a shorthand notation,
(3.34)2i|∫Ii{πf(t,xu′,u′)-πf(t,xu,u)}dt|2≤2i|∫Ii|πf(t,xu′,u′)-πf(t,xu,u′)|dt|2+2i|∫Ii{πf(t,xu,u′)-πf(t,xu,u)}dt|2≤2i∫IiM+∥xu′-xu∥2dt+2·2iD˘∫Ii1Hu′,udt≤M+∥xu′-xu∥22+D˘σ*(u′,u).(B) Properties of the “Linear” Perturbationsqu′,u
Letε′′>0 be arbitrarily given, let k be any number in (0,1], and let u′′, u∈U′. Let us first prove the following consequence of Lemma 3.2. (We drop writing · for ω.) For all s,
(3.35)|∫0s1Im{[f(t,xu(t),uk(t))-f(t,xu(t),u(t))]-k[f(t,xu(t),u′′(t))-f(t,xu(t),u(t))]}{∫0s}|2<ε′′k2m+1,
where uk on Im, m=0,1,2,…, is defined by uk∶=u′′1Cm+u(1-1Cm), with the sets Cm being as follows. They are obtained by replacing J′ by Im, (hence [0,s] by Im∩[0,s]), and ε by ε′′k/2m+1 in (3.10), that is, in Lemma 3.2, and denoting the corresponding subset C by Cm⊂Im, with g in Lemma 3.2 being equal to f(t,xu(t),u′′(t))-f(t,xu(t),u(t)). Here,  meas (Cm)=kmeas(Im)=k/2m+1, so σˇ(uk,u)≤σ*(uk,u)≤k.
Letu′∈U′. Because (3.35) holds for some Cm when u is replaced by u′, we get that for some uk∈U′, for all s,
(3.36)|∫0s1Im{{∫0s}[f(t,xu(t),uk(t))-f(t,xu(t),u(t))]-∫0s[kf(t,xu(t),u′′(t))+(1-k)f(t,xu(t),u′(t))-f(t,xu(t),u(t))]}dt|2≤ε′′k2m+1.
From (3.35) and (3.27) it follows that, for any t,
(3.37)|quk,u(t)-kqu′′,u(t)|2≤Dsups≤t|∫0s∑m=0∞1Im{[f(sˇ,xu(sˇ),uk(sˇ))-f(sˇ,xu(sˇ),u(sˇ))-k[f(sˇ,xu(sˇ),u′′(sˇ))-f(sˇ,xu(sˇ),u(sˇ))]}dsˇ{∫0s∑m=0∞}|2≤Dε′′k∑m=0∞12m+1=Dε′′k,withσ*(uk,u)≤k,
and similarly, from (3.36) and (3.29) it follows that
(3.38)supt|quk,u(t)-[kqu′′,u(t)+(1-k)qu′,u(t)]|2≤Dε′′k.
From (3.38) it follows, in a shorthand notation, that
(3.39)2i|∫Ii{πq˙uk,u-(kπq˙u′′,u+(1-k)πq˙u′,u)}dt|2≤M+Dε′′k2+ε′′k2.
To see this, note that(3.40)∥πfx(t,xu,u)quk,u-πfx(t,xu,u)[kqu′′,u+(1-k)qu′,u]∥2≤M+∥quk,u-[kqu′′,u+(1-k)qu′,u]∥2≤M+Dε′′k,
(by (2.2) and (3.38)), so
(3.41)2i|∫Iiπfx(t,xu,u)quk,u-πfx(t,xu,u)[kqu′′,u+(1-k)qu′,u]dt|2≤M+Dε′′k2.
Note also thatπq˙uk,u=πfx(t,xu,u)quk,u+πf(t,xu,uk)-πf(t,xu,u) and
(3.42)kπq˙u′′,u+(1-k)πq˙u′,u=πfx(t,xu,u)[kqu′′,u+(1-k)qu′,u]+kπf(t,xu,u′′)+(1-k)πf(t,xu,u′)-πf(t,xu,u).
Then (3.39) follows from (3.36) and (3.41).
Ifui∈U′, i=1,…,i*, i* arbitrary, and ki∈(0,1), ∑iki=1, then for any ε*>0, it is easily seen that we can obtain, for some u^∈U′, that
(3.43)2i|∫Ii{πq˙u^,u(t,·)-∑i=1i*kiπq˙ui,u(t,·)}dt|2≤ε*.
(For i*=3,by (3.39), we can first obtain a control uˇ such that 2i|∫Ii{πq˙uˇ,u-∑i=12ki/(k1+k2)πq˙ui,u}dt|2≤ε*/2, and then by (3.39) we can obtain a control u^ such that 2i|∫Ii{πq˙u^,u-[(k1+k2)πq˙uˇ,u+k3πqu3,u]}dt|2≤ε*/2, hence 2i|∫Ii{πq˙u^,u-[(k1+k2)(∑i=12ki/(k1+k2)πq˙ui,u)+k3πqu3,u]}dt|2≤ε*/2+(k1+k2)ε*/2≤ε*. Continuing this argument, we get (3.43) for general i*.)
Evidently, we can obtain for anyε*>0, for some u^∈U′, that both
(3.44)(3.43)holdsand|qu^(1,·)-(∑i=1i*kiqui,u(1,·))|2≤ε*.
Finally, let the numberc′∈[0,1) satisfy 2M+DD˘(1-c′)≤α/4, c′≥c (for c and α, see (2.10)), and let u∈U**∶={u∈U′:σ*(u,u*)≤α/16M+DD˘}. We want to prove the inequality (shorthand notation)
(3.45)∥πf(·,x*,u^)-πf(·,x*,u*)-[πfx(·,xu,u)qu^,u+πf(·,xu,u^)-πf(·,xu,u*)]∥2≤α2
whenever u^ equals u on [0,c′).
Now,|πfx(t,xu,u)qu^,u|2≤M+|qu^,u|2≤2M+DD˘(1-c′)≤α/4, see (3.26). Next, for u∈U**, ∥xu-x*∥2≤α/8M+ when σˇ(u,u*)≤σ*(u,u*)≤α/16M+DD˘, see (3.25), so ∥πf(t,x*,u^)-πf(t,x*,u*)-[πf(t,xu,u^)-πf(t,xu,u*)]∥2≤2M+∥xu-x*∥2≤α/4.
Using the two inequalities involvingα/4, we get (3.45). And from this property and (2.10) it easily follows that (shorthand notation), for all u∈U**,
(3.46)1[c′,1][zˇ(t,·)+Bα/2]⊂clco{1[c′,1][πfx(t,xu,u)qu^,u+πf(t,xu,u^)-πf(t,xu,u*){πfx(t,xu,u)qu^,u+πf(t,xu,u^)}]:u^∈U′{1[c′,1][πfx(t,xu,u)qu^,u+πf(t,xu,u^)}}
(cl = closure in ∥·∥2-norm). To see this, apply Lemma 11.1 in Seierstad [8]. (Intuitively this lemma says that if a ball is contained in the closed convex hull of a set, and the elements of the set are slightly perturbed then a slightly smaller ball is contained in the closed convex hull of the set of perturbed elements.)(C) Relations between Exact and Linear Perturbations
Letu′′, u be given elements in U′. Let ε>0 be arbitrarily given. Define ε′=ε/max{K+,K-} where the constants K+ and K- are specified in the proof below. Let us first prove that for any r>0, small enough, for any δ∈(0,r], there exists a u′∈U′ such that
(3.47)supt|δ-1(xu′(t,·)-xu(t,·)-δqu′′,u(t,·))|2≤K-ε′δ≤ε,σ*(u′,u)≤δ.
Writeσ-xqdBt=∑jσ-xjqdBtj. Define, in a shorthand notation,
(3.48)ξ1(t,ω)=δ-1{f(t,xu+δqu′′,u,u)-f(t,xu,u)-fx(t,xu,u)δqu′′,u},ξ1(t)=[E(∫0tξ1(s)ds)2]1/2,ξ2(t)∶={E(∫0t[δ-1{σ-(t,xu+δqu′′,u)-σ-(t,xu)-σ-x(t,xu)δqu′′,u}]dBt)2}1/2.
There exists aδ′∈(0,ε′] such that
(3.49)ξi(t)≤ε′,i=1,2,whenδ∈(0,δ′],uniformlyint,
by Lemmas B and C in the appendix.
In (3.35) let k=δ, ε′′=ε′ and let u′=uδ=uk (so σˇ(u′,u)≤σ*(u′,u)≤δ). We will prove (3.47) for this u′.
Let(3.50)ξ3(t)∶=[δ-1{f(t,xu+δqu′′,u,u′)-f(t,xu,u)-fx(t,xu,u)δqu′′,u-f(t,xu,u′)+f(t,xu,u)}],
and let ξ3(t)∶=[E(∫0tξ3(s)ds)2]1/2. On ∁Hu′,u, ξ3(t)=ξ1(t), while, by (2.2),
(3.51)1Hu′,u|ξ3(t)-ξ1(t)|≤1Hu′,uδ-1{f(t,xu+δqu′′,u,u′)-f(t,xu+δqu′′,u,u)-f(t,xu,u′)+f(t,xu,u)}≤1Hu′,u2M+|qu′′,u|.
So, by (3.49), uniformly in t,
(3.52)ξ3(t)=[E(∫0t{ξ1(s)+ξ3(s)-ξ1(s)}ds)2]1/2≤[E(∫0tξ1(s)ds)2]1/2+[E(∫0t1Hu′,u{ξ3(s)-ξ1(s)}ds)2]1/2≤ε′+[E(∫0t1Hu′,u2M+|qu′′,u|ds)2]1/2≤ε′+4M+DD˘δ≤K~ε′,
where K~∶=1+4M+DD˘ (recall meas(Hu′,u)≤δ, and ∥qu′′,u∥2≤2DD˘; see (3.26)).
Consider now(3.53)δ-1(xu′-xu-δqu′′,u)=∫0tδ-1{f(s,xu′,u′)-f(s,xu,u)-fx(s,xu,u)δqu′′,u-δf(s,xu,u′′)+δf(s,xu,u){f(s,xu′,u′)}}ds+∫0tδ-1[σ-(s,xu′)-σ-(s,xu)-σ-x(s,xu)δqu′′,u]dBs.
By(A.2) in the appendix, Lemma A, for some constants Dˇ>0, K-*>0,
(3.54)∥δ-1(xu′-xu-δqu′′,u)∥2≤Dˇ[ξ3(t)+ξ2(t)+ε′]≤DˇK-*ε′,
To see this, in Lemma A letzˇ=xu′, let hˇ=f(t,x,u′), let yˇ=xu+δqu′′,u, let α*=α=0, σ*=σ-, and let
(3.55)β=[f(s,xu,u)-f(s,xu+δqu′′,u,u′)+fx(s,xu,u)δqu′′,u+δf(s,xu,u′′)-δf(s,xu,u)],β*=σ-(s,xu)-σ-(s,xu+δqu′′,u)+σ-x(s,xu)δqu′′,u.
Note that(3.56)yˇ(t)=∫0t[f(s,xu,u)-f(s,xu+δqu′′,u,u′)]ds+∫0t[σ-(s,xu)-σ-(s,xu+δqu′′,u)]dBs+∫0t[fx(s,xu,u)δqu′′,u+δf(s,xu,u′′)-δf(s,xu,u)]ds+∫0tσ-x(s,xu)δqu′′,udBs+∫0tf(s,xu+δqu′′,u,u′)ds+∫0tσ-(s,xu+δqu′′,u)dBs.
Hence, we have(3.57)yˇ=∫0tβds+∫0tβ*dBs+∫0thˇ(s,yˇ)ds+∫0tσ*(s,yˇ)dBs.
Observe that|∫0tδ-1β*dBs|2≤ξ2(t) (see definition subsequent to (3.47)), and that β=-δξ3+δf(s,xu,u′′)-δf(s,xu,u)-(f(s,xu,u′)-f(s,xu,u)), so |∫0tδ-1β|2≤ξ3(t)+ε′, because u′=uδ=uk satisfies (3.35) for ε′=ε′′. Using also (3.49), (3.52) yields (3.54) for K-*=K~+2; hence, (3.47) for K-=DˇK-*.
Next, let us prove that for anyr>0 small enough, for any δ∈(0,r], there exists a u′=uδ∈U′ such that
(3.58)(3.47)holds,and2i|∫Iiδ-1[πx˙u′-πx˙u-δπq˙u′′,u]ds|2≤ε.
Whenδ′∈(0,ε′] is small enough, then |πξ1(t)|2≤ε′, uniformly in t when δ∈(0,δ′], by (3.26) and Lemma 3.3 above (ξ1 defined subsequent to (3.47), we use that ξ1=δ-1{∫01‍[fx(t,xu+θδqu′′,u,u)-fx(t,xu,u)]δqu′′,udθ}). Moreover, by (3.47), ∥xu′-xu-δqu′′,u∥2≤K-ε′δ, so, by (2.2),
(3.59)∥πf(·,xu+δqu′′,u,u)-πf(·,xu′,u)∥2≤M+K-ε′δ.
Hence, using the definition ofξ1 referred to and |πξ1(t)|2≤ε′, we get ∥ξ4∥2≤ε′+M+K-ε′, where
(3.60)ξ4(t)=δ-1{πf(t,xu′,u)-πf(t,xu,u)}-πfx(t,xu,u)qu′′,u.
Let(3.61)ξ5(t)∶=δ-1{πf(t,xu′,u′)-πf(t,xu,u)-πfx(t,xu,u)δqu′′,u-πf(t,xu,u′)+πf(t,xu,u)}.
Thenξ5(t)=ξ4(t) on ∁Hu′,u. On Hu′,u, by (2.2), (3.25), and (3.26), for all t,
(3.62)|ξ5(t)|2≤M+δ-1|xu′-xu|2+M+|qu′′,u|2≤δ-1M+2DD˘σˇ(u′,u)+M+2DD˘≤4M+DD˘,
as σˇ(u′,u)≤δ. Hence, using the inequalities for |ξ5|2 and ∥ξ4∥2 above and Jensen’s inequality,
(3.63)2i[E(∫Iiξ5(t)dt)2]1/2≤2i[E(∫J1Ii1∁Hu′,uξ4(t)dt)2]1/2+2i[E(∫J1Ii1Hu′,uξ5(t)dt)2]1/2≤(12)[ε′+M+K-ε′]+2M+DD˘σ*(u′,u).
Finally, recalling thatσ*(u′,u)≤δ, for
(3.64)ξ6(t)∶=δ-1{πf(t,xu′,u′)-πf(t,xu,u)-πfx(t,xu,u)δqu′′,u-πδf(t,xu,u′′)+πδf(t,xu,u)},
we have that
(3.65)2i[E(∫Iiξ5(t)-ξ6(t)dt)2]1/2≤2i[E(∫Iiδ-1[δπf(t,xu,u′′)-δπf(t,xu,u)-(πf(t,xu,u′)-πf(t,xu,u))]dt)2]1/2≤ε′2,
by (3.35). Hence, using σ*(u′,u)≤δ≤ε′ and (3.63),
(3.66)2i|∫Iiξ6(t)dt|2≤2i|∫Iiξ5(t)dt|2+2i|∫Ii(ξ6(t)-ξ5(t))dt|2≤(12)[ε′+MK-ε′]+2M+DD˘σ*(u′,u)+ε′2=(1+MK-2+2M+DD˘)ε′,
so for K+∶=(1+MK-/2+2M+DD˘), (3.58) has been proved.(D) Continuity ofu→qu′,u at u*
Defineδ(t)∶=qu′,u-qu′,u*. Let us first prove that
(3.67)∥δ(·,·)∥2→0,whenu→u*inσ*-metric.
Now,(3.68)δ(t)=∫0t{fx(s,xu,u)qu′,u+f(s,xu,u′)-f(s,xu,u)}ds-∫0t[fx(s,x*,u*)qu′,u*+f(s,x*,u′)-f(s,x*,u*)]ds+∫0t{σ-x(s,xu)qu′,u-σ-x(s,xu*)qu′,u*}dBs=∫0t[fx(s,xu,u)qu′,u-fx(s,xu,u)qu′,u*]ds+∫0t[fx(s,xu,u)-fx(s,xu,u*)]qu′,u*ds+∫0t[fx(s,xu,u*)-fx(s,x*,u*)]qu′,u*ds+∫0t[f(s,xu,u′)-f(s,x*,u′)]ds+∫0t[f(s,x*,u*)-f(s,xu,u*)]ds+∫0t1Hu,u*[f(s,xu,u*)-f(s,xu,u)]ds+∑j∫0t‍[σ-xj(s,xu)qu′,u-σ-xj(s,xu)qu′,u*]dBsj+∑j∫0t[σ-xj(s,xu)-σ-xj(s,x*)]qu′,u*dBsj.
Using Ito’s isometry, Jensen’s inequality, and the algebraic inequality(∑i=1N|ai|)2≤N∑jai2, then for some number k (only dependent on the number of addends)
(3.69)E|δ(t)|2≤k∫0tM+2E|qu′,u-qu′,u*|2ds+k|∫0t1Hu,u*2M+2E|qu′,u*|2ds+kE∫0t|{fx(s,xu,u*)-fx(s,x*,u*)}qu′,u*|2ds+k∫0t2M+2E|xu-x*|2ds+k|∫0t1Hu,u*E|f(s,xu,u*)-f(s,xu,u)|2ds+k∑j∫0tM+2E(qu′,u-qu′,u*)2ds+k∑jE∫0t[{σ-xj(s,xu)-σ-xj(s,x*)}qu′,u*]2ds.
Hence,(3.70)Eδ(t)2≤k(1+n′)M+2∫0tEδ(s)2ds+∫0tγ(s)ds,
where
(3.71)γ(t)∶=k1Hu,u*2M+2E|qu′,u*|2+kE[{fx(t,xu,u*)-fx(t,x*,u*)}qu′,u*]2+2kM+2E|xu-x*|2+k1Hu,u*E[f(t,xu,u*)-f(t,xu,u)]2+k∑jE[{σ-xj(t,xu)-σ-xj(t,x*)}qu′,u*]2.
Now,∥xu(·,·)-xu*(·,·)∥2→0 when u→u* in σˇ-metric, see (3.25). Hence, by the Basic assumption A1, when u→u* in σˇ-metric, then for each t, it is easily seen, using Lebesgue’s dominated convergence theorem (and, if necessary, Remark M in the appendix), that the terms in curly brackets converge to zero in L2, by the bound M+ on fx and σ-x; see (3.25) and (2.2). Since E|qu′,u*|2≤2DD˘, then, for each t, the product of the two terms in curly brackets with qu′,u* converge to zero in L1 when u→u*. Hence, the expectation of the two products converge to zero when u→u* in σˇ-metric. Since γ(·) is bounded, by (3.24), (3.25), and (3.26), then ∫01γ(s)ds→0 when u→u* in σˇ-metric (by dominated convergence again). By Gronwall’s inequality, for some constant D,
(3.72)Eδ(t)2≤D∫0tγ(s)ds≤D∫01γ(s)ds.
So (3.67) holds.
Next, we want to prove thatδi satisfies
(3.73)supi2i|δi(·)|2→0whenu→u*inσ*-metric,
where
(3.74)δi(ω)∶=∫Ii[πfx(t,xu(t,ω),u(t,ω))qu′,u+πf(t,xu,u′)-πf(t,xu,u)]dt-∫Ii[πfx(t,x*(t,ω),u*(t,ω))qu′,u*+πf(t,x*,u′)-πf(t,x*,u*)]dt.
Now, in a shorthand notation,(3.75)δi(ω)=∫Ii[πfx(t,xu,u)qu′,u-πfx(t,xu,u)qu′,u*]dt+∫Ii{πfx(t,xu,u)-πfx(t,xu,u*)}qu′,u*dt+∫Ii{πfx(t,xu,u*)-πfx(t,x*,u*)}qu′,u*dt+∫Ii[πf(t,xu,u′)-πf(t,x*,u′)]dt+∫Ii[πf(t,x*,u*)-πf(t,xu,u*)]dt+∫Ii1Hu,u*[πf(t,xu,u*)-πf(t,xu,u)]dt.
From this we get that(3.76)|δi|2≤∫IiM+∥qu′,u(·,·)-qu′,u*(·,·)∥2dt+∫Ii1Hu,u*2M+∥qu′,u*(·,·)∥2dt+|∫Ii{πfx(t,xu(t,·),u*(t,·))-πfx(x*(t,·),u*(t,·))}qu′,u*(t,·)|2dt+∫Ii2M+∥xu(·,·)-x*(·,·)∥2dt+∫Ii1Hu,u*|f(t,xu(t,·),u*(t,·))-f(t,xu(t,·),u(t,·))|2dt.
For some increasing nonnegative error functione(·)≤2M+, the third integrand is smaller than e(supt|xu(t,ω)-x*(t,ω)|)supt|qu′,u*(t,ω)|; see B1 and B4 in the global assumptions. We then get that
(3.77)2i|δi|2≤(M+2)∥qu′,u-qu′,u*∥2+M+∥qu′,u*∥22i∫Ii1Hu,u*+|e(supt|xu(t,·)-xu*(t,·)|)(supt|qu′,u*(t,·)|)|2·2i∫1Iidt+M+∥xu(t,·)-x*(t,·)∥2+2iD˘∫Ii1Hu,u*dt.
Now, whenσ*(u,u*)→0, ∥qu′,u(·,·)-qu′,u*(·,·)∥2→0 by (3.67), and ∥xu(·,·)-x*(·,·)∥2*→0 (see (3.25)). Then the term e(supt|xu(t,ω)-xu*(t,ω)|)→0 in P-measure and then also in L2-norm (by the bound 2M+), so e(supt|xu(t,ω)-xu*(t,ω)|)supt|qu′,u*(t,ω)|→0 in L1-norm, and then also in L2-norm, as the term is bounded by the L2-function 2M+supt|qu′,u*(t,ω)|, (see (3.26)). Thus 2iδi→0, uniformly in i; that is, (3.73) holds.(E) Final Proof Steps
Forz(·)∈L2(Ω,Y), define Π1(z(·))∶=E[z(·)∣Φ1-1/2] and Πi(z(·))∶=E[z(·)∣Φ1-1/2i]-E[z(·)∣Φ1-1/2i-1], i>1. Define
(3.78)|z(·)|2∶=supi2i|Πiz(·)|2forz(·)∈L2(Ω,Y),|z(·,·)|2∶=supi2i|∫Ii|z(t,·)|dt|2forz(·,·)∈B∞,
for B∞ see (2.8). Furthermore, let L2 be the subset of L2(Ω,Y) consisting of all element z(·)∈L2(Ω,Y) such that |z(·)|2∶=supi2i|Πiz(·)|2<∞, and such that z(·)=limk→∞∑1≤i≤kΠiz(·)=limk→∞E[z(·)∣Φ1-1/2k], (limit in |·|2-norm). It is easily seen that elements of the type ∫Jy(t,ω)dt, y(t,ω) progressively measurable, ∥y(·,·)∥2<∞, precisely make up the set L2. To see this, using Jensen’s inequality three times (and for any α(t)≥0, that essupt(α(t))1/2=(essuptα(t))1/2, note that for any interval J′,
(3.79)(E[E[∫J′y(t,·)dt∣Φ1-1/2k]]2)1/2≤(E[E[(∫J′y(t,·)dt)2∣Φ1-1/2k]])1/2=(E[meas(J′)∫J′(y(t,·))2dt])1/2=(meas(J′))1/2(E[∫J′y(t,·)2dt])1/2≤(meas(J′))1/2(∫J′essuptE|y(t,·)|2dt)1/2=meas(J′)(essuptE|y(t,·)|2)1/2=meas(J′)∥y(·,·)∥2,
so, in particular, |Π1∫Jy(t,·)dt|2≤∥y(·,·)∥2. This yields also, for j>1, that
(3.80)|Πj∫Jy(t,·)dt|2=|Πj∑0≤i<∞∫Iiy(t,·)dt|2=|∑j-1≤i<∞Πj∫Iiy(t,·)dt|2≤|∑j-1≤i<∞E[∫Iiy(t,·)dt∣Φ1-1/2j]|2+|∑j-1≤i<∞E[∫Iiy(t,·)dt∣Φ1-1/2j-1]|2≤2·∑j-1≤i<∞(12i+1)∥y(·,·)∥2=2·(12j-1)∥y(·,·)∥2,
so |∫Jy(t,ω)dt|2≤4∥y(·,·)∥2<∞. Moreover, similarly,
(3.81)|∫Jy(t,ω)dt-∑1≤j≤kΠj∫Jy(t,ω)dt|2=|∫Jy(t,ω)dt-E[∫Jy(t,ω)dt∣Φ1-1/2k]|2=|∫Jy(t,ω)dt-∫01-1/2ky(t,ω)dt-E[∫1-1/2k1y(t,ω)dt∣Φ1-1/2k]|2≤(22k)∥y(·,·)∥2,
so ∫Jy(t,ω)dt is an L2-limit of ∑1≤j≤kΠj∫Jy(t,ω)dt. Hence, ∫Jy(t,ω)dt∈L2. Finally, if z(ω)∈L2, then z(ω)=∫Jγ(t,ω)dt=limk→∞∫01-1/2k+1γ(t,ω)dt, for
(3.82)γ(t,ω)∶=2∑m≥12mΠmz(ω)1[1-1/2m,1-1/2m+1)(t),
where ∥γ(·,·)∥2≤2·2|z(·)|,  γ(·,·)  is  progressively measurable.
LetΘ be the linear map from B∞into L2(Ω,Y) defined by z(·,·)→∫01z(t,ω)dt. In (3.80) we have just proved that Θ has norm ≤4 for the norms ∥·∥2 and |·|22 (or for ∥·∥2→2|·|, as we will express it). We also have that the norm on Θ is ≤8 for the norms |·|2→2|·|, as we will see. Let z(·,·)∈B∞ and define zi=∫Iiz(t,ω). Then, by Jensen’s inequality,
(3.83)|Π1∑i=0∞zi(·)|2≤∑i=0∞|Π1zi(·)|2≤∑i=0∞(E[Π1(zi(·))2])1/2=∑i=0∞(E[(zi(·))2])1/2≤∑i=0∞(12i)·2i|∫Iiz(·,·)dt|2≤2·2|z(·,·)|,
while for j>1,
(3.84)|Πj∑i=0∞zi(·)|2=|Πj∑i=j-1∞zi(·)|2=|∑i=j-1∞{E[zi(·)∣Φ1-1/2j]-E[zi(·)∣Φ1-1/2j-1]}|2≤∑i=j-1∞{|zi(·)|2+|zi(·)|2}≤2∑i=j-1∞|∫Iiz(·,·)|2≤∑i=j-1∞2·2|z(·,·)|2i=8·(12j)·2|z(·,·)|,
so supj2j|Πj∫Jz(t,·)dt|2≤8·2|z(·,·)|.
Now, forz∈B2(0,α(1-c′)/4)⊂L2, (B2(·,·) a ball in |·|2-norm), we have
(3.85)z=∫c′1γ′(s,·)ds,γ′(s,ω)=2(1-c′)-1∑i=0∞2iΠiz1[1-(1-c′)/2i,1-(1-c′)/2i+1)∈Bα/2.
Hence, by (3.46), for u∈U**,
(3.86)∫c′1zˇdt+B2(0,α(1-c′)4)⊂clco{∫c′1[πfx(t,xu,u)qu^,u+πf(t,xu,u^)-πf(t,xu,u*)]dt:u^∈U′}
(cl = closure in |·|2; note that γ′′(·,·)→∫c′1γ′′(s,·)ds is continuous in ∥·∥2→2|·|, as shown above). Observe, finally, that when u∈U′ satisfies σ*(u,u*)≤α(1-c′)/64D˘, then, using |Θ|≤8 (for |·|2→2|·|) and (3.24) yields
(3.87)|∫J1[c′,1](πf(t,xu,u)-πf(t,xu,u*))dt|2≤8supi2i|∫Ii1[c′,1](πf(t,xu,u)-πf(t,xu,u*))dt|2≤16D˘supi2i∫Ii1[c′,1]1Hu,u*dt≤8D˘σ*(u,u*)≤α(1-c′)8.
Hence, using (3.86) and the last string of inequalities and a simple argument (or Lemma 11.1 in Seierstad [8] again), we get
(3.88)∫c′1zˇds+B2(0,α(1-c′)8)⊂clco{∫c′1[πfx(t,xu,u)qu^,u+πf(t,xu,u^)-πf(t,xu,u)]dt:u^∈U′}⊂clco{∫01‍[πfx(t,xu,u)qu^,u+πf(t,xu,u^)-πf(t,xu,u)]dt:u^∈U′}=clco{πqu^,u(1):u^∈U′},
(cl = closure in |·|2) for u∈U′, σ*(u,u*)≤μ*∶=min{α(1-c′)/64D˘,α/16M+DD˘} (the last fraction defined in connection with (3.45)).
To obtain the conclusion in Theorem2.1, we will now apply Corollary I in the appendix, and for this end, we make the following identifications: Y^=L2, γ*=μ*, K~*=K~*′=1, A~=U′, σ-*=σ*, the norm |·| on Y^ equal to |·|2, a~′=u′, a~*=u*, a~=u, a~+=u′′, H^(a~)=πΘx˙u, q^a~′,a~=πΘq˙u′,u, Hˇ(a~)=E[a·Θx˙u], and qˇa~′,a~=E[a·Θq˙u′,u]. By |Θ|≤8 for |·|2→2|·| and (3.58), it follows that the property (A.20) is satisfied, and by |Θ|≤8 and (3.44) it follows that (A.21) is satisfied, both (A.20) and (A.21) in the manner required in Corollary I. Moreover, for zˇ*∶=∫c′1zˇ(t,·)dt, B2(zˇ*,α(1-c′)/8)⊂clcoπqU′,u(1,·), by (3.88) for u∈clB(u*,μ*). By (3.33) and (3.25), H^ and Hˇ are continuous, and by (3.73), (3.67), a→q~a′,a is continuous at a~*=u*. The required boundedness of qˇa′,a is satisfied because of (3.26). The space (U′,σˇ) is complete by well-known arguments; see Lemma 5.1 in Seierstad [8] and Lemma 1, page 202 in Clarke [9]. Moreover, if un is a Cauchy sequence in σ*, then it is a Cauchy sequence in σˇ. Let u be its σˇ-limit. Then, for all i, limm→∞2i+1∫Ii1Hun,umdt=2i+1∫Ii1Hun,udt. Now, for any ε>0, for m, n≥ some N, for all i, 2i+1∫Ii1Hun,umdt≤ε, and so also 2i+1∫Ii1Hun,udt≤ε. Thus the space (U′,σ*) is complete. Hence all conditions in Corollary I are satisfied. Thus, for some Λ0≥0, some nonzero continuous linear functional ν on L2, (Λ0,ν)≠0, for all u∈U′, Λ0E[a·qu,u*(1,·)]+〈πqu,u*(1,·),ν〉≤0. Because qu,u*(1,·)=∫01C(1,t,·)[f(t,x*(t,·),u(t,·))-f(t,x*(t,·),u*(t,·))]dt, the conclusion in Theorem 2.1 follows.Proof of Remark2.2.
Note that (2.10) implies B2(0,α(1-c′)/8)⊂clco{πqU′,u*(1)}. See (3.88). If Λ0=0, the maximum condition (2.11) implies 〈clco{πqU′,u*(1)},ν〉≤0 and hence 〈B2(0,α(1-c′)/8),ν〉≤0⇒ν=0, a contradiction. Observe that on L2|·|2≤|·|2 (for v∈L2, |v|2=|∑i=1∞Πiv|2≤∑i=1∞(1/2i)2i|Πiv|2≤|v|2). If we show for a |·|2-dense subset Q of B2(0,α(1-c′)/8) that 〈q-,ν〉≤k|q|2 for q-∈Q for some k independent of q-, then 〈±q-,ν〉≤k|±q-|2 for q-∈B2(0,α(1-c′)/8), implying |·|2-continuity on L2. The maximum condition implies 〈πqU′,u*(1),EaqU′,u*(1)},(ν,Λ0)〉≤0; hence, for any ϕ∈B2(0,α(1-c′)/8)∩πcoqU′,u*(1), there exists a ψ∈coqU′,u*(1) such that ϕ=πψ and 〈πψ,ν〉+Λ0E〈ψ,a〉≤0, so 〈ϕ,ν〉=〈πψ,ν〉≤-Λ0E〈ψ,a〉≤Λ0|a||ψ|2≤Λ0|a|2DD˘, by (3.26).Proofs of Remark2.3 and the following remark can be found in the appendix.Remark 3.4 (exact attainability).
In Theorem2.1, drop the assumption that u* is optimal (and the optimization problem). Then, for each z(t,ω)∈int(clco{πq˙u,u*:u∈U′}), z(·,·)∈L2prog(J×Ω,Y), ∥z(·,·)∥2<∞, (cl and int = interior both corresponding to ∥·∥2), for all r>0, for some number γ∈(0,r] and some control u∈U′, πxu(T,ω)=y~+γ∫0Tz(t,ω)dta.s.Example
Lett∈[0,1]=J, let dXt1=-u2dt, let dXt2=u+Xt3, let dXt3=σdBt, let σ be a nonzero constant, let u∈ℝ, let X0i=0, let i=1,2,3, and let us maximize EX11 subject to X12=1, a.s. This trivial problem was solved in Seierstad [10] using the HJB equation. Let {Φt}t be the natural filtration generated by Bt and let U′′={v(·,·)∈L2prog(J×Ω,ℝ):∥v(·,·)∥2≤1}. Let us merely show that the solution presented in Seierstad [10], namely u*=1-Xt*3, (Xt*3=σBt), satisfies the necessary conditions for U′=U′′+u*. (So Xt*2 is deterministic and equals t.) Evidently, the conditions in Remark 2.2 are satisfied, so Λ0 can be put equal to 1, and ν is |·|2-continuous.
The3×3-matrix  Ct satisfies dCt=DCt where the 3×3-matrix D consists of the partial derivatives of the drift terms in the three-state equation above, the only nonzero element in D being the element D23=1. Hence, Cii(t,s)=1, i=1,2,3, Cij=0, i≠j, except for C23(t,s) which equals t-s. Now, ν*=(1,ν,0), so the maximum condition (2.11) reduces to 〈∫01(u-u*)dt,ν〉-∫01(u2-u*2)dt≤0, u=u(·,·)∈U′. Writing p-=(p1-,p2-,p3-), p1-≡1, the time-pointwise version (2.12) of this condition becomes p2-(t,ω)[u(t,ω)-u*(t,ω)]-(u(t,ω)2-u*(t,ω)2)≤0, which evidently requires that 2u*(t,ω)=p2-(t,ω). Now, for u*=1-Xt*3, p2-(t,ω)=2-2Xt*3, ν=limt→1p2-(t,ω)=2-2X1*3. This p2- does satisfy p2-(t,ω)=(E[(1,ν,0)C(1,t)∣Φt])2=E[ν∣Φt]=2-2Xt*3. Moreover, p3-(t,ω)∶=(E[(1,ν,0)C(1,t)∣Φt])3=(E[(1-t)ν∣Φt])3=(1-t)p2-(t,ω). (Concerning (2.14), it is evidently satisfied by q1=0, q2=-2σ, q3=-(1-t)2σ, dp1=dp1-=0, p1=p1-≡1, dp2=dp2-=-2σdBt, p2=p2-=2-2σBt, dp3=dp3-=-p2-(t,ω)+(1-t)dp2-(t,ω)=-p2(t,ω)-(1-t)2σdBt).



---

*Source: 729490-2012-12-13.xml*