# Optimal Control with Time Delays via the Penalty Method

Authors: Mohammed Benharrat; Delfim F. M. Torres
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

We prove necessary optimality conditions of Euler-Lagrange type for a problem of the calculus of variations with time delays, where the delay in the unknown function is different from the delay in its derivative. Then, a more general optimal control problem with time delays is considered. Main result gives a convergence theorem, allowing us to obtain a solution to the delayed optimal control problem by considering a sequence of delayed problems of the calculus of variations.


---

## Body

## 1. Introduction

Over the past years, there has been an increasing interest in time delay problems of the calculus of variations and control [1–4]. Such interest is explained for their importance in control and engineering [5–8]. Indeed, time delays are inherent in various real systems, such as control systems and optimal control problems in engineering [9, 10].In this paper we improve recent optimality conditions for time delay variational problems. In [11] necessary optimality conditions of Euler-Lagrange, DuBois-Reymond, and Noether type were obtained for problems of the calculus of variations with a time delay. The results of [11] were then extended to delayed variational problems with higher order derivatives in [2]. Here we model time delay variational problems in a more realistic way: while in [2, 11] the delay on functions and their derivatives (and control variables) is always the same, here we consider different delays for the functions and derivatives/controls.The text is organized as follows. In Section2 we formulate the delayed problem of the calculus of variations, where the delay in the unknown functions is different from the delay in their derivatives. The main result in this section is Theorem 4, which provides necessary optimality conditions of Euler-Lagrange type. Control strategies via an exterior penalty method are then investigated in Section 3. The idea is to replace the optimal control problem with time delays by a series of delayed problems of the calculus of variations. The main result gives a convergence theorem that allows us to obtain a solution to delayed optimal control problems with linear delayed control systems, by considering a sequence of variational problems with time delays of the type considered before in Section 2 (see Theorem 7). We end with Section 4 of conclusions.


## 2. Calculus of Variations with Time Delays

We consider the following fundamental problem of the calculus of variations with time delays, where the delay in the function we are looking for is different from the delay in its derivative:(1)min⁡∫0⊤‍L(t,x(t),x(t-τ1),x˙(t),x˙(t-τ2))dt
subject to
(2)x(t)=θ1(t),t∈[-τ1,-τ2]=:I1,x(t)=θ2(t),t∈[-τ2,0]=:I2,x(⊤)=α,
where L:[0,⊤]×R4N→R, (t,a,a-,b,b-)→L(t,a,a-,b,b-), is the Lagrangian, ⊤>0 is fixed in R, τ1 and τ2 are two given positive real numbers such that τ2<τ1<⊤, and θ1(·) and θ2(·) are given piecewise smooth functions. Let I:=[0,⊤], L2(I,RN) be the Lebesgue space of measurable functions such that
(3)∥x∥L2=(∫0⊤‍∥x(t)∥RN2dt)1/2<∞
and H1(I,RN) the Sobolev space of functions having their weak first derivative lying in L2(I,RN) and represented by
(4)x(t)=x(τ)+∫τt‍x˙(s)ds
for all τ and t in I. We denote that(i)
H is the space of all functions x:[-τ1,⊤]→RN such that x/I1∈L2(I1,RN), x/I2∈H1(I2,RN), and x/I∈H1(I,RN), which is a Hilbert space with the norm
(5)∥x∥H=(∥x/I1∥L2(I1,RN)2+∥x/I2∥H1(I2,RN)2+∥x/I∥H1(I,RN)2)1/2;(ii)
D:={x(·)∈H:x/I1=θ1,x/I2=θ2,andx(⊤)=α};(iii)
J:H→R is the functional
(6)J(x(·))=∫0⊤‍L(t,x(t),x(t-τ1),x˙(t),x˙(t-τ2))dt.Our problem (1) and (2) takes then the following form:
(7)min⁡x(·)∈DJ(x(·)).
We make the following assumptions on the data of problem (7): (A1)
LagrangianL is a C1 Carathéodory mapping; that is, it is of class C1 in (a,a-,b,b-) for almost all t∈[0,⊤] and is measurable in t for every (a,a-,b,b-);(A2)
there existγi(·)∈L2(I,R+), i=1,…,5, such that a.e. in t∈I(8)|L(t,a,a-,b,b-)|≤γ1(t),∥∂iL(t,a,a-,b,b-)∥≤γi(t),i=2,…,5,where∂iL is the partial derivative of L with respect to its ith argument.Definition 1 (cone of tangents).
LetZ be a normed space, A⊂Z, and a∈A¯. The cone of tangents T(A,a) is the set of all z∈Z with the property that there is a sequence (an) in A converging strongly to a and a sequence of nonnegative numbers (αn) such that αn(an-a)→z.Lemma 2.
The setD is an affine linear subspace of H and the cone of tangents T(D,x(·)) is given by
(9)T(D,x(·))={v(·)∈H:v(·)/I1=0,v(·)/I2=0,v(⊤)=0}.Proof.
Letv(·)∈T(D,x(·)). Then there exist (xn(·))n⊂D and λn≥0 such that xn(·)→x(·) in D implies that λn(xn(·)-x(·))→v(·) in H. Since xn(·),x(·)∈D for all n, we have
(10)xn(τ)=x(τ)=θ1(τ),τ∈[-τ1,-τ2],xn(τ)=x(τ)=θ2(τ),τ∈[-τ2,0],xn(⊤)=x(⊤)=α.
Hence,
(11)λn(xn(τ)-x(τ))=0,τ∈[-τ1,-τ2],λn(xn(τ)-x(τ))=0,τ∈[-τ2,0],λn(xn(⊤)-x(⊤))=0.
Therefore, v(·)∈H with v(τ)=0 for all τ∈[-τ1,-τ2], v(τ)=0 for almost all τ∈[-τ2,0], and v(⊤)=0. Thus,
(12)T(D,x(·))⊂{v(·)∈H:v(·)/I1=0,v(·)/I2=0,v(⊤)=α}=K.
Conversely, let v(·)∈K for x(·)∈D. Define xn(·)=x(·)+(1/n)v(·). Then n(xn(·)-x(·))=v(·) with v(·)∈H. Hence, v(·)∈T(D,x(·)).For convenience, we introduce the operator[·]τ1τ2 defined by
(13)[x]τ1τ2(t)=(t,x(t),x(t-τ1),x˙(t),x˙(t-τ2)).Proposition 3.
Under conditions(A1) and (A2), the mapping J(·) is Fréchet differentiable and
(14)J′(x(·))(v(·))=∫0⊤‍(〈∂2L[x]τ1τ2(t),v(t)〉+〈∂3L[x]τ1τ2(t),v(t-τ1)〉+〈∂4L[x]τ1τ2(t),v˙(t)〉+〈∂5L[x]τ1τ2(t),v˙(t-τ2)〉)dt.Proof.
Letv(·)∈H. We have
(15)J′(x(·);v(·))=lim⁡λ→0+1λ(J(x(·)+λv(·))-J(x(·)))=lim⁡λ→0+∫0⊤‍1λ[L[x+λv]τ1τ2(t)-L[x]τ1τ2(t)]dt.
Define
(16)Ψλ(t)=1λ[L[x+λv]τ1τ2(t)-L[x]τ1τ2(t)]
and
(17)Ψ(t)=〈∂2L[x]τ1τ2(t),v(t)〉+〈∂3L[x]τ1τ2(t),v(t-τ1)〉+〈∂4L[x]τ1τ2(t),v˙(t)〉+〈∂5L[x]τ1τ2(t),v˙(t-τ2)〉.
Then, Ψλ(t)→Ψ(t) as λ→0+ for almost all t∈[0,⊤].  On the other hand,
(18)|Ψλ(t)|≤g(t)
a.e. in t∈[0,⊤] with
(19)g(t)=γ(t)[∥v(t)∥RN+∥v(t-τ1)∥RN+∥v˙(t)∥RN+∥v(t-τ2)∥RN]
a function not depending on λ, and
(20)|Ψλ(t)|≤g(t)+1
for almost all  t∈[0,⊤] and λ sufficiently small. Since [0,⊤] has finite measure, Lebesgue’s theorem yields that
(21)∫0⊤‍Ψλ(t)dt⟶∫0⊤‍Ψ(t)dt
as λ→0+. Hence,
(22)J′(x(·))(v(·))=∫0⊤(〈∂2L[x]τ1τ2(t),v(t)〉‍+〈∂3L[x]τ1τ2(t),v(t-τ1)〉+〈∂4L[x]τ1τ2(t),v˙(t)〉+〈∂5L[x]τ1τ2(t),v˙(t-τ2)〉)dt.
This is the directional derivative of J in the direction v. To finish the proof, we need to show that J′(x(·);v(·)) is linear and bounded in v and continuous in x. The linearity is obvious. We begin by proving that J′(x(·);·) is bounded from H to R:
(23)|J′(x(·);v(·))|≤∫0⊤|〈∂2L[x]τ1τ2(t),v(t)〉|dt‍+∫0⊤|〈∂3L[x]τ1τ2(t),v(t-τ1)〉|dt‍+∫0⊤|〈∂4L[x]τ1τ2(t),v˙(t)〉|dt+∫0⊤|〈∂5L[x]τ1τ2(t),v˙(t-τ2)〉|dt≤∫0⊤γ2(t)∥v(t)∥RNdt+∫0⊤γ3(t)∥v(t-τ1)∥RNdt+∫0⊤γ4(t)∥v˙(t)∥RNdt+∫0⊤γ5(t)∥v˙(t-τ2)∥RNdt≤∫0⊤γ2(t)∥v(t)∥RNdt+∫-τ1⊤-τ1γ3(t+τ1)∥v(t)∥RNdt+∫0⊤γ4(t)∥v˙(t)∥RNdt+∫-τ2⊤-τ2γ5(t+τ2)∥v˙(t)∥RNdt≤∫0⊤γ2(t)∥v(t)∥RNdt+∫-τ1-τ2γ3(t+τ1)∥v(t)∥RNdt+∫-τ20γ3(t+τ1)∥v(t)∥RNdt+∫0⊤-τ1γ3(t+τ1)∥v(t)∥RNdt+∫0⊤γ4(t)∥v˙(t)∥RNdt+∫-τ20γ5(t+τ2)∥v˙(t)∥RNdt≤M∥v(·)∥H.
We still need to prove the continuity of J′(·). Let xn(·)→x(·) in H. Then,
(24)|[J′(xn(·))-J′(x(·))](v(·))|≤∫0⊤|〈∂2L[xn-x]τ1τ2(t),v(t)〉|‍dt+∫0⊤|〈∂3L[xn-x]τ1τ2(t),v(t-τ1)〉|‍dt+∫0⊤|〈∂4L[xn-x]τ1τ2(t),v˙(t)〉|‍dt+∫0⊤|〈∂5L[xn-x]τ1τ2(t),v˙(t-τ2)〉|‍dt≤∫0⊤∥∂2L[xn-x]τ1τ2(t)∥∥v(t)∥RN‍dt+∫0⊤∥∂3L[xn-x]τ1τ2(t)∥∥v(t-τ1)∥RNdt+∫0⊤∥∂4L[xn-x]τ1τ2(t)∥∥v˙(t)∥RN‍dt+∫0⊤∥∂5L[xn-x]τ1τ2(t)∥∥v˙(t-τ2)∥RN‍dt=I2+I3+I4+I5,
where
(25)I2≤∥∂2L[xn-x]τ1τ2(·)∥∥v(t)∥L2,I3≤∥∂3L[xn-x]τ1τ2(·)∥∥v(t)∥L2([-τ1,⊤],RN),I4≤∥∂4L[xn-x]τ1τ2(·)∥∥v˙(t)∥L2,I5≤∥∂3L[xn-x]τ1τ2(·)∥∥v˙(t)∥L2([-τ1,⊤],RN).
On the other hand, xn(·)→x(·) in H. From Lebesgue’s theorem, there exists k1,k2,k3⊂N such that k1⊂k2⊂k3 and
(26)xk(t)⟶x(t),a.e.t∈[0,⊤],∀k∈k1,x˙k(t)⟶x˙(t),a.e.t∈[0,⊤],∀k∈k1,xk(t)⟶x(t),a.e.t∈[-τ2,0],∀k∈k2,x˙k(t)⟶x˙(t),a.e.t∈[-τ2,0],∀k∈k2,xk(t)⟶x(t),a.e.t∈[-τ1,-τ2],∀k∈k3.
Hence,
(27)x˙k(t-τ2)⟶x˙(t-τ2),a.e.t∈[0,τ2],∀k∈k2,xk(t-τ1)⟶x(t-τ1),a.e.t∈[0,τ1],∀k∈k3.
Since L(t,·,·,·) is C1-Carathéodory, assumption (A2) assures from Lebesgue’s theorem that
(28)∥∂iL[xn-x]τ1τ2(·)∥⟶0,i=2,3,4,5.
This implies that I2+I3+I4+I5→0. Then, J′(xk(·))→J′(x(·)). The proof is complete.Theorem 4 (necessary optimality conditions of Euler-Lagrange type for problem (1) and (2)).
Under conditions(A1) and (A2), if x-(·) is a minimizer to problem (1) and (2), then x-(·) satisfies the following Euler-Lagrange equations with time delay:
(29)ddt{∂4L[x-]τ1τ2(t)+∂5L[x-]τ1τ2(t+τ2)}=∂2L[x-]τ1τ2(t)+∂3L[x-]τ1τ2(t+τ1),a.e.t∈[0,⊤-τ1],ddt{∂4L[x-]τ1τ2(t)+∂5L[x-]τ1τ2(t+τ2)}=∂2L[x-]τ1τ2(t),a.e.t∈]⊤-τ1,⊤-τ2],ddt∂4L[x-]τ1τ2(t)=∂2L[x-]τ1τ2(t),a.e.t∈]⊤-τ2,⊤].Proof.
Ifx-(·) is a minimizer to problem (1) and (2), then
(30)J′(x-(·))(v(·))=0
for all v(·)∈T(D,x-(·)); that is,
(31)∫0⊤‍(〈p2(t),v(t)〉+〈p3(t),v(t-τ1)〉+〈p4(t),v˙(t)〉+〈p5(t),v˙(t-τ2)〉)dt=0
for all v(·)∈T(D,x-(·)) with pi(t)=∂iL[x-]τ1τ2(t), i=2,3,4,5. Integration by parts yields
(32)∫0⊤‍〈p4(t),v˙(t)〉dt=-∫0⊤‍〈p˙4(t),v(t)〉dt,∫0⊤‍〈p5(t),v(t-τ2)〉dt=〈p5(⊤),v(⊤-τ2)〉+∫0⊤‍〈p˙5(τ),v(τ-τ2)〉dτ.
By (31) and (32), we obtain that
(33)∫0⊤‍(〈p2(t),v(t)〉+〈p3(t),v(t-τ1)〉-〈p˙4(t),v(t)〉+〈p˙5(t),v(t-τ2)〉)dt+〈p5(t),v(⊤-τ2)〉=0
for all v(·)∈T(D,x-(·)). On the other hand,
(34)∫0⊤〈p3(t),v(t-τ1)〉‍dt=∫-τ1⊤-τ1〈p3(τ+τ1),v(τ)〉dτ=∫-τ10〈p3(τ+τ1),v(τ)〉‍dτ+∫0⊤-τ1〈p3(τ+τ1),v(τ)〉dτ=∫0⊤-τ1〈p3(τ+τ1),v(τ)〉‍dτ,∫0⊤〈p˙5(t),v(t-τ2)〉‍dt=∫-τ2⊤-τ2〈p˙5(τ+τ2),v(τ)〉‍dτ=∫-τ20〈p˙5(τ+τ2)v(τ)〉‍dτ+∫0⊤-τ2〈p˙5(τ+τ2),v(τ)〉‍dτ=∫0⊤-τ2〈p˙5(τ+τ2),v(τ)〉‍dτ.
Hence,
(35)∫0⊤‍〈p2(t)-p˙4(t),v(t)〉dt+∫0⊤-τ1‍〈p3(t+τ1),v(t)〉dt-∫0⊤-τ2‍〈p˙5(t+τ2),v(t)〉dt+〈p5(⊤),v(⊤-τ2)〉=0
for all v(·)∈T(D,x-(·)). Put
(36)p-3(t+τ1)={p3(t+τ1)ift∈[0,⊤-τ1],0ift∈]⊤-τ1,⊤],q(t+τ2)={p˙5(t+τ2)ift∈[0,⊤-τ2],0ift∈]⊤-τ2,⊤].
Then,
(37)∫0⊤‍〈p2(t)-p˙4(t)+p-3(t+τ1)-q(t+τ2),v(t)〉dt+〈p5(⊤),v(⊤-τ2)〉=0
for all v(·)∈T(D,x-(·)). In particular, for v such that v(τ)=0 for almost all τ∈[-τ1,0] and v(τ)=0 for almost all τ∈[⊤-τ2,0], we have
(38)p2(t)-p˙4(t)+p-3(t+τ1)-q(t+τ2)=0a.e.t∈[0,⊤]
or
(39)p˙4(t)+p˙5(t+τ2)=p2(t)+p3(t+τ1)a.e.t∈[0,⊤-τ1],p˙4(t)+p˙5(t+τ2)=p2(t)a.e.t∈]⊤-τ1,⊤-τ2],p˙4(t)=p2(t)a.e.t∈]⊤-τ2,⊤].
The proof is complete.


## 3. Optimal Control with Time Delays

Now we prove existence of an optimal solution to more general problems of optimal control with time delays. The result is obtained via the exterior penalty method [12, 13] and Theorem 4. The optimal control problem with time delays is defined as follows:
(40)min⁡∫0⊤‍l(t,x(t),x˙(t-τ2),u(t))dt
subject to
(41)x˙(t)=Ax(t-τ1)+Bu(t),t∈[0,⊤]=:I,x(t)=θ1(t),t∈[-τ1,-τ2]=:I1,x(t)=θ2(t),t∈[-τ2,0]=:I2,x(⊤)=α,
where x(·)∈H, u(·)∈U0={u(·)∈L2([0,⊤],U):u(0)=0}, A is an N×N matrix, B is an N×m matrix, and l:[0,⊤]×RN×RN×Rm→R, (t,a,b-,c)→l(t,a,b-,c). The final time ⊤>0 is fixed in R, τ1 and τ2 are two given positive real numbers such that τ2<τ1<⊤, and, as before, θ1(·) and θ2(·) are given piecewise smooth functions. In the sequel, we denote by θ(·) the function defined by θ(t)=θ1(t), t∈I1, and θ(t)=θ2(t), t∈I2. We make the following assumptions on the data of the problem. (H1)
The mappingl is a C1-Carathéodory mapping; that is, l is C1 in (a,b-,c) for almost all t∈[0,T] and is measurable in t for every (a,b-,c)∈RN×RN×Rm.(H2)
There existγi(·)∈L2(I,R+),i=1,…,5, such that
(42)|l(t,a,b-,c)|≤γ1(t),∥∂il(t,a,b-,c)∥≤γi(t)a.e.t∈I,i=2,…,4,where∂il is the partial derivative of l with respect to its ith argument, i=2,…,4.(H3)
There existsρ>0 such that for almost all t∈[0,T] and for all (a,b-,c)∈RN×RN×Rm(43)l(t,a,b-,c)≥ρ∥c∥Rm.(H4)
l(t,a,b-,c) is convex in (b-,c).Using the exterior penalty function method, we consider the following sequence of unconstrained optimal control problems corresponding to (40) and (41):
(Pn)inf⁡∫0⊤‍l(t,x(t),x˙(t-τ2),u(t))dt+cn2∫0⊤‍∥x˙(t)-Ax(t-τ1)-Bu(t)∥RN2dt,x(t)=θ1(t)a.e.t∈I1,x(t)=θ2(t)a.e.t∈I2,x(⊤)=α,x(·)∈H,u(·)∈U0,
where cn+1≥cn, cn→∞. Denote
(44)Ln(t,a,a-,b,b-,c):=l(t,a,b-,c)+cn2∥b-Aa-+Bc∥RN2,Jn(x(·),u(·)):=∫0⊤‍Ln(t,x(t),x(t-τ1),x˙(t),x˙(t-τ2),u(t))dt,D:={x(·)∈H:x(⊤)=α}.
The sequence of unconstrained optimal control problems takes then the following form:
(Pn)inf⁡Jn(x(·),u(·)),x(·)∈D,u(·)∈U0,n∈N.Lemma 5.
The cone of tangentsT(U0,u(·)) is given by
(45)T(U0,u(·))={w(·)∈L2(I,Rm):w(0)=0}.Proof.
It is similar to the proof of Lemma2.It is well known that the penalty function method is a very effective technique for solving constrained optimization problems via unconstrained ones. The main question is the convergence of the sequence of solutions of the unconstrained optimal control problems to the original/constrained problem. Before giving the convergence theorem, we begin with some preparatory results, which are a direct consequence of the necessary optimality conditions given by Theorem4.Proposition 6.
For everyn, if (xn(·),un(·))∈D×U0 is an optimal solution to (Pn), then (i)
(46)ddtϕn(t)=A*ϕn(t+τ1)+1cnan(t)-1cnen(t)a.e.t∈[0,⊤-τ1],ddtϕn(t)=1cnan(t)-1cnen(t)a.e.t∈]⊤-τ1,⊤-τ2],ddtϕn(t)=1cnan(t)a.e.t∈]⊤-τ2,⊤],B*ϕn(t)=1cnbn(t)a.e.t∈[0,⊤],where(47)ϕn(t)=x˙n(t)-Axn(t-τ1)+Bun(t),an(t)=la′(t,xn(t),x˙n(t-τ2),un(t)),en(t)=lb-′(t,xn(t),x˙n(t-τ2),un(t)),bn(t)=lc′(t,xn(t),x˙n(t-τ2),un(t));(ii)
there existsM>0 such that ∥ϕn(t)∥X≤M for all t∈[0,⊤] and all n sufficiently large.Proof.
(i) Let(xn(·),un(·))∈D×U0 be an optimal solution to (Pn). Then, by Lemmas 2 and 5 and Theorem 4, we obtain the necessary conditions of item (i) for problem (Pn).
(ii) Sinceun(0)=0 and θ˙(0+) exists, x˙n(0) is defined and there exists k>0 such that ∥ϕn(0)∥≤k. By the first equation of item (i), we have
(48)ϕn(t)=ϕn(0)+A*∫0t‍ϕn(τ+τ1)dτ+1cn∫0t‍(an(τ)+en(τ))dτ,t∈[0,⊤-τ1].
Consequently,
(49)∥ϕn(t)∥≤∥ϕn(0)∥+∥A*∥∫0t∥ϕn(τ+τ1)∥‍dτ+Rn≤k+α∫0t∥ϕn(τ+τ1)∥‍dτ+Rn
for all t∈[0,⊤-τ1] with k=∥ϕn(0)∥, α=∥A*∥, and Rn=(1/cn)(∥γ2(·)∥L2+∥γ3(·)∥L2). By Gronwall’s lemma, we obtain that
(50)∥ϕn(t)∥≤(k+Rn)exp⁡(α(⊤-τ1))∀t∈[0,⊤-τ1].
The second and third equalities of item (i) give
(51)∥ϕn(t)∥≤∥ϕn(⊤-τ1)∥+Rn∀t∈]⊤-τ1,⊤].
Now, inequalities (50) and (51) imply that
(52)∥ϕn(t)∥≤Mn∀n,∀t∈[0,⊤]
with
(53)Mn=kexp⁡(α(⊤-τ1))+Rn(1+exp⁡(α(⊤-τ1))).
Since Rn→0, there exists M>0 such that
(54)∥ϕn(t)∥RN≤M
for all t∈[0,⊤] and for all n large.We are now ready to prove the convergence theorem, which reads as follows.Theorem 7 (penalty convergence theorem).
If hypotheses(H1)–(H4) hold and problem (40) and (41) has a finite value, then the sequence (xn(·),un(·))n of solutions to (Pn) contains a subsequence (xk(·),uk(·))k such that (i)
xk(·)→x(·) strongly in C(I,RN);(ii)
uk(·)→u(·) weakly in L2(I,Rm);(iii)
x˙k(·)→x˙(·) weakly in L2(I,RN);with (x(·),u(·)) a solution to problem (40) and (41).Proof.
Let(xn(·),un(·))∈D×U0 be an optimal solution to (Pn) for every n. By Proposition 6,
(55)∥x˙n(t)∥≤M+∥A∥∥xn(t-τ1)∥+∥B∥∥un(t)∥≤M+β∥xn(t-τ1)∥+σ∥un(t)∥.
Because
(56)xn(t)=θ(0)+∫0tx˙n(τ)dτ‍∀t∈[0,⊤],xn(t)=θ(t)a.e.t∈[-τ1,0],
it follows that
(57)∥xn(t)∥≤∥θ(0)∥+∫0t∥x˙n(τ)∥‍dτ≤∥θ(0)∥+M⊤+β∫0t∥xn(τ-h)∥dτ+σ∫0t∥un(τ)∥dτ.
On the other hand, if M denotes the finite value of (40) and (41), then
(58)∫0⊤‍l(t,xn(t),un(t))dt≤Jn(xn(·),un(·))≤M.
By assumption (H3), there exists K>0 such that
(59)∥un(·)∥L2≤K.
Thus,
(60)∥xn(t)∥≤∥θ(0)∥+M⊤+σ⊤K+β∫-τ10‍∥θ(τ)∥dτ+β∫0t-τ1‍∥xn(τ)∥dτ.
By Gronwall’s lemma, we obtain that
(61)∥xn(t)∥≤ψfornsufficientlylarge,∀t∈[0,⊤],
where
(62)ψ=(∥θ(0)∥+M⊤+σ⊤K+βτ1∥θ(·)∥)exp⁡(β(⊤-τ1)).
Similarly, for n sufficiently large,
(63)∥x˙n(t)∥≤M+β∥xn(t-τ1)∥+σ∥un(t)∥.
For all t∈[0,h], we have
(64)∥x˙n(t)∥≤M+γ(t)+β∥θ(t-τ1)∥+σ∥un(t)∥=ω(t).
Since ω(·)∈L2(I,R) and (un(·))n is bounded in L2(I,Rm), with [0,h] of finite measure, there exists ϱ>0 such that
(65)∥x˙n(·)∥≤ϱinL2([0,τ1],RN),fornsufficientlylarge.
For all t∈[τ1,⊤] we have
(66)∥x˙n(t)∥≤M+γ(t)+βψ+σ∥un(t)∥.
As before, we can assert that
(67)∃ϱ^>0:∥x˙n(·)∥≤ϱ^inL2([τ1,⊤],RN)fornsufficientlylarge.
By (65) and (67), there exists η>0 such that
(68)∥x˙n(·)∥≤η
in L2([0,⊤],RN) for n sufficiently large. Therefore, there exists a subsequence (x˙k(·))k of (x˙n(·))n converging to σ(·)∈L2(I,RN). Since xn(t)=θ(0)+∫0t‍x˙n(τ)dτ for all t∈I, by the use of (61), the sequence (xn(·))n is equi-bounded and equi-continuous (because (x˙n(·))n is bounded in L2(I,RN)). Ascoli’s theorem implies that
(69)xk(·)⟶x(·)stronglyinC(I,RN).
Since
(70)xk(0)=θ(0),∫0t‍x˙n(τ)dτ⟶∫0t‍σ(τ)dτ,
we obtain that x(t)=θ(0)+∫0t‍σ(τ)dτ and x˙(t)=σ(t) a.e. t∈I. The sequence (un(·))n is bounded in L2(I,Rm). Thus, there exists a subsequence (uk(·))k such that uk(·)→u(·) weakly in L2(I,RN). To complete the proof, we show that (x(·),u(·)) is an optimal solution to (40)-(41). By Proposition 6, we have
(71)B*ϕk(t)=1ckbk(t)a.e.t∈I.
Hence,
(72)∫0t‍∥B*ϕk(τ)∥dτ=1ck∫0t‍∥bk(τ)∥dτ≤1ckM
with M=⊤∥γ4(·)∥L2. We conclude that
(73)∫0t‍B*ϕk(τ)dτ⟶0∀t∈I.
On the other hand,
(74)∫0t‍B*ϕk(τ)dτ⟶∫0t‍B*ϕ(τ)dτ∀t∈I.
Consequently,
(75)∫0t‍B*ϕ(τ)dτ=0∀t∈I.
This implies that
(76)B*ϕ(t)=0∀t∈I.
Thus,
(77)ϕ(t)=0∀t∈I,x˙(t)=Ax(t-τ1)+Bu(t)∀t∈I
and x(t)=θ(t) a.e. t∈[-τ1,0], x(⊤)=α. Then, (x(·),u(·)) is an admissible pair and
(78)∫I‍l(t,x(t),x˙(t-τ2),u(t))dt≥M.
On the other hand,
(79)∫I‍l(t,xk(t),x˙k(t-τ2),uk(t))dt≤M.
Now the hypotheses (H1), (H2), and (H4), together with Lebesgue’s theorem, assert that
(80)∫I‍l(t,x(t),x˙(t-τ2),u(t))dt≤M;
that is,
(81)∫I‍l(t,x(t),x˙(t-τ2),u(t))dt=M.
This implies that the pair (x(·),u(·)) is a solution to problem (40) and (41).


## 4. Conclusion

New optimality conditions for problems of the calculus of variations and optimal control with time delays, where the delay in the unknown function differs from the delay in its derivative/control, were obtained. The proofs are first given in the simpler context of the delayed calculus of variations and then extended to delayed optimal control problems by using a penalty method. New results include a convergence theorem (see Theorem7), which is of great practical interest because it allows us to obtain a solution to a delayed optimal control problem by considering a sequence of simpler problems of the calculus of variations. Previous results in the literature [2, 3, 11] consider the delay in the unknown function to be the same as the delay in its derivative. There is, however, no justification for the delays to be the same. In contrast with those results, here we consider the case of multiple time delays. Moreover, the procedure of our proofs is completely different from the case of one time delay only, which relies on the Lagrange multiplier method. Such approach introduces a new unknown function, the Lagrange multiplier, for which it is hard to set the interpolation space. Indeed, the Lagrange multiplier must be carefully selected in order to be possible to obtain an accurate solution. Otherwise, the resulting system of equations may become singular, in particular if the number of degrees of freedom is too large. Here we use a penalty method, which requires only the choice of one scalar parameter. Big values of this parameter are used in order to impose the boundary conditions in a proper manner. Furthermore, in our case the use of the penalty method replaces a constrained optimization problem (the delayed optimal control problem) by a sequence of unconstrained problems of the calculus of variations with time delay whose solutions converge to the solution of the original constrained problem. Similarly to [11], our results can be easily extended for controls with time delay.



---

*Source: 250419-2014-09-02.xml*