# Solutions to the Hamilton-Jacobi equation for Bolza problems with discontinuous time dependent data

Julien Bernis, Piernicola Bettiol

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Julien Bernis, Piernicola Bettiol. Solutions to the Hamilton-Jacobi equation for Bolza problems with discontinuous time dependent data. ESAIM: Control, Optimisation and Calculus of Variations, 2020, 26, pp.66. 10.1051/cocv/2019041. hal-02946118

HAL Id: hal-02946118

https://hal.science/hal-02946118

Submitted on 22 Sep 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# SOLUTIONS TO THE HAMILTON-JACOBI EQUATION FOR BOLZA PROBLEMS WITH DISCONTINUOUS TIME DEPENDENT DATA

JULIEN BERNIS AND PIERNICOLA BETTIOL*

**Abstract.** We consider a class of optimal control problems in which the cost to minimize comprises both a final cost and an integral term, and the data can be discontinuous with respect to the time variable in the following sense: they are continuous w.r.t. $t$ on a set of full measure and have everywhere left and right limits. For this class of Bolza problems, employing techniques coming from viability theory, we give characterizations of the value function as the unique generalized solution to the corresponding Hamilton-Jacobi equation in the class of lower semicontinuous functions: if the final cost term is extended valued, the generalized solution to the Hamilton-Jacobi equation involves the concepts of lower Dini derivative and the proximal normal vectors; if the final cost term is a locally bounded lower semi-continuous function, then we can show that this has an equivalent characterization in a viscosity sense.

**Mathematics Subject Classification.** 49J15, 49J21, 49L20.

Received November 30, 2018. Accepted June 23, 2019.

## 1. INTRODUCTION

Consider the non autonomous Bolza problem:

$$ (P_{S,x_0}) \quad \begin{cases} \text{Minimize} & \int_S^T L(t, x(t), \dot{x}(t))dt + g(x(T)) \\ \text{over arcs} & x \in W^{1,1}([S, T], \mathbb{R}^n) \text{ satisfying} \\ & \dot{x}(t) \in F(t, x(t)) \text{ for almost every } t \in [S, T], \\ & x(S) = x_0, \end{cases} $$

in which $[S, T]$ is a given interval, $x_0 \in \mathbb{R}^n$ is a given initial datum, $g: \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ and $L: [S, T] \times \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ are given functions, and $F: [S, T] \times \mathbb{R}^n \to \mathbb{R}^n$ is a given multivalued function. The reference problem $(P_{S,x_0})$ can be embedded in a family of problems $(P_{t,x})$ parametrized by pairs of initial data $(t, x) \in [S, T] \times \mathbb{R}^n$. This leads to the concept of the value function for $(P_{t,x})$ $V: [S, T] \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$, which, for all $(t, x) \in [S, T] \times \mathbb{R}^n$, is defined taking the infimum cost for $(P_{t,x})$:

$$ V(t, x) := \inf \left\{ \int_t^T L(s, x(s), \dot{x}(s))ds + g(x(T)) \mid x(\cdot) \text{ F-trajectory on } [t, T], x(t) = x \right\}. $$

Keywords and phrases: Hamilton-Jacobi equation, discontinuous data, value function.

Laboratoire de Mathématiques, Université de Bretagne Occidentale, 6 Avenue Victor Le Gorgeu, 29200 Brest, France.

* Corresponding author: piernicola.bettiol@univ-brest.fr
Here, an F-trajectory on the interval $[s, t] \subset [S, T]$ is an absolutely continuous arc $x(\cdot) : [s, t] \to \mathbb{R}^n$ which satisfies the reference differential inclusion $\dot{x}(\sigma) \in F(\sigma, x(\sigma))$ for a.e. $\sigma \in [s, t]$. We shall consider characterizations of $V(\cdot, \cdot)$ as the unique solution – in a suitable generalized sense – to the Hamilton-Jacobi equation:

$$ \left\{ \begin{array}{l} \partial_t \varphi(t, x) + \inf_{v \in F(t,x)} \{\partial_x \varphi(t, x) \cdot v + L(t, x, v)\} = 0 \\ \varphi(T, x) = g(x), \end{array} \right. \qquad (\text{HJE}) $$

when we may have a discontinuous behaviour of F and L w.r.t. the time variable t. Many techniques have been employed to characterize the value function as solution to (HJE), mainly coming from viscosity solutions theory and viability theory. In both contexts a lot of work has been done including the case of discontinuous time dependence problems (see for instance the monographs [2, 5, 9, 20] and the references therein). In this paper we employ nonsmooth analysis tools and a viability approach to provide value function characterizations involving the notions of lower Dini derivative (also called contingent epiderivative), proximal subdifferential, and Fréchet subdifferential and superdifferential. An important feature is that we allow the final cost function g to be a lower semicontinuous function, possibly extended valued, incorporating implicit terminal constraints. As a consequence the natural class of functions in which we study the value function is the set of lower semicontinuous functions.

In presence of extended terminal costs, the first result, using viability theory, characterizing lower semicontinuous value functions as solutions to (HJE) in a generalized sense which involves the contingent epiderivatives, is obtained in [12]. In the same paper we can find also characterizations using (Fréchet) subdifferentials, and eventually both subdifferentials and superdifferentials leading to a comparison with viscosity solutions for continuous value functions. These results have been achieved for the Mayer problem (i.e. for $L = 0$) assuming velocity sets F which are continuous in $(t, x)$. A further significant contribution is [9], in which appropriate invariance theorems allow to characterize the value function also considering proximal subdifferentials.

Passing to discontinuous time-dependent optimal control problems, the relevance of the role of lower Dini derivatives to deal with measurable time-dependence was highlighted by [21]. Simple examples illustrate that the value function might not be the unique lower semicontinuous generalized (according to the concepts above-mentioned) solution to (HJE) in an ‘almost everywhere’ sense (cf. the discussion in [4]). However, uniqueness properties of the solution can be derived for the mere measurable time dependent case imposing additional conditions on the class of functions which are candidate to be solutions, such as the epigraph of the candidate solution is absolutely continuous w.r.t. t, see [14].

A different perspective has been recently suggested in [4] for the intermediate case (between the continuous one and the merely measurable one) when the multifunction $t \rightsquigarrow F(t, x)$ has everywhere one-sided limits, for all $x$, and is continuous on the complement of a zero-measure subset of $[S, T]$ (without necessarily imposing further a priori regularity conditions such as the absolute continuity of the epigraph of the candidate solutions). In this context, considering optimal control problems with a final cost term (i.e. $L = 0$), the value function turns out to be the unique lower semicontinuous solution to (HJE) taking into account ‘everywhere in t’ characterizations which involve the concepts of lower Dini derivative and the proximal subdifferential. Further important features of the results obtained in [4] are: the presence of left and right limits $F(t^+, x)$ and $F(t^-, x)$ (the role of which cannot be exchanged) in the characterizing conditions and the presence of the horizontal proximal subdifferentials in the concept of the proximal solution.

The main objective of this paper is to explore lower semicontinuous characterizations of the value function in the context of non-autonomous Bolza problems, in which the velocity set F satisfies the same assumptions as in [4]. The Lagrangian L is assumed to have the same behaviour in t ($L(\cdot, x, v)$ is continuous on a set of full measure and has everywhere left and right limits), but is just continuous w.r.t. x. In addition, L satisfies standard conditions in v (such as convexity and boundedness on bounded sets). We observe that it would be natural to invoke a well-known augmentation technique and rewrite the reference Bolza problem ($P_{S,x_0}$) in
a Mayer form:

$$ egin{cases} 
\text{Minimize } g(x(T)) + y(T) \\
\text{over arcs } (x, y) \in W^{1,1}([S, T], \mathbb{R}^{n+1}) \text{ satisfying} \\
(\dot{x}(t), \dot{y}(t)) \in G(t, x(t)) \text{ for almost every } t \in [S, T], \\
x(S) = x_0, \quad y(S) = 0
\end{cases} 
$$

where $G(t, x) := \{(v, w) | v \in F(t, x), w \ge L(t, x, v)\}$. Even if this method provides a good insight of a correct outcome, previous results on the Mayer problem are not applicable in our case. On the other hand, keeping the Bolza formulation of the reference problem allows us, for instance, to impose weaker assumptions on the Lagrangian $L$, avoiding additional (and more restrictive) Lipschitz continuity conditions of $L$ w.r.t. the state variable $x$, that would be otherwise necessary to impose if we passed to the Mayer form, and which is typically required in previous work for the Mayer problem (cf. [4, 12, 13, 14]). Therefore, the mere state augmentation technique does not simplify the task: we would add a step in the analysis and eventually end up with a (possibly more involved) problem, with exactly the same difficulties as we left the reference optimal control problem in the Bolza form.

Our first main result (see Thm. 2.1) provides a characterization of lower semicontinuous extended valued value functions involving both the notions of generalized solution in terms of lower Dini derivatives and in terms of proximal normals to epigraph sets (confirming that a result consistent with ([4], Thm. 2.2) can be obtained also for the class of Bolza problems considered here). The second main result of our paper gives a positive answer to an important question (highlighted in [4]): it was not known whether to achieve an extended-sense viscosity solution characterization of lower semicontinuous value functions would require employing horizontal Fréchet subderivatives (and superderivatives). Theorem 2.2 gives (together with the examples in Sect. 2.4) an answer to this issue and represents, at the same time, an extension to earlier viscosity solutions characterizations such as in [12, 14] (and [13] for the state constraints free case), to locally bounded lower semicontinuous value functions for Bolza problems with $F$ and $L$ discontinuous in $t$ and a discontinuous final cost term $g$.

To complete the huge picture of this strand, we recall that the viability approach is applicable also to characterize value functions for state constrained optimal control problems (cf. [4, 13, 15, 20]). In this case, the analysis requires some compatibility conditions of the velocity sets $F$ with the state constraint (called ‘existence of inward/outward pointing conditions’), which conveys more restrictive assumptions on $F$ and is based on some distance estimates results. The discussion on these technical aspects together with the appropriate assumptions which allow to revisit our results in the context of the state constrained Bolza problems goes beyond the main purpose of the present paper and is part of an ongoing project (cf. [3]).

The paper is organized as follows. In Section 2 we display the employed notation, the invoked assumptions (together with an hypotheses reduction technique), our main results (Thms. 2.1 and 2.2) accompanied by some refinements and a discussion based on three illustrative examples. The third section is dedicated to some preliminary results. Section 4 provides the proof of Theorem 2.1, which is split into three main steps. The proof of Theorem 2.2 is provided in Section 5.

## 2. Main results

### 2.1. Notation

In the paper we write $\mathbb{R}_+$ the set of non negative real numbers, i.e. $\{x \in \mathbb{R} | r \ge 0\}$, and $\mathbb{B}$ for the closed unit ball in $\mathbb{R}^n$. We denote the Lebesgue subsets of $[S, T]$ and the Borel subsets of $\mathbb{R}^m$ by $\mathcal{L}$ and $\mathcal{B}^m$ respectively. The (associated) product $\sigma$-algebra of sets in $[S, T] \times \mathbb{R}^m$ is written $\mathcal{L} \times \mathcal{B}^m$. We denote by $L^p([\alpha, \beta], \mathbb{R}^n)$ the space of $L^p$ functions for the Lebesgue measure, that are defined on $[\alpha, \beta]$, and take values in $\mathbb{R}^n$. We write $W^{1,1}([\alpha, \beta], \mathbb{R}^n)$, the space of absolutely continuous function for the Lebesgue measure endowed with
the norm:

$$ \|f\|_{W^{1,1}} := |f(\alpha)| + \int_{\alpha}^{\beta} |\dot{f}(s)|ds, \text{ for all } f \in W^{1,1}([\alpha, \beta], \mathbb{R}^n). $$

Let $D \subset \mathbb{R}^m$, we denote by $\co D$, $\overline{D}$ and $\co \overline{D}$ respectively the convex hull, the closure and the closed convex hull of $D$. The polar cone $D^*$ to a subset $D$ is given by:

$$ D^* := \{v \in \mathbb{R}^m \mid \forall w \in D, v \cdot w \le 0\}. $$

For arbitrary nonempty closed sets in $\mathbb{R}^n$, $C'$ and $C$, we denote by $d_H(C, C')$ the 'Hausdorff distance' between $C$ and $C'$:

$$ d_H(C, C') := \inf\{\beta > 0 \mid C' \subset C + \beta\mathbb{B}\} \lor \inf\{\beta > 0 \mid C \subset C' + \beta\mathbb{B}\}. $$

Take a closed set $C \subset \mathbb{R}^m$ and $x \in \mathbb{R}^m$. Then $\min_{y \in C}\{|x - y|\}$ is the distance of $x$ from the set $C$ and is written $d_C(x)$. If $f: C \subset \mathbb{R}^m \to \mathbb{R}$ is a locally bounded function, we denote its lower (resp. upper) semicontinuous envelope by:

$$ f_*(x) := \liminf_{y \xrightarrow{C} x} f(y) \left( \text{resp. } f^*(x) := \limsup_{y \xrightarrow{C} x} f(y) \right), \text{ for every } x \in C. $$

The notation $y \xrightarrow{C} x$ means that we are considering convergent sequences $(y_i)_{i \in \mathbb{N}}$ such that $y_i \to x$, and each element $y_i$ belongs to $C$. An increasing function $\omega: \mathbb{R}_+ \to \mathbb{R}_+$ is a modulus of continuity if $\lim_{s \to 0} \omega(s) = 0$.

We also recall some basic concepts and tools coming from nonsmooth analysis (detailed dissertations of which can be found in the monographs [1, 5, 7, 9, 20]). Consider a set $D \subset \mathbb{R}^m$, a point $x \in \overline{D}$ and a multifunction $G(\cdot): D \rightsquigarrow \mathbb{R}^m$. The limit inferior and the limit superior of $G(\cdot)$ at $x$ along $D$ (in the Kuratowski sense) are the sets

$$ \liminf_{y \xrightarrow{D} x} G(y) := \{v \in \mathbb{R}^m \mid \limsup_{y \xrightarrow{D} x} d_{G(y)}(v) = 0\}, $$

$$ \limsup_{y \xrightarrow{D} x} G(y) := \{v \in \mathbb{R}^m \mid \liminf_{y \xrightarrow{D} x} d_{G(y)}(v) = 0\}. $$

The *Bouligand tangent cone* (alternatively referred to as contingent cone) $T_C(x)$ to a closed set $C \subset \mathbb{R}^m$ at $x \in C$ is defined by:

$$ T_C(x) := \{v \in \mathbb{R}^m \mid \liminf_{h \to 0^+} \frac{d_C(x + hv)}{h} = 0\} = \limsup_{h \to 0^+} \frac{C - x}{h}. $$

The proximal normal cone to $C$ at $x \in C$, denoted $N_C^P(x)$, is defined by:

$$ N_C^P(x) := \{\eta \in \mathbb{R}^m \mid \exists M \ge 0 \text{ s.t. } \eta \cdot (y - x) \le M|y - x|^2, \forall y \in C\}. $$

The strict normal cone $\hat{N}_C(x)$ to $C$ at $x$ is defined as follows

$$ \hat{N}_C(x) := \{\eta \in \mathbb{R}^m \mid \limsup_{y \xrightarrow{C} x} |y - x|^{-1} \eta \cdot (y - x) \le 0\}. $$
We have $\hat{N}_C(x) = [T_C(x)]^*$ and

$$N_C^P(x) \subset \hat{N}_C(x). \qquad (2.1)$$

Consider an extended valued function $\varphi : \mathbb{R}^m \to \mathbb{R} \cup \{\pm\infty\}$. We write $\text{dom}(\varphi) := \{x \in \mathbb{R}^m | \varphi(x) \neq \pm\infty\}$, $\text{epi}\varphi := \{(x, r) \in \mathbb{R}^{m+1} | r \geq \varphi(x)\}$, and $\text{hyp}\varphi := \{(x, r) \in \mathbb{R}^{m+1} | r \leq \varphi(x)\}$. Take $x \in \text{dom}(\varphi)$ and $d \in \mathbb{R}^m$. The lower Dini derivative (also called the contingent epiderivative, cf. [1, 5, 16]) of $\varphi$ at $x$ in the direction $d \in \mathbb{R}^m$, denoted $D_\uparrow\varphi(x, d)$, is defined by:

$$D_\uparrow\varphi(x, d) := \liminf_{\substack{h \downarrow 0 \\ e \to d}} h^{-1}(\varphi(x + he) - \varphi(x)).$$

Similarly, one can also define the upper Dini derivative (alternatively referred to as the contingent hypoderivative), of $\varphi$ at $x$ in the direction $d \in \mathbb{R}^m$, denoted $D_\downarrow\varphi(x, d)$:

$$D_\downarrow\varphi(x, d) := \limsup_{\substack{h \downarrow 0 \\ e \to d}} h^{-1}(\varphi(x + he) - \varphi(x)).$$

We evoke the following useful relations (see [1]):

$$T_{\text{epi} \varphi}(x, \varphi(x)) = \text{epi} D_{\uparrow} \varphi(x, \cdot), \qquad (2.2)$$

and

$$T_{\text{hyp} \varphi}(x, \varphi(x)) = \text{hyp} D_{\downarrow} \varphi(x, \cdot). \qquad (2.3)$$

We recall also that if $U$ is an extended valued function defined on $[S, T] \times \mathbb{R}^n$, taking $\epsilon \in \{1, -1\}$, then for $(t, x) \in \text{dom}(U)$ we can use a simpler expression for $D_\uparrow U((t, x), (\epsilon, d))$:

$$D_\uparrow U((t, x), (\epsilon, d)) = \liminf_{\substack{h \downarrow 0 \\ e \to d}} h^{-1}(U(t + \epsilon h, x + he) - U(t, x)).$$

## 2.2. Hypotheses

In this paper we shall invoke the following hypotheses: for every given positive number $R_0$, there exist functions $c_F(\cdot) \in L^1([S, T], \mathbb{R}_+)$ and $k_F(\cdot) \in L^1([S, T], \mathbb{R}_+)$, a modulus of continuity $\omega(\cdot) : \mathbb{R}_+ \to \mathbb{R}_+$, and constants $c_0 > 0$, $M_0 > 0$ such that:

(H1): (i) The multivalued function $F : [S, T] \times \mathbb{R}^n \rightsquigarrow \mathbb{R}^n$ takes convex, closed, nonempty values. For every $x \in \mathbb{R}^n$, $F(\cdot, x)$ is Lebesgue mesurable on $[S, T]$.

(ii) The function $g : \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ is lower semicontinuous, with nonempty domain.

(H2): (i) For almost every $t \in [S, T]$ and $x \in \mathbb{R}^n$

$$F(t, x) \subset c_F(t)(1 + |x|)\mathbb{B}.$$

(ii) For all $(t, x) \in [S, T] \times R_0\mathbb{B}$

$$F(t, x) \subset c_0\mathbb{B}.$$
(H3): (i)

$d_H(F(t, x'), F(t, x)) \le \omega(|x - x'|)$, for all $x, x' \in R_0\mathbb{B}$ and $t \in [S, T]$.

(ii)

$F(t, x') \subset F(t, x) + k_F(t)|x - x'|\mathbb{B}$, for all $x, x' \in R_0\mathbb{B}$ and for a.e. $t \in [S, T]$.

(H4): (i) For each $x \in \mathbb{R}^n$, $s \in [S, T[$, and $t \in ]S, T]$ the following limits (in the sense of Kuratowski) exist and are nonempty

$$F(s^+, x) := \lim_{s' \downarrow s} F(s', x) \quad \text{and} \quad F(t^-, x) := \lim_{t' \uparrow t} F(t', x).$$

(ii) For almost every $s \in [S, T[$ and $t \in ]S, T]$, and every $x \in \mathbb{R}^n$ we have

$$F(s^+, x) = F(s, x) \quad \text{and} \quad F(t^-, x) = F(t, x).$$

(H5): (i) The Lagrangian $L: [S, T] \times \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ is $\mathcal{L} \times \mathcal{B}^{n+n}$-measurable. For every $t \in [S, T]$ and $x \in \mathbb{R}^n$, $L(t, x, \cdot)$ is convex.

ii) $L$ is locally bounded in the following sense

$$|L(t, x, v)| \le M_0, \quad \text{for all} \quad (t, x, v) \in [S, T] \times R_0\mathbb{B} \times 2c_0\mathbb{B}.$$

(H6): (i) $|L(t, x', v) - L(t, x, v)| \le \omega(|x - x'|)$, for all $x, x' \in R_0\mathbb{B}$, $t \in [S, T]$ and $v \in c_0\mathbb{B}$.

(ii) $L(t^-, x, v) := \lim_{t' \uparrow t} L(t', x, v)$ exists for every $(t, x, v) \in ]S, T] \times R_0\mathbb{B} \times c_0\mathbb{B}$, and $L(t^-, x, v) = L(t, x, v)$ for a.e. $t \in ]S, T]$ and for all $(x, v) \in R_0\mathbb{B} \times c_0\mathbb{B}$.

(iii) $L(s^+, x, v) := \lim_{s' \downarrow s} L(s', x, v)$ exists for every $(s, x, v) \in [S, T[ \times R_0\mathbb{B} \times c_0\mathbb{B}$, and $L(s^+, x, v) = L(s, x, v)$ for a.e. $s \in [S, T[$ and for all $(x, v) \in R_0\mathbb{B} \times c_0\mathbb{B}$.

### 2.2.1. A priori boundedness and hypotheses reduction technique

We observe that condition (H2) guarantees a well-known a priori uniform boundedness property for the $F$-trajectories. More precisely, if we take initial data $(t, x) \in [S, T] \times \mathbb{R}^n$ and an $F$-trajectory $y \in W^{1,1}([t, T], \mathbb{R}^n)$ such that $y(t) = x$, then for every $s \in [t, T]$, $y(s) \in (1 + |x|) \exp \left( \int_S^T c_F(s)ds \right) \mathbb{B}$. Set $R_0 := (1 + |x|) \exp \left( \int_S^T c_F(s)ds \right)$, then, owing to (H2) ii), for almost every $s \in [t, T]$, $\dot{y}(s) \in c_0\mathbb{B}$. As a consequence, once we fix the initial data $(t, x)$, using a standard hypotheses reduction argument (cf. [4] or [20]), when we are interested in studying the behaviour of the value function at $(t, x)$, we can impose much stronger assumptions. More precisely, we introduce the multifunction $\hat{F}: [S, T] \times \mathbb{R}^n \rightsquigarrow \mathbb{R}^n$

$$\hat{F}(s, y) := \begin{cases} F(s, y) & \text{if } |y| \le R_0 \\ F(s, R_0y/|y|) & \text{if } |y| > R_0, \end{cases}$$

and the function $\tilde{L}: [S, T] \times \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$

$$\hat{L}(s, y, v) := \begin{cases} L(s, y, v) & \text{if } |y| \le R_0 \\ L(s, R_0y/|y|, v) & \text{if } |y| > R_0. \end{cases}$$
The multifunction $\hat{F}(\cdot, \cdot)$ and the function $\hat{L}(\cdot, \cdot, \cdot)$ satisfy hypotheses (H1), (H2)*, (H3)*, (H4), (H5)* and (H6)*, where we denote by (H2)*, (H3)*, (H5)* and (H6)* the global (stronger) version of conditions (H2), (H3), (H5) and (H6), in which we have removed the constant $R_0$.

The data of the problem ($P_{t,x}$) involving either ($F, L$) or ($\hat{F}, \hat{L}$) do coincide in a neighbourhood of the reference point ($t, x$). It follows that in the forthcoming analysis we can invoke the more restrictive version of conditions (H1)–(H6) without loss of generality.

## 2.3. Characterizations of lower semicontinuous value functions

We consider the following family of minimization problems indexed by initial data $(t, x) \in [S, T] \times \mathbb{R}^n$:

$$ (P_{t,x}) \quad \begin{cases} \text{Minimize } \int_t^T L(s, x(s), \dot{x}(s))ds + g(x(T)) \\ \text{over the arcs } x \in W^{1,1}([t, T], \mathbb{R}^n) \text{ satisfying} \\ \dot{x}(s) \in F(s, x(s)), \text{ for almost every } s \in [t, T], \\ x(t) = x. \end{cases} $$

We recall that the value function $V: [S, T] \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ is defined by the infimum cost for ($P_{t,x}$):

$$ V(t, x) = \inf(P_{t,x}), \quad \text{for all } (t, x) \in [S, T] \times \mathbb{R}^n. $$

The first result provides a characterization of lower semicontinuous extended valued value functions in a generalized sense involving the concepts of Dini derivative and proximal normal (to the epigraph); these are sometimes referred to as “lower Dini solutions” and “proximal solutions” (cf. [7, 20]).

**Theorem 2.1. (Characterization of lower semicontinuous extended valued value functions)** Assume (H1)–(H6). Let $U: [S, T] \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ be an extended valued function. Then the assertions (a), (b) and (c) below are equivalent.

(a) The function $U$ is the value function for ($P_{t,x}$): $U = V$.

(b) The function $U$ is lower semicontinuous and satisfies:

i) for every $(t, x) \in ([S, T] \times \mathbb{R}^n) \cap \text{dom}(U)$

$$ \inf_{v \in F(t^+, x)} [D_\uparrow U((t, x), (1, v)) + L(t^+, x, v)] \le 0; $$

ii) for every $(t, x) \in ([S, T] \times \mathbb{R}^n) \cap \text{dom}(U)$

$$ \sup_{v \in F(t^-, x)} [D_\uparrow U((t, x), (-1, -v)) - L(t^-, x, v)] \le 0; \quad (2.4) $$

iii) For all $x \in \mathbb{R}^n$, $U(T, x) = g(x)$.

(c) The function $U$ is lower semicontinuous and satisfies:

i) for every $(t, x) \in ([S, T] \times \mathbb{R}^n) \cap \text{dom}(U)$

$$ \xi^0 + \inf_{v \in F(t^+, x)} [\xi^1 \cdot v + \lambda L(t^+, x, v)] \le 0, \text{ for all } (\xi^0, \xi^1, -\lambda) \in N_{\text{epi}}^P((t, x), U(t, x)); $$
ii) for every $(t, x) \in (]S, T[ \times \mathbb{R}^n) \cap \text{dom}(U)$

$$ \xi^0 + \inf_{v \in F(t^-, x)} [\xi^1 \cdot v + \lambda L(t^-, x, v)] \ge 0, \text{ for all } (\xi^0, \xi^1, -\lambda) \in N_{\text{epi } U}^P((t, x), U(t, x)); \quad (2.5) $$

iii) for every $x \in \mathbb{R}^n$,

$$ \liminf_{\{(t', x') \to (S, x) \mid t' > S\}} U(t', x') = U(S, x), $$

and

$$ \liminf_{\{(t', x') \to (T, x) \mid t' < T\}} U(t', x') = U(T, x) = g(x). $$

We consider now the case when the final cost is lower semicontinuous and locally bounded. In this case it is immediate to see that the value function acquires the same properties. In presence of a locally bounded candidate $U$ to be a solution to an Hamilton-Jacobi equation, a well-known approach in viscosity solutions theory suggests to consider its lower and upper semicontinuous envelopes and check whether the properties of supersolution and subsolution in the viscosity sense are satisfied (cf. [2]). From the perspective developed in our paper, this idea leads to a notion of viscosity solution expressed in terms of strict normals to the epigraph and the hypograph of the candidate solution $U$.

**Theorem 2.2. (Characterization of lower semicontinuous locally bounded value functions)** Assume (H1)-(H6). Suppose, in addition, that $g$ is locally bounded and satisfies $(g^*)_* = g$. Let $U : [S, T] \times \mathbb{R}^n \to \mathbb{R}$ be a locally bounded function. Then, the assertions (a), (b) and (c) of Theorem 2.1 are equivalent to (d) below.

(d) $U$ is lower semicontinuous and satisfies:

i) For every $(t, x) \in ]S, T[ \times \mathbb{R}^n$

$$ \xi^0 + \inf_{v \in F(t^+, x)} [\xi^1 \cdot v + \lambda L(t^+, x, v)] \le 0, \text{ for all } (\xi^0, \xi^1, -\lambda) \in \hat{N}_{\text{epi } U}((t, x), U(t, x)); $$

ii) for every $(t, x) \in ]S, T[ \times \mathbb{R}^n$

$$ - \xi^0 + \sup_{v \in F(t^+, x)} [-\xi^1 \cdot v - \lambda L(t^+, x, v)] \le 0, \text{ for all } (-\xi^0, -\xi^1, \lambda) \in \hat{N}_{\text{hyp } U^*}((t, x), U^*(t, x)); \quad (2.6) $$

iii) for every $x \in \mathbb{R}^n$,

$$ \liminf_{\{(t', x') \to (S, x) \mid t' > S\}} U(t', x') = U(S, x), $$

$$ U(T, x) = g(x), $$

and

$$ U^*(T, x) = g^*(x). $$

If the condition $(g^*)_* = g$ in Theorem 2.2 is removed, then the implication is valid only in one sense.

**Proposition 2.3.** Assume (H1)-(H6) are satisfied and that $g$ is locally bounded. Let $U : [S, T] \times \mathbb{R}^n \to \mathbb{R}$ be a locally bounded function. Then, the assertions (a), (b) or (c) of Theorem 2.1 imply (d) of Theorem 2.2.
Imposing also the lower semicontinuity of $L$ w.r.t. $t$, we obtain the following result.

**Proposition 2.4.** Assume (H1)-(H6). If, in addition, we suppose that $L(\cdot, x, v)$ is a lower semicontinuous function for all $(x,v) \in R_0B \times c_0B$, then the assertions of Theorem 2.1, Theorem 2.2 and Proposition 2.3 remain valid when we replace $L(t^+, x, v)$ by $L(t, x, v)$ in conditions (b) i), (c) i) and (d) i).

**Remark 2.5.** (i) The characterizations (c) and (d) of the value function $V(\cdot, \cdot)$ are expressed in terms of proximal normals to its epigraph, and strict normals to its epigraph and hypograph. Invoking the well-known relations between subdifferentials (superdifferentials) of a given function and the normal vectors to its epigraph (and hypograph) these properties can be alternatively rewritten considering proximal subdifferentials, and (Fréchet) subdifferentials and superdifferentials of $V(\cdot, \cdot)$ at points $(t, x)$ belonging to the domain of $V(\cdot, \cdot)$. In [4], for instance, where the velocity set $F$ has the same discontinuous behaviour, characterizations of the values function are provided by conditions involving both horizontal and non-horizontal proximal subdifferentials. Here, we prefer to use the formulation with normal vectors because it summarizes in a concise way the characterization of interest, which in our case has to take into consideration also the Lagrangian term $L$. Moreover, the normal vectors expression highlights the somewhat ‘abnormal’ feature of the horizontal normal vectors $(\xi^0, \xi^1, \lambda = 0)$, which corresponds to the case in which the Lagrangian disappears in conditions (c) and (d). The contribution of horizontal normals can be easily removed when ‘$F$ is continuous’ (cf. [12, 20]) owing to the well-known (Rockafellar) horizontal approximation theorem (cf. [9]), and it is not clear whether this simplification procedure would be in general applicable in the discontinuous context (cf. the issue raised in Remark 2.2-(d) of [4]).

(ii) Conditions in (b), (c) and (d) are formulated taking into account particular left and right limits w.r.t. $t$ of $F$ and $L$. For the Mayer problem, in [4] it is shown that the role of the left/right limits is crucial to characterize the value function, and assertions (b) and (c) become in general false if we try to exchange the role of those limits. As one may expect, our results for Bolza problems are consistent with [4]. We underline the fact that also for the viscosity solutions characterization (d) the role of the right limit is crucial as illustrated by Example 2.6. Finally we observe that, in (b) i), (c) i) and (d) i) of Theorems 2.1 and 2.2 we can avoid consideration of the limits of $L$ w.r.t. $t$, imposing also the lower semicontinuity of $L$ in $t$ (see Prop. 2.4).

(iii) The characterization (d) provided by Theorem 2.2 concerns lower semicontinuous value functions for optimal control problems having a terminal cost $g$ which is locally bounded and satisfies the condition $(g^*)_* = g$. A natural question would be:

Is that possible to characterize $V(\cdot, \cdot)$ in the sense of Theorem 2.2 for optimal control problems removing the conditions 'g is locally bounded or $(g^*)_* = g'?'

If $g$ is a lower semicontinuous extended valued function (taking the value $+\infty$ at some points), the issue of interpreting the viscosity subsolution replacing the condition (d)-ii) immediately arises and it is not clear how we have to interpret $U^*$. Taking the lim sup operator we would lose crucial information on the boundary of $\text{dom}(V)$ and the viability approach would not be applicable or give the desired information. On the other hand, if we consider the smaller (extended valued) upper semicontinuous function bigger than $V$ on the domain of $V$, under some circumstances (such as $V$ is continuous on its domain and $\text{dom}(V)$ is a closed set) we would be induced to end up with the function $V^-$ which coincides with $V$ on $\text{dom}(V)$ and takes the value $-\infty$ on $[S, T] \times \mathbb{R}^n \setminus \text{dom}(V)$. The latter technique would not help either, as clarified by Example 2.7. Condition $(g^*)_* = g$ can be removed if we are interested in proving that the value function is a viscosity solution in the sense of (d) of Theorem 2.2 (as established by Prop. 2.3). However, condition $(g^*)_* = g$ becomes far from being just a technical hypothesis and emerges as crucial if we want a characterization (comparison result) for the value function. This point is illustrated in Example 2.8.

(iv) The results above are still valid if we start from a slightly more general context in which the Lagrangian in now extended valued $L: [S, T] \times \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ and assumption (H5)-ii) is replaced by a ‘local
boundedness a.e. in $t'$ in the following sense: there exists a set of full measure $E \subset [S, T]$ such that

$$|L(t, x, v)| \le M_0, \quad \text{for all } (t, x, v) \in E \times R_0\mathbb{B} \times 2c_0\mathbb{B}.$$

Indeed, using (H6) we can reduce attention to the case in which $L$ is locally bounded in the sense of (H5), and then the analysis remains the same.

(v) Assertions (c) and (d) of Theorems 2.1 and 2.2 can be easily reformulated in terms of an Hamiltonian function

$$H_{\lambda}(t, x, p) := \inf_{v \in F(t,x)} [p \cdot v + \lambda L(t, x, v)].$$

Observe that under our assumptions $H_{\lambda}(\cdot, x, p)$ turns out to be continuous on the complement of a zero-measure subset of $[S, T]$ and has everywhere one-sided limits $H_{\lambda}(t^{+}, x, p)$ and $H_{\lambda}(t^{-}, x, p)$.

## 2.4. Examples

**Example 2.6.** Consider the optimal control problem

$$ (P_{t_0, x_0}) \quad \left\{ \begin{array}{ll} \text{Minimize } g(x(1)) + \int_0^1 L(t, x(t), \dot{x}(t))dt & \text{over arcs } x(\cdot) \in W^{1,1}([t_0, 1], \mathbb{R}) \text{ such that} \\ \dot{x}(t) \in F(t) & \text{for a.e. } t \in [t_0, 1], \\ x(t_0) = x_0, & \end{array} \right. $$

where $t_0 \in [0, 1]$, $x_0 \in \mathbb{R}$,

$$ F(t) := \begin{cases} [-1, 1], & \text{if } 0 \le t \le \frac{1}{2}, \\ [-\frac{1}{2}, \frac{1}{2}], & \text{if } \frac{1}{2} < t \le 1, \end{cases} $$

$$ g(x) := \begin{cases} 1 + x, & \text{if } x > 0, \\ x, & \text{if } x \le 0, \end{cases} $$

and

$$ L(t, x, v) := \begin{cases} (v+1)^2, & \text{if } 0 < t \le \frac{1}{2}, \\ (v+\frac{1}{2})^2 + 2, & \text{if } \frac{1}{2} < t \le 1. \end{cases} $$

The value function $V: [0, 1] \times \mathbb{R} \to \mathbb{R}$ is

$$ V(t, x) = \begin{cases} x + t + \frac{5}{4}, & \text{if } 0 \le t \le \frac{1}{2} \quad \text{and} \quad x + t - \frac{3}{4} > 0, \\ x + t + \frac{1}{4}, & \text{if } 0 \le t \le \frac{1}{2} \quad \text{and} \quad x + t - \frac{3}{4} \le 0, \\ x - \frac{3t}{2} + \frac{5}{2}, & \text{if } \frac{1}{2} < t \le 1 \quad \text{and} \quad x + \frac{t}{2} - \frac{1}{2} > 0, \\ x - \frac{3t}{2} + \frac{3}{2}, & \text{if } \frac{1}{2} < t \le 1 \quad \text{and} \quad x + \frac{t}{2} - \frac{1}{2} \le 0. \end{cases} $$

As a result of a routine analysis, one can see that conditions (b)–(c) of Theorem 2.1 and condition (d) of Theorem 2.2 are satisfied by $V$. Here, we only display some calculations at the point $(t_0, x_0) = (\frac{1}{2}, \frac{1}{4})$, which is
of particular interest since it carries information about the discontinuous behaviour of the data $F$, $L$ and $g$ at the same time. Consider, for instance, (d) ii) of Theorem 2.2. Take any $(u, v) \in \mathbb{R}^2$. We have:

$$D_{\downarrow} V^* \left( \left( \frac{1}{2}, \frac{1}{4} \right), (u, v) \right) = \begin{cases} u + v, & \text{if } u \le 0 \text{ and } u + v \ge 0, \\ -\infty, & \text{if } u \le 0 \text{ and } u + v < 0, \\ -\frac{3u}{2} + v, & \text{if } u > 0 \text{ and } \frac{u}{2} + v \ge 0, \\ -\infty, & \text{if } u > 0 \text{ and } \frac{u}{2} + v < 0, \end{cases}$$

and hence:

$$T_{\text{hyp}} V^* \left( \left( \frac{1}{2}, \frac{1}{4} \right), V^* \left( \frac{1}{2}, \frac{1}{4} \right) = 2 \right) = \left\{ (u, v, \ell) \in \mathbb{R}^3 \mid \begin{cases} u \le 0 \\ u + v \ge 0 \\ \ell \le u + v \end{cases} \text{ or } \begin{cases} u > 0 \\ \frac{u}{2} + v \ge 0 \\ \ell \le -\frac{3u}{2} + v \end{cases} \right\}.$$

By polarity, we deduce that:

$$\hat{N}_{\text{hyp}} V^* \left( \left( \frac{1}{2}, \frac{1}{4} \right), 2 \right) = \left\{ (-\xi^0, -\xi^1, \lambda) \mid \lambda \in \mathbb{R}_+, \xi^1 \ge \lambda, \xi^1 \ge \xi^0, -\xi^0 + \frac{\xi^1}{2} - 2\lambda \le 0 \right\}.$$

Consistently with condition (d) ii) in Theorem 2.2, the value function satisfies:

$$-\xi^0 + \max_{v \in [-\frac{1}{2}, \frac{1}{2}]} \left\{ -\xi^1 v - \lambda \left[ \left( v + \frac{1}{2} \right)^2 + 2 \right] \right\} = -\xi^0 + \frac{\xi^1}{2} - 2\lambda \le 0,$$

for every $(-\xi^0, -\xi^1, \lambda) \in \hat{N}_{\text{hyp}} V^* \left( \left( \frac{1}{2}, \frac{1}{4} \right), 2 \right)$.

On the other hand, switching the roles of $F(\frac{1}{2}^+)$ and $F(\frac{1}{2}^-)$ in the analysis above, we would not obtain the validity of condition (d) ii) since, taking the vector $(\frac{3}{2}, -1, 1) \in \hat{N}_{\text{hyp}} V^* \left( \left( \frac{1}{2}, \frac{1}{4} \right), 2 \right)$, we obtain:

$$\frac{3}{2} + \max_{v \in [-1,1]} \left\{ -v - \left[ \left( v + \frac{1}{2} \right)^2 + 2 \right] \right\} = \frac{3}{2} - \frac{5}{4} = \frac{1}{4} > 0.$$

Similarly, switching the roles of $L(\frac{1}{2}, x_0, v)$ and $L((\frac{1}{2})^+, x_0, v)$ for the same normal vector, we would not obtain condition (d) ii) either:

$$\frac{3}{2} + \max_{v \in [-\frac{1}{2}, \frac{1}{2}]} \left\{ -v - (v+1)^2 \right\} = \frac{3}{2} + \frac{1}{4} = \frac{7}{4} > 0.$$

Even if we switched limits for both $L$ and $F$, condition (d) ii) would not be satisfied since we have:

$$\frac{3}{2} + \max_{v \in [-1,1]} \left\{ -v - (v+1)^2 \right\} = \frac{3}{2} + 1 = \frac{5}{2} > 0.$$

This example shows that condition (d) ii) must involve the right limits $F(t^+, x)$ and $L(t^+, x, v)$, for, if the limits were taken from the other side, the assertion would be false in general. Similar considerations show the fundamental significance of the right limits also in condition (d) i) of Theorem 2.2.

Theorem 2.2 provides a characterization for the class of lower semicontinuous functions which are also locally bounded. One might wonder whether this result can be generalized to the class of lower semicontinuous extended
valued functions, like for the characterization provided by Theorem 2.1. The major difficulty comes from interpreting the concept of viscosity subsolution (which would correspond to condition (d) ii)) on the boundary of the domain of the candidate to be value function. The notion of viscosity subsolution used in our paper involves consideration of the upper semicontinuous envelope $V^*$, which has a clear meaning if $V$ is locally bounded. On the other hand, if $V$ were extended valued (with a closed nonempty domain), one might be tempted to take into account the upper semicontinuous extended valued function $V^-$:

$$V^-(t, x) := \begin{cases} V(t, x), & \text{if } (t, x) \in \text{dom}(V), \\ -\infty, & \text{elsewhere.} \end{cases}$$

The following simple example shows that this would not provide the desired effect, even if $F$ is continuous and the value function $V$ is continuous on $\text{dom}(V)$.

**Example 2.7.** Consider the optimal control problem:

$$\begin{cases} \text{Minimize } g(x(1)) \\ \text{over arcs } x(\cdot) \in W^{1,1}([t_0, 1], \mathbb{R}) \text{ such that} \\ \dot{x}(t) \in F(t) & \text{for a.e. } t \in [t_0, 1], \\ x(t_0) = x_0, & \end{cases}$$

where $t_0 \in [0, 1], x_0 \in \mathbb{R}$,

$$g(x) := \begin{cases} +\infty, & \text{if } x > 0, \\ x, & \text{if } x \le 0, \end{cases}$$

and for all $(t, x, v) \in [0, 1] \times \mathbb{R} \times \mathbb{R}$,

$$F(t) := [-1, 1].$$

The value function $V: [0, 1] \times \mathbb{R} \to \mathbb{R} \cup \{+\infty\}$ is:

$$V(t, x) = \begin{cases} +\infty, & \text{if } x + t - 1 > 0, \\ x + t - 1, & \text{if } x + t - 1 \le 0. \end{cases}$$

Then

$$V^-(t, x) = \begin{cases} -\infty, & \text{if } x + t - 1 > 0, \\ x + t - 1, & \text{if } x + t - 1 \le 0. \end{cases}$$

Let us consider $(t_0, x_0) \in ]0, 1[ \times \mathbb{R}$ such that $x_0 + t_0 + 1 = 0$. We have:

$$\hat{N}_{\text{hyp } V^{-}}((t_0, x_0), V(t_0, x_0) = 0) = \{(-\xi^0, -\xi^0, \lambda) \mid \lambda \in \mathbb{R}_+, \xi^0 \le \lambda\}.$$

However if we use $(1, 1, 1) \in \hat{N}_{\text{hyp } V^{-}}((t_0, x_0), V(t_0, x_0))$, then condition (d) ii) is violated since:

$$1 + \max_{v \in [-1, 1]} \{v\} = 1 + 1 > 0.$$

Observe that the issue here is not due to the fact that horizontal vectors might be involved in the characterization, indeed the vector $(1, 1, 1) \in \hat{N}_{\text{hyp } V^{-}}((t_0, x_0), V(t_0, x_0))$, considered above, is definitely non-horizontal and corresponds to the superdifferential $p = (-1, -1) \in \partial_{+}V(t_0, x_0)$.
**Example 2.8.** Consider the Mayer problem:

$$
\left\{
\begin{array}{l}
\text{Minimize } g(x(1)) \\
\text{over arcs } x(\cdot) \in W^{1,1}([t_0, 1], \mathbb{R}) \text{ such that} \\
\dot{x}(t) \in F(t) \quad \text{for a.e. } t \in [t_0, 1], \\
x(t_0) = x_0,
\end{array}
\right.
$$

where $t_0 \in [0, 1]$, $x_0 \in \mathbb{R}$,

$$
g(x) := \begin{cases} 1, & \text{if } x \neq 0, \\ 0, & \text{if } x = 0, \end{cases}
$$

and for all $(t, x) \in [0, 1] \times \mathbb{R}$,

$$
F(t) := [0, 1].
$$

The value function $V: [0, 1] \times \mathbb{R} \to \mathbb{R}$ is:

$$
V(t, x) = \begin{cases} 0, & \text{if } x + 1 - t \ge 0 \text{ and } x \le 0, \\ 1, & \text{if } x + 1 - t < 0 \text{ or } x > 0. \end{cases}
$$

One can easily check that $V$ is a viscosity solution, *i.e.* satisfies (d) i)–iii). Consider the function $U : [0,1] \times \mathbb{R} \to \mathbb{R}$:

$$
U(t, x) := \begin{cases} 0, & \text{if } x = 0, \\ \frac{1}{2}, & \text{if } x \neq 0, x + 1 - t \ge 0 \text{ and } x \le 0, \\ 1, & \text{if } x + 1 - t < 0 \text{ or } x > 0. \end{cases}
$$

Then $U$ is also a viscosity solution in the sense of condition (d). This shows that, if we do not have the property $(g^*)_* = g$, we do not obtain the uniqueness of the viscosity solution in the sense of (d).

3. PRELIMINARY RESULTS

We observe that, under our reference assumptions (H1)–(H6) (or under their more restrictive form provided
by the *a priori* boundedness argument), for every $(t, x) \in [S, T] \times \mathbb{R}^n$, the problem

$$
\inf_{x(\cdot) \text{ F-trajectory on } [t,T], x(t)=x} J(x(\cdot)) := \int_t^T L(s, x(s), \dot{x}(s))ds + g(x(T))
$$

has a minimizer. This is due to the fact that, with respect to the $W^{1,1}$ topology, the set of $F$-trajectories
$\{x(\cdot) \text{ F-trajectory on } [t,T], x(t)=x\}$ is compact (cf. [7], Thm. 6.39 or [20], Thm. 2.5.3) and the functional $J(\cdot)$
is lower semicontinuous.

Taking into account (H5)*, we can state a local Lipschitz regularity lemma for the function $L(s, y, \cdot)$ (locally uniformly with respect to $(s, y)$), the proof of which is based on standard arguments on convex functions, and therefore it is omitted. We just observe that the role of the number $2c_0$ (instead of the simpler $c_0$) allows to deduce the Lipschitz regularity of $L$ in $v$ in a ball with the smaller radius $c_0$.
**Lemma 3.1.** Assume (H5)*. Then, there exists a positive constant $k_L$ such that for every $(s, y) \in [S, T] \times \mathbb{R}^n$, and $v, v' \in c_0\mathbb{B}$:

$$|L(s, y, v) - L(s, y, v')| \le k_L|v - v'|. \quad (3.1)$$

In the following lemma we establish a further (uniform) regularity property of the Lagrangian, which we will invoke several times in our analysis.

**Lemma 3.2.** (i) Assume that $L$ satisfies (H5)* i), (H6)* i) and (H6)* iii) . Let $t \in [S, T[$ and $x \in \mathbb{R}^n$. Then, for every $\varepsilon > 0$, there exists $\delta > 0$ such that, for every $y \in x + \delta\mathbb{B}$, for every real $s \in ]t, t + \delta] \cap [S, T]$, and every $u \in c_0\mathbb{B}$:

$$L(s, y, u) \ge L(t^+, x, u) - \varepsilon. \quad (3.2)$$

(ii) Assume that $L$ satisfies (H5)* i) and (H6)* i)-ii). Let $t \in ]S, T[$ and $x \in \mathbb{R}^n$. Then, for every $\varepsilon > 0$, there exists $\delta > 0$ such that, for every $y \in x + \delta\mathbb{B}$, for every $s \in [t - \delta, t \cap [S, T]$, and every $u \in c_0\mathbb{B}$:

$$L(s, y, u) \ge L(t^-, x, u) - \varepsilon. \quad (3.3)$$

(iii) Assume that $L$ satisfies (H5)* and (H6)* i), and that $L(\cdot, x, v)$ is lower semicontinuous for all $(x, v) \in \mathbb{R}_0\mathbb{B} \times c_0\mathbb{B}$. Let $t \in [S, T[$ and $x \in \mathbb{R}^n$. Then, for every $\varepsilon > 0$, there exists $\delta > 0$ such that, for every $y \in x + \delta\mathbb{B}$, for every $s \in [t - \delta, t + \delta] \cap [S, T]$, and for every $u \in c_0\mathbb{B}$:

$$L(s, y, u) \ge L(t, x, u) - \varepsilon. \quad (3.4)$$

*Proof.* We start proving i). Fix any $\varepsilon > 0$. Take any $v \in c_0\mathbb{B}$. Invoking (H6)* iii), there exists $0 < \delta_1(v, \varepsilon) < 1$ such that, for all $s \in ]t, t + \delta_1(v, \varepsilon)] \cap [S, T]$, we have

$$L(s, x, v) \ge L(t^+, x, v) - \frac{\varepsilon}{4}. \quad (3.5)$$

Invoking Lemma 3.1 we also know that, for all $\tau \in [S, T]$ and $v, u \in c_0\mathbb{B}$,

$$|L(\tau, x, v) - L(\tau, x, u)| \le k_L|u - v|. \quad (3.6)$$

Set $\delta_2(v, \varepsilon) := \min\{\delta_1(v, \varepsilon); \frac{\varepsilon}{4k_L}\}(> 0)$. Then, combining inequalities (3.5) and (3.6) (this is used twice, i.e. for $\tau = t^+$ and $\tau = s$) yields: for every $s \in ]t, t + \delta_2(v, \varepsilon)] \cap [S, T]$, and every $u \in (v + \delta_2(v, \varepsilon)\mathbb{B}) \cap c_0\mathbb{B}$ we have

$$L(s, x, u) \ge L(t^+, x, u) - \frac{3}{4}\varepsilon. \quad (3.7)$$

Using the compactness of $c_0\mathbb{B}$, from the open cover of the set $c_0\mathbb{B} \subset \bigcup_{v \in c_0\mathbb{B}} v + \delta_2(v, \varepsilon)\mathbb{B}$ ($\mathbb{B}$ is the open unit ball) we can extract a finite subcover:

$$c_0\mathbb{B} \subset \bigcup_{j=1}^{N} v_j + \delta_2(v_j, \varepsilon)\mathbb{B}.$$

Define $\delta_3 := \min_{j=1,...,N} \delta_2(v_j, \varepsilon)$. We obtain:

$$L(s, x, u) \ge L(t^+, x, u) - \frac{3}{4}\varepsilon, \quad (3.8)$$
for all $s \in ]t, t + \delta_3] \cap [S, T]$, and every $u \in c_0\mathbb{B}$. From (H6)* i), we know that there exists $0 < \delta \le \delta_3$ such that $\omega(\delta) \le \frac{1}{4}\varepsilon$, and so

$$|L(s, y, u) - L(s, x, u)| \le \frac{1}{4}\varepsilon, \text{ for all } y \in x + \delta\mathbb{B}.$$

As a consequence, from this inequality and from (3.8), we deduce the validity of (3.2). The proofs of ii) and iii) follow along the same lines. Indeed, in the first step of the proof, we can use respectively (H6)* ii) and the lower semicontinuity of $L(\cdot, x, v)$ instead of (H6)* iii) to obtain (3.5) on the suitable time interval. $\square$

We now introduce the auxiliary Lagrangian $L^-$ which will be used as a technical tool in the characterization of solutions to the Hamilton-Jacobi equation. Take any $(t, x) \in ]S, T] \times \mathbb{R}^n$. We consider the following modulus of continuity of $F$ with respect to time (from the left) $\theta_t^- : [0, t - S] \to \mathbb{R}_+$, defined by: for every $h \in [0, t - S]$,

$$\theta_t^-(h) := \begin{cases} \sup_{0<t-s\le h \atop |x-y|\le c_0h} d_H(F(s,y), F(t^-, x)), & \text{if } h \ne 0, \\ 0, & \text{otherwise.} \end{cases} \quad (3.9)$$

Write $K := \exp \left( \int_S^T k_F(s)ds \right)$. If we take also a vector $v \in F(t^-, x)$, we define the following set:

$$Z(t, x, v) := \{z(\cdot) F\text{-trajectory on } [S, t] \mid z(t) = x\} \quad (3.10)$$

$$\text{and } \|x + (\cdot - t)v - z(\cdot)\|_{L^\infty([t-h, t], \mathbb{R}^n)} \le K\theta_t^-(h)h, \text{ for all } h \in [0, t - S].$$

The Lagrangian $L^- : ]S, T] \times \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ is defined as follows: for every $(t, x, v) \in ]S, T] \times \mathbb{R}^n \times \mathbb{R}^n$,

$$L^-(t, x, v) := \begin{cases} \liminf_{h \downarrow 0} h^{-1} \inf \left\{ \int_{t-h}^t L(s, z(s), \dot{z}(s))ds \mid z \in Z(t, x, v) \right\}, & \text{if } v \in F(t^-, x), \\ L(t^-, x, v), & \text{otherwise.} \end{cases} \quad (3.11)$$

The map $L^-$ arises in a somehow natural way in some crucial steps of our analysis (cf. the proofs of Prop. 3.3 and Thm. 2.1). A similar auxiliary Lagrangian function was introduced in [10, 11] to investigate characterization of solutions to Hamilton-Jacobi equations in the context of calculus of variations. In our framework the expression of $L^-$ is more involved since we have to take account of the velocity constraint given by the differential inclusion $\dot{z}(s) \in F(s, z(s))$ and the possible different (from the left and from the right) limit behaviour of $F$ w.r.t. $t$.

**Proposition 3.3.** Suppose that (H1), (H2)*, (H3)*, (H4) and (H5)* are satisfied.

(i) Then, for all $(t, x, v) \in ]S, T] \times \mathbb{R}^n \times \mathbb{R}^n$, we have $L^-(t, x, v) \in \mathbb{R}$ and

$$L(t^-, x, v) \le L^-(t, x, v). \quad (3.12)$$

(ii) If, in addition, $L$ satisfies (H6)* i)-ii), then for every $(t, x, v) \in ]S, T] \times \mathbb{R}^n \times \mathbb{R}^n$ we also obtain

$$L^-(t, x, v) \le L(t^-, x, v). \quad (3.13)$$

*Proof.* (i) Consider $(t, x) \in ]S, T] \times \mathbb{R}^n$. We can assume that $v \in F(t^-, x)$, since otherwise the stated inequality immediately follows from the definition of $L^-$. Using Filippov existence theorem (cf. [20], Thm. 2.4.3), we have $Z(t, x, v) \ne \emptyset$. As a consequence, we obtain $\inf \left\{ \int_{t-h}^t L(s, z(s), \dot{z}(s))ds \mid z \in Z(t, x, v) \right\} \ne +\infty$, for all $h \in ]0, t - S]$.
Invoking the a priori uniform boundedness of the F-trajectories, it is straightforward to see that all the arcs in Z(t, x, v) are uniformly bounded and uniformly Lipschitz continuous. Since L is bounded in the sense of condition (H5)*, we deduce that there exists a constant M₀ > 0 such that, for every z(·) ∈ Z(t, x, v),

$$|L(s, z(s), \dot{z}(s))| \leq M_0, \text{ for almost every } s \in [S, t].$$

It follows that, for all $h \in ]0, t-S]$:
$$\left| \inf \left\{ \int_{t-h}^{t} L(s, z(s), \dot{z}(s))ds \mid z \in Z(t, x, v) \right\} \right| \leq hM_0, \text{ which implies that}$$
$$|L^{-}(t, x, v)| \leq M_0, \text{ and therefore } L^{-}(t, x, v) \in \mathbb{R}.$$

We now establish (3.12). Let $\varepsilon > 0$. For every $h \in ]0, t-S]$, small enough, we choose $z_h \in W^{1,1}([S, t], \mathbb{R}^n)$ an $\varepsilon h$-minimizer for the following Lagrange problem:

$$\inf \left\{ \int_{t-h}^{t} L(s, z(s), \dot{z}(s))ds \mid z \in Z(t, x, v) \right\}.$$

Invoking again the a priori boundedness of F-trajectories, the family $(\dot{z}_h)_{h \in ]0, t-S]}$ is bounded in $\mathbb{L}^{\infty}$ by $c_0$. Using (3.3) of Lemma 3.2 and Jensen's inequality, we obtain for all $h \in ]0, t-S]$:

$$\frac{1}{h} \int_{t-h}^{t} L(s, z_h(s), \dot{z}_h(s))ds \geq \frac{1}{h} \int_{t-h}^{t} L(t^{-}, x, \dot{z}_h(s))ds - \varepsilon \geq L \left( t^{-}, x, \frac{1}{h} \int_{t-h}^{t} \dot{z}_h(s)ds \right) - \varepsilon.$$

From standard analysis, we also know that $\lim_{h \downarrow 0} \frac{1}{h} \int_{t-h}^{t} \dot{z}_h(s)ds = v$. Passing to the limit inferior in the last equation, we have:

$$\varepsilon + L^{-}(t, x, v) \geq L(t^{-}, x, v) - \varepsilon,$$

which confirms (3.12) since $\varepsilon$ is arbitrary.

(ii) Consider $(t, x) \in ]S, T] \times \mathbb{R}^n$. Again, we can restrict attention to the case $v \in F(t^{-}, x)$, since otherwise the assertion easily follows from the definition of $L^{-}$, and claim that

$$L^{-}(t, x, v) \leq \lim_{h \downarrow 0} \sup_{\substack{0 < t - v \leq h \\ |x - y| \leq c_0 h}} L(s, y, v). \quad (3.14)$$

Indeed, using Filippov existence theorem, we can find an F-trajectory $z \in W^{1,1}([S, t], \mathbb{R}^n)$, such that $z(t) = x$ and for every $h \in [0, t-S]$:

$$\|z - (x + (\cdot - t)v)\|_{\mathbb{L}^{\infty}([t-h, t], \mathbb{R}^n)} \leq \int_{t-h}^{t} |\dot{z}(s) - v|ds \leq K\theta_{t}^{-}(h)h, \quad (3.15)$$

where $K = \exp(\int_{S}^{T} k_{F}(s)ds)$. From Lemma 3.1, there exists $k_L > 0$ such that for every $(t', x') \in [S, T] \times \mathbb{R}^n$, $v, v' \in c_0\mathbb{B}$:

$$|L(t', x', v) - L(t', x', v')| \leq k_L |v - v'|.$$
As a consequence, for every $h \in ]0, t-S]$, we have:

$$ 
\begin{aligned}
\inf \left\{ \int_{t-h}^{t} L(s, z(s), \dot{z}(s)) ds \mid z \in Z(t, h, v) \right\} &\le \int_{t-h}^{t} L(s, z(s), \dot{z}(s)) ds, \\
&\le \int_{t-h}^{t} L(s, z(s), v) ds + \int_{t-h}^{t} k_L |\dot{z}(s) - v| ds \\
&\le h \sup_{\substack{0 < t-s \le h \\ |x-y| \le c_0 h}} L(s, y, v) + h k_L K \theta_t^-(h).
\end{aligned}
$$

Dividing across by $h$, passing to the limit inferior as $h$ goes to 0, yields (3.14). If $L$ satisfies satisfies also (H6)* i)-ii), then:

$$ L^{-}(t, x, v) \le \lim_{h \downarrow 0} \sup_{\substack{0 < t-s \le h \\ |x-y| \le c_0 h}} L(s, y, v) = L(t^{-}, x, v), $$

which confirms (3.13).

<figure><img src="image_3.png" /></figure>

We conclude this section recalling a well-known result, referred to as the Weak Invariance/Global Viability Theorem (cf. [1] or [20]).

**Theorem 3.4** (Weak invariance theorem). Take a multifunction $\Gamma : \mathbb{R}^k \rightsquigarrow \mathbb{R}^k$, an interval $[S, T]$ and a closed set $D \subset \mathbb{R}^k$. Assume:

(i) The graph of $\Gamma$ is closed and $\Gamma(x)$ is a nonempty, convex set for each $x \in \mathbb{R}^k$;

(ii) there exists $c > 0$ such that

$$ \Gamma(x) \subset c(1 + |x|)\mathbb{B} \quad \text{for all } x \in \mathbb{R}^k; $$

(iii) for every $x \in D$ we have

$$ \min_{v \in \Gamma(x)} \zeta \cdot v \le 0 \quad \text{for all } \zeta \in N_D^P(x). $$

Then, given any $x_0 \in D$, there exists an absolutely continuous function $x(\cdot)$ satisfying

$$ 
\left\{ 
\begin{array}{ll}
\dot{x}(t) & \in \Gamma(x(t)) \quad \text{for a.e. } t \in [S, T], \\
x(S) & = x_0, \\
x(t) & \in D \quad \text{for all } t \in [S, T].
\end{array}
\right. $$

## 4. PROOF OF THEOREM 2.1

The proof has the following structure: we first show that the value function satisfies property (b) of Theorem 2.1. We subsequently prove that condition (b) implies condition (c). Finally, if a lower semicontinuous function $U$ satisfies (c) then we show that it coincides with the value function. Each step is highlighted by a proposition or a theorem statement.

### 4.1. The value function satisfies (b) of Theorem 2.1

**Proposition 4.1.** Assume (H1)-(H6). Let $V : [S, T] \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ be the value function of the problem. Then $V$ satisfies (b) i)-iii) of Theorem 2.1.
Proof. From the definition of V it immediately follows that $V(T, \cdot) = g(\cdot)$ confirming (b) iii). The lower semi-continuity of V can be deduced by standard arguments (see for instance [17], Thm. 1.1). We have to prove that V satisfies (b) i) and (b) ii) of Theorem 2.1.

**Step 1.** The first part of this step is somewhat standard (cf. [4, 20]). We briefly reproduce this analysis since, in the second part of this step, it has to be properly combined with suitable properties on the Lagrangian L, mainly described by Lemma 3.2.

Take $(t, x) \in ([S, T \times \mathbb{R}^n) \cap \text{dom}(V)$. Let $y \in W^{1,1}([t, T], \mathbb{R}^n)$ be a minimizing F-trajectory for $(P_{t,x})$, whose existence is guaranteed by our assumptions on F and L (see Sect. 3). Using the principle of optimality, for every $\delta \in ]0, T - t]$, we have:

$$V(t + \delta, y(t + \delta)) - V(t, x) = \int_{t+\delta}^{t} L(s, y(s), \dot{y}(s))ds.$$

From the fundamental theorem of calculus we also have for every $\delta \in ]0, T - t]$:

$$\delta^{-1}(y(t + \delta) - y(t)) = \delta^{-1} \int_{t}^{t+\delta} \dot{y}(s)ds.$$

Let $(\delta_i)_{i \in \mathbb{N}}$ be a strictly decreasing sequence of positive real numbers that converges to 0. For every integer $i \in \mathbb{N}$, let us define $v_i \in \mathbb{R}^n$ by:

$$v_i := \delta_i^{-1} \int_t^{t+\delta_i} \dot{y}(s)ds.$$

From the a priori boundedness of the F-trajectories guaranteed by the hypotheses reduction of Section 2, $|\dot{y}(s)| \le c_0$ for almost every $s \in [t, T]$. From this inequality, we deduce that the sequence $(v_i)_{i \in \mathbb{N}}$ is bounded by $c_0$. Then, there exists a vector $\bar{v} \in c_0\mathbb{B}$ such that, up to a subsequence, $v_i \xrightarrow{i \to +\infty} \bar{v}$.

Take any $p \in \mathbb{R}^n$ and $i \in \mathbb{N}$. Since $F(s^+, y(s)) = F(s, y(s))$ almost everywhere for each $i \in \mathbb{N}$, we have:

$$p \cdot v_i = \delta_i^{-1} \int_t^{t+\delta_i} p \cdot \dot{y}(s)ds \le \delta_i^{-1} \int_t^{t+\delta_i} \max_{v \in F(s^+, y(s))} p \cdot v ds. \quad (4.1)$$

But since the function $s \mapsto \max_{v \in F(s^+, y(s))} p \cdot v$ is right continuous at $s = t$, letting $i$ go to $+\infty$ in equation (4.1), we have:

$$p \cdot \bar{v} \le \max_{v \in F(t^+, y(t))} p \cdot v.$$

Employing the characterization of the closed convex hull of a set by the support function ([19], Thm. 13.1), we deduce that $\bar{v} \in F(t^+, y(t)) = F(t^+, x)$.

Using the definition of $D_\uparrow V((t, x), (1, \bar{v}))$ and the principle of optimality, we obtain:

$$\begin{align}
D_\uparrow V((t, x), (1, \bar{v})) + L(t^+, x, \bar{v}) &\le \liminf_{i \to \infty} \delta_i^{-1} (V(t + \delta_i, y(t + \delta_i)) - V(t, x)) + L(t^+, x, \bar{v}), \nonumber \\
&= \liminf_{i \to \infty} \delta_i^{-1} \int_{t+\delta_i}^{t} L(s, y(s), \dot{y}(s))ds + L(t^+, x, \bar{v}), \nonumber \\
&= \liminf_{i \to \infty} -\delta_i^{-1} \int_{t}^{t+\delta_i} L(s, y(s), \dot{y}(s))ds + L(t^+, x, \bar{v}). \tag{4.2}
\end{align}$$
Fix any $\varepsilon > 0$. We consider the constant $\delta_\varepsilon > 0$ given by Lemma 3.2 i) for the reference pair $(t, x) \in [S, T[ \times \mathbb{R}^n$. Since $v_i \xrightarrow{i \to +\infty} \bar{v}$ and $L(t^+, x, \cdot)$ is continuous, there exists $N_0 \in \mathbb{N}$ such that:

$$|L(t^+, x, v_i) - L(t^+, x, \bar{v})| \le \varepsilon, \text{ for all } i \ge N_0,$$

and from the continuity of the F-trajectory $y(\cdot)$, we can choose an integer $N \ge N_0$ such that for every integer $i \ge N$: $\delta_i < \delta_\varepsilon$, and for all $s \in [t, t + \delta_i]$, $|y(s) - x| \le \delta_\varepsilon$. Since for almost every $s \in [t, T]$, $|\dot{y}(s)| \le c_0$, Lemma 3.2 guarantees that for almost every $s \in [t, t + \delta_\varepsilon] \cap [t, T]$,

$$L(s, y(s), \dot{y}(s)) \ge L(t^+, x, \dot{y}(s)) - \varepsilon.$$

Thus, for any integer $i \ge N$:

$$\delta_i^{-1} \int_t^{t+\delta_i} L(s, y(s), \dot{y}(s))ds \ge \delta_i^{-1} \int_t^{t+\delta_i} L(t^+, x, \dot{y}(s))ds - \varepsilon.$$

Applying Jensen's inequality to the convex function $L(t, x, \cdot)$, we also obtain:

$$\delta_i^{-1} \int_t^{t+\delta_i} L(t^+, x, \dot{y}(s))ds \ge L \left( t^+, x, \delta_i^{-1} \int_t^{t+\delta_i} \dot{y}(s)ds \right) - \varepsilon = L(t^+, x, v_i) - \varepsilon.$$

We deduce that $-\delta_i^{-1} \int_t^{t+\delta_i} L(s, y(s), \dot{y}(s))ds + L(t^+, x, \bar{v}) \le 2\varepsilon$ for every integer $i \ge N$, and so, from (4.2) we obtain:

$$D_\uparrow V((t, x), (1, \bar{v})) + L(t^+, x, \bar{v}) \le 2\varepsilon.$$

Since $\varepsilon$ is arbitrary, this confirms (b)-i).

**Step 2.** Let $(t, x) \in \text{dom}(V) \cap [S, T] \times \mathbb{R}^n$. Let $\tilde{v} \in F(t^-, x)$. For every $s \in [S, t]$, set $y(s) = x + (s - t)\tilde{v}$. Hypotheses on the multifunction $F$ allow us to use the Filippov existence theorem: there exists an F-trajectory $\tilde{z}(\cdot)$ that satisfies $\tilde{z}(t) = x$, such that for every $h \in ]0, t - S[$,

$$\|\tilde{z} - y\|_{L^\infty([t-h, t], \mathbb{R}^n)} \le K \left( \int_{t-h}^t d_{F(s, y(s))}(\tilde{v})ds \right) \le K \theta_t^-(h)h,$$

where $K = \exp(\int_S^T k_F(s)ds)$ and $\theta_t^-$ is the modulus of continuity defined in (3.9). Recalling the hypotheses reduction and definition of $\tilde{Z}(t, x, \tilde{v})$ given in (3.10) (see Sect. 2), it follows that $\tilde{z}(\cdot) \in Z(t, x, \tilde{v}) \ne \emptyset$. For any $h \in ]0, t - S[$, there exists an $h^2$ minimizer $z_h(\cdot) \in Z(t, x, \tilde{v})$ of the Lagrange problem:

$$\inf \left\{ \int_{t-h}^{t} L(s, z(s), \dot{z}(s))ds \mid z \in Z(t, x, \tilde{v}) \right\}$$

For any $h \in ]0, t - S[$, we write $v_h := h^{-1}(x - z_h(t - h))$. We obtain:

$$|\tilde{v} - v_h| = h^{-1}|z_h(t - h) - y(t - h)| \le K \theta_t^-(h).$$

We deduce $v_h \xrightarrow{h \downarrow 0} \tilde{v}$.
Using the principle of optimality applied to the F-trajectories $z_h(\cdot)$, we also have:

$$V(t-h, x-hv_h) - V(t, x) \leq \int_{t-h}^{t} L(s, z_h(s), \dot{z}_h(s))ds, \text{ for every } h \in [0, t-S].$$

It follows that for every $h \in [0, t-S[$,

$$h^{-1}(V(t-h, x-hv_h) - V(t, x)) \leq h^{-1} \inf \left\{ \int_{t-h}^{t} L(s, z(s), \dot{z}(s))ds \mid z \in Z(t, x, \tilde{v}) \right\} + h.$$

Hence, passing to the limit inferior when $h$ goes to 0 and recalling the definition of $L^-$ in (3.11),

$$D_{\uparrow}V((t, x), (-1, -\tilde{v})) \leq \liminf_{h \to 0} h^{-1} (V(t-h, x-hv_h) - V(t, x)) \leq L^{-}(t, x, \tilde{v}).$$

As a consequence, owing to ii) of Proposition 3.3, we obtain:

$$D_{\uparrow}V((t, x), (-1, -\tilde{v})) \leq L(t^{-}, x, \tilde{v}),$$

which establishes the validity of (b)-ii), concluding the proof of Proposition 4.1. $\square$

## 4.2. The value function is a proximal solution

In this subsection we prove that any lower semicontinuous function $U : [S, T] \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ satisfying condition (b) from Theorem 2.1 also verifies condition (c) from Theorem 2.1, i.e. is a proximal solution.

**Proposition 4.2.** Assume (H1)-(H6) and let $U : [S, T] \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ be a lower semicontinuous function satisfying (b) i)-iii) from Theorem 2.1. Then $U$ is a proximal solution to (HJE), i.e. satisfies (c) i)-iii).

We shall make use of two technical lemmas, which provide consequences of properties (b) i) and (b) ii) of Theorem 2.1.

**Lemma 4.3.** Let $U : [S, T] \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ be a lower semicontinuous function. Take any $(t, x) \in ([S, T \times \mathbb{R}^n) \cap \text{dom}(U)$. Then, there exists $v \in F(t^+, x)$, a sequence $(v_i)_{i \in \mathbb{N}}$ in $\mathbb{R}^n$ converging to $v$, and a strictly decreasing sequence $(h_i)_{i \in \mathbb{N}}$ in $\mathbb{R}_+$, converging to 0 as $i$ goes to $+\infty$, such that:

$$\lim_{i \to +\infty} h_i^{-1}(U(t+h_i, x+h_i v_i) - U(t, x)) = \left( \inf_{w \in F(t^+, x)} D_{\uparrow}U((t, x), (1, w)) + L(t^+, x, w) \right) - L(t^+, x, v).$$

Assume, in addition, that $U$ satisfies (b) i). Then we have:

$$\lim_{i \to +\infty} h_i^{-1}(U(t+h_i, x+h_i v_i) - U(t, x)) \leq -L(t^+, x, v). \quad (4.3)$$

*Proof.* Fix any $(t, x) \in ([S, T \times \mathbb{R}^n) \cap \text{dom}(U)$. Write $\Delta := \inf_{w \in F(t^+, x)} D_{\uparrow}U((t, x), (1, w)) + L(t^+, x, w)$. Let $(\varepsilon_j)_{j \in \mathbb{N}}$ in $\mathbb{R}_+$ be a strictly decreasing sequence that converges to 0. For any $j \in \mathbb{N}$, there exists a vector $v_j \in F(t^+, x)$ such that:

$$\Delta \leq D_{\uparrow}U((t, x), (1, v_j)) + L(t^+, x, v_j) \leq \Delta + \varepsilon_j.$$

Since $F(t^+, x)$ is compact, there exists $\tilde{v}$ in $F(t^+, x)$ for which, up to a subsequence, $(v_j)_{j \in \mathbb{N}}$ converges to $\tilde{v}$. By definition of the limit inferior, for each $j \in \mathbb{N}$, there exists a sequence $(v_{j,i})_{i \in \mathbb{N}}$ in $\mathbb{R}^n$ converging to $v_j$ and a
strictly decreasing sequence $(h_{j,i})_{i \in \mathbb{N}}$ in $\mathbb{R}_+$ converging to 0 such that:

$$ \lim_{i \to +\infty} h_{j,i}^{-1}(U(t+h_{j,i}, x+h_{j,i}v_{j,i}) - U(t,x)) = \liminf_{v' \to v_j, h' \downarrow 0} h'^{-1}(U(t+h', x+h'v') - U(t,x)). $$

It follows that we can construct a sequence $(\varphi(j))_{j \in \mathbb{N}}$ for which the subsequence $(h_{j,\varphi(j)})_{j \in \mathbb{N}}$ is strictly decreasing, converges to 0, and such that for every $j \in \mathbb{N}$:

$$ |v_j - v_{j,\varphi(j)}| \le \varepsilon_j, $$

$$ h_{j,\varphi(j)}^{-1}(U(t+h_{j,\varphi(j)}, x+h_{j,\varphi(j)}v_{j,\varphi(j)}) - U(t,x)) \in [\Delta - \varepsilon_j - L(t^+, x, v_j), \Delta + 2\varepsilon_j - L(t^+, x, v_j)]. \quad (4.4) $$

Write $\tilde{h}_j := h_{j,\varphi(j)}$ and $\tilde{v}_j := v_{j,\varphi(j)}$ for each $j \in \mathbb{N}^*$. As a consequence, we have $\lim_{j \to +\infty} \tilde{v}_j = \tilde{v}$ and $\lim_{j \to +\infty} \tilde{h}_j = 0$. Moreover, using the continuity of $L(t^+, x, \cdot)$, we obtain $\lim_{j \to +\infty} L(t^+, x, v_j) = L(t^+, x, \tilde{v})$. Therefore, letting $j$ go to $+\infty$ in (4.4) yields:

$$ \lim_{j \to +\infty} \tilde{h}_j^{-1}(U(t + \tilde{h}_j, x + \tilde{h}_j \tilde{v}_j) - U(t, x)) = \Delta - L(t^+, x, \tilde{v}). $$

If $U$ satisfies (b) i), we have $\Delta \le 0$, which implies

$$ \lim_{j \to +\infty} \tilde{h}_j^{-1}(U(t + \tilde{h}_j, x + \tilde{h}_j \tilde{v}_j) - U(t, x)) \le -L(t^+, x, \tilde{v}), $$

and concludes the proof of the lemma.

**Lemma 4.4.** Let $U: [S, T] \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ be a lower semicontinuous function. Assume that $U$ satisfies (b) ii). Let $(t, x) \in ([S, T] \times \mathbb{R}^n) \cap \text{dom}(U)$. Then for every $v \in F(t^-, x)$, there exists a sequence $(v_i)_{i \in \mathbb{N}}$ in $\mathbb{R}^n$ converging to $v$ and a decreasing sequence $(h_i)_{i \in \mathbb{N}}$ in $\mathbb{R}_+$ which converges to 0, such that:

$$ \lim_{i \to +\infty} h_i^{-1}(U(t - h_i, x - h_i v_i) - U(t, x)) \le L(t^-, x, v). \quad (4.5) $$

*Proof.* Consider any $(t, x) \in ([S, T] \times \mathbb{R}^n) \cap \text{dom}(U)$ and $v \in F(t^-, x)$. We have:

$$ D_\uparrow U((t, x), (-1, -v)) \le L(t^-, x, v). $$

Using the definition of $D_\uparrow U$, there exists a sequence $(v_i)_{i \in \mathbb{N}}$ in $\mathbb{R}^n$ converging to $v$ and a decreasing sequence $(h_i)_{i \in \mathbb{N}}$ in $\mathbb{R}_+$, converging to 0, such that:

$$ \liminf_{h \downarrow 0, v_h \to v} h^{-1}(U(t-h, t-hv_h) - U(t,x)) = \lim_{i \to +\infty} h_i^{-1}(U(t-h_i, t-h_iv_i) - U(t,x)) \le L(t^-, x, v), $$

which concludes the proof.

We are now ready to prove Proposition 4.2. The proof is split into three steps.

**Step 1.** We first claim that (c) i) from Theorem 2.1 holds: for every $(t, x) \in ([S, T] \times \mathbb{R}^n) \cap \text{dom}(U)$, for every proximal vector $(\xi^0, \xi^1, -\lambda) \in N_{\text{epi}}^P((t, x), U(t, x))$, we have

$$ \xi^0 + \min_{v \in F(t^+, x)} \xi^1 \cdot v + \lambda L(t^+, x, v) \le 0. $$
Take any $(t, x) \in (]S, T[ \times \mathbb{R}^n) \cap \text{dom}(U)$ and $(\xi^0, \xi^1, -\lambda) \in N_{\text{epi } U}^P((t, x), U(t, x))$. We necessarily have $\lambda \in \mathbb{R}_+$. Invoking the definition of the proximal normal cone, there exists $M \in \mathbb{R}_+$ such that for every $(t', x', \alpha') \in \text{epi } U$:

$$ \xi^0 \cdot (t' - t) + \xi^1 \cdot (x' - x) - \lambda(\alpha' - U(t, x)) \leq M(|t - t'|^2 + |x' - x|^2 + |\alpha' - U(t, x)|^2). \quad (4.6) $$

From Lemma 4.3, there exists $v \in F(t^+, x)$, a sequence $(v_i)_{i \in \mathbb{N}}$ in $\mathbb{R}^n$ that converges to $v$ and a strictly decreasing sequence $(h_i)_{i \in \mathbb{N}}$ in $\mathbb{R}_+$, converging to 0, such that:

$$ \lim_{i \to +\infty} h_i^{-1} (U(t+h_i, x+h_i v_i) - U(t,x)) \leq -L(t^+, x, v). $$

In particular we have:

$$ \lim_{i \to +\infty} h_i^{-1} |U(t+h_i, x+h_i v_i) - U(t,x)|^2 = 0, $$

since it is the product of the sequence $([h_i^{-1}(U(t+h_i, x+h_i v_i) - U(t,x))]^2)_{i \in \mathbb{N}}$, which converges and is therefore bounded, with the sequence $(h_i)_{i \in \mathbb{N}}$, that converges to 0.

Taking the particular values $(t+h_i, x+h_i v_i, U(t+h_i, x+h_i v_i))$ for $(t', x', \alpha')$ in (4.6), and dividing across by $h_i$, for every $i \in \mathbb{N}$, we obtain:

$$ \xi^0 + \xi^1 \cdot v_i \leq \lambda h_i^{-1} (U(t+h_i, x+h_i v_i) - U(t,x)) + M h_i (1 + |v_i|^2) + M h_i^{-1} |U(t+h_i, x+h_i v_i) - U(t,x)|^2. $$

Letting the integer $i$ go to $+\infty$, we have: $\xi^0 + \xi^1 \cdot v = \lim_{i \to +\infty} (\xi^0 + \xi^1 \cdot v_i) \leq -\lambda L(t^+, x, v)$, and thus we obtain: $\xi^0 + \min_{v \in F(t^+, x)} \xi^1 \cdot v + \lambda L(t^+, x, v) \leq 0$, which confirms the claim of step 1.

**Step 2.** We now prove that (c) ii) is satisfied: for every $(t, x) \in (]S, T[ \times \mathbb{R}^n) \cap \text{dom}(U)$, for every proximal vector $(\xi^0, \xi^1, -\lambda) \in N_{\text{epi } U}^P((t, x), U(t, x))$, we have

$$ \xi^0 + \min_{v \in F(t^-, x)} \xi^1 \cdot v + \lambda L(t^-, x, v) \geq 0. $$

Consider any $(t, x) \in (]S, T[ \times \mathbb{R}^n) \cap \text{dom}(U)$ and $(\xi^0, \xi^1, -\lambda) \in N_{\text{epi } U}^P((t, x), U(t, x))$. Take any $v \in F(t^-, x)$. Owing to Lemma 4.4, we can find two sequences $(v_i)_{i \in \mathbb{N}}$ and $(h_i)_{i \in \mathbb{N}}$ satisfying (4.5). Employing the same arguments used in the first step, there exists $M \in \mathbb{R}_+$ such that for every $i \in \mathbb{N}$:

$$ -(\xi^0 + \xi^1 \cdot v_i) \leq \lambda h_i^{-1} (U(t-h_i, x-h_i v_i) - U(t,x)) + M h_i (1 + |v_i|^2) + M h_i^{-1} |U(t-h_i, x-h_i v_i) - U(t,x)|^2, $$

where $\lim_{i \to +\infty} h_i^{-1} |U(t-h_i, x-h_i v_i) - U(t,x)|^2 = 0$. Bearing in mind (4.5), letting $i$ go to $+\infty$, we obtain:

$$ -(\xi^0 + \xi^1 \cdot v) \leq \lambda L(t^-, x, v). $$

Thus we have $\xi^0 + \xi^1 \cdot v + \lambda L(t^-, x, v) \geq 0$ and consequently:

$$ \xi^0 + \inf_{w \in F(t^-, x)} \xi^1 \cdot w + \lambda L(t^-, x, w) \geq 0. $$

**Step 3.** To conclude the proof we have to consider the boundary conditions (c) iii). Take any $x \in \mathbb{R}^n$. Using the lower semicontinuity of $U$, we have:

$$ \liminf_{\{(t', x') \to (S, x) \mid t' > S\}} U(t', x') \geq U(S, x), \quad \text{and} \quad \liminf_{\{(t', x') \to (T, x) \mid t' < T\}} U(t', x') \geq U(T, x). $$
If $(S, x) \notin \text{dom}(U)$, then we immediately obtain:

$$ \liminf_{\{(t', x') \to (S, x) \mid t' > S\}} U(t', x') \le U(S, x) = +\infty. $$

If $(S, x) \in \text{dom}(U)$, then using Lemma 4.3, we can find $v \in F(S^+, x)$, a sequence $(v_i)_{i \in \mathbb{N}}$ in $\mathbb{R}^n$ converging to $v$, and a strictly decreasing sequence $(h_i)_{i \in \mathbb{N}}$ in $\mathbb{R}_+$ converging to 0, such that (4.3) holds at $t = S$. As a consequence $\limsup_{i \to +\infty} U(S + h_i, x + h_i v_i) \le U(S, x)$. Thus we have:

$$ U(S, x) \le \liminf_{\{(t', x') \to (S, x) \mid t' > S\}} U(t', x') \le \liminf_{i \to +\infty} U(S + h_i, x + h_i v_i) \le U(S, x), $$

which gives the first equality in (c) iii) from Theorem 2.1.

If $(T, x) \notin \text{dom}(U)$, clearly we have: $\liminf_{\{(t', x') \to (T, x) \mid t' < T\}} U(t', x') \le U(T, x)$, so we consider the case when $(T, x) \in \text{dom}(U)$. Then fix any $v \in F(T^-, x)$. Using Lemma 4.4, we deduce the existence of a sequence $(v_i)_{i \in \mathbb{N}}$ in $\mathbb{R}^n$ converging to $v$ and a decreasing sequence $(h_i)_{i \in \mathbb{N}}$ in $\mathbb{R}_+$, that converges to 0, such that (4.5) holds at $t = T$.

Since $L(t^-, x, v)$ is finite, from (4.5) we deduce that $\limsup_{i \to +\infty} U(T - h_i, x - h_i v_i) \le U(T, x)$. It follows that

$$ U(T, x) \le \liminf_{\{(t', x') \to (T, x) \mid t' < T\}} U(t', x') \le \liminf_{i \to +\infty} U(T - h_i, x - h_i v_i) \le U(T, x). $$

Using the relation $U(T, x) = g(x)$, given by the fact $U$ satisfies (b) iii), we obtain the last desired boundary condition at $t = T$.

### 4.3. A proximal solution coincides with the value function: comparison results

We display the last part of the proof which consists in showing that if a lower semicontinuous function $U : [S, T] \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ satisfies (c) of Theorem 2.1, then it coincides with the value function for $(P_{t,x})$. We observe that for the inequality $V(t, x) \le U(t, x)$ conditions (H6) ii) and iii) are not necessary, but they are required for the opposite inequality. More precisely we will prove the following result.

**Theorem 4.5.** Assume (H1)–(H5) and (H6) i). Let $U : [S, T] \times \mathbb{R}^n \to \mathbb{R} \cup \{+\infty\}$ be a lower semicontinuous function.

(i) Suppose that $U$ satisfies (c) i), and that, for all $x \in \mathbb{R}^n$,

$$ U(T, x) = g(x) \quad \text{and} \quad \liminf_{\{(t', x') \to (S, x) \mid t' > S\}} U(t', x') = U(S, x). $$

Then $V(t, x) \le U(t, x)$ for any $(t, x) \in ([S, T] \times \mathbb{R}^n) \cap \text{dom}(U)$.

(ii) Assume, in addition, that $L$ satisfies (H6) ii) and iii). Suppose that $U$ satisfies (c) ii), and for all $x \in \mathbb{R}^n$

$$ \liminf_{\{(t', x') \to (T, x) \mid t' < T\}} U(t', x') = U(T, x) = g(x). $$

Then $V(t, x) \ge U(t, x)$ for any $(t, x) \in ([S, T] \times \mathbb{R}^n) \cap \text{dom}(U)$.

Theorem 4.5 contains two ‘comparison results’ establishing the last part of the proof of Theorem 2.1 with the implication ‘(c) ⇒ (a)’. Combined with Propositions 4.1 and 4.2, it provides uniqueness result for the characterization of the value function in the class of lower semicontinuous functions, as summarized in the Corollary below.
**Corollary 4.6.** Assume that (H1)–(H6) are satisfied. Then the value function V is the unique lower semicontinuous function solution to (HJE) in the sense of (b)–(c) of Theorem 2.1.

*Proof of Theorem 4.5 i).* In order to establish the first comparison result, bearing in mind the hypotheses reduction of Section 2, we introduce an auxiliary multivalued function: $Q : [S, T] \times \mathbb{R}^n \rightsquigarrow \mathbb{R}^n \times \mathbb{R}$ defined by:

$$Q(\tau, x) := \begin{cases} \{(v, -\eta) | v \in F(S^+, x), M_0 \ge \eta \ge L(S^+, x, v)\}, & \text{if } \tau = S, \\ \co \{(v, -\eta) | v \in \{F(\tau^-, x) \cup F(\tau^+, x)\}, M_0 \ge \eta \ge \tilde{L}(\tau, x, v)\}, & \text{if } \tau \in ]S, T[, \\ \{(v, -\eta) | v \in F(T^-, x), M_0 \ge \eta \ge L(T^-, x, v)\}, & \text{if } \tau = T, \end{cases}$$

where $\tilde{L}(\tau, x, v) := \min\{L(\tau^+, x, v), L(\tau^-, x, v)\}$. A routine analysis allows to verify that the multifunction $Q$ takes as values nonempty convex sets with elements which are (uniformly) bounded by $c := \sqrt{c_0^2 + M_0^2}$; moreover the graph of $Q$ is closed.

Take any $(t_0, x_0) \in (]S, T[ \times \mathbb{R}^n) \cap \text{dom}(U)$. The crucial point of Theorem 4.5 i) is establishing the applicability of the Weak Invariance Theorem 3.4 for the following differential inclusion:

$$\left\{ \begin{array}{l} (\dot{\tau}, \dot{x}, \dot{l})(t) \in \Gamma(\tau(t), x(t), l(t)), \quad \text{for a.e. } t \in [t_0, T], \\ (\tau(t), x(t), l(t)) \in \text{epi } U, \quad \text{for all } t \in [t_0, T], \\ (\tau(t_0), x(t_0), l(t_0)) = (t_0, x_0, U(t_0, x_0)), \end{array} \right.$$

where $\Gamma : [S, T] \times \mathbb{R}^{n+1} \rightsquigarrow \mathbb{R}^{n+2}$ is defined by

$$\Gamma(\tau, x, \ell) := \begin{cases} \co(\{(0,0,0)\} \cup (\{1\} \times Q(S, x))), & \text{if } \tau = S, \\ \{1\} \times Q(\tau, x), & \text{if } \tau \in ]S, T[, \\ \co(\{(0,0,0)\} \cup (\{1\} \times Q(T, x))), & \text{if } \tau = T. \end{cases}$$

Clearly the multifunction $\Gamma$ inherits the following properties from $Q$: the graph of $\Gamma$ is closed, for all $(\tau, x, \ell) \in [S, T] \times \mathbb{R}^{n+1}$, $\Gamma(\tau, x, \ell)$ is nonempty convex set and $\Gamma(\tau, x, \ell) \subset (c+1)\mathbb{B}$. The ‘inward pointing condition’ iii) of Theorem 3.4 is also satisfied. Indeed, for $\tau = S, T$ the construction of $\Gamma$ immediately yields the required inequality (for instance taking $w = 0$ that belongs to both $\Gamma(S, x, \ell)$ and $\Gamma(T, x, \ell)$). On the other hand, since $U$ satisfies (c) i) of Theorem 2.1, for every $(\tau, x) \in \text{dom}(U)$, and every $(\xi^0, \xi^1, -\lambda) \in N_{\text{epi } U}^P((\tau, x), U(\tau, x))$, there exists $\bar{v} \in F(\tau^+, x)$ (recall that $F(\tau^+, x)$ is compact) such that:

$$\xi^0 + \xi^1 \cdot \bar{v} + \lambda L(\tau^+, x, \bar{v}) \le 0 \quad (4.7)$$

and $\min_{w \in \Gamma(\tau, x, \ell)} (\xi^0, \xi^1, -\lambda) \cdot w \le \xi^0 + \xi^1 \cdot \bar{v} + \lambda L(\tau^+, x, \bar{v})$. Then, the Weak Invariance Theorem 3.4 is applicable and there exists $(\tau(\cdot), x(\cdot), \ell(\cdot)) \in W^{1,1}([t_0, T], \mathbb{R} \times \mathbb{R}^n \times \mathbb{R})$ satisfying $\tau(t) = t$ and

$$\left\{ \begin{array}{l} (\dot{x}(t), \dot{l}(t)) \in Q(t, x(t)), \text{ for a.e. } t \in [t_0, T] \\ x(t_0) = x_0, \ell(t_0) = U(t_0, x_0) \\ \ell(t) \ge U(t, x(t)) \quad \text{for all } t \in [t_0, T]. \end{array} \right.$$
Taking into account the definition of the multivalued function $Q$ and the hypotheses on both $F$ and $L$, we deduce that $x(\cdot)$ is an $F$-trajectory and that $\dot{l}(s) \le -L(s, x(s), \dot{x}(s))$ for a.e. $s \in [t_0, T]$. Hence we have:

$$g(x(T)) = U(T, x(T)) \le l(T) = l(t_0) + \int_{t_0}^{T} \dot{l}(s)ds \le U(t_0, x_0) - \int_{t_0}^{T} L(s, x(s), \dot{x}(s))ds,$$

which implies:

$$g(x(T)) + \int_{t_0}^{T} L(s, x(s), \dot{x}(s))ds \le U(t_0, x_0).$$

Thus we obtain:

$$V(t_0, x_0) \le U(t_0, x_0).$$

If $(S, x_0)$ belongs to $\text{dom}(U)$, we pick a decreasing sequence $(h_i)_{i \in \mathbb{N}}$ in $\mathbb{R}_+$ that converges to 0 and a sequence $(y_i)_{i \in \mathbb{N}}$ in $\mathbb{R}^n$ that converges to $x_0$ such that:

$$\lim_{i \to +\infty} U(S + h_i, y_i) = \liminf_{\{(t', x') \to (S, x) \mid t' > S\}} U(t', x') = U(S, x_0).$$

From what precedes, for every integer $i \in \mathbb{N}$, we have:

$$V(S + h_i, y_i) \le U(S + h_i, y_i).$$

Passing to the limit inferior in that last equation yields:

$$V(S, x_0) = \liminf_{\{(t', x') \to (S, x) \mid t' > S\}} V(t', x') \le \liminf_{i \to +\infty} V(S + h_i, y_i) \le \lim_{i \to +\infty} U(S + h_i, y_i) = U(S, x_0).$$

Note that we also have $g(x_0) = V(T, x_0) \le U(T, x_0) = g(x_0)$. $\square$

*Proof of Theorem 4.5 ii).* Pick $(\bar{t}, \bar{x}) \in ([S, T] \times \mathbb{R}^n) \cap \text{dom}(U)$ and let $x \in W^{1,1}([\bar{t}, T], \mathbb{R}^n)$ be an $F$-trajectory such that $x(\bar{t}) = \bar{x}$. We want to prove:

$$U(\bar{t}, \bar{x}) \le g(x(T)) + \int_{\bar{t}}^{T} L(s, x(s), \dot{x}(s))ds.$$

We can assume that $g(x(T)) < +\infty$, otherwise we automatically have the desired inequality. Using the fact that $\liminf_{\{(t', x') \to (T, x) \mid t' < T\}} U(t', x') = U(T, x) = g(x)$, we can find a sequence of points $(T_i, y_i)$ in $[\bar{t}, T] \times \mathbb{R}^n$ such that $\lim_{i \to +\infty} (T_i, y_i) = (T, x(T))$ and $\lim_{i \to +\infty} U(T_i, y_i) = U(T, x(T))$. Invoking Filippov's existence theorem and arguing as in [4], we obtain a subsequence of $F$-trajectories $x_i(\cdot)$ on $[\bar{t}, T]$ such that $x_i(T_i) = y_i$, for all $i$, and $\|x_i(\cdot) - x(\cdot)\|_{W^{1,1}} \to 0$ as $i \to +\infty$.

The multivalued function $F$ satisfies the assumptions which allow to apply Carathéodory's parametrization theorems ([1], Thms. 9.6.2 and 9.7.2). Hence there exists a mesurable function:

$$f : [S, T] \times \mathbb{R}^n \times \mathbb{B} \to \mathbb{R}^n,$$
such that:

$$
\left\{
\begin{array}{l}
\text{For every } (t, x) \in [S, T] \times \mathbb{R}^n, F(t, x) = f(t, x, \mathbb{B}); \\
\text{For every } (x, u) \in \mathbb{R}^n \times \mathbb{B}, f(\cdot, x, u) \text{ is mesurable}; \\
\text{For every } (t, u) \in [S, T] \times \mathbb{B}, f(t, \cdot, u) \text{ is } 10nk_F(t)-\text{Lipschitz}; \\
\text{For every } (t, x) \in [S, T] \times \mathbb{R}^n, (u, u') \in \mathbb{B}^2, |f(t, x, u) - f(t, x, u')| \le 5n \max_{v \in F(t,x)} |v| |u - u'|.
\end{array}
\right.
\qquad (4.8)
$$

Under our hypotheses, for all $(s, x, u) \in ]S, T[ \times \mathbb{R}^n \times \mathbb{B}$, we have (cf. [4]):

$$
\lim_{s' \downarrow s, x' \to x, u' \to u} f(s', x', u') \in F(s^{-}, x) \text{ and } \lim_{s' \uparrow s, x' \to x, u' \to u} f(s', x', u') \in F(s^{+}, x).
$$

Fix $i \ge 0$. Since $x_i(\cdot)$ is an $F$-trajectory, for almost every $t \in [\bar{t}, T]$:

$$
\dot{x}_i(t) \in f(t, x_i(t), \mathbb{B}).
$$

and, using Filippov's selection theorem (cf. [20], Thm. 2.3.13), there exists a mesurable selection $u_i : [\bar{t}, T] \to \mathbb{B}$
such that:

$$
\dot{x}_i(t) = f(t, x_i(t), u_i(t)), \text{ for almost every } t \in [\bar{t}, T].
$$

Let $\varepsilon > 0$. Lusin's theorem (cf. [7], Prop. 6.14) allows us to find a pair of functions $(x_i^\varepsilon, u_i^\varepsilon)$ defined on $[\bar{t}, T]$ such that $x_i^\varepsilon(T_i) = x_i(T_i)$ and

$$
\left\{
\begin{array}{l}
\dot{x}_i^\varepsilon(t) = f(t, x_i^\varepsilon(t), u_i^\varepsilon(t)) \text{ for almost every } t \in [\bar{t}, T]; \\
\text{the control } u_i^\varepsilon \text{ is continuous}; \\
\|x_i - x_i^\varepsilon\|_{L^\infty([\bar{t}, T], \mathbb{R}^n)} \le \varepsilon; \\
\text{meas}(\{t \in [\bar{t}, T] \mid u_i(t) - u_i^\varepsilon(t) \ne 0\}) \le \varepsilon.
\end{array}
\right.
\qquad (4.9)
$$

For every $(t, x) \in [\bar{t}, T] \times \mathbb{R}^n$, we write $v^+(t, x) := f(t^+, x, u_i^\varepsilon(t))$, $v^-(t, x) := f(t^-, x, u_i^\varepsilon(t))$. We define two multivalued functions $F_i^\varepsilon : [\bar{t}, T_i] \times \mathbb{R}^n \rightsquigarrow \mathbb{R}^n$, $\Lambda_i^\varepsilon : [\bar{t}, T_i] \times \mathbb{R}^n \rightsquigarrow \mathbb{R}$ by the relations: for every $(t, x) \in [\bar{t}, T_i] \times \mathbb{R}^n$:

$$
F_i^\varepsilon(t, x) := \co \{v^-(t, x), v^+(t, x)\},
$$

$$
\Lambda_i^\varepsilon(t, x) := \co \{L(t^-, x, v^-(t, x)), L(t^+, x, v^+(t, x))\}.
$$

Then we set a new multifunction $\Gamma_i^\varepsilon : [0, T_i - \bar{t}] \times \mathbb{R}^n \times \mathbb{R} \rightsquigarrow \mathbb{R} \times \mathbb{R}^n \times \mathbb{R}$ defined for every $(t, x, \ell) \in [0, T_i - \bar{t}] \times \mathbb{R}^n \times \mathbb{R}$:

$$
\Gamma_i^\varepsilon(t, x, \ell) = \begin{cases} \co \{(0,0,0) \cup \{1\} \times -F_i^\varepsilon(\bar{t}, x) \times \Lambda_i^\varepsilon(\bar{t}, x)\}, & \text{if } t = T_i - \bar{t}, \\ \{1\} \times -F_i^\varepsilon(T_i - t, x) \times \Lambda_i^\varepsilon(T_i - t, x), & \text{if } t \in [0, T_i - \bar{t}]. \end{cases}
$$

The multivalued function $\Gamma_i^\varepsilon$ is convex, compact valued and has closed graph. We consider the following
differential inclusion:

$$
\left\{
\begin{array}{l}
(\dot{\tau}(t), \dot{y}(t), \dot{l}(t)) \in \Gamma_i^\varepsilon(\tau(t), y(t), l(t)), \text{ for a.e. } t \in [0, T_i - \bar{t}], \\
\tau(0) = 0, y(0) = x_i^\varepsilon(T_i) = x_i(T_i), \ell(0) = U(T_i, x(T_i)), \\
\ell(t) \ge U(T_i - \tau(t), y(t)), \text{ for all } t \in [0, T_i - \bar{t}].
\end{array}
\right.
\qquad (4.10)
$$
We define the arc

$$t \mapsto (\tau_i^\epsilon(t), y_i^\epsilon(t), \ell_i^\epsilon(t)) := \left( t, x_i^\epsilon(T_i - t), U(T_i, x_i(T_i)) + \int_0^t L(T_i - s, x_i^\epsilon(T_i - s), \dot{x}_i^\epsilon(T_i - s))ds \right).$$

The arc $(\tau_i^\epsilon, y_i^\epsilon, \ell_i^\epsilon)$ is the unique $\Gamma_i^\epsilon$-trajectory with initial condition $(0, x(T_i), U(T_i, x_i(T_i)))$. Owing to the 'hypotheses reduction' argument of Section 2, we deduce that there exists a constant $c := \sqrt{c_0^2 + M_0^2}$ such that $\Gamma_i^\epsilon(t, x, \ell) \subset (c+1)\mathbb{B}$, for every $(t, x, \ell)$.

For every $(\tau, x) \in [0, T_i - \bar{t}] \times \mathbb{R}^n$, we set $\tilde{U}(\tau, x) := U(T_i - \tau, x)$. Therefore the last condition in (4.10) can be interpreted as the inclusion $(\tau(t), y(t), \ell(t)) \in \text{epi } \tilde{U}$ for all $t \in [0, T_i - \bar{t}]$. We claim that the Weak Invariance Theorem 3.4 is applicable to the differential inclusion (4.10). We have already observed that the assumptions i) and ii) of this theorem are satisfied. We show now that $\Gamma_i^\epsilon$ also satisfies the last ('inward pointing') condition iii). That is, for every pair $(\tau, x) \in ([0, T_i - \bar{t}] \times \mathbb{R}^n) \cap \text{dom}(\tilde{U})$ and every $\ell \ge \tilde{U}(\tau, x)$:

$$\min_{w \in \Gamma_i^\epsilon(\tau, x, \ell)} (\xi^0, \xi^1, -\lambda) \cdot w \le 0, \text{ for all } (\xi^0, \xi^1, -\lambda) \in N_{\text{epi } \tilde{U}}^P((\tau, x), \ell). \quad (4.11)$$

Indeed, let $(\tau, x) \in ([0, T_i - \bar{t}] \times \mathbb{R}^n) \cap \text{dom}(\tilde{U})$ and $(\xi^0, \xi^1, -\lambda) \in N_{\text{epi } \tilde{U}}^P((\tau, x), \tilde{U}(\tau, x))$ (we recall that we can always reduce to the case $\ell = \tilde{U}(\tau, x)$), which is equivalent to say:

$$(-\xi^0, \xi^1, -\lambda) \in N_{\text{epi } U}^P((T_i - \tau, x), U(T_i - \tau, x)).$$

We notice that $v^-(T_i - \tau, x) \in F_i^\epsilon(T_i - \tau, x) \cap F((T_i - \tau)^-, x)$ and, bearing in mind $U$ satisfies (2.5) of condition (c) ii), we obtain:

$$-\xi^0 + \xi^1 \cdot v^-(T_i - \tau, x) + \lambda L((T_i - \tau)^-, x, v^-(T_i - \tau, x)) \ge 0.$$

Hence we can confirm (4.11) by choosing $w = (1, -v^-(T_i - \tau, x), L((T_i - \tau)^-, x, v^-(T_i - \tau, x)))$. As a consequence we can apply the Weak Invariance Theorem obtaining that the arc $(\tau_i^\epsilon(\cdot), y_i^\epsilon(\cdot), \ell_i^\epsilon(\cdot))$ is the solution to (4.10). For $t = T_i - \bar{t}$, by a change of variable, we have:

$$U(T_i, x_i(T_i)) + \int_{\bar{t}}^{T_i} L(s, x_i^\epsilon(s), \dot{x}_i^\epsilon(s))ds \ge U(\bar{t}, x_i^\epsilon(\bar{t})), \text{ for all } \epsilon. \quad (4.12)$$

Invoking condition (H2)*, for all $t \in [\bar{t}, T]$, $\max_{v \in F(t, x_i(t))} |v| \le c_0$. So, for almost every $s \in [\bar{t}, T]$ we have:

$$\begin{aligned} |\dot{x}_i^\epsilon(s) - \dot{x}_i(s)| &= |f(s, x_i^\epsilon(s), u_i^\epsilon(s)) - f(s, x_i(s), u_i(s))| \\ &\le |f(s, x_i^\epsilon(s), u_i^\epsilon(s)) - f(s, x_i(s), u_i^\epsilon(s))| + |f(s, x_i(s), u_i^\epsilon(s)) - f(s, x_i(s), u_i(s))| \\ &\le 10nk_F(s)|x_i^\epsilon(s) - x_i(s)| + 5n \max_{v \in F(s, x_i(s))} |v| |u_i^\epsilon(s) - u_i(s)| \\ &\le 10nk_F(s)\epsilon + 5nc_0|u_i^\epsilon(s) - u_i(s)|. \end{aligned}$$

Since meas($\{t \in [\bar{t}, T] | u_i(t) - u_i^\epsilon(t) \ne 0\}$) $\le \epsilon$ and $\|u_i - u_i^\epsilon\|_{L^\infty} \le 2$, this implies that

$$\dot{x}_i^\epsilon \xrightarrow{\epsilon \to 0} \dot{x}_i.$$
As a consequence, up to a subsequence, $\dot{x}_i^\varepsilon(\cdot)$ converges to $\dot{x}_i(\cdot)$ almost everywhere in $[\bar{t}, T]$. Using (H6)* and (3.1), we can apply Lebesgue's dominated convergence theorem, and we obtain:

$$ \int_{\bar{t}}^{T_i} L(s, x_i^\varepsilon(s), \dot{x}_i^\varepsilon(s))ds \xrightarrow{\varepsilon \to 0} \int_{\bar{t}}^{T_i} L(s, x_i(s), \dot{x}_i(s))ds. $$

Passing to the limit inferior in (4.12), bearing in mind that $x_i^\varepsilon(\bar{t}) \xrightarrow{\varepsilon \to 0} x_i(\bar{t})$, and $U$ is lower semicontinuous, we obtain

$$ U(T_i, x_i(T_i)) + \int_{\bar{t}}^{T_i} L(s, x_i(s), \dot{x}_i(s))ds \geq U(\bar{t}, x_i(\bar{t})), \text{ for all } i. $$

Then, as $i \to +\infty$,

$$ g(x(T)) + \int_{\bar{t}}^{T} L(s, x(s), \dot{x}(s))ds \geq U(\bar{t}, \bar{x}). $$

Since $x(\cdot)$ was an arbitrary $F$-trajectory satisfying $x(\bar{t}) = \bar{x}$, we deduce that

$$ V(\bar{t}, \bar{x}) \geq U(\bar{t}, \bar{x}). $$

This concludes the proof.

□

## 5. PROOFS OF THEOREM 2.2, AND PROPOSITIONS 2.3 AND 2.4

*Proof of Theorem 2.2.* The proof is organized as follows: in Step 1 we show that, assuming hypotheses (H1)–(H6), the value function $V$ is a viscosity solution in the sense of condition (d) of Theorem 2.2. In Step 2, we prove that if a lower semicontinuous function $U$ satisfies (d) i) and (d) iii) of Theorem 2.2, then $V \leq U$. In Step 3 we prove that if we impose the additional assumption $(g^*)_* = g$, then any lower semicontinuous function $U$ satisfying (d) ii) and (d) iii) satisfies $U \leq V$.

### Step 1: The value function $V$ satisfies (d) i)–iii).

Assume that hypotheses (H1)–(H6) are satisfied. We first observe that, from the *a priori* boundedness of the $F$-trajectories, and the local boundedness of $g$ and $L$, it immediately follows that $V$ is locally bounded.

Let $(t, x) \in ]S, T[ \times \mathbb{R}^n$. We invoke the 'hypotheses reduction' of Section 2, and we can apply the same argument of the proof of Step 1 of Proposition 4.1, obtaining the existence of a vector $v \in F(t^+, x)$ such that:

$$ D_{\uparrow} V((t, x), (1, v)) \leq -L(t^+, x, v). $$

This and the relation (2.2) imply that:

$$ (1, v, -L(t^+, x, v)) \in T_{\text{epi}} V((t, x), V(t, x)). $$

Let $(\xi^0, \xi^1, -\lambda) \in \hat{N}_{\text{epi}} V((t, x), V(t, x))$. Since $\hat{N}_{\text{epi}} V((t, x), V(t, x))$ is the polar cone to the set $T_{\text{epi}} V((t, x), V(t, x))$ we have:

$$ (\xi^0, \xi^1, -\lambda) \cdot (1, v, -L(t^+, x, v)) \leq 0. $$
This easily implies that:

$$ \xi^0 + \min_{v \in F(t^+, x)} [\xi^1 \cdot v + \lambda L(t^+, x, v)] \le 0, \text{ for all } (\xi^0, \xi^1, -\lambda) \in \hat{N}_{\text{epi}} V((t, x), V(t, x)), $$

which confirms (d) i).

Let $(t, x) \in ]S, T[ \times \mathbb{R}^n$ and $\tilde{v} \in F(t^+, x)$. There exists a sequence $(t_i, x_i)_{i \in \mathbb{N}}$ in $]S, T[ \times \mathbb{R}^n \setminus \{(t, x)\}$ that converges to $(t, x)$ such that:

$$ \lim_{i \to +\infty} V(t_i, x_i) = V^*(t, x). $$

We claim that we can extract a subsequence such that $t_i > t$ for all $i \in \mathbb{N}$. Let us assume that $t_i \le t$ for every $i \in \mathbb{N}$ and take a strictly decreasing sequence $(\tau_i)_{i \in \mathbb{N}}$ in $]t, T]$ that converges to $t$. Fix any $i \in \mathbb{N}$, and take an F-trajectory $x_i(\cdot) \in W^{1,1}([t_i, T], \mathbb{R}^n)$ such that $x_i(t_i) = x_i$. Using the principle of optimality, we obtain:

$$ V(t_i, x_i) - \int_{t_i}^{\tau_i} L(s, x_i(s), \dot{x}_i(s))ds \le V(\tau_i, x_i(\tau_i)). $$

Using the local boundedness of $L$ given by condition (H5)*, there exists $M_0 > 0$ such that for every $i \in \mathbb{N}$:

$$ V(t_i, x_i) - M_0|\tau_i - t_i| \le V(\tau_i, x_i(\tau_i)). $$

Passing to the limit superior and using the upper semicontinuity of $V^*$, we obtain:

$$ V^*(t, x) = \limsup_{i \to +\infty} V(t_i, x_i) \le \limsup_{i \to +\infty} V(\tau_i, x_i(\tau_i)) \le \limsup_{i \to +\infty} V^*(\tau_i, x_i(\tau_i)) \le V^*(t, x). $$

Hence $\limsup_{i \to +\infty} V(\tau_i, x_i(\tau_i)) = V^*(t, x)$ and there exists a subsequence $(i_k)_{k \in \mathbb{N}}$ for which:

$$ V(\tau_{i_k}, x_{i_k}(\tau_{i_k})) \xrightarrow{k \to +\infty} V^*(t, x). $$

Fix any $\tilde{v} \in F(t^+, x)$. Then for every $i \in \mathbb{N}$, there exists $v_i \in F(t_i^+, x_i)$ such that $\lim_{i \to +\infty} v_i = \tilde{v}$. For every $i \in \mathbb{N}$, we consider the arc

$$ y_i(s) := x_i + (s - t_i)v_i, \text{ for all } s \in [t_i, T]. $$

Using the Filippov existence theorem, for every $i \in \mathbb{N}$ there exists an F-trajectory $z_i(\cdot)$ that satisfies $z_i(t_i) = x_i$ and such that for every $h \in ]0, T - t_i]$

$$ \|z_i - y_i\|_{L^\infty([t_i, t_i+h], \mathbb{R}^n)} \le K \left( \int_{t_i}^{t_i+h} d_{F(s, y_i(s))}(v_i)ds \right), $$

where $K = \exp(\int_S^T k_F(s)ds)$. From the a priori boundedness of F-trajectories, we can pick $R_0 > 0$ such that, for every $i \in \mathbb{N}$, $|y_i(s)| \le R_0$ for every $s \in [t_i, T]$. Observe also that $|\dot{y}_i(s)| \le c_0$, for any $i \in \mathbb{N}$ and for almost every $s \in [t_i, T]$.

For every $i \in \mathbb{N}$, we define $\delta_i = \max\{|V(t_i, x_i) - V^*(t, x)|, |x_i - x|, |t_i - t|\}$. Take a strictly decreasing sequence $(h_i)_{i \in \mathbb{N}}$ that converges to 0 such that $h_i \ge \sqrt{\delta_i}$.
We recall the definition of $D_{\downarrow}V^*((t,x), (1, \tilde{v}))$:

$$D_{\downarrow}V^*((t,x), (1, \tilde{v})) = \limsup_{\substack{h \downarrow 0 \\ e \to 1, w \to \tilde{v}}} h^{-1} [V^* (t + he, x + hw) - V^* (t, x)].$$

Fix any $i \in \mathbb{N}$, and set $w_i = \frac{1}{h_i} \int_{t_i}^{t_i+h_i} \dot{z}_i(s)ds$. Note that we have:

$$|v_i - w_i| \le K \left( \int_{t_i}^{t_i+h_i} d_{F(s,y_i(s))}(v_i)ds \right) \le K\theta_i(h_i),$$

where

$$\theta_i(h) := \begin{cases} \sup_{0<s-t_i \le h} d_H(F(s,y), F(t_i^+, x_i)), & \text{if } h \ne 0, \\ 0, & \text{otherwise.} \end{cases}$$

There exists $\tau \in [t_i, t_i + h_i]$ and $z \in \mathbb{R}^n$ verifying $|x_i - z| \le c_0h_i$ such that:

$$\theta_i(h_i) \le d_H(F(\tau, z), F(t_i^+, x_i)) + \frac{1}{i+1}.$$

Hence we obtain:

$$\theta_i(h_i) \le d_H(F(\tau, z), F(t^+, x)) + d_H(F(t^+, x), F(t_i^+, x_i)) + \frac{1}{i+1}.$$

We notice that:

$$\tau - t \le (\tau - t_i) + (t_i - t) \le h_i + h_i^2 \text{ and } |x - z| \le |z - x_i| + |x_i - x| \le h_i c_0 + h_i^2.$$

This yields:

$$\theta_i(h_i) \le \sup_{\substack{0<s-t \le h_i+h_i^2 \\ |y-x| \le c_0h_i+h_i^2}} d_H(F(s,y), F(t^+, x)) + \sup_{\substack{0<s-t \le h_i^2 \\ |y-x| \le h_i^2}} d_H(F(t^+, x), F(s^+, y)) + \frac{1}{1+i},$$

which implies that $\theta_i(h_i) \xrightarrow{i \to +\infty} 0$. Recalling that for every $i \in \mathbb{N}$, $|\tilde{v} - w_i| \le \theta_i(h_i) + |v_i - \tilde{v}|$, we obtain:

$$w_i \xrightarrow{i \to +\infty} \tilde{v}.$$

For every $i \in \mathbb{N}$, we define: $e_i := 1 - \frac{t-t_i}{h_i}$ and $\tilde{w}_i := w_i - \frac{x-x_i}{h_i}$, and immediately notice that $\lim_{i \to +\infty} (e_i, \tilde{w}_i) = (1, \tilde{v})$. This yields:

$$\begin{aligned} D_{\downarrow}V^*((t,x), (1, \tilde{v})) &\ge \limsup_{i \to +\infty} h_i^{-1} [V^* (t + h_i e_i, x + h_i \tilde{w}_i) - V^* (t, x)], \\ &= \limsup_{i \to +\infty} h_i^{-1} [V^* (t_i + h_i, x_i + h_i w_i) - V^* (t, x)], \\ &\ge \limsup_{i \to +\infty} h_i^{-1} [V (t_i + h_i, x_i + h_i w_i) - V^* (t, x)]. \end{aligned}$$
Fix $i \in \mathbb{N}$. We have:

$$V(t_i + h_i, x_i + h_i w_i) - V^*(t, x) \geq V(t_i + h_i, x_i + h_i w_i) - V(t_i, x_i) - \delta_i. \quad (5.1)$$

Using the principle of optimality, we obtain:

$$V(t_i + h_i, x_i + h_i w_i) - V(t_i, x_i) \geq - \int_{t_i}^{t_i+h_i} L(s, z_i(s), \dot{z}_i(s))ds.$$

Hence, dividing across equation (5.1) by $h_i$, passing to the limit superior in this inequality while recalling $\frac{\delta_i}{h_i} \leq \sqrt{\delta_i}$, we obtain:

$$D_{\downarrow} V^*((t, x), (1, \tilde{v})) \geq - \liminf_{i \to +\infty} \frac{1}{h_i} \int_{t_i}^{t+h_i} L(s, z_i(s), \dot{z}_i(s))ds. \quad (5.2)$$

We recall that from Lemma 3.1, there exists $k_L > 0$ such that for every $(t', x') \in [S, T] \times \mathbb{R}^n$, and $v, v' \in c_0\mathbb{B}$:

$$|L(t', x', v) - L(t', x', v')| \leq k_L |v - v'|.$$

As a consequence, for every $i \in \mathbb{N}$ we have:

$$\int_{t_i}^{t_i+h_i} L(s, z_i(s), \dot{z}_i(s))ds \leq \int_{t_i}^{t_i+h_i} L(s, z_i(s), \tilde{v})ds + \int_{t_i}^{t_i+h_i} k_L|\dot{z}_i(s) - \tilde{v}|ds, \\ \leq h_i \sup_{\substack{|z-x| \leq c_0h_i+h_i^2 \\ 0<s-t \leq h_i+h_i^2}} L(s, z, \tilde{v}) + h_i k_L(K\theta_i(h_i) + |v_i - \tilde{v}|).$$

Dividing across by $h_i$, passing to the limit inferior as $i$ goes to $+\infty$ gives:

$$\liminf_{i \to +\infty} \frac{1}{h_i} \int_{t_i}^{t+h_i} L(s, z_i(s), \dot{z}_i(s))ds \leq L(t^+, x, \tilde{v}). \quad (5.3)$$

Combining (5.2) and (5.3) we obtain:

$$D_{\downarrow} V^*((t, x), (1, \tilde{v})) \geq -L(t^+, x, \tilde{v}).$$

From the relation (2.3), this implies that:

$$(1, \tilde{v}, -L(t^+, x, \tilde{v})) \in T_{\text{hyp}} V^*((t, x), V^*(t, x))$$

Let $(-\xi^0, -\xi^1, \lambda) \in \hat{N}_{\text{hyp}} V^*((t, x), V^*(t, x))$. Necessarily we have $\lambda \in \mathbb{R}_+$. Hence using the polarity relation between the contingent cone and the strict normal cone we have:

$$-\xi^0 - \xi^1 \cdot \tilde{v} - \lambda L(t^+, x, \tilde{v}) \leq 0.$$

This relation being valid for all $v \in F(t^+, x)$, we obtain:

$$-\xi^0 + \max_{v \in F(t^+, x)} [-\xi^1 \cdot \tilde{v} - \lambda L(t^+, x, \tilde{v})] \leq 0,$$
which confirms (d) ii).

To prove that $V$ satisfies (d) iii), only the assertion $V^*(T, \cdot) = g^*(\cdot)$ remains to be proved. Since $V(T, \cdot) = g(\cdot)$, it is obvious that $V^*(T, x) \ge g^*(x)$ for every $x \in \mathbb{R}^n$. We prove that the converse inequality is also satisfied. Fix any $x \in \mathbb{R}^n$. There exists a sequence $(t_i, x_i)_{i \in \mathbb{N}}$ in $[S, T] \times \mathbb{R}^n \setminus \{(T, x)\}$ converging to $(T, x)$ such that:

$$ \lim_{i \to +\infty} V(t_i, x_i) = \limsup_{(t,y) \to (T,x)} V(t, y) = V^*(T, x). $$

For every $i \in \mathbb{N}$ there exists an $F$-trajectory $x_i(\cdot) \in W^{1,1}([t_i, T], \mathbb{R}^n)$ such that $x_i(t_i) = x_i$. By the principle of optimality:

$$ V(t_i, x_i) - \int_{t_i}^t L(s, x_i(s), \dot{x}_i(s))ds \le V(t, x_i(t)), \quad \text{for all } t \in [t_i, T]. $$

Using again condition (H5)*, we know that there exists a constant $M_0 > 0$ such that for every $i \in \mathbb{N}$:

$$ V(t_i, x_i) - M_0 |T - t_i| \le V(T, x_i(T)) = g(x_i(T)). $$

Using the fact $\lim_{i \to +\infty} x_i(T) = x$, we pass to the limit superior as $i$ tends to $+\infty$ and obtain:

$$ V^*(T, x) \le \limsup_{i \to +\infty} g(x_i(T)) \le \limsup_{y \to x} g(y) = g^*(x), $$

which achieves to show that $V$ satisfies (d) iii).

**Step 2.** We show that if $U$ satisfies (d) i) and (d) iii), then for every $(\bar{t}, \bar{x}) \in [S, T] \times \mathbb{R}^n$ we have $V(\bar{t}, \bar{x}) \le U(\bar{t}, \bar{x})$. Bearing in mind (2.1), from (2.6) we deduce that for all $(t, x) \in ]S, T[ \times \mathbb{R}^n$

$$ \xi^0 + \min_{v \in F(t^+, x)} [\xi^1 \cdot v + \lambda L(t^+, x, v)] \le 0, \text{ for all } (\xi^0, \xi^1, -\lambda) \in N_{\text{epi } U}((t, x), U(t, x)). $$

This implies that $U$ satisfies (c) i) from Theorem 2.1. Since $U$ satisfies (d) iii), we can use the same arguments employed in the proof of Theorem 4.5 i) we have:

$$ V(\bar{t}, \bar{x}) \le U(\bar{t}, \bar{x}), \text{ for every } (\bar{t}, \bar{x}) \in [S, T] \times \mathbb{R}^n. $$

**Step 3.** We prove that if $U$ satisfies (d) ii) and (d) iii), then for every $(\bar{t}, \bar{x}) \in [S, T] \times \mathbb{R}^n$ we have $U(\bar{t}, \bar{x}) \le V(\bar{t}, \bar{x})$.

Using (d) iii), we can restrict attention to the case when $(\bar{t}, \bar{x}) \in ]S, T[ \times \mathbb{R}^n$. Let $x \in W^{1,1}([\bar{t}, T], \mathbb{R}^n)$ be an $F$-trajectory such that $x(\bar{t}) = \bar{x}$. We want to prove that:

$$ U(\bar{t}, \bar{x}) \le g(x(T)) + \int_{\bar{t}}^T L(s, x(s), \dot{x}(s))ds. $$

We can find a sequence $(\xi_j)_{j \in \mathbb{N}^*}$ in $\mathbb{R}^n$, converging to $x(T)$, such that:

$$ \lim_{j \to +\infty} g^*(\xi_j) = (g^*)_*(x(T)). $$

Applying Carathéodory's parametrization theorem and Filippov's selection theorem, we can find a mesurable function $u(\cdot)$ such that $\dot{x}(t) = f(t, x(t), u(t))$ for almost every $t \in [\bar{t}, T]$, for a Lipschitz continuous parametrization $f$ of $F$ satisfying (4.8). Applying Lusin's theorem, for every $j \in \mathbb{N}^*$ we construct a pair of functions $(z_j, u_j)$
defined on $[\bar{t}, T]$ such that:

$$ egin{cases} 
\dot{z}_j(t) = f(t, z_j(t), u_j(t)) & \text{for almost every } t \in [\bar{t}, T] \text{ and } z_j(\bar{t}) = x(\bar{t}); \\ 
\text{the control } u_j \text{ is continuous}; \\ 
\|x - z_j\|_{L^\infty([\bar{t}, T], \mathbb{R}^n)} \le \frac{1}{j}; \\ 
\text{meas}(\{t \in [S, T] \mid u(t) - u_j(t) \ne 0\}) \le \frac{1}{j}. 
\end{cases} 
\tag{5.4} $$

For every $j \in \mathbb{N}^*$, we define $y_j(\cdot) \in W^{1,1}([0, T - \bar{t}], \mathbb{R}^n)$ as the solution to the following differential equation:

$$ egin{cases} 
\dot{y}(s) = -f(T - s, y(T - s), u_j(T - s)) & \text{for a.e. } s \in [0, T - \bar{t}], \\ 
y(0) = \xi_j. 
\end{cases} $$

For every $j \in \mathbb{N}^*$, we note $\bar{x}_j := y_j(T - \bar{t})$ and define $x_j(\cdot) \in W^{1,1}([\bar{t}, T], \mathbb{R}^n)$ by:

$$ x_j(s) := y_j(T - s), $$

which implies that $x_j(\cdot)$ is the solution to the following differential equation:

$$ egin{cases} 
\dot{y}(s) = f(s, y(s), u_j(s)) & \text{for a.e. } s \in [\bar{t}, T], \\ 
y(0) = \bar{x}_j. 
\end{cases} $$

Owing to the Lipschitz continuity of $f$ and the properties of $(u_j)_{j \in \mathbb{N}^*}$ we have:

$$ \|x_j - x\|_{W^{1,1}([\bar{t}, T], \mathbb{R}^n)} \xrightarrow{j \to +\infty} 0. $$

For every $(t, x) \in [\bar{t}, T] \times \mathbb{R}^n$, we write $v^+(t, x) := f(t^+, x, u_j(t))$, $v^-(t, x) := f(t^-, x, u_j(t))$.

We then define two multivalued functions $F_j : [\bar{t}, T] \times \mathbb{R}^n \rightsquigarrow \mathbb{R}^n$, $\Lambda_j : [\bar{t}, T] \times \mathbb{R}^n \rightsquigarrow \mathbb{R}$ by the relations: for every $(t, x) \in [\bar{t}, T] \times \mathbb{R}^n$:

$$ F_j(t, x) := \co \{v^-(t, x), v^+(t, x)\}, $$

$$ \Lambda_j(t, x) := -\co \{L(t^-, x, v^-(t, x)), L(t^+, x, v^+(t, x))\}. $$

Then we set a new multifunction $\Gamma_j : [\bar{t}, T] \times \mathbb{R}^n \times \mathbb{R} \rightsquigarrow \mathbb{R} \times \mathbb{R}^n \times \mathbb{R}$ defined for every $(t, x, \ell) \in [\bar{t}, T] \times \mathbb{R}^n \times \mathbb{R}$:

$$ \Gamma_j(t, x, \ell) := \begin{cases} \{1\} \times F_j(t, x) \times \Lambda_j(t, x), & \text{if } t \in [\bar{t}, T], \\ \co \{(0, 0, 0) \cup \{1\} \times F_j(T, x) \times \Lambda_j(T, x)\}, & \text{if } t = T. \end{cases} $$

Observe that the multivalued function $\Gamma_j$ is convex, compact valued and has closed graph.

We consider the following differential inclusion:

$$ egin{cases} 
(\dot{\tau}(t), \dot{y}(t), \dot{\ell}(t)) \in \Gamma_j(\tau(t), y(t), \ell(t)), & \text{for a.e. } t \in [\bar{t}, T], \\ 
\tau(\bar{t}) = \bar{t}, y(\bar{t}) = \bar{x}_j, \ell(\bar{t}) = U^*(\bar{t}, \bar{x}_j), \\ 
(\tau(t), y(t), \ell(t)) \in \text{hyp } U^*, & \text{for all } t \in [\bar{t}, T]. 
\end{cases} 
\tag{5.5} $$

Observe that the last condition in (5.5) means that $\ell(t) \le U^*(\tau(t), y(t))$, for all $t \in [\bar{t}, T]$.
We define the arc on $[\bar{t}, T]$

$$t \mapsto (\tau_j(t), x_j(t), \ell_j(t)) := \left(t, x_j(t), U^*(\bar{t}, \bar{x}_j) - \int_{\bar{t}}^t L(s, x_j(s), \dot{x}_j(s))ds\right).$$

Observe that $(\tau_j, x_j, \ell_j)$ is the unique $\Gamma_j$-trajectory with initial condition $(\bar{t}, \bar{x}_j, U^*(\bar{t}, \bar{x}_j))$.

Assumptions i) and ii) of Weak Invariance Theorem 3.4 are satisfied from the discussion above and from the fact that the ‘hypotheses reduction’ of Section 2 guarantee also that $\Gamma_j(\tau, x, \ell) \subset (c+1)\mathbb{B}$, for every $(\tau, x, \ell)$, where $c := \sqrt{c_0^2 + M_0^2}$. The ‘inward pointing’ condition iii) is also satisfied, we prove the validity of the following property: for every pair $(\tau, x) \in [\bar{t}, T] \times \mathbb{R}^n$ and every $\ell \le U^*(\tau, x)$

$$\min_{w \in \Gamma_j(\tau, x, \ell)} (-\xi^0, -\xi^1, \lambda) \cdot w \le 0, \text{ for all } (-\xi^0, -\xi^1, \lambda) \in N_{\text{hyp}}^P(U^*, (\tau, x), \ell). \quad (5.6)$$

Let $(\tau, x) \in [\bar{t}, T] \times \mathbb{R}^n$ and $(-\xi^0, -\xi^1, \lambda) \in N_{\text{hyp}}^P((\tau, x), U^*(\tau, x))$ (we recall that we can always reduce to the case $\ell = U^*(\tau, x)$). We notice that $v^+(\tau, x) \in F_j(\tau, x) \cap F(\tau^+, x)$ and, bearing in mind (2.1) and that $U$ satisfies (2.6), we obtain:

$$-\xi^0 - \xi^1 \cdot v^+(\tau, x) - \lambda L(\tau^+, x, v^+(\tau, x)) \le 0.$$

So (5.6) is confirmed since $(1, v^+(\tau, x), -L(\tau^+, x, v^+(\tau, x))) \in \Gamma_j(\tau, x, U^*(\tau, x))$.

As a consequence, the Weak Invariance Theorem 3.4 is applicable to the differential inclusion (5.5), and we can conclude that the arc $(\tau_j, x_j, \ell_j)$ is a solution to the constrained differential inclusion (5.5). It follows that at $t = T$:

$$U^*(\bar{t}, \bar{x}_j) - \int_{\bar{t}}^{T} L(s, x_j(s), \dot{x}_j(s))ds \le U^*(T, x_j(T)) = g^*(x_j(T)), \text{ for every } j. \quad (5.7)$$

Since $\|x_j - x\|_{W^{1,1}([\bar{t}, T], \mathbb{R}^n)} \xrightarrow{j \to +\infty} 0$, by Lebesgue's dominated convergence theorem we have:

$$\int_{\bar{t}}^{T} L(s, x_j(s), \dot{x}_j(s))ds \xrightarrow{j \to +\infty} \int_{\bar{t}}^{T} L(s, x(s), \dot{x}(s))ds.$$

Since $U \le U^*$, and $U$ is lower semicontinuous, passing to the limit inferior in (5.7) yields:

$$U(\bar{t}, \bar{x}) \le \liminf_{j \to +\infty} g^*(\xi_j) + \int_{\bar{t}}^{T} L(s, x(s), \dot{x}(s))ds.$$

Recalling that $\lim_{j \to +\infty} g^*(\xi_j) = (g^*)_*(x(T))$ and $(g^*)_* = g$, we obtain:

$$U(\bar{t}, \bar{x}) \le g(x(T)) + \int_{\bar{t}}^{T} L(s, x(s), \dot{x}(s))ds,$$

which implies

$$U(\bar{t}, \bar{x}) \le V(\bar{t}, \bar{x}),$$

and concludes the proof. $\square$
*Proof of Proposition 2.3.* The proof immediately follows from the proof of Theorem 2.2, observing that condition $(g^*)_* = g$ is used only in Step 3. □

*Proof of Proposition 2.4.* The proof of Proposition 2.4 follows along the same lines as the proof of Theorem 2.1 and Theorem 2.2, replacing $\tilde{L}$ with $L$ in the definition of the multivalued function $Q$ (steps ‘(c) ⇒ (a)’, ‘(d) ⇒ (a)’, and proof of Thm. 4.5 i)), and taking into account that, when $L$ is lower semicontinuous with respect to the time variable, we have $L(t, x, v) \le L(t^+, x, v)$, for all $(t, x, v) \in [S, T[ \times \mathbb{R}^n \times \mathbb{R}^n$ (steps ‘(a) ⇒ (b)’, ‘(b) ⇒ (c)’ and ‘(a) ⇒ (d)’). □

## REFERENCES

[1] J.-P. Aubin and H. Frankowska, Set-valued analysis. Springer Science & Business Media (2009).

[2] M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of the Hamilton-Jacobi Equations. Birkhäuser, Boston (1997).

[3] J. Bernis, *Chimie Organique et Supramoléculaire*. Ph.D. thesis, Université de Bretagne Occidentale (2007).

[4] P. Bettiol and R.B. Vinter, The Hamilton Jacobi Equation For Optimal Control Problems with Discontinuous Time Dependence. *SIAM J. Control Optim.* **55** (2017) 1199–1225.

[5] G. Cannarsa and C. Sinestrari, Semiconcave functions, Hamilton-Jacobi equations, and optimal control. Springer Science & Business Media (2004).

[6] L. Cesari, Optimization-theory and applications: problems with ordinary differential equations, vol. 17. Springer Science & Business Media (2012).

[7] F. H. Clarke, Functional analysis, calculus of variations and optimal control. Vol. 264 of *Graduate Texts in Mathematics*. Springer Science & Business Media (2013).

[8] F.H. Clarke, Y.S. Ledyaev, R.J. Stern and P.R. Wolenski, Qualitative properties of trajectories of control systems: a survey. *J. Dynam. Control Syst.* **1** (1995) 1–48.

[9] F.H. Clarke, Y.S. Ledyaev, R.J. Stern and P.R. Wolenski, Nonsmooth Analysis and Control Theory. Vol. 178 of *Graduate Texts in Mathematics*. Springer Verlag, New York (1998).

[10] G. Dal Maso and H. Frankowska, Value functions for Bolza problems with discontinuous Lagrangians and Hamilton-Jacobi inequalities. *ESAIM: COCV* **5** (2000) 369–393.

[11] G. Dal Maso, H. Frankowska, et al., Autonomous integral functionals with discontinuous nonconvex integrands: Lipschitz regularity of minimizers, Du Bois-Reymond necessary conditions, and Hamilton-Jacobi equations. *Appl. Math. Optim.* **48** (2003) 39–66.

[12] H. Frankowska, Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations. *SIAM J. Control Optim.* **31** (1993) 257–272.

[13] H. Frankowska and M. Mazzola, Discontinuous solutions of Hamilton–Jacobi–Bellman equation under state constraints. *Calc. Var. Partial Differ. Equ.* **46** (2013) 725–747.

[14] H. Frankowska, S. Plaskacz and T. Rzezuchowski, Measurable viability theorems and the Hamilton-Jacobi-Bellman equation. *J. Differ. Equ.* **116** (1995) 265–305.

[15] H. Frankowska and R.B. Vinter, Existence of neighbouring feasible trajectories: applications to dynamic programming for state constrained optimal control problems. *J. Optim. Theory Appl.* **104** (2000) 21–40.

[16] A.D. Ioffe, Calculus of Dini subdifferentials of functions and contingent coderivatives of set valued maps. *Nonlinear Anal. Theory Methods Appl.* **8** (1984) 517–539.

[17] C. Olech, Weak lower semicontinuity of integral functionals. *J. Optim. Theory Appl.* **19** (1976) 3–16.

[18] S. Plaskacz, M. Quincampox, et al., On representation formulas for Hamilton Jacobi’s equations related to calculus of variations problems. *Topol. Methods Nonlinear Anal.* **20** (2002) 85–118.

[19] R.T. Rockafellar, Convex analysis. Princeton University Press (2015).

[20] R. Vinter, Optimal control. Springer Science & Business Media (2010).

[21] R. Vinter and P. Wolenski, Hamilton-Jacobi theory for optimal control problems with data measurable in time. *SIAM J. Control Optim.* **28** (1990) 1404–1419.