<figure><img src="image_1.png" /></figure>

HAL
open science

# First order Mean Field Games on networks

Yves Achdou, Paola Mannucci, Claudio Marchi, Nicoletta Tchou

► To cite this version:

Yves Achdou, Paola Mannucci, Claudio Marchi, Nicoletta Tchou. First order Mean Field Games on networks. 2023. hal-03729443v3

HAL Id: hal-03729443

https://hal.science/hal-03729443v3

Preprint submitted on 20 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# First order Mean Field Games on networks

YVES ACHDOU,\* PAOLA MANNUCCI,† CLAUDIO MARCHI,‡ NICOLETTA TCHOU §

November 20, 2023

## Abstract

This paper is devoted to finite horizon deterministic mean field games in which the state space is a network. The agents control their velocity, and when they occupy a vertex, they can enter into any incident edge. The running and terminal costs are assumed to be continuous in each edge but not necessarily globally continuous on the network. A Lagrangian formulation is proposed and studied. It leads to relaxed equilibria consisting of probability measures on admissible trajectories. The existence of such relaxed equilibria is obtained. The proof requires the existence of optimal trajectories and a closed graph property for the map which associates to each point the set of optimal trajectories starting from that point.

To any relaxed equilibrium corresponds a mild solution of the mean field game, i.e. a pair $(u, m)$ made of the value function $u$ of a related optimal control problem, and a family $m = (m(t))_t$ of probability measures on the network. Given $m$, the value function $u$ is characterized by a Hamilton-Jacobi problem on the network. Regularity properties of $u$ and a weak form of a Fokker-Planck equation satisfied by $m$ are investigated.

**Keywords:** deterministic mean field games, networks, Lagrangian formulation, first order Hamilton-Jacobi equations on networks.

**2010 AMS Subject classification:** 35F50, 35Q91, 35R02, 49K20, 49L25, 49N80, 91A16.

## Contents

1 Introduction 2

1.1 Notations 4

2 Deterministic optimal control on networks 4

2.1 Existence of optimal trajectories 9

2.2 First properties 11

2.3 Euler-Lagrange conditions 15

2.4 Lipschitz regularity of optimal trajectories 17

2.5 Closed graph property 26

2.6 Lipschitz continuity of the value function 29

2.7 Local semi-concavity of the value function away from the vertex 35

<footnote>\*Université de Paris Cité and Sorbonne Université, CNRS, Laboratoire Jacques-Louis Lions, (LJLL), F-75006 Paris, France, achdou@ljll-univ-paris-diderot.fr</footnote>

<footnote>†Dipartimento di Matematica “Tullio Levi-Civita”, Università di Padova, mannucci@math.unipd.it</footnote>

<footnote>‡Dipartimento di Matematica “Tullio Levi-Civita”, Università di Padova, claudio.marchi@unipd.it</footnote>

<footnote>§Univ Rennes, CNRS, IRMAR - UMR 6625, F-35000 Rennes, France, nicoletta.tchou@univ-rennes1.fr</footnote>
<table>
  <tbody>
    <tr>
      <td>2.8</td>
      <td>Regularity of u along optimal trajectories and optimal synthesis</td>
      <td>36</td>
    </tr>
    <tr>
      <td>2.9</td>
      <td>The PDE satisfied by u on G × [0, T)</td>
      <td>39</td>
    </tr>
    <tr>
      <td>2.9.1</td>
      <td>Relaxed controls</td>
      <td>39</td>
    </tr>
    <tr>
      <td>2.9.2</td>
      <td>Hamiltonians</td>
      <td>39</td>
    </tr>
    <tr>
      <td>2.9.3</td>
      <td>Hamilton-Jacobi equations on G × [0, T)</td>
      <td>39</td>
    </tr>
    <tr>
      <td>2.9.4</td>
      <td>Viscosity solution of (2.80)</td>
      <td>40</td>
    </tr>
    <tr>
      <td>2.9.5</td>
      <td>Main result</td>
      <td>40</td>
    </tr>
    <tr>
      <td><b>3</b></td>
      <td><b>Relaxed Mean Field Games equilibria</b></td>
      <td><b>44</b></td>
    </tr>
    <tr>
      <td>3.1</td>
      <td>Setting and notations</td>
      <td>44</td>
    </tr>
    <tr>
      <td>3.2</td>
      <td>Relaxed MFG equilibrium</td>
      <td>49</td>
    </tr>
    <tr>
      <td>3.3</td>
      <td>Preliminary results</td>
      <td>49</td>
    </tr>
    <tr>
      <td>3.4</td>
      <td>A closed graph property</td>
      <td>50</td>
    </tr>
    <tr>
      <td>3.5</td>
      <td>Proof of Theorem 3.5</td>
      <td>53</td>
    </tr>
    <tr>
      <td>3.6</td>
      <td>Proof of Theorem 3.6</td>
      <td>55</td>
    </tr>
    <tr>
      <td>3.7</td>
      <td>Mild solutions</td>
      <td>55</td>
    </tr>
    <tr>
      <td>3.8</td>
      <td>Regularity of u in the interior of the edges</td>
      <td>58</td>
    </tr>
    <tr>
      <td>3.9</td>
      <td>Properties of m</td>
      <td>59</td>
    </tr>
    <tr>
      <td>3.10</td>
      <td>The continuity equation</td>
      <td>63</td>
    </tr>
  </tbody>
</table>

# 1 Introduction

The theory of Mean Field Games (MFGs in short) introduced in the pioneering articles of Lasry and Lions [25, 26, 27], deals with the asymptotic behaviour of differential games, either deterministic or stochastic, as the number of players tends to infinity. The major part of the literature on deterministic MFGs addresses situations in which the state space is either $\mathbb{R}^d$ or the flat torus $\mathbb{R}^d/\mathbb{Z}^d$, and in which the dynamics of the players is strongly controllable. In such cases, the mean field game is determined by the pair made of the distribution of states at all times and the optimal value of a representative agent. The latter quantities satisfy a system of PDEs coupling a continuity equation (forward in time) and a Hamilton-Jacobi (HJ) equation (backward in time), see [20].

Assuming that the dynamics are strongly controllable, Cannarsa et al, [16, 17, 18], have studied MFGs in which the agents are constrained to remain in the closure of a regular bounded open domain of $\mathbb{R}^d$. With such state constraints, the distribution of states may become singular, as it was first observed in [1], and it becomes difficult to write boundary conditions for the continuity equation (see also Section 3 below for some examples of formation and propagation of Dirac masses). For this reason, Cannarsa et al, following ideas contained in [13, 14, 21], introduce a notion of relaxed equilibrium which is defined in a Lagrangian setting rather than with PDEs. The evolution of the game is described in terms of probability measures defined on a set of admissible trajectories, instead of time-dependent probability measures defined on the state space. In the same vein, Mazanti and Santambrogio, [30], obtain the existence of relaxed equilibria for minimal time MFGs, in which each agent aims at exiting a given closed subset of a general compact metric space in minimal time and faces congestion effects (her speed cannot exceed a bound depending on the density of players). See also [22] for similar models in the Euclidean setting. In [5], the authors of the present paper prove the existence of relaxed equilibria for deterministic state constrained MFGs in which the agents control their acceleration. This is an example of state constrained MFGs in which the strong controllability property does not hold.
The present paper aims at studying relaxed equilibria for deterministic MFGs in which the state space is a network, i.e. a subset of $\mathbb{R}^d$ made of a finite number of edges and vertices. Optimal control problem on junctions, networks or stratified sets is a rather recent field which contains a number of interesting open problems (see [4, 24, 23, 12, 28, 29, 32, 11]). The aforementioned paper [30] on minimal time MFGs also applies to networks. Stochastic MFGs on networks (each agent is subject to an independent noise) have been studied in [2] (see also [21, 3] for infinite horizon problems). Finally, in the recent preprint [7], Gomes et al study a class of stationary MFG on networks and their relationship with Wardrop equilibria. The present paper can be considered as a first step of a more general research project on deterministic MFGs on networks that we intend to pursue.

For simplicity, we hereafter focus on a junction, i.e. $N$ half-lines in $\mathbb{R}^d$ glued together at a single vertex, say the origin. Yet, all the results below may be generalized for general networks with more than one vertices and edges of possibly finite lengths. Given the time evolution of the distribution of the players, each agent solves an optimal problem with finite time horizon. We assume that the agents control their velocity. In particular, when an agent is at the vertex, she can choose either to remain still or to enter any edge. The running and terminal costs depend on the distribution of agents in a non local, regularizing manner, but are not supposed to be continuous across the vertex (the costs may change from one edge to the next). We also restrict ourselves to running costs which depend quadratically on the velocity. Finally, there is a distinct running cost for staying at the vertex.

The first part of the present paper is devoted to optimal control problems on the network, (which arise if the distribution of states in the MFG is given). The main results concerning optimal control are as follows: the existence of an optimal trajectory for any initial state, a closed graph property for the map which associates to each point on the network the set of optimal trajectories starting from that point, Euler-Lagrange conditions for the optimal control, the characterization of the value function of the optimal control problem as the generalized viscosity solution of an Hamilton-Jacobi problem posed on the network with suitable conditions at the vertex (the definition of generalized viscosity solution will be recalled), the local or global Lipschitz regularity of the value function. The second part of the paper deals with relaxed equilibria for MFGs on the network. The existence of the latter is proved using Kakutani's fixed point theorem applied to a suitable multivalued map, which requires in particular a closed graph property. To any relaxed equilibrium, it is then possible to associate a family of time-dependent probability measures on the state space $(m(t))_t$ and the value function $u$ of a suitable optimal control problem involving $m$. All the results of the first part of the paper apply to the latter optimal control problem. In particular, some regularity properties of $u$ can be deduced. It is also possible to prove that $m$ solves a continuity equation in a weak sense and to give information on the propagation of its singularities. The pair $(u, m)$ is named a *mild solution* of the MFG, see [16].

This paper is organized as follows. The remaining part of Section 1 contains the description of the geometry and the definition of some notations. Section 2 is devoted to optimal control problems. In particular, we obtain the existence of an optimal trajectory for every starting point, a closed graph property for the map that associates to each point the set of optimal trajectories, and study the value function (mainly, its characterization as the viscosity solution of a HJ problem on the network and some regularity properties). Section 3 concerns deterministic MFGs on the junction. Relying on the results of Section 2, we prove the existence of a relaxed MFG equilibrium and study the related mild solutions.
## 1.1 Notations

Throughout this paper, the notation $C_b$ means continuous and bounded.

**The junction.** We adopt the notations of [6]. In the whole paper, the state space is a junction in $\mathbb{R}^d$ with $N$ ($N > 1$) semi-infinite straight edges, denoted by $(J_i)_{i=1,...,N}$. Let the edge $J_i$ be the closed half-line $\mathbb{R}^+e_i$, and the vectors $e_i$ be two by two distinct unit vectors in $\mathbb{R}^d$. The junction $\mathcal{G}$ is obtained by gluing the half-lines $J_i$ at the origin $O$:

$$ \mathcal{G} = \bigcup_{i=1}^{N} J_i. $$

For a vector $\xi$ aligned with a given $e_i$, we set $\bar{\xi} = \xi \cdot e_i$.

The geodetic distance $d(x, y)$ between two points $x, y$ of $\mathcal{G}$ is

$$ d(x, y) = \begin{cases} |x - y| & \text{if } x, y \text{ belong to the same edge } J_i \\ |x| + |y| & \text{if } x, y \text{ belong to different edges } J_i \text{ and } J_j. \end{cases} $$

If $\varphi$ is a function defined on $J_i$, we will sometimes use the same notation $\varphi$ for the function $\mathbb{R}^+ \ni \bar{x} \mapsto \varphi(\bar{x}e_i)$.

**Gradient of a function.** Let $C^1(\mathcal{G})$ be the set of continuous functions $\varphi \in C(\mathcal{G})$ such that, for every $i = 1, ..., N$, the restriction of $\varphi$ to the edge $J_i$, $\varphi|_{J_i}$ belongs to $C^1(\mathbb{R}^+)$; moreover, for $\varphi \in C^1(\mathcal{G})$, we set

$$ (1.1) \qquad D\varphi(x) = \begin{cases} D\varphi|_{J_i} & \text{if } x \in J_i \setminus \{O\}, \\ (D\varphi|_{J_1}, \dots, D\varphi|_{J_N}) & \text{if } x = O. \end{cases} $$

Observe that $D\varphi(x)$ is 1-dimensional when the point $x$ lies in the interior of a given edge while it is $N$-dimensional when $x$ coincides with the vertex $O$.

In a similar manner, let $C^1(\mathcal{G} \times [0, T])$ be the set of continuous functions $\varphi \in C(\mathcal{G} \times [0, T])$ such that for any $1 \le i \le N$, the restriction $\varphi|_{J_i \times [0, T]}$ belongs to $C^1(J_i \times [0, T])$.

# 2 Deterministic optimal control on networks

We consider optimal control problems on $\mathcal{G}$ with horizon $T > 0$ and different running costs in the edges and at the vertex. The set of controls, the dynamics and the running cost associated to a given edge $J_i$ are respectively denoted by $A_i$, $\tilde{F}_i$ and $\tilde{\ell}_i$. For the sake of simplicity, we shall focus on the case where $\tilde{f}_i = \alpha$, i.e. the agent directly chooses its velocity, and where the running cost is $\tilde{\ell}_i(x, t, \alpha) = \ell_i(x, t) + |\alpha|^2/2$ (it depends separately on the control and on the state variable). However, what follows may be easily extended to a more general setting, namely

* a network instead of a simple junction

* functions $\tilde{f}_i$ with a linear or sublinear growth at infinity and such that $\tilde{f}_i(A_i)$ contains a neighborhood of 0 (strong controllability assumption)

* running costs which depend separately on the control and the state variable and are strongly convex in the control.
More precisely, we make the following assumptions:

[H0] In order to avoid confusion between the control sets, we set $A_i = \{i\} \times \mathbb{R}$ for $i = 0, \dots, N$. Hence, the sets $A_i$ are disjoint. We set $A = \cup_{i=0}^N A_i$. For $a = (i, \bar{a}) \in A$, we set $|a| = |\bar{a}|$ and, with an abuse of notations, we shall write indifferently $\bar{a}e_i$ and $(i, \bar{a})$.

Let $f_i : J_i \times A_i \to \mathbb{R}$ be defined by $f_i(x, a) = \bar{a}$ for $a = (i, \bar{a})$. We will use the notation $F_i(x)$ for the set $\{f_i(x, a)e_i, a \in A_i\} = \mathbb{R}e_i$ for $x \in J_i$ ($i = 1, \dots, N$). We also set $F_0(O) = \{0_{\mathbb{R}^d}\}$.

[H1] For $i = 1, \dots, N$, the running costs $\ell_i$ belong to $C_b(J_i \times [0, T])$. Let us also introduce a specific cost for staying at the origin, namely $\ell_* : [0, T] \to \mathbb{R}$, continuous and bounded.

For $i = 1, \dots, N$, the terminal costs $g_i$ belong to $C_b(J_i)$. Let $g_*$ be a fixed number.

In the remaining part of Section 2, we will always assume that the costs satisfy the minimal hypotheses made in this paragraph and will not repeat them. We will specify when additional hypotheses are needed.

Let us now recall a general version of Filippov implicit function lemma, which will be useful to prove Theorem 2.2 below. For the proof, we refer the reader to [31].

**Theorem 2.1.** Let $I \subset \mathbb{R}$ be an interval and $\gamma : I \to \mathbb{R}^d \times \mathbb{R}^d$ be a measurable function. Let $A$ be a metric space. Let $K$ be a closed subset of $\mathbb{R}^d \times A$ and $\Psi : K \to \mathbb{R}^d \times \mathbb{R}^d$ be continuous. Assume that $\gamma(I) \subset \Psi(K)$, then there is a measurable function $\Phi : I \to K$ such that

$$ \Psi \circ \Phi(t) = \gamma(t) \quad \text{for a.a. } t \in I. $$

Let us introduce the set

$$ (2.1) \qquad M = \{(x, a) : x \in \mathcal{G}; a \in A_i \text{ if } x \in J_i \setminus \{O\}, \text{ and } a \in A \text{ if } x = O\}. $$

Note that $M$ is closed. Moreover, since the sets $A_i$ are disjoint, for each $(x, a) \in M$, there exist a unique $i \in \{1, \dots, N\}$ and a unique $\bar{a} \in \mathbb{R}$ such that $(x, a) = (x, (i, \bar{a}))$. Let the function $f$ be defined on $M$ by

$$ f(x, a) = \begin{cases} f_i(x, a)e_i, & \text{if } x \in J_i \setminus \{O\}, \\ f_i(O, a)e_i, & \text{if } x = O \text{ and } a \in A_i, i \neq 0, \\ 0_{\mathbb{R}^d}, & \text{if } x = O \text{ and } a \in A_0, \end{cases} $$

for $(x, a) \in M$. Since the sets $A_i$ are disjoint, $f$ is continuous on $M$. Let $\tilde{F}(x)$ be defined by

$$ \tilde{F}(x) = \begin{cases} F_i(x) & \text{if } x \in J_i \setminus \{O\}, \\ \bigcup_{i=0}^N F_i(O) & \text{if } x = O. \end{cases} $$

For $x \in \mathcal{G}$, let the set of admissible paths starting from $x$ be

$$ (2.2) \qquad Y_{x,0} = \left\{ y_x \in W^{1,2}([0, T]; \mathcal{G}) : \begin{array}{l} \dot{y}_x(t) \in \tilde{F}(y_x(t)), \\ y_x(0) = x. \end{array} \text{ for a.e. } t \in [0, T], \right\}. $$

**Theorem 2.2.** If [H0] and [H1] hold, then

1. For any $x \in \mathcal{G}$, $Y_{x,0}$ is nonempty
2. For any $x \in \mathcal{G}$, for any $y_x \in Y_{x,0}$, there exists a measurable function $\Phi : [0, T] \to M$, $\Phi = (\phi_1, \phi_2)$ such that

$$ \phi_2 = (i, \bar{\phi}_2), \text{ with } \bar{\phi}_2 \in \mathbb{R}, \text{ when } \phi_1 \in J_i \setminus \{O\} \\ (y_x(s), \dot{y}_x(s)) = (\phi_1(s), f(\phi_1(s), \phi_2(s))), \quad \text{for a.e. } s, $$

which means in particular that $y_x$ is a continuous representation of $\phi_1$

3. Almost everywhere in $[0, T]$,

$$ \dot{y}_x(s) = \sum_{i=1}^{N} 1_{\{y_x(s) \in J_i \setminus \{O\}}}(s) \bar{\phi}_2(s) e_i $$

4. Almost everywhere on $\{s : y_x(s) = O\}$, $f(O, \phi_2(s)) = 0$.

*Proof.* The proof of point 1 is easy, because $0 \in \tilde{F}(x)$ for every $x \in \mathcal{G}$.
The proof of point 2 is a consequence of Theorem 2.1, with $K = M$, $I = [0, T]$, $\gamma(s) = (y_x(s), \dot{y}_x(s))$ and $\Psi(x, a) = (x, f(x, a))$.
Point 2 implies

$$ \dot{y}_x(s) = \sum_{i=1}^{N} 1_{\{y_x(s) \in J_i \setminus \{O\}\}}(s) \bar{\phi}_2(s) e_i + 1_{\{y_x(s)=O\}}(s) f(O, \phi_2(s)), $$

and from Stampacchia's theorem, $f(O, \phi_2(s)) = 0$ almost everywhere in $\{s : y_x(s) = O\}$.
This yields points 3 and 4. $\square$

**Remark 2.3.** It is worth noticing that in Theorem 2.2, a solution $y_x$ can be associated with several control laws $\phi_2$ which may be different even on sets with positive measure. Actually, for a.e. $s \in \{s \in [0, T] | y_x(s) \in J_i \setminus \{O\}\}$, the control $\phi_2(s)$ is uniquely defined as $\phi_2(s) = \dot{y}_x(s)$ and belongs to $\mathbb{R}e_i$ (for $i = 1, \dots, N$). On the other hand, for a.e. $s \in \{s \in [0, T] | y_x(s) = O\}$, the control $\phi_2(s)$ is 0 by Stampacchia theorem, and it can be arbitrarily chosen in any $A_i$, for $i = 0, \dots, N$.

For any $x \in \mathcal{G}$ and $t_1, t_2 \in [0, T]$ with $t_1 < t_2$, consider the set of admissible trajectories (namely, pairs made of controls and paths) on the interval $[t_1, t_2]$ which start from $x$ at $t_1$:
(2.3)

$$ \Gamma_{t_1, t_2}[x] = \left\{ (y_x, \alpha) \in L^2([t_1, t_2], M) : \begin{array}{l} y_x \in W^{1,2}([t_1, t_2]; \mathcal{G}), \\ y_x(s) = x + \int_{t_1}^{s} f(y_x(\tau), \alpha(\tau))d\tau \quad \text{in } [t_1, t_2] \end{array} \right\}. $$

For simplicity, when $t_2 = T$, we write $\Gamma_{t_1}[x]$ instead of $\Gamma_{t_1,T}[x]$ and, when $t_2 = T$ and $t_1 = 0$, we drop the subscript: $\Gamma[x] = \Gamma_{0,T}[x]$.

Finally, the set of all admissible trajectories starting at time $t = 0$ is defined as follows:

$$ (2.4) \qquad \Gamma = \bigcup_{x \in \mathcal{G}} \Gamma[x]. $$
**Remark 2.4** (concatenation of two admissible trajectories). For $0 \le t_1 \le t_2 \le t_3 \le T$ and $x \in \mathcal{G}$, if $(y_1, \alpha_1) \in \Gamma_{t_1,t_2}[x]$ and $(y_2, \alpha_2) \in \Gamma_{t_2,t_3}[y_1(t_2)]$, the trajectory $(\tilde{y}, \tilde{\alpha})$ defined by

$$ \tilde{y}(s) = \begin{cases} y_1(s) & \text{for } s \in [t_1, t_2] \\ y_2(s) & \text{for } s \in [t_2, t_3] \end{cases} \quad \text{and} \quad \tilde{\alpha}(s) = \begin{cases} \alpha_1(s) & \text{for } s \in [t_1, t_2] \\ \alpha_2(s) & \text{for } s \in [t_2, t_3] \end{cases} $$

belongs to $\Gamma_{t_1,t_3}[x]$.

**The cost functional.** For $t \in [0, T]$, the cost associated to the trajectory $(y_x, \alpha) \in \Gamma_t[x]$ is

$$ (2.5) \quad J_t(x; (y_x, \alpha)) = \int_t^T \left[ \sum_{i=1}^N \ell_i(y_x(\tau), \tau) \mathbb{1}_{y_x(\tau) \in J_i \setminus \{O\}} + \ell_O(\tau) \mathbb{1}_{y_x(\tau)=O} \right] d\tau + \int_t^T \frac{|\alpha(\tau)|^2}{2} d\tau + g(y_x(T)) $$

where

$$ (2.6) \quad \ell_O(\tau) = \min\{\ell_*(\tau), \min_{i=1,\dots,N} \ell_i(O, \tau)\} $$

$$ (2.7) \quad g(y) = \sum_{i=1}^{N} g_i(y) \mathbb{1}_{y \in J_i \setminus \{O\}} + \min\{g_*, \min_{i=1,\dots,N} g_i(O)\} \mathbb{1}_{y=O}, $$

recalling that $g_*$ and $\ell_*$ are introduced in assumption (H$_1$). For brevity, defining

$$ (2.8) \quad L(x,t) = \sum_{i=1}^{N} \ell_i(x,t) \mathbb{1}_{x \in J_i \setminus \{O\}} + \ell_O(t) \mathbb{1}_{x=O} \quad \forall (x,t) \in \mathcal{G} \times [0,T], $$

enables one to write

$$ J_t(x; (y_x, \alpha)) = \int_t^T \left( L(y_x(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau + g(y_x(T)). $$

**Remark 2.5.** The arguments below would also apply for costs of the form

$$ J_t(x; (y_x, \alpha)) = \int_t^T \left[ \sum_{i=1}^N \ell_i(y_x(\tau), \tau) \mathbb{1}_{y_x(\tau) \in J_i \setminus \{O\}} + \sum_{i=0}^N \ell_i(O, \tau) \mathbb{1}_{y_x(\tau)=O, \alpha(\tau) \in A_i} \right] d\tau + \int_t^T \frac{|\alpha(\tau)|^2}{2} d\tau + g(y_x(T)), $$

where we have set $\ell_0(O, \tau) = \ell_*(\tau)$.

**The value function.** The value function of the optimal control problem is

$$ (2.9) \quad u(x,t) = \inf_{(y,\alpha) \in \Gamma_t[x]} J_t(x;(y,\alpha)). $$

Set

$$ (2.10) \quad \Gamma_t^{\text{opt}}[x] = \{(y, \alpha) \in \Gamma_t[x] : J_t(x; (y, \alpha)) = \min_{(\hat{y}, \hat{\alpha}) \in \Gamma_t[x]} J_t(x; (\hat{y}, \hat{\alpha}))\}. $$

For simplicity, we drop the subscript when $t = 0$: $\Gamma_t^{\text{opt}}[x] = \Gamma_t^{\text{opt}}[x]$.
**Remark 2.6.** The value function $u$ is bounded. Indeed, the trajectory associated to the control $\alpha \equiv 0$ is admissible and provides an upper bound for the value function, because the costs $\ell_i$ and $g$ are bounded. From this, it stems that the optimal controls, if they exist, are uniformly bounded in $L^2(0, T)$.

**Remark 2.7** (restriction of optimal trajectories). For $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$ and $\bar{t} \in [t, T]$, $(y_{|[\bar{t},T]}, \alpha_{|[\bar{t},T]}) \in \Gamma_t^{\text{opt}}[y(\bar{t})]$. Indeed, assume by contradiction that there exists a trajectory $(\bar{y}, \bar{\alpha}) \in \Gamma_t^{\text{opt}}[y(\bar{t})]$ such that $J_{\bar{t}}(y(\bar{t})); (\bar{y}, \bar{\alpha})) < J_{\bar{t}}(y(\bar{t})); (y_{|[\bar{t},T]}, \alpha_{|[\bar{t},T]}))$. Then, by Remark 2.4, the concatenation $(\tilde{y}, \tilde{\alpha})$ of $(y, \alpha)$ with $(\bar{y}, \bar{\alpha})$, defined by

$$ \tilde{y}(s) = \begin{cases} y(s) & \text{for } s \in [t, \bar{t}] \\ \bar{y}(s) & \text{for } s \in [\bar{t}, T] \end{cases}, \quad \text{and} \quad \tilde{\alpha}(s) = \begin{cases} \alpha(s) & \text{for } s \in [t, \bar{t}] \\ \bar{\alpha}(s) & \text{for } s \in [\bar{t}, T] \end{cases} $$

belongs to $\Gamma_t[x]$ and consequently there holds

$$ \begin{aligned} u(x,t) &= J_t(x; (y, \alpha)) = \int_t^{\bar{t}} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau + J_{\bar{t}}(y(\bar{t})); (y_{|[\bar{t},T]}, \alpha_{|[\bar{t},T]}) \\ &> \int_t^{\bar{t}} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau + J_{\bar{t}}(y(\bar{t})); (\bar{y}, \bar{\alpha})) = J_t(x; (\tilde{y}, \tilde{\alpha})), \end{aligned} $$

which contradicts the optimality of $(y, \alpha)$.

**Remark 2.8.** From Remark 2.7, we deduce that for any $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$, there holds

$$ u(x,t) = u(y(\bar{t}), \bar{t}) + \int_t^{\bar{t}} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau \quad \forall \bar{t} \in [t, T]. $$

**Remark 2.9.** The concatenation of two optimal trajectories yields an optimal trajectory. More precisely, for any $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$, $\hat{t} \in (t, T)$ and $(\hat{y}, \hat{\alpha}) \in \Gamma_t^{\text{opt}}[y(\hat{t})]$, the concatenation $(y_0, \alpha_0)$ of $(y, \alpha)$ and $(\hat{y}, \hat{\alpha})$ belongs to $\Gamma_t^{\text{opt}}[x]$. Indeed, from Remark 2.8,

$$ \begin{aligned} u(x,t) &= u(y(\hat{t}), \hat{t}) + \int_t^{\hat{t}} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau \\ &= \int_{\hat{t}}^T \left( L(\hat{y}(\tau), \tau) + \frac{|\hat{\alpha}(\tau)|^2}{2} \right) d\tau + g(\hat{y}(T)) + \int_t^{\hat{t}} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau \\ &= J_t(x; (y_0, \alpha_0)), \end{aligned} $$

i.e. $(y_0, \alpha_0)$ is optimal for $u(x, t)$.

**Lemma 2.10.** If [H0] and [H1] hold, then for any $x \in \mathcal{G}$: $\lim_{t \to T^{-}} u(x, t) = g(x)$.

*Proof.* Fix $x \in \mathcal{G}$. Since $y$ corresponding to control $\alpha = 0$ is admissible,

$$ u(x,t) \le \int_t^T L(y(\tau), \tau) d\tau + g(x), $$

and because $L$ is bounded, this implies that $\limsup_{t \to T^{-}} u(x, t) \le g(x)$.

On the other hand, for any $\epsilon \in (0, 1)$, let $(y_t^\epsilon, \alpha_t^\epsilon)$ be an $\epsilon$-optimal trajectory for $u(x, t)$. The same arguments as in Remark 2.6 yield that there exists a constant $C$ (independent of $t$ and of $\epsilon$) such that: $\|\alpha_t^\epsilon\|_{L^2(t,T)} \le C$ so, in particular, $y_t^\epsilon(\cdot)$ is 1/2-Hölder continuous with constant $C$. Hence,

$$ \liminf_{t \to T^{-}} u(x,t) \ge \liminf_{t \to T^{-}} \left( \int_t^T L(y_t^\epsilon(\tau), \tau) d\tau + g(y_t^\epsilon(T)) \right) - \epsilon = g(x) - \epsilon. $$

Letting $\epsilon$ tend to 0 yields the desired result. $\square$
## 2.1 Existence of optimal trajectories

**Proposition 2.11.** For each point $(x, t) \in \mathcal{G} \times [0, T]$, there exists an optimal trajectory, namely there exists $(y_x, \alpha) \in \Gamma_t[x]$ such that $u(x, t) = J_t(x; (y_x, \alpha))$. In other words, $\Gamma^{\text{opt}}[x] \neq \emptyset$.

*Proof.* Fix $(x, t) \in \mathcal{G} \times [0, T]$ and consider a minimizing sequence $(y^n, \alpha^n) \in \Gamma_t[x]$, i.e. $u(x, t) = \lim_{n \to \infty} J_t(x; (y^n, \alpha^n))$. From Remark 2.6, there exists a constant $C$, independent of $n$, such that

$$ (2.11) \qquad \int_t^T \frac{|\alpha^n(\tau)|^2}{2} d\tau \leq C. $$

This implies that $y^n$ are uniformly bounded and uniformly 1/2-Hölder continuous, because

$$ (2.12) \qquad y^n(s) = x + \int_t^s \sum_{i=1}^N 1_{\{y^n(\tau) \in J_i \setminus \{O\}\}} \bar{\alpha}^n(\tau) e_i d\tau. $$

There exist $\alpha \in L^2([t, T]; \mathbb{R}^d)$ and $y_x \in C^{1/2}([t, T]; \mathbb{R}^d)$ such that, possibly up to the extraction of subsequences, $\sum_{i=1}^N 1_{\{y^n(\cdot) \in J_i \setminus \{O\}\}} \bar{\alpha}^n(\cdot) e_i$ converge to $\alpha$ in the weak topology of $L^2([t, T]; \mathbb{R}^d)$ and $y_n$ uniformly converge to $y_x$. In particular, letting $n \to \infty$ in (2.12) yields

$$ (2.13) \qquad y_x \in W^{1,2}([t,T]; \mathbb{R}^d), \quad \text{with} \quad y_x(s) = x + \int_t^s \alpha(\tau) d\tau \in \mathcal{G}, $$

because $\mathcal{G}$ is closed. We now claim that

$$ (2.14) \qquad (y_x, \alpha) \in \Gamma_t[x]. $$

To obtain (2.14), it suffices to prove that $\alpha$ is an admissible control, i.e. that $(y_x(s), \alpha(s)) \in M$ for a.e. $s \in (t, T)$. To this end, let us argue differently whether $y_x(s)$ coincides or not with $O$.

Consider $s \in (t, T)$ such that $y_x(s) \in J_i \setminus \{O\}$ for some $i = 1, \dots, N$. Since the $y^n$ are uniformly 1/2-Hölder continuous and uniformly converge to $y_x$, we deduce that, for $\varepsilon > 0$ sufficiently small and for any $n$ sufficiently large, there holds

$$ y^n(\tau) \in J_i \setminus \{O\} \qquad \forall \tau \in (s - \varepsilon, s + \varepsilon). $$

In particular, for $n$ sufficiently large, $\alpha^n(\tau) = \bar{\alpha}^n(\tau)e_i$ for $\tau \in (s - \varepsilon, s + \varepsilon)$. Letting $n \to \infty$, we conclude that $\alpha(\tau)$ is aligned with $e_i$ for $\tau \in (s - \varepsilon, s + \varepsilon)$. Define the compact set

$$ (2.15) \qquad E = \{s \in (t, T) : y_x(s) = O\}. $$

From (2.13), Stampacchia's theorem yields that $\alpha(s) = 0$ for a.a. $s \in E$.

Hence, we may write for instance $\alpha = 0e_1$ in $E$. The claim (2.14) is proved.

Let us now check that $(y_x, \alpha)$ is an optimal trajectory, i.e. that

$$ (2.16) \qquad u(x,t) = J_t(x; (y_x, \alpha)). $$

In order to prove (2.16), it is useful to decompose $u(x, t)$ as follows

$$ (2.17) \qquad u(x,t) = \lim_{n \to \infty} \left[ \int_t^T \frac{|\alpha^n(\tau)|^2}{2} d\tau + \sum_{i=1}^5 I_i \right], $$
where

$$I_1 = \int_t^T \sum_{i=1}^N \ell_i(y^n(\tau), \tau) \mathbb{1}_{y^n(\tau) \in J_i \setminus \{O\}} \mathbb{1}_{y_x(\tau) \in J_i \setminus \{O\}} d\tau,$$

$$I_2 = \int_t^T \sum_{i=1}^N \ell_i(y^n(\tau), \tau) \mathbb{1}_{y^n(\tau) \in J_i \setminus \{O\}} \mathbb{1}_{y_x(\tau) \in G \setminus J_i} d\tau,$$

$$I_3 = \int_t^T \sum_{i=1}^N \ell_i(y^n(\tau), \tau) \mathbb{1}_{y^n(\tau) \in J_i \setminus \{O\}} \mathbb{1}_{y_x(\tau) = O} d\tau,$$

$$I_4 = \int_t^T \ell_O(\tau) \mathbb{1}_{y^n(\tau) = O} d\tau,$$

$$I_5 = g(y^n(T)),$$

and study separately the different contributions in the right hand side of (2.17). It is well known that the convergence in the weak topology of $L^2([t, T]; \mathbb{R}^d)$ entails

$$(2.18) \qquad \int_t^T \frac{|\alpha(\tau)|^2}{2} d\tau \le \liminf_{n \to \infty} \int_t^T \frac{|\alpha^n(\tau)|^2}{2} d\tau.$$

Concerning $I_1$, the uniform convergence of $y^n$ to $y_x$ as $n \to \infty$ and the continuity of $\ell_i$ ensure that, for any $\tau \in [t, T]$,

$$\ell_i(y^n(\tau), \tau) \mathbb{1}_{y^n(\tau) \in J_i \setminus \{O\}} \mathbb{1}_{y_x(\tau) \in J_i \setminus \{O\}} \to \ell_i(y_x(\tau), \tau) \mathbb{1}_{y_x(\tau) \in J_i \setminus \{O\}} \quad \text{as } n \to \infty.$$

Since the $\ell_i$'s are bounded, the dominated convergence theorem yields

$$(2.19) \qquad I_1 \to \int_t^T \sum_{i=1}^N \ell_i(y_x(\tau), \tau) \mathbb{1}_{y_x(\tau) \in J_i \setminus \{O\}} d\tau \quad \text{as } n \to \infty.$$

As for $I_2$, again the uniform convergence of $y^n$ to $y_x$ and the continuity of $\ell_i$ ensure that the integrand tends to zero as $n \to \infty$. Again the dominated convergence theorem yields

$$(2.20) \qquad I_2 \to 0 \quad \text{as } n \to \infty.$$

Let us now consider the term $I_5$ and argue differently whether $y_x(T)$ coincides or not with $O$. If $y_x(T) \in J_i \setminus \{O\}$ for some $i \in \{1, \dots, N\}$ then, the uniform convergence of $y^n$ to $y_x$ and the continuity of $g_i$ entail $g(y^n(T)) = g_i(y^n(T)) \to g_i(y_x(T)) = g(y_x(T))$ as $n \to \infty$. If $y_x(T) = O$, again by the uniform convergence of $y^n$ to $y_x$ and by the definition of $g$ in (2.7), for any $\varepsilon > 0$, we get $g(y^n(T)) \ge g(O) - \varepsilon = g(y_x(T)) - \varepsilon$ for $n$ sufficiently large. In both cases,

$$(2.21) \qquad \liminf_{n \to \infty} I_5 \ge g(y_x(T)).$$

On the other hand,

$$I_3 + I_4 = \int_t^T \left[ \sum_{i=1}^N \ell_i(y^n(\tau), \tau) \mathbb{1}_{y^n(\tau) \in J_i \setminus \{O\}} + \ell_O(\tau) \mathbb{1}_{y^n(\tau) = O} \right] \mathbb{1}_{y_x(\tau) = O} d\tau \\ + \int_t^T \ell_O(\tau) \mathbb{1}_{y^n(\tau) = O} \mathbb{1}_{y_x(\tau) \ne O} d\tau.$$
Observe that $1_{y^n(\cdot)=O}1_{y_x(\cdot)\neq O} \to 0$ as $n \to \infty$. Hence, from the dominated convergence theorem,

$$ \int_t^T \ell_O(\tau) 1_{y^n(\tau)=O} 1_{y_x(\tau)\neq O} d\tau \to 0 \quad \text{as } n \to \infty. $$

Assume for a while that

$$ (2.22) \quad \liminf_{n \to \infty} \int_t^T \left[ \sum_{i=1}^N \ell_i(y^n(\tau), \tau) 1_{y^n(\tau) \in J_i \setminus \{O\}} + \ell_O(\tau) 1_{y^n(\tau)=O} \right] 1_{y_x(\tau)=O} d\tau \\ \geq \int_t^T \ell_O(\tau) 1_{y_x(\tau)=O} d\tau. $$

From (2.17), (2.18) and (2.22),

$$ u(x,t) \geq \int_t^T \left[ \frac{|\alpha(\tau)|^2}{2} + \sum_{i=1}^N \ell_i(y_x(\tau), \tau) 1_{y_x(\tau) \in J_i \setminus \{O\}} + \ell_O(\tau) 1_{y_x(\tau)=O} \right] d\tau + g(y_x(T)) $$

which is equivalent to (2.16).

There remains to prove (2.22). Recall that the set E has been defined in (2.15). Since $y^n$ uniformly converge to $y_x$, $\|y^n - y_x\|_{L^\infty(E)}$ is arbitrary small for n sufficiently large. Then the continuity of $\ell_i$ implies that for any $\varepsilon > 0$,

$$ \ell_i(y^n(\tau), \tau) > \ell_O(\tau) - \varepsilon, \quad \forall \tau \in E, $$

for n sufficiently large, which implies inequality (2.22). $\square$

**Remark 2.12.** The following statement can be seen as the “converse” of Remark 2.8. If there exist $t_1 \in [0, T]$ and $(y_1, \alpha_1) \in \Gamma_{t,t_1}[x]$ such that

$$ u(x,t) = u(y_1(t_1), t_1) + \int_t^{t_1} \left( L(y_1(\tau), \tau) + \frac{|\alpha_1(\tau)|^2}{2} \right) d\tau, $$

then, there exists $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$ with $(y_1, \alpha_1) = (y, \alpha)$ on $(t, t_1)$. Indeed, consider the concatenation $(y, \alpha)$ of $(y_1, \alpha_1)$ with $(\bar{y}, \bar{\alpha})$ where $(\bar{y}, \bar{\alpha})$ is any trajectory in $\Gamma_{t_1}^{\text{opt}}[y_1(t_1)]$. from Remark 2.4, $(y, \alpha)$ is admissible for $u(x, t)$. Since the above equality can be written

$$ \begin{aligned} u(x,t) &= \int_{t_1}^{T} \left( L(\bar{y}(\tau), \tau) + \frac{|\bar{\alpha}(\tau)|^2}{2} \right) d\tau + g(\bar{y}(T)) + \int_{t}^{t_1} \left( L(y_1(\tau), \tau) + \frac{|\alpha_1(\tau)|^2}{2} \right) d\tau \\ &= J_t(x; (y, \alpha)), \end{aligned} $$

$(y, \alpha)$ is optimal for $u(x, t)$.

## 2.2 First properties

This paragraph is devoted to the dynamic programming principle and the continuity of $u$. Let us stress that the structure of the control set plays a crucial role in what follows.

**Proposition 2.13 (Dynamic programming principle).** Assume [H0] and [H1]. For any $(x, t) \in \mathcal{G} \times [0, T]$ and $\bar{t} \in [t, T]$, there holds

$$ (2.23) \quad u(x,t) = \inf_{(y,\alpha) \in \Gamma_{t,\bar{t}}[x]} \left\{ u(y(\bar{t}), \bar{t}) + \int_{\bar{t}}^{\bar{t}} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau \right\}. $$
Proof. (i). For any $(y, \alpha) \in \Gamma_t[x]$, there holds

$$
\begin{align*}
J_t(x, (y, \alpha)) &= \int_t^{\bar{t}} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau + J_{\bar{t}}(y(\bar{t}), (y_{|[\bar{t},T]}, \alpha_{|[\bar{t},T]})) \\
&\geq \int_t^{\bar{t}} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau + u(y(\bar{t}), \bar{t}) \\
&\geq \inf_{(z, \beta) \in \Gamma_{t, \bar{t}}[x]} \left\{ u(z(\bar{t}), \bar{t}) + \int_t^{\bar{t}} \left( L(z(\tau), \tau) + \frac{|\beta(\tau)|^2}{2} \right) d\tau \right\},
\end{align*}
$$

where $(y_{|[\bar{t},T]}, \alpha_{|[\bar{t},T]})$ is the restriction of the trajectory $(y, \alpha)$ in the interval $[\bar{t}, T]$. Taking the infimum in $(y, \alpha) \in \Gamma_t[x]$ leads to (2.23) with the $\ge$ sign instead of $=$.

Let us now prove the reverse inequality. Consider $(y, \alpha) \in \Gamma_{t, \bar{t}}[x]$. For $(\bar{y}, \bar{\alpha}) \in \Gamma_{\bar{t}}^{\text{opt}}[y(\bar{t})]$ (whose existence is ensured by Proposition 2.11),

$$
u(y(\bar{t}), \bar{t}) + \int_{t}^{\bar{t}} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau = J_{t}(x, (\tilde{y}, \tilde{\alpha})) \geq u(x, t),
$$

where $(\tilde{y}, \tilde{\alpha})$ is the concatenation of the trajectory $(y, \alpha)$ on $[t, \bar{t}]$ and of the trajectory $(\bar{y}, \bar{\alpha})$ on $[\bar{t}, T]$. Recall from Remark 2.4 that $(\tilde{y}, \tilde{\alpha}) \in \Gamma_t[x]$. The proof is completed by taking the infimum in $(y, \alpha) \in \Gamma_{t, \bar{t}}[x]$. $\square$

**Proposition 2.14** (Continuity of the value function). If [H0] and [H1] hold, then the function $u$ is continuous in $\mathcal{G} \times [0, T)$.

Proof. Consider $(x_1, t_1)$, $(x_2, t_2) \in \mathcal{G} \times [0, T_1)$, with $T_1 < T$ and $\delta := d(x_1, x_2) < (T-T_1)/2$. Without any loss of generality, we may assume that both $x_1$ and $x_2$ belong to the same edge, say $e_1$. Consider $(y_2, \alpha_2) \in \Gamma_{t_2}^{\text{opt}}[x_2]$. Consider the trajectory that starts in $x_1$ at time $t_1$ and corresponds to the control

$$
\alpha_1(s) = \begin{cases} \begin{cases} e_1 & \text{if } x_1 \le x_2 \\ -e_1 & \text{if } x_1 > x_2 \end{cases} & \text{for } s \in [t_1, t_1 + \delta], \\ \frac{T-t_2}{T-t_1-\delta} \alpha_2 \left( \frac{T-t_2}{T-t_1-\delta} s - \frac{T(\delta+t_1-t_2)}{T-t_1-\delta} \right) & \text{for } s \in (t_1 + \delta, T], \end{cases}
$$

thus

$$
y_1(s) = \begin{cases} x_1 + \frac{x_2 - x_1}{\delta} (s - t_1) & \text{for } s \in [t_1, t_1 + \delta], \\ x_2 + \int_{t_1 + \delta}^{s} \frac{T-t_2}{T-t_1-\delta} \alpha_2 \left( \frac{T-t_2}{T-t_1-\delta} \tau - \frac{T(\delta+t_1-t_2)}{T-t_1-\delta} \right) d\tau & \text{for } s \in [t_1 + \delta, T]. \end{cases}
$$

Observe that, for $s \in [t_1 + \delta, T]$, there holds

$$
y_1(s) = x_2 + \int_{t_2}^{\frac{T-t_2}{T-t_1-\delta}} \frac{s - \frac{T(\delta+t_1-t_2)}{T-t_1-\delta}}{T-t_1-\delta} \alpha_2(\theta) \, d\theta = y_2 \left( \frac{T-t_2}{T-t_1-\delta} s - \frac{T(\delta+t_1-t_2)}{T-t_1-\delta} \right),
$$

and that $(y_1, \alpha_1) \in \Gamma_{t_1}(x_1)$ with $y_1(T) = y_2(T)$. On the other hand,

$$
\begin{align}
\|\alpha_1\|_{L^2(t_1,T)}^2 &= \delta + \int_{t_1+\delta}^{T} \left( \frac{T-t_2}{T-t_1-\delta} \right)^2 \left| \alpha_2 \left( \frac{T-t_2}{T-t_1-\delta} s - \frac{T(\delta+t_1-t_2)}{T-t_1-\delta} \right) \right|^2 d\tau \nonumber \\
&= \delta + \frac{T-t_2}{T-t_1-\delta} \|\alpha_2\|_{L^2(t_2,T)}^2 \nonumber \\
(2.24) \qquad \qquad \qquad \qquad &= \delta + \|\alpha_2\|_{L^2(t_2,T)}^2 + \frac{t_1-t_2+\delta}{T-t_1-\delta} \|\alpha_2\|_{L^2(t_2,T)}^2. \nonumber
\end{align}
$$
Let us estimate

$$J_{t_1}(x_1; (y_1, \alpha_1)) - J_{t_2}(x_2; (y_2, \alpha_2)) = \sum_{i=1}^{4} I_i,$$

where

$$I_1 = \int_{t_1}^{t_1+\delta} L(y_1(\tau), \tau) d\tau, \quad I_2 = \frac{\|\alpha_1\|_{L^2(t_1,T)}^2 - \|\alpha_2\|_{L^2(t_2,T)}^2}{2},$$

$$I_3 = \int_{t_1+\delta}^{T} L(y_1(\tau), \tau) d\tau, \quad I_4 = -\int_{t_2}^{T} L(y_2(\tau), \tau) d\tau,$$

recalling that $y_1(T) = y_2(T)$. From the boundedness of the running cost and (2.24), there holds $|I_1| \le K\delta$ for some constant $K$, and

$$|I_2| \le \frac{\delta}{2} + \frac{\|\alpha_2\|_{L^2(t_2,T)}^2 |t_1 - t_2| + \delta}{2(T - T_1)}.$$

On the other hand, after a change of variable,

$$I_3 = \int_{t_1+\delta}^{T} L \left( y_2 \left( \frac{T-t_2}{T-t_1-\delta} \tau - \frac{T(\delta+t_1-t_2)}{T-t_1-\delta} \right), \tau \right) d\tau \\ = \frac{T-t_1-\delta}{T-t_2} \int_{t_2}^{T} L \left( y_2(\theta), \frac{T-t_1-\delta}{T-t_2}\theta + \frac{T(\delta+t_1-t_2)}{T-t_2} \right) d\theta,$$

which implies that

$$|I_3 + I_4| = \left| \frac{t_2 - t_1 - \delta}{T - t_2} \int_{t_2}^{T} L \left( y_2(\theta), \frac{T - t_1 - \delta}{T - t_2}\theta + \frac{T(\delta + t_1 - t_2)}{T - t_2} \right) d\theta \right. \\ \left. + \int_{t_2}^{T} \left[ L \left( y_2(\theta), \frac{T - t_1 - \delta}{T - t_2}\theta + \frac{T(\delta + t_1 - t_2)}{T - t_2} \right) - L(y_2(\theta), \theta) \right] d\theta \right| \\ \le \frac{|t_2 - t_1| + \delta}{T - T_1} K + T\omega \left( \frac{|t_2 - t_1| + \delta}{T - T_1} 2T \right),$$

where $\omega$ is a modulus of continuity common to all the costs $\ell_i$ in $B(0, R)$ with $R > |x_1| + |x_2| + \max_{s \in [0, T]} |y_2(s)|$. In conclusion,

$$|J_{t_1}(x_1; (y_1, \alpha_1)) - J_{t_2}(x_2; (y_2, \alpha_2))| \le \tilde{K}(|t_2 - t_1| + \delta) + \tilde{\omega}(|t_2 - t_1| + \delta)$$

for a suitable constant $\tilde{K}$ (depending only on $T_1$) and a suitable modulus of continuity $\tilde{\omega}$ (depending on $T_1$, $|x_1|$ and $|x_2|$). From the optimality of $(y_2, \alpha_2)$,

$$\begin{aligned} u(x_1, t_1) &\le J_{t_1}(x_1; (y_1, \alpha_1)) \le J_{t_2}(x_2; (y_2, \alpha_2)) + \tilde{K}(|t_2 - t_1| + \delta) + \tilde{\omega}(|t_2 - t_1| + \delta) \\ &\le u(x_2, t_2) + \tilde{K}(|t_2 - t_1| + \delta) + \tilde{\omega}(|t_2 - t_1| + \delta). \end{aligned}$$

Reversing the role of $(x_1, t_1)$ and $(x_2, t_2)$, we get

$$|u(x_1, t_1) - u(x_2, t_2)| \le \tilde{K}(|t_2 - t_1| + \delta) + \tilde{\omega}(|t_2 - t_1| + \delta),$$

and the proof is done.

<figure><img src="image_3.png" /></figure>

**Remark 2.15** (Hölder/Lipschitz continuity). If the running costs $\ell_i$ are $\theta$-Hölder continuous with respect to time for $\theta = (0, 1]$, the same arguments as above can be used for proving that the value function is locally $\theta$-Hölder continuous with respect to $(x, t)$ in $G \times [0, T)$.
The following property will not be used in the remaining part of the paper.

**Lemma 2.16.** Fix $(x, t) \in \mathcal{G} \times [0, T)$ and $(y, \alpha) \in \Gamma_t[x]$ and consider a sequence $\{(x_n, t_n)\}_{n \in \mathbb{N}}$, such that $(x_n, t_n) \in \mathcal{G} \times [0, T)$, $\delta'_n = d(x_n, x) + |t_n - t| \to 0$ as $n \to \infty$. There exists a sequence $\{(y_n, \alpha_n)\}_{n \in \mathbb{N}}$, such that $(y_n, \alpha_n) \in \Gamma_{t_n}[x_n]$

$$ (i) \quad \sup_{[t_n \vee t, T]} d(y_n(\cdot), y(\cdot)) \le \delta_n + |t_n - t| + \|\alpha\|_2 \sqrt{\delta'_n}, \quad y_n(T) = y(T) $$

$$ (2.25) \quad (ii) \quad \|\alpha_n\|_2^2 \le \|\alpha\|_2^2 + \delta'_n \left(1 + \frac{\|\alpha\|_2^2}{T - \delta'_n}\right) $$

$$ (iii) \quad \lim_{n \to \infty} J_{t_n}(x_n; (y_n, \alpha_n)) = J_t(x; (y, \alpha)). $$

*Proof of Lemma 2.16.* We adapt the arguments in the proof of Lemma 2.26. It is enough to focus on the situation in which all the points $x_n$ and $x$ belong to the edge $e_1$, and all the $t_n$ are either smaller or larger than $t$. Set $\delta_n = d(x_n, x)$.

Case 1: $t_n \le t$, $\forall n \in \mathbb{N}$. Let us introduce the control

$$ \alpha_n(s) = \begin{cases} 0, & \text{for } s \in [t_n, t], \\ \begin{cases} e_1, & \text{if } \bar{x}_n \le \bar{x}, \\ -e_1, & \text{if } \bar{x}_n > \bar{x}, \end{cases} & \text{for } s \in [t, t + \delta_n], \\ \frac{T-t}{T-t-\delta_n} \alpha \left( s \frac{T-t}{T-t-\delta_n} - \delta_n \frac{T}{T-t-\delta_n} \right), & \text{for } s \in (t + \delta_n, T], \end{cases} $$

and let $y_n$ be the corresponding path starting from $x_n$ at time $t_n$. Clearly, $(y_n, \alpha_n) \in \Gamma_{t_n}[x_n]$, and

$$ y_n(s) = \begin{cases} x_n, & \text{for } s \in [t_n, t], \\ x_n + (x - x_n)\delta_n^{-1}(s - t), & \text{for } s \in [t, t + \delta_n], \\ y \left( s \frac{T-t}{T-t-\delta_n} - \delta_n \frac{T}{T-t-\delta_n} \right), & \text{for } s \in [t + \delta_n, T]. \end{cases} $$

The bounds in (2.25)-(i) and (ii) are obtained with the same arguments as above. Moreover,

$$ J_{t_n}(x_n; (y_n, \alpha_n)) - J_t(x; (y, \alpha)) = \sum_{i=1}^{5} I_i, $$

where, for $i = 1, \dots, 4$, the terms $I_i$ are analogous to the corresponding ones in (2.59), while $I_5 = \int_{t_n}^t L(x_n, 0, \tau) d\tau$. Then $|I_5| \le K|t - t_n|$ for a suitable constant $K$, since the costs $\ell_i$ are bounded functions. The same calculations as in the proof of Lemma 2.26 lead to the desired result.

Case 2: $t_n \ge t$, $\forall n \in \mathbb{N}$. It is clear that $t + \delta'_n = t_n + \delta_n$. Consider the control

$$ \alpha_n(s) = \begin{cases} \begin{cases} e_1, & \text{if } \bar{x}_n \le \bar{x}, \\ -e_1, & \text{if } \bar{x}_n > \bar{x}, \end{cases} & \text{for } s \in [t_n, t + \delta'_n], \\ \frac{T-t}{T-t-\delta'_n} \alpha \left( s \frac{T-t}{T-t-\delta'_n} - \delta'_n \frac{T}{T-t-\delta'_n} \right), & \text{for } s \in (t + \delta'_n, T], \end{cases} $$

and let $y_n$ be the corresponding path starting from $x_n$ at time $t_n$. Then $(y_n, \alpha_n) \in \Gamma_{t_n}[x_n]$ and

$$ y_n(s) = \begin{cases} x_n + (x - x_n)\delta_n^{-1}(s - t), & \text{for } s \in [t, t + \delta'_n], \\ y \left( s \frac{T-t}{T-t-\delta_n} - \delta_n \frac{T}{T-t-\delta_n} \right), & \text{for } s \in [t + \delta'_n, T]. \end{cases} $$

The desired result is obtained with the same calculations as in the proof of Lemma 2.26.

□
## 2.3 Euler-Lagrange conditions

Below, we address situations in which it is possible to write the Euler-Lagrange conditions for an optimal trajectory. They will consist of a family of differential equations along with a condition at the horizon. The following lemma deals with the Euler-Lagrange condition in time intervals $[t_1, t_2] \subset (0, T)$ for which an optimal trajectory lies in the interior of a given edge.

**Lemma 2.17.** Consider $i \in \{1, \dots, N\}$, and assume that the function $\ell_i$ is differentiable with respect to its first argument with $\partial_x \ell_i \in C(J_i \times [0, T])$. Consider any $(x, t) \in \mathcal{G} \times [0, T]$ and any $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$ such that, for some $t_1, t_2 \in (t, T)$, there holds

$$y(s) \in J_i \setminus \{O\}, \quad \forall s \in [t_1, t_2].$$

Then, the control $\alpha$ is $C^1$ in $(t_1, t_2)$ and

$$(2.26) \qquad \alpha'(s) = \partial_x \ell_i(y(s), s), \quad \forall s \in (t_1, t_2).$$

*Proof.* Fix $\tilde{t} \in (t_1, t_2)$ and consider $\alpha_1 \in L^2(t, T)$, with $\alpha_1(s) \in \mathbb{R}e_i$ a.e. in $(t_1, t_2)$, $\alpha_1(s) = 0$ for $s \notin (t_1, t_2)$ and $\int_{t_1}^{t_2} \alpha_1 ds = 0$. In $(t_1, t_2)$, both $\alpha$ and $\alpha_1$ are aligned with $e_i$ and can be written $\alpha(s) = \bar{\alpha}(s)e_i$ and $\alpha_1(s) = \bar{\alpha}_1(s)e_i$ with $\bar{\alpha}(s), \bar{\alpha}_1(s) \in \mathbb{R}$. For $h \in \mathbb{R}$, with $|h|$ sufficiently small, the control $\alpha_h(\cdot) := \alpha(\cdot) + h\alpha_1(\cdot)$ is admissible for $(x, t)$ because $|y|_{t_1, t_2}$ is bounded from below by a positive number. Let $y_h$ denote the trajectory corresponding to the control $\alpha_h$. It is clear that $y_h(T) = y(T)$. Then, since $(y, \alpha)$ is optimal,

$$0 \le \frac{J_t(y_h, \alpha_h) - J_t(y, \alpha)}{h} = \int_{t_1}^{t_2} \left( \bar{\alpha}(s)\bar{\alpha}_1(s) + \frac{h\bar{\alpha}_1(s)^2}{2} + \frac{\ell_i(y_h(s), s) - \ell_i(y(s), s)}{h} \right) ds.$$

Since $y_h(s) = y(s) + h \int_{t_1}^s \alpha_1(\tau)d\tau$ for $s \in [t_1, t_2]$, we deduce from the regularity of $\ell_i$ with respect to the state variable that

$$\int_{t_1}^{t_2} \frac{\ell_i(y_h(s), s) - \ell_i(y(s), s)}{h} ds = \int_{t_1}^{t_2} \partial_x \ell_i(y(s), s) \int_{t_1}^{s} \bar{\alpha}_1(\tau) d\tau ds + o(1),$$

where $o(1)$ is a function of $h$ that tends to 0 as $h \to 0$. Integrating by parts the last integral and observing that $\int_{t_1}^{t_2} \bar{\alpha}_1 ds = 0$ yields

$$\int_{t_1}^{t_2} \frac{\ell_i(y_h(s), s) - \ell_i(y(s), s)}{h} ds = - \int_{t_1}^{t_2} \left( \int_{t_1}^{s} \partial_x \ell_i(y(\theta), \theta) d\theta \right) \bar{\alpha}_1(s) ds.$$

Inserting the latter in (2.27) and letting $h \to 0$ leads to

$$0 \le \int_{t_1}^{t_2} \left[ \bar{\alpha}(s) - \int_{t_1}^{s} \partial_x \ell_i(y(\theta), \theta) d\theta \right] \bar{\alpha}_1(s) ds,$$

for every $\alpha_1$ supported in $[t_1, t_2]$ with $\int_{t_1}^{t_2} \alpha_1 ds = 0$. The linearity of the constraint then implies

$$0 = \int_{t_1}^{t_2} \left[ \bar{\alpha}(s) - \int_{t_1}^{s} \partial_x \ell_i(y(\theta), \theta) d\theta \right] \bar{\alpha}_1(s) ds,$$

i.e. that $s \mapsto \bar{\alpha}(s) - \int_{t_1}^s \partial_x \ell_i(y(\theta), \theta) d\theta$ is orthogonal in $L^2(t_1, t_2)$ to $V = \{f \in L^2(t_1, t_2) : \int_{t_1}^{t_2} f = 0\} = \mathbb{R}^{\perp L^2(t_1, t_2)}$. Hence, this function is constant and (2.26) is proved. $\square$
**Remark 2.18.** A consequence of (2.26) is that $\alpha$ is Lipschitz continuous in each interval $[t_1, t_2] \subset [0, T]$ such that $y(t) \neq O$ for $t \in (t_1, t_2)$.

**Remark 2.19.** If we only suppose that for some $p \in [1, \infty]$, $\ell_i(\cdot, t)$ is bounded in $W_{\text{loc}}^{1,p}(J_i)$ uniformly with respect to $t \in [0, T]$, then $y \in W^{2,p}(t_1, t_2)$ and (2.26) holds for almost all $s \in (t_1, t_2)$.

The following lemma deals with the transversality condition for an optimal trajectory which stays in the interior of a given edge near the horizon $T$.

**Lemma 2.20.** We keep the assumptions of Lemma 2.17 and we also assume that $g_i \in C^1(J_i)$. Consider any $(x, t) \in \mathcal{G} \times [0, T]$ and any $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$ such that $y(T) \in J_i \setminus \{O\}$. Then, there holds

$$ (2.28) \qquad \alpha(T) = -\partial_x g_i(y(T)). $$

*Proof.* The arguments are similar to those in the proof of Lemma 2.17. Since $t \mapsto y(t)$ is continuous with $y(T) \in J_i \setminus \{O\}$, there exists $\delta > 0$ such that $y(s) \in J_i \setminus \{O\}$ for $s \in [T - \delta, T]$. Consider $\alpha_1 \in L^2(t, T)$ with $\alpha_1(s) \in \mathbb{R}e_i$ a.e. in $(T - \delta, T)$, $\alpha_1(s) = 0$ a.e. in $(t, T - \delta)$. In $(T - \delta, T)$, both $\alpha$ and $\alpha_1$ are aligned with $e_i$ and we may write $\alpha(s) = \bar{\alpha}(s)e_i$ and $\alpha_1(s) = \bar{\alpha}_1(s)e_i$. As before, for $h \in \mathbb{R}$ with $|h|$ sufficiently small, the control $\alpha_h(\cdot) := \alpha(\cdot) + h\alpha_1(\cdot)$ is admissible for $(x, t)$. Let $y_h$ be the trajectory corresponding to the control $\alpha_h$. We deduce from the optimality of $(y, \alpha)$ that

$$ 0 \le \int_{T-\delta}^{T} \left( \bar{\alpha}(s)\bar{\alpha}_1(s) + \frac{h\bar{\alpha}_1(s)^2}{2} + \frac{\ell_i(y_h(s), s) - \ell_i(y(s), s)}{h} \right) ds + \frac{g_i(y_h(T)) - g_i(y(T))}{h}. $$

Since $y_h(s) = y(s) + h \int_{T-\delta}^{s} \alpha_1(\tau)d\tau$ for $s \in [T - \delta, T]$, arguing as in the proof of Lemma 2.17 leads to

$$ \frac{g_i(y_h(T)) - g_i(y(T))}{h} = \partial_x g_i(y(T)) \int_{T-\delta}^{T} \bar{\alpha}_1(\tau)d\tau + O(h), $$

and

$$ \int_{T-\delta}^{T} \frac{\ell_i(y_h(s), s) - \ell_i(y(s), s)}{h} ds = \int_{T-\delta}^{T} \partial_x \ell_i(y(s), s) \int_{T-\delta}^{s} \bar{\alpha}_1(\tau) d\tau ds + O(h) \\ = \int_{T-\delta}^{T} \partial_x \ell_i(y(\theta), \theta) d\theta \int_{T-\delta}^{T} \bar{\alpha}_1(\tau) d\tau - \int_{T-\delta}^{T} \left( \int_{T-\delta}^{s} \partial_x \ell_i(y(\theta), \theta) d\theta \right) \bar{\alpha}_1(s) ds, $$

where the last equality is obtained after an integration by parts. Combining the latter three inequalities and letting $h \to 0$ yield

$$ 0 \le \left( \int_{T-\delta}^{T} \partial_x \ell_i(y(\theta), \theta) d\theta + \partial_x g_i(y(T)) \right) \int_{T-\delta}^{T} \bar{\alpha}_1(s) ds + \\ \int_{T-\delta}^{T} \left[ \bar{\alpha}(s) - \int_{T-\delta}^{s} \partial_x \ell_i(y(\theta), \theta) d\theta \right] \bar{\alpha}_1(s) ds. $$

Since $y(s) \in J_i \setminus \{O\}$ for $s \in [T - \delta, T]$, we infer from (2.26) that

$$ 0 \le (\bar{\alpha}(T) + \partial_x g_i(y(T))) \int_{T-\delta}^{T} \bar{\alpha}_1(s) ds. $$

This yields (2.28) since $\alpha_1$ is arbitrary. $\square$
## 2.4 Lipschitz regularity of optimal trajectories

We now aim at proving that for any $(x, t) \in \mathcal{G} \times [0, T]$, any trajectory $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$ is such that $\alpha$ is bounded in $(t, T)$, with a bound that depends locally uniformly on $x$. The essential arguments are the Euler-Lagrange and the transversality conditions obtained in Section 2.3 and a key estimate on the initial velocity of an optimal trajectory, locally independent of the starting point, see Lemma 2.22 below.

**Theorem 2.21.** Assume that for all $i = 1, \dots, N$, $g_i \in C^1(J_i)$ with $\partial_x g_i \in C_b(J_i)$, $\ell_i$ is differentiable with respect to its first argument with $\partial_x \ell_i \in C_b(J_i \times [0, T])$, and let $M_g, M_\ell, L_g$ and $L_\ell$ be defined by

$$ (2.29) \qquad \begin{aligned} M_g &= \|g\|_{L^\infty(\mathcal{G})}, & L_g &= \max_{i=1,\dots,N} \|\partial_x g_i\|_{L^\infty(J_i)}, \\ M_\ell &= \|L\|_{L^\infty(\mathcal{G} \times [t,T])}, & L_\ell &= \max_{i=1,\dots,N} \|\partial_x \ell_i\|_{L^\infty(J_i \times [t,T])}. \end{aligned} $$

For any $(x, t) \in \mathcal{G} \times [0, T]$ and for any trajectory $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$, the control $\alpha$ belongs to $L^\infty(t, T)$. Moreover, there exists a positive constant $V$ (depending only on $L_g, M_\ell, L_\ell, d(x, O)$ and $(T - t)^{-1}$) such that

$$ \|\alpha\|_{\infty} \leq V. $$

*Proof.* Consider a trajectory $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$. Set

$$ (2.30) \qquad V_* = L_g + (T - t)L_\ell. $$

Let us split the interval $[t, T]$ in order to distinguish the times $s$ for which $y(s) \in J_i \setminus \{O\}$, $i = 1, \dots, N$, and $y(s) = O$. More precisely, set

$$ I_0 = \{s \in [t, T] : y(s) = O\}, \quad I_i = \{s \in [t, T] : y(s) \in J_i \setminus \{O\}\} \quad \text{for } i = 1, \dots, N. $$

Since $y(\cdot)$ is continuous, the set $I_0$ is closed and each $I_i$ can be written as the disjoint union of a (possibly infinite) family of subintervals of $[t, T]$, open in $[t, T]$.

We aim at bounding $\|\alpha\|_{\infty}$. For that, we consider the following different cases:

1. From Stampacchia theorem, $\alpha(s) = 0$ for a.e. $s \in I_0$.

2. Assume that, for some $t_1 \in (t, T)$ and for some $i \in \{1, \dots, N\}$, $(t_1, T] \subset I_i$. This implies in particular that $y(T) \in J_i \setminus \{O\}$. From the Euler-Lagrange condition (2.26) and the transversality condition (2.28),

$$ \alpha(s) = -\partial_x g_i(y(T)) + \int_T^s \partial_x \ell_i(y(\tau), \tau) d\tau. $$

From the assumptions made on $g_i$ and $\ell_i$, this implies that $\|\alpha\|_{L^\infty(t_1, T)} \leq V_*$.

3. Assume that for some $i \in \{1, \dots, N\}$, $[t, T] \subset I_i$. Then the same argument as in the previous point yields that $\|\alpha\|_{L^\infty(t, T)} \leq V_*$.

4. Assume that, for some $t_1, t_2 \in [t, T]$, $y(t_1) = y(t_2) = O$, and for some $i \in \{1, \dots, N\}$, $(t_1, t_2) \subset I_i$. From Lemma 2.17, the function $s \mapsto \alpha(s)$ is continuous on $(t_1, t_2)$. Then, from standard calculus, we deduce that there exists $t_3 \in (t_1, t_2)$ such that $\alpha(t_3) = 0$. Then (2.26) implies that for $s \in (t_1, t_2)$,

$$ \alpha(s) = \int_{t_3}^{s} \partial_x \ell_i(y(\tau), \tau) d\tau, $$

therefore that $\|\alpha\|_{L^\infty(t_1, t_2)} \leq V_*$.
5. Assume that, for some $t_1 \in (t, T)$, $y(t_1) = O$, and for some $i \in \{1, \dots, N\}$, $[t, t_1) \subset I_i$. From Remark 2.18, the control $\alpha$ is Lipschitz continuous and the bound (2.26) holds in $(t, t_1)$. In particular, $\alpha(t)$ is well defined. Take $\alpha(s) = \bar{\alpha}(s)e_i$ and $y(s) = \bar{y}(s)e_i$ for $s \in [t, t_1]$. It is clear that

$$ (2.31) \qquad \bar{\alpha}(t) - L_{\ell}(s-t) \le \bar{\alpha}(s) \le \bar{\alpha}(t) + L_{\ell}(s-t) \quad \forall s \in [t, t_1). $$

We distinguish two subcases

(a) If $\bar{\alpha}(t)$ is nonnegative, then since $\bar{y}(t_1) = 0 < \bar{y}(t)$, there exists $t_2 : t \le t_2 < t_1$ such that $\bar{\alpha}(t_2) = 0$. As above $\bar{\alpha}(s) = \int_{t_2}^s \partial_x \ell_i(y(\tau), \tau) d\tau$, which yields $\|\bar{\alpha}\|_{L^\infty(t,t_1)} \le V_*$

(b) If $\bar{\alpha}(t)$ is negative, then we can apply Lemma 2.22 below, which yields the desired bound on $\|\bar{\alpha}\|_{L^\infty(t,t_1)}$.

By using the fact that $[t, T] = \cup_{i=0}^N I_i$, the observations above on $I_0$ and $I_i$, and by combining all the points above, we get the desired estimate on $\|\bar{\alpha}\|_{L^\infty(0,T)}$. $\square$

**Lemma 2.22.** Keeping the assumptions of Theorem 2.21, we also assume that, for some $t_1 \in (t, T]$, $y(t_1) = O$, and for some $i \in \{1, \dots, N\}$, $y(s) \in J_i \setminus \{O\}$ for $s \in [t, t_1)$, and that $\alpha(t) \cdot e_i < 0$. Then, for some positive constant $C$ (depending only on $(T-t)^{-1}$, $d(x, O)$, $M_\ell$, $L_\ell$, $L_g$ defined in (2.29)), there holds

$$ \alpha(s) \cdot e_i \ge -C \quad \text{in } [t, t_1). $$

*Proof of Lemma 2.22.* Set $x = \bar{x}e_i$, $\alpha(t) = -\bar{v}e_i$ with $\bar{v} > 0$, and for any $s \in [t, t_1)$, let $\bar{\alpha}(s), \bar{y}(s)$ be the real numbers such that $\alpha(s) = \bar{\alpha}(s)e_i$ and $y(s) = \bar{y}(s)e_i$.

Hence, from Lemma 2.17, the claim is equivalent to the existence of some positive $C$ (depending only on $M_\ell$, $L_\ell$, $L_g$, $d(x, O)$ and $(T-t)$), such that

$$ \bar{v} \le C. $$

From (2.26), for $s \in [t, t_1)$ there holds

$$ (2.32) \qquad \begin{cases} (i) & -\bar{v} - L_{\ell}(s-t) \le \bar{\alpha}(s) \le -\bar{v} + L_{\ell}(s-t), \\ (ii) & \bar{x} - \bar{v}(s-t) - \frac{L_{\ell}(s-t)^2}{2} \le \bar{y}(s) \le \bar{x} - \bar{v}(s-t) + \frac{L_{\ell}(s-t)^2}{2}. \end{cases} $$

Let us start by some useful estimates. We claim that, for $\bar{v} \ge 2L_\ell T$ there holds

$$ (2.33) \qquad t + \frac{4\bar{x}}{5\bar{v}} \le t_1 \le t + \frac{4\bar{x}}{3\bar{v}}. $$

Indeed, the left inequality in (2.32)-(ii) with $s = t_1$ yields

$$ \bar{x} \le (t_1 - t) \left[ \bar{v} + \frac{L_{\ell}(t_1 - t)}{2} \right] \le (t_1 - t) \left[ \bar{v} + \frac{L_{\ell}T}{2} \right] \le (t_1 - t) \frac{5\bar{v}}{4}. $$

Analogously, the right inequality in (2.32)-(ii) with $s = t_1$ yields

$$ -\bar{x} \le (t_1 - t) \left[ -\bar{v} + \frac{L_{\ell}(t_1 - t)}{2} \right] \le (t_1 - t) \left[ -\bar{v} + \frac{L_{\ell}T}{2} \right] \le -(t_1 - t) \frac{3\bar{v}}{4}. $$

This concludes the proof of (2.33).
We now claim that, for $\bar{v} \ge \max\{2L_{\ell}T, 4\bar{x}/(3T)\}$, there holds

$$ (2.34) \qquad \bar{\alpha}(s) \le -\frac{\bar{v}}{2} \quad \forall s \in [t, t_1) \quad \text{and} \quad \int_t^{t_1} \frac{\bar{\alpha}(s)^2}{2} ds \ge \frac{\bar{v}\bar{x}}{10}. $$

Indeed, observe first that (2.32)-(i) entails

$$ \bar{\alpha}(s) \le -\bar{v} + L_{\ell}(t_1 - t) \quad \forall s \in [t, t_1). $$

From estimate (2.33) and our choice of $\bar{v}$,

$$ \bar{\alpha}(s) \le -\bar{v} + \frac{2\bar{x}}{3T} \le -\frac{\bar{v}}{2} \quad \forall s \in [t, t_1). $$

where we have successively used that $\bar{v} \ge 2L_{\ell}T$ and that $3T\bar{v} \ge 4\bar{x}$. Next, we deduce from the first inequality in (2.34) and (2.33) that

$$ \int_t^{t_1} \frac{\bar{\alpha}(s)^2}{2} ds \ge \frac{\bar{v}^2(t_1 - t)}{8} \ge \frac{\bar{v}\bar{x}}{10}, $$

and (2.34) is proved.

We are now going to find estimates on $\bar{v}$ by proposing suitable competitors for the optimal control problem defining $u(x, t)$. Let $V_*$ be the constant defined in (2.30).

If $\bar{v} \le \max\{2L_{\ell}T, 4\bar{x}/(3T), 40\bar{x}/(T-t), 20V_*\}$, there is nothing to do. We are left with estimating $\bar{v}$ in the case when

$$ (2.35) \qquad \bar{v} > \max \left( 2L_{\ell}T, \frac{4\bar{x}}{3T}, \frac{40\bar{x}}{T-t}, 20V_* \right). $$

The arguments below differ according to the behaviour of $(y, \alpha)$ after time $t_1$.

Case A: $d(y(s), O) \le \bar{x}$ for $s \in [t_1, T]$.

Recall that the case under focus is when (2.35) holds. Consider the control

$$ \alpha_1(s) = -\bar{v}/20e_i \quad \text{in } [t, t + 20\bar{x}/\bar{v}], \qquad \alpha_1(s) = 0 \quad \text{in } [t + 20\bar{x}/\bar{v}, T]. $$

Let $(y_1, \alpha_1)$ be the corresponding trajectory. Observe that $(y_1, \alpha_1)$ is admissible for $(x, t)$, so the optimality of $(y, \alpha)$ entails

$$ \begin{aligned} 0 \le J_t(x; (y_1, \alpha_1)) - J_t(x; (y, \alpha)) \le & \int_t^{t+20\bar{x}/\bar{v}} \left( \frac{\bar{v}^2}{800} + \ell_i(y_1(s), s) - L(y(s), s) \right) ds \\ & - \int_t^{t_1} \frac{\bar{\alpha}(s)^2}{2} ds + \int_{t+20\bar{x}/\bar{v}}^T (\ell_O(s) - L(y(s), s)) ds \\ & + g(O) - g(y(T)). \end{aligned} $$

Since $\bar{v} > \max\{2L_{\ell}T, 4\bar{x}/(3T)\}$, (2.34) implies that

$$ (2.36) \qquad 0 \le -\frac{3\bar{x}\bar{v}}{40} + 40M_{\ell}\frac{\bar{x}}{\bar{v}} + \int_{t+20\bar{x}/\bar{v}}^{T} (\ell_O(s) - L(y(s), s)) ds + g(O) - g(y(T)). $$

Denoting by $I$ the last integral, (2.8) and (2.6) yield

$$ \begin{aligned} I &= \sum_{i=1}^{N} \int_{t+20\bar{x}/\bar{v}}^{T} [\ell_O(s) - \ell_i(y(s), s)] \mathbb{1}_{y(s) \in J_i \setminus \{O\}} ds \\ &\le \sum_{i=1}^{N} \int_{t+20\bar{x}/\bar{v}}^{T} [\ell_i(0, s) - \ell_i(y(s), s)] \mathbb{1}_{y(s) \in J_i \setminus \{O\}} ds \\ &\le T L_{\ell} \bar{x}, \end{aligned} $$
where the latter inequality follows from the definition of case A. Similarly, $g(O)-g(y(T)) \le L_g \bar{x}$. Injecting these estimates in (2.36), we get

$$0 \le -\frac{3\bar{v}^2}{40} + (TL_{\ell} + L_g)\bar{v} + 40M_{\ell},$$

which implies that $\bar{v} \le \frac{20}{3}(\sqrt{((TL_{\ell} + L_g))^2 + 12M_{\ell}} + (TL_{\ell} + L_g))$. We have proven that in case A,

$$(2.38) \quad \bar{v} \le \max \left( 2L_{\ell}T, \frac{4\bar{x}}{3T}, \frac{40\bar{x}}{T-t}, 20V_{*}, \frac{20}{3} \left( \sqrt{((TL_{\ell} + L_g))^2 + 12M_{\ell}} + (TL_{\ell} + L_g) \right) \right).$$

Case B: $\exists \tau \in (t_1, T]$ such that $d(y(\tau), O) > \bar{x}$. Recall that (2.35) holds. For later use, set

$$\begin{align*}
\tau_2 &= \inf\{\tau \in [t_1, T] : d(y(\tau), O) > \bar{x}\}, \\
l &\in \{1, \dots, N\} \text{ such that } y(\tau_2) \in J_l \setminus \{O\}, \\
\tau_1 &= \inf\{s \in [t_1, T] : y(\tau) \in J_l \setminus \{O\} \quad \forall \tau \in (s, \tau_2]\}.
\end{align*}$$

In other words, $\tau_2$ is the first time larger than $t_1$ at which the trajectory reaches a distance to the origin greater than $\bar{x}$ and $\tau_1$ is the time at which the trajectory enters in $J_l \setminus \{O\}$ and remains there up to time $\tau_2$ (note that the trajectory can also visit $J_l \setminus \{O\}$ before $\tau_1$).

Let us distinguish three subcases.

Subcase B1: $\tau_1 \ge t + 20\bar{x}/\bar{v}$. Consider the control

$$\alpha_1(s) = -\frac{\bar{v}}{20}e_i \text{ in } [t, t+20\bar{x}/\bar{v}), \quad \alpha_1(s) = 0 \text{ in } [t+20\bar{x}/\bar{v}, \tau_1), \quad \alpha_1(s) = \alpha(s) \text{ in } [\tau_1, T],$$

and let $(y_1, \alpha_1)$ be the corresponding trajectory, which is clearly admissible for $(x, t)$. The optimality of $(y, \alpha)$ entails

$$0 \le J_t(x; (y_1, \alpha_1)) - J_t(x; (y, \alpha)) \le \int_t^{t+20\bar{x}/\bar{v}} \left( \frac{\bar{v}^2}{800} + \ell_i(y_1(s), s) - L(y(s), s) \right) ds \\ - \int_t^{t_1} \frac{\bar{\alpha}(s)^2}{2} ds + \int_{t+20\bar{x}/\bar{v}}^{\tau_1} (\ell_O(s) - L(y(s), s)) ds.$$

Then, from (2.34),

$$0 \le \left(\frac{\bar{v}}{40} - \frac{\bar{v}}{10}\right) \bar{x} + 40 \frac{M_{\ell}}{\bar{v}} \bar{x} + \int_{t+20\bar{x}/\bar{v}}^{\tau_1} (\ell_O(s) - L(y(s), s)) ds.$$

As above, we deduce that

$$0 \le -\frac{3\bar{v}^2}{40} + TL_{\ell}\bar{v} + 40M_{\ell},$$

which proves that in Subcase B1,

$$(2.39) \quad \bar{v} \le \max \left( 2L_{\ell}T, \frac{4\bar{x}}{3T}, \frac{40\bar{x}}{T-t}, 20V_{*}, \frac{20}{3} \left( \sqrt{(TL_{\ell})^2 + 12M_{\ell}} + TL_{\ell} \right) \right).$$

Subcase B2: $\tau_1 < t + 20\bar{x}/\bar{v}$ and $y(s) \in J_l \setminus \{O\}$ for $s \in (\tau_1, T]$. Consider the control

$$(2.40) \quad \alpha_1(s) = -\frac{\bar{v}}{20}e_i \text{ in } [t, t+20\bar{x}/\bar{v}], \quad \alpha_1(s) = a\alpha(as+b) \text{ in } (t+20\bar{x}/\bar{v}, T],$$
with

$$a = \frac{T - \tau_1}{T - t - 20\bar{x}/\bar{v}}, \quad \text{and} \quad b = T \frac{\tau_1 - t - 20\bar{x}/\bar{v}}{T - t - 20\bar{x}/\bar{v}},$$

(note that $a > 1$). Let $(y_1, \alpha_1)$ be the corresponding trajectory. There holds

$$y_1(s) = (\bar{x} - \bar{v}(s-t)/20)e_i \quad \text{in } [t, t+20\bar{x}/\bar{v}], \quad y_1(s) = y(as+b) \quad \text{in } [t+20\bar{x}/\bar{v}, T].$$

In particular, $(y_1, \alpha_1)$ is admissible for $(x, t)$. The optimality of $(y, \alpha)$ entails

$$ \begin{aligned} 0 &\le J_t(x; (y_1, \alpha_1)) - J_t(x; (y, \alpha)) \\ &\le \int_t^{t+20\bar{x}/\bar{v}} \left( \frac{\bar{v}^2}{800} + \ell_i(y_1(s), s) \right) ds + \int_{t+20\bar{x}/\bar{v}}^T \left( \frac{a^2|\alpha(as+b)|^2}{2} + L(y_1(s), s) \right) ds \\ &\quad - \int_t^{\tau_1} \frac{\bar{\alpha}(s)^2}{2} ds - \int_{\tau_1}^{\tau_1} \frac{|\alpha(s)|^2}{2} ds - \int_{\tau_1}^T \frac{|\alpha(s)|^2}{2} ds - \int_t^{\tau_1} L(y(s), s) ds - \int_{\tau_1}^T L(y(s), s) ds \\ &\le \left( \frac{\bar{v}}{40} - \frac{\bar{v}}{10} \right) \bar{x} + \int_t^{t+20\bar{x}/\bar{v}} \ell_i(y_1(s), s) ds + \int_{t+20\bar{x}/\bar{v}}^T \left( \frac{a^2|\alpha(as+b)|^2}{2} + L(y_1(s), s) \right) ds \\ &\quad (2.41) - \int_{\tau_1}^T \frac{|\alpha(s)|^2}{2} ds - \int_t^{\tau_1} L(y(s), s) ds - \int_{\tau_1}^T L(y(s), s) ds. \end{aligned} $$

Similarly as above,

$$(2.42) \quad \int_t^{t+20\bar{x}/\bar{v}} \ell_i(y_1(s), s) ds - \int_t^{\tau_1} L(y(s), s) ds \le M_\ell \left( \frac{20\bar{x}}{\bar{v}} + (\tau_1 - t) \right) \le 40M_\ell \frac{\bar{x}}{\bar{v}}.$$

On the other hand,

$$\int_{t+20\bar{x}/\bar{v}}^{T} \frac{a^2|\alpha(as+b)|^2}{2} ds - \int_{\tau_1}^{T} \frac{|\alpha(s)|^2}{2} ds = (a-1) \int_{\tau_1}^{T} \frac{|\alpha(s)|^2}{2} ds \le \frac{40\bar{x}}{\bar{v}(T-t)} \int_{\tau_1}^{T} \frac{|\alpha(s)|^2}{2} ds,$$

where the latter inequality comes from the fact that $a-1 \le \frac{20\bar{x}/\bar{v}}{T-t-20\bar{x}/\bar{v}}$ and that $\bar{v}(T-t) > 40\bar{x}$.

Recall that $V_*$ is the constant defined in (2.30), and that $\|\alpha\|_\infty \le V_*$ in $[\tau_1, T]$. Then, from the latter inequality, we deduce

$$(2.43) \quad \int_{t+20\bar{x}/\bar{v}}^{T} \frac{a^2|\alpha(as+b)|^2}{2} ds - \int_{\tau_1}^{T} \frac{|\alpha(s)|^2}{2} ds \le \frac{20\bar{x}}{\bar{v}} V_*^2.$$

On the other hand,

$$\int_{t+20\bar{x}/\bar{v}}^{T} L(y_1(s), s) ds - \int_{\tau_1}^{T} L(y(s), s) ds = \int_{t+20\bar{x}/\bar{v}}^{T} L(y(as+b), s) ds - \int_{\tau_1}^{T} L(y(s), s) ds = I_1 + I_2$$

for

$$I_1 = - \int_{\tau_1}^{t+20\bar{x}/\bar{v}} L(y(s), s) ds$$

$$I_2 = \int_{t+20\bar{x}/\bar{v}}^{T} (L(y(as+b), s) - L(y(s), s)) ds.$$

Since $0 < t + 20\bar{x}/\bar{v} - \tau_1 \le 20\bar{x}/\bar{v}$,

$$I_1 \le 20M_\ell \frac{\bar{x}}{\bar{v}}.$$
On the other hand, since both $y(as+b)$ and $y(s)$ belong to $J_l \setminus \{O\}$ for $s \in (t+20\bar{x}/\bar{v}, T]$,
there holds
(2.44)

$$d(y(as+b), y(s)) \le \int_{as+b}^{s} |\alpha(\theta)| d\theta \le V_*(T-s) \frac{t+20\bar{x}/\bar{v}-\tau_1}{T-t-20\bar{x}/\bar{v}} \le \frac{V_*(T-s)}{T-t-20\bar{x}/\bar{v}} \frac{20\bar{x}}{\bar{v}} \le 40V_* \frac{\bar{x}}{\bar{v}},$$

where the last inequality comes from the fact that $\bar{v}(T-t) > 40\bar{x}$. This implies that

$$I_2 \le 40V_*L_\ell T \frac{\bar{x}}{\bar{v}}.$$

Hence,

$$(2.45) \qquad \int_{t+\bar{x}/K_0}^{T} L(y_1(s), s)ds - \int_{\tau_1}^{T} L(y(s), s)ds \le 20(M_\ell + 2L_\ell V_*T) \frac{\bar{x}}{\bar{v}}.$$

Injecting (2.42), (2.43) and (2.45) in (2.41), we obtain

$$0 \le -\frac{3\bar{v}\bar{x}}{40} + 20(3M_{\ell} + V_{*}^{2} + 2V_{*}L_{\ell}T)\frac{\bar{x}}{\bar{v}},$$

thus

$$(2.46) \qquad \bar{v} \le \max \left( 2L_{\ell}T, \frac{4\bar{x}}{3T}, \frac{40\bar{x}}{T-t}, 20V_{*}, 20\sqrt{\frac{2}{3}(3M_{\ell} + V_{*}^{2} + 2L_{\ell}V_{*}T)} \right).$$

Subcase B3: $\tau_1 < t + 20\bar{x}/\bar{v}$ and $\exists \tau \in (\tau_1, T)$ such that $y(\tau) = O$. Set

$$\tau_3 = \min\{\tau \in (\tau_1, T] : y(\tau) = O\},$$

i.e. $\tau_3$ is the first time greater than $\tau_1$ at which the trajectory $y(\cdot)$ reaches the vertex.
Clearly, from the definition of $\tau_1$, $y(\tau) \in J_l \setminus \{O\}$ for $\tau \in (\tau_1, \tau_3)$ and $\tau_3 > \tau_2$. As in the previous cases,

$$\tau_2 > \tau_1 + \frac{\bar{x}}{V_*} \ge t + \frac{\bar{x}}{V_*}.$$

Since $\bar{v} > 20V_*$, we know that $\tau_2 > t + 20\bar{x}/\bar{v}$. Hence,

$$\tau_1 < t + 20 \frac{\bar{x}}{\bar{v}} < \tau_2 < \tau_3.$$

Consider the trajectory $(y_1, \alpha_1)$ defined in (2.40). Note that

$$y_1(s) = y(as+b) \in J_l \setminus \{O\} \quad \forall s \in I_* := \left[ t + 20 \frac{\bar{x}}{\bar{v}}, \frac{\tau_3(T - t - 20\bar{x}/\bar{v}) - T(\tau_1 - t - 20\bar{x}/\bar{v})}{T - \tau_1} \right].$$

Observe that $\tau_3 \in I_*$ and $y(\cdot) - y_1(\cdot) = (\bar{y}(\cdot) - \bar{y}_1(\cdot))e_\ell$ in $I_*$ with $\bar{y}(t + 20\bar{x}/\bar{v}) - \bar{y}_1(t + 20\bar{x}/\bar{v}) > 0$ and $\bar{y}(\tau_3) - \bar{y}_1(\tau_3) < 0$. We deduce that there exists $\tau_4 \in (t + 20\bar{x}/\bar{v}, \tau_3)$ such that $y(\tau_4) = y_1(\tau_4)$.

We can now choose a competitor $(y_2, \alpha_2)$ as the trajectory corresponding to the control

$$\alpha_2(s) = \alpha_1(s) \text{ in } [t, \tau_4], \quad \alpha_2(s) = \alpha(s) \text{ in } (\tau_4, T].$$

Note that there holds: $y_2(s) \in J_l \setminus \{O\}$ for $s \in [t, t + 20\bar{x}/\bar{v})$, $y_2(t + 20\bar{x}/\bar{v}) = O$, $y_2(s) \in J_l \setminus \{O\}$ and $y_2(s) = y(as + b)$ for $s \in [t + 20\bar{x}/\bar{v}, \tau_4)$, $y_2(s) = y(s)$ for $s \in [\tau_4, T]$.
The optimality of $(y, \alpha)$ entails

$$
\begin{align}
0 &\le J_t(x; (y_2, \alpha_2)) - J_t(x; (y, \alpha)) \nonumber \\
  &\le \int_t^{t+20\bar{x}/\bar{v}} \left( \frac{\bar{v}^2}{800} + \ell_i(y_2(s), s) \right) ds + \int_{t+20\bar{x}/\bar{v}}^{\tau_4} \left( \frac{a^2|\alpha(as+b)|^2}{2} + L(y_2(s), s) \right) ds \nonumber \\
  &\quad - \int_t^{t_1} \frac{\bar{\alpha}(s)^2}{2} ds - \int_{t_1}^{\tau_1} \frac{|\alpha(s)|^2}{2} ds - \int_{\tau_1}^{\tau_4} \frac{|\alpha(s)|^2}{2} ds - \int_t^{\tau_1} L(y(s), s) ds \nonumber \\
  &\quad - \int_{\tau_1}^{\tau_4} L(y(s), s) ds. \tag{2.47}
\end{align}
$$

As above,

$$
\int_t^{t+20\bar{x}/\bar{v}} \frac{\bar{v}^2}{800} ds - \int_t^{t_1} \frac{\bar{\alpha}(s)^2}{2} ds \leq -\frac{3\bar{v}\bar{x}}{40}.
$$

The same arguments as those used for obtaining (2.42),(2.43) lead to

$$
\begin{align*}
\int_t^{t+20\bar{x}/\bar{v}} \ell_i(y_2(s), s) ds - \int_t^{\tau_1} L(y(s), s) ds &\le 40 M_\ell \frac{\bar{x}}{\bar{v}}, \\
\int_{t+20\bar{x}/\bar{v}}^{\tau_4} \frac{a^2 |\alpha(as+b)|^2}{2} ds - \int_{\tau_1}^{\tau_4} \frac{|\alpha(s)|^2}{2} ds &\le 20 V_*^2 \frac{\bar{x}}{\bar{v}}.
\end{align*}
$$

On the other hand,

$$
\begin{align*}
\Lambda &:= \int_{t+20\bar{x}/\bar{v}}^{\tau_4} L(y_2(s), s) ds - \int_{\tau_1}^{\tau_4} L(y(s), s) ds \\
&= \int_{t+20\bar{x}/\bar{v}}^{\tau_4} L(y(as+b), s) ds - \int_{\tau_1}^{\tau_4} L(y(s), s) ds \\
&= - \int_{\tau_1}^{t+20\bar{x}/\bar{v}} L(y(s), s) ds + \int_{t+20\bar{x}/\bar{v}}^{\tau_4} [L(y(as+b), s) - L(y(s), s)] ds \\
&\le 20 M_\ell \frac{\bar{x}}{\bar{v}} + \int_{t+20\bar{x}/\bar{v}}^{\tau_4} [\ell_l(y(as+b), s) - \ell_l(y(s), s)] ds,
\end{align*}
$$

where the last inequality is due to the fact that both $y(as+b)$ and $y(s)$ belong to $J_l \setminus \{O\}$ for $s \in (t + 20\bar{x}/\bar{v}, \tau_4)$. Observe that estimate (2.44) holds on $[t + 20\bar{x}/\bar{v}, \tau_4]$, hence

$$
\Lambda \le 20(M_{\ell} + 2L_{\ell}V_{*}T) \frac{\bar{x}}{\bar{v}}.
$$

Injecting all these estimates in (2.47), we obtain

$$
0 \le -\frac{3\bar{v}\bar{x}}{40} + 20 \left( 3M_{\ell} + V_{*}^2 + 2L_{\ell}V_{*}T \right) \frac{\bar{x}}{\bar{v}},
$$

thus, in Subcase B3,

$$
(2.48) \quad \bar{v} \le \max \left( 2L_{\ell}T, \frac{4\bar{x}}{3T}, \frac{40\bar{x}}{T-t}, 20V_{*}, 20\sqrt{\frac{2}{3}(3M_{\ell} + V_{*}^2 + 2L_{\ell}V_{*}T)} \right).
$$

Finally, in all cases, $\bar{v}$ is smaller than the maximal value of the right hand sides in (2.38),(2.39),(2.46),(2.48). $\square$
If, in addition to the assumptions made in Theorem 2.21, the final cost is continuous on the whole network $G$ (thus Lipschitz continuous on $G$ because of the other assumptions), then it turns out that the optimal controls are uniformly bounded in the whole time interval $[0, T]$:

**Theorem 2.23.** We keep the assumptions of Theorem 2.21 and also assume that $g \in C^0(G)$. Then, the same result of Theorem 2.21 holds true with a constant $V$ independent of $(T-t)$, namely: there exists a constant $V_\# > 0$ (dependent on $L_g, M_g, L_\ell, d(x, O)$ but independent of $(T-t)$) such that

$$ \|\alpha\|_{\infty} \leq V_{\#} \quad \forall (y, \alpha) \in \Gamma_{t}^{\text{opt}}[x]. $$

*Proof.* We consider the same cases as in the proof of Theorem 2.21. Cases (1)-(4) and (5) – (a) are dealt with using the same arguments as in the proof of Theorem 2.21. In Case (5) – (b), we apply Lemma 2.24 below. □

**Lemma 2.24.** Under the assumptions of Theorem 2.23, the statement of Lemma 2.22 holds true with a constant $V_\#$ independent of $(T-t)$.

*Proof.* We borrow some notations of Lemma 2.22. In particular, we set: $x = \bar{x}e_i$, $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$, $\alpha(t) = -\bar{v}e_i$ with $\bar{v} > 0$, $\alpha(s) = \bar{\alpha}(s)e_i$ for $s \in [t, t_1)$. (Recall: $y(t_1) = O$). By Lemma 2.17, without any loss of generality, we assume $\bar{v}$ so large to have $\bar{\alpha}(s) < 0$ for $s \in [t, t_1)$.

Note that points (1)-(4) in the proof of Theorem 2.23 ensure that there exists a positive constant $V_1$ (dependent on $L_g, M_g, L_\ell, d(x, O)$ but independent of $(T-t)$ and of $(T-t_1)$) such that: $|\alpha(s)| \le V_1$ for $s \in [t_1, T]$.

We proceed constructing a competitor $(y_1, \alpha_1)$. For a constant $\mu \ge V_1$ which will suitably chosen later on, we introduce the trajectory $(y_1, \alpha_1) \in \Gamma_{t,t_1}[x]$ obeying to the control $\alpha_1(s) = \bar{\alpha}_1(s)e_i$ with

$$ \bar{\alpha}_1(s) = \begin{cases} \bar{\alpha}(s) & \text{if } \bar{\alpha}(s) \ge -\mu \\ 0 & \text{otherwise.} \end{cases} $$

Clearly, if $y_1(t_1) = y(t_1)$, then $\alpha(\cdot) = \bar{\alpha}(\cdot)$ a.e. in $[t, t_1]$ and there is nothing to prove. So we consider $y_1(t_1) \ne O$. We take $y_1(s) = \bar{y}_1(s)e_i$ for $s \in [t, t_1]$. Since $|\bar{\alpha}_1(\cdot)| \le |\bar{\alpha}(\cdot)|$ in $[t, t_1]$, $y_1(t_1) \in J_i \setminus \{O\}$, namely $\bar{y}_1(t_1) > 0$. Recalling $y(t_1) = O$ and $\bar{\alpha}(\cdot) < 0$ in $[t, t_1)$,

$$ \bar{y}_1(t_1) = [y_1(t_1) - y(t_1)] \cdot e_i = - \int_t^{t_1} \bar{\alpha}(s) \mathbf{1}_{\{\bar{\alpha}(s) < -\mu\}} ds = \int_t^{t_1} |\bar{\alpha}(s)| \mathbf{1}_{\{\bar{\alpha}(s) < -\mu\}} ds =: A $$

and also

$$ (2.49) \qquad d(y_1(s), y(s)) \le - \int_t^s \bar{\alpha}(\tau) \mathbf{1}_{\{\bar{\alpha}(\tau) < -\mu\}} d\tau \le A \quad \forall s \in [t, t_1]. $$

In order to construct our competitor after time $t_1$, we need an auxiliary trajectory. We consider the trajectory $(y_2, \alpha_2)$ starting at point $y_1(t_1) = Ae_i$ at time $t_1$ and obeying to the control $\alpha_2(s) = -V_1e_i$ for $s \in [t_1, t_1+A/V_1]$. Clearly, $y_2(s) \in e_i \setminus \{O\}$ for $s \in [t_1, t_1+A/V_1)$ with $y_2(t_1+A/V_1) = O$. We set

$$ t_2 = \min\{T, t_1 + A/V_1, \min\{s \in [t_1, T] : y_2(s) = y(s)\}\} $$

namely $t_2$ is the first moment among: the time horizon $T$, the instant $t_1 + A/V_1$ when $y_2$ reaches $O$ and the first moment when the trajectories $y(\cdot)$ and $y_1(\cdot)$ intersect. On the
interval $[t_1, t_2)$, we define our competitor $(y_1, \alpha_1)$ as: $y_1(s) = y_2(s)$. We note that, for $s \in [t_1, t_2)$, there holds

$$ (2.50) \qquad d(y_1(s), y(s)) \le d(y_1(s), O) + d(O, y(s)) \le A - V_1(s - t_1) + V_1(s - t_1) \le A. $$

Let us now argue differently according to the different situations in the definition of time $t_2$. Case (a): $t_2 = T$. In this case, our competitor is already completely constructed. By the optimality of $(y, \alpha)$,

$$ (2.51) \qquad 0 \le J_t(x; (y_1, \alpha_1)) - J_t(x; (y, \alpha)) = \sum_{i=1}^{5} I_i $$

where

$$ \begin{align*} I_1 &= \int_t^{t_1} \frac{|\alpha_1(s)|^2 - |\alpha(s)|^2}{2} ds, & I_2 &= \int_t^{t_1} (\ell_i(y_1(s), s) - \ell_i(y(s), s)) ds, \\ I_3 &= \int_{t_1}^{t_2} \frac{|\alpha_1(s)|^2 - |\alpha(s)|^2}{2} ds, & I_4 &= \int_{t_1}^{t_2} (\ell_i(y_1(s), s) - L(y(s), s)) ds, \\ I_5 &= g(y_1(T)) - g(y(T)). \end{align*} $$

From our choice of $\alpha_1$ in $[t, t_1]$, the Lipschitz continuity of $\ell_i$ and (2.49),

$$ I_1 = - \int_t^{t_1} \frac{|\alpha(s)|^2}{2} \mathbb{1}_{\{\bar{\alpha}(s) < -\mu\}} ds, \quad I_2 \le L_{\ell}T \|d(y_1(s), y(s))\|_{L^{\infty}(t, t_1)} \le L_{\ell}TA. $$

Moreover, we note $t_2 - t_1 \le A/V_1$ because of $t_2 = T$. From our choice of $\alpha_1$ in $[t_1, t_2]$,

$$ I_3 \le \int_{t_1}^{t_2} \frac{|\alpha_1(s)|^2}{2} ds = \frac{V_1^2}{2}(t_2 - t_1) \le \frac{V_1 A}{2}. $$

In order to estimate $I_4$ and $I_5$, observe that for $s \in [t_1, t_2]$ $y_1(s)$ and $y(s)$ may belong to different edges. For this reason, nothing better than

$$ I_4 \le 2M_{\ell}(t_2 - t_1) \le 2M_{\ell}A/V_1 \quad \text{and} \quad I_5 \le L_g d(y_1(T), y(T)) \le L_g A, $$

can be obtained, where the latter estimate is due to the global Lipschitz continuity of $g$ and (2.50) (here, the continuity of $g$ in the vertex plays a crucial role). Replacing all these estimates in (2.51), by the definition of $A$, we get

$$ 0 \le \int_t^{t_1} \left( -\frac{|\alpha(s)|}{2} + (L_{\ell}T + V_1/2 + 2M_{\ell}/V_1 + L_g) \right) |\alpha(s)| \mathbb{1}_{\{\bar{\alpha}(s) < -\mu\}} ds. $$

Hence, if $\{\bar{\alpha}(s) < -\mu\} \cap [t, t_1]$ has positive measure and $\mu > 2(L_{\ell}T + V_1/2 + 2M_{\ell}/V_1 + L_g)$, then we get the desired contradiction.

Case (b): $t_2 = \min\{s \in [t_1, T] : y_2(s) = y(s)\}$. We need to construct $(y_1, \alpha_1)$ also on $(t_2, T]$; we choose: $(y_1(s), \alpha_1(s)) = (y(s), \alpha(s))$ for $s \in (t_2, T]$. Note that also in this case, $t_2 - t_1 \le A/V_1$. Following the same calculations as those of case-(a), we end the proof.

Case (c): $t_2 = t_1 + A/V_1$ with $t_2 < T$ and $t_2 < \min\{s \in [t_1, T] : y_2(s) = y(s)\}$. Observe that $y_1(t_2) = O$ and $y(t_2) \in J_j \setminus \{O\}$ for some $j \in \{1, \dots, N\}$ with $|y(t_2)| \le A$. In this case, we need to construct our competitor also in the interval $[t_2, T]$. To this end,
we need another auxiliary trajectory; let $y_3(\cdot)$ be the path that starts at $O$ at time $t_2$ and obeying to the control $\alpha_3(s) = |\alpha(s)|e_j$ for $s \in [t_2, T]$. We set

$$t_3 = \min \{T, \min\{s \in [t_2, T] : y_3(s) = y(s)\}\}$$

and we define

$$(y_1(s), \alpha_1(s)) = (y_3(s), \alpha_3(s)) \quad \text{for } s \in (t_2, t_3].$$

Note that, in the interval $[t_2, t_3]$ both $y_1(s)$ and $y(s)$ belong to the same edge $J_j$; moreover, by $y_1(t_2) = O$ and $|y(t_2)| \le A$, for $s \in [t_2, t_3]$ there holds (2.52)

$$d(y_1(s), y(s)) \le d \left( \int_{t_2}^{s} |\alpha(\tau)| d\tau e_j, \left( A + \int_{t_2}^{s} \alpha(\tau) d\tau \right) e_j \right) = A + \int_{t_2}^{s} (\alpha(\tau) - |\alpha(\tau)|) d\tau \le A.$$

Now, we split our arguments according to the different situations in the definition of time $t_3$.

**Case (c1):** $t_3 = T$. From the optimality of $(y, \alpha)$,

$$(2.53) \qquad 0 \le J_t(x; (y_1, \alpha_1)) - J_t(x; (y, \alpha)) = \sum_{i=1}^{7} I_i$$

where: for $i = 1, \dots, 5$, the $I_i$'s are the same as those of case (a) (in particular, the estimates obtained in case (a) still hold true because $t_2 - t_1 = A/V_1$) and

$$I_6 = \int_{t_2}^{t_3} \left( \frac{|\alpha_1(s)|^2 - |\alpha(s)|^2}{2} \right) ds, \quad I_7 = \int_{t_2}^{t_3} \left( L(y_1(s), s) - \ell_j(y(s), s) \right) ds.$$

Our definition of $\alpha_3$ entails: $I_6 = 0$. Moreover, thanks to assumption (2.6) on the structure of $\ell_O$, the Lipschitz continuity of $\ell_i$ and (2.52),

$$I_7 \le \int_{t_2}^{t_3} (\ell_j(y_1(s), s) - \ell_j(y(s), s)) ds \le L_{\ell}TA.$$

Replacing all these estimates in (2.53), we get

$$0 \le \int_{t}^{t_1} \left( -\frac{|\alpha(s)|}{2} + (2L_{\ell}T + V_1/2 + 2M_{\ell}/V_1 + L_g) \right) |\alpha(s)| 1_{\{\bar{\alpha}(s) < -\mu\}} ds.$$

Hence, if $\{\bar{\alpha}(s) < -\mu\} \cap [t, t_1]$ has positive measure and $\mu > 2(2L_{\ell}T + V_1/2 + 2M_{\ell}/V_1 + L_g)$, then we get the desired contradiction.

**Case (c2):** $t_3 = \min\{s \in [t_2, T] : y_3(s) = y(s)\}$ with $t_3 < T$. We define our competitor on $[t_3, T]$ as the trajectory starting at $y_1(t_3) = y(t_3)$ at time $t_3$ and obeying to the control $\alpha_1(s) = \alpha(s)$ for $s \in [t_3, T]$. We end our proof using the same calculations of case (c1). $\square$

## 2.5 Closed graph property

Let us now investigate a closed graph property of the multi-valued map $x \mapsto \Gamma^{\text{opt}}[x]$ defined in (2.10).

**Proposition 2.25.** Fix $x \in \mathcal{G}$ and a sequence $\{x_n\}_{n \in \mathbb{N}}$ with $x_n \in \mathcal{G}$ and $x_n \to x$ as $n \to \infty$. Consider $(y_n, \alpha_n) \in \Gamma^{\text{opt}}[x_n]$ for any $n \in \mathbb{N}$. Assume that, as $n \to \infty$, $y_n$ uniformly converge to a path $y$. Then, the $y$ belongs to $Y_{x,0}$ (defined in (2.2)). Moreover, there exists a measurable function $\alpha$ such that $(y, \alpha)$ belongs to $\Gamma^{\text{opt}}[x]$ defined in (2.10).
An intermediate step in the proof of Proposition 2.25 is Lemma 2.26 below which deals with the approximation of admissible trajectories. The proof of Lemma 2.26 is postponed after that of Proposition 2.25.

**Lemma 2.26.** Fix $x \in G$ and $(y, \alpha) \in \Gamma[x]$; consider a sequence $\{x_n\}_{n \in \mathbb{N}}$ of points $x_n \in G$ such that $\delta_n := d(x_n, x) \to 0$ as $n \to \infty$. Then, there exists a sequence $\{(y_n, \alpha_n)\}_{n \in \mathbb{N}}$ such that, for any $n \in \mathbb{N}$, $(y_n, \alpha_n) \in \Gamma[x_n]$,

$$ (i) \quad \sup_{[0,T]} d(y_n(\cdot), y(\cdot)) \le \delta_n + \|\alpha\|_2 \sqrt{\delta_n} \quad \text{with} \quad y_n(T) = y(T) \\ (2.54) \quad (ii) \quad \|\alpha_n\|_2^2 \le \|\alpha\|_2^2 + \delta_n \left(1 + \frac{\|\alpha\|_2^2}{T-\delta_n}\right) \\ (iii) \quad \lim_{n \to \infty} J_0(x_n; (y_n, \alpha_n)) = J_0(x; (y, \alpha)). $$

*Proof of Proposition 2.25.* Consider $x, x_n$, $(y_n, \alpha_n)$ and $y$ as in the statement. We wish to prove that there exists a control $\alpha$ such that

i) $(y, \alpha)$ belongs to $\Gamma[x]$,

ii) $(y, \alpha)$ is optimal for $J_0$, i.e. $J_0(x, (y, \alpha)) \le J_0(x, (\hat{y}, \hat{\alpha}))$ for every $(\hat{y}, \hat{\alpha}) \in \Gamma[x]$.

Fix any $(\hat{y}, \hat{\alpha}) \in \Gamma[x]$. Lemma 2.26 ensures that there exists a sequence $\{(\hat{y}_n, \hat{\alpha}_n)\}_{n \in \mathbb{N}}$ such that $(\hat{y}_n, \hat{\alpha}_n) \in \Gamma[x_n]$ and

$$ (2.55) \qquad \hat{y}_n \to \hat{y} \text{ uniformly in } [0, T] \text{ as } n \to \infty, \qquad \|\hat{\alpha}_n\|_2 \le \|\hat{\alpha}\|_2 + o_n(1), \\ \qquad \limsup_{n \to \infty} J_0(x_n; (\hat{y}_n, \hat{\alpha}_n)) \le J_0(x; (\hat{y}, \hat{\alpha})) $$

where $o_n(1)$ is a sequence such that $\lim_n o_n(1) = 0$. On the other hand, the optimality of $(y_n, \alpha_n)$ yields

$$ (2.56) \qquad J_0(x_n; (y_n, \alpha_n)) \le J_0(x_n; (\hat{y}_n, \hat{\alpha}_n)). $$

From the observations above, we deduce that $J_0(x_n; (y_n, \alpha_n))$ are uniformly bounded and, in particular that there exists a constant $C$, independent of $n$, such that $\int_t^T |\alpha_n(\tau)|^2 d\tau \le C$. Hence, repeating the same arguments as those in the proof of Proposition 2.11 (in particular, for obtaining (2.14)), we deduce that $\{\alpha_n\}_{n \in \mathbb{N}}$ converges to some control $\alpha$ in the weak topology of $L^2([0, T], \mathbb{R}^d)$ and $(y, \alpha) \in \Gamma[x]$. Hence, point i) is proved. Taking the $\liminf_n$ in (2.56) and using (2.55), we also deduce $J_0(x, (y, \alpha)) \le J_0(x, (\hat{y}, \hat{\alpha}))$. Thanks to the arbitrariness of $(\hat{y}, \hat{\alpha}) \in \Gamma[x]$, we deduce point ii). $\square$

*Proof of Lemma 2.26.* Without any loss of generality, we may assume that, (possibly after extracting a subsequence that we still denote $\{x_n\}$) all the points $x$ and $x_n$ belong to the same edge (for simplicity, say $J_1$) for $n$ sufficiently large, so $x = \bar{x}e_1$, $x_n = \bar{x}_n e_1$ for $\bar{x}, \bar{x}_n \in \mathbb{R}^+$. Indeed, if $x = O$, we may argue edge by edge since there are finitely many edges. Set $\delta_n = d(x, x_n) = |\bar{x} - \bar{x}_n|$. Let us now introduce a control $\alpha_n$ such that the corresponding path $y_n$ is admissible (i.e. it takes its values on the network).

Set

$$ \alpha_n(s) = \begin{cases} \begin{cases} e_1 & \text{if } \bar{x}_n \le \bar{x} \\ -e_1 & \text{if } \bar{x}_n > \bar{x} \end{cases} & \text{for } s \in [0, \delta_n] \\ \frac{T}{T-\delta_n} \alpha \left( (s-\delta_n) \frac{T}{T-\delta_n} \right) & \text{for } s \in (\delta_n, T] \end{cases} $$
(note that here the structure $A_i = \{i\} \times \mathbb{R}$ plays a crucial role) and let $y_n$ start from $x_n$ and correspond to $\alpha_n$:

$$y_n(s) = x_n + \int_t^s \alpha_n(\tau) d\tau.$$

Observe that for $s \in [0, \delta_n]$,

$$y_n(s) = x_n + (s/\delta_n)(x - x_n)$$

in particular, $y_n(\delta_n) = x$. From the definition of $\alpha_n$, we get after a change of variable that, for $s \in [\delta_n, T]$,

$$\begin{align}
y_n(s) &= x + \int_{\delta_n}^{s} \frac{T}{T - \delta_n} \alpha \left( (\tau - \delta_n) \frac{T}{T - \delta_n} \right) d\tau = x + \int_{0}^{(s-\delta_n)\frac{T}{T-\delta_n}} \alpha(\tau) d\tau \tag{2.57} \\
&= y \left( (s - \delta_n) \frac{T}{T - \delta_n} \right). \nonumber
\end{align}$$

The trajectory $(y_n, \alpha_n)$ is admissible and

$$(2.58) \qquad y_n(T) = y(T).$$

The trajectory $y_n$ starts at $x_n$, moves with speed 1 until it reaches the point $x$ at time $\delta_n$ (clearly, in this time interval it always remains in the edge $J_1$) and, from time $\delta_n$, becomes a time-rescaled version of the trajectory $y$ such that $y_n(T) = y(T)$.

Let us now estimate $d(y(s), y_n(s))$. For $s \in [0, \delta_n]$,

$$\begin{align*}
d(y(s), y_n(s)) &\le d(y_n(s), x) + d(y(s), x) \le (\delta_n - s) + \int_0^s |\bar{\alpha}(\tau)| d\tau \\
&\le (\delta_n - s) + \|\alpha\|_2 \sqrt{s}
\end{align*}$$

(Cauchy-Schwarz inequality is used in the last line).

This and (2.57) imply that for $s \in [\delta_n, T]$,

$$\begin{align*}
d(y(s), y_n(s)) &= d \left( y(s), y \left( \left( s - \delta_n \right) \frac{T}{T - \delta_n} \right) \right) \le \int_{(s-\delta_n)\frac{T}{T-\delta_n}}^s |\bar{\alpha}(\tau)| d\tau \\
&\le \|\alpha\|_2 \sqrt{\delta_n} \sqrt{\frac{T-s}{T-\delta_n}}.
\end{align*}$$

The latter two inequalities easily imply the bound (2.54)-(i).

Next, by definition of $\alpha_n$,

$$\begin{align*}
\|\alpha_n\|_2^2 &= \delta_n + \int_{\delta_n}^s \left(\frac{T}{T - \delta_n}\right)^2 \alpha^2 \left( (\tau - \delta_n) \frac{T}{T - \delta_n} \right) d\tau = \delta_n + \int_0^T \frac{T}{T - \delta_n} \alpha(\tau)^2 d\tau \\
&= \delta_n + \|\alpha\|_2^2 + \frac{\delta_n}{T - \delta_n} \|\alpha\|_2^2,
\end{align*}$$

which easily implies the bound of (2.54)-(ii).

We now prove (2.54)-(iii). From (2.58), $g(y_n(T)) = g(y(T))$. Hence,

$$(2.59) \qquad J_0(x_n; (y_n, \alpha_n)) - J_0(x; (y, \alpha)) = \sum_{i=1}^4 I_i$$
where

$$I_1 = \int_0^{\delta_n} \left[ \sum_{i=1}^N \ell_i(y_n(\tau), \tau) \mathbf{1}_{y_n(\tau) \in J_i \setminus \{O\}} + \ell_O(\tau) \mathbf{1}_{y_n(\tau)=O} \right] d\tau,$$

$$I_2 = \frac{\|\alpha_n\|_2^2 - \|\alpha\|_2^2}{2},$$

$$I_3 = \int_{\delta_n}^{T} \left[ \sum_{i=1}^{N} \ell_i(y_n(\tau), \tau) \mathbf{1}_{y_n(\tau) \in J_i \setminus \{O\}} + \ell_O(\tau) \mathbf{1}_{y_n(\tau)=O} \right] d\tau,$$

$$I_4 = - \int_0^T \left[ \sum_{i=1}^N \ell_i(y(\tau), \tau) \mathbf{1}_{y(\tau) \in J_i \setminus \{O\}} + \ell_O(\tau) \mathbf{1}_{y(\tau)=O} \right] d\tau.$$

The boundedness of $\ell_i$ implies

$$|I_1| \le K \delta_n,$$

for $K = \sum_{i=1}^N \|\ell_i\|_\infty + \|\ell_*\|_\infty$. On the other hand, (2.54)-(ii) entails

$$|I_2| \le \frac{\delta_n}{2} \left( 1 + \frac{\|\alpha\|_2^2}{T - \delta_n} \right).$$

The definition of $\alpha_n$ and (2.57) yield

$$I_3 = \int_{\delta_n}^{T} \left[ \sum_{i=1}^{N} \ell_i \left( y \left( \left( \tau - \delta_n \right) \frac{T}{T - \delta_n} \right), \tau \right) \mathbf{1}_{y \left( \left( \tau - \delta_n \right) \frac{T}{T - \delta_n} \right) \in J_i \setminus \{O\}} + \ell_O(\tau) \mathbf{1}_{y \left( \left( \tau - \delta_n \right) \frac{T}{T - \delta_n} \right) = O} \right] d\tau,$$

which becomes after a change of variable,

$$I_3 = \int_0^T \left[ \sum_{i=1}^N \ell_i \left( y(\theta), \frac{T - \delta_n}{T} \theta + \delta_n \right) \mathbf{1}_{y(\theta) \in J_i \setminus \{O\}} + \ell_O \left( \frac{T - \delta_n}{T} \theta + \delta_n \right) \mathbf{1}_{y(\theta)=O} \right] \left( 1 - \frac{\delta_n}{T} \right) d\theta.$$

Let $\mathcal{G}'$ a bounded subset of $\mathcal{G}$ such that $y(s)$ belongs to $\mathcal{G}'$ for all $s \in [0, T]$ and let $\omega$ be a common modulus of continuity of the $\ell_i$ in $J_i \cap \mathcal{G}'$. The latter observation and the definition of $I_4$ yield

$$|I_3 + I_4| \le \int_0^T (N+1)\omega \left( \delta_n \frac{T-\theta}{T} \right) d\theta + \delta_n(N+1)K \le (N+1)T\omega(\delta_n).$$

Combining all the estimates with (2.59) and taking the lim sup, we complete the proof of (2.54)-(iii). □

## 2.6 Lipschitz continuity of the value function

We investigate the Lipschitz continuity of the value function $u$. We will see below that special assumptions will be needed for it to hold up to the horizon $T$.

**Proposition 2.27.** Under the same assumption as in Theorem 2.21, the value function is locally Lipschitz continuous in $\mathcal{G} \times [0, T)$.
**Remark 2.28.** Note that in contrast with Remark 2.15, we do not suppose that the costs $\ell_i$ are Lipschitz continuous with respect to time.

*Proof.* The proof borrows some ideas of [17, Proposition 4.1] and is split into several steps. For brevity, we set

$$ (2.60) \qquad \mathcal{G}_{\delta} = \{x \in \mathcal{G} : d(x, O) \le \delta\}. $$

**Step 1.** We first prove that $u(\cdot, t)$ is locally Lipschitz continuous in $J_i \setminus \{O\}$ locally uniformly with respect to $t \in [0, T)$. More precisely, having fixed $T_1 \in (0, T)$ and $R > 0$, we wish to prove that for any $t \in [0, T_1]$, $x_0 = \bar{x}_0 e_i \in J_i \cap \mathcal{G}_R \setminus \{O\}$ and $r > 0$ sufficiently small, the function $u(\cdot, t)$ is Lipschitz continuous on $(\bar{x}_0 - r, \bar{x}_0 + r)e_i$ with a Lipschitz constant which depends only on the parameters of the problem and on $T_1$ and $R$ (it is independent of $\bar{x}_0$, $r$, $t$ and $i$).

For that, fix some $r$, $0 < r < \bar{x}_0/4$. Observe that $(\bar{x}_0 - 4r, \bar{x}_0 + 4r)e_i \subset J_i \setminus \{O\}$. Consider $\bar{x}, \bar{x}_1 \in (\bar{x}_0 - r, \bar{x}_0 + r)$, with $\bar{x} \ne \bar{x}_1$ and $|\bar{x} - \bar{x}_1| \le 2(T - t)V$, where $V$ is the constant found in Theorem 2.21 for the set $\mathcal{G}_{5R/4} \times [0, T_1]$. Set $x = \bar{x}e_i$, $x_1 = \bar{x}_1e_i$ and $\tau = \frac{|\bar{x} - \bar{x}_1|}{2V}$. For $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$, let $(y_1, \alpha_1)$ be the trajectory starting at $x_1$ at time $t$ and associated to the control

$$ \alpha_1(s) = \alpha(s) + (x - x_1)/\tau \quad \text{for } s \in [t, t + \tau], \quad \alpha_1(s) = \alpha(s) \quad \text{for } s \in (t + \tau, T]. $$

From Theorem 2.21, $y$ does not reach the origin $O$ before time $t + \frac{3r}{V}$. On the other hand, $\tau \le \frac{r}{V}$. Hence, in the time interval $(t, t + \tau)$, $y$ stays in $J_i \setminus \{O\}$.

It is clear that $y(\cdot) = y_1(\cdot)$ in $(t + \tau, T]$. We claim that

$$ (i) \quad (y_1, \alpha_1) \in \Gamma_t[x_1]. $$

$$ (ii) \quad d(y(\cdot), y_1(\cdot)) \le |\bar{x} - \bar{x}_1| \quad \text{in } (t, T] $$

Let us prove (i). From the observation above, it is enough to prove that $y_1(s) \in J_i$ for $s \in [t, t + \tau]$. We observe that

$$ \begin{aligned} d(y_1(s), x_0) &\le d(y_1(s), x_1) + |\bar{x}_1 - \bar{x}_0| \le \int_t^s \left| \alpha(\theta) + \frac{x-x_1}{\tau} \right| d\theta + r \\ &\le \int_t^s \left| \alpha(\theta) \right| d\theta + \frac{s-t}{\tau} |\bar{x} - \bar{x}_1| + r \le V(s-t) + |\bar{x} - \bar{x}_1| + r \\ &\le 4r, \end{aligned} $$

where the last inequality is due to our choice of $\tau$. The inequality found above yields that $y_1(s) \in J_i$ for $s \in [t, t + \tau]$, then (i).

Let us now prove (ii). For $s \in (t + \tau, T]$, (ii) is obvious. For $s \in (t, t + \tau]$, there holds

$$ d(y(s), y_1(s)) = \left| (\bar{x} - \bar{x}_1) - \int_t^s \frac{\bar{x} - \bar{x}_1}{\tau} ds \right| = \left| (\bar{x} - \bar{x}_1) \frac{\tau - (s - t)}{\tau} \right| \le |\bar{x} - \bar{x}_1|. $$

The claims (i) and (ii) are proved.

By definition of $u$, and recalling that in the interval $(t, t + \tau)$ both $y$ and $y_1$ stay in $J_i \setminus \{O\}$,

$$ (2.61) \qquad u(x_1, t) - u(x, t) \le \int_t^{t+\tau} \left( \frac{|\alpha_1(s)|^2}{2} - \frac{|\alpha(s)|^2}{2} + \ell_i(y_1(s), s) - \ell_i(y(s), s) \right) ds. $$
The definition of $\alpha_1$ and Theorem 2.21 imply that

$$ 
\begin{aligned}
\int_t^{t+\tau} \left( \frac{|\alpha_1(s)|^2}{2} - \frac{|\alpha(s)|^2}{2} \right) ds &\le \frac{1}{2} \int_t^{t+\tau} \left( \frac{|\bar{x} - \bar{x}_1|^2}{\tau^2} + 2 \frac{|\bar{x} - \bar{x}_1||\alpha(s)|}{\tau} \right) ds \\
&\le \frac{1}{2} \frac{|\bar{x} - \bar{x}_1|^2}{\tau} + |\bar{x} - \bar{x}_1|V \\
&\le 2V|\bar{x} - \bar{x}_1|,
\end{aligned}
$$

where the last inequality is due to the choice of $\tau$. On the other hand, assumption (2.29) and point (ii) entail

$$ \int_t^{t+\tau} (\ell_i(y_1(s), s) - \ell_i(y(s), s)) ds \le L_\ell |\bar{x} - \bar{x}_1|\tau = \frac{L_\ell |\bar{x} - \bar{x}_1|^2}{2V} \le \frac{L_\ell r}{V} |\bar{x} - \bar{x}_1| \le \frac{L_\ell R}{4V} |\bar{x} - \bar{x}_1| $$

because $|\bar{x} - \bar{x}_1| \le 2r \le \bar{x}_0/2 \le R/2$. The latter two inequalities and (2.61) yield

$$ u(x_1, t) - u(x, t) \le \left( 2V + \frac{L_\ell R}{4V} \right) |\bar{x} - \bar{x}_1|. $$

Reversing the role of $x$ and $x_1$, we obtain the desired Lipschitz continuity with constant $2V + L_\ell R/4V$, and complete Step 1.

**Step 2.** We observe that the Lipschitz constant found in Step 1 is independent of $\bar{x}_0$, provided that $\bar{x}_0 \in \mathcal{G}_R$. Hence, $u(\cdot, t)$ is Lipschitz continuous in $(\mathcal{G}_R \cap J_i) \setminus \{O\}$ with the same Lipschitz constant as above.

**Step 3.** By the continuity of $u$ (see Proposition 2.14), $u(\cdot, t)$ is Lipschitz continuous in $\mathcal{G}_R$ with Lipschitz constant $2V + L_\ell R/4V$. Note that this Lipschitz constant depends implicitly on $T_1$ through $V$.

**Step 4.** We now prove the Lipschitz continuity in time of $u$ for $t \in [0, T_1]$. Consider $x \in \mathcal{G}_R$ and $t_1, t_2 \in [0, T_1]$. Without loss of generality, we may assume that $t_1 \le t_2$.

Consider $(y, \alpha) \in \Gamma_{t_1}^{\text{opt}}[x]$. Observe that $y(t_2) \in \mathcal{G}_{R+VT}$. Let $W \ge V$ be the constant found in Theorem 2.21 for the set $\mathcal{G}_{5(R+VT)/4} \times [0, T_1]$. Obviously,

$$ |u(x, t_2) - u(x, t_1)| \le |u(x, t_2) - u(y(t_2), t_2)| + |u(y(t_2), t_2) - u(x, t_1)|. $$

From Step 3 and Theorem 2.21,

$$ 
\begin{aligned}
|u(x, t_2) - u(y(t_2), t_2)| &\le \left( 2W + \frac{L_\ell(R + VT)}{4W} \right) d(x, y(t_2)) \le \left( 2W + \frac{L_\ell(R + VT)}{4W} \right) \int_{t_1}^{t_2} |\alpha(s)| ds \\
&\le \left( 2VW + \frac{L_\ell(R + VT)}{4} \right) |t_2 - t_1|.
\end{aligned}
$$

On the other hand, the Dynamic Programming Principle (see Proposition 2.13) ensures that

$$ |u(y(t_2), t_2) - u(x, t_1)| \le \int_{t_1}^{t_2} \left( \frac{|\alpha(s)|^2}{2} + |L(y(s), s)| \right) ds \le \left( \frac{V^2}{2} + M_\ell \right) |t_2 - t_1|. $$

From the latter three inequalities, we deduce that

$$ |u(x, t_2) - u(x, t_1)| \le \left( 2VW + \frac{V^2}{2} + \frac{L_\ell(R + VT)}{4} + M_\ell \right) |t_2 - t_1|. $$

Hence, Step 4 is done.

**Step 5.** We achieve the proof by combining the results obtained in steps 3 and 4. $\square$
If furthermore the terminal cost g is continuous on $\mathcal{G}$, then the Lipschitz continuity of u w.r.t. $(x, t)$ holds locally in $x$ and globally in $t \in [0, T]$:

**Corollary 2.29.** Under the assumptions of Theorem 2.23, the value function u is locally Lipschitz continuous in $\mathcal{G} \times [0, T]$.

*Proof.* Since u is continuous on $\mathcal{G} \times [0, T]$, it is enough to repeat the proof of Proposition 2.27 using Theorem 2.23 instead of Theorem 2.21. $\square$

The following proposition, which will not be used in the remaining part of the paper, addresses the local Lipschitz continuity of the value function with respect to $x$ up to the horizon $T$, provided that the terminal cost $g$ is Lipschitz continuous on $\mathcal{G}$ and the running costs $\ell_i$ are Lipschitz continuous w.r.t. $x$, but without assuming $C^2$ continuity of the costs in $J_i \setminus \{O\}$. Note that its proof does not rely on the optimality conditions stated in Lemmas 2.17 and 2.20, in contrast with Corollary 2.29.

**Proposition 2.30.** If the terminal cost $g$ is Lipschitz continuous in $\mathcal{G}$ with Lipschitz constant $L_g$ and the costs $\ell_i$ are bounded ($\|\ell_i\|_{\infty} \le M_\ell$) and Lipschitz continuous in $x$ with Lipschitz constant $L_\ell$, then, the value function is locally Lipschitz continuous with respect to $x$ in $\mathcal{G} \times [0, T]$.

*Proof.* For what follows, let us fix $v$ an arbitrary positive constant.

There is no loss of generality in assuming that $x_1$ and $x_2$ belong to the same edge, say $J_i$, i.e. $x_1 = \bar{x}_1e_i$ and $x_2 = \bar{x}_2e_i$. From Remark 2.6, there exists $C > 0$ such that for every $(y, \alpha) \in \Gamma_t^{\text{opt}}[x], \|\alpha\|_2 \le C$ and $y$ is $1/2$-Hölder continuous with Hölder constant $C$. Let us distinguish several cases.

**Case 1:** $x_1, x_2 \in J_i \setminus \{O\}$ with $\bar{x}_1, \bar{x}_2 \ge C(T-t)^{1/2}$. Consider $(y_2, \alpha_2) \in \Gamma_t^{\text{opt}}[x_2]$. Since $d(y_2(s), x_2) \le C(T-t)^{1/2}$ for every $s \in [t, T]$, the control $\alpha_2$ is also admissible for $(x_1, t)$ because $d(x_1, O) \ge C(T-t)^{1/2}$. Let $y_1$ be the path starting from $x_1$ at time $t$ and associated to the control $\alpha_2$. For $s \in [t, T]$, both $y_2(s)$ and $y_1(s)$ belong to $J_i$, and $d(y_2(s), y_1(s)) = d(x_2, x_1) = |\bar{x}_2 - \bar{x}_1|$. By definition of u, there holds

$$u(x_1, t) - u(x_2, t) \le \int_t^T |\ell_i(y_1(s), s) - \ell_i(y_2(s), s)| ds + |g(y_1(T)) - g(y_2(T))| \\ \le (L_\ell T + L_g) d(x_1, x_2).$$

The proof is completed by reversing the roles of $x_1$ and $x_2$.

**Case 2:** $x_1, x_2 \in J_i$ with $\bar{x}_1 \le 2C(T-t)^{1/2}$ and $x_2 = O$. For $(y_2, \alpha_2) \in \Gamma_t^{\text{opt}}[x_2]$, set

$$\alpha_1(s) = -\max\{v, |\alpha_2(s)|\}e_i, \quad \text{for } s \in [t, t_*],$$

where $v$ is the constant fixed above. Let $y_1$ be the path defined on $[t, t_*]$ such that $y_1(t) = x_1$ and corresponding to the control $\alpha_1$. The time $t_*$ is defined by

$$t_* = \min\{T, \min\{s \in [t, T] : y_1(s) = y_2(s)\}, \min\{s \in [t, T] : y_1(s) = O\}\}.$$

Then

$$(2.62) \qquad t_* - t \le \frac{\bar{x}_1}{v} = \frac{d(x_1, x_2)}{v}.$$

because $y_1(s) \ne O$ for $s \in [t, t_*)$ and $\alpha_1(s) \cdot e_i \le -v$ for $s \in [t, t_*]$.
The definition of $\alpha_2$ also implies that
(2.63)
$$d(y_1(s), y_2(s)) \le d(y_1(s), O) + d(y_2(s), O) \le \bar{x}_1 - \int_t^s |\alpha_2(\tau)| d\tau + \int_t^s |\alpha_2(\tau)| d\tau = d(x_1, x_2),$$
for $s \in [t, t_*]$. Again from (2.62),

$$ (2.64) \quad \int_t^{t_*} \left[ \frac{|\alpha_1(s)|^2}{2} - \frac{|\alpha_2(s)|^2}{2} + L(y_1(s), s) - L(y_2(s), s) \right] ds \le \left[ \frac{v^2}{2} + 2M_\ell \right] \frac{d(x_1, x_2)}{v}. $$

The following arguments will differ according to the value of $t_*$.

**Subcase 2-a:** $t_* = T$. From (2.63) and the Lipschitz continuity of $g$,

$$ g(y_1(T)) - g(y_2(T)) \le L_g d(x_1, x_2). $$

This inequality and (2.64) yield

$$ u(x_1, t) - u(x_2, t) \le \left( \frac{4M_\ell + v^2}{2v} + L_g \right) d(x_1, x_2). $$

**Subcase 2-b:** $t_* = \min\{s \in [t, T] : y_1(s) = y_2(s)\} < T$. In this case, set $\alpha_1(s) = \alpha_2(s)$ for $s \in (t_*, T]$. Clearly, $y_1(s) = y_2(s)$ for $s \in (t_*, T]$. This and (2.64) imply

$$ u(x_1, t) - u(x_2, t) \le (2M_\ell + v^2/2) \frac{|\bar{x}_2 - \bar{x}_1|}{v}. $$

**Subcase 2-c:** $t_* = \min\{s \in [t, T] : y_1(s) = O\} < \min\{T, \min\{s \in [t, T] : y_1(s) = y_2(s)\}\}$. Then, $y_2(t_*)$ belongs to some $J_j \setminus \{O\}$ with $j \ne i$ and $y_1(t_*) = O$. Indeed, should $y_2(t_*)$ belong to $J_i \setminus \{O\}$, then there would exist a time $\tau \in (t, t_*)$ such that $y_1(\tau) = y_2(\tau)$, in contradiction with the definition of $t_*$, and $y_1(t_*) = y_2(t_*) = O$ has been addressed in **Subcase 2-b**.

Let us define $(y_1, \alpha_1)$ by

$$ \alpha_1(s) = |\alpha_2(s)|e_j \quad \text{for } t \in (t_*, t_{**}], $$

where $t_{**} = \min\{T, \min\{s \in (t_*, T] : y_1(s) = y_2(s)\}\}$. Note that, in $[t_*, t_{**}]$, both $y_1(\cdot)$ and $y_2(\cdot)$ belong to $J_i$ with $y_2(\cdot) \ne O$. Here again, the arguments differ according to the cases in the definition of $t_{**}$.

**Subcase 2-c1:** $t_{**} = T$. For $s \in [t_*, t_{**}]$, there holds

$$ (2.65) \qquad \begin{aligned} d(y_1(s), y_2(s)) &= y_2(t_*) \cdot e_j + \int_{t_*}^s \alpha_2(\tau) \cdot e_j d\tau - \int_{t_*}^s |\alpha_2(\tau)| d\tau \\ &\le y_2(t_*) \cdot e_j = d(y_2(t_*), y_1(t_*)) \le d(x_1, x_2), \end{aligned} $$

the last inequality stemming from (2.63). Taking into account estimate (2.64), we get

$$ \begin{aligned} u(x_1, t) - u(x_2, t) &\le (2M_\ell + \frac{v^2}{2})d(x_1, x_2) + \int_{t_*}^{t_{**}} [L(y_1(s), s) - L(y_2(s), s)] ds \\ &\quad + g(y_1(T)) - g(y_2(T)) \\ &\le (L_g + 2M_\ell + \frac{v^2}{2})d(x_1, x_2) + \int_{t_*}^{t_{**}} [\ell_j(y_1(s), s) - \ell_j(y_2(s), s)] ds \end{aligned} $$
where the last inequality is due to the Lipschitz continuity of g and (2.65). Then the Lipschitz continuity of $\ell_j$ and (2.65) again lead to

$$ (2.66) \qquad u(x_1, t) - u(x_2, t) \le (TL_\ell + L_g + 2M_\ell + \frac{v^2}{2})|\bar{x}_2 - \bar{x}_1|. $$

**Subcase 2-c2:** $t_{**} < T$. Hence, $y_1(t_{**}) = y_2(t_{**})$. Set $(y_1, \alpha_1) = (y_2, \alpha_2)$ on $(t_{**}, T]$. The same calculations as in Subcase 2-c1 yield (2.66).

**Case 3:** $x_1, x_2 \in J_i$ with $0 < \bar{x}_2 < \bar{x}_1 \le 2C(T-t)^{1/2}$. Consider $(y_2, \alpha_2) \in \Gamma_t^{\text{opt}}[x_2]$ and define the path $y_1$ starting at $x_1$ at time $t$ and corresponding to the control

$$ \alpha_1(s) = -|\alpha_2(s)|e_i \quad \text{for } s \in [t, t_*], $$

where

$$ t_* = \min\{T, \min\{s \in [t, T] : y_2(s) = O\}, \min\{s \in [t, T] : y_1(s) = y_2(s)\}\}. $$

Observe that, for $s \in [t, t_*)$, both $y_1(s)$ and $y_2(s)$ belong to $J_i \setminus \{O\}$, and

$$ (2.67) \qquad d(y_1(s), y_2(s)) \le \bar{x}_1 - \int_t^s |\alpha_2(\tau)|d\tau - \bar{x}_2 - \int_t^s \alpha_2(\tau) \cdot e_i d\tau \le \bar{x}_1 - \bar{x}_2 = d(x_1, x_2). $$

This implies

$$ (2.68) \qquad \int_{t}^{t_*} \left[ \frac{|\alpha_1(s)|^2}{2} - \frac{|\alpha_2(s)|^2}{2} + L(y_1(s), s) - L(y_2(s), s) \right] ds \le L_\ell T d(x_1, x_2). $$

Let us argue differently according to the cases in the definition of $t_*$.

**Subcase 3-a:** $t_* = T$. Arguing as in Subcase 2-a and using (2.67)-(2.68) leads to the desired result.

**Subcase 3-b:** $t_* = \min\{s \in [t, T] : y_2(s) = O\} < T$. Combining the conclusions in **Case 2** and (2.67)-(2.68) leads to the desired result.

**Subcase 3-c:** $t_* = \min\{s \in [t, T] : y_1(s) = y_2(s)\} < T$. The conclusion follows by setting $(y_1, \alpha_1) = (y_1, \alpha_1)$ on $(t_*, T]$.

**Case 4:** $x_1, x_2 \in J_i$ with $0 < \bar{x}_1 < \bar{x}_2 \le 2C(T-t)^{1/2}$. Consider $(y_2, \alpha_2) \in \Gamma_t^{\text{opt}}[x_2]$ and the trajectory $(y_1, \alpha_1)$ such that $\alpha_1(s) = |\alpha_2(s)|e_i$ on $[t, t_*]$, where

$$ t_* = \min\{T, \min\{s \in [t, T] : y_1(s) = y_2(s)\}\}. $$

Note that, in $[t, t_*]$, $\alpha_1(s) \cdot e_i \ge 0$. Hence, $y_2$ cannot hit the vertex O before crossing $y_1$. For $s \in [t, t_*]$ and

$$ (2.69) \qquad d(y_1(s), y_2(s)) = x_2 + \int_{t}^{s} \alpha_2(\tau)d\tau - x_1 - \int_{t}^{s} |\alpha_2(\tau)|d\tau \le x_2 - x_1 = d(x_1, x_2). $$

This implies

$$ (2.70) \qquad \int_{t}^{t_*} \left[ \frac{|\alpha_1(s)|^2}{2} - \frac{|\alpha_2(s)|^2}{2} + L(y_1(s), s) - L(y_2(s), s) \right] ds \le L_\ell T d(x_1, x_2). $$

The arguments differ according to the cases in the definition of $t_*$.

**Subcase 4-a:** $t_* = T$. Arguing as in Subcase 2-a and using by (2.69)-(2.70) yields the desired result.

**Subcase 4-b:** $t_* = \min\{s \in [t, T] : y_1(s) = y_2(s)\}$. The result follows from the same arguments as in Subcase 3-c using (2.69)-(2.70). The proof is complete. $\square$
## 2.7 Local semi-concavity of the value function away from the vertex

Here, we wish to prove that the value function $u$ is semi-concave with respect to $x$ with a linear modulus of semi-concavity, locally in $J_i \setminus \{O\}$ and for $t$ bounded away from the horizon $T$. For the definition of semi-concavity and the main related properties, we refer the reader to the monograph [19].

**Proposition 2.31.** We keep the assumptions of Theorem 2.21 and assume furthermore that for all $i$ and $s$, $\ell_i(\cdot, s) \in C^{1,1}(J_i)$ and that $\|\partial_{xx}^2 \ell_i\|_{L^\infty(J_i \times [0,T])} < \infty$. Consider $t \in [0, T)$ and $x, y \in J_i \setminus \{O\}$ with $0 < r \le |x|, |y| \le R$. Under the same assumptions as in Theorem 2.21, there exists a constant $C$ (depending on $r, R$ and on $T-t$) such that

$$ \lambda u(x,t) + (1-\lambda)u(y,t) - u(\lambda x + (1-\lambda), t) \le C\lambda(1-\lambda)|x-y|^2, \quad \forall \lambda \in [0,1]. $$

The main technical part of the proof of Proposition 2.31 makes use of the following lemma. Recall that by (2.26) $\alpha(t)$ is well defined.

**Lemma 2.32.** Consider $x \in J_i \setminus \{O\}$ for some $i = 1, \dots, N$, $t \in [0, T)$ and $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$. Set $x = \bar{x}_{e_i}$. Under the same assumptions as in Proposition 2.31, there exists a constant $C$ (depending on $|\bar{x}|$ and on $T-t$) such that

$$ (2.71) \qquad u(x+h,t) - u(x,t) + \alpha(t) \cdot h \le C|h|^2 $$

for any $h = \bar{h}_{e_i}$ with $|\bar{h}|$ sufficiently small.

*Proof of Proposition 2.31.* Our arguments are reminiscent of [18, Corollary 3.2]. Lemma 2.32 implies that there exists a constant $C$ (depending on $r, R$ and on $T-t$) such that

$$ \frac{1}{2}u(x+h,t) + \frac{1}{2}u(x-h,t) - u(x,t) \le C|h|^2, \quad \forall h, |h| \le |x|. $$

Since $u$ is continuous (see Proposition 2.14), the latter inequality is equivalent to (2.71), see [19, Theorem 2.1.10]. $\square$

*Proof of Lemma 2.32.* The arguments are reminiscent of the proof of [18, Lemma 3.1]. Consider $t, x, (y, \alpha)$ as in the statement.

Take $h = \bar{h}_{e_i}$ with $|h| < \bar{x}/2$, and set

$$ t_* = \left( \frac{T-t}{2} \right) \wedge \left( \frac{\bar{x}}{2V} \right), $$

where $V$ is the constant associated to $G_{\bar{x}}$, see (2.60). Consider the trajectory $(y_h, \alpha_h)$ starting at $x+h$ at time $t$ with the control

$$ (2.72) \qquad \alpha_h(s) = \alpha(s) - h/t_* \quad \text{for } s \in [t, t+t_*], \quad \text{and} \quad \alpha_h(s) = \alpha(s) \quad \text{for } s \in [t+t_*, T]. $$

One easily checks that

$$ (2.73) \qquad y_h(s) = y(s) + h \frac{t_* - s + t}{t_*} \quad \text{for } s \in [t, t + t_*], \quad \text{and} \quad y_h(s) = y(s) \quad \text{for } s \in [t + t_*, T], $$

and that $y_h(s) \in J_i \setminus \{O\}$ for all $s \in [t, T]$. Therefore,

$$ u(x+h,t) - u(x,t) \le \int_t^{t+h} \left( \frac{|\alpha_h(s)|^2 - |\alpha(s)|^2}{2} + \ell_i(y_h(s),s) - \ell_i(y(s),s) \right) ds. $$
On the other hand, since $\alpha|_{[t,t+t_*]} \in W^{1,\infty}$,

$$
\begin{align*}
\alpha(t) \cdot h &= - \int_t^{t+t_*} \frac{d}{ds} [\alpha(s) \cdot (y_h(s) - y(s))] ds \\
&= - \int_t^{t+t_*} [\alpha'(s) \cdot (y_h(s) - y(s)) + \alpha(s) \cdot (\alpha_h(s) - \alpha(s))] ds \\
&= - \int_t^{t+t_*} [\partial_x \ell_i(y(s), s) e_i \cdot (y_h(s) - y(s)) + \alpha(s) \cdot (\alpha_h(s) - \alpha(s))] ds,
\end{align*}
$$

where the latter identity is due to Euler-Lagrange condition (2.26).

Combining the latter two observations leads to

$$
u(x+h,t) - u(x,t) + \alpha(t) \cdot h \le \int_t^{t+t_*} \frac{|\alpha_h(s) - \alpha(s)|^2}{2} ds + \int_t^{t+t_*} \ell_i(y_h(s),s) - \ell_i(y(s),s) ds \\
\qquad - \int_t^{t+t_*} \partial_x \ell_i(y(s),s) e_i \cdot (y_h(s) - y(s)) ds.
$$

In what follows, $C$ is a constant which may change from line to line and depends only on $\bar{x}$ and $T-t$. The regularity of $\ell_i$ implies

$$
\begin{align*}
u(x+h,t) - u(x,t) + \alpha(t) \cdot h &\le \int_t^{t+t_*} \left( \frac{|\alpha_h(s) - \alpha(s)|^2}{2} + \frac{\|\partial_{xx}^2 \ell_i\|_{\infty}}{2} |y_h(s) - y(s)|^2 \right) ds \\
(2.74) \qquad &\le C \|y_h - y\|_{W^{1,2}([t,t+t_*], \mathcal{G})}^2 \\
&\le C|h|^2,
\end{align*}
$$

the last line being obtained thanks to (2.72) and (2.73). The desired inequality is proved.

<figure><img src="image_4.png" /></figure>

2.8 Regularity of u along optimal trajectories and optimal synthesis

Here, we investigate some regularity properties of $u$ in the interiors of the edges. The following lemma is reminiscent of [20, Lemma 4.9].

**Lemma 2.33.** Consider $t \in [0, T)$, $x \in J_i \setminus \{O\}$ for some $i = 1, \dots, N$, $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$ and set

$$
t_* = T \wedge \min\{\tau \in [t, T] : y(\tau) = O\}.
$$

Under the same assumptions as in Proposition 2.31, the following properties hold:

(i) For any $s \in (t, t_*)$, $\alpha|_{(s,t_*)}$ is the unique optimal control for $u(y(s), s)$ up to time $t_*$. In other words for any $(y_1, \alpha_1) \in \Gamma_s^{\text{opt}}[y(s)]$, $\alpha_1$ coincides with $\alpha$ in $(s, t_*)$

(ii) $\partial_x u(x, t)$ exists if and only if the set

$$
\mathcal{A}(x) = \{ \alpha(t) : (y, \alpha) \in \Gamma_t^{\text{opt}}[x] \}
$$

is as singleton. Moreover, in this case, $\mathcal{A} = \{-\partial_x u(x,t)e_i\}$.

(iii) For any $s \in (t, t_*)$, the function $u(\cdot, s)$ is differentiable at $y(s)$ with $\partial_x u(y(s), s)e_i = -\alpha(s)$.
Proof. (i). The arguments are similar to the proof of [20, Lemma 4.9-(1)], so we refer the reader to that paper for the details and focus only on the main new aspects.

For any $s \in (t, t_*)$, consider $(y_1, \alpha_1) \in \Gamma_t^{\text{opt}}[y(s)]$ and set $t_{*,1} = T \wedge \min\{\tau \in [t, T] : y_1(\tau) = O\}$. For $0 < h < (s-t) \wedge (t_* \wedge t_{*,1} - s)$, we consider the following control

$$ \alpha_h(\tau) = \begin{cases} \frac{\alpha(\tau)}{y_1(s+h)-y(s-h)} & \text{if } \tau \in [t, s-h] \\ \frac{\alpha_1(\tau)}{2h} & \text{if } \tau \in (s-h, s+h) \\ \alpha_1(\tau) & \text{if } \tau \in [s+h, T] \end{cases} $$

and the corresponding trajectory $(y_h, \alpha_h)$ which is admissible for $u(x, t)$, from the choice of $h$. Let $(y_0, \alpha_0)$ stand for the concatenation of $(y, \alpha)$ and $(y_1, \alpha_1)$ at time $s$. From Remark 2.9, $(y_0, \alpha_0) \in \Gamma_t^{\text{opt}}[x]$. Comparing the costs associated $(y_0, \alpha_0)$ and to $(y_h, \alpha_h)$ and letting $h$ tend to 0 permits to prove that $\alpha(s) = \alpha_1(s)$, see [20]. Then, from Lemma 2.17, $y(\cdot)$ and $y_1(\cdot)$ satisfy the same second order differential equation with the same initial conditions: $y(s) = y_1(s)$ and $y'(s) = \alpha(s) = \alpha_1(s) = y'_1(s)$. Therefore, $y(\tau) = y_1(\tau)$ and $\alpha(\tau) = \alpha_1(\tau)$ for $\tau \in (s, t_*)$, and $t_* = t_{*,1}$.

(ii). Assume that $\partial_x u(x, t)$ exists. We wish to prove that $\mathcal{A}(x)$ is a singleton. Let $(y, \alpha)$ belong to $\Gamma_t^{\text{opt}}[x]$. By the local semi-concavity of $u$, see Lemma 2.32,

$$ u(x+h, t) - u(x, t) + \alpha(t)h \le Ch^2 \quad \text{for } h \text{ sufficiently small.} $$

Then, from [19, Proposition 3.3.4], we infer: $-\bar{\alpha}(t) \in D^+u(x, t)$. Moreover, since $u(\cdot, t)$ is differentiable at $x$, $D^+u(x, t)$ is a singleton. Hence, $\mathcal{A}(x)$ is the singleton $\{-\partial_x u(x, t)e_i\}$.

Conversely, assume that $\mathcal{A}(x)$ is a singleton. We wish to prove that $u$ is differentiable at $(x, t)$. To this end, we claim that, if $p \in D^*u(x, t)$, then the unique solution to

$$ (2.75) \qquad \xi''(\tau) = \partial_x \ell_i(\xi(\tau), \tau)e_i, \quad \xi(t) = x, \quad \xi'(t) = -pe_i $$

is such that there exists $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$ with $y(\tau) = \xi(\tau)$ for $0 \le \tau \le t_{*,\xi} := T \wedge \min\{\tau \in [t, T] : \xi(\tau) = O\}$.

Before proving the claim, let us first see how to use this intermediate result to conclude: since $\mathcal{A}(x)$ is a singleton, if the claim is true, then also $D^*u(x, t)$ is a singleton and it coincides with $\mathcal{A}(x)$. Then [19, Proposition 3.3.4] yields that $u$ is differentiable at $(x, t)$ with $\partial u(x, t)e_i = -\alpha(t)$ for every $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$ and the proof of (ii) is complete. There remains to prove the claim above: since $p \in D^*u(x, t)$, there exists a sequence $\{x_n\}_{n \in \mathbb{N}}$ with $x_n \to x$ and $\partial_x u(x_n, t) \to p$ as $n \to \infty$. Consider the unique solution to

$$ (2.76) \qquad \xi_n''(\tau) = \partial_x \ell_i(\xi_n(\tau), \tau)e_i, \quad \xi_n(t) = x_n, \quad \xi'_n(t) = -\partial_x u(x_n, t)e_i. $$

Since $u$ is differentiable at $(x_n, t)$, we have already proved that $\mathcal{A}(x_n)$ is the singleton $\{-\partial_x u(x_n, t)e_i\}$. On the other hand, from Lemma 2.17, any trajectory $(y_n, \alpha_n) \in \Gamma_t^{\text{opt}}[x_n]$ satisfies (2.76) on $[t, t_{*,n})$ where $t_{*,n} = T \wedge \min\{\tau \in [t, T] : y_n(\tau) = O\}$. Observe now that, from Theorem 2.21, there exists $t_{*,\min} > t$ such that $t_{*,n} \ge t_{*,\min}$ for any $n$. Hence, $y_n(\tau) = \xi_n(\tau)$ for $\tau \in [t, t_{*,\min}]$. From the uniform Lipschitz continuity of optimal trajectories (see Theorem 2.21), we deduce that $y_n$ uniformly converges to $y$ as $n \to \infty$. Next, Proposition 2.25 ensures that there exists a measurable function $\alpha$ such that $(y, \alpha) \in \Gamma_t^{\text{opt}}[x]$. Passing to the limit in (2.76), we infer that $y(\tau) = \xi(\tau)$ in $[t, t_{*,\min}]$. The claim is proved.

(iii). It is enough to combine the previous two statements (see also [20, Remark 4.10]). □
**Corollary 2.34.** Consider two optimal trajectories $\gamma_i \in \Gamma^{\text{opt}}[x_i]$ such that $\gamma_1(t) = \gamma_2(t) \in J_k \setminus \{O\}$ for some $t \in (0, T)$. Let $I_i$, $i = 1, 2$, be the largest open interval containing $t$ such that $\gamma_i(s) \in J_k \setminus \{O\}$ for $s \in I_i$. Under the same assumptions as in Proposition 2.31, $I_1 = I_2$.

*Proof.* There exists $\delta > 0$ such that both $\gamma_1(s)$ and $\gamma_2(s)$ lie in $J_k \setminus \{O\}$ for $s \in (t-\delta, t+\delta)$. Let us prove first that $\gamma_1$ and $\gamma_2$ coincide in $(t, t+\delta)$. For that, let $\gamma_3$ be the concatenation of $\gamma_{1|_{[0,t]}}$ and $\gamma_{2|_{[t,T]}}$. From Lemma 2.33-(i), $\gamma_1 = \gamma_3$ in $(t-\delta, t+\delta)$. This implies that $\gamma_1 = \gamma_2$ in $(t, t+\delta)$.

As a second step, from the latter result and Euler-Lagrange optimality condition, we deduce that $\gamma_1$ and $\gamma_2$ coincide also in $(t-\delta, t)$.

By a standard connectivity argument, $I_1 = I_2$ and $\gamma_1$ and $\gamma_2$ coincide in this interval. $\square$

We now tackle the counterpart of [20, Lemma 4.11] on optimal synthesis in the time interval in which the trajectory remains in the interior of a given edge. We first need the following definition:

**Definition 2.1.** Consider $(x, t) \in \mathcal{G} \times [0, T]$ and $t_1 \in (t, T)$. We say that the trajectory $(y, \alpha) \in \Gamma_{t,t_1}[x]$ is optimal for $u(x, t)$ on the interval $(t, t_1)$ if there exists $(\tilde{y}, \tilde{\alpha}) \in \Gamma_t^{\text{opt}}[x]$ with $(y, \alpha) = (\tilde{y}, \tilde{\alpha})$ on $(t, t_1)$.

**Lemma 2.35.** The assumptions are the same as in Proposition 2.31. Consider $t \in [0, T)$, $x \in J_i \setminus \{O\}$ for some $i = 1, \dots, N$.

If $u(\cdot, t)$ is differentiable at $x$, then there is a unique $t_* \in (t, T]$ and a unique $y$ such that

$$ (2.77) \qquad y'(s) = -\partial_x u(y(s), s) \quad \text{a.e. in } (t, t_*), \quad \text{and} \quad y(t) = x, $$

and $t_* = T \wedge \min\{\tau \in [t, T] : y(\tau) = O\}$.

The trajectory $(y, y')$ is optimal for $u(x, t)$ on the interval $(t, t_*)$ in the sense of Definition 2.1.

*Proof.* The first part of the statement is a consequence of Lemma 2.33-(ii) and -(iii).

The arguments for proving the optimality of $y$ on $(t, t_*)$ are reminiscent of [20, Lemma 4.11]. Hence, we focus on the main new aspects and refer the reader to [20] for the details. From Proposition 2.27, $u$ is Lipschitz continuous on each interval $[t, t_1] \subset [t, T)$. Hence, also $y$ is Lipschitz continuous on $[t, t_1]$.

The same arguments as in the proof of [20, Lemma 4.11] yield

$$ \frac{d}{ds}u(y(s), s) = -\frac{1}{2}|y'(s)|^2 - \ell_i(y(s), s), $$

for a.a. $s \in (t, t_*)$. Integrating this inequality on $(t, t_*)$ leads to

$$ u(x,t) = u(x,t) + \int_{t}^{t_{*}} \left( \frac{|y'(s)|^2}{2} + \ell_{i}(y(s), s) \right) ds. $$

From Remark 2.12, we infer that $(y, y')$ is optimal on $(t, t_*)$. $\square$
## 2.9 The PDE satisfied by $u$ on $G \times [0, T)$

The aim of this paragraph is to prove that the value function is the unique viscosity solution (in a suitable sense that will defined) of Hamilton-Jacobi equations in the network, with a suitable transmission condition at the origin.

### 2.9.1 Relaxed controls

To start with, let us recall the definition of the relaxed controls introduced in [4]. They will be used to construct the Hamiltonians involved in the Hamilton-Jacobi equations on $G \times [0, T)$. For $x \in J_i$, $i = 1, \dots, N$, set

$$FL_i(x,t) = \overline{\text{co}}\{(a, a^2/2) : a \in \mathbb{R}\}, \quad \text{and} \quad FL_i^{\perp}(x,t) = FL_i(x,t) \cap \{(\zeta, \xi) \in \mathbb{R}^2 : \zeta \ge 0\}.$$

Here, the notation co stands is used for the convex hull. It can be easily checked that

$$FL_i(x,t) = \{(\zeta, \xi) \in \mathbb{R}^2 : \zeta \ge \zeta^2/2\}, \quad \text{and} \quad FL_i^{\perp}(x,t) := \{(\zeta, \xi) \in \mathbb{R}^2 : \zeta \ge 0, \zeta \ge \zeta^2/2\}.$$

For $x = O$, set

$$FL(O,t) = \bigcup_{i=1}^{N} FL_i^{\perp}(O,t).$$

### 2.9.2 Hamiltonians

For $x \in J_i$, $i = 1, \dots, N$, $t \in [0, T]$, $\bar{p} \in \mathbb{R}$, $p = (p_1, \dots, p_N \in \mathbb{R}^N$, set

$$H_i(x, t, \bar{p}) = \sup_{(\zeta, \xi) \in FL_i(x,t)} \{-\bar{p}\zeta - \xi - \ell_i(x,t)\},$$

$$H_i^{\perp}(x, t, \bar{p}) = \sup_{(\zeta, \xi) \in FL_i^{\perp}(x,t)} \{-\bar{p}\zeta - \xi - \ell_i(x,t)\},$$

$$H_O(t,p) = \max\{-\ell_*(t), \max_{i=1,...,N} \{-\ell_i(O,t)\}, \max_{i=1,...,N} \{H_i^{\perp}(O,t,p_i)\}\} \\ = \max\{-\ell_O(t), \max_{i=1,...,N} \{H_i^{\perp}(O,t,p_i)\}\}.$$

Elementary calculus yields

$$(2.78) \quad H_i(x, t, \bar{p}) = \sup_{a \in \mathbb{R}} \left\{ -\bar{p}a - \frac{|a|^2}{2} - \ell_i(x, t) \right\} = \frac{|p|^2}{2} - \ell_i(x, t) \quad \forall x \in J_i,$$

$$(2.79) \quad H_i^{\perp}(O, t, \bar{p}) = \max_{\bar{\alpha} \ge 0} \{-\bar{\alpha}\bar{p} - \ell_i(O, t) - |\bar{\alpha}|^2/2\} = \begin{cases} \frac{|\bar{p}|^2}{2} - \ell_i(O, t) & \text{if } \bar{p} \le 0, \\ -\ell_i(O, t) & \text{if } \bar{p} > 0. \end{cases}$$

### 2.9.3 Hamilton-Jacobi equations on $G \times [0, T)$

We are interested in the system of first-order PDEs on $G \times (0, T)$:

$$(2.80) \quad \begin{cases} -\partial_t u + H_i(x, t, Du) = 0, & \text{if } x \in J_i \setminus \{O\}, \\ -\partial_t u + H_O(t, Du) = 0, & \text{if } x = O, \end{cases}$$

where $Du(x, t)$ is defined in (1.1) and is a 1-dimensional (resp. $N$-dimensional) object if $x \in J_i \setminus \{O\}$ (resp. $x = O$).
### 2.9.4 Viscosity solution of (2.80)

**Definition 2.2.** A function $u \in C(\mathcal{G} \times (0, T))$ is a viscosity subsolution (resp. supersolution) of (2.80) if for every function $\varphi \in C^1(\mathcal{G} \times (0, T))$ touching $u$ from above (resp. below) at $(x, t) \in \mathcal{G} \times (0, T)$, there holds

$$ 
\begin{array}{lll}
-\partial_t \varphi(x,t) + H_i(x,t, D\varphi) & \leq 0 & (\text{resp. } \geq 0) \quad \text{if } x \in J_i \setminus \{O\}, \\
-\partial_t \varphi + H_O(t, D\varphi) & \leq 0 & (\text{resp. } \geq 0) \quad \text{if } x = O.
\end{array}
$$

A function $u \in C(\mathcal{G} \times (0, T))$ is a viscosity solution of (2.80) if it is both a viscosity subsolution and a viscosity supersolution of (2.80).

### 2.9.5 Main result

**Theorem 2.36.** Under assumptions [H0] and [H1], the value function $u$ defined in (2.9) is a viscosity solution of (2.80) in $\mathcal{G} \times (0, T)$. Moreover, for all $x \in \mathcal{G}$, $t \mapsto u(x, t)$ is continuous in $[0, T]$ and $u(x, T) = g(x)$.

*Proof.* We borrow some arguments from the proof of [23, Theorem 6.4]. Clearly, the standard theory on viscosity solutions can be applied in $\mathcal{G} \setminus \{O\}$, so it suffices to focus on the origin $O$.

**Step 1:** $u$ is a supersolution at $O$. Let $\varphi \in C^1(\mathcal{G} \times [0, T])$ be a function touching $u$ from below at $(O, \bar{t})$, for some $\bar{t} \in (0, T)$. Without loss of generality, since $u$ is bounded, we may assume that $u - \varphi$ achieves a global minimum at $(O, \bar{t})$ with value 0, i.e. $\varphi(x, t) \leq u(x, t) \quad \forall (x, t) \in \mathcal{G} \times [0, T]$ and $\varphi(O, \bar{t}) = u(O, \bar{t})$. Let $(y, \alpha) \in \Gamma_{\bar{t}}[O]$ be an optimal trajectory for $u(O, \bar{t})$. The Dynamic Programming Principle in Proposition 3.12-(i) and Remark 2.7 ensure

$$ u(O, \bar{t}) = u(y(s), s) + \int_{\bar{t}}^{s} \left[ L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right] d\tau, \quad \forall s \in [\bar{t}, T], $$

which entails

$$ \varphi(y(s), s) - \varphi(O, \bar{t}) + \int_{\bar{t}}^{s} \left[ L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right] d\tau \leq 0, \quad \forall s \in [\bar{t}, T]. $$

With the same arguments as in [23, Theorem 6.4 (proof)], we deduce (2.81)

$$ \int_{\bar{t}}^{s} \left[ \partial_t \varphi(y(\tau), \tau) + D\varphi(y(\tau), \tau) \cdot \alpha(\tau) + L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right] d\tau \leq 0, \quad \forall s \in [\bar{t}, T], $$

setting $D\varphi(\tau, y(\tau)) \cdot \alpha(\tau) = 0$ for a.a. $\tau \in \{\tau \in [\bar{t}, T] : y(\tau) = O\} =: \mathcal{T}_0$, which makes sense because from Stampacchia theorem, $\alpha(\tau) = 0$ for a.a. $\tau \in \mathcal{T}_0$.

From the uniform bound of the optimal control in $L^2$, see Remark 2.6, there holds

$$ d(y(\tau), O) \leq \int_{\bar{t}}^{\tau} |\alpha(s)| ds \leq C(\tau - \bar{t})^{1/2}, \quad \forall \tau \in [\bar{t}, T]. $$

Hence, from the regularity of $\varphi$, there exists a constant $K$ such that, for $\psi = \varphi, \partial_t \varphi, D\varphi$,

$$ (2.82) \qquad |\psi(y(\tau), \tau) - \psi(O, \bar{t})| \leq K(\tau - \bar{t})^{1/2}, \quad \forall \tau \in [\bar{t}, T]. $$
It is convenient to introduce the following sets of times:

$\mathcal{T}_0^s = \{\tau \in (\bar{t}, s) : y(\tau) = O\}$ and $\mathcal{T}_i^s = \{\tau \in (\bar{t}, s) : y(\tau) \in J_i \setminus \{O\}\}$ for $i = 1, \dots, N$.

Note that $\mathcal{T}_0^s$ is closed while if $i > 0$, then $\mathcal{T}_i^s$ is open, and that $(\bar{t}, s) = \bigcup_{i=0}^N \mathcal{T}_i^s$. Hence (2.81) becomes:

$$ (2.83) \qquad \sum_{i=0}^{N} \int_{\mathcal{T}_i^s} \xi(\tau) d\tau \le 0, \quad \forall s \in [\bar{t}, T], $$

where

$$ \xi(\tau) = \partial_t \varphi(y(\tau), \tau) + D\varphi(y(\tau), \tau) \cdot \alpha(\tau) + L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2}. $$

In (2.83), let us address separately the terms corresponding to $i = 0$ and $i = 1, \dots, N$.

Consider $i = 0$ first. From Stampacchia theorem, $\alpha(\tau) = 0$ and $L(y(\tau), \tau) = \ell_O(\tau)$ for a.a. $\tau \in \mathcal{T}_0^s$. Hence,

$$ \int_{\mathcal{T}_0^s} \xi(\tau) d\tau = \int_{\mathcal{T}_0^s} [\partial_t \varphi(O, \tau) + \ell_O(\tau)] d\tau \ge \int_{\mathcal{T}_0^s} [\partial_t \varphi(O, \bar{t}) + \ell_O(\bar{t})] d\tau - (s - \bar{t})\omega(s - \bar{t}), $$

where the inequality is due to (2.82) and to the continuity of $\ell_O$, and where $\omega$ is a modulus of continuity depending on the constant $K$ in (2.82) and on the modulus of continuity of $\ell_O$. On the other hand, the definition of $H_O$ guarantees

$$ \ell_O(\bar{t}) \ge -H_O(\bar{t}, p) \quad \forall p \in \mathbb{R}^N. $$

The latter two observations imply that

$$ (2.84) \qquad \int_{\mathcal{T}_0^s} \xi(\tau) d\tau \ge |\mathcal{T}_0^s| (\partial_t \varphi(O, \bar{t}) - H_O(\bar{t}, D\varphi(\bar{t}, O))) - (s - \bar{t})\omega(s - \bar{t}). $$

Consider now $i \in \{1, \dots, N\}$. For a.a. $\tau \in \mathcal{T}_i^s$, the control $\alpha(\tau)$ has the form $\alpha(\tau) = \bar{\alpha}(\tau)e_i$ with $\bar{\alpha}(\tau) \in \mathbb{R}$. From (2.82), Remark 2.6 and the continuity of $\ell_i$, there exists a modulus of continuity $\omega$ such that

$$ \begin{align} \int_{\mathcal{T}_i^s} \xi(\tau) d\tau &= \int_{\mathcal{T}_i^s} \left[ \partial_t \varphi(y(\tau), \tau) + D\varphi|_{J_i}(y(\tau), \tau)\bar{\alpha}(\tau) + \ell_i(y(\tau), \tau) + \frac{|\bar{\alpha}(\tau)|^2}{2} \right] d\tau \tag{2.85} \\ &\ge \int_{\mathcal{T}_i^s} \left[ \partial_t \varphi(O, \bar{t}) + D\varphi|_{J_i}(O, \bar{t})\bar{\alpha}(\tau) + \ell_i(O, \bar{t}) + \frac{|\bar{\alpha}(\tau)|^2}{2} \right] d\tau - (s - \bar{t})\omega(s - \bar{t}). \end{align} $$

Thanks to the convexity of the set $FL_i$, the same arguments as those in [23, eq.(6.22)] (as a matter of fact, it is enough to use Jensen inequality in the present case), lead to the existence of $(\zeta_i, \xi_i) \in FL_i(O, \bar{t})$ such that

$$ \int_{\mathcal{T}_i^s} D\varphi|_{J_i}(O, \bar{t})\bar{\alpha}(\tau) d\tau = D\varphi|_{J_i}(O, \bar{t}) \int_{\mathcal{T}_i^s} \bar{\alpha}(\tau) d\tau = |\mathcal{T}_i^s| D\varphi|_{J_i}(O, \bar{t})\zeta_i, $$

$$ \int_{\mathcal{T}_i^s} |\bar{\alpha}(\tau)|^2/2 d\tau = |\mathcal{T}_i^s|\xi_i. $$
Note that the path $y(\bar{t}) = O$ and that during the interval $(\bar{t}, s)$ may enter and exit several edges. However, if $y(s) \in J_i \setminus \{O\}$ for $s \in (t_1, t_2)$ and $y(t_1) = y(t_2) = O$, then, there holds

$$ \int_{t_1}^{t_2} \bar{\alpha}(\tau) d\tau = 0, $$

and consequently

$$ \int_{\mathcal{T}_i^s} \bar{\alpha}(\tau) d\tau = \begin{cases} y(s) & \text{if } y(s) \in J_i, \\ 0 & \text{otherwise,} \end{cases} $$

which implies that $\zeta_i \ge 0$. Therefore,

$$ \begin{aligned} \int_{\mathcal{T}_i^s} \left[ D\varphi|_{J_i}(O, \bar{t})\bar{\alpha}(\tau) + \ell_i(O, \bar{t}) + \frac{|\bar{\alpha}(\tau)|^2}{2} \right] d\tau &= |\mathcal{T}_i^s| \left[ D\varphi|_{J_i}(O, \bar{t})\zeta_i + \ell_i(O, \bar{t}) + \xi_i \right] \\ &\ge -|\mathcal{T}_i^s| H_i^{\downarrow}(O, \bar{t}, D\varphi|_{J_i}(O, \bar{t})). \end{aligned} $$

The latter inequality and (2.85) yield

$$ \begin{aligned} \int_{\mathcal{T}_i^s} \xi(\tau) d\tau &\ge |\mathcal{T}_i^s| \left[ \partial_t \varphi(O, \bar{t}) - H_i^{\downarrow}(O, \bar{t}, D\varphi|_{J_i}(O, \bar{t})) \right] - (s - \bar{t})\omega(s - \bar{t}) \\ &\ge |\mathcal{T}_i^s| \left[ \partial_t \varphi(O, \bar{t}) - H_O(\bar{t}, D\varphi(O, \bar{t})) \right] - (s - \bar{t})\omega(s - \bar{t}). \end{aligned} $$

This, (2.84) and (2.83) then imply that

$$ \begin{aligned} (N+1)(s-\bar{t})\omega(s-\bar{t}) &\ge \left(\sum_{i=0}^{N} |\mathcal{T}_i^s|\right) \left[ \partial_t \varphi(O, \bar{t}) - H_O(\bar{t}, D\varphi(O, \bar{t})) \right] \\ &\ge (s-\bar{t}) \left[ \partial_t \varphi(O, \bar{t}) - H_O(\bar{t}, D\varphi(O, \bar{t})) \right], \end{aligned} $$

the last line is obtained because $(\bar{t}, s) = \cup_{i=0}^N \mathcal{T}_i^s$ and $\mathcal{T}_i^s \cap \mathcal{T}_j^s = \emptyset$ for $i \ne j$. Dividing t by $(s - \bar{t})$ and letting s tend to $\bar{t}^+$ yield

$$ -\partial_t \varphi(O, \bar{t}) + H_O(\bar{t}, D\varphi(O, \bar{t})) \ge 0, $$

i.e. the desired inequality.

**Step 2:** u is a subsolution at O. Let $\varphi \in C^1(G \times [0, T])$ be a function touching u from above at $(O, \bar{t})$, for some $\bar{t} \in (0, T)$. As above, it may be assumed that $\varphi(x, t) \ge u(x, t) \quad \forall (x, t) \in G \times [0, T]$ and $\varphi(O, \bar{t}) = u(O, \bar{t})$. The Dynamic Programming Principle in Proposition 2.13 ensures that for any $s \in (\bar{t}, T)$ and any $(y, \alpha) \in \Gamma_{\bar{t},s}[O]$:

$$ u(O, \bar{t}) \le u(y(s), s) + \int_{\bar{t}}^{s} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau. $$

This implies that, for any $s \in (\bar{t}, T)$ and any $(y, \alpha) \in \Gamma_{\bar{t},s}[O]$,

$$ (2.86) \qquad \varphi(y(s), s) - \varphi(O, \bar{t}) + \int_{\bar{t}}^{s} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau \ge 0. $$

Note that (2.86) can be written

$$ (2.87) \qquad \varphi(y(s), s) - \varphi(O, \bar{t}) + \sum_{i=0}^{N} \int_{\mathcal{T}_i^s} \left( L(y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau \ge 0, $$
where the sets $\mathcal{T}_i^s$ are defined as in Step 1 and depend upon the trajectory $(y, \alpha)$. The arguments below will differ whether $(y, \alpha) \in \Gamma_{\bar{t},s}[O]$ remains at $O$ or enters in a given edge $J_i$.

Case (a): the trajectory remains at $O$. For any $s \in (\bar{t}, T]$, consider the trajectory $(y, \alpha) \in \Gamma_{\bar{t},s}[O]$ with $\alpha(\cdot) = 0$. Clearly, $y(\cdot) = O$ and $(\bar{t}, s) = \mathcal{T}_0^s$. Then (2.87) becomes

$$ \varphi(O, s) - \varphi(O, \bar{t}) + \int_{\bar{t}}^{s} \ell_O(\tau) d\tau \geq 0. $$

From the continuity of $\ell_O$ with respect to $t$,

$$ \varphi(O, s) - \varphi(O, \bar{t}) + (s - \bar{t})\ell_O(\bar{t}) \geq -(s - \bar{t})\omega(s - \bar{t}), $$

for some modulus of continuity $\omega$. Dividing by $(s - \bar{t})$, letting $s \to \bar{t}^+$ taking into account the regularity of $\varphi$ yield

$$ (2.88) \qquad \partial_t \varphi(O, \bar{t}) + \ell_O(\bar{t}) \geq 0. $$

Case (b): the trajectory enters in a given edge. Fix $i \in \{1, \dots, N\}$. For any $n \in \mathbb{N} \setminus \{0\}$, fix $\bar{a} \in (0, n)$. For any $s \in (\bar{t}, T]$, consider the trajectory $(y, \alpha) \in \Gamma_{\bar{t},s}[O]$ with $\alpha(\tau) = \bar{a}e_i$ for $\tau \in (\bar{t}, s)$. Clearly, $\alpha(\tau) \in A_i$ and $y(\tau) \in J_i \setminus \{O\}$ for $\tau \in (\bar{t}, s)$. Thus $(\bar{t}, s) = \mathcal{T}_i^s$. Note that here the unboudedness of $J_i$ is not essential. Indeed, if $J_i$ had a finite length $l_i$, then it would be enough to choose $s \le \bar{t} + l_i/\bar{a}$. By the same arguments as in Step 1 (see (2.81)), inequality (2.87) can be written

$$ \int_{\bar{t}}^{s} \left[ \partial_t \varphi(y(\tau), \tau) + D\varphi_{|J_i}(y(\tau), \tau)\bar{a} + \ell_i(y(\tau), \tau) + \frac{\bar{a}^2}{2} \right] d\tau \geq 0, \quad \forall s \in [\bar{t}, T]. $$

As in Step 1, taking into account Remark 2.6, estimate (2.82) and the uniform continuity of $\ell_i$ in any neighbourhood of $O$, we get

$$ \int_{\bar{t}}^{s} \left[ \partial_t \varphi(O, \bar{t}) + D\varphi_{|J_i}(O, \bar{t})\bar{a} + \ell_i(O, \bar{t}) + \frac{\bar{a}^2}{2} \right] d\tau \geq -(s - \bar{t})\omega(s - \bar{t}), \quad \forall s \in [\bar{t}, T] $$

for a suitable modulus of continuity $\omega$. Dividing the previous inequality by $(s - \bar{t})$ and letting $s \to \bar{t}^+$ yield

$$ \partial_t \varphi(O, \bar{t}) + D\varphi_{|J_i}(O, \bar{t})\bar{a} + \ell_i(O, \bar{t}) + \frac{\bar{a}^2}{2} \geq 0. $$

Since $\bar{a} \in (0, n)$ is arbitrary,

$$ \partial_t \varphi(O, \bar{t}) - \sup_{\bar{a} \in (0, n)} \left\{ -D\varphi_{|J_i}(O, \bar{t})\bar{a} - \ell_i(O, \bar{t}) - \frac{\bar{a}^2}{2} \right\} \geq 0, $$

and, since $n$ is arbitrary,

$$ \partial_t \varphi(O, \bar{t}) - \sup_{\bar{a} \geq 0} \left\{ -D\varphi_{|J_i}(O, \bar{t})\bar{a} - \ell_i(O, \bar{t}) - \frac{\bar{a}^2}{2} \right\} \geq 0. $$

Then (2.79) yields

$$ \partial_t \varphi(O, \bar{t}) - H_i^{\downarrow}(O, \bar{t}, D\varphi_{|J_i}(O, \bar{t})) \geq 0. $$
Since $i$ is arbitrary, and from inequality (2.88), we deduce

$$ \partial_t \varphi(O, \bar{t}) - \max \left\{ \max \left\{ -\ell_*(\bar{t}), \max_{i=1,\dots,N} \left\{ -\ell_i(O, \bar{t}) \right\} \right\}, \max_{i=1,\dots,N} \left\{ H_i^\downarrow(x, t, D\varphi|_{J_i}(O, \bar{t})) \right\} \right\} \ge 0, $$

i.e. the desired inequality.

<figure><img src="image_5.png" /></figure>

**Remark 2.37.** If $g \in C(\mathcal{G})$, then there is a unique viscosity solution $u$ of (2.80) such that $u \in C_b(\mathcal{G} \times [0, T])$ and $u(\cdot, T) = g$, see e.g. [23]. We have not found any relevant literature on uniqueness when $g$ is not continuous and plan to address this topic in a future work.

# 3 Relaxed Mean Field Games equilibria

We are now ready to tackle Mean Field Games. Relying on the results contained in Section 2, we prove that there exists a relaxed MFG equilibrium and study the related mild solutions.

## 3.1 Setting and notations

**Probability sets and evaluation map.** Let $\mathcal{P}(\mathcal{G})$ denote the set of Borel probability measures on $\mathcal{G}$ endowed with the narrow topology. Similarly, $\mathcal{P}(\Gamma)$ stands for the set of Borel probability measures on $\Gamma$. For $t \in [0, T]$, the evaluation map $e_t: \Gamma \to \mathcal{G}$ is defined by $e_t(y_x, \alpha) = y_x(t)$. For any $\mu \in \mathcal{P}(\Gamma)$ and $t \in [0, T]$, the Borel probability measure $m^\mu(t)$ on $\mathcal{G}$ is defined by $m^\mu(t) = e_t\#\mu$.

**Costs.** The running cost and the terminal cost depend on the distribution of the population. We consider the costs $L_i \in C(\mathcal{P}(\mathcal{G}); C_b(\mathcal{G} \times [0, T]))$, for $i = 1, \dots, N$, and $L_* \in C(\mathcal{P}(\mathcal{G}); C([0, T]))$. Similarly, let $G_i: \mathcal{P}(\mathcal{G}) \to C_b(\mathcal{G})$, $i = 1, \dots, N$, and $G_* : \mathcal{P}(\mathcal{G}) \to \mathbb{R}$ be continuous functions. The images of $m \in \mathcal{P}(\mathcal{G})$ by $L_i$, respectively by $G_i$ are denoted by $L_i[m](\cdot, \cdot)$, respectively $G_i[m](\cdot)$, and we introduce similar notations for $L_*$ and $G_*$. Let the real number $K$ be defined as follows:

$$ (H_1^{\text{MFG}}) \quad K = \max \left( \sup_{m \in \mathcal{P}(\mathcal{G})} \|L_*[m]\|_{L^\infty}, \max_{i=1,\dots,N} \sup_{m \in \mathcal{P}(\mathcal{G})} \|L_i[m]\|_{L^\infty}, \sup_{m \in \mathcal{P}(\mathcal{G})} \|G_*[m]\|_{L^\infty}, \max_{i=1,\dots,N} \sup_{m \in \mathcal{P}(\mathcal{G})} \|G_i[m]\|_{L^\infty} \right) \in \mathbb{R}^+. $$

For brevity, we write

$$ \begin{align} L[m](x,t) &= \sum_{i=1}^{N} L_i[m](x,t) \mathbb{1}_{x \in J_i \setminus \{O\}} + L_O[m](t) \mathbb{1}_{x=O}, \tag{3.1} \\ G[m](x) &= \sum_{i=1}^{N} G_i[m](x) \mathbb{1}_{x \in J_i \setminus \{O\}} + \min \left\{ G_*[m], \min_{i=1,\dots,N} G_i[m](O) \right\} \mathbb{1}_{x=O}, \end{align} $$

for $x \in \mathcal{G}$ and $t \in [0, T]$, where

$$ L_O[m](\tau) = \min \left\{ L_*[m](\tau), \min_{i=1,\dots,N} L_i[m](O, \tau) \right\}. $$
**Admissible paths.** Let us introduce the sets of admissible *paths*

$$ (3.2) \quad \tilde{\Gamma}_C[x] = \{y \in Y_{x,0} : d(y(s), O) \le C, \forall s \in [0, T], \|\dot{y}\|_2 \le C\}, \quad \tilde{\Gamma}_C = \bigcup_{x \in \mathcal{G}} \tilde{\Gamma}_C[x], $$

and endow $\tilde{\Gamma}_C$ with the topology of uniform convergence. Note that a *path* is the sole $y \in Y_{y(0),0}$ while a *trajectory* is formed by the couple $(y, \alpha) \in \Gamma$.

**Lemma 3.1.** For every positive constant $C$, the set $\tilde{\Gamma}_C$ is compact.

*Proof.* Fix $C > 0$ and consider a sequence $\{y_n\}_{n \in \mathbb{N}}$, with $y_n \in \tilde{\Gamma}_C$. Possibly for a sub-sequence (still denoted by $y_n$), $\{\dot{y}_n\}_n$ converges in the weak topology of $L^2([0, T], \mathbb{R}^d)$ to some $\alpha \in L^2([0, T], \mathbb{R}^d)$, with $\|\alpha\|_2 \le C$. Then, $\{y_n\}_n$ converges uniformly to some $y \in C([0, T], \mathcal{G})$. Clearly, $\alpha = \dot{y}$. The same arguments as in the proof of Proposition 2.11 yield that the path $y$ is admissible, i.e. $y \in Y_{y(0),0}$, and consequently that $y$ belongs to $\tilde{\Gamma}_C$. $\square$

**Lipschitz admissible paths.** Given two positive constants $V$ and $C$, let us introduce the sets of Lipschitz admissible *paths*

$$ (3.3) \quad \Gamma_{C,V}^{\text{Lip}}[x] = \{y \in \tilde{\Gamma}_C[x] : \|y'\|_{\infty} \le V\}, \quad \Gamma_{C,V}^{\text{Lip}} = \bigcup_{x \in \mathcal{G}} \Gamma_{C,V}^{\text{Lip}}[x], $$

and endow $\Gamma_{C,V}^{\text{Lip}}$ with the topology of uniform convergence. The same arguments as in Lemma 3.1 yield that $\Gamma_{C,V}^{\text{Lip}}$ is compact.

**The set $\mathcal{P}(\tilde{\Gamma}_C)$ and the associated costs.** Let $\mathcal{P}(\tilde{\Gamma}_C)$ denote the set of probability measures on $\tilde{\Gamma}_C$ endowed with the narrow topology. For $t \in [0, T]$, the evaluation map $e_t : \tilde{\Gamma}_C \to \mathcal{G}$ is defined by $e_t(y) = y(t)$. For any $\mu \in \mathcal{P}(\tilde{\Gamma}_C)$ and $t \in [0, T]$, define the Borel probability measure $m^\mu(t)$ on $\mathcal{G}$ by $m^\mu(t) = e_t^\mu \mu$. Clearly, $\text{supp}(m^\mu(t)) \subset \{x \in \mathcal{G} : d(x, O) \le C\}$. It is possible to prove that, if $\mu \in \mathcal{P}(\tilde{\Gamma}_C)$, then the map $t \mapsto m^\mu(t)$ belongs to $C^{1/2}([0, T], \mathcal{P}(\mathcal{G}))$, see Lemma 3.8 below. Hence, for all $(y, \alpha) \in \Gamma$, the functions $t \mapsto F_i[m^\mu(t)](y(t))$ are continuous and bounded by the constant $K$ introduced in $(H_1^{\text{MFG}})$. With $\mu \in \mathcal{P}(\tilde{\Gamma}_C)$ and $(y, \alpha) \in \Gamma[x]$, we associate the cost

$$ (3.4) \quad J^\mu(x; (y, \alpha)) = \int_0^T \left( L[m^\mu(\tau)](y(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau + G[m^\mu(T)](y(T)). $$

**Remark 3.2.** We recall that for each $y \in \tilde{\Gamma}_C[x]$ there exists $\alpha \in L^2([0, T], \mathbb{R}^d)$ such that $(y, \alpha) \in \Gamma[x]$, from Theorem 2.2 and Remark 2.3. Such a control $\alpha$ is unique for a.e. $t \in \{t \in [0, T] : y(t) \ne O\}$, which is not the case in $\{t \in [0, T] : y(t) = O\}$. However, the associated cost is independent of the choice of this control, namely: for any $y \in \tilde{\Gamma}_C[x]$, there holds

$$ J^\mu(x; (y, \alpha_1)) = J^\mu(x; (y, \alpha_2)) \quad \forall (y, \alpha_1), (y, \alpha_2) \in \Gamma[x]. $$

For every $y \in \tilde{\Gamma}_C[x]$, we define $\alpha_y$ the control such that $(y, \alpha_y) \in \Gamma[x]$ and $\alpha_y(t) = 0 \in A_0$ for a.e. $t \in \{t \in [0, T] : y(t) = O\}$. Note that this control is uniquely defined up to a set of null measure.
**Optimal trajectories.** Fix $\mu \in \mathcal{P}(\tilde{\Gamma}_C)$; for any $x \in \mathcal{G}$, let us set

$$ (3.5) \qquad \Gamma^{\mu, \text{opt}}[x] = \left\{ (y, \alpha) \in \Gamma[x] : J^{\mu}(x; (y, \alpha)) = \min_{(\tilde{y}, \tilde{\alpha}) \in \Gamma[x]} J^{\mu}(x; (\tilde{y}, \tilde{\alpha})) \right\} $$

where $J^\mu$ is defined in (3.4). Proposition 2.11 entails that for each $\mu \in \mathcal{P}(\tilde{\Gamma}_C)$ and $x \in \mathcal{G}$, the set $\Gamma^{\mu, \text{opt}}[x]$ of optimal trajectories starting from $x$ is not empty. We set $\Gamma^{\mu, \text{opt}} = \cup_{x \in \mathcal{G}} \Gamma^{\mu, \text{opt}}[x]$.

**Remark 3.3.** From assumption $(H_1^{\text{MFG}})$, there exists a positive constant $\tilde{C}$ such that, for every $\mu \in \mathcal{P}(\tilde{\Gamma}_C)$, $x \in \mathcal{G}$ and $(y, \alpha) \in \Gamma^{\mu, \text{opt}}[x]$, there holds $\|\alpha\|_2 \le \tilde{C}$. In particular, if $m_0 \in \mathcal{P}(\mathcal{G})$ has compact support, then for every $\mu \in \mathcal{P}(\tilde{\Gamma}_C)$, $x \in \text{supp}(m_0)$ and $(y, \alpha) \in \Gamma^{\mu, \text{opt}}[x]$, there holds $y \in \tilde{\Gamma}_{\tilde{C}}[x]$ (possibly after taking a larger value of the constant $\tilde{C}$).

**The set $\mathcal{P}_{m_0}(\tilde{\Gamma}_C)$.** We assume

$$ (H_2^{\text{MFG}}) \qquad m_0 \in \mathcal{P}(\mathcal{G}) \qquad \text{has compact support.} $$

Let $\mathcal{P}_{m_0}(\tilde{\Gamma}_C)$ denote the set of measures $\mu \in \mathcal{P}(\tilde{\Gamma}_C)$ such that $e_0\#\mu = m_0$. In general, $\mathcal{P}_{m_0}(\tilde{\Gamma}_C)$ may be empty. However, in the present framework, this is not the case:

**Lemma 3.4.** Under assumptions $(H_1^{\text{MFG}})$ and $(H_2^{\text{MFG}})$, for $C$ sufficiently large, $\mathcal{P}_{m_0}(\tilde{\Gamma}_C)$ is not empty.

*Proof.* The proof consists of adapting some arguments in [16, Remark 3.2]. For $C \ge \tilde{C}$ (where $\tilde{C}$ is the constant introduced in Remark 3.3), consider the map: $j: \text{supp}(m_0) \to \tilde{\Gamma}_C$, $j(x)(t) = x$ for any $t \in [0, T]$. Set $\tilde{m}_0 = m_{0|\text{supp}(m_0)}$, the restriction of $m_0$ to its support. Observe that $e_0\#(j\#\tilde{m}_0) = m_0$, hence $(j\#\tilde{m}_0) \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$. $\square$

**The set $\mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})$.** We assume $(H_2^{\text{MFG}})$. Let $\mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})$ denote the set of measures $\mu \in \mathcal{P}(\Gamma_{C,V}^{\text{Lip}})$ such that $e_0\#\mu = m_0$. Adapting the arguments in the proof of Lemma 3.4, we obtain that, for $C$ and $V$ sufficiently large, $\mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})$ is not empty.

Let us give an example, particularly simple because the agents do not interact, in which the distribution of states may develop a singularity.

**Example 3.1.** In a junction with two edges, consider the costs: $L_1[m] = -1$, $L_2[m] = 1$, $L_*[m] = -1$ and $G_i[m] = 0$ ($i = *, 1$) and $G_2[m] = m(J_2)$ for every $m \in \mathcal{P}(\mathcal{G})$. Assume that the initial distribution of states is uniform on $[0, 1/2]e_1 \cup [0, 1/2]e_2$. Fix $\bar{x} \in (0, 1/2]$. Let $(y, \alpha)$ be an optimal trajectory starting at $\bar{x}e_2$ at time $t=0$.

We claim that, for $T$ sufficiently large, $(y, \alpha)$ reaches $O$ at time $t_{\bar{x}} = \bar{x}/2$ and stops there. Indeed, either $y(\cdot) = \bar{x}e_2$ in $[0, T]$ (and the corresponding cost is equal to $T$) or there exists $s_1 \in (0, T]$ such that $s_1 = \min\{s \in [0, T] : y(s) = O\}$ because the other possibilities are less convenient. In the latter case, $y(\cdot) = O$ in $[s_1, T]$ is the optimal choice among all the trajectories $(\tilde{y}, \tilde{\alpha})$ such that $\tilde{\alpha}(s) = \alpha(s)$ if $s \in [0, s_1]$.

Then, from the Euler-Lagrange condition in Lemma 2.17, there holds $\alpha(\cdot) = -\bar{\alpha}e_2$ in $(0, s_1)$ for a constant $\bar{\alpha} > 0$. Hence, $s_1 = \bar{x}/\bar{\alpha}$. The resulting cost is

$$ \frac{\bar{x}\bar{\alpha}}{2} + \frac{2\bar{x}}{\bar{\alpha}} - T, $$
whose minimum w.r.t. $\bar{\alpha} \in (0, +\infty)$ is attained when $\bar{\alpha} = 2$. With this choice of $\bar{\alpha}$, the cost is $2\bar{x} - T$ which is the minimal one, provided that $T$ is sufficiently large. Our claim is completely proved.

Therefore, the distribution of agents develops a singularity at the vertex O immediately after time $s = 0$: for $s \in (0, T]$, the singularity is $c(s)\delta_O$ (here, $\delta_O$ is the Dirac delta at O) with $c(s) = 2s$ for $s \in (0, 1/4]$ and $c(s) = 1/2$ for $s \in [1/4, T]$.

Analogously, for $L_1 = L_2 = 1$ and $L_* = -1$, a Dirac delta immediately appears at O and after the time $1/4$, the whole population is concentrated at O.

In the next example, again without interactions, the distribution of states develops a singularity that, after a while, starts travelling inside the edges.

**Example 3.2.** Consider a network with two vertices $V_1$ and $V_2$ and three edges $J_1, J_2$ and $J_3$ such that $J_1 \cap J_2 = V_1$, $J_2 \cap J_3 = V_2$, $J_1 \cap J_3 = \emptyset$. For simplicity, assume that $V_1$ coincides with the origin O. The edges $J_1$ and $J_3$ are unbounded while the edge $J_2$ has length equal to 1, say $J_2 = [0, 1]e_2$ for some unit vector $e_2$ (i.e. $V_1 = 0e_2$ and $V_2 = e_2$). The running cost $L$ and the terminal cost $G$ are defined on the three edges as follows: for any measure $m$ on the network

$$L[m](x,t) = \begin{cases} 0, & \text{if } x \in J_1 \cup \{V_1\}, \\ k_L, & \text{if } x \in (J_2 \cup J_3 \cup \{V_2\}) \setminus \{V_1\}, \end{cases} \\ G[m](x,t) = \begin{cases} 0, & \text{if } x \in (J_1 \cup J_2 \cup \{V_1\}) \setminus \{V_2\}, \\ -k_G, & \text{if } x \in J_3 \cup \{V_2\}, \end{cases}$$

for some positive constants $k_L$ and $k_G$ which will be chosen later on. Note that these costs fulfill the assumptions (2.6) and (2.7). The time horizon $T$ will be chosen suitably large later on.

Assume for a moment that, for $T > (2k_L)^{-1/2}$ and $k_G > 2\sqrt{2k_L}$, for any $t \in [0, T - (2k_L)^{-1/2}]$, any $(y, \alpha) \in \Gamma_t^{\text{opt}}[V_1]$ is such that

$$(3.6) \quad y(s) = \begin{cases} V_1, & \text{for } s \in [t, T - (2k_L)^{-1/2}], \\ (2k_L)^{1/2} (-T + (2k_L)^{-1/2} + s) e_2, & \text{for } s \in (T - (2k_L)^{-1/2}, T], \end{cases}$$

i.e. the trajectory remains at $V_1$ up to time $T - (2k_L)^{-1/2}$ and enters afterwards in $J_2$ with constant velocity, so to reach $V_2$ at time $T$.

Under the latter assumption, let us prove that, for $T$ sufficiently large, if $(y, \alpha) \in \Gamma_0^{\text{opt}}[\bar{x}e_2]$, with $\bar{x} \in [0, 1]$, then

$$y(s) = \begin{cases} \begin{aligned} & -(2k_L)^{1/2}s + \bar{x} \end{aligned} e_2, & \text{for } s \in [0, \bar{x}(2k_L)^{-1/2}], \\ V_1, & \text{for } s \in [\bar{x}(2k_L)^{-1/2}, T - (2k_L)^{-1/2}], \\ (2k_L)^{1/2} \begin{aligned} & -T + (2k_L)^{-1/2} + s \end{aligned} e_2, & \text{for } s \in (T - (2k_L)^{-1/2}, T], \end{cases}$$

i.e., the trajectory moves towards $V_1$ with velocity $(2k_L)^{1/2}$, reaches $V_1$ at time $(2k_L)^{-1/2}\bar{x}$ and remains there until time $T - (2k_L)^{-1/2}$, then moves towards $V_2$ with velocity $(2k_L)^{1/2}$ and reaches $V_2$ at the horizon $T$. Clearly, if $m_0$ is supported in $J_2$, then (3.7) entails that all the agents first reach $V_1$, (so a singularity appears in the distribution), then all together start to move toward $V_2$ at time $T - (2k_L)^{-1/2}$.

Let us prove (3.7). Since $\bar{x}e_2 \in J_2 \setminus \{V_1, V_2\}$, Euler-Lagrange condition in Lemma 2.17 implies that the control $\alpha$ is constant on an interval $[0, \tau)$, for some $\tau \in (0, T]$.
Let us list all the possible strategies and compare the corresponding costs.

Strategy A: $y(s) = \bar{x}e_2$ and $\alpha(s) = 0$ for all $s \in [0, T]$. The cost is $J_A = k_L T$.

Strategy B: $\alpha$ is constant on $[0, \tau)$, where $\tau = \min\{T, \min\{s > 0 : y(s) \in \{V_1, V_2\}\}\}$.
Note that, if $y$ remains in $J_2 \setminus \{V_1, V_2\}$ in the whole interval $[0, T]$, then the cost $J_A$ is not larger. Thus we may assume $\tau = \min\{s > 0 : y(s) \in \{V_1, V_2\}\} < T$. We distinguish two subcases whether $y(\tau) = V_1$ or $y(\tau) = V_2$.

Strategy B1: $y(\tau) = V_2$. Euler-Lagrange conditions yields $\alpha(s) = (1 - \bar{x})/\tau e_2$ and $y(s) = \tau^{-1}[\bar{x}\tau + (1 - \bar{x})s]e_2$ on $[0, \tau)$. It is then clear that $y(s) = V_2$ for $s \in [\tau, T]$, because the other possibilities lead to higher costs. The corresponding cost is $(1 - \bar{x})^2/(2\tau) + k_L T - k_G$. Since the latter quantity is strictly decreasing w.r.t. $\tau$, its minimum in $\tau$ is achieved by $\tau = T$. Hence the optimal cost with Strategies of type B1 is $J_{B1} = (1 - \bar{x})^2/(2T) + k_L T - k_G$.

Strategy B2: $y(\tau) = V_1$. Euler-Lagrange condition yields $\alpha(s) = -(\bar{x}/\tau)e_2$ and $y(s) = \bar{x}(1 - s/\tau)e_2$ on $[0, \tau)$. Then (3.6) implies that

$$y(s) = \begin{cases} V_1 & \text{for } s \in [\tau, T - (2k_L)^{-1/2}], \\ (2k_L)^{1/2} \left(-T + (2k_L)^{-1/2} + s\right) e_2 & \text{for } s \in (T - (2k_L)^{-1/2}, T]. \end{cases}$$

The cost corresponding to this trajectory is

$$\frac{1}{2} \frac{\bar{x}^2}{\tau} + k_L \tau + \frac{1}{2} \sqrt{2k_L} + k_L T - k_L \left( T - \frac{1}{\sqrt{2k_L}} \right) - k_G,$$

and its minimum w.r.t. $\tau \in [0, T]$ is achieved by $\tau = \bar{x}/\sqrt{2k_L}$. Hence the optimal cost in Strategy B2 is $J_{B2} = (1 + \bar{x})\sqrt{2k_L} - k_G$.

Conclusion. Comparing the costs $J_A$, $J_{B1}$ and $J_{B2}$, we obtain that $J_{B1} > J_{B2}$ and $J_A > J_{B2}$ for $k_L > 1$, $k_G$ satisfying the assumptions before (3.6) and $T$ sufficiently large. Hence, the optimal trajectory is that of Strategy B2.

There remains to prove (3.6). To this end, let us distinguish several possible strategies.

Strategy $\tilde{A}$: $y(s) = V_1$ in $[t, T]$. The cost is $J_{\tilde{A}} = 0$.

Strategy $\tilde{B}$: Immediately or after a while, the trajectory $y$ enters in $J_1 \setminus \{V_1\}$ and remains in $J_1$. Since the cost associated to the kinetic energy is higher than with Strategy $\tilde{A}$, Strategy $\tilde{B}$ is strictly suboptimal.

Strategy $\tilde{C}$: Immediately or after a while, the trajectory $y$ enters in $J_2 \setminus \{V_1\}$ and is such that $y(T) \in (J_1 \cup J_2) \setminus J_3$, in particular, $y(T) \neq V_2$. Since the cost associated to the kinetic energy is higher than with Strategy $\tilde{A}$, Strategy $\tilde{C}$ is strictly suboptimal.

Strategy $\tilde{D}$: Immediately or after a while, the trajectory $y$ enters in $J_2$ and is such that $y(T) = V_2$. Hence,

* $y(s) = V_1$ on $[t, s_1]$ for some $s_1 \in [t, T]$

* for $s_2 = \min\{s \in (s_1, T] : y(s) = V_2\}$, there holds: $y(s_2) = V_2$ and $y(s) \in J_2 \setminus \{V_1\}$ for $s \in (s_1, s_2)$. Then, from Euler-Lagrange condition, $\alpha(s) = (s_2 - s_1)^{-1}e_2$ for $s \in (s_1, s_2)$

* $y(s) = V_2$ for $s \in [s_2, T]$ because all the other possibilities result in a higher cost.

The resulting cost is

$$\frac{1}{2} \frac{1}{s_2 - s_1} + k_L T - k_L s_1 - k_G.$$
Let us minimize the latter cost w.r.t. $s_1 \in [t, T)$ and $s_2 \in (s_1, T]$. Since it is strictly decreasing w.r.t. $s_2$, let us choose $s_2 = T$ so there remains to minimize $\frac{1}{2} \frac{1}{T-s_1} + k_L T - k_L s_1 - k_G$ with respect to $s_1 \in [t, T)$. The minimum is reached at $s_1 = T - (2k_L)^{-1/2}$, and takes the value $J_{\tilde{D}} = \sqrt{2k_L - k_G}$ which is less than $J_{\tilde{A}}$ from the assumption on $k_G$. Strategy $\tilde{E}$: Immediately or after a while, the trajectory $y$ enters in $J_2$ and is such that $y(T) \in J_3 \setminus \{V_2\}$. Comparing the resulting cost with that of Strategy $\tilde{D}$, one can check that Strategy $\tilde{E}$ is strictly suboptimal.

## 3.2 Relaxed MFG equilibrium

Fix $\mu \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$; for any $x \in \mathcal{G}$, let us set

$$ (3.8) \qquad \Gamma_C^{\mu, \text{opt}}[x] = \left\{ y \in \tilde{\Gamma}_C[x] : J^\mu(x; (y, \alpha_y)) = \min_{(\tilde{y}, \tilde{\alpha}) \in \Gamma[x]} J^\mu(x; (\tilde{y}, \tilde{\alpha})) \right\} $$

where $J^\mu$ is defined in (3.4) and $\alpha_y$ is a control such that $(y, \alpha_y) \in \Gamma[x]$ (see Remark 3.2).

**Definition 3.1.** The complete probability measure $\mu \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$ is a relaxed mean field game equilibrium associated with the initial distribution $m_0$ if

$$ (3.9) \qquad \operatorname{supp}(\mu) \subset \bigcup_{x \in \operatorname{supp}(m_0)} \Gamma_C^{\mu, \text{opt}}[x]. $$

The following two theorems address the existence of MFG equilibria under different hypothesis.

**Theorem 3.5.** Assume $(H_0)$, $(H_1^{\text{MFG}})$ and in $(H_2^{\text{MFG}})$; consider $C \ge \tilde{C}$ (where $\tilde{C}$ is the constant introduced in Remark 3.3). Then, there exists a relaxed mean field equilibrium $\mu \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$.

The proof of Theorem 3.5 is postponed to subsection 3.5.

**Theorem 3.6.** Keeping the assumptions of Theorem 3.5, we also assume that, for some positive constant $K$,

$$ (H_3^{\text{MFG}}) \quad \left\{ \begin{array}{lll} G_i[m] \in C^2(J_i) & \text{and} & L_i[m](\cdot, t) \in C^2(J_i) \quad \forall m \in \mathcal{P}(\mathcal{G}), t \in [0, T] \\ & & \sup_{m \in \mathcal{P}(\mathcal{G})} \max_{i=1, \dots, N} \|\partial G_i[m]\|_{\infty} \le K \\ & & \sup_{m \in \mathcal{P}(\mathcal{G})} \max_{i=1, \dots, N} \sup_{t \in [0, T]} \|\partial L_i[m](\cdot, t)\|_{\infty} \le K, \end{array} \right. $$

Then, there exists a relaxed MFG equilibrium $\mu \in \mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})$, where the constants $V$ and $C$ appear respectively in Theorem 2.21 with $t = 0$ and $x \in \text{supp}(m_0)$ and in Theorem 3.5.

The proof of Theorem 3.6 is postponed to subsection 3.6.

## 3.3 Preliminary results

**Lemma 3.7.** Let a sequence of probability measures $\{\mu_n\}_{n \in \mathbb{N}}$, $\mu_n \in \mathcal{P}(\tilde{\Gamma}_C)$, be narrowly convergent to $\mu \in \mathcal{P}(\tilde{\Gamma}_C)$ as $n \to \infty$. For all $t \in [0, T]$, the sequence $\{m^{\mu_n}(t)\}_{n \in \mathbb{N}}$ is narrowly convergent to $m^\mu(t)$.
**Proof.** Adapting the arguments of [5, Lemma 3.1] leads to

$$ \int_G f(x) dm^{\mu_n}(t)(x) = \int_{\tilde{\Gamma}_C} f(y(t)) d\mu_n(y) \rightarrow \int_{\tilde{\Gamma}_C} f(y(t)) d\mu(y) = \int_G f(x) dm^{\mu}(t)(x), $$

for all $f \in C_b^0(\mathcal{G}; \mathbb{R})$.

□

**Lemma 3.8.** Assume $(H_2^{\text{MFG}})$. There holds

$$ \sup_{\mu \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)} \text{Wass}_1(m^{\mu}(t), m^{\mu}(s)) \le C|t-s|^{\frac{1}{2}} \quad \forall t, s \in [0, T]. $$

Similarly,

$$ \sup_{\mu \in \mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})} \text{Wass}_1(m^{\mu}(t), m^{\mu}(s)) \le V|t-s| \quad \forall t, s \in [0, T]. $$

**Proof.** Consider any $\mu \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$. For any $t, s \in [0, T]$, there holds

$$ \begin{aligned} \sup_{\phi} \int_{\mathcal{G}} \phi(x) [dm^{\mu}(t) - dm^{\mu}(s)](x) &= \sup_{\phi} \int_{\tilde{\Gamma}_C} [\phi(y(t)) - \phi(y(s))] d\mu(y) \\ &\le \int_{\tilde{\Gamma}_C} |y(t) - y(s)| d\mu(y) \le |t-s|^{\frac{1}{2}} \|\alpha\|_2 \end{aligned} $$

where the supremum is performed over all the continuous 1-Lipschitz function. Owing to the definition of $\tilde{\Gamma}_C$ in (3.2) and to the arbitrariness of $\mu \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$, the latter relation entails the first statement. The second statement is obtained in a singular way. □

It is useful to recall the disintegration theorem:

**Theorem 3.9.** Let $X$ and $Y$ be Radon metric spaces, $\pi: X \rightarrow Y$ be a Borel map, $\mu$ be a probability measure on $X$. Set $\nu = \pi_{\#}\mu$. There exists a $\nu$-almost everywhere uniquely defined Borel measurable family of probability measures $(\mu_y)_{y \in Y}$ on $X$ such that

$$ \mu_y(X \setminus \pi^{-1}(y)) = 0, \quad \text{for } \nu\text{-almost all } y \in Y, $$

and for every Borel function $f: X \rightarrow [0, +\infty]$,

$$ \int_X f(x) d\mu(x) = \int_Y \left( \int_X f(x) d\mu_y(x) \right) d\nu(y) = \int_Y \left( \int_{\pi^{-1}(y)} f(x) d\mu_y(x) \right) d\nu(y). $$

Recall that $(\mu_y)_{y \in Y}$ is a Borel family of probability measures if for any Borel subset $B$ of $X$, $Y \ni y \mapsto \mu_y(B)$ is a Borel function from $Y$ to $[0, 1]$.

## 3.4 A closed graph property

Choosing $C \ge \tilde{C}$, where $\tilde{C}$ is the constant introduced in Remark 3.3, we first establish a closed graph property for the map $\Gamma_C^{\mu, \text{opt}}[x]$.
**Proposition 3.10.** Consider $\mu \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$ and $x \in \text{supp}(m_0)$. Consider also a sequence of probability measures $\{\mu_n\}_{n \in \mathbb{N}}$, with $\mu_n \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$, narrowly convergent to $\mu$ as $n \to \infty$ and a sequence of points $\{x_n\}_{n \in \mathbb{N}}$, with $x_n \in \mathcal{G}$ and $x_n \to x$ as $n \to \infty$. Let $\{y_n\}_{n \in \mathbb{N}}$ be a sequence of paths such that $y_n \in \tilde{\Gamma}_C^{\mu_n, \text{opt}}[x_n]$ and $y_n$ uniformly converge to some path $y$ as $n \to \infty$. Then, $y$ belongs to $\Gamma_C^{\mu_n, \text{opt}}[x]$, namely any trajectory $(y, \alpha_y)$ is an optimal trajectory for $J^\mu$. In other words, the multivalued map $(x, \mu) \Rightarrow \Gamma_C^{\mu_n, \text{opt}}[x]$ enjoys the closed graph property.

*Proof of Proposition 3.10.* There are similar arguments as in the proof of Proposition 2.25, so we will mailnly focus on the new aspects. We wish to prove that

$$ (i) \quad y \in \tilde{\Gamma}_C[x], \quad (ii) \quad (y, \alpha_y) \text{ is optimal for } J^\mu. $$

From the definition of $\tilde{\Gamma}_C$, the controls $\alpha_{y_n}$ are uniformly bounded in $L^2$. The same arguments as in the proof of Proposition 2.25 show that, possibly up to a subsequence (still denoted by $\alpha_{y_n}$), $\{\alpha_{y_n}\}_n$ converges in the weak topology of $L^2((0, T), \mathbb{R}^d)$ to some control $\alpha_y$, with $\|\alpha_y\|_2 \le C$, that $(y, \alpha_y) \in \Gamma[x]$ and $y \in \tilde{\Gamma}_C[x]$. The proof of point (i) is done.

Concerning (ii), it suffices to prove that

$$ J^{\mu}(x; (y, \alpha_y)) \le J^{\mu}(x; (\hat{y}, \hat{\alpha})) \quad \forall (\hat{y}, \hat{\alpha}) \in \Gamma[x]. $$

Fix any $(\hat{y}, \hat{\alpha}) \in \Gamma[x]$. Lemma 2.26 ensures that there exists a sequence $\{\hat{y}_n, \hat{\alpha}_n\}_{n \in \mathbb{N}}$ such that $(\hat{y}_n, \hat{\alpha}_n) \in \Gamma[x_n]$, $\hat{y}_n(T) = \hat{y}(T)$ and

$$ \hat{y}_n \to \hat{y} \text{ uniformly in } [0, T] \text{ as } n \to \infty, \quad \|\hat{\alpha}_n\|_2 \le \|\hat{\alpha}\|_2 + o_n(1), $$

where $o_n(1)$ is a sequence such that $\lim_n o_n(1) = 0$. Since $y_n \in \tilde{\Gamma}_C^{\mu_n, \text{opt}}[x_n]$,

$$ (3.10) \qquad J^{\mu_n}(x_n; (y_n, \alpha_{y_n})) \le J^{\mu_n}(x_n; (\hat{y}_n, \hat{\alpha}_n)). $$

Let us now study separately the two sides of (3.10). For the right hand side, the construction and the properties of $(\hat{y}_n, \hat{\alpha}_n)$ entail

$$ J^{\mu_n}(x_n; (\hat{y}_n, \hat{\alpha}_n)) \le J^{\mu}(x; (\hat{y}, \hat{\alpha})) + \sum_{i=1}^{4} \bar{I}_i $$

where, for $\delta_n = d(x, x_n)$,

$$ \bar{I}_1 = \int_0^{\delta_n} L[m^{\mu_n}(\tau)](\hat{y}_n(\tau), \tau)d\tau, \quad \bar{I}_2 = \frac{\|\alpha_n\|_2^2 - \|\alpha\|_2^2}{2} \le o_n(1), $$

$$ \bar{I}_3 = \int_{\delta_n}^{T} L[m^{\mu_n}(\tau)](\hat{y}_n(\tau), \tau)d\tau, \quad \bar{I}_4 = - \int_{0}^{T} L[m^{\mu}(\tau)](\hat{y}(\tau), \tau)d\tau. $$

The boundedness of $L$ implies that $\lim_{n \to \infty} \bar{I}_1 = 0$. Then, arguing as in the proof of Lemma 2.26,

$$ \bar{I}_3 = \int_0^T \left[ \sum_{i=1}^N L_i \left[ m^{\mu_n} \left( \frac{T - \delta_n}{T} \xi + \delta_n \right) \right] \left( \hat{y}(\xi), \frac{T - \delta_n}{T} \xi + \delta_n \right) \mathbb{1}_{\hat{y}(\xi) \in J_i \setminus \{O\}} \right. \\ \left. + L_O \left[ m^{\mu_n} \left( \frac{T - \delta_n}{T} \xi + \delta_n \right) \right] \left( \frac{T - \delta_n}{T} \xi + \delta_n \right) \mathbb{1}_{\hat{y}(\xi) = O} \right] \left( 1 - \frac{\delta_n}{T} \right) d\xi $$

$$ = \int_0^T L \left[ m^{\mu_n} \left( \frac{T - \delta_n}{T} \xi + \delta_n \right) \right] \left( \hat{y}(\xi), \frac{T - \delta_n}{T} \xi + \delta_n \right) \left( 1 - \frac{\delta_n}{T} \right) d\xi, $$
and consequently,

$$ \bar{I}_3 + \bar{I}_4 = \bar{I}_5 + \bar{I}_6 + \bar{I}_7 + \bar{I}_8 $$

where

$$
\begin{align*}
\bar{I}_5 &= -\frac{\delta_n}{T} \int_0^T L \left[ m^{\mu_n} \left( \frac{T - \delta_n}{T} \xi + \delta_n \right) \right] \left( \hat{y}(\xi), \frac{T - \delta_n}{T} \xi + \delta_n \right) d\xi, \\
\bar{I}_6 &= \int_0^T \left( L \left[ m^{\mu_n} \left( \frac{T - \delta_n}{T} \xi + \delta_n \right) \right] \left( \hat{y}(\xi), \frac{T - \delta_n}{T} \xi + \delta_n \right) - L \left[ m^{\mu_n}(\xi) \right] \left( \hat{y}(\xi), \frac{T - \delta_n}{T} \xi + \delta_n \right) \right) d\xi, \\
\bar{I}_7 &= \int_0^T \left( L \left[ m^{\mu_n}(\xi) \right] \left( \hat{y}(\xi), \frac{T - \delta_n}{T} \xi + \delta_n \right) - L \left[ m^{\mu}(\xi) \right] \left( \hat{y}(\xi), \frac{T - \delta_n}{T} \xi + \delta_n \right) \right) d\xi, \\
\bar{I}_8 &= \int_0^T \left( L \left[ m^{\mu}(\xi) \right] \left( \hat{y}(\xi), \frac{T - \delta_n}{T} \xi + \delta_n \right) - L \left[ m^{\mu}(\xi) \right] \left( \hat{y}(\xi), \hat{\alpha}(\xi), \xi \right) \right) d\xi.
\end{align*}
$$

The boundedness of $L$ entails: $|\bar{I}_5| = o_n(1)$. From Lemma 3.8, the assumptions on the costs $L_i$ and Lebesgue dominated convergence theorem, $|\bar{I}_6| = o_n(1)$. From Lemma 3.7, again the assumptions on the costs $L_i$ and Lebesgue dominated convergence theorem, $|\bar{I}_7| = o_n(1)$. Finally, since $\hat{y}$ is bounded and $L_i[m]$ are continuous, $|\bar{I}_8| = o_n(1)$.

To summarize, there holds

$$ (3.11) \qquad \limsup_n J^{\mu_n}(x_n; (\hat{y}_n, \hat{\alpha}_n)) \le J^{\mu}(x; (\hat{y}, \hat{\alpha})). $$

The left hand side of (3.10) is addressed with arguments from the proof of Proposition 2.11. By definition of cost (3.4),

$$ (3.12) \qquad J^{\mu_n}(x_n; (y_n, \alpha_{y_n})) = \int_0^T \frac{|\alpha_{y_n}(\tau)|^2}{2} d\tau + \sum_{i=1}^5 \hat{I}_i, $$

where

$$
\begin{align*}
\hat{I}_1 &= \int_0^T \sum_{i=1}^N L_i[m^{\mu_n}(\tau)](y_n(\tau), \tau) \mathbb{1}_{y_n(\tau) \in J_i \setminus \{O\}} \mathbb{1}_{y(\tau) \in J_i \setminus \{O\}} d\tau, \\
\hat{I}_2 &= \int_0^T \sum_{i=1}^N L_i[m^{\mu_n}(\tau)](y_n(\tau), \tau) \mathbb{1}_{y_n(\tau) \in J_i \setminus \{O\}} \mathbb{1}_{y(\tau) \in G \setminus J_i} d\tau, \\
\hat{I}_3 &= \int_0^T \sum_{i=1}^N L_i[m^{\mu_n}(\tau)](y_n(\tau), \tau) \mathbb{1}_{y_n(\tau) \in J_i \setminus \{O\}} \mathbb{1}_{y(\tau)=O} d\tau, \\
\hat{I}_4 &= \int_0^T L_O[m^{\mu_n}(\tau)](\tau) \mathbb{1}_{y_n(\tau)=O} d\tau, \\
\hat{I}_5 &= G[m^{\mu_n}(\tau)](y_n(T)).
\end{align*}
$$

The convergence in the weak topology of $L^2([t, T]; \mathbb{R}^d)$ entails

$$ \int_0^T \frac{|\alpha(\tau)|^2}{2} d\tau \le \liminf_{n \to \infty} \int_0^T \frac{|\alpha_n(\tau)|^2}{2} d\tau. $$

Recall from Lemma 3.7 that, for each $t \in [0, T]$, the map $\mathcal{P}(\tilde{\Gamma}_C) \ni \mu \mapsto m^\mu(t) \in \mathcal{P}(G)$ is continuous. Hence, by our assumption, for every $i \in \{1, \dots, N\}$, $L_i[m^{\mu_n}(t)](\cdot, \cdot)$ and $G_i[m^{\mu_n}(T)](\cdot)$ converge uniformly respectively to $L_i[m^\mu(t)](\cdot, \cdot)$ and to $G_i[m^\mu(T)](\cdot)$ as $n \to \infty$. Therefore, the dominated convergence theorem yields

$$ \hat{I}_1 \to \int_0^T \sum_{i=1}^N L_i[m^\mu(\tau)](y(\tau), \tau) \mathbb{1}_{y(\tau) \in J_i \setminus \{O\}} d\tau \quad \text{and} \quad \hat{I}_2 \to 0, \quad \text{as } n \to \infty. $$
The same arguments as in the proof of Proposition 2.11 and the definition of $G[m]$ in (3.1)
imply

$$
\liminf_{n \to \infty} \hat{I}_5 \geq G[m^\mu(T)](y(T)).
$$

Furthermore,

$$
\hat{I}_3 + \hat{I}_4 = \int_0^T \left[ \sum_{i=1}^N L_i[m^{\mu_n}(t)](y_n(\tau), \tau) \mathbb{1}_{y_n(\tau) \in J_i \setminus \{O\}} + L_O[m^{\mu_n}(t)](\tau) \mathbb{1}_{y_n(\tau)=O} \right] \mathbb{1}_{y(\tau)=O} d\tau \\
+ \int_0^T L_O[m^{\mu_n}(t)](\tau) \mathbb{1}_{y_n(\tau)=O} \mathbb{1}_{y(\tau) \neq O} d\tau.
$$

Again the dominated convergence theorem ensures

$$
\int_0^T L_O[m^{\mu_n}(t)](\tau) \mathbb{1}_{y_n(\tau)=O} \mathbb{1}_{y(\tau) \neq O} d\tau \to 0 \quad \text{as } n \to \infty.
$$

Then, from Fatou's Lemma and the boundedness of $L_i$,

$$
\liminf_{n \to \infty} \int_0^T \left[ \sum_{i=1}^N L_i[m^{\mu_n}(\tau)](y_n(\tau), \tau) \mathbb{1}_{y_n(\tau) \in J_i \setminus \{O\}} \right. \\
\left. \qquad + L_O[m^{\mu_n}(\tau)](\tau) \mathbb{1}_{y_n(\tau)=O} \right] \mathbb{1}_{y(\tau)=O} d\tau \ge \int_0^T L_O[m^{\mu_n}(\tau)](\tau) \mathbb{1}_{y(\tau)=O} d\tau.
$$

Combining all the observations above with (3.12) yields

$$
\begin{equation} \tag{3.13}
\begin{aligned}
\liminf_{n \to \infty} J^{\mu_n}(x_n; (y_n, \alpha_{y_n})) &\ge \int_0^T \left[ \frac{|\alpha(\tau)|^2}{2} + \sum_{i=1}^N L_i[m^\mu(\tau)](y(\tau), \tau) \mathbb{1}_{y(\tau) \in J_i \setminus \{O\}} \right. \\
&\qquad \left. + L_O[m^\mu(\tau)](\tau) \mathbb{1}_{y(\tau)=O} \right] d\tau + G[m^\mu(T)](y(T)) \\
&= J^\mu(x; (y, \alpha_y)).
\end{aligned}
\end{equation}
$$

In conclusion, (3.10), (3.11) and (3.13) entail

$$
J^{\mu}(x; (y, \alpha_{y})) \leq J^{\mu}(x; (\hat{y}, \hat{\alpha})).
$$

Since $(\hat{y}, \hat{\alpha}) \in \Gamma[x]$ is arbitrary, we get $J^{\mu}(x; (y, \alpha_y)) = \min_{(\hat{y}, \hat{\alpha}) \in \Gamma[x]} J^{\mu}(x; (\hat{y}, \hat{\alpha}))$ which is
 equivalent to (ii). $\square$

3.5 Proof of Theorem 3.5

Let us first recall some notations. For every $\mu \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$, let $J^\mu$ be the associated cost as in (3.4); for any $x \in \mathcal{G}$, let $\Gamma_C^{\mu, \text{opt}}[x]$ be the set of optimal paths starting from $x$ for the cost $J^\mu$ as in (3.5). Proposition 2.11 ensures: $\Gamma_C^{\mu, \text{opt}}[x] \neq \emptyset$ for every $x \in \mathcal{G}$. It is worth recalling that the set $\tilde{\Gamma}_C$ is compact, from Lemma 3.1. By Prokhorov theorem [9, Theorem 5.1.3], $\mathcal{P}(\tilde{\Gamma}_C)$ is also compact.

The multivalued map $E : \mathcal{P}_{m_0}(\tilde{\Gamma}_C) \Rightarrow \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$ is defined as follows:

$$
(3.14) \qquad E(\mu) = \{ \hat{\mu} \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C) : \text{supp } \hat{\mu}_x \subset \Gamma_C^{\mu, \text{opt}}[x] \text{ } m_0 \text{ - a.e. } x \in \mathcal{G} \},
$$

where {$\hat{\mu}_x$}$_{x \in \mathcal{G}}$ is the family of Borel probability measures on $\mathcal{P}_{m_0}(\tilde{\Gamma}_C)$ obtained apply-
ing the disintegration Theorem 3.9 to $\mu$, $X$, $Y$ and $\pi$ being replaced respectively by $\hat{\mu}$,
$\mathcal{P}_{m_0}(\tilde{\Gamma}_C)$, $\mathcal{G}$ and $e_0$ (so, clearly, $\nu$ coincides with $m_0$). The proof of the theorem amounts to
proving that the map $E$ admits a fixed point. Let us assume for the moment the following
properties
(i) for every $\mu \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$, the set $E(\mu)$ is not empty and convex

(ii) the map $E$ enjoys the closed graph property.

Then, Kakutani fixed point theorem ensures that $E$ admits a fixed point $\mu$. Without any loss of generality, we can complete the measure $\mu$ and obtain a relaxed MFG equilibrium. It remains to prove the above mentioned two properties.

(i). Recall that $\Gamma_C^{\mu, \text{opt}}[x] \neq \emptyset$ for every $x \in \mathcal{G}$ and that the map $\Gamma_C^{\mu, \text{opt}}[\cdot]$ has the closed graph property, from Proposition 2.11 and Proposition 2.25, Therefore, the result [8, Theorem 8.1.4] guarantees that the map $\Gamma_C^{\mu, \text{opt}}[\cdot]$ has a Borel measurable selection denoted $x \mapsto y_x^\mu$ for every $x \in \mathcal{G}$. We introduce a measure $\hat{\mu}$ on $\tilde{\Gamma}_C$ as follows:

$$ \hat{\mu}(B) = \int_{\mathcal{G}} \delta_{y_x^{\mu}}(B) m_0(dx) \quad \forall \text{ Borel } B \subset \tilde{\Gamma}_C, $$

where $\delta_{y_x^\mu}(\cdot)$ is the Dirac delta-function centered in $y_x^\mu$. Note that $\hat{\mu}_x = \delta_{y_x^\mu}$ for $m_0$-a.e. $x \in \mathcal{G}$. Hence, $\hat{\mu}$ belongs to $E(\mu)$.

Let us now prove that $E(\mu)$ is convex. Fix $\mu^1, \mu^2 \in E(\mu)$ and $\lambda \in [0, 1]$. By easy calculation, one obtains $\lambda\mu^1 + (1 - \lambda)\mu^2 \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$. On the other hand, for $i = 1, 2$, since $\mu^i \in E(\mu)$, by the disintegration theorem 3.9, there exists a Borel measurable family $\{\mu_x^i\}_{x \in \mathcal{G}}$ of probability measures (which is $m_0$-a.e. uniquely defined and “disintegrate” $\mu^i$ with respect to $m_0$) and a set $A_i \subset \mathcal{G}$ such that $m_0(A_i) = 0$ and $\text{supp} \mu_x^i \subset \Gamma_C^{\mu, \text{opt}}[x]$ for every $x \in \mathcal{G} \setminus A_i$. Therefore, the measure $\lambda\mu^1 + (1 - \lambda)\mu^2$ can be disintegrated as follows: for each Borel function $f$ on $\tilde{\Gamma}_C$, there holds

$$ \int_{\tilde{\Gamma}_C} f(\gamma)(\lambda\mu^1 + (1 - \lambda)\mu^2)(d\gamma) = \int_{\mathcal{G}} \left( \int_{\tilde{\Gamma}_C} f(\gamma)(\lambda\mu_x^1 + (1 - \lambda)\mu_x^2)(d\gamma) \right) m_0(dx) $$

with $m_0(A_1 \cup A_2) = 0$ and

$$ \text{supp} (\lambda\mu_x^1 + (1 - \lambda)\mu_x^2) \subset \Gamma_C^{\mu, \text{opt}}[x] \quad \forall x \in \mathcal{G} \setminus (A_1 \cup A_2). $$

Hence, $\lambda\mu^1 + (1 - \lambda)\mu^2$ belongs to $E(\mu)$, so $E(\mu)$ is convex.

(ii). Consider a sequence $\{\mu_n\}_{n \in \mathbb{N}}$ of probability measures $\mu_n \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$ which narrowly converges to some $\mu \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$ as $n \to \infty$. Consider also a sequence $\{\hat{\mu}_n\}_{n \in \mathbb{N}}$, with $\hat{\mu}_n \in E(\mu_n)$ for any $n \in \mathbb{N}$, which narrowly converges to some $\hat{\mu} \in \mathcal{P}_{m_0}(\tilde{\Gamma}_C)$ as $n \to \infty$. Our aim is to prove that $\hat{\mu}$ belongs to $E(\mu)$.

By the disintegration theorem, there exists a $m_0$-a.e. uniquely defined Borel measurable family of measures $\{\hat{\mu}_x\}_{x \in \mathcal{G}}$ on $\tilde{\Gamma}_C$ and $A \subset \mathcal{G}$ such that: $m_0(A) = 0$, $\hat{\mu}_x(\tilde{\Gamma}_C \setminus e_0^{-1}(\{x\})) = 0$ for every $x \in \mathcal{G} \setminus A$ and

$$ \int_{\tilde{\Gamma}_C} f(y)\hat{\mu}(dy) = \int_{\mathcal{G}} \left( \int_{\tilde{\Gamma}_C[x]} f(y)\hat{\mu}_x(dy) \right) m_0(dx). $$

Consider $x \in \mathcal{G} \setminus A$ and $\hat{y} \in \text{supp} \hat{\mu}_x$. Kuratowski theorem ([9, Proposition 5.1.8]) ensures that there exists a sequence $\{y_n\}_{n \in \mathbb{N}}$, with $y_n \in \text{supp} \hat{\mu}_n$, which converges to $\hat{y}$ in the topology of $\tilde{\Gamma}_C$. Let $x_n = e_0(y_n)$. Since $\hat{\mu}_n \in E(\mu_n)$, there holds: $y_n \in \tilde{\Gamma}_C^{\mu_n, \text{opt}}[x_n]$. By Proposition 3.10, we infer $\hat{y} \in \tilde{\Gamma}_C^{\mu, \text{opt}}[x]$. By the arbitrariness of $\hat{y} \in \text{supp} \hat{\mu}_x$, we obtain $\text{supp} \hat{\mu}_x \subset \tilde{\Gamma}_C^{\mu, \text{opt}}[x]$ and consequently, by the arbitrariness of $x \in \mathcal{G} \setminus A$, that $\hat{\mu}$ belongs to $E(\mu)$. $\square$
## 3.6 Proof of Theorem 3.6

This paragraph contains the proof of Theorem 3.6. We proceed adapting the proof of Theorem 3.5 and using some ideas from [17, Theorem 4.1]. We consider the multivalued map E, defined in (3.14) which has the closed graph property (see point (ii) in the proof of Theorem 3.5). We then introduce the multivalued map $E_0$ as the restriction of E to the set $P_{m_0}(\Gamma_{C,V}^{\text{Lip}})$ where C and V are chosen as in the statement of the theorem. The proof consists of checking that $E_0$ fulfills the hypotheses of Kakutani fixed point theorem. To this end, we need to check that

(i) $E_0(\mu) \subset P_{m_0}(\Gamma_{C,V}^{\text{Lip}})$, $\forall \mu \in P_{m_0}(\Gamma_{C,V}^{\text{Lip}})$

(ii) $P_{m_0}(\Gamma_{C,V}^{\text{Lip}})$ is compact

(iii) $E_0(\mu)$ is a not empty convex set, $\forall \mu \in P_{m_0}(\Gamma_{C,V}^{\text{Lip}})$

(iv) $E_0$ has the closed graph property.

Let us successively address the four properties.

(i). Consider $\mu \in P_{m_0}(\Gamma_{C,V}^{\text{Lip}})$. As in the proof of Theorem 3.5, we see that $\tilde{\Gamma}_C^{\mu,\text{opt}}[x] \neq \emptyset$ for any $x \in \text{supp}(m_0)$. On the other hand, from Theorem 2.21, $\tilde{\Gamma}_C^{\mu,\text{opt}}[x] \subset \Gamma_{C,V}^{\text{Lip}}[x]$. Hence, $\text{supp}(\hat{\mu}) \subset \Gamma_{C,V}^{\text{Lip}}$, for any $\hat{\mu} \in E_0(\mu)$. Invoking [17, Lemma 4.1], we get: $\hat{\mu} \in P_{m_0}(\Gamma_{C,V}^{\text{Lip}})$ and the proof of (i) is achieved.

(ii). From Lemma 3.1, the set $\tilde{\Gamma}_C$ is compact. Then, from Ascoli-Arzelà Theorem, $\Gamma_{C,V}^{\text{Lip}}$ is compact. From Prokhorov theorem, $P^{\text{Lip}}(\Gamma_{C,V}^{\text{Lip}})$ is compact so, in particular, $P_{m_0}(\Gamma_{C,V}^{\text{Lip}})$ is compact.

(iii) and (iv). These properties have already been obtained in the proof of Theorem 3.5. We refer the reader to that proof for the details.

In conclusion, by Kakutani theorem, there exists a fixed point of the map $E_0$, namely a relaxed MFG equilibrium in $P_{m_0}(\Gamma_{C,V}^{\text{Lip}})$. $\square$

## 3.7 Mild solutions

Let $\mu \in P_{m_0}(\Gamma)$ be a relaxed MFG equilibrium whose existence is guaranteed by Theorem 3.5. We consider the value function naturally associated to $\mu$:

$$ (3.15) \qquad u(x,t) = \inf_{(y_x, \alpha) \in \Gamma_t[x]} J_t^\mu(x; (y_x, \alpha)), $$

where $J_t^\mu$ is the cost defined in (3.4).

**Definition 3.2.** Let $\mu \in P_{m_0}(\Gamma)$ be a relaxed MFG equilibrium. The pair $(u, m)$ is the associated mild solution if $u$ is the value function defined in (3.15) and $m \in C([0, T], P(\mathcal{G}))$ is defined by $m(t) = e_t \# \mu$.

**Remark 3.11.** Lemma 3.8 ensures that $m \in C^{1/2}([0, T], P(\mathcal{G}))$.

For simplicity of notations, we set

$$ (3.16) \qquad \ell_i(x,t) = L_i[m^\mu(t)](x,t) \quad \text{and} \quad g_i(x) = G_i[m^\mu(T)](x) \quad \forall (x,t) \in \mathcal{G} \times [0, T] $$

and we shall also use the abridged notation $L$ as in (2.8). The costs $\ell_i$ are those payed by the agents in the MFG. By Lemma 3.8, the functions $\ell_i$ fulfill assumption $(H_1)$.
The purpose of this section is to derive several properties of the value function from the results of Section 2. As a preliminary step, invoking Proposition 2.13, Proposition 2.14, Remark 2.15 and Lemma 3.8, we obtain the following proposition:

**Proposition 3.12.** Under the assumptions of Theorem 3.5, the value function $u$ defined in (3.15) has the following properties:

(i) (Dynamic programming principle)

$$u(x,t) = \inf_{(y_x, \alpha) \in \Gamma_{t, \bar{t}}[x]} \left\{ u(y_x(\bar{t}), \bar{t}) + \int_t^{\bar{t}} \left( L(y_x(\tau), \tau) + \frac{|\alpha(\tau)|^2}{2} \right) d\tau \right\}$$

where

$$\Gamma_{t, \bar{t}}[x] = \left\{ \begin{array}{ll} (y_x, \alpha) \in L^2([t, \bar{t}], M) : & y_x \in W^{1,2}([t, \bar{t}]; \mathcal{G}), \\ & y_x(s) = x + \int_t^s \alpha(\tau)d\tau \quad \text{in } [t, \bar{t}] \end{array} \right\};$$

(ii) the value function is continuous in $\mathcal{G} \times [0, T)$.

Applying Theorem 2.36, it can now be proved that $u$ solves the HJ problem associated with the costs $\ell_i$.

**Theorem 3.13.** Under the same assumptions as in Theorem 3.5, the value function $u$ defined in (3.15) is a solution to (2.80) with the costs $\ell_i$ defined in (3.16). Moreover, for all $x \in \mathcal{G}$, $t \mapsto u(x, t)$ is continuous in $[0, T]$ and $u(x, T) = g(x)$ with the costs $g_i$ defined in (3.16).

**Corollary 3.14.** Under the same assumptions as in Theorem 3.6, there holds

(a) $u$ is locally Lipschitz continuous in $\mathcal{G} \times [0, T)$

(b) if, moreover, $G[m]$ is Lipschitz continuous for every $m \in \mathcal{P}(\mathcal{G})$, then $u$ is locally Lipschitz continuous in $\mathcal{G} \times [0, T]$.

*Proof.* Assumption $(H_3^{\text{MFG}})$ entails that the costs $\ell_i$ and $g_i$ associated to $\mu$ in (3.16) fulfill the assumption of Theorem 2.21. Hence, for proving points (a) and (b), it is enough to apply respectively Proposition 2.27 and Corollary 2.29. $\square$

**Remark 3.15** (Uniqueness of the mild solution). We say that $F: \mathcal{G} \times \mathcal{P}(\mathcal{G}) \to \mathbb{R}$ is monotone if, for any $m_1, m_2 \in \mathcal{P}(\mathcal{G})$, there holds $\int_{\mathcal{G}} (F(x, m_1) - F(x, m_2))(m_1 - m_2)(dx) \ge 0$. The strict monotonicity holds if furthermore $\int_{\mathcal{G}} (F(x, m_1) - F(x, m_2))(m_1 - m_2)(dx) = 0$ if and only if $F(\cdot, m_1) \equiv F(\cdot, m_2)$. If for all $t$, $(x, m) \mapsto L[m](x, t)$ and $(x, m) \mapsto G[m](x)$ are strictly monotone, then it can be proved with the same arguments as in [16, Theorem 4.1 and Remark 4.1], that if $(u_1, m_1)$ and $(u_2, m_2)$ are mild solutions respectively associated to two relaxed equilibria $\mu_1$ and $\mu_2$, then $u_1 = u_2$. Under a more restrictive monotonicity assumption on $L$, it can also be proved that $m_1 = m_2$.

It is worth noticing that the uniqueness of the mild solution does not imply the uniqueness of the relaxed MFG equilibrium as shown in the following example.
**Example 3.3.** Let us exhibit two probabilities $\mu_1, \mu_2 \in \mathcal{P}(\Gamma)$ such that

$$ (3.17) \qquad \mu_1 \neq \mu_2, \quad \text{and} \quad e_t \# \mu_1 = e_t \# \mu_2, \quad \forall t \in [0, T]. $$

For $0 < t_1 \le t_2 < T$, consider four paths $\gamma_i \in \Gamma$ ($i = 1, \dots, 4$) such that

$$ \begin{array}{lclcl} \gamma_1 = \gamma_2 & \text{and} & \gamma_3 = \gamma_4 & \text{on } [0, t_1] \\ \gamma_1 = \gamma_2 = \gamma_3 = \gamma_4 & & \text{on } [t_1, t_2] \\ \gamma_1 = \gamma_3 & \text{and} & \gamma_2 = \gamma_4 & \text{on } [t_2, T], \end{array} $$

and such that $\gamma_1$ does not coincide with $\gamma_3$ on $[0, t_1]$ and with $\gamma_2$ on $[t_2, T]$. Then (3.17) holds for the probabilities on $\Gamma$ defined by

$$ \mu_1 = \frac{1}{4} \sum_{i=1}^{4} \delta_{\gamma_i}, \quad \text{and} \quad \mu_2 = \frac{1}{2} \delta_{\gamma_1} + \frac{1}{2} \delta_{\gamma_4}. $$

Let us provide examples of strictly monotone operators.

**Example 3.4.** Fix a function $\kappa \in C_0^\infty(\mathbb{R})$ with $0 \le \kappa \le 1$, $\kappa'(0) = \kappa''(0) = 0$ and let $K : \mathcal{G} \times \mathcal{G} \to \mathbb{R}$ be defined by $K(x, y) = \kappa(d(x, y))$ where $d$ is the distance in $\mathcal{G}$. The function $K$ has the following properties:

$$ K(x, y) = K(y, x), \quad \forall x, y \in \mathcal{G}. $$

For any $m \in \mathcal{P}(\mathcal{G})$, define $K * m : \mathcal{G} \to \mathbb{R}$ by

$$ K * m(x) = \int_{\mathcal{G}} K(x, y)m(dy). $$

Let the running cost $L : \mathcal{P}(\mathcal{G}) \times \mathcal{G} \to \mathbb{R}$ be defined by

$$ L[m](x) = \int_{\mathcal{G}} \ell(y, K * m(y)) K(x, y)dy $$

where $\ell : \mathcal{G} \times \mathbb{R} \to \mathbb{R}$ is a smooth function such that $w \mapsto \ell(y, w)$ is strictly increasing for every $y \in \mathcal{G}$. It is standard that

$$ (3.18) \qquad \int_{\mathcal{G}} (L[m_1](x) - L[m_2](x))(m_1 - m_2)(dx) \ge 0, $$

and the equality in (3.18) holds true if and only if $L[m_1](x) = L[m_2](x)$ for every $x \in \mathcal{G}$. Note that $x \mapsto L[m](x)$ is continuous on $\mathcal{G}$, and $C^2$ on $J_i \setminus \{0\}$, $i = 1, \dots, N$.

**Example 3.5.** With $L$ defined in Example 3.4, consider

$$ \tilde{L}[m](x) = L[m](x) + \sum_{i=1}^{N} a_i \mathbb{1}_{x \in J_i \setminus \{0\}} $$

for a collection $(a_i)_{1 \le i \le N}$ of positive weights. Clearly $\tilde{L}$ is strictly monotone, Lipschitz continuous w.r.t. $m$ (for the Wasserstein distance $Wass_1$) and $\tilde{L}[m]$ is discontinuous in $x$ at $O$.
## 3.8 Regularity of $u$ in the interior of the edges

In what follows, we collect several properties of the value function in a mild solution, starting with easy consequences of the results contained in Section 2. Then we aim at obtaining more accurate information at the points $(x, t)$ such that $x$ lies in the support of $m(t)$, i.e. the points that are actually hit by optimal trajectory.

**Lemma 3.16.** We make the same assumptions as in Theorem 3.6. Let $\mu \in \mathcal{P}_{m_0}(\Gamma)$ be a relaxed MFG equilibrium and $(u, m)$ be the related mild solution.

(a) The function $u$ is locally semi-concave in $(J_i \setminus \{O\}) \times (0, T)$

(b) Lemma 2.33 holds replacing $\Gamma_t^{opt}[x]$ with $\Gamma_t^{opt,\mu}[x]$, in particular, the characterization of the optimal control with the $\partial_x u$ away from the vertex

(c) Corollary 2.34 holds replacing $\Gamma_t^{opt}[x]$ with $\Gamma_t^{opt,\mu}[x]$

(d) Lemma 2.35 holds.

*Proof.* Assumption $(H_3^{MFG})$ entails that the costs $\ell_i$ and $g_i$ associated to $\mu$ in (3.16) fulfill the assumption of Theorem 2.21. We end the proof by applying Proposition 2.31, Lemma 2.33, Corollary 2.34 and Lemma 2.35. $\square$

Next, let us prove that $u$ is a bilateral subsolution (see [10, Definition III.2.27]) of the Hamilton-Jacobi equation and is differentiable at least at the points $(x, t)$ such that $x$ belongs to the support of $m(t)$ and does not coincide with $O$. To this end, some new notations are useful. Set

$$ 
\begin{align*}
\text{supp}(m(t)) &= \{x \in \mathcal{G} : \forall \omega, \text{ open neighborhood of } x, m(t)(\omega) > 0\} \\
Q_m &= \{(x,t) \in \mathcal{G} \times (0,T) : x \in (\mathcal{G} \setminus \{O\}) \cap \text{supp}(m(t))\} \\
\partial Q_m &= \{(O,t) : t \in (0,T], O \in \text{supp}(m(t))\}
\end{align*}
$$

and introduce the subdifferential of $u$ at $(x, t) \in Q_m$ as

$$ D^+u(x,t) = \left\{ (\pi, q) \in \mathbb{R}^2 : \limsup_{y \to x, \theta \to t} \frac{u(y, \theta) - u(x, t) - \pi(\theta - t) - q(\bar{y} - \bar{x})}{|\bar{y} - \bar{x}| + |\theta - t|} \le 0 \right\}, $$

where $x = \bar{x}e_j$, $y = \bar{y}e_j$ (note that $j$ is uniquely defined, from the definition of $Q_m$).

**Remark 3.17.** Similar arguments as those in the proof of [18, Theorem 4.5] yield that for any $t \in (0, T)$, for $\mu$-a.e. $\gamma \in \Gamma$, the point $(\gamma(t), t)$ belongs to $Q_m \cup \partial Q_m$.

**Proposition 3.18.** The assumptions are those of Theorem 3.6 and Corollary 3.14. Let $\mu \in \mathcal{P}_{m_0}(\Gamma)$ be a relaxed MFG equilibrium and $(u, m)$ be the associated mild solution. Then, for any $(x, t) \in Q_m$,

(a) there holds
$$ -p_1 + H(x, t, p_2) = 0 \quad \forall (p_1, p_2) \in D^+u(x, t) $$

(b) $u$ is differentiable at $(x, t)$.
Proof. (a). The arguments are reminiscent of those used in [18, Theorem 4.1]. Fix $(x, t) \in Q_m$ and consider $(p_1, p_2) \in D^+u(x, t)$. Without any loss of generality, let us assume that $x \in J_i$ with $x = \bar{x}e_i$. From Theorem 3.13, $u$ is a viscosity solution to problem (2.80) with the cost $\ell_i$ defined in (3.16) and $g = G[m^\nu(T)]$. Hereafter, for simplicity, we refer to (2.80) as the HJ-problem. Since $u$ is a viscosity subsolution to the HJ-problem,

$$-p_1 + H(x, t, p_2) \leq 0.$$

Let us now prove the reverse inequality. Since $x \in \text{supp}(m(t))$, there exists a trajectory $(\gamma, \gamma') \in \Gamma^{\text{opt}}[\gamma(0)]$ with $\gamma(t) = x$. Let $r$ be small enough such that $\gamma(t-s) \in J_i \setminus \{O\}$ for every $s \in [0, r]$; we write $\gamma(t-s) = \bar{\gamma}(t-s)e_i$. From the definition of the subdifferential,

$$u(\gamma(t-s), t-s) - u(x, t) \leq -p_1 s - p_2(x - \bar{\gamma}(t-s)) + o(r) = -p_1 s - p_2 \int_{t-s}^{t} \gamma'(\tau)d\tau + o(r).$$

On the other hand, from Remark 2.7, $(\gamma_{|[t-s,T]}, \gamma'_{|[t-s,T]})$ and $(\gamma_{|[t,T]}, \gamma'_{|[t,T]})$ belong respectively to $\Gamma_{t-s}^{\text{opt}}[\gamma(t-s)]$ and to $\Gamma_t^{\text{opt}}[\gamma(t)]$. Hence,

$$u(\gamma(t-s), t-s) - u(x, t) = \int_{t-s}^{t} \left( \frac{|\gamma'(\tau)|^2}{2} + \ell[m(\tau)](\gamma(\tau)) \right) d\tau.$$

The latter two observations yield

$$\int_{t-s}^{t} \left( \frac{|\gamma'(\tau)|^2}{2} + \ell_i[m(\tau)](\gamma(\tau)) \right) d\tau \leq -p_1 s - p_2 \int_{t-s}^{t} \gamma'(\tau)d\tau + o(r).$$

Next, the regularity of $m$ (see Theorem 3.6) and of $\gamma$ in $(t-s, t)$ (see the Euler-Lagrange relation in Lemma 2.17) entail

$$\ell_i[m(\tau)](\gamma(\tau)) = \ell_i[m(t)](x) + O(s), \quad \text{and} \quad \gamma'(\tau) = \gamma'(t) + o(1)$$

for any $\tau \in (t-s, t)$. This implies

$$\frac{1}{s} \int_{t-s}^{t} \left( \frac{|\gamma'(t)|^2}{2} + \ell_i[m(t)](x) \right) d\tau + o(1) \leq -p_1 - p_2\gamma'(t) + o(1).$$

Letting $s \to 0^+$, we infer

$$0 \leq -p_1 - p_2\gamma'(t) - \frac{|\gamma'(t)|^2}{2} - \ell_i[m(t)](x) \leq -p_1 + H(x, t, p_2)$$

where the last inequality comes from the definition of $H$, see (2.78).

(b). Point (b) is obtained with the arguments in the proof of [18, Proposition 4.2] replacing [18, Theorem 4.1] and [18, Corollary 4.1] respectively with point (a) and Lemma 3.16-(a). $\square$

<figure><img src="image_6.png" /></figure>

## 3.9 Properties of $m$

Consider a mild solution $(u, m)$ associated to a given relaxed MFG equilibrium $\mu \in \mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})$. Here, we wish to investigate the behaviour of the point masses of $m$ if they exist.
Let us recall from Example 3.1 and Example 3.2 that $m$ may develop a singularity of the form of a point mass at the origin and that the latter singularity may be transported into the edges. Below, we prove that each singular point conserves its mass when it travels in the interior of an edge. This implies that point masses cannot appear/vanish in the interior of a given edge. In particular, the creation of a point mass can occur only at the vertex. Finally, we provide an example with two vertices in which $m$ is a Dirac mass at the first vertex until some time $t_1$, a Dirac mass at the second vertex after $t_2 > t_1$, and in which there is no mass points between the two vertices at all $t$, $t_1 < t < t_2$.

**Theorem 3.19.** Under the assumptions of Theorem 3.6, let $\mu \in \mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})$ be a relaxed MFG equilibrium and $(u, m)$ be the corresponding mild solution. Consider $x \in \text{supp}(m(t)) \cap J_j \setminus \{O\}$ for some $j = 1, \dots, N$ and $t \in (0, T)$. The following holds:

(a) there exists $x_0 \in \text{supp}(m_0)$ and $\gamma \in \Gamma_{C,V}^{\text{Lip}}$ with $(\gamma, \alpha_\gamma) \in \Gamma^{\mu, \text{opt}}[x_0]$ and $\gamma(t) = x$ (recall that the control $\alpha_\gamma$ was introduced in Remark 3.2)

(b) for $i = 1, 2$, consider $x_{0,i}$ and $\gamma_i$ satisfying point (a) and denote $t_{*,i} = \sup\{s \in [0, t]; \gamma_i(s) = O\}$ and $t_{*,i} = 0$ if the latter set is empty and, similarly, $t^{*,i} = \inf\{s \in [t, T]; \gamma_i(s) = O\}$ and $t^{*,i} = T$ if the latter set is empty. Then, there holds

$$t_{*,1} = t_{*,2} =: t_* \qquad t^{*,1} = t^{*,2} =: t^* \qquad \text{and} \qquad \gamma_1 = \gamma_2 \quad \text{on } (t_*, t^*)$$

(c) for every $s \in (t_*, t^*)$ and every $\gamma$ as in point (a), there holds: $m(s)(\{\gamma(s)\}) = m(t)(\{x\})$.

*Proof.* (a). For every positive $r$, $m(t)(B(x, r)) > 0$. Hence,

$$0 < m(t)(B(x, r)) = \int_G 1_{\{\xi \in G \cap B(x, r)\}} m(t)(d\xi) \\ = \int_{\{y \in \Gamma_{C,V}^{\text{Lip}} : (y, \alpha_y) \in \Gamma^{\mu, \text{opt}}, y(0) \in \text{supp}(m_0)\}} 1_{\{y : y(t) \in B(x, r)\}} \mu(dy)$$

where the last equality comes from the definition of $m$. Consequently the set

$$E = \{y \in \Gamma_{C,V}^{\text{Lip}} : (y, \alpha_y) \in \Gamma^{\mu, \text{opt}}, y(0) \in \text{supp}(m_0), y(t) \in B(x, r)\}$$

is not empty for every $r > 0$. We infer that there exist a sequence $\{x_n\}_n$, with $x_n \in B(x, 1/n)$ and a sequence $\gamma_n \in \Gamma_{C,V}^{\text{Lip}}$ with $(\gamma_n, \alpha_{\gamma_n}) \in \Gamma^{\mu, \text{opt}}$, $\gamma_n(0) \in \text{supp}(m_0)$ and $\gamma_n(t) = x_n$. By standard arguments, we see that, as $n \to \infty$, $\gamma_n$ uniformly converge to some path $\gamma \in \Gamma_{C,V}^{\text{Lip}}$ with $\gamma(0) \in \text{supp}(m_0)$ and $\gamma(t) = x$. From the stability of optimal trajectories, $(\gamma, \alpha_\gamma)$ belongs to $\Gamma^{\mu, \text{opt}}$. Point (a) is proved.

(b). It is a direct consequence of Lemma 3.16-(c).

(c). Consider $s \in (t_*, t^*)$. The definition of $m$ entails

$$(3.19) \qquad m(t)(\{x\}) = \int_{\{y \in \Gamma_{C,V}^{\text{Lip}} : (y, \alpha_y) \in \Gamma^{\mu, \text{opt}}, y(0) \in \text{supp}(m_0)\}} 1_{\{y : y(t) = \gamma(t)\}} \mu(dy)$$

where $\alpha_y$ is the control defined in Remark 3.2. From point (b), there holds

$$\{y \in \Gamma_{C,V}^{\text{Lip}} : (y, \alpha_y) \in \Gamma^{\mu, \text{opt}}, y(0) \in \text{supp}(m_0), y(t) = \gamma(t)\} \\ = \{y \in \Gamma_{C,V}^{\text{Lip}} : (y, \alpha_y) \in \Gamma^{\mu, \text{opt}}, y(0) \in \text{supp}(m_0), y(s) = \gamma(s)\}$$
for every $s \in (t_*, t^*)$. Combining the latter identity and (3.19) yields

$$m(t)(\{x\}) = \int_{\{y \in \Gamma_{C,V}^{\text{Lip}} : (y, \alpha_y) \in \Gamma^{\mu, \text{opt}}, y(0) \in \text{supp}(m_0)\}} 1_{\{y : y(s) = \gamma(s)\}} \mu(dy) = m(s)(\{\gamma(s)\})$$

for every $s \in (t_*, t^*)$ which is our statement.

<figure><img src="image_7.png" /></figure>

The following result is direct consequence of Theorem 3.19-(c).

**Proposition 3.20.** Under the hypotheses of Theorem 3.6, let $\mu \in \mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})$ be a relaxed MFG equilibrium and $(u, m)$ be the corresponding mild solution. For every $\gamma$ as in Theorem 3.19-(a) such that there exists $t \in (0, T)$ with $\gamma(t) \in J_i \setminus \{O\}$, there holds

$$m(s)(\{\gamma(s)\}) = m(s')(\{\gamma(s')\}) \quad \forall s, s' \in (t_*, t^*)$$

where

$$t_* = \sup\{s \in [0, t] : \gamma(s) = O\}, \quad t^* = \inf\{s \in [t, T] : \gamma(s) = O\}$$

$(t_* = 0$ and respectively $t^* = T$ when the corresponding set is empty). This implies that if $m$ has a point mass, then the latter is conserved as long as it stays in the interior of a given edge.

We now focus on the case when no optimal trajectory hits $x$ at time $t$.

**Proposition 3.21.** Let $\mu \in \mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})$ be a relaxed MFG equilibrium and $(u, m)$ be the corresponding mild solution. If $m(t)(\{x\}) = 0$ and point (a) in Theorem 3.19 does not hold, then there exists $\delta > 0$ such that

$$m(s)(\{x\}) = 0 \quad \forall s \in (t - \delta, t + \delta).$$

*Proof.* From Theorem 3.19-(a), $x \notin \text{supp}(m(t))$. Then, there exists a positive number $r$ such that $B(x, r) \cap \text{supp}(m(t)) = \emptyset$ and, consequently, the set

$$E = \{y \in \Gamma_{C,V}^{\text{Lip}} : (y, \alpha_y) \in \Gamma^{\mu, \text{opt}}, y(0) \in \text{supp}(m_0), y(t) \in B(x, r)\}$$

is negligible for the measure $\mu$. Taking into account the uniform Lipschitz continuity of the optimal trajectories, we obtain that there exists a sufficiently small $\delta > 0$ such that

$$E(s) := \{y \in \Gamma_{C,V}^{\text{Lip}} : (y, \alpha_y) \in \Gamma^{\mu, \text{opt}}, y(0) \in \text{supp}(m_0), y(s) \in B(x, r/2)\} \subset E$$

for every $s \in (t - \delta, t + \delta)$. Since $\mu$ is complete, $E(s)$ is also negligible for $\mu$ and

$$m(s)(B(x, r/2)) = 0 \quad \forall s \in (t - \delta, t + \delta),$$

which achieves the proof.

<figure><img src="image_8.png" /></figure>

We now provide an example with two vertices in which

* there is a Dirac mass at the first vertex which disappears

* a Dirac mass arises at the second vertex

* no Dirac mass travels in the edge between the two vertices.
**Example 3.6.** With the same network as in Example 3.2, consider the costs which do not depend on m (no interaction between the agents):

$$L[m](x) = \begin{cases} K_L & \text{if } x \in G \setminus J_2, \\ 1 & \text{if } x \in J_2 \setminus \{V_1, V_2\}, \\ 0 & \text{if } x \in \{V_1, V_2\}, \end{cases} \quad \text{and} \quad G[m](x) = \begin{cases} K_G & \text{if } x \neq V_2, \\ 0 & \text{if } x = V_2, \end{cases}$$

where $K_L > 1$ and $K_G$ are positive constants that will chosen later (note that they fulfill assumptions (2.6) and (2.7)). The time horizon $T$ will be chosen later. Take $m_0 = \delta_{V_1}$. It is obvious that every optimal trajectory $(y, \alpha)$ with $y(0) = V_1$ must remain at $V_1$ until a time $\tau_1 \in (0, T]$, then move inside $J_2$ if $\tau_1 < T$ so to reach $V_2$ at some time $\tau_2 \in (\tau_1, T]$ and finally remain at $V_2$ until $T$. The constants $T$, $K_L$ and $K_G$ will be chosen sufficiently large so that $\tau_1 < T$.

From the Euler-Lagrange condition (2.26), there exists $c \in \mathbb{R}$ such that $\alpha(s) = c$ for $s \in (\tau_1, \tau_2)$ with $\tau_2 = \tau_1 + 1/c$. Hence,

$$y(s) = \begin{cases} V_1 & \text{for } s \in [0, \tau_1], \\ c(s - \tau_1)e_2 & \text{for } s \in (\tau_1, \tau_1 + 1/c], \\ V_2 & \text{for } s \in (\tau_2, T]. \end{cases}$$

Let us minimize the cost $J_0(V_1; (y, \alpha))$ with respect to $\tau_1$ and $c$. There holds

$$J_0(V_1; (y, \alpha)) = \int_{\tau_1}^{\tau_1+1/c} \left( \frac{c^2}{2} + 1 \right) ds = \frac{c}{2} + \frac{1}{c}.$$

Hence, the minimum of $J_0(V_1; (y, \alpha))$ is achieved by $c = \sqrt{2}$ independently of $\tau_1$.

Let us take $T$ larger than $1 + 1/\sqrt{2}$ and introduce the family $\{y_\tau\}_{\tau \in [0,1]}$

$$y_{\tau}(s) = \begin{cases} V_1 & \text{for } s \in [0, \tau], \\ \sqrt{2}(s - \tau)e_2 & \text{for } s \in (\tau, \tau + 1/\sqrt{2}], \\ V_2 & \text{for } s \in (\tau + 1/\sqrt{2}, T], \end{cases}$$

which are all optimal from the above calculations. There exists a positive constant $C$ sufficiently large such each $y_\tau$ belongs to $\tilde{\Gamma}_C$. Define the measure $\mu$ on $\tilde{\Gamma}_C$ (defined in (3.2)) as follows: for all Borel set $A \subset \tilde{\Gamma}_C$,

$$\mu(A) = \mathcal{L}(\{\tau \in [0, 1] : y_{\tau} \in A\}),$$

where $\mathcal{L}$ is the Lebesgue measure. The measure $\mu$ fulfills

$$\bullet \ \mathrm{supp}(\mu) \subset \Gamma_C^{\mu, \mathrm{opt}}[V_1]$$

$$\bullet \ e_0 \# \mu(\{V_1\}) = \mu(\{\gamma \in \tilde{\Gamma}_C : \gamma(0) = V_1\}) = \mathcal{L}(\{\tau \in [0,1] : y_\tau(0) = V_1\}) = 1 = m_0(\{V_1\});$$

therefore, the measure $\mu$ is a relaxed MFG equilibrium. Let $(u, m)$ be the corresponding mild solution. We claim that for all $x \in G \setminus \{V_1, V_2\}$ and all $t \in [0, T]$

$$m(t)(\{x\}) = 0.$$

Indeed,

$$m(t)(\{x\}) = \mu(\{\gamma \in \tilde{\Gamma}_C : \gamma(t) = x\}) = \mathcal{L}(\{\tau \in [0,1] : y_\tau(t) = x\}) = 0,$$

the last equality is true since the set $\{\tau \in [0,1] : y_\tau(t) = x\}$ contains at most one value.
### 3.10 The continuity equation

Consider a mild solution $(u, m)$ associated to some relaxed MFG equilibrium $\mu \in \mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})$. Here, we make the same hypotheses as in Theorem 3.6, and we study the evolution of the distribution $m(t)$. We obtain that $m$ satisfies (in a suitable weak sense) a continuity equation in which the drift is given as the optimal feedback from the Hamilton-Jacobi equation.

**Theorem 3.22.** Under the hypotheses of Theorem 3.6, let $\mu \in \mathcal{P}_{m_0}(\Gamma_{C,V}^{\text{Lip}})$ be a relaxed MFG equilibrium and $(u, m)$ be a related mild solution. Then, for every $\phi \in C^\infty(\mathcal{G} \times [0, T])$ such that $\text{supp}(\phi(\cdot, t))$ is contained in a compact subset of $\mathcal{G}$ independent of $t$, there holds

$$ (3.20) \quad m(t)({\{O\}})\partial_t\phi(O,t) + \sum_{i=1}^N \int_{\mathcal{G}} \mathbb{1}_{x \in J_i \setminus \{O\}} [\partial_t\phi(x,t) - Du(x,t)D\phi(x,t)] m(t)(dx) \\ = \frac{d}{dt} \left[ \int_{\mathcal{G}} \phi(x,t)m(t)(dx) \right]. $$

For any $i = 1, \dots, N$,

$$ (3.21) \quad q_i = \frac{d}{dt} \left( \int_{\Gamma} \mathbb{1}_{\gamma(\cdot) \in J_i \setminus \{O\}} \mu(d\gamma) \right) = \frac{d}{dt} \left( m(\cdot) (J_i \setminus \{O\}) \right), $$

is well defined in $\mathcal{D}'(0, T)$, and there holds

$$ (3.22) \quad \frac{d}{dt} [m(\cdot)(\{O\})] + \sum_{i=1}^{N} q_i = 0 $$

in the sense of $\mathcal{D}'(0, T)$.

**Remark 3.23.** In fact, in the proof of Theorem 3.22, we will also obtain that, for all $i = 1, \dots, N$,

$$ (3.23) \quad \frac{d}{dt} \left( \int_{\Gamma} \mathbb{1}_{\gamma(\cdot) \in J_i \setminus \{O\}} \phi(\gamma(\cdot), \cdot) \mu(d\gamma) \right) = \\ \phi(O, \cdot)q_i + \int_{\mathcal{G}} \mathbb{1}_{x \in J_i \setminus \{O\}} [\partial_t\phi(x, \cdot) - Du(x, \cdot)D\phi(x, \cdot)] m(\cdot)(dx) $$

in the sense of $\mathcal{D}'(0, T)$.

**Remark 3.24.** Equation (3.20) implies in particular that for all $\phi \in C_0^\infty(\mathcal{G} \times (0, T))$,

$$ \int_0^T m(t)(\{O\})\partial_t\phi(O,t)dt \\ + \sum_{i=1}^N \int_0^T \int_{\mathcal{G}} \mathbb{1}_{x \in J_i \setminus \{O\}} [\partial_t\phi(x,t) - Du(x,t)D\phi(x,t)] m(t)(dx)dt = 0. $$

Before giving the proof of Theorem 3.22, let us state a few useful lemmas.

**Lemma 3.25.** For any $x \in \mathcal{G}$, $t \mapsto m(t)(\{x\})$ is a measurable bounded function on $[0, T]$. In particular, it admits a derivative in $\mathcal{D}'(0, T)$. Moreover, for all $x \in \mathcal{G}$, the set $\{t \in [0, T] : m(t)(\{x\}) > 0\}$ is measurable.