# A Lagrangian approach for aggregative mean field games of controls with mixed and final constraints

J. Frédéric Bonnans, Justina Gianatti, Laurent Pfeiffer

► To cite this version:

J. Frédéric Bonnans, Justina Gianatti, Laurent Pfeiffer. A Lagrangian approach for aggregative mean field games of controls with mixed and final constraints. SIAM Journal on Control and Optimization, 2023, 61 (1), pp.105-134. 10.1137/21M1407720. hal-03174167v2

HAL Id: hal-03174167

https://inria.hal.science/hal-03174167v2

Submitted on 17 Jan 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A LAGRANGIAN APPROACH FOR AGGREGATIVE MEAN FIELD GAMES OF CONTROLS WITH MIXED AND FINAL CONSTRAINTS

*

J. FRÉDÉRIC BONNANS†, JUSTINA GIANATTI‡, AND LAURENT PFEIFFER†

**Abstract.** The objective of this paper is to analyze the existence of equilibria for a class of deterministic mean field games of controls. The interaction between players is due to both a congestion term and a price function which depends on the distributions of the optimal strategies. Moreover, final state and mixed state-control constraints are considered, the dynamics being nonlinear and affine with respect to the control. The existence of equilibria is obtained by Kakutani's theorem, applied to a fixed point formulation of the problem. Finally, uniqueness results are shown under monotonicity assumptions.

**Key words.** Mean field games of controls, aggregative games, constrained optimal control, Lagrangian equilibria

**AMS subject classifications.** 49K15, 49N60, 49N80, 91A07, 91A16

**1. Introduction.** In this article we consider a Nash equilibrium problem involving a large number $N$ of agents, each of them solving a deterministic optimal control problem involving control-affine nonlinear dynamics, final state constraints, and mixed state-control constraints. The agents may only differ from each other by their initial condition. The interaction between the agents is induced by a price variable and a congestion term, which are determined by the collective behavior of the agents. Our mathematical analysis focuses on an equilibrium problem which models the asymptotic limit when $N$ goes to infinity and when each isolated agent is supposed to have no impact on the coupling terms (the price variable and the congestion term). Therefore the problem falls into the class of mean field games (MFGs), which have received considerable attention in the literature since their introduction in the pioneering works by Lasry and Lions [26, 27, 28] and Caines, Huang and Malhamé [23].

Our work addresses two main difficulties. The first difficulty of our model is the interaction induced by the price variable. In the cost function of each agent, the price penalizes linearly the control variable. It is defined as a monotonic function of some aggregative term that can be interpreted as a demand. Here it is the average value of the controls exerted by all agents. This kind of interaction is similar to the one in Cournot models in economics, where companies without market power compete on the amount of some product. Our model is representative from games in energy markets involving a large number of small storage devices and some endogenous price depending on the average speed of charge of the devices. See for instance [2, 18, 29, 31]. The second difficulty is the presence of mixed control-state constraints and final state constraints. They appear naturally in applications in electrical engineering: for example, when the storage devices must be fully (or partially) loaded at the end of the time frame. In the appendix, we motivate the use of mixed constraints with an

<footnote>*The first author was partially supported by the FiME Lab Research Initiative (Institut Europlace de Finance). This article benefited from the support of the FMJH Program PGMO and from the support to this program from EDF.</footnote>

<footnote>†Université Paris-Saclay, CNRS, CentraleSupélec, Inria, Laboratoire des signaux et systèmes, 91190, Gif-sur-Yvette, France (frederic.bonnans@inria.fr, laurent.pfeiffer@inria.fr).</footnote>

<footnote>‡CIFASIS-CONICET-UNR, Ocampo y Esmeralda, S2000EZP, Rosario, Argentina (gianatti@cifasis-conicet.gov.ar).</footnote>
example involving gas storages.

In most MFG models proposed in the literature, the agents interact only through their position (their state variable). Mean field game models with interaction through the states and controls are now commonly called *MFGs of controls*. The terminologies *extended MFGs* and *strongly coupled MFGs* are also employed. Let us review the articles dedicated to such models. In [20], a stationary second order MFG of controls is studied. A deterministic MFG of controls is considered in [21]. An existence result has been obtained for a quite general MFG model in [16]. A uniqueness result is provided in [5]. The works [24, 25] analyse the existence and uniqueness of classical solutions in the second order case. An existence result is provided in the monograph [17, Section 4.6], for MFGs described by forward backward stochastic differential equations. The particular price interaction investigated in this article has been studied in [7] in the second order case and in [22] in the case of a degenerate diffusion and potential congestion terms.

Most MFG models consist of a coupled system of partial differential equations (PDEs), the Fokker-Planck equation and the Hamilton-Jacobi-Bellman (HJB) equation. The presence of final and mixed constraints in the underlying optimal control problem makes it difficult to characterize the behavior of a representative agent with the classical HJB approach. We therefore rely on a Lagrangian formulation of the problem, rather than on a PDE approach. More precisely, our equilibrium problem is posed on the set of Borel probability measures on the space of state-control trajectories. The Lagrangian approach has been employed in several references dealing with deterministic MFGs. Variational MFGs are studied in [4]. The article [30] deals with minimal-time MFGs. The three articles [12, 13, 14] deal with state-constrained MFGs and with the connection between the Lagrangian and the PDE formulations. In [15] MFGs with linear dynamics are considered and in [1] state-constrained MFGs with control on the acceleration are studied.

At a methodological level, the common feature of almost all studies dedicated to MFGs of controls is the introduction of an auxiliary mapping, which allows to put the equilibrium problem in a reduced form that can be handled with a fixed point approach. In the PDE approach, the auxiliary mapping allows to express the control of a representative agent at a given time $t$ in function of its current state $x$, the equilibrium distribution (of the states) and the gradient of the value function (see for example [17, Lemma 4.60], [16, Lemma 5.2] or [7, Lemma 5]). This relation is in general not explicit, contrary to MFGs with interaction through the state variable only. In the probabilistic approach of [21, Assumption G], the auxiliary mapping depends on $t$, $x$, and a pair of random variables $(X_t, P_t)$, whose distribution coincides with the distribution of pairs of state-costate of all agents in the game. In [17, Lemma 4.61], the auxiliary mapping directly depends on the distribution of $(X_t, P_t)$. Our roadmap is the same as the one used in the references mentioned above: we introduce an auxiliary mapping (of the same nature as the one in [17]) which allows to write the equilibrium problem in a reduced form which is then tractable with a fixed point argument. After reformulation, the equilibrium problem is posed on the set of Borel probability measures on the space of state-costate trajectories.

Our article is one of the very few publications dealing with first order MFGs of controls and Lagrangian formulation for these problems. (i) The article of Gomes and Voskanyan [21] is the closest to our work. Their analysis relies in a quite crucial manner on some regularity properties of the value function associated with the underlying optimal control problem (Lipschitz continuity, semiconcavity) which are easily demonstrated in their framework without constraints. Those properties are not
needed in the Lagrangian framework. They could probably be established, but under stronger qualification conditions than those in force in the present work. Incidentally, the initial distribution of the agents must have a density in [21], which is not the case in the present work. (ii) Carmona and Delarue have an existence result, for an MFG of controls posed as a forward-backward stochastic differential equation, see [17, Proposition 4.64]. This model relies on Pontryagin's principle, which is a sufficient condition only under convexity assumptions on the underlying optimal control problem (see the assumption SMP [17, page 161]), which we do not need. Let us mention that their other result [17, Proposition 4.64] concerns the second order case. (iii) In a recent work, Graber, Mullenix and Pfeiffer have obtained the existence of a solution for an MFG of controls formulated as a coupled system of possibly degenerate PDEs. This work is restricted to the potential case, when the local congestion term is the derivative of some convex function. It also relies on a periodicity condition on the data functions, which we do not need here. (iv) Recently in [32], the authors study the existence of a Lagrangian equilibrium for an MFG of controls, following a variational approach instead of solving a fixed point problem, as proposed here.

The paper is organized as follows: In Section 2 we present the problem that we address here, referred to as MFGC. We introduce the main notation and we define the notion of Lagrangian equilibria for MFGC that we use throughout this work. In Section 3 we study the optimal control problem associated with an individual player, providing optimality conditions and regularity of solutions. Defining an auxiliary notion of equilibria, by a fixed point argument, in Section 4 we prove the existence of Lagrangian equilibria. In Section 5, under additional monotonicity assumptions we analyze the uniqueness of solutions.

## 2. Description of the aggregative MFGC problem.

**2.1. Preliminaries.** Let $(X, d)$ be a separable metric space. We denote by $\mathcal{P}(X)$ the set of Borel probability measures on $X$. Given $p \in [1, +\infty)$, it is defined $\mathcal{P}_p(X)$ as the set of probability measures $\mu$ on $X$ such that

$$ \int_X d(x, x_0)^p d\mu(x) < +\infty, $$

for some (and thus any) $x_0 \in X$. The Monge-Kantorovich distance on $\mathcal{P}_p(X)$ is given by

$$ d_p(\mu, \nu) = \inf_{\pi \in \Pi(\mu, \nu)} \left[ \int_X d(x, y)^p d\pi(x, y) \right]^{\frac{1}{p}}, $$

where $\Pi(\mu, \nu)$ denotes the set of probability measures on $X \times X$ with first and second marginals equal to $\mu$ and $\nu$ respectively. In this paper, we work with $p = 1$. For all $\mu, \nu \in \mathcal{P}_1(X)$, we have the following formula (see [19, Theorem 11.8.2]):

$$ d_1(\mu, \nu) = \sup \left\{ \int_X f(x) d\mu(x) - \int_X f(x) d\nu(x) \mid f: X \to \mathbb{R} \text{ is 1-Lipschitz} \right\}. $$

We recall the definition of narrow convergence of measures. We say that the sequence $(\mu_n)_{n \in \mathbb{N}} \subset \mathcal{P}(X)$ narrowly converges to $\mu \in \mathcal{P}(X)$ if

$$ \lim_{n \to \infty} \int_X f(x) d\mu_n(x) = \int_X f(x) d\mu(x), \quad \forall f \in C_b^0(X), $$

where $C_b^0(X)$ denotes the set of all continuous and bounded real functions defined on $X$. Throughout this work we endow the space $\mathcal{P}_1(X)$ with the narrow topology.
As a consequence of [3, Proposition 7.1.5], for any compact set $K \subset X$, we have for all $p \ge 1$, $\mathcal{P}(K) = \mathcal{P}_p(K)$ and $d_p$ metricizes the narrow convergence of probability measures on the set $\mathcal{P}(K)$. In addition, $\mathcal{P}(K)$ is compact.

**2.2. MFG equilibria and main notation.** We start by defining the optimal control problem that each agent aims to solve, assuming that the price and the distribution of the other players are known. The problem takes the form of a constrained minimization problem parameterized by the initial condition $x_0 \in \mathbb{R}^n$, the agents distribution $m \in C([0, T]; \mathcal{P}_1(\mathbb{R}^n))$ and the price $P \in L^\infty(0, T; \mathbb{R}^m)$.

Let $\Gamma := H^1(0, T; \mathbb{R}^n) \subset C(0, T; \mathbb{R}^n)$ be equipped with the supremum norm, denoted by $\| \cdot \|_\infty$. Given $x_0 \in \mathbb{R}^n$, we define $\Gamma[x_0]$ by

$$ \Gamma[x_0] = \{\gamma \in \Gamma : \gamma(0) = x_0\}. $$

We take $L^2(0, T; \mathbb{R}^m)$ as the control space, which we denote by $\mathcal{U}$. We denote by $\mathcal{K}[x_0]$ the feasible set that is defined by

$$ \mathcal{K}[x_0] = \left\{ \begin{array}{ll} \dot{\gamma}(t) = a(\gamma(t)) + b(\gamma(t))v(t), & \text{for a.e. } t \in (0, T), \\ \gamma(0) = x_0, & \\ (\gamma, v) \in \Gamma \times \mathcal{U} : \begin{array}{l} c(\gamma(t), v(t)) \le 0, \\ g_1(\gamma(T)) = 0, \\ g_2(\gamma(T)) \le 0 \end{array} & \text{for a.e. } t \in (0, T), \end{array} \right\}. $$

The dynamics coefficients are $a: \mathbb{R}^n \to \mathbb{R}^n$ and $b: \mathbb{R}^n \to \mathbb{R}^{n \times m}$ (note that $b_i(x) \in \mathbb{R}^n$ will denote the $i$-th column of $b(x)$). The final equality and inequality constraint functions are, respectively, $g_1: \mathbb{R}^n \to \mathbb{R}^{n_{g_1}}$, and $g_2: \mathbb{R}^n \to \mathbb{R}^{n_{g_2}}$, and the state-control constraint function is $c: \mathbb{R}^n \times \mathbb{R}^m \to \mathbb{R}^{n_c}$. Now we define the cost functional $J[m, P]: \Gamma \times \mathcal{U} \to \mathbb{R}$ as

$$ J[m, P](\gamma, v) = \int_{0}^{T} \left( L(\gamma(t), v(t)) + \langle P(t), v(t) \rangle + f(\gamma(t), m(t)) \right) dt + g_0(\gamma(T), m(T)). $$

Here $L: \mathbb{R}^n \times \mathbb{R}^m \to \mathbb{R}$ represents the running cost of the agents, $f: \mathbb{R}^n \times \mathcal{P}_1(\mathbb{R}^n) \to \mathbb{R}$, the congestion function, and $g_0: \mathbb{R}^n \times \mathcal{P}_1(\mathbb{R}^n) \to \mathbb{R}$ is the final cost. Therefore, the optimal control problem that each agent addresses is

$$ (2.1) \qquad \min_{(\gamma,v) \in \mathcal{K}[x_0]} J[m, P](\gamma, v). $$

The set of optimal trajectories for this minimization problem is denoted by

$$ (2.2) \qquad \Gamma[m, P, x_0] = \{\bar{\gamma} \in \Gamma[x_0] : \exists \bar{v} \in \mathcal{U}, (\bar{\gamma}, \bar{v}) \text{ is a solution to (2.1)}\}. $$

**2.2.1. Lagrangian MFGC equilibria.** In the previous paragraph, we have described the optimization problem, for a particular player, given the price and the agents distribution. We describe now how the price is related to the collective behavior of all agents and give a Lagrangian description of our mean field game.

Let $m_0 \in \mathcal{P}_1(\mathbb{R}^n)$ be the initial distribution of the agents. We fix a price function $\psi: \mathbb{R}^m \to \mathbb{R}^m$, which is assumed to be bounded. For $t \in [0, T]$, the mapping $e_t: \Gamma \times \mathcal{U} \to \mathbb{R}^n$ is given by $e_t(\gamma, v) = \gamma(t)$. We define the set

$$ \mathcal{P}_{m_0}(\Gamma \times \mathcal{U}) = \{\eta \in \mathcal{P}_1(\Gamma \times \mathcal{U}) : e_0 \# \eta = m_0\}. $$
Given $\eta \in \mathcal{P}_{m_0}(\Gamma \times \mathcal{U})$, we define the cost functional $J^\eta = J[m^\eta, P^\eta]$, where the coupling terms $m^\eta: t \in [0, T] \mapsto m_t^\eta \in \mathcal{P}_1(\mathbb{R}^n)$ and $P^\eta \in L^\infty(0, T; \mathbb{R}^m)$ are given by

$$ (2.3) \qquad m_t^\eta = e_t \# \eta \quad \text{and} \quad P^\eta = \Psi\left(\int_{\Gamma \times \mathcal{U}} v \, d\eta(\gamma, v)\right). $$

The continuity of the mapping $t \mapsto m_t^\eta$ will be ensured by Lemma 4.2. In the definition of $P^\eta$, $\int_{\Gamma \times \mathcal{U}} v \, d\eta(\gamma, v)$ is a Bochner integral with value in $L^2(0, T; \mathbb{R}^m)$ (which is well defined since $\eta \in \mathcal{P}_1(\Gamma \times \mathcal{U})$) and the mapping $\Psi: \theta \in L^2(0, T; \mathbb{R}^m) \to \Psi[\theta] \in L^\infty(0, T; \mathbb{R}^m)$ denotes the Nemytskii operator associated with the price function $\psi: \mathbb{R}^m \to \mathbb{R}^m$, defined by $\Psi[\theta](t) = \psi(\theta(t))$, for a.e. $t \in (0, T)$.

Given $x_0 \in \mathbb{R}^n$ and $\eta \in \mathcal{P}_{m_0}(\Gamma \times \mathcal{U})$, we denote by $\Gamma^\eta[x_0]$ the set of optimal state-control trajectories associated with the cost $J^\eta$ and set of constraints $\mathcal{K}[x_0]$:

$$ \Gamma^\eta[x_0] = \{ (\bar{\gamma}, \bar{v}) \in \mathcal{K}[x_0] : J^\eta(\bar{\gamma}, \bar{v}) \le J^\eta(\gamma, v) \quad \forall (\gamma, v) \in \mathcal{K}[x_0] \}. $$

**DEFINITION 2.1.** We call Lagrangian MFGC equilibrium any distribution $\eta \in \mathcal{P}_{m_0}(\Gamma \times \mathcal{U})$ supported on optimal trajectories, i.e.

$$ \text{supp}(\eta) \subset \bigcup_{x \in \text{supp}(m_0)} \Gamma^\eta[x]. $$

The main objective of this paper is to prove the existence of a Lagrangian MFGC equilibrium, under the assumptions described in the following subsection.

**2.3. Assumptions.** For a given normed vector space $X$, we denote by $\bar{B}_X(R)$ the closed ball of radius $R$ and center 0. When the context is clear, we simply write $\bar{B}(R)$. Given $R > 0$, we denote $V(R) = \text{conv}\{(x, v) : |x| \le R, c(x, v) \le 0\}$. Finally, **1** stands for a vector of all ones, of appropriate dimension, and inequality between vectors means component-wise inequality.

We consider the following assumptions:

(H1) Convexity assumptions

(i) There exists $C > 0$ such that for all $x \in \mathbb{R}^n$, the mapping $L(x, \cdot)$ is strongly convex with parameter $1/C$ and for all $(x, v) \in \mathbb{R}^n \times \mathbb{R}^m$,

$$ L(x, v) \ge (1/C) |v|^2 - C. $$

(ii) For all $x \in \mathbb{R}^n$ and $i = 1, ..., n_c$, the mapping $c_i(x, \cdot)$ is convex.

(iii) The mapping $\psi$ is monotone, i.e. $\langle \psi(y) - \psi(x), y - x \rangle \ge 0$, for all $x$ and $y$ in $\mathbb{R}^m$.

(H2) Regularity assumptions

(i) The mappings $L$ and $c$ are twice continuously differentiable.

(ii) The mappings $a, b, g_0, g_1$, and $g_2$ are continuously differentiable.

(iii) For all $m \in \mathcal{P}_1(\mathbb{R}^n)$, the mapping $f(\cdot, m)$ is continuously differentiable. The mappings $f$ and $D_x f$ are continuous with respect to both variables.

(iv) The mapping $\psi$ is continuous.

(H3) Boundedness and growth assumptions

(i) Let $R > 0$. Then there exists $C(R) > 0$ such that, for all $(x, v) \in V(R)$,

$$ |D_x L(x, v)| \le C(R)(1 + |v|^2), $$

$$ |D_v L(x, v)| \le C(R)(1 + |v|). $$
(ii) Let $R > 0$. Then there exists $C(R) > 0$ such that, for all $(x, v)$ and $(\tilde{x}, \tilde{v})$ in $V(R) + \bar{B}(1)$,

$$|D_x c(x, v)| + |D_v c(v, x)| \leq C(R), \\ |D_v c(x, v) - D_x c(\tilde{x}, \tilde{v})| \leq C(R) |(x, v) - (\tilde{x}, \tilde{v})|.$$

(iii) There exists $C > 0$ such that for all $x \in \mathbb{R}^n$,

$$|a(x)| \leq C(1 + |x|) \quad \text{and} \quad |b(x)| \leq C(1 + |x|).$$

(iv) The support $K_0$ of $m_0$ is bounded.

(v) There exists $C > 0$ such that for all $x_0 \in K_0$, $m \in C(0, T; \mathcal{P}_1(\mathbb{R}^n))$ and $(\gamma, v) \in \mathcal{K}[x_0]$,

$$\int_0^T f(\gamma(t), m(t))dt + g_0(\gamma(T), m(T)) \geq -C \\ \|D_x f(\gamma(t), m(t))\|_{L^\infty(0,T;\mathbb{R}^n)} \leq C.$$

(vi) The mapping $\psi$ is bounded.

(H4) Feasibility assumptions

(i) Let $R > 0$. Then there exists a constant $C(R) > 0$ such that, for all $x \in \bar{B}(R)$, there exists $v \in \bar{B}(C(R))$ satisfying $c(x, v) \leq 0$.

(ii) There exists $C > 0$ such that for all $x_0 \in K_0$, $m \in C(0, T; \mathcal{P}(\mathbb{R}^n))$ with $m(0) = m_0$, and for all $P \in L^\infty(0, T; \mathbb{R}^m)$ satisfying

$$(2.4) \qquad \|P\|_{L^\infty(0,T;\mathbb{R}^m)} \leq \sup_{\theta \in \mathbb{R}^m} |\psi(\theta)|,$$

there exists $(\gamma_0, v_0) \in \mathcal{K}[x]$ such that $J[m, P](\gamma_0, v_0) \leq C$.

(H5) Qualification assumptions

(i) There exists $C > 0$ such that for all $x_0 \in K_0$, $(\gamma, v) \in \mathcal{K}[x_0]$, and $z_1 \in \mathbb{R}^{n_{g_1}}$, there exists a pair $(y, w) \in H^1(0, T; \mathbb{R}^n) \times L^\infty(0, T; \mathbb{R}^m)$ solution of the linearized state equation

$$(2.5) \quad \left\{ \begin{array}{l} \dot{y}(t) = (Da(\gamma(t)) + \sum_{i=1}^{m} Db_i(\gamma(t))v_i(t))y(t) + b(\gamma(t))w(t), \\ y(0) = 0, \end{array} \right. $$

such that $Dg_1(\gamma(T))y(T) = z_1$, and

$$\|y\|_{H^1(0,T;\mathbb{R}^n)} \leq C|z_1| \quad \text{and} \quad \|w\|_{L^\infty(0,T;\mathbb{R}^n)} \leq C|z_1|.$$

(ii) There exists $C > 0$ such that for all $x_0 \in K_0$ and $(\gamma, v) \in \mathcal{K}[x_0]$, there exists $(y, w) \in H^1(0, T; \mathbb{R}^n) \times L^\infty(0, T; \mathbb{R}^m)$ satisfying (2.5) such that

$$\left\{ \begin{array}{l} Dg_1(\gamma(T))y(T) = 0, \\ g_2(\gamma(T)) + Dg_2(\gamma(T))y(T) \leq -1/C\mathbf{1}, \\ c(\gamma(t), v(t)) + Dc(\gamma(t), v(t))(y(t), w(t)) \leq -1/C\mathbf{1}. \end{array} \right. $$

In addition, $\|y\|_{H^1(0,T;\mathbb{R}^n)} \leq C$ and $\|w\|_{L^\infty(0,T;\mathbb{R}^n)} \leq C$.
(iii) There exists $C > 0$ such that for all $(x, v) \in \mathbb{R}^n \times \mathbb{R}^m$ satisfying $c(x, v) \le 0$, and for all $\omega \in \mathbb{R}^{|I(x,v)|}$,

$$|D_v c_{I(x,v)}(x, v)^\top \omega| \ge (1/C)|\omega|,$$

where $I(x, v) = \{i = 1, \dots, n_c : c_i(x, u) = 0\}$.

(iv) Given $R > 0$, let $x \in \bar{B}(R)$, and let $v \in \mathbb{R}^m$ be such that $c(x, v) \le 0$. Then there exist $C > 0$, $\varepsilon > 0$, and $w \in \bar{B}(C)$ satisfying

$$c(x, v) + D_v c(x, v) w \le -\varepsilon \mathbf{1}.$$

In addition the constants $C > 0$ and $\varepsilon > 0$ only depend on $R$.

*Remark 2.2.* Let us comment on the nature and the motivation of some of the assumptions introduced above.

1. The first step of our analysis consists in finding a bound in $L^2$ for the optimal controls associated with problem (2.1). This bound must be uniform with respect to $m$, $P$, $x_0$. We proceed with the standard approach from the calculus of variations, which requires:

* the existence of a feasible pair $(\gamma, v)$ with a uniformly bounded cost: this is ensured by (H4)-(ii).

* a lower bound of the cost function $J[m, P](\gamma, v)$ which holds for any feasible pair: this is ensured by (H1)-(i) and (H3)-(v). This also requires a bound on $P$, which is why we also impose that $\psi$ is bounded with Assumption (H3)-(vi).

2. Assumptions (H5)-(i) and (H5)-(ii) together are qualification conditions (for the mixed and final constraints) in the form of Mangasarian-Fromovitz qualification conditions (see [11, Section 2.3.4]). They were employed in a similar context in [9, Section 3].

3. Assumptions (H5)-(iii) and (H5)-(iv) are both qualification conditions for the constraints $c(x, v) \le 0$ for a fixed value of $x$, this is why those qualification conditions only involve partial derivatives of $c$ with respect to $v$. They are used in particular in Lemma 3.1 and Lemma 3.2. They respectively take the form of linear independence qualification conditions and inward pointing conditions. Assumption (H5)-(iii) was used in [8, Equation 2.30] and Assumption (H5)-(iv) was used in [10, Definition 2.5] for example, in similar contexts.

*Remark 2.3.* 1. As was pointed out above, some of the assumptions are used to derive a priori bounds on the optimal controls associated with problem (2.1). However, if the set of feasible controls is bounded, these bounds are much easier to obtain and some simplifications can be done. Assume that there exists a constant $C > 0$ such that for any $(x, v) \in \mathbb{R}^n \times \mathbb{R}^m$, $c(x, v) \le 0 \Rightarrow |v| \le C$. Then it is easy to verify that for any $x_0 \in K_0$, for any $(\gamma, v) \in \mathcal{K}[x_0]$, it holds

$$\|\gamma\|_{L^\infty(0,T;\mathbb{R}^n)} \le C \quad \text{and} \quad \|v\|_{L^\infty(0,T;\mathbb{R}^n)} \le C,$$

increasing if necessary the value of $C$. In this case, the following simplifications can be considered:

* Assumptions (H3)-(i) and (H3)-(ii) are satisfied.

* Assumption (H3)-(v) can be ignored. We already have that $\gamma(t)$ takes values in a bounded set. Moreover, one can require in this assumption
that $m(t)$ lies in a set of probability measure with support included into a bounded set; such a set is compact for the topology of $d_1$. Therefore the bounds follow directly from the continuity of $f$, $D_x f$, and $g_0$.

* It is not necessary to impose that $\psi$ is bounded.

* Assumption (H4)-(ii) boils down to a feasibility assumption (the bound $J[m, P](\gamma_0, v_0) \le C$ is then automatically satisfied).

2. The verification of Assumption (H4)-(ii) is made easier when $f$ is known to be bounded. Then it suffices to assume that there exists $C > 0$ such that for all $x_0 \in K_0$, there exists $(\gamma_0, v_0) \in \mathcal{K}[x_0]$ with $\|v_0\|_{L^2(0,T;\mathbb{R}^m)} \le C$. In such a case, it is easy to deduce a bound of $\gamma_0$ in $L^\infty(0,T;\mathbb{R}^n)$ and finally a bound of $J[m, P](\gamma_0, v_0)$, with the help of Assumption (H3)-(i).

3. In some situations, one can find a convex set $X \subseteq \mathbb{R}^n$ such that for any $x_0 \in K_0$, for any $(\gamma, v) \in \mathcal{K}[x_0]$, for any $t \in [0, T]$, $\gamma(t) \in X$. In this case, the variable $x$ appearing in Assumptions (H1)-(i), (H1)-(ii), (H4)-(i), (H5)-(iii), (H5)-(iv) can be restricted to $X' := (X + B_{\mathbb{R}^n}(\delta))$, where $\delta$ is chosen arbitrarily small. The statements of Lemma 3.1 and Lemma 3.2 remain true for $x \in X$.

*Remark 2.4.* For the sake of simplicity in the presentation of this article, we consider time-independent data, but most of the results remain valid if the above assumptions hold uniformly with respect to time.

**3. The optimal control problem.** In this section, we study the optimal control problem (2.1) that an individual player aims to solve. Throughout this section, we fix a triplet $(m, P, x_0) \in C(0, T; \mathcal{P}_1(\mathbb{R}^n)) \times L^\infty(0, T; \mathbb{R}^m) \times K_0$ such that (2.4) holds.

**3.1. Some technical results.** The next lemma is a metric regularity property, obtained from the Mangasarian-Fromovitz qualification condition (H5)-(iv), which implies Robinson's qualification condition (see [11, Section 2.3.4]). Thus the lemma is a particular case of the Robinson-Ursescu stability theorem [11, Theorem 2.87].

**LEMMA 3.1.** Let $R > 0$. There exist $\delta > 0$ and $C > 0$ such that for all $(x, \tilde{x}, \tilde{v}) \in \bar{B}(R)^2 \times \mathbb{R}^m$ such that $c(\tilde{x}, \tilde{v}) \le 0$ and $|x - \tilde{x}| \le \delta$, there exists $v \in \mathbb{R}^m$ such that

$$c(x, v) \le 0 \quad \text{and} \quad |v - \tilde{v}| \le C|x - \tilde{x}|.$$

Moreover, for fixed $\tilde{x}$ and $\tilde{v}$, $v$ can be constructed as a continuous function of $x$.

*Proof.* Let $R > 0$. The constant $\varepsilon$ used below, as well as all constants $C > 0$, depend only on $R$. Let $(x, \tilde{x}, \tilde{v}) \in \bar{B}(R)^2 \times \mathbb{R}^m$ be such that $c(\tilde{x}, \tilde{v}) \le 0$. By Assumptions (H5)-(iv), there exist $w \in \mathbb{R}^m$, $C > 0$, and $\varepsilon > 0$ such that

$$c(\tilde{x}, \tilde{v}) + D_v c(\tilde{x}, \tilde{v}) w \le -\varepsilon \quad \text{and} \quad |w| \le C.$$

Let $\theta \in [0, 1]$ and let $v_\theta = \tilde{v} + \theta w$. We have

$$(3.1) \qquad c(x, v_\theta) = c(\tilde{x}, \tilde{v}) + \theta D_v c(\tilde{x}, \tilde{v}) w + a_\theta + b_\theta,$$

where

$$a_\theta = \int_0^1 D_x c(\tilde{x} + s(x - \tilde{x}), \tilde{v} + s\theta w)(x - \tilde{x})ds,$$

$$b_\theta = \theta \int_0^1 [D_v c(\tilde{x} + s(x - \tilde{x}), \tilde{v} + s\theta w) - D_v(\tilde{x}, \tilde{v})] w ds.$$
By Assumption (H3)-(ii), we obtain

$$|a_{\theta}| \leq C|x - \tilde{x}| \quad \text{and} \quad |b_{\theta}| \leq C\theta(|x - \tilde{x}| + \theta).$$

It follows from (3.1) that

$$ (3.2) \qquad \begin{aligned} c(x, v_{\theta}) &= (1 - \theta)c(\tilde{x}, \tilde{v}) + \theta[c(\tilde{x}, \tilde{v}) + D_v c(\tilde{x}, \tilde{v})w] + a_{\theta} + b_{\theta} \\ &\leq -\theta\varepsilon + C|x - \tilde{x}| + C\theta^2. \end{aligned} $$

Let us define $\delta = \frac{\varepsilon^2}{4C^2}$ and $\theta = \frac{2C|x-\tilde{x}|}{\varepsilon}$, where $C$ is the constant appearing in the right-hand side of (3.2). We assume now that $|x-\tilde{x}| \leq \delta$ and we fix $v = v_{\theta}$. It remains to verify that $c(x, v) \leq 0$. Note first that $\theta \leq \frac{2C\delta}{\varepsilon} \leq \frac{\varepsilon}{2C}$, by definition of $\delta$. It follows from (3.2) that

$$ c(x, v) \leq -\theta\varepsilon + \underbrace{(C\theta)\theta}_{\leq \varepsilon/2} + C|x - \tilde{x}| \leq -\frac{\theta\varepsilon}{2} + C|x - \tilde{x}| = 0, $$

which concludes the proof. □

LEMMA 3.2. (i) For all $x \in \mathbb{R}^n$ and for all $r \in \mathbb{R}^m$, there exists a unique pair $(v, \nu) \in \mathbb{R}^m \times \mathbb{R}^{n_c}$ such that the following holds:

$$ (3.3) \qquad D_v L(x, v)^{\top} + r + D_v c(x, v)^{\top} \nu = 0, \quad \nu \geq 0, \quad \text{and} \quad \langle \nu, c(x, v) \rangle = 0. $$

We denote it $(v[x, r], \nu[x, r])$.

(ii) Let $R > 0$. The mapping $(x, r) \in \bar{B}(R) \mapsto (v[x, r], \nu[x, r])$ is Lipschitz continuous.

(iii) There exists $C > 0$ such that for all $x \in \mathbb{R}^n$, for all $r_1$ and $r_2$ in $\mathbb{R}^m$, it holds

$$ \langle v_2 - v_1, r_2 - r_1 \rangle + \frac{1}{C} |v_2 - v_1|^2 \leq 0, $$

where $v_j = v[x, r_j]$, for $j = 1, 2$.

Proof. (i) Let $(x, r) \in \mathbb{R}^n \times \mathbb{R}^m$. Consider the optimization problem:

$$ (3.4) \qquad \inf_{v \in \mathbb{R}^m} L(x, v) + \langle r, v \rangle, \quad \text{subject to: } c(x, v) \leq 0. $$

As a consequence of Assumption (H1)-(i), the above cost function is coercive. By Assumption (H4)-(i), there exists $v_0$ such that $c(x, v_0) \leq 0$. Therefore, (3.4) possesses a solution $v$. As a consequence of the qualification assumption (H5)-(iii), the optimality conditions exactly take the form of (3.3). This proves the existence part of the first part of the theorem. Now take a pair $(v, \nu)$ satisfying (3.3). Then, by the strong convexity of $L(x, \cdot)$ and by the convexity of the mappings $c_i(x, \cdot)$, $v$ is the unique solution to (3.4) and $\nu$ is the associated Lagrange multiplier, it is also unique as a consequence of (H5)-(iii).

(ii) Let us prove the Lipschitz continuity of $v[\cdot, \cdot]$, $\nu[\cdot, \cdot]$. We mainly rely on results of [11]. We first reformulate (3.3) as a generalized equation: given $(x, r)$, the pair $(v, \nu)$ satisfies (3.3) if and only if

$$ (3.5) \qquad 0 \in \Phi(v, \nu; x, r) + N(\nu), $$

where $\Phi(v, \nu; x, r) = (D_v L(x, v)^{\top} + r + D_v c(x, v)^{\top} \nu, -c(x, v)) \in \mathbb{R}^{m+n_c}$ and $N(\nu) = \{(0, z) \in \mathbb{R}^{m+n_c}: z \leq 0, \langle z, \nu \rangle = 0\}$, if $\nu \geq 0$, and $N(\nu) = \emptyset$ otherwise. By [11,
Proposition 5.38], $(\bar{v}, \bar{v})$ is a strongly regular solution of (3.5) (in the sense of [11, Definition 5.12]). Note that the required sufficient second-order optimality conditions follow from the strong convexity of $L(x, \cdot)$ and the convexity of $c_i(x, \cdot)$. It follows then from [11, Theorem 5.13] that $v[\cdot, \cdot]$ and $\nu[\cdot, \cdot]$ are locally Lipschitz continuous, and therefore Lipschitz continuous on any compact set, as was to be proved.

(iii) Let us subtract equality (3.3), for $r_1$, from equality (3.3), for $r_2$, and consider the scalar product of the result with $v_2 - v_1$. We obtain

$$ \underbrace{(D_v L(x, v_2) - D_v L(x, v_1))(v_2 - v_1)}_{(a)} + \langle r_2 - r_1, v_2 - v_1 \rangle - \underbrace{\langle D_v c(x, v_2)(v_1 - v_2), v_2 \rangle}_{(b)} - \underbrace{\langle D_v c(x, v_1)(v_2 - v_1), v_1 \rangle}_{(c)} = 0, $$

where $\nu_j = \nu[x, r_j]$, for $j = 1, 2$. To conclude the proof, we just need to bound from below the term (a) and to bound from above (b) and (c). By Assumption (H1)-(i), we have (a) $\ge \frac{1}{C}|v_2 - v_1|^2$, for some constant $C > 0$ independent of $x, r_1$, and $r_2$. Using the complementarity condition, the convexity of $c$ with respect to its second variable (Assumption (H1)-(ii)), and the nonnegativity of $\nu_2$, we further obtain that

$$ (b) = \langle c(x, v_2) + D_v c(x, v_2)(v_1 - v_2), v_2 \rangle \le \langle c(x, v_1), v_2 \rangle \le 0. $$

Similarly, (c) $\le 0$. This concludes the proof. □

Remark 3.3. The twice differentiability of $L$ and $c$, required in Assumption (H2)-(i) is only used for the application of [11, Proposition 5.38] in the proof of Lemma 3.2. It is sufficient to assume that $L$ and $c$ are continuously differentiable if $c$ does not depend on $x$ (i.e. if we just have control constraints instead of mixed state-control constraints). In that case, the Lipschitz continuity is deduced from [11, Proposition 4.32].

**3.2. Estimates for the optimal solutions.** The goal of this section is to derive some a priori bounds for solutions $(\bar{\gamma}, \bar{v})$ to the optimal control problem (2.1) and for the associated costate and Lagrange multipliers. They will be crucial for the construction of an appropriate set of probability measures on state-costate trajectories. We follow a rather standard methodology. The coercivity of $L$, together with other feasibility and bound conditions allows to show the existence of a solution and to derive a bound of $\bar{v}$ in $L^2(0, T; \mathbb{R}^m)$. Then we provide first-order necessary optimality conditions and a bound on the associated costate $p$, with the help of the qualification conditions. We finally obtain bounds of $\bar{\gamma}$ and $p$ in $W^{1,\infty}(0, T; \mathbb{R}^n)$ and $\bar{v}$ in $L^\infty(0, T; \mathbb{R}^m)$.

We recall that throughout this section the triplet $(m, P, x_0)$ is fixed and satisfies (2.4). Note that all constants $C$ used in this section are independent of $(m, P, x_0)$.

**PROPOSITION 3.4.** *The optimal control problem (2.1) has (at least) one solution. There exist two constants $M_1 > 0$ and $C > 0$, independent of $m, P$, and $x_0$, such that for all solutions $(\bar{\gamma}, \bar{v})$ to (2.1),*

$$ (3.6) \qquad \|\bar{\gamma}\|_{L^{\infty}(0,T;\mathbb{R}^n)} \le M_1 \quad \text{and} \quad \|\bar{v}\|_{L^2(0,T;\mathbb{R}^m)} \le C. $$

*Proof.* The constants $C > 0$ used in this proof only depend on the data of the problem. Let $(\gamma_0, v_0) \in \mathcal{K}[x_0]$ satisfy Assumption (H4)-(ii). Let $(\gamma_k, v_k)_{k \in \mathbb{N}}$ be a minimizing sequence. Without loss of generality, we can assume that

$$ J[m, P](\gamma_k, v_k) \le J[m, P](\gamma_0, v_0), \quad \forall k \in \mathbb{N}. $$
Using Assumption (H1)-(i), the boundedness of P, and Assumption (H3)-(v), we deduce that

$$ 
\begin{aligned}
C &\ge J[m, P](\gamma_0, v_0) \ge J[m, P](\gamma_k, v_k) \\
&\ge \frac{1}{C} \|v_k\|_{L^2(0,T)}^2 - C \|v_k\|_{L^1(0,T)} - C \ge \frac{1}{C} \|v_k\|_{L^2(0,T)}^2 - C,
\end{aligned}
$$

for some independent constants C. It follows that $v_k$ is bounded in $L^2(0, T; \mathbb{R}^m)$. By Grönwall's lemma and Assumption (H3)-(iii), there exists a constant $C > 0$ such that $\|\gamma_k\|_{L^\infty(0,T;\mathbb{R}^n)} \le C$. The state equation further implies that $\|\gamma_k\|_{H^1(0,T;\mathbb{R}^n)} \le C$. Extracting a subsequence if necessary, there exist $(\bar{\gamma}, \bar{v}) \in H^1(0, T; \mathbb{R}^n) \times L^2(0, T; \mathbb{R}^m)$ and a $C > 0$ such that

$$ \|\bar{\gamma}\|_{H^1(0,T;\mathbb{R}^n)} \le C \quad \text{and} \quad \|\bar{v}\|_{L^2(0,T;\mathbb{R}^m)} \le C $$

and such that $(\gamma_k, v_k) \to (\bar{\gamma}, \bar{v})$ for the weak topology of $H^1(0, T; \mathbb{R}^n) \times L^2(0, T; \mathbb{R}^m)$. Since $H^1(0, T; \mathbb{R}^n)$ is compactly embedded in $L^\infty(0, T; \mathbb{R}^n)$, we deduce that $\gamma_k$ converges uniformly to $\bar{\gamma}$.

Let us prove that $c(\bar{\gamma}(t), \bar{v}(t)) \le 0$ for a.e. $t \in (0, T)$. Let $\varphi \in L^\infty(0, T; \mathbb{R}^{n_c})$ be such that $\varphi(t) \ge 0$ for a.e. $t \in (0, T)$. We have

$$ \int_0^T \langle \varphi(t), c(\bar{\gamma}(t), \bar{v}(t)) \rangle dt = a_k + b_k + \int_0^T \langle \varphi(t), c(\gamma_k(t), v_k(t)) \rangle dt \le a_k + b_k, $$

where, skipping the time arguments

$$ a_k = \int_0^T \langle \varphi, c(\bar{\gamma}, \bar{v}) - c(\bar{\gamma}, v_k) \rangle dt \quad \text{and} \quad b_k = \int_0^T \langle \varphi, c(\bar{\gamma}, v_k) - c(\gamma_k, v_k) \rangle dt. $$

Note that all these integrals are well-defined as a consequence of Assumption (H3)-(ii). Also by Assumption (H3)-(ii), we easily verify that $D_v c(\bar{\gamma}(\cdot), \bar{v}(\cdot)) \in L^2(0, T; \mathbb{R}^{n_c \times m})$. Therefore, by the convexity of the mappings $c_i(x, \cdot)$ in Assumption (H1)-(ii),

$$ a_k \le \int_0^T \langle \varphi(t), D_v c(\bar{\gamma}(t), \bar{v}(t)) (\bar{v}(t) - v_k(t)) \rangle dt \underset{k \to \infty}{\longrightarrow} 0. $$

By Assumption (H3)-(ii), we also have

$$ |b_k| \le C \|\varphi\|_{L^\infty(0,T;\mathbb{R}^{n_c})} \|\gamma_k - \bar{\gamma}\|_{L^\infty(0,T;\mathbb{R}^n)} \underset{k \to \infty}{\longrightarrow} 0. $$

It follows that for all $\varphi \ge 0$, $\int_0^T \langle \varphi, c(\bar{\gamma}(t), \bar{v}(t)) \rangle dt \le 0$. Therefore, $c(\bar{\gamma}(t), \bar{v}(t)) \le 0$, for a.e. $t \in (0, T)$. With similar arguments, we prove that $(\bar{\gamma}, \bar{v})$ is feasible and that

$$ J[m, P](\bar{\gamma}, \bar{v}) \le \lim_{k \to \infty} J[m, P](\gamma_k, v_k), $$

which concludes the proof of optimality of $(\bar{\gamma}, \bar{v})$. Repeating the above arguments, we show that any solution to (2.1) satisfies the bound (3.6). □

We next state optimality conditions for the optimal control problem. The proof of the following proposition is deferred to the appendix in Section A. In the rest of the section, we write $c[t]$ instead of $c(\bar{\gamma}(t), \bar{v}(t))$ (for a specified pair $(\bar{\gamma}, \bar{v})$). We use the same convention for $a, b, g_0, g_1$, and $g_2$.
**PROPOSITION 3.5.** Let $(\bar{\gamma}, \bar{v})$ be a solution to (2.1). There exists a quintuplet

$$ (p, \lambda_0, \lambda_1, \lambda_2, \nu) \in W^{1,2}(0, T; \mathbb{R}^n) \times \mathbb{R} \times \mathbb{R}^{n_{g_1}} \times \mathbb{R}^{n_{g_2}} \times L^{\infty}(0, T; \mathbb{R}^{n_c}) $$

such that $(p, \lambda_0) \neq (0, 0)$ and such that the adjoint equation

(3.7)

$$ \left\{ \begin{array}{l} p(T)^{\top} = \lambda_0 D g_0[T] + \lambda_1^{\top} D g_1[T] + \lambda_2^{\top} D g_2[T] \\ - \dot{p}(t)^{\top} = \lambda_0 D_x L[t] + \lambda_0 D_x f[t] + p(t)^{\top} (D a[t] + \sum_{i=1}^{m} D b_i[t] v_i(t)) + \nu(t)^{\top} D_x c[t], \end{array} \right. $$

the stationary condition

$$ (3.8) \qquad \lambda_0 D_v L[t] + \lambda_0 P(t)^{\top} + p(t)^{\top} b[t] + \nu(t)^{\top} D_v c[t] = 0, $$

and the following sign and complementarity conditions

$$ (3.9) \qquad \left\{ \begin{array}{ll} \lambda_0 \ge 0, & \langle \lambda_2, g_2(\bar{\gamma}(T)) \rangle = 0, \\ \lambda_2 \ge 0, & \langle \lambda_2, g_2(\bar{\gamma}(T)) \rangle = 0, \\ \nu(t) \ge 0, & \langle c(\bar{\gamma}(t), \bar{v}(t)), \nu(t) \rangle = 0, \quad \text{for a.e. } t \in (0, T) \end{array} \right. $$

are satisfied. Moreover, if $\lambda_0 \neq 0$, then $p \in W^{1,\infty}(0, T; \mathbb{R}^n)$.

The goal of the last two results in this subsection is to obtain uniform bounds for the optimal solutions and their associated multipliers.

**PROPOSITION 3.6.** Let $(\bar{\gamma}, \bar{v})$ be a solution to (2.1). There exists a quintuplet $(p, \lambda_0, \lambda_1, \lambda_2, \nu)$ satisfying the optimality conditions of the above proposition and such that $\lambda_0 = 1$. Moreover, for such a quintuplet, we have

$$ \|p\|_{L^{\infty}(0,T;\mathbb{R}^n)} \le M_2 $$

for some constant $M_2$ independent of $(\bar{\gamma}, \bar{v})$ and $(p, \lambda_0, \lambda_1, \lambda_2, \nu)$.

*Proof.* The proof essentially relies on the qualification conditions (H5)-(i) and (H5)-(ii). All constants $C$ used in the proof are independent of $(p, \lambda_0, \lambda_1, \lambda_2, \nu)$ and $(\bar{\gamma}, \bar{v})$. Let $(y, w)$ satisfy the linearized equation (2.5) (for $(\gamma, v) = (\bar{\gamma}, \bar{v})$). By integration by parts we have

$$ \begin{align*} \langle p(T), y(T) \rangle &= \int_0^T \langle \dot{p}(t), y(t) \rangle + \langle p(t), \dot{y}(t) \rangle dt \\ &= -\lambda_0 \int_0^T (D_x L[t] + D_x f[t]) y(t) dt \\ &\quad - \int_0^T p(t)^\top (D a[t] + \sum_{i=1}^m D b_i[t] \bar{v}_i(t)) y(t) dt - \int_0^T \nu(t)^\top D_x c[t] y(t) dt \\ &\quad + \int_0^T p(t)^\top (D a[t] + \sum_{i=1}^m D b_i[t] \bar{v}_i(t)) y(t) dt + \int_0^T p(t)^\top b[t] w(t) dt. \end{align*} $$

The second and the fourth integral cancel out. Injecting the optimality condition (3.8) in the last integral, we obtain:

$$ (3.10) \qquad \begin{aligned} \langle p(T), y(T) \rangle = & - \lambda_0 \int_0^T (DL[t](y(t), w(t)) + \langle P(t), w(t) \rangle + D_x f[t]y(t)) dt \\ & - \int_0^T \nu(t)^\top Dc[t](y(t), w(t)) dt. \end{aligned} $$
The main feature of this formula is that the right-hand side is independent of $p$. Let $(y, w)$ satisfy Assumption (H5)-(ii). By (H3)-(i), (H3)-(vi) and (H3)-(v) we have

$$ \int_0^T (DL[t](y(t), w(t)) + \langle P(t), w(t) \rangle + D_x f[t]y(t)) dt \\ \leq C \|y\|_{\infty} \int_0^T (1 + |\bar{v}(t)|^2) dt + C \|w\|_{\infty} \int_0^T (1 + |\bar{v}(t)|) dt \leq C, $$

where the last inequality holds by Proposition 3.4 and (H5)-(ii). By (H5)-(ii) and the complementarity conditions (3.9) we obtain

$$ \int_0^T \nu(t)^\top Dc[t](y(t), w(t)) dt = \int_0^T \nu(t)^\top (c[t] + Dc[t](y(t), w(t))) dt \\ \leq -\frac{1}{C} \|\nu\|_{L^1(0,T; \mathbb{R}^{n_c})}. $$

Therefore,

$$ (3.11) \qquad \langle p(T), y(T) \rangle \geq -C\lambda_0 + \frac{1}{C} \|\nu\|_{L^1(0,T;\mathbb{R}^{n_c})}. $$

Moreover, we deduce from the terminal condition for $p$ that

$$ \begin{align} \langle p(T), y(T) \rangle &= \lambda_0 Dg_0[T]y(T) + \langle \lambda_1, Dg_1[T]y(T) \rangle + \langle \lambda_2, Dg_2[T]y(T) \rangle \nonumber \\ &= \lambda_0 Dg_0[T]y(T) + \langle \lambda_2, g_2[T] + Dg_2[T]y(T) \rangle \tag{3.12} \\ &\leq C\lambda_0 - \frac{1}{C}|\lambda_2|. \nonumber \end{align} $$

The last inequality holds by (H2)-(ii), Proposition 3.4 and (H5)-(ii). It follows from (3.11) and (3.12) that

$$ (3.13) \qquad |\lambda_2| \leq C\lambda_0 \quad \text{and} \quad \|\nu\|_{L^1(0,T;\mathbb{R}^{n_c})} \leq C\lambda_0. $$

Now, let us consider $(y, w)$ satisfying (H5)-(i) with $z_1 = \lambda_1/|\lambda_1|$. We have

$$ \|y\|_{L^{\infty}(0,T;\mathbb{R}^n)} \leq C \quad \text{and} \quad \|w\|_{L^{\infty}(0,T;\mathbb{R}^n)} \leq C. $$

Since $Dc[t]$ is bounded in $L^{\infty}(0,T; \mathbb{R}^{n_c} \times (n+m))$ (by Assumption (H3)-(ii)), we have

$$ (3.14) \qquad \|Dc[\cdot](y(\cdot), w(\cdot))\|_{L^{\infty}(0,T;\mathbb{R}^{n_c})} \leq C. $$

Formula (3.10), together with the bound on $\|\nu\|_{L^1(0,T;\mathbb{R}^{n_c})}$ and (3.14) yields

$$ (3.15) \qquad \langle p(T), y(T) \rangle \leq C\lambda_0. $$

It follows from the terminal condition and the estimate on $|\lambda_2|$ that

$$ \begin{align} \langle p(T), y(T) \rangle &= \lambda_0 Dg_0[T]y(T) + \langle \lambda_1, Dg_1[T]y(T) \rangle + \langle \lambda_2, Dg_2[T]y(T) \rangle \nonumber \\ &\geq -C\lambda_0 + |\lambda_1|. \tag{3.16} \end{align} $$

Combining (3.15) and (3.16), we deduce that

$$ (3.17) \qquad |\lambda_1| \leq C\lambda_0. $$

If $\lambda_0 = 0$, then $\lambda_1 = 0$, $\lambda_2 = 0$, and $\nu = 0$. Thus $p(T) = 0$ and $\dot{p}(t) = 0$ a.e. and therefore $p = 0$, in contradiction with $(p, \lambda_0) \neq (0, 0)$. We deduce that $\lambda_0 > 0$. The optimality conditions being invariant by multiplication of a positive constant, we deduce the existence of a quintuplet satisfying (3.7)-(3.8)-(3.9) and $\lambda_0 = 1$. Bounds of $|\lambda_1|$, $|\lambda_2|$, and $\|\nu\|_{L^1(0,T;\mathbb{R}^{n_c})}$ directly follow from (3.13) and (3.17). Then we obtain a bound of $|p(T)|$ and finally a bound of $\|p\|_{L^{\infty}(0,T;\mathbb{R}^n)}$ with Grönwall's lemma. $\square$
DEFINITION 3.7. Given a solution $(\bar{\gamma}, \bar{v})$ to (2.1), we call associated costate any $p$ for which there exists $(\lambda_0, \lambda_1, \lambda_2, \nu)$ such that the optimality conditions in Proposition 3.5 hold true and $\lambda_0 = 1$.

In order to obtain more regularity on $(\bar{\gamma}, \bar{v})$, we need to express the optimal control as an auxiliary function of the state and costate, which is deduced from Lemma 3.2.

LEMMA 3.8. Let $(\bar{\gamma}, \bar{v})$ and $(p, \lambda_0, \lambda_1, \lambda_2, \nu)$ be as in Proposition 3.6. There exists $C > 0$ independent of $(\bar{\gamma}, \bar{v})$ and $(p, \lambda_0, \lambda_1, \lambda_2, \nu)$ such that

$$ \|\bar{v}\|_{L^{\infty}(0,T;\mathbb{R}^n)} \leq C, \quad \|\nu\|_{L^{\infty}(0,T;\mathbb{R}^n)} \leq C. $$

In addition, there exist constants $M_3 > 0$ and $M_4 > 0$, such that

$$ (3.18) \qquad \|\dot{\bar{\gamma}}\|_{L^{\infty}(0,T;\mathbb{R}^n)} \leq M_3, \quad \|\dot{p}\|_{L^{\infty}(0,T;\mathbb{R}^n)} \leq M_4. $$

*Proof.* It follows from the optimality condition (3.8) and Lemma 3.2 that

$$ (3.19) \qquad \begin{aligned} \bar{v}(t) &= v[\bar{\gamma}(t), P(t) + b(\bar{\gamma}(t))^{\top} p(t)], \\ \nu(t) &= v[\bar{\gamma}(t), P(t) + b(\bar{\gamma}(t))^{\top} p(t)]. \end{aligned} $$

Lemma 3.2 further implies that $\|\bar{v}\|_{L^{\infty}(0,T;\mathbb{R}^m)} \leq C$ and $\|\nu\|_{L^{\infty}(0,T;\mathbb{R}^{n_c})} \leq C$. The estimates (3.18) follow. □

4. Existence of MFGC equilibria. In this section, we prove the main result of the paper. We first construct the auxiliary function announced in the introduction. Then, applying Kakutani's fixed point theorem, we prove the existence of an auxiliary MFGC equilibrium (defined in Subsection 4.2), which will imply the existence of a Lagrangian one.

4.1. Auxiliary function. We set $B = \bar{B}(M_1) \times \bar{B}(M_2) \subset \mathbb{R}^n \times \mathbb{R}^n$, where $M_1$ and $M_2$ are given by Lemma 3.4 and Proposition 3.6, respectively.

LEMMA 4.1. (i) Let $\mu \in \mathcal{P}(\mathbb{R}^n \times \mathbb{R}^n)$ be such that $\text{supp}(\mu) \subseteq B$. There exists a unique $P \in \mathbb{R}^m$ such that

$$ (4.1) \qquad P = \psi\left(\int_B v[x, P + b(x)^{\top}q]d\mu(x, q)\right), $$

where $v[\cdot, \cdot]$ is the mapping introduced in Lemma 3.2. In the sequel, the unique solution to (4.1) is denoted by $\mathbf{P}[\mu]$.

(ii) The mapping $\mu \in \mathcal{P}(B) \mapsto \mathbf{P}[\mu]$ is uniformly continuous.

The mapping $\mathbf{P}$ is the auxiliary function which will allow us later to define a new notion of equilibrium. Let us motivate its introduction. In the above lemma, the probability measure $\mu$ represents the distribution of the agents with respect to their state and costate at a given time $t$. At equilibrium, an agent with state $x$ and costate $q$ utilizes the control $v[x, P(t) + b(x)^{\top}q]$, by Pontryagin's principle. Therefore the price relation (2.3) leads to the fixed point equation $P(t) = \psi(\int_B v[x, P(t)+b(x)^{\top}q]d\mu(x, q))$ introduced above.

*Proof of Lemma 4.1.* Let us first prove the existence of a solution. Let $C > 0$ denote a bound of $|\psi|$ (Assumption (H3)-(vi)). Consider the map

$$ \chi: P \in \bar{B}_{\mathbb{R}^m}(C) \mapsto \psi\left(\int_B v[x, P + b(x)^{\top}q]d\mu(x, q)\right) \in \bar{B}_{\mathbb{R}^m}(C). $$
By Lemma 3.2, the mapping $v[\cdot, \cdot]$ is continuous. Therefore, by the Schauder fixpoint theorem, there exists $P \in \bar{B}_{\mathbb{R}^m}(C)$ such that $P = \chi(P)$, which proves the existence of a solution to (4.1).

Let us prove next the uniqueness and the uniform continuity. Let $\mu_1$ and $\mu_2$ be in $\mathcal{P}(B)$. Let $P_1$ and $P_2$ denote two solutions of (4.1), for $\mu = \mu_1$ and $\mu = \mu_2$, respectively. For $j = 1, 2$, consider the maps

$$v_j: (x, q) \in B \mapsto v[x, P_j + b(x)^\top q].$$

Note that by construction, $P_j = \psi(\int_B v_j d\mu_j)$. Let us first note that there exists a constant $C > 0$, independent of $\mu_1, \mu_2, v_1$, and $v_2$ such that

$$\|v_j\|_{L^\infty(B)} \le C \quad \text{and} \quad v_j \text{ is } C\text{-Lipschitz.}$$

This is a consequence of Assumption (H2)-(ii) and Lemma 3.2. This implies, together with the monotonicity of $\psi$ (Assumption (H1)-(iii)) that

$$0 \le \langle \psi(\int_B v_2 d\mu_2) - \psi(\int_B v_1 d\mu_2), \int_B (v_2 - v_1) d\mu_2 \rangle \\ = \langle P_2 - P_1, \underbrace{\int_B (v_2 - v_1) d\mu_2}_{=: (a)} \rangle + \langle \psi(\int_B v_1 d\mu_1) - \psi(\int_B v_1 d\mu_2), \underbrace{\int_B (v_2 - v_1) d\mu_2}_{=: (b)} \rangle.$$

Lemma 3.2 yields

$$(a) \le \int_B \langle v_2(x, q) - v_1(x, q), (P_2 + b(x)^\top q) - (P_1 + b(x)^\top q) \rangle d\mu_2(x, q) \\ \le -\frac{1}{C} \int_B |v_2(x, q) - v_1(x, q)|^2 d\mu_2(x, q).$$

Let $C$ denote a bound of $\|v_1\|_{L^\infty(B)}$. Since $\psi$ is continuous, it is uniformly continuous on $B$. Therefore, there exists a function $\omega: [0, \infty) \to [0, \infty)$ such that for all $x$ and $y$ in $\bar{B}_{\mathbb{R}^m}(C)$, $|\psi(y) - \psi(x)| \le \omega(|y - x|)$, such that $\omega(0) = 0$ and such that $\omega$ is right-continuous at 0. We have

$$(b) \le \omega(|\int_B v_1 d\mu_1 - \int_B v_1 d\mu_2|) \Big|_{\int_B (v_2 - v_1) d\mu_2}.$$

Using further the Lipschitz continuity of $v_1$ and Cauchy-Schwarz inequality, we deduce that

$$(b) \le \omega(Cd_1(\mu_1, \mu_2)) \left( \int_B |v_2 - v_1|^2 d\mu_2 \right)^{1/2}.$$

Since $0 \le (a) + (b)$, we deduce that

$$\int_B |v_2 - v_1|^2 d\mu_2 \le (C\omega(Cd_1(\mu_1, \mu_2)))^2 =: \tilde{\omega}(d_1(\mu_1, \mu_2)).$$

Finally, we have

$$|P_2 - P_1| \le \omega(|\int_B v_2 d\mu_2 - \int_B v_1 d\mu_1|) \\ \le \omega(|\int_B v_2 - v_1 d\mu_2| + |\int_B v_1 d\mu_2 - \int_B v_1 d\mu_1|) \\ \le \omega(\tilde{\omega}(d_1(\mu_1, \mu_2) + Cd_1(\mu_1, \mu_2))).$$

If $\mu_1 = \mu_2$, then $P_1 = P_2$. The uniqueness of the solution to (4.1) follows. The uniform continuity of $P$ also follows, which concludes the proof. □
**4.2. Auxiliary MFGC equilibria.** In order to analyze the existence of La-grangian MFGC equilibria, we propose here a new notion of equilibrium, that we call auxiliary equilibrium. We set

$$ \tilde{\Gamma} = H^1(0, T; \mathbb{R}^n) \times H^1(0, T; \mathbb{R}^n). $$

We equip $\tilde{\Gamma}$ with the supremum norm, defined by $\max(\|\gamma\|_{\infty}, \|p\|_{\infty})$ for a given pair $(\gamma, p) \in \tilde{\Gamma}$. We denote it (by extension) $\|(\gamma, p)\|_{\infty}$. For any $x_0 \in \mathbb{R}^n$, we define

$$ \tilde{\Gamma}[x_0] = \{(\gamma, p) \in \tilde{\Gamma} : \gamma(0) = x_0\}. $$

Given $t \in [0, T]$, we consider the mappings $\tilde{e}_t: \tilde{\Gamma} \to \mathbb{R}^n$ and $\hat{e}_t: \tilde{\Gamma} \to \mathbb{R}^n \times \mathbb{R}^n$ defined by $\tilde{e}_t(\gamma, p) = \gamma(t)$ and $\hat{e}_t(\gamma, p) = (\gamma(t), p(t))$, for all $(\gamma, p) \in \tilde{\Gamma}$. We denote

$$ \mathcal{P}_{m_0}(\tilde{\Gamma}) = \{\kappa \in \mathcal{P}_1(\tilde{\Gamma}) : \tilde{e}_0 \sharp \kappa = m_0\}. $$

We consider the following compact subset of $\tilde{\Gamma}$,

$$ (4.2) \quad \tilde{\Gamma}_B := \{(\gamma, p) \in \tilde{\Gamma} : \|\gamma\|_{\infty} \le M_1, \|p\|_{\infty} \le M_2, \|\dot{\gamma}\|_2 \le T^{\frac{1}{2}} M_3, \|\dot{p}\|_2 \le T^{\frac{1}{2}} M_4\}, $$

where $M_1, M_2, M_3$ and $M_4$ were introduced in Proposition 3.4, Proposition 3.6 and Lemma 3.8.

Given a distribution $\kappa \in \mathcal{P}_1(\tilde{\Gamma})$ with $\text{supp}(\kappa) \subset \tilde{\Gamma}_B$, we set for $t \in [0, T]$

$$ \tilde{m}_t^\kappa = \tilde{e}_t \sharp \kappa \in \mathcal{P}(\mathbb{R}^n), \quad \mu_t^\kappa = \hat{e}_t \sharp \kappa \in \mathcal{P}(\mathbb{R}^n \times \mathbb{R}^n). $$

**LEMMA 4.2.** Let $\kappa \in \mathcal{P}_{m_0}(\tilde{\Gamma})$ with $\text{supp}(\kappa) \subset \tilde{\Gamma}_B$. Then $\tilde{m}_t^\kappa$ and $\mu_t^\kappa$ are $\frac{1}{2}$-Hölder continuous w.r.t. $t \in [0, T]$.

*Proof.* Recalling that $B = \bar{B}(M_1) \times \bar{B}(M_2) \subset \mathbb{R}^n \times \mathbb{R}^n$, since $\text{supp}(\kappa) \subset \tilde{\Gamma}_B$, we obtain $\text{supp}(\mu_t^\kappa) \subset B$ and $\text{supp}(\tilde{m}_t^\kappa) \subset \bar{B}(M_1)$ for all $t \in [0, T]$.

For all $s, t \in [0, T]$ we have

$$ \begin{align*} d_1(\mu_t^\kappa, \mu_s^\kappa) &= \sup_{\varphi \in \text{Lip}_1(\mathbb{R}^n \times \mathbb{R}^n)} \int_B \varphi(x, q)(d\mu_t^\kappa - d\mu_s^\kappa)(x, q) \\ &= \sup_{\varphi \in \text{Lip}_1(\mathbb{R}^n \times \mathbb{R}^n)} \int_{\Gamma} [\varphi(\gamma(t), p(t)) - \varphi(\gamma(s), p(s))]d\kappa(\gamma, p) \\ &\le \int_{\Gamma} \max\{|\gamma(t) - \gamma(s)|, |p(t) - p(s)|\}d\kappa(\gamma, p) \\ &\le T^{\frac{1}{2}} \max\{M_3, M_4\}|t - s|^{\frac{1}{2}}. \end{align*} $$

The last inequality holds by the assumption $\text{supp}(\kappa) \subset \tilde{\Gamma}_B$. Similarly the result follows for $\tilde{m}_t^\kappa$.

Given $\kappa \in \mathcal{P}(\tilde{\Gamma}_B)$, by the above lemma, we obtain $\tilde{m}^\kappa \in C([0, T]; \mathcal{P}_1(\mathbb{R}^n))$. Setting $\tilde{P}^\kappa \in L^\infty(0, T; \mathbb{R}^m)$ given by

$$ (4.3) \qquad \tilde{P}^\kappa(t) = \mathbf{P}[\mu_t^\kappa], $$

where $\mathbf{P}$ is defined in Lemma 4.1, by Lemma 4.1 and Lemma 4.2, we obtain $\tilde{P}^\kappa \in C(0, T; \mathbb{R}^m)$. Defining the functional $\tilde{J}^\kappa := J[\tilde{m}^\kappa, \tilde{P}^\kappa]$, we can consider the set of
optimal trajectories and associated adjoint states $\tilde{\Gamma}^{\kappa}[x_0]$ given by

$$ \tilde{\Gamma}^{\kappa}[x_0] = \{ (\bar{\gamma}, p) \in \tilde{\Gamma}[x_0] : \bar{\gamma} \in \Gamma[\tilde{m}^{\kappa}, \tilde{P}^{\kappa}, x_0] \text{ and } p \text{ is an associated costate with } \bar{\gamma} \}, $$

where the meaning of “associated costate” is given in Definition 3.7 and $\Gamma[\tilde{m}^{\kappa}, \tilde{P}^{\kappa}, x_0]$ was defined in (2.2).

**DEFINITION 4.3.** A measure $\kappa \in \mathcal{P}_{m_0}(\tilde{\Gamma})$ is an auxiliary MFGC equilibrium if

$$ \text{supp}(\kappa) \subset \bigcup_{x \in \text{supp}(m_0)} \tilde{\Gamma}^{\kappa}[x]. $$

We now establish the relationship between the notion of Lagrangian and auxiliary MFGC equilibria. Given $\kappa \in \mathcal{P}(\tilde{\Gamma}_B)$, let $V^{\kappa}: \tilde{\Gamma} \to \mathcal{U}$ be defined by

$$ V^{\kappa}(\gamma, p) = v[\gamma, \tilde{P}^{\kappa} + b(\gamma)^{\top}p], $$

where the r.h.s. is the Nemytskii operator associated with the auxiliary mapping introduced in Lemma 4.1. Let $\pi_1: \tilde{\Gamma} \to \Gamma$ be such that $\pi_1(\gamma, p) = \gamma$. Then, we define $\eta[\kappa] = (\pi_1, V^{\kappa}) \nparallel \kappa \in \mathcal{P}_1(\Gamma \times \mathcal{U})$.

**LEMMA 4.4.** Let $\kappa \in \mathcal{P}(\tilde{\Gamma})$ be an auxiliary MFGC equilibrium. Then, $\eta[\kappa] \in \mathcal{P}_1(\Gamma \times \mathcal{U})$ is a Lagrangian MFGC equilibrium.

*Proof.* For the sake of simplicity we note $\eta$ instead of $\eta[\kappa]$. The main point is to prove that $\tilde{P}^{\kappa} = P^{\eta}$, where $P^{\eta}$ was introduced in (2.3). By the definition of $\eta$, it is supported on regular curves, thus

$$ \begin{aligned} P^{\eta}(t) &= \psi\left(\int_{\Gamma \times \mathcal{U}} v(t)d\eta(\gamma, v)\right) \\ &= \psi\left(\int_{\tilde{\Gamma}} v\left[\gamma(t), \tilde{P}^{\kappa}(t) + b(\gamma(t))^{\top}p(t)\right] d\kappa(\gamma, p)\right) \\ &= \psi\left(\int_{\mathbb{R}^n \times \mathbb{R}^n} v\left[x, \tilde{P}^{\kappa}(t) + b(x)^{\top}q\right] d\mu_t^{\kappa}(x, q)\right) = \tilde{P}^{\kappa}(t). \end{aligned} $$

The last equality follows from (4.1) and (4.3). It is clear that $\tilde{m}^{\kappa} = m^{\eta}$, then $J^{\eta} = \tilde{J}^{\kappa}$ ($J^{\eta}$ is defined in Section 2.2.1). Since $\kappa$ is an auxiliary MFGC equilibrium, any $(\bar{\gamma}, \bar{p}) \in \text{supp}(\kappa)$ defines an optimal pair $(\bar{\gamma}, V^{\kappa}(\bar{\gamma}, \bar{p}))$ for $J^{\eta}$. We conclude that $\eta$ is a Lagrangian MFGC equilibrium. $\square$

In Section 4.4 we show the existence of auxiliary MFGC equilibria, applying Kakutani’s fixed point theorem. The next technical section provides some convergence results to prove that the assumptions of Kakutani’s theorem hold.

### 4.3. Convergence properties.

**LEMMA 4.5.** Let $(\kappa^i)_{i \in \mathbb{N}}$ be a sequence contained in $\mathcal{P}_1(\tilde{\Gamma})$ such that $\text{supp}(\kappa^i) \subset \tilde{\Gamma}_B$ for all $i \in \mathbb{N}$. Assume that $(\kappa^i)_{i \in \mathbb{N}}$ narrowly converges to $\kappa$. Then,

$$ \sup_{t \in [0, T]} d_1(\tilde{m}_t^{\kappa^i}, \tilde{m}_t^{\kappa}) \to 0 \quad \text{and} \quad \sup_{t \in [0, T]} d_1(\mu_t^{\kappa^i}, \mu_t^{\kappa}) \to 0. $$

*Proof.* We start proving that for any $\bar{\kappa}, \hat{\kappa} \in \mathcal{P}_1(\tilde{\Gamma}_B)$ we have

$$ (4.4) \quad \sup_{t \in [0, T]} d_1(m_t^{\bar{\kappa}}, m_t^{\hat{\kappa}}) \le d_1(\bar{\kappa}, \hat{\kappa}), \quad \text{and} \quad \sup_{t \in [0, T]} d_1(\mu_t^{\bar{\kappa}}, \mu_t^{\hat{\kappa}}) \le d_1(\bar{\kappa}, \hat{\kappa}). $$
We show the result for $\mu_t^{\bar{\kappa}}$ and $\mu_t^{\hat{\kappa}}$, and then the result for $m_t^{\bar{\kappa}}$ and $m_t^{\hat{\kappa}}$ is straightforward. By the Kantorovich-Rubinstein formula, for any $t \in [0, T]$ we have

$$d_1(\mu_t^{\bar{\kappa}}, \mu_t^{\hat{\kappa}}) = \sup_{\varphi \in \text{Lip}_1(\mathbb{R}^n \times \mathbb{R}^n)} \int_{\mathbb{R}^n \times \mathbb{R}^n} \varphi(x, q) d(\mu_t^{\bar{\kappa}} - \mu_t^{\hat{\kappa}})(x, q) \\ = \sup_{\varphi \in \text{Lip}_1(\mathbb{R}^n \times \mathbb{R}^n)} \int_{\bar{\Gamma}} \varphi(\gamma(t), p(t)) d(\bar{\kappa} - \hat{\kappa})(\gamma, p) \le d_1(\bar{\kappa}, \hat{\kappa}).$$

In the last inequality we use the fact that given $\varphi \in \text{Lip}_1(\mathbb{R}^n \times \mathbb{R}^n)$, the mapping $(\gamma, p) \mapsto \varphi(\gamma(t), p(t))$ belongs to $\text{Lip}_1(\bar{\Gamma})$, for all $t \in [0, T]$.

Since $(\kappa^i)_{i \in \mathbb{N}} \subset \mathcal{P}_1(\bar{\Gamma}_B)$ narrowly converges to $\kappa$, by [3, Proposition 7.1.5], we obtain $d_1(\kappa^i, \kappa) \to 0$. The conclusion follows with (4.4). □

LEMMA 4.6. Let $(\kappa^i)_{i \in \mathbb{N}} \subset \mathcal{P}_{m_0}(\bar{\Gamma})$, $\kappa \in \mathcal{P}_{m_0}(\bar{\Gamma})$ be such that $\text{supp}(\kappa^i) \subset \bar{\Gamma}_B$ for all $i$ and $\text{supp}(\kappa) \subset \bar{\Gamma}_B$. Assume that $\kappa^i$ narrowly converges to $\bar{\kappa}$. Let $(x_i)_{i \in \mathbb{N}} \subset K_0$ be a sequence such that $x_i \to \bar{x}$ and let $(\gamma_i, p_i)_{i \in \mathbb{N}} \subset \bar{\Gamma}^{\kappa^i}[x_i]$ (defined in section 4.2) be a sequence such that $(\gamma_i, p_i) \to (\bar{\gamma}, \bar{p})$ uniformly on $[0, T]$. Then $(\bar{\gamma}, \bar{p}) \in \bar{\Gamma}^{\bar{\kappa}}[\bar{x}]$.

Proof. We have to prove that there exists $\bar{v} \in L^2(0, T; \mathbb{R}^m)$ such that $(\bar{\gamma}, \bar{v}) \in \mathcal{K}[\bar{x}]$ and

$$\tilde{J}^{\bar{\kappa}}(\bar{\gamma}, \bar{v}) \le \tilde{J}^{\bar{\kappa}}(\gamma, v) \quad \forall (\gamma, v) \in \mathcal{K}[\bar{x}].$$

In addition, we have to prove that $\bar{p}$ is the costate associated with $(\bar{\gamma}, \bar{v})$, in the sense of Definition 3.7.

Since $(\gamma_i, p_i)_{i \in \mathbb{N}} \subset \bar{\Gamma}^{\kappa^i}[x_i]$, there exists for all $i \in \mathbb{N}$ a control $v_i \in L^2(0, T; \mathbb{R}^m)$ such that $(\gamma_i, v_i) \in \mathcal{K}[x_i]$ and $(\gamma_i, v_i)$ is optimal for $\tilde{J}^{\kappa^i}$. By Proposition 3.4, since $(x_i)_{i \in \mathbb{N}} \subset K_0$, we have $\|\gamma_i(t)\|_{\infty} \le M_1$ and $\|v_i\|_2 \le C$, for all $i \in \mathbb{N}$. Therefore, there exists $\bar{v} \in L^2(0, T; \mathbb{R}^m)$ such that, up to a subsequence, $v_i \to \bar{v}$. By Lemma 3.8, the sequence $(\gamma_i)_{i \in \mathbb{N}}$ is a bounded sequence in $H^1(0, T; \mathbb{R}^n)$, since $\gamma_i \to \bar{\gamma}$ in $C(0, T; \mathbb{R}^n)$, it follows that $\bar{\gamma} \in H^1$ and $\dot{\gamma}_i \to \dot{\bar{\gamma}}$ in $L^2(0, T; \mathbb{R}^n)$. In addition, by (H2)-(ii), (H3)-(iii), the uniform convergence of $\gamma_i$ to $\bar{\gamma}$ and the weak convergence of $v_i$ to $\bar{v}$ we obtain

$$a(\gamma_i) + b(\gamma_i)v_i \to a(\bar{\gamma}) + b(\bar{\gamma})\bar{v}, \quad \text{in } L^2(0, T; \mathbb{R}^n),$$

which implies that $\dot{\bar{\gamma}}(t) = a(\bar{\gamma}(t)) + b(\bar{\gamma}(t))\bar{v}(t)$, for a.e. $t \in (0, T)$. It is clear that $\bar{\gamma}(0) = \bar{x}$.

Furthermore, for all $i \in \mathbb{N}$ there exists $(\lambda_1^i, \lambda_2^i, \nu_i) \in \mathbb{R}^{n_{g_1}} \times \mathbb{R}^{n_{g_2}} \times L^\infty(0, T; \mathbb{R}^{n_c})$ such that (3.7), (3.8) and (3.9) hold for $(\gamma_i, v_i, p_i)$ and $\lambda_0^i = 1$. By the proof of Proposition 3.6, we obtain that $(\lambda_1^i, \lambda_2^i)_{i \in \mathbb{N}}$ is bounded, then there exists a subsequence, still denoted $(\lambda_1^i, \lambda_2^i)_{i \in \mathbb{N}}$, that converges to $(\bar{\lambda}_1, \bar{\lambda}_2)$.

By Lemma 3.2 and (3.8), we deduce

$$v_i(t) = v[\gamma_i(t), \tilde{P}^{\kappa^i}(t) + b(\gamma_i(t))^{\top} p_i(t)], \quad \nu_i(t) = v[\gamma_i(t), \tilde{P}^{\kappa^i}(t) + b(\gamma_i(t))^{\top} p_i(t)].$$

By our assumptions, Lemma 4.1 and Lemma 4.5, the sequences $(\gamma_i)$, $(p_i)$ and $(\tilde{P}^{\kappa^i})$ are bounded and they converge to $\bar{\gamma}$, $\bar{p}$ and $\tilde{P}^{\bar{\kappa}}$, uniformly over $[0, T]$. By Lemma 3.2, the mappings $v[\cdot, \cdot]$ and $\nu[\cdot, \cdot]$ are Lipschitz continuous over bounded sets, then

$$(4.5) \qquad v_i(t) \to \bar{v}(t) = v[\bar{\gamma}(t), \tilde{P}^{\bar{\kappa}}(t) + b(\bar{\gamma}(t))^{\top} \bar{p}(t)],$$

and

$$\nu_i(t) \to \bar{v}(t), \text{ where } \bar{v}(t) = v[\bar{\gamma}(t), \tilde{P}^{\bar{\kappa}}(t) + b(\bar{\gamma}(t))^{\top} \bar{p}(t)],$$
uniformly over $[0, T]$. In addition by Lemma 4.5, $\sup_{t \in [0,T]} d_1(\tilde{m}_t^{\kappa^i}, \tilde{m}_t^{\bar{\kappa}}) \to 0$. Therefore by (H2) and (H3), we can pass to the limit in (3.7). By similar arguments we can pass to the limit in (3.8) and (3.9). Finally we can conclude that $(\bar{p}, 1, \bar{\lambda}_1, \bar{\lambda}_2, \bar{\nu})$ satisfies the adjoint equation, the stationary condition and the complementarity condition for $(\bar{\gamma}, \bar{v})$.

Now, we prove the optimality of $(\bar{\gamma}, \bar{v})$ for $\tilde{J}^{\bar{\kappa}}$. First we show that

$$ (4.6) \qquad \tilde{J}^{\bar{\kappa}}(\bar{\gamma}, \bar{v}) = \lim_{i \to \infty} \tilde{J}^{\kappa^i}(\gamma_i, v_i). $$

By the uniform convergence of the sequence $(\gamma_i)$, Lemma 4.5 and (H2) we have

$$ \int_0^T f(\gamma_i(t), \tilde{m}_t^{\kappa^i}) dt \to \int_0^T f(\bar{\gamma}(t), \tilde{m}_t^{\bar{\kappa}}) dt \quad \text{and} \quad g_0(\gamma_i(T), \tilde{m}_T^{\kappa^i}) \to g_0(\bar{\gamma}(T), \tilde{m}_T^{\bar{\kappa}}). $$

Skipping the time arguments, we have

$$ \int_0^T \left[ \langle \tilde{P}^{\kappa^i}, v_i \rangle - \langle \tilde{P}^{\bar{\kappa}}, \bar{v} \rangle \right] dt = \int_0^T \left[ \langle \tilde{P}^{\kappa^i} - \tilde{P}^{\bar{\kappa}}, v_i \rangle + \langle \tilde{P}^{\bar{\kappa}}, v_i - \bar{v} \rangle \right] dt. $$

By Lemma 4.1, Lemma 4.5, the uniform convergence in (4.5) and the boundedness of the sequences $(\tilde{P}^{\kappa^i})$ and $(v_i)$ we conclude that $\int_0^T \langle \tilde{P}^{\kappa^i}, v_i \rangle dt \to \int_0^T \langle \tilde{P}^{\bar{\kappa}}, \bar{v} \rangle dt$. By (H2)-(i) and the uniform convergence of $(\gamma_i)$ and $(v_i)$ to $\bar{\gamma}$ and $\bar{v}$, respectively, we deduce that $\int_0^T L(\gamma_i, v_i)dt \to \int_0^T L(\bar{\gamma}, \bar{v})dt$. Combining the above estimates, (4.6) follows.

Now, let $(\hat{\gamma}, \hat{v}) \in \mathcal{K}[\bar{x}]$ be an optimal solution for $\tilde{J}^{\bar{\kappa}}$ and initial condition $\bar{x}$. By (H5)(i)-(ii), Robinson's constraint qualification (see [11, (2.163)]) holds at $\hat{v}$. By [11, Theorem 2.87] and (H2) we conclude that there exists a sequence $(\hat{v}_i)_{i \in \mathbb{N}} \subset L^{\infty}(0, T; \mathbb{R}^m)$ such that $\|\hat{v}_i - \hat{v}\|_{\infty} \to 0$, and the sequence $(\hat{\gamma}_i)_{i \in \mathbb{N}}$, given by

$$ \left\{ \begin{array}{l} \dot{\hat{\gamma}}_i(t) = a(\hat{\gamma}_i(t)) + b(\hat{\gamma}_i(t))\hat{v}_i(t), \quad \text{for a.e. } t \in [0, T] \\ \hat{\gamma}_i(0) = x_i \end{array} \right. $$

is such that $(\hat{\gamma}_i, \hat{v}_i) \in \mathcal{K}[x_i]$. In addition, by our assumptions and Grönwall's Lemma we deduce that $(\hat{\gamma}_i)_{i \in \mathbb{N}}$ is uniformly bounded in $L^{\infty}(0, T; \mathbb{R}^n)$ and $\|\hat{\gamma}_i - \hat{\gamma}\|_{\infty} \to 0$.

By the optimality of $(\gamma_i, p_i) \in \tilde{\Gamma}^{\kappa^i}[x_i]$, we have

$$ (4.7) \qquad \tilde{J}^{\kappa^i}(\gamma_i, v_i) \le \tilde{J}^{\kappa^i}(\hat{\gamma}_i, \hat{v}_i) \quad \forall i \in \mathbb{N}. $$

Since $\|\hat{v}_i - \hat{v}\|_{\infty} \to 0$ and $\|\hat{\gamma}_i - \hat{\gamma}\|_{\infty} \to 0$, arguing as above we obtain,

$$ (4.8) \qquad \lim_{i \to \infty} \tilde{J}^{\kappa^i}(\hat{\gamma}_i, \hat{v}_i) = \tilde{J}^{\bar{\kappa}}(\hat{\gamma}, \hat{v}). $$

By (4.6), (4.7) and (4.8), we deduce

$$ \tilde{J}^{\bar{\kappa}}(\bar{\gamma}, \bar{v}) = \lim_{i \to \infty} \tilde{J}^{\kappa^i}(\gamma_i, v_i) \le \lim_{i \to \infty} \tilde{J}^{\kappa^i}(\hat{\gamma}_i, \hat{v}_i) = \tilde{J}^{\bar{\kappa}}(\hat{\gamma}, \hat{v}). $$

Then, $(\bar{\gamma}, \bar{v})$ is optimal, which finally proves that $(\bar{\gamma}, \bar{p}) \in \tilde{\Gamma}^{\bar{\kappa}}[\bar{x}]$. $\square$

**4.4. Existence results.** In this section, we characterize auxiliary MFGC equilibria as fixed points of a set-valued map. Applying Kakutani's fixed point theorem, we prove the existence of such equilibria.
By [3, Theorem 5.3.1] (Disintegration Theorem), for any $\kappa \in \mathcal{P}_{m_0}(\tilde{\Gamma})$, there exists a $m_0$-a.e. uniquely determined Borel measurable family $\{\kappa_x\}_{x \in \mathbb{R}^n} \subset \mathcal{P}(\tilde{\Gamma})$ such that

$$ \mathrm{supp}(\kappa_x) \subset \tilde{\Gamma}[x], \quad m_0\text{-a.e. } x \in \mathbb{R}^n, $$

and for any Borel mapping $\varphi: \tilde{\Gamma} \to [0, +\infty]$,

$$ \int_{\tilde{\Gamma}} \varphi(\gamma, p) \mathrm{d}\kappa(\gamma, p) = \int_{\mathbb{R}^n} \left( \int_{\tilde{\Gamma}[x]} \varphi(\gamma, p) \mathrm{d}\kappa_x(\gamma, p) \right) \mathrm{d}m_0(x). $$

Following the lines of [12], we define the set-valued map $E: \mathcal{P}_{m_0}(\tilde{\Gamma}) \rightrightarrows \mathcal{P}_{m_0}(\tilde{\Gamma})$ as

$$ E(\kappa) = \{\hat{\kappa} \in \mathcal{P}_{m_0}(\tilde{\Gamma}) : \mathrm{supp}(\hat{\kappa}_x) \subset \tilde{\Gamma}^\kappa[x], m_0\text{-a.e. } x \in \mathbb{R}^n\}. $$

It follows that $\kappa$ is an auxiliary MFGC equilibrium if and only if $\kappa \in E(\kappa)$.

**THEOREM 4.7.** *There exists at least one auxiliary MFGC equilibrium.*

*Proof.* Arguing as in [12, Lemma 3.5], for any $\kappa \in \mathcal{P}_{m_0}(\tilde{\Gamma})$ the set $E(\kappa)$ is a nonempty convex set. By Proposition 3.4, Proposition 3.6 and Lemma 3.8 we have

$$ E(\kappa) \subset \mathcal{P}_{m_0}(\tilde{\Gamma}_B), \quad \forall \kappa \in \mathcal{P}_{m_0}(\tilde{\Gamma}), $$

where $\tilde{\Gamma}_B$ was introduced in (4.2). By Lemma 4.6, and [12, Lemma 3.6], we conclude that the map $E: \mathcal{P}_{m_0}(\tilde{\Gamma}) \rightrightarrows \mathcal{P}_{m_0}(\tilde{\Gamma})$ has closed graph.

Finally, since the set $\tilde{\Gamma}_B$ is a compact subset of $\tilde{\Gamma}$, we obtain that $\mathcal{P}_{m_0}(\tilde{\Gamma}_B)$ is a nonempty compact convex set. Then, we can apply Kakutani's fixed point theorem, to deduce that there exists $\hat{\kappa} \in \mathcal{P}_{m_0}(\tilde{\Gamma}_B)$ such that $\hat{\kappa} \in E(\hat{\kappa})$. $\square$

**Remark 4.8.** Let us comment on the impossibility to employ a similar fixed point approach directly based on the notion of Lagrangian equilibria (Definition 2.1). Consider a probability distribution $\eta$ of state-control trajectories. From the definition of $P^\eta$, there is no regularity property (with respect to time) to expect, since the controls in problem (2.1) are taken in $L^2(0, T; \mathbb{R}^m)$. Consequently, it is not possible to use relation (3.19) to derive any regularity property for the optimal controls with respect to the criterion $J[m^\eta, P^\eta]$ and thus it does not seem possible to construct an appropriate compact set of probability distributions of state-control trajectories, on which some fixed point relation could be defined.

**5. Uniqueness.** As usual in the MFG theory, by adding some monotonicity assumptions we can obtain uniqueness results.

**DEFINITION 5.1.** A function $\varphi: \mathbb{R}^n \times \mathcal{P}(\mathbb{R}^n) \to \mathbb{R}$ is monotone if

$$ \int_{\mathbb{R}^n} (\varphi(x, m_1) - \varphi(x, m_2)) \mathrm{d}(m_1 - m_2)(x) \geq 0, \quad \forall m_1, m_2 \in \mathcal{P}(\mathbb{R}^n). $$

It is strictly monotone if it is monotone and

$$ \int_{\mathbb{R}^n} (\varphi(x, m_1) - \varphi(x, m_2)) \mathrm{d}(m_1 - m_2)(x) = 0, $$

if and only if $\varphi(x, m_1) = \varphi(x, m_2)$ for all $x \in \mathbb{R}^n$.

An example of strictly monotone function can be found in [12].
**THEOREM 5.2.** Assume that $f$ and $g_0$ are strictly monotone and $\psi$ is also strictly monotone (i.e. for all $x$ and $y \in \mathbb{R}^m$ with $x \neq y$, $\langle \psi(y) - \psi(x), y - x \rangle > 0$). Let $\eta_1, \eta_2 \in \mathcal{P}_{m_0}(\Gamma \times \mathcal{U})$ be Lagrangian MFGC equilibria for $m_0$, then $P^{\eta_1} = P^{\eta_2}$ and $J^{\eta_1} = J^{\eta_2}$.

*Proof.* Let us define $u^i(x) = \inf_{(\gamma,v) \in \mathcal{K}[x]} J^{\eta_i}(\gamma, v)$, $i = 1, 2$. Let $(\gamma, v) \in \text{supp}(\eta_1)$, then

$$u^1(\gamma(0)) = \int_0^T (L(\gamma(t), v(t)) + \langle P^{\eta_1}(t), v(t) \rangle + f(\gamma(t), m_t^{\eta_1})) dt + g_0(\gamma(T), m_T^{\eta_1}),$$
$$u^2(\gamma(0)) \le \int_0^T (L(\gamma(t), v(t)) + \langle P^{\eta_2}(t), v(t) \rangle + f(\gamma(t), m_t^{\eta_2})) dt + g_0(\gamma(T), m_T^{\eta_2}).$$

Integrating w.r.t. $\eta_1$ we obtain

$$\int_{\Gamma \times \mathcal{U}} (u^1(\gamma(0)) - u^2(\gamma(0))) d\eta_1(\gamma, v) + \int_{\Gamma \times \mathcal{U}} \int_0^T \langle P^{\eta_2}(t) - P^{\eta_1}(t), v(t) \rangle dt d\eta_1(\gamma, v) \\ \ge \int_{\Gamma \times \mathcal{U}} (g_0(\gamma(T), m_T^{\eta_1}) - g_0(\gamma(T), m_T^{\eta_2})) d\eta_1(\gamma, v) \\ + \int_{\Gamma \times \mathcal{U}} \int_0^T (f(\gamma(t), m_t^{\eta_1}) - f(\gamma(t), m_t^{\eta_2})) dt d\eta_1(\gamma, v).$$

By the definition of $m^{\eta_1}$ we obtain

$$\int_{\mathbb{R}^n} (u^1(x) - u^2(x)) dm_0(x) + \int_{\Gamma \times \mathcal{U}} \int_0^T \langle P^{\eta_2}(t) - P^{\eta_1}(t), v(t) \rangle dt d\eta_1(\gamma, v) \\ \ge \int_{\mathbb{R}^n} (g_0(x, m_T^{\eta_1}) - g_0(x, m_T^{\eta_2})) dm_T^{\eta_1}(x) \\ + \int_0^T \int_{\mathbb{R}^n} (f(x, m_t^{\eta_1}) - f(x, m_t^{\eta_2})) dm_t^{\eta_1}(x) dt.$$

Arguing in a similar way for $\eta_2$, we deduce

$$\int_0^T \int_{\Gamma \times \mathcal{U}} \langle P^{\eta_2}(t) - P^{\eta_1}(t), v(t) \rangle d(\eta_1 - \eta_2)(\gamma, v) dt \\ \ge \int_{\mathbb{R}^n} (g_0(x, m_T^{\eta_1}) - g_0(x, m_T^{\eta_2})) d(m_T^{\eta_1} - m_T^{\eta_2})(x) \\ (5.1) \qquad + \int_0^T \int_{\mathbb{R}^n} (f(x, m_t^{\eta_1}) - f(x, m_t^{\eta_2})) d(m_t^{\eta_1} - m_t^{\eta_2})(x) dt.$$

By the definition of $P^{\eta_i}$ we deduce

$$\int_0^T \int_{\Gamma \times \mathcal{U}} \langle P^{\eta_2}(t) - P^{\eta_1}(t), v(t) \rangle d(\eta_1 - \eta_2)(\gamma, v) dt \\ = \int_0^T \left\langle \psi \left( \int v d\eta_2(\gamma, v) \right) - \psi \left( \int v d\eta_1(\gamma, v) \right), \int v d\eta_1(\gamma, v) - \int v d\eta_2(\gamma, v) \right\rangle dt$$
and the r.h.s. is non-positive, by Assumption (H1)-(iii). In addition, since $f$ and $g_0$ are monotone, we deduce that the three terms in (5.1) vanish. Since $f$ and $g_0$ are strictly monotone we obtain for all $x \in \mathbb{R}^n$ and a.e. $t \in (0, T)$,

$$f(x, m_t^{\eta_1}) = f(x, m_t^{\eta_2}) \quad \text{and} \quad g_0(x, m_T^{\eta_1}) = g_0(x, m_T^{\eta_2}).$$

By the strict monotony of $\psi$ we have

$$\int v d\eta_1(\gamma, v) = \int v d\eta_2(\gamma, v), \quad \text{a.e. } t \in (0, T),$$

which in particular implies $P^{\eta_1} = P^{\eta_2}$. The result follows. □

*Remark 5.3.* As noted in [12], if we assume that $\psi$ is strictly monotone, $g_0$ is monotone and $f$ satisfies

$$\int_{\mathbb{R}^n} (f(x, m_1) - f(x, m_2)) d(m_1 - m_2)(x) \le 0 \Rightarrow m_1 = m_2,$$

then, following the ideas of the above proof, we obtain $P^{\eta_1} = P^{\eta_2}$ and $m^{\eta_1} = m^{\eta_2}$.

**6. Conclusion.** We have proved the existence of a Lagrangian equilibrium for an MFG of controls with final state and mixed state-control constraints, and a class of nonlinear dynamics. Using auxiliary mappings and a priori estimates on optimal state-costate trajectories, we have reformulated the problem as a fixed point problem on a compact set of probability measures on state-costate trajectories. As explained in Remark 5.3, this reformulation was necessary, in the absence of smoothing properties of the price interaction.

A future direction of research may concern the characterization of the equilibrium with a system of coupled partial differential equations (HJB equation and continuity equation), as it is done for example in [14]. In this reference, a feedback control is constructed thanks to the differentiability of the value function, itself obtained with the strict convexity of the Hamiltonian. This last property is however lost (in general) in the presence of mixed state-control constraints. Another difficulty would arise from the treatment of final-state constraints. The recent work [6] may contain useful tools in that direction; this article deals with optimal control problems with final-state constraints: it provides a characterization of the value function as well as sensitivity relation.

In some future work, one could also address the extension of our aggregative MFG model to the case of pure state constraints, as those considered in [12]. As we already pointed out, our analysis relies in a crucial way on some a priori estimates on the costate, whose evolution is not impacted by the price variable. Proving the regularity of the costate, in the presence of pure state constraints and a merely measurable price function, seems however to be a great challenge.

**Appendix A. Proof of optimality conditions.**

We provide in this section a proof of the optimality conditions stated in Proposition 3.5. An important difficulty is the fact that optimal controls are not a priori known to be bounded (we are not able to prove the boundedness of optimal controls without having the optimality conditions at hand). It is therefore not possible to formulate the optimal control problem as an abstract problem satisfying a qualification condition in $L^\infty$ and to derive easily optimality conditions, as it is done in [10] for example. It turns out that the optimal control problem can be naturally formulated
as an optimal control problem for which the dynamic constraint takes the form of a differential inclusion. This enables us to use the associated optimality conditions, referred to as *extended Euler-Lagrange* conditions in the literature. More precisely, our analysis is based on [33, Theorem 7.5.1], which covers the case of unbounded controls and requires few regularity assumptions.

We first introduce two definitions of cones, used for the expression of the optimality conditions for problems with differential inclusions. Given a closed subset $K$ of $\mathbb{R}^l$ and $x \in K$, we call proximal normal cone of $K$ at $x$ the set $N_K^P(x)$ defined by

$$N_K^P(x) = \{ p \in \mathbb{R}^l : \exists C > 0, \forall y \in K, \langle p, y - x \rangle \le C |y - x|^2 \}.$$

That is, $p \in N_K^P(x)$ if and only if, for some $C > 0$,

$$(A.1) \qquad x \in \operatorname{argmin} \{\langle -p, y \rangle + C|y - x|^2 : y \in K\}.$$

The limiting normal cone $N_K(x)$ is defined by

$$N_K(x) = \left\{ p \in \mathbb{R}^l \mid \exists (x_k, p_k)_{k \in \mathbb{N}} \text{ such that: } \begin{array}{l} (x_k, p_k) \to (x, p), \text{ as } k \to \infty \\ x_k \in K, p_k \in N_K^P(x_k) \quad \forall k \in \mathbb{N} \end{array} \right\}.$$

*Proof of Proposition 3.5. Step 1:* reformulation of the optimal control problem. Let us fix a solution $(\bar{\gamma}, \bar{v}) \in H^1(0, T; \mathbb{R}^n) \times L^2(0, T; \mathbb{R}^m)$ to (2.1). In order to alleviate the notation, we first define

$$\tilde{L}(t, x, v) = L(x, v) + \langle P(t), v \rangle + f(x, m(t)),$$

for all $(x, v) \in \mathbb{R}^{n+m}$ and for a.e. $t \in (0, T)$.

We work with an augmented state variable $y = (y^{(1)}, y^{(2)}, y^{(3)}) \in \mathbb{R}^{n+m+1}$. We consider a set-valued map $F: [0, T] \times \mathbb{R}^{n+m+1} \to \mathbb{R}^{n+m+1}$ defined as $F(t, y) = \{\xi(t, y^{(1)}, v, z) : (v, z) \in \mathbb{R}^m \times \mathbb{R}, c(y^{(1)}, v) \le 0, z \ge 0\}$, where for $(x, v, z) \in \mathbb{R}^{n+m+1}$

$$\xi^{(1)}(t, x, v, z) = a(x) + b(x)v, \quad \xi^{(2)}(t, x, v, z) = v, \quad \xi^{(3)}(t, x, v, z) = \tilde{L}(t, x, v) + z.$$

The component $\xi^{(1)}$ coincides with the dynamics of the original state variable. The second component has a technical purpose, it allows in particular to prove easily that $F(t, y)$ is closed (which would be delicate otherwise, since the controls are not necessarily bounded). The third component allows to put the problem in Mayer form. The initial condition associated with the new state variable is defined by $\bar{y}_0 = (x_0, 0, 0) \in \mathbb{R}^{n+m+1}$. Let $K \subseteq \mathbb{R}^{2(n+m+1)}$ be given by

$$K = \{(y_i, y_f) \in \mathbb{R}^{2(n+m+1)} : y_i = \bar{y}_0, g_1(y_f^{(1)}) = 0, g_2(y_f^{(1)}) \le 0\}.$$

We define $\Phi: \mathbb{R}^{2(n+m+1)} \to \mathbb{R}$ by $\Phi(y_i, y_f) = g_0(y_f^{(1)}) + y_f^{(3)}$. The optimal control problem (2.1) can finally be reformulated as follows:

(A.2)

$$\inf_{y \in H^1(0, T; \mathbb{R}^{n+m+1})} \Phi(y(0), y(T)), \quad \text{subject to:} \quad \left\{ \begin{array}{ll} \dot{y}(t) \in F(t, y(t)), & \text{for a.e. } t \in (0, T), \\ (y(0), y(T)) \in K. & \end{array} \right.$$

More precisely, the trajectory $\bar{y}$, defined by

$$\left\{ \begin{array}{l} \bar{y}^{(1)}(t) = \bar{\gamma}(t) \\ \bar{y}^{(2)}(t) = \int_{0}^{t} \bar{v}(s)ds \\ \bar{y}^{(3)}(t) = \int_{0}^{t} \tilde{L}(s, \bar{\gamma}(s), \bar{v}(s))ds \end{array} \right.$$
is a solution to (A.2). Denoting $\bar{\xi} = \dot{y}$, we note that $\bar{\xi}(t) = \xi(t, \bar{\gamma}(t), \bar{v}(t), 0)$.

**Step 2:** verification of the technical conditions of [33, Theorem 7.5.1]. It is easily verified that for a.e. $t \in (0, T)$, $F(t, y)$ is non-empty and convex, as a consequence of Assumptions (H1)-(i), (H1)-(ii), and (H4)-(i). It is also easily verified that $F$ is measurable and has a closed graph. It remains to show that there exist $\eta > 0$ and $k \in L^1(0, T)$ such that

$$ (A.3) \qquad F(t, \tilde{y}) \cap (\dot{\tilde{y}}(t) + \eta k(t) \bar{B}(1)) \subseteq F(t, y) + k(t)|\tilde{y} - y|\bar{B}(1), $$

for a.e. $t \in (0, T)$ and for all $y$ and $\tilde{y}$ such that $|y - \bar{y}(t)| \le \eta$ and $|\tilde{y} - \bar{y}(t)| \le \eta$. Let $t \in (0, T)$, let $y$ and $\tilde{y}$ be such that $|y - \bar{y}| \le \delta/2$ and $|\tilde{y} - \bar{y}| \le \delta/2$, where $\delta$ is given by Lemma 3.1, with $R = \|\bar{\gamma}\|_{L^\infty(0,T;\mathbb{R}^n)}$. Let $\bar{k}(t) = 1 + |\bar{v}(t)|^2$. Let $\tilde{\xi} \in F(t, \tilde{y}) \cap (\dot{\tilde{y}}(t) + \bar{k}(t)\bar{B}(1))$. Let $\bar{v} \in \mathbb{R}^m$ and $\tilde{z} \in \mathbb{R}$ be such that $\tilde{\xi} = \xi(t, \tilde{y}^{(1)}, \tilde{v}, \tilde{z})$, $c(\tilde{y}^{(1)}, \tilde{v}) \le 0$ and $\tilde{z} \ge 0$. Since $|\tilde{\xi} - \bar{\xi}(t)| \le \bar{k}(t)$, we deduce that

$$ |\tilde{\xi}^{(2)} - \bar{\xi}^{(2)}(t)| = |\bar{v} - \bar{v}(t)| \le \bar{k}(t). $$

Therefore

$$ |\bar{v}| \le |\bar{v} - \bar{v}(t)| + |\bar{v}(t)| \le \bar{k}(t) + \frac{1}{2} + \frac{1}{2}|\bar{v}(t)|^2 \le \frac{3}{2}\bar{k}(t). $$

We also have $|y^{(1)} - \tilde{y}^{(1)}| \le |y - \bar{y}(t)| + |\tilde{y} - \bar{y}(t)| \le \delta$. Thus by Lemma 3.1, there exists $v \in \mathbb{R}^m$ such that $c(y^{(1)}, v) \le 0$ and $|v - \tilde{v}| \le C|y^{(1)} - \tilde{y}^{(1)}|$ (note that all constants $C$ involved for the verification of (A.3) are independent of $(t, \tilde{y}, \tilde{v}, y, v)$). Let $\xi = \xi(t, y^{(1)}, v, \tilde{z})$. We have $\xi \in F(t, y)$. It remains to bound $|\xi - \bar{\xi}|$. We first have

$$ \begin{aligned} |\xi^{(1)} - \bar{\xi}^{(1)}| &\le |a(y^{(1)}) - a(\tilde{y}^{(1)})| + |b(y^{(1)})| \cdot |v - \tilde{v}| + |b(y^{(1)}) - b(\tilde{y}^{(1)})| \cdot |\bar{v}| \\ &\le C(|y - \bar{y}| + |v - \tilde{v}|)(1 + |\bar{v}(t)|^2) \\ &\le C|y - \tilde{y}|\bar{k}(t), \end{aligned} $$

by (H2)-(ii). The same estimate can be established for $|\xi^{(3)} - \bar{\xi}^{(3)}|$ (with the help of Assumption (H3)-(i)) and for $|\xi^{(2)} - \bar{\xi}^{(2)}|$, thus

$$ (A.4) \qquad |\xi - \bar{\xi}| \le C|y - \tilde{y}|\bar{k}(t). $$

The inclusion (A.3) follows, taking $k(t) = C\bar{k}(t)$ and $\eta = \min(\delta/2, 1/C)$, where $C$ is the constant appearing in the right-hand side of (A.4).

**Step 3:** abstract optimality conditions and interpretation. Applying [33, Theorem 7.5.1], we obtain the existence of $\bar{p} \in W^{1,1}(0, T; \mathbb{R}^{n+m+1})$ and $\lambda_0 \ge 0$ such that:

(i) $(\bar{p}, \lambda_0) \ne (0, 0)$,

(ii) $-\dot{p}(t) \in \text{conv}\{q : (q - \bar{p}(t)) \in N_{\text{Gr}(F(t, \cdot))}(\bar{y}(t), \bar{\xi}(t))\}$,

(iii) $(-\bar{p}(0), \bar{p}(T)) \in \lambda_0 \nabla \Phi(\bar{y}(0), \bar{y}(T)) + N_K(\bar{y}(0), \bar{y}(T))$,

where $\text{Gr}(F(t, \cdot)) = \{(y, \xi) : \xi \in F(t, y)\}$. We let the reader verify that the condition (iii) (together with Assumptions (H5)-(i) and (H5)-(ii)) implies the existence of $\lambda_1 \in \mathbb{R}^{n_{g_1}}$ and $\lambda_2 \in \mathbb{R}^{n_{g_2}}$, $\lambda_2 \ge 0$, such that

$$ \bar{p}^{(1)}(T)^\top = \lambda_0 D g_0(\bar{y}^{(1)}(T)) + \lambda_1^\top D g_1(\bar{y}^{(1)}(T)) + \lambda_2^\top D g_2(\bar{y}^{(1)}(T)), $$

$$ \bar{p}^{(2)}(T)^\top = 0, $$

$$ \bar{p}^{(3)}(T)^\top = \lambda_0, $$

and such that $\langle g_2(\bar{y}^{(1)}(T)), \lambda_2 \rangle = 0$. For the interpretation of the adjoint equation (condition (ii)), we need to examine the limiting normal cone of the graph of $F(t, \cdot)$.
Let $y \in \mathbb{R}^{n+m+1}$, let $\xi \in F(t, y)$, and let $(q, -p) \in N_{\mathrm{Gr}(F(t, \cdot))}(y, \xi)$. Let $y_k \to y$, $\xi_k = \xi(t, y_k^{(1)}, v_k, z_k) \to \xi$, $\xi_k \in F(t, y_k)$, $(q_k, p_k) \to (q, p)$ be such that $(q_k, -p_k) \in N_{\mathrm{Gr}(F(t, \cdot))}^P(y_k, \xi_k)$. By definition of the proximal normal cone, see (A.1), $(y_k, \xi_k)$ is for some $C > 0$ (depending on $k$) solution of the minimization problem

$$ \min_{(y, \xi) \in \mathrm{Gr}(F(t, \cdot))} \sum_{i=1}^{3} \left( \langle -q_k^{(i)}, y^{(i)} \rangle + \langle p_k^{(i)}, \xi^{(i)} \rangle + C(|y - y_k|^2 + |\xi - \xi_k|^2) \right). $$

In view of the expression of the mult mapping $F$, this holds if and only if, for some $(v_k, z_k) \in \mathbb{R}^m \times \mathbb{R}$, $(y_k, v_k, z_k) \in \mathbb{R}^{n+m+1} \times \mathbb{R}^m \times \mathbb{R}$ is solution of

$$ \begin{aligned} & \min_{(y,v,z)} \sum_{i=1}^{3} \langle -q_k^{(i)}, y^{(i)} \rangle + \langle p_k^{(1)}, a(y^{(1)}) + b(y^{(1)})v \rangle \\ & \qquad + \langle p_k^{(2)}, v \rangle + \langle p_k^{(3)}, \tilde{L}(t, y^{(1)}, v) + z \rangle + C(|y - y_k|^2 + |\xi - \xi_k|^2), \\ & \text{s.t. } c(y^{(1)}, v) \le 0 \text{ and } z \ge 0. \end{aligned} $$

Since this problem is qualified, we obtain the existence of $\nu_k \in \mathbb{R}^{n_c}$, $\nu_k \ge 0$, such that the following stationarity and complementarity conditions hold:

* Stationarity with respect to $z$: $p_k^{(3)} \ge 0$.

* Stationarity with respect to $v$:

$$ (A.5) \quad (p_k^{(1)})^{\top} b(y_k^{(1)}) + (p_k^{(2)})^{\top} + p_k^{(3)} D_v \tilde{L}(y_k^{(1)}, v_k) + \nu_k^{\top} D_v c(y_k^{(1)}, v_k) = 0. $$

* Stationarity with respect to $y^{(1)}$:

$$ -(q_k^{(1)})^{\top} + (p_k^{(1)})^{\top} \left( Da(y_k^{(1)}) + \sum_{i=1}^{m} Db(y_k^{(1)})v_{k,i} \right) + p_k^{(3)} D_x \tilde{L}(y_k^{(1)}, v_k) + \nu_k^{\top} D_x c(y_k^{(1)}, v_k) = 0. $$

* Stationarity with respect to $y^{(2)}$: $q_k^{(2)} = 0$.

* Stationarity with respect to $y^{(3)}$: $q_k^{(3)} = 0$.

* Complementarity: $\langle c(y_k^{(1)}, v_k), \nu_k \rangle = 0$.

The inward pointing condition, Assumption (H5)-(iv), yields a uniform bound on $\nu_k$ (with respect to $k$). This allows to pass to the limit in the above relations, using the continuity assumptions on $\tilde{L}$, $a$, $b$, and $c$ (note that $\xi_k^{(2)} \to \xi^{(2)}$ implies that $v_k \to v$). We deduce that $\dot{p}^{(2)} = 0$ and $\dot{p}^{(3)} = 0$, thus $\bar{p}^{(2)}(t) = 0$ and $\bar{p}^{(3)}(t) = \lambda_0$. Since $\bar{p} \in W^{1,1}(0, T; \mathbb{R}^{n+m+1})$, it belongs to $L^{\infty}(0, T; \mathbb{R}^{n+m+1})$. Passing to the limit in (A.5) we obtain

$$ (A.6) \quad \lambda_0 D_v \tilde{L}(\bar{y}^{(1)}(t), \bar{v}(t)) + (\bar{p}^{(1)}(t))^{\top} b(\bar{y}^{(1)}(t)) + \bar{v}(t)^{\top} D_v c(\bar{y}^{(1)}(t), \bar{v}(t)) = 0. $$

If $\lambda_0 \ne 0$, $\bar{v}(t) = \nu[\bar{y}^{(1)}(t), \lambda_0 P(t) + (\bar{p}^{(1)}(t))^{\top} b(\bar{y}^{(1)}(t))]$, by Lemma 3.2. Since $\nu[\cdot, \cdot]$ is Lipschitz continuous on bounded sets, we obtain $\bar{v} \in L^{\infty}(0, T; \mathbb{R}^{n_c})$. Analogously, $\bar{v} \in L^{\infty}(0, T; \mathbb{R}^m)$, therefore we deduce that $\bar{p} \in W^{1,\infty}(0, T; \mathbb{R}^{n+m+1})$.

If $\lambda_0 = 0$, we denote by $\bar{v}_I(t)$ the components of $\bar{v}(t)$ whose indices belong to the set $I(\bar{y}^{(1)}(t), \bar{v}(t))$. Skipping the time arguments, from (A.6) we deduce

$$ \bar{v}_I = - \left[ D_v c_I(\bar{y}^{(1)}, \bar{v}) D_v c_I(\bar{y}^{(1)}, \bar{v})^{\top} \right]^{-1} D_v c_I(\bar{y}^{(1)}, \bar{v}) b(\bar{y}^{(1)})^{\top} \bar{p}^{(1)}. $$

The matrix $[D_v c_I(\bar{y}^{(1)}, \bar{v}) D_v c_I(\bar{y}^{(1)}, \bar{v})^{\top}]$ is uniformly invertible by (H5)-(iii). Since $\bar{y}^{(1)}, \bar{p}^{(1)} \in L^{\infty}(0, T; \mathbb{R}^n)$, by (H2)-(i), (H3)-(ii) and (H3)-(iii) we deduce that $\bar{v} \in$
$L^\infty(0, T; \mathbb{R}^{n_c})$. In this case, we only have $\bar{v} \in L^2(0, T; \mathbb{R}^m)$, so we obtain $\bar{p} \in W^{1,2}(0, T; \mathbb{R}^{n+m+1})$. □

## Appendix B. Application to a gas storage problem.

**B.1. Setting.** Consider the case when the scalar state $\gamma(t)$ represents a scaled energy storage, with value in $[0, 1]$ and integrator dynamics

$$ \dot{\gamma}(t) = v(t). $$

Therefore $a(x) = 0$ and $b(x) = 1$. In addition we have limitations on the efficiency of pumping depending on the storage level, namely

$$ \varphi_1(\gamma(t)) \leq v(t) \leq \varphi_2(\gamma(t)), $$

with $\varphi_1$ and $\varphi_2$ decreasing and of class $C^1: [0, 1] \to \mathbb{R}$, with negative (resp. positive) values except for $\varphi_1(0) = \varphi_2(1) = 0$, and for some $\alpha_1 > 0$ and $\alpha_2 > 0$:

$$ (B.1) \qquad -\alpha_1 x \leq \varphi_1(x); \quad \varphi_2(x) \leq \alpha_2(1-x). $$

In particular we have the uniform bound

$$ (B.2) \qquad v_m := \varphi_1(1) \leq v(t) \leq \varphi_2(0) := v_M, $$

For example, we could take $\varphi_1(x) = -\alpha_1 x$ and $\varphi_2(x) = \alpha_2(1-x)$. Since these constraints imply that the state remains between 0 and 1 (assuming of course that $\gamma(0) \in [0, 1]$), we can discard the pure state constraint $\gamma(t) \in [0, 1]$.

In what follows we will assume that the support of $m_0$ is a compact subset of $(0, 1)$. It follows that for some $\varepsilon_X > 0$, any trajectory $(\gamma, v)$ satisfying the mixed state and control constraints is such that $\gamma(t) \in [\varepsilon_X, 1 - \varepsilon_X]$, for all $t \in [0, T]$. So point 3 of Remark 2.3 applies with $X = [\varepsilon_X, 1 - \varepsilon_X]$, taking $\delta \in (0, \varepsilon_X)$ in the definition of $X'$.

The two mixed constraints are expressed in the format of this paper as

$$ (B.3) \qquad c_1(x, v) = \varphi_1(x) - v; \quad c_2(x, v) = v - \varphi_2(x). $$

They cannot be active simultaneously, since $\varphi_1$ and $\varphi_2$ have opposite sign, and do not have zero value simultaneously. It follows that

$$ \delta := \min_{x \in [0,1]} [\varphi_2(x) - \varphi_1(x))] $$

is positive. Consequently any $(x, v)$ such that $c(x, v) \leq 0$ satisfies

$$ (B.4) \qquad c_1(x, v) + c_2(x, v) = \varphi_1(x) - \varphi_2(x) \leq -\delta. $$

A classical constraint is to have a minimal storage at the end of the period, say $\gamma(T) \geq \frac{1}{2}$, corresponding to

$$ (B.5) \qquad g_2(x) := \frac{1}{2} - x. $$

There is no equality constraint on the final state. Also, $f(x, m) = 0$ and $g_0(x, m) = -\pi x$, where $\pi$ can be interpreted as a final price. For $\psi$ we can take the identity, which is not bounded but, since is applied to a set of bounded controls $v$, we can redefine it as a bounded, continuous monotone operator. We can also take for instance $L(x, v) = v^2/2$.
**B.2. Checking hypotheses.** The non obvious hypotheses are (H5)-(ii) and (H5)-(iv). In view of Remark 2.3, it is enough to check this latter assumption for $x \in X'$. Assume that $c_1(x, v) \le c_2(x, v)$, i.e. the second constraint is “more active” than the first one. Taking $w = -\delta/4$, by (B.4) we have $c_1(x, v) \le -\delta/2$, so that

$$ (B.6) \qquad c_1(x, v) + D_v c_1(x, v)w \le -\delta/4. $$

Also,

$$ (B.7) \qquad c_2(x, v) + D_v c_2(x, v)w = c_2(x, v) - \delta/4 \le -\delta/4. $$

If $c_2(x, v) \le c_1(x, v)$, taking $w = \delta/4$ we obtain similar estimates. Hypothesis (H5)-(iv) follows.

We next discuss (H5)-(ii). Remember that we ignore the first condition since there is no equality constraint for the final state. Given $\varepsilon = (\varepsilon_1, \varepsilon_2)$ with $\varepsilon_1 \ge 0$ and $\varepsilon_2 \ge 0$, set

$$ (B.8) \qquad \kappa_1(t) := \varphi_1(\gamma(t)) - v(t) + \varepsilon_1; \quad \kappa_2(t) := -\varepsilon_1 + \varphi_2(\gamma(t)) - v(t). $$

In our setting, the third condition of (H5)-(ii), when $\varepsilon_1 := 1/C$, can be expressed as

$$ (B.9) \qquad \kappa_1(t) + D\varphi_1(\gamma(t))y(t) \le w(t) \le \kappa_2(t) + D\varphi_2(\gamma(t))y(t), $$

with $y(t) = \int_0^t w(s)ds$. Observe that

$$ (B.10) \qquad \kappa_1(t) - \varepsilon_1 \le 0 \le \kappa_2(t) + \varepsilon_1. $$

Let

$$ (B.11) \qquad F(t, y) := \min(\kappa_2(t) + D\varphi_2(\gamma(t))y, \varepsilon_2). $$

Since $F(t, y)$ is a Lipschitz function of $y$, by the Cauchy-Lipschitz theorem, the ODE

$$ (B.12) \qquad \dot{y}(t) = F(t, y(t)), \quad t \in (0, T); \quad y(0) = 0 $$

has a unique solution $y_\varepsilon(t)$, and we denote $w_\varepsilon(t) := \dot{y}_\varepsilon(t) = F(t, y_\varepsilon(t))$. By the definition of $F$, $(y_\varepsilon(t), w_\varepsilon(t))$ satisfies the second linearized mixed constraint.

**LEMMA B.1.** We have that:

(i) If $\varepsilon = 0$, then $y_0(t) = 0$ for all $t \in [0, T]$.

(ii) If $\varepsilon_1 = 0$ and $\varepsilon_2 > 0$, then $y_\varepsilon(t) \ge 0$ for all $t$, and is always equal to 0 iff the second mixed constraint is always active.

(iii) If $\varepsilon_1 = 0$ and $\varepsilon_2 > 0$, Let $t_0 := \inf\{t \in [0, T]; y(t) > 0\}$. If the second mixed constraint is not always active, then $t_0 < T$, and $y(t) > 0$, for all $t \in (t_0, T]$. In particular $y(T) > 0$.

(iv) If $\varepsilon_1 = 0$ and $\varepsilon_2 > 0$, then the first mixed linearized constraint is never active.

(v) Assume that the second mixed constraint is not always active. Given $\varepsilon_2 > 0$, small enough, if $\varepsilon_1 > 1$ is small enough, then $(y_\varepsilon, w_\varepsilon)$ satisfies (H5)-(ii).

*Proof.* (i) Let $z(t)$ denote the zero function on $[0, T]$. Then, in view of (B.10):

$$ (B.13) \qquad F(t, z(t))) = \min(\kappa_2(t), \varepsilon_2) = \min(\kappa_2(t), 0) = 0. $$

This establishes point (i).

(ii) Since $F$ is a nondecreasing function of $\varepsilon_2$, and $y(t)$ is one dimensional, we deduce
that $y_{\varepsilon}(t)$ is a nondecreasing function of $\varepsilon_2$, and is therefore nonnegative (since $y_0 = 0$).
We have $y_{\varepsilon} = z$ iff

$$
(B.14) \qquad 0 = F(t, 0) = \min(\kappa_2(t), \varepsilon_2), \quad \text{for all } t \in [0, T],
$$

that is, iff the upper bound is always active.

(iii) Observe that since $\kappa_2(t) \ge 0$, and $D\varphi_2(\gamma(t))y_\varepsilon(t) \le 0$:

$$
(B.15) \quad \dot{y}(t) = F(t, y(t)) \ge \min(D\varphi_2(\gamma(t))y_\varepsilon(t), \varepsilon_2) = D\varphi_2(\gamma(t))y_\varepsilon(t) \ge -Cy_\varepsilon(t).
$$

Let $y_{\varepsilon}(t_1) > 0$. It follows that for $t \in [t_1, T]$, $y_{\varepsilon}(t) \ge e^{-C(t-t_1)}y_{\varepsilon}(t_1)$. Point (iii)
follows.

(iv) Since $F$ is a Lipschitz function of $(y, \varepsilon)$, there exists $C > 0$ such that, for all $\varepsilon \in \mathbb{R}_+^2$:

$$
(B.16) \quad \max_{t \in [0, T]} (|y_{\varepsilon}(t)| + |D\varphi_1(\gamma(t))y_{\varepsilon}(t)| + |D\varphi_2(\gamma(t))y_{\varepsilon}(t)|) \le C_0|\varepsilon|.
$$

Since $y_{\varepsilon}(t) \ge 0$ and $D\varphi_1(\gamma(t)) \le 0$, we have

$$
(B.17) \qquad \kappa_1(t) + D\varphi_1(\gamma(t))y_\varepsilon(t) \le \kappa_1(t) \le \varepsilon_1.
$$

We distinguish two cases.

(a) If $w_{\varepsilon}(t) = \varepsilon_2$ the result holds, whenever $\varepsilon_1 < \varepsilon_2$.

(b) If $w_{\varepsilon}(t) = \kappa_2(t) + D\varphi_2(\gamma(t))y_{\varepsilon}(t)$, since $\kappa_2(t) - \kappa_1(t) \ge \delta - 2\varepsilon_1$, we get

$$
(B.18) \qquad w(t) \ge \kappa_1(t) + \delta - 2\varepsilon_1 - C_0|\varepsilon|.
$$

So, the result holds provided that

$$
(B.19) \qquad 2\varepsilon_1 + C_0|\varepsilon| \le \delta.
$$

(v) Fix $\varepsilon_2 > 0$ small enough. For $\varepsilon_1 > 0$ small enough, we have by a continuity argument that the first linearized mixed constraint holds (as well as the second one by the definition of $F(t, y)$), and that $y(T) > 0$. The conclusion follows. $\square$

REFERENCES

[1] Y. Achdou, P. Mannucci, C. Marchi, and N. Tchou. Deterministic mean field games with control on the acceleration and state constraints. arXiv preprint:eprint 2104.07292, 2021.

[2] C. Alasseur, I. Ben Tahar, and A. Matoussi. An extended mean field game for storage in smart grids. *Journal of Optimization Theory and Applications*, 184(2):644–670, 2020.

[3] L. Ambrosio, N. Gigli, and G. Savare. *Gradient Flows: In Metric Spaces and in the Space of Probability Measures*. Lectures in Mathematics. ETH Zürich. Birkhäuser Basel, 2008.

[4] J.-D. Benamou, G. Carlier, and F. Santambrogio. Variational mean field games. In *Active Particles*, Volume 1, pages 141–171. Springer, 2017.

[5] C. Bertucci, J.-M. Lasry, and P.-L. Lions. Some remarks on mean field games. *Communications in Partial Differential Equations*, 44(3):205–227, 2019.

[6] O. Bokanowski, A. Désilles, and H. Zidani. Relationship between maximum principle and dynamic programming in presence of intermediate and final state constraints. *ESAIM: Control, Optimisation and Calculus of Variations*, 27:91, 2021.

[7] J. Bonnans, S. Hadikhanloo, and L. Pfeiffer. Schauder estimates for a class of potential mean field games of controls. *Appl. Math. Optim.*, 83(3):1431–1464, 2021.

[8] J. Bonnans and A. Hermant. Second-order analysis for optimal control problems with pure state constraints and mixed control-state constraints. *Annales de l'Institut Henri Poincare (C) Non Linear Analysis*, 26(2):561–598, 2009.
[9] J. Bonnans and N. Osmolovskii. Second-order analysis of optimal control problems with control and initial-final state constraints. *Journal of Convex Analysis*, 17(3):885–913, 2010.

[10] J. F. Bonnans, X. Dupuis, and L. Pfeiffer. Second-order necessary conditions in Pontryagin form for optimal control problems. *SIAM Journal on Control and Optimization*, 52(6):3887–3916, 2014.

[11] J. F. Bonnans and A. Shapiro. *Perturbation analysis of optimization problems*. Springer Science & Business Media, 2000.

[12] P. Cannarsa and R. Capuani. Existence and uniqueness for mean field games with state constraints. In *PDE models for multi-agent phenomena*, pages 49–71. Springer, 2018.

[13] P. Cannarsa, R. Capuani, and P. Cardaliaguet. $C^{1,1}$ smoothness of constrained solutions in the calculus of variations with application to mean field games. *Mathematics in Engineering*, 1(1):174–203, 2019.

[14] P. Cannarsa, R. Capuani, and P. Cardaliaguet. Mean field games with state constraints: from mild to pointwise solutions of the pde system. *Calculus of Variations and Partial Differential Equations*, 60(3):1–33, 2021.

[15] P. Cannarsa and C. Mendico. Mild and weak solutions of mean field games problem for linear control systems. *arXiv preprint:eprint 1907.02654*, 2019.

[16] P. Cardaliaguet and C.-A. Lehalle. Mean field game of controls and an application to trade crowding. *Mathematics and Financial Economics*, 12(3):335–363, 2018.

[17] R. Carmona and F. Delarue. *Probabilistic Theory of Mean Field Games with Applications I-II*. Springer, 2018.

[18] A. De Paola, D. Angeli, and G. Strbac. Distributed control of micro-storage devices with mean field games. *IEEE Transactions on Smart Grid*, 7(2):1119–1127, 2015.

[19] R. M. Dudley. *Real Analysis and Probability*. Cambridge Studies in Advanced Mathematics. Cambridge University Press, 2 edition, 2002.

[20] D. A. Gomes, S. Patrizi, and V. Voskanyan. On the existence of classical solutions for stationary extended mean field games. *Nonlinear Anal.: Theory, Methods and Appl.*, 99:49–79, 2014.

[21] D. A. Gomes and V. K. Voskanyan. Extended deterministic mean-field games. *SIAM Journal on Control and Optimization*, 54(2):1030–1055, 2016.

[22] P. Graber, A. Mullenix, and L. Pfeiffer. Weak solutions for potential mean field games of controls. *NoDEA, Nonlinear Differ. Equ. Appl.*, 28(5):34, 2021. Id/No 50.

[23] M. Huang, R. P. Malhamé, and P. E. Caines. Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. *Commun. Inf. Syst.*, 6(3):221–252, 2006.

[24] Z. Kobeissi. Mean field games with monotonous interactions through the law of states and controls of the agents. *arXiv preprint: 2006.12949*, 2020.

[25] Z. Kobeissi. On classical solutions to the mean field game system of controls. *Communications in Partial Differential Equations*, pages 1–36, 2021.

[26] J.-M. Lasry and P.-L. Lions. Jeux à champ moyen. i – le cas stationnaire. *Comptes Rendus Mathematique*, 343(9):619 – 625, 2006.

[27] J.-M. Lasry and P.-L. Lions. Jeux à champ moyen. ii – horizon fini et contrôle optimal. *Comptes Rendus Mathematique*, 343(10):679 – 684, 2006.

[28] J.-M. Lasry and P.-L. Lions. Mean field games. *Japanese J. of Math.*, 2(1):229–260, 2007.

[29] K. Liu, N. Oudjane, and C. Wan. Approximate Nash equilibria in large nonconvex aggregative games. *arXiv preprint: 2011.12604*, 2020.

[30] G. Mazanti and F. Santambrogio. Minimal-time mean field games. *Mathematical Models and Methods in Applied Sciences*, 29(08):1413–1464, 2019.

[31] D. Paccagnan, M. Kamgarpour, and J. Lygeros. On aggregative and mean field games with applications to electricity markets. In *2016 European Control Conference (ECC)*, pages 196–201. IEEE, 2016.

[32] F. Santambrogio and W. Shim. A Cucker–Smale inspired deterministic mean field game with velocity interactions. *SIAM Journal on Control and Optimization*, 59(6):4155–4187, 2021.

[33] R. Vinter. *Optimal control*. Springer Science & Business Media, 2010.