# Extragridient methods for mean field games of controls and mean field type FBSDEs

Charles Meynard

<figure><img src="image_1.png" /></figure>

## To cite this version:

Charles Meynard. Extragridient methods for mean field games of controls and mean field type FBS-DEs. 2026. hal-05498903v3

HAL Id: hal-05498903

https://hal.science/hal-05498903v3

Preprint submitted on 13 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
EXTRAGRADIENT METHODS FOR MEAN FIELD GAMES OF CONTROLS
AND MEAN FIELD TYPE FBSDES

MEYNARD CHARLES

ABSTRACT. In this paper we present a numerical scheme to solve coupled mean field forward-backward stochastic differential equations driven by monotone vector fields. This is based on an adaptation of so called extragradient methods by characterizing solutions as zeros of monotone variational inequalities in a Hilbert space. We first introduce the procedure in the context of mean field games of controls and highlight its connection to the fictitious play. Under sufficiently strong monotonicity assumptions, we demonstrate that the sequence of approximate solutions converges exponentially fast. Then we extend the method and main results to general forward backward systems of stochastic differential equations that do not necessarily stem from optimal control.

# 1. INTRODUCTION

## 1.1. General introduction.
In this article, we present a numerical method to approximate the solution of the coupled mean field forward-backward stochastic differential equation (FBSDE)

$$
(1.1) \qquad \begin{cases} X_t = X_0 - \int_0^t F(X_s, U_s, \mathcal{L}(X_s, U_s))ds + \sqrt{2\sigma}dB_t, \\ U_t = g(X_T, \mathcal{L}(X_T)) + \int_t^T G(X_s, U_s, \mathcal{L}(X_s, U_s))ds - \int_t^T Z_sdB_s, \end{cases}
$$

for a given initial condition $X_0 \in L^2(\Omega, \mathbb{R}^d)$, monotone coefficients $(F, G, g)$, where $\mathcal{L}(X, U)$ indicates the joint law of the random variables $X, U$ and $(Z_t)_{t \in [0,T]}$ is uniquely defined in such a fashion that the process $(U_t)_{t \in [0,T]}$ is progressively adapted with respect to the filtration generated by the Brownian motion $(B_t)_{t \ge 0}$

In recent years, there has been considerable interest in mean field forward backward systems [19, 9].
Usually, such systems arise from the study of stochastic optimal control problems with an interacting
population, whether it be mean field control [10], or mean field games [36, 34] for which this probabilistic
formulation consists in an alternative to the study of the master equation [16]. In this case, the
associated problem can usually be formulated as follows

$$
(1.2) \qquad \begin{cases} X_t = X_0 - \int_0^t \nabla_p H(X_s, U_s, \mathcal{L}(X_s))ds + \sqrt{2\sigma}dB_t, \\ U_t = \nabla_x u(T, X_T, \mathcal{L}(X_T)) + \int_t^T \nabla_x H(X_s, U_s, \mathcal{L}(X_s))ds - \int_t^T Z_sdB_s, \end{cases}
$$

where $(x, p, \mu) \mapsto H(x, p, m)$ is the Hamiltonian associated to the control problem of players and $u(T, \cdot)$ the terminal value function for a given player. There are however many advantages to the added genrality of considering coefficients $(F, G, g)$ that do not fall into this categoryn even for mean field games. Indeed the class of extended mean field games [37] is contained in (1.1), in particular this includes mean field games of control [41]. Conceptually, this dependence of the coefficients on the law of the solution makes the study of these systems difficult beyond short horizons of time [19, 14]. The existence and uniqueness of solutions to coupled mean field FBSDEs over arbitrarily long intervals of time remains a challenging problem, and relies most often on monotonicity assumptions. In the particular case of mean field games, a theory of wellposedness has been developed mostly through arguments based on partial differential equations (PDEs) in the flat monotone regime [16]. On the other hand, another notion of monotonicity has been used to study forward backward systems directly [36, 9, 39] through the Hilbertian approach. In a series of paper [43, 27, 40], this approach was extended to mean field games under the notion of displacement monotonicity. The method we present in this article relies extensively on this last notion of monotonicity we refer to as $L^2$-monotonicity when coefficients are not gradients.
Part of this article is also dedicated to the numerical approximation of mean field FBSDEs with common noise and the development of approximation scheme for this problem. In mean field games these models arise from the addition of a noise impacting the dynamics of all players in the same fashion [21, 36]. Models in which such noise is purely additive have led to master equations of second order [16] and have been studied extensively in the literature on both MFGs [16, 13, 38, 17] and mean field type FBSDEs [20, 39]. On the other hand common noise can also come from an additional stochastic process, as in [11, 12]. This leads us to consider systems of the form

$$ (1.3) \quad \begin{cases} X_t = X_0 - \int_0^t F(X_s, U_s, \mathcal{L}(X_s, U_s|\mathcal{F}_s^0))ds + \sqrt{2\sigma}dB_t, \\ U_t = g(X_T, \mathcal{L}(X_T|\mathcal{F}_T^0)) + \int_t^T G(X_s, U_s, \mathcal{L}(X_s, U_s|\mathcal{F}_s^0))ds - \int_t^T Z_s d(B_s, W_s), \\ p_t = p_0 - \int_0^t b(p_s)ds + \sqrt{2\sigma^0}dW_s \end{cases} $$

where $(\mathcal{F}_t^0)_{t \ge 0}$ represents the filtration of the common noise and $(W_s)_{s \ge 0}$ is a $\mathcal{F}^0$-Brownian motion. In particular it was proven in [12] that this class of system includes the case of MFGs with an additive common noise.

In the specific case of mean field games, many numerical schemes have been proposed to solve the system (1.2). In the absence of common noise, several methods based on an equivalent PDE formulation have been proposed [3, 1] and [2] for MFGs of controls. Another approach is to rely on the fictitious play algorithm presented in [18]. On the other hand, probabilistic schemes have also been developed to solve FBSDEs of the form (1.1). Ranging from iterative methods based on Picard iterations [4, 24] to cost minimization with machine learning [22, 23]. Although the method we present in this article is also purely probabilistic, it relies instead on the monotonicity of coefficients and is inspired by the family of extra-gradient algorithms [32, 44]. Let us insist that there already exist an extensive literature on the application of these algorithms to monotone games, see [8, 29, 5, 30] and the references therein. In the case of MFGs our work can be seen as an extension to infinite dimensional monotone games.

## 1.2. Main contributions.
In this article, we present a probabilistic numerical method to solve mono-tone FBSDEs of the form (1.1). In the case of mean field games (1.1), it can be seen as a natural extension of previously developed algorithm for monotone games. For this method, we give theoretical convergence rates. In particular we show that under sufficiently strong monotonicity assumptions lin-ear convergence is achieved by the algorithm including for mean field games of controls. The numerical scheme we propose is not sensitive to the addition of common noise, in the sense that the convergence rate is unchanged (although the simulation cost for each iteration is larger from a practical viewpoint).

## 1.3. Organization of the paper.
In Section 2 we start with some reminders on MFGs of controls. Then we expose the numerical scheme proposed in this particular context. We give explicit convergence rates and in particular show that whenever the game is displacement monotone, strongly in the controls, linear convergence is achieved. Then, we extend our results to the general case of (1.1) by taking inspiration from the former case study. In section 3 we treat the case of mean field type FBSDEs with an independent common noise. Formally the presence of a common noise does not impact monotonicity, consequently the extension is straightforward. In a last short section, we present some numerical results illustrating the convergence bounds obtained in the previous sections.

## 1.4. Notation.

- let $k \in \mathbb{N}$, $k > 0$, for the canonical product on $\mathbb{R}^k$ we use the notation

$$ x \cdot y = \sum_i x_i y_i, $$

and the following notation for the induced norm

$$ |x| = \sqrt{x \cdot x}. $$

- Let $\mathcal{P}(\mathbb{R}^d)$ be the set of (Borel) probability measures on $\mathbb{R}^d$, for $q \ge 0$, we use the usual notation

$$ \mathcal{P}_q(\mathbb{R}^d) = \{ \mu \in \mathcal{P}(\mathbb{R}^d), \int_{\mathbb{R}^d} |x|^q \mu(dx) < +\infty \}, $$

for the set of all probability measures with a finite qth moment.
- For two measures $\mu, \nu \in \mathcal{P}(\mathbb{R}^d)$ we define $\Gamma(\mu, \nu)$ to be the set of all probability measures $\gamma \in \mathcal{P}(\mathbb{R}^{2d})$ satisfying

$$ \gamma(A \times \mathbb{R}^d) = \mu(A) \quad \gamma(\mathbb{R}^d \times A) = \nu(A), $$

for all Borel set $A$ on $\mathbb{R}^d$.

- The Wasserstein distance between two measures belonging to $\mathcal{P}_q(\mathbb{R}^d)$ is defined as

$$ W_q(\mu, \nu) = \left( \inf_{\gamma \in \Gamma(\mu, \nu)} \int_{\mathbb{R}^{2d}} |x - y|^q \gamma(dx, dy) \right)^{\frac{1}{q}}. $$

In what follows $\mathcal{P}_q(\mathbb{R}^d)$ is always endowed with the associated Wasserstein distance, $(\mathcal{P}_q, W_q)$ being a complete metric space.

- We say that a function $U : \mathcal{P}_q(\mathbb{R}^d) \to \mathbb{R}^d$ is Lipschitz if

$$ \exists C \ge 0, \quad \forall (\mu, \nu) \in (\mathcal{P}_q(\mathbb{R}^d))^2, \quad |U(\mu) - U(\nu)| \le C W_q(\mu, \nu). $$

- Consider $(\Omega, \mathcal{F}, \mathbb{P})$ a probability space,

- We define

$$ L^q(\Omega, \mathbb{R}^d) = \{ X : \Omega \to \mathbb{R}^d, \quad \mathbb{E}[|X|^q] < +\infty \}. $$

- Whenever a random variable $X : \Omega \to \mathbb{R}^d$ is distributed along $\mu \in \mathcal{P}(\mathbb{R}^d)$ we use equivalently the notations $\mathcal{L}(X) = \mu$ or $X \sim \mu$.

- $\mathcal{M}_{d \times n}(\mathbb{R})$ is the set of all matrices of size $d \times n$ with reals coefficients, with the notation $\mathcal{M}_n(\mathbb{R}) \equiv \mathcal{M}_{n \times n}(\mathbb{R})$.

- $C_b(\mathbb{R}^d, \mathbb{R}^k)$ is the set of all continuous bounded functions from $\mathbb{R}^d$ to $\mathbb{R}^k$.

**1.5. Preliminary results.** In this article we fix an horizon $T > 0$ and consider a complete probability space $(\Omega, \mathcal{F}, \mathbb{P})$ endowed with a $(d+d^0)$-dimensional Brownian motion $(B_t, W_t)_{0 \le t \le T}$. In particular we remind that the space $L^2(\Omega, \mathbb{R})$ endowed with the inner product

$$ \langle \cdot, \cdot \rangle : (X, Y) \mapsto \mathbb{E}[XY], $$

is a Hilbert space. We use the notation $H = (L^2(\Omega, \mathbb{R}), \langle \cdot, \cdot \rangle)$ and the induced norm is denoted by

$$ \forall X \in H, \quad \|X\| = \sqrt{\langle XX \rangle}. $$

We also use the notation $H^T$ for the Hilbert space of square integrable $\mathcal{F}$-adapted random processes on $[0, T]$

$$ \left\{ (X_s)_{s \in [0, T]}, \quad \mathbb{E} \left[ E \int_0^T |X_s|^2 ds \right] < +\infty \text{ and } (X_s)_{s \in [0, T]} \text{ is } \mathcal{F}\text{-adapted} \right\}, $$

endowed with the inner product

$$ \langle X, Y \rangle^T = \mathbb{E} \left[ \int_0^T X_t Y_t dt \right]. $$

**Definition 1.1.** For a function $F : \mathbb{R}^d \times \mathcal{P}_2(\mathbb{R}^d) \to \mathbb{R}^d$, we define its lift on $H$, $\tilde{F}$ to be

$$ \tilde{F} : \left\{ \begin{array}{ll} H \to H, \\ X \mapsto F(X, \mathcal{L}(X)). \end{array} \right. $$

We remind the following adaptation of Lemma 2.3 of [43]

**Proposition 1.2.** Let $f : \left\{ \begin{array}{l} \mathbb{R}^{2d} \to \mathbb{R} \\ (x, y) \mapsto f(x, y) \end{array} \right. $ be a continuous function, such that

$$ \forall (x, y) \in \mathbb{R}^{2d}, \quad f(x, y) = f(y, x), \quad f(x, x) = 0. $$

Suppose that for some $\mu \in \mathcal{P}_2(\mathbb{R}^d)$ with full support on $\mathbb{R}^d$ the following holds

$$ \forall (X, Y) \in \mathcal{H}^2, X \sim \mu, Y \sim \mu, \quad \mathbb{E}[f(X, Y)] \ge 0, $$

then the inequality is satisfied pointwise

$$ \forall (x, y) \in \mathbb{R}^d, \quad f(x, y) \ge 0. $$

In particular this direct corollary
**Corollary 1.3.** Let $F: \mathbb{R}^d \times \mathcal{P}_2(\mathbb{R}^d) \to \mathbb{R}^d$ be a continuous function. Suppose that there exists a constant $C_F$ such that

$$ \forall (X, Y) \in \mathcal{H}^2, \quad \|\tilde{F}(X) - \tilde{F}(Y)\| \le C_F \|X - Y\|. $$

Then for the same constant $C_F$

$$ \forall (\mu, x, y) \in \mathcal{P}_2(\mathbb{R}^d) \times (\mathbb{R}^d)^2, \quad |F(x, \mu) - F(y, \mu)| \le C_F |x - y|. $$

*Proof.* It suffices to apply the above proposition to

$$ f: (x, y) \mapsto C_F^2 |x - y|^2 - |F(x, \mu) - F(y, \mu)|^2, $$

for fixed $\mu$ with full support in $\mathbb{R}^d$. For measure that do not have full support, the result then follows by density and the continuity of $F$ in $\mathcal{P}_2(\mathbb{R}^d)$ as in [43]. □

**1.5.1.** *A primer on monotone variational inequalities in Hilbert spaces.*

Let $(\mathcal{H}, \langle \cdot, \cdot \rangle_{\mathcal{H}})$ indicate a Hilbert space. Consider $v: \mathcal{H} \to \mathcal{H}$ a continuous function and suppose we are interested in the following problem

$$ (1.4) \qquad \text{find } x^* \in \mathcal{H}, \forall x \in \mathcal{H}, \langle v(x^*), x - x^* \rangle_{\mathcal{H}} \ge 0. $$

Since $\mathcal{H}$ is reflexive, this is clearly equivalent to finding a $x^*$ such that

$$ v(x^*) = 0_{\mathcal{H}}. $$

If we further assume that $v$ is monotone non-degenerate, i.e.

$$ \forall x, y \in \mathcal{H}, x \neq y, \langle v(x) - v(y), x - y \rangle_{\mathcal{H}} > 0, $$

then there can exists at most one $x^*$ solving (1.4). In particular if we assume that $v$ is strongly monotone, that is

$$ \exists \beta > 0, \quad \forall x, y \in \mathcal{H}, \langle v(x) - v(y), x - y \rangle_{\mathcal{H}} \ge \beta \|x - y\|^2, $$

then $v$ is an invertible operator and it follows that there exists a $x^*$ solution to (1.4). A very natural question is how to compute $x^*$ in this setting. To that end let us fix some $x_0 \in \mathcal{H}, \gamma > 0$ and introduce the following sequence

$$ \forall n \ge 1, \quad x_{n+1} = x_n - \gamma v(x_{n+1}). $$

Letting

$$ v_{\gamma}: x \mapsto x + \gamma v(x), $$

this is equivalent to

$$ \forall n \ge 1, \quad x_{n+1} = v_{\gamma}^{-1}(x_n). $$

However since $v$ is $\beta$-strongly monotone, it follows that

$$ \forall x, y \in \mathcal{H}, \quad \langle v_{\gamma}(x) - v_{\gamma}(y), x - y \rangle \ge (1 + \gamma\beta) \|x - y\|^2. $$

This implies that

$$ \|v_{\gamma}^{-1}\|_{Lip} \le \frac{1}{1 + \gamma\beta}. $$

Since $v_{\gamma}^{-1}$ is a contraction on $\mathcal{H}$, we deduce from the definition of $(x_n)_{n \in \mathbb{N}}$ that it converges exponentially to the fixed point of $v_{\gamma}^{-1}$ which also happens to be $x^*$. By this method we have constructed a sequence $(x_n)_{n \in \mathbb{N}}$ which converges to the unique solution $x^*$ of (1.4). The main issue with the above construction is that the definition of $x_{n+1}$ from $x_n$ is implicit. In fact since it may require a fixed point procedure, computing $x_{n+1}$ at each iteration can be just as hard as computing $x^*$ directly. This is where extra-gradient method come into play, the key idea is to approximate the sequence with

$$ x_{n+1} \approx x_n - \gamma v(x_n - \gamma v(x_n)), $$

at each step. The definition of $x_{n+1}$ is now explicit in function of $x_n$, and it turns out that this approximation yields good results both in theory and practise. In particular, we remind this quite general inequality from [6]
**Lemma 1.4.** Let $(\mathcal{H}, \langle \cdot, \cdot \rangle_{\mathcal{H}})$ be a Hilbert space, and the sequence $(X_n, X_{n+\frac{1}{2}}, Y_n)_{n \in \mathbb{N}} \subset \mathcal{H}^3$ be defined through

$$ 
\left\{ 
\begin{array}{l}
X_{n+\frac{1}{2}} = X_n - \gamma_n V_n, \\
Y_{n+1} = Y_n - V_{n+\frac{1}{2}}, \\
X_{n+1} = \gamma_{n+1} Y_{n+1},
\end{array}
\right. 
$$

for $(\gamma_n, V_n, V_{n+\frac{1}{2}})_{n \in \mathbb{N}} \subset \mathcal{H}^2$ and $(\gamma_n)_{n \in \mathbb{N}} \subset \mathbb{R}^+$ a non increasing sequence of step size. Then the following holds for any $x \in \mathcal{H}$

$$ \sum_{i=1}^{n} \langle V_{i+\frac{1}{2}}, X_{i+\frac{1}{2}} - x \rangle_{\mathcal{H}} \le \frac{\|x - X_1\|_{\mathcal{H}}^2}{2\gamma_1} + \left( \frac{1}{2\gamma_{n+1}} - \frac{1}{2\gamma_1} \right) \|x\|_{\mathcal{H}}^2 + \frac{1}{2} \sum_{i=1}^{n} \left( \gamma_i \|V_{n+\frac{1}{2}} - V_n\|_{\mathcal{H}}^2 - \frac{1}{\gamma_i} \|X_{n+\frac{1}{2}} - X_n\|_{\mathcal{H}}^2 \right) $$

*Proof.* Although the original proof is conducted for a sequence in $\mathbb{R}^d$, since it only uses classical results on vector spaces endowed with an inner product, there is no difficulty in extending the proof to a general Hilbert space. $\square$

**Remark 1.5.** Whenever $\gamma_n \equiv \gamma$ is a constant sequence and $V_n = v(X_n)$ for some function $v$, this is equivalent to

$$ \forall n \ge 1, \quad X_{n+1} = X_n - \gamma v(X_n - \gamma v(X_n)). $$

The above inequality allows to work in a very general framework compared to our very limited introduction on extragrandient methods.

## 2. DECOUPLING ALGORITHM

In this first part, we present the main idea behind the algorithm we introduce to solve FBSDEs numerically. We present it first in the context of mean field games of control, see [42] for a presentation of mean field games of controls in the displacement monotone setting. In this context, the proposed method is very natural. Let us emphasize once again that there is already an extensive litterarure on extragrandient methods for monotone games [7, 6], and the extension to mean field games is quite straightforward. In this section we also consider dynamics without common noise.

**2.1. A motivating example from mean field games.** We start with some reminders on mean field games of controls in the displacement monotone framework, from the formulation of the equilibrium condition to the associated FBSDE. In particular, we adopt a strategy which is slightly different from usual. Instead of introducing an implicit fixed point mapping to write the forward backward system, we show that the equilibrium can be expressed explicitely in function of the coefficients through the use of an infinite dimensional inverse. Although this formulation is strictly equivalent to more standard ones for mean field games of controls, we believe this makes the exposition clearer and more straightforward (and in particular totally explicit).

**2.1.1. Uniqueness and characterization of optimal controls.** Let us first recall exactly the formulation of the mean field game problem in this setting. For a given flow of measures $(m_t)_{t \in [0,T]} \in \mathcal{P}_2([0,T], \mathbb{R}^{2d})$ we consider the cost

$$ J(\alpha, (m_t)_{t \in [0,T]}) = \left\{ 
\begin{array}{l}
\mathbb{E} \left[ g(X_T^\alpha, \mu_T) + \int_0^T L(X_t^\alpha, \alpha_t, m_t) dt \right], \\
X_t^\alpha = X_0 - \int_0^t \alpha_t dt + \sqrt{2\sigma} B_t, \\
\mu_T = \pi_d m_T.
\end{array}
\right. $$

where $\pi_d m$ indicates the marginal of $m$ over the first $d$ variables for any $m \in \mathcal{P}_2(\mathbb{R}^{2d})$. We suppose that $J(\alpha, (m_t)_{t \in [0,T]})$ represents the cost paid by a player with control $\alpha$ whenever the distribution of all other players and their controls is $(m_t)_{t \in [0,T]}$. The mean field game problem consists in finding an equilibrium, that is, a distribution $(m_t^*)_{t \in [0,T]}$ such that after solving

$$ \inf_{\alpha} J(\alpha, (m_t^*)_{t \in [0,T]}), $$

the players are still be distributed along $(m_t^*)_{t \in [0,T]}$. There exist several mathematically equivalent rigorous definitions of this problem. In this article we focus on formulating it directly at the level of
controls, which is natural for displacement monotone MFGs. Namely, solving the mean field game problem is equivalent to finding a control $(\alpha_t^*)_t \in [0,T] \subset H_{\mathcal{F}^\infty}^T$ such that

$$ (2.1) \qquad \alpha^* = \underset{\alpha}{\operatorname{arginf}} J(\alpha, (\mathcal{L}(X_t^{\alpha^*}, \alpha_t^*))_{t \in [0,T]}). $$

Clearly, this definition results from a fixed point of an optimisation problem characteristic of mean field games. At first glance, the existence of such a control, let alone the uniqueness is not clear. Let us start with a standard result on finite dimensional optimal control

**Lemma 2.1.** Suppose that $(x, \alpha) \mapsto (g(x, \mu), L(x, \alpha, m)) \in C^{1,1}(\mathbb{R}^{2d}, \mathbb{R}^2)$ uniformly in $m \in \mathcal{P}_2(\mathbb{R}^{2d})$ - If $(\alpha_t)_{t \in [0,T]} \subset H^T$ satisfies

$$ (2.2) \qquad \alpha \in \underset{\alpha' \in H^T}{\operatorname{arginf}} J(\alpha', (m_t)_{t \in [0,T]}), $$

the following holds

$$ (2.3) \quad \forall \alpha' \in H^T, \quad \langle \nabla_x g(X_T^\alpha, \mu_T), X_T^{\alpha'} - X_T^\alpha \rangle + \int_0^T \left\langle \begin{pmatrix} \nabla_x L \\ \nabla_\alpha L \end{pmatrix} (X_t^\alpha, \alpha_t, m_t), \begin{pmatrix} X_t^{\alpha'} - X_t^\alpha \\ \alpha_t' - \alpha_t \end{pmatrix} \right\rangle dt \ge 0, $$

where

$$ \mu_T(dx) = \int_{y \in \mathbb{R}^d} m_T(dx, dy), $$

indicates the marginal of $m_T$ over its first $d$ variables.

- If, furthermore, $x \mapsto g(x, \mu)$ is convex uniformly in $\mu \in \mathcal{P}_2(\mathbb{R}^d)$, and $(x, \alpha) \mapsto L(x, \alpha, m)$ is convex, strongly in $\alpha$, uniformly in $m \in \mathcal{P}_2(\mathbb{R}^{2d})$ then for any flow of measures $(m_t)_{t \in [0,T]} \subset \mathcal{P}_2(\mathbb{R}^{2d})$, there exists a unique solution to the optimal control problem

$$ \inf_{\alpha} J(\alpha, (m_t)_{t \in [0,T]}), $$

and it is the unique control satisfying (2.3).

*Proof.* Let us assume that there exists a control $\alpha \in H^T$ satisfying (2.2), for a given control $\alpha' \in H^T$, (2.3) is obtained by observing that for any $\lambda \in (0, 1]$

$$ \frac{1}{\lambda} \left( J(\alpha + \lambda(\alpha' - \alpha), (m_t)_{t \in [0,T]}) - J(\alpha, (m_t)_{t \in [0,T]}) \right) \ge 0. $$

The result follows naturally by taking the limit as $\lambda \to 0$. As for the second proposition, under the additional convexity assumptions

$$ \alpha \mapsto J(\alpha, (m_t)_{t \in [0,T]}), $$

is strongly convex on $H^T$ and the existence and uniqueness of a minimizer follow from standard optimization theory. Finally let us assume that there exist two controls $\alpha^1, \alpha^2$ satisfying (2.3), then the convexity of coefficients implies that

$$ \|\alpha^1 - \alpha^2\|_T \le 0, $$

$$ \text{yielding } \alpha^1 = \alpha^2 \text{ in } H_{\mathcal{F}^\infty}^T. \quad \square $$

We now make an assumption on the displacement monotonicity of $L$ and $g$.

**Hypothesis 2.2.**

$$ \exists c_L > 0, \quad \forall (X, Y, \alpha, \alpha') \in H^4, \\ \langle \nabla_x g(X, \mathcal{L}(X)) - \nabla_x g(Y, \mathcal{L}(Y)), X - Y \rangle \ge 0, $$

$$ \left\langle \begin{pmatrix} \nabla_x L \\ \nabla_\alpha L \end{pmatrix} (X, \alpha, \mathcal{L}(X, \alpha)) - \begin{pmatrix} \nabla_x L \\ \nabla_\alpha L \end{pmatrix} (Y, \alpha', \mathcal{L}(Y, \alpha')), \begin{pmatrix} X - Y \\ \alpha - \alpha' \end{pmatrix} \right\rangle \ge c_L \|\alpha - \alpha'\|^2 $$

**Remark 2.3.** Hypothesis 2.2 implies the convexity of $x \mapsto g(x, \mu)$ and the $c_L$-strong convexity of $(x, \alpha) \mapsto L(x, \alpha, m)$ in $\alpha$ uniformly in $m \in \mathcal{P}_2(\mathbb{R}^{2d})$, as a direct consequence of Proposition 1.2.

We now show that under these assumptions, there exists a unique solution to a FBSDE involving the Hilbertian inverse of $\nabla_\alpha L$.

**Hypothesis 2.4.** the derivatives $\nabla_x g, \nabla_x L, \nabla_\alpha L$ are well defined and continuous in all their arguments and $(x, \alpha, m) \mapsto (\nabla_x g(x, \mu), \nabla_x L(x, \alpha, m), \nabla_\alpha L(x, \alpha, m))$ is Lipschitz,
**Lemma 2.5.** Under Hypotheses 2.2 and 2.4, the Hilbertian mapping

$$ \alpha \mapsto \nabla_{\alpha} L(X, \alpha, \mathcal{L}(X, \alpha)), $$

admits a continuous inverse in $\alpha \in H$ for a given $X \in H$ denoted

$$ \alpha \mapsto \nabla_{\alpha} \tilde{L}^{-1}(X, \alpha). $$

Moreover, there exists a Lipschitz continuous function $L_{inv}: \mathbb{R}^{2d} \times \mathcal{P}_2(\mathbb{R}^{2d}) \to \mathbb{R}^d$ such that

$$ \forall (\alpha, X) \in H^{2d}, \quad \nabla_{\alpha} \tilde{L}^{-1}(X, \alpha) = L_{inv}(X, \alpha, \mathcal{L}(X, \alpha)), $$

and for any initial condition $X_0 \in H$ there exists a unique strong solution $(X_t, U_t, Z_t)_{t \in [0, T]}$ to the forward backward system

$$ (2.4) \qquad \left\{ \begin{array}{l} X_t = X_0 - \int_0^t \nabla_{\alpha} \tilde{L}^{-1}(X_s, U_s) ds + \sqrt{2\sigma} dB_t, \\ U_t = \nabla_x g(X_T, \mathcal{L}(X_T)) + \int_t^T \nabla_x L(X_s, \nabla_{\alpha} \tilde{L}^{-1}(X_s, U_s), \mathcal{L}(X_t, \nabla_{\alpha} \tilde{L}^{-1}(X_s, U_s))) ds - \int_t^T Z_s dB_s. \end{array} \right. $$

*Proof.* Let us first observe that $\nabla_{\alpha} \tilde{L}^{-1}$ is well defined and Lipschitz as a function from $H^{2d}$ into $H^d$. For a given $X \in H^d$, the invertibility follows from the strong monotonicity of $\alpha \mapsto \nabla_{\alpha} L(X, \alpha)$ on $H^d$. The fact that the mapping $(X, U) \mapsto \nabla_{\alpha} \tilde{L}^{-1}(X, U)$ is Lipschitz on $H^{2d}$ follows from the following inequality

$$ \forall (X, Y, \alpha, \alpha') \in H^{4d}, $$

$$ \frac{C_F^2}{2c_L} \|X - Y\|^2 + \langle \nabla_{\alpha} \tilde{L}(X, \alpha) - \nabla_{\alpha} \tilde{L}(Y, \alpha'), \alpha - \alpha' \rangle \geq \frac{c_L}{2} \|\alpha - \alpha'\|^2, $$

where $C_F$ indicates the Lipschitz norm of $X \mapsto \nabla_{\alpha} \tilde{L}(X, \alpha)$ which is bounded independently of $\alpha \in H^d$ by assumption. The fact that $\nabla_{\alpha} \tilde{L}^{-1}$ can be expressed as the lift of Lipschitz continuous function follows from [39] Lemma 3.10. In fact, one can directly check that for any $(x, a) \in \mathbb{R}^{2d}$, $(X, \alpha) \in H^{2d}$, $L_{inv}$ is given by

$$ L_{inv}(x, p, \mathcal{L}(X, \alpha)) = D_p H(x, p), $$

Consequently, (2.4) is a FBSDE with Lipschitz coefficients. It is standard [14, 20] that under this condition, wellposedness holds over a sufficiently short horizon of time and we now prove that this system is monotone yielding wellposedness on any time interval. Proceeding with

$$ F(X, U) = \nabla_{\alpha} \tilde{L}^{-1}(X, U), G(X, U) = \nabla_x L(X_s, \nabla_{\alpha} \tilde{L}^{-1}(X_s, U_s), \mathcal{L}(X_t, \nabla_{\alpha} \tilde{L}^{-1}(X_s, U_s))). $$

By the definition of $F = \tilde{L}^{-1}$ and the displacement monotonicity of $L$

$$ \forall (X, Y, \alpha, \alpha') \in H^4, $$

$$ \langle \begin{pmatrix} \tilde{F} \\ \tilde{G} \end{pmatrix} (X, U) - \begin{pmatrix} F \\ G \end{pmatrix} (Y, V), \begin{pmatrix} X - Y \\ U - V \end{pmatrix} \rangle \geq c_L \|F(X, U) - F(Y, V)\|^2. $$

Since $g$ is also displacement monotone by assumption, we believe the existence and uniqueness of a solution to (2.4) for any $T > 0$ follow from standard techniques. However since we couldn't find this exact result in the literature, we briefly explain how to obtain it in Lemma A.1 in the Appendix. $\square$

**Remark 2.6.** Since we are considering the Hilbertian inverse of $\alpha \mapsto \nabla_{\alpha} L(X, \alpha, \mathcal{L}(X, \alpha))$ and not the inverse in a $\alpha \in \mathbb{R}^d$ for a given measure $m \in \mathcal{P}_2(\mathbb{R}^{2d})$, it is important to note that we do not have the equality

$$ \nabla_{\alpha} \tilde{L}^{-1} = D_p H, $$

where $(x, p, m) \mapsto H(x, p, m)$ is the Hamiltonian associated to $L$. Let us also remark that it is not necessary to assume that

$$ \alpha \mapsto \nabla_{\alpha} L(X, \alpha, \mathcal{L}(X, \alpha)), $$

is Lipschitz. Indeed, strong monotonicity implies that its inverse is Lipschitz even if $\nabla_{\alpha} L$ is not.

We now present how the FBSDE (2.4) relates to equilibria of the mean field game of control (2.1).
**Theorem 2.7.** Under Hypotheses 2.2 and 2.4, there exists a unique solution to the mean field game of control (2.1). It is the unique control satisfying (2.5)

$$ \forall \alpha' \in (H^T)^d, \quad \langle \nabla_x g(X_T^\alpha, \mathcal{L}(X_T^\alpha)), X_T^{\alpha'} - X_T^\alpha \rangle + \int_0^T \left\langle \begin{pmatrix} \nabla_x L \\ \nabla_\alpha L \end{pmatrix} (X_t^\alpha, \alpha_t, \mathcal{L}(X_t^\alpha, \alpha_t)), \begin{pmatrix} X_t^{\alpha'} - X_t^\alpha \\ \alpha_t' - \alpha_t \end{pmatrix} \right\rangle dt \ge 0. $$

and is given by

$$ \forall t \le T, \quad \alpha_t = \nabla_\alpha \tilde{L}^{-1}(X_t, U_t), $$

where $(X_t, U_t, Z_t)_{t \in [0,T]}$ is the unique solution to (2.4).

*Proof.* The fact that any control solution to the mean field game problem must satisfy (2.5) is a direct consequence of Lemma 2.1 in the displacement monotone setting. Uniqueness of a solution to the mean field game problem then follows from the fact that, under our displacement monotonicity assumption on $L, g$, there can be at most one control satisfying (2.5). Finally it remains to check that (2.5) holds for

$$ \forall t \le T, \quad \alpha_t = \nabla_\alpha \tilde{L}^{-1}(X_t, U_t), $$

which is a direct consequence of the definition of $(X_t, U_t)_{t \in [0,T]}$.

□

Existence and uniqueness results of a solution to the mean field game of control under the standing assumptions is already well known, see for example the recent work [31]. The main novelty here is that the forward backward system can be made explicit by using the Hilbertian inverse instead of relying on an implicit fixed point map. In our opinion, this makes the exposition more straightforward and will also be relevant later in the article.

**2.1.2.** *An algorithm from the theory of monotone variational inequalities.* For a given control $(\alpha_t)_{t \in [0,T]} \in (H^T)^d$, we first introduce the following backward process

$$ U_t^\alpha = \nabla_x g(X_T^\alpha, \mathcal{L}(X_T^\alpha)) + \int_t^T \nabla_x L(X_s^\alpha, \alpha_s, \mathcal{L}(X_s^\alpha, \alpha_s)) ds - \int_t^T Z_s^\alpha dB_s. $$

The existence and uniqueness of a pair $(U^\alpha, Z^\alpha)_{t \in [0,T]}$ for a given control is classic in the theory of decoupled forward backward systems, see [46] Theorem 4.3.1. Indeed since the pair $(\alpha_s, X_s^\alpha)_{s \in [0,T]}$ is known in this case, the above system can be viewed as a standard backward Lipschitz SDE of the form

$$ (2.6) \qquad Y_t = \xi - \int_t^T f(\omega, Y_s, Z_s) ds - \int_t^T Z_s dB_s, $$

with

$$ f(\omega, Y_s, Z_s) = -\nabla_x L(X_s^\alpha(\omega), \alpha_s(\omega), \mathcal{L}(X_s^\alpha, \alpha_s)). $$

We now introduce the following functional on $(H^T)^d$

$$ v: \begin{cases} (H^T)^d \to (H^T)^d \\ (\alpha_t)_{t \in [0,T]} \mapsto (\nabla_\alpha L(X_t^\alpha, \alpha_t, \mathcal{L}(X_t^\alpha, \alpha_t)) - U_t^\alpha)_{t \in [0,T]} \end{cases}. $$

**Lemma 2.8.** Under Hypothesis 2.4, there exists a constant $C_v$ depending only on the Lipschitz constants of $\nabla_x g, \nabla_x L$ and $T$ only such that

$$ \forall (\alpha, \alpha') \in (H^T)^{2d}, \quad \|v(\alpha) - v(\alpha')\|_T \le C_v \|\alpha - \alpha'\|_T. $$

Furthermore, if Hypothesis 2.2 holds then $v$ is $c_L$-strongly monotone on $(H^T)^d$ and a control $\alpha^* \in (H^T)^d$ is a solution to the mean field game of control problem (2.1) if and only if

$$ \forall \alpha \in (H^T)^d \quad \langle v(\alpha^*), \alpha - \alpha^* \rangle \ge 0. $$

*Proof.* Let us first prove the statement on the Lipschitz regularity of $v$. It is quite evident that for any two controls $(\alpha, \alpha') \in (H^T)^{2d}$,

$$ \|X_t^\alpha - X_t^{\alpha'}\|_T \le \|\alpha - \alpha'\|_T. $$

Then applying Ito's Lemma for fixed $t \in [0, T]$

$$ \|U_T^\alpha - U_T^{\alpha'}\|^2 = \|U_t^\alpha - U_t^{\alpha'}\|^2 - \mathbb{E} \left[ \int_t^T \left( \nabla_x \tilde{L}(X_s^\alpha, \alpha_s) - \nabla_x \tilde{L}(X_s^{\alpha'}, \alpha'_s) \right) \cdot \left( U_s^\alpha - U_s^{\alpha'} \right) ds \right] + \int_t^T \|Z_s^\alpha - Z_s^{\alpha'}\|^2 ds $$
which implies that

$$ \forall t \in [0, T], \quad \|U_t^\alpha - U_t^{\alpha'}\|^2 \le (\|\nabla_x g\|_{Lip} + T\|\nabla_x L\|_{Lip})\|\alpha - \alpha'\|_T^2 + \frac{\|\nabla_x L\|_{Lip}}{4} \int_t^T \|U_s^\alpha - U_s^{\alpha'}\|^2 ds, $$

and the conclusion follows from Gronwall lemma. For the second part of the lemma, by definition of $(U_t^\alpha)_{t \in [0, T]}$,

$$ \forall \alpha, \alpha' \in (H^T)^d, $$

$$ \langle v(\alpha), \alpha' - \alpha \rangle^T = \langle \nabla_x g(X_T^\alpha, \mathcal{L}(X_T^\alpha)), X_T^{\alpha'} - X_T^\alpha \rangle + \int_0^T \left\langle \begin{pmatrix} \nabla_x L \\ \nabla_\alpha L \end{pmatrix} (X_t^\alpha, \alpha_t, \mathcal{L}(X_t^\alpha, \alpha_t)), \begin{pmatrix} X_t^{\alpha'} - X_t^\alpha \\ \alpha_t' - \alpha_t \end{pmatrix} \right\rangle dt, $$

and the result follows from Hypothesis 2.2 and Theorem 2.7 □

With this, we have proved that the problem of solving for the mean field game equilibrium can be reformulated as solving a variational inequality on the space of controls $(H^T)^d$. Taking inspiration from the dual extrapolation algorithm introduced in [32], we present an algorithm that converges to the mean field game optimal control starting from any initial condition

**Theorem 2.9.** Take an initial condition $\alpha_1 \in (H^T)^{2d}$, for $n \ge 1$ we introduce the sequences

$$ \left\{ \begin{array}{l} \alpha_{n+\frac{1}{2}} = \alpha_n - \gamma v(\alpha_n), \\ \alpha_{n+1} = \alpha_n - \gamma v(\alpha_{n+\frac{1}{2}}). \end{array} \right. $$

If

$$ \gamma \le \frac{1}{\|v\|_{Lip,H^T}}, $$

then under Hypotheses 2.4, 2.2, letting

$$ \bar{\alpha}_n = \frac{1}{n} \sum_{i=1}^{n} \alpha_{i+\frac{1}{2}}, $$

the following holds

$$ \forall n \ge 1, \quad \|\alpha^* - \bar{\alpha}_n\|_T^2 \le \frac{\|\alpha^* - \alpha_1\|_T^2}{2\gamma c_L n}. $$

*Proof.* Let us first observe by an application of Lemma 1.4, for any control $\alpha \in (H^T)^d$,

$$ \forall n \ge 1, \quad \sum_{i=1}^{n} \langle v(\alpha_{i+\frac{1}{2}}), \alpha_{i+\frac{1}{2}} - \alpha \rangle \le \frac{\|\alpha - \alpha_1\|_T^2}{2\gamma}. $$

On the other hand, introducing the notation

$$ \bar{\alpha}_n = \frac{1}{n} \sum_{i=1}^{n} \alpha_{i+\frac{1}{2}}, $$

$$ \begin{align*} \sum_{i=1}^{n} \langle v(\alpha_{i+\frac{1}{2}}), \alpha_{i+\frac{1}{2}} - \alpha \rangle &= \sum_{i=1}^{n} \langle v(\alpha_{i+\frac{1}{2}}) - v(\alpha), \alpha_{i+\frac{1}{2}} - \alpha \rangle + \langle v(\alpha), \alpha_{i+\frac{1}{2}} - \alpha \rangle \\ &\ge n \langle v(\alpha), \bar{\alpha}_n - \alpha \rangle + c_L \sum_{i=1}^{n} \|\alpha - \alpha_{i+\frac{1}{2}}\|_T^2 \\ &\ge n \langle v(\alpha), \bar{\alpha}_n - \alpha \rangle + n c_L \|\bar{\alpha}_n - \alpha\|_T^2, \end{align*} $$

where we used the strong monotonicity of $v$ on $(H^T)^d$. Putting together those two inequalities and evaluating for $\alpha = \alpha^*$ solution of the mean field game problem yields

$$ \forall n \ge 1, \quad \|\alpha^* - \bar{\alpha}_n\|^2 \le \frac{\|\alpha^* - \alpha_1\|^2}{2\gamma c_L n}. $$

□
**Remark 2.10.** A corollary of this proof is that

$$ \sum_{n=1}^{+\infty} \|\alpha^* - \alpha_{n+\frac{1}{2}}\|_T^2 < +\infty, $$

which implies that eventually, the sequence of last iterates $(\alpha_{n+\frac{1}{2}})_{n \in \mathbb{N}}$ converges to $\alpha^*$ with

$$ \forall C > 0, \exists n_0, \forall n \ge n_0, \quad \|\alpha_{n+\frac{1}{2}} - \alpha^*\|_T^2 \le \frac{C}{n}, $$

but this does not give an explicit rate, hence the choice of working rather on $(\bar{\alpha}_n)_{n \in \mathbb{N}}$

The above iterative procedure can be generalized to other monotonicity conditions, as we will see later. In the specific setting of strong monotonicity in the control, we can show that for as sufficiently small step size, the convergence of the last iterate is exponential

**Theorem 2.11.** Under the same assumptions as Theorem 2.9, If

$$ \gamma < \min \left( \frac{1}{2\|v\|_{Lip}}, \frac{c_L}{\|v\|_{Lip}^2} \right), $$

then there exists a constant $\lambda \in (0, 1)$ depending only on $\gamma$, $\|v\|_{Lip}$ and $c_L$ such that

$$ \forall n \in \mathbb{N}, \quad \|\alpha_n - \alpha^*\| \le \lambda^n \|\alpha_0 - \alpha^*\|. $$

*Proof.* In the constant step regime let us first remark that the sequence $(\alpha_i)_{i \in \mathbb{N}}$ is defined through

$$ \alpha_{i+1} = \alpha_i - \gamma v(\alpha_i - \gamma v(\alpha_i)). $$

Letting $\alpha^*$ be the control associated to (2.7), it follows that

$$ \begin{aligned} \|\alpha_{i+1} - \alpha^*\|_T^2 &= \|\alpha_i - \alpha^* - \gamma v(\alpha_i - \gamma v(\alpha_i))\|_T^2 \\ &= \|\alpha_i - \alpha^*\|_T^2 + \gamma^2 \|v(\alpha_i - \gamma v(\alpha_i))\|_T^2 - 2\gamma \langle \alpha_i - \alpha^*, v(\alpha_i - \gamma v(\alpha_i)) \rangle^T. \end{aligned} $$

Using the fact that

$$ v(\alpha^*) = 0, $$

we get

$$ \|\alpha_{i+1} - \alpha^*\|_T^2 = \|\alpha_i - \alpha^*\|_T^2 + \gamma^2 \|v(\alpha_i - \gamma v(\alpha_i))\|_T^2 - 2\gamma \langle \alpha_i - \alpha^*, v(\alpha_i - \gamma v(\alpha_i)) - v(\alpha^*) \rangle^T. $$

Using the strong monotonicity of $v$,

$$ \|\alpha_{i+1} - \alpha^*\|_T^2 \le \|\alpha_i - \alpha^*\|_T^2 + \gamma^2 \|v(\alpha_i - \gamma v(\alpha_i))\|_T^2 - 2\gamma c_L \|\alpha_i - \gamma v(\alpha_i) - \alpha^*\|^2 - 2\gamma^2 \langle v(\alpha_i), v(\alpha_i - \gamma v(\alpha_i)) - v(\alpha^*) \rangle^T $$

Which implies that

$$ \begin{aligned} \|\alpha_{i+1} - \alpha^*\|_T^2 \le & (1 - 2\gamma c_L) \|\alpha_i - \alpha^*\|_T^2 - \gamma^2 \|v(\alpha_i - \gamma v(\alpha_i))\|_T^2 - 2\gamma^3 c_L \|v(\alpha_i)\|^2 + 4\gamma^2 c_L \langle \alpha_i - \alpha^*, v(\alpha_i) \rangle^T \\ & - 2\gamma^2 \langle v(\alpha_i) - v(\alpha_i - \gamma v(\alpha_i)), v(\alpha_i - \gamma v(\alpha_i)) \rangle^T. \end{aligned} $$

Since $v$ is Lipschitz clearly

$$ \begin{aligned} \|\alpha_{i+1} - \alpha^*\|_T^2 \le & (1 - 2\gamma c_L + 4\gamma^2 c_L \|v\|_{Lip}) \|\alpha_i - \alpha^*\|_T^2 - \gamma^2 \|v(\alpha_i - \gamma v(\alpha_i))\|_T^2 \\ & - 2\gamma^3 c_L \|v(\alpha_i)\|^2 + 2\gamma^3 \|v\|_{Lip} \|v(\alpha_i)\| \|v(\alpha_i - \gamma v(\alpha_i))\|, \end{aligned} $$

and we finally obtain

$$ \|\alpha_{i+1} - \alpha^*\|_T^2 \le (1 - 2\gamma c_L + 4\gamma^2 c_L \|v\|_{Lip}) \|\alpha_i - \alpha^*\|_T^2 + \gamma^2 \left( \gamma \frac{\|v\|_{Lip}^2}{c_L} - 1 \right) \|v(\alpha_i - \gamma v(\alpha_i))\|_T^2. $$

In particular, letting

$$ \gamma^* = \min \left( \frac{1}{2\|v\|_{Lip}}, \frac{c_L}{\|v\|_{Lip}^2} \right), $$

for any $\gamma < \gamma^*$ the sequence $(\alpha_i)_{i \in \mathbb{N}}$ converges exponentially at the following rate

$$ \forall n \in \mathbb{N}, \quad \|\alpha_n - \alpha^*\|^2 \le (1 - 2\gamma c_L + 4\gamma^2 c_L \|v\|_{Lip}^2)^n \|\alpha_0 - \alpha^*\|^2. $$

□
**Remark 2.12.** It is well known that another regime of wellposedness for the mean field game problem 2.1 is a strong monotonicity condition on the forward variable $X$. Namely, if coefficients are Lipschitz, $\nabla_{\alpha} \tilde{L}$ has a well defined, Lipschitz inverse and the following monotonicity condition is satisfied

$$ \exists c_L > 0, \quad \forall (X, Y, \alpha, \alpha') \in H^4, \\ \langle \nabla_x g(X, \mathcal{L}(X)) - \nabla_x g(Y, \mathcal{L}(Y)), X - Y \rangle \ge c_L \|X - Y\|^2, \\ \langle \begin{pmatrix} \nabla_x L \\ \nabla_{\alpha} L \end{pmatrix} (X, \alpha, \mathcal{L}(X, \alpha)) - \begin{pmatrix} \nabla_x L \\ \nabla_{\alpha} L \end{pmatrix} (Y, \alpha', \mathcal{L}(Y, \alpha')), \begin{pmatrix} X - Y \\ \alpha - \alpha' \end{pmatrix} \rangle \ge c_L \|X - Y\|^2 $$

then there exists a unique strong solution to (2.4)(see [12] for example). Although most results presented in this article extend easily to this setting (in particular the convergence of $(\bar{\alpha}_n)_{n \in \mathbb{N}}$), it is unclear to us whether this Theorem on the exponential convergence of last iterates remains true. This is why we presented first a more general result on the convergence of averaged controls $(\bar{\alpha}_n)_{n \in \mathbb{N}}$.

**2.1.3. Link with the fictitious play.** The algorithm we present is clearly different from the fictitious play algorithm in mean field game. The most obvious difference being that we consider dynamic on controls rather than on the distribution of players, but the learning procedure is also quite different. Nevertheless, there is a strong link between the two algorithms, namely the fictitious play can also be interpreted as solving a variational inequality.

In this section letting $(\mu_t)_{t \in [0,T]} \subset \mathcal{P}_2(\mathbb{R}^d)$ we consider the cost

$$ J(\alpha, \mu_t)_{t \in [0,T]} = \begin{cases} \mathbb{E} \left[ g(X_T^\alpha, \mu_T) + \int_0^T \mathbf{L}(X_t^\alpha, \alpha_t, \mu_t) dt \right] \\ X_t^\alpha = X_0 - \int_0^t \alpha_t dt + \sqrt{2\sigma} B_t \end{cases} $$

with

$$ \mathbf{L}(x, \alpha, \mu) = L(x, \alpha) + f(x, \mu), $$

and the mean field game problem of finding a flow of measure $(\mu^*)_{t \in [0,T]}$ such that

$$ \left\{ \begin{array}{l} \alpha^* = \underset{\alpha}{\operatorname{arginf}} J(\alpha, (\mu^*)_{t \in [0,T]}), \\ \forall t \in [0, T], \quad \mu_t^* = \mathcal{L}(X_t^{\alpha^*}). \end{array} \right. $$

Let

$$ P_a = \left\{ (m_t)_{t \in [0,T]} \subset \mathcal{P}_2(\mathbb{R}^{2d}), \exists \alpha \in H^T, m = (\mathcal{L}(X_t^\alpha, \alpha_t))_{t \in [0,T]} \right\}, $$

the set of all possible distribution. Indicating the duality product between $f \in C_b(\mathbb{R}^d)$ and $\mu \in \mathcal{P}(\mathbb{R}^d)$ by

$$ (f, \mu) = \int_{\mathbb{R}^d} f(x) \mu(dx), $$

we introduce the following duality product between flot of measures $(\mu_t)_{t \in [0,T]} \in C([0,T], \mathcal{P}_2(\mathbb{R}^{2d}))$ and functions $(f,g) \in C([0,T], C(\mathbb{R}^2, \mathbb{R})) \times C(\mathbb{R}^{2d}, \mathbb{R})$

$$ ((f,g), \mu)_T = (g, \mu_T) + \int_0^T (f, \mu_t) dt. $$

For a given $m \in P_a$ with first marginal $\mu$ we define

$$ v(m) = ((t, x, \alpha) \mapsto \mathbf{L}(x, \alpha, \mu_t), x \mapsto g(x, \mu_T)), $$

and by definition, it follows that for any $m' \in P_a$ there exists a control $(\alpha_t)_{t \in [0,T]}$ such that

$$ (v(m), m')_T = J((\alpha_t)_{t \in [0,T]}, (\mu_t)_{t \in [0,T]}). $$

In particular solving the mean field game problem is equivalent to finding $m^* \in P_a$ such that

$$ \forall m \in P_a, \quad (v(m^*), m - m^*)_T \ge 0, $$

Moreover let us observe that for any $m, n \in P_a$, with first marginal being respectively $\mu^m, \mu^n$

$$ (v(m) - v(n), m - n)_T = (g(\cdot, \mu_T^m) - g(\cdot, \mu_T^n), \mu_T^m - \mu_T^n) + \int_0^T (f(\cdot, \mu_t^m) - f(\cdot, \mu_t^n), \mu_t^m - \mu_t^n) dt. $$

Consequently, if both $f$ and $g$ are flat monotone then $v$ is a monotone functional from $P_a$ into $C([0, T], C(\mathbb{R}^2, \mathbb{R})) \times C(\mathbb{R}^{2d}, \mathbb{R})$ with the duality product $(\cdot, \cdot)_T$.
**2.2. Beyond mean field games: monotone FBSDE.** In the case of mean field game, there is a canonical monotone functional arising from the optimal control problem behind the game. We are now interested in extending the algorithm presented to general monotone FBSDEs. We consider the following system

$$ (2.7) \qquad \begin{cases} X_t = X_0 - \int_0^t F(X_s, U_s, \mathcal{L}(X_s, U_s))ds + \sqrt{2\sigma}dB_t, \\ U_t = g(X_T, \mathcal{L}(X_T)) + \int_t^T G(X_s, U_s, \mathcal{L}(X_s, U_s))ds - \int_t^T Z_sdB_s. \end{cases} $$

Clearly, the forward backward system associated to the mean field game of control falls into a particular case of (2.7). We place ourselves under the following assumptions

**Hypothesis 2.13.** $F, G: \mathbb{R}^{2d} \times \mathcal{P}_2(\mathbb{R}^{2d}) \to \mathbb{R}^d$ are Lipschitz and $g: \mathbb{R}^d \times \mathcal{P}_2(\mathbb{R}^d) \to \mathbb{R}^d$ is Lipschitz

We also assume that coefficients are $L^2$-monotone,

**Hypothesis 2.14.**

$$ \begin{gathered} \exists c_F > 0, \quad \forall (X, Y, U, V) \in H^4, \\ \langle g(X, \mathcal{L}(X)) - g(Y, \mathcal{L}(Y)), X - Y \rangle \ge 0, \\ \langle \left( \begin{pmatrix} F \\ G \end{pmatrix} \right) (X, Y, \mathcal{L}(X, Y)) - \left( \begin{pmatrix} F \\ G \end{pmatrix} \right) (Y, V, \mathcal{L}(Y, V)), \begin{pmatrix} X - Y \\ U - V \end{pmatrix} \rangle \ge c_F \|U - V\|^2 \end{gathered} $$

Obviously, this assumption corresponds to the displacement monotonicity assumption on the La-grangian $L$ in the case of MFGs. It has already been observed numerous times that the wellposedness of this system under $L^2$-monotonicity does not depend on whether or not the coefficients $F, G$ are gradients. First in Lions' Lectures with the Hilbertian approach to master equations [36, 14], then in [9] with a purely probabilistic approach, and more recently in [39] for weak solutions.

**Theorem 2.15.** Under Hypotheses 2.13 and 2.14, for any $X_0 \in H$ there exists a unique strong solution $(X_t, U_t, Z_t)_{t \in [0, T]}$ to the FBSDE (2.7).

*Proof.* The existence of a strong solution is a direct consequence of [39] Lemma 3.21 combined with Lemma 3.9. As for uniqueness it is a natural consequence of the strong monotonicity of coefficients. Let us fix an initial condition $X \in H$ and assume that there exist two solutions $(X_t^i, U_t^i, Z_t^i)_{t \in [0, T]}$ for $i = 1, 2$ starting from $X_0^i = X$. Defining

$$ \forall t \in [0, T], \quad V_t = (U_t^1 - U_t^2) \cdot (X_t^1 - X_t^2), $$

using the monotonicity of $(F, G)$, $g$ and applying Ito's Lemma to $(V_t)_{t \in [0, T]}$ we get

$$ \forall t \in [0, T], \quad 0 \le \mathbb{E}[V_T] \le \mathbb{E}[V_t] \le \mathbb{E}[V_0] = 0. $$

Then using this time the strong monotonicity of $(F, G)$ in $U$ it follows that

$$ \mathbb{E} \left[ \int_0^T |U_t^1 - U_t^2|^2 dt \right] = 0, $$

and by a direct application of Gronwall's lemma we also have

$$ \forall t \in [0, T], \quad \|X_t^1 - X_t^2\| = 0, $$

hence the uniqueness of a strong solution.

□

A natural question at this point is whether we can associate a monotone variational inequality to this system as we did in the case of mean field games 2.1. To that end, we introduce the following parametrization

$$ \forall \alpha \in H^T, \quad X_t^\alpha = X_0 - \int_0^t \alpha_s ds + \sqrt{2\sigma}B_t. $$

In the case of MFGs, this comes from the formulation of the control problem. Here, it can be seen as a linearization of the FBSDE (2.7). In particular, $(X_t^*)_{t \in [0, T]} \stackrel{H^T}{=} (X_t)_{t \in [0, T]}$ if and only if

$$ (2.8) \qquad \mathbb{E} \left[ \int_0^T |\alpha_t - F(X_t, U_t, \mathcal{L}(X_t, U_t))|^2 dt \right] = 0. $$

Let us now remark the following
**Lemma 2.16.** under Hypotheses 2.13 and 2.14, the Hilbertian lift of $F$, $\tilde{F}$ has a well defined inverse in $U \in H$ for any given $X \in U$. Denoting this inverse by $\tilde{F}_u^{-1}$, the strong monotonicity of $\tilde{F}$ on $H$ yields

$$ \forall (X, Y, \alpha, \alpha') \in H^4, \quad \|\tilde{F}_u^{-1}(X, \alpha) - \tilde{F}_u^{-1}(Y, \alpha')\| \le C (\|X - Y\| + \|\alpha - \alpha'\|), \\ \text{where } C \text{ is a constant depending only on } \|\tilde{F}\|_{\text{Lip}} \text{ and } c_F. $$

Since it has now been established that $\tilde{F}$ is invertible, (2.8) implies that a control $\alpha$ solves the FBSDE (2.7) if and only if

$$ (U_t)_{t \in [0,T]} = (\tilde{F}_u^{-1}(X_t^\alpha, \alpha_t))_{t \in [0,T]}. $$

In light of this new representation, we introduce the following linear form

**Lemma 2.17.** For a given $\alpha \in (H^T)^d$, let $l(\alpha)$ be defined by

$$ \forall \alpha' \in (H^T)^d, \quad l(\alpha)(\alpha') = \\ \langle g(X_T^\alpha), X_T^{\alpha'} \rangle + \int_0^T \langle \tilde{F}_u^{-1}(X_t^\alpha, \alpha_t), \alpha_t' \rangle + \langle \tilde{G}(X_t^\alpha, \tilde{F}_u^{-1}(X_t^\alpha, \alpha_t)), X_t^{\alpha'} \rangle dt. $$

Under Hypotheses 2.14 and 2.13

$$ - \forall (\alpha, \alpha') \in H^T $$

$$ (l(\alpha) - l(\alpha'))(\alpha - \alpha') \ge c_F \|\tilde{F}_u^{-1}(X_t^\alpha, \alpha) - \tilde{F}_u^{-1}(X_t^{\alpha'}, \alpha')\|_T^2 $$

- Let

$$ \forall t \in [0, T] \quad \alpha_t^* = \tilde{F}(X_t, U_t), $$

for $(X_t, U_t, Z_t)_{t \in [0,T]}$ the unique solution to (2.7), then for any $(\alpha', \alpha'') \in (H^T)^{2d}$

$$ l(\alpha^*)(\alpha' - \alpha'') = 0. $$

**Proof.** This first statement is a direct consequence of the monotonicity of coefficients. Following Hypothesis 2.14, we already know that for all $(X, Y, U, V) \in H^{4d}$,

$$ \langle \tilde{F}(X, U) - \tilde{F}(Y, V), U - V \rangle + \langle \tilde{G}(X, U) - \tilde{G}(Y, V), X - Y \rangle \ge c_F \|U - V\|^2. $$

By choosing $U = \tilde{F}_u^{-1}(X, \alpha)$, $V = \tilde{F}_u^{-1}(Y, \alpha')$ for some $\alpha, \alpha' \in H^d$, we get

$$ \langle \alpha - \alpha', \tilde{F}_u^{-1}(X, \alpha) - \tilde{F}_u^{-1}(Y, \alpha) \rangle + \langle \tilde{G}(X, \tilde{F}_u^{-1}(X, \alpha)) - \tilde{G}(Y, \tilde{F}_u^{-1}(Y, \alpha)), X - Y \rangle \ge c_F \|\tilde{F}_u^{-1}(X, \alpha) - \tilde{F}_u^{-1}(Y, \alpha)\|^2, $$
and the first statement follows naturally.

As for the second statement, it suffices to remark that

$$ \|X_t^{\alpha*} - X_t\|_T = 0. $$

□

We now have all the tools at hand to present a monotone functional associated to the FBSDE (2.7)

**Lemma 2.18.** Under Hypotheses 2.14 and 2.13, for any $\alpha \in (H^T)^d$, $X_0 \in H$ the system

$$ (2.9) \qquad \left\{ \begin{array}{l} X_t^\alpha = X_0 - \int_0^t \alpha_s ds + \sqrt{2\sigma} dB_t, \\ U_t^\alpha = \tilde{g}(X_T^\alpha) + \int_t^T \tilde{G}(X_s^\alpha, \tilde{F}_u^{-1}(X_s^\alpha, \alpha_s)) ds - \int_t^T Z_s^\alpha dB_s, \end{array} \right. $$

has a unique solution $(X_t^\alpha, U_t^\alpha, Z_t^\alpha)_{t \in [0,T]}$. Moreover, letting

$$ v: \left\{ \begin{array}{l} (H^T)^d \to (H^T)^d \\ (\alpha_t)_{t \in [0,T]} \mapsto (\tilde{F}_u^{-1}(X_t^\alpha, \alpha_t) - U_t^\alpha)_{t \in [0,T]} \end{array} \right. , $$

- there exists a constant depending only on $T$, $\|(\tilde{F}, \tilde{G})\|_{\text{Lip}}$, $\|\tilde{g}\|_{\text{Lip}}$ and $c_F$ such that

$$ \forall (\alpha, \alpha') \in (H^T)^2, \quad \|v(\alpha) - v(\alpha')\|_T \le C \|\alpha - \alpha'\|_T. $$

$$ - \forall (\alpha, \alpha' \alpha'') \in (H^T)^3 $$

$$ \langle v(\alpha), \alpha' - \alpha'' \rangle_T = l(\alpha)(\alpha' - \alpha''). $$
*Proof.* Since the system (2.9) is a decoupled forward backward system with Lipschitz coefficients the wellposedness follows once again from [46] Theorem 4.3.1. Then, the first statement follows from an application of Gronwall's lemma. The second statement is just a consequence of the definition of $(U_t^\alpha)_{t \in [0,T]}$ and Ito's lemma. $\square$

We may now state the corresponding theorem to Theorem 2.9 in the case of FBSDEs

**Theorem 2.19.** Let $v$ be defined as in Lemma 2.18, for a given initial condition $\alpha_1 \in (H^T)^d$ and $n \ge 1$ we introduce the sequences

$$ \left\{ \begin{array}{l} \alpha_{n+\frac{1}{2}} = \alpha_n - \gamma v(\alpha_n), \\ \alpha_{n+1} = \alpha_n - \gamma v(\alpha_{n+\frac{1}{2}}), \end{array} \right. $$

If

$$ \gamma \le \frac{1}{\|v\|_{Lip,H^T}}, $$

then under Hypotheses 2.14, and 2.13, letting

$$ \bar{U}^n = \frac{1}{n} \sum_{i=1}^{n} F_u^{-1}(X^{\alpha_{i+\frac{1}{2}}}, \alpha_{i+\frac{1}{2}}), $$

the following holds

$$ \forall n \ge 1, \quad \|U_n - \bar{U}^n\|_T^2 \le \frac{\|F(X_n, U_n) - \alpha_1\|_T^2}{2\gamma c_L n}. $$

- Letting

$$ \forall t \in [0, T], \quad \bar{X}_t^n = X_0 - \int_0^t \tilde{F}(\bar{X}_s^n, \bar{U}_s^n) ds + \sqrt{2\sigma} B_t, $$

there exists a constant $C$ depending only on $T$ and $\|\tilde{F}\|_{Lip}$ such that

$$ \|\bar{X}_t^n - X_t\|_T^2 \le C \frac{\|F(X_t, U_t) - \alpha_1\|_T^2}{2\gamma c_L n}. $$

*Proof.* We follow the same idea as in Theorem 2.9 of using the monotone functional $v$ combined with Lemma 1.4. This yields

$$ \forall \alpha \in H^T, \quad n\langle v(\alpha), \bar{\alpha}_n - \alpha \rangle + c_F \sum_{i=1}^n \|\tilde{F}_u^1(X^\alpha, \alpha) - \tilde{F}_u^1(X^{\alpha_{i+\frac{1}{2}}}, \alpha_{i+\frac{1}{2}})\|_T^2 \le \sum_{i=1}^n \langle v(\alpha_{i+\frac{1}{2}}), \alpha_{i+\frac{1}{2}} - \alpha \rangle \le \frac{\|\alpha - \alpha_1\|_T^2}{2\gamma}. $$

Evaluating this expression for $\alpha^* = F(X, U)$ where $(X_t, U_t, Z_t)_{t \in [0,T]}$ is the unique strong solution to (2.7) we get

$$ n \|U_n - \frac{1}{n} \sum_{i=1}^{n} \tilde{F}_u^1(X^{\alpha_{i+\frac{1}{2}}}, \alpha_{i+\frac{1}{2}})\|_T^2 \le \sum_{i=1}^{n} \|U_n - \tilde{F}_u^1(X^{\alpha_{i+\frac{1}{2}}}, \alpha_{i+\frac{1}{2}})\|_T^2 \le \frac{\|\tilde{F}(X_n, U_n) - \alpha_1\|_T^2}{2\gamma c_F}, $$

proving the first claim. The second claim follows naturally from an application of Gronwall Lemma. $\square$

**Remark 2.20.** Clearly this procedure requires precise knowledge of $F_u^{-1}$ to lead to a reasonable algorithm. Although this was not a problem in mean field games of control, as no inverse was used in the definition of $v$, it appears that this algorithm is not very suitable to compute the solution of Forward backward systems where the forward driver $F$ depends on the law of the backward process $U$. In this case the Hilbertian inverse is in general not trivial, whereas when $F$ doesn't depend on the law of the backward process, $\tilde{F}_u^{-1}$ is just

$$ (X, U) \mapsto F_u^{-1}(X, U, \mathcal{L}(X)) $$

where $u \mapsto F_u^{-1}(x, u, \mu)$ indicates the inverse of $u \mapsto F(x, u, \mu)$ in $\mathbb{R}^d$ for a given pair $(x, \mu) \in \mathbb{R}^d \times \mathcal{P}_2(\mathbb{R}^d)$.

**Remark 2.21.** Let us observe that, if the following monotonicity condition

$$ \begin{array}{l} \exists c_F > 0, \quad \forall (X, Y, U, V) \in H^4, \\ \langle g(X, \mathcal{L}(X)) - g(Y, \mathcal{L}(Y)), X - Y \rangle \ge 0, \\ \langle \begin{pmatrix} F \\ G \end{pmatrix} (X, Y, \mathcal{L}(X, Y)), \begin{pmatrix} F \\ G \end{pmatrix} (Y, V, \mathcal{L}(Y, V)), \begin{pmatrix} X - Y \\ U - V \end{pmatrix} \rangle \ge c_F \|F(X, U, \mathcal{L}(X, U)) - F(Y, V, \mathcal{L}(Y, V))\|^2, \end{array} $$
is satisfied instead. Then

$$ \forall (\alpha, \alpha') \in H^T, \quad \langle v(\alpha) - v(\alpha'), \alpha - \alpha' \rangle \ge c_F \|\alpha - \alpha'\|^2, $$

and an analogue to Theorem 2.11, on the exponential convergence to the solution of the FBSDE (2.7) follows naturally. The Wellposedness of (2.7) under such monotonicity condition is proven in the appendix (A.1). In particular, this monotonicity condition is always satisfied whenever coefficients are Lipschitz and the pair $(F, G)$ is strongly monotone in $(X, U)$

In the case of mean field games, there exists a canonical monotone functional coming from the optimal control representation of the problem. In a sense this corresponds to a potential regime compared to general FBSDEs. In this latter case, it is unclear to us whether the parametrization we introduce (2.9) yields the best results. Although it certainly feels like a natural extension to MFGs, depending on the problem there might exist parametrizations leading to much better convergence rates.

Before ending this section we present a lemma that will prove useful to estimate the convergence rate of a sequence of approximate solutions

**Lemma 2.22.** Under Hypotheses 2.13 and 2.14, letting $(X_s, U_s, Z_s)_{s \in [0,T]}$ be the unique strong solution to (2.7), there exists a constant $C$ depending only on $c_F, T$ and the Lipschitz constants of $F, G, g$ such that for any control $\alpha \in H^T$

$$ \forall t \in [0, T], \quad \|U_t^\alpha - U_T\|, \|X_t^\alpha - X_t\| \le C \|v(\alpha)\|_T. $$

*Proof.* By definition of $v(\alpha)$

$$ \forall t \in [0, T], \quad \alpha_t = F(X_t^\alpha, U_t^\alpha + v(\alpha)_t, \mathcal{L}(X_t^\alpha, U_t^\alpha + v(\alpha)_t)). $$

Naturally, this is equivalent to the fact that the pair $(X_t^\alpha, U_t^\alpha)_{t \in [0, T]}$ solves

$$ (2.10) \quad \left\{ \begin{array}{l} X_t^\alpha = X_0 - \int_0^t F(X_s^\alpha, U_s^\alpha + v(\alpha)_s, \mathcal{L}(X_s^\alpha, U_s^\alpha + v(\alpha)_s))ds + \sqrt{2\sigma}dB_t, \\ U_t^\alpha = g(X_T^\alpha, \mathcal{L}(X_T^\alpha)) + \int_t^T G(X_s^\alpha, U_s^\alpha + v(\alpha)_s, \mathcal{L}(X_s^\alpha, U_s^\alpha + v(\alpha)_s))ds - \int_t^T Z_s^\alpha dB_s. \end{array} \right. $$

Let $(X_t, U_t)_{t \in [0, T]}$ be the unique solution to (2.7) we define

$$ \forall t \in [0, T], \quad V_t = (U_t - U_t^\alpha) \cdot (X_t - X_t^\alpha). $$

By Ito's Lemma,

$$ \forall t \in [0, T], \quad V_T = V_t - \int_t^T (\Delta F_s \cdot (U_s - U_s^\alpha) + \Delta G_s \cdot (X_s - X_s^\alpha)) ds, $$

with the notation

$$ \forall s \in [0, T], \quad \Delta F_s = (X_s, U_s, \mathcal{L}(X_s, U_s)) - (X_s^\alpha, U_s^\alpha + v(\alpha)_s, \mathcal{L}(X_s^\alpha, U_s^\alpha + v(\alpha)_s)), $$

and an analogue definition for $(\Delta G_s)_{s \in [0, T]}$. By the monotonicity of coefficients and their Lipschitz regularity, there exists a constant $C_{Lip}$ depending only on the Lipschitz norms of $F, G$ such that

$$ (2.11) \quad \forall t \in [0, T], \quad 0 \le \mathbb{E}[V_T] \le \mathbb{E}[V_t] + C_{Lip} \underbrace{\int_0^T \|v(\alpha)_s\| (\|X_s - X_s^\alpha\| + \|U_s - U_s^\alpha\|) ds}_{C_T^\alpha}. $$

Using the strong monotonicity of $(F, G)$ we deduce that

$$ \forall t \in [0, T], \quad c_F \int_0^t |U_s - U_s^\alpha|^2 ds + V_t \le V_0 + C_{Lip} \int_0^t \|v(\alpha)_s\| (\|X_s - X_s^\alpha\| + \|U_s - U_s^\alpha\|) ds. $$

Using (2.11), it follows that

$$ (2.12) \quad \forall t \in [0, T], \quad \frac{c_F}{2} \int_0^t \|U_s - U_s^\alpha\|^2 ds \le \mathbb{E}[V_0] + C_{Lip}^2 \left( \frac{2}{c_F} + 1 \right) \|v(\alpha)\|_T + \int_0^t \|X_s - X_s^\alpha\|^2 ds + C_T^\alpha. $$

Since

$$ \forall t \in [0, t], \quad \|X_t^\alpha - X_t\|^2 \le C_{Lip} \int_0^t (\|X_s^\alpha - X_s\|^2 + 2\|U_s^\alpha - U_s\|^2 + 2\|v(\alpha)_s\|^2) ds, $$
estimating the term depending on $(U_s^\alpha - U_s)_{s \in [0,t]}$ with (2.12), we get by an application of Gronwall's lemma that there exists a constant $C$ depending on $c_F, T$ and the Lipschitz constants of $(F, G)$ such that

$$ (2.13) \qquad \forall t \in [0, T], \quad \|X_t^\alpha - X_t\|^2 \le C (\|v(\alpha)\|_T + C_T^\alpha). $$

Integrating from 0 to $T$ and using Cauchy-schwartz inequality we get

$$ \int_0^T \|X_s - X_s^\alpha\|^2 ds \le \left( \|v(\alpha)\|_T^2 (C + C_{Lip}^2 C^2) + C \int_0^T \|v(\alpha)_s\| \|U_s^\alpha - U_s\| ds \right). $$

In particular, by plugging back this estimate in (2.13),

$$ \forall t \in [0, T], \quad \|X_t^\alpha - X_t\|^2 \le C' \left( \|v(\alpha)\|_T^2 + \int_0^T \|v(\alpha)_s\| \|U_s^\alpha - U_s\| ds \right), $$

for a constant $C'$ with the same dependencies as $C$. Using this estimate, we can estimate $\int_0^T \|U_s^\alpha - U_s\|^2 ds$ in a similar fashion and we conclude to the existence of a constant $C''$ depending on $c_F, T, \|(F, G, g)\|_{Lip}$ only such that

$$ \forall t \in [0, T], \quad \|X_t - X_t^\alpha\|, \|U_t - U_t^\alpha\| \le C'' \|v(\alpha)\|_T. $$

□

**2.3. Decreasing step size.** So far, we have been working with a constant step size $\gamma$. This yields a convergence of the forward backward system in $(H^T)^d$ with a rate of $O(\frac{1}{\sqrt{n}})$. Although choosing a decreasing step size leads to a worse converge rate, it allows to weaken some assumption. In particular, the step size can then be chosen independently of the Lipschitz norm of $v$. This is particularly interesting for FBSDEs posed on large time intervals since we expect that $\|v\|_{Lip,H^T}$ grows linearly with the horizon $T$.

**Corollary 2.23.** Let $v$ be defined as in 2.18, for a given initial condition $(\alpha_1, Y_1) \in (H^T)^{2d}$ and $n \ge 1$ we introduce the sequences

$$ \left\{ \begin{array}{l} \alpha_{n+\frac{1}{2}} = \alpha_n - \gamma_n v(\alpha_n), \\ Y_{n+1} = Y_n - v(\alpha_{n+\frac{1}{2}}), \\ \alpha_{n+1} = \gamma_{n+1} Y_{n+1}, \end{array} \right. $$

If $(\gamma_n)_{n \in \mathbb{N}} \subset \mathbb{R}^+$ is a sequence of decreasing step such that there exists $C \in \mathbb{R}^+$, $\delta \in (0, 1)$ such that

$$ \forall n \in \mathbb{N}, \quad \frac{1}{\gamma_n} \le C_n n^\delta, $$

and

$$ \exists n_0, \quad \gamma_{n_0} \le \|v\|_{Lip,H^T}, $$

then under Hypotheses 2.14, 2.13, letting

$$ \bar{U}^n = \frac{1}{n} \sum_{i=1}^{n} F_u^{-1}(X^{\alpha_{i+\frac{1}{2}}}, \alpha_{i+\frac{1}{2}}), $$

and $(X_t, U_t, Z_t)_{t \in [0, T]}$ be the unique strong solution to (2.7), the following holds

- There exists a constant $C_v$ depending on the coefficients and $(\gamma_n)_{n \in \mathbb{N}}$ such that

$$ \forall n \ge 1, \quad \|U_n - \bar{U}^n\|_T^2 \le \frac{\|\tilde{F}(X_n, U_n) - \alpha_1\|_T^2}{2\gamma_1 c_F n} + \frac{C_\gamma}{2n^{1-\delta}} \|U_n\|_T^2 + \frac{C_v}{c_F n}. $$

- Letting

$$ \forall t \in [0, T], \quad \bar{X}_t^n = X_0 - \int_0^t \tilde{F}(\bar{X}_s^n, \bar{U}_s^n) ds + \sqrt{2\sigma} B_t, $$

there exists a constant $C$ depending only on $T$ and $\|\tilde{F}\|_{Lip}$ such that

$$ \|\bar{X}_t^n - X_t\|_T^2 \le C \left( \frac{\|\tilde{F}(X_t, U_t) - \alpha_1\|_T^2}{2\gamma_1 c_F n} + \frac{C_\gamma}{2n^{1-\delta}} \|U_t\|_T^2 + \frac{C_v}{c_F n} \right). $$
Proof. Applying Lemma 1.4, we find that

$$c_F n \|U\| - \frac{1}{n} \sum_{i=1}^{n} \bar{F}_u^1(X^{\alpha_{i+\frac{1}{2}}}, \alpha_{i+\frac{1}{2}}) \|T\|^2 \le \frac{\|\bar{F}(X, U) - \alpha_1\|_T^2}{2\gamma_1} + \frac{1}{2\gamma_{n+1}} \|U\|_T^2 \\ + \frac{1}{2} \sum_{i=1}^{n} \left( \gamma_i \|v(\alpha_{i+\frac{1}{2}}) - v(\alpha_i)\|_T^2 - \frac{1}{\gamma_i} \|\alpha_{i+\frac{1}{2}} - \alpha_i\|_T^2 \right).$$

Letting

$$C_v = \frac{1}{2} \sum_{i=1}^{n_0} \left( \gamma_i \|v(\alpha_{i+\frac{1}{2}}) - v(\alpha_i)\|_T^2 - \frac{1}{\gamma_i} \|\alpha_{i+\frac{1}{2}} - \alpha_i\|_T^2 \right)$$

It follows that

$$\|U\| - \frac{1}{n} \sum_{i=1}^{n} \bar{F}_u^1(X^{\alpha_{i+\frac{1}{2}}}, \alpha_{i+\frac{1}{2}}) \|T\|^2 \le \frac{\|\bar{F}(X, U) - \alpha_1\|_T^2}{2\gamma_1 c_f n} + \frac{C_\gamma}{2n^{1-\delta}} \|U\|_T^2 + \frac{C_v}{c_F n}.$$

The rest of the proof is a straightforward consequence of this result

□

**Remark 2.24.** the condition

$$\exists n_0, \quad \gamma_{n_0} \le \frac{1}{\|v\|_{Lip,H^T}},$$

is always true as soon as the sequence tends to 0. If the operator $v$ is only assumed to be $\eta$ Holder for some $\eta \in (0, 1)$, then this result can also be adapted by requiring that

$$\sum_{n=1}^{+\infty} \gamma_n^{\frac{1+\eta}{1-\eta}} < +\infty,$$

instead. By an application of Young's inequality

$$\sum_{i=1}^{n} \gamma_i \|\alpha_{i+\frac{1}{2}} - \alpha_i\|^{2\eta} \le \frac{1}{1-\eta} \sum_{i=1}^{n} \gamma_i^{\frac{1+\eta}{1-\eta}} + \frac{1}{\eta} \sum_{i=1}^{n} \frac{1}{\gamma_i} \|\alpha_{i+\frac{1}{2}} - \alpha_i\|^2.$$

For monotone FBSDEs with Holder coefficients we refer to [39]

**2.4. Outside of the mean field regime.** Let us mention that this method is still valid outside of the mean field regime, namely this allows to compute the characteristics of system of PDEs of the form

$$(2.14) \qquad \partial_t U + F(x, U) \cdot \nabla_x U = G(x, U),$$

in the monotone regime. For mean field games, this corresponds to the study of finite state space MFGs [36]. Computing solutions to this system can be challenging as the regularity of this PDE stems from its monotonicity. In general this means that either the interval of time considered must be sufficiently small, or the approximation used must be monotonicity preserving. Finally even in the so called potential regime, that is, when (2.14) arises from the gradient of a convex HJB equation [45]

$$\partial_t u + H(x, \nabla_x u) = 0,$$

the method we propose appears new. Although it can only be applied to the framework of convex HJB equation compared to more general method such as shooting algorithms [15], it exhibits global convergence independently of the chosen initial control.

Another point which is relevant in the finite dimensional setting is that $v$ can be monotone without requiring that all coefficients are. Consider the following HJB equation for simplicity

$$(2.15) \qquad \left\{ \begin{array}{ll} \partial_t u + H(x, \nabla_x u) - \sigma \Delta_x u = 0, & (t, x) \in (0, T) \times \mathbb{R}^d, \\ u(T, x) = g(x) & \forall x \in \mathbb{R}^d. \end{array} \right.$$

We place ourselves under the following assumptions

**Hypothesis 2.25.** Let $L$ be the fenchel conjugate of $H$ in (2.15),

- there exists two constant $c_\alpha > 0, c_x \ge 0$ such that

$$\forall (x, \alpha) \in \mathbb{R}^{2d} \quad \begin{pmatrix} D_{x,x}^2 L & D_{x,\alpha}^2 L \\ D_{\alpha,x}^2 & D_{\alpha}^2 L \end{pmatrix} (x, \alpha) \ge \begin{pmatrix} -c_x I_d & 0 \\ 0 & c_\alpha I_d \end{pmatrix},$$

in the sense of distributions.
- there exists $c_g \ge 0$ such that

$$ \forall (x, y) \in \mathbb{R}^{2d}, \quad (g(x) - g(y)) \cdot (x - y) \ge -c_g |x - y|^2. $$

- $\nabla_x L, \nabla_\alpha L, \nabla_x g$ are lipschitz.

- $\sigma > 0$.

Those assumptions, especially on $\nabla_\alpha L$ are much stronger than what is needed for the wellposedness (in a classical sense) of (2.15), nevertheless they are quite standard in optimal control and it is a convenient setting to state the following:

**Lemma 2.26.** Under Hypothesis 2.25, there exists a unique solution $u \in C([0, T] \times \mathbb{R}^d) \cup C^{1,2}((0, T) \times \mathbb{R}^d)$ to (2.15). Moreover $x \mapsto \nabla_x u(t, x)$ is Lipschitz uniformly in $t \in [0, T]$. In particular for any initial condition $X \in H$ there exists a unique solution to the forward backward system

$$ (2.16) \qquad \left\{ \begin{array}{ll} dX_t = -D_p H(X_t, U_t)dt + \sqrt{2\sigma}dB_t, & X_0 = X, \\ dU_t = D_x H(X_t, U_t) - Z_t dB_t, & U_T = \nabla_x g(X_T), \end{array} \right. $$

whose decoupling field is given by $(t, x) \mapsto \nabla_x u(t, x)$.

*Proof.* In this setting the uniqueness of a solution to (2.15) follows from the theory of viscosity solutions [25]. The existence is also quite standard. For example, it suffices to use any short time existence result [14] combined with schauder type estimates [33]. The fact that $\nabla_x u$ is the decoupling field associated to the FBSDE (2.16) is then simply a consequence of Ito's lemma. $\square$

Since the existence of a solution to (2.16) for any initial condition has been established we now show the convergence of the algorithm introduced in this section

**Lemma 2.27.** Fix an initial condition $X_0 \in H^d$ and let $v$ be defined as in 2.8 for $L, g$. Under Hypothesis 2.25, if

$$ c_v = c_\alpha - T(c_x + c_g) > 0, $$

and

$$ \gamma \le \min \left( \frac{1}{2\|v\|_{Lip}}, \frac{c_v}{\|v\|_{Lip}^2} \right), $$

then there exists a $\lambda \in (0, 1)$ depending only on $c_v, \gamma, \|v\|_{Lip}$ such that for any $\alpha_1 \in (H^T)^d$ letting

$$ \left\{ \begin{array}{l} \alpha_{n+\frac{1}{2}} = \alpha_n - \gamma v(\alpha_n), \\ \alpha_{n+1} = \alpha_n - \gamma v(\alpha_{n+\frac{1}{2}}). \end{array} \right. $$

the following holds

$$ \forall n \in \mathbb{N}, \quad n \ge 1, \quad \|\alpha_n - \alpha^*\| \le \lambda^n \|\alpha_1 - \alpha^*\| $$

for $\alpha^* = D_p H(X_t, U_t)$ the optimal control associated to (2.16).

*Proof.* Since coefficients are all Lipschitz, it is evident that $v$ is Lipschitz. Now let us take two controls $\alpha, \alpha'$, using Hypothesis 2.25, we get

$$ \langle v(\alpha) - v(\alpha'), \alpha - \alpha' \rangle^T \ge c_\alpha \|\alpha - \alpha'\|_T^2 - c_x \|X^\alpha - X^{\alpha'}\|_T^2 - c_g \|X_T^\alpha - X_{T'}^{\alpha'}\|_T^2. $$

By definition

$$ \forall t \in [0, T], \quad \|X_t^\alpha - X_t^{\alpha'}\|^2 \le \|\alpha - \alpha'\|_T^2. $$

As such

$$ \langle v(\alpha) - v(\alpha'), \alpha - \alpha' \rangle^T \ge c_v \|\alpha - \alpha'\|_T^2. $$

Since this is true for any controls $\alpha, \alpha', v$ is Lipschitz continuous and strongly monotone, and we are in the conditions of applications of Theorem (2.11). $\square$

Let us remark that even in mean field games, $v$ may be monotone even when coefficients aren't. Indeed the above result is adapted easily to the mean field setting on short intervals of time. So long as the monotonicity of $v$ is conserved, the existence and uniqueness of a solution are not particularly hard to prove, this phenomenon is sometimes refered to as the semi-monotone regime in short time [42].
### 3. FBSDEs WITH COMMON NOISE

We now turn to the problem of FBSDEs with common noise

$$ (3.1) \quad \begin{cases} X_t = X_0 - \int_0^t F(X_s, p_s, U_s, \mathcal{L}(X_s, U_s|\mathcal{F}_s^0))ds + \sqrt{2\sigma}dB_t, \\ U_t = g(X_T, p_T, \mathcal{L}(X_T|\mathcal{F}_T^0)) + \int_t^T G(X_s, p_s, U_s, \mathcal{L}(X_sU_s|\mathcal{F}_s^0))ds - \int_t^T Z_s d(B_s, W_s), \\ p_t = p_0 - \int_0^t b(p_s)ds + \sqrt{2\sigma^0}dW_s, \end{cases} $$

for $X_0 \in H^d$, $q_0 \in H^{d^0}$ initial conditions independent of each other and of $(W_s, B_s)_{s \ge 0}$. In this case the filtration associated to common noise is $(\mathcal{F}_t^0)_{t \in [0,T]}$ which is the augmented filtration associated to $(\tilde{\mathcal{F}}_t^0)_{t \in [0,T]}$ defined by

$$ \tilde{\mathcal{F}}_t^0 = \sigma(q_0, (W_s)_{s \le t}). $$

**Remark 3.1.** We insist on the fact that this class of FBSDEs includes the case of additive common noise often studied in mean field games. Indeed, consider the following problem

$$ (3.2) \quad \begin{cases} X_t = X_0 - \int_0^t F(X_s, U_s, \mathcal{L}(X_s, U_s|\mathcal{F}_s^0))ds + \sqrt{2\sigma}dB_t + \sqrt{2\sigma^0}dW_t, \\ U_t = g(X_T, \mathcal{L}(X_T|\mathcal{F}_T^0)) + \int_t^T G(X_s, U_s, \mathcal{L}(X_s, U_s|\mathcal{F}_s^0))ds - \int_t^T Z_s d(B_s, W_s), \\ \mathcal{F}_t^0 = \sigma((W_s)_{s \le t}). \end{cases} $$

Introducing the two processes

$$ \forall t \in [0, T], \quad \begin{cases} p_t = \sqrt{2\sigma^0}W_t, \\ Y_t = X_t - p_t, \end{cases} $$

Formally, the triple $(Y_s, p_s, Z_s)_{s \in [0,T]}$ is a solution to

$$ \left\{ \begin{array}{l} Y_t = X_0 - \int_0^t F(Y_s + p_s, U_s, T(\cdot, p_s)\#\mathcal{L}(Y_s, U_s|\mathcal{F}_s^0))ds + \sqrt{2\sigma}dB_t, \\ p_t = \sqrt{2\sigma^0}W_t, \\ U_t = g(Y_T + p_T, (id_{\mathbb{R}^d} + p_T)\#\mathcal{L}(X_T|\mathcal{F}_T^0)) + \int_t^T G(Y_s + p_s, U_s, T(\cdot, p_s)\#\mathcal{L}(Y_s, U_s|\mathcal{F}_s^0))ds - \int_t^T Z_s d(B_s, W_s), \\ \mathcal{F}_t^0 = \sigma((p_s)_{s \le t}), \end{array} \right. $$

where $f_{\#\mu}$ indicates the pushforward of a measure $\mu$ by the function $f$ and

$$ (id_{\mathbb{R}^d} + p) : x \mapsto x + p, \quad T(\cdot, p) : (x, y) \mapsto (x + p, y). $$

In particular if $(F, G, g)$ are Lipschitz functions on $\mathbb{R}^{2d} \times \mathcal{P}_2(\mathbb{R}^{2d})$ then the coefficients are still Lipschitz in all arguments after this transformation. A rigorous equivalence between these two formulations is established in [12] Lemma 5.3.

We work under the following assumption

**Hypothesis 3.2.** - Hypothesis 2.14 is satisfied by

$$ (x, u, m) \mapsto (g(x, p, \pi_d m), F(x, p, u, m), G(x, p, u, m),) $$

uniformly in $p \in \mathbb{R}^{d^0}$.

- $(F, G, g, b)$ are Lipschitz in all their arguments for $W_2$ in the measure argument.

It has already been observed in [11], that the addition of such common noise does not perturb the monotonicity of the system. Hence results of wellposedness for monotone FBSDEs are easily adapted to this particular system where the common noise $(p_t)_{t \ge 0}$ evolves independently of the rest of the dynamics

**Lemma 3.3.** Under Hypothesis 3.2, for any initial condition $X_0 \in H^d$, $q_0 \in H^{d^0}$, independent of each other and of $(W_t, B_t)_{t \ge 0}$, there exists a unique strong solution to the FBSDE (3.1)

*Proof.* For a proof of this result based on the existence of a Lipschitz decoupling field to the FBSDE (3.1), we refer to [39] Theorem 3.33 and the subsection 3.4.1 of said work. □
Let us remark that under the standing assumptions, the function

$$U \mapsto F(X, p, U, \mathcal{L}(X, U)),$$

is invertible on $H^d$ for any $(X, p) \in H^d \times \mathbb{R}^{d^0}$ by the monotonicity of $F$, with this Hilbertian inverse being denoted by $F_u^{-1}(X, p, \cdot)$. In particular following [39] Lemma 3.10 there exists a Lipschitz function $F_{inv}: \mathbb{R}^d \times \mathbb{R}^{d^0} \times \mathbb{R}^d \times \mathcal{P}_2(\mathbb{R}^{2d}) \to \mathbb{R}^d$ such that

$$\forall (X, \alpha, p) \in H^{2d} \times \mathbb{R}^{d^0}, \quad F_{inv}(X, p, \alpha, \mathcal{L}(X, \alpha)) = \tilde{F}_u^{-1}(X, p, \alpha, \mathcal{L}(X, \alpha))$$

Following the previous section, for a given $(\alpha_t)_{t \in [0,T]} \in (H^T)^d$, we introduce the dynamics

$$(3.3) \qquad \left\{ \begin{array}{l} X_t^\alpha = X_0 - \int_0^t \alpha_s ds + \sqrt{2\sigma} dB_s, \\ p_t = p_0 - \int_0^t b(p_s) ds + \sqrt{2\sigma^0} dW_s, \\ U_t^\alpha = g(X_T^\alpha, p_T, \mathcal{L}(X_T^\alpha | \mathcal{F}_T^0)) - \int_t^T G_F(X_s^\alpha, p_s, \alpha_s, \mathcal{L}(X_s^\alpha, \alpha_s | \mathcal{F}_s^0)) ds - Z_s d(B_s, W_s), \end{array} \right.$$

where

$$\tilde{G}_F(X_s^\alpha, p_s, \alpha_s, \mathcal{L}(X_s^\alpha, \alpha_s | \mathcal{F}_s^0)) = G(X_s^\alpha, p_s, F_{inv}(X_s^\alpha, p_s, \alpha_s, \mathcal{L}(X_s^\alpha, \alpha_s | \mathcal{F}_s^0)), \mathcal{L}(X_s^\alpha, F_{inv}(X_s^\alpha, p_s, \alpha_s, \mathcal{L}(X_s^\alpha, \alpha_s | \mathcal{F}_s^0)) | \mathcal{F}_s^0)).$$

**Corollary 3.4.** Under Hypothesis 3.2, for any $\alpha \in (H^T)^d$, $X_0 \in H$ the system (3.3) has a unique solution $(X_t^\alpha, U_t^\alpha, Z_t^\alpha)_{t \in [0,T]}$. Moreover, let

$$(3.4) \qquad v: \left\{ \begin{array}{l} (H^T)^d \to (H^T)^d \\ (\alpha_t)_{t \in [0,T]} \mapsto (F_u^{-1}(X_t^\alpha, \alpha_t) - U_t^\alpha)_{t \in [0,T]} \end{array} \right. .$$

$v$ is Lipschitz on $(H^T)^d$ and there exists a constant $C$ such that if

$$\gamma \le \frac{1}{\|v\|_{Lip, H^T}},$$

for any $\alpha_0 \in (H^T)^d$, the procedure

$$\left\{ \begin{array}{l} \alpha_{n+\frac{1}{2}} = \alpha_n - \gamma v(\alpha_n), \\ \alpha_{n+1} = \alpha_n - \gamma v(\alpha_{n+\frac{1}{2}}), \\ \bar{U}^n = \frac{1}{n} \sum_{i=1}^n F_u^{-1}(X^{\alpha_{i+\frac{1}{2}}}, \alpha_{i+\frac{1}{2}}) \end{array} \right.$$

satisfies

$$\forall n \ge 0, \quad C \|U - \bar{U}^n\|_T^2 \le \frac{\|\tilde{F}(X, U, \cdot) - \alpha_0\|_T^2}{\gamma n}.$$

*Proof.* This is almost a direct adaptation of Theorem 2.19. The core of the proof remaining unchanged in the presence of an additional common noise. In particular it is straightforward to show that $v$ is still Lipschitz in this setting. Instead let us focus on showing that the monotonicity of $v$ is unchanged in this new problem. Let $\alpha, \alpha' \in H^T$, by definition of the forward backward system (3.1)

$$\langle v(\alpha) - v(\alpha'), \alpha - \alpha' \rangle^T = \langle g(X_T^\alpha, p_T, \mathcal{L}(X_T^\alpha | \mathcal{F}_T^0)) - g(X_T^{\alpha'}, p_T, \mathcal{L}(X_T^{\alpha'} | \mathcal{F}_T^0)), X_T^\alpha - X^{(\alpha')}_T \rangle \\ + \mathbb{E} \left[ \int_0^T \left( F_{inv}(X_s^\alpha, p_s, \alpha_s, \mathcal{L}(X_s^\alpha, \alpha_s | \mathcal{F}_s^0)) - F_{inv}(X_s^{\alpha'}, p_s, \alpha_s', \mathcal{L}(X_s^{\alpha'}, \alpha_s' | \mathcal{F}_s^0)) \right) \cdot (\alpha_s - \alpha_s') ds \right] \\ + \mathbb{E} \left[ \int_0^T \left( \tilde{G}_F(X_s^\alpha, p_s, \alpha_s, \mathcal{L}(X_s^\alpha, \alpha_s | \mathcal{F}_s^0)) - \tilde{G}_F(X_s^{\alpha'}, p_s, \alpha_s', \mathcal{L}(X_s^{\alpha'}, \alpha_s' | \mathcal{F}_s^0)) \right) \cdot (X_s^\alpha - X_s^{\alpha'}) ds \right].$$

For fixed $p \in \mathbb{R}^{d^0}$, the monotonicity of the pair

$$(X, \alpha) \mapsto (F_{inv}(X, \alpha, p, \mathcal{L}(X, \alpha)), \tilde{G}_F(X, \alpha, p, \mathcal{L}(X, \alpha))),$$
has been established in Lemma 2.16. Thus, using an equivalent definition of $L^2$-monotonicity via the conditional law $(\mathcal{L}(X_s^\alpha, \alpha_s | \mathcal{F}_s^0))_{s \in [0,T]}$ (to avoid having to deal with conditional random variables) we deduce that

$$ \mathbb{E} \left[ \int_0^T \left( \begin{array}{c} F_{inv}(X_s^\alpha, p_s, \alpha_s, \mathcal{L}(X_s^\alpha, \alpha_s | \mathcal{F}_s^0)) - F_{inv}(X_s^{\alpha'}, p_s, \alpha'_s, \mathcal{L}(X_s^{\alpha'}, \alpha'_s | \mathcal{F}_s^0)) \\ \tilde{G}_F(X_s^\alpha, p_s, \alpha_s, \mathcal{L}(X_s^\alpha, \alpha_s | \mathcal{F}_s^0)) - \tilde{G}_F(X_s^{\alpha'}, p_s, \alpha'_s, \mathcal{L}(X_s^{\alpha'}, \alpha'_s | \mathcal{F}_s^0)) \end{array} \right) \cdot \begin{pmatrix} \alpha_s - \alpha'_s \\ X_s^\alpha - X_s^{\alpha'} \end{pmatrix} ds \middle| F_T^0 \right] \\ \geq c_F \mathbb{E} \left[ \int_0^T \left| F_{inv}(X_s^\alpha, p_s, \alpha_s, \mathcal{L}(X_s^\alpha, \alpha_s | \mathcal{F}_s^0)) - F_{inv}(X_s^{\alpha'}, p_s, \alpha'_s, \mathcal{L}(X_s^{\alpha'}, \alpha'_s | \mathcal{F}_s^0)) \right|^2 \middle| \mathcal{F}_T^0 \right] \quad \text{a.s.} $$

The rest of the proof follows from arguments presented in the proof of Theorem 2.19 by using the above monotonicity estimate. □

**Remark 3.5.** The adaptation of previously obtained convergence results to the case of FBSDEs with common noise is deceptively simple. Although the proof of convergence is quite similar, numerically this addition leads to significant changes in the algorithm considered.

**3.1. numerical setup.** In this section we present some numerical results for the method introduced in this article. We first discretize the time interval considered $[0, T]$ with a time step $\Delta_t$ and work with discretized control $(\alpha_i)_{i \in \{1, \dots, N_t\}} = (\alpha(i\Delta_t))_{i \in \{1, \dots, N_t\}}$. The different processes are then computed with an explicit Euler scheme. Let us present the scheme more precisely in the case study of FBSDEs (2.7)

$$ \left\{ \begin{array}{l} X_t = X_0 - \int_0^t F(X_s, U_s, \mathcal{L}(X_s, U_s))ds + \sqrt{2\sigma}dB_t, \\ U_t = g(X_T, \mathcal{L}(X_T)) + \int_t^T G(X_s, U_s, \mathcal{L}(X_s, U_s))ds - \int_t^T Z_sdB_s. \end{array} \right. $$

We suppose that the map $F^{-1} : \mathbb{R}^{2d} \times \mathcal{P}_2(\mathbb{R}^{2d}) \to \mathbb{R}^d$ which is such that $F^{-1} = \tilde{F}^{-1}$ is known. The existence of such a map follows from Remark 2.6, however as we already mentioned in Remark 2.20, computing it can be a challenge whenever $F$ depends on the law of $(U_t)_{t \in [0,T]}$. (although this is never a problem for mean field games, since in this case $F^{-1} = \nabla_\alpha L$).

First we fix $N_p$ the number of paths simulated and introduce $(X_0^i)_{i=1:N_p} \subset \mathbb{R}^{N_p}$, where each $X_0^i$ is an independent realization of a random variable with law $\mathcal{L}(X_0)$. We also introduce $(G^{i,j})_{i=1:N_p, j=1:N_t}$ a sequence of independent gaussian increments sampled from a normal distribution with mean 0 and covariance $\sqrt{\Delta_t}I_d$. These will play the role of the Brownian increments

$$ W_{(i+1)\Delta_t} - W_{i\Delta_t}. $$

A discrete control $(\alpha^{i,j})_{i=1:N_p, j=1:N_t}$ gives the control at the timestep $j$ along the path $i$. We define $X^{i,0} = X_0^i$ and for $j = 1:N_p$

$$ X^{i,j} = X^{i,j-1} + \Delta_t \alpha^{i,j} + \sqrt{2\sigma} G^{i,j}. $$

To approximate the backward process $(U_t^\alpha)_{t \in [0,T]}$, we first define

$$ U^{i,N_t} = g(X^{i,N_t}, \frac{1}{N_p} \sum_k \delta_{X^{k,N_t}}), $$

where $\delta_x$ indicates the dirac measure centered in $x \in \mathbb{R}^d$. Then the sequence is constructed by backward induction for $j = 0:(N_t - 1)$,

$$ U^{i,j} = \mathbb{E} \left[ U^{i,j+1} + \Delta_t G(X^{i,j+1}, F^{-1}(X^{i,j+1}, \alpha^{i,j+1}), \frac{1}{N_p} \sum_k \delta_{(X^{k,j+1}, \alpha^{k,j+1})}), \frac{1}{N_p} \sum_k \delta_{(X^{k,j+1}, \alpha^{k,j+1})}) \middle| \mathcal{F}_j \right], $$

where $\mathcal{F}_j = \sigma((X_0^i, G^{i,k}, \alpha^{i,k})_{i=1:N_p, k \le j})$. The conditional expectation is approximated as follows, letting $(f_\omega)_{\omega \in \mathcal{X}}$ be a parametrized family of functions on some space $\mathcal{X}$, for fixed $j$ we approximate $(U^{i,j})_{i=1:N_p}$ with

$$ \forall i \in \{1, \dots, N_p\}, \quad U^{i,j} \approx f_{\omega^*}(X^{i,j}), $$

where

$$ \omega^* = \operatorname{arginf}_{\omega \in \mathcal{X}} J(\omega), $$
for

$$J(\omega) = \frac{1}{N_p} \sum_k \left| U^{k,j+1} + \Delta_t G(X^{k,j+1}, F^{-1}(X^{k,j+1}, \alpha^{k,j+1}, \frac{1}{N_p} \sum_l \delta_{(X^{l,j+1}, \alpha^{l,j+1})}), \frac{1}{N_p} \sum_l \delta_{(X^{l,j+1}, \alpha^{l,j+1})}) - f_{\omega}(X^{k,j}) \right|^2$$

Clearly this is based on the idea that for two random variables $X, Y \in H$,

$$\mathbb{E}[Y|X] = \inf_{\text{measurable } f} \mathbb{E}[|Y - f(X)|^2].$$

The parametrized family $(f_{\omega})_{\omega \in \mathcal{X}}$ can range from neural networks to any sufficiently rich regression method, for general results on regression based methods for BSDEs we refer to [28]. Now that we have explained how the trajectories are calculated we introduce the following discretized operator

$$v^{N_t, N_p}((X_0^i, G^{i,j})_{i=1:N_p, j=1:N_t}) : \left\{ \begin{array}{ll} \mathbb{R}^{N_p \times N_t \times d} \longrightarrow \mathbb{R}^{N_p \times N_t \times d} & \\ (\alpha^{i,j})_{i=1:N_p, j=1:N_t} \longmapsto (F^{-1}(X^{i,j}, \alpha^{i,j}, \frac{1}{N_p} \sum_k \delta_{(X^{k,j}, \alpha^{k,j})}) - U^{i,j})_{i=1:N_p, j=1:N_t} & . \end{array} \right.$$

We can now introduce the associated algorithm to solve the FBSDE (2.7)

**Algorithm 1 Monotone FBSDE solver**

fix an error level $\varepsilon_0$.

fix a decreasing sequence of step $(\gamma_n)_{n \in \mathbb{N}} \subset \mathbb{R}^+$.

$N_{\text{iter}} \leftarrow 0$

Simulate $N_p$ independent realizations $(X_0^i)_{i=1:N_p}$ of $X_0$.

Simulate $N_p \times N_t$ realizations $(G^{i,j})_{i=1:N_p, j=1:N_t}$ from independent gaussian random variable $G^{i,j} \sim \mathcal{N}(0, \sqrt{\Delta_t}I_d)$.

Choose an initial control $(\alpha^{i,j})_{i=1:N_p, j=1:N_t}$ progressively adapted.

set $\bar{\alpha} = \alpha$.

$$\varepsilon_{\text{error}} = \|v^{N_t, N_t}((X_0^i, G^{i,j})_{i=1:N_p, j=1:N_t})(\bar{\alpha})\|_T$$

$$\text{Initialize } Y = \frac{1}{\gamma_1} \alpha$$

while $\varepsilon_{\text{error}} > \varepsilon_0$ do

$$N_{\text{iter}} \leftarrow N_{\text{iter}} + 1$$

$$\alpha \leftarrow \alpha - \gamma_{N_{\text{iter}}} v^{N_t, N_p}((X_0^i, G^{i,j})_{i=1:N_p, j=1:N_t})(\alpha)$$

$$\bar{\alpha} \leftarrow \left(1 - \frac{1}{N_{\text{iter}} + 1}\right) \bar{\alpha} + \frac{1}{N_{\text{iter}} + 1} \alpha$$

$$Y \leftarrow Y - v^{N_t, N_p}((X_0^i, G^{i,j})_{i=1:N_p, j=1:N_t})(\alpha)$$

$$\alpha \leftarrow \gamma_{N_{\text{iter}}} Y$$

$$\varepsilon_{\text{error}} \leftarrow \|v^{N_t, N_t}((X_0^i, G^{i,j})_{i=1:N_p, j=1:N_t})(\bar{\alpha})\|_T$$

end while

Return $\bar{\alpha}$.

Where we used the notation

$$\|\alpha\|_T = \sqrt{\frac{\Delta_t}{N_p} \sum_{i,j} \|\alpha^{i,j}\|^2},$$

which is the corresponding $L^2$-norm for discretized random processes. In general the true error is difficult to compute, however thanks to Lemma 2.22, it can be estimated up to a multiplicative constant by $\|v(\alpha)\|_T$. In particular this allows us to estimate the convergence rate of the algorithm efficiently.

3.1.1. FBSDEs with common noise. Let us briefly explain the main differences in this setting. Since we must be able to compute an approximation of the conditional law at each backward step, this implies that the generation of the discrete idiosyncratic noise $(G^{i,j})_{i=1:N_p, j=1:N_t}$ and the discrete common noise $(G_0^{k,j})_{k=1:N_0, j=1:N_t}$ must be done independently (where we choose to simulate $N_0$ trajectories of the common noise). A discrete control is now also a collection $(\alpha^{i,j,k})_{i=1:N_p, j=1:N_t, k=1:N_0}$ which depends on the trajectories of common noise. We now present an algorithm in pseudocode to compute the function

$$v^{N_t, N_p, N_0} : \mathbb{R}^{N_t} \times \mathbb{R}^{N_p} \times \mathbb{R}^{N_0} \rightarrow \mathbb{R}^{N_t} \times \mathbb{R}^{N_p} \times \mathbb{R}^{N_0},$$

discretization of the associated functional (3.4)
<caption>**Algorithm 2** Computation of $v^{N_t, N_p, N_0}$</caption>

INPUT: $(\alpha^{i,j,k})_{i=1:N_p, j=1:N_t, k=1:N_0}$

-Compute the associated forward processes $(X^{i,j,k}, p^{j,k})_{i=1:N_p, j=1:N_t, k=1:N_0}$ iteratively through

$$ \begin{cases} X^{i,j+1,k} = X^{i,j,k} + \Delta_t \alpha^{i,j,k} + \sqrt{2\sigma} G^{i,j}, \\ p^{j+1,k} = p^{j,k} + \Delta_t b(p^{j,k}) + \sqrt{2\sigma^0} G_0^{k,j}. \end{cases} $$

-The backward process $(U^{i,j,k})_{i=1:N_p, j=1:N_t, k=1:N_0}$ is computed recursively as follows: given $(U^{i,j+1,k})_{i=1:N_p, j=1:N_t}$, $(U^{i,j,k})_{i=1:N_p, j=1:N_t}$ is computed by regression of

$$ U^{i,j+1,k} + \Delta_t G_F(X^{i,j+1,k}, p^{j+1,k}, \alpha^{i,j+1,k}, \frac{1}{N_p} \sum_l \delta_{(X^{l,j+1,k}, \alpha^{l,j+1,k})}) $$

into $(X^{i,j,k}, p^{j,k})_{i=1:N_p, k=1:N_0}$

-Return $(F_{inv}(X^{i,j,k}, p^{j,k}, \alpha^{i,j,k}, \frac{1}{N_p} \sum_l \delta_{(X^{l,j,k}, \alpha^{l,j,k})}) - U^{i,j,k})_{i=1:N_p, j=1:N_t, k=1:N_0}$.

Clearly this algorithm is much more costly compared to the version without common noise. First the memory complexity is now of the order of $O(N_t \times N_p \times N_0)$. Moreover at each backward time step, we compute the regression of a function in $\mathbb{R}^d \times \mathbb{R}^{d^0}$

## 3.2. Mean field forward backward stochastic differential equations.
Compared to the deterministic case, the most time consuming step consists in approximating the backward process. Although the method we propose allows to iterate on decoupled FBSDEs, we still need to estimate a conditional expectation at each time step. In the following test, it is estimated with polynomial regression on the first 10 Hermite polynomials.

### 3.2.1. System considered.
We consider the following example

$$ (3.5) \qquad \left\{ \begin{array}{l} X_t = X_0 - \int_0^t aY_s ds + \sqrt{2\sigma} dB_s, \\ Y_t = bX_T + \int_t^T (cX_s + \mathbb{E}[f(X_s - \mathbb{E}[X_s])]) ds - Z_s dB_s, \end{array} \right. $$

In this case the solution is known explicitely. Indeed Let us first assume that $(Y_s)_{s \in [0,T]}$ can be written in feedback form as

$$ (3.6) \qquad \forall t \in [0, T], \quad Y_t = \eta(t)X_t + \theta(t), $$

for smooth functions of time $\eta, \theta$. It follows directly that

$$ \bar{X}_t = X_t - \mathbb{E}[X_t], $$

is given by

$$ \bar{X}_t = \bar{X}_0 e^{-a \int_0^t \eta(s) ds} + \sqrt{2\sigma^x} \int_0^t e^{-\int_s^t a\eta(u) du} dW_s. $$

In particular, there exists a continuous function $e: [0, T] \to \mathbb{R}$ depending on $\eta, a, \mathcal{L}(X_0)$ only such that

$$ \forall t \in [0, T], \quad \mathbb{E}[f(X_t - \mathbb{E}[X_t])] = \mathbb{E}[f(\bar{X}_t)] = e(t). $$

Using the representation formula (3.6), we deduce that

$$ \forall t \in [0, T], \quad 0 = \int_t^T \left( \left(c + \frac{d\eta}{ds}(s) - a\eta^2(s)\right) X_s + e(s) - a\eta(s)\theta(s) + \frac{d\theta}{ds}(s) \right) ds + \int_t^T (\sqrt{2\sigma\eta}(s) - Z_s) dW_s. $$

In the end it suffices to solve the following system

$$ \left\{ \begin{array}{l} c + \frac{d\eta}{ds}(s) - a\eta^2(s) = 0, \quad \eta(T) = b, \\ e(s) - a\eta(s)\theta(s) + \frac{d\theta}{ds}(s) = 0, \quad \theta(T) = 0. \end{array} \right. $$

In particular if $a, b, c \ge 0$ and $f$ is either monotone or Lipschitz with $\|f\|_{Lip} \le c$, we are in the monotone regime and the system admits a unique solution on any interval of time. Since monotonicity
also implies uniqueness of a solution to (3.5), we have exhibited the unique solution to this mean field FBSDE. For $c = 0$

$$ egin{cases} 
\eta(t) = \frac{b}{1 + ab(T - t)}, \\
\bar{X}_t = \bar{X}_0 \left( 1 - \frac{abt}{1 + abT} \right) + \sqrt{2\sigma}(1 + ab(T - t)) \int_0^t \frac{1}{1 + ab(T - s)} dW_s, \\
\theta(t) = \frac{1}{1 + ab(T - t)} \int_{T-t}^T e(s)(1 + ab(T - s))ds \\
e(t) = \mathbb{E}[f(\bar{X}_t)].
\end{cases} $$

Choosing $X_0 = x \in \mathbb{R}$,

$$ \bar{X}_t \sim \mathcal{N} \left( 0, \frac{2\sigma}{ab} \left( 1 + ab(T-t) - \frac{(1+ab(T-t))^2}{1+abT} \right) \right), $$

and $e(t)$ can be computed very accurately with gaussian quadrature.

### 3.2.2. Numerical results.
We now present some numerical result for the system (3.5) with the parameters $\sigma = a = b = x_0 = 1$, $c = 0$, $T = 10$ and

$$ \forall x \in \mathbb{R}, \quad f(x) = atan(x - 1), $$

for

$$ \gamma = 0.08. $$

Following the algorithm 1 with $N_t = 100$, $N_p = 10000$ and letting

$$ \varepsilon_{error}(n) = \|v^{N_t,N_p}(\alpha_n)\|_T, $$

the sequence of errors on the last iterate, we plot its logarithm in function of the number of iterations

<figure><img src="image_3.png" /><figcaption>FIGURE 1. log-linear plot of the last iterate error in function of the number of iterations</figcaption></figure>

After a small number of iterations, the linear fit is almost perfect. This emphasizes that the last iterates converge exponentially fast when the conditions for applying Theorem 2.11 are met.

Let us emphasize that the number of iterations may appear large only because we considered a time interval of $[0, 10]$ and even then, computations are quite fast. Since the functions $\theta, \eta$ can be computed very accurately in this setting let us also compare our numerical results to the solution of (3.5). For the same parameters, the functions $\eta, \theta$ are estimated numerically by doing a linear regression of $(Y^{i,j})_{j=1:N_p}$ onto $(X^{i,j})_{j=1:N_p}$ for fixed $1 \in \{1, \dots, N_t\}$.
<figure><img src="image_4.png" /><figcaption>FIGURE 3. $\theta(t)$</figcaption></figure>

<figure><img src="image_5.png" /></figure>

Plot of the functions $t \mapsto \theta(t)$, $t \mapsto \eta(t)$ in blue and of the computed values using our algorithm with the red crosses. Since $\varepsilon_{error}$ is very small ($\sim 10^{-13}$) the difference between the true curve and the computed values does not come from the number of iterations but is rather inherent to the discretization (both in time and on the space of random paths) of the problem.

## 4. PERSPECTIVES

In this article we have presented a reinterpretation of monotone FBSDEs as the solution of a monotone variational inequality in a Hilbert space. This point of view allows for a straightforward adaptation of extra-gradient methods to mean field type FBSDEs. Under sufficiently strong assumptions we present a decoupling algorithm converging exponentially fast to the solution of the problem. However in general the convergence rate is of the order of $\frac{1}{\sqrt{n}}$ for $n$ the number of iterations. A challenge for future research is to improve this convergence rate. To that end an interesting lead consists in studying a version of the algorithm with an adaptive step size. For example, although the context is very different it is shown in [35] that the convergence rate of fictitious play [18] can be tremendously improved by adding a linear search for the best step size at each step. Another possibility is to find better parameterizations of the problem. In [26] the authors proved that better convergence rates can be obtained by studying FBSDEs under different probability measures using Girsanov Theorem. Perhaps such an idea can also be applied in our setting.

## ACKNOWLEDGEMENTS

I would like to thank François Delarue for the discussions we had while working on this paper,

Charles Meynard acknowledge the financial support of the European Research Council (ERC) under the European Union's Horizon Europe research and innovation program (ELISA project, Grant agreement No. 101054746). Views and opinions expressed are however those of the author only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them

## REFERENCES

[1] Yves Achdou, Fabio Camilli, and Italo Capuzzo-Dolcetta. Mean field games: Convergence of a finite difference method. *SIAM Journal on Numerical Analysis*, 51(5):2585-2612, 2013.

[2] Yves Achdou and Ziad Kobeissi. Mean field games of controls: Finite difference approximations. *Mathematics in Engineering*, 3(3):1-35, 2021.

[3] Yves Achdou and Alessio Porretta. Convergence of a finite difference scheme to weak solutions of the system of partial differential equations arising in mean field games. *SIAM Journal on Numerical Analysis*, 54(1):161-186, 2016.

[4] Andrea Angiuli, Christy V. Graves, Houzhi Li, Jean-François Chassagneux, Franccois Delarue, and René A. Carmona. Cemrcs 2017: numerical probabilistic approach to mfg. *ESAIM: Proceedings and Surveys*, 2018.
[5] Kimon Antonakopoulos, Elena Veronica Belmega, and Panayotis Mertikopoulos. Adaptive extra-gradient methods for min-max optimization and games. In *International Conference on Learning Representations*, 2021.

[6] Kimon Antonakopoulos, Thomas Pethick, Ali Kavis, Panayotis Mertikopoulos, and Volkan Cevher. Sifting through the noise: Universal first-order methods for stochastic variational inequalities. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, *Advances in Neural Information Processing Systems*, volume 34, pages 13099–13111. Curran Associates, Inc., 2021.

[7] Waïss Azizian, Ioannis Mitliagkas, Simon Lacoste-Julien, and Gauthier Gidel. A tight and unified analysis of gradient-based methods for a whole spectrum of differentiable games. In Silvia Chiappa and Roberto Calandra, editors, *Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics*, volume 108 of *Proceedings of Machine Learning Research*, pages 2863–2873. PMLR, 2020.

[8] Francis Bach and Kfir Y Levy. A universal algorithm for variational inequalities adaptive to smoothness and noise. In Alina Beygelzimer and Daniel Hsu, editors, *Proceedings of the Thirty-Second Conference on Learning Theory*, volume 99 of *Proceedings of Machine Learning Research*, pages 164–194. PMLR, 2019.

[9] A. Bensoussan, S.C.P. Yam, and Z. Zhang. Well-posedness of mean-field type forward-backward stochastic differential equations. *Stochastic Processes and their Applications*, 125(9):3327–3354, 2015.

[10] Alain Bensoussan, Jens Frehse, and Philip Yam. *Mean Field Games and Mean Field Type Control Theory*. Springer, New York, 2013.

[11] C. Bertucci and C. Meynard. Noise through on additional variable for mean field games master equation on finite state space, 2024.

[12] C. Bertucci and C. Meynard. A study of common noise in mean field games, 2024.

[13] Charles Bertucci. Monotone solutions for mean field games master equations: continuous state space and common noise. *Communications in Partial Differential Equations*, 48(10-12):1245–1285, 2023.

[14] Charles Bertucci, Jean-Michel Lasry, and Pierre-Louis Lions. On lipschitz solutions of mean field games master equations. *Journal of Functional Analysis*, 287(5):110486, 2024.

[15] Jean-Baptiste Caillau, Roberto Ferretti, Emmanuel Trélat, and Hasnaa Zidani. Chapter 15 - an algorithmic guide for finite-dimensional optimal control problems. In Emmanuel Trélat and Enrique Zuazua, editors, *Numerical Control: Part B*, volume 24 of *Handbook of Numerical Analysis*, pages 559–626. Elsevier, 2023.

[16] P. Cardaliaguet, F. Delarue, J.-M. Lasry, and P.-L. Lions. The master equation and the convergence problem in mean field games. *Princeton University Press*, 2019.

[17] Pierre Cardaliaguet and Panagiotis Souganidis. Monotone solutions of the master equation for mean field games with idiosyncratic noise. *SIAM Journal on Mathematical Analysis*, 54(4):4198–4237, 2022.

[18] Cardaliaguet, Pierre and Hadikhanloo, Saeed. Learning in mean field games: The fictitious play. *ESAIM: COCV*, 23(2):569–591, 2017.

[19] René Carmona and François Delarue. Mean field forward-backward stochastic differential equations. *Electronic Communications in Probability*, 18:1–15, 2013.

[20] René Carmona and François Delarue. *Probabilistic Theory of Mean Field Games with Applications II*. Springer, 2018.

[21] René Carmona, François Delarue, and Daniel Lacker. Mean field games with common noise. *The Annals of Probability*, 44(6):3740–3803, 2016.

[22] René Carmona and Mathieu Laurière. Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: Ii-the finite horizon case. *The Annals of Applied Probability*, 32(6):4065–4105, 2022.

[23] René Carmona and Mathieu Laurière. Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: Ii-the finite horizon case. *The Annals of Applied Probability*, 32(6):4065–4105, 2022.

[24] Jean-François Chassagneux, Dan Crisan, and François Delarue. Numerical method for FBSDEs of McKean-Vlasov type. *The Annals of Applied Probability*, 29(3):1640–1684, 2019.

[25] Michael G. Crandall, Hitoshi Ishii, and Pierre-Louis Lions. User's guide to viscosity solutions of second order partial differential equations. *Bulletin of the American Mathematical Society*, 27(1):1–67, 1992.

[26] François Delarue and Athanasios Vasileiadis. Exploration noise for learning linear-quadratic mean field games. *Mathematics of Operations Research*, 50(3):1762–1831, 2025.

[27] W. Gangbo, A. Mészáros, C. Mou, and J. Zhang. Mean field games master equations with non-separable hamiltonians and displacement monotonicity. *Annals of Probability*, 50, 2022.

[28] Emmanuel Gobet, Jean-Philippe Lemor, and Xavier Warin. A regression-based monte carlo method to solve backward stochastic differential equations. *The Annals of Applied Probability*, 15(3):2172–2202, 2005.

[29] Eduard Gorbunov, Hugo Berard, Gauthier Gidel, and Nicolas Loizou. Stochastic extragridient: General analysis and improved rates. In Gustav Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera, editors, *Proceedings of The 25th International Conference on Artificial Intelligence and Statistics*, volume 151 of *Proceedings of Machine Learning Research*, pages 7865–7901. PMLR, 2022.

[30] Saeed Hadikhanloo, Rida Laraki, Panayotis Mertikopoulos, and Sylvain Sorin. Learning in nonatomic games, part i: Finite action spaces and population games. *Journal of Dynamics and Games*, 9(4):433–460, 2022.

[31] Joe Jackson and Alpár R. Mészáros. Quantitative convergence for displacement monotone mean field games of control, 2025.

[32] G. M. Korpelevich. The extragridient method for finding saddle points and other problems. *Ekonomika i Matematicheskie Metody*, 12:747–756, 1976.

[33] N. V. Krylov. *Lectures on Elliptic and Parabolic Equations in Hölder Spaces*, volume 12 of *Graduate Studies in Mathematics*. American Mathematical Society, 1996.

[34] Jean-Michel Lasry and Pierre-Louis Lions. Mean field games. *Japanese Journal of Mathematics*, 2(3):229–260, 2007.
[35] Pierre Lavigne and Laurent Pfeiffer. Generalized conditional gradient and learning in potential mean field games. Applied Mathematics & Optimization, 88(3), 2023.

[36] P.-L. Lions. Lectures at collège de france, 2007-2011.

[37] P.-L. Lions and P. Souganidis. Extended mean-field games. Rendiconti Lincei, 3:611-625, 2020.

[38] Alpár R. Mészáros M. Bansil and C. Mou. Global well-posedness of displacement monotone degenerate mean field games master equations, 2023.

[39] C. Meynard. Monotone solutions to mean field games master equation in the 12-monotone setting, 2025.

[40] Chenchen Mou and Jianfeng Zhang. Wellposedness of second order master equations for mean field games with nonsmooth data. *Memoirs of the American Mathematical Society*, 2019.

[41] Chenchen Mou and Jianfeng Zhang. Mean field games of controls: Propagation of monotonicities, 2022.

[42] Chenchen Mou and Jianfeng Zhang. Mean field games of controls: Propagation of monotonicities. *Probability, Uncertainty and Quantitative Risk*, 7(3):247-274, 2022.

[43] A. Mészáros and C. Mou. Mean field games systems under displacement monotonicity. *SIAM Journal on Mathematical Analysis*, 56(1):529-553, 2024.

[44] Yurii Nesterov. Dual extrapolation and its applications to solving variational inequalities and related problems. *Mathematical Programming*, 109(2):319-344, 2007.

[45] R. Tyrrell Rockafellar and Peter R. Wolenski. Convexity in hamilton-jacobi theory i: Dynamics and duality. *SIAM Journal on Control and Optimization*, 39(5):1323-1350, 2000.

[46] Jianfeng Zhang. *Backward Stochastic Differential Equations: From Linear to Fully Nonlinear Theory*, volume 86 of *Probability Theory and Stochastic Modelling*. Springer, 2017.

A. WELLPOSEDNESS OF A MONOTONE FBSDE

**Lemma A.1.** Let $(F, G, g) : \mathbb{R}^{2d} \times \mathcal{P}_2(\mathbb{R}^{2d}) \to \mathbb{R}^d$ be Lipschitz functions and consider the following FBSDE

$$ (A.1) \qquad \begin{cases} dX_t = -\tilde{F}(X_t, U_t)dt + \sigma dB_t, X_0 = X, \\ dU_t = \tilde{G}(X_t, U_t)dt - Z_t dB_t, U_T = g(X_T). \end{cases} $$

If the following monotonicity condition holds

$$ \begin{gathered} \exists c_F > 0, \quad \forall (X, Y, U, V) \in H^4, \\ \langle \tilde{g}(X) - \tilde{g}(Y), X - Y \rangle \ge 0, \\ \langle \begin{pmatrix} F \\ G \end{pmatrix} (X, Y) - \begin{pmatrix} \tilde{F} \\ \tilde{G} \end{pmatrix} (Y, V), \begin{pmatrix} X - Y \\ U - V \end{pmatrix} \rangle \ge c_F \| \tilde{F}(X, U) - \tilde{F}(Y, V) \|^2, \end{gathered} $$

then for any $T > 0$, and for any initial condition $X \in H$ independent of $(B_t)_{t \ge 0}$ there exists a unique solution to the FBSDE (A.1).

*Proof.* Let us present an a priori estimate for the forward backward system (A.1). We fix two initial conditions $X, Y \in H$ and assume that there exist strong solutions to (A.1) $(X_t, U_t)_{t \in [0,T]}$ (resp. $(Y_t, V_t)_{t \in [0,T]}$) with initial condition $X$ (resp. $Y$). We now introduce the following process

$$ \forall t \in [0, T], \quad I_t = (U_t - V_t) \cdot (X_t - Y_t). $$

By Ito's lemma,

$$ \forall t \in [0, T], \quad \mathbb{E}[I_T] = \mathbb{E}[I_t] - \int_t^T \langle \begin{pmatrix} \tilde{F} \\ \tilde{G} \end{pmatrix} (X_s, U_s) - \begin{pmatrix} \tilde{F} \\ \tilde{G} \end{pmatrix} (Y_s, V_s), \begin{pmatrix} X_s - Y_s \\ U_s - V_s \end{pmatrix} ds. $$

Using the monotonicity of coefficients, we get that

$$ \forall t \in [0, T], \quad 0 \le \mathbb{E}[I_T] \le \mathbb{E}[I_t], $$

and

$$ c_F \mathbb{E} \left[ \int_0^T |\tilde{F}(X_s, U_s) - \tilde{F}(Y_s, V_s)|^2 ds \right] \le \mathbb{E}[I_0]. $$

By definition of $(X_s, Y_s)_{s \in [0, T]}$, it follows naturally that

$$ \forall t \in [0, T], \quad \|X_t - Y_t\| \le \|X - Y\| + \frac{t}{c_F} \sqrt{\mathbb{E}[I_0]}. $$

Using this estimate and by a backward application of Gronwall lemma, there exists a constant $C$ depending only on $\|(G, g)\|_{Lip}$, $c_F$ such that

$$ \forall t \in [0, T], \quad \|U_t - V_t\|^2 \le C e^{CT} (\|X - Y\|^2 + \mathbb{E}[I_0]). $$
Since by Cauchy schwartz inequality

$$ \forall \lambda > 0, \quad \mathbb{E}[I_0] \le \frac{\lambda}{2} \|X - Y\|^2 + \frac{1}{2\lambda} \|U_0 - V_0\|^2, $$

choosing $\lambda = Ce^{CT}$ and evaluating for $t = 0$ we get

$$ \|U_0 - V_0\|^2 \le (2Ce^{CT} + C^2e^{2CT})\|X - Y\|^2. $$

This shows that there can be at most one strong solution for a given initial condition. Since this a priori estimate is valid for any initial condition and on any time interval, it follows naturally from ideas introduced in [14, 39] that the FBSDE (A.1) admits a Lipschitz decoupling field $U : [0, T] \times \mathbb{R}^d \times \mathcal{P}_2(\mathbb{R}^d) \to \mathbb{R}^d$. In particular, this allows to conclude that there indeed exists a unique strong solution to (A.1) for any initial condition $X \in H$. $\square$