# Mean-field games for high-density crowds: The discount factor reproduces experiments but raises numerical issues

Cécile Appert-Rolland, Matteo Butano, Denis Ullmo

► To cite this version:

Cécile Appert-Rolland, Matteo Butano, Denis Ullmo. Mean-field games for high-density crowds: The discount factor reproduces experiments but raises numerical issues. EPJ Web of Conferences, 2025, Traffic and Granular Flow 2024 (TGF'24), 334, pp.04022. 10.1051/epjconf/202533404022. hal-05262060

HAL Id: hal-05262060

https://hal.science/hal-05262060v1

Submitted on 25 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Mean-field games for high-density crowds: the discount factor reproduces experiments but raises numerical issues

Cécile APPERT-ROLLAND<fnref n="1" />,*, Matteo BUTANO<fnref n="1" />,<fnref n="2" />, and Denis ULLMO<fnref n="2" />,**

**Abstract.** Among the various types of crowd models, mean-field games have proved especially appropriate to reproduce some experimental results in high density crowds that involve anticipation on a time scale longer than usually involved in collision avoidance. It is possible to tune the level of anticipation with a so-called discount factor. However, the introduction of such a discount factor raises numerical issues close to solid boundaries (walls). A way to circumvent the difficulty is presented.

## 1 Introduction

Pedestrian displacements involve several timescales. In [1] it was proposed to distinguish the strategic level, where pedestrians plan a task and the associated displacement; the tactical level, where the route choice is made; and the operational level, which describes the actual walking behavior and collision avoidance. In most microscopic modeling approaches, the tactical level will result in the determination of a desired velocity, while at the operational level the actual velocity will deviate from the desired one due to collision avoidance strategies.

However, in some cases, the strategic and tactical levels are not so clearly separated. For example, during an evacuation, pedestrians may anticipate congestion at the nearest exit and decide to take a detour to avoid it. Another example is the avoidance of a moving obstacle by a dense crowd, such as in the experiment of [2, 3], in which a cylindrical obstacle was moving in a dense, static crowd. It was observed that individuals moved toward high-density regions long before being pushed by the cylinder, at odds with what classical agent-based models would do, because agents anticipate that in total it will be the most efficient strategy to let the obstacle go.

Mean-field games (MFG) [4–7] have been proposed as a possible tool for modeling crowds [8, 9] and have been shown to be able to capture such anticipation [10, 11] even in the simplest version of the model. However, a more refined description can be obtained by using a so-called discount factor, which allows to adjust the level of anticipation. Indeed, this factor has allowed for the reproduction of different experimental observations under different conditions within a single framework [12].

<footnote n="1">¹Université Paris-Saclay, CNRS, IJCLab (UMR 9012), 91405 Orsay, France</footnote>

<footnote n="2">²Université Paris-Saclay, CNRS, LPTMS (UMR 8626), 91405 Orsay, France</footnote>

<footnote>*e-mail: Cecile.Appert-Rolland@ijclab.in2p3.fr</footnote>

<footnote>**e-mail: denis.ullmo@universite-paris-saclay.fr</footnote>
In this paper, after general considerations on the various types of anticipation contained in classical modeling approaches (section 2), we shall recall how quadratic MFG are formulated (section 3) and how equations can be handled in the permanent regime (section 4). Then, we will explain how introducing of a discount factor creates numerical difficulties near solid boundaries and eventually give some perspective on how to solve these numerical issues (section 5).

## 2 Various levels of anticipation

Agent-based models for pedestrian dynamics span from the historical social force model [13] to velocity based models [14, 15]. These models have in common that each pedestrian is embedded with a desired velocity that she would follow if she were alone, and a collision avoidance strategy that causes her to deviate from it. The latter usually does not anticipate much beyond the next most imminent collision. Longer-term anticipation can be incorporated into the desired velocity, with several types of hypotheses.

In order to illustrate the various levels of anticipation that are encountered, we consider the case of an evacuation from a room with two doors (see Fig. 1).

<figure><img src="image_2.png" /><figcaption>**Figure 1.** Evacuation of a room with two doors. Several cases are considered where the choice of the desired velocity for the hatched-blue pedestrian is conditioned by (a) the shortest path; (b) some congestion formed at the current time; (c) some congestion that could form in the future; (d) a more complex case with possible congestions at both exits.</figcaption></figure>

The simplest way to define the desired velocity is to have it follow the shortest path (Fig. 1a).

However, if a congestion forms at the current time in front of the closest exit, using the other exit may allow for a faster exit from the room (Fig. 1b). This type of optimization, which is based on the system's current state, is often used in macroscopic models, such as for example the Hughes' model [16, 17]. But such optimization can also be performed in
the framework of microscopic models [1, 18, 19]. The desired velocity can be chosen so as to minimize an estimate of the travel time taking congestion effects into account [19]. The approach used by Hoogendoorn et al [1, 18] corresponds to an optimal control problem, in which each agent optimizes its future path by minimizing a cost, based on the current positions and velocities of the other pedestrians. In all these approaches, the agent does not anticipate the movement of others. The optimization must therefore be repeated periodically to account for the new state of the system (see the simulations in [1]).

Anticipation of the future behavior of other pedestrians can be incorporated through the use of game theory approaches [1, 18]. Games are relevant when there is a competition between agents. In crowds, pedestrians compete for space. Each agent optimizes her strategy so as to minimize a cost involving the future positions of others (Fig. 2a). The behavior of others is then predicted under the assumption that they also optimize a similar, known cost. Solving the game usually consists in finding a Nash equilibrium, i.e. a set of strategies for all agents such that none of them would benefit from making an individual strategy change.

When the number of pedestrians is large, predicting the individual behavior of each agent is very time consuming. One alternative is then to replace the knowledge of individual positions by a density field - hence the name Mean-field Game [8] (Fig. 2b).

<figure><img src="image_3.png" /><figcaption>**Figure 2.** Evacuation of a room with two doors. In game theory (a), the hatched-blue pedestrian will anticipate that others will have the same strategy as she would have if she were in their place. In a mean-field game (b), others are not seen individually but are replaced by a density field against which the blue pedestrian competes for space.</figcaption></figure>

## 3 A basic Mean-field Game model

The mean-field games formalism has been introduced by mathematicians [4–6], and describe games where each agent plays against a density field representative of the other players.

In one of the simplest setting, each agent is assumed to move with a speed $v_{des} + \sigma\xi$, i.e. her desired velocity is modified by a Gaussian noise but no direct interaction between agents is explicitly taken into account at the operational level. The desired velocity is obtained from the optimization by each agent of an individual cost

$$c[v_{des}(\cdot)](\mathbf{x}, t) = \mathbb{E} \left\{ \int_{t}^{T} \mathcal{L}(\mathbf{x}_{\tau}, \tau)[\rho_{\tau}(\cdot)]e^{\gamma(t-\tau)} d\tau + e^{\gamma(t-T)} c_{T}(\mathbf{x}_{T}) \right\} \quad (1)$$

in which the expectation value is taken on the noise $\xi$, and the coordinate $\mathbf{x}$ refers to the initial position of the agent at time $t$.
In this expression, the running cost $\mathcal{L}$ and the terminal cost $c_T$ are weighted by an exponential decreasing with time on a time scale $1/\gamma$. Such a discount factor reflects the fact that events close in time should have a higher influence on the agents' strategy choice [1, 8, 18]. Another interpretation of the discount factor is related to partial blindness [8] and thus limited access to information. Indeed, in the aforementioned moving cylinder experiment, tuning the discount parameter $\gamma$ allowed to account for the variable level of information that pedestrians have depending on whether they face or turn their back to the incoming moving obstacle, with a good qualitative agreement with the experimental data [12].

In Eq. (1), the running cost $\mathcal{L}(\mathbf{x}_\tau, \tau)[\rho]$ depends on the density $\rho$ and it is only through this term that interactions between agents are taken into account. Besides, the fact that the running cost depends quadratically on the control parameter $v_{\text{des}}$, as in Eq.(2), makes the corresponding MFG particularly tractable - see [7] for a review on Quadratic Mean Field Games. To fix the ideas, an example of possible running cost is

$$ \mathcal{L}(\mathbf{x}_\tau, \tau)[\rho] = \frac{\mu}{2}(v_{\text{des}}(\mathbf{x}_\tau, \tau))^2 - g\rho(\mathbf{x}_\tau, \tau) - U_0(\mathbf{x}_\tau, \tau) \quad (2) $$

where $U_0$ is a potential that can take infinite values under moving (cylinder) or fixed (walls) obstacles. Here $g < 0$ would ensure that pedestrians are reluctant to be in highly crowded regions.

The individual cost that an individual has to pay for the optimal strategy - or equivalently optimal desired velocity $v_{\text{des}}^*(\cdot)$- is called the value function

$$ u(\mathbf{x}, t) = \min_{v_{\text{des}}(\cdot)} c[v_{\text{des}}(\cdot)](\mathbf{x}, t). $$

Note that the value function depends only on the pair $(\mathbf{x}, t)$, where $\mathbf{x}$ is the location of the agent considered at time $t$, but not on the label $i$ of the agent.<fnref n="1" /> Indeed all pedestrians are identical in the sense that they would choose the same desired velocity if located at the same initial position.

Using Bellman's Principle of Optimality [20], the game described above can be formulated in the form of a system of two coupled equations, which, when $\gamma = 0$, reads

$$ \partial_t \rho = \frac{1}{\mu} \nabla_x (\rho \nabla_x u) + \frac{\sigma^2}{2} \Delta_x \rho \quad \text{(forward in time)} \quad (3) $$

$$ \partial_t u = \frac{1}{2\mu} (\nabla_x u)^2 - \frac{\sigma^2}{2} \Delta_x u + g\rho + U_0 \quad \text{(backward in time)} \quad (4) $$

with the initial/final conditions

$$ \rho(t=0, \mathbf{x}) = \rho_0(\mathbf{x}) $$

$$ u(t=T, \mathbf{x}) = c_T(\mathbf{x}). $$

The first equation (3) gives the evolution of the density field. Note that although the game is based on the optimizing of an individual cost (Eq. (1)), the MFG is actually a macroscopic model.

The second equation, called the Hamilton-Jacobi-Bellman (HJB) equation, gives the evolution of the value function $u$. Note that the sign in front of the Laplacian is positive for the

<footnote n="1">¹At the initial time $t = 0$, agents had no precise knowledge of their future position at time $t > 0$, as it was depending on the future realization of noise. They knew only the probability distribution of their location at later times. Once at time $t > 0$, of course they know which position they occupy. This extra knowledge is included in the new optimization that they perform at time $t$.</footnote>
first equation but negative for HJB. Also, for HJB, it is the final value that is given as temporal boundary condition. This indicates that, while the density equation is forward in time, as usual, the HJB equation is backward in time. This special forward/backward structure leads to specific challenges both from a conceptual point of view as well as in the numerical implementation of the model.

Finally, the desired velocity of a given pedestrian for which the cost is optimal is obtained as

$$v_{\text{des}}^{*}(\mathbf{x}, t) = -\frac{1}{\mu} \nabla_{\mathbf{x}} u(\mathbf{x}, t)$$

Again, the pedestrian is identified by the fact that it is in position $\mathbf{x}$ at time $t$.

## 4 The permanent regime

In this section we shall address the case of the permanent, or steady-state regime. This is particularly relevant in the case of the cylinder crossing a static crowd, as we expect the crowd's deformation around the cylinder to be stationary in the cylinder's frame. But we can also have permanent regimes in rooms or parts of buildings where pedestrians would enter and exit on different sides, with a constant incoming flow.

In this permanent regime, both the density $\rho_{\text{st}}(\mathbf{x})$ and velocity fields are stationary. However, since the value function is an integral over time, it can still depend on time, which for $\gamma = 0$ can be written as $u(t, \mathbf{x}) = u_{\text{st}}(\mathbf{x}) - \lambda t$, with $\lambda$ a constant usually fixed by the asymptotic behavior of the value function (for the cylinder experiment $\lambda = -g\rho_0$). Both density and value function are governed in the permanent regime by the time-independent equations which are obtained by setting $\partial_t \rho \to 0$ in Eq. (3) and $\partial_t u \to -\lambda$ in Eq. (4).

Solving the permanent regime system of equations is conceptually simpler than solving the time-dependent one since the dimensionality of the problem is reduced by one. However, a direct implementation of a discrete difference scheme shows strong instability and typically does not converge. The reason for this instability can be understood as follows: Since $u_{\text{st}}(\mathbf{x})$ is the value function, it represents the optimal cost an individual can hope to pay if located at $\mathbf{x}$. An impenetrable obstacle, such as the cylinder in the cylinder-crossing experiment, or the walls of a room, is modeled by an instantaneous cost $U_0(\mathbf{x}) \to \infty$. A consequence is that the value function $u_{\text{st}}(\mathbf{x})$ will also diverge to $\infty$ as one approaches an impenetrable obstacle, and one can show that this divergence is logarithmic. The reason why the divergence is also felt near but outside of the obstacle is that the noise in the pedestrian dynamics gives some diffusion at the macroscopic level. Due to this diffusion, a perturbation is felt at a typical distance $h = \sqrt{\mu\sigma^4/2g\rho_0}$ called the healing length. Here this logarithmic divergence of $u_{\text{st}}$ makes finite difference methods very impractical near impenetrable obstacles.

When $\gamma = 0$, one possible way to address the difficulties generated by the log divergence is to perform a change of variables from the original ($u_{\text{st}}$, $\rho_{\text{st}}$) variables to a new set of variables ($\Phi_{\text{st}}$, $\Gamma_{\text{st}}$) defined as

$$\Phi_{\text{st}}(\mathbf{x}) = \exp\left(-\frac{1}{\mu\sigma^2} u_{\text{st}}(\mathbf{x})\right), \qquad (5)$$

$$\Gamma_{\text{st}}(\mathbf{x}) = \rho_{\text{st}}(\mathbf{x}) \exp\left(\frac{1}{\mu\sigma^2} u_{\text{st}}(\mathbf{x})\right), \qquad (6)$$

implying thus that $\rho_{\text{st}}(\mathbf{x}) = \Gamma_{\text{st}}(\mathbf{x})\Phi_{\text{st}}(\mathbf{x})$. Eq. (5) is a rather standard Cole-Hopf transformation [21, 22] allowing to linearize the equation, and Eq. (6) can be understood as an Hermitization<fnref n="2" /> of the Kolmogorov equation. In terms of the ($\Phi_{\text{st}}$, $\Gamma_{\text{st}}$) variables [7], the mean field

<footnote n="2">²Hermitization allows to have diagonalizable matrices, which are more easy to handle.</footnote>
game equations take a form that resembles very much the stationary Non-Linear Schrödinger equation (NLS) describing for instance the mean field approximation of interacting bosons.

Beyond the physical understanding that can emerge from such a representation, we immediately see from Eq. (5) that as $u_{\text{st}} \to \infty$, we get $\Phi_{\text{st}} \to 0$. The logarithmic divergence thus disappears for $\Phi_{\text{st}}$, and it can be shown that $\Gamma_{\text{st}}$ also nicely tend to zero near the impenetrable obstacles. As a consequence, a rather standard finite difference scheme can be used to solve this system of equations [23].

# 5 The problem with $\gamma \neq 0$

Now, let us consider the case of a finite discount parameter $\gamma \neq 0$, as in Eq. (1). At first glance, this modification seems to imply only marginal changes to the formalism, namely: i) the addition of a term $\gamma u_{\text{st}}$ on the right-hand side of Eq. (4), and ii) the fact that $\lambda$ should now be taken equal to zero. However, it turns out that this $\gamma u_{\text{st}}$ term completely spoils the transformation to the $(\Phi_{\text{st}}, \Gamma_{\text{st}})$ variables, preventing us from following the same route as in the $\gamma = 0$ case.

As we know that the difficulty here lies in the logarithmic divergence of the value function near impenetrable obstacles, it is useful to focus on a 'small distance' $l$ near this impenetrable obstacle, namely to make the change of variable $x = lx'$, together with $u_{\text{st}} = \mu\sigma^2u'_{\text{st}}$ so that equations are dimensionless. Doing so, one observes that if $l \ll h$, $h$ being the healing length, the effects of interactions between pedestrians become negligible, which is expected as the motion is dominated by the proximity to the impenetrable obstacle rather than by interactions. In the same way, and more importantly for our concern, if $l \ll \sigma/\sqrt{\gamma}$, the impact of a finite discount parameter $\gamma$ can be seen to be negligible.

This remark suggests a possible approach to solving numerically the MFG equation in the permanent regime for a finite discount parameter $\gamma$. As illustrated in Fig. 3, one could divide the space into two overlapping regions:

* a first region very close to the impenetrable obstacle, within a distance $l \ll \sigma/\sqrt{\gamma}$. In this region, the effects of a non-zero $\gamma$ are negligible, so that a transformation to the $(\Phi_{\text{st}}, \Gamma_{\text{st}})$ variables can be performed;

* a second region made of the remaining of the spatial domain (with a little bit of overlap with the former). There, we are sufficiently far from the impenetrable obstacle not to be bothered by the logarithmic divergence, so that a direct calculation with the $(u_{\text{st}}, \rho_{\text{st}})$ variables does not raise a particular problem.

The overlap between the two regions allows to couple the solutions obtained on both sides. This approach have been implemented successfully in [12] to analyze the role of the discount factor.

# 6 Conclusion

In conclusion, we have seen that a natural way to implement the anticipation processes that have been demonstrated in the cylinder crossing experiment [2, 3], and which should also occur naturally in the context of room evacuation, is to approach them through a game theory. For dense crowds, where the number of pedestrians is large, a mean-field approximation is both necessary and effective, leading to a mean-field game model of the crowd behavior.

Although the mean-field game description is much simpler than its many-body counterpart, its forward-backward structure makes it conceptually not straightforward, and logarithmic divergence near impenetrable barriers leads to instabilities in numerical schemes.
<figure><img src="image_4.png" /><figcaption>**Figure 3.** Schematic of the different regions near the cylindrical obstacle. Dots correspond to the discretization of space used in the numerical scheme. Cyan area ($r < R$): discretization points lying under the cylindrical obstacle. Yellow area ($R < r < R + l$): neighborhood of the impenetrable obstacle sufficiently close to the boundary that the effect of the discount factor is irrelevant. Prune area ($r > R+l$): numerical domain sufficiently far from the impenetrable obstacle that the logarithmic divergence is not a problem. From [23].</figcaption></figure>

Within a quadratic model with “perfect information” (here in the sense of a discount parame-
ter $\gamma = 0$), introducting of a new set of variables allows us to map the problem onto something
very reminiscent of the nonlinear Schrödinger equation, which not only brings physical in-
sights but also solves the divergence problem. For non-zero $\gamma$ this approach cannot be applied
to the entire space. However, a mixed approach using the new variables near the impenetrable
barrier, where the terms associated with the discount factor are negligible, and the original
variables beyond that, provides an elegant way to solve the mean-field game equations nu-
merically, opening the way to a larger range of applications. A more thorough study of this
numerical approach will be published later.

References

[1] S.P. Hoogendoorn, P.H.L. Bovy, Transportation Research Part B: Methodological 38, 169 (2004), https://doi.org/10.1016/S0191-2615(03)00007-9

[2] A. Nicolas, M. Kuperman, S. Ibáñez, S. Bouzat, C. Appert-Rolland, Sci. Rep. 9, 105 (2019), https://doi.org/10.1038/s41598-018-36711-7

[3] C. Appert-Rolland, J. Pettré, A.H. Olivier, W. Warren, A. Duigou-Majumdar, E. Pinsard, A. Nicolas, Collective Dynamics 5, 1 (2020), https://doi.org/10.17815/CD.2020.109

[4] J.M. Lasry, P.L. Lions, C. R. Acad. Sci. Paris, Ser. I **343**, 619 (2006), https://doi.org/10.1016/j.crma.2006.09.019

[5] J.M. Lasry, P.L. Lions, C. R. Acad. Sci. Paris, Ser. I **343**, 679 (2006), https://doi.org/10.1016/j.crma.2006.09.018

[6] M. Huang, R. Malhamé, P. Caines, Commun. Inf. Syst. **6**, 221 (2006).

[7] D. Ullmo, I. Swiecicki, T. Gobron, Phys. Rep. **799**, 1 (2019), https://doi.org/10.1016/j.physrep.2019.01.001
[8] A. Lachapelle, M.T. Wolfram, Transp. Res. part B: method. **45**, 1572 (2011), https://doi.org/10.1016/j.trb.2011.07.011

[9] M. Burger, M.D. Francesco, P. Markowich, M.T. Wolfram, Discrete Contin. Dyn. Syst. Ser. B **19**, 1311 (2014), https://doi.org/10.3934/dcdsb.2014.19.1311

[10] T. Bonnemain, M. Butano, T. Bonnet, I. Echeverría-Huarte, A. Seguin, A. Nicolas, C. Appert-Rolland, D. Ullmo, Phys. Rev. E **107**, 024612 (2023), https://doi.org/10.1103/PhysRevE.107.024612

[11] M. Butano, T. Bonnemain, C. Appert-Rolland, A. Nicolas, D. Ullmo, *Modeling of obstacle avoidance by a dense crowd as a Mean-Field Game*, in *Traffic and Granular Flow '22*, edited by K. Rao, A. Seyfried, A. Schadschneider (Springer, Singapore, 2024), Vol. 443 of *Lecture Notes in Civil Engineering*, pp. 93–100, https://doi.org/10.1007/978-981-99-7976-9_12

[12] M. Butano, C. Appert-Rolland, D. Ullmo, SciPost Phys. **16**, 104 (2024), https://doi.org/10.21468/SciPostPhys.16.4.104

[13] D. Helbing, P. Molnar, Phys. Rev. E **51**, 4282 (1995), https://doi.org/10.1103/PhysRevE.51.4282

[14] S. Paris, J. Pettré, S. Donikian, Computer Graphics Forum - Eurographics **26**, 665 (2007), https://doi.org/10.1111/j.1467-8659.2007.01090.x

[15] J. van den Berg, M.C. Lin, D. Manocha, *Reciprocal velocity obstacles for real-time multi-agent navigation*, in *Proceedings of the 2008 IEEE International Conference on Robotics and Automation* (2008), pp. 1928–1935, https://doi.org/10.1109/ROBOT.2008.4543489

[16] R. Hughes, Transp. Res. B **36**, 507 (2002), https://doi.org/10.1016/S0191-2615(01)00015-7

[17] M.D. Francesco, P.A. Markowich, J.F. Pietschmann, M.T. Wolfram, J. of Diff. Equations **250**, 1334 (2011), https://doi.org/10.1016/j.jde.2010.10.015

[18] S.P. Hoogendoorn, P.H.L. Bovy, Optim. Control Appl. Meth. **24**, 153 (2003), https://doi.org/10.1002/oca.727

[19] T. Kretz, A. Große, S. Hengst, L. Kautzsch, A. Pohlmann, P. Vortisch, Advances in Complex Systems **14**, 733 (2011), https://doi.org/10.1142/S0219525911003281

[20] R. Bellman, *Dynamic Programming* (Princeton University Press, 1957)

[21] E. Hopf, Communications on Pure and Applied Mathematics **3**, 201–230 (1950), https://doi.org/10.1002/cpa.3160030302

[22] J.D. Cole, Quarterly of Applied Mathematics **9**, 225–236 (1951), https://doi.org/10.1090/qam/42889

[23] M. Butano, Ph.D. thesis, Graduate School of Physics, University Paris-Saclay (2024), http://www.theses.fr/2024UPASP064