# Vaccination in a Large Population: Mean Field Equilibrium versus Social Optimum

Bruno Gaujal, Josu Doncel, Nicolas Gast

► To cite this version:

Bruno Gaujal, Josu Doncel, Nicolas Gast. Vaccination in a Large Population: Mean Field Equilibrium versus Social Optimum. NETGCOOP 2020 - 10th International Conference on NETwork Games, Control and Optimization, Sep 2021, Cargèse, France. pp.1-9. hal-02938850

HAL Id: hal-02938850

https://inria.hal.science/hal-02938850

Submitted on 15 Sep 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
# Vaccination in a Large Population: Mean Field Equilibrium versus Social Optimum

Bruno Gaujal<fnref n="2" /> and Josu Doncel<fnref n="1" /> and Nicolas Gast<fnref n="2" />

**Abstract.** We analyze a virus propagation dynamics in a large population of agents (or nodes) with three possible states (Susceptible, Infected, Recovered) where agents may choose to vaccinate. We show that this system admits a unique symmetric equilibrium when the number of agents goes to infinity. We also show that the vaccination strategy that minimizes the social cost has the same threshold structure as the mean field equilibrium, but with a shorter threshold. This implies that, to encourage optimal vaccination behavior, vaccination should always be subsidized.

## 1 SIRV Dynamics

A large number $N$ of agents (that can be nodes in a communication network, persons in a crowd, or abstract players in a mathematical game) are subject to interactions: they meet (or communicate with) each other according to a uniform process, described in the following. Each agent (or player<fnref n="3" />) has 3 possible states: Susceptible, Infected, Recovered ($S, I, R$).

When an agent in state $S$ meets an agent in state $I$, it gets infected. An agent in state $I$ will eventually recover and go to state $R$. An agent in state $R$ stays in $R$ forever. This is a classical SIR model of virus propagation among the agents.

This model can be seen as over-simplistic but actually, it has a good predictive power for human epidemics and other diffusion processes. This model was introduced as early as in 1927 by Kermack and McKendrick in a series of papers [1] and "because of their seminal importance to the field of theoretical epidemiology, these articles were republished in the Bulletin of Mathematical Biology in 1991" (from wikipedia). This so-called SIR model has been studied ever since, being the subject of papers in mathematics, computer science, health studies and bio-mathematics.

Here, we consider a stochastic evolution of the population, whose state changes are driven by Poisson events and players can decide to vaccinate (hence the $V$ in the name of our model).

<footnote n="1">¹ University of the Basque Country, Leioa 48940, Spain</footnote>

<footnote n="2">² Univ. Grenoble Alpes, Inria, CNRS, LIG, F-38000 Grenoble, France</footnote>

<footnote>— A player encounters other players with rate $\gamma$ (activity of the player). A slight generalization is possible without any impact on the following analysis. The</footnote>

<footnote n="3">³ both terms will be used in the following, interchangeably.</footnote>
rate of encounters with susceptible players is equal to $\gamma_1$ and the rate of encounters with infected players is equal to $\gamma_2(< \gamma_1)$ to take into account a lesser activity of infected players. If the first player is Susceptible and the second is Infected in an encounter, then the first one becomes Infected.

- An Infected agent Recovers at rate $\rho$.

- A Susceptible agent can decide to get vaccinated. It chooses its vaccination rate $\pi(t) \in [0, \theta]$.

- Once an agent is vaccinated or recovered, its state becomes $R$ in both cases and does not change after this point.

Similar models have been studied in [2,3], although the vaccination policies in these papers do not depend explicitly on the susceptible population.

Let $(m_S(t), m_I(t), m_R(t))$ be the proportion of the agents in states $S, I, R$ respectively at time $t$. The Markovian evolution of one player is given by a non-homogeneous continuous time Markov chain displayed in Figure 1.

<figure><img src="image_1.png" /><figcaption>Fig. 1: Markov chain driving the evolution of one agent in the population</figcaption></figure>

## 2 Cost Functions and Objectives

In addition to the state evolution, the system is endowed with cost functions.

The cost of being infected is $c_I$ per time unit.

The vaccination cost is linear in its rate: For one player, the cost to vaccinate at rate $\pi$ is $c_V\pi$.

We are now ready to state the problem to be solved: *under full information and common knowledge (the state of all players at time t and their vaccination strategy is known to all)*, each player wants to choose a vaccination strategy that *minimizes its cost up to a time horizon T*.

This is not a well defined problem because the optimal strategy of any player depends on the strategy of any other player, who in turn is trying to optimize its vaccination strategy that depends on the first player's strategy. One classical way to get around this difficulty is to consider "stable points", that is Nash equilibria and social optimum.

**Definition 1 (Symmetric Nash Equilibrium (SNE))**. A Symmetric Nash Equilibrium is a vaccination strategy $\pi_{NE}$ such that if all the players use $\pi_{NE}$, then any player's optimal strategy is to use $\pi_{NE}$.
**Definition 2 (Social Optimal).** A social optimal is a vaccination strategy $\pi_{SO}$ imposed upon all players that minimizes the sum of the costs of all the players.

SNE always exist in our SIRV model (the proof is based on *Kakutani fixed point theorem*, not detailed here). SO always exist in our SIRV model (The proof uses the compacity of the strategy space for the weak topology and is not detailed here). For more details on these two proofs of existence, see [4].

Unfortunately both strategies are very hard to compute when $N$ is large because of the combinatorial explosion of the state space.

## 3 Mean Field Limit

Since the players are indistinguishable, the state of the system is given by the population distribution, $(m_S(t), m_I(t), m_R(t))$.

When $N \to \infty$, $(m_S(t), m_I(t), m_R(t))$ behaves as a fluid, whose evolution follows the Kolmogorov equations of the individual Markov chain (see for example [4,5]). Under a given vaccination strategy $\pi$.

$$ 
\begin{cases}
\dot{m}_S(t) = -\gamma m_S(t) m_I(t) - \pi(t) m_S(t) \\
\dot{m}_I(t) = \gamma m_S(t) m_I(t) - \rho m_I(t) \\
\dot{m}_R(t) = \rho m_I(t) + \pi(t) m_S(t).
\end{cases}
\quad (1) $$

There are some technicalities here, because $\pi(t)$ may not be continuous. In this case, one can still use the Carathéodory Existence Theorem to show that the solution of these equation is well defined and unique once $(m_S(0), m_I(0), m_R(0))$ are given (see [4] for a rigorous proof).

When $\pi(\cdot) \equiv 0$, this yields the classical SIR dynamics of *Kermack and McKendrick* (1927). Even in this simple case, completely closed form solutions have not been found (see the recent results in [6]). However, the mean field framework is still easier to analyze qualitatively than the case with a finite number of players because, at the limit, the strategy chosen by a single player will not alter the behavior of the whole population. To make this more precise, we introduce the best response correspondence in the mean field case.

### 3.1 Best Response

Let us pick one player among the infinite population (it is called player 0 in the following). Let us define the state probabilities $(p_S^0, p_I^0, p_R^0)$ of Player 0 as follows: For $X \in \{S, I, R\}$,

$$ p_X^0(t) = \mathbb{P}(\text{Player 0 is in state } X \text{ at time } t). $$

If player 0 uses strategy $\pi_0$ while the population uses strategy $\pi$, then the state probabilities $(p_S^0, p_I^0, p_R^0)$ of Player 0 has an evolution given by its local Kolmogorov equation:
$$
\begin{cases}
\dot{p}_S^0(t) = -\gamma p_S^0(t) m_I(t) - \pi_0(t) p_S^0(t) \\
\dot{p}_I^0(t) = \gamma p_S^0(t) m_I(t) - \rho p_I^0(t) \\
\dot{p}_R^0(t) = \rho p_I^0(t) + \pi_0(t) p_S^0(t).
\end{cases}
$$

The population distribution $(m_S(t), m_I(t), m_R(t))$ used in these equations are the solutions of the mean field limit equations (1) and do not depend on the strategy chosen by Player 0.

Using the foregoing notations, the expected individual cost for Player 0 is:

$$
W(\pi_0, \pi) = \int_0^T \left( c_V \pi_0(t) p_S^0(t) + c_I p_I^0(t) \right) dt,
$$

where $c_V$ is the vaccination cost and $c_I$ is the unit time cost of being infected, as defined earlier.

A best response of Player 0 to a population using strategy $\pi$ is a strategy $\pi_0^*$ that minimizes its cost.

This best response can be computed using a Bellman optimality equation. If we denote by $W_X^*(t)$ the optimal total cost from $t$ to $T$ of Player 0 when it is in state $X$ at time $t$, they satisfy

$$
W_R^*(T) = W_S^*(T) = W_I^*(T) = 0.
$$

and

$$
- \dot{W}_S^*(t) = \min_{\pi_0(t)} [\pi_0(t) (c_V - W_S^*(t)) + \gamma m_I(t) (W_I^*(t) - W_S^*(t))] \quad (2)
$$

$$
- \dot{W}_I^*(t) = c_I - \rho W_I^*(t). \qquad (3)
$$

The best response strategy when in state $S$ at time $t$ is given by

$$
\pi_0^*(t) = \arg \min_{\pi_0(t)} [\pi_0(t) (c_V - W_S^*(t)) + \gamma m_I(t) (W_I^*(t) - W_S^*(t))]. \quad (4)
$$

Let us denote by $BR(\pi) = \pi_0^*$, the best responses of Player 0 to $\pi$.

**Definition 3 (Mean Field Equilibirum (MFE))**. If $\pi \in BR(\pi)$ then $\pi$ is a mean field equilibrium.

3.2 Threshold Policy

By analyzing the Bellman Equations (2)-(3), we can show that the best response
strategy is unique and has a specific structure.

**Lemma 1.** For any population strategy $\pi$, the best-response $\pi_0^* \in BR(\pi)$ is a threshold strategy: There exists a critical time $t_c^*(\pi)$ s.t.

$$
\pi_0^*(t) = \begin{cases} \theta & \text{if } t < t_c^0(\pi), \\ 0 & \text{if } t > t_c^0(\pi). \end{cases}
$$
The full proof is available in [7], is essentially based on the analysis of

$$ \pi_0^*(t) = \arg\min_{\pi_0(t)} [\pi_0(t)(c_V - W_S^*(t)) + \gamma m_I(t)(W_I^*(t) - W_S^*(t))], $$

which implies that $\pi_0(t)$ must be 0 whenever $c_V > W_S^*(t)$ and $\theta$ whenever $c_V < W_S^*(t)$.

This induces the following theorem.

**Theorem 1.** SIRV has a unique mean-field equilibrium. This equilibrium is deterministic (players do not randomize their decisions) and is a threshold strategy. The threshold will be denoted by $\tau_c^{MFE}$.

In 2004, Francis ([2] showed that the stable point for each player in the SIRV equations has a threshold. This result precedes the introduction of mean field games and mean field equilibria, only defined in 2007 in [8]. It is quite remarkable (and also natural) that the two notions should coincide.

Theorem 1 is proved in the long version of this paper, available as a Research Report [7].

We believe that $t_c^{MFE}$ cannot be computed in closed form as a function of $T$, $c_I$, $c_V$, $\theta$, $\rho$ and $\gamma$. However, the complexity reduction from finding a stable strategy (whose domain has an infinite dimension) to determining one real number makes numerical computations feasible (see Section 5).

# 4 Social Optimal Strategy

We denote by $C(\pi)$ the average cost incurred by the population under strategy $\pi$, i.e.,

$$ C(\pi) = \int_{0}^{T} (c_I m_I(t) + c_V \pi(t) m_S(t)) dt. $$

A social optimal strategy minimizes the total cost:

$$ \pi^{opt} \in \arg\min_{\pi} C(\pi). $$

It is known that the solution of this problem is a bang-bang strategy [9] (only vaccination rates 0 and $\theta$ will be used). A further analysis of the dynamics shows the following result:

**Proposition 1.** The strategy that minimizes the total cost is a threshold strategy: There exists a critical time $\tau^{opt}$ s.t.

$$ \pi^{opt} = \begin{cases} \theta & \text{if } t < \tau^{opt}, \\ 0 & \text{if } t > \tau^{opt}. \end{cases} $$
Proof. (Sketch) The proof is based on the Pontryagin maximum principle: If $\pi^{opt}$ is an optimal strategy, then there exist two Lagrange multipliers $\lambda_S(t)$ and $\lambda_I(t)$ such that $\lambda_S(T) = 0$, $\lambda_I(T) = 0$ and for any $t < T$,

$$ 
\begin{aligned}
-\dot{\lambda}_S &= c_V \pi^{opt}(t) + (-\gamma m_I^{opt}(t) - \pi^{opt}(t))\lambda_S + \gamma m_I^{opt}(t)\lambda_I \\
-\dot{\lambda}_I &= c_I - \gamma m_S^{opt}(t)\lambda_S + (\gamma m_S^{opt}(t) - \rho)\lambda_I \\
\pi^{opt}(t) &= \arg \min [c_V \pi(t)m_S^{opt}(t) + c_I m_I^{opt}(t) \\
&\quad + (\gamma m_S^{opt}(t)m_I^{opt}(t) - \pi(t)m_S^{opt}(t))\lambda_S \\
&\quad + (\gamma m_S^{opt}(t)m_I^{opt}(t) - \rho m_I^{opt}(t))\lambda_I],
\end{aligned}
$$

where $m_I^{opt}(t)$, $m_S^{opt}(t)$ are the proportions of the population in states I and S respectively, at time t, under the optimal strategy. By straightforward simplifications, one gets

$$ -\dot{\lambda}_S = \inf_{\pi} (\pi(t)(c_V - \lambda_S) + \gamma m_I^{opt}(t)(\lambda_I - \lambda_S)) \quad (5) $$

$$ -\dot{\lambda}_I = c_I - \rho\lambda_I + \gamma m_S^{opt}(t)(\lambda_I - \lambda_S) \quad (6) $$

$$ \pi^{opt}(t) = \arg \min (\pi(t)m_S^{opt}(t)(c_V - \lambda_S)). \quad (7) $$

One can notice a similarity between Equations (5)-(6) for the Lagrange multipliers and Equations (2)-(3) for the costs of the best response.

This induces a similar shape for the strategy that optimizes the social cost: There exists a critical time $\tau^{opt}$ s.t.

$$ \pi^{opt}(t) = \theta \text{ if } t < \tau^{opt} \text{ and } 0 \text{ if } t > \tau^{opt}. $$

**Proposition 2.** The threshold of the optimal social cost is larger than the threshold of the mean field equilibrium: $\tau^{opt} \ge \tau^{MFE}$.

Proof. Again, the proof is based on the fact that Equations (5)-(6) and Equations (2)-(3) are similar up to the additional term $\gamma m_S^{opt}(t)(\lambda_I - \lambda_S)$ for $\lambda_I$. Using this, the comparison between the optimal strategy and the mean field equilibrium bowls down to the comparisons of the Lagrange Multipliers $\lambda_S$, $\lambda_I$ and the costs $J_S$, $J_I$.

One easy case is when $c_V$ is larger than $c_I/\rho$. In this case, for all $t$, $\lambda_S \le \lambda_I \le c_I/\rho \le c_V$ so that the jump time of the mean field equilibrium is $\tau^{eq} = 0$. Therefore, the socially optimal jump time $\tau^{opt}$ can only be larger than $\tau^{eq}$.

Let us now consider the case when $c_V < c_I/\rho$. In this case, $\tau^{opt}$ is the time when $\lambda_S$ gets below $c_V$. By examining the Lagrange multipliers $\lambda_S$ and $\lambda_I$ between $\tau^{opt}$ and $T$, one can show that they must satisfy the following properties:

$$ - \lambda_S(T) = 0, \lambda_I(T) = 0, $$

$$ - \forall t \in [\tau^{opt}, T], \lambda_S(t) \le \lambda_I(t). $$

Indeed, if there is a time $t$ such that $\lambda_S(t) = \lambda_I(t)$, then their derivatives become comparable ($\dot{\lambda}_S(t) \le \dot{\lambda}_I(t)$). Therefore, the additional term $\gamma m_S^{opt}(t)(\lambda_I - \lambda_S)$
$\lambda_S$) in (5) remains positive so that $\lambda_I(t) \le J_I(t)$, $\forall \tau^{opt} \le t \le T$. In turn this implies that $\lambda_S(t) \ge J_S(t)$, $\forall \tau^{opt} \le t \le T$.

This implies that $J_S(\tau^{opt}) \le \lambda_S(\tau^{opt}) = c_V$. Finally, this implies that $\tau^{MFE}$ (the time when $J_S$ crosses level $c_V$) is smaller than $\tau^{opt}$, with equality only when $\tau^{opt} = 0$.

This result should be intuitive: Each member of the population has a personal incentive to stop vaccinating when the individual risk becomes low enough, while the social planner has a higher standard for the virus disappearance before deciding to stop the vaccination.

# 5 Numerical Comparisons

## 5.1 Comparison of MFE and Social Optimum

In this section, we report a numerical evaluation of the threshold of the two strategies<fnref n="4" />. We consider the same system parameters as in [3], which is based on the epidemiological study of the H1N1 epidemic of 2009-2010 in France : $\rho = 36.5$, $\gamma = 73$, $\theta = 10$, $c_I = 36.5$ and $c_V = 0.5$. Besides, we consider that the proportion of susceptible and infected population at time 0 are both equal to 0.4.

For these parameters, we compute the optimal strategy and the mean field equilibrium over a time horizon of a year, that is, $T = 1$. The results are reported in Figure 2a where the population state space is divided into three regions that represent the decisions taken by both strategies at time 0, as a function of the initial state. In the white region, both strategies vaccinate at maximum rate. In the dark gray region, the strategy of the social optimum is to vaccinate at maximum rate and the strategy of the equilibrium is to not vaccinate. In the light gray region, both strategies are to not vaccinate.

We also plot the trajectories corresponding to both strategies when the proportion of infected population and of susceptible population at time 0 are both equal to 0.4. In Figure 2a (see Figure 2b for a zoomed figure), we plot with a solid line the behavior of the equilibrium vaccination strategy, and with a dashed line, the behavior of the social optimum. The obtained cost for the equilibrium vaccination strategy for the parameters under consideration is 0.6824, whereas for the social optimum vaccination strategy is 0.6818.

## 5.2 Mechanism Design

For any vaccination cost $c_V$, while the other parameters remain fixed, we denote by $\tau^{opt}(c_V)$ (resp. $\tau^{MFE}(c_V)$) the jump time of the socially optimal strategy (resp. equilibrium strategy). It can be shown that in both cases, the jump times are decreasing in $c_V$: the more costly is the vaccination, the less people vaccinate

<footnote n="4">⁴ The codes to reproduce these experiments are available at https://github.com/josudoncel/MeanFieldGameAnalysisSIRModelVaccinations</footnote>
<figure><img src="image_2.png" /><figcaption>Fig. 2: Population dynamics under the equilibrium strategy (dashed line) and the socially optimal strategy (solid line). Three zones are displayed: (i) in the white region, the social optimum and the equilibrium vaccinate with maximum rate; (ii) in the dark gray region, the social optimum vaccinates with maximum rate, while the equilibrium does not vaccinate; and (iii) in the light gray region, neither the social optimum nor the equilibrium vaccinates. $m_I(0) = m_S(0) = 0.4$.</figcaption></figure>

(for the socially optimal situation as well as for the mean field equilibrium). Figure 3 confirms that the jump times decrease with $c_V$ and also shows that the jump times are never equal for this range of parameters.

Therefore, if the vaccination decisions are let to individuals, then vaccination should be subsidized, by offering a subsidy $g$ off the vaccination cost so that both jump times coincide, i.e.,

$$ \tau^{MFE}(c_V - g) = \tau^{opt}(c_V). $$

For example, with the same parameters as in the simulation of Figure 2, and for $c_V = 0.8$, the jump time of the social optimum is 0.0106, while the jump time of the equilibrium is 0. When $c_V = 0.65$, the jump time of the equilibrium is 0.034. This simulation shows that the subsidy required to encourage selfish individuals to vaccinate optimally consists of a reduction of the vaccination cost of $g = 0.15$.

## References

1. W. O. Kermack and A. G. McKendrick, “A contribution to the mathematical theory of epidemics,” in Proceedings of the Royal Society of London A: mathematical, physical and engineering sciences, vol. 115, no. 772. The Royal Society, 1927, pp. 700–721.

2. P. J. Francis, “Optimal tax/subsidy combinations for the flu season,” Journal of Economic Dynamics and Control, vol. 28, no. 10, pp. 2037–2054, 2004.
<figure><img src="image_3.png" /><figcaption>Fig. 3: Jump times comparison when $c_V$ varies from 0.01 to 1. The jump time of the mean field equilibrium (MFE) is represented with a solid line and the jump time of the social optimum (OPT) with a dotted line. The horizontal distance is the subsidy to be granted to incentive players to use the optimal vaccination strategy.</figcaption></figure>

3. L. Laguzet and G. Turinici, “Individual vaccination as Nash equilibrium in a SIR model: the interplay between individual optimization and societal policies,” 2015, hal-01100579, version 1. Available: https://hal.archives-ouvertes.fr/hal-01100579v1.

4. J. Doncel, N. Gast, and B. Gaujal, “Discrete Mean Field Games: Existence of Equilibria and Convergence,” *Journal of Dynamics and Games*, vol. 6, no. 3, pp. 1–19, 2019. [Online]. Available: https://hal.inria.fr/hal-01277098

5. D. A. Gomes, J. Mohr, and R. R. Souza, “Discrete time, finite state space mean field games,” *Journal de Mathématiques Pures et Appliquées*, vol. 93, no. 3, pp. 308–328, 2010. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S002178240900138X

6. L. F. Harko T. and M. M.K., “Exact analytical solutions of the susceptible-infected-recovered (sir) epidemic model and of the sir model with equal death and birth rate,” *Applied Mathematics and Computation*, no. 236, p. 184194, 2014.

7. J. Doncel, N. Gast, and B. Gaujal, “A Mean-Field Game Analysis of SIR Dynamics with Vaccination,” 2017, working paper or preprint. [Online]. Available: https://hal.inria.fr/hal-01496885

8. J.-M. Lasry and P.-L. Lions, “Mean field games,” *Japanese Journal of Mathematics*, vol. 2, no. 1, pp. 229–260, 2007.

9. H. Behncke, “Optimal control of deterministic epidemics,” *Optimal control applications and methods*, vol. 21, no. 6, pp. 269–285, 2000.