Title: Strategic Distribution Shift of Interacting Agents via Coupled Gradient Flows
Venue: NeurIPS 2023
Accepted: yes
Keywords: distribution shift

partial differential equations
Abstract: We propose a novel framework for analyzing the dynamics of distribution shift in real-world systems that captures the feedback loop between learning algorithms and the distributions on which they are deployed. Prior work largely models feedback-induced distribution shift as adversarial or via an overly simplistic distribution-shift structure. In contrast, we propose a coupled partial differential equation model that captures fine-grained changes in the distribution over time by accounting for complex dynamics that arise due to strategic responses to algorithmic decision-making, non-local endogenous population interactions, and other exogenous sources of distribution shift. We consider two common settings in machine learning: cooperative settings with information asymmetries, and competitive settings where a learner faces strategic users. For both of these settings, when the algorithm retrains via gradient descent, we prove asymptotic convergence of the retraining procedure to a steady-state, both in finite and in infinite dimensions, obtaining explicit rates in terms of the model parameters. To do so we derive new results on the convergence of coupled PDEs that extends what is known on multi-species systems. Empirically, we show that our approach captures well-documented forms of distribution shifts like polarization and disparate impacts that simpler models cannot capture.

Strategic Distribution Shift of Interacting Agents via Coupled Gradient Flows

Abstract

We propose a novel framework for analyzing the dynamics of distribution shift in real-world systems that captures the feedback loop between learning algorithms and the distributions on which they are deployed. Prior work largely models feedbackinduced distribution shift as adversarial or via an overly simplistic distribution-shift structure. In contrast, we propose a coupled partial differential equation model that captures fine-grained changes in the distribution over time by accounting for complex dynamics that arise due to strategic responses to algorithmic decision-making, non-local endogenous population interactions, and other exogenous sources of distribution shift. We consider two common settings in machine learning: cooperative settings with information asymmetries, and competitive settings where a learner faces strategic users. For both of these settings, when the algorithm retrains via gradient descent, we prove asymptotic convergence of the retraining procedure to a steady-state, both in finite and in infinite dimensions, obtaining explicit rates in terms of the model parameters. To do so we derive new results on the convergence of coupled PDEs that extends what is known on multi-species systems. Empirically, we show that our approach captures well-documented forms of distribution shifts like polarization and disparate impacts that simpler models cannot capture.

1 Introduction

In many machine learning tasks, there are commonly sources of exogenous and endogenous distribution shift, necessitating that the algorithm be retrained repeatedly over time. Some of these shifts occur without the influence of an algorithm; for example, individuals influence each other to become more or less similar in their attributes, or benign forms of distributional shift occur [Qui+]. Other shifts, however, are in response to algorithmic decision-making. Indeed, the very use of a decision-making algorithm can incentivize individuals to change or mis-report their data to achieve desired outcomes- a phenomenon known in economics as Goodhart's law. Such phenomena have been empirically observed, a well-known example being in [CC11], where researchers observed a population in Columbia strategically mis-reporting data to game a poverty index score used for distributing government assistance. Works such as [Mil+20; Wil+21], which investigate the effects of distribution shift over time on a machine learning algorithm, point toward the need for evaluating the robustness of algorithms to distribution shifts. Many existing approaches for modeling distribution shift focus on simple metrics like optimizing over moments or covariates [DY10; LHL21; BBS09]. Other methods consider worst-case scenarios, as in distributionally robust optimization [AZ22; LFG22; DN21; Kuh+19]. However, when humans respond to algorithms, these techniques may not

be sufficient to holistically capture the impact an algorithm has on a population. For example, an algorithm that takes into account shifts in a distribution's mean might inadvertently drive polarization, rendering a portion of the population disadvantaged.

Motivated by the need for a more descriptive model, we present an alternative perspective which allows us to fully capture complex dynamics that might drive distribution shifts in real-world systems. Our approach is general enough to capture various sources of exogenous and endogenous distribution shift including the feedback loop between algorithms and data distributions studied in the literature on performative prediction [Per+20; IYZ21; Ray+22; Nar+22; MPZ21], the strategic interactions studied in strategic classification [Har+16; Don+18], and also endogenous factors like intra-population dynamics and distributional shifts. Indeed, while previous works have studied these phenomena in isolation, our method allows us to capture all of them as well as their interactions. For example, in [Zrn+21], the authors investigate the effects of dynamics in strategic classification problemsbut the model they analyze does not capture individual interactions in the population. In [IYZ21], the authors model the interaction between a population that repeatedly responds to algorithmic decision-making by shifting its mean. Additionally, [Ray+22] study settings in which the population has both exogenous and endogenous distribution shifts due to feedback, but much like the other cited work, the focus remains on average performance. Each of these works fails to account for diffusion or intra-population interactions that can result in important qualitative changes to the distribution.

Contributions. Our approach to this problem relies on a detailed non-local PDE model of the data distribution which captures each of these factors. One term driving the evolution of the distribution over time captures the response of the population to the deployed algorithm, another draws on models used in the PDE literature for describing non-local effects and consensus in biological systems to model intra-population dynamics, and the last captures a background source of distribution shift. This is coupled with an ODE, lifted to a PDE, which describes the training of a machine learning algorithm results in a coupled PDE system which we analyze to better understand the behaviors that can arise among these interactions.

In one subcase, our model exhibits a joint gradient flow structure, where both PDEs can be written as gradients flows with respect to the same joint energy, but considering infinite dimensional gradients with respect to the different arguments. This mathematical structure provides powerful tools for analysis and has been an emerging area of study with a relatively small body of prior work, none of which related to distribution shifts in societal systems, and a general theory for multi-species gradient flows is still lacking. We give a brief overview of the models that are known to exhibit this joint gradient flow structure: in [DS20] the authors consider a two-species tumor model with coupling through Brinkman's Law. A number of works consider coupling via convolution kernels [FF13; Giu+22; JPZ22; CHS18; DT20; Dou+23] and cross-diffusion [LY22; AB21; MKB14], with applications in chemotaxis among other areas. In the models we consider here, the way the interaction between the two populations manifests is neither via cross-diffusion, nor via the non-local self-interaction term. A related type of coupling has recently appeared in [HPS22; HPS23], however in the setting of graphs. Recent work [Dom+21] provides particle-based methods to approximately compute the solution to a minimax problem where the optimization space is over measures; following that work, [WC23] provides another particle-based method using mirror descent-ascent to solve a similar problem. Other recent work [Lu23] proves that a mean-field gradient ascent-descent scheme with an entropy annealing schedule converges to the solution of a minimax optimization problem with a timescale separation parameter that is also time-varying; in contrast, our work considers fixed timescale separation setting. [GG23] show that the mean-field description of a particle method for solving minimax problems has proveable convergence guarantees in the Wasserstein-Fisher-Rao metric. Each of these references considers an energy functional that is linear in the distribution of each species respectively; our energy includes nonlinearities in the distributions via a self-interaction term as well as diffusion for the population. Moreover, the above works introduce a gradient flow dynamic as a tool for obtaining and characterizing the corresponding steady states, whereas in our setting we seek to capture the time-varying behavior that models distributions shifts. In the other subcase, we prove exponential convergence in two competitive, timescale separated settings where the algorithm and strategic population have conflicting objectives. We show numerically that retraining in a competitive setting leads to polarization in the population, illustrating the importance of fine-grained modeling.

2 Problem Formulation

Machine learning algorithms that are deployed into the real world for decision-making often become part of complex feedback loops with the data distributions and data sources with which they interact. In an effort to model these interactions, consider a machine learning algorithm that has loss given by L ( z, x ) where x ∈ R d are the algorithm parameters and z ∈ R d are the population attributes, and the goal is to solve

where X is the class of model parameters and ρ ( z ) is the population distribution. Individuals have an objective given by J ( z, x ) in response to a model parameterized by x , and they seek to solve

When individuals in the population and the algorithm have access to gradients, we model the optimization process as a gradient-descent-type process. Realistically, individuals in the population will have nonlocal information and influences, as well as external perturbations, the effects of which we seek to capture in addition to just minimization. To address this, we propose a partial differential equation (PDE) model for the population, that is able to capture nonlocal interactions between individuals on the level of a collective population. To analyse how the population evolves over time, a notion of derivative in infinite dimensions is needed. A natural, and in this context physically meaningful, way of measuring the dissipation mechanism for probability distributions is the Wasserstein-2 metric (see Definition 4). The following expression appears when computing the gradient of an energy functional with respect to the Wasserstein-2 topology.

Definition 1. [First Variation] For a map G : P ( R d ) ↦→ R and fixed probability distribution ρ ∈ P ( R d ) , the first variation of G at the point ρ is denoted by δ ρ G [ ρ ] : R d → R , and is defined via the relation

for all ψ ∈ C ∞ c ( R d ) such that ∫ d ψ = 0 , assuming that G is regular enough for all quantities to exist.

Here, P ( R d ) denotes the space of probability measures on the Borel sigma algebra. Using the first variation, we can express the gradient in Wasserstein-2 space, see for example [Vil03, Exercise 8.8].

Lemma 1. The gradient of an energy G : P 2 ( R d ) → R in the Wasserstein-2 space is given by

Here, P 2 ( R d ) denotes the set of probability measures with bounded second moments, also see Appendix A.2. As a consequence, the infinite dimensional steepest descent in Wasserstein-2 space can be expressed as the PDE

All the coupled gradient flows considered in this work have this Wasserstein-2 structure. In particular, when considering that individuals minimize their own loss, we can capture these dynamics via a gradient flow in the Wasserstein-2 metric on the level of the distribution of the population. Then for given algorithm parameters x ∈ R d , the evolution for this strategic population is given by

where E ( ρ ) is a functional including terms for internal influences and external perturbations. In real-world deployment of algorithms, decision makers update their algorithm over time, leading to an interaction between the two processes. We also consider the algorithm dynamics over time, which we model as

In this work, we analyze the behavior of the dynamics under the following model. The algorithm suffers a cost f 1 ( z, x ) for a data point z under model parameters x in the strategic population, and a cost f 2 ( z, x ) for a data point in a fixed, non-strategic population. The strategic population is denoted by ρ ∈ P , and the non-strategic population by ¯ ρ ∈ P . The algorithm aims to minimize

where the norm is the vector inner product ∥ x ∥ 2 = ⟨ x, x ⟩ and β > 0 weights the cost of updating the model parameters from its initial condition.

We consider two settings: ( i ) aligned objectives, and ( ii ) competing objectives. Case ( i ) captures the setting in which the strategic population minimization improves the performance of the algorithm, subject to a cost for deviating from a reference distribution ˜ ρ ∈ P . This cost stems from effort required to manipulate features, such as a loan applicant adding or closing credit cards. On the other hand, Case ( ii ) captures the setting in which the strategic population minimization worsens the performance of the algorithm, again incurring cost from distributional changes.

2.1 Case (i): Aligned Objectives

In this setting, we consider the case where the strategic population and the algorithm have aligned objectives. This occurs in examples such as recommendation systems, where users and algorithm designers both seek to develop accurate recommendations for the users. This corresponds to the population cost

where KL ( · | · ) denotes the Kullback-Leibler divergence. Note that the KL divergence introduces diffusion to the dynamics for ρ . The weight α > 0 parameterizes the cost of distribution shift to the population. To account for nonlocal information and influence among members of the population, we include a kernel term E ( ρ ) = 1 2 ∫ ρW ∗ ρ d z , where ( W ∗ ρ )( z ) = ∫ W ( z -¯ z )d ρ (¯ z ) is a convolution integral and W is a suitable interaction potential.

2.2 Case (ii): Competing Objectives

In settings such as online internet forums, where algorithms and users have used manipulative strategies for marketing [Del06], the strategic population may be incentivized to modify or mis-report their attributes. The algorithm has a competitive objective, in that it aims to maintain performance against a population whose dynamics cause the algorithm performance to suffer. When the strategic population seeks an outcome contrary to the algorithm, we model strategic population cost as

A significant factor in the dynamics for the strategic population is the timescale separation between the two "species"-i.e., the population and the algorithm. In our analysis, we will consider two cases: one, where the population responds much faster than the algorithm, and two, where the algorithm responds much faster than the population. We illustrate the intermediate case in a simulation example.

3 Results

We are interested in characterizing the long-time asymptotic behavior of the population distribution, as it depends on the decision-makers action over time. The structure of the population distribution gives us insights about how the decision-makers actions influences the entire population of users. For instance, as noted in the preceding sections, different behaviors such as bimodal distributions or large tails or variance might emerge, and such effects are not captured in simply looking at average performance. To understand this intricate interplay, one would like to characterize the behavior of both the population and the algorithm over large times. Our main contribution towards this goal is a novel analytical framework as well as analysis of the long-time asymptotics.

A key observation is that the dynamics in (2) and (3) can be re-formulated as a gradient flow; we lift x to a probability distribution µ by representing it as a Dirac delta µ sitting at the point x . As a result, the evolution of µ will be governed by a PDE, and combined with the PDE for the population, we obtain a system of coupled PDEs,

where δ ρ and δ µ are first variations with respect to ρ and µ according to Definition 1. The natural candidates for the asymptotic profiles of this coupled system are its steady states, which - thanks to the gradient flow structure - can be characterized as ground states of the corresponding energy functionals. In this work, we show existence and uniqueness of minimizers (maximizers) for the functionals under suitable conditions on the dynamics. We also provide criteria for convergence and explicit convergence rates. We begin with the case where the interests of the population and algorithm are aligned, and follow with analogous results in the competitive setting. We show convergence in energy, which in turn ensures convergence in a product Wasserstein metric. For convergence in energy, we use the notion of relative energy and prove that the relative energy converges to zero as time increases.

Definition 2 (Relative Energy) . The relative energy of a functional G is given by G ( γ | γ ∞ ) = G ( γ ) -G ( γ ∞ ) , where G ( γ ∞ ) is the energy at the steady state.

Since we consider the joint evolution of two probability distributions, we define a distance metric W on the product space of probability measures with bounded second moment.

Definition 3 (Joint Wasserstein Metric) . The metric over P 2 ( R d ) ×P 2 ( R d ) is called W and is given by

for all pairs ( ρ, µ ) , (˜ ρ, ˜ µ ) ∈ P 2 ( R d ) ×P 2 ( R d ) , and where W 2 denotes the Wasserstein-2 metric (see Definition 4). We denote by W ( R d ) := ( P 2 ( R d ) ×P 2 ( R d ) , W ) the corresponding metric space.

3.1 Gradient Flow Structure

In the case where the objectives of the algorithm and population are aligned , we can write the dynamics as a gradient flow by using the same energy functional for both species. Let G a ( ρ, µ ) : P ( R d ) ×P ( R d ) ↦→ [0 , ∞ ] be the energy functional given by

This expression is well-defined as the relative entropy KL ( ρ | ˜ ρ ) can be extended to the full set P ( R d ) by setting G a ( ρ, µ ) = + ∞ in case ρ is not absolutely continuous with respect to ˜ ρ .

In the competitive case we define G c ( ρ, x ) : P ( R d ) × R d ↦→ [ -∞ , ∞ ] by

In settings like recommender systems, the population and algorithm have aligned objectives; they seek to minimize the same cost but are subject to different dynamic constraints and influences, modeled by the regularizer and convolution terms. In the case where the objectives are aligned, the dynamics are given by

Note that (4) is a joint gradient flow, because the dynamics can be written in the form

where γ = ( ρ, µ ) and where the gradient and divergence are taken in both variables ( z, x ) . We discuss the structure of the dynamics (4) as well as the meaning of the different terms appearing in the energy functional G a in Appendix A.1.

In other settings, such as credit score reporting, the objectives of the population are competitive with respect to the algorithm. Here we consider two scenarios; one, where the algorithm responds quickly relative to the population, and two, where the population responds quickly relative to the algorithm. In the case where the algorithm can immediately adjust optimally (best-respond) to the distribution, the dynamics are given by

Next we can consider the population immediately responding to the algorithm, which has dynamics

In this time-scale separated setting, model (5) represents a dyamic maximization of G c with respect to ρ in Wasserstein-2 space, and an instantaneous minimization of G c with respect to the algorithm parameters x . Model (6) represents an instantaneous maximization of G c with respect to ρ and a dynamic minimization of G c with respect to the algorithm parameters x . The key results on existence and uniqueness of a ground state as well as the convergence behavior of solutions depend on convexity (concavity) of G a and G c . The notion of convexity that we will employ for energy functionals in the Wasserstein-2 geometry is (uniform) displacement convexity , which is analogous to (strong) convexity in Euclidean spaces. One can think of displacement convexity for an energy functional defined on P 2 as convexity along the shortest path in the Wasserstein-2 metric (linear interpolation in the Wasserstein-2 space) between any two given probability distributions. For a detailed definition of (uniform) displacement convexity and concavity, see Section A.2. In fact, suitable convexity properties of the input functions f 1 , f 2 , W and ˜ ρ will ensure (uniform) displacement convexity of the resulting energy functionals appearing in the gradient flow structure, see for instance [Vil03, Chapter 5.2].

We make the following assumptions in both the competitive case and aligned interest cases. Here, I d denotes the d × d identity matrix, Hess ( f ) denotes the Hessian of f in all variables, while ∇ 2 x f denotes the Hessian of f in the variable x only.

Assumption 1 (Convexity of f 1 and f 2 ) . The functions f 1 , f 2 ∈ C 2 ( R d × R d ; [0 , ∞ )) satisfy for all ( z, x ) ∈ R d × R d the following:

There exists constants λ 1 , λ 2 ≥ 0 such that Hess ( f 1 ) ⪰ λ 1 I 2 d and ∇ 2 x f 2 ⪰ λ 2 I d ;

There exist constants a i > 0 such that x · ∇ x f i ( z, x ) ≥ -a i for i = 1 , 2 ;

Assumption 2 (Reference Distribution Shape) . The reference distribution ˜ ρ ∈ P ( R d ) ∩ L 1 ( R d ) satisfies log ˜ ρ ∈ C 2 ( R d ) and ∇ 2 z log ˜ ρ ( z ) ⪯ -˜ λ I d for some ˜ λ > 0 .

Assumption 3 (Convex Interaction Kernel) . The interaction kernel W ∈ C 2 ( R d ; [0 , ∞ )) is convex, symmetric W ( -z ) = W ( z ) , and for some D > 0 satisfies

We make the following observations regarding the assumptions above:

The convexity in Assumption 3 can be relaxed and without affecting the results outlined below by following a more detailed analysis analogous to the approach in [CMV03].

If f 1 and f 2 are strongly convex, the proveable convergence rate increases, but without strict or strong convexity of f 1 and f 2 , the regularizers KL ( ρ | ˜ ρ ) and ∫ ∥ x -x 0 ∥ 2 2 d x provide the convexity guarantees necessary for convergence.

For concreteness, one can consider the following classical choices of input functions to the evolution:

Using the log-loss function for f 1 and f 2 satisfies Assumption 1.

Taking the reference measure ˜ ρ to be the normal distribution satisfies Assumption 2, which ensures the distribution is not too flat.

Taking quadratic interactions W ( z ) = 1 2 | z | 2 satisfies Assumption 3.

Remark 1 (Cauchy-Problem) . To complete the arguments on convergence to equilibrium, we require sufficient regularity of solutions to the PDEs under consideration. In fact, it is sufficient if we can show that equations (4) , (5) , and (6) can be approximated by equations with smooth solutions. Albeit tedious, these are standard techniques in the regularity theory for partial differential equations, see for example [CMV03, Proposition 2.1 and Appendix A], [OV00], [Vil03, Chapter 9], and the references therein. Similar arguments as in [DV00] are expected to apply to the coupled gradient flows considered here, guaranteeing existence of smooth solutions with fast enough decay at infinity, and we leave a detailed proof for future work.

3.2 Analysis of Case (i): Aligned Objectives

The primary technical contribution of this setting consists of lifting the algorithm dynamics from an ODE to a PDE, which allows us to model the system as a joint gradient flow on the product space of probability measures. The coupling occurs in the potential function, rather than as cross-diffusion or non-local interaction as more commonly seen in the literature for multi-species systems.

Theorem 2. Suppose that Assumptions 1-3 are satisfied and let λ a := λ 1 +min( λ 2 + β, α ˜ λ ) > 0 . Consider solutions γ t := ( ρ t , µ t ) to the dynamics (4) with initial conditions satisfying γ 0 ∈ P 2 ( R d ) × P 2 ( R d ) and G a ( γ 0 ) < ∞ . Then the following hold:

(a) There exists a unique minimizer γ ∞ = ( ρ ∞ , µ ∞ ) of G a , which is also a steady state for equation (4) . Moreover, ρ ∞ ∈ L 1 ( R d ) , has the same support as ˜ ρ , and its density is continuous.

(b) The solution γ converges exponentially fast in G ( · | γ ) and W ,

t a ∞ G a ( γ t | γ ∞ ) ≤ e -2 λ a t G a ( γ 0 | γ ∞ ) and W ( γ t , γ ∞ ) ≤ ce -λ a t for all t ≥ 0 , where c > 0 is a constant only depending on γ 0 , γ ∞ and the parameter λ a .

Proof. (Sketch) For existence and uniqueness, we leverage classical techniques in the calculus of variations. To obtain convergence to equilibrium in energy, our key result is a new HWI-type inequality, providing as a consequence generalizations of the log-Sobolev inequality and the Talagrand inequality. Together, these inequalities relate the energy (classically denoted by H in the case of the Boltzmann entropy), the metric (classically denoted by W in the case of the Wasserstein-2 metric) and the energy dissipation (classically denoted by I in the case of the Fisher information) 1 . Combining these inequalities with Gronwall's inequality allows us to deduce convergence both in energy and in the metric W .

3.3 Analysis of Case (ii): Competing Objectives

In this setting, we consider the case where the algorithm and the strategic population have goals in opposition to each other; specifically, the population benefits from being classified incorrectly. First, we will show that when the algorithm instantly best-responds to the population, then the distribution of the population converges exponentially in energy and in W 2 . Then we will show a similar result for the case where the population instantly best-responds to the algorithm.

In both cases, we begin by proving two Danskin-type results (see [Dan67; Ber71]) which will be used for the main convergence theorem, including convexity (concavity) results. To this end, we make the following assumption ensuring that the regularizing component in the evolution of ρ is able to control the concavity introduced by f 1 and f 2 .

Assumption 4 (Upper bounds for f 1 and f 2 ) . There exists a constant Λ 1 > 0 such that

and for any R > 0 there exists a constant c 2 = c 2 ( R ) ∈ R such that

Equipped with Assumption 4, we state the result for a best-responding algorithm.

Theorem 3. Suppose Assumptions 1-4 are satisfied with α ˜ λ > Λ 1 . Let λ b := α ˜ λ -Λ 1 . Define G b ( ρ ) := G c ( ρ, b ( ρ )) . Consider a solution ρ t to the dynamics (5) with initial condition ρ 0 ∈ P 2 ( R d ) such that G b ( ρ 0 ) < ∞ . Then the following hold:

(a) There exists a unique maximizer ρ ∞ of G b ( ρ ) , which is also a steady state for equation (5) . Moreover, ρ ∞ ∈ L 1 ( R d ) , has the same support as ˜ ρ , and its density is continuous.

(b) The solution ρ t converges exponentially fast to ρ ∞ with rate λ b in G b ( · | ρ ∞ ) and W 2 ,

where c > 0 is a constant only depending on ρ 0 , ρ ∞ and the parameter λ b .

Proof. (Sketch) The key addition in this setting as compared with Theorem 2 is proving that G b ( ρ ) is bounded below, uniformly displacement concave and guaranteeing its smoothness via Berge's Maximum Theorem. This is non-trivial as it uses the properties of the best response b ( ρ ) . A central observation for our arguments to work is that δ ρ G b [ ρ ] = ( δ ρ G c [ ρ, x ]) | x = b ( ρ ) . We can then conclude using the direct method in the calculus of variations and the HWI method.

Here, the condition that α ˜ λ must be large enough corresponds to the statement that the system must be subjected to a strong enough regularizing effect.

In the opposite case, where ρ instantly best-responds to the algorithm, we show Danskin-like results for derivatives through the best response function and convexity of the resulting energy in x which allows to deduce convergence.

Theorem 4. Suppose Assumptions 1-4 are satisfied with α ˜ λ > Λ 1 , and that r ( x ) is differentiable (as shown by example conditions in Lemmas 27 and 28). Define G d ( x ) := G c ( r ( x ) , x ) . Then it holds:

(a) There exists a unique minimizer x ∞ of G d ( x ) which is also a steady state for (6) .

(b) The vector x ( t ) solving the dynamics (6) with initial condition x (0) ∈ R d converges exponentially fast to x ∞ with rate λ d := λ 1 + λ 2 + β > 0 in G d and in the Euclidean norm:

These two theorems illustrate that, under sufficient convexity conditions on the cost functions, we expect the distribution ρ and the algorithm x to converge to a steady state. In practice, when the distributions are close enough to the steady state there is no need to retrain the algorithm.

While we have proven results for the extreme timescale cases, we anticipate convergence to the same equilibrium in the intermediate cases. Indeed, it is well known [Bor09] (especially for systems in Euclidean space) that for two-timescale stochastic approximations of dynamical systems, with appropriate stepsize choices, converge asymptotically, and finite-time high probability concentration bounds can also be obtained. These results have been leveraged in strategic classification [Zrn+21] and Stackelberg games [FCR20; FR21; Fie+21]. We leave this intricate analysis to future work.

In the following section we show numerical results in the case of a best-responding x , best-responding ρ , and in between where x and ρ evolve on a similar timescale. Note that in these settings, the dynamics do not have a gradient flow structure due to a sign difference in the energies, requiring conditions to ensure that one species does not dominate the other.

4 Numerical Examples

We illustrate numerical results for the case of a classifier, which are used in scenarios such as loan or government aid applications [CC11], school admissions [PS13], residency match [Ree18], and recommendation algorithms [LSW10], all of which have some population which is incentivized to submit data that will result in a desirable classification. For all examples, we select classifiers

of the form x ∈ R , so that a data point z ∈ R is assigned a label of 1 with probability q ( z, x ) = (1 + exp ( -b ⊤ z + x )) -1 where b > 0 . Let f 1 and f 2 be given by

Note that Hess ( f 1 ) ⪰ 0 and ∇ 2 x f 2 ⪰ 0 , so λ 1 = λ 2 = 0 . Here, the strictness of the convexity of the functional is coming from the regularizers, not the cost functions, with ˜ ρ a scaled normal distribution. We show numerical results for two scenarios with additional settings in the appendix. First we illustrate competitive interests under three different timescale settings. Then we simulate the classifier taking an even more naïve strategy than gradient descent and discuss the results. The PDEs were implemented based on the finite volume method from [CCH15].

4.1 Competitive Objectives

In the setting with competitive objectives, we utilize G c ( ρ, x ) with W = 0 , f 1 and f 2 as defined above with b = 3 fixed as it only changes the steepness of the classifier for d = 1 , and α = 0 . 1 and β = 0 . 05 . In Figure 1, we simulate two extremes of the timescale setting; first when ρ is nearly best-responding and then when x is best-responding. The simulations have the same initial conditions and end with the same distribution shape; however, the behavior of the strategic population differs in the intermediate stages. When ρ is nearly best-responding, we see that the distribution quickly shifts

pbest-responding

0.75

t=0.0

p

t=1.0

p

t=1.7

-Q

t=5.0

p

obabil

0.50

p

p

p

p

0.25

X

od

od

od

prol

0.00

-2

0

2

4

2

0

2

4

-2

0

2

4

2

0

2

4

Z

Z

Z

Z

xbest-responding

lity

0.75

t=0.0

p

t=3.0

p

t=5.0

p

t=15.0

p

obabil

0.50

d

d

p

p

0.25

od

d

od

prob

0.00

-2

0

2

4

-2

0

2

4

-2

0

2

4

-2

0

2

4

Z

Z

Z

Z

mass over the classifier threshold. Then the classifier shifts right, correcting for the shift in ρ , which then incentivizes ρ to shift more mass back to the original mode. In contrast, when x best-responds, the right-hand mode slowly increases in size until the system converges.

Figure 2 shows simulation results from the setting where ρ and x evolve on the same timescale. We observe that the distribution shift in ρ appears to fall between the two extreme timescale cases, which we expect. We highlight two important observations for the competitive case. One, a single-mode

pandxatsimilarrates

probability

0.75

t=0.0

-Q

t=10.0

p

t=16.7

p

t=50.0

p

0.50

d

d

d

d

0.25

od

od

od

0.00

2

0

2

4

-2

0

2

4

-2

0

2

4

-2

0

2

4

Z

Z

Z

Z

distribution becomes bimodel, which would not be captured using simplistic metrics such as the mean and variance. This split can be seen as polarization in the population, a phenomenon that a mean-based strategic classification model would not capture. Two, the timescale on which the

classifier updates significantly impacts the intermediate behavior of the distribution. In our example, when x updated slowly relative to the strategic population, the shifts in the population were greater than in the other two cases. This suggests that understanding the effects of timescale separation are important for minimizing volatility of the coupled dynamics.

4.2 Naïve Behavior

In this example, we explore the results of the classifier adopting a non-gradient-flow strategy, where the classifier chooses an initially-suboptimal value for x and does not move, allowing the strategic population to respond. All functions and parameters are the same as in the previous example. When

Classifier Loss

SS07

-1.8

1.9

Population Loss

1.0

SS07

0.8

0

10

20

30

40

50

time

gradient descent

naive

od

0.6

p

probability

p

0.4

0.2

0.0

-2

0

2

4

Z

Figure 3: Although the classifier starts with a larger cost by taking the naive strategy, the final loss is better. This illustrates how our model can be used to compare robustness of different strategies against a strategic population.

comparing with the gradient descent strategy, we observe that while the initial loss for the classifier is worse for the naive strategy, the final cost is better. While this results is not surprising, because one can view this as a general-sum game where the best response to a fixed decision may be better than the equilibrium, it illustrates how our method provides a framework for evaluating how different training strategies perform in the long run against a strategic population.

5 Future Directions, Limitations, and Broader Impact

Our work presents a method for evaluating the robustness of an algorithm to a strategic population, and investigating a variety of robustness using our techniques opens a range of future research directions. Our application suggests many questions relevant to the PDE literature, such as: (1) Does convergence still hold with the gradient replaced by an estimated gradient? (2) Can we prove convergence in between the two timescale extremes? (3) How do multiple dynamic populations respond to an algorithm, or multiple algorithms? In the realm of learning algorithms, our framework can be extended to other learning update strategies and presents a way to model how we can design these update strategies to induce desired behaviors in the population.

A challenge in our method is that numerically solving high-dimensional PDEs is computationally expensive and possibly unfeasible. Here we note that in many applications, agents in the population do not alter more than a few features due to the cost of manipulation. We are encouraged by the recent progress using deep learning to solve PDEs, which could be used in our application.

Broader Impacts Modeling the full population distribution rather than simple metrics of the distribution is important because not all individuals are affected by the algorithm in the same way. For example, if there are tails of the distribution that have poor performance even if on average the model is good, we need to know how that group is advantaged or disadvantaged relative to the rest of the population. Additionally, understanding how people respond to algorithms offers an opportunity to incentivise people to move in a direction that increases social welfare.

Acknowledgments and Disclosure of Funding

LC is supported by an NDSEG fellowhip from the Air Force Office of Scientific Research. FH is supported by start-up funds at the California Institute of Technology. LR is supported by ONR YIP N00014-20-1-2571 P00003 and NSF Awards CAREER 1844729, and CPS 1931718. EM acknowledges support from NSF Award 2240110. We are grateful for helpful discussions with José A. Carrillo.

References

[Ala40] Leon Alaoglu. 'Weak Topologies of Normed Linear Spaces'. In: The Annals of Mathematics 41.1 (Jan. 1940), p. 252. ISSN: 0003486X. DOI: 10.2307/1968829 .

[Dan67] John M. Danskin. The Theory of Max-Min and its Application to Weapons Allocation Problems . Red. by M. Beckmann et al. Vol. 5. Ã-konometrie und Unternehmensforschung / Econometrics and Operations Research. Berlin, Heidelberg: Springer, 1967. ISBN: 9783642460944 9783642460920. DOI: 10.1007/978-3-64246092-0 .

[Ber71] Dimitri P. Bertsekas. 'Control of Uncertain Systems with a Set-Membership Description of Uncertainty'. PhD thesis. Cambridge, MA, USA: MIT, 1971.

[Pos75] E. Posner. 'Random coding strategies for minimum entropy'. In: IEEE Transactions on Information Theory 21.4 (July 1975), pp. 388-391. ISSN: 1557-9654. DOI: 10.1109/ TIT.1975.1055416 .

[Lie83] Elliott H. Lieb. 'Sharp Constants in the Hardy-Littlewood-Sobolev and Related Inequalities'. In: Annals of Mathematics 118.2 (1983), pp. 349-374. ISSN: 0003-486X. DOI: 10.2307/2007032 .

[Rud91] Walter Rudin. Functional analysis . 2nd ed. International series in pure and applied mathematics. New York: McGraw-Hill, 1991. 424 pp. ISBN: 9780070542365.

[McC97] Robert J. McCann. 'A Convexity Principle for Interacting Gases'. In: Advances in Mathematics 128.1 (June 1, 1997), pp. 153-179. ISSN: 0001-8708. DOI: 10.1006/ aima.1997.1634 .

[BB00] Jean-David Benamou and Yann Brenier. 'A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem'. In: Numerische Mathematik 84.3 (Jan. 1, 2000), pp. 375-393. ISSN: 0945-3245. DOI: 10.1007/s002110050002 .

[DV00] Laurent Desvillettes and Cedric Villani. 'On the spatially homogeneous landau equation for hard potentials part i : existence, uniqueness and smoothness'. In: Communications in Partial Differential Equations 25.1 (Jan. 1, 2000), pp. 179-259. ISSN: 0360-5302. DOI: 10.1080/03605300008821512 .

[OV00] F. Otto and C. Villani. 'Generalization of an Inequality by Talagrand and Links with the Logarithmic Sobolev Inequality'. In: Journal of Functional Analysis 173.2 (June 1, 2000), pp. 361-400. ISSN: 0022-1236. DOI: 10.1006/jfan.1999.3557 .

[Gho02] B. K. Ghosh. 'Probability Inequalities Related to Markov's Theorem'. In: The American Statistician 56.3 (2002), pp. 186-190. ISSN: 00031305.

[CMV03] José A. Carrillo, Robert J. McCann, and Cédric Villani. 'Kinetic equilibration rates for granular media and related equations: entropy dissipation and mass transportation estimates'. In: Revista Matemática Iberoamericana 19.3 (Dec. 2003), pp. 971-1018. ISSN: 0213-2230.

[Vil03] Cédric Villani. Topics in optimal transportation . Vol. 58. Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2003, pp. xvi+370. ISBN: 0-8218-3312-X. DOI: 10.1090/gsm/058 .

[Ste04] J. Michael Steele. The Cauchy-Schwarz Master Class: An Introduction to the Art of Mathematical Inequalities . Cambridge University Press, Apr. 26, 2004. 320 pp. ISBN: 9780521546775.

[AB06] 'Correspondences'. In: Infinite Dimensional Analysis: A Hitchhiker's Guide . Ed. by Charalambos D. Aliprantis and Kim C. Border. Berlin, Heidelberg: Springer, 2006, pp. 555-590. ISBN: 9783540295877. DOI: 10.1007/3-540-29587-9_17 .

[CMV06] José A. Carrillo, Robert J. McCann, and Cedric Villani. 'Contractions in the 2Wasserstein Length Space and Thermalization of Granular Media'. In: Archive for Rational Mechanics and Analysis 179.2 (Feb. 1, 2006), pp. 217-263. ISSN: 1432-0673. DOI: 10.1007/s00205-005-0386-1 .

[Del06] Chrysanthos Dellarocas. 'Strategic Manipulation of Internet Opinion Forums: Implications for Consumers and Firms'. In: Management Science 52.10 (Oct. 2006), pp. 15771593. ISSN: 0025-1909, 1526-5501. DOI: 10.1287/mnsc.1060.0567 .

[De 07] Camillo De Lellis. 'Ordinary Differential Equations with Rough Coefficients and the Renormalization Theorem of Ambrosio'. In: Séminaire Bourbaki . Vol. 59. 972. Mar. 2007, pp. 175-204.

[BCL09] Andrea L. Bertozzi, Jose A. Carrillo, and Thomas Laurent. 'Blow-up in multidimensional aggregation equations with mildly singular interaction kernels*'. In: Nonlinearity 22.3 (Feb. 2009), p. 683. ISSN: 0951-7715. DOI: 10.1088/0951-7715/22/3/009 .

[BBS09] Steffen Bickel, Michael Brückner, and Tobias Scheffer. 'Discriminative Learning Under Covariate Shift'. In: The Journal of Machine Learning Research 10 (Dec. 1, 2009), pp. 2137-2155. ISSN: 1532-4435.

[Bor09] Vivek S Borkar. Stochastic approximation: a dynamical systems viewpoint . Vol. 48. Springer, 2009.

[DY10] Erick Delage and Yinyu Ye. 'Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems'. In: Operations Research 58.3 (June 2010), pp. 595-612. ISSN: 0030-364X, 1526-5463. DOI: 10.1287/opre.1090. 0741 .

[LSW10] Juan Lang, Matt Spear, and S. Felix Wu. 'Social Manipulation of Online Recommender Systems'. In: Social Informatics . Ed. by Leonard Bolc, Marek Makowski, and Adam Wierzbicki. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer, 2010, pp. 125-139. ISBN: 9783642165672. DOI: 10.1007/978-3-642-16567-2_10 .

[CC11] Adriana Camacho and Emily Conover. 'Manipulation of Social Program Eligibility'. In: American Economic Journal: Economic Policy 3.2 (2011), pp. 41-65. ISSN: 1945-7731.

[Car+11] J. A. Carrillo et al. 'Global-in-time weak measure solutions and finite-time aggregation for nonlocal interaction equations'. In: Duke Mathematical Journal 156.2 (Feb. 1, 2011). ISSN: 0012-7094. DOI: 10.1215/00127094-2010-211 .

[BCY12] D. Balagu'e, J. Carrillo, and Y. Yao. 'Confinement for repulsive-attractive kernels'. In: Discrete and Continuous Dynamical Systems-series B (2012).

[BLL12] Andrea L. Bertozzi, Thomas Laurent, and Flavien Lager. 'Aggregation and spreading via the newtonian potential: the dynamics of patch solutions'. In: Mathematical Models and Methods in Applied Sciences 22 (supp01 Apr. 2012), p. 1140005. ISSN: 0218-2025. DOI: 10.1142/S0218202511400057 .

[FF13] Marco Di Francesco and Simone Fagioli. 'Measure solutions for non-local interaction PDEs with two species'. In: Nonlinearity 26.10 (Oct. 1, 2013), pp. 2777-2808. ISSN: 0951-7715, 1361-6544. DOI: 10.1088/0951-7715/26/10/2777 .

[PS13] Parag A. Pathak and Tayfun Sönmez. 'School Admissions Reform in Chicago and England: Comparing Mechanisms by Their Vulnerability to Manipulation'. In: American Economic Review 103.1 (Feb. 2013), pp. 80-106. ISSN: 0002-8282. DOI: 10.1257/ aer.103.1.80 .

[CCH14] José Antonio Carrillo, Michel Chipot, and Yanghong Huang. 'On global minimizers of repulsive-attractive power-law interaction energies'. In: Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 372.2028 (Nov. 13, 2014), p. 20130399. ISSN: 1364-503X, 1471-2962. DOI: 10.1098/rsta. 2013.0399 .

[MKB14] Alan Mackey, Theodore Kolokolnikov, and Andrea L. Bertozzi. 'Two-species particle aggregation and stability of co-dimension one solutions'. In: Discrete and Continuous Dynamical Systems 19.5 (2014), pp. 1411-1436.

[CCH15] José A. Carrillo, Alina Chertock, and Yanghong Huang. 'A Finite-Volume Method for Nonlinear Nonlocal Equations with a Gradient Flow Structure'. In: Communications in Computational Physics 17.1 (Jan. 2015), pp. 233-258. ISSN: 1815-2406, 1991-7120. DOI: 10.4208/cicp.160214.010814a .

[San15] Filippo Santambrogio. 'Optimal Transport for Applied Mathematicians. Calculus of Variations, PDEs and Modeling'. In: (2015).

[Har+16] Moritz Hardt et al. 'Strategic Classification'. In: Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science . ITCS '16. New York, NY, USA: Association for Computing Machinery, Jan. 14, 2016, pp. 111-122. ISBN: 9781450340571. DOI: 10.1145/2840728.2840730 .

[CHS18] José A. Carrillo, Yanghong Huang, and Markus Schmidtchen. 'Zoology of a Nonlocal Cross-Diffusion Model for Two Species'. In: SIAM Journal on Applied Mathematics 78.2 (Jan. 2018), pp. 1078-1104. ISSN: 0036-1399, 1095-712X. DOI: 10.1137/ 17M1128782 .

[Don+18] Jinshuo Dong et al. 'Strategic classification from revealed preferences'. In: Proceedings of the 2018 ACM Conference on Economics and Computation . 2018, pp. 55-70.

[Ree18] Alex Rees-Jones. 'Suboptimal behavior in strategy-proof mechanisms: Evidence from the residency match'. In: Games and Economic Behavior . Special Issue in Honor of Lloyd Shapley: Seven Topics in Game Theory 108 (Mar. 1, 2018), pp. 317-330. ISSN: 0899-8256. DOI: 10.1016/j.geb.2017.04.011 .

[Kuh+19] Daniel Kuhn et al. 'Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning'. In: Operations Research & Management Science in the Age of Analytics . Ed. by Serguei Netessine, Douglas Shier, and Harvey J. Greenberg. INFORMS, Oct. 2019, pp. 130-166. ISBN: 9780990615330. DOI: 10.1287/educ. 2019.0198 .

[DS20] Tomasz Debiec and Markus Schmidtchen. 'Incompressible Limit for a Two-Species Tumour Model with Coupling Through Brinkman's Law in One Dimension'. In: Acta Applicandae Mathematicae 169.1 (Oct. 1, 2020), pp. 593-611. ISSN: 1572-9036. DOI: 10.1007/s10440-020-00313-1 .

[DT20] Manh Hong Duong and Julian Tugaut. 'Coupled McKean-Vlasov diffusions: wellposedness, propagation of chaos and invariant measures'. In: Stochastics 92.6 (Aug. 17, 2020), pp. 900-943. ISSN: 1744-2508. DOI: 10.1080/17442508.2019.1677663 .

[FCR20] Tanner Fiez, Benjamin Chasnov, and Lillian Ratliff. 'Implicit learning dynamics in stackelberg games: Equilibria characterization, convergence analysis, and empirical study'. In: International Conference on Machine Learning . PMLR. 2020, pp. 31333144.

[Mil+20] John Miller et al. 'The Effect of Natural Distribution Shift on Question Answering Models'. In: Proceedings of the 37th International Conference on Machine Learning . International Conference on Machine Learning. PMLR, Nov. 21, 2020, pp. 6905-6916.

[Per+20] Juan Perdomo et al. 'Performative Prediction'. In: Proceedings of the 37th International Conference on Machine Learning . International Conference on Machine Learning. PMLR, Nov. 21, 2020, pp. 7599-7609.

[AB21] Abdulaziz Alsenafi and Alethea B. T. Barbaro. 'A Multispecies Cross-Diffusion Model for Territorial Development'. In: Mathematics 9.12 (Jan. 2021), p. 1428. ISSN: 22277390. DOI: 10.3390/math9121428 .

[Dom+21] Carles Domingo-Enrich et al. A mean-field analysis of two-player zero-sum games . May 6, 2021. DOI: 10.48550/arXiv.2002.06277 . arXiv: 2002.06277[cs,math, stat] .

[DN21] John C. Duchi and Hongseok Namkoong. 'Learning models with uniform performance via distributionally robust optimization'. In: The Annals of Statistics 49.3 (June 2021), pp. 1378-1406. ISSN: 0090-5364, 2168-8966. DOI: 10.1214/20-AOS2004 .

[FR21] Tanner Fiez and Lillian J Ratliff. 'Local convergence analysis of gradient descent ascent with finite timescale separation'. In: Proceedings of the International Conference on Learning Representation . 2021.

[Fie+21] Tanner Fiez et al. 'Global convergence to local minmax equilibrium in classes of nonconvex zero-sum games'. In: Advances in Neural Information Processing Systems 34 (2021), pp. 29049-29063.

[IYZ21] Zachary Izzo, Lexing Ying, and James Zou. 'How to Learn when Data Reacts to Your Model: Performative Gradient Descent'. In: Proceedings of the 38th International Conference on Machine Learning . International Conference on Machine Learning. PMLR, July 1, 2021, pp. 4641-4650.

[LHL21] Qi Lei, Wei Hu, and Jason D. Lee. 'Near-Optimal Linear Regression under Distribution Shift'. In: ArXiv (June 23, 2021).

[MPZ21] John P Miller, Juan C Perdomo, and Tijana Zrnic. 'Outside the echo chamber: Optimizing the performative risk'. In: International Conference on Machine Learning . PMLR. 2021, pp. 7710-7720.

[Wil+21] Olivia Wiles et al. 'A Fine-Grained Analysis on Distribution Shift'. In: ArXiv (Oct. 21, 2021).

[Zrn+21] Tijana Zrnic et al. 'Who Leads and Who Follows in Strategic Classification?' In: Advances in Neural Information Processing Systems . Vol. 34. Curran Associates, Inc., 2021, pp. 15257-15269.

[AZ22] Alekh Agarwal and Tong Zhang. 'Minimax Regret Optimization for Robust Machine Learning under Distribution Shift'. In: Proceedings of Thirty Fifth Conference on Learning Theory . Conference on Learning Theory. PMLR, June 28, 2022, pp. 27042729.

[Giu+22] Valeria Giunta et al. 'Local and Global Existence for Nonlocal Multispecies AdvectionDiffusion Models'. In: SIAM Journal on Applied Dynamical Systems 21.3 (Sept. 2022), pp. 1686-1708. ISSN: 1536-0040. DOI: 10.1137/21M1425992 .

[HPS22] Georg Heinze, Jan-Frederik Pietschmann, and Markus Schmidtchen. Nonlocal crossinteraction systems on graphs: Nonquadratic Finslerian structure and nonlinear mobilities . Jan. 11, 2022. DOI: 10.48550/arXiv.2107.11289 . arXiv: 2107.11289[math] .

[JPZ22] Ansgar Jungel, Stefan Portisch, and Antoine Zurek. 'Nonlocal cross-diffusion systems for multi-species populations and networks'. In: Nonlinear Analysis 219 (June 1, 2022), p. 112800. ISSN: 0362-546X. DOI: 10.1016/j.na.2022.112800 .

[LY22] Guanlin Li and Yao Yao. 'Two-species competition model with chemotaxis: wellposedness, stability and dynamics'. In: Nonlinearity 35.3 (Feb. 3, 2022), p. 1329. ISSN: 0951-7715. DOI: 10.1088/1361-6544/ac4a8d .

[LFG22] Fengming Lin, Xiaolei Fang, and Zheming Gao. 'Distributionally Robust Optimization: A review on theory and applications'. In: Numerical Algebra, Control and Optimization 12.1 (2022), pp. 159-212.

[Nar+22] Adhyyan Narang et al. 'Learning in Stochastic Monotone Games with DecisionDependent Data'. In: International Conference on Artificial Intelligence and Statistics . PMLR. 2022, pp. 5891-5912.

[Ray+22] Mitas Ray et al. 'Decision-dependent risk minimization in geometrically decaying dynamic environments'. In: Proceedings of the AAAI Conference on Artificial Intelligence . Vol. 36. 7. 2022, pp. 8081-8088.

[CFG23] J. Carrillo, Alejandro Fernandez-Jimenez, and D. G'omez-Castro. 'Partial mass concentration for fast-diffusions with non-local aggregation terms'. In: Apr. 10, 2023.

[Dou+23] Marie Doumic et al. Multispecies cross-diffusions: from a nonlocal mean-field to a porous medium system without self-diffusion . June 12, 2023. DOI: 10.48550/arXiv. 2306.01777 . arXiv: 2306.01777[math] .

[GG23] Camilo Garcia Trillos and Nicolas Garcia Trillos. On adversarial robustness and the use of Wasserstein ascent-descent dynamics to enforce it . Jan. 9, 2023. DOI: 10.48550/ arXiv.2301.03662 . arXiv: 2301.03662[cs,math] .

[HPS23] Georg Heinze, Jan-Frederik Pietschmann, and Markus Schmidtchen. 'Nonlocal crossinteraction systems on graphs: Energy landscape and dynamics'. In: Kinetic and Related Models (KRM) (2023).

[Lu23] Yulong Lu. Two-Scale Gradient Descent Ascent Dynamics Finds Mixed Nash Equilibria of Continuous Games: A Mean-Field Perspective . Jan. 8, 2023. DOI: 10.48550/arXiv. 2212.08791 . arXiv: 2212.08791[cs,math,stat] .

[WC23] Guillaume Wang and Lenaic Chizat. An Exponentially Converging Particle Method for the Mixed Nash Equilibrium of Continuous Games . Apr. 13, 2023. DOI: 10.48550/ arXiv.2211.01280 . arXiv: 2211.01280[cs,math] .

[Qui+] Joaquin Quinonero-Candela et al. Dataset Shift in Machine Learning . MIT Press.

A General structure and preliminaries

In this section, we give more details on the models discussed in the main article, and introduce definitions and notation that are needed for the subsequent proofs.

A.1 Structure of the dynamics

For the case of aligned objectives, the full coupled system of PDEs (4) can be written as

In other words, the population ρ in (7a) is subject to an isotropic diffusive force with diffusion coefficient α > 0 , a drift force due to the time-varying confining potential ∫ f 1 d µ ( t ) -α log ˜ ρ , and a self-interaction force via the interaction potential W . If we consider the measure µ to be given and fixed in time, this corresponds exactly to the type of parabolic equation studied in [CMV03]. Here however the dynamics are more complex due to the coupling of the confining potential with the dynamics (7b) for µ ( t ) via the coupling potential f 1 . Before presenting the analysis of this model, let us give a bit more intuition on the meaning and the structure of these dynamics.

In the setting where µ represents a binary classifier, we can think of the distribution ¯ ρ as modelling all those individuals carrying the true label 1, say, and the distribution ρ ( t ) as modelling all those individuals carrying a true label 0, say, where 0 and 1 denote the labels of two classes of interest. The term ∫ f 1 ( z, x ) µ ( t, d x ) represents a penalty for incorrectly classifying an individual at z with true label 0 when using the classifier µ ( t, x ) . In other words, ∫ f 1 ( z, x ) µ ( t, d x ) ∈ [0 , ∞ ) is increasingly large the more z digresses from the correct classification 0. Similarly, ∫ f 1 ( z, x ) ρ ( t, d z ) ∈ [0 , ∞ ) is increasingly large if the population ρ shifts mass to locations in z where the classification is incorrect. The terminology aligned objectives refers to the fact that in (7) both the population and the classifier are trying to evolve in a way as to maximize correct classification. Analogously, the term ∫ f 2 ( z, x )¯ ρ (d z ) is large if x would incorrectly classify the population ¯ ρ that carries the label 1. A natural extension of the model (7) would be a setting where also the population carrying labels 1 evolves over time, which is simulated in Section E.2. Most elements of the framework presented here would likely carry over the setting of three coupled PDEs: one for the evolution of ρ ( t ) , one for the evolution of ¯ ρ ( t ) and one for the classifier µ ( t ) .

The term

forces the evolution of ρ ( t ) to approach ˜ ρ . In other words, it penalizes (in energy) deviations from a given reference measure ˜ ρ . In the context of the application at hand, we take ˜ ρ to be the initial distribution ρ ( t = 0) . The solution ρ ( t ) then evolves away from ˜ ρ over time due to the other forces that are present. Therefore, the term KL ( ρ | ˜ ρ ) in the energy both provides smoothing of the flow and a penalization for deviations away from the reference measure ˜ ρ .

The self-interaction term W ∗ ρ introduces non-locality into the dynamics, as the decision for any given individual to move in a certain direction is influenced by the behavior of all other individuals in the population. The choice of W is application dependent. Very often, the interaction between two individuals only depends on the distance between them. This suggests a choice of W as a radial function, i.e. W ( z ) = ω ( | z | ) . A choice of ω : R → R such that ω ′ ( r ) > 0 corresponds to an attractive force between individuals, whereas ω ′ ( r ) < 0 corresponds to a repulsive force. The statement | z | ω ′ ( | z | ) = z · ∇ z W ( z ) ≥ -D in Assumption 3 therefore corresponds to a requirement that the self-interaction force is not too repulsive. Neglecting all other forces in (7a), we obtain the non-local interaction equation

which appears in many instances in mathematical biology, mathematical physics, and material science, and it is an equation that has been extensively studied over the past few decades, see for example [Car+11; BCY12; CCH14; BCL09; BLL12; CMV06; CFG23] and references therein. Using the results from these works, our assumptions on the interaction potential W can be relaxed in many ways, for example by allowing discontinuous derivatives at zero for W , or by allowing W to be negative.

The dynamics (7b) for the algorithm µ is a non-autonomous transport equation,

where the time-dependence in the velocity field

comes through the evolving population ρ ( t ) . This structure allows to obtain an explicit solution for µ ( t ) in terms of the initial condition µ 0 and the solution ρ ( t ) to (7a) using the method of characteristics.

Proposition 5. Assume that there exists a constant c > 0 such that

Then the unique distributional solution µ ( t ) to (7b) is given by

where Φ( t, s, x ) solves the characteristic equation

Proof. Thanks to Assumption 1, we have that v ∈ C 1 ( R × R d ; R d ) , and by (8), we have

By classical Cauchy-Lipschitz theory for ODEs, this guarantees the existence of a unique global solution Φ( t, s, x ) solving (10). Then it can be checked directly that µ ( t ) as defined in (9) is a distributional solution to (7b).

In the characteristic equation (10), Φ( s, t, x ) is a parametrization of all trajectories: if a particle was at location x at time t , then it is at location Φ( s, t, x ) at time s . Our assumptions on f 1 , f 2 and ¯ ρ also ensure that Φ( s, t, · ) : R d → R d is a C 1 -diffeomorphism for all s, t ∈ R . For more details on transport equations, see for example [De 07].

Remark 2. Consider the special case where µ 0 = δ x 0 for some initial position x 0 ∈ R d . Then by Proposition 5, the solution to (7b) is given by µ ( t ) = δ x ( t ) , where x ( t ) := Φ( t, 0 , x 0 ) solves the ODE

which is precisely of type (3) .

For the case of competing objectives, the two models we consider can be written as

for (5), and

for (6).

A.2 Definitions and notation

Here, and in what follows, I d denotes the d × d identity matrix, and id denotes the identity map. The energy functionals we are considering are usually defined on the set of probability measures on R d , denoted by P ( R d ) . If we consider the subset P 2 ( R d ) of probability measures with bounded second moment,

then we can endow this space with the Wasserstein-2 metric.

Definition 4 (Wasserstein-2 Metric) . The Wasserstein-2 metric between two probability measures µ, ν ∈ P 2 ( R d ) is given by

where Γ is the set of all joint probability distributions with marginals µ and ν , i.e. µ (d z ) = ∫ γ (d z, z ′ )d z ′ and ν (d z ′ ) = ∫ γ ( z, d z ′ )d z .

The restriction to P 2 ( R d ) ensures that W 2 is always finite. Then the space ( P 2 ( R d ) , W 2 ) is indeed a metric space. We will make use of the fact that W 2 metrizes narrow convergence of probability measures. To make this statement precise, let us introduce two common notions of convergence for probability measures, which are a subset of the finite signed Radon measures M ( R d ) .

Definition 5. Consider a sequence ( µ n ) ∈ M ( R d ) and a limit µ ∈ M ( R d ) .

(Narrow topology) The sequence ( µ n ) converges narrowly to µ , denoted by µ n ⇀ µ , if for all continuous bounded functions f : R d → R ,

(Weak-∗ topology) The sequence ( µ n ) converges weakly-∗ to µ , denoted by µ n ∗ ⇀ µ , if for all continuous functions vanishing at infinity (i.e. f : R d → R such that for all ϵ > 0 there exists a compact set K ϵ ⊂ R d such that | f ( z ) | < ϵ on R d \ K ϵ ), we have

Let us denote the set of continuous functions on R d vanishing at infinity by C 0 ( R d ) , and the set of continuous bounded functions by C b ( R d ) . Note that narrow convergence immediately implies that µ n ( R d ) → µ ( R d ) as the constant function is in C b ( R d ) . This is not necessarily true for weak-∗ convergence. We will later make use of the Banach-Alaoglu theorem [Ala40], which gives weak-∗ compactness of the unit ball in a dual space. Note that M ( R d ) is indeed the dual of C 0 ( R d ) endowed with the sup-norm, and P ( R d ) is the unit ball in M ( R d ) using the dual norm. Moreover, if we can ensure that mass does not escape to infinity, the two notions of convergence in Definition 5 are in fact equivalent.

Lemma 6. Consider a sequence ( µ n ) ∈ M ( R d ) and a measure µ ∈ M ( R d ) . Then µ n ⇀µ if and only if µ n ∗ ⇀µ and µ n ( R d ) → µ ( R d ) .

This follows directly from Definition 5. Here, the condition µ n ( R d ) → µ ( R d ) is equivalent to tightness of ( µ n ) , and follows from Markov's inequality [Gho02] if we can establish uniform bounds on the second moments, i.e. we want to show that there exists a constant C > 0 independent of n such that

Definition 6 (Tightness of probability measures) . A collection of measures ( µ n ) ∈ M ( R d ) is tight if for all ϵ > 0 there exists a compact set K ϵ ⊂ R d such that | µ n | ( R d \ K ϵ ) < ϵ for all n ∈ N , where | µ | denotes the total variation of µ .

Another classical result is that the Wasserstein-2 metric metrizes narrow convergence and weak-∗ convergence of probability measures, see for example [San15, Theorem 5.11] or [Vil03, Theorem 7.12].

Lemma 7. Let µ n , µ ∈ P 2 ( R d ) . Then W 2 ( µ n , µ ) → 0 if and only if

Remark 3. Note that µ n ⇀µ can be replaced by µ n ∗ ⇀µ in the above statement thanks to the fact that the limit µ is a probability measure with mass 1, see Lemma 6.

Next, we consider two measures µ, ν ∈ P ( R d ) that are atomless , i.e µ ( { z } ) = 0 for all z ∈ R d . By Brenier's theorem [BB00] (also see [Vil03, Theorem 2.32]) there exists a unique measurable map T : R d → R d such that T # µ = ν , and T = ∇ ψ for some convex function ψ : R d → R . Here, the push-forward operator ∇ ψ # is defined as

for all Borel-measurable functions f : R d ↦→ R + . If ρ 1 = ∇ ψ # ρ 0 , we denote by ρ s = [(1 -s ) id + s ∇ ψ ] # ρ 0 the discplacement interpolant between ρ 0 and ρ 1 . We are now ready to introduce the notion of displacement convexity, which is the same as geodesic convexity in the geodesic space ( P 2 ( R d ) , W 2 ) . We will state the definition here for atomless measures, but it can be relaxed to any pair of measures in P 2 using optimal transport plans instead of transport maps. In what follows, we will use s to denote the interpolation parameter for geodesics, and t to denote time related to solutions of (4), (5) and (6).

Definition 7 (Displacement Convexity) . A functional G : P ↦→ R is displacement convex if for all ρ 0 , ρ 1 that are atomless we have

where ρ s = [(1 -s ) id + s ∇ ψ ] # ρ 0 is the displacement interpolant between ρ 0 and ρ 1 . Further, G : P ↦→ R is uniformly displacement convex with constant η > 0 if

where ρ s = [(1 -s ) id + s ∇ ψ ] # ρ 0 is the displacement interpolant between ρ 0 and ρ 1 .

Remark 4. In other words, G is displacement convex (concave) if the function G ( ρ s ) is convex (concave) with ρ s = [(1 -s id + s ∇ ψ ] # ρ 0 being the displacement interpolant between ρ 0 and ρ 1 . Contrast this with the classical notion of convexity (concavity) for G , where we require that the function G ((1 -s ) ρ 0 + sρ 1 ) is convex (concave).

In fact, if the energy G is twice differentiable along geodesics, then the condition d 2 d s 2 G ( γ s ) ≥ 0 along any geodesic ( ρ s ) s ∈ [0 , 1] between ρ 0 and ρ 1 is sufficient to obtain displacement convexity. Similarly, when d 2 d s 2 G ( ρ s ) ≥ ηW 2 ( ρ 0 , ρ 1 ) 2 , then G is uniformly displacement convex with constant η > 0 . For more details, see [McC97] and [Vil03, Chapter 5.2].

A.3 Steady states

The main goal in our theoretical analysis is to characterize the asymptotic behavior for the models (4), (5) and (6) as time goes to infinity. The steady states of these equations are the natural candidates to be asymptotic profiles for the corresponding equations. Thanks to the gradient flow structure, we expect to be able to make a connection between ground states of the energy functionals, and the steady states of the corresponding gradient flow dynamics. More precisely, any minimizer or maximizer is in particular a critical point of the energy, and therefore satisfies that the first variation is constant on disconnected components of its support. If this ground state also has enough regularity (weak differentiability) to be a solution to the equation, it immediately follows that it is in fact a steady state.

To make this connection precise, we first introduce what exactly we mean by a steady state.

Definition 8 (Steady states for (4)) . Given ρ ∞ ∈ L 1 + ( R d ) ∩ L ∞ loc ( R d ) with ∥ ρ ∞ ∥ 1 = 1 and µ ∞ ∈ P 2 ( R d ) , then ( ρ ∞ , µ ∞ ) is a steady state for the system (4) if ρ ∞ ∈ W 1 , 2 loc ( R d ) , ∇ W ∗ ρ ∞ ∈ L 1 loc ( R d ) , ρ ∞ is absolutely continuous with respect to ˜ ρ , and ( ρ ∞ , µ ∞ ) satisfy

in the sense of distributions.

Here, L 1 + ( R d ) := { ρ ∈ L 1 ( R d ) : ρ ≥ 0 } .

Definition 9 (Steady states for (5)) . Let ρ ∞ ∈ L 1 + ( R d ) ∩ L ∞ loc ( R d ) with ∥ ρ ∞ ∥ 1 = 1 . Then ρ ∞ is a steady state for the system (5) if ρ ∞ ∈ W 1 , 2 loc ( R d ) , ∇ W ∗ ρ ∞ ∈ L 1 loc ( R d ) , ρ ∞ is absolutely continuous with respect to ˜ ρ , and ρ ∞ satisfies

in the sense of distributions, where b ( ρ ∞ ) := argmin x G c ( ρ ∞ , x ) .

Definition 10 (Steady states for (6)) . The vector x ∞ ∈ R d is a steady state for the system (6) if it satisfies

In fact, with the above notions of steady state, we can obtain improved regularity for ρ ∞ .

Lemma 8. Let Assumptions 1-3 hold. Then the steady states ρ ∞ for (4) and (5) are continuous.

Proof. We present here the argument for equation (5) only. The result for (4) follows in exactly the same way by replacing f 1 ( z, b ( ρ ∞ )) with -∫ f 1 ( z, x )d µ ∞ ( x ) .

Thanks to our assumptions, we have f 1 ( · , b ( ρ ∞ )) + α log ˜ ρ ( · ) ∈ C 1 , which implies that ∇ ( f 1 ( · , b ( ρ ∞ )) + α log ˜ ρ ( · )) ∈ L ∞ loc . By the definition of a steady state, ρ ∞ ∈ L 1 ∩ L ∞ loc and thanks to Assumption 3 we have W ∈ C 2 , which implies that ∇ W ∗ ρ ∞ ∈ L ∞ loc . Let

Then by the aforementioned regularity, we obtain h ∈ L 1 loc ∩ L ∞ loc . By interpolation, it follows that h ∈ L p loc for all 1 < p < ∞ . This implies that div ( h ) ∈ W -1 ,p loc . Since ρ ∞ is a weak W 1 , 2 loc -solution of (13), we have

and so by classic elliptic regularity theory we conclude ρ ∞ ∈ W 1 ,p loc . Finally, applying Morrey's inequality, we have ρ ∞ ∈ C 0 ,k where k = p -d p for any d < p < ∞ . Therefore ρ ∞ ∈ C ( R d ) (after possibly being redefined on a set of measure zero).

B Proof of Theorem 2

For ease of notation, we write G a : P ( R d ) ×P ( R d ) ↦→ [0 , ∞ ] as

with potential given by V ( x ) := ∫ f 2 ( z, x )d¯ ρ ( z ) + β 2 ∥ x -x 0 ∥ 2 .

In order to prove the existence of a unique ground state for G a , a natural approach is to consider the corresponding Euler-Lagrange equations

where c 1 , c 2 are constants that may differ on different connected components of supp( ρ ) and supp( µ ) . These equations are not easy to solve explicitly, and we are therefore using general non-constructive techniques from calculus of variations. We first show continuity and convexity properties for the functional G a (Lemma 9 and Proposition 10), essential properties that will allow us to deduce existence and uniqueness of ground states using the direct method in the calculus of variations (Proposition 11). Using the Euler-Lagrange equation 14, we then prove properties on the support of the ground state (Corollary 12). To obtain convergence results, we apply the HWI method: we first show a general 'interpolation' inequality between the energy, the energy dissipation and the metric (Proposition 13); this fundamental inequality will then imply a generalized logarithmic Sobolev inequality (Corollary 14) relating the energy to the energy dissipation, and a generalized Talagrand inequality (Corollary 15) that allows to translate convergence in energy into convergence in metric. Putting all these ingrediends together will then allow us to conclude for the statements in Theorem 2.

Lemma 9 (Lower semi-continuity) . Let Assumptions 1-3 hold. Then the functional G a : P × P → R is lower semi-continuous with respect to the weak-∗ topology.

Proof. We split the energy G a into three parts: (i) KL ( ρ | ˜ ρ ) , (ii) the interaction energy W , and (iii) the potential energy V . For (i), lower semi-continuity has been shown in [Pos75]. For (ii), we can directly apply [San15, Proposition 7.2] using Assumption 3. For (iii), note that V and f 1 are lower semi-continuous and bounded below thanks to Assumption 1, and so the result follows from [San15, Proposition 7.1].

Proposition 10 (Uniform displacement convexity) . Let α, β > 0 . Fix γ 0 , γ 1 ∈ P 2 ×P 2 and let Assumptions 1-3 hold. Along any geodesic ( γ s ) s ∈ [0 , 1] ∈ P 2 ×P 2 connecting γ 0 to γ 1 , we have for all s ∈ [0 , 1]

As a result, the functional G a : P × P → R is uniformly displacement convex with constant λ a > 0 .

where we define

Proof. Let γ 0 and γ 1 be two probability measures with bounded second moments. Denote by ϕ, ψ : R d → R the optimal Kantorovich potentials pushing ρ 0 onto ρ 1 , and µ 0 onto µ 1 , respectively:

The now classical results in [BB00] guarantee that there always exists convex functions ϕ, ψ that satisfy the conditions above. Then the path ( γ s ) s ∈ [0 , 1] = ( ρ s , µ s ) s ∈ [0 , 1] defined by

is a W -geodesic from γ 0 to γ 1 .

The first derivative of V along geodesics in the Wasserstein metric is given by

and taking another derivative we have

where we denoted D s ( z, x ) := Hess ( f 1 )((1 -s ) z + s ∇ ϕ ( z ) , (1 -s ) x + s ∇ ψ ( x )) , and the last inequality follows from Assumption 1 and the optimality of the potentials ϕ and ψ .

Following [CMV03; Vil03] and using Assumption 2, the second derivatives of the diffusion term and the interaction term along geodesics are given by

Putting the above estimates together, we obtain (15).

Remark 5. Alternatively, one could assume strong convexity of W , which would improve the lower-bound on the second derivative along geodesics.

Proposition 11. (Ground state) Let Assumptions 1-3 hold for α, β > 0 . Then the functional G a : P ( R d ) × P ( R d ) → [0 , ∞ ] admits a unique minimizer γ ∗ = ( ρ ∗ , µ ∗ ) , and it satisfies ρ ∗ ∈ P 2 ( R d ) ∩ L 1 ( R d ) , µ ∗ ∈ P 2 ( R d ) , and ρ ∗ is absolutely continuous with respect to ˜ ρ .

̸

Proof. We show existence of a minimizer of G a using the direct method in the calculus of variations. Denote by γ = ( ρ, µ ) ∈ P×P ⊂ M×M a pair of probability measures as a point in the product space of Radon measures. Since G a ≥ 0 on P × P (see Assumption 1) and not identically + ∞ everywhere, there exists a minimizing sequence ( γ n ) ∈ P × P . Note that ( γ n ) is in the closed unit ball of the dual space of continuous functions vanishing at infinity ( C 0 ( R d ) × C 0 ( R d )) ∗ endowed with the dual norm ∥ γ n ∥ ∗ = sup | ∫ f d ρ n + ∫ g d µ n | ∥ ( f,g ) ∥ ∞ over f, g ∈ C 0 ( R d ) with ∥ ( f, g ) ∥ ∞ := ∥ f ∥ ∞ + ∥ g ∥ ∞ = 0 . By the Banach-Alaoglu theorem [Rud91, Thm 3.15] there exists a limit γ ∗ = ( ρ ∗ , µ ∗ ) ∈ M×M = ( C 0 × C 0 ) ∗ and a convergent subsequence (not relabelled) such that γ n ∗ ⇀γ ∗ . In fact, since KL ( ρ ∗ | ˜ ρ ) < ∞ it follows that ρ ∗ is absolutely continuous with respect to ˜ ρ , implying ρ ∗ ∈ L 1 ( R d ) thanks to Assumption 2. Further, µ ∗ has bounded second moment, else we would have

inf γ ∈P×P G a ( γ ) = ∞ which yields a contradiction. It remains to show that ∫ d ρ ∗ = ∫ d µ ∗ = 1 to conclude that γ ∗ ∈ P × P . To this aim, it is sufficient to show tightness of ( ρ n ) and ( µ n ) , preventing the escape of mass to infinity as we have ∫ d ρ n = ∫ d µ n = 1 for all n ≥ 1 . Tightness follows from Markov's inequality [Gho02] if we can establish uniform bounds on the second moments, i.e. we want to show that there exists a constant C > 0 independent of n such that

To establish (17), observe that thanks to Assumption 2, there exists a constant c 0 ∈ R (possibly negative) such that -log ˜ ρ ( z ) ≥ c 0 + ˜ λ 4 ∥ z ∥ 2 for all z ∈ R d . Then

Therefore, using ∫ d ρ n = ∫ d µ n = 1 and writing ζ := min { α ˜ λ 4 , β 2 } > 0 , we obtain the desired uniform upper bound on the second moments of the minimizing sequence,

This concludes the proof that the limit γ ∗ satisfies indeed γ ∗ ∈ P ×P , and indeed ρ ∗ ∈ P 2 ( R d ) as well. Finally, γ ∗ is indeed a minimizer of G a thanks to weak-* lower-semicontinuity of G a following Lemma 9.

Next we show uniqueness using a contradiction argument. Suppose γ ∗ = ( ρ ∗ , µ ∗ ) and γ ′ ∗ = ( ρ ′ ∗ , µ ′ ∗ ) are minimizers of G a . For s ∈ [0 , 1] , define γ s := ((1 -s ) id + sT, (1 -s ) id + sS ) # γ ∗ , where T, S : R d ↦→ R d are the optimal transport maps such that ρ ′ ∗ = T # ρ ∗ and µ ′ ∗ = S # µ ∗ . By Proposition 10 the energy G a is uniformly displacement convex, and so we have

̸

If γ ∗ = γ ′ ∗ and s ∈ (0 , 1) , then strict inequality holds by applying similar arguments as in [McC97, Proposition 1.2]. However, the strict inequality G a ( γ s ) < G a ( γ ∗ ) for γ ∗ = γ ′ ∗ is a contradiction to the minimality of γ ∗ . Hence, the minimizer is unique.

Remark 6. If λ 1 > 0 , then the strict convexity of f 1 can be used to deduce uniqueness, and the assumptions on -log ˜ ρ can be weakened from strict convexity to convexity.

Corollary 12. Let Assumptions 1-3 hold. Any minimizer γ ∗ = ( ρ ∗ , µ ∗ ) of G a is a steady state for equation (4) according to Definition 8 and satisfies supp( ρ ∗ ) = supp(˜ ρ ) .

̸

Proof. By Proposition 11, we have ρ ∗ , µ ∗ ∈ P 2 , as well as ρ ∗ ∈ L 1 + , ∥ ρ ∗ ∥ 1 = 1 , and that ρ ∗ is absolutely continuous with respect to ˜ ρ . Since W ∈ C 2 ( R d ) , it follows that ∇ W ∗ ρ ∗ ∈ L 1 loc . In order to show that γ ∗ is a steady state for equation (4), it remains to prove that ρ ∗ ∈ W 1 , 2 loc ∩ L ∞ loc . As γ ∗ is a minimizer, it is in particular a critical point, and therefore satisfies equations (14). Rearranging, we obtain (for a possible different constant c 1 [ ρ ∗ , µ ∗ ] = 0 ) from (14a) that

Then for any compact set K ⊂ R d ,

As f 1 ≥ 0 by Assumption 1 and W ≥ 0 by Assumption 3, the last two terms are finite. The first supremum is finite thanks to continuity of ˜ ρ . Therefore ρ ∗ ∈ L ∞ loc . To show that ρ ∗ ∈ W 1 , 2 loc , note that for any compact set K ⊂ R d , we have ∫ K | ρ ∗ ( z ) | 2 d z < ∞ as a consequence of ρ ∗ ∈ L ∞ loc . Moreover, defining T [ γ ]( z ) := -1 α (∫ f 1 ( z, x ) µ ( x ) + W ∗ ρ ( z ) ) ≤ 0 , we have

̸

which is bounded noting that exp(2 T [ γ ∗ ]) ≤ 1 and that T [ γ ∗ ]( · ) , ∇ T [ γ ∗ ]( · ) and ∇ ˜ ρ are in L ∞ loc , where we used that f 1 , ( · , x ) , W ( · ) , ˜ ρ ( · ) ∈ C 1 ( R d ) by Assumptions 1-3. We conclude that ρ ∗ ∈ W 1 , 2 loc , and indeed ( ρ ∗ , µ ∗ ) solves (12) in the sense of distributions as a consequence of (14).

Next, we show that supp( ρ ∗ ) = supp(˜ ρ ) using again the relation (18). Firstly, note that supp( ρ ∗ ) ⊂ supp(˜ ρ ) since ρ ∗ is absolutely continuous with respect to ˜ ρ . Secondly, we claim that exp [ -1 α (∫ f 1 ( z, x ) µ ∗ ( x ) + W ∗ ρ ∗ ( z ) )] > 0 for all z ∈ R d . In other words, we claim that ∫ f 1 ( z, x ) µ ∗ ( x ) < ∞ and W ∗ ρ ∗ ( z ) < ∞ for all z ∈ R d . Indeed, for the first term, fix any z ∈ R d and choose R > 0 large enough such that z ∈ B R (0) . Then, thanks to continuity of f 1 according to Assumption 1, we have

For the second term, note that by Assumption 3, we have for any z ∈ R d and ϵ > 0 ,

where the last equality follows by choosing the optimal ϵ = √ 2 D . We conclude that

which is finite for any fixed z ∈ R d thanks to the fact that ρ ∗ ∈ P 2 ( R d ) . Hence, supp( ρ ∗ ) = supp(˜ ρ ) .

Remark 7. If we have in addition that ˜ ρ ∈ L ∞ ( R d ) , then the minimizer ρ ∗ of G a is in L ∞ ( R d ) as well. This follows directly by bounding the right-hand side of (18) .

The following inequality is referred to as HWI inequality and represents the key result to obtain convergence to equilibrium.

Proposition 13 (HWI inequality) . Define the dissipation functional

Assume α, β > 0 and let λ a as defined in (15) . Let γ 0 , γ 1 ∈ P 2 ×P 2 such that G a ( γ 0 ) , G a ( γ 1 ) , D a ( γ 0 ) < ∞ . Then

Proof. For simplicity, consider γ 0 , γ 1 that have smooth Lebesgue densities of compact support. The general case can be recovered using approximation arguments. Let ( γ s ) s ∈ [0 , 1] denote a W -geodesic between γ 0 , γ 1 . Following similar arguments as in [CMV03] and [OV00, Section 5] and making use of the calculations in the proof of Proposition 10, we have

where

Note that the dissipation functional can then be written as

Using the double integral Cauchy-Schwarz inequality [Ste04], we obtain

Next, we compute a Taylor expansion of G a ( γ s ) when considered as a function in s and use the bound on d 2 d s 2 G a from (15):

Remark 8. The HWI inequality in Proposition 13 immediately implies uniqueness of minimizers for G a in the set { γ ∈ P × P : D a ( γ ) < + ∞} . Indeed, if γ 0 is such that D a ( γ 0 ) = 0 , then for any other γ 1 in the above set we have G a ( γ 0 ) ≤ G a ( γ 1 ) with equality if and only if W ( γ 0 , γ 1 ) = 0 .

Corollary 14 (Generalized Log-Sobolev inequality) . Denote by γ ∗ the unique minimizer of G a . With λ a as defined in (15) , any product measure γ ∈ P 2 ×P 2 such that G ( γ ) , D a ( γ ) < ∞ satisfies

Proof. This statement follows immediately from Proposition 13. Indeed, let γ 1 = γ ∗ and γ 0 = γ in (20). Then

Corollary 15 (Talagrand inequality) . Denote by γ ∗ the unique minimizer of G a . With λ a as defined in (15) , it holds

for any γ ∈ P 2 ×P 2 such that G a ( γ ) < ∞ .

Proof. This is also a direct consequence of Proposition 13 by setting γ 0 = γ ∗ and γ 1 = γ . Then G a ( γ ∗ ) < ∞ and D a ( γ ∗ ) = 0 , and the result follows.

Proof of Theorem 2. The entropy term ∫ ρ log ρ produces diffusion in ρ for the corresponding PDE in (4). As a consequence, solutions ρ t to (4) and minimizers ρ ∗ for G a have to be L 1 functions. As there is no diffusion for the evolution of µ t , solutions may have a singular part. In fact, for initial condition µ 0 = δ x 0 , the corresponding solution will be of the form µ t = δ x ( t ) , where x ( t ) solves the ODE (3) with initial condition x 0 . This follows from the fact that the evolution for µ t is a transport equation (also see Section A.1 for more details). Results (a) and (b) are the statements in Proposition 11, Corollary 12 and Corollary 15. To obtain (c), we differentiate the energy G a along solutions γ t to the equation (4):

where the last bound follows from Corollary 14. Applying Gronwall's inequality, we immediately obtain decay in energy,

Finally, applying Talagrand's inequality (Corollary 15), the decay in energy implies decay in the product Wasserstein metric,

where c > 0 is a constant only depending on γ 0 , γ ∗ and the parameter λ a .

C Proof of Theorem 3

In the case of competing objectives, we rewrite the energy G c ( ρ, x ) : P ( R d ) × R d ↦→ [ -∞ , ∞ ] as follows:

where

Note that for any fixed ρ ∈ P , the energy G c ( ρ, · ) is strictly convex in x , and therefore has a unique minimizer. Define the best response by

and denote G b ( ρ ) := G c ( ρ, b ( ρ )) . We begin with auxiliary results computing the first variations of the best response b and then the different terms in G b ( ρ ) using Definition 1.

Lemma 16 (First variation of the best response) . The first variation of the best response of the classifier at ρ (if it exists) is

where Q ( ρ ) ⪰ ( β + λ 1 + λ 2 ) I d is a symmetric matrix, constant in z and x , defined as

In particular, we then have for any ψ ∈ C ∞ c ( R d ) with ∫ ψ d z = 0 that

Proof. Let ψ ∈ C ∞ c ( R d ) with ∫ ψ d z = 0 and fix ϵ > 0 . Any minimizer of G c ( ρ + ϵψ, x ) for fixed ρ must satisfy

) d

δ

ρ

∇

x

G

c

[

ρ

+

ϵψ, b

(

ρ

+

ϵψ

)]

ψ

(

z

z

+

∇

2

x

G

c

(

ρ

+

ϵψ, b

(

ρ

+

ϵψ

))

δ

ρ

b

[

ρ

+

ϵψ

](

z

)

ψ

(

z

Next, we explicitly compute all terms involved in (22). Computing the derivatives yields

Note that ∇ 2 x G c is invertible by Assumption 1, which states that f 1 and f 2 have positive-definite Hessians. Inverting this term and substituting these expressions into (22) for ϵ = 0 gives

Finally, the lower bound on Q ( ρ ) follows thanks to Assumption 1.

Remark 9. If we include the additional assumption that f i ∈ C 3 ( R d × R d ; [0 , ∞ )) for i = 1 , 2 , then the Hessian of b [ ρ ] is well-defined. More precisely, the Hessian is given by

where u [ ρ, ψ ] = ∫ ∇ x f 1 ( z, b [ ρ ]) ψ ( z )d z and

Therefore, we can Taylor expand b [ ρ ] up to second order and control the remainder term of order ϵ 2 .

Differentiating in ϵ , we obtain

∫

∫

) d

z

= 0

.

(22)

Lemma 17 (First variation of G b ) . The first variation of G b is given by

where

and

Proof. Webegin with general expressions for Taylor expansions of b : P ( R d ) → R d and f i ( z, b ( · )) : P ( R d ) → R for i = 1 , 2 around ρ . Let ψ ∈ T with T = { ψ : ∫ ψ ( z )d z = 0 } . Then

and

We compute explicitly each of the first variations:

(i) Using (24), we have

(ii) Similarly, using again (24),

(iii) Finally, from (23) it follows that

Finally, the expression for δ ρ P [ ρ ] follows by direct computation

Lemma 18. Denote G b ( ρ ) := G c ( ρ, b ( ρ )) with b ( ρ ) given by (5) . Then δ ρ G b [ ρ ] = δ ρ G c [ ρ ] | x = b ( ρ ) .

Proof. We start by computing δ ρ G c ( · , x )[ ρ ]( z ) for any z, x ∈ R d :

Next, we compute δ ρ G b . Using Lemma 17, the first variation of G b is given by

Note that

and by the definition of the best response b ( ρ ) , we have ∇ x G x ( ρ, x ) | x = b ( ρ ) = 0 . Substituting into the expression for δ ρ G b and using (25), we obtain

This concludes the proof.

Lemma 19 (Uniform boundedness of the best response) . Let Assumption 1 hold. Then for any ρ ∈ P ( R d ) , we have

Proof. By definition of the best response b ( ρ ) , we have

To show that that b ( ρ ) is uniformly bounded, we take the inner product of the above expression with b ( ρ ) itself

Using Assumption 1 to bound the two integrals, together with using Young's inequality to bound the first term on the right-hand side, we obtain

which concludes the proof after rearranging terms.

Lemma20 (Upper semi-continuity) . Let Assumptions 1-3 hold. The functional G c : P ( R d ) × R d → [ -∞ , + ∞ ] is upper semi-continuous when P ( R d ) × R d is endowed with the product topology of the weak-∗ topology and the Euclidean topology. Moreover, the functional G b : P ( R d ) → [ -∞ , + ∞ ] is upper semi-continuous with respect to the weak-∗ topology.

Proof. The functional G c : P ( R d ) × R d → [ -∞ , + ∞ ] is continuous in the second variable thanks to Assumption 1. Similarly, ∫ f 1 ( z, x )d ρ ( z ) + ∫ f 2 ( z, x )d¯ ρ ( z ) is continuous in ρ thanks to [San15, Proposition 7.1] using the continuity of f 1 and f 2 . Further, -P is upper semi-continuous using [Pos75] and [San15, Proposition 7.2] thanks to Assumptions 2 and 3. This concludes the continuity properties for G c .

The upper semi-continuity of G b then follows from a direct application of a version of Berge's maximum theorem [AB06, Lemma 16.30]. Let R := ∥ x 0 ∥ 2 + 2( a 1 + a 2 ) β > 0 . We define φ : ( P ( R d ) , W 2 ) ↠ R d as the correspondence that maps any ρ ∈ P ( R d ) to the closed ball B R (0) ⊂ R d . Then the graph of φ is Gr φ = P ( R d ) ×{ B R (0) } . With this definition of φ , the range of φ is compact and φ is continuous with respect to weak-∗ convergence, and so it is in particular upper hemicontinuous. Thanks to Lemma 19, the best response function b ( ρ ) is always contained in B R (0) for any choice of ρ ∈ P ( R d ) . As a result, maximizing -G c ( ρ, x ) in x over R d for a fixed ρ ∈ P ( R d ) reduces to maximizing it over B R (0) . Using the notation introduced above, we can restrict G c to G c : Gr φ → R and write

Because G c ( ρ, x ) is upper semi-continuous when P ( R 2 ) × R d is endowed with the product topology of the weak-∗ topology and the Euclidean topology, [AB06, Lemma 16.30] guarantees that G b ( · ) is upper semi-continuous in the weak-∗ topology.

Proposition 21. Let α, β > 0 and assume Assumptions 1-4 hold with the parameters satisfying α ˜ λ > Λ 1 . Fix ρ 0 , ρ 1 ∈ P ( R d ) . Along any geodesic ( ρ s ) s ∈ [0 , 1] ∈ P 2 ( R d ) connecting ρ 0 to ρ 1 , we have for all s ∈ [0 , 1]

As a result, the functional G b : P 2 ( R d ) → [ -∞ , + ∞ ] is uniformly displacement concave with constant λ b > 0 .

Proof. Consider any ρ 0 , ρ 1 ∈ P 2 ( R d ) . Then any W 2 -geodesic ( ρ s ) s ∈ [0 , 1] connecting ρ 0 with ρ 1 solves the following system of geodesic equations:

where ρ s : R d → R and v s : R d ↦→ R d . The first derivative of G b along geodesics can be computed explicitly as

The left-hand side of the inner product is zero by definition of the best response b ( ρ s ) to ρ s , see (26). Therefore

Differentiating a second time, using (28) and integration by parts, we obtain

where

From (16), we have that

and thanks to Assumption 4 it follows that

This leaves L 2 to bound; we first consider the term d d s b ( ρ s ) :

Defining u ( ρ s ) ∈ R d by

using the results from Lemma 16 for ∇ z δ ρ b [ ρ s ] , Assumption 1 and the fact that Q ( ρ ) is constant in z and x , we have

Combining all terms together, we obtain

Remark 10. Under some additional assumptions on the functions f 1 and f 2 , we can obtain an improved convergence rate. In particular, assume that for all z, x ∈ R d ,

there exists a constant Λ 2 ≥ λ 2 ≥ 0 such that ∇ 2 x f 2 ( z, x ) ⪯ Λ 2 I d ;

there exists a constant σ ≥ 0 such that ∥∇ x ∇ z f 1 ( z, x ) ∥ ≥ σ .

Then we have -Q ( ρ s ) -1 ⪯ -1 / ( β +Λ 1 +Λ 2 ) I d . Using Lemma 16, we then obtain a stronger bound on L 2 as follows:

This means we can improve the convergence rate in (27) to λ b := α ˜ λ + σ 2 β +Λ 1 +Λ 2 -Λ 1 .

Proposition 22 (Ground state) . Let Assumptions 1-4 hold for α ˜ λ > Λ 1 ≥ 0 and β > 0 . Then there exists a unique maximizer ρ ∗ for the functional G b over P ( R d ) , and it satisfies ρ ∗ ∈ P 2 ( R d ) ∩ L 1 ( R d ) and ρ ∗ is absolutely continuous with respect to ˜ ρ .

Proof. Uniqueness of the maximizer (if it exists) is guaranteed by the uniform concavity provided by Lemma 21. To show existence of a maximizer, we use the direct method in the calculus of variations, requiring the following key properties for G b : (1) boundedness from above, (2) upper semi-continuity, and (3) tightness of any minimizing sequence. To show (1), note that ∇ 2 z ( f 1 ( z, x ) + α log ˜ ρ ( z )) ⪯ -( α ˜ λ -Λ 1 ) I d for all z, x ∈ R d × R d by Assumptions 2 and 4, and so

with c 0 ( x ) := f 1 (0 , x ) + α log ˜ ρ (0) + 1 α ˜ λ -Λ 1 ∥∇ z [ f 1 (0 , x ) + α log ˜ ρ (0)] ∥ 2 . Therefore,

To estimate each of the remaining terms on the right-hand side, denote R := ∥ x 0 ∥ 2 + 2( a 1 + a 2 ) β and recall that ∥ b ( ρ ) ∥ ≤ R for any ρ ∈ P ( R d ) thanks to Lemma 19. By continuity of f 1 and log ˜ ρ , there exists a constant c 1 ∈ R such that

The second term is controlled by c 2 thanks to Assumption 4. And the third term can be bounded directly to obtain

This concludes the proof of (1). Statement (2) was shown in Lemma 20. Then we obtain a minimizing sequence ( ρ n ) ∈ P ( R d ) which is in the closed unit ball of C 0 ( R d ) ∗ and so the Banach-Anaoglu theorem [Rud91, Theorem 3.15] there exists a limit ρ ∗ in the Radon measures and a subsequence (not relabeled) such that ρ n ∗ ⇀ρ ∗ . In fact, ρ ∗ is absolutely continuous with respect to ˜ ρ as otherwise G b ( ρ ∗ ) = -∞ , which contradicts that G b ( · ) > -∞ somewhere. We conclude that ρ ∗ ∈ L 1 ( R d ) since ˜ ρ ∈ L 1 ( R d ) by Assumption 2. To ensure ρ ∗ ∈ P ( R d ) , we require (3) tightness of the minimizing sequence ( ρ n ) . By Markov's inequality [Gho02] it is sufficient to establish a uniform bound on the second moments:

To see this we proceed in a similar way as in the proof of Proposition 10. Defining

we have K ( ρ ) = -G b ( ρ ) + ∫ f 2 ( z, b ( ρ ))d¯ ρ ( z ) + β 2 ∥ b ( ρ ) -x 0 ∥ 2 . Then using again the bound on b ( ρ ) from Lemma 19,

where the last inequality is thanks to Assumption 4. Hence, using the estimates (29) and (30) from above, and noting that the sequence ( ρ n ) is minimizing ( -G b ) , we have

which uniformly bounds the second moments of ( ρ n ) . This concludes the proof for the estimate (31) and also ensures that ρ ∗ ∈ P 2 ( R d ) .

Corollary 23. Any maximizer ρ ∗ of G b is a steady state for equation (5) according to Definition 9, and satisfies supp( ρ ∗ ) = supp(˜ ρ ) .

Proof. To show that ρ ∗ is a steady state we can follow exactly the same argument as in the proof of Corollary 12, just replacing -1 α ∫ f 1 ( z, x ) µ ∗ ( x ) with + 1 α ∫ f 1 ( z, b ( ρ ∗ ) . It remains to show that supp( ρ ∗ ) = supp(˜ ρ ) . As ρ ∗ is a maximizer, it is in particular a critical point, and therefore satisfies that δ ρ G b [ ρ ∗ ]( z ) is constant on all connected components of supp( ρ ∗ ) . Thanks to Lemma 18, this means there exists a constant c [ ρ ∗ ] (which may be different on different components of supp( ρ ∗ ) ) such that

̸

Rearranging, we obtain (for a possible different constant c [ ρ ∗ ] = 0 )

Firstly, note that supp( ρ ∗ ) ⊂ supp(˜ ρ ) since ρ ∗ is absolutely continuous with respect to ˜ ρ . Secondly, note that exp 1 α f 1 ( z, b ( ρ ∗ )) ≥ 1 for all z ∈ R d since f 1 ≥ 0 . Finally, we claim that exp ( -1 α W ∗ ρ ∗ ( z ) ) > 0 for all z ∈ R d . In other words, we claim that W ∗ ρ ∗ ( z ) < ∞ for all z ∈ R d . This follows by exactly the same argument as in Corollary 12, see equation (19). We conclude that supp( ρ ∗ ) = supp(˜ ρ ) .

Remark 11. If we have in addition that ˜ ρ ∈ L ∞ ( R d ) and f 1 ( · , x ) ∈ L ∞ ( R d ) for all x ∈ R d , then the maximizer ρ ∗ of G b is in L ∞ ( R d ) as well. This follows directly by bounding the right-hand side of (32) .

With the above preliminary results, we can now show the HWI inequality, which implies again a Talagrand-type inequality and a generalized logarithmic Sobolev inequality.

Proposition 24 (HWI inequalities) . Define the dissipation functional

Assume α, β > 0 such that α ˜ λ > Λ 1 + σ 2 , and let λ b as defined in (27) . Denote by ρ ∗ the unique maximizer of G b .

(HWI) Let ρ 0 , ρ 1 ∈ P 2 ( R d ) such that G b ( ρ 0 ) , G b ( ρ 1 ) , D b ( ρ 0 ) < ∞ . Then

(logSobolev) Any ρ ∈ P 2 ( R d ) such that G ( ρ ) , D b ( ρ ) < ∞ satisfies

(Talagrand) For any ρ ∈ P 2 ( R d ) such that G b ( ρ ) < ∞ , we have

Proof. The proof for this result follows analogously to the arguments presented in the proofs of Proposition 13, Corollary 14 and Corollary 15, using the preliminary results established in Proposition 21 and Proposition 22.

Proof of Theorem 3. Following the same approach as in the proof of Theorem 2, the results in Theorem 3 immediately follow by combining Proposition 22, Corollary 23 and Proposition 24 applied to solutions of the PDE (5).

D Proof of Theorem 4

The proof for this theorem uses similar strategies as that of Theorem 3, but considers the evolution of an ODE rather than a PDE. Recall that for any x ∈ R d the best response r ( x )( · ) ∈ P ( R d ) in (6) is defined as

where the energy G c ( ρ, x ) : P ( R d ) × R d ↦→ [ -∞ , ∞ ] is given by

Lemma 25. Let Assumptions 2- 4 hold and assume α ˜ λ > Λ 1 . Then for each x ∈ R d there exists a unique maximizer ρ ∗ := r ( x ) solving argmax ˆ ρ ∈P 2 G c (ˆ ρ, x ) . Further, r ( x ) ∈ L 1 ( R d ) , supp( r ( x )) = supp(˜ ρ ) , and there exists a function c : R d ↦→ R such that the best response ρ ∗ ( z ) = r ( x )( z ) solves the Euler-Lagrange equation

Proof. Equivalently, consider the minimization problem for F ( ρ ) = -∫ f 1 ( z, x ) d ρ ( z ) + αKL ( ρ | ˜ ρ ) + 1 2 ∫ ρW ∗ ρ with some fixed x . Note that we can rewrite F ( ρ ) as

where V ( z, x ) := -( f 1 ( z, x ) + α log ˜ ρ ( z )) is strictly convex in z for fixed x by Assumptions 2 and 4. Together with Assumption 3, we can directly apply the uniqueness and existence result from [CMV03, Theorem 2.1 (i)].

The result on the support of r ( x ) and the expression for the Euler-Lagrange equation follows by exactly the same arguments as in Corollary 12 and Corollary 23.

Lemma 26. The density of the best response r ( x ) is continuous on R d for any fixed x ∈ R d .

Proof. Instead of solving the Euler-Lagrange equation (36), we can also obtain the best response r ( x ) as the long-time asymptotics for the following gradient flow:

Following Definitions 8 and 9, we can characterize the steady states ρ ∞ of the PDE (37) by requiring that ρ ∞ ∈ L 1 + ( R d ) ∩ L ∞ loc ( R d ) with ∥ ρ ∞ ∥ 1 = 1 such that ρ ∞ ∈ W 1 , 2 loc ( R d ) , ∇ W ∗ ρ ∞ ∈ L 1 loc ( R d ) , ρ ∞ is absolutely continuous with respect to ˜ ρ , and ρ ∞ satisfies

in the sense of distributions. Noting that because the energy functional F ( ρ ) differs from G a ( ρ, µ ) only in the sign of f 1 ( z, x ) if viewing G a ( ρ, µ ) as a function of ρ only. Note that F ( ρ ) is still uniformly displacement convex in ρ due to Assumption 4. Then the argument to obtain that ρ ∞ ∈ C ( R d ) follows exactly as that of Lemma 8.

Lemma 27. Let i ∈ { 1 , ..., d } . If the energy H i : P ( R d ) → R d given by

admits a critical point at x ∈ R d , then the best response r ( x ) ∈ P ( R d ) is differentiable in the i th coordinate direction at x ∈ R d . Further, the critical point of H i is in the subdifferential ∂ x i r ( x ) .

Proof. First, note that DF [ r ( x )]( x )( u ) = 0 for all directions u ∈ C ∞ c ( R d ) and for all x ∈ R d thanks to optimality of r ( x ) . Here, DF denotes the Fréchet derivative of F , associating to every ρ ∈ P ( R d ) the bounded linear operator DF [ ρ ] : C ∞ c → R

and we note that F ( ρ ) depends on x through the potential V . Fixing an index i ∈ { 1 , ..., d } , and differentiating the optimality condition with respect to x i we obtain

Both terms can be made more explicit using the expressions for the Fréchet derivative of F :

and for the second term note that the second Fréchet derivative of F at ρ ∈ P ( R d ) along directions u, v ∈ C ∞ c ( R d ) such that (supp( u ) ∪ supp( v )) ⊂ supp( ρ ) is given by

In other words, assuming supp( r ( x )) = supp(˜ ρ ) = R d , relation (40) can be written as

For ease of notation, given r ( x ) ∈ P ( R d ) , we define the function g : P ( R d ) → L 1 loc ( R d ) by

The question whether the partial derivative ∂ x i r ( x ) exists then reduces to the question whether there exists some ρ ∗ ∈ P ( R d ) such that ρ = ρ ∗ solves the equation

and for some constant c > 0 . This is precisely the Euler-Lagrange condition for the functional H i defined in (39), which has a solution thanks to the assumption of Lemma 27.

We observe that the first term in H i is precisely (up to a constant) the χ 2 -divergence with respect to r ( x ) ,

Depending on the shape of the best response r ( x ) , the χ 2 -divergence may not be displacement convex. Similarly, the last term -∫ ∂ x i f 1 ( z, x )d ρ ( z ) in the energy H i is in fact displacement concave due to the convexity properties of f 1 in z . The interaction term is displacement convex thanks to Assumption 3. As a result, the overall convexity properties of H i are not known in general. Proving the existence of a critical for H i under our assumptions on f 1 , f 2 , ˜ ρ and W would be an interesting result in its own right, providing a new functional inequality that expands on the literature of related functional inequalities such as the related Hardy-LittlewoodSobolev inequality [Lie83].

It remains to show that H i indeed admits a critical point. Next, we provide examples of additional assumptions that would guarantee for Lemma 27 to apply.

Lemma 28. If either C := sup R d | W ( z ) | < ∞ , or

then for each x ∈ R d and for large enough α > 0 , the best response r ( x ) is differentiable with the gradient coordinate ∂ x i r ( x ) given by the unique coordinate-wise solutions of the Euler-Lagrange condition for H i .

Proof. We will show this result using the Banach Fixed Point Theorem for the mapping T i : L 1 ( R d ) → L 1 ( R d ) for each fixed i ∈ { 1 , ..., d } given by

noting that ρ ∗ = T i ( ρ ∗ ) is the Euler-Lagrange condition for a critical point of H i . It remains to show that T i is a contractive mapping. For the first assumption, note that

Similarly, for the second assumption we estimate

which requires a bound on ∥ W ∗ r ( x ) ∥ ∞ . Using

we conclude that T i is a contraction map for large enough α . In both cases, we can then apply the Banach Fixed-Point Theorem to conclude that ∇ x r ( x ) exists and is unique.

Lemma 29. Let r ( x ) as defined in (6) . If r ( x ) is differentiable in x , then we have ∇ x G d ( x ) = ( ∇ x G c ( ρ, x )) | ρ = r ( x ) .

Proof. We start by computing ∇ x G d ( x ) . We have

where we used that r ( x ) solves the Euler-Lagrange equation (36) and that r ( x ) ∈ P ( R d ) for any x ∈ R d so that ∫ r ( x )( z )d z is independent of x .

Lemma 30. Let Assumption 1 hold. Then G d : R d → R ∪ { + ∞} is strongly convex with constant λ d := λ 1 + λ 2 + β > 0 .

Proof. The energy G c ( ρ, x ) is strongly convex in x due to our assumptions on f 1 , f 2 , and the regularizing term ∥ x -x 0 ∥ 2 2 . This means that for any ρ ∈ P ,

Selecting ρ = r ( x ′ ) , we have

Since G c ( r ( x ′ ) , x ) ≤ G c ( r ( x ) , x ) by definition of r ( x ) , we obtain the required convexity condition:

Proof of Theorem 4. For any reference measure ρ 0 ∈ P , we have

and therefore, G d is coercive. Together with the strong convexity provided by Lemma 30, we obtain the existence of a unique minimizer x ∞ ∈ R d . Convergence in norm now immediately follows also using Lemma 30: for solutions x ( t ) to (6), we have

A similar result holds for convergence in entropy using the Polyák-Łojasiewicz convexity inequality

which is itself a direct consequence of strong convexity provided in Lemma 30. Then

E Additional Simulation Results

We simulate a number of additional scenarios to illustrate extensions beyond the setting with provable guarantees and in the settings for which we have results but no numerical implementations in the main paper. First, we simulate the aligned objectives setting in one dimension, corresponding to (4). Then we consider two settings which are not covered in our theory: (1) the previously-fixed distribution ¯ ρ is also time varying, and (2) the algorithm does not have access to the full distributions of ρ and ¯ ρ and instead samples from them to update. Lastly, we illustrate a classifier with the population attributes in two dimensions, which requires a different finite-volume implementation [CCH15, Section 2.2] than the one dimension version of the PDE due to flux in two dimensions.

E.1 Aligned Objectives

Here we show numerical simulation results for the aligned objectives case, where the population and distribution have the same cost function. In this setting, the dynamics are of the form

where f 1 and f 2 are as defined in section 4.1, and W = 1 20 (1 + z ) -1 , a consensus kernel. Note that W does not satisfy Assumption 3, but we still observe convergence in the simulation. This is expected; in other works such as [CMV03], the assumptions on W are relaxed and convergence results proven given sufficient convexity of other terms. The regularizer ˜ ρ is set to ρ 0 , which models a penalty for the effort required of individuals to alter their attributes. The coefficient weights are α = 0 . 1 and β = 1 , with discretization parameters dz = 0 . 1 , dt = 0 . 01 .

0.8

Q

d

0.6

-p

probability

X

0.4

0.2

0.0

-4

-2

0

2

4

6

8

Z

In Figure 4, we observe the strategic distribution separating itself from the stationary distribution, improving the performance of the classifier and also improving the performance of the population itself. The strategic distribution and classifier appear to be stationary by time t = 40 .

E.2 Multiple Dynamical Populations

We also want to understand the dynamics when both populations are strategic and respond to the classifier. In this example, we numerically simulate this and in future work we hope to prove additional results regarding convergence. This corresponds to modeling the previously-fixed distribution ¯ ρ as time-dependent; let this distribution be τ ∈ P 2 . We consider the case where ρ is competitive with x and τ is aligned with x , with dynamics given by

We use W = 0 and f 1 , f 2 as in section 4.1 and the same discretization parameters as in Section E.1. In Figure 5,

1.0

~Q

0.8

p

0.6

T

X

prob

0.4

0.2

0.0

2

-1

0

1

2

3

4

5

Z

we observe that the τ population moves to the right, assisting the classifier in maintaining accurate scoring. In contrast, ρ also moves to the right, rendering the right tail to be classified incorrectly, which is desirable for individuals in the ρ population but not desirable for the classifier. While we leave analyzing the long-term behavior mathematically for future work, the distributions and classifier appear to converge by time t = 20 .

E.3 Sampled Gradients

In real-world applications of classifiers, the algorithm may not know the exact distribution of the population, relying on sampling to estimate it. In this section we explore the effects of the classifier updating based on an approximated gradient, which is computed by sampling the true underlying distributions ρ and ¯ ρ . We use the same parameters for the population dynamics as in section 4.1, and for the classifier we use the approximate gradient

First, we simulate the dynamics with the classifier and the strategic population updating at the same rate, using α = 0 . 05 , β = 1 , and the same consensus kernel as used previously, with the same discretization parameters as in E.1. In Figure 6, we observe no visual difference between the two results with n = 4 versus n = 40 samples, which suggests that not many samples are needed to estimate the gradient.

Next, we consider the setting where the classifier is best-responding to the strategic population.

Unlike the first setting, we observe in Figure 7 a noticeable difference between the evolution of ρ t with n = 4 versus n = 40 samples. This is not surprising because optimizing with a very poor estimate of the cost function at each time step would cause x t to vary wildly, and this method fails to take advantage of correct "average" behavior that gradient descent provides.

sampledgradientunderequalrate,n=4

0.75

t=0.0

p

t=6.0

p

t=10.0

p

t=30.0

p

lity

Dabil

0.50

d

p

d

p

prob

0.25

X

od

od

od

0.00

2

0

2

4

2

0

2

4

2

0

2

4

-2

0

2

4

Z

Z

Z

Z

sampledgradientunderequalrate,n=40

0.75

t=0.0

p

t=6.0

p

t=10.0

p

t=30.0

p

obabi

0.50

d

p

p

p

0.25

od

od

od

prol

X

0.00

-2

0

2

4

2

0

2

4

-2

0

2

4

-2

0

2

4

Z

Z

Z

Z

sampled gradientunderequalrate,n=4

ility

0.75

t=0.0

p

t=6.0

p

t=10.0

-Q

t=30.0

p

0.50

p

p

p

p

obabi

0.25

od

od

od

prol

X

0.00

2

0

2

4

-2

0

2

4

-2

0

2

4

2

0

2

4

Z

Z

Z

Z

sampledgradientunderequalrate,n=40

0.75

t=0.0

p

t=6.0

p

t=10.0

p

t=30.0

p

0.50

d

p

p

p

obabi

0.25

od

od

od

prol

0.00

2

0

2

4

-2

0

2

4

-2

0

2

4

-2

0

2

4

Z

Z

Z

Z

E.4 Two-dimensional Distributions

In practice, individuals may alter more that one of their attributes in response to an algorithm, for example, both cancelling a credit card and also reporting a different income in an effort to change a credit score. We model this case with z ∈ R 2 and x ∈ R 2 , and simulate the results for the setting where the classifier and the population are evolving at the same rate. While this setting is not covered in our theory, it interpolates between the two timescale extremes.

We consider the following classifier:

with W = 0 . Again, the reference distribution ˜ ρ corresponds to the initial shape of the distribution, instituting a penalty for deviating from the initial distribution. We use α = 0 . 5 and β = 1 for the penalty weights, run for t = 4 with dt = 0 . 005 and dx = dy = 0 . 2 for the discretization. In this case, the strategic population is competing with the classifier, with dynamics given by

In Figure 8, we observe the strategic population increasing mass toward the region of higher probability of being labeled "1" while the true underlying label is zero, with the probability plotted at time t = 4 . This illustrates

Po

p

-5

-5

0.04

0.04

0

2

0

0.02

0.02

5

5

-5

0

5

-5

0

5

Z1

Z1

d

probability

-5

5

0.04

0.4

0

0

0.02

0.2

5

-5

0.0

-5

0

5

-5

0

5

Z1

Z1

similar behavior to the one-dimensional case, including the distribution splitting into two modes, which is another example of polarization induced by the classifier. Note that while in this example, x ∈ R 2 and we use a linear classifier; we could have x ∈ R d with d > 2 and different functions for f 1 and f 2 which yield a nonlinear classifier; our theory in the timescale-separated case holds as long as the convexity and smoothness assumptions on f 1 and f 2 are satisfied.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider a the problem where the change in a distribution for an objective can be modeled as a set of coupled nonlinear parabolic PDEs. These methods have nonlocal interactions and describe a model of the influence of the model on the population and vice-versa. Additionally, these correspond to the evolution of the measures associated with the Wasserstein gradient flow that minimizes a set of defined energy functionals, which relate to the original optimization problem. Such equations have been well studied in terms of the granular media equation and a wide theory has been devoted to existence and uniqueness of their solution. Two scenarios are considered: one with cooperation and another with an adversarial behavior of the population. The main results of the paper are convergence rates to the steady state for the coupled PDEs. The behavior of both the optimization algorithm and the population are studied in some numerical experiments at the end of the paper. The numerical experiments illustrate the importance of considering such a model rather than relying on more simple summary statistics for notions of distributional shift.\n\nSTRENGTHS:\nThe proposed model is a very elegant model for describing distribution shift and the feedback mechanisms between the shift in population and objective functions. It provides a much richer class of perturbations than what is generally seen in the literature in cases such as distributionally robust optimization or in adversarial optimization. The model also has implications in providing additional distribution information (beyond low order moments) at equilibrium. The authors show the importance of this where different geometries of the distribution can imply different effects on the population distribution. Overall, the paper is well written, provides nice ideas, and describes a more unique take on the problem of distribution shifts.\n\nWEAKNESSES:\nThe numerical evaluation is a bit limited, but I think that’s not a problem since most of the results are theoretical. Still, it would be nice to see some more concrete applications of the theory by simulating the PDEs described, where, for example, all components are not simulated. It also appears that this may be difficult to apply to real world scenarios as the authors alluded in their limitations. A real application where all components need to be estimated was not discussed, but that's beyond the scope of the paper. \n\nSome of the assumptions are somewhat strong, but these are usually made to provide convergence and existence of solutions of these PDEs. In that regard, processes that satisfy these assumptions may not be entirely general, but that’s not a big deal since the authors clearly constrain their analysis based on the assumptions they make.\n\nQUESTIONS:\nThe second equation should be the argmin of x \\in R^d?\n\nIs there intuition on how the different functionals could be estimated e.g. for a particular application? Or is it assumed that these are already known? \n\nRelated to the previous question, how difficult would it be to apply to a real scenario, beyond the simulations the authors provided? This may not be a focus of the paper, but seems like something worth discussing. \n\nCould particle methods be used to solve the problem in high dimensions as is sometimes done in high dimensional cases? For example, this was explored in [1] and I was wondering if it could be applied in this scenario by approximating the associated McKean-Vlasov process (there might be an issue with some of the terms in the PDEs)? \n\n[1] Crucinio et al, Solving Fredholm Integral Equations of the First Kind via Wasserstein Gradient Flows, 2022\n\nLIMITATIONS:\nThe authors sufficiently discuss limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-04"}

comments: SUMMARY:
This paper studies the long-term behavior of a dynamical system where there are distribution shifts in response to algorithmic decision-making. The authors derive PDEs to capture distributional changes and describe the result of the algorithm for both the coorperative setting and the competitive setting. Under certain regularization conditions, it is shown that the PDEs converge to unique steady-state distribution. Finally, some numerical simulations are conducted to iilustrate the problem.

STRENGTHS:
1. The paper is well-written and not hard to read. All definitions, theoretical results and experiments are accompanied by explanations when necessary. Moreover, the authors motivate the problem of interest in a very convincing way.

2. The paper is technically solid. The process of interaction between algorithm and population is rigorously modelled and its asymptotic behavior is characterized.

WEAKNESSES:
The authors do not include many discussions on the insights provided by the theoretical findings. This is probably because the limit distribution of the PDEs are difficult to characterize. Moreover, the setting considered is over-simplified since it only contains one agent. Given that this paper is kind of a first step towards mathematical analysis of algorithm interaction, I think these flaws are totally acceptable and can be left for future work.

QUESTIONS:
1. Many real-world applications involve discrete rather than continuous time. Is it possible to generalize the analysis in this paper to discrete time setting?

2. Is it reasonable to assume that $f_1,f_2$ are convex?

LIMITATIONS:
The authors have adequately addressed the limitations and potential negative societal impact of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

comments: SUMMARY:
This paper presents a framework for analyzing dynamics of distribution shift that captures feedback loop between learning algorithms and the distributions that they are deployed on using a coupled differential equations model. The paper considers both co-operative and competitive settings and suggests that a learner updated with gradient descent converges to steady state both in finite and infinite dimensions (in terms of model parameters).

STRENGTHS:
The paper presents a rigorous treatment of the problem and presents general results, although I am not an expert in this area and cannot present a rating with any meaningful confidence.

WEAKNESSES:
The paper works with several assumptions that I do not see a clear cut comparison of how these assumption compares against prior work, so it is unclear how general these results are.

QUESTIONS:
One way to make the paper appeal to a general audience is to have a discussion surrounding claims and assumptions comparing this work against prior results in the literature.

LIMITATIONS:
n/a
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new framework for analyzing the interaction and feedback loop between learning algorithms and the distributions on which they are deployed. The framework in the paper is more general than prior models, allowing to capture more complex interactions and shifts in the algorithm and distribution. This is done using a partial differential equation that modifies the algorithm\\distribution in a way that optimizes its cost with respect to the choice of distribution\\algorithm. Under several assumptions which include strong convexity, the authors were able to prove that gradient flow converges and provide fast convergence rates. The authors are provide experimental results for a simple 1-dimensional setting which demonstrate that their framework can explain general phenomenon which are not explained by prior models, such as polarization.\n\nSTRENGTHS:\nThe paper is well-written and easy to follow. Moreover, as mentioned above, the framework in the paper offers several advantages compared to prior frameworks, allowing to capture more complex interactions and shifts of the distribution. This model is therefore able (via simple examples) to theoretically explain forms of distribution shifts which we see in practice such as polarization.\n\nWEAKNESSES:\n1. While the framework allows for complex forms of shifts of the model and distribution, the analysis assumes that the model\\distribution changes to be the exact minimizer of a certain cost function. This may be a little strict in general. Can the same analysis work when the shifts are approximate minimizers for the cost function?\n\n2. Extreme timescale setting: the analysis in the paper only considers two extreme timescale settings: (i) the algorithm responds much faster than the population, and (ii) the population responds much faster than the algorithm. This is somewhat strict and doesn’t allow for intermediate changes in the algorithm\\distribution. \n\n3. Assumptions: the paper has many assumptions, some of which are less common. The authors should give some motivation and justification for their assumptions, and provide examples where all of these assumptions hold.\n\n4. The experiments are a little simple (1-dimensional classifiers) and it would be more interesting to see an example which is more relevant in practice. This is perhaps one of the limitations of this framework: even though it is able to capture more complex structures of shifts, it is harder to simulate for more sophisticated examples and models (e.g. high dimensional classifiers, other timescale settings, and so on). The authors should therefore also discuss the limitation of their framework compared to existing ones. \n\n5. Experimental discussion: the experimental section jumps too fast to the experiments without explaining the objectives of these experiments or discussing its connections to the previous theoretical results. The authors should explain their main objectives in this setting, discuss connection to the theory in the previous section, and also compare to existing prior work (e.g. show how the example in section 4.1 cannot be captured by prior frameworks).\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nSee above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-26"}