A Fisher-Rao gradient flow for entropic mean-field min-max games
Authors: Razvan-Andrei Lascu
Mateusz B. Majka
Lukasz Szpruch
Venue: Accepted by TMLR ()

Abstract: Gradient flows play a substantial role in addressing many machine learning problems. We examine the convergence in continuous-time of a Fisher-Rao (Mean-Field Birth-Death) gradient flow in the context of solving convex-concave min-max games with entropy regularization. We propose appropriate Lyapunov functions to demonstrate convergence with explicit rates to the unique mixed Nash equilibrium.

Published in Transactions on Machine Learning Research (08/2024)
A Fisher-Rao gradient flow for entropic mean-field min-max games
rl2029 uk School of Mathematical and Computer Sciences, Heriot- Watt University, Edinburgh, UK, and Morwell Institute for Mathematical Sciences, Edinburgh, UK
Razvan-Andred Lascu
Mateusz B. Majka
mh m.jajka &hor ac. uk School of Mathemafical and Computer Sciences, Heriot-Waid University, Edinburgh, UK, and Marwell Institute for Mathematical Sciences, Edinburgh, UK
Lukasz Szpruch
Lszpruch@edd. uk School of Mathematics, University of Edinburgh, UK, and The Alan Turing Institute, UK and Simtopia, UK
Reviewed on OpenReview: https: // openreview. net/ forum? Ad=Afc2CuRRaR
Abstract
Gradient flows play a substantial role in addressing many machine learning problems. We examine the convergence contimous-time of Pisher- Rao (Mean-Field Birth-Death) gra- dient flow in the context of solving convex-concave min-max games with entropy regulariza- lion. We propose appropriate Lyapunov functions to demonstrale convergence with explicit rates to the unique mixed Nash equilibrium.
1 Introduction
The rapid progress of machine learning (ML) techniques such Generative Adversarial Networks (GANs)
(Goodfellow et al. 2014), adversarial learning (Madry et al. 2018), multi-agent reinforcement learning Zhang et al. 2021 has propelled surge of interest in the study of optimization problems on the space of probability measures recent years. Particularly noteworthy are the numerous works, e.g., (Hsieh el al.
2019 Domingo-Fann Enrich et al. 2020 Wang & Chizat, 2023 Lu 2023 Trillos Trillos 2023 Kim et al. (024). illustrating how training GANs and addressing adversarial robustness can be cast as min-max games over probability measures. In this setting, understanding the time evolution of the players' initial strategies to the equilibrium of the game leverages the use of gradient flows on the space of probability measures. A discussion about the applications of gradient flows in optimization and sampling can be found in recent
survey (Trillss et al. 2023
The Fisher-Rao (FR) gradient flow has been recently studied the context of mean-field optimization problems (Liu al. 2023), accclcrating Langevin-basod sampling from multi-modal distributions (Lu al. 2019 2023), and training shallow neural networks in the mean-field regime (Rotskoff et al., 2019). The motivation for employing FR dynamics in sampling from multi-modas distributions is their ability to globally transport the mass of probability density between modes without traversing low-probability regions (Lu) et al. 2019). In the context of training neural networks, the benefit of using FR dynamics is similar in that they have the potential capability to avoid local minima the loss function Rotskoff et. al. 2019).
The present paper aims extend these results and focuses on the contimuous-time convergence of the Fisher-Roo (FR) gradient flow the unique mixed Nash equilibrium of an entropy-rogularized min-max game.

---

Published in Transactions on Machine Learning Research (08/2024)
1.1 Notation and setup
For any 2 C Rd, denote by (2) the space of probability measures on E which are absolutely con- tinuous with respect to the Lebesgue measure. Following a standard convention, elements in Pac denote probability mcasures as well as their densitics. Let X, y S Rd and fix two reference probability measures (dz) & e === E (X)) and p(dy) & e-Ur(w) E ()c where U : :x + R and UP :y + R are two measurable functions. The relative entropy DxL(-1=> P(2) - [0, oo] with respect to n is given by
{ [xx 1x log
p(r)dz, if is absolutely continuous with respect to T x())
M(xrr
(KL(U|T=
(00,
otherwise,
and we define (HL(H)p) analogously for any P E P()). Let F P(Y) x PD) + R be a convex-concave (possibly non-linear) function and a > be a regularization parameter. We consider the min-max problem
o² min max V (v, with V (v, F(u.u) + 2 (DRI.(U))) - (Rc((l))) (EP()))))))
(1)
In order to solve
11, one typically aims to identify mixed Nash equilibria (MNEs) (von Neumann et. al. characterized by pairs of measures (vG (G) E P(X) x P()) that satisfy
1944
Nash 1951
(V S V (va 12) V (V, for all (u, E P(X) P(J)).
(2)
It is important to highlight that when F is bilinear and O, i.e., (2 = fy fxx Ma, s)o(dz))(dy)) for some measurable function f f : x - I, measures characterized by 2 represent MNEs the classical sense of a two-player zero-sum game, Results concerned with the existence and uniqueness an MNE for (T)
are presented in Appendix A in Lascu et al. (2023) (alternatively, see Appendix C.2, C.3 in Domingo-Enrin Enrichh et al. (2020) for the case when F is bilinear). Tt is also proved in Lascu et. al. (2023) that V T-converges to F as 1 O, under mild assumptions F, π and p.
In optimization, the monotonic decrease of the objective function along the gradient flow is key to proving convergence. However, for min-max games the monotonic decrease no longer holds due to the conflicting actions of the players. Hence, a suitable Lyapunov function is needed. common choice is the so-called Nikaidò-Isoda (NI) error (Nikaidó & Isoda 1955). which, for all (M, E (X)) x P ()), can be defined as (V(V'n))
p) = mmax VO (V, min ('EP())
v'E(X)
From the saddle point condition 2 it follows that NI(U, 0 and NI(v, μ4) 0 if and only if (v, is
MNE. We will also consider alternative Lyapunov function given by (6)
In what follows, we will introduce the FR gradient flow on the space (PaclX) x (ao())), where FR is the Fisher-Rao Rao distance defined by (3)
1.2 Fisher-Rao (mean-field birth-death) gradient flow
The Fishcr-Rau metric was introduced by Rao 1992) via the Fisher information matrix, and since then has been extensively studied in the context of information geometry (Amari 2016 Ay et al. 2017). In this work, we are mainly focusing on the the dynamical formulation of the Fisher- Rao metric (see e.g. Section 2.2 in Gallouët & Monsaingeon| (2017), Appendix C.2.1 in Yan et al. (2023) and Section 3.3 in KKodratyyv & Vorotnikov (2019)). For any λu A1 (Mc(MM, with M R., the variational representation of the FR distance is given by
- (dd)) (dr)dds: O.Ae de (rr: (r= re(e))h((te)) }. (3)
FRo (Ao. A1) = inf
where the infimum taken over all curves [0, 1] 3 + (M1, E Pac(M) 12(M:A) solving
0,7₂ = d ( relu))o(dy)
(4)
2

---

Published in Transactions on Machine Learning Research (08/2024)
in the distributional sense, such that t ++ A is weakly continuous with endpoints Ao and X1. The reaction term ri (3r) E R is scalar that dictates how much mass is created/destrroed at time t > 0 and position x B R. The integral in (4) guarantees that X is a probability measure for all 2 0.
Inspired by Liu et al. 2023), we consider the Fisher-Rao (mean-field birth-death) gradient flow on the space (Pa x Pac ()))))). FR) in the setting of (1). As opposed to the mean-field Best Response dynamics studied in Lascu et al. (2023), , which is another flow that can be used to solve [ and which relies on introducing fixed point perspective on min-max games, the FR dynamics utilize gradient flow (M4, in the Fisher-Rao geometry, As a first attempt at defining Fisher- Rao gradient flow for solving (L) consider
[)vr(z) 8V (Vi, x)), 0u (1,, ())),
5V" Av
(5)
(op((v)=
with initial condition (vo, no) E (x) x ()). We adopt the convention that
Jx 8V dv (v,, x), = / oV oμ (vr, () = =
since the flat derivatives of V are uniquely defined up to an additive shift (see Definition B.1 in Section Thus, the total mass 1 of probability measures is still preserved along the gradient flow. A formal derivation of the Fisher-Rao gradient flow can be found in Appendix C
Remark 1.1. Suppose v is bilinear payoff function and any mired strategies (V, supported finite sets of m and n pure strategies X = { "Im and y = {vi, Vin.V2.".. Unt. respectively. Then V E P(X) and E P(Y) can be approzimated by the empirical measures v" = J. Em= de, and μ* = 1 i = oy respectively, and hence the Fisher- Rao gradient flow (5) retrieves the replicator dynamics studied in evolutionary game theory [Cressman 8 Tuo 2014; Hofbwuer & Sigmmndd 1998), see also [Abe et al. 2022
for a related replicator appicattormmatntt mutant dynamies.
1.2.1 Sketch of convergence proof for the FR gradient flow
For the sake of presenting an intuitive hcuristic argument, we ignore here for now that the flat derivatives (v. ar)) + sye de (v, μ1, x) and (u, + dy 51" (v, may not exist for the V defined relative entropy term DKL being only lower semicontinuous with respect to the weak convergence topology (for this reason, in our analysis Section 2 auxiliary functions a and b). Nevertheless, will now demonstrate that choosing the flow (Vi, p4)20 as in
due to the
we will replace these two derivatives with appropriately defined
5 makes the function
+ (KL( + DKL(UEPR)
(6)
decrease in t, under the assumption of convexity-concavee of F. Let (v, E x Then assuming the existence of the flow (Mes Pc)22 satisfying 5 and the differentiablity of the map + DKL (xl4) + (H(p)) we formally have that
d dt (Dki((//x)) + DKI.(A)m))
= - d: (u(x)) v(x)
dx+ + / O. (w()) (w()) log [(V)
v1(z)) dx
dy
= =- / (p(x) - vx(x)) dev((z) dx - / (u(v) -M((i)) (eple(u) dy
vx(x)
y
(r(v)
= f 8V
dy (vr, Hi, z)(v v.) / 8V
(vi (vi, v) (d)
where the second equality follows from the fact that I ( = (yddy = 0. Then, assuming that + F(v, μ)) and μ r> J(), are convex and concave, respectivcly (see Assumption D., we observe that
v シ (o n) and -+ (v u) are o-strongly-convex and o-strongly-coneave relative to DRI., respectively
(see Lemma
3.1 in Section A that is
(v, V (vt, {
8V
d²
(dx()) + DKL(vlp)). dv (1,, (v)xxx)))
3

---

Published in Transactions on Machine Learning Research (08/2024)
-V & J/ óu (,, -dldd) a²
oV
2 (KL(H))))
Therefore, we obtain that
d dt (DKL(vlp)) (KL(()))) (v, (o(, + (ou.u)) (v(,H)
o²
0²
DKI(U)ps) DKI((llu). (7)
2
Setting (v, (G) in 7 using the the saddle point condition and applying Gronwall's inequality gives
2 i.e, (" (m S 0,
(KL(V)lV) + DKI(U)le)) se - (VKL(V (n) + D
(8)
Since (KI (Kr((Glm) 2 0 with equality if and only if v Vn and μ 14r follows that the unique MNE of is achieved with exponential rate e -
To prove convergence in terms of the N1 error, first observe that, for any (v, p) E P(X) x P()), we can write (2 - (v, (v), (V (o + (v, V (v, kT) V (v,
By Theorem 2.2 and Assumption
2 there exist Ci C2 > 0 depending such that
8V (v),
(v. (r, w) s C2.7.
S C1
5v
for all (u) E P(X) P()), and all (x,y) E X x Y. Then, by Lemma 3.1 and Pinsker's inequality,
show that
(vt, - Vo (V( VV V (u, < 2C, / + (KI(l) (ouu)
where C₂ maxa((a J. Hence, by we obtain
(o(,,, V (v, 20.e of " DKL(VGlUo) (KL(H)(H)) + (V,, (V,
(9)
Since (VSHHC) is the MNE of we have
max V (vt, min V (o (2) (vG
Therefore, maximizing over (v, E P(X) x P()) in
gives
N1(p, S -0 . V + DKL (u
Finally, observe that, by 18
we have
(vLlv)) + DKL(HElup) < oc.
1.3 Our contribution
We prove the existence of the FR gradient flow (Pt, pr)co and show that it converges with rates O (( (e-$1) and o (e +t the unique MNE of with respect to the Lyapunov functions + (KI(V)lV)) + DRI( (4)) and t + NT pt)-
$1)
4

---

Published in Transactions on Machine Learning Research (08/2024)
1.4 Related works
Recent intensive research has been dedicated to examining the convergence of the Wasserstein gradient flow to MNEs within the specific formulation of game ( in which F is bilinear (F(v, μ) - ly fx f(x, ())()) and regularized by entropy rather than relative entropy DKL. The spaces X' and y are assumed to be either compact smooth manifolds without boundary, embedded in the Euclidean space, Euclidean tori, while f cxhibits sufficient regularity, typically being at least continuously differentiable with Lipschitz conditions satisfied by Va. and Vy This line of research is explored in works such as Domingo-Enree et al. (2020)
Ma & Ying 2022 : Lu 2023
: Wang & Chizat
2023
In this context, Ma Ying 2022): : Lu (023 investigate the convergence of the Wasscrstein gradient flow,
proving exponential convergence to the MNE when the players' flows (u))220 and (μ4))20 converge at different
rates. In Ma & Ying (2022), the analysis considers the scenario where one player's flow reaches equilibrium
while the other remains governed by the Wasserstein gradient flow equation. Notably, Ma & Ying (2022) Theorem 5) shows the convergence (without explicit rate) of the flow (c)20) to the unique MNE under these separated dynamics.
On the other hand, Lu (0233) examines the situation where the players' flows evolve at varying speeds but with finite timescale separation, meaning that neither player has reached equilibrium. Here, Lu (2023 Theorem 2.1) proves exponential convergence of the finitely timescale-separateed Wasserstein gradient flow to the unique MNE, with the convergence rate depending upon regularization and timescale separation parameters. In contrast to Ma & Ying (2022); Lu 2023), we prove that the Fisher-Rao gradient flow (5) converges exponentially fast to the unique MNE while players' dynamics converge at the same speed.
Other works such as (Domingo-Enrice Ennich et al. 2020 Wang & Chizat 2023) focused on the convergence of the Wasserstein--isher--e
(WFR) gradient flow, combining both the Wasserstein gradient flow (allowing parti- cles to diffuse in space) and the Fisher-Rao gradient flow (forcing particles to evade low probability regions). Assuming that F is bilinear and o 0, Enniih et al. (2020) investigates the Wasserstein-Fisher Rao gradient flow's convergence (without giving explicit rates). For to > 0 (dependent on the parameters governing the individual contributions of the Wasserstein and Fisher- Rao components in the WFR gradi- ent flow), and under circunstances where the Fisher-Rao component predominates over the Wasserstein component, Domingo-Entin Enrich et al.] (2020) prove a characterization result of e-approximate MNE for the WFR gradient flow. More precisely; Domingo-Enrich et al. (2020) Theorem 2) establishes that the pair to 16 vyds, to So p.ds) is an e-approximate MNE of the game, i.e., NI 1 f v,ds, 1 to
11
1.
to So uads) S e, for
any arbitrarily chosen >>0 0.
(2023 introduces a proximal point method that can be viewed as a discrete-time counterpart of the WFR gradient flow. Working within the framework of bilinear F, with o = 0, and unique MNE, Wang & Chizal (2023) Theorem 2.2) establishes the local exponential convergence of the iterates to the unique MNE of the game with respect to the NI error and the WFR distance, provided that the initialization is done in close vicinity to the MNE. The algorithm proposed by WWng & Chizat (2023)
Lastly, Wang & Chizat
assurees that both players update the positions and weights of the particles simultaneously. However, this is not the only possible discretization for the WFR gradient flow. As highlighted in Lascu et al. (2024) for the discrete-time Fisher-Rao gradient flow, the convergence rates of the dynamics in a game differ depending on the order the players move: either simultancously (players move at the same time) or sequentially (cach player moves upon observing the opponents' moves). Although the continuous-time analysis of the gradient flow docs not capture these two situations, it can serve as guide for designing discrete-time approximations of implementable algorithms.
Moreover, it is argued in Wang & Chizat 2023 that the results of Theorem 2.2 hold only for discrete-time algorithms, and do not imply the convergence of the contimuuu--time WFR gradient flow because sending the step-sizes of the time discretization to zero will force the initializaaion (vo, (D) ) to be already an MNE (see the discussion after Theorem 2.2). Unlike the assumption in Waan & Chizat (2023) that guarantees convergence of the discrete-time algorithm as long as the initialization is close to the MNE, i.c., NI(2 I ro, for some To > 0, depending on the step-sizes, our warm start condition (Assumption 4 for the Fisher-Raa gradient flow only imposes absolute continuity and comparability with priori known reference measures and p.
5

---

Published in Transactions on Machine Learning Research (08/2024)
Furthermore, Wang & Chizat 2023) discuss that if (Vos Mo) is supported the entire space X' and V. respectively, then it is expected that the convergence of the discrete-time Fisher-Rao dynamics occurs at sub-linear rate in the worst case, and not an exponential rate (see Appendix D Also, if the strategy spaces X' and y are finite and the initialization (o))/o) is supported on a large mumber of points uniformly covering and V, respectively, there is no guaranlee for last-iterate convergence of discrete-time Fisher-Rao dynamics to an MNE of Vo. As showed by Wei et al. (2021); Daskalakis & Panageas (2019). last-iterate convergence guarantees for games on finite strategy spaces X' and y hold under the assumption that the MNE is unique, which may not hold for Vo.
MNE
While it scems natural to combine the Wasserstein and the Fisher-Rao gradient flows, rigorously proving explicit convergence rates for the continuous-time WFR gradient flow for games such as L is challenging and, to our knowledge, still open problem. In this work, we provide an additional step by establishing last-iterate exponential convergence of the Fisher-Rao gradient flow the unique MNE measured in both KL divergence and NI error (sce Theorem 2.3
2
Main results
As we explained the introduction, we study the convergence of the FR gradient flow to the unique MNE of the reguaax-eg two-player zero-sum game given by (1, where F P (X) x x (D) E. is a linelrnear function and 0. Throughout the paper, have the following assumptions.
Assumption (Convexixx- of P). Suppose F admits first order flat derivotives with respect to both and μ stated in Definition B.1 Furthermore, suppose that F is conver in and concove in p, i.e., for any v, / E E (2) and any p.p E P (Y), we have.
F(V. - -FV,)) 2 / 5v
6F (v. (V. -vn) vddxx)
(10)
(o.))) & I, 8µ (v. (v. pdd))
5F
(11)
Assumption (Boundedness of first order flat dcrivatives). Suppose F admits first order flat derivatives with respect to both and μ stated in Desinition B.1 and there exist conslords Cuy Cp 0 such that for all (v, p)) E P(X) P(Y) and for all (x, X V, we have
8F ōu (v, (, CCn,
8F (v, u.w) (v, S C...
op
Assumption 3 (Boundedness of second order flat derivatives). Suppose F admits second order flat deriva- tives stated in Definition B.1 and there erist constants Cuvs Cape Coges Cuv > O such that for all (v. E P(X) x P(Y) and for all (2, (x', (x',y') E X' x V, we have
F duz (V, (1,,, s Cu.,
F dn2 (v, (V,
s Cuups
F ōvōp
8FF (v, p1 R. Cp...
(U, v))) S SCuas
Note that the order of the flat derivatives in / and can be interchanged due to Lascu et al. 2023 Lemma B.2). Using Assumption 3 it is straightforward to check that there exist constants C² C₁, > 0 such that for
all (v, ()) E (P)( x Pao ()), (1/, p/) E (X) x Pac ()) and all (x,y) y) E X V, we have that
8F (v, μ, x)-
8F
(v/ μl x)
< C (TV(R, TV(H,,))
(12)
dv
pv
5F
8F
(v. (0.14,8)
(v' /l
+ TV(,,14).
(13)
8y
6

---

Published in Transactions on Machine Learning Research (08/2024)
Remark 2.1. An ezample of function F which satisfies Assumptions 1 2
3
is F(2,
ly Ix (x. v)d(dr))(dy) for function f X x y - IR which is bounded bul possibly non-conver-non- concave. Indeed, Assumption 1 is trivially satisfied by such F, while Assumptions 2 and & hold due to the boundedness of f. This type of objective function F is prototypical in applications including the training of GANs (see, e.g, Arjousky al. (2077) Hsich al. (2019) and distribulionally robust oplimization (see,
e. Maddy al. 2018)] Sinha et al. 2018))-
Another example that can be viewed
particular case of our general framework is the objective function of Wasserstein- GANs (WGANs) with gradient penalty (Gulrajoni et al., 2017 Petzka et al. 2018). Following Conforti et al. (2023); Kazeykina et al. (2022), GANs can be framed as min-mar problem over the space of probability measures. Consider the following class of parameterized functions as the class of choices for the discriminator:
(x EiYY)) Law(Y) = E P(V)).
The map EY =)) is the output of two-layer newral network with parameters Y and bounded, continuous and non-constare activation function f. The objective function of the GAN reads
J/ f(o.))( -D)(d))((dd) PEP(X) PEP())
min max PEP(2) MEP()) Ey Ey~nIf(Y, (dx) min max
where V' is the distribution of the true data. Assuming that X 3 IIE f(x,v) E R is differentiable and I-Lipschitz, we retrieve the WGAN. The gradient penalty regularization terms proposed by Gulrajani et al. ;; Petzka et al. (2018) are added in order to ensure that the discriminator remains 1 1-Lipschitz along the interpolation between the generated and true data. Thus, the objective function con be expressed as
{1, [ -(,, + Ne J, J. (IV (V-))d)))(
min max HEP())
+A(1- - 1)9 U (IV=, will - 1)² {(dr))(du))
where 9 : R - R is twice differentiable concave function with bounded derivatives, >> 0 is regularization porameter and & E [0, 1]. Since g is concove, the term inside is linear in Jr and independent of v, it follows that the third term in the optimization problem is concave in p.. This fact together with the lincarity in v and μ of the first two lerms shows that the objective function satisfies Assumplion 1 Since f is bounded, differentiable and 1-Lipschitz, and is twice differentiable with bounded derivatives, Assumptions and are also satisfied.
Assumption 4 (Ratio condition). Suppose (v) P(()))(()) are absolutely continuous and comparable with x and p, respectively, in the sense that
There erist constants Tu > 0 such that
inf vo(x)
Po(3) NEY ply)
2 To:
inf
2 Tu.
(14)
ZEX ((x)
2. There exist constants Ry, Rp > 1 such that
Mo (x(
Mo(y) yEy p(y)
sup x (π)
< Rv:
sup
< Ry.
(15)
It can be proved (see, e.g., Lascu et al. fixed point equations
2023
Proposition A.1)) that the MNE (va uz) of satisfies the
2 8F r2 dz Vo -U* -U7
(x(x))
exp
(16)
(.(w) = Z'(v", 112)
1
exp
@ 8p
2 5F
(v+ 16. - UP(U)) -v)(
(17)
7

---

Published in Transactions on Machine Learning Research (08/2024)
where (V2,, and Z' Z'(v,, MG) are normalizing constants.
16 and 17 and Assumption 2 we deduce that Assumption 4 is equivalent to assuming that there exist constants re, Tu > O and Ry, Ry such that for all (x, v) E X x
Combining
Mo(x)
poly)
S Re, Fa Tu S (x)
S Rg. 1%
(18)
We emphasize that Assumption is natural in the context of Fisher-Rao flows. From (5) we observe that the support of the measures along the gradient flow does not increase. Thus, it is essential that the initialization
is comparable with the MNE (cf. (18) as reflected in Assumption 4 It is observed in Liu et al. (2023) that Assumption can be understood as a "warm start" type of condition.
Returning to the question of flat differentiability of v which was raised in Subsection 1.2 if we assume that F is flat differentiable with respect to both v and μ (see Assumption [ then the maps (u, u, 2)) + a(v, Hi a) == (ul(
oF () of log su are well-defined and formally correspond to the flat derivatives 5V (v, p,,) and 5V- 5y (v, 14, :), respectively, for those measures and j for which such derivatives exist (note that we will only need to consider a and b along our gradient flow (Pi,He)ro0 so our argument can be interpreted as stating that, while Vo is not flat differentiable everywhere, it is indeed flat differentiable along our gradient flow). The relative entropy terms DKL appear in the definition of and b as normalizing constants ensure that Jx a(v, j1, (dx)))(x)) = 0 and fy b(v, p. ()((d)) = O, since we adopt the convention that the flat derivatives of are uniquely defined up to an additive shift (see Definition B.1 in Section B). Motivated by this discussion, we define the Fisher-Rao gradient flow Me)e20 on the space (Pac x (ac( by
p(a)
DKI. and (v, p4 3) + b(v, 14, v) 5F
Bp (v, (,,)), - log u(a)
(())
p(
6v
(,,1(z) = -a(v,, He, (x), @.we(()) b(urs Mr, (c)),,
(19)
with initial condition (Vo. po) E Pac(2) x P))( We will establish the existence a solution to 19 in Theorem 2.2 As we will demonstrate, the flow of densities (e) differentiable time (cf. equation (35), and hence the solution to I9h can be interpreted just a classical solution an ordinary differential equation.
The following result extends Liu et al. 20233 Theorem 2.1) to the case of two-player zero-sum games by showing that the Fisher-Rao gradient flow 199 admits a unique solution (9),00
Theorem 2.2. Suppose that Assumption 2 and condition
150 from Assumption hold. Then for
each (Vo, mo) E ()a((X)= x Pun())) there exisls a unique pair of continuous and differentiable in time flows (vr, He)efo,.0)) E Pac(X) x Pac()) satisfying the system of equations (19
Moreover, for t > 0,
4 (x & 2log Rs + σ² Cv. DKI(HelP) s 2 log Ry + σ²
02
4
(20)
and there exist constants u Bi > such that for all t > 0,
vx(x) =EX (x)
sup me(y) DEY plu)
sup
< RI,,
& RI,
(21)
Additionally, if condition for all 1. 0,
14 from Assumption holds, then there crist constants Ti,,, Fi,a > 0 such that
inf (x( EE (2)
inf Helv) DEY p())
Tius
2 Ti,
(22)
The bounds obtained in this theorem allow us to prove that the map 1 + (KI())) + (RE(l)))) is differentiable along the FR dynamics 19
Next, we state the other main result of this paper. We prove two different types of convergence results for the FR dynamics (19) in min-max games given by 1 one in terms of the players' strategies and the other
8

---

Published in Transactions on Machine Learning Research (08/2024)
in terms of the payoff function. Whereas the proof of the existence of the flow (Theorem 2.2 follows
route similar to Liu et al. (2023), the proof of convergence in Theorem 2.3 is significantly different from Liu et al. (023) since, as indicated in the introduction, in the present paper, convergence has to be studied with respect to appropriately chosen Lyapunov functions and, moreover, we do not rely on the Polyak- Soyyak-ojjsiewicc inequality.
Theorem 2.3. Suppose that Assumplion
and hold and lel (v, E Pac(X) Pac(D). Then the map t + (KL(v(e) + DKL (plpe) is differentiable along the FR dynamics 19
Suppose furthermore that
Assumption 1 holds. Then, for all t > 0, we have
DRI(VG + (Rille SE . (DRI(VE vo) + DRL (n)),
NI(U, mo) S 20 +f DKL
po) + DKL (HElpo)
(23)
In game theoretic language, the first result of Theorem 2.3 says that convergence of the FR dynamics 19 to the unique MNE (vS KG) in terms of the strategies Vt and Pe is achieved with exponential rate depending on while the second result shows exponential convergence in terms of the payoff function V
Remark 2.4. Exponential convergence of the single mean-field birth-death flow (vi)12o with respect to DKL ()) can be shown to also hold the selling Liu el al. (2023), however il was not studied in Liu et al. 2023), which considered only convergence of V (v)) for a convex energy function Vo :(((R))) R.
3 Proof of Theorem 2.3
Before present the proof of Theorem 2.3 begin with technical lemma, which extends Liu et al. 2023 Lemma 2.5) to the min-max games setup, and which is proved in Section A The proof of Theorem 2.3 done in two steps:
First, we show that, for fixed (v, p.), the map +> (KL() + (kll)) differentiable when (e)s2 is defined the FR flow 19
Second, we show that 4 (DKL(5)))) (K)())))) is bounded from above by - (DRu(L)))))) + (xI()))))) and then we apply Gronwall's inequality to obtain exponen- tial convergence. Subsequently, we establish exponential convergence for t + N1 (1c,
Lemma 3.1 (Relative o-strong- o-strong--onvexitty-oo convexity-coneaaity to DKL). For Va given by (1. if Assumption holds, then vo salisfies the following inequalilies for all (u, (V/ (() Pac(A) Pac())):
(v'p),))("''v)())) + VO (v(,,)))VVV 2 S
σ²
2 (z
("(u (3,,))) / b(v. (dly)))) 2 DKL(H(Iu)
o²
Proof of Theorem 2.3 Step 1: Differentiability of DKL with respect to the FR flow 19 Suppose that Assumption and hold. In order to show the differentiability of t + DKL(vlut) for fixed Pac(2)) with respect
19 it suffices show that there exists an integrable function f X - such that
v(T) ) a (1x)) log v((x))
$ {(xx),
for all t O. Indeed, using 19 and 36
we have that
a: (v(x) log m(x)
(x) (x)
8F
σ² log
[x(
o²
DkL(eslm))
v(x)
v(x)
(vr Mc, x)
ve (x))))
dv
((x)
o²
3C, +
ro, log R1,e) 2 Rv)) u(x)) fx)).
2
9

---

Published in Transactions on Machine Learning Research (08/2024)
An identical argument gives the differentiability of (KL())))) for fixed μ E Pac())). Then, we have that the map 1 + (vl()) + (pl(p) is differentiable.
Step 2: Convergence of the FR flow: Since t + DRI(UlPs) + (RI(H) is differentiable, we have that v(x))
d dt (DKL(U)p)) (KI((Au))) 1/x d: (M(s) log v1(z))
dr dx+ / 0: (m(v) log s(v)
dy
[e(())
= -- 1 (v(x) vx())) ()v((z) dr / (u(y) -ue(v)) (cple() dy (x)
(r(1)
=/x
-v))(d)) / blv,, 20 - -
where in the second equality we used the fact that Sx dvvs(t)de = -y (b) = 0. If a > 0 and Assumption holds then, using Lemma 3.1 in the equation above, we obtain
d
o²
o²
dt (DKL()|)) + (ullm) & V V 14) - Vo (V 1) (KL(U)m) DKL(Hlpe).
(24)
Setting (1) (G) in 14 and using the saddle point condition 12 gives
a² DKL(USlu)
a² 2 DKL(HGlu))
(DEL (K (4) + DKL (pt
Hence, by Gronwall's inequality, we obtain that
(1 + DRI(HGlPe) t (n) + DRL (m)))
(25)
To prove convergence in terms of the N1 error, first observe that, for any (v, E P(X) P(J), write (V.,)) - v (x) = (v,, - V (v's ()) + V (Va - (v, 1²) + V (v, (3) V (u,
By Theorem 2.2 and Assumption
there exist C1,o, C2 > 0 depending on such that
8V 8v (v,,H,,)) S C1.a:
ov
(v. Mr, v) S Cz.a,
for all (v, E P(X) P()), and all (x, E X y. Then, by Lemma 3.1 we have
x dv (vr μ. x) () (drr) o² $ CI TV(U v1) S S1, DKI(v))ur)
/ 8V
V -
DKL (vglu)
and
(V( - Vo (v, < /
8V
ōp (v. - -u) o² 2
DKI
S C2. TV( ue) & C2.0 v 2KI
where the last inequalities hold due to Pinsker's inequality and since o DKI. a² I DRI. DKI, (Hōlu)) 2 0, for all t 0. Setting C. = max (C1,a, C2,7,, and using the incquality va + vb S 2((a + b), we get
Vd (v, & 20, (DLL(v(l + DKL + (v, (4) - (' (v. + V (V, U1) - (V,
(26)
20,e of DDLL(U) + DEL
where the last inequality follows from Note that since (Vz (C)) is the MNE 1 we have
max V (V3 (1) min V (v, (C) V (VZ,
10

---

Published in Transactions on Machine Learning Research (08/2024)
Therefore, maximizing over (1) in 26 gives
min V (1, S 20oe-at +: DRL
NI(Pr, pe) = max V (Ve, p)
(o) + (KI(Hs))o)
Finally, observe that, by
18 we have
(KI(U + DRI,(HGll < co.
Acknowledgements
R-AL was supported by the EPSRC Centre for Doctoral Training in Mathematical Modelling, Analysis and Computation (MAC-MIGS) funded by the UK Engineering and Physical Sciences Research Council (grant FP/S023291 Heriot-Watt University and the University of Edinburgh. T. acknowledges the support of the UKRI Prosperity Partnership Scheme (FAIR) under EPSRC Grant EP/V056883/1 and the Alan Turing Institute.
References
Kenshi Abe, Mitsuki Sakamoto, and Atsushi Iwasaki. Mutation-driven follow the regularized leader for last- ilerate convergence in vero-sum games. In The 38th Conference on Uncerlainly in Artificial Intelligence, 2022.
Shun-ichi Amari. Information Geometry and IIs Applications. Springer Publishing Company, Incorporated, 1st edition, 2016. ISBN 4431559779.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generalive adversarial networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 214-223. PMLR, 06-11 Aug 2017.
Cyril Aubin-Franin Frankoskan,,wkki, Anna Korba, and Flavien Léger. Mirror descent with relative smoothness in measure spaces, with application to sinkhorn and em. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Newral Information Processing Systems, volume 35, pp. 17263- 17275. Curran Associates, Inc., 2022.
N. Ay, J. Jost, H.V. Lē, and L. Schwachhöfer. Information Geometry. Ergebnisse der Mathematik und ihrer Grenzgebiete. Folge Series of Modern Surveys in Mathematics. Springer International Publishing, 2017.
René A. Carmona and François Delarue. Prolabilistic Theory of Mean Field Games with Applications I: Meon Field FBSDEs, Control, and Games. Springer International Publishing, 2018.
Lénaic Chizat. Convergence Rates Gradient Methods for Convex Optimization the Space of Measures. Open Journal of Mathematical Optimization, 3:8, 2022.
Giovanni Conforti, Anna Kazeykina, and Zhenjie Ren. Game on random environment, mean-field langevin
system, and noural networks. Mathematics of Operations Research, 48(1):78-99) -99, 2023. doi: (0.1287/moor:
Ross Cressman and Yi Tao. The replicator equation and other game dynamics. Proceedings of the National Academy of Sciences, 111(supplmeen 3):10810 10817, 2014.
Constantinos Daskalakis and Ioannis Panagcas. Last-iterate convergence: Zero-sum games and constrained min-max optimization. In Information Technology Convergence and Services, 2019.
Carles Domingo-Enrichl Samy Jelassi, Arthur Mensch, Grant Rotskoff, and Joan Bruna. A mean-field analysis of two-player zero-sum games. In Advances in Neural Information Processing Systems, volume 33, pp. 20215 5-2226. Curran Associates, Inc., 2020.
11

---

Published in Transactions on Machine Learning Research (08/2024)
Thomas O. Gallouët and Léonard Monsaingeon. A JKO splitting scheme for Kantorovich-Fisher Rao gradient flows. STAM J. Math. Anal, 10(2):1100-1133) 2017. ISSN 0036-1410. doi: 10.1137/16M106660
lan Goodfellow, Jean Pouget-Abaue Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (eds.), Adounces Neural Information Processing Systems, volume 27.
Curran Associates, Inc., 2014.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein gans. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp. 5769 5779, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Josef Hofbauer and Karl Sigmund. Evolutionary Games and Population Dynamics. Cambridge University Press, 1998.
Ya-Ping Hsich, Chen Liu, and Volkan Cevher. Finding mixed Nash equilibria of generative adversarial networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings of Machine Learning Reseorch, pp. PMLR, 09-15 Jun 2019.
Anna Kazeykina, Zhenjie Ren, Xiaolu Tan, and Junjian Yang. Ergodicity of the underdamped mean-field Langevin dynamics, April 2022.
Juno Kim, Kakei Yamamoto, Kazusato Oko, Zhuoran Yang, and Taiji Suzuki. Symmetric mean-field langevin dynamics for distributional minimax problems. In The Twelfth International Conference on Learning Representations, 2024.
Stanislav Kondratyev and Dmitry Vorotnikov. Spherical heatinger-aantroovieel gradient flows. SIAM Journal on Mathemalicul Analysis, (33):0053-2083) 2019.
Razvan-Andrei Lascu, Mateusz B. Majka, and Lukasz Szpruch. Entropic mean-field min-max problems via best response flow, 2023. arXiv:2306.030:3
Razvan-Andrei Lascu, Mateusz B. Majka, and Lukasz Szpruch. Mirror descent-ascent for mean-field min-max problems, 2024. arXiv:2402.08106.
Linshan Liu, Mateusz B. Majka, and Fukasz Sapruch. Polyak- tojasiewicz inequality the space of measures and convergence of field birth-death processes. Applied Mathematics and Optimization, 87(3), March 2023. ISSN 0095-4616.
Yulong Lu. Two-scale gradient descent ascent dynamics finds mixed nash equilibria of continuous games: A mean-field perspective. In Proceedings of the 40th International Conference on Machine Learning, 2023.
Yulong Lu, Jianfeng Lu, and James Nolen. Accelerating Langevin Sampling with Birth-death, May 2019. arXiv:1905.0986: 09863.
Yulong Lu, Dejan Slepčev, and Lihan Wang. Birth-death dynamics for sampling: global convergence, approximations and their asymptotics. Nonlinearity, 36:5731-5772, 09 2023.
Chao Ma and Lexing Ying. Provably convergent quasistatic dynamics for mean-field two-player vero-sum games, In International Conference on Learning Representations, 2022.
Alcksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
John Nash. Non-cooperative games. Annals of Mathematics, 54(2):286 295, 1951. ISSN 0003486X.
12

---

Published in Transactions on Machine Learning Research (08/2024)
Hukukane Nikaidô and Kazuo Isoda. Note on non-cooperative convex game. Pacific Journal of Mathematics, 1:807-815, 1955.
Felix Otto. The geometry of dissipative evolution equations: The porous medium equation. Communications in Partial Differential Equations, 26(1-2):011-77, 2001.
Henning Petzka, Asja Fischer, and Denis Lukovnikov. On the regularization of wasserstein GANs. International Conference on Learning Representations, 2018.
C. Radhakrishma Rao. Information and the Accurucy Allainable in the Estimation of Stalistical Paramelers, pp. 235-2477. Springer New York, New York, NY, 1992.
Grant Rotskoff, Samy Jelassi, Joan Bruna, and Eric Vanden- Eijnden. Neuron birth-death dynamics accel- erates gradient descent and converges asymptotically. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 Proceedings of Machine Learning Research, pp. 5508-5517. PMLR, 09-15 Jun 2019.
Aman Sinha, Hongseok Namkoong, and John Duchi. Certifialle distributional robustness with principled adversarial training. In International Conference on Learning Representations, 2018.
Camilo Garcia Trillos and Nicolas Garcia Trillos. On adversarial robustness and the of wasserstein ascent-descent dynamics to enforce it, 2023. arXiv:2301.03662.
Nicolás García Trillos, Bahram Hosscini, and Daniel Sanz-Alonso. From optimization to sampling through gradient flows. Notices of the American Mathemalical Sociely, 2023.
A.B. Tsybakov. Introduction to Nonparametric Estimation. Springer Series in Statistics. Springer New York, 2008. ISBN 9750387790527.
John von Neumann, Oskar Morgenstern, and Ariel Rubinstein. Theory of Games and Economic Behavior (60th Anniversary Commemorative Edition). Princeton University Press, 1944. ISBN 9780691130613.
David Siška and Lukasz Szpruch. Gradient Flows for Regularized Stochastic Control Problems. arXiv e-prints, art arXiv:2006.059566, June 2020.
Guillaume Wang and Lénaic Chizat. An exponentially converging particle method for the mixed nash equilibrium of continuous games, 2023. arXiv:2211.01280..
Chen- Yu Wei, Chung- Wei Lee, Mengxiao Zhang, and Haipeng Luo. Linear last-iterate convergence con- strained saddle-point optimization. In International Conference on Learning Representations, 2021.
Yuling Yan, Kaizheng Wang, and Philippe Rigollet. Learning gaussian mixtures using the wasserstein-fisher rao gradient flow, 2023. arXiv:2301.017
Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. Multi-agent reinforcement learning: A selective overview of theories and algorithms, 2021. arXiv:1911.10635.
A Technical results and proofs
In this section, we present the proofs of the remaining results formulated in Section of the paper.
A.1 Proof of Lemma 3.1
In this subsection, present the proof of Lemma 3.1
13

---

Published in Transactions on Machine Learning Research (08/2024)
Proof of Lemma 3.1 Using 10 from Assumption ! it follows that
oF /x
(v, H, 2, - 2) - x²
o²
vO (O(v', p4) - V (v, ub) 2
DKL(V'IT)
DKL(v/T)
x 8v
2
2
= =/2 (
OF
0² log
u(xx)
0²
/ log
[xxx
(v, 14, x)
(v' - (d)
(v' - ())
ov
T(x))
2
x
((x))
0² / log
(1(x) (π))
o²
f log
M(x))
v'(dx)
v(dx)
x
2 x
(
= . ( OF
(+) + d²
log (((x))
(m(e))
((())) ))
- ()x) (v'v))(rr) + x²
2 DRL(V'IV)
6v
2
J alv, (dd)) + 2 DRi.(Upp)
0²
Similarly, using 11 from Assumption it follows that
(v, - Va (v, < /
8F
o² (v, (v, - m)(d)) -2 DKL(w'lp) 0²
DKD (KL((lp)
= =1,
O8F
o²
(((y)
p(v) (u'- -d) + a²
/ log 2 y
(M(U)
(v, u.
log
(1l - (w)
2
p(y)
0²
/
n'(v)
1d' (dy) + o² /
(uy)
/ log
log
u(dy)
2
p(u)
2 y
p(u)
= /
OF
a²
(u(y)
p(y) )) (1' -d)(d))
a² DKI(u' Iu)
(V, 14,
log
2
2
11 blv, n. v, m)(d))
o²
-DK (r' In).
2
j
A.2 Existence and uniqueness of the FR flow
In this subsection, we present the proof of our main result concerning the existence and uniqueness of the Fisher-Rao (FR) flow, i.e., Theorem 2.2 We construct a Picard iteration which is proved to be well-defined in Lemma A.1 Lemma A.2 shows that the Picard iteration mapping admits a unique fixed point in an
appropriate metric. Then in order to conclude the proof of Theorem 2.2 we show the ratio condition (21)
The proof of Theorem 2.2 is an adaptation of the proof of Liu al 20223
Theorem 2.1) to the min-max
setting
Proof of Theorem 2.2 Step 1: Eristence of the gradient flow and bound 20 on [0, In order to prove the
existence of a solution (vts Mx)e20 to
(2))(()) = - 6E
(yt, Me, x) + o8a log o²
v((2) [(x)
(Vol)))))) v((x), + 2 DKL(Help)) pe(3).
No
2
(27)
@spe(v)= =
-Op (ut, He, v)
(E
I log
(e())) plu)
o²
we first notice that (27 is equivalent to
b log (x(x = of (ve, H4, x) + 0²
log
[=(())
o² DKL(Us)TT),
5z
2
x(x)
2
(28)
0 log
6E oa
(ve, y)
7²
log
[=(p)
+
o²
2 DKI,(Help))
I
p(y)
By Duhamel's s formula, 28 is equivalent to
log (x) =e ---5"
= log v0(x) fo g²
_ " (t=s)
(22
6F
(Vs, x) log T(x) DKL (vs (T) ds,
8v
loggu()) =e --- - (o)g) + f 8² a (1-8)
2 e
du (vs, Ms: y) + p(y) + D (KI(uusp) Ip)) ds.
14

---

Published in Transactions on Machine Learning Research (08/2024)
Based on these formulas, will define Picard iteration scheme. To this end, let first fix T 0 and (0) (0)
choose a pair of flows of probability measures (v) Mi )cE(o.T] such that
T JN" (KL r)ds < 00,
L
T
(x)ds
DKL (ug (p)ds < co.
For each n 1, we fix v0 = -10 = uo and 4o (n) = 40 - no (with vo and yo satisfying condition 15 from
(0)
Assumption 40 and define (v(") Me )eeo, by
(m) (m)
log V (z) (x) = E o vz()
/ o² of (1-o)
(2 2 8F
(29)
(v(m-t), (((-1)), ,0) - log - ( ds,
2
02 dv
log 14 (n) (1) -- 3' (og))())
- (t-s)
L'
0² pe==
8F
(30)
,,(m)))))))m))1)) v) + log pog) + DKI.(uL"-11111 ds.
2
0 81
We have the following result.
Lemma A.1. The sequence of flows (Iv("), (n) Dee10.T) that for all n 2 1 and all t. E [0, we have
given by 29
and 30
is well-defined and such
n=0
" (T) 2 log Ry
Co. Dru 2og R2 2² Cp.
Proof of Lemma A.1 The proof follows from the same induction argument used prove Liu al. (2023
Lemma 3.1).
0, we consider the sequence of flows (lv)") ,pr"") ) n-0 in (P(X))00)) x P(y) u.21, TVI..11). where, for any (v, E p(x)o.M x P(y) the distance Ty'u.11 is defined by
For fixed T
Tylo.T] ((vr, Me)eet.)) (4)e)) [" (TV ['" TV dtdtt.
Since (P(X), TV) is complete, to conclude that (P(x))..7) ",TT So TV(,,, vf)dt) and (P()))(0.T) fo (V(,,,1)) are complete. Therefore,
apply the argument from Siška & Szpruch 2020 Lemma A.5) withp=
one deduce that (P. (0)) TVIo.Ti) is also complete. We consider the Picard iteration mapping & (n(---)) Mr (n-1) Jee(u.n) = (n) u4 (n) )esju,n defined via (2 and (30 and show that ø admits
unique fixed point (vt, pe)aeto,ry in (P(x))0..) (())(..1), TVlL.T)) which is the solution to 19
Lemma A.2. The mapping & admits a unique fixed point Heheeor)) in (P(X))O.T) x ((7) TDIO.T).
Proof of Lemma
Step 1: The sequence of flows (lv8") .14 (m) )eetorT)) n=o is Cauchy sequence in (P(X))O0T)) x 0.7) TVl0211.
From
29 we have
/-' 0² (1-))
logu (m) (x) - -log (n- (2) =
j 2
x
2
8F
(1
1) 1 100-11pp11 1m x)
6F
(v
(n (n-2) 2)
11
2m - DRI (v!--11)) + (RI(u(a-2)) (0mmmmmm))mm ds.
σ 5v
15

---

Published in Transactions on Machine Learning Research (08/2024)
Multiplying both sides by ((n0) (x) and integrating with respect to x, we obtain
(1) lee ((m--1)== Dut(uv
- ./² 0² -
(t x)
2
/
oF
(uf=-1),
-1--")),,_ .0)
oF
(ufa-3), _1(aa)) (lm-2),90 ,0) o8
=1
2
02
x
ōv
DKL((u (T) + ( ds. (31)
Moreover, note that
Jx
OF
(ufm-1), 1) 1) (1(m-1),00 x) x))= dv
oF
(2! .u! ( 2 ( (dz)
x 8v
= /
8F
(1/m-1)) 1) (0--1),,])) ,x) 8v (v:n 1) ! x oF
5F
dg (n! (1 |/ 2
x) xz)
8F
(uin-2) , (/5-2),,00 x)) v. (dx) (dx)
x
6v
ov
Se 1 Jo
(' F2F (a--1) 11! ( + > (1!am1) يl! ) I. w) dP - ) (dw) (dx)
-
ōqud
1
{
8²F F (2)
2) + x (v(a-1) ula 1
( 2) x 2) ai (imuin) - 2ln-2) (dz)ut mh (dx).
dv2
x
Similarly, again from 29 we have
02 22 (1=s)x
log (n- (x) log 11 (x)
= (x))
0 2
x
8F
(2(n-2)
(
x)
5F
1 (1(m-1)
( ( 22) vi 1 (dx)
σ2 x
dv
8v
- DRI((un-2) ()) + (uf 177) ds.
Multiplying both sides by vi" (x)) and integrating with respect to x, we obtain
DkL(v(n-1)110 -
1/"
0²
o (t (t-s) s)
2 (F
/
(v-m
2), 116 116-21,,0 2) dv (v! (n 1) 1),,,, ()== (x) v/ ( (dz)
5F
2
02 x
8v
-DLL(1 (an-(((a2211)) 21 + DRI(US -))))--))))))))))))<)) ds. (32)
Similarly as before, we note that
6F
(u(m-
, 11 (1- 23 x)
8F
(v! 11 1) u1! ( .)) ((m))((d))
x 5y
dv
Lx
OF
(u(m
1 (1-1))
xx)
8F
(w(n-1) 1) pl 2) x)) of
(w!a u1 ((1-)) x)) dv
oF
(v/m-22) ,00-22,,00-00 11/0-2) xx) u (dx)())))
8v
dy
=- -- J/y 50
8²F
(n- _ 2) + > -
- ( .
m. w) x dP (اسسrي) - ي (()) (d (dr)
ouōv
-
-1x Jx
FFF
v (n=2) + > (1 -n(m-2) : 11 (1-
x x x dp (-! - (dz)u² (m)) (dz).
daz2
16

---

Published in Transactions on Machine Learning Research (08/2024)
Combining 31 and 32
we obtain
- {- x
(vt
(v( 1) (vt ()
(v(((=("))) (1(""""11)))) =- - -
S |
FFF ((n-1)), 166 2) + x (16- - 11 (1) Ft w) yP (
- (m!--1)-11b- 11(-221 (dw) (u(²) - u v (dx)
(
ōuōv
1
8²F 8u2
(n)=))
+ d (1fn-1) - 2 u 2 2,2 2) di (-rui1) - 12n (dz) (v(t) - us v/m-11) (dx) ds.
x x 0
Hence, due to Assumption 3 we get
DKL(U (() ( + DRL(U De
(1 (- ((0 (Cmur TV(x"="),
TV(il") (i), Pt
((( + Cup TV(x 1, v(m ds
S max Crv) TV(U us ((011)) f e -- (TV(u!= 11! 2)) + TV 1), ,), u)n ds.
By the Pinsker- Csiisar inequality, (V (M) (/--11) S DKI,(u(") /e
(1 ((-11)), and hence
(m) ( (/a-1)) /' ( TV (1 1), , ve
(m) vi 11-1))= max(Cp,e: C.,u (v 4 (v
TV(( (V( ds.
which gives
TV(, (1) ve ) 1
max{Crm, Grwd f (1-)) ( (TV(u(--1) (T) TV( ,/mm)2)) )ds.
An almost identical argument leads to
1 TV( (x) .M ( < z maxx(Cu,u Crad Lf e of (e-s) (V((u 1, 2)) + TV(u!n-1), , 2)mm))) (TV()
If we set Cmax (a C.N + max Maax(Cunp (Curp Cuuk: and add the previous two inequalities, we obtain
TV 60) ((--1)) + TV (n) .1 1n-1))
Cmax <
f e- e-
( (VV(ub"""), 2)) + TV( 2/n ds.
4
Cinxx
n-1
(TV
dtn- dtzdt
4
max
n-1
t (n-
ln-
(V (00 TV (1) (0)
(1)
( p )dm
2)!
Cmax
(n 1n-12 2) /
mn-2
1 + TV 10)
)dn-
4
where in the third inequality we bounded p!a-2 dtn-1 S So dtn-1 and in the fourth inequality we bounded fn Se St Hence, we obtain
TV (m) 11 ytt + 1' TV( (x), (1 )dt
.14
Cmux
n-1 -1
(n - 1)! (S"" Po J- P )
<
(0) Pin
4
17

---

Published in Transactions on Machine Learning Research (08/2024)
Using the definition of Tylu. the last inequality becomes (m) Tylom (1v(*), :14 )oEOT,T) (ve 1)-1) H
(m-1))
Cmax
n-1
(n TVlo.T (1v
T
( 14 hejo (vg H4 heetom) ,K
4
By choosing sufficiently large, conclude that ((v)") , ml
(((")) )eeta.)) n=0 is Cauchy sequence. By
completeness of (P(X)10.T) x P())...),, TV)O.TT). the sequence admits limit point (ve, (c) e (P(X))1.11 x u2) TVIu.11)
Slep 2: The limil poinl (ve, Me)oel, is a fixed point of d. From Step 1, we obtain that for Lebesgue-almost all t [0, have
TV(1 11) + O, TV(4 me) - 0 as n + 0c. (m)
Thus, by 12 and 13 we have that for Lebesgue-almost all t E [0, and any fixed (ar, y) E X' x V/ the Pi (n-1) 2)) and &F (v(a-1) Mi (1(-1)), 2) from (29 and (30) converge 8F
flat derivatives 6F
du (vt, M2, x)
4F and oz (1t, Me, v) respectively, theorem and obtain
Using Assumption 2 we apply the dominated convergence
lim 200
/ dv
oF (w!m
p (()-1)) x)ds
I
6F
(Ve, Mas Ms, x)ds,
8v
lim J
8F (v! (1-)) (n-1) .y) I'"
8F
u
(Ue, Ms, Vdds,
n n-00
81
8p
for Lebesgue-almost all E [0, and any fixed (I, E X x y
Moreover, in flow of probability measures for all n, we oblain for all n and L E [0, T), and any fixed (x, v) E X x V, that
29 and
30 using Assumption 2 155 Lemma A.1 and the fact that (v("),, 14 )ec(o.T] is
(n) (x) (x) <(x)
(a) (x) log
3log Rs + g Co.
(v) ply) < 3 log R. + g2 Cr.
(n) (v) log
Pi
Furthermore, since (v("), (m)) )e(o, considered to be absolutely continuous with respect to the Lebesgue measure for all n, it follows that convergence in TV distance implies convergence in L'. Then due to the dominated convergence theorem, we deduce that DKL (v, (0(")/m and DKL. (u(")) (p) converge to DKI(( (r) and DKL (M lp), respectively, for Lebesgue-almost all t E [0, T7, as - co. Hence, using again Lemma A.1 and the dominated convergence theorem, we obtain
lim I (vL (x)ds= /.' (KL(vs/Tdds
lim I' (kl = 1 DRI(Hslp)ds
n n-
for Lebesgue-almost all & 6 [0,
Therefore, letting n + oo in 29 and 30 and using the fact that x + log is continuous on (0, oo), we conclude that (e) is fixed point of ø.
The faxed point (Mt, Me)eejo,T of d is unique. Suppose, for the contrary, that & admits two fixed points (Vt, Pe)t and (De, P such that vo = in and μo = Po- Then repeating the same calculations from Step 1, we arrive at.
Slep 3:
(v.( + (KL(E))))) S max(Cu. Cr. TV D.) S - (TV (M) + TV D.)) ds.
18

---

Published in Transactions on Machine Learning Research (08/2024)
By the Pinsker-Csizsse inequality, (V m) DKL((s(()), and hence
(V,,, v2)) ds, (v,, & max Cr.c (V(,, ''" -- (TV(usi (V)
which gives
TV v2)) - max{Cuuvr Cev) S e-S(U-0) (TV(Pe: Pa) + TV (.)) ds.
4
An almost identical argument leads to
1 [V(m,)))) & 7 max|Cr,pr Cpa f e ((-)) (.) +V),, ds.
If we set Canx max Cu. }+ + max Cuuper Caul, and add the previous two inoqualities, obtain
TV( (V),, Cmax ('" E (t-x) (TV(us: P.) TV (vs, (,)) ds.
For each & E T), denote ((1) S c (TV(Hs; + TV(,,, v.)) ds. Observe that ʃ 2 0 and f(0) 0.
Then, by Gronwall's lemma, we obtain
0 f3 5 SE Smax f(0)
and hence
TV(,, + TV (b) = O,
for Tebesgue-almost all i E [0, 7), which implics
1/2 4= D2 14 Tit, for Lebesgue-almost all t e T). Therefore, the fixed point (ve, Me Me)) of ø must be unique.
From Steps 1, 2 and 3, we obtain the existence and uniqueness pair of flows (V,, (eleeeo satisfying 27 for any T >
Returning to the proof of Theorem recall from Step of Lemmn A.2 that for Lebesgue-almmon all e [0, 7] we have
TV (a) m)) + O, TV(4")) (m) .P) + O
as + 00,
which implies
weakly as + 00. Hence, using the lower semi-continuity of the entropy, we obtain
5/ + vts Pe +
(vo(T) < lim (KL( (T) S 2loog Rd + C.. (m)
4
(33)
n-000
Ø²
(ue(p))p) < lim inf (u
(1) (p) < 2og R2 + -² Cp.
-/2
4
(34)
where both second inequalities follow from Lemma A.1 In order to ensure that the solution (M2, ph (ejo. can be extended to all t 2 0, we first need to prove the bound on the ratios vl/ and h/p in 21
Step Ratio condition
21 Using
29 and 30
we see that for any E [0, T] we have
vi (n) (x)) = e log
vo(x)
22 (t-s)
11) .)) - (kI(((m-)) ds, 2 8F (n 1)
log
(Φx)
(x)
0 ōv
log
Mi (n) (g)
e " log
Mo(y)
+
0²
e
0² (t-s)
2 8F
(u nn-1) -1 (1(--1,,)) + (kf ds.
plv)
p(y)
2
r 8μ
19

---

Published in Transactions on Machine Learning Research (08/2024)
Using Assumption
15
33
and
34
obtain
log (r(x) x(x) S log R, + zz C..
6
CC..
r²
log Me(3) p(v)
6
S 3 log R2 + 0² Cn.
Hence we can choose R == 1 + exp log R2 + C2) and RI = + exp (3log R + of C.). Note
Bi RI. are conveniently chosen so that log R1, log Br, in our subsequent calculations Obtain- ing a lower bound on v((a) and pclu) follows similarly, by using (14p instead of (15).
p(y)
Step 3: Existence of the gradient flow oo). In order to complete our proof, note that the unique (7 can also be expressed
solution (Vt, pe)eejo71
vx(x) vo(x) exp (-fo"
(
OF (Ues Me. (e.))
σ²
Va(x)
o² DKL(Us/TT ds):
log
8v
+(x)
2
(35)
mcly) = mu(y) exp (%
6F
0²
+ a²
(Vo:
No: v)
log
DKa(Halp) ds
2
p(y)
2
Then it follows that the flows (Vt, (t),ee)) are continuous and differentiable in time.
From
2
33 34 and 21 we obtain for any t E [0,2]
8F
o²
log [x(x))
0²
(Ee, Me, x)
(KL(F)
8v
T(x)
s 3C, + a² 2
(max{] loor,,, R1 2log Rτ) =: Ov. (36)
8F
0²
(y)
o² DKI(mil))
(vt, Mr.
log
+
8p
2
p(y)
0²
330 (max [lag// (ogrr,ul, log R1 + 2 2log R,) CV
This gives |[ellay I Il wwollveeCvi and || $ || (v,e and shows that and μe do not explode in any finite time, hence we obtain a global solution (M,,/4) (v),/4))es0.oo)) te(0, which is continuous and differentiable in
time. In particular, the bounds in (33 (34), (21) and (22) hold for all t > 0.
B Notation and definitions
In this section we recall some important definitions. Following Carmona & Delarue (2018) Definition 5.43), we start with the notion of differentiability on the space of probability measure that utilize throughout the paper.
Definition B.1. Fix p 2 O. For any M C Rdd let Pp(M) be the space of probability measures M with fivile p-lh momenls. A function F: (P(M) + R admils firsl-order flat derivative Pp(M), if there exists 6F a function ou (P(M) x M + R, such that
1. the map (M(M) x XM (m,z) + &F (m,z) is jointly continuous with respect to the product topology, 8m where Po(M) is endowed with the weak topology,
2. For any m PP(M), there exists C > O such that, for all M, have
8F
(m, x))
< (1 (1)))
8m
20

---

Published in Transactions on Machine Learning Research (08/2024)
3. For all m, m' E Pp(M), it holds that
F(m')= - F(m) = S.¹ Ju
8F (m + e(m' - m), x)) (m' m) (dr)de.
(37)
6m
5E is then called the flat derivative of F on Pp(M). We note that dm erists up an additive The functional 6m constant, and thus we make the normalizing convention JM 8m (m, x)m(dx)
If, for any fixed M, the map m + 6F (m, x) satisfies Definition B.1 &e
say that F admits second-order
flat derivative denoted by M x M + such that. 8m
Consequently, by Definition B.1 there exists a functional g2
PP(MM x
dm
8F (m', x)
8F 5m (m, x)) = [-' J
F 5m
(v =(m' - m), x, x') (m' - m) (dx')de )de.
(38)
8m
Example B.2. We provide some simple examples of functions F: : Pp(M) + I' and their flat derivatives.
1. Let F(m) = JM f(x)m(dx), where f M - IR is continuous and has p-th order growth. Then
8F 5m
8²F (m, x)) =
(m, x)) f(x),
dm
2. Let F(V,))) Su ((1)))))))) Then
FF
6F (,,
Jmx [(,, ((),))) 82F 8u2 (v, 1.
8FF (1, v) y) = 0.
5v (U, (u, = ((,, ((x,))((dy),
ou
8n12
Let F(V, (JM SM f(z, ()d(d)))(d with f continuous and g twice continuously differen- tiable. Then using the chain rule we oblain
5F
x) = Un f(a, fl. (e((er))dd)))))) I. f(x, ((x.))((dy) 2))
ōv
8F
Vn Sao Il sla, (ln /,, ((x,
(1, v, = =g 8q
ōςō (,, = F2F
82F
ōvôn (v. (v.,, Unu fla s(r, (olla)))(()))) f(z,v)
+ S f(x. viu(duno" (oreee fla Fla slonaaaaaa)l)) / f(x, v)(d)))
8FF Rv2 (U, p,,,, (,,,,,)))--) Un I(,, ((w I I( whlldd)d)) I s(a', (uld))
F 2p2 (u, = (Nr fi (( / [(x, [(,, ((),)))(d)))
/ S1, (((d))
Definition B.3 (TV distance between probability measures; TTybakov 2008
Definition 2.4). Let (M, Assume that is a-finite
be a measurable space and let P and Q be probability measures on AM, measure on (M,A) such that P and Q are absolutely continuous with respect to and let and q denote their probabilily densily fanctions, respectively. The lotal varialion dislance between P and Q is defined as:
TV(P,Q) = sup PP(A) - Q(A)) = sup 1 (p q)dc AEA
(p-9))mul
AEA
21

---

Published in Transactions on Machine Learning Research (08/2024)
C A formal derivation of the Fisher-Rao gradient flow
In this section, we follow Lu et al. 2019); Yan et al. (2023
Gallouët & Monsaingeon (2017 formally
derive the Risher-Rao gradient flow of lunction G (M) + I, with M MCR. R. Following Otto 2001
define the tangent space (MM) at any A E Pac by
fumctions & on M such that S. {(z)d==
(M)
The tangent space (M) can also be identified with
(M) {5 ((=(() - (ax) (((x)) - S.x. r(o))())))) with r L2 (MiN)). (M)
On this tangent space, define the Riemannian metric oR (MR(M) (FR(M) +R at A by
9x (EL, = Sm. (1(x))E(())
1(x) dx, for f E (M).
(39)
Then for any E((x) X(x) (re(z) (=(((((())) for i 1,2, the metric F becomes
(2), = //x (())))())MM))d Su. (x)))(()dd / ra(x))(x)de.
With the metric RfR, the Fisher-Rao distance FR, defined by 3), can be viewed the geodesic distance on (Pc(M))gXP)
Now, we derive the Fisher-Ras gradient flow for given function G Pac(M) - R, which we assume is flat differentiable (ef. Definition B.1. Let (Ae)eo be a smooth curve starting from Ao = E Pac(M) with
arbitrary tangent vector
- 0))=))o00 = ((x) (r(00) / r(o))(w)de)
(40)
The metric gradient grad FR G E (2(M of G with respect to of defined by
d
G(X:)
=
FR (gradFR FR G(A).)).
(41)
dt
1==
and the gradient flow of G on (Pac(M), FR) is defined by Then, by the chain rule,
(x)((z) grad FR (().)
(42)
40 and
39
we obtain
d
G(A))
8G I (A. (1.2
8G / (1,))((x)d ofth 8G
(A.
dt
M dx
M 5x
==
Since & E (TFR(M is arbitrary, it follows from 41 that
8G
grad FR G())(z) 82
(A, (),)))(x),
and consequently by
42
8G
0,7((x) = 8A ())
22

---

Published in Transactions on Machine Learning Research (08/2024)
D Example: Convergence of discrete-time Fisher-Rao dynamics for a non-interactive min-max game
In this section, we provide an example of game for which the convergence of the discrete-time Fisher-Rao in the non-regularized setting, and 1974
dynamics to an MNE (measured in NI error) occurs rate O I
rate O ( (to the unique MNB) when regularization is added to the objective function. Here, 7) > 0 and a > 0 denote the discretization step-size and regularization parameter, respectively.
To be precise in what follows, we use the terms "discrete-time Fisher-Rao" and "Mirror Rescent-Asceni as synonyms. As is argued in Wang & Chizat 2023) (at the end of the first bullet point on page 8) a worst-case example which shows convergence of the Fisher-Rao dynamics discrete time in terms of the NI
error at rate O 7m can be constructed by following the arguments from Proposition 5.5, Setting II (see the proof of Proposition 5.1 for this specific setting) in Chizat 2022).
Chizat 2022 Proposition 5.5) discusses the case of minimizing a (non-lincar) convex function along the iterates of the Mirror Descent algorithm (see Algorithm in (Chizat 2022 ) In order to establish the connection between the setting in Chizat (2022) and the min-max games setting, we note that removing the terms corresponding to the Wasserstein flow in the Conic Particle Mirror-Prox (CP-MP) algorithm in Wang & Chizat (2023) (see the update rule below Lemma 2.1) and taking L = 1 retrieves the Mirror Descent- Ascent (MDA) scheme with step-size ry > 0 for min-max games in which players update simultancously. The zcro-step-size limit of the MDA scheme is prociscly the Fisher-Rao gradient flow.
Following Setting in the proof of Proposition 5.1 in Chizat 2022 we construct an example for the convergence of the Fisher-Rao dynamics in discrete time terms of the NI error al a rate O mm for the min-max setting in Wang & Chizat (2023 as follows.
First, recall that the min-max game considered in Wang & Chizat 2023 is given by setting F fy Jx (x,, v)u(dz)u(dy) and a = 0 in i.c.,
min max F (H, with F (v. v) = J (((d))))))) (EP(X) NEP())
(43)
Consider [(x, (xx - (v), where & is any smooth function such that $(x) = |لr| when ||al| S 1/
and (xx) 2 1/2 otherwise. For this choice of f, initial measures (44), (44,²²) supported on the entire space X' and V, and any step-size 7, > 0, the simultaneous MDA update reads
n= arg min (ʃ (x) (dxx) + + DKL (K, (()),
+++ arg max { v2) - DKL )²
Hence, we can write (x) x u (z(z)) exp()xp(-nung ( and (2) x vn exp(-nm) [|vl)), which are Gaussian distributions of variance 1 Therefore, for n given by 43). we can compute
7jn
NI(H", max (v" ,M) (()))"")) / T ((y) (dg)+ / (c)))"(dr min / ((y)u(dy) min J (()))(d)). (0,0)
#
Note that v + (dy) and p f (z))((dz) have unique minimizers v" S, and μ² = ōg, respectively, and (0) = 0. Now, assume for example that (n², 20) are uniform distributions on y and X, respectively (recall that X, are assumed to be compact in Waan Chizat (2023)). Then, following the proof of (Chizal, 2022
Proposition 5.5), we obtain
| (dy) ~ 2mn*
and analogously
J (zx))(((r)) ~ 2nm
23

---

Published in Transactions on Machine Learning Research (08/2024)
Hence,
(v", :1) 1100-F40 (v),,)") / ((y))"(dy) (dz)))(dr)
_ mm
NI(H"," max (p,)
Now, consider the entropic regularization of 43 i.c.,
min max F" (H, with F (1, = J J f(z v)o(dz)v(d)) + DKI Dkk(v))))))) MEP(X) PEP())
where > 0 is the regularization parameter and p are defined as in straightforward calculation shows that
Using the definition of f,
(1; = -g)
where
(2) = / ((z) + (KI(H)T)
(v) / ((y)v(dy) + (klp))))
In this case, the simultancous MDA update reads
[att = arg min, { 6F (n", - (dxx + & DRI((()))) 5p
yat1= = arg max, {- 5v (v" ,),, - D" (dy) DKL(U/
(v
Note that since the game is non-interactive due to the definition of f, we essentially need to solve two independent optimization problems. Observe that F and Fş are both o-strongly convex relative to DKL in the sense of Lemma 3.1 i.e., for any n', 4 E P(X), we have
(7 - FT /
5F 8μ (n, 2) ()(d)) + a DRL(p')p),
and analogously for Fg. Thus, we can follow the proof of (Aubin-Frankowski the distinction that the Bregman divergence in Aubin Mubii--FFFaaknnn Frankowski et al. (2022) is actually the KL divergence DKL in our case. More specifically, in our sctup, equation (39) from the proof of (Aubin-Frankowe) Frankowski et al. 2022 Theorem 4), becomes
al. 2022
Theorem 4) with
Fd (2 S ) (n) (KI(ula")) + DRI(ulu") " DRI. (mla"tt)
for any P E P(X). Setting Ho arg min, (H) in the incquality above, using the fact that F (pt) S F (n*+1), and applying the discrete-time Gronwall's lemma with the assumption on S 1, show that
DRI(H)(p") I exp(-omm) DRI(HlHp)
Analogously, we have
DKL(V)(U") S exp(-omn) DKL(U))O))
Therefore, we obtain
DKL(GGH"") + s exp(-omm) (DKL(H)la") + (x
Then, we argue in the proof of Theorem 2.3, we can consoquently show that
1 (lom) / DKI + (KI(O)
NI(p", ) < 20, exp
In conclusion, in the case where there no interaction between players, adding regularization leads to exponential convergence of the discrete-time Fisher-Raa dynamics to the unique MNE of the game, whereas no regularization guarantees only O convergence rate to an MNE.
24


Reviews:
[{"review_id": "41lV0DU3JA", "reviewer": "Reviewer_Eqm9", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is clearly written. The proofs are well detailed and easy to follow. The convergence of the continuous flows for min-max games is not immediate since the dynamics can exhibit a cycling behavior [MPP18], making the result of this paper interesting.\n\nThe main identifiable weakness of the work is in the fact that implementable particle based discretizations of Wassertein-Fisher-Rao gradient flow already exist in the litterature [Wang, Chizat 2023]. The motivation for this result therefore needs to be argued further, why is it still interesting to establish convergence of the flow for strongly monotone settings?  For instance, the authors could discuss the fact that Wang and Chizat's work requires lower bounded step sizes (see paragraph after Thm 2.2 in Wang, Chizat) and also compare with the rates obtained when $\\sigma = 0$  in WC23 (i.e no Wassertein term) to establish the cost of discretization. \n\nThe second weakness is the lack of discussion around the NI bound. First, the NI bound is established without using the strong convexity, it is thus unclear if linear rates achieved for discretized particle schemes are impossible for the flow or if it simply a loose analysis. Second it is effectively vacuous as the left hand side involves a term than can be made arbitrarily large by putting mass on low probability regions of $\\mu_0, \\nu_0$. The authors should discuss why the quantity on the left hand side is not effectively $+\\infty$ and why it remains intersting.\n\n\n[MPP18] Cycles in adversarial regularized learning, Mertikopoulos, Papadimitriou, Pilouras", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper analyzes the convergence properties of Fisher-Rao gradient flow for regularized min-max games. The existence and unicity of the flow is established using a contractive map fixed point argument. Under a warm start condition, linear convergence to the unique equilibrium is established. A bound on the duality gap of the averaged iterates is also proposed.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The two weaknesses detailed in the previous section need to be discussed in the paper. \n\nI believe the discussion of the NI bound is critical and doable. \n\nImproved comparisons with particle Wassertein-Fisher-Rao schemes (where the Wassertein term is diminished) will strengthen the work and justify the need for this analysis, especially since the existence of the flow is established using standard Picard iteration arguments and would thus benefit from a strong motivation.", "broader_impact_concerns": "The work is of a theoretical nature.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0epGGwp05O", "reviewer": "Reviewer_Fcxx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. The convergence analysis via Lyapunov function is clear. The proof sketch provided in the introduction really benefits the paper presentation. \n2. The convergence analysis seems novel. \n\nWeakness:\n1. This paper would benefit from a more in-depth derivation of Fisher Rao gradient flow. This paper introduced the variational formulation of Fisher Rao distance. However, it does not derive the formulation of Fisher Rao gradient flow, but rather directly lays out the form. It would be great to derive that form, showing why the gradient with respect to Fisher Rao metric takes that form.\n\n2. The assumptions seem a bit strong. I understand that this paper aims to be general by considering a general functional. It would be great if some examples of functionals, beyond the bilinear objective, satisfying all these assumptions are introduced. I am especially nervous about Asssumption 3, which involves second order functional derivatives. It seems that the notion of functional derivative is different from the commonly seen Frechet derivative. I would encourage the authors also put the introduction of this version of derivative in the main paper (e.g., the background) and give a few concrete examples.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper explores the convergence of a Fisher-Rao (FR) gradient flow within the context of solving convex-concave min-max games, where the objective is a bivariate functional of the two input distributions, combined with KL divergence regularizations. The authors prove that  Fisher-Rao gradient flow finds the mixed (and regularized) Nash equilibrium at a sublinear rate, where the convergence is measured via the NI function. To establish this result, the authors use KL divergence as the Lyapunov function and prove that it converges at a linear rate along the course of FR gradient flow.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Add more discussions about the assumptions. See Point 2 above. \n\n2. Highlight that the functional derivative is \"flat derivatives\" and discuss this concept in details. \n\n3. Discuss how to implement this Fisher Rao gradient flow, especially using particle methods. Can it be implemented using particle methods? Or do we need to maintain a density? \n\n4. Question: Is the assumption that $\\int \\frac{\\delta V}{\\delta \\mu} (\\mu_t, \\nu_t) = 0$ without loss of generality? Can everything go through without this condition? Also, should this condition hold for all $(\\mu_t, \\nu_t)$?\n\n5. It seems that the proof analysis is very similar to that of Replicator Dynamics with entropy regularization for zero sum (or even monotone) matrix games. See, e.g., \"Mutation-Driven Follow the Regularized Leader for Last-Iterate Convergence\nin Zero-Sum Games\". I was wondering if FR gradient descent is a general version of replicator dynamics? Concretely, if distributions $\\mu$, $\\nu$ are supported on a finite set, and $F$ is bilinear, then your problem becomes a zero-sum matrix game. What is the corresponding algorithm and does that recover replicator dynamics? It would be great if you could add a section discussing this special case in detail. I believe this would benefit this paper a lot.", "broader_impact_concerns": "No concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "w8rQZmjmEX", "reviewer": "Reviewer_sTP3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\nThe mathematical results in this paper are new and correct to me. In particular, the long time convergence result is not achieved in previous works that consider the corresponding Wasserstein flows (as a comparison, the result of Lu 2023 holds only when the two dynamics have a timescale separation).\n\nThe setting of the problem is also interesting. There have been various works considering various min-max flows for finding mixed Nash equilibria, and this work is a nice addition to the field.\n\n\nWeakness: It would be nice if the authors could elaborate more about the motivation for considering Fisher-Rao gradient flows. The dynamics in Domingo-Enrich already have Fisher-Rao terms, which should be mentioned at an earlier and more visible place. Also for sampling purposes, the potential ability of the dynamics overcoming metastability is the main motivation (one could modify this sentence for other purposes like training neural networks). A small discussion along these lines would be nice.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work considers the Fisher-Rao gradient descent-ascent flow for finding the mixed Nash equilibria of the entropic-regularized min-max game. More precisely, regarding the mixed behavior of agents as a probability distribution over the set of strategies, the problem can be formulated as \\begin{equation*} \\min_{\\nu \\in \\mathcal{P}(\\mathcal{X})} \\max_{\\mu \\in \\mathcal{P}(\\mathcal{Y})}V^\\sigma (\\nu,\\mu), \\qquad \\text{ with } V^\\sigma(\\nu,\\mu) = F(\\nu,\\mu)+\\frac{\\sigma^2}{2}(\\mathrm{KL}(\\nu|\\pi) - \\mathrm{KL}(\\mu|\\rho)).\\end{equation*}\n\tHere $F$ is a sufficiently regular functional that is convex in $\\nu$ and concave in $\\mu$, $\\sigma>0$ is the regularization parameter, and $\\rho,\\pi$ are reference measures. The gradient flow dynamics then has the form (after suitable normalization)\n\t\\begin{equation*} \\partial_t \\nu_t (x)  = -\\nu_t(x) \\frac{\\delta V^\\sigma}{\\delta \\nu_t}(\\nu_t,\\mu_t,x), \\qquad \\partial_t \\mu_t(y)  = \\mu_t(y) \\frac{\\delta V^\\sigma}{\\delta \\nu_t}(\\nu_t,\\mu_t,y).\\end{equation*}\n\tThe authors proved that, if the initial conditions $(\\nu_0,\\mu_0)$ are pointwisely bounded above and below by $\\pi,\\rho$, then for any fixed $\\sigma$, the gradient flow dynamics is well-posed for all time and converges to the unique Nash equilibrium $(\\nu_\\sigma^\\star, \\mu_\\sigma^\\star)$ exponentially with rate $e^{-\\frac{\\sigma^2t}{2}}$. Finally, the Nikaidò-Isoda error also converges to zero with rate $\\frac{1}{t}$.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. p.2,  l.6, regarding existence and uniqueness of mixed Nash equilibria, the authors only listed Lascu et al. 2023 as reference; they should also cite Domingo-Enrich et al. 2020 here, even though their setting is slightly more restrictive (i.e. $F(\\nu,\\mu) = \\int \\int K(x,y) d\\nu(x)d\\mu(y)$).\n\n2. p.2, at the very beginning of Section 1.2. Fisher-Rao metric has been considered for a very long time, as it plays an important role in information geometry. I would suggest the authors to cite the following works instead:\n\n * Information Geometry and Its Applications, Amari.\n * Information Geometry. Nihat Ay, Jürgen Jost, Hông Vân Lê, Lorenz Schwachhöfer.\n\nRegarding the Benamou-Brenier formulation, besides Gallouët & Monsaingeon 2017, one could also call\n\t\t\n * Spherical Hellinger-Kantorovich Gradient Flows, Kondratyev & Vorotnikov.\n\nI find citing Liu et al. 2023 here gives too much credit to their work at this specific place.\n\t\n3. p.3, l.3, KL is lower semicontinuous under which topology of probability measures? I would prefer the authors to specify here.\n\n4. p.5, Assumptions 2 and 3, please refer to Definition B.1 so that the first and second variations make sense.\n\n5. p.5, the line between (13) and (14), I would prefer to say ``the MNE of (1) satisfies the fixed point equations $\\ldots$''. Also brackets are missing around the number 1. \n\n6. p.6, Theorem 2.2. What is the precise notion of solution of (17), in terms of function space and  time regularity? You also need initial condition to have finite KL divergence, correct?\n\n7. p.7, the first line in the second paragraph of Remark 2.5, do you mean ``explicit'' instead of ''implicit''?\n\n8. Bottom of p.15, the second to last inequality should be $\\log \\frac{\\nu_t(x)}{\\pi(x)} \\le \\log R_\\nu + \\frac{2}{\\sigma^2}C_\\nu  + 2\\log R_\\nu + \\frac{4}{\\sigma^2} C_\\nu.$ Namely, besides the first term, other terms on the r.h.s. should have an extra factor $\\frac{2}{\\sigma^2}$. Same issue for the last inequality of p.15. Corresponding changes are needed at the top of p.16.\n\n9. p.17, l.3, I suggest the authors to emphasize ``for __any__ fixed $x\\in \\mathcal{M} \\ldots$\"", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: We had good reviews and rebuttals and dialogue (and paper revisions). In the end, the reviewers were not unanimous in their recommendations but they were all close enough, with enough of a lean toward acceptance. In particular, we found that the paper meets the criteria for TMLR: enough interest, and accurate and convincing evidence.  So I'm pleased to recommend acceptance.

However, the reviewers still had a few "wishlist" items.  I think it is in the authors' best interest to address these issues, since they will make it a stronger paper and increase its impact.  In particular,

Reviewer `Fcxx` mentioned requested changes:
- a better introduction of the technical ingredients such as Fisher-Rao gradient and Flat derivative
- a better motivation of the problem setting and assumptions
- a clear articulation of the technical novelty of this paper.
- "implementation via particle method" was hard to follow, and it would be nice to develop the argument and make it more rigorous
- Fixing the issues with the WGAN-GP example (see their comments for details)

Reviewer `Eqm9` mentions fleshing out more about the motivation for continuous time (see their comments).
Decision: Accept (poster)