Uniform-in-time propagation of chaos for the mean-field gradient Langevin dynamics
Authors: Taiji Suzuki
Atsushi Nitanda
Denny Wu
Venue: ICLR 2023 poster (2023)
Keywords: Neural network optimization
mean-field regime
interacting particle system
propagation of chaos

Abstract: The mean-field Langevin dynamics is characterized by a stochastic differential equation that arises from (noisy) gradient descent on an infinite-width two-layer neural network, which can be viewed as an interacting particle system. In this work, we establish a quantitative weak propagation of chaos result for the system, with a finite-particle discretization error of $\mathcal{O}(1/N)$ \textit{uniformly over time}, where $N$ is the width of the neural network. This allows us to directly transfer the optimization guarantee for infinite-width networks to practical finite-width models without excessive overparameterization. On the technical side, our analysis differs from most existing studies on similar mean field dynamics in that we do not require the interaction between particles to be sufficiently weak to obtain a uniform propagation of chaos, because such assumptions may not be satisfied in neural network optimization. Instead, we make use of a logarithmic Sobolev-type condition which can be verified in appropriate regularized risk minimization settings. 

Published as a conference paper at ICLR 2023
UNIFORM-IN-TIME PROPAGATION OF CHAOS FOR THE MEAN-FIEL GRADIENT LANGEVIN DYNAMICS
Taiji Suzuki
Atsushi Nitanda
The University of Tokyo
Kyushu Institute of Technology RIKEN Center for Advanced Intelligence Project ni tandaaaa kyutech. ac jp
RIKEN Center for Advanced Intelligence Project t jiiiimmmsst i w-t.kyo..c...... jp
Denny Wu
The University of Toronto Vector Institute for Artificial intelligence dennywuecs. wtw.lttcsttorrtoooo www.nto.r...o...o.oo edu
ABSTRACT
The mean-field Langevin dynamics is characterized by stochastic differential equation that arises from (noisy) gradient descent on an infinite-width two-layer neural network, which can be viewed as interacting particle system. In this work, we establish quantitative weak propagation of chaos result for the system, with finite- finite-particle discretization eπror of (1//N) uniformly over time, where is the width of the neural network. This allows us to directly transfer the learn- ing guarantee for anfinite-witth networks to practical finite-width models without excessive overparameterrizatio. On the technical side, our analysis differs from most existing studies on similar mean field dynamics in that do not require the interaction between particles to be sufficiently weak to obtain a uniform propa- gation of chaos, because such assumptions may not satisfied in neural network optimization. Instead, we make use of a logarithmic Sobolev-type condition which can be verified in appropriate regularized risk minimization settings.
INTRODUCTION
Mean-field neural networks. We consider the optimization of a two-layer neural network in the mean-field regime, which is represented as an average over neurons:
1 N
fx(2) = N
h=(2;,,
where given the input Rdd each neuron computes a nonlinear transformation based on trainable parameters E Rdd for example, we may set h,(2)) tanh(w + b) for (w. b) "'+
Importantly, the mean-field parameterization allows for the parameters to move away from initial- ization during gradient descent and hence Iearn informative features (Yang and Hu, 2020) even when the network width is large (N + oo), in contrast to the Neural Tangent Kernel (NTK) pa- rameterization (Jacot et al., 2018) 11) (corresponding to 1/VN prefactor), which freezes the model initialization under overparameterrizaioon This feature learning ability enables mean-field neural networks to outperform the NTK counterpart (or linear estimators in general) in learning wide range of target functions (Ghorbani et al., 2019; Li al., 2020: Abbe al., 2022; Ba et al., 2022).
Optimization guarantees for mean-field field neural networks are typically obtained by lifting the finite- width model to the infinite-dimensional space of parameter distributions and then exploiting convex- ity of the objective function. Using this viewpoint, convergence of gradient flow on infinite-widie neural networks to the global optimal solution can be shown under appropriate conditions (Nitanda and Suzuki, 2017: Chizat and Bach, 2018; Mei et al., 2018; Rotskoff and Vanden-Eijndee 2018; Sirignano and Spiliopoulos, 2020). However, most existing results are qualitative in nature, in that they do not characterize the rate of convergence and the finite-particle discretization error.

---

Published as a conference paper at ICLR 2023
Mean-field Langevin dynamies. An often-studied optimization method for mean-field neural net- works is the noisy particle gradient descent (NPGD) algorithm (Mei et al., 2018; Hu et al.. 2019; Chen et al., 2020b), where Gaussian noise is injected to the gradient encourage "exploration" and enable global optimality to be shown under milder conditions than the noiseless case. The large particle and vanishing step size limit is termed the mean-field Langevin dynamics (Hu al., 2019), which globally minimizes an regropy-regularized convex functional in the space of measures.
Recently, Nitanda et al. (2022); Chizat (2022) established exponential convergence for the mean- field Langevin dynamics under certain logarithmic Sobolev inequalities which can be easily verified in regularized risk minimization problems using two-layer neural networks (I). This represents significant step towards a quantitative optimization analysis of neural networks in the presence of feature learning, yet the limitation also clear: these results are obtained from directly analyzing the large particle limit (i.e., the limiting McKean- stochastic differential equation), and cannot be easily transferred to practical finite-width networks. In fact, naively applying the quantitative results in Mei et al. (2018; 2019) leads to discretization error bounds that blow up exponentially in time, rendering the guarantee vacuous beyond the very early stages of gradient descent learning. There- fore, for the purpose of characterizing the optimization behavior of unite-width neural networks, it important to derive finite- sorite-prttile discretization error bound that holds uniformly over time, that is, the error remains stable even when is large.
1.1 OUR CONTRIBUTIONS
In this paper, we establish finite-particle guarantees for the mean-field Langevin dynamics via a propagation of chaos calculation (Sznitman, 1991) which controls the weak error between the empirical distribution of the interacting particle system and the corresponding infinite particle limit along the optimization trajectory. This allows us to bound the differ- ence in the function value between the width neural net- work optimized by NPGD and the infinite-width counterpart.
training errot
1
(PDA)
i.id In particular, starting from N initialized particles Xo ~ 140. if we denote the paiteceerticle model at time of optimiza- tion as fx,, and its corresponding infinite-particle limit as ful then our propagation of chaos result is the following.
10a
10 10a
10
number steps
Figure I: Training error of two-layer NNs optimized by NPGD. Solid curve: mean over 100 runs. Translucent curve: individual runs. Dashed black line: global optimum approximated by the PDA algorithm (Nitanda et al., 2021).
Theorem (informal). Under suitable regularity conditions, [[(fx (2) ((x)))]]= (1/N) for any t> 0 and 2 Rdd
We make the following remarks on the main theorem:
To our knowledge, we provide the first rigorous uniform-in-time propagation of chaos result in the context of mean-field neural networks. This is in contrast prior works where the discretization error typically increases as optimization proceeds (e.g., 1fx.-fxx (expp(t)- N-1/2 as in Mei et al. (2018, Theorem 3)). The theorem implies that as the width N becomes larger, the difference between the finite-width and indhhite-witth model output diminishes rapidly, shown in Figure 1.
Our analysis assumes a modified Log-Sobolev condition which is satisfied in regularized risk minimization problems using neural network when the convex regularizer on the parameters has super-quadratic tail. Noticeably, we do not impose any constraint on the strength of regularization and interaction; this differs from many existing results where uniform propagation of chaos is only achieved under weak interaction or large noise (Eberle et al., 2019; Delarue and Tse, 2021).
Due to the space constraint, we defer discussions on additional related works to Appendix A.
2 PRELIMINARIES
In this section, we formulate the problem setting and introduce some useful notations for the fol- lowing sections. We optimize a two-layer neural network by minimizing the empirical or expected risk in a supervised learning setting, where the input is included in a set c Id and the output is in bounded set y c R. As defined in the Introduction, (x)) Z hz(x) E y represents
2

---

Published as a conference paper at ICLR 2023
neuron (particle) with parameters RR, and the mean mean-field neural network is written as the average over N neurons: (x) , E (x,),, where X (xx) Rd denotes the col- lection of parameters and 2 E Z. The continuous limit of the neural network is obtained by taking N + 00, and in analogy the law of large numbers, fx converges the following integral form:
Sx(=)= ] (x))d(()))
where is probability measure (IR )B representing the weight of cach parameter. Let P the set of probability measures on (R2 (B)))), and Pp be those with a finite p-th moment (p 2 1). As typical for the mean-field analysis, we aim to optimize the density function μ so that the neural network f accurately predicts the output y E ) from the input & E 2.
In the following, we take (regularized) empirical risk minimization note that the exact same analysis applies to the minimization expected risk. Let ((2,)) y>> x + the logistic loss 8(2, y) = log(1 + exp( ((2-)y)) ) for classification. For each (z, yi) in the given training data (zi, 8/ )=1 Z V, we use the notation (x) and (f() to indicate ft2, (x) and (((()) 9i) respectively, Our goal is to find an approximate minimizer of the following objective over P:
concrete example, and
R be convex loss function, such as the squared loss {(z,)) v) (= y) for regression,
F(R) = F(a)==- Σ ((tW) r(x))u(x))
(1)
where d > is the regularization strength and r(-) a convex regularizer. More specifically, we will analyze the mean-field Langevin dynamics which solves entropy-regularized version of (I).
worth noting that this entropy-regularized objective can also be globally optimized by the re- cently proposed particle gradient-type methods in Nitanda et al. (2021); Oko et al. (2022), for which finite-width convergence rates have been provided. However, those methods employ an intricate double-loop structure which does not mirror the uomeonly-used gradient descent algorithm. There- fore, an important question to be addressed is whether noisy gradient descent also enjoys similar quantitative convergence guarantee this is precisely the motivation of the current paper.
MEAN-FIELD GRADIENT LANGEVIN DYNAMICS
Derivation of the continuous dynamics. The basic idea of the mean-field Langevin dynamics is to optimize the aforementioned objective via Wasserstein gradient flow over a set of measures P. To define the gradient with respect to the measure, we introduce the first-variation &G of functional
G P2 + at E Py (for a given 2 1) as continuous functional Py x RRd + R that satisfies GlEv + (1 - e)p)
/
8G
lim 0
(w)(x)d(v 1n) 5yz
6
for any v E Pg. If there exists such functional
&G
bμ we say G admits a firit-variatinn at μr, simply 8G
G is differentiable at μ.. To avoid the ambiguity of öü up to constant shift, we follow the convention of imposing / de (n)dæ O. our setting, the first-variation the objective F is given by
8F (n)(x) 8μ
Σ +(an))())+Arre)))) n
We track F(Ja) along trajectory of measures (uu)r in Pa following continuity equation:
0,4- V -uuv))
where Rd + d is vector field included in (3())), and the time-derivative and the divergence operator are defined in weak sense, that is, for any continuously differentiable function with compact support, dodju ddu,, = - V v,du,dr. Then, the time-derivative of G(u)) can be written
a,G(m)= f V G(p))
(2)
We refer readers to Villani (2009); Ambrosio et al. (2005); Bakry et al. (2014) for more details. In F(u)) can be seen gradient direction in the measure space (endowed with this sense, V oy
3

---

Published as a conference paper at ICLR 2023
Wasserstein metric). The mean-field Langevin dynamics approximately minimizes the objective F based on the Wasserstein gradient flow. Specifically, define the nonlinear drift term:
bi,, ())
8F oy
(1)
Σ (h) + Vr(x),
the mean-field Langevin dynamics is then given by the following stochastic differential equation:
dX;; b(X,, He) + 22ddW,, Law(X;),
(3a)
(3b)
for Xo No. where Law(X) denotes the distribution (probability law) of the random variable X, and (Wo) is the d-dimensional standard Brownian motion. The existence and uniqueness of the solution are ensured by Theorem 3.3 of Huang al. (2021) (see also Corollary for more details).
For concise presentation in the subsequent analysis, we follow the notation in (Delarue and Tse, 2021) and denote the law pr in the mean-field Langevin dynamics with the initial condition μs as [4 m(L, Mo). It is known that μa, satisfies the following nonlinear Fokker- Planck equation:
0em(t, no) = AAm(t, no) + [m((,
(4)
with m0( (o) = po (this is again defined in a weak sense, that is, f dd(mm(t, (b) mm,, ms,,Ho)) = Se [(AØ b(. m(T, mo)) Vo)dm((T do for smooth test function f with compact support). This dynamics is an example of distribution dependent SDEs originating from the study interact- ing particle systems which dates back to 1950s (Kahn and Harris, 1951; Kac, 1956; 1959; McKean, 1966; 1967). A fundamental characterization of the mean-field Langevin dynamics is that it is a Wasserstein gradient flow that minimizes the following objective (Mei al., 2018; Hu al., 2019):
((u) = F()) + AEnt(u),
(5)
log(du(2)/dz)dd(} is the negative entropy of μ.. Indeed, it is known that V Vdl(m) V + XV log()) XV log(H) + b(- m1) (e.g, Theorem 4.16 of Ambrosio et al. (2005))
where ())
dy 6z
and the continuity equation corresponding to j4 can be rewritten as 0./4: = V [(AV log(Ue) + dy pe), which, in combination with the identity (2), yields that
b(:,))))mu] V (V (C(u)
/ 5L(H)
a c(m)
duc.
We therefore see that μ4 decreases L(vx) unless 8C(x) O, which is crucial property that guaran- tees the convergence of br to the global optimal solution (Lemma 1). This can be seen as nonlinear extension of the usual gradient Langevin dynamics (c.g.. see Bakry et al. (2014)), where F()) is a linear functional in the form of F(4)
L(x)du with some objective function L.
is easy to see that the objective L can be reformulated as the following objective (up to constant) that employs the KL divergence from distribution characterized by the regularization term
C(w) Σ (.(f)) + AKL(H,P))
(6)
where v, is a distribution with density proportional exp(- (Ai/r/)//)))))) and KL(.,-) is the KL diver- gence (relative entropy) defined as KL, v) log( dl/dd)dd.
Particle discretization. One of the main difficulties to simulate the mean-field Langevin dynamics that we cannot access the exact information of μ in the practical setting. Instead, we approximate this lintti-dimensional objective by finite set of particles, which yields the following SDE:
1 N
ax -b(A, dd V2XWW; M
x
and X; ~ A4g- Our goal is to quantify the finite mnite-particle approximation error due to replacing μ4 with pi The main mechanism of this approximation is the propagation of chaos (Sznitman, 1991).
4

---

Published as a conference paper at ICLR 2023
which roughly refers to the phenomenon that as the number of particles N oc, the correlation between particles vanishes and μiب" + ft. However, it is far from trivial to establish a quantitative estimate of such approximation that provides meaningful guarantee for the finite-particle system. In the following sections, we will show that under appropriate conditions, the phiite-prttcle error can be (weakly) controlled uniformly over time, which allows to transfer learning guarantees in the mean-field limit to finite-width neural networks that are not excessively overparameterized.
4 MAIN ASSUMPTIONS AND LOGARITHMIC SOBOLEV INEQUALITY
In this section we present our main theoretical result the quantitative propagation of chaos. First, we introduce the main assumption in our analysis.
Assumption 1, We assume his r, - and satisfy the following conditions:
1. Convexity of loss: & is a convex function.
Boundedness and smoothness: There exists O such that |1hellcc I B. 11Vhsllo I B, VVV halloo S B, (ax() 195()) (fW))) s B uniformly over P.
3. Regularity of r: The regularization term r is convex function satisfying cr||æ||2+6 <r(a)= S
C, - ||=||²
Vr(x) x c|||=||2+8 and 0 N VV = C1 + ||>||) 1 for constants
0 < 8 and Cr, C, > O.
We make the following remarks on the assumptions.
The loss convexity is standard assumption to ensure that the objective L is convex with respect to I (note that this does not imply convexity with respect to the parameters {a:}/= of the network).
The second assumption is satisfied for standard two-layer models under the following conditions: (i) ||=|| I C: (fi) smooth loss function, such as the squared loss and logistic loss. For example, we may set (x)
tanh(ro ( 2)) with smooth activation function a and x (r, w), or h=()) = (w z + b) with smooth and bounded activation and x = (w,b) b).
The constraint on r(-) requires the regularization term to have a super-quadratic tail, which is satisfied, for example, by r(x) = | While this does not cover the standard weight decay, we note that such regularizers with stronger tail growth have been employed in the theoretical analysis of neural networks (Chen et al., 2020a; Allen-Zhe and Li, 2022). The purpose of this assumption is ensure good isoperimetry of μr along the trajectory (see Corollary 1).
The infinite differentiability condition is imposed only for the simplicity of our analysis.
Proximal Gibbs distribution. An important quantity in the convergence analysis is the proximal Gibbs distribution: for P, we define the proximal density function
1
1 18F(1)) (x:) exp (-x
Pa(x)=
Z(H)
where Z(1) is the normalization constant. One may check that this corresponds to the minimizer 8y
of the linearized potential: min f 5F(()) dv + XEnt(v). For a given Pt, we denote by te its proximal Gibbs measure, that is, the probability measure with the density Ppc- Then, we have the following characterization of the minimizer of L.
Proposition 1. Under Assumption 1, the functional L has a unique minimizer in P that is absolutely continuous with respect the Lebesgue measure. Moreover, p² P248 the optimal solution if and only if nC" is absolutely continuous and its density function is given by Py-
This proposition can be shown in the same manner as Proposition 2.5 of Hu et al. (2019). We remark that although this prior result assumed r to have at most quadratic growth, its proof does not require such growth condition but requires only the integrability of u, and (x) (x)())xx)).
Many convergence properties of the mean-field Langevin dynamics can be characterized by prop- erties of jC and the proximal Gibbs distribution ür. We first introduce the logarithmic Sobolev inequality (LSI) which will be very useful in the subsequent analysis.
'This condition can be casily relaxed to |=r| < r(x) S C+(1 + || with 8 of 8' > 0. Here consider 8 8 just for simplicity of presentation.
5

---

Published as a conference paper at ICLR 2023
Definition (Logarithmic Sobolev inequality). Let p(0) be smooth probability density function R.. p(0) (or its corresponding probability measure on (R. B((d))) satisfies the LSI with constant
a' > O if and only if. for any smooth function Fd R with Ep co, it holds that
2 a'
p0 2 log( (b)) - Bepl log(E S P
B [
We can verify that in our setting, the proximal Gibbs measure satisfies the LSI condition. Proposition Under Assumption 1, Py satisfies the log-Sobolev inequality with a constant a that depends on d, Cr, B, X, 8. If additionally VV ž then the LSI holds with a = 221 exp (- (- ).
The proof is given in Corollary 5 in the Appendix. Note what similar characterization was obtained in Nitanda et al. (2022); Chizat (2022) under a quadratic regularizer r(z) = ||x:| via the standard Bakry-Emery and Holley-Stroock arguments (Bakry and Emery, 1985; Holley and Stroock, 1987) (see also Corollary 5.7.2 and 5.1.7 of Bakry et al. (2014)). However, our Assumption does not entail v r = and thus our proof follows a different strategy. This LSI condition is crucial to the geometric ergodicity of the mean-field Langevin dynamics described in Theorem below.
5 CONVERGENCE GUARANTEE FOR FINITE-WIDIH NEURAL NETWORKS
To present our (weak) convergence result, we first introduce an objective function in the form of U(t, μ) == p))(t,
where +b P >> IR is assumed to be sufficiently smooth, that is, twice differentiable with respect to μ and x, and the derivatives are bounded supz,..... su. | Ois
5" 5"((
6y (II,,..., xx)] < C
for As 0, 1, 2 and ji = 0, 1, uniformly over all μ E with some constant C (see Delarue and Tse (2021) for related definition).
Example 1. Under Assumption we allow for the following objective functions.
(i) Neural network function value: x()x) = f (z()dp((z) with a fixed E Z.
(ii) Training and test error: 2(p)) = (E,,
Y)] where P is a distribution on Z x V. For a smooth loss f, & satisfies the smoothness condition. If = E (x..)))) then is the training loss, and if P is the test distribution, is the test loss.
We proceed by bounding the the weak difference between the finite-particle system at time / and the optimal n² (see Proposition 1): ([ )] $(x"), ). We utilize the following decomposition:
(( (n²) Elu(t, 1 2 (p
(0, u(t, μ
(8)
(i). orgodicity term
(II). propagation of chaos term
where the ergodicity term (I) monitors the convergence of the infinite-particle dynamics (3) (starting from wg") to the optimal solution μ², and the propagation of chaos term (II) controls the fluctu- ation due to the sarte-particle update (4). The two terms are bounded separately in the ensuing subsections. Note that while we focus two-layer neural networks under Assumption 1, the same computation can be performed under certain isoperimetric conditions on the trajectory. In particular,
Analysis of the ergodicity term (I) only requires the proximal Gibbs measure jú to satisfy the LSI. This condition can be easily verified for both quadratic regularizer as in (Nitanda et al., 2022; Chizat, 2022) and super-quadratic regularizers as shown in Proposition 2.
To control the propagation of chaos term (II), we require an LSI condition on l4 along the trajectory. Similar assumption also appeared Lacker and Flem (2022) to obtain a uniform-in- time evaluation, and is very challenging to establish in the mean-field neural network setting. We prove this assumption by transferring the LSI constant from ea to μ4 via a super LSI condition, which is verified under the super-quadratie regularization in Assumption I (see Lemma 2).
5.1 BOUNDING THE ERGODICITY TERM (1)
Let W, (p(,.v) v) denote the p- p-Wsssesttein distance between 14 E Pp- We first show that mit, 1 converges n² in exponential order (geometric ergodicity) in the following sense.

---

Published as a conference paper at ICLR 2023
Lemma 1. Under Assumption f.ror p4 m(t, ) it holds that L(pe) < oc for any 0, and
((u)) - [(²") S exp(- (T)) p²)? (T](,,1)) S exp( exp(-2a>\t - (T)) (2
B² for any t > To where To is arbitrary, (/2(t) 2x 1-exp(-B-p B2c) LSI constant of Ri given in Proposition 2.
1 exp((1911) and the
2
The proof can be found in Corollary 4 in the Appendix. This is an extension of the "entropy sand- wich" argument in Nitanda et al. (2022); Chizat (2022), in which the right hand side of the bound is given by (L vc") instead of the Wasserstein distance. However, in our setting, 140 1 is discrete distribution and thus the KL divergence from μ not finite. To resolve this issue, our anal- ysis shows an upper bound of the KL divergence at f > To via the Wasserstein distance W2 (Po, (²") (see Corollary 3). Then, we obtain the following theorem on the convergence of term (I).
Theorem (Geometric ergodicity). Under Assumption 1, the term (I) converges
Et m ) ) +(x²) s 2 (2((T) exp(-eXpp axt To) ,b"),
for any t >> To where 0 is arbitrary positive real number.
Proof. By Otto-Villani's theorem (Otto and Villani, 2000), LSI implies Talagrand's inequality:
W2(n, n²) S v a [KL(H, Also, the smoothness of & entails F(x) P(a") S CW2(u,
(see Lemma 10). The assertion is obtained by combining Lemma and Talagrand's inequality. Li
5.2 BOUNDING THE PROPAGATION CHAOS TERM (II)
Bounding the second term (II) is much more involved. We utilize the following evaluation adapted from Delarue and Tse (2021) (see Arnaudon and Del Moral (2020) for similar calculation):
(4, (t,, (
1 N
Σ
d
Iu (t- s, 1. (c. p. (dx) ds. {
Intuitively, the integrand on the right hand side approximately represents (0, p - n for small that is, how small time difference between the finite particle and continuous limit propagates to the terminal time t. Here, for a linear operator acting on a function f: RR - R, we write (() q(f). Then from Delarue and Tse (2021) (see also Appendix C.2, C.3) it holds that
8-u (t, (o),
(e.)) 0 (2)) 8n2
σつΦ ( (t: po: E, (t: (t:) + 5$ 8p
(t (t;
8r2
where d0t) and d are linear operators defined by
5 (t: E, x,21) = d on (m(t: (o))
8 (1: Iı, (0) ((.)) 0(m,), fn (m(t; (1),)))
for a smooth test function Φ Rd + R, where m)t. = ʃ f( (d)))m((, (x) (we will also use the same notation for general measure p). The dynamics of these operators is characterized by Proposition 6 in Appendix C.I, which is adapted from Delarue and Tse (2021). To obtain uniform-in-time evaluation of term (II), we aim show a rapid decay of d!") and d.9).
Isoperimetry of μr via super LSI. Our strategy is to establish the boundedness of the integral 8u (t - 8, 2 (a, x) (dx) ds by proving the exponential convergence of 16' (..))) on"
8'u D(m)): (12)): ond (t k. N) (z, However, this requires local evaluation around m(t s; s;,"d", for which we cannot exploit "global" properties such the log-Sobolev condition of the optimal
7

---

Published as a conference paper at ICLR 2023
solution jr" Instead, our current analysis requires m(t m; ) to also satisfy the LSI condition, which is technically demanding to establish. To our knowledge, similar condition has only been recently verified in Guillin et al. (2021); Lacker and Flem (2022) for a limited class of interaction potentials which cannot cover the case of mean-field neural networks.
To overcome this difficulty, impose stronger- ceseeer-haanquueee than-quadratic tail growth condition the regu- larizer, i.e., r(z) = 2[[|==]| (see third point in Assumption 1). Under this assumption, we can show that the proximal Gibbs measure Jr, associated with f4, mm( 8; u satisfies the super logarithmic Sobolev inequality defined below.
Definition (super logarithmic Sobolev inequality (super LSI)), We say that a probability measure satisfies super log-Sobolev inequality if there exists monotonically non-increasing function B (0, 00) + E such that for any & satisfying Eylø²] 1 and 1 [ < oc, it holds that
ul log o²) Sr / |V 3(r) (vr > 0).
It is known that super LSI implies LSI if there exists such that ((r) O.
Lemma 2. For any P, the probability measure ji corresponding to the proximal Gibbs density
Pu satisfies the super log-Sobolev inequality with ((r) C' 1426 log(r /2) with constant
6C" ) C² > O. Furthermore, it satisfies the log- log-Sobolev inequality with LSI constant à exp(- 1+25
The proof is given in Appendix B.3. An important consequence of this lemma is that frr and ja, have bounded density ratio when / is sufficiently large. This implies that many properties of th also inherited by μa- Crucially, the bound on density ratio is strong enough for the LSI condition to be transferred from tu to Ht, which allows us establish the exponential convergence of af")) and d²)
Corollary 1. Under Assumption 1, there exists some To > 0 depending d, B, 8, X, Cr. Cr and Qo depending on W2(Ho, v(") such that for all 2 To + Qo we have
1
dj4
2 (Vx ERRR)
SC'e To)),
V2
dj
no
where C' is some positive constant. Moreover, for To Qo 44 satisfies (a/2)-LSSI.
Uniform-in-time propagation of chaos. Equipped with the LSI condition on Pex we can now control term (II) in the error decomposition by proving exponential convergence of a!") and ( In particular, LSI implies the Poincaré inequality which then roughly ensures the KL-divergence behaves like a strongly convex function around μ4. Therefore, small perturbation from He expo-
d
nentially converges to 0 as grows, which entails the fast convergence of d'") and (2) because these quantities represent infinitesimal displacement of pr.
Theorem 2 (Uniform Propagation of Chaos). Suppose that the support of po is bounded. Then for any 0 and 1 sis i it holds that
[
(D ( 8uz2 (t 105-s,K -.. (x., u. (dx) = =O(exp(->ats -To)/2)).
8
E
with some constant To > This implies that
(4, - u(t, H = O(N
The proof of this theorem can be found in Appendix C.6. addition, note that the informal theorem stated in Section is direct consequence of the above theorem (see Corollary for details).
5.3 PUTTING THINGS TOGETHER
By combining the previous calculations, we arrive at the following characterization on the difference between the finite-width neural network and the optimal (infinite-widde solution μ²" Corollary 2. Under Assumption 1, if the initial distribution μe has bounded support. then we have
Eu(O, ph NN) x(n") < C C2 20- exp(-xxX -o));
for some constants C1, C2 and any To >
8

---

For the figures, we report the training or test error without the regularization terms. In Figure I in the Introduction, we plot the training error for r(x) ||>/|| whereas in Figure 2, we plot the test error for r(z) = ||=|| Observe that even though our current theoretical analysis does not cover the Published as a conference paper at ICLR 2023
Noticeably, since both Theorem 1 and Theorem 2 do not require sufficiently large regularizations, our finite-particle guarantee holds for any choice of regularization strength A, A1; in contrast, prior works on uniform propagation of chaos typically assume weak interaction large noise, which limits the applicability to the optimization of neural networks in the mean-field regime.
As important consequence of our general convergence guarantee, know that the finite particle dynamics also exhibits a similar decay of the training and test losses compared to the continuous limit up O(1/N) discretization error; this can be straightforwardly checked by considering the special setting where ((n) = E(zz,))-(((uu(2) Y)] (see Example 1).
Remark 1. Our current convergence result holds in expectation. To obtain a high probability state- ment, as discussed in Arnaudon and Del Moral (2020), one may apply a martingale concentra- tion inequality fe.g., see Lemma 3.2 of Nishiyama (1997)) to obtain guarantee in the form of with probability 1 -c
(( (t) Elu(0, Utt,, n (ovooo((()///
NUMERICAL EXPERIMENTS
We provide empirical support for our propagation of chaos result in synthetic student-teacher setting. We consider the empirical risk minimization problem, where the training labels are generated by teacher model which is Gaus- sian function defined as (z) exp (-Il==all") We set n = 2000,, = 20. The loss is chosen to the squared error, and for the regularization term we set r(x) ||> or r(x) = ||=|| and the regularization strength x = A = 10-2 The student model is two-layer neural net- work with tanh activation, and the width N is taken to be (16,32, 32, 64, 128, 256, 512, 1024, 2048}. We optimize the student model using NPGD with step size = 10-2 The global optimal solution μ² is approximated via the particle dual averaging (PDA) algorithm (Nitanda et al.. 2021): we set the model width N 2048 and number of outer loop steps T 250; we scale the number of inner loop steps T, with t, and the step size Tte with 1/Vt, where is the outer loop iteration.
2
test error
10:2
M- 16
6 60-10
M- 2048
(PDA)
x 10
10a
10>
number of steps
Figure 2: Test error of NNs optimized by NPGD (r(z) = ||=||()) Solid curve: mean over 100 runs. Translucent curve: individual runs.
latter setting, the empirical trends are almost identical: as the width N increases, the performance of the finite-width network improves and approaches that of the (approximate) optimal solution /²"
CONCLUSION
In this paper, we established the first uniform-in-time propagation of chaos for the mean-field Langevin dynamics in the context of neural network optimization. In contrast to most existing works, our analysis gives quantitative discretization error and does not blow up through time, and do not impose the commonny-assunne a super logarithmic Sobolev inequality that is satisfied by a regularization term with super-quadratic tail. This condition then enables us establish good isoperimetry of the intermediate solution yt, which gives an exponential convergence of the eтror propagation.
weak interaction condition. This is achieved by utilizing
Limitations and future directions. Our current analysis requires a super-quadratic tail of the regularization term, which does not cover the commonly-used l regularization (weight decay). We note that after our initial submission, Chen al. (2022a) developed a different proof technique based on the tensorization of LSI which handles the case of quadratic regularization. Another important future direction to extend the analysis discrete-timg dynamics. Finally, the mean-field Langevin dynamics has found applications beyond neural network optimization (Chizat et al., 2022): hence we are optimistic that our technique can provide unite-particle guarantee for interacting particle algorithms in other applications.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENT
IS was partially supported by JSPS KAKENHI (20H00576) and 1ST CREST. AN was partially supported by JSPS Kakenhi (22H03650) and PRESTO (IPMJPR1928). DW was partially sup- ported by a Borealis Al Fellowship.
REFERENCES
E. Abbe, E. B. Adsera, and T. Misiakiewicz. The merged-staircase property: a necessary and nearly sufficient condition for sgd learning of sparse functions on two-layer neural networks. In Confer- ence on Learning Theory, pages 4782-4887. PMLR, 2022.
S. Akiyama and T. Suzuki. On learnability via gradient method for two-layer relu neural networks in teacher-student setting. In International Conference on Machine Learning, pages 152-162. PMLR, 2021.
Z. Allen-Zhu and Y. Li. Feature purification: How adversarial training performs robust deep learn- ing. In 2021 IEEE 62nd Annual Symposium Foundations of Computer Science (FOCS), pages 977-988. IEEE, 2022.
L. Ambrosio, N. Gigli, and G. Savare. Gradient Flows: In Metric Spaces and in the Space of Probability Measures. Lectures in Mathematics. ETH Zürich. Birkhäuser Basel, 2005.
M. Amaudon and P. Del Moral. second order analysis of Mckean-Vlason semigroups. The Annals of Applied Probability, 30(6):2613-2664 2020.
Ba, M. A. Erdogdu, T. Suzuki, Z. Wang, D. Wu, and Yang. High-dimensional asymptotics of feature learning: How gradient step improves the representation. In Advances in Neural Information Processing Systems, 2022.
D. Bakry and M. Emery. Diffusions hypercontractives. In J. Azéma and M. Yor, editors, Séminaire de Probabilités XIX 1983/84, pages 177-206, Berlin, Heidelberg, 1985. Springer Berlin Heidel- berg. ISBN 978-3-540-39397-9)
D. Bakry, I. Gentil, M. Ledoux, et al. Analysis and geometry of Markov diffuusio operators, volume 103. Springer, 2014.
F. Chen, 7. Ren, and S. Wang. Uniform-in-time propagation of chaos for mean field langevin dynamics. arXiv preprint arXiv:22/2.03050, 2022a.
M. Chen, Y. Bai, J. D. Lee, T. Zhao, H. Wang, C. Xiong, and R. Socher. Towards understanding hier- archical learning: Benefits of neural representations. Advances in Neural Information Processing 2020a.
Systems, 33:22134-222
Z. Chen, Y. Cao, Q. Gu, and T. Zhang. A generalized neural tangent kernel analysis for two-layer neural networks. arXiv preprint arXiv:2002.04020 2020b.
7. Chen, G. M. Rotskoff, J. Bruna, and F. Vanden-Eijnden. A dynamical central limit theorem for shallow neural networks. arXiv preprint arXiv:2008. 09623, 2020c.
Z. Chen, E. Vanden-Eijnden, and J. Bruna. On feature learning in neural networks with global convergence guarantees. arXiv preprint arXiv:2204.100 10782, 2022b.
L. Chizat. Sparse optimization on measures with over-parameterized gradient descent. arXiv preprint 1rXiv:1907.10300 2019.
L. Chizat. Mean-field langevin dynamics: Exponential convergence and annealing. Transac- tions on Machine Learning Research, 2022. URL https: opw.reveeeeeeeeeeeeeeee...
/ttttt///////oeererrrie net forum?
id Ad=BDqzLHI LHgzHHlgge Lm..Em.
L. Chizat and F. Bach. On the global convergence of gradient descent for over weer-paammttrizeed models using optimal transport. In Advances Neural Information Processing Systems 31, pages 3040- 3050, 2018.
10

---

Published as a conference paper at ICLR 2023
L. Chizat, S. Zhang, M. Heitz, and G. Schiebinger. Trajectory inference via mean-field langevin in path space. arXiv preprint arXiv:2205.07 146, 2022.
V. De Bortoli, A. Durmus, X. Fontaine, and U. Simsekli. Quantitative propagation of chaos for sgd in wide neural networks. Advances in Neural Information Processing Systems, 33:278-288, 2020.
Delarue and A. Tse. Uniform in time weak propagation of chaos on the torus. arXiv preprint arXiv:2104. 14973, 2021.
M. G. Delgadino, R. S. Gvalani, G. A. Pavliotis, and S. A. Smith. Phase transitions, logarithmic propagation of chaos for weakly interacting diffusions.
sobolev inequalities, and arXiv preprint arXiv:2 /2.06304, 2021.
A. Durmus, A. Eberle, A. Guillin, and R. Zimmer. An elementary approach to uniform in time propagation of chaos. Proceedings of the American Mathematical Society, 148((2):5387-5398, 2020.
A. Eberle, A. Guillin, and R. Zimmer. Quantitative harris-type theorems for diffusions and mekean- vlasov processes. Transactions of the American Mathematical Society, 371(10):7135-7773 2019.
H. Föllmer. Time reversal on wiener space. In Stochastic processes-mathemeen and physics, pages 119-129. Springer, 1986.
B. Ghorbani, S. Mei, T. Misiakiewicz, and Montanari. Limitations of lazy training of two-layers neural network. In Advances in Neural Information Processing Systems, pages 911 1-9121, 2019.
A. Guillin, P. L. Bris, and P. Monmarché. Uniform in time propagation of chaos for the 2d vortex model and other singular stochastic systems. arXiv preprint arXiv:2108. 08675, 2021.
A. Guillin, W. Liu, L. Wu, and C. Zhang. Uniform poincaré and logarithmic sobolev inequalities for mean field particle systems. The Annals of Applied Probability, 1290)11590-1614 2022.
R. Holley and D. Stroock. Logarithmic sobolev inequalities and stochastic ising models. Journal of statistical physics, 46(5-6): 159-1194, 1987.
K. Hu, Z. Ren, D. Siska, and L. Szpruch. Mean-field langevin dynamics and energy landscape of neural networks. arXiv preprint 1rXiv:1905.07769, 2019.
X. Huang, P. Ren, and F.-Y. Wang. Distribution dependent stochastic differential equations. Fron- tiers of Mathematics in China, 16(2):257-301, 2021.
A. Jacot, F. Gabriel, and C. Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In Advances in Neural Information Processing Systems 31, pages 8580-8589, 2018.
R. Jordan, D. Kinderlehrer, and F. Otto. The variational formulation of the fokker-planck equation. SIAM journal on mathematical analysis, 29(1):1-17, 1998.
M. Kac. Foundations of kinetic theory. In Proceedings The matical statisties and probability, volume 3, pages 171-197, 1956.
The third Berkeley symposium on mathe-
M. Kac. Probability and related topics in physical sciences, volume 1. American Mathematical
Soc., 1959.
H. Kahn and T. E. Harris. Estimation of particle transmission by random sampling. National Bureau of Standards applied mathematics series, 12:27-30, 1951.
Y.-H. Kim. note on the moment inequalities for stochastic integral. Math. Inequal. Appl, 16(4): 1023-1029, 2013.
D. Lacker. Hierarchies, entropy, and quantitative propagation of chaos for mean field diffusions. arXiv preprint arXiv:2105.0245 2021.
D. Lacker and L. L. Flem. Sharp uniform-in inttmm----time propagation chaos. arXiv preprint arXiv:2205. 22002205..27...... 2022.

---

Published as a conference paper at ICLR 2023
Y. Li, T. Ma, and H. R. Zhang. Learning over-parametrized two-layer neural networks beyond ntk. 2020.
In Proceedings of Conference on Learning Theory 33, pages
H. P. McKean. A class of markov processes associated with nonlinear parabolic equations. Pro- ceedings of the National Academy of Sciences, 1676);1007-191, 1966.
H. P. McKean. Propagation of chaos for a class of non-linear parabolic equations. Stochastic Differential Equations (Lecture Series in Differential Equations, Session 7, Catholic Univ., 1967), pages 41-57, 1967.
S. Mei, A. Montanari, and P.-M. Nguyen. A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences, 115(33):E7665 2018.
S. Mei, T. Misiakiewicz, and A. Montanari. Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit. arXiv preprint (:XXix:902.00011 2019.
S. Mischler. An introduction to evolution PDEs, Chapter 0: On the Gronwall lemma, 2019. URL https: /www. ceremade........................ dwwphine httt:///eetteseeeeeeeeeeeeieeeie hww.:/2220ccchhp..oooo
N. Nishikawa, T. Suzuki, A. Nitanda, and D. Wu. Two-layer neural network infinite dimensional data: global optimization guarantee in the mean-field regime. In Advances in Neural Information Processing Systems.
Y. Nishiyama. Some central limit theorems for -aauue semimartingales and their applications. Probability theory and related fields, 108(4):459-494, 1997.
A. Nitanda and T. Suzuki. Stochastic particle gradient descent for infinite ensembles. arXiv preprint arXiv: 171.0037,, 2017.
A. Nitanda, D. Wu, and T. Suzuki. Partiele dual averaging: Optimization of mean field neural net- work with global convergence rate analysis. Advances in Neural Information Processing Systems, 1e119608-19621., 2021.
A. Nitanda, D. Wu, and Suzuki. Convex analysis of the mean field langevin dynamics. G. Camps-Vall Valls, F. J. R. Ruiz, and I. Valera, editors, Proceedings of The 25th International Con- ference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 9741-9757. PMLR, 28 30 Mar 2022.
In
K. Oko, T. Suzuki, A. Nitanda, and D. Wu. Particle stochastic dual coordinate ascent: Exponen- tial convergent algorithm for mean field neural network optimization. In International Confer- ence on Learning Representations, 2022. URL https: /atpss////ppeeveeeeee net forum?irmioo POQp7AJwz3.
Otto and C. Villani. Generalization of an inequality by talagrand and links with the logarithmic sobolev inequality. Journal of Functional Analysis, 173(2):361-40) 2000.
H. T. Pham and P.-M. Nguyen. Limiting fluctuation and trajectorial stability of multilayer neural networks with mean field training. Advances in Neural Information Processing Systems, 34: 4843-4855, 2021.
Ren and F-Y. Wang. Bismut formula for lions derivative of distribution dependent sdes and applications. Journal of Differential Equations, 2775 2019.
Ren and F-YY. Wang. Exponential convergence in entropy and Wasserstein Mckean-Viaan sdes. Nonlinear Analysis, 206: 12259, 2021.
G. M. Rotskoff and E. Vanden-Fijjnden. Trainability and accuracy of neural networks: An interacting particle system approach. arXiv preprint 10X0v:1805.00915, 2018.
G. M. Rotskoff, S. Jelassi, J. Bruma, and E. Vanden-Eijnden Global convergence of neuron birth- death dynamics. In Proceedings of International Conference on Machine Learning 36, pages 9689-9698, 2019.
12

---

Published as a conference paper at ICLR 2023
S. Salem. A gradient flow approach of uniform in time propagation of chaos for particles in double a well confinement. arXiv preprint arXiv:1810.08944, 2018.
J. Sirignano and K. Spiliopoulos. Mean field analysis of neural networks: A central limit theorem. Stochastic Processes and their Applications, 130(3): (820)):820-1852 2020.
A.-S. Sznitman. Topics in propagation of chaos. Ecole d'été de probabilités de Saint-Flour XIX-1989 pages 165-251. Springer, 1991.
A. Tse. Higher order regularity of nonlinear fokker pdes with respect to the measure com- ponent. Journal de Mathématiques Pures et Appliquées, 150: 150:134-180, 2021.
C. Villani. Oplimal transport: old and new, volume 338. Springer, 2009.
F. Wang. Functional inequalities, Markov semigroups and spectral theory. The Science Series of the Contemporary Elite Youth. Elsevier, 2005.
E-Y. Wang. Integration by parts formula and shift harnack inequality for stochastic equations. The Annals of Probability, 42(3):994-1019 2014.
F.-Y. Wang. Distribution dependent sdes for landau type equations. Stochastic Processes and their Applications, 128(():595-6)) -211,99556661111 2018.
C. Wei, J. D. Lee, Q. Liu, and T. Ma. Regularization matters: Generalization and optimization of neural nets vs their induced kernel. In Advances in Neural Information Processing Systems, pages 9712-9724, 2019.
G. Yang and E. J. Hu. arXiv:2011.14552 2020.
Feature learning in infinite-width neural networks. arXiv preprint
13

---

Published as a conference paper at ICLR 2023
TABLE OF CONTENTS
Introduction
1.1 Our contributions
2
Preliminaries
3
Mean-field gradient Langevin dynamics
4
Main Assumptions and Logarithmic Sobolev Inequality
5
Convergence guarantee for finite-width neural networks
5.1
Bounding the ergodicity term (I)
5.2
Bounding the propagation of chaos term (II)
5.3
Putting things together
6
Numerical experiments
7
Conclusion
Additional Related Works
15
A.1 Optimization of mean-field neural networks
15
A.2
Interacting particle systems and propagation of chaos
15
B Basic properties of the solution
15
B.1 Boundedness and uniqueness the solution
15
B.2 Geometric ergodicity
18
Super logarithmic Sobolev inequality
19
C Computation of Error Terms
24
C.1 Dynamics of the derivative terms
24
C.2 First order differentiation
25
C,3 Second order differentiation
28
C.4 General convergence guarantee
30
C.5 Proof of Lemma
33
C.6 Combining all bounds together
34
Auxiliary lemmas
36
14

---

Published as a conference paper at ICLR 2023
Appendix
A ADDITIONAL RELATED WORKS
A.1 OPTIMIZATION OF MEAN-FIELD NEURAL NETWORKS
Mean-field analysis of two-layer neural networks describes the optimization dynamics partial differential equation (PDE) of the parameter distribution, from which convergence to the global op- timal solution be shown (Nitanda and Suzuki, 2017; Chizat and Bach, 2018; Mei et al., 2018; Rotskoff and Vanden-Eijnden, 2018). However, quantitative convergence results usually requires additional assumptions on the learning problem (Chizat, 2019; Akiyama and Suzuki, 2021; Chen et al., 2022b), or modification of the dynamics (Rotskoff et al., 2019; Wei et al., 2019). For the entropy-regularize
objective (5), efficient optimization algorithms have been proposed in Nitanda et al. (2021); Oko et al. (2022); Nishikawa et al.; noticeably, the quantitative convergence rate guar- antees for these particle-based methods remain valid in both finite-width and discrete-time settings.
If we restrict ourselves the standard gradient descent-based based methods, then fluctuation around the mean-field limit has been studied in Rotskoff and Vanden- Eijnenn (2018); Sirignano and Spiliopou- los (2020); Pham and Nguyen (2021). !). Closely related to our work are the quantitative propagation of chaos results for two-layer neural network from De Bortoli et al. (2020); Chen et al. (2020c). In particular, De Bortoli et al. (2020) studied the impact of learning rate in the stochastic gradient descent update, but did not provide a convergence rate or uniform control of the discretization error time (due to the lack of regularization). Chen al. (2020c) showed that the long-time fluctua- tion induced by finite width can be controlled assuming that the mean-field dynamics converges at specific rate, but such condition very challenging establish in their setting. In contrast, our result provides a uniform-in inettee---imeeee bound on the finite-width discretization error under conditions that can be verified for regularized risk minimization problems.
A.2 INTERACTING PARTICLE SYSTEMS AND PROPAGATION OF CHAOS
Propagation of chaos has been analyzed mainly in the context of McKean- Vlasov equations whose drift term has the form of b(x, /)) = (v f W vv)d(v))l. Generally, neural network optimization not included in this class, but techniques to analyze such equations can be applied to the neural network setting. Many existing works analyze the discretization error in a bounded time horizon (see Lacker (2021) and references therein). That is, for a fixed time horizon T, has been shown that su,te(o.r7] (p,") 1 (px)) S Cr/N. However, the constant Cr depends on T and the dependency is often exponential.
To obtain a uniform in time control, there are roughly two approaches: (i) the uniform log-Sobolev (or Poincaré) inequality approach, and (ii) the local Taylor expansion approach. The first approach (i) directly derives the LSI constant of the N N-particle dynamics (X = (x;) and show that the constant can be bounded from below uniformly over all N. For example, Guillin et al. (2022) (see also references therein) established a uniform LSI constant under a weak interaction assumption. Ren and Wang (2021); Delgadino et al. (2021) showed geometric ergodicity based on similar evaluation, Salem (2018) considered a uniform WJ-inequality instead of the log-Sobolev inequality also based on weak interaction conditions. The second approach (ii) is what we employed; in particular, we follow the framework developed in Arnaudon and Del Moral (2020); Delarue and Tse (2021). In addition, Durmus et al. (2020) devised different technique using a sophisticated coupling argument. We note that these prior results all assume weak interaction between particles establish celform-ii-timme evaluation, and thus cannot be applied to the neural network setting.
B BASIC PROPERTIES OF THE SOLUTION
B. BOUNDEDNESS AND UNIQUENESS OF THE SOLUTION
Proposition 3 (Theorem 2.1 and Corollary 4.3 of Wang (2018), adapted). Suppose that there exist K1, K2, K3 E C(O, oo); (0, oo)) such that
15

---

Published as a conference paper at ICLR 2023
2((b(x, μu) b(y, v). x- v) S (itt)|1=- -11- K2()) -)] yll any t2 and u.v. E P2.
2, a,
R
1010 & (1(t)(1+ + p(ll [19) for any E P2 and 2 0.
Then, the mean-filed Langevin dynamics (3) has a unique strong solution.
Moreover, for two different solutions x(") and x(2) with different initial distributions Pu Po
(1)
(2)
(k) Law(Xi ( (k 1,2) equivalent for t 0 and satisfy the
P2. the corresponding laws ji following contraction property:
Wa(ue (1) (2))² S (1)WW)(( .40 10))), KL(P)) 1 (2(())2(11)) )2
(9)
for 1. > O, where wh (0, oo) (0) (j = 1,2) depends only on K1 K2. K3, X and is increasing function.
Note that it is possible that lim,- 2/2(t) co. Indeed, (t) is given (t) tt2(t) exx((t(())))) + )e(o)))
2 (t) 2A
As consequence of this proposition, obtain the following corollary.
Corollary 3. Under Assumption 1, the mean-fled Langevin dynamics (3) has a unique strong solu- tion. Moreover, the two distributions ! and fil") corresponding to different initial distributions in P2 are equivalent and satisfy the contraction property (9).
In particular, μr is equivalent to μ² and hence is equivalent to the Lebesgue measure. Therefore, Hs has a density that is positive for all x E RR, and if nu E Pa. then the density of b4 satisfies dut (x) E C ((0, 0o) x Idd RR). (t, x)) + de
Proof. Let H(,,M) =
(zf))yy()) (((((()) We just need to check the two conditions in Proposi- tion 3. The first condition can be checked as follows: by noticing the convexity of r, we have
-2(b(z, b(y, v), v)
= --2(VH(2, (a) - VH 1), x- - 3) (Vr(() Vr(y), - y) =-2(VH (z. p) VH 14, v) - 2(V 14) (H),,)),I
- -VAi(Vr(z) - Vr(y), -y)
< <BB 1kc vll + 2B S2 polllll 1 wll (:: Assumption and convexity of r)
< 22B vlI² + W2H, - y|l-
which yields the first condition. Next, the second condition be guaranteed ||(00, μ1]l < (0, μill + A vr(O]l < B² + ACr. Therefore, applying Proposition 3, we obtain the first assertion.
As for the second assertion, we first note that n² is an invariant measure of the mean-field Langevin dynamics. Hence, if A4 14, then (4 n² for any t > 0. Moreover, recall that P,- je by Proposition 1. Combining these relations, we have μ Pn- b4 > 0) when the initial distribution satisfies 140 = n²" Therefore, Mr with general initial distribution is equivalent to μ² by Proposition 3. Finally, (f) + due Jordan et al. (1998).
de (x) E C' ((0, (0) x Rd R) follows from Theorem 5.1 of
In the subsequent analysis, it is important ensure the boundedness of the moments of X,. Indeed, we have the following estimate.
Lemma 3. Under Assumption for any p 2 2, [||| []] < oo implies
E
sup IE(O,T]
|X,||
< <
for any T > 0.
16

---

Published as a conference paper at ICLR 2023
This is direct consequence of Theorem 2.1 of Wang (2018). Therefore, have that pe E Pp as long as /4 E Pp- Indeed, we are interested in situation where 8o0 = S # Di_1 X and hence 2 because a discrete measure has finite moment for any 2.
Ae Pp for any
If p = 2 or p = 2 + 6, we have a sharper uniform bound as follows.
Lemma 4. Under Assumption I, we have the following uniform boundedness of the moments:
sup [[I|X, < max t>0
1
+ic(1+8/22 + λicō + 2Ad IB[1 [B]l|xo|1?]
B²
XIe(I 6/2)
sup E[r( S max t>0
Blr((oo);
C.(2+8) (1(((
(B²
(1+ 11+22)110r- + AC, +
2 (AC,)
6/2
2 (21c/2)145
2+8
[c,(1+00]
The same bounds also hold with respect to B[|| and B|r(X;))
Proof. Let H((x) 1. Eブ= (hy (z). By the formula of the infinitesimal generator, we have
= -X 6(Xe, pc] = 2Ad.
di
By Young's inequality, the right hand side can be bounded as 2E [x_ (VH,(X,) A vr(X,))) + 2Ad
23 x|| c X, 2+ + 2Ad
4Ba
2A10(1 + 6/2) X, 131
+ 2A
(1+ 1011111111
2Ad
4A1c(1 + 5/2)
2
B²
+ A10ō + 2\d A1d(1 + x
+1c(1+8/22)
Hence, we obtain that
B*
E [|III Alo(1 + 8/2)
+ A10ō + 2Xd v B|l|xoo|
Aic(1 5/2)
In the same vein, we can show the bound for r(X;) follows. First note that
d
dt [ = - (X.))6(X,, u:)] AB[T-(VVTr(X
By Young's inequality, the right hand side be bounded
- E -E(,"x(X,) (VH,)))))) A, + AC, Bill X, X (1])
< |1
I1))
- +C, X (11)
(B²C,) 2+6
>1c +
2(1 8)
5
$ 2+ 8 (Aic/2) ( 146
Blll X, 2+0 - A1cr
E
X,
2+0
+
2+0
+
5
2
8
6/2
\1c +
+CC +
(AC,)
2+6
2+0
xc,(1+0)
2(2 - 6)
(B²C,)2+6
oNier + +XC
2+8 2 (AC,)
(2+8)/2 (A+,)
8
6/2
2 8 (>1c//2)144
148
de (1 + +8)
(1G + +8)
1|
2
(B.((x))/C.-1
This gives the second bound.
The argument be applied X3 which concludes the assertion.
17

---

Published as a conference paper at ICLR 2023
Lemma 5. Under Assumption for Muu Pp with it holds that
[ (I .d..)/.............. =
V (lo((())))) = tv22
E
and, if 2 1, it also holds that
B log(u) <0,
for any t > O.
Proof. First, note that (Au E Pp ensures differentiability μ4g The characterization of V log(()) is given by the integration by parts formula investigated by Föllmer (1986) (see also Lemma 6.2 of Hu et al. (2019), Wang (2014) and Theorem 5.1 of Wang (2018)).
As for the moment bound, first note that
|bb( +01+|1 = 0(1 + [|x]|
by the assumption. Then, by Jensen's inequality and the moment inequality of stochastic integral (Kim, 2013), we have that, for q = p/8.
E[| V Vlog((+))11
Cr.g
(1+ a
2X
1/ (1+ s(1
Cr.q
1x,,1
< <0 (: Lemma 3),
tv2x
[gg=1)
9/2 مو
where Cz
According Lemma 3 and the remark following the lemma, have Me E Pp for any p 2 in our situation where No is discrete measure like μ4 ua" Hence, we may assume B|llV ( < oo for any p 2 2. In particular, the Fisher divergence I(Helle") well- defined for any > 0 (but not defined for t 0).
B.2 GEOMETRIC ERGODICITY
For p. v E P where v is absolutely continuous with respect to μ and thus can be written as de fdp, the Fisher divergence of with respect to p is defined as
T(vlln) = 4 / / Vogg()))
Proposition 4 (Geometric ergodicity of the mean-field Langevin dynamics (Nitanda et al., 2022;
Chizat, 2022)).
((4)) - ((=*)== S (opp-22X))(C(uo)) L(1²)). XKI(U)la") C(u) L(x*) AKT(H(lpp)
Although Nitanda et al. (2022); Chizat (2022) assumed r(z) @(|x= we can adapt the same argument also to situation. Indeed, the quadraticity of the regularization term is used to ensure the well-posedness the solution, and in our setting this is ensured by Corollary 3, which yields the assertion of the proposition.
Combining Corollary and Propositions 4, we obtain the following corollary.
Corollary 4. Under Assumption 1, for any initial condition μlu E P with Wz(Ho p²) < co, it holds that L(4) < oo, and
C(px) L(p²) <
(b) p²)
KL(H,, 11*) < exp(-2axl6
for any t. To where To > 0 can be arbitrary.
18

---

Published as a conference paper at ICLR 2023
Proof. We know that μ² = Py- M4 (t 0) when the initial distribution is fff μ from the proof of Corollary 3. Plugging this relation into Corollary 3 and Propositions gives the assertion.
As remarked in Section in the convergence analysis we need assume an LSI (or Poincaré inequality) on instead of fı.. This is not generally ensured. However, we can verify this condition the samip satisfies the super log- tog-Sobolev inequality. Indeed, the super log-Sobolev inequality entails the iitrraconntractving yielding an _ -convergence of the density ratio between Ar and fi- This is remarkably useful to transfer the L.II property of jút to jtt-
B.3 SUPER LOGARITHMIC SOBOLEV INEQUALITY
Definition 3 (super log-Sobolev inequality (super LSI)). We say that probability measure satisfies super log-Sobolev inequality if there exits a monotonically non-increasing function 3 (0, +B E such that
1lg2 log
f 13) ST /VI Vsdu V + B(r)
(Vr > O, AS D(E), (1 =1).
Proposition 5. Ifa probability measure is given by exp(-V) where ((x)= >1r(2) (x) with a convex function r : Rd + IR satisfying A Vr(x) x 2 c||x||2+é for 8 0 and h R2d + R satisfying
V S C < 0o, then p satisfies the super tog-Soboler inequality with B(r)
C" 44:26 10g(r/2) where C' > 0 is constant depending on d, C, In purticular, it satisfies the Nog-Sobolev inequality with a constant a' >> 0 such that B(2/a²) = 0.
Proof. This can be proven by adapting Corollary 5.7.5 of Wang (2005). Let P. be the semigroup that corresponds to the generator L Φ = Aø - VV- - Vg. Then, we have that
L ||| - (A1 Vr + VH)- (2x)
< d - |0||21|| 2+ + 2 V H ||||||
2+8 5
[VH
20||x||2+8 + e|| 2
(: Young's 's inequality)
c
+ C²
2||
Corollary 5.7.5 of Wang (2005) implies that ||Pe|l<>0))= (w)
exppc'tt(222)))))
for constant c' > 0 depending d, c, C. Indeed, (5.7.9) of Wang (2005) holds for c + d c and 7(7) + 1 +6/2 in their notations, which yields the bound.
Then, Theorem 5.1.7 of Wang (2005) states that the super log-Sobolev incquality holds for ((r) 2 log //P//all= (1) < 2 log(c') 2 2 log(r/2). By resetting C" +- log(c'), we obtain the assertion.
Due to our assumption the regularization term in Assumption 1, we know that the proximal Gibbs measure satisfies the super LSI condition.
Corollary 5. ju satisfies the super log-Sobolev inequality with 3(r) C² addition, it satisfies the log-Sobolev inequality with the LSI-constant ā = exp(
4+26
log(r/2). In
AC
)
4426
Let Ps.t (s <) be the semigroup associated with Xx, i.c., (Poef((x) = E[f(X))X, x] and H.Po = pef.
Theorem 3. There exists to E (0, such that
[
3 D2
PPo.oott wot.tallll2aaaa -L= (p.) S exp
dp + Cto
for some constant C In particular, there exists Co > 0 such that || ( S
Co oo uniformly over > 0.
19

---

Published as a conference paper at ICLR 2023
Proof.
We first note that
(A(D- (4(p- 1)r
(o( S - +
p (x( +
(10)
for all f E ((L such that / 2 0 (which we denote as f D+), < < 0c, (by definition invariant measure corresponding to Li, is Pt, i.e., Lu 0).
Let to = 12 dp I for 7(0)
1
We define p(7) and (T) functions on [0, () such that
2p'
p(())
((( 70 p(p())
p(0) = 2,
N'(r)= = (T)
4(plt)-1)sp(z) p(T)
N())) = 0.
p(T)2
Then, for f E D+. if we rewrite top one has
dr || /3-7,5/|(LMt)>)p
1 (Ps (Ps (Ps-r.s))pin MP(p))
((()
|| | pe0psp/--
p(T))lPP-T..l {1 [1p(7)-1
p(r)
p(T)
+ p(T) P| 7,a ( /
((p(()
PP- d dr Uos.
(II)
The last term of the right hand side can be bounded as
/
p(T) P. P f
p(r)
dr ds---da
(a.r) / |Po-nsfp())
< p((T) ||Po-t.of -ll
(-p(())
En (g)))))) (L hi) Po-rdo
IC 1
p(T) |Po-- ||
[ /
(P-r.af((r) Po- C
(12)
p(T)
By the backward Kolmogorov equation, and combining these inequalities (11) and (12) with the super log-Sobolev inequality (10), we have
d (c- (o- I.. (P..-aflllon dr
p'(7)e-N(()
(PP- log
Po- Ps-r,af)
Po. (1 (a-))
p(T)
+ p'(T)
Mo- ((Po-T. p() Ly,,, Po-r,ff)
(T)(T((
1o00 fp
log | P. (
(((T)
+ + ((T)
f (1111117 (, ,) / |Pansslp(() d P
1
P
p(T)
C
N(r)
p(T)
Hence, we obtain that
P (Fn-r) S (3 (1fliaa))))))) s (
Here, notice that
p(s-to) 7(p)
/ to p'(T))(((T)) p(T)
dr= = / 1 po
dp
dr
ldr = to-
Jo
20

---

Published as a conference paper at ICLR 2023
On the other hand, since also have to = 2(0) lim- No p(r) = oo and hence
dp by its definition, it must be the case that p
/
B
(1p-1)7(p)
3 (27(p))
lim N(T)
P
dp
dp.
7/to
p
2 p²
This yields the first assertion.
By Corollary 5 we can see that 1 8(27(p))
dp
J2 poo 8(1P))
dp 2
J2 + Co < co. This yields the second assertion.
Let p(t, x, v) be the density function of the distribution X₁ conditioned by Xo x supp(uo)
with respect to tr, i.e. p(t, dF (6x) (y). Then, we have that
di.
|/oll (oll /coll+ lo, x, ()) - 1)ʃ (v) (dy) s ioo.o Pro - Potto /x)++o)())))
ECol| -
(13)
where used the bound || [.oo (b4a)l12(00)--1oe I Co and the fact that the support of jo is the whole space IRd. We will show that the right hand side converges to 0 in an exponential order by taking its differentiation with respect to
P. - (z. = =2 f (P - (.s)(Ps, D.f)dd. + [(Pars ((Pa.S-uuf7) -i.
We evaluate each term as follows.
(i) The second term in the right hand side can be evaluated as
1 (Pa.f 0.f)
|Leas- -uf) ( n
5y (lsn),s))ssstttt) is
$ c [Prses - -ius) Σ (hy))
3=1
s sc / (Past - (af)?iuyda Σ
/
Vh (V log(Hs) - V (og((s))dd4
s - - R.f (M l(a.)
(14)
(ii) Next, we evaluate the first term. By the Poincaré inequality (PI), have that
| - Ddji..
/ (Pstssass p..
= / >IIVPE.f
2 Pa.e + (Ps. = Ps.e - TMJ + juJ - P. (ssPss
Aall|Paef /r. 12: (P..f [ - (P.(P..))) + (Pel - (P..))))))))
Nall|Pael - - / 2( - 11. + (Hr. - -r.(P,,()))
Pall|Pass jar /..111 (i.(Pa,.f) [1,
Pa.. - -;ef122a(a.) Xa()o(P...f) -i.ff)?
21

---

Published as a conference paper at ICLR 2023
Denote the total variation norm of two probability measures j and by || 2///r,, notice that, if 8 2 to. it holds that
lū, (Pe..) [u,
[ s.t- Ub Pt- 1o (Pt- - (it Pi Pi
11 to IVv Pt Po + 1iit Pt tott
< 1| - th (0) If (3(()) lite to-
$ Coll - || |I (2(()) + 1i.f Pe- (P-
(15)
and, ift to. the same bound without the first term in the right hand side holds. By Pinsker's inequality || 2/|TV= I VKI(ull>) /2, we have that
|/ 1|V I (KL(ūallu*")
V2 / x- (oF(m), (.)
1
8F(us) ())
djt, + togg(((n"///(,))
8p
8j
sup
BF("")
(x)
6F(mm)
(I)
[1v, - Jw 10.0 S S² exp((=ale [C"esp(-chs)WW(lnnn
(16)
where C² > 0 is a constant depending on To, a, A. Moreover, if we let ir = ps then
/ log(Ar/ 00,
0, O,KL(Rrllk")
(A bu V) log(A= dz
/ (- A- (-AVlog log((>) 6, v log(p=//1 dr
- / (Vloo((+)) - Vlog(u" 1|
À / (Vlog(a") + x -,) (V log(A>) V log(u" )dda
- |Vlog(A+) log(n /d
A / (Vlog(m)) - v log(A>+)) T (V log(Ar) - V ))dff
À / |/ log((,))) v log(u dp + / [Vlog(ūr) log(e" lIg(
2
S -aKTT(ū-lle")) sup V log(ūr) V (z)()")<>))1²
< -XaKL(A(lle)) C exp( ((AAaa)))i W2 2²)
where we used the same argument as Eq. (16) in the first incquality. This evaluation, together with the bound of (L [|x²)) in Eq. (16), implies that
KI(A-lle") < C' max (exppppx((1p)( (p,l""))) exxp((22xxr)
C' exp(-a>\( 5) + W2
Therefore,
P - lllv S P l|v + I1 fi
S exp(-aaii - to s)/2 W2) ("),
Next, we evaluate the second term of the right hand side of Eq. (15), 1 pi- (Pt) We notice that
0, (aadn/ddr//f(Is---- Vi +
22

---

Published as a conference paper at ICLR 2023
(-AVlog((a) - b.) (T) ((Pas))ddr + (Pael)aiidd
I (Pacf)0oiidr
sc f S |Panfliddr ((ulll & (l. v V (Mell(a))
where the first inequality is obtained by the same argument as (14) and the last inequality is by Theorem and the fact that the density ratio between Pr, and ä.r are bounded from above and below for any T, r' because of the boundedness of 6F Hence, as Eq. (22) and Eq. (23) below, we have oμ
l (Pt- -f+fl
J. (i.aal/i)d
C to
s C -xpeep(-aa(( - to (1)) v / p "I))
Therefore, by applying these bounds the right hand side of Eq. (15), we arrive at (Ps(Pss)) f² s (exp(-Aac( - 21 + W2(Ho: 1 "aaio,""")))") 1)111
(iii) Finally, by combining the bounds (i) and (ii), we obtain that
0,1 Psef - (l.f((22pp)) 2 PA jr ].ffl1?p(p)) C.a exp( + s))(1 + W2 1I (z)))
which yields that, by taking the differentiation with respect in the reverse direction,
P Pel11 .)
$ -2da - (x-a---------- ( + C'Aa + s))(1 11
and thus, if we write C. = C S/ (Pssds
| - 11+22
S eexp(-2dar + Crotroll Prttortttoo T / 11122(
+ +daC ( pttoo
exp(->ac s)) + W2 : ,²)) 1) (22 -2aa(t 22a((t-))+Ciro--- -C. -C
fl expt(Ctta)l) [1 + C(1 We(Ho, # ))
As in Eq, (23) below, the right hand side can be bounded by
C exp((2Xaa)]l/11 (1 + W2(Ho, / exp((a(1 + W2(Po, 1²)))
for constants C₁ and Cg. This is further bounded
C3 exp(-22art exp(Ca(1 + W2(Ho. H ((((Ho.H")))ll ())
with constants C3 and C4. Therefore, arrive at P Petto (a(()))) S exp(-aXt) ( + W2(po, μ "))) (LP
where we used that the density ratio between ü., and j., are bounded from above and below for any T' because of the boundedness of åf and C/2 = C1/2. by
Therefore, by applying this bound to the right hand side (13), we have that
sup |p(1 + to x, y) 3.9
S C' exp( exp(-aXt)Qo..
(17)
23

---

Published as a conference paper at ICLR 2023
for constant C' and cxp(C ,²)), which can be checked by noticing the density
of bu is smooth and taking f(z) = 1f - x'll I E} for arbitrary x' E Id and letting + 0. This
indicates that the density function of Me with respect to f4 satisfies
S C" exp(- - -0))
Importantly, this observation indicates that μr satisfies the (a/2)-LSI for sufficiently large such as 1/v2] via the Holley-Stroon bounded perturbation
C" exp((=aX( (0) )Q < min| 2 principle (e.g., Proposition 5.1.6 of Bakry al. (2014)).
Corollary 6. Under Assumption 1, there exits To depending on d, B, a, X, Gr. Cr such that pa satisfies the (a/2)-LSS condition for 2 To + log( (og(Qo))az ax) Moreover,
du
du
S
(x)
22,
C C'eep(-aX(l- Too)).
V2
djt
djt
for all 1. 2 To + log @n)/ad, where C' is a constant.
By setting Q = log log(Qu)/ad, we obtain Corollary in the main text.
C COMPUTATION OF ERROR TERMS
C.I DYNAMICS OF THE DERIVATIVE TERMS
Recall that d' and d are linear operators defined by
(t.Po. (. ((s)) A (m(t:))
d (t: Po. T1. (6) a (1a) Anf (m(t: (21,
for smooth test function Φ: Rd E. where m(t, (0) f o(x (())dm(t, ())).
Given a linear operator g, we introduce differential operator Ln as follows,
L,9 xāq V (b()))))) V (1 (- ())))
which is defined in a weak sense, i.e., (0) = ( -()))) b(- Vo- (.))
Vol))u(dy)) for test function Rd IR with appropriate regularity condition. Following Delarue and Tse (2021), we know that these operators obey the following dynamics.
Proposition 6. d(t)) and d. follows the following differential equation: for t >> 0,
(t: (t; = =m (t; d (0; = =(D2),,
where (D',), is defined by (D)),(0) = O2, 0(x), and
0, (t: 21, 22, )- Lm((e,) d3 (1;
+V
(t; 22) 11
db
(t;m,))))
+V
( (t:H),
21) 56
m (t.,,,)) (t: (1), (1:
+V.
((t, p))
op
8h
m(
d (0; p1 x1, x22) =
The above equations should be interpreted in weak sense,i.e. when O.q: Lm((ta)) - Ps means that qe (e((tt.)) qá(á(s, )) = S 4- (,, ))dr + 4r (+ (I ((T ))dr + Jr ( ))dr for a smooth test function @ [0. oo) Rd R.
The well- coll-posedness this differential equation justified in Delarue and Tse (2021); Tse (2021) for mean-field dynamics the d-dimensional torus. Although dynamics is defined IR and the regularization term has unbounded gradient, the arguments there can be applied because is convex and does not depend on the distribution. Here omit the technical details.
24

---

Published as a conference paper at ICLR 2023
C.2 FIRST ORDER DIFFERENTIATION
Let H.(x) - + &f (xx). Here, we evaluate the first order derivative di (1; x). For that purpose, we define an operator d (ti,, C,) where & Rd E Rd p P (we omit the argument z if there is no confusion)
a (tHHE, = mm,,, edg - (11=)))(0)lE0
de
= v 84
mll, (x).
One may check that (t; (t: p, e2 where is the indicator vector that has 1 in its i-th coordinate and 0 in other coordinates, In our case, we are interested in a setting
N
N40
N
which is a discrete measure with support consisting of N points. that case, dl Mo: E, x) can be reformulated as the following gradient flow
Po ((x))) 81
6 m(t, (o)))
N Σ
m(t, (o)
where E(Xo) = Ng when Xo = and E(Xo) otherwise. Clearly, & E 1,2 Accordingly, we define the following SDE:
dx; = - -ix; + 2AdW,,
" Law (X)), Xi Xo e((X)))
Then, if define of lim xi X. it holds that
dd (t) = Blos vo(X.)).
We can see that the infinitesimal displacement of follows the following differential equation:
duf = 05 1o)=xx + 84 (X,,
dt,
(18)
from which we can derive moment bounds for og. In particular, for p 2
d 1|
8 of b(X ōy
1
of Vb(x,
28
p dt
8 b(xx,)))()))
-1115111-2
(155 v
X(05
VV
r
r(X,)of
of
8y
- 1P B II].
where we used Assumption and convexity of r in the last inequality.
First, by taking expectation of both sides for p
we know that E |||| |||l| yields that
||| ||| _ ()) exp(2 [[] Then, for p > 1, noticing that of = 0 for Xo +
and || (1 1 L o5 |||. we have that, for x' *
1|| | = =
(19)
I <x 1) B²t B²t) P| = I'1 /'
- B) t s)]ds
If exp(2pB2s) exp((2p - 1)B² [1] 1p
25

---

Published as a conference paper at ICLR 2023
I c2p (cxxx(() 1|B "1|l
< B exp ex) ([ex !]
s (exx((2pB*) 1]|
(20)
In the same vein, when Xo = x, we have that
|o O(exp(2pD*t))
(21)
for 2 1 and > 0.
By Corollary 3, mit,ed; - (1 e)uo) has smooth density for t > O, which we denote by n/? Corollary also asserts that / (x)) > and is equivalent to je for any O. Let
(1) Gr (z')
1
Dc Hi (e)
(x')
(1(x')
For concise presentation, we introduce the abbreviated notation
()) = Ey, 1n.1 o).
(
Let Pij be the distribution of X; conditioned by X6 = x' + =((x'). We define the condi-
de Mi (x")) (-=- Accordingly, we define (2) Le (6) == Bun (o 0). Then, we can see that ft,0 (1) Nv4 ((1x3)))
(1) tional version of ft,ar 20,2/e (t,zle((x") (x") Pe ( C V (e)
(1)
Lemma 6 (Bismut formula), Suppose Assumption holds. Then for a bounded measurable function f: Rdd R. μo E P2 and O, m(t (I + (f) is differentiable with respect at 0, and we have
d m(t (1 =0000 -
-01= =E [5(x)) S
c5 .dW,
de
where
5 (V2A)=1
T5.
5b بية
(X,, Mo (g( (2)
t
Also, 2e (x)))²)) = EBJ[ 55 dw, = x'), Ve (x")= = Eiso' c5 dW,[X, x",No = 2'1, and
B Bi (x.)) =] K(INN?
(1)
and
x') (()) for x' + where K(t) is constant depending t.
Proof. The first assertion is obtained by the Bismut formula with respect to the Lions derivative by the initial distribution No (Ren and Wang. 2019). In particular, Theorem 2.1 of Ren and Wang (2019) yields the assertion by setting = s/t / in their notation. We note that although they assumed b(x, p)) and their derivatives bounded, the same argument can be directly applied our setting because r(x) is a convex function that forces X, to contract to origin instead of diverging.
As for the second assertion, I f(x),, log (x))d4 for all f (note that (x(x)) > O for all x E Rdd by Corollary 3). This indi- cates that O (og(p))(z) E cf aW.|X, xr] almost surely. In the same vein, also have the characterization of the conditional version (= Indeed, may consider dynamics of X, (X,.X) and apply the same argument on μ4 to the distribution of X,
first observe that d d
f(x d.p.la)
ke(x) die
(1)
The moment bound can be ensured by noticing that
( Big 1 (X.) = x = E E L <5 dW, Xo,
2 = Xo = x
< EE (1' & .dw.)) IXo
26

---

Published as a conference paper at ICLR 2023
S C12 / B1115119140 - x:ds
C2
S (1 (1+ 3) z|ds
x 12
x 12 (1 (1+ (1 =0 O [1,2(1 + t) N2 +t)
Ct.2
1
where the first inequality is due to Jensen's inequality, the second inequality is by the moment inequality of stochastic integral (Kim, 2013) (Ct.g 2003/ ). and the last inequality is due Eq. (21) and Assumption 1. When Xo x, Eq. (20) gives |1] | x'1 ((exp(2pB2tt), which gives the bound for Xo + x. Lo AāØ by V
Let
for o: Rd + R. Then, the derivative of qi.z with respect to can be evaluated as
d
Ep Ep 11
0 Ø
[[
dt
Ot dc
0 0
[[f(X;)]
de Ōt
B((AS - b, V))(()))
dc
67 = f a -, v)f
8b. (1)
(q fdu
We can also see that
d E B dt
10 fl
( -A
V)s
8b. (1)
V fdud
8u
Note that the second term in the right hand side is 6bg (1) instead of bb, oh
((t,z)=)) We refer readers
to Tse (2021) for higher order regularity of the nonlinear PDEs induced by the derivative with
respect to the initial distribution in the torus setting. Therefore, by applying Theorem 4, we obtain the following convergence bound.
Lemma 7. Under Assumption 1, it holds that, for any > 7 with sufficiently small T > 0,
(1) (1) (X.)) (ÂA,, exp(- (X))) Bi
= OA no exp(- -taatt (o)i--To)222211) 22)
where APD (
Proof. We apply Theorem 4 in Appendix C.4. We first note that the conditions of Theorem 4 hold for & = C' C't (2 2²)) by Corollary 4 and 6, = 0. Moreover, by Corollary 6, we can assume a, a/2 for t 2 To + log(Q:)/(aa) and a = 0 otherwise. Next, we check the integrability of TT ( with respect to I. According to Nitanda et al. (2022); Chizat (2022),
dt (C(m) -((=))()))))00)))))) -121(4 O.
which implies that
[r IHHollaedds < 1 (C(He) -(p")) - -((+)) ((₂*)))
Hence,
S( [THulldssdss s / (1-11) (C(pe) (((m)) - C(()))
12
27

---

Published as a conference paper at ICLR 2023
so (x TO)) ( - -(((m)
(22)
By taking + + To and ='= To for 0,1,... and taking summation over k. we obtain
(usll,,,dds x o (V((m))) ( -(xx))
0(1 v (22(T) x²)) 0,(1), (23)
where omitted the dependence on X, in the order symbol. Therefore, Theorem 4 yields that (x)))) O exp(- - [l)33/4
for sufficiently small O, where Ant exp(O((2(voo,'')) Combining this evaluation and Lemma 6 completes the proof.
However, observe that the bound in Lemma 7 has O(N) factor in the right hand side. We can
remove that factor by considering B[ (X.))² instead of Elue,2 (X.) Recall that 140 = 1
* E Xy. Here we define Pol, == N-I Ei=li X
Lemma 8. Under Assumption 1, it holds that, for any > with sufficientll small O,
Bl (x,)11² = OAAp exp(->a(t To)/2)1le13).
where Apo ( ,*) W2(Mor., 1)).
The proof which can be found in Appendix C.5.
We finally remark that combining Eq. (21) and Lemma 8, we know that for any & C&(RR) and t 0, it holds that
dt E.) & O (Ano exp(-Aa(t To)/))]((ll
(24)
where ||oll,o,1 = max
C.3 SECOND ORDER DIFFERENTIATION
Now we evaluate the second order derivatives. Let æ x2 Rd fixed. For e (21, with en 2 and _ gl2! E Rdd, we note that
82 U(t (2))
V
22
&
(m cidm, + +20ds - (1---- 61 - =2)) (=000
dez
2IT
(m((, 620ng + - (d + +1
<2) ((1)))
22
Dez
5m
82
t (m po) ())))) (t: No. 111),, d()) (1: no: 5 5/21))
5m
8 @( (0)
+ 5m
where
( (t: Ho, Moo..l. 111 -|21 (2)
02
= = T
de Jc2 m (l, E10 t2g02z
(
By Corollary 3, mtt, θz + 62θ,, - (1 -1 - =2))0) has a smooth density for O, which is (ei
denoted by 1 (2)) Corollary also asserts that 14 we write
(x) 0 and equivalent μ for any t > 0.
0²
1e (x) Mc(x)
[1] AV
#4
(e1.22)
(x)
de
28

---

Published as a conference paper at ICLR 2023
then we notice that d po: ,1,
Ey, (e, p). Here, we denote by
( (0) = (t: po: | xx) = V de mt, Ed0g - (1 c) e)(o)) Then, by taking the derivative of ae (t;) with respect to 62, we know ge (and ) follows the following dynamics:
dee) (t: ,00, ,1,) x2) = dt Eu (1 L(())]
11,
dt
-Ep Enl Lo(0]] /
86,
(d(2) (d(0 (tipo:ss)) (0,,,,,,,,,00 Veduu
8f
8bb (dl))
8b:
BBp 00 (d (1)
d2,11)
(d()))) vΦ)
d())
(d Váj
82 t.[1] ((1)) V
8p
1,2]
and also
(2, (-) O.
(2)
Now, let
(
and consider situation where T1 x2
(8
We write L.,4,, to indicate (1)
(1)
for
(t: no. el,, xt) when Ia = Xg. Then, we have that
d Bu l (2) L(())
dt
=Bn, lui (2) (r)))) - -f 1
8b,
( Voduu
N
N
Exp
11,111,
(1)
8bc (1)
8p
(1.12)))
VΦ
N
E
Ep
(1)
8bc
8p
(1,1)
(1)
VΦ
i=1
8
E
(1)
(1)
VΦ
822
By Eq. (24),
1
Expc
11,[1],8 5h
8b.
(2).))
(1)
VΦ
50
Apo (1) 11
N
N
8b,
E
(2,
(1)
VΦ
5
Appo (0)(1=)))2)]133)) T. [[aaalll
N
In the same vein, we also have
bb
0 Ag exp(- - Io)) /Bp. /Ep. [|]
( VΦ <
Me
8p
Then applying Theorem 4. obtain the following lemma. Lemma 9, Under Assumption 1, it holds that
El(a (2) (X.)) OApso exp(-Aa(() (2) ( where Apn exp(OO p²) "2(Ho,.,H'))
Proof. From the argument above, the assumption in Theorem guaranteed for 6, = exp(-Xt/2)) The other conditions are also satisfied as in the proof of Lemma 7. Hence, for arbitrary small positive time T > 0, we have that
Ellg (2) (xx;)) 0(1 + El (0 (X))) O(1,,
due to Theorem 4-(i) and g (2)
0.
29

---

Published as a conference paper at ICLR 2023
On the other hand, Corollary 4 yields that C(ue) L(u") S (02((T))22(10, u.). Combining this with the argument in Eq. (23), we see that
Vilee S so 50(11
Then, by the same argument as Lemma and Theorem 4-(ii,, we obtain the assertion.
C.4 GENERAL CONVERGENCE GUARANTEE
(1) We can see that = 4i,z and = qi()) satisfy
d
Eu.lu0l = B[gcLc(0)) /
8bt 8μ
(q.)
Voduu + rr())
(25)
dt
where (r)( I Ceepí -appp--aaat) V Br [oll F]. and with slight abuse of notation we write (r())
Ey, [R We define
Di(1) =-
Σ Ep 1uc D2(1) := dpe.
where - - (f..)
In addition, recall the satisfies the Poincaré inequality (PI) with constant a if for all smooth functions f Rdd + R, have
(a) S -B,
is well-known that LSI implies PI with the same constant.
The following theorem provides an upper bound on DI and D2 under the Poincaré inequality. Theorem 4. Given Assumption and suppose that d dt ",,l S Ct. qt satisfies Eq. (25) with rx(ge) satisfying Ir (r:) < 0, IIV V [2] with sequence 1/2 2 de 2 0 and C 2 0, and pe satisfies PPI for at 2 (ax = also allowed in the case that μt does not satisfy the LSI), then the following bounds hold:
(i)
D2(1) S exp ( x
C
D2(0) + 2
(
2X
(ii)
dt (Di + AD2(1)) - 2(1 - -.) (D) + AD2(())
+ B² 2BA T(4 (1) + + 2B² de
D2(1) +
C²
- 8z-
1- 5z
In particular, it holds that, for S1,
f' C2
Dz(t) + AD2(t) <
S. cxp(AA- A,,)()((T))))))) + +.)((D(())+))((()))
2
where
A, = -2(1 8,) + + (AV((Holie)) + + 8, ds,
1 8,
and Ci = 1,
Proof.
By substituting o +- 9t. holds that
d dt / -2B Erc( -
2 duc - 2
8b: (g:) Vgdaa + 2r,(g))
di ot
30

---

Published as a conference paper at ICLR 2023
Here, the first two terms in the right hand side can be evaluated as
n. l0 Le - (()
due
-2 (11 b, (g:)) .
- bb ()a))
=28 (AA b, ()()) - 2T (A(Ag)) A/V by )g
2. [
Part (i). By the assumption, the Souchy-Schwarce incquality, and the arithmetic-gcometrie mean inequality, we can see that
8bt (g))
Vqedut + 2re(
8μ
Ba
C
E
[ot]
yr, uu, [1| Viell 13] +
XE
[
B
C
Vg+
mu [4 +
X 8..
Therefore,
d
B
C²
D2(t) N
D2(t)
X 81-
dt
This yields that
Ba
C2
I 5,ds) D2(0) + C
S 8, exp
Ba +
C
[ ordr) ds
D2(1) exp
(t (t-
A
X
2
(
B+
C²
C
B
C
S exp
+ 2Λ
D2(0) +
(
C
exp
2X
2A
This gives the first inequality.
Part (ii). Next, we evaluate the time differentiation of D as
d
(t)
dt
Σ
(e 19ch,1)
dt
j=1
d Fe laihsl + Em, [gchy]
d
Σ En [9c
Ey (e (aassnn - 6y vhi)
Tb
Σ f=1
(4: Vhg
Be, D2
8p
Ey, 140,, / Vag Vhydju
22
Em, 19 f a.16 + +VVlog(ue)) Vh,du,
+ A
J=1
2 (C, (qt 113 + Bene,
(wheee (Qe)aj= := / Vhu Vhydme).
Here, we notice that
Σ Ee 1e, / ao(be + AVlog(uz)) Vhydu
31

---

Published as a conference paper at ICLR 2023
S A Σ V G // |V IV - V Vlog(m)))&dmm) / s.dueefallVi ll l 1=1
< AnB' AAB'VVTun VTuel (i ) B. AnB" V I (+r) D2(().
In addition, note that
yn lachelle'e / Vgr Vhydæ = / |va: ōu
obe
dr.
Therefore,
(1i + (D2(t))
dt
s no 11 - 4A fVu: Vil
ob. (1e ddp - 21² / /Vail/ap
8u
+ +2B 12BB\\I( (7. (1)) + BBe + (x:(
s 2
- 41 / Vat
8bc
(et d1t - 21² / Vall'dp
n2 1,
8y
+ 2B Av T(u ((T) D2(1) + D2(1 + 26,1 / /IVenl/ddye + =00
O²
2 / 11V-==
1 5b.
djp + (1- (1))))
5b,
2
(q:)
+ v 5.
(9t)
dpe
5e 5p
D2(1) C²
B² (12BAVII(Hllle
2 5₁
2 /
1 5b.
(q:) + /1 1 8,1 Vg
oz dp
6₁
C2
+ B²
2BXV (Mellpe) [1) Ez + 2B²
D2(1) + dc.
1 8,
When be satisties Px (which is implied by a, a,LLSD, it holds that
2
/
1 Σ
du $ /lle,
8b, (a)) + 02AV
04
01
Eachy + 02
dup
n i=1
for any 01, 0g > O, which gives that
d
(Di + (t)(t))
dt
2
s S-20 /
v1 de Σ
he 14:11] + v1- -6290
dic
J=1
8r
C²
+ B²
2BX ((eel
+ < +
2B²
D2(1)=
5z
1 1-
2
1
C Enc h [9hh,] f hakgdHe
<0
1
n²
1,==1
4)a,
1
Σ
Ey 1 ( - -(1-)))a1 / oddme
6r
C² 8,
+ B 2BAV ((pelii.)
+ + 2B2
D2(t)
1- 6,
2
211 - 6.))o (1)
32

---

Published as a conference paper at ICLR 2023
C² + B² 2BAv 2BAyT(uPlll ((ullit))) + + 2B 6, 1- D2(t)
2 5,.
Using AD2(t) & (1( + (t2 and Grönwall's inequality (Mischler, 2019), we arrive at
DI(1) + AD2(1) S (-¹ C² d (x)(())(D..O)) + AD2(0)),
where
8, A, A, = -2(1 - 6,)2a, + (VI((ssl(e)) + + 1 - 8,
ds,
for constant C1 max (2B3, 3/A.
C.5 PROOF OF LEMMA
Recall that in Lemma we obtained a bound on [g (Xx))) that contains a factor of N. Now (1)
refine this result by considering bound Eigi, (1)
(X))o] S Ellg (1) (x))llé))
Define the events I (Xu x}} and If (Xo x). We let jel7, be the conditional
distribution of X₁ conditioned by Iι and Mali be that conditioned by Ií. For notation simplic-
ity, we write = Gt 9 = ae and (()z(()) = Er'to P(Xo = x' X2 = -) (.) = (1)
(1)
(1)
Ex'dee En (.) ).
())
()). Accordingly, write azz (0) Bai laq Φ) and 21/z (Ø)
Buaiz 191/z Ø) for test function ø.
We control Bllg (x,)lI by utilizing the following bound:
El (X.) < N V (x.)) = x] + N N-1
N v 1 (x.)) + x]). (26)
We first evaluate (1) positive sequence 6, 0,
[[ (x,)) z]. If we let D2(t) Ex,, 1421, then it holds that, for
d
f
8b
E di
v, [
-2 [ [1?]
(4:) Vayz duaz,
s 22 1zzz || 2B² DDz(t), V /EMIEE [/V
< 22(1 8, 16 p V ; 11 + B3 (2(()((28.)
- -2a,, (1 - 6,) |9 + Ba D2()
Thus, Grönwall's inequality (see also the proof of Theorem 4) yields that
Epzz 10 [-
ct B²))(s)
exp(At - AA,dss + exp((xppp),........e 19
26
where A, = So - 2(1-5))aa,ds Here, we recall that Dz S Apo N (xp(-aa)(tTT)(334)))
by Lemma 7. Hence, if we set a constant which can depend on a, X, To. This argument and Lemma 6 give
(o) then A, -2Aa(s [To(s-TTo++ for
Epgg, ( À No N exxp(-oX(( (5)
(27)
= Next, we evaluate E[(g I4 X2 zz. Let Let. = ΣØ 91zz (eg) and D2,,(t)
7Z dprpzy. For (1,e()) D2,a(t) (t), we can see that
Σ Eming /012g / (0, AV {olg((uzzz) Vhydpugy
33

---

Published as a conference paper at ICLR 2023
deetz / / VVog((t) V Vlog(H1);; (1d Whylla /oo |1hylloo
V Pe.
<
j=1
- AnB
(( (2,
=.
We also notice that
Σ 9
dt "he S B',,D,
j=1
Hence by the same reasoning the proof of Theorem 4, we have
d
(D),(() + 2D2,c)
dt
--2 2 (C (1 b11) 18. -2x
ôbc
22
(g:)
Varzyduzz + B2 (2,
--
n²
8u
E 1 |12] - 22/ /
öb, (4:)
8p
N --1 1
/
8b,
(4t IT) + AV
dupzz B²AS, ()) D2,(()
N
2 -/ /
1
8b,
A
8b /
(auz, Varzdduu - 2 N / |V Iddmues
X2
(9HIZT)
dPHz
N
N
8u
N
1
8b.
S
(1t IF)
+ -V
duqr= + B2(2S,, 6)) D2,((1)
1
+ 4XB
Briizs [1aez m1ll
Vaqzz ldd/qp
N
5 N 1 1
/
8br
(at Iạ) + AV
dunpz B2 +z)
+ 4AB²
N2 Epizy 14 11 //
/ |
|dd
N -
8b. (1t 24) + XV 1
< - 2
dugpz + B2 + (2.)
N
(V App -BBp--Ao((-77///16)611
2
Vijz 12duuy;;
where we used Eq. (27) the last inequality. Then, by noticing that the same argument as Corollary holds for the conditional distribution Jazz since Eq. (17) holds uniformly for any æ E ( we may apply the same reasoning the proof of Theorem 4 to obtain that, for sufficiently small T > O, the following holds
(t, + (t2,((t)
Am expl-aX(t To)/2 + C(1 We(porr; /² (0)))( + D1, + D2,c(T))
S exxp(O(W2(PO,,) 1=)) + W2(Horzy (²))) (xpp(-u(((-To)/))]1 where we used Lemma 6 in the last inequality.
(28)
Finally, by combining the inequalities (27) and (28) to Eq. (26), obtain the assertion.
C.6 COMBINING ALL BOUNDS TOGETHER
Recall that
d
[(eens 8n2
I7 (t s, n. (,1x)) (o),0)) u (dx) ds.
(0, U(t,, un
N Σ
34

---

Published as a conference paper at ICLR 2023
Hence, by applying Lemma 7 and 9 with μo u E = {(1) =g(²]= C2 we obtain
/
L'M (1 uz )(r )e, 1/ (dx)
0 (2))) 1 (aa)) 8122
/
8
8n2 (mlt s, us )) s: 2,2), (t ,: 15,z))
80h
(m(t -8.1 N (dd - 8: 1h E, a. a,,,,,,)),, (dx)
= -OA expl expl->a(tt - --To)/ -oo)/2))
where Auy = 1 N exp(O 11) + W2(u xxx; Now we only need to evaluate the term B-
Suppose C' is constant such that Any S 1/2 N exp(C ,²) + W2 81 (xpp"") [)). Then, Since v (x) S exp(-xxc ||=r|
verify Ang eexp (3C, /B,
BRN [|| + B,,- [
Ey- [|a] = (1) and thus we need to evaluate E exp C Eny 11Ix This can be upper
bounded by E exp 9C² + 1 [IXIP])] = E exp
9Ca
+ 2N Exn [Ix:1p)] Let Q. =
2
Σ 1=
| By Ito's formula, we have
N
]
d dl
1
18
E
exp
2N
1=1
E
exp (Qt/2)
1 N
Σ b(X),
A
Σ
(d Ea (1X21)))
N
2N
1=1
1 n
2/2 (d + (x:11)) )
E
exp (Q./2)
Σ
B² B
(2 8)
||
N
Then, by Young's inequality, there exists constant C" depending on B, 6, c. d such that
-- - (2 8)Ac, 2+ (d+ C' (2 + 8) 12
2
Moreover, by Jensen's inequality, the term related to || can be further bounded by
(2+5)/2
1 N
--Q(220)) (2+8)/22
N
1=1
i=
In summary, we arrive at
d
|exp (Q./2) < Elexp(Q,/2)(CC -cspp(///)(( (2220)/22
E
dt
with another constant
(2+6)ee. 2
Here, by noticing that
C' - -Q (2+0))2 < 2C'1[Q S (2C' -00//////4+
holds that
d B (xxp (Q./2)l < Elexp(Q./2//2'110 < (202/0)2/1200) C']] dt
< E { 2C' exC - CCeexp((//))
This means that
300C/ee7 Elexp(Q./2)] S max(2expl (2C' /c) 2/(2+5)], Elexp(Qn/2)]] 0(1)
35

---

Published as a conference paper at ICLR 2023
One side remark is that further show exponential decay of the term related to Elexp(Qo/2)) but the above sufficient for our purpose. Finally, arrive at
(ttt,tt))) - o E[Awy] /' exp(->a(t- To)/
B (4)
1 (1 B[+
- exp(AATo/2)
aa
By selecting the initial distribution so that TE[A satisfied if the support of μo bounded, e.g., 16 EE 6p.
MD =(1), the right hand side is O(1). This is
D AUXILIARY LEMMAS
Lemma 10. Suppose that P E has smooth first-variation such that | for any vo, V1 E P2, it holds that
llxo S C. Then,
(w1) (x2)) S CW2(vo, 1/).
Proof. By the Benamou- -enaiiu-BBrenier formula, for any v1, My E P2, it holds that
(v v0) inf /Ilvel/ddudt [ 0,v, + V (1,,,) = e (0)
where the infimum is taken over all curves v4 [0. 1] + P2 continuous with respect to the weak topology (Ambrosio al., 2005, Chapter 8). Then, for any us satisfying dey + V (vav)) (t
[0, 1]), we note that
d = f v 82 (ve)dre, and thus
dt
(T) V((o))
1/¹ / T (ve
8p
S.¹ / [Ivell
(ve)
dezdit
5fc
V. / ||ve| Idvdd.
By taking infimum with respect to Ut, we obtain the inequality.
fx, the informal theorem stated in Section formally corresponds to fuN in the main text. Then, we have the following corollary as direct consequence of Theorem 2.
Corollary 7. Under the same setting as Theorem 2, we have
[l(f - (z)) O(INN),
where μt m(t, p and the expectation is taken over the dynamics of nd with fixed initial condition N.
Proof. By conditioning the initial distribution nr the evolution of μe deterministic, Hence, for fixed E Rddd holds that
El(Snx (z) - (2)) (x) - Sm)
2Exx/u - (=) 2R[J, (2) (=)))))
36

---

Published as a conference paper at ICLR 2023
+ - ( (f)
=
2[ (1) (=)).
Therefore, by considering ((4)= (x) and ((v) = (fu(z) - (=))), Theorem 2 indicates that
(() -()) - (2) - fz.
O(1/N),
which yields the assertion.
37


Reviews:
[{"review_id": "zgH8SFdVfz", "reviewer": "Reviewer_FGcq", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers the mean-field regime of an overparametrized two layers neural network $f_X$ where $X\\in\\mathbb{R}^p$ stands for the trainable parameters. Indeed, it have been shown previously taking the number of neurons $N\\to \\infty$ that optimizing such network can be seen as minimizing the functional over the space of probability laws \n$$\nF(\\mu)=\\frac{1}{n} \\sum_{i=1}^n \\ell_i\\left(f_\\mu\\right)+\\lambda_1 \\int r(x) \\mathrm{d} \\mu(x) \\quad , \\quad f_\\mu(z)=\\int f_x(z) \\mathrm{d} \\mu(x).\n$$\nIn particular, the paper aims to extend and complete existing literature regarding convergence of the neurons following the continuous limit of the \"noisy\" gradient descent dynamics to a mean-field Langevin equation which is the gradient flow dynamics associated with $F$. \nMore precisely, the authors consider the particle system\n\n$$\n\\mathrm{d} \\hat{X}_t^i=-b(\\hat{X}_t^i, \\mu_t^N ) \\mathrm{d} t +\\sqrt{2 \\lambda} \\mathrm{d} W_t^i \n$$\n\n$$\n\\mu_t^N=\\frac{1}{N} \\sum_{i=1}^N \\delta_{\\hat{X}_t^i}, \n$$\n\nwhere $W_t^i$ are independent Brownian motion and $b$ corresponds to the gradient of the first variation $F$\n$$\nb(x, \\mu)=\\nabla \\frac{\\delta F}{\\delta \\mu}(\\mu)(x)=\\frac{1}{n} \\sum_{j=1}^n \\ell_j^{\\prime}\\left(f_\\mu\\right) \\nabla h_j(x)+\\lambda_1 \\nabla r(x).\n$$\n\nThey then show that for a nice class of functional $\\Phi$, a uniform propagation holds for the sequence of dynamics $(\\mu_t^N)$: there exist $C_1,C_2,C_3$ such that for any $t \\geq 0$ and $N \\geq 1$,\n$$\n|\\mathbb{E}\\left[\\Phi\\left(\\mu_t^N\\right)\\right]-\\Phi\\left(\\mu^*\\right) |\\leq C_1 e^{C_2t} + C_3/N.\n$$\n\nwhere $\\mu^*$ is the minimizer of \n$$\n\\mathcal{L}(\\mu)=\\frac{1}{n} \\sum_{i=1}^n \\ell_i\\left(f_\\mu\\right)+\\lambda \\mathrm{KL}\\left(\\mu, \\nu_r\\right),\n$$\n\nand the limit of the mean-field Langevin dynamics\n\\begin{aligned}\n&\\mathrm{d} X_t=-b\\left(X_t, \\mu_t\\right) \\mathrm{d} t+\\sqrt{2 \\lambda} \\mathrm{d} W_t, \\\\\n&\\mu_t=\\operatorname{Law}\\left(X_t\\right).\n\\end{aligned}", "summary_of_the_review": "Overall I think that it could be a nice contribution to the literature on propagation of chaos.\n\nHowever, in my opinion, the writing of the paper has to be improved and the relevance of the contributions for the machine community should be better emphasized. \n\nSecond, I do not really know if ICLR is a good venue for the paper which I think should be suitable for more theoretical conferences or journals. Especially because the proofs of the results are quite delicate and require a relatively long time to be verified. To be honest, I did not have the time to do it and I am afraid no reviewer has either. \n\n", "strength_and_weaknesses": "Strength:\n- the result is mathematically interesting and brings some new tools for establishing uniform propagation of chaos for particle systems which emerged from overparametrized neural network optimization.\n\nWeakness:\n- I am not really sure how interesting this result is to the machine learning community. I think the authors should better motivate the question they are addressing and provide some insights why their results may help in understanding of the optimization of overparametreized neural networks.  Ideally, could the authors give practical implications of their findings? \n- The condition on the regularization term is not really satisfactory. In particular, the authors say that previous works were limited by assuming either weak interaction or large noise. To my view, the authors get rid of this constraint by imposing a very strong regularization term which always counterbalances the interaction drift. This choice could potentially miss important and interesting phase-transition phenomena. \n- I felt that some parts of the paper would needed to be rewritten and  proofread if the authors want that their paper is accessible to the machine learning community. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "rDrds709_r", "reviewer": "Reviewer_LXtr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a uniform-in-time analysis of the propagation of chaos for the mean-field Langevin dynamics to conduct neural network optimization. The authors avoid the double-loop structure and enable to deal with convergence guarantee based on finite-width neural network with vanilla noisy gradient descent algorithms.", "summary_of_the_review": "Convergence analysis for stochastic gradient descent on a finite-width neural network where the upper bound is table w.r.t. time.\n\n", "strength_and_weaknesses": "**Pros:**\n1. The first quantitative discretization error ensures analysis based on neural networks of finite/ limited neurons. The uniform nature ensures that the deviation of the output reduces rapidly.\n2. The analysis leverages the advantages of propagation of chaos and overcomes the bottleneck suffered by the Growall inequality in Mei'18 and the error bound remains stable when the time t is large.\n\n**Cons:**\nThe assumption of the super-quadratic tail of the regularization term is strong indeed.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "mrNZ9xOgGt", "reviewer": "Reviewer_713A", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper considers two-layer networks with one output dimension in the mean-field parametrization. In the limit of infinite hidden-layer width, these can be described in the infinite dimensional space of parameter distributions (across hidden unit weights) by a nonlinear Fokker-Plank equation, which has been done in the past. The paper justifies the relevance of this approach by computing bounds on the difference between finite-size discretization and the infinite-width limit, showing that the difference in any sufficiently smooth test function and for any time in the training process scales as $1/N$, given relatively mild assumptions. This extends previous finite-size bounds which suffered from an exponential dependence on training time.", "summary_of_the_review": "This paper seems mathematically sound and provides a relevant result for the study of feature learning in two layer networks in MF-parametrization. It is suitable for ICLR. The technical description of the novelty in approach should be improved.", "strength_and_weaknesses": "**Strengths:**\n - Technically demanding proof with relevant outcome (1/N scaling) under mild conditions.\n\n**Weaknesses:**\n - The progress in proof technique is stated multiple times in abstract/intro/conclusion (not requiring weak interaction while showing uniform propagation of chaos), but not technically discussed/explained in the relevant technical section 4 or appendices. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "a2GIJYd8ASh", "reviewer": "Reviewer_k2b8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the convergence of the mean-field gradient Langevin dynamics for the regularized mean-field neural network model. Under certain regularity assumption, it is proved that the dynamics in mean-field limit and the finite particle approximation are close for all time t>0, and certain statistic of these two dynamics are close with the order of O(1/N). As a consequence, it can be show that the finite particle approximation of the mean-field dynamics exponentially convergence towards the optimal solution up to an error of O(1/N).", "summary_of_the_review": "This paper shows that the dynamics in mean-field limit and the finite particle approximation are close for all time t>0, and certain statistic of these two dynamics are close with the order of O(1/N). This result is interesting, but is obtained under a relatively strong assumptions on the parameter regularizer.", "strength_and_weaknesses": "Strength: This paper provides the first rigorous uniform-in-time propagation of chaos result in the context of mean-field neural networks. This is achieved without adding strong entropy regularization or imposing strong assumptions on the complicated interaction term. \n\nWeakness: To achieve the claimed result, the convex regularizer on parameters is assumed to have a super-quadratic tail, an assumption that is not satisfied for the most common $\\ell_2$ regularizer.\n\nSome minor comments:\n1. Below the equation after Eq.(4), there is a statement \"We therefore see that $\\mu_t$ decreases ... $\\neq 0$.\". It is difficult to understand. \n2. In the paragraph of \"Particle discretization\" on page 4, it is assume that $\\hat X_0^i \\sim \\mu_0$, but in Lemma 1, it is claimed that you replaced the KL divergence on the R.H.S. with the Wasserstein distance since $\\mu_0$ is a discrete distribution. Why can't we simply take $\\mu_0$ to be some simple continuous initial distribution?\n3. At the end of Proposition 1, I guess it should be $\\mu^* = p_{\\mu^*}$.\n4. The concept of \"test function\" used in section 4.1 is not the same as the ones commonly used for defining weak convergences. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "YCs5fIdGo6F", "reviewer": "Reviewer_tT78", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proves uniform-in-time chaos in the context of mean-field (MF) Langevin dynamics, motivated by noisy gradient descent for 2-layer MF neural nets. The analysis exploits the fact that a super log-Sobolev inequality (LSI) for the so-called proximal Gibbs measure, enabled by a super-quadratic regularizer, translates to the LSI of the corresponding flow measure at any sufficiently large time.", "summary_of_the_review": "The paper proves the first uniform-in-time chaos result ever known for MF 2-layer neural nets. This is a good result with new technical insights, though I’m unable to go through the proofs. I would expect researchers in the area take serious interests in this result, even if the strong regularizer does not appear convincing to be the right way to understanding the phenomenon.", "strength_and_weaknesses": "This paper is the first I have known of to prove a uniform-in-time type of result for MF neural networks. This is a good result, since there is currently no applicable tool from the uniform-in-time chaos literature. Uniformity in time is also a highly desirable property, since one drawback of typical MF analyses is that the approximation between the large-width neural nets and the MF limit blows up exponentially with time. The observation that $\\mu_t$ satisfies LSI thanks to the super-quadratic regularizer, leading to Theorem 2, is an interesting one. The work also circumvents the trouble of doing the large-$N$ approximation at each time $t$ by going for a large (ideally infinite) $t$, hence allowing for the error decomposition as well as the exponential convergence result that was proven by previous works.\n\nI do not have much to complain about the paper, nor do I have sufficient time and background to check the proofs. Of course, there are typical concerns about the (potentially bad) dependency among the constants, as well as the rather unusually strong regularizer, but I would not worry much about it for now in light of a new result, the first of its kind. I have two questions:\n\n- The numerical illustration basically shows that the approximation is increasingly better for larger $N$, but it doesn’t quite show that the approximation is getting better / no worse with time. Is there a way to better illustrate this?\n\n- The strong tail of the regularizer essentially shapes the curvature at infinity and intuitively prevents bad things to happen at infinity. However the analysis centers around the optimal solution $\\mu*$, for which the faraway region likely matters much less. Is there a way to intuitively understand this supposed dilemma? One possibility is that the regularizer might not be needed for uniform-in-time chaos.\n\nHere my concern is that the strong regularizer is more of a technical device than something insightful, so there is some doubt whether this is the right way to understand uniform-in-time chaos.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster