Title: Stochastic Approximation Algorithms for Systems of Interacting Particles
Venue: NeurIPS 2023
Accepted: yes
Keywords: Stochastic Approximation

Mean-Field Dynamics

Dynamical Systems

Neural Networks

Sampling
Abstract: Interacting particle systems have proven highly successful in various machine
learning tasks, including approximate Bayesian inference and neural network optimization. However, the analysis of these
systems often relies on the simplifying assumption of the \emph{mean-field} limit, where particle
numbers approach infinity and infinitesimal step sizes are used. In practice, discrete time steps,
finite particle numbers, and complex integration schemes are employed, creating a theoretical gap
between continuous-time and discrete-time processes. In this paper, we present a novel framework
that establishes a precise connection between these discrete-time schemes and their corresponding
mean-field limits in terms of convergence properties and asymptotic behavior. By adopting a dynamical system perspective, our framework seamlessly integrates various numerical schemes that are typically analyzed independently. 
For example, our framework provides a unified treatment of optimizing an infinite-width two-layer neural network and sampling via Stein Variational Gradient descent, which were previously studied in isolation.

Stochastic Approximation Algorithms for Systems of Interacting Particles

Abstract

Interacting particle systems have proven highly successful in various machine learning tasks, including approximate Bayesian inference and neural network optimization. However, the analysis of these systems often relies on the simplifying assumption of the mean-field limit, where particle numbers approach infinity and infinitesimal step sizes are used. In practice, discrete time steps, finite particle numbers, and complex integration schemes are employed, creating a theoretical gap between continuous-time and discrete-time processes. In this paper, we present a novel framework that establishes a precise connection between these discretetime schemes and their corresponding mean-field limits in terms of convergence properties and asymptotic behavior. By adopting a dynamical system perspective, our framework seamlessly integrates various numerical schemes that are typically analyzed independently. For example, our framework provides a unified treatment of optimizing an infinite-width two-layer neural network and sampling via Stein Variational Gradient descent, which were previously studied in isolation.

1 Introduction

Dynamics of interacting particle systems are central to many challenges in modern machine learning. These range from algorithm design for approximate Bayesian inference, to the study of equilibria in games [26, 31, 34]. Moreover, researchers have gained valuable insights by interpreting the training process of over-parametrized two-layer neural networks as a system of interacting particles, thereby advancing our intuition in this domain [11, 12, 36-38, 46, 51].

Analyzing the behavior of such systems poses a significant challenge. To simplify the analysis, researchers often turn to the concept of the mean-field limit . In this approach, the number of particles is increased to infinity, the step-size of the algorithm shrinks to zero, and results are deduced from this continuous limit. Specifically, an initial probability distribution is assigned to the infinitely many particles, and a continuous-time evolution is derived for this distribution. The resulting evolutionary equation serves as a powerful tool for gaining insights into the system's asymptotic behavior, significantly aiding in the comprehension of systems with a large number of interacting particles [1, 5, 22, 26, 47].

Although the mean-field limit has offered valuable insights into the aforementioned problems, its rigorous justification necessitates examining systems with both a finite number of particles and the discrete-time algorithms commonly used in practice. Unfortunately, to the best of our knowledge, while a substantial body of work exists on finite-particle systems (see, e.g., the review papers [8, 9] and references therein), these studies are confined to analyzing continuous-time dynamics. As a result, a gap between theory and practice has emerged, and a precise link between continuous and discrete-time schemes is still largely lacking.

§ Contributions. Our paper aims to bridge this gap by rigorously establishing the convergence of discrete-time algorithms to their continuous-time counterparts in terms of long-term behavior.

To accomplish this, we draw inspiration from the field of dynamical system theory, which traces its origins back to early developments in statistics [42] and has recently found success in various domains of machine learning, including optimization theory, games, and sampling [21, 23, 24].

This dynamical system framework offers two significant advantages. Firstly, it provides a flexible framework capable of accommodating a wide range of practical schemes commonly employed in real-world scenarios. Secondly, this framework enables a unified treatment of various machine learning tasks that were previously studied in isolation. Notably, it allows us to address tasks such as training neural networks and approximate Bayesian inference using techniques like Stein variational gradient descent in a coherent and unified manner.

In summary, our paper makes the following contributions:

We introduce a comprehensive framework for analyzing a broad class of algorithms called stochastic approximation schemes . These schemes are widely utilized in simulating systems of interacting particles, and our framework provides a unified approach to analyze their behavior.

Under mild assumptions on the discrete-time schemes and the mean-field dynamics, we prove the convergence of these schemes towards their respective mean-field limits. The convergence is established in terms of the 2-Wasserstein distance , a metric commonly used to measure the dissimilarity between probability distributions.

Since our framework is specifically tailored to address a wide range of fields, we instantiate our main theorem and provide novel guarantees to a diverse array of interacting particle systems across domains such as the Stein variational gradient descent, the training of wide two-layer neural networks, and the examination of game equilibria.

2 Illustration: Training Two-Layer Neural Networks via Noisy SGD

To illustrate the particle system under study and its corresponding mean-field limit, let us consider the example of training a two-layer neural network with 𝑁 neurons and squared loss using the noisy SGD algorithm. The network takes an input 𝑧 2 ℝ 𝑑 GLYPH<0> 1 and computes function:

Here, θ = ' 𝜃 1 GLYPH<150> GLYPH<149> GLYPH<149> GLYPH<149> GLYPH<150> 𝜃 𝑁 ' denotes the collection of neurons' parameters, and 𝜅 is an activation function (such as sigmoid or tanh ). To train this network, one ideally minimizes the regularized risk 𝐿 ' θ ' B 𝔼 ' 𝑦GLYPH<150>𝑧 'GLYPH<24> D 1 2 ' 𝑦 GLYPH<0> ℎ θ ' 𝑧 '' 2 , 𝜆𝑈 ' θ ' , where D is the data distribution, and 𝑈 ' θ ' = 1 𝑁 P 𝑖 𝑈 ' 𝜃 𝑖 ' is a regularizer, such as 𝑈 ' 𝜃 ' = 1 2 j 𝜃 j 2 . Applying the noisy gradient descent algorithm to the regularized risk then results in the update rule:

where 𝛾 𝑘 , 1 is the step-size, 𝜎 GLYPH<21> 0 is the noise level, and f 𝜉 𝑖 𝑘 , 1 g is a collection of i.i.d. standard Gaussians. To cast this algorithm as a system of interacting particles, we define the interaction kernel 𝑊 and the potential 𝑉 as

It is then easily seen (cf. [36, Eqn. 4]) that the update rule (SGD 𝜎 ) can be rewritten as

By considering neurons as particles, the equation (2) reveals that the training dynamics of each particle is influenced by the potential 𝑉 and the interaction energy 𝑊 with other particles. This interpretation highlights that (SGD 𝜎 ) represents the evolution of a system of interacting particles. In a broader context, one may consider more sophisticated algorithms, several of which (see Section 4 for examples) can be formulated as

where 𝑃 𝑖 𝑘 , 1 is the (random or deterministic) perturbation in evaluating the gradient. These algorithms are usually called stochastic approximation algorithms . For example, Noisy SGD corresponds to setting 𝑃 𝑖 𝑘 , 1 = r 𝜃 𝑖 e 𝐿 ' θ 𝑘 ' GLYPH<0> r 𝜃 𝑖 𝐿 ' θ 𝑘 ' , where e 𝐿 is the loss of a random batch of data.

§ Continuous-time limit and the mean-field approximation. We now go from the discretetime algorithm (3) to a continuous-time process and then derive the mean-field approximation. First, observe that the risk 𝐿 ' θ ' depends on ' 𝜃 1 GLYPH<150> GLYPH<149> GLYPH<149> GLYPH<149> GLYPH<150> 𝜃 𝑁 ' only through their empirical measure b 𝜇 = 1 𝑁 P 𝑖 𝛿 𝜃 𝑖 . For example,

where we have introduced the drift function 𝑏 . To simplify the analysis, previous studies then consider the idealized setting where an infinitesimal step-size is employed, thereby further reducing (3) to the following system of stochastic differential equations (SDEs): p

where b 𝜇 𝑡 = 1 𝑁 P 𝑖 𝛿 𝜃 𝑖 𝑡 is the empirical measure of the particles at time 𝑡 , and f 𝑊 𝑖 · g is a collection of i.i.d. standard Brownian motions. In the over-parametrized regime where the number of particles 𝑁 becomes very large, one can approximate the initial setting of the particles b 𝜇 0 with a probability density 𝜌 0 , and consider the following mean-field approximation defined over ℝ 𝑑 as p

This dynamics captures the behavior of an individual particle 𝜃 within a density of particles distributed according to 𝜌 . The analysis of (6) turns out to be considerably simpler compared to the system of SDEs in (5), e.g., via studying the evolutionary PDE 𝜕 𝑡 𝜌 𝑡 = r · ' 𝜌 𝑡 𝑏 ' · GLYPH<150> 𝜌 𝑡 '' , 𝜎 2 Δ 𝜌 𝑡 . As a result, it has attracted significant interest in the field of deep learning theory [11, 12, 36-38].

In conclusion, the mean-field dynamics (6) offers a powerful and elegant framework, but its validity rests on two simplifying assumptions: an infinite number of particles and a step-size approaching zero. Arigorous justification of these two steps is by no means trivial. While the existing literature has made progress in addressing the infinite particle issue, the second assumption has received comparatively less attention. Bridging this gap is one of the primary objectives of our paper. Specifically, we aim to establish the Wasserstein convergence of the discrete-time dynamics (3) to the same limit sets 1 as the continuous-time particle dynamics (5), under mild conditions on the drift 𝑏 and perturbations f 𝑃 𝑖 𝑘 , 1 g , as well as the step-size rule 𝛾 𝑘 , 1 .

3 Dynamics of Systems of Interacting Particles

This section presents the fundamental master theorem that forms the basis for all the applications discussed in Section 4. Our objectives are two-fold. Firstly, we aim to provide a set of assumptions for the discrete-time scheme that can be easily verified by practical algorithms. Secondly, we establish a set of assumptions on the mean-field dynamics, which, as demonstrated in Section 4, are readily implied by the standard assumptions in the finite particle regime. The key result of our paper asserts that, under these assumptions, any discrete-time scheme converges to its continuous counterpart, thus closing the existing theoretical gap.

3.1 The algorithmic template

In this paper, we study the stochastic approximation algorithms for simulating systems of interacting particles of the form:

, 1 b , 1 b , 1 where 𝑏 : ℝ 𝑑 GLYPH<2> 𝒫 2 ' ℝ 𝑑 ' ! ℝ 𝑑 is the (non-local) drift, 2 𝜎 : ℝ 𝑑 GLYPH<2> 𝒫 2 ' ℝ 𝑑 ' ! ℝ 𝑑 GLYPH<2> 𝑑 is the (statedependent and non-local) diffusion coefficient, 𝑃 𝑖 𝑘 , 1 is the noise and bias in evaluating the drift, and b 𝜇 𝑘 is the empirical measure of the particles at iteration 𝑘 . The system (SAA) can be written in a more succinct way by stacking all the variables in a larger vector. That is, define 𝑥 B ' 𝑥 𝑖 ' 𝑖 2» 𝑁 … 2 ' ℝ 𝑑 ' GLYPH<10> 𝑁 , and let 𝜇 𝑥 be the empirical distribution corresponding to the 𝑁 vectors in 𝑥 , and define the aggregated drift and diffusion terms as

Define f 𝑃 𝑘 , 1 g and f 𝜉 𝑘 , 1 g analogously. We can then rewrite (SAA) as:

3.2 Dynamical system theory

By considering infinitesimal step-sizes and neglecting the perturbations 𝑃 𝑖 𝑘 , 1 , we can derive a system of corresponding SDEs as follows:

with the corresponding aggregated version

It is important to emphasize that even though we have rearranged the particles into a unified vector and introduced the concepts of aggregated drift and diffusion, the original process retains its exchangeability . This property implies that the distribution of 𝑋 𝑖 𝑡 in (Sys-SDE) remains invariant under permutations of the particles (for further details, refer to [8, Def. 2.1]).

The primary objective of our paper is to rigorously establish the convergence of the stochastic approximation scheme (PSAA) to its continuous-time counterpart (PSDE). To accomplish this, we employ the dynamical system theory introduced by Benaïm and Hirsch [4]. First, we construct a continuous-time interpolated process associated with the discrete-time algorithm (PSAA):

Here, 𝜏 𝑘 = P 𝑘 𝑗 = 1 𝛾 𝑗 represents the cumulative time until step 𝑘 . It is worth noting that we construct (Int) in such a way that it is adapted to the same filtration ' ℱ 𝑡 ' as the Brownian motion.

To compare (PSAA) with (PSDE), we integrate (PSDE) using the following approach: For a fixed 𝑡 GLYPH<21> 0 , we define 𝑊 ' 𝑡 ' 𝑠 = 𝑊 𝑡 , 𝑠 GLYPH<0> 𝑊 𝑡 and denote the solution of (PSDE) as the flow :

It is important to observe that the flow starts at 𝑋 𝑡 and continues according to the true SDE.

We now introduce the central concept in our paper, which is the asymptotic pseudotrajectory theory of Benaïm and Hirsch [4].

Definition 1 (Wasserstein asymptotic pseudotrajectory) . We say the stochastic process ' 𝑋 𝑡 ' 𝑡 GLYPH<21> 0 is a Wasserstein asymptotic pseudotrajectory (WAPT) of the flow Φ if for any fixed 𝑇 ¡ 0 ,

where W 2 'GLYPH<1> GLYPH<150> GLYPH<1>' denotes the 2-Wasserstein distance between two distributions.

The notion of WAPT provides a measure of 'asymptotic closeness' between two stochastic processes. In particular, (8) requires that ' 𝑋 𝑡 ' 𝑡 GLYPH<21> 0 closely tracks the flow Φ ' 𝑡 ' 𝑠 over arbitrarily long time intervals 𝑇 with arbitrary precision. The key aspect of the WAPT is that it serves as a tool specifically designed to establish the convergence of a stochastic approximation scheme to its continuous-time counterparts. In particular, it is known that proving the convergence of a stochastic approximation algorithm (PSAA) to its continuous-time counterparts (PSDE) can be accomplished by demonstrating the following two conditions [3, 4]:

The interpolation (Int) satisfies the WAPT condition with respect to the corresponding flow.

The iterates f 𝑥 𝑘 g 𝑘 's in (PSAA) have bounded second moments.

Below, we present a set of general conditions that are straightforward to verify and ensure the satisfaction of the above two conditions.

Remark. There are two major reasons for choosing W 2 as the distance in (8): Firstly, the 2-Wasserstein space is a metric space on which McKean-Vlasov equations can be seen as a flow , both aspects indispensable for invoking the dynamical system theory of Benaïm and Hirsch [3, 4]. Secondly, it is a popular metric in the propagation of chaos literature. This allows a seamless transition from convergence guarantees for stochastic approximation schemes to their mean-field limit counterparts via combining our results with the propagation of chaos results in the literature, see (13).

3.3 Technical Assumptions

We proceed to present our technical assumptions and discuss their generality.

§ On the mean-field dynamics. We begin by introducing three assumptions that pertain to the drift and diffusion coefficients of the continuous-time dynamics (Sys-SDE):

Assumption 1 (Lipschitzness of drift and diffusion) . There is some 𝐿 ¡ 0 such that for all 𝑥GLYPH<150> 𝑦 2 ℝ 𝑑 and all 𝜇GLYPH<150> 𝜈 2 𝒫 2 ' ℝ 𝑑 ' ,

Assumption 2 (Drift growth condition) . For all 𝜇 2 𝒫 2 ' ℝ 𝑑 ' , there is some 𝐶 𝑣 ¡ 0 such that

Assumption 3 (Boundedness of the diffusion) . There is some 𝐾 ¡ 0 such that for all 𝑥 2 ℝ 𝑑 and all 𝜇 2 𝒫 2 ' ℝ 𝑑 ' , k 𝜎 ' 𝑥GLYPH<150> 𝜇 'k 𝐹 GLYPH<20> 𝐾 .

We note that the first assumption is standard and is commonly used to prove existence of strong solutions for the mean-field equation (see [26, Thm. 3.3]). The other two assumptions are exceedingly weak and are satisfied by all the applications we consider.

§ On the stochastic approximation schemes. The following assumptions concern the timediscretization scheme and the induced noise and bias.

Assumption 4 (Noise and bias) . The perturbation 𝑃 𝑘 , 1 decomposes into noise and bias as 𝑃 𝑘 , 1 = 𝑈 𝑘 , 1 , 𝜀 𝑘 , 1 , where the noises f 𝑈 𝑘 , 1 g form a martingale difference sequence, i.e., 𝔼 » 𝑈 𝑘 , 1 j 𝑈 𝑘 … = 0 , and have second moments uniformly bounded by 𝑀 𝑈 . In addition, the bias terms satisfy 𝜀 𝑘 2 ℱ 𝜏 𝑘 and

Assumption 5 (Step-sizes) . The step-sizes are decreasing and satisfy the Robbins-Monro summability conditions

Moreover, we require, for some constant 𝑃 ¡ 0 ,

The condition specified in equation (9) is algorithm-dependent, and as we will prove in Section 4, it is satisfied by numerous practical schemes. In Assumption 5, equation (10) is a commonly used condition in the literature [42], while (11) imposes a mild growth condition on the step size, which remains satisfied even for slowly-decreasing step-sizes such as 𝛾 𝑘 , 1 GLYPH<24> ' p 𝑘 log 𝑘 ' GLYPH<0> 1 . Therefore, this condition is not overly restrictive and accommodates a wide range of scenarios.

§ On dissipativity. In the context of dynamical system theory, it is important to ensure that the iterates of stochastic approximation schemes have bounded second moments. In the literature, this requirement is often met by imposing dissipativity -type conditions. Building upon this concept, we introduce the following definition:

Definition 2 (Average Dissipativity) . We call the drift 𝑏 to be ' 𝛼GLYPH<150> 𝛽 ' -dissipative on average for some 𝛼 ¡ 0 and 𝛽 2 ℝ , if for all probability measures 𝜇 2 𝒫 2 ' ℝ 𝑑 ' , it holds

The concept of average dissipativity, as introduced in Definition 2, provides a novel formulation specifically designed to capture the dissipativity property of a drift function that depends on measures , as in equation (Sys-SDE). In contrast to the traditional notion of dissipativity [19], which focuses on the dissipative behavior of individual particles in isolation, this formulation allows for a more fine-grained control over the collective behavior exhibited by 𝑁 particles, each running in parallel with the same stochastic approximation scheme. In Section 4, we will provide concrete examples from the applications of machine learning to demonstrate the satisfaction of average dissipativity.

3.4 Main Results

We are now ready to state our main theorem, whose proof can be found in Appendix B.

Theorem 1. Consider the algorithm (SAA) , where the drift 𝑏 and diffusion 𝜎 satisfy Assumptions 1-3, and the step-sizes f 𝛾 𝑘 , 1 g and the perturbations f 𝑃 𝑖 𝑘 , 1 g satisfy Assumptions 4 and 5. Then the following holds:

The interpolation (Int) of iterates of the algorithm is a WAPT of the flow in (Flow) .

Moreover, if the drift 𝑏 is dissipative on average (see Definition 2), the iterates of (PSAA) are bounded in second moments, and their limit set is included in that of the original SDE (PSDE) .

Proof sketch. The main step of the proof is the construction of the Picard process , which is inspired by [23] and defined as follows:

The proof is completed in four steps: first, we prove that the Picard process closely tracks the flow (Flow), and then we bound the distance between the Picard process and the interpolation (Int). By using martingale convergence arguments, employing our assumption on bias Assumption 4, and Grönwall inequality, we conclude the proof of the WAPT property. Lastly, we show that dissipativity on average ensures a uniform bound on the second moments of the iterates. The convergence is then implied by invoking [23, Theorem 3]. GLYPH<4>

In summary, according to Theorem 1, the convergence of the stochastic approximation scheme in (SAA) can be reduced to its continuous-time counterpart in (Sys-SDE) when the assumptions stated in Section 3.3 are satisfied.

Our Theorem 1 can be combined with existing results in the finite particle regime to establish the overall convergence towards the desired mean-field limit. To see this, let 𝑀 𝑖GLYPH<150>𝑁 𝑡 be 𝑁 independent processes, synchronously coupled with (Sys-SDE), each starting from 𝑋 𝑖 0 and following

where 𝜇 𝑡 is the mean-field solution . A common phenomenon in the study of interacting particle systems, known as uniform propagation of chaos [8, 22, 27, 47], implies the existence of a constant 𝐶 such that for every 𝑁 :

where 𝑋 𝑖GLYPH<150>𝑁 𝑡 represents the particles following the continuous-time dynamics in (Sys-SDE). Letting 𝜇 1 represent the limit of the mean-field equation, a straightforward application of the triangle inequality argument, which we defer to Appendix B.2, yields:

In other words, as the number of particles approaches infinity, the law of the empirical distribution of the particles following the discrete-time algorithm (SAA) also converges to the mean-field solution.

§ A note on the literature. The rich body of literature on McKean-Vlasov SDEs and interacting particle systems offers considerable insights on the convergence of Euler-Maruyama and Milstein type numerical schemes to their limiting mean-field equations, e.g., [2, 28, 41]. However, it is noteworthy that our study diverges in several key respects: Firstly, our work emphasizes on generic stochastic and biased drift oracles. This contrasts with the deterministic and unbiased drift oracles considered in the aforementioned studies, making our algorithmic approach broader in scope. Secondly, while those studies present strong finite-time error bounds, our convergence results focus on providing asymptotic guarantees. Lastly, we incorporate different underlying assumptions. For instance, we need global Lipschitz drifts to ensure globally integrable flows, while one-sided Lipschitz drifts are allowed in the works by [2, 28, 41]. However, our growth condition in Assumption 2 requires control on average, whereas the works mentioned assume stronger pointwise controls.

In the light of these distinctions, we believe that our work complements this body of literature.

4 Applications

The goal of this section is to demonstrate the wide-ranging applicability of our framework across diverse domains such as machine learning, game theory, and physics, which were previously analyzed in isolation. In each of these applications, we demonstrate that standard assumptions in their respective domains meet the necessary conditions to invoke Theorem 1. All proofs are deferred to Appendix C.

4.1 Two-Layer Neural Networks and Mean-Field Langevin

Our first application is providing a rigorous guarantee for the training dynamics of wide two-layer neural networks as alluded to in Section 2. To begin, let us quickly recall the notations therein: The noisy SGD iterates in (SGD 𝜎 ) for wide two-layer neural networks can be viewed as approximations of the mean-field dynamics (6) through the discrete-time system (SAA), whose drift term 𝑏 'GLYPH<1> GLYPH<150> GLYPH<1>' is defined in (1). The corresponding continuous-time and finite-particle dynamics is (5), which has been extensively studied in recent years; see [11, 12, 36-38, 46, 51] and references therein. Under the standard assumptions as in [11, 12, 37], a simple application of Theorem 1 then yields the convergence of (SGD 𝜎 ):

Corollary 1. Let 𝜅 'GLYPH<1>' denote the activation function. Assume that (1) 𝜅 and 𝜅 0 are Lipschitz and bounded, (2) the data has bounded support, and (3) j 𝑎 𝜅 0 ' 𝑎 'j is bounded. Then the discrete-time scheme (SGD 𝜎 ) converges in W 2 to the same limit sets as the continuous-time (5) .

The mean-field dynamics for two-layer neural networks converges to a stationary point in Wasserstein space due to its gradient flow structure [12]. In addition, it is known that under fairly mild assumptions, the resulting limit becomes a unique global risk minimizer [12]. By employing uniform propagation of chaos for neural networks [46], it is observed that the limit sets of the continuous-time 𝑁 -particle dynamics exhibit similar generalization error to the global minimizer. Remarkably, Corollary 1 validates that noisy SGD and other discretizations following template (SAA) eventually converge to this limit set, providing justification for the observed generalization behavior in neural networks.

§ Comparison to prior work. Our result above cannot be directly compared to the existing analysis conducted on the discrete-time scheme (SGD 𝜎 ), namely [37], due to the assumption A1 therein, which excludes step-size rules of the form 𝛾 𝑘 , 1 GLYPH<24> 𝑘 GLYPH<0> 𝛽 where 𝛽 2 ' 1 GLYPH<157> 2 GLYPH<150> 1 … . Conversely, the bounds provided in [37] for a constant step-size are non-asymptotic (albeit doubly-exponential), resulting in stronger conclusions compared to our asymptotic results. Therefore, these two analyses complement each other and contribute to a more comprehensive understanding of the training dynamics exhibited by neural networks.

4.2 Stein Variational Gradient Descent

Sampling from a distribution 𝜋 / 𝑒 GLYPH<0> 𝑉 is a crucial task in various machine learning applications. An effective method that has demonstrated practical success in this regard is the Stein variational gradient descent (SVGD) [29, 30]. Intuitively, this method emulates the steepest descent for the KL divergence in continuous-time with a continuous probability measure . In practice, the algorithm is implemented as an interacting particle system as follows (see [33] for a derivation). Let 𝐾 : ℝ 𝑑 GLYPH<2> ℝ 𝑑 ! ℝ be a positive definite kernel . The SVGD algorithm updates the set of 𝑁 particles as

where r 2 𝐾 is the gradient of 𝐾 with respect to its second input. The corresponding continuous-time dynamics is then:

which is a special case of (Sys-SDE) with 𝑏 ' 𝑥GLYPH<150> 𝜇 ' = 'r 2 𝐾 GLYPH<3> 𝜇 '' 𝑥 ' GLYPH<0> ' 𝐾 GLYPH<3> ' 𝜇 r 𝑉 ''' 𝑥 ' and 𝜎 GLYPH<17> 0 (no diffusion), where GLYPH<3> denotes the convolution operator. We now prove:

Corollary 2. Suppose that (1) r 𝑉 ' 𝑥 ' is Lipschitz, (2) 𝑉 is dissipative, 3 (3) for some 𝐶 ¡ 0 , jr 𝑉 ' 𝑥 'j GLYPH<20> 𝐶 ' 1 , j 𝑥 j 2 ' , (4) k 𝐾 k1 GLYPH<150> kr 2 𝐾 k1 GLYPH<150> kr 2 𝐾 k1 GLYPH<159> 1 , (5) jr 2 𝐾 ' 𝑥GLYPH<150> 𝑦 'j GLYPH<20> 𝜂 GLYPH<157> j 𝑥 GLYPH<0> 𝑦 j , and (6) 𝐾 ' 𝑥GLYPH<150> 𝑦 ' GLYPH<20> 𝜂 GLYPH<157> j 𝑥 GLYPH<0> 𝑦 j 2 for some 𝜂 ¡ 0 . Then the iterates in (SVGD 𝑘 ) converge in W 2 to the same limit sets as the continuous-time process (SVGD 𝑡 ) .

Similar to neural networks training discussed in Section 4.1, the mean-field SVGD also exhibits a gradient flow structure, and in this case, the target distribution 𝜋 serves as the only limit set for the mean-field SVGD [29]. Combining Corollary 2 with a uniform propagation of chaos argument for SVGD then confirms that the iterates of (SVGD 𝑘 ), or any discretization of (SVGD 𝑡 ) that satisfies Assumption 4, effectively converge to a distribution that closely approximates 𝜋 in terms of W 2 . This outcome underscores the effectiveness of these algorithms in sampling.

§ Comparison to prior work. While there has been extensive theoretical work on the convergence of SVGD, most of it focuses on either the population limit (i.e., when 𝑁 !1 in (SVGD 𝑘 )) or the vanishing step-size (which directly examines the properties of (SVGD 𝑡 )) [10, 14, 25, 29, 43, 45]. However, the convergence behavior of the discrete iterates (SVGD 𝑘 ) remains a challenging task with limited success. To the best of our knowledge, the only existing work in this direction is [44], but its result is not directly comparable to ours. Although our assumptions on the bounded derivatives of 𝑉 and 𝑊 are similar to those in [44], the difference lies in the requirement imposed on the target distribution 𝑒 GLYPH<0> 𝑉 . Specifically, Shi and Mackey [44] assume a T1-inequality on 𝑒 GLYPH<0> 𝑉 [50], whereas our work requires 𝑉 to be dissipative. Notably, our approach holds a significant advantage over [44] in terms of simplicity: The result of [44] is only applicable to a highly specific and fairly complicated step-size rule (see [44, Cor 2]), whereas our sole requirement on 𝛾 𝑘 , 1 is the standard Assumption 5. However, it should be noted that the bounds in [44] are non-asymptotic (albeit doubly-exponential in 𝑁 and exponential in 𝑘 ), while we can only handle asymptotic convergence.

Remark. It is worth mentioning that Corollary 2 can be straightforwardly extended to the Stochastic Particle-Optimization Sampling algorithm of Zhang et al. [53], which is identical to SVGD but includes a constant diffusion term 𝜎 ¡ 0 . Since the analysis remains the same, we omit the details.

4.3 Two-Player Zero-sum Continuous Games

Min-max learning appears in several important machine learning tasks such as Generative Adversarial Networks [18] and adversarial training [35]. These learning problems can be formulated as a continuous zero-sum game between a min-player and a max-player. The min-player selects strategies from the set X GLYPH<26> ℝ 𝑑 , while the max-player selects strategies from the set Y GLYPH<26> ℝ 𝑑 , with the goal of finding a saddle point of a function 𝐾 ' 𝑥GLYPH<150> 𝑦 ' :

However, solving (14) becomes challenging and sometimes impossible when 𝐾 is non-convex in 𝑥 and non-concave in 𝑦 , as a solution to (14) may not even exist. To address this issue, mixed Nash equilibriums (MNEs) are introduced, where the pure strategies are replaced by probability distributions over the sets of strategies, which exist under mild assumptions on 𝐾 [13, 17].

Specifically, an MNE is represented by a pair of measures ' 𝜇 ★ GLYPH<150> 𝜈 ★ ' 2 𝒫 2 ' X ' GLYPH<2> 𝒫 2 ' Y ' , forming a saddle point of the functional 𝐸 ' 𝜇GLYPH<150> 𝜈 ' B ∬ 𝐾 ' 𝑥GLYPH<150> 𝑦 ' 𝜇 ' 𝑑𝑥 ' 𝜈 ' 𝑑𝑦 ' . The quest for efficient solutions to the MNE problem in machine learning has led to the development of particle-based methods that offer approximate solutions [13, 20, 34]. However, the existing studies have primarily focused on analyzing the continuous-time dynamics of these methods. Below, we shift our focus to the more practical setting of the discrete-time system of interacting particles in these work:

where 𝜉 𝑖 𝑘 , 1 and 𝜁 𝑖 𝑘 , 1 are independent standard Gaussians, 𝜏 ¡ 0 is a hyperparameter chosen by the user, and 𝛼 is the scale difference between the two players [34]. We will additionally consider the optimistic version of (GDA 𝑘 ) in [20], which has shown empirical benefits over (GDA 𝑘 ):

:

At first glance, it may seem that (GDA 𝑘 ) and (OGDA 𝑘 ) are distinct algorithms that cannot be analyzed together. However, our subsequent corollary reveals that these systems actually converge to the same

continuous-time dynamics:

where 𝑊 𝑖 𝑡 and 𝐵 𝑗 𝑡 are collections of i.i.d. Brownian motions. The key enabling factor for this unified treatment is to allow for a non-zero bias in the algorithmic template (SAA).

To facilitate the analysis, we introduce a pairing of players' particles by setting 𝑄 𝑖 𝑡 B ' 𝑋 𝑖 𝑡 GLYPH<150>𝑌 𝑖 𝑡 ' 2 ℝ 2 𝑑 , and define the drift 𝑏 : ℝ 2 𝑑 GLYPH<2> 𝒫 2 ' ℝ 2 𝑑 ' ! ℝ 2 𝑑 as

and the diffusion 𝜎 ' 𝑞GLYPH<150> 𝜇 ' B p 2 𝜏 diag ' 𝐼 𝑑 GLYPH<2> 𝑑 GLYPH<150> p 𝛼 𝐼 𝑑 GLYPH<2> 𝑑 ' . By doing so, we can then cast (GDA 𝑡 ) in template of (Sys-SDE).

Corollary 3. Assume that (1) r 𝑥 𝐾 and r 𝑦 𝐾 are Lipschitz, and (2) r 𝑥 𝐾 and GLYPH<0>r 𝑦 𝐾 are dissipative, or (2') the domains X and Y are bounded. Then the algorithms (GDA 𝑘 ) and (OGDA 𝑘 ) converge in W 2 to the same limits of (GDA 𝑡 ) .

We note that the assumptions in (3) are mild and are even required for analying the continuous-time dynamics; see e.g., [34]. Consequently, our Corollary 3 provides a rigorous foundation for the consideration of continuous-time dynamics in existing studies such as [13, 20, 34].

Remark. The schemes (GDA 𝑘 ) and (OGDA 𝑘 ) above rely on simultaneous updates, i.e., from ' 𝑥 𝑘 GLYPH<150> 𝑦 𝑘 ' , one obtains ' 𝑥 𝑘 , 1 GLYPH<150> 𝑦 𝑘 , 1 ' . However, empirical evidence suggests that alternating updates i.e., following ' 𝑥 𝑘 GLYPH<150> 𝑦 𝑘 ' ! ' 𝑥 𝑘 , 1 GLYPH<150> 𝑦 𝑘 ' ! ' 𝑥 𝑘 , 1 GLYPH<150> 𝑦 𝑘 , 1 ' , often preforms better. Our framework allows for this flexibility, as it is easy to cast the alternating (GDA 𝑘 ) and (OGDA 𝑘 ) as stochastic approximation schemes satisfying Assumption 4, see [24, Proposition 4] for an example of this argument.

4.4 Kinetic Equations

In this section, we study the kinetic equations defined on the space of probability measures. Initially emerging from the physics community, these equations have recently gained attention in the machine learning community due to their connection to Wasserstein gradient flows [15, 32, 39, 40, 48, 49, 52]. Here, we denote 𝜌 as a probability density, and we examine three distinct 'energy' functionals: an internal energy U , a potential energy V , and an interaction energy W . These functionals are defined as follows:

The most interesting scenario is when 𝑈 ' 𝑠 ' = 𝑠 log 𝑠 represents the entropy , in which case the Wasserstein gradient flow with finite particles becomes [5, 7]:

By setting 𝑏 ' 𝑥GLYPH<150> 𝜇 ' B GLYPH<0>r 𝑉 ' 𝑥 ' GLYPH<0> 'r 𝑊 GLYPH<3> 𝜇 '' 𝑥 ' and 𝜎 ' 𝑥GLYPH<150> 𝜇 ' GLYPH<17> p 2 , we again see that (Kin-SDE) is a special case of (Sys-SDE).

In the physics community, the equation is commonly simulated using a system of interacting particles through the proximal point method [6, 48]:

Note that the right-hand side of (Kin-Prox) involves the next iterates 𝑥 𝑖 𝑘 , 1 so that, as opposed to the simple Euler discretization, it is an implicit rule. The key factor that enables the application of our framework to these implicit schemes is the observation that (Kin-Prox) can be formulated as a stochastic approximation scheme in (SAA) by incorporating a non-zero bias term; see Appendix C. Our next result provides a rigorous guarantee for these methods:

Corollary 4. Assume that (1) 𝑉 is dissipative and r 𝑉 is 𝐿 -Lipschitz, (2) 𝑊 is symmetric and r 𝑊 is 𝐿 -Lipschitz, (3) There exists some 𝑀 𝑊 GLYPH<21> 0 such that for all 𝑥GLYPH<150> 𝑦 2 ℝ 𝑑 , hr 𝑊 ' 𝑥 ' GLYPH<0> r 𝑊 ' 𝑦 ' GLYPH<150> 𝑥 GLYPH<0> 𝑦 i GLYPH<21> GLYPH<0> 𝑀 𝑊 GLYPH<149> Then, the iterates (Kin-Prox) converge in W 2 to the same limit sets as (Kin-SDE) .

We remark that the convergence of (Kin-Prox) under these standard assumptions is known [15, 48]. However,we emphasize that the aforementioned results only apply to deterministic updates, while our proof is robust enough to accommodate updates with finite-variance noise . This flexibility sets our approach apart, allowing for a broader range of practical applications.

5 Conclusion and Future Directions

In conclusion, our work has successfully bridged the gap between continuous- and discrete-time schemes by establishing the convergence of discrete-time algorithms to their continuous-time counterparts, drawing inspiration from dynamical system theory. This achievement offers a flexible framework that can accommodate practical schemes and unify tasks from various domains, including machine learning, game theory, and physics. By introducing a comprehensive framework for analyzing stochastic approximation schemes, providing convergence proofs, and presenting easily verifiable conditions at the finite particle level, our contributions enhance the understanding and application of stochastic approximation schemes for simulating interacting particle systems.

In our future works, we will explore the exciting possibilities offered by the dynamical system theory of [4] to derive convergence rates through 𝜆 -pseudotrajectories. This avenue of research will allow us to establish exponential convergence for dynamics using smaller step-sizes after a 'burn-in' time, thereby sharpening our understanding for the long-term behavior of these practical schemes.

Additionally, we aim to relax the Lipschitzness assumption on the drift and expand the scope of guaranteed algorithms, such as the Ensemble Kalman Sampler [16]. We also aim to explore further applications in natural sciences, including stochastic mean-field FitzHugh-Nagumo models and networks of Hodgkin-Huxley neurons [1]. These models are important because they provide a mathematical description of neuronal dynamics, contribute to our understanding of neurological disorders, and inform the development of brain-computer interfaces. By leveraging our novel framework, we hope to offer rigorous guarantees for the algorithms employed in these domains while also designing new, efficient approaches that can support computational neuroscience research.

Acknowledgments and Disclosure of Funding

This work was supported by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program grant agreement No 815943. YPH acknowledges funding through an ETH Foundations of Data Science (ETH-FDS) postdoctoral fellowship.

References

Javier Baladron, Diego Fasoli, Olivier Faugeras, and Jonathan Touboul. Mean Field description of and propagation of chaos in recurrent multipopulation networks of Hodgkin-Huxley and Fitzhugh-Nagumo neurons, October 2011.

Jianhai Bao, Christoph Reisinger, Panpan Ren, and Wolfgang Stockinger. First-order convergence of milstein schemes for McKean-vlasov equations and interacting particle systems. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences , 477 (2245), January 2021. doi: 10.1098/rspa.2020.0258. URL https://doi.org/10.1098/ rspa.2020.0258 .

Michel Benaïm. Dynamics of stochastic approximation algorithms. In Seminaire de probabilites XXXIII , pages 1-68. Springer, 2006.

Michel Benaïm and Morris W Hirsch. Asymptotic pseudotrajectories and chain recurrent flows, with applications. Journal of Dynamics and Differential Equations , 8(1):141-176, 1996.

Dario Benedetto, Emanuelle Caglioti, José A Carrillo, and Mario Pulvirenti. A non-maxwellian steady distribution for one-dimensional granular media. Journal of statistical physics , 91: 979-990, 1998.

Lev M Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR computational mathematics and mathematical physics , 7(3):200-217, 1967.

Patrick Cattiaux, Guillin Arnaud, and Malrieu Florent. Probabilistic approach for granular media equations in the non uniformly convex case, March 2006.

Louis-Pierre Chaintron and Antoine Diez. Propagation of chaos: a review of models, methods and applications. i. models and methods. Kinetic and Related Models , 2022. doi: 10.3934/krm. 2022017.

Louis-Pierre Chaintron and Antoine Diez. Propagation of chaos: A review of models, methods and applications. II. Applications. Kinetic and Related Models , 15(6):1017, 2022. ISSN 1937-5093, 1937-5077. doi: 10.3934/krm.2022018.

Sinho Chewi, Thibaut Le Gouic, Chen Lu, Tyler Maunu, and Philippe Rigollet. Svgd as a kernelized wasserstein gradient flow of the chi-squared divergence. Advances in Neural Information Processing Systems , 33:2098-2109, 2020.

Lénaïc Chizat. Mean-field langevin dynamics: Exponential convergence and annealing. Transactions on Machine Learning Research , 2022.

Lénaïc Chizat and Francis Bach. On the Global Convergence of Gradient Descent for Overparameterized Models using Optimal Transport. In Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018.

Carles Domingo-Enrich, Samy Jelassi, Arthur Mensch, Grant Rotskoff, and Joan Bruna. A mean-field analysis of two-player zero-sum games, May 2021.

Andrew Duncan, Nikolas Nüsken, and Lukasz Szpruch. On the geometry of stein variational gradient descent. arXiv preprint arXiv:1912.00894 , 2019.

Alain Durmus, Andreas Eberle, Arnaud Guillin, and Raphael Zimmer. An elementary approach to uniform in time propagation of chaos. Proceedings of the American Mathematical Society , 148(12):5387-5398, 2020.

Alfredo Garbuno-Inigo, Franca Hoffmann, Wuchen Li, and Andrew M Stuart. Interacting langevin diffusions: Gradient structure and ensemble kalman sampler. SIAM Journal on Applied Dynamical Systems , 19(1):412-441, 2020.

Irving L Glicksberg. A further generalization of the kakutani fixed theorem, with application to nash equilibrium points. Proceedings of the American Mathematical Society , 3(1):170-174, 1952.

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM , 63(11):139-144, 2020.

Jack K. Hale. Asymptotic Behavior of Dissipative Systems . American Mathematical Society, 1988.

Ya-Ping Hsieh, Chen Liu, and Volkan Cevher. Finding mixed nash equilibria of generative adversarial networks. In International Conference on Machine Learning , pages 2810-2819, 2019.

Ya-Ping Hsieh, Panayotis Mertikopoulos, and Volkan Cevher. The limits of min-max optimization algorithms: Convergence to spurious non-critical sets. In International Conference on Machine Learning , pages 4337-4348. PMLR, 2021.

M. Kac. Foundations of Kinetic Theory. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Volume 3: Contributions to Astronomy and Physics , volume 3.3, pages 171-198. University of California Press, January 1956.

Mohammad Reza Karimi, Ya-Ping Hsieh, and Andreas Krause. A dynamical system view of langevin-based non-convex sampling. arXiv preprint arXiv:2210.13867 , 2022.

Mohammad Reza Karimi, Ya-Ping Hsieh, Panayotis Mertikopoulos, and Andreas Krause. The dynamics of Riemannian Robbins-Monro algorithms. In COLT 2022-35th Annual Conference on Learning Theory , pages 1-31, 2022.

Anna Korba, Adil Salim, Michael Arbel, Giulia Luise, and Arthur Gretton. A non-asymptotic analysis for stein variational gradient descent. Advances in Neural Information Processing Systems , 33:4672-4682, 2020.

Daniel Lacker. Mean field games and interacting particle systems. preprint , 2018.

Daniel Lacker and Luc Le Flem. Sharp uniform-in-time propagation of chaos. Probability Theory and Related Fields , February 2023. ISSN 1432-2064. doi: 10.1007/s00440-023-01192-x.

Gunther Leobacher, Christoph Reisinger, and Wolfgang Stockinger. Well-posedness and numerical schemes for one-dimensional McKean-vlasov equations and interacting particle systems with discontinuous drift. BIT Numerical Mathematics , 62(4):1505-1549, May 2022. doi: 10.1007/s10543-022-00920-4. URL https://doi.org/10.1007/s10543-022-00920-4 .

Qiang Liu. Stein Variational Gradient Descent as Gradient Flow, November 2017.

Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural information processing systems , 29, 2016.

Qiang Liu and Dilin Wang. Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm, September 2019.

Wei Liu, Liming Wu, and Chaoen Zhang. Long-Time Behaviors of Mean-Field Interacting Particle Systems Related to McKean-Vlasov Equations. Communications in Mathematical Physics , 387(1):179-214, October 2021. ISSN 1432-0916. doi: 10.1007/s00220-021-04198-5.

Jianfeng Lu, Yulong Lu, and James Nolen. Scaling limit of the stein variational gradient descent: The mean field regime. SIAM Journal on Mathematical Analysis , 51(2):648-671, 2019.

Yulong Lu. Two-Scale Gradient Descent Ascent Dynamics Finds Mixed Nash Equilibria of Continuous Games: A Mean-Field Perspective, January 2023.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017.

Song Mei, Andrea Montanari, and Phan-Minh Nguyen. A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences , 115(33): E7665-E7671, 2018.

Song Mei, Theodor Misiakiewicz, and Andrea Montanari. Mean-field theory of two-layers neural networks: Dimension-free bounds and kernel limit, February 2019.

Atsushi Nitanda, Denny Wu, and Taiji Suzuki. Convex Analysis of the Mean Field Langevin Dynamics. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics , pages 9741-9757. PMLR, May 2022.

Felix Otto. The geometry of dissipative evolution equations: the porous medium equation. Communications in Partial Differential Equations , 26(1-2):101-174, 2001. doi: 10.1081/ PDE-100002243.

Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning , 11(5-6):355-607, 2019.

Christoph Reisinger and Wolfgang Stockinger. An adaptive euler-maruyama scheme for McKean-vlasov SDEs with super-linear growth and application to the mean-field FitzHugh-nagumo model. Journal of Computational and Applied Mathematics , 400:113725, January 2022. doi: 10.1016/j.cam.2021.113725. URL https://doi.org/10.1016/j.cam.2021.113725 .

Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics , pages 400-407, 1951.

Adil Salim, Lukang Sun, and Peter Richtarik. A convergence theory for svgd in the population limit under talagrand's inequality t1. In International Conference on Machine Learning , pages 19139-19152. PMLR, 2022.

Jiaxin Shi and Lester Mackey. A finite-particle convergence rate for stein variational gradient descent. CoRR , abs/2211.09721, 2022. doi: 10.48550/arXiv.2211.09721. URL https: //doi.org/10.48550/arXiv.2211.09721 .

Lukang Sun, Avetik Karagulyan, and Peter Richtarik. Convergence of stein variational gradient descent under a weaker smoothness condition. In International Conference on Artificial Intelligence and Statistics , pages 3693-3717. PMLR, 2023.

Taiji Suzuki, Atsushi Nitanda, and Denny Wu. Uniform-in-time propagation of chaos for the mean-field gradient Langevin dynamics. In ICLR , 2023.

Alain-Sol Sznitman. Topics in propagation of chaos. In Donald L. Burkholder, Etienne Pardoux, Alain-Sol Sznitman, and Paul-Louis Hennequin, editors, Ecole d'Eté de Probabilités de SaintFlour XIX - 1989 , Lecture Notes in Mathematics, pages 165-251, Berlin, Heidelberg, 1991. Springer. ISBN 978-3-540-46319-1. doi: 10.1007/BFb0085169.

Juan Luis Vázquez. The porous medium equation: mathematical theory . Oxford University Press on Demand, 2007.

Cédric Villani. Topics in optimal transportation . Number 58. American Mathematical Soc., 2003.

Cédric Villani. Optimal transport: old and new , volume 338. Springer Science & Business Media, 2008.

Blake Woodworth, Suriya Gunasekar, Jason D Lee, Edward Moroshko, Pedro Savarese, Itay Golan, Daniel Soudry, and Nathan Srebro. Kernel and rich regimes in overparametrized models. In Conference on Learning Theory , pages 3635-3673. PMLR, 2020.

Rentian Yao, Xiaohui Chen, and Yun Yang. Mean-field nonparametric estimation of interacting particle systems. In Conference on Learning Theory , pages 2242-2275. PMLR, 2022.

Jianyi Zhang, Ruiyi Zhang, Lawrence Carin, and Changyou Chen. Stochastic ParticleOptimization Sampling and the Non-Asymptotic Convergence Theory. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , pages 18771887. PMLR, June 2020.

Kelvin Shuangjian Zhang, Gabriel Peyré, Jalal Fadili, and Marcelo Pereyra. Wasserstein Control of Mirror Langevin Monte Carlo. arXiv:2002.04363 [cs, math, stat] , February 2020.

A Properties of Mean-field Transfer to Particle System

LemmaA.1. The Assumptions 1-3 transfer seamlessly to the aggregated notions of drift and diffusion given in (7) :

If 𝑏 ' · GLYPH<150> · ' and 𝜎 ' · GLYPH<150> · ' satisfy Assumption 1, then 𝒃 ' · ' and 𝝈 ' · ' are 𝐿 ' p 𝑁 , 1 ' -Lipschitz.

If 𝑏 ' · GLYPH<150> · ' satisfies Assumption 2, then 𝒃 ' · ' satisfies the same condition with the constant p 𝑁𝐶 𝑣 .

If 𝜎 ' · GLYPH<150> · ' satisfies Assumption 3, then the same holds for 𝝈 ' · ' with constant p 𝑁𝐾 .

Proof. We prove these statements separately:

1. Let 𝑥GLYPH<150> 𝑦 2 ' ℝ 𝑑 ' GLYPH<10> 𝑁 and define 𝑎 𝑖 = j 𝑥 𝑖 GLYPH<0> 𝑦 𝑖 j . First, notice that W 2 2 ' 𝜇 𝑥 GLYPH<150> 𝜇 𝑦 ' GLYPH<20> 1 𝑁 P 𝑎 2 𝑖 , as the average on the right-hand-side corresponds to the specific coupling of 𝑥 𝑖 $ 𝑦 𝑖 . Now, observe that

Let a = ' 𝑎 1 GLYPH<150> GLYPH<149> GLYPH<149> GLYPH<149> GLYPH<150> 𝑎 𝑁 ' , and notice that the last quantity above is equal to

This means that

For the diffusion, it suffices to notice that

The rest of the proof is similar to the one for the drift.

2. We have

where in the last inequality, we used Cauchy-Schwarz. This implies h 𝑥GLYPH<150> 𝒃 ' 𝑥 'i GLYPH<20> 𝐶 𝑣 p 𝑁 'j 𝑥 j , 1 ' .

3. It is easy to see that

Lemma A.2. If 𝑏 ' · GLYPH<150> · ' is ' 𝛼GLYPH<150> 𝛽 ' -dissipative on average, then 𝒃 ' · ' is ' 𝛼GLYPH<150>𝑁𝛽 ' -dissipative in the usual sense, that is, for all 𝑥 2 ' ℝ 𝑑 ' GLYPH<10> 𝑁 , h 𝑥GLYPH<150> 𝒃 ' 𝑥 'i GLYPH<20> GLYPH<0> 𝛼 j 𝑥 j 2 , 𝑁𝛽 .

Proof. Observe that for 𝑥 2 ' ℝ 𝑑 ' GLYPH<10> 𝑁 we have

This means that h 𝑥GLYPH<150> 𝒃 ' 𝑥 'i GLYPH<20> GLYPH<0> 𝛼 j 𝑥 j 2 , 𝑁𝛽 .

GLYPH<4>

B The Main Theorem

B.1 Proof of Theorem 1

Recall the Picard process (Picard):

We break down the proof into four steps: first, we prove that the Picard process is close to the flow (Flow), and then we bound the distance between the Picard process and the interpolation (Int). We then conclude the proof of the WAPT property. Lastly, we prove that stability is implied by dissipativity.

§ Distance of Picard from Flow.

where we used Lipschitzness of 𝒃 and 𝝈 (implied by Assumption 1 and Lemma A.1), Itô's isometry (see, e.g., [54, Lemma 3.4]), and Lemma A.1.

§ Distance of Picard to Interpolation. We place a bar above a symbol to denotes its piecewise constant interpolation.

where Δ 𝑃 ' 𝑡GLYPH<150> 𝑠 ' is the accumulated noise and bias from time 𝑡 to time 𝑡 , 𝑠 , which is equal to

with 𝑛 = 𝑚 ' 𝑡 , 𝑠 ' and 𝑘 = 𝑚 ' 𝑡 ' . It is shown in [23] that lim 𝑡 !1 𝔼 GLYPH<12> GLYPH<12> Δ 𝑃 ' 𝑡GLYPH<150> 𝑠 ' GLYPH<12> GLYPH<12> 2 = 0 , a.s. Continuing to bound the inside of the integral, we have

where we used the fact that conditional expectation is a contraction in 𝐿 2 , and

Moreover, by Assumption 3 we have 𝔼 tr ' 𝝈 ' 𝑥 𝑘 ' > 𝝈 ' 𝑥 𝑘 '' = O ' 1 ' . We thus get by Lemma B.1

This implies

with 𝐴 𝑡 ! 0 as 𝑡 !1 , a.s.

§ Concluding the proof of APT. By Grönwall inequality,

as 𝑡 !1 . Since we get the desired result.

§ Stability. Lemma A.2 implies that under dissipativity on average, the iterates are stable, and as in [23, Theorem 3], we get the desired convergence result.

B.2 On Propagation of Chaos

Theorem 1 shows that the law 𝜇 𝑘 of 𝑥 𝑘 2 ' ℝ 𝑑 ' GLYPH<10> 𝑁 converges in the Wasserstein space to the limit-set (or the internally chain-transitive (ICT) set) 𝑆 GLYPH<26> 𝒫 2 '' ℝ 𝑑 ' GLYPH<10> 𝑁 ' of the corresponding flow:

By looking only at the first particle of 𝑥 𝑘 , namely, 𝑥 1 𝑘 , and given that the dynamics is exchangeable, it follows that

where 𝜋 1 ' 𝑑𝑥 1 GLYPH<150> 𝑑𝑦 1 ' = ∫ 𝜋 ' 𝑥GLYPH<150> 𝑦 ' 𝑑𝑥 2 𝑑𝑦 2 GLYPH<1> GLYPH<1> GLYPH<1> 𝑑𝑥 𝑁 𝑑𝑦 𝑁 , and we used the exchangability in deducing that the law of 𝑦 𝑖 are the same as the first marginal of 𝜇 , for all 𝑖 = 1 GLYPH<150> GLYPH<149> GLYPH<149> GLYPH<149> GLYPH<150>𝑁 . Hence, as the limit-set 𝑆 0 of the first component of the SDE 𝑋 1 𝑡 is a subset of the marginal of 𝑆 ,

This means that the first particle converges in law to the ICT sets of the corresponding SDE. Assuming a uniform propagation of chaos, we also know that the law of 𝑋 1 𝑡 has a distance of 𝑂 ' 1 GLYPH<157> 𝑁 ' from the mean-field equation, and hence, we get that the law of the particles following the discrete algorithm have controllable distance from the mean-field dynamics.

B.3 Supporting Lemmas

Lemma B.1. Suppose Assumptions 1-5 hold. One has 𝔼 j 𝒃 ' 𝑥 𝑘 'j 2 = O ' 1 GLYPH<157> 𝛾 𝑘 , 1 ' , 𝔼 j 𝜀 𝑘 , 1 j 2 = O ' 𝛾 𝑘 , 1 ' , and 𝔼 j 𝑃 𝑘 , 1 j 2 = O ' 1 ' .

Proof. We repeatedly use the fact that 𝔼 j 𝒃 ' 𝑥 𝑘 'j 2 GLYPH<20> 2 𝐿 2 𝔼 j 𝑥 𝑘 j 2 , 𝔼 j 𝒃 ' 𝑥 0 'j 2 C 2 𝐿 2 𝔼 j 𝑥 𝑘 j 2 , 𝐶 0 . By Assumption 4, 𝔼 j 𝜀 𝑘 , 1 j 2 GLYPH<20> O ' 𝛾 2 𝑘 , 1 ' 𝑎 𝑘 , O ' 𝛾 𝑘 , 1 ' , and we have

Moreover, as p 𝑝 , 𝑞 GLYPH<20> p 𝑝 , p 𝑞 , we have

Assumption 3 also implies that 𝔼 j 𝝈 ' 𝑥 𝑘 ' 𝜉 𝑘 , 1 j 2 GLYPH<20> 𝐶 𝜎 .

Define 𝑎 𝑘 B 𝔼 j 𝑥 𝑘 j 2 . Then,

Plugging the bounds from (B.2) and (B.3) into (B.4) gives

for some 𝑃GLYPH<150> 𝑄GLYPH<150> 𝑅 ¡ 0 that do not depend on 𝑘 .

We now prove 𝑎 𝑘 GLYPH<20> 𝑀 GLYPH<157> 𝛾 𝑘 , 1 for some fixed 𝑀 ¡ 0 via induction. Suppose this is the case for 𝑘 . For 𝑘 , 1 we have

The last inequality is equivalent to the fact that the following quadratic equation (in p 𝑀 ) has a bounded largest root (and the bound shall not depend on 𝑘 ):

Notice that by Assumption 5, the leading coefficient is negative, and the larger root is computed as

The claim for 𝒃 follows by Lipschitzness, from which the claim for the bias and perturbation follows directly. GLYPH<4>

C Proofs of Results for Applications

C.1 Two-Layer Neural Networks and Mean-field Langevin

Proof of Corollary 1. Smoothness of drift: We start by showing Lipschitzness with respect to the measure parameter of the drift. First, observe that r 𝜃 𝑊 ' 𝜃GLYPH<150> · ' is Lipschitz:

due to the boundedness of 𝜅 0 and 𝑧 , and Lipschitzness of 𝜅 .

Now, consider 𝜇GLYPH<150> 𝜈 2 𝒫 2 ' ℝ 𝑑 ' and let 𝜋 be the optimal coupling (in W 2 sense) between them. Then, for a fixed 𝜃 2 ℝ 𝑑 ,

Next, we show for a fixed measure 𝜇 2 𝒫 2 ' ℝ 𝑑 ' , 𝑏 ' · GLYPH<150> 𝜇 ' is Lipschitz in the first input.

Let us treat each term separately. We have

Similarly,

Thus,

showing 𝑏 satisfies Assumption 1.

Growth control: First, let us calculate

As 𝜑 , 𝜅 0 , and supp ' D ' are bounded, we can see that

thus, satisfying Assumption 2.

Dissipativity on average: Here we use the extra assumption that j 𝑎 𝜅 0 ' 𝑎 'j is bounded. We directly bound the terms 𝜅 0 'h 𝜃GLYPH<150> 𝑧 i'h 𝜃GLYPH<150> 𝑧 i above and obtain

C.2 Stein Variational Gradient Descent

Proof of Corollary 2. While the first term in the drift is standard to work with (see Section 4.4), it is the second term in the drift that makes it difficult to analyze. Specifically, we prove the dissipativity on average only for empirical measures. While this would be enough for our purposes (and Theorem 1 goes through), it is an interesting future direction to see when does dissipativity hold in a more general setup. Moreover, for simplicity, we only consider the case where the kernel 𝐾 is of the form 𝐾 ' 𝑥GLYPH<150> 𝑦 ' = ℎ ' 𝑥 GLYPH<0> 𝑦 ' , for some function ℎ .

Below, we first prove that 𝑏 is dissipative on average, which implies that the law of the iterates will be in a compact subset of 𝒫 2 ' ℝ 𝑑 ' . Then, we show that 𝑏 is smooth on this compact subset.

Dissipativity on average: Due to 𝐾 being symmetric, r 2 𝐾 ' 𝑥GLYPH<150> 𝑦 ' = GLYPH<0>r 2 𝐾 ' 𝑦GLYPH<150> 𝑥 ' . We thus have

Thus,

by Cauchy-Schwarz and the assumption that jr 2 𝐾 ' 𝑥GLYPH<150> 𝑦 'j GLYPH<20> 𝜂 GLYPH<157> j 𝑥 GLYPH<0> 𝑦 j . With similar arguments, and using dissipativity of 𝑉 , we have

As 𝐾 ' 𝑥GLYPH<150> 𝑥 ' = ℎ ' 0 ' and 𝐾 ' 𝑥GLYPH<150> 𝑦 ' ¡ 0 for all 𝑥GLYPH<150> 𝑦 , and that 𝜇 is an empirical measure 𝜇 = 1 𝑁 P 𝑁 𝑖 = 1 𝛿 𝑥 𝑖 , the last quantity is equal to

In total, we derive that 𝑏 is dissipative on average.

Smoothness of the drift: We have, for 𝜇 in a compact set of 𝒫 2 ' ℝ 𝑑 '

Moreover, take 𝜇GLYPH<150> 𝜈 in the same compact set, and let 𝜋 be the optimal coupling (in W 2 sense). Then,

C.3 Two-player Zero-sum Continuous Games

Proof of Corollary 3. Recall that

Smoothness of drift: For a fixed 𝜇 2 𝒫 2 ' ℝ 𝑑 ' ℝ 2 𝑑 and 𝑞GLYPH<150> 𝑟 2 ℝ 2 𝑑 we have

where we used 𝐿 -Lipschitzness of r 𝑥 𝐾 and r 𝑦 𝐾 .

Now, for a fixed 𝑞 2 ℝ 2 𝑑 , and 𝜇GLYPH<150> 𝜈 2 𝒫 2 ' ℝ 𝑑 ' ℝ 2 𝑑 with optimal coupling 𝜋 , we have

Average dissipativity of drift: Suppose r 𝑥 𝐾 and GLYPH<0>r 𝑦 𝐾 are ' 𝑎GLYPH<150> 𝛽 ' -dissipative. Then

implying that 𝑏 ' · GLYPH<150> · ' is ' 𝑎𝛼GLYPH<150> 2 𝛽 ' -dissipative on average.

If, on the other hand, the domains X and Y are bounded, observe that by Cauchy-Schwarz

where 𝑀 = sup 𝑞 1 2 X GLYPH<150>𝑞 2 2 Y j 𝑞 1 j jr 𝑥 𝐾 ' 𝑞 1 GLYPH<150> 𝑞 0 2 'j , 𝛼 j 𝑞 2 j jr 𝑦 𝐾 ' 𝑞 0 1 GLYPH<150> 𝑞 2 'j . Also denoting by 𝑅 = sup 𝑞 2 X GLYPH<2> Y j 𝑞 j 2 , we see that for any 𝛼 ¡ 0 , 𝑏 ' · GLYPH<150> · ' is ' 𝛼GLYPH<150>𝑀 , 𝛼𝑁 ' -dissipative on average, as

Optimistic algorithm fits Assumption 4 : Recall the iterates

where Ξ 𝑖 𝑘 , 1 = ' 𝜉 𝑖 𝑘 , 1 GLYPH<150> 𝜁 𝑖 𝑘 , 1 ' . Notice that the bias of this iteration is

For brevity, let us write ℱ 𝑘 B ℱ 𝜏 𝑘 . We have

Moreover, we have

Combining the last two inequalities, we have

Since 𝛾 𝑘 , 1 ! 0 , we can assume that 12 𝛾 2 𝑘 , 1 GLYPH<20> 1 GLYPH<157> 2 , which implies

which is exactly what we are after.

C.4 Kinetic Equations

Proof of Corollary 4. Smoothness of the drift: Let 𝑥GLYPH<150> 𝑦 2 ℝ 𝑑 and 𝜇GLYPH<150> 𝜈 2 𝒫 2 ' ℝ 𝑑 ' and set 𝜋 be an optimal coupling between 𝜇 and 𝜈 (in 𝑊 1 sense). Then

By 𝐿 -Lipschitzness of r 𝑉 , the first term is bounded by 𝐿 j 𝑥 GLYPH<0> 𝑦 j . For the second term, using the coupling, we can write it as

Putting these together we get

Average dissipativity of the drift: First we show that for 𝑥 2 ℝ 𝑑 and a probability measure 𝜇 , we have

This holds, since

where in the penultimate inequality we used the assumption (which implies hr 𝑊 ' 𝑥 ' GLYPH<150> 𝑥 i GLYPH<21> GLYPH<0> 𝑀 𝑊 ), and in the last one, we used the that 𝑊 is symmetric (which implies r 𝑊 'GLYPH<0> 𝑧 ' = GLYPH<0>r 𝑊 ' 𝑧 ' ), and used Fubini's theorem to exchange integrals. Bringing the last term to the left and dividing by 2 shows (C.1).

GLYPH<4>

To show average dissipativity, it suffices to observe

Proximal algorithm fits Assumption 4 : Note that this implicit algorithm corresponds to the following proximal step

By defining the perturbation as

we see that the algorithm (Kin-Prox) fits the template (SAA). For brevity, let us write ℱ 𝑘 B ℱ 𝜏 𝑘 . We only have to show that

We have

For brevity, let

noticing that 𝜀 𝑖 𝑘 , 1 = 𝑓 ' 𝑥 𝑖 𝑘 , 1 ' GLYPH<0> 𝑓 ' 𝑥 𝑖 𝑘 ' . By the update rule (Kin-Prox)

Moreover, we have that j 𝑓 ' 𝑥 𝑖 𝑘 , 1 'j 2 GLYPH<20> 2 j 𝑓 ' 𝑥 𝑖 𝑘 , 1 ' GLYPH<0> 𝑓 ' 𝑥 𝑖 𝑘 'j 2 , 2 j 𝑓 ' 𝑥 𝑖 𝑘 'j 2 . Since 𝛾 𝑘 , 1 ! 0 , we can assume that 16 𝐿 2 𝛾 2 𝑘 , 1 GLYPH<159> 1 2 . All in all, this gives

This implies that

Summing over 𝑖 and observing that P j 𝑓 ' 𝑥 𝑖 'j = j 𝒃 ' 𝑥 𝑘 'j

𝑘 2 2 concludes the proof. GLYPH<4>

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper analyses discretisations of mean-field type SDEs arising in several areas of machine learning. The main contribution is a convergence result (Theorem 1) stating that under appropriate conditions on the drift and diffusion coefficients, the discretised dynamics convergence in 2-Wasserstein distance, in an infinite time horizon limit, to the continuous-time system of interacting particles. Using the classical uniform propagation of chaos for interacting particle systems to show convergence to the mean field dynamics. Applications to Two-Layer NNs training by SGD, Stein Variational Gradient Descent, Two-Player Zero-sum Continuous Games and Kinetic Equations are presented.\n\nSTRENGTHS:\nAs outlined by the authors, SDEs of mean-field type arise in several areas of machine learning and statistics as continuous-time and infinite-number-of-particle limits of discrete stochastic difference equations. Therefore, a convergence analysis of the discrete schemes to the their continuous-time counterparts is of high relevance and importance within the modern machine learning landscape. The paper is well-written and clear, its structure is easy to follow. The four examples on Two-Layer NNs, Stein Variational Gradient Descent, Two-Player Zero-sum Continuous Games and on Kinetic Equations nicely demonstrate the applicability of the main result (Theorem 1) to several topics/areas of modern machine learning.\n\nWEAKNESSES:\nThere is a rich body of literature on SDE discretisation scheme for McKean-Vlasov SDEs and interacting particle systems [1, 2, 3], that is completely ignored by the authors. The results of these papers concern convergence of Euler-Marayama and/or Milstein type numerical schemes to the limiting mean-field equations when the step size of the solver goes to zero and the number of particles goes to infinity. As far as I see, the main difference in the anlyses is in the notion of convergence in time: the authors consider as convergence criterion the *Wasserstein asymptotic pseudotrajectory* (WAPT), which is a large time behaviour from dynamical systems theory, while in the aforementioned series of paper the convergence is in terms of discretisation step, which is more classical in (numerical) stochastic analysis. Albeit the two notions of convergences are different, I think an in-depth discussion and comparison between the two is required.\n\nI invite the authors to initiate a conversation on the above during the rebuttal period. If a rigorous and fair comparison/discussion is eventually presented, I will happily increase my rating.\n\n**References**\n\n[1] Bao, Jianhai, et al. \"First-order convergence of Milstein schemes for McKean–Vlasov equations and interacting particle systems.\" Proceedings of the Royal Society A 477.2245 (2021): 20200258.\n\n[2] Reisinger, Christoph, and Wolfgang Stockinger. \"An adaptive Euler–Maruyama scheme for McKean–Vlasov SDEs with super-linear growth and application to the mean-field FitzHugh–Nagumo model.\" Journal of Computational and Applied Mathematics 400 (2022): 113725.\n\n[3] Leobacher, Gunther, Christoph Reisinger, and Wolfgang Stockinger. \"Well-posedness and numerical schemes for one-dimensional McKean–Vlasov equations and interacting particle systems with discontinuous drift.\" BIT Numerical Mathematics 62.4 (2022): 1505-1549.\n\nQUESTIONS:\n- Can the authors comment on the choice of 2-Wasserstein distance in the WAPT convergence criterion?\n- The authors make repetitive use of the notion of a *limit set*; albeit this is classical in the theory of dynamical systems, it is worth defining it in the main body of the paper.\n- The paper would benefit from the addition of an *idea-of-proof* paragraph following Theorem 1 (still defering the details to the appendix).\n- (This is not necessarily a question targeted to this paper, but more generally to the community working at the intersection between NNs and mean-field dynamics) What are the issues in analysing more-than-two layers feedforward NNs using the language of mean-field SDEs? What about more general architectures such ResNets, RNNs etc. ?\n\nLIMITATIONS:\nThe authors adequately addressed limitations of their contribution and discussed future work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

comments: SUMMARY:
This paper develops a theoretical mathematical framework to characterize the convergence properties of discrete particle systems to their mean-field limit.

STRENGTHS:
The mathematical theory in this paper is beyond my scope, but it appears to be mathematically sound. The paper is well-written.

WEAKNESSES:
I believe that this paper would benefit from some applied tests/results to show practical relevance. For example, how would it help training a GAN? Do the theoretical convergence results help actual training? How do the results compare to actual training? Are the bounds tight relative to actual convergence?

QUESTIONS:
I think it would be helpful to provide some examples of how these these theoretical results can help inform and guide ML development, etc. For example, how can I use these convergence guarantees as a design a NN?

LIMITATIONS:
Limitations do not seem to be explicitly addressed by the authors.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work considers the convergence of discrete time interacting particle systems to their respective continuous time limits (i.e, McKean-Vlasov type equations) under general assumptions which are applicable to varied contexts like neural networks, kinetic theory, game theory and sampling algorithms.  The finite particle + finite step size algorithms are considered to be stochastic approximations of the mean field limit and convergence is analyzed in terms of dynamical systems theory.  This work considers the notion of weak asymptotic pseudo-trajectory to show that the stochastic approximations are close to mean field limit under general conditions. These are then applied to various specific contexts like neural networks and SVGD to derive convergence bounds.\n\nSTRENGTHS:\nThe generality of the assumptions and the framework is the main contribution of this work. This enables the authors to derive several useful results in varied domains under a common framework. This can be a useful tool in establishing convergence to mean field limits in new problems without requiring elementary analysis.\n\nWEAKNESSES:\n1. The notation in the algorithmic template is extremely confusing. Shortening $b(x,\\mu)$ to just $b(x)$ makes it very confusing. Population level SA is also a bad terminology since it confuses the reader about whether this is the mean-field limit (i.e, $n \\to \\infty$) or not (because the mean field limit is often referred to as the population limit).\n\n2. WAPT as the notion of convergence requires more justification. This is so since the continuous process begins at $X_t$, the $t$-th time instant of the discrete time process and the uniform convergence is established as $t \\to \\infty$. What if the initial deviation in the stochastic approximation ensures that $X_t$ itself is not likely to be reached by PSDE ? \n\n3. Assumption 5 is a bit non-standard. Also, I think there is a typo here. Assumption (11) is not satisfied for any decreasing step size sequence since $\\gamma_{k+1}/\\gamma_{k+2} > 1$. Please clarify and state what exact step sizes are allowed.\n\n4. Under specific settings, much stronger results can be derived for convergence when the algorithm is designed specially or under specific assumptions like logarithmic sobolev inequalities (even with finite particles and constant step sizes). This framework precludes such analyses.  (See [A1,A2]). I am not very well versed in the game theory or kinetic theory literature, so I will abstain from commenting on these results. \n\n[A1]  Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction.\n\n[A2]  Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation\n\nQUESTIONS:\nPlease answer the questions posed in the weaknesses section.\n\nLIMITATIONS:\nThe authors have a good discussion on the applicability of their work. I think they should discuss the drawbacks of WAPT better.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper fills a theoretical gap between the application of ideas interacting particle systems to algorithms in machine learning--algorithms, like SGVD, that are almost always realized as discrete-time routines with a finite number of particles--and the substantial existing body of theoretical work on finite particle systems with continuous dynamics. These latter works have yielded valuable insights about, e.g., the training process of two-layer neural networks, algorithm design for approximate Bayesian inference, or the nature of equilibria in games. However, they have not rigorously established the convergence of discrete-time to continuous. This paper establishes that convergence via Benaïm and Hirsch's notion of a Wasserstein asymptotic pseudo-trajectory (WAPT), which gives a measure of asymptotic closeness (in the Wasserstein-2 sense) between two stochastic processes. Specifically, via Theorem 1, the convergence of the family of (discrete-time) stochastic approximation algorithms (SAA) is reduced to its continuous time counterpart. Combining Theorem 1 with existing results in the literature yields the conclusion that the empirical distribution of particles following the discrete-time SAA converges to the mean-field solution, as desired.\n\nSTRENGTHS:\nThe central originality of this paper lies in its adaptation of WAPT to solve an open problem in the mean-field theory of discrete-time IPS in machine learning.\nBroadly, I found the clarity and elegance of the mathematical exposition to be exceptionally good. The paper persuasively argues for the importance of a rigorous theory of convergence, and smoothly introduces concepts and definitions needed for understanding Theorem 1, while re-orienting the reader by summarizing previous results at effective moments. After stating the Theorem, the applications of the theory to two-layer NNs, SGVD, games, and kinetic equations were clear and enlightening.\nThe significance of the presented comprehensive framework is high, as the future directions section makes clear.\n\nWEAKNESSES:\nI found the paper without major weaknesses. In terms of the overall presentation of results, I was surprised to see interacting particle systems (IPS) as the frame for this theory rather than simply continuous-time Markov jump processes. I can see the value in specializing to interacting particle systems, but some readers may be deeply acquainted with continuous Markov jump processes and be largely unaware of the IPS literature. Bringing that connection onto the screen by mentioning it in the technical background may help orient readers with a more general stochastic process background.\n\nIn the same vein, a brief mention of the relationship to multi-agent systems could be of value in ensuring that this work reaches the wide readership for which its theory is relevant.\n\nQUESTIONS:\n- For readers unfamiliar with pseudo-trajectories from the dynamical systems literature, a brief footnote or aside giving a terse formal definition may be helpful. The intuitive explanation is mostly clear--the requirement than the orbit X(t) closely tracks the flow over arbitrarily long time intervals T with arbitrary precision--but a formal statement that uses a constant like \\lambda (as I see in line 352!) would likely not strain the reader too much.\n- It may be valuable, in the future directions section or in an appendix, to discuss what kinds of algorithms do not fall under the algorithmic template of the SAA family, as a way to highlight other gaps in the existing mean-field theory.\n\nLIMITATIONS:\nThe authors have adequately address the limitations of their work. I see no potential negative societal impacts.", "TITLE": "Official Review", "RECOMMENDATION": 9, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-13"}