Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a uniform-in-time analysis of the propagation of chaos for the mean-field Langevin dynamics to conduct neural network optimization. The authors avoid the double-loop structure and enable to deal with convergence guarantee based on finite-width neural network with vanilla noisy gradient descent algorithms.
strength_and_weaknesses: **Pros:**
1. The first quantitative discretization error ensures analysis based on neural networks of finite/ limited neurons. The uniform nature ensures that the deviation of the output reduces rapidly.
2. The analysis leverages the advantages of propagation of chaos and overcomes the bottleneck suffered by the Growall inequality in Mei'18 and the error bound remains stable when the time t is large.

**Cons:**
The assumption of the super-quadratic tail of the regularization term is strong indeed.


clarity,_quality,_novelty_and_reproducibility: The authors clearly stated the contributions and limitations of the proof. 

Novelty: I am not an expert in neural network theory and I am not able to evaluate the novelty.

The technical tools based on gradient flow stuff seem standard and reasonable.

Typos: $\mu_t$ decreases $\mathcal{L}(\mu_t)$ unless $\frac{\delta \mathcal{L}(\mu)}{\delta\mu}=0$?
summary_of_the_review: Convergence analysis for stochastic gradient descent on a finite-width neural network where the upper bound is table w.r.t. time.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the convergence of the mean-field gradient Langevin dynamics for the regularized mean-field neural network model. Under certain regularity assumption, it is proved that the dynamics in mean-field limit and the finite particle approximation are close for all time t>0, and certain statistic of these two dynamics are close with the order of O(1/N). As a consequence, it can be show that the finite particle approximation of the mean-field dynamics exponentially convergence towards the optimal solution up to an error of O(1/N).
strength_and_weaknesses: Strength: This paper provides the first rigorous uniform-in-time propagation of chaos result in the context of mean-field neural networks. This is achieved without adding strong entropy regularization or imposing strong assumptions on the complicated interaction term. 

Weakness: To achieve the claimed result, the convex regularizer on parameters is assumed to have a super-quadratic tail, an assumption that is not satisfied for the most common $\ell_2$ regularizer.

Some minor comments:
1. Below the equation after Eq.(4), there is a statement "We therefore see that $\mu_t$ decreases ... $\neq 0$.". It is difficult to understand. 
2. In the paragraph of "Particle discretization" on page 4, it is assume that $\hat X_0^i \sim \mu_0$, but in Lemma 1, it is claimed that you replaced the KL divergence on the R.H.S. with the Wasserstein distance since $\mu_0$ is a discrete distribution. Why can't we simply take $\mu_0$ to be some simple continuous initial distribution?
3. At the end of Proposition 1, I guess it should be $\mu^* = p_{\mu^*}$.
4. The concept of "test function" used in section 4.1 is not the same as the ones commonly used for defining weak convergences. 
clarity,_quality,_novelty_and_reproducibility: Clarity: The presentation of this paper is overall clearly written. Section 4.1.2 is difficult to follow.

Novelty: While this paper achieves the uniform-in-time type control between the mean-field dynamics and its finite-particle counterpart without adding strong entropy regularization or imposing strong assumptions on the complicated interaction term, the super-quadratic tail of the convex regularizer seems strong to me. I was not able to go through the details of the proof and I believe it is not easy. However, a strong regularization basically confines the parameter in a compact space (especially the gradients are assumed to be bounded) and LSI naturally holds in that case. I am not certain how meaningful this result is.


summary_of_the_review: This paper shows that the dynamics in mean-field limit and the finite particle approximation are close for all time t>0, and certain statistic of these two dynamics are close with the order of O(1/N). This result is interesting, but is obtained under a relatively strong assumptions on the parameter regularizer.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proves uniform-in-time chaos in the context of mean-field (MF) Langevin dynamics, motivated by noisy gradient descent for 2-layer MF neural nets. The analysis exploits the fact that a super log-Sobolev inequality (LSI) for the so-called proximal Gibbs measure, enabled by a super-quadratic regularizer, translates to the LSI of the corresponding flow measure at any sufficiently large time.
strength_and_weaknesses: This paper is the first I have known of to prove a uniform-in-time type of result for MF neural networks. This is a good result, since there is currently no applicable tool from the uniform-in-time chaos literature. Uniformity in time is also a highly desirable property, since one drawback of typical MF analyses is that the approximation between the large-width neural nets and the MF limit blows up exponentially with time. The observation that $\mu_t$ satisfies LSI thanks to the super-quadratic regularizer, leading to Theorem 2, is an interesting one. The work also circumvents the trouble of doing the large-$N$ approximation at each time $t$ by going for a large (ideally infinite) $t$, hence allowing for the error decomposition as well as the exponential convergence result that was proven by previous works.

I do not have much to complain about the paper, nor do I have sufficient time and background to check the proofs. Of course, there are typical concerns about the (potentially bad) dependency among the constants, as well as the rather unusually strong regularizer, but I would not worry much about it for now in light of a new result, the first of its kind. I have two questions:

- The numerical illustration basically shows that the approximation is increasingly better for larger $N$, but it doesn’t quite show that the approximation is getting better / no worse with time. Is there a way to better illustrate this?

- The strong tail of the regularizer essentially shapes the curvature at infinity and intuitively prevents bad things to happen at infinity. However the analysis centers around the optimal solution $\mu*$, for which the faraway region likely matters much less. Is there a way to intuitively understand this supposed dilemma? One possibility is that the regularizer might not be needed for uniform-in-time chaos.

Here my concern is that the strong regularizer is more of a technical device than something insightful, so there is some doubt whether this is the right way to understand uniform-in-time chaos.
clarity,_quality,_novelty_and_reproducibility: The paper is generally clear. The result is new and significant.
summary_of_the_review: The paper proves the first uniform-in-time chaos result ever known for MF 2-layer neural nets. This is a good result with new technical insights, though I’m unable to go through the proofs. I would expect researchers in the area take serious interests in this result, even if the strong regularizer does not appear convincing to be the right way to understanding the phenomenon.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper considers the mean-field regime of an overparametrized two layers neural network $f_X$ where $X\in\mathbb{R}^p$ stands for the trainable parameters. Indeed, it have been shown previously taking the number of neurons $N\to \infty$ that optimizing such network can be seen as minimizing the functional over the space of probability laws 
$$
F(\mu)=\frac{1}{n} \sum_{i=1}^n \ell_i\left(f_\mu\right)+\lambda_1 \int r(x) \mathrm{d} \mu(x) \quad , \quad f_\mu(z)=\int f_x(z) \mathrm{d} \mu(x).
$$
In particular, the paper aims to extend and complete existing literature regarding convergence of the neurons following the continuous limit of the "noisy" gradient descent dynamics to a mean-field Langevin equation which is the gradient flow dynamics associated with $F$. 
More precisely, the authors consider the particle system

$$
\mathrm{d} \hat{X}_t^i=-b(\hat{X}_t^i, \mu_t^N ) \mathrm{d} t +\sqrt{2 \lambda} \mathrm{d} W_t^i 
$$

$$
\mu_t^N=\frac{1}{N} \sum_{i=1}^N \delta_{\hat{X}_t^i}, 
$$

where $W_t^i$ are independent Brownian motion and $b$ corresponds to the gradient of the first variation $F$
$$
b(x, \mu)=\nabla \frac{\delta F}{\delta \mu}(\mu)(x)=\frac{1}{n} \sum_{j=1}^n \ell_j^{\prime}\left(f_\mu\right) \nabla h_j(x)+\lambda_1 \nabla r(x).
$$

They then show that for a nice class of functional $\Phi$, a uniform propagation holds for the sequence of dynamics $(\mu_t^N)$: there exist $C_1,C_2,C_3$ such that for any $t \geq 0$ and $N \geq 1$,
$$
|\mathbb{E}\left[\Phi\left(\mu_t^N\right)\right]-\Phi\left(\mu^*\right) |\leq C_1 e^{C_2t} + C_3/N.
$$

where $\mu^*$ is the minimizer of 
$$
\mathcal{L}(\mu)=\frac{1}{n} \sum_{i=1}^n \ell_i\left(f_\mu\right)+\lambda \mathrm{KL}\left(\mu, \nu_r\right),
$$

and the limit of the mean-field Langevin dynamics
\begin{aligned}
&\mathrm{d} X_t=-b\left(X_t, \mu_t\right) \mathrm{d} t+\sqrt{2 \lambda} \mathrm{d} W_t, \\
&\mu_t=\operatorname{Law}\left(X_t\right).
\end{aligned}
strength_and_weaknesses: Strength:
- the result is mathematically interesting and brings some new tools for establishing uniform propagation of chaos for particle systems which emerged from overparametrized neural network optimization.

Weakness:
- I am not really sure how interesting this result is to the machine learning community. I think the authors should better motivate the question they are addressing and provide some insights why their results may help in understanding of the optimization of overparametreized neural networks.  Ideally, could the authors give practical implications of their findings? 
- The condition on the regularization term is not really satisfactory. In particular, the authors say that previous works were limited by assuming either weak interaction or large noise. To my view, the authors get rid of this constraint by imposing a very strong regularization term which always counterbalances the interaction drift. This choice could potentially miss important and interesting phase-transition phenomena. 
- I felt that some parts of the paper would needed to be rewritten and  proofread if the authors want that their paper is accessible to the machine learning community. 
clarity,_quality,_novelty_and_reproducibility: I think the writing and presentation of the document should be improved. In particular, section 4.1.2, which is two pages of the main document, is in my opinion too difficult to parse.
In addition, Some details which are given are inessential whereas more would have required more care. 

- For example, it is a bit odd in my opinion to note that a distribution is also called a probability law (p4). 
- Why referring particularly to Huang for existence and uniqueness of solutions for (3) while no assumption and detail are given and there are many other results on this topic.
- The use of both $\lambda$ and $\lambda_1$ is really necessary? I think you could only introduce one of them since you work with a generic $r$.
- From assumption on $r$, it is hard to guess functions satisfying such a condition except $\Vert x\Vert^{2\alpha}$, for $\alpha >1$.
- The authors make use of higher order derivative of function $\Phi :\mathcal{P} \to \mathbb{R}$ with respect to a measure $\mu$ but I did not catch how it is defined only based on the definition of the first order derivative. In fact one of the main arguments is based on a second order differential computation derived in Delarue and Tse. However, I was not able to understand the tools used by the authors and how they use them to show Theorem 2 in the main paper. Finally, I think the authors should really comment more on what their contributions differ from those of Delarue and Tse.
- I did not follow how the smoothness of $\Phi$ implies that $\Phi(\mu) - \Phi(\mu^*) \leq C W_2(\mu,\mu^{*})$ (end of p6).
- the notation $\mu_t$ and $m(t,\mu_0)$ is really confusing.
- Finally, I was not been able to recover the informal statement of the authors:
$$
\mathbb{E}[(f_{X_t}-f_{\mu_t})^2] = O(1/N).
$$
In my opinion, it should be 
$$
\vert \mathbb{E}[f_{X_t}-f_{\mu_t}] \vert = O(1/N).
$$

summary_of_the_review: Overall I think that it could be a nice contribution to the literature on propagation of chaos.

However, in my opinion, the writing of the paper has to be improved and the relevance of the contributions for the machine community should be better emphasized. 

Second, I do not really know if ICLR is a good venue for the paper which I think should be suitable for more theoretical conferences or journals. Especially because the proofs of the results are quite delicate and require a relatively long time to be verified. To be honest, I did not have the time to do it and I am afraid no reviewer has either.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper considers two-layer networks with one output dimension in the mean-field parametrization. In the limit of infinite hidden-layer width, these can be described in the infinite dimensional space of parameter distributions (across hidden unit weights) by a nonlinear Fokker-Plank equation, which has been done in the past. The paper justifies the relevance of this approach by computing bounds on the difference between finite-size discretization and the infinite-width limit, showing that the difference in any sufficiently smooth test function and for any time in the training process scales as $1/N$, given relatively mild assumptions. This extends previous finite-size bounds which suffered from an exponential dependence on training time.
strength_and_weaknesses: **Strengths:**
 - Technically demanding proof with relevant outcome (1/N scaling) under mild conditions.

**Weaknesses:**
 - The progress in proof technique is stated multiple times in abstract/intro/conclusion (not requiring weak interaction while showing uniform propagation of chaos), but not technically discussed/explained in the relevant technical section 4 or appendices. 


clarity,_quality,_novelty_and_reproducibility: **Quality, Novelty:** yes.

**Clarity:** In general clear presentation of context and results, however the central technical advance is advertised but not clearly detailed (see weaknesses).

**Reproducibility:** The calculations are clear and reproducible. The code for the numerical experiments should be made available.


**Additional small comments:**  There are a number possible typos \
 -p.1: $h_z(x)=\tanh(r\sigma(w^Tx)) $ seems a typo (while on p.5 below A1 $\tanh(r)\sigma(w^Tx)$ is given) \
-p.2: "using neural network"; "$M$ neurons" ; also $x\in \mathbb{R}^p$ and $X \subset \mathbb{R}^d $ should both be $\mathbb{R}\times\mathbb{R}^d$? \
-p.4: "provides meaningful guarantee" \
-p.5 in Def.1: $\mathbb{R}^p$ should be $\mathbb{R}^d$? \
-p.6: "under quadratic regularizer" \
-p.9: "[our] analysis do not cover" \
-p.13: "xB"; "having form"; "such the"; "(1) uniform"; "(2) local"; "[they showed a..] but it also requires" \
-p.20: "We also a"; "for related quantity" \

summary_of_the_review: This paper seems mathematically sound and provides a relevant result for the study of feature learning in two layer networks in MF-parametrization. It is suitable for ICLR. The technical description of the novelty in approach should be improved.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Reviewers overall agree this paper should be accepted. I think the reviews summarize very well the strengths and weaknesses of the paper as well as points that the authors should include in the revised version. This paper is a valuable addition to the conference. 
justification_for_why_not_higher_score: While this is a very nice paper I did not see any particular reason for a highlight. 


justification_for_why_not_lower_score: No reason to reject.
Decision: Accept: poster