Paper: Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems (ICLR-2024)

Reviewer comment (Theory):
In the footnote on page 6, you mention that the typical rate for the Wasserstein distance between a measure and an i.i.d.-sampled empirical measure is $N^{-1/d}$. However, in this case, the particles in $ \mu_{ \overline{\mathscr{X}}\_k}$ are highly correlated due to coupled dynamics. The correlation exists not only between particles in $ \overline{\mathscr{X}}\_k$ for a fixed $k$, but also between particles at different times (i.e., $ \overline{\mathscr{X}}\_k$ and $ \overline{\mathscr{X}}\_j$ for $j \neq k$). Thus, it is unclear what the convergence rate of $W_1(\mu_{ \overline{\mathscr{X}}\_k},   \mathbb{E} [ \mu_{ \overline{\mathscr{X}}\_k} ] )$ would be.

Author response:
**1.1-2.** $\mathbb{E}[\mu\_{\overline{\mathscr{X}}\_k}]$ is the expected output of MFL-AG, hence Theorem 3.7 quantifies a sort of bias. While this does not provide the full picture, we put the expectation inside $W_1$ and carefully designed our approach (using uniform LLN, short-term perturbation, etc) to bypass the curse of dimensionality. We mentioned the i.i.d. sample bound simply to illustrate that the CoD is unavoidable for $W_1(\mu\_{\mathscr{X}\_k}, \mathbb{E}[\mu\_{\mathscr{X}\_k}])$ or $W_1(\mu\_{\mathscr{X}\_k}, \mu^*)$ even in the ideal situation (no chaos) where the particles are perfectly independently sampled from the true distribution.

Nonetheless, we see the utility in obtaining a bound for $W_1(\mu\_{\mathscr{X}\_k}, \mu^*)$. Thus, we have added a completely new section which fleshes out the above discussion and develops similar bounds for $W_1(\mu\_{\mathscr{X}\_k}, \mu^*)$ to Theorem 3.7 save for the CoD, which is unavoidable in the weakly interacting setting of propagation of chaos. Please see Appendix C.6.

Of course, in practice we may also simply run the algorithm multiple ($M$) times and take the average of the outputs, which would also bypass the issue and yield the standard $1/\sqrt{M}$ convergence.

We also remark that while the statement of Theorem 3.7 does not directly imply i.i.d. behavior of particles, the philosophy of propagation of chaos plays a key role in the proof. For example when developing the uniform law of large numbers we consider the system as a perturbed version of the gradient-stopped process, whose particles are independent when conditioned on the history up to a certain point in time. Hence, we emphasize the utility of our results lies not only in Theorem 3.7 but also the novel perturbation analysis developed throughout Appendix C.

Reviewer comment (Theory):
The discussion on top of page 7 is not very clear. You state that you expand around $\mu, \hat{\nu} $, but the first-order condition in (3) is only for MNE. It is unclear why  $ \frac{\delta }{\delta \mu} \mathcal{L}\_\lambda(\mu, \hat{\nu}) =  \frac{\delta }{\delta \hat{\nu}} \mathcal{L}\_\lambda(\mu, \hat{\nu}) = 0$ would hold for an arbitrary $\mu$. Do you mean you expand around $  \mu_*, \nu_*$? It is also unclear why higher-order terms can be ignored. Currently, I do not understand how the upper bound on $W_1$ implies an upper bound on NI error. Since NI error upper bounds the KL-divergence, and KL-divergence upper bounds $W_1$, I doubt the correctness of claiming that NI error can be upper bounded by the sum of $W_1$ in general.

Author response:
**1.3-5.** Thank you for your astute observations. The upper bound does not hold in general; the sketched idea only works for the *unregularized* NI error (without KL terms), which we have not considered in the paper. At present, there seems to be no clean way to bound entropic terms (KL, NI) regarding $\mathbb{E[\mu\_{\mathscr{X}\_k}]}, \mathbb{E[\nu\_{\mathscr{Y}\_k}]}$. We apologize for the mistake, and the claims have been deleted as discussing unregularized error will be too confusing. (Propagation of chaos is often presented in terms of Wasserstein distance throughout the literature. We merely wanted to strengthen the remark that the squared distance $W_1^2$ (and not $W_1$) is a natural unit of measurement on the left-hand side of Theorem 3.7 in view of Talagrand's and sandwich inequalities.)

Reviewer comment (Theory):
In the case where $\mathcal{L}$ is bilinear and assumes good regularity (e.g., the kernel is uniformly bounded), is it possible to upper bound  $NI( \mathbb{E}[ \mu\_{ \overline{\mathscr{X}}\_k } ], \mathbb{E}[\nu_{ \overline{\mathscr{Y}}_k } ])$ by the sum of $W_1$ in the propagation of chaos results?

Author response:
Thank you for taking the time to read through our responses! We agree that the current statement is not ideal to present our results, however alternatives are somewhat unclear. $\text{NI}(\mu\_{\mathscr{X}\_k}, \nu\_{\mathscr{Y}\_k})$ is lower bounded by KL which is lower bounded by $W_2^2$, so it is impossible to avoid the curse of dimensionality for NI error. A compromise could be $\text{NI}(\mathbb{E}[\mu\_{\mathscr{X}\_k}], \mathbb{E}[\nu\_{\mathscr{Y}\_k}])$ or $\text{KL}(\mathbb{E}[\mu\_{\mathscr{X}\_k}]\Vert\mu^*)$, however entropic terms involving $\mathbb{E}[\mu\_{\mathscr{X}\_k}]$ cannot be controlled with only the uniform LLN.

In fact, we can show that **unregularized** NI error $\text{NI}\_0(\mathbb{E}[\mu\_{\mathscr{X}\_k}], \mathbb{E}[\nu\_{\mathscr{Y}\_k}]) = \max\_{\nu'}\mathcal{L}(\mathbb{E}[\mu\_{\mathscr{X}\_k}],\nu')-\min\_{\mu'}\mathcal{L}(\mu',\mathbb{E}[\nu\_{\mathscr{Y}\_k}])$, if it is defined, can be upper bounded by $W_1$ as discussed above. This is the corresponding result to [1, Conversion to a Wasserstein distance bound], while this cannot be extended to regularized NI due to additional entropic terms.

Define $\text{NI}\_0(\mu,\nu)=\max\_{\nu'}\mathcal{L}(\mu,\nu')-\min\_{\mu'}\mathcal{L}(\mu',\nu)$ and suppose $\mathcal{L}$ is strongly convex-concave so that it possesses a unique saddle point $(\mu\_0^*,\nu\_0^*)$. More precisely, suppose $\mathcal{L}(\mu^*,\nu)\geq \mathcal{L}(\mu\_0^*,\nu) + \kappa W_1^2(\mu^*,\mu\_0^*)$, then we have the chain of inequalities

$$\mathcal{L}(\mu_0^*,\nu^*) + \kappa W_1^2(\mu^*,\mu_0^*) \leq \mathcal{L}(\mu^*,\nu^*)\leq \mathcal{L}(\mu^*,\nu^*)+\lambda\text{KL}(\mu^*\Vert\rho^\mu) \leq \mathcal{L}(\mu_0^*,\nu^*)+\lambda\text{KL}(\mu_0^*\Vert\rho^\mu)$$

which shows that $W_1(\mu^*,\mu_0^*)=O(\sqrt{\lambda})$. Furthermore, we have $\max_{\nu'}\mathcal{L}(\mu,\nu')-\mathcal{L}(\mu_0^*,\nu_0^*)\leq \max_{\nu'} (\mathcal{L}(\mu,\nu')-\mathcal{L}(\mu_0^*,\nu'))\leq M_\mu W_1(\mu,\mu_0^*)$. Combining and adding with the corresponding result for $\nu$ gives that $\text{NI}_0(\mu,\nu)\leq O(W_1(\mu,\mu^*)+W_1(\nu,\nu^*)+\sqrt{\lambda})$.

We hope this can clarify the reviewer's concerns.

Reviewer comment (Theory):
The discussion on top of page 7 should be made more formal and rigorous. The current ambiguous discussion makes it hard to understand the utility of the propagation of chaos results.

Author response:
**1.3-5.** Thank you for your astute observations. The upper bound does not hold in general; the sketched idea only works for the *unregularized* NI error (without KL terms), which we have not considered in the paper. At present, there seems to be no clean way to bound entropic terms (KL, NI) regarding $\mathbb{E[\mu\_{\mathscr{X}\_k}]}, \mathbb{E[\nu\_{\mathscr{Y}\_k}]}$. We apologize for the mistake, and the claims have been deleted as discussing unregularized error will be too confusing. (Propagation of chaos is often presented in terms of Wasserstein distance throughout the literature. We merely wanted to strengthen the remark that the squared distance $W_1^2$ (and not $W_1$) is a natural unit of measurement on the left-hand side of Theorem 3.7 in view of Talagrand's and sandwich inequalities.)

Reviewer comment (Theory):
In Chen et al. (2022), they provide propagation of chaos in KL-divergence under extra regularity assumptions (Theorem 2.4). In this paper's setting, do you expect a similar propagation of chaos in KL divergence? Also, Chen et al. (2022) obtain an $O(1/N)$ error, optimal in order of $N$. Is it possible to improve the bounds to $O(1/N)$, or is there a lower bound on the convergence rate for propagation of chaos?

Author response:
**Q1.** Proving propagation of chaos in KL divergence would be ideal; this is the approach taken in [1] as well, and is much more direct and simpler. We tried to emulate this framework at first, but quickly found out key concepts such as their $N$-particle proximal distribution cannot be applied. This is due to the dependence on the entire history and cannot be fixed by adding stronger assumptions. Thus we had to resort to more technically involved methods (uniform LLN, short-time perturbation analysis) leading to our $W_1$ result. While morally there seems to be no reason why the result would not hold in KL, our current tools are not enough to guarantee this.

For the uniform LLN (Proposition 3.6) at least, the $O(1/\sqrt{N})$ rate is optimal. The worst-case LLN error is determined by the change in Lipschitz constant of the function $F(\mu_{\mathscr{X}}, \nu_{\mathscr{X}})$ when each particle is removed, which in turn is tightly bounded by the leave-one-out $W_1$ error due to W1-Lipschitzity of $F(\cdot,\cdot)$. Since the leave-one-out $W_2^2$ error is $O(1/N)$ (which is central in Chen's argument), the LLN error must be $O(1/\sqrt{N})$. Since we did not quite use the full power of the uniform LLN in subsequent analyses, however, it is possible that a different approach could circumvent this and achieve $O(1/N)$, which seems an interesting direction for future work. Nevertheless, we emphasize this is the first history-dependent propagation of chaos result with explicit rates in any setting. Existing related results in the SPDE literature are asymptotic and do not provide error bounds [3].

Reviewer comment (Theory):
Is it possible to apply the annealing techniques in Lu (2022) to achieve the MNE of the unregularized object (assuming it exists)?

Author response:
**Q3.** We have performed some rough computations; it is indeed possible to achieve the unregularized MNE using MFL-AG with an annealing rate $\lambda_t=O(1/\log t)$ as in Lu's work or [4]. This results in a proximal gap (Prop. 3.3) of $O(1/(\log t)^2)$ and NI error convergence rate (Theorem 3.4) of $O(1/\log t)$. Interestingly, this also absorbs the slow $O(1/t)$ rate of the regularized problem. A caveat is that we must assume $\delta\mathcal{L}/\delta\mu$ is bounded rather than Lipschitz as in Assumption 3 (a usually stronger condition) in order to avoid the use of Talagrand's inequality and instead use TV distance bounds as in Appendix D. For MFL-ABR, we can similarly anneal $\lambda_k$ (keeping it fixed in the inner loop) to achieve $O(1/\log k)$ in the outer loop.

Reviewer comment (Writing):
The description of the equation can be more comprehensive. Usually, we should describe all the notations of the equation at least right after it. For example, you do not mention what $\rho^{\mu}$ and $\rho^{\nu}$ are in equation 1.

Author response:
**W2.** Thank you for your suggestions. The description of Equation 1 and other potentially confusing notational issues have been fixed.

Reviewer comment (Writing):
Could you please explain more about what "SYMMETRIC" means in your proposed method and title? Is it mainly from the same temperature or regularization strength $\lambda$ for both players?

Author response:
**Q1.** Symmetry means that both $\mu,\nu$ have same learning rates (flow speeds). Until our paper, existing minimax MFLD results only studied imbalanced flows, that is $\partial_t\mu_t = (\text{equation for }\mu_t)$ and $\partial_t\nu_t = \eta\times(\text{equation for }\nu_t)$ for very large $\eta$ [1] or the static limit $\eta\to\infty$ [2]. In such regimes $\nu$ can be considered already nearly optimized for any $\mu$ and the difficulty of controlling two coupled flows are mostly removed. However, needless to say the setting is quite artificial. Symmetric updates are much more difficult to analyze and do not converge in general for even simple problems such as minmax$(x^\top y)$ for $x,y\in\mathbb{R}^d$ [3].

Reviewer comment (Theory):
Then how compatible your proposed method and proof can be extended to when there exists $\lambda_1$ and $\lambda_2$ for two players respectively? Since you mentioned that the problems you are looking for naturally arise for example solving robust learning or zero-sum games in reinforcement learning. Usually, in the robustness via adversarial learning, the regularization strength is relevant to the strength of the adversary. For example, if the adversary is attacked on soft actor-critic (SAC)[1], which originally had its regularization term (temperature), then it is not flexible to have the regularization terms for the target agent and the adversary the same values.

Author response:
**Q2.** Our method easily extends to different regularization strengths $\lambda_\mu,\lambda_\nu$ with minimal modifications on the derivations and bounds (this has been added to the first page footnote). Intuitively, the regularizations take place in their respective spaces $\mathcal{X}$ and $\mathcal{Y}$ so there is no need for the strength parameters to be equal. We only took equal $\lambda$ for simplicity and to adhere to preceding works.

Reviewer comment (Evaluation):
Could you analyze why MFL-AG can have a lower NI error compared with MFL-ABR and does it mean that MFL-AG performs better than MFL-ABR in this experiment? Furthermore, please suggest what kind of numerical experiment is more suitable with MFL-ABR against MFL-AG, and vice versa.

Author response:
**Q3.** MFL-AG indeed performed better than MFL-ABR in this particular experiment, however we found empirically that this generally depended on the choice of hyperparameters. For example, speeding up the inner loop of MFL-ABR by decreasing iterations and increasing $\eta$, or speeding up the outer loop by increasing $\beta$ could accelerate convergence of MFL-ABR, but also led to instability in some runs. In contrast, MFL-AG always exhibited a "slow but steady" $O(1/t)$ convergence.

In addition, MFL-ABR and MFL-DA performed better for problems where $Q(x,y)$ factored into a sum (or product) of separate potentials for $x,y$. In this case the dynamics decouples into two single-optimization problems which are known to converge exponentially. In contrast, MFL-AG was preferable for potentials with complex interactions between $x,y$. For example for the sigmoid potential in Section 6, $\mu$ pulls $\nu$ toward itself while $\nu$ pushes away, leading to a more complicated interacting dynamics that is better solved by the more stable MFL-AG.

Reviewer comment (Theory):
The assumptions over the two measures used in entropic regularization are quite strong. The result of Theorem 3.7 has an expectation inside $W_1$, which is somewhat hard to interpret: it would seem to be saying that the bias of the procedure is small, but not the variance, so many repetitions of the particle dynamics would be needed to estimate the expectation of some function $f$. The outer-loop particle discretization error is not accounted for in the algorithm from Section 4.

Author response:
**Assumptions.** Our assumptions on the regularizer functions and functional $\mathcal{L}$ are standard in the mean-field literature, corresponding to L2 regularization in the case of two-layer neural networks [1,2]. In fact, they are more relaxed compared to some works which require super-quadratic growth of regularizers [3] or uniform boundedness of functional derivatives [4]. Lu and Domingo-Enrich's works do not have an L2 regularizer only due to the fact that they impose the strong assumptions that $\mathcal{X,Y}$ are *compact manifolds with Ricci curvature lower bounded*, which also guarantees an LSI constant.

**Outer loop.** Please see our discussion with Reviewer q6kk on the outer loop discretization issue.

**Expectation.** The expectation inside $W_1$ is purely due to the following fundamental fact: the expected Wasserstein distance between any $d$-dimensional distribution and a size $N$ sample is $O(N^{-1/d})$, so any attempt to directly control distance of $\widehat{\mu}$ to $\mu^*$ will immediately incur the curse of dimensionality even if $\widehat{\mu}$ is perfectly i.i.d. sampled from the true distribution. We thus carefully designed our approach to completely avoid this dependency. Nonetheless, if the expectation is outside we can still obtain a similar bound save for the CoD. We have added a new section (Appendix C.6) which discusses the effect of CoD and develops corresponding results and proofs.

Of course, as you mentioned in practice we may also simply run the algorithm multiple ($M$) times and take the average of the outputs, which would also bypass the issue and yield the standard $1/\sqrt{M}$ convergence.

Reviewer comment (Theory):
I do not understand what is going on in Lemma C.7 (it might be because I do not understand $\Pi$ fully -- see below).

Author response:
**Q2.** We apologize for our mistake - the projection is supposed to be $\mathscr{X}\mapsto X^i$, that is the projection of the $N$-particle vector to its $i$th coordinate (averaged over all $i$). The corresponding pullback operator on functions is $\Pi^* f(\mathscr{X})=\frac{1}{N}\sum_{i=1}^N f(X^i)$. An explicit definition is given at the beginning of Appendix C.2. We hope this clarifies the role of Lemma C.7 as well.

Reviewer comment (Writing):
Some of the notation is unnecessarily hard to parse. For example, on Page 5: *Denote ordered sets of $N$ particles* -- it would seem that the authors mean a vector in $\mathcal{X}^N$. Also on the same page: it seems that the authors introduce the subscript $1:k$ to identify particle positions at different times, but then don't use this notation consistently at the bottom of the page (and also elsewhere).

Author response:
**Notation.** The notation has been updated. The notation $(\mathscr{X},\mathscr{Y})\_{1:k}$ has been changed to $(\mathscr{X}\_{1:k},\mathscr{Y}\_{1:k})$ in the main text and reintroduced in the appendix. The functional has been changed from $L$ to $\mathcal{L}$, and the use of $\mu,\nu$ in sub/superscripts to denote problem quantities has been clarified on the first page.

Reviewer comment (Writing):
$L_\mu$ and $L_\nu$ are used to denote Lipschitz constants in the first/second variable, but also, $L$ is the functional and $(\mu,\nu)$ are used to denote arbitrary elements of the space of probability measures.

Author response:
**Notation.** The notation has been updated. The notation $(\mathscr{X},\mathscr{Y})\_{1:k}$ has been changed to $(\mathscr{X}\_{1:k},\mathscr{Y}\_{1:k})$ in the main text and reintroduced in the appendix. The functional has been changed from $L$ to $\mathcal{L}$, and the use of $\mu,\nu$ in sub/superscripts to denote problem quantities has been clarified on the first page.

Reviewer comment (Theory):
The definition of $\Pi$ in page 6 as “the average of the pushforward operators along the projections $X \mapsto X^i$” does not seem correct. How can you project to a higher-dimensional space?

Author response:
**Q2.** We apologize for our mistake - the projection is supposed to be $\mathscr{X}\mapsto X^i$, that is the projection of the $N$-particle vector to its $i$th coordinate (averaged over all $i$). The corresponding pullback operator on functions is $\Pi^* f(\mathscr{X})=\frac{1}{N}\sum_{i=1}^N f(X^i)$. An explicit definition is given at the beginning of Appendix C.2. We hope this clarifies the role of Lemma C.7 as well.

Reviewer comment (Experiments):
*(Q1)* Algorithm 1 and elsewhere – rather than sample subsets of $\mathcal{X}_k,\mathcal{Y}_k$, maybe you could just use a weighted empirical measure where set $\mathcal{X}_k$ has weight proportional to $\beta_k$?

Author response:
**Q1.** It is indeed possible to weighted-concatenate the particles instead of random sampling in both Algorithm 1 and the outer loop of Algorithm 2. Since our discretization guarantee for MFL-AG is written in terms of expected distribution, it applies to both methods, and concatenation may help stabilize the output.

The main issue is that the number of total particles $NK$ will then grow proportionally with the number of epochs $K$, so that with a per-epoch memory/compute constraint $C$ the user may have to set a smaller $N=O(C/K)$ to begin with. Whereas with random sampling the two algorithms only require 4 arrays of length $N=O(C)$ (as discussed in Section 5.1), so it is hard to say which is superior.

If particle weighting is allowed, another natural consideration is to update the masses themselves and study the resulting discrete Wasserstein-Fisher-Rao gradient dynamics, which is an interesting direction for future work.

Reviewer comment (Theory):
*(Q2)* Can you clarify what is $\Pi$ and explain specifically the proof of Lemma C.7?

Author response:
**Q2.** We apologize for our mistake - the projection is supposed to be $\mathscr{X}\mapsto X^i$, that is the projection of the $N$-particle vector to its $i$th coordinate (averaged over all $i$). The corresponding pullback operator on functions is $\Pi^* f(\mathscr{X})=\frac{1}{N}\sum_{i=1}^N f(X^i)$. An explicit definition is given at the beginning of Appendix C.2. We hope this clarifies the role of Lemma C.7 as well.

Reviewer comment (Theory):
The requirement of convexity combined with strictly bounded gradients is quite strict, as it essentially forces the problem to be roughly linear. This is a major restriction and the authors should do their best to circumvent this, or otherwise explain why this assumption was necessary in their opinion.

Author response:
The functionals studied in even ordinary MFLD works (such as that for optimizing neural networks) are always convex; this is due to convexity of the loss function and an intended consequence of lifting to the space of measures, not due to simplicity of the underlying problem. Furthermore, minimax MFLD is often studied explicitly for bi-linear objectives, that is $\mathcal{L}(\mu,\nu)=\iint Q(x,y) d\mu(x)d\nu(y)$ where the underlying payoff $Q$ can be any nonconvex-nonconcave $C^1$ function. This covers all (2-player) zero-sum continuous games, which in turn generalizes all finite zero-sum games.

In fact, **all** preceding minimax MFLD works [1,2,3] only considered bi-linear $\mathcal{L}$ (linear-convex in [4]), and even in this simple case it was unknown how to design a symmetric descent-ascent type algorithm until our paper. Hence, our work can actually be considered much more general.

Furthermore, our assumptions in the gradient are implied by the assumptions on $Q$ in these works and are also generally considered weaker than sup norm control, which is also a common assumption in the MFLD literature. In fact, they are more relaxed compared to some works which require super-quadratic regularization or uniform boundedness of functional derivatives. (The works [1,2,3] avoid L2 regularization only because they impose the strong assumption that $\mathcal{X,Y}$ are compact manifolds without boundary, which automatically ensures various isoperimetric inequalities.)

We agree that nonconvex MFLD is also of great interest, however extremely little is known for even the single-optimization case as of yet and should be considered as a separate direction of study.

Reviewer comment (Theory):
It is unclear why the authors prefer to use \(L^1\) Lipschitz-ness and obtain their result in Wasserstein-1. It may be more natural to frame all results in \(W_2\), so clarification on whether \(W_1\) was chosen for a specific reason would be useful.

Author response:
Indeed, framing the result in $W_2$ as in [5] could be more natural. However unlike [5], for our dynamics we only have indirect control over the empirical-proximal distribution gap via Prop. 3.6, which yields a $W_1$ bound by Kantorovich duality when converting to a bound for the empirical measures. Our uniform LLN analysis (as opposed to entropy-based bounds) is a novel approach that allows us to overcome the challenges inherent in the extremely complex historically-dependent particle interactions. Please note that we have added a new section on bounding expected distance and curse of dimensionality (Appendix C.6); in this case, a stronger moment bound will allow strengthening to $W_2$.

Reviewer comment (Theory):
The dependencies on \(N\) are worse when compared to standard mean-field results. The dependence on \(\eta\) in the first term is also not ideal; for instance, uniform-in-time propagation of chaos for the mean-field Langevin dynamics would look like \(k^2/N^2\) (in \(W_2^2\) under mild assumptions).

Author response:
The worse rates arise from controlling the full history dependence, not minimax dynamics a priori. For the uniform LLN (Proposition 3.6) at least, the $O(1/\sqrt{N})$ rate is optimal. The worst-case LLN error is determined by the change in Lipschitz constant of the function $F(\mu_{\mathscr{X}}, \nu_{\mathscr{X}})$ when each particle is removed, which in turn is tightly bounded by the leave-one-out $W_1$ error due to W1-Lipschitzity of $F(\cdot,\cdot)$. Since the leave-one-out $W_2^2$ error is $O(1/N)$, the LLN error must be $O(1/\sqrt{N})$.

Since we did not quite use the full power of the uniform LLN in subsequent analyses, however, it is possible that a different approach could circumvent this and achieve better rates as well as getting rid of the dependency on $\eta$. Nevertheless, we emphasize this is the first history-dependent propagation of chaos result with explicit rates in any setting. The few existing results in the SPDE literature are asymptotic and do not provide error bounds [6].

Reviewer comment (Theory):
It is possible that writing an analogue of the BBGKY hierarchy in this setting and conducting the analysis that way could sharpen the rates. Under the convex-concavity assumption and bounded gradients assumptions, the analysis may be tractable using such an approach.

Author response:
**Kinetic dynamics.** While not quite the same as BBGKY, a kinetic mean-field system has been considered in the context of machine learning optimization very recently in [7]. The studied underdamped MFLD corresponds to a momentum gradient descent, and exponential convergence and uniform-in-time propagation of chaos results are obtained. Although from an algorithmic perspective the use of momentum may potentially help to speed up the slow $O(1/t)$ rate of MFL-AG, it is unclear if the convergence rates (derived using entropic hypocoercivity) improves upon ordinary MFLD in general, and the propagation of chaos result requires some higher-order regularity assumptions. Adapting these techniques to deal with history dependence is also highly nontrivial (as was for ordinary MFLD for our paper). Nonetheless, applying kinetic MFLD to develop faster minimax algorithms is a very interesting and promising direction for future work.

Reviewer comment (Presentation):
Figure 1 should be moved to the main text, as it is a core part of the contribution and should not be placed in the appendix.

Author response:
The Figure has been moved to the main text.

Reviewer comment (Theory):
The dependency of \(K\) on key problem parameters should be clarified in the main text. It seems from the Appendices that it should be roughly \(\epsilon^2\). Additionally, the dimension dependence, which appears to be fairly mild, should be highlighted.

Author response:
The dependency of $C_1$ on $\eta$ mentioned after Theorem 3.7 means that in order to bound all 3 terms as $\epsilon$, we need to take $\eta=O(\epsilon^2)$ which in turn roughly requires $K=O(\epsilon^{-1-1/\alpha})$ to assure $C_1/K<\epsilon$. This has been clarified in the main text.

Dimension dependence only appears in the LSI constant as $\alpha = O(d)$ due to Lipschitz perturbations, and can be averted entirely if we assume bounded perturbations by the Holley-Stroock argument. We have added this in the paper.

Reviewer comment (Theory):
In Theorem 4.1, the reason for taking \(k\) as \(1/\epsilon \log \epsilon\) should be explained. Shouldn’t \(\log 1/\epsilon\) suffice?

Author response:
In Theorem 4.1, bounding discretization error (3rd term) requires $C\beta<\epsilon$ so that $k=\frac{1}{\epsilon}\log\frac{1}{\epsilon}$ is needed to ensure the outer loop convergence $\exp(-\beta k)<\epsilon$.

Reviewer comment (Theory):
In Lemma C.2, there is no need to refer to Gronwall’s lemma, as induction suffices.

Author response:
Lemma C.2 has been reworded as per your suggestions.

Reviewer comment (Writing):
A definition of “convexity” should be given in the Appendix, e.g., geodesic convexity in \(W_2\), or convexity with respect to TV distance.

Author response:
The definition of convexity has been added.