SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We gave a unified theoretical framework to bound the optimization error of the single-loop mean-field Langevin dynamics (MFLD) that is applicable to the finite-particle, discrete-time, and stochastic gradient algorithm. Our analysis is general enough to cover several important learning problems such as the optimization of mean-field neural networks, density estimation via MMD minimization, and variational inference via KSD minimization. We considered three versions of the algorithms (F-MFLD, SGD-MFLD, SGLD-MFLD); and despite the fact that our analysis deals with a more general setting (mean-field interactions), we are able to recover and even improve existing convergence guarantees when specialized to the standard gradient Langevin dynamics.
TS was partially supported by JSPS KAKENHI (20H00576) and JST CREST (JPMJCR2115). AN was partially supported by JSPS KAKENHI (22H03650). We thank the anonymous reviewers for their careful reading of our manuscript and their insightful comments.
E. Abbe, E. B. Adsera, and T. Misiakiewicz. The merged-staircase property: a necessary and nearly sufficient condition for sgd learning of sparse functions on two-layer neural networks. In Conference on Learning Theory, pages 4782-4887. PMLR, 2022.
M. Arbel, A. Korba, A. SALIM, and A. Gretton. Maximum mean discrepancy gradient flow. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and
1.1 Our Contributions 2
2 Problem Setting 3
2.1 Some Applications of MFLD 4
2.2 Practical implementations of MFLD 5
3 Main Assumptions and Theoretical Tools 6
3.1 Proximal Gibbs Measure & Logarithmic Sobolev inequality 6
4 Main Result: Convergence Analysis 7
4.1 General Recipe for Discretization Error Control 7
4.2 F-MFLD and SGD-MFLD 8
4.3 SVRG-MFLD 9
5 Conclusion 10
A Verification of Assumptions 15
B Proof of Theorem 2 15
B.1 Evaluation of Objective Decrease 15
B.2 Stochastic Gradient Error (Term C) 16
B.2.1 Analysis under Assumption 4 17
B.3 Putting Things Together 22
C Proof of Theorem 3: F-MFLD and SGD-MFLD 23
D Proof of Theorem 4: SVRG-MFLD 24
E Auxiliary lemmas 26
E.1 Properties of the MFLD Iterates 26
E.2 Wasserstein Distance Bound 27
E.3 Logarithmic Sobolev Inequality 29
E.4 Uniform Log-Sobolev Inequality 30
A Verification of Assumptions
Assumption 2 can be satisfied in the following settings for the three examples presented in Section 2.1.
(i) mean-field neural network. The neurons  $h_x(\cdot)$  and their gradients are bounded (e.g., tanh activation), and the first derivative of the loss is Lipschitz continuous (e.g., squared loss, logistic loss). That is, there exists  $C > 0$  such that  $\sup_z\sup_x|h_x(z)|\leq C$ ,  $\sup_z\sup_x\| \nabla_xh_x(z)\| \leq C$  and  $\| \nabla h(x) - \nabla h(x^{\prime})\| \leq C\| x - x^{\prime}\|$  for all  $x,x^{\prime}\in \mathbb{R}^{d}$ , and  $\sup_{y,z}\sup_{\mu \in \mathcal{P}}|\partial_f\ell (f_\mu (z),y)|\leq C$ ,  $|\partial_f\ell (f_\mu (z),y) - \partial_f\ell (f_{\mu '}(z),y)|\leq C|f_\mu (z) - f_{\mu '} (z)|$  and  $|\partial_f^2\ell (f_\mu (z),y)|\leq C$  for all  $(y,z)$  and  $\mu ,\mu^{\prime}\in \mathcal{P}$ .
(ii) MMD minimization. The kernel  $k$  is smooth and has light tail, e.g., the Gaussian RBF kernel.
(iii) KSD minimization. The kernel  $k$  has a light tail such that  $\sup_{z,z'} \max \{ |W_{\mu^*}(z,z')|, \| \nabla_z W_{\mu^*}(z,z') \|, \| \nabla_z \nabla_z^\top W_{\mu^*}(z,z') \|_{\mathrm{op}} \} \leq C$ ; for example,  $k(z,z') = \exp \left( -\frac{\|z\|^2}{2\sigma_1^2} - \frac{\|z'\|^2}{2\sigma_1^2} - \frac{\|z-z'\|^2}{2\sigma_2^2} \right)$ , and  $\max \{ \| \nabla \log(\mu^*(z)) \|, \| \nabla^{\otimes 2} \log(\mu^*(z)) \|_{\mathrm{op}}, \| \nabla^{\otimes 3} \log(\mu^*(z)) \|_{\mathrm{op}} \} \leq C (1 + \| z \|)$ .
This section gives the proof of Theorem 2. For notation simplicity, we write  $\eta$  to indicate  $\eta_{k}$ . Recall that for  $\mathcal{X} = (X^i)_{i=1}^N$ , the proximal Gibbs distribution  $\mu_{\mathcal{X}}$  is defined by
$$
p _ {\mathcal {X} ^ {\prime}} (X) \propto \exp \left(- \frac {1}{\lambda} \frac {\delta F (\mu_ {\mathcal {X} ^ {\prime}})}{\delta \mu} (X)\right).
$$
We also define another version of the proximal Gibbs distribution corresponding to  $\mathcal{F}^N$  as
$$
p ^ {(N)} (\mathcal {X}) \propto \exp \left(- \frac {N}{\lambda} F (\mu_ {\mathcal {X}})\right).
$$
B.1 Evaluation of Objective Decrease
By the same argument as Chen et al. (2022), we have that
$$
- \frac {1}{\lambda} \nabla \frac {\delta F (\mu_ {\mathcal {X}})}{\delta \mu} (X ^ {i}) = \nabla \log (p _ {\mathcal {X}} (X _ {i})) = \nabla_ {i} \log (p ^ {(N)} (\mathcal {X})) = - \frac {N}{\lambda} \nabla_ {i} F (\mu_ {\mathcal {X}}),
$$
where  $\nabla_{i}$  is the partial derivative with respect to  $X_{i}$ . Therefore, we have
$$
\begin{array}{l} \nabla_ {i} \frac {\delta \mathcal {F} ^ {N} (\boldsymbol {\mu} ^ {(N)})}{\delta \boldsymbol {\mu}} (\mathcal {X}) = N \nabla_ {i} F (\boldsymbol {\mu} _ {\mathcal {X}}) + \lambda \nabla_ {i} \log (\boldsymbol {\mu} ^ {(N)} (\mathcal {X})) \\ = \nabla \frac {\delta F (\mu_ {\mathcal {X}})}{\delta \mu} (X _ {i}) + \lambda \nabla_ {i} \log (\mu^ {(N)} (\mathcal {X})) \\ = - \lambda \nabla_ {i} \log (p ^ {(N)} (\mathcal {X})) + \lambda \nabla_ {i} \log (\mu^ {(N)} (\mathcal {X})). \tag {10} \\ \end{array}
$$
Remembering that  $\mathcal{X}_k = (X_k^i)_{i=1}^N$  is updated by  $X_{k+1}^i = X_k^i - \eta v_k^i + \sqrt{2\lambda \eta} \xi_k^i$ , the solutions  $X_k^i$  and  $X_{k+1}^i$  can be interpolated by the following continuous time dynamics:
$$
\widetilde {\mathcal {X} _ {0}} = \mathcal {X} _ {k},
$$
$$
\mathrm {d} \widetilde {X} _ {t} ^ {i} = - v _ {k} ^ {i} \mathrm {d} t + \sqrt {2 \lambda} \mathrm {d} W _ {t} ^ {i},
$$
for  $0 \leq t \leq \eta$ . Then,  $\widetilde{\mathcal{X}_{\eta}}$  obeys the same distribution as  $\mathcal{X}_{k+1}$ . Let  $\tilde{\mu}_t^{(N)}$  be the law of  $\widetilde{\mathcal{X}_t}$ . The Fokker-Planck equation of the dynamics yields that
$$
\frac {\mathrm {d} \tilde {\mu} _ {t} ^ {(N)}}{\mathrm {d} t} (\mathcal {X} | \mathcal {X} _ {0: k}, \omega_ {0: k}) = \sum_ {i = 1} ^ {N} \nabla_ {i} \cdot \left(\tilde {\mu} _ {t} ^ {(N)} (\mathcal {X} | \mathcal {X} _ {0: k}, \omega_ {0: k}) v _ {k} ^ {i}\right) + \lambda \sum_ {i = 1} ^ {n} \Delta_ {i} \tilde {\mu} _ {t} ^ {(N)} (\mathcal {X} | \mathcal {X} _ {0: k}, \omega_ {0: k}).
$$
Hence, by taking expectation with respect to  $\mathcal{X}_{0:k}$ , it holds that
$$
\begin{array}{l} \frac {\mathrm {d} \tilde {\mu} _ {t} ^ {(N)}}{\mathrm {d} t} (\mathcal {X}) \\ = \sum_ {i = 1} ^ {N} \lambda \nabla_ {i} \cdot \left(\tilde {\mu} _ {t} ^ {(N)} (\mathcal {X}) \left(\nabla_ {i} \log \left(\frac {\tilde {\mu} _ {t} ^ {(N)}}{p ^ {(N)}} (\mathcal {X})\right)\right)\right) \\ + \sum_ {i = 1} ^ {N} \nabla_ {i} \cdot \left\{ \right.\tilde {\mu} _ {t} ^ {(N)} (\mathcal {X}) \left( \right.\mathbb {E} _ {\mathcal {X} _ {0: k} | \widetilde {\mathcal {X}} _ {t}} \left[ \right. v _ {k} ^ {i} (\mathcal {X} _ {0: k}, \omega_ {k} ^ {i}) \left. \right| \widetilde {\mathcal {X}} _ {t} = \mathcal {X}, \omega_ {k} \left. \right] - \nabla \frac {\delta F (\mu_ {\widetilde {\mathcal {X}} _ {t}})}{\delta \mu} (X ^ {i})\left. \right)\left. \right\}, \\ \end{array}
$$
where we omitted the influence of  $\omega_{0:k}$  to  $\tilde{\mu}_t^{(N)}$ , which should be written as  $\tilde{\mu}_t^{(N)}(\mathcal{X}|\omega_{0:k})$  in a more precise manner. Combining this and Eq. (10) yields the following decomposition,
$$
\begin{array}{l} \begin{array}{c} \mathrm {d} \mathcal {F} ^ {N} (\tilde {\mu} _ {t} ^ {(N)}) \\ \hline \mathrm {d} t \end{array} \\ \leq - \underbrace {\frac {3 \lambda^ {2}}{4} \sum_ {i = 1} ^ {N} \mathbb {E} _ {\mathcal {X} \sim \tilde {\mu} _ {t} ^ {(N)}} \left[ \left\| \nabla_ {i} \log \left(\frac {\tilde {\mu} _ {t} ^ {(N)}}{p ^ {(N)}} (\mathcal {X})\right) \right\| ^ {2} \right]} _ {=: A} \\ + \sum_ {i = 1} ^ {N} \underbrace {\mathbb {E} _ {\widetilde {\mathcal {X}} _ {t} , \widetilde {\mathcal {X}} _ {0}} \left[ \left\| \nabla_ {i} \frac {\delta F (\tilde {\mu} _ {0} ^ {(N)})}{\delta \mu} (\widetilde {X} _ {0} ^ {i}) - \nabla_ {i} \frac {\delta F (\tilde {\mu} _ {t} ^ {(N)})}{\delta \mu} (\widetilde {X} _ {t} ^ {i}) \right\| ^ {2} \right]} _ {=: B} \\ - \sum_ {i = 1} ^ {N} \lambda \underbrace {\mathbb {E} _ {\widetilde {\mathcal {X}} _ {t} , \widetilde {\mathcal {X}} _ {0}} \left[ \left\langle \nabla_ {i} \log \left(\frac {\widetilde {\mu} _ {t} ^ {(N)}}{p ^ {(N)}} (\widetilde {\mathcal {X}} _ {t})\right) , v _ {k} ^ {i} (\mathcal {X} _ {0 : k} , \omega_ {k} ^ {i}) - \nabla \frac {\delta F (\mu_ {\widetilde {\mathcal {X}} _ {0}})}{\delta \mu} (\widetilde {X} _ {0} ^ {i}) \right\rangle \right]} _ {=: C}. \\ \end{array}
$$
Evaluation of term  $A$ : By the leave-one-out argument in the proof of Theorem 2.1 of Chen et al. (2022), the first term of the right hand side can be upper bounded by
$$
\begin{array}{l} - \frac {3 \lambda^ {2}}{4 N} \sum_ {i = 1} ^ {N} \mathbb {E} _ {\mathcal {X} \sim \tilde {\mu} _ {t} ^ {(N)}} \left[ \left\| \nabla_ {i} \log \left(\frac {\tilde {\mu} _ {t} ^ {(N)}}{p ^ {(N)}} (\mathcal {X})\right) \right\| ^ {2} \right] \\ \leq - \frac {\lambda^ {2}}{4 N} \sum_ {i = 1} ^ {N} \mathbb {E} _ {\mathcal {X} \sim \tilde {\mu} _ {t} ^ {(N)}} \left[ \left\| \nabla_ {i} \log \left(\frac {\tilde {\mu} _ {t} ^ {(N)}}{p ^ {(N)}} (\mathcal {X})\right) \right\| ^ {2} \right] \\ - \frac {\lambda \alpha}{2} \left(\frac {1}{N} \mathcal {F} ^ {N} \left(\tilde {\mu} _ {t} ^ {(N)}\right) - \mathcal {F} \left(\mu^ {*}\right)\right) + \frac {C _ {\lambda}}{N}, \tag {11} \\ \end{array}
$$
with a constant  $C_{\lambda}$ . The proof is given in Lemma 7 for completeness, where we see that the inequality holds with  $C_{\lambda} = 2\lambda L\alpha (1 + 2c_{L}\bar{R}^{2}) + 2\lambda^{2}L^{2}\bar{R}^{2}$ .
Evaluation of term  $B$ : By Lemma 2, we have
$$
\mathbb {E} _ {\widetilde {\mathcal {X}} _ {t}, \widetilde {\mathcal {X}} _ {0}} \left[ \left\| \nabla \frac {\delta F (\widetilde {\mu} _ {0} ^ {(N)})}{\delta \mu} (\widetilde {X} _ {0} ^ {i}) - \nabla \frac {\delta F (\widetilde {\mu} _ {t} ^ {(N)})}{\delta \mu} (\widetilde {X} _ {t} ^ {i}) \right\| ^ {2} \right] \leq \delta_ {\eta_ {k}},
$$
for  $\delta_{\eta_k} = C_1L^2 (\eta_k^2 +\lambda \eta_k) = O(L^2 (\eta_k^2 +\eta_k\lambda))$
B.2 Stochastic Gradient Error (Term C)
Now we evaluate the final term C. First, we derive a bound without Assumption 4. By the Cauchy-Schwarz inequality, we have that
$$
\lambda \mathbb {E} _ {\widetilde {\mathcal {X} _ {t}}, \widetilde {\mathcal {X} _ {0}}} \left[ \left\langle \nabla_ {i} \log \left(\frac {\tilde {\mu} _ {t} ^ {(N)}}{p ^ {(N)}} (\widetilde {\mathcal {X} _ {t}})\right), v _ {k} ^ {i} (\mathcal {X} _ {0: k}, \omega_ {k} ^ {i}) - \nabla \frac {\delta F (\mu_ {\widetilde {\mathcal {X} _ {0}}})}{\delta \mu} (\widetilde {X} _ {0} ^ {i}) \right\rangle \right]
$$
$$
\begin{array}{l} \leq \frac {\lambda^ {2}}{4} \mathbb {E} _ {\widetilde {\mathcal {X}} _ {t}, \widetilde {\mathcal {X}} _ {0}} \left[ \left\| \nabla_ {i} \log \left(\frac {\tilde {\mu} _ {t} ^ {(N)}}{p ^ {(N)}} (\widetilde {\mathcal {X}} _ {t})\right) \right\| ^ {2} \right] + \mathbb {E} _ {\widetilde {\mathcal {X}} _ {t}, \widetilde {\mathcal {X}} _ {0}} \left[ \left\| v _ {k} ^ {i} \left(\mathcal {X} _ {0: k}, \omega_ {k} ^ {i}\right) - \nabla \frac {\delta F \left(\mu_ {\widetilde {\mathcal {X}} _ {0}}\right)}{\delta \mu} \left(\widetilde {X} _ {0} ^ {i}\right) \right\| ^ {2} \right] \\ \leq \frac {\lambda^ {2}}{4} \mathbb {E} _ {\widetilde {\mathcal {X} _ {t}}, \widetilde {\mathcal {X} _ {0}}} \left[ \left\| \nabla_ {i} \log \left(\frac {\tilde {\mu} _ {t} ^ {(N)}}{p ^ {(N)}} (\widetilde {\mathcal {X} _ {t}})\right) \right\| ^ {2} \right] + \sigma_ {v, k} ^ {2}. \\ \end{array}
$$
B.2.1 Analysis under Assumption 4
Next, we derive a tighter result under the additional Assumption 4. We decompose term  $C$  as follows:
$$
\begin{array}{l} \mathbb {E} _ {\widetilde {\mathcal {X}} _ {t}, \widetilde {\mathcal {X}} _ {0}} \left[ \left\langle \nabla_ {i} \log \left(\frac {\tilde {\mu} _ {t} ^ {(N)}}{p ^ {(N)}} (\widetilde {\mathcal {X}} _ {t})\right), v _ {k} ^ {i} \left(\mathcal {X} _ {0: k}, \omega_ {k} ^ {i}\right) - \nabla \frac {\delta F \left(\mu_ {\widetilde {\mathcal {X}} _ {0}}\right)}{\delta \mu} \left(\widetilde {X} _ {0} ^ {i}\right) \right\rangle \right] \\ = \mathbb {E} _ {\widetilde {\mathcal {X} _ {t}}, \widetilde {\mathcal {X} _ {0}}} \left[ \left\langle \nabla_ {i} \log \left(\tilde {\mu} _ {t} ^ {(N)} (\widetilde {\mathcal {X} _ {t}})\right), v _ {k} ^ {i} \left(\mathcal {X} _ {0: k}, \omega_ {k} ^ {i}\right) - \nabla \frac {\delta F \left(\mu_ {\widetilde {\mathcal {X} _ {0}}}\right)}{\delta \mu} \left(\widetilde {X} _ {0} ^ {i}\right) \right\rangle \right] \\ - \mathbb {E} _ {\widetilde {\mathcal {X}} _ {t}, \widetilde {\mathcal {X}} _ {0}} \left[ \left\langle \nabla_ {i} \log \left(p ^ {(N)} (\widetilde {\mathcal {X}} _ {t})\right), v _ {k} ^ {i} \left(\mathcal {X} _ {0: k}, \omega_ {k} ^ {i}\right) - \nabla \frac {\delta F \left(\mu_ {\widetilde {\mathcal {X}} _ {0}}\right)}{\delta \mu} \left(\widetilde {X} _ {0} ^ {i}\right) \right\rangle \right]. \\ \end{array}
$$
Since  $\nabla_{i}\log (\tilde{\mu}_{t}^{(N)}) = \nabla_{i}\tilde{\mu}_{t}^{(N)} / \tilde{\mu}_{t}^{(N)}$ , this is equivalent to
$$
\begin{array}{l} \mathbb {E} _ {\widetilde {\mathcal {X}} _ {t}, \widetilde {\mathcal {X}} _ {0}} \left[ \left\langle \left(\tilde {\mu} _ {t} ^ {(N)} (\widetilde {\mathcal {X}} _ {t}) ^ {- 1} \nabla_ {i} \tilde {\mu} _ {t} ^ {(N)} (\widetilde {\mathcal {X}} _ {t}) - \nabla_ {i} \log (p ^ {(N)}) (\widetilde {\mathcal {X}} _ {t})\right), v _ {k} ^ {i} (\mathcal {X} _ {0: k}, \omega_ {k} ^ {i}) - \nabla \frac {\delta F (\mu_ {\widetilde {\mathcal {X}} _ {0}})}{\delta \mu} (\widetilde {X} _ {0} ^ {i}) \right\rangle \right] \\ = \underbrace {\int \mathbb {E} _ {\widetilde {\mathcal {X}} _ {0} | \widetilde {\mathcal {X}} _ {t}} \left[ \left\langle \int \nabla_ {i} \tilde {\mu} _ {t} ^ {(N)} \left(\widetilde {\mathcal {X}} _ {t} \mid \mathcal {X} _ {0 : k} ^ {\prime}\right) \mu^ {(N)} \left(\mathrm {d} \mathcal {X} _ {0 : k} ^ {\prime}\right) , v _ {k} ^ {i} \left(\mathcal {X} _ {0 : k} , \omega_ {k} ^ {i}\right) - \nabla \frac {\delta F \left(\mu_ {\mathcal {X} _ {k}}\right)}{\delta \mu} \left(X _ {k} ^ {i}\right) \right\rangle \right] \mathrm {d} \widetilde {\mathcal {X}} _ {t}} _ {C - (\mathrm {I})}} \\ - \underbrace {\mathbb {E} _ {\widetilde {\mathcal {X} _ {0}} , \widetilde {\mathcal {X} _ {t}}} \left[ \left\langle \nabla_ {i} \log \left(p ^ {(N)}\right) \left(\widetilde {\mathcal {X} _ {t}}\right) , v _ {k} ^ {i} \left(\mathcal {X} _ {0 : k} , \omega_ {k} ^ {i}\right) - \nabla \frac {\delta F \left(\mu_ {\widetilde {\mathcal {X} _ {0}}}\right)}{\delta \mu} \left(\widetilde {X} _ {0} ^ {i}\right) \right\rangle \right]} _ {C - (\mathrm {I I})}, \tag {12} \\ \end{array}
$$
where  $\mathcal{X}_{0:k}^{\prime}$  is an independent copy of  $\mathcal{X}_{0:k}$ .
Evaluation of  $C$ -( $I$ ):
(1) The first term  $(C - (\mathrm{I}))$  of the right hand side of Eq. (12) can be evaluated as
$$
\begin{array}{l} \int \mathbb {E} _ {\widetilde {\mathcal {X}} _ {0} | \widetilde {\mathcal {X}} _ {t}} \left[ \left\langle \int \nabla_ {i} \tilde {\mu} _ {t} ^ {(N)} \left(\widetilde {\mathcal {X}} _ {t} | \mathcal {X} _ {0: k} ^ {\prime}\right) \mu^ {(N)} \left(\mathrm {d} \mathcal {X} _ {0: k} ^ {\prime}\right), v _ {k} ^ {i} \left(\mathcal {X} _ {0: k}, \omega_ {k} ^ {i}\right) - \nabla \frac {\delta F \left(\mu_ {\mathcal {X} _ {k}}\right)}{\delta \mu} \left(X _ {k} ^ {i}\right) \right\rangle \right] \mathrm {d} \widetilde {\mathcal {X}} _ {t} \\ = \int \mathbb {E} _ {\widetilde {\mathcal {X}} _ {0} | \widetilde {\mathcal {X}} _ {t}} \left[ \left\langle \int \nabla_ {i} \tilde {\mu} _ {t} ^ {(N)} \left(\widetilde {\mathcal {X}} _ {t} | \mathcal {X} _ {0: k} ^ {\prime}\right) \mu^ {(N)} \left(\mathrm {d} \mathcal {X} _ {0: k} ^ {\prime}\right), v _ {k} ^ {i} \left(\left(\widetilde {\mathcal {X}} _ {t}, \mathcal {X} _ {0: k - 1}\right), \omega_ {k} ^ {i}\right) - \nabla \frac {\delta F \left(\mu_ {\widetilde {\mathcal {X}} _ {t}}\right)}{\delta \mu} \left(\widetilde {X} _ {t} ^ {i}\right) \right\rangle \right] \mathrm {d} \widetilde {\mathcal {X}} _ {t} \\ + \int \mathbb {E} _ {\widetilde {\mathcal {X} _ {0}} | \widetilde {\mathcal {X} _ {t}}} \left[ \left\langle \int \nabla_ {i} \tilde {\mu} _ {t} ^ {(N)} \left(\widetilde {\mathcal {X} _ {t}} \mid \mathcal {X} _ {0: k} ^ {\prime}\right) \mathrm {d} \mu^ {(N)} \left(\mathcal {X} _ {0: k} ^ {\prime}\right), \right. \right. \\ \end{array}
$$
$$
\left. v _ {k} ^ {i} \left(\mathcal {X} _ {0: k}, \omega_ {k} ^ {i}\right) - \nabla \frac {\delta F \left(\mu_ {\mathcal {X} _ {k}}\right)}{\delta \mu} \left(X _ {k} ^ {i}\right) - v _ {k} ^ {i} \left(\left(\widetilde {\mathcal {X} _ {t}}, \mathcal {X} _ {0: k - 1}\right), \omega_ {k} ^ {i}\right) + \nabla \frac {\delta F \left(\mu_ {\widetilde {\mathcal {X} _ {t}}}\right)}{\delta \mu} \left(\widetilde {X} _ {t} ^ {i}\right)\right\rangle\left. \right] d \widetilde {\mathcal {X} _ {t}}, \tag {13}
$$
where  $(\widetilde{\mathcal{X}}_t,\mathcal{X}_{0:k - 1}) = (\mathcal{X}_0,\mathcal{X}_1,\ldots ,\mathcal{X}_{k - 1},\widetilde{\mathcal{X}}_t)$  which is obtained by replacing  $\mathcal{X}_k$  of  $\mathcal{X}_{0:k}$  to  $\widetilde{\mathcal{X}}_t$ . First, we evaluate the first term in the right hand side of Eq. (13). Let
$$
D _ {m} ^ {i} F (\mathcal {X}) := \frac {\delta F (\mu_ {\mathcal {X}})}{\delta \mu} (X ^ {i}),
$$
for  $\mathcal{X} = (X^i)_{i=1}^N$ , and let  $D_m F(\mathcal{X}) = (D_m^i F(\mathcal{X}))_{i=1}^N$ . Let
$$
\Delta v ^ {i} := v _ {k} ^ {i} \left(\mathcal {X} _ {0: k} ^ {\prime}, \omega_ {k} ^ {i}\right) - D _ {m} ^ {i} F \left(\mathcal {X} _ {k} ^ {\prime}\right),
$$
and let  $\Delta v = (\Delta v^i)_{i=1}^N$ . We also define
$$
Z _ {i} = (\sqrt {2 \lambda t}) ^ {- 1} [ \widetilde {X} _ {t} ^ {i} - (X _ {k} ^ {\prime i} - t v _ {k} ^ {i} (\mathcal {X} _ {0: k} ^ {\prime}, \omega_ {k} ^ {i})) ].
$$
Then  $Z_{i} \sim N(0,1)$  conditioned by  $\mathcal{X}_{0:k}^{\prime}$  and is independent of  $\omega_{k}^{i}$ . We also define
$$
v _ {k} ^ {i} (\hat {\mathcal {X}}, \omega_ {k} ^ {i}) := \mathbb {E} _ {\mathcal {X} _ {0: k - 1} | \hat {\mathcal {X}}, \omega_ {1: k}} [ v _ {k} ^ {i} ((\hat {\mathcal {X}}, \mathcal {X} _ {0: k - 1}), \omega_ {k} ^ {i}) ],
$$
where the expectation is taken conditioned by  $\omega_{1:k}$  but we omit  $\omega_{1:k-1}$  from the left hand side for the simplicity of notation. Since the density of the conditional distribution  $\tilde{\mu}_t^{(N)}(\widetilde{\mathcal{X}}_t|\mathcal{X}_{0:k}')$  is proportional to  $\exp\left(-\sum_{i=1}^N \frac{\|\widetilde{X}_t^i - (X_k'^i - tv_k^i(\mathcal{X}_{0:k}',\omega_k^i))\|^2}{2(2\lambda t)}\right)$ , it holds that, under second order differentiability of  $v_k^i$  and  $D_mF$ ,
$$
\begin{array}{l} \int \left\langle \int - \frac {\widetilde {X} _ {t} ^ {i} - (X _ {k} ^ {\prime i} - t v _ {k} ^ {i} (\mathcal {X} _ {0 : k} ^ {\prime} , \omega_ {k} ^ {i}))}{2 \lambda t} \tilde {\mu} _ {t} ^ {(N)} (\widetilde {\mathcal {X} _ {t}} | \mathcal {X} _ {0: k} ^ {\prime}) \mu^ {(N)} (\mathrm {d} \mathcal {X} _ {0: k} ^ {\prime}), \right. \\ v _ {k} ^ {i} (\widetilde {\mathcal {X}} _ {t}, \omega_ {k} ^ {i}) - \nabla \frac {\delta F (\mu_ {\widetilde {\mathcal {X}} _ {t}})}{\delta \mu} (\widetilde {X} _ {t} ^ {i}) \Bigg > \mathrm {d} \widetilde {\mathcal {X}} _ {t} \\ = \mathbb {E} _ {Z, \mathcal {X} _ {0: k} ^ {\prime}} \left[ \left\langle - \frac {Z _ {i}}{\sqrt {2 \lambda t}}, v _ {k} ^ {i} (\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z - t \Delta v, \omega_ {k} ^ {i}) - \right. \right. \\ \left. D _ {m} ^ {i} F (\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z - t \Delta v) \right\rangle \Bigg ] \\ = \mathbb {E} _ {Z, \mathcal {X} _ {0: k} ^ {\prime}} \left[ \left\langle - \frac {Z _ {i}}{\sqrt {2 \lambda t}}, \right. \right. \\ v _ {k} ^ {i} (\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z, \omega_ {k} ^ {i}) - D _ {m} ^ {i} F (\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z) \\ + v _ {k} ^ {i} (\mathcal {X} _ {k} ^ {\prime} - t v _ {k} (\mathcal {X} _ {0: k} ^ {\prime}, \omega_ {k}) + \sqrt {2 \lambda t} Z, \omega_ {k} ^ {i}) - v _ {k} ^ {i} (\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z, \omega_ {k} ^ {i}) \\ \left. \left. - D _ {m} ^ {i} F \left(\mathcal {X} _ {k} ^ {\prime} - t v _ {k} \left(\mathcal {X} _ {0: k} ^ {\prime}, \omega_ {k}\right) + \sqrt {2 \lambda t} Z\right) + D _ {m} ^ {i} F \left(\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F \left(\mathcal {X} _ {k} ^ {\prime}\right) + \sqrt {2 \lambda t} Z\right) \right\rangle \right]. \tag {14} \\ \end{array}
$$
By Assumption 4, we can evaluate
$$
\begin{array}{l} \mathbb {E} _ {\omega_ {k}} \left[ \left\| v _ {k} ^ {i} (\mathcal {X} _ {k} ^ {\prime} - t v _ {k} (\mathcal {X} _ {0: k} ^ {\prime}, \omega_ {k}) + \sqrt {2 \lambda t} Z, \omega_ {k} ^ {i}) - v _ {k} ^ {i} (\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z, \omega_ {k} ^ {i}) \right. \right. \\ \left. \left. - D _ {m} ^ {i} F (\mathcal {X} _ {k} ^ {\prime} - t v _ {k} (\mathcal {X} _ {0: k} ^ {\prime}, \omega_ {k}) + \sqrt {2 \lambda t} Z) + D _ {m} ^ {i} F (\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z) \right\rVert \right] \\ \leq \mathbb {E} _ {\mathcal {X} _ {0: k - 1} ^ {\prime \prime}, \omega_ {k}} \left[ \left\| v _ {k} ^ {i} \left(\left(\mathcal {X} _ {k} ^ {\prime} - t v _ {k} \left(\mathcal {X} _ {0: k} ^ {\prime}, \omega_ {k}\right) + \sqrt {2 \lambda t} Z, \mathcal {X} _ {0: k - 1} ^ {\prime \prime}\right), \omega_ {k} ^ {i}\right) \right. \right. \\ - v _ {k} ^ {i} \big ((\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z, \mathcal {X} _ {0: k - 1} ^ {\prime \prime} \big), \omega_ {k} ^ {i} \big) \\ \left. \left. - D _ {m} ^ {i} F (\mathcal {X} _ {k} ^ {\prime} - t v _ {k} (\mathcal {X} _ {0: k} ^ {\prime}, \omega_ {k}) + \sqrt {2 \lambda t} Z) + D _ {m} ^ {i} F (\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z) \right\rVert \right] \\ \end{array}
$$
$$
\begin{array}{l} \leq \mathbb {E} _ {\mathcal {X} _ {0: k - 1} ^ {\prime \prime}, \omega_ {k}} \left[ \sum_ {j = 1} ^ {N} \left\| \nabla_ {j} ^ {\top} v _ {k} ^ {i} \left(\left(\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z, \mathcal {X} _ {0: k - 1} ^ {\prime \prime}\right), \omega_ {k} ^ {i}\right) \right. \right. \\ - \nabla_ {j} ^ {\top} D _ {m} ^ {i} F ((\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z, \mathcal {X} _ {0: k - 1} ^ {\prime \prime})) \Big \| _ {\mathrm {o p}} \| t \Delta v _ {j} \| \\ \left. + Q t ^ {2} \left(\frac {1}{N} \sum_ {j = 1} ^ {N} \| \Delta v _ {j} \| ^ {2} + \| \Delta v _ {i} \| ^ {2}\right) \right], \\ \end{array}
$$
where Jensen's inequality is used in the first inequality. By using Assumption 4 again, the first term in the right hand side can be evaluated as
$$
\begin{array}{l} \sum_ {j = 1} ^ {N} \mathbb {E} _ {\mathcal {X} _ {0: k - 1} ^ {\prime \prime}, \omega_ {k}} \left[ \frac {a _ {j}}{2} \| \nabla_ {j} ^ {\top} v _ {k} ^ {i} ((\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z, \mathcal {X} _ {0: k - 1} ^ {\prime \prime}), \omega_ {k} ^ {i}) \right. \\ - \nabla_ {j} ^ {\top} D _ {m} ^ {i} F ((\mathcal {X} _ {k} ^ {\prime} - t D _ {m} F (\mathcal {X} _ {k} ^ {\prime}) + \sqrt {2 \lambda t} Z, \mathcal {X} _ {0: k - 1} ^ {\prime \prime})) \| _ {\mathrm {o p}} ^ {2} + \frac {1}{2 a _ {j}} \| t \Delta v _ {j} \| ^ {2} \Biggr ] \\ \end{array}
$$
$$
\leq (N - 1) \left(\frac {a _ {- i}}{2 N ^ {2}} \tilde {\sigma} _ {v, k} ^ {2} + \frac {1}{2 a _ {- i}} t ^ {2} \sigma_ {v, k} ^ {2}\right) + \left(\frac {a _ {i}}{2} \tilde {\sigma} _ {v, k} ^ {2} + \frac {1}{2 a _ {i}} t ^ {2} \sigma_ {v, k} ^ {2}\right),
$$