SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zebang Shen*
zebang.shen@inf.ethz.ch
Zhenfu Wang*
zwang@bicomr.pku.edu.cn
The McKean-Vlasov equation (MVE) describes the collective behavior of particles subject to drift, diffusion, and mean-field interaction. In physical systems, the interaction term can be singular, i.e. it diverges when two particles collide. Notable examples of such interactions include the Coulomb interaction, fundamental in plasma physics, and the Biot-Savart interaction, present in the vorticity formulation of the 2D Navier-Stokes equation (NSE) in fluid dynamics. Solving MVEs that involve singular interaction kernels presents a significant challenge, especially when aiming to provide rigorous theoretical guarantees. In this work, we propose a novel approach based on the concept of entropy dissipation in the underlying system. We derive a potential function that effectively controls the KL divergence between a hypothesis solution and the ground truth. Building upon this theoretical foundation, we introduce the Entropy-dissipation Informed Neural Network (EINN) framework for solving MVEs. In EINN, we utilize neural networks (NN) to approximate the underlying velocity field and minimize the proposed potential function. By leveraging the expressive power of NNs, our approach offers a promising avenue for tackling the complexities associated with singular interactions. To assess the empirical performance of our method, we compare EINN with SOTA NN-based MVE solvers. The results demonstrate the effectiveness of our approach in solving MVEs across various example problems.
Scientists use Partial Differential Equations (PDEs) to describe natural laws and predict the dynamics of real-world systems. As PDEs are of fundamental importance, a growing area in machine learning is the use of neural networks (NN) to solve these equations [Han et al., 2018; Zhang et al., 2018; Raissi et al., 2020; Cai et al., 2021; Karniadakis et al., 2021; Cuomo et al., 2022]. An important category of PDEs is the McKean-Vlasov equation (MVE), which models the dynamics of a stochastic particle system with mean-field interactions
$$
\mathrm {d} \mathbf {X} _ {t} = - \nabla V (\mathbf {X} _ {t}) \mathrm {d} t + K * \bar {\rho} _ {t} (\mathbf {X} _ {t}) \mathrm {d} t + \sqrt {2 \nu} \mathrm {d} \mathbf {B} _ {t}, \quad \bar {\rho} _ {t} = \operatorname {L a w} (\mathbf {X} _ {t}). \tag {1}
$$
Here  $\mathbf{X}_t\in \mathcal{X}$  denotes a random particle' position,  $\mathcal{X}$  is either  $\mathbb{R}^d$  or the torus  $\Pi^d$  (a cube  $[-L,L]^d$  with periodic boundary condition),  $V:\mathbb{R}^d\to \mathbb{R}$  denotes a known potential,  $K:\mathbb{R}^d\rightarrow \mathbb{R}^d$  denotes some interaction kernel and the convolution operation is defined as  $h*\phi = \int_{\mathcal{X}}h(\pmb {x} - \pmb {y})\phi (\pmb {y})d\pmb {y},$ $\{\mathbf{B}_t\}_{t\geq 0}$  is the standard  $d$  -dimensional Wiener process with  $\nu \geq 0$  being the diffusion coefficient, and  $\bar{\rho}_t:\mathcal{X}\to \mathbb{R}$  is the law or the probability density function of the random variable  $\mathbf{X}_t$  and the initial data  $\bar{\rho}_0$  is given. Under mild regularity conditions, the density function  $\bar{\rho}_t$  satisfies the MVE
$$
(\mathbf {M V E}) \quad \partial_ {t} \bar {\rho} _ {t} (\boldsymbol {x}) + \operatorname {d i v} \left(\bar {\rho} _ {t} (- \nabla V (\boldsymbol {x}) + K * \bar {\rho} _ {t} (\boldsymbol {x}))\right) = \nu \Delta \bar {\rho} _ {t} (\boldsymbol {x}), \tag {2}
$$
*Authors are listed in alphabetic order.
where  $\mathrm{div}$  denotes the divergence operator,  $\mathrm{div}h(\pmb {x}) = \sum_{i = 1}^{d}\partial h_{i} / \partial x_{i}$  for a velocity field  $h:\mathbb{R}^d\to$ $\mathbb{R}^d$  ,  $\Delta$  denotes the Laplacian operator defined as  $\Delta \phi = \operatorname {div}\bigl (\nabla \phi \bigr)$  , where  $\nabla \phi$  denotes the gradient of a scalar function  $\phi :\mathbb{R}^d\rightarrow \mathbb{R}$  . Note that all these operators are applied only on the spatial variable  $\pmb{x}$  In order to describe dynamics in real-world phenomena such as electromagnetism [Golse, 2016] and fluid mechanics [Majda et al., 2002], the interaction kernels  $K$  in the MVE can be highly singular, i.e.  $\| K(\pmb {x})\| \to \infty$  when  $\| \pmb {x}\| \to 0$  . Two of the most notable examples are the Coulomb interactions
$$
\text {(C o u l o m b K e r n e l)} \quad K (\boldsymbol {x}) = - \nabla g (\boldsymbol {x}), \text {w i t h} g (\boldsymbol {x}) = \left\{ \begin{array}{l l} \left(\left(d - 2\right) S _ {d - 1} (1)\right) ^ {- 1} \| \boldsymbol {x} \| ^ {- (d - 2)}, & d \geq 3, \\ - \left(2 \pi\right) ^ {- 1} \log \| \boldsymbol {x} \|, & d = 2, \end{array} \right. \tag {3}
$$
with  $S_{d - 1}(1)$  denoting the surface area of the unit sphere in  $\mathbb{R}^d$ , and the vorticity formulation of the 2D Navier-Stokes equation (NSE) where the interaction kernel  $K$  is given by the Biot-Savart law
$$
\text {(B i o t - S a v a r t K e r n e l)} \quad K (\boldsymbol {x}) = \frac {1}{2 \pi} \frac {\boldsymbol {x} ^ {\perp}}{\| \boldsymbol {x} \| ^ {2}} = \frac {1}{2 \pi} \left(- \frac {x _ {2}}{\| \boldsymbol {x} \| ^ {2}}, \frac {x _ {1}}{\| \boldsymbol {x} \| ^ {2}}\right), \tag {4}
$$
where  $\pmb{x} = (x_{1}, x_{2})$  and  $\| \pmb{x}\|$  denotes the Euclidean norm of a vector.
Classical methods for solving MVEs, including finite difference, finite volume, finite element, spectral methods, and particle methods, have been developed over time. A common drawback of these methods lies in the constraints of their solution representation: Sparse representations, such as less granular grids, cells, meshes, fewer basis functions, or particles, may lead to an inferior solution accuracy; On the other hand, dense representations incur higher computational and memory costs.
As a potent tool for function approximation, NNs are anticipated to overcome these hurdles and handle higher-dimensional, less regular, and more complex systems efficiently [Weinan et al. 2021]. The most renowned NN-based algorithm is the Physics Informed Neural Network (PINN) [Raissi et al. 2019]. The philosophy behind the PINN method is that solving a PDE system is equivalent to finding the root of the corresponding differential operators. PINN tackles the latter problem by directly parameterizing the hypothesis solution with an NN and training it to minimize the  $\mathcal{L}^2$  functional residual of the operators. As a versatile PDE solver, PINN may fail to exploit the underlying dynamics of the PDE, which possibly leads to inferior performance on task-specific solvers. For example, on the 2D NSE problem, a recent NN-based development [Zhang et al. 2022] surpasses PINN and sets a new SOTA empirical performance, which however lacks rigorous theoretical substantiation. Despite the widespread applications of PINN, rigorous error estimation guarantees are scarce in the literature. While we could not find results on the MVE with the Coulomb interaction, only in a very recent paper [De Ryck et al. 2023], the authors establish for NSE that the PINN loss controls the discrepancy between a candidate solution and the ground truth. We highlight that their result holds average-in-time, meaning that at a particular timestamp  $t\in [0,T]$ , a candidate solution with small PINN loss may still significantly differ from the true solution. In contrast, all guarantees in this paper are uniform-in-time. Moreover, there is a factor in the aforementioned guarantee that exponentially depends on the total evolving time  $T$ , while the factor in our guarantee for the NSE is independent of  $T$ . We highlight that these novel improvements are achieved for the proposed EINN framework since we take a completely different route from PINN: Our approach is explicitly designed to exploit the underlying dynamics of the system, as elaborated below.
Our approach Define the operator
$$
\mathcal {A} [ \rho ] \stackrel {\text {d e f}} {=} - \nabla V + K * \rho - \nu \nabla \log \rho . \tag {5}
$$
By noting  $\Delta \bar{\rho}_t = \mathrm{div}(\bar{\rho}_t \nabla \log \bar{\rho}_t)$ , we can rewrite the MVE in the form of a continuity equation
$$
\partial_ {t} \bar {\rho} _ {t} (\boldsymbol {x}) + \operatorname {d i v} \left(\bar {\rho} _ {t} (\boldsymbol {x}) \mathcal {A} \left[ \bar {\rho} _ {t} \right] (\boldsymbol {x})\right) = 0. \tag {6}
$$
For simplicity, we will refer to  $\mathcal{A}[\bar{\rho}_t]$  as the underlying velocity. Consider another time-varying hypothesis velocity field  $f:\mathbb{R}\times \mathbb{R}^d\to \mathbb{R}$  and let  $\rho_t^f$  be the solution to the continuity equation
$$
\text {(h y p o t h e s i s s o l u t i o n)} \quad \partial_ {t} \rho_ {t} ^ {f} (\boldsymbol {x}) + \operatorname {d i v} \left(\rho_ {t} ^ {f} (\boldsymbol {x}) f (t, \boldsymbol {x})\right) = 0, \rho_ {0} ^ {f} = \bar {\rho} _ {0} \tag {7}
$$
for  $t \in [0, T]$ , where we recall that the initial law  $\bar{\rho}_0$  is known. We will refer to  $\rho_t^f$  as the hypothesis solution and use the superscript to emphasize its dependence on the hypothesis velocity field  $f$ . We propose an Entropy-dissipation Informed Neural Network framework (EINN), which trains an NN parameterized hypothesis velocity field  $f_{\theta}$  by minimizing the following EINN loss
$$
\left(\operatorname {E I N N} \text {l o s s}\right) R \left(f _ {\theta}\right) \stackrel {\text {d e f}} {=} \int_ {0} ^ {T} \int_ {\mathcal {X}} \| f _ {\theta} (t, \boldsymbol {x}) - \mathcal {A} \left[ \rho_ {t} ^ {f _ {\theta}} \right] (\boldsymbol {x}) \| ^ {2} \rho_ {t} ^ {f _ {\theta}} (\boldsymbol {x}) \mathrm {d} \boldsymbol {x} \mathrm {d} t. \tag {8}
$$
The objective (8) is obtained by studying the stability of carefully constructed Lyapunov functions. These Lyapunov functions draw inspiration from the concept of entropy dissipation in the system, leading to the name of our framework. We highlight that we provide a rigorous error estimation guarantee for our framework for MVEs with singular kernels (3) and (4), showing that when  $R(f_{\theta})$  is sufficiently small,  $\rho_t^{f_\theta}$  recovers the ground truth  $\bar{\rho}_t$  in the KL sense, uniform in time.
Theorem 1 (Informal). Suppose that the initial density function  $\bar{\rho}_0$  is sufficiently regular and the hypothesis velocity field  $f_t(\cdot) = f(t, \cdot)$  is at least three times continuously differentiable both in  $t$  and  $x$ . We have for the MVE with a bounded interaction kernel  $K$  or with the singular Coulomb (3) or Biot-Savart (4) interaction, the KL divergence between the hypothesis solution  $\rho_t^f$  and the ground truth  $\bar{\rho}_t$  is controlled by the EINN loss for any  $t \in [0, T]$ , i.e. there exists some constant  $C > 0$ ,
$$
\sup  _ {t \in [ 0, T ]} \mathbf {K L} \left(\rho_ {t} ^ {f}, \bar {\rho} _ {t}\right) \leq C R (f). \tag {9}
$$
Having stated our main result, we elaborate on the difference between EINN and PINN in terms of information flow over time, which explains why EINN achieves better theoretical guarantees: In PINN, the residuals at different time stamps are independent of each other and hence there is no information flow from the residual at time  $t_1$  to the one at time  $t_2 (> t_1)$ . In contrast, in the EINN loss (8), incorrect estimation made in  $t_1$  will also affect the error at  $t_2$  through the hypothesis solution  $\rho_t^f$ . Such an information flow gives a stronger gradient signal when we are trying to minimize the EINN loss, compared to the PINN loss. It partially explains why we can obtain the novel uniform-in-time estimation as opposed to the average-in-time estimation for PINN and why the constant  $C$  in the NSE case is independent of  $T$  for EINN (Theorem 2), but exponential in  $T$  for PINN.
Contributions. In summary, we present a novel NN-based framework for solving the MVEs. Our method capitalizes on the entropy dissipation property of the underlying system, ensuring robust theoretical guarantees even when dealing with singular interaction kernels. We elaborate on the contributions of our work from theory, algorithm, and empirical perspectives as follows.
1. (Theory-wise) By studying the stability of the MVEs with bounded interaction kernels or with singular interaction kernels in the Coulomb (3) and the Biot-Savart case (4) (the 2D NSE) via entropy dissipation, we establish the error estimation guarantee for the EINN loss on these equations. Specifically, we design a potential function  $R(f)$  of a hypothesis velocity  $f$  such that  $R(f)$  controls the KL divergence between the hypothesis solution  $\rho_t^f$  (defined in equation (7)) and the ground truth solution  $\bar{\rho}_t$  for any time stamp within a given time interval  $[0,T]$ . A direct consequence of this result is that  $R(f)$  can be used to assess the quality of a generic hypothesis solution to the above MVEs and  $\rho_t^f$  exactly recovers  $\bar{\rho}_t$  in the KL sense given that  $R(f) = 0$ .
2. (Algorithm-wise) When the hypothesis velocity field is parameterized by an NN, i.e.  $f = f_{\theta}$  with  $\theta$  being some finite-dimensional parameters, the EINN loss  $R(f_{\theta})$  can be used as the loss function of the NN parameters  $\theta$ . We discuss in detail how an estimator of the gradient  $\nabla_{\theta}R(f_{\theta})$  can be computed so that stochastic gradient-based optimizers can be utilized to train the NN. In particular, for the 2D NSE (the Biot-Savart case [4]), we show that the singularity in the gradient computation can be removed by exploiting the anti-derivative of the Biot-Savart kernel.
3. (Empirical-wise) We compare the proposed approach, derived from our novel theoretical guarantees, with SOTA NN-based algorithms for solving the MVE with the Coulomb interaction and the 2D NSE (the Biot-Savart interaction). We pick specific instances of the initial density  $\bar{\rho}_0$ , under which explicit solutions are known and can be used as the ground truth to test the quality of the hypothesis ones. Using NNs with the same complexity (depth, width, and structure), we observe that the proposed method significantly outperforms the included baselines.
2 Entropy-dissipation Informed Neural Network
In this section, we present the proposed EINN framework for the MVE. To understand the intuition behind our design, we first write the continuity equation (7) in a similar form as the MVE (6):
$$
\left. \partial_ {t} \rho_ {t} ^ {f} (\boldsymbol {x}) + \operatorname {d i v} \left(\rho_ {t} ^ {f} (\boldsymbol {x}) \left(\mathcal {A} \left[ \rho_ {t} ^ {f} \right] (\boldsymbol {x}) + \delta_ {t} (\boldsymbol {x})\right)\right)\right) = 0, \tag {10}
$$
where  $f$  is the hypothesis velocity (recall that  $f_{t}(\cdot) = f(t,\cdot)$ ) and
$$
\text {(P e r t u r b a t i o n)} \quad \delta_ {t} (\boldsymbol {x}) \stackrel {\text {d e f}} {=} f _ {t} (\boldsymbol {x}) - \mathcal {A} \left[ \rho_ {t} ^ {f} \right] (\boldsymbol {x}) \tag {11}
$$
can be regarded as a perturbation to the original MVE system. Consequently, it is natural to study the deviation of the hypothesis solution  $\rho_t^f$  from the true solution  $\bar{\rho}_t$  using an appropriate Lyapunov
function  $L(\rho_t^f, \bar{\rho}_t)$ . The functional relation between this deviation and the perturbation is termed as the stability of the underlying dynamical system, which allows us to derive the EINN loss (8). Following this idea, the design of the EINN loss can be determined by the choice of the Lyapunov function  $L$  used in the stability analysis. In the following, we describe the Lyapunov function used for the MVE with the Coulomb interaction and the 2D NSE (MVE with Biot-Savart interaction). The proof of the following results are the major theoretical contributions of this paper and will be elaborated in the analysis section 3.
- For the MVE with the Coulomb interaction, we choose  $L$  to be the modulated free energy  $E$  (defined in equation (23)) which is originally proposed in [Bresch et al., 2019a] to establish the mean-field limit of a corresponding interacting particle system. We have (setting  $L = E$ )
$$
\frac {\mathrm {d}}{\mathrm {d} t} E \left(\rho_ {t} ^ {f}, \bar {\rho} _ {t}\right) \leq \frac {1}{2} \int_ {\mathcal {X}} \rho_ {t} ^ {f} (\boldsymbol {x}) \| \delta_ {t} (\boldsymbol {x}) \| ^ {2} \mathrm {d} \boldsymbol {x} + C E \left(\rho_ {t} ^ {f}, \bar {\rho} _ {t}\right), \tag {12}
$$
where  $C$  is a universal constant depending on  $\nu$  and  $(\bar{\rho}_t)_{t\in [0,T]}$ .
- For the 2D NSE (MVE with the Biot-Savart interaction), we choose  $L$  as the KL divergence. Our analysis is inspired by [Jabin and Wang, 2018] which for the first time establishes the quantitative mean-field limit of the stochastic interacting particle systems where the interaction kernel can be in some negative Sobolev space. We have
$$
\frac {\mathrm {d}}{\mathrm {d} t} \mathbf {K L} \left(\rho_ {t} ^ {f}, \bar {\rho} _ {t}\right) \leq - \frac {\nu}{2} \int_ {\mathcal {X}} \rho_ {t} ^ {f} (\boldsymbol {x}) \| \nabla \log \frac {\rho_ {t} ^ {f}}{\bar {\rho} _ {t}} (\boldsymbol {x}) \| ^ {2} + C \mathbf {K L} \left(\rho_ {t} ^ {f}, \bar {\rho} _ {t}\right) + \frac {1}{\nu} \int_ {\mathcal {X}} \rho_ {t} ^ {f} (\boldsymbol {x}) \| \delta_ {t} (\boldsymbol {x}) \| ^ {2} \mathrm {d} \boldsymbol {x}, \tag {13}
$$
where again  $C$  is a universal constant depending on  $\nu$  and  $(\bar{\rho}_t)_{t\in [0,T]}$ .
After applying Gronwall's inequality on the above results, we can see that the EINN loss (8) is precisely the term derived by stability analysis of the MVE system with an appropriate Lyapunov function. In the next section, we elaborate on how a stochastic approximation of  $\nabla_{\theta}R(f_{\theta})$  can be efficiently computed for a parameterized hypothesis velocity field  $f = f_{\theta}$  so that stochastic optimization methods can be utilized to minimize  $R(f_{\theta})$ .
2.1 Stochastic Gradient Computation with Neural Network Parameterization
While the choice of the EINN loss (8) is theoretically justified through the above stability study, in this section, we show that it admits an estimator which can be efficiently computed. Define the flow map  $X_{t}$  via the ODE  $\mathrm{d}\pmb{x}(t) = f_t(\pmb{x}(t);\theta)\mathrm{d}t$  with  $\pmb{x}(0) = \pmb{x}_0$  such that  $\pmb{x}(t) = X_{t}(\pmb{x}_{0})$ . From the definition of the push-forward measure, one has  $\rho_t^f = X_t\# \bar{\rho}_0$ . Recall the definitions of the EINN loss  $R(f)$  in equation (8) and the perturbation  $\delta_t$  in equation (11). Use the change of variable formula of the push-forward measure in (a) and Fubini's theorem in (b). We have
$$
R (f) = \int_ {0} ^ {T} \| \delta_ {t} \| _ {\rho_ {t} ^ {f}} ^ {2} d t \stackrel {(a)} {=} \int_ {0} ^ {T} \| \delta_ {t} \circ X _ {t} \| _ {\rho_ {0}} ^ {2} d t \stackrel {(b)} {=} \int \int_ {0} ^ {T} \| \delta_ {t} \circ X _ {t} (\boldsymbol {x} _ {0}) \| ^ {2} d t d \bar {\rho} _ {0} (\boldsymbol {x} _ {0}). \tag {14}
$$
Consequently, by defining the trajectory-wise loss (recall  $\pmb{x}(t) = X_t(\pmb{x}_0)$ )
$$
R (f; \boldsymbol {x} _ {0}) = \int_ {0} ^ {T} \| \delta_ {t} \circ X _ {t} (\boldsymbol {x} _ {0}) \| ^ {2} d t = \int_ {0} ^ {T} \| \delta_ {t} (\boldsymbol {x} (t)) \| ^ {2} d t, \tag {15}
$$
we can write the potential function  $\boxed{8}$  as an expectation  $R(f) = \mathbb{E}_{\boldsymbol{x}_0\sim \bar{\rho}_0}[R(f;\boldsymbol{x}_0)]$ . Similarly, when  $f$  is parameterized as  $f = f_{\theta}$ , we obtain the expectation form  $\nabla_{\theta}R(f_{\theta}) = \mathbb{E}_{\boldsymbol{x}_0\sim \bar{\rho}_0}[\nabla_{\theta}R(f_{\theta};\boldsymbol{x}_0)]$ .
We show  $\nabla_{\theta}R(f_{\theta};\pmb{x}_0)$  can be computed accurately, via the adjoint method (for completeness see the derivation of the adjoint method in appendix  $\boxed{\mathbf{D}}$ ). As a recap, suppose that we can write  $R(f_{\theta};\pmb{x}_0)$  in a standard ODE-constrained form  $R(f_{\theta};\pmb{x}_0) = \ell (\theta) = \int_0^T g(t,\pmb{s}(t),\theta)dt$ , where  $\{\pmb {s}(t)\}_{t\in [0,T]}$  is the solution to the ODE  $\frac{d}{dt}\pmb {s}(t) = \psi (t,\pmb {s}(t);\theta)$  with  $\pmb {s}(0) = \pmb {s}_0$ , and  $\psi$  is a known transition function. The adjoint method states that the gradient  $\frac{d}{d\theta}\ell (\theta)$  can be computed as
$$
\text {(A d j o i n t M e t h o d)} \quad \frac {d \ell}{d \theta} = \int_ {0} ^ {T} a (t) ^ {\top} \frac {\partial \psi}{\partial \theta} (t, s (t); \theta) + \frac {\partial g}{\partial \theta} (t, s (t); \theta) d t. \tag {16}
$$
where  $a(t)$  solves the final value problems  $\frac{d}{dt} a(t)^{\top} + a(t)^{\top}\frac{\partial\psi}{\partial s} (t,\mathbf{s}(t);\theta) + \frac{\partial g}{\partial s} (t,\mathbf{s}(t);\theta) = 0, a(T) = 0$ . In the following, we focus on how  $R(f_{\theta};\pmb{x}_0)$  can be written in the above ODE-constrained form.
Write  $R(f_{\theta};x_0)$  in ODE-constrained form Expanding the definition of  $\delta_t$  in equation (11) gives
$$
\delta_ {t} (\boldsymbol {x} (t)) = f _ {t} (\boldsymbol {x} (t)) - \left(- \nabla V (\boldsymbol {x} (t)) + K * \rho_ {t} ^ {f} (\boldsymbol {x} (t)) - \nu \nabla \log \rho_ {t} ^ {f} (\boldsymbol {x} (t))\right). \tag {17}
$$
Note that in the above quantity,  $f$  and  $V$  are known functions. Moreover, it is known that  $\nabla \log \rho_t^f(\boldsymbol{x}(t))$  admits a closed form dynamics (e.g. see Proposition 2 in [Shen et al., 2022])
$$
\frac {d}{d t} \nabla \log \rho_ {t} ^ {f} (\boldsymbol {x} (t)) = - \nabla \left(\operatorname {d i v} f _ {t} (\boldsymbol {x} (t); \theta)\right) - \left(\mathcal {J} _ {f _ {t}} (\boldsymbol {x} (t); \theta)\right) ^ {\top} \nabla \log \rho_ {t} ^ {f} (\boldsymbol {x} (t)), \tag {18}
$$
which allows it to be explicitly computed by starting from  $\nabla \log \bar{\rho}_0(x_0)$  and integrating over time (recall that  $\bar{\rho}_0$  is known). Here  $\mathcal{J}_{f_t}$  denotes the Jacobian matrix of  $f_{t}$ . Consequently, all we need to handle is the convolution term  $K*\rho_t^f (\pmb {x}(t))$ .
A common choice to approximate the convolution operation is via Monte-Carlo integration: Let  $\pmb{y}_i(t) \stackrel{\mathrm{iid}}{\sim} \rho_t^f$  for  $i = 1, \dots, N$  and denote an empirical approximation of  $\rho_t^f$  by  $\mu_N^{\rho_t^f} = \frac{1}{N} \sum_{i=1}^{N} \delta_{\pmb{y}_i(t)}$ , where  $\delta_{\pmb{y}_i(t)}$  denotes the Dirac measure at  $\pmb{y}_i(t)$ . We approximate the convolution term in equation (17) in different ways for the Coulomb and the Biot-Savart interactions:
1. For the Coulomb type kernel (3), we first approximate  $K * \rho_t^f$  with  $K_c * \rho_t^f$ , where
$$
K _ {c} (\boldsymbol {x}) \stackrel {\text {d e f}} {=} \left\{ \begin{array}{l l} K (\boldsymbol {x}) & \text {i f} \| \boldsymbol {x} \| > c, \\ 0 & \text {i f} \| \boldsymbol {x} \| \leq c. \end{array} \right. \tag {19}
$$
If  $\rho_t^f$  is bounded in  $\mathcal{X}$ , we have
$$
\sup  _ {\boldsymbol {x} \in \mathcal {X}} \| (K - K _ {c}) * \rho_ {t} ^ {f} (\boldsymbol {x}) \| = \sup  _ {\boldsymbol {x} \in \mathcal {X}} \| \int_ {\| \boldsymbol {x} - \boldsymbol {y} \| \leq c} \frac {\boldsymbol {x} - \boldsymbol {y}}{\| \boldsymbol {x} - \boldsymbol {y} \| ^ {d}} \rho_ {t} ^ {f} (\boldsymbol {y}) d \boldsymbol {y} \| \leq \| \rho_ {t} ^ {f} \| _ {\mathcal {L} ^ {\infty} (\mathcal {X})} \int_ {\| \boldsymbol {y} \| \leq c} \frac {1}{\| \boldsymbol {y} \| ^ {d - 1}} d \boldsymbol {y}.
$$
To compute the integral on the right-hand side, we will switch to polar coordinates  $(r,\psi)$ :
$$
\int_ {| \boldsymbol {y} | \leq c} \frac {1}{| \boldsymbol {y} | ^ {d - 1}} \mathrm {d} \boldsymbol {y} = \int_ {0} ^ {c} \mathrm {d} r \frac {1}{r ^ {d - 1}} \int_ {\Psi} \mathrm {d} \psi J _ {(r, \psi)} \leq \int_ {0} ^ {c} \mathrm {d} r = c. \tag {20}
$$