SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Christian Fiedler<sup>1</sup> Michael Herty<sup>2</sup> Sebastian Trimpe<sup>3</sup>
$^{1,3}$ Institute for Data Science in Mechanical Engineering (DSME) RWTH Aachen University, Aachen, Germany {fiedler,trimpe}@dsme.rwth-aachen.de
$^{2}$  Institute for Geometry and Practical Mathematics (IGPM) RWTH Aachen University, Aachen, Germany herty@igpm.rwth-aachen.de
In many applications of machine learning, a large number of variables are considered. Motivated by machine learning of interacting particle systems, we consider the situation when the number of input variables goes to infinity. First, we continue the recent investigation of the mean field limit of kernels and their reproducing kernel Hilbert spaces, completing the existing theory. Next, we provide results relevant for approximation with such kernels in the mean field limit, including a representative theorem. Finally, we use these kernels in the context of statistical learning in the mean field limit, focusing on Support Vector Machines. In particular, we show mean field convergence of empirical and infinite-sample solutions as well as the convergence of the corresponding risks. On the one hand, our results establish rigorous mean field limits in the context of kernel methods, providing new theoretical tools and insights for large-scale problems. On the other hand, our setting corresponds to a new form of limit of learning problems, which seems to have not been investigated yet in the statistical learning theory literature.
Models with many variables play an important role in many fields of mathematical and physical sciences. In this context, going to the limit of infinitely many variables is an important analysis and modeling approach. Interacting particle systems are a classic example; these are usually modeled as dynamical systems describing the temporal evolution of many interacting objects. In physics, such systems were first investigated in the context of gas dynamics, cf. [1]. Since even small volumes of gases typically contain an enormous number of molecules, a microscopic modeling approach quickly becomes infeasible and one considers the evolution of densities instead [2]. In the past decades, interacting particle systems arising from many different domains have been considered, for example, animal movement (inter alia swarms of birds, schools of fish, colonies of microorganisms) [3, 4], social and political dynamics [5, 6], crowd modeling and control (pedestrian movement, gathering at large events like football games or concerts) [7-9], swarms of robots [10-12] or vehicular traffic (in particular, traffic jams) [13]. There is now a vast literature on such applications, and we refer to the surveys [14-16] as starting points. A prototypical example of such a system is given by  $\dot{x}_i = \frac{1}{M}\sum_{j=1}^{M}\phi(x_i,x_j)(x_j - x_i)$ , for  $i = 1,\dots,M$ , where  $M \in \mathbb{N}_{+}$  particles or agents are modelled by their state  $x_i \in \mathbb{R}^d$ ,  $i = 1,\dots,M$ , evolving according to some interaction rule  $\phi:\mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$ . Typical questions then concern the long-term behavior of such systems, in particular, emergent phenomena like consensus or alignment [17]. While first-principles modeling
has been very successful for interacting particle systems in physical domains, using this approach to model the interaction rules in complex domains like social and opinion dynamics, pedestrian and animal movement or vehicular traffic, can be problematic. Therefore, learning interaction rules from data has been recently intensively investigated, for example, in the pioneering works [18, 19]. The data consists typically of (sampled) trajectories of the particle states, potentially with measurement noise, and the goal is to learn a good approximation of the interaction rule  $\phi$ .
Our work is motivated by a related problem. Frequently, the state of such a complex multiagent system can be easily measured or estimated, e.g., by video recordings or image snapshots for bird swarms or schools of fish, and microscopy recordings for microorganism colonies; aerial imaging for human crowds (e.g., via quadcopters); and polling and social media analysis for opinion dynamics. However, some interesting features of the whole system might be more difficult to measure. For example, how a swarm of birds or a school of fish will react to an external stimulus (like an approaching predator), given the current state of the population. Such a reaction could be a change of density or spread of the population, or a change in mean velocity. Another example is given by features of a society in opinion dynamics (average happiness, aggression potential, susceptibility to adversarial interventions), given the current "opinion state". Measuring such features can be difficult, for example, due to a required intervention. Formally, such a feature is a functional  $F_{M}:(\mathbb{R}^{d})^{M}\to \mathbb{R}$  of the current state of the system, and since the state is often easy to measure, it would be useful to have an explicit mapping from state to feature of interest. However, since first principles modeling is unlikely to be successful in the domains considered here, it is promising to learn such a mapping from data. We can formalize this as a standard supervised learning task: The data set consists of  $D_N^{[M]} = ((\vec{x}_1,y_1),\ldots ,(\vec{x}_N,y_N))$  where  $\vec{x}_n\in (\mathbb{R}^d)^M$  are snapshot measurements of the particle states (corresponding to the input of the functional) and  $y_{n}\in \mathbb{R}$  is the value of the functional of interest, potentially with measurement noise, at snapshot state  $\vec{x}_n$ . Let us assume an additive noise model, i.e.,  $y_{n} = F_{M}(\vec{x}_{n}) + \epsilon_{n}$  for  $n = 1,\dots ,N$ , where  $\epsilon_1,\ldots ,\epsilon_N\in \mathbb{R}$  are noise variables. This is now a regression problem that could be solved, for example, using a Support Vector Machine (SVM) [20]. Note that for this we need a kernel  $k_{M}:(\mathbb{R}^{d})^{M}\times (\mathbb{R}^{d})^{M}\rightarrow \mathbb{R}$  on  $(\mathbb{R}^d)^M$ .
Similarly to classical physical examples like gas dynamics, the case of a large number of particles is also relevant in modern complex interacting particle systems. Since this poses computational and modeling challenges, it can be advantageous to go also here to a kinetic level and model the evolution of the particle distribution instead of every individual particle. It is well-established how to derive a kinetic partial differential equation from ordinary differential equations systems on the particle level, for example, using the Boltzmann equation or via a mean field limit, cf. [17] for an overview in the context of multi-agent systems. Formally, instead of trajectories of particle states of the form  $[0,T]\ni t\mapsto \vec{x} (t)\in (\mathbb{R}^d)^M$ , we then have trajectories of probability measures  $[0,T]\ni t\mapsto \mu (t)\in \mathcal{P}(\mathbb{R}^d)$ . This immediately raises the question of whether the learning setup outlined above also allows a corresponding kinetic limit. More precisely, let  $K\subseteq \mathbb{R}^{d}$  be compact and assume that all particles remain confined to this compactum, i.e.,  $x_{i}(t)\in K$  for all  $i = 1,\dots ,M$  and all  $t\in [0,T]$  under the microscopic dynamics. If the underlying dynamics have a mean field limit, then it is reasonable to assume that the finite-input functionals  $F_{M}:K^{M}\to \mathbb{R}$  converge also in mean field to some  $F:\mathcal{P}(K)\to \mathbb{R}$  for  $M\rightarrow \infty$ , see Section 2 for a precise definition of this notion. In turn, we can now formulate a corresponding learning problem on the mean field level: A data set is then given by  $D_N = ((\mu_1,y_1),\ldots ,(\mu_N,y_N))$ , where  $\mu_{n}\in \mathcal{P}(K)$  are snapshots of the particle state distribution over time and  $y_{n}\in \mathbb{R}$  are again potentially noisy measurements of the functional. Assuming an additive noise model, this corresponds to  $y_{n} = F(\mu_{n}) + \epsilon_{n}$ ,  $n = 1,\dots ,N$ . If we want to use an SVM on the kinetic level, we need a kernel  $k:\mathcal{P}(K)\times \mathcal{P}(K)\to \mathbb{R}$  on probability distributions. There are several options available for this, see e.g. [21]. However, assuming that all ingredients of the learning problem arise as a mean field limit, this naturally leads to the question of whether a mean field limit of kernels exists, and what this means for the relation of the learning problems on the finite-input and kinetic level. In [22], this reasoning has motivated the introduction and investigation of the mean field limit of kernels. In the present work, we extend the theory of these kernels and investigate them in the context of statistical learning theory. In particular, since in practice one would use the mean field kernels on microscopic data with large, but finite  $M$ , we need convergence results of the various objects appearing in statistical learning with kernels. Exactly such results are provided in Section 4.
This means the dynamics on the level of individual particles.
Finally, we would like to stress that the technical developments here are independent of the motivation outlined above, in that they apply to mean field limits of functions and kernels that do not necessarily arise from the dynamics of interacting particle systems.
Contributions Our contributions cover three closely related aspects. 1) We extend and complete the theory of mean field limit kernels and their RKHSs (Section 2). In Theorem 2.3, we precisely describe the relationship between the RKHS of the finite-input kernels and the RKHS of the mean field kernel, completing the results from [22]. In particular, this allows us to interpret the latter RKHS as the mean field limit of the former RKHSs. Furthermore, in Lemma 2.4 and 2.5, we provide inequalities for the corresponding RKHS norms, which are necessary for  $\Gamma$ -convergence arguments. 2) We provide results relevant for approximation with mean field limit kernels (Section 3). With Proposition 3.1, we give a first result on the approximation power of mean field limit kernels, and in Theorem 3.3 we can also provide a representative theorem for these kernels. For its proof, we use a  $\Gamma$ -convergence argument, which is to the best of our knowledge the first time this technique has been used in the context of kernel methods. 3) We investigate the mean field limit of kernels in the context of statistical learning theory (Section 4). We first establish an appropriate mean field limit setup for statistical learning problems, based on a slightly stronger mean field limit existence result than available so far, cf. Proposition 2.1. To the best of our knowledge, this is a new form of a limit for learning problems. In this setup, we then provide existence, uniqueness, and representative theorems for empirical and (using an apparently new notion of mean field convergence of probability distributions) infinite-sample solutions of SVMs, cf. Proposition 4.3 and 4.5. Finally, under a uniformity assumption, we can also establish convergence of the minimal risks in Proposition 4.7.
Our developments are relevant from two different perspectives: on the one hand, they constitute a theoretical proof-of-concept that the mean field limit can be "pulled through" the (kernel-based) statistical learning theory setup. In particular, this demonstrates that rigorous theoretical results can be transferred through the mean field limit, similar to works in the context of control of interacting particle systems, see e.g. [23]. On the other hand, our setup appears to be a new variant of a large-number-of-variables limit in the context of machine learning, complementing established settings like infinite-width neural networks [24].
Due to space constraints, all proofs and some additional technical results have been placed in the supplementary material.
2 Kernels and their RKHSs in the mean field limit
Setup and preliminaries Let  $(X,d_X)$  be a compact metric space and denote by  $\mathcal{P}(X)$  the set of Borel probability measures on  $X$ . We endow  $\mathcal{P}(X)$  with the topology of weak convergence of probability measures. Recall that for  $\mu_n,\mu \in \mathcal{P}(X)$ , we say that  $\mu_{n}\rightarrow \mu$  weakly if for all bounded and continuous  $f:X\to \mathbb{R}$  (since  $X$  is compact, this is equivalent to  $f$  continuous) we have  $\lim_{n\to \infty}\int_X\phi (x)\mathrm{d}\mu_n(x)\to \int_X\phi (x)\mathrm{d}\mu (x)$ . The topology of weak convergence can be metrized by the Kantorovich-Rubinstein metric  $d_{\mathrm{KR}}$ , defined by
$$
d _ {\mathrm {K R}} \left(\mu_ {1}, \mu_ {2}\right) = \sup  \left\{\int_ {X} \phi (x) \mathrm {d} \left(\mu_ {1} - \mu_ {2}\right) (x) \mid \phi : X \rightarrow \mathbb {R} \text {i s} 1 \text {- L i p s c h i t z} \right\}.
$$
Note that since  $X$  is compact and hence separable, the Kantorovich-Rubinstein metric is equal to the 1-Wasserstein metric here. Furthermore,  $\mathcal{P}(X)$  is compact in this topology. For  $M\in \mathbb{N}_{+}$  and  $\vec{x}\in X^M$ , denote the  $i$ -th component of  $\vec{x}$  by  $x_{i}$ , and define the empirical measure for  $\vec{x}$  by  $\hat{\mu} [\vec{x}] = \frac{1}{M}\sum_{i = 1}^{M}\delta_{x_i}$  where  $\delta_{x}$  denotes the Dirac measure centered at  $x\in X$ . The empirical measures are dense in  $\mathcal{P}(X)$  w.r.t. the Kantorovich-Rubinstein metric. Additionally, define  $d_{\mathrm{KR}}^2:\mathcal{P}(X)^2\times \mathcal{P}(X)^2\to \mathbb{R}_{\geq 0}$  by  $d_{\mathrm{KR}}^2 ((\mu_1,\mu_1'),(\mu_2,\mu_2')) = d_{\mathrm{KR}}(\mu_1,\mu_2) + d_{\mathrm{KR}}(\mu_1',\mu_2')$ , and note that  $(\mathcal{P}(X)^2,d_{\mathrm{KR}}^2)$  is a compact metric space. Moreover, denote the set of permutations on  $\{1,\ldots ,M\}$  by  $\mathcal{S}_M$ , and for a tuple  $\vec{x}\in X^M$  and permutation  $\sigma \in S_M$  define  $\sigma \vec{x} = (x_{\sigma (1)},\dots,x_{\sigma (M)})$ . Finally, we recall some well-known definitions and results from the theory of reproducing kernel Hilbert spaces, following [20, Chapter 4]. For an arbitrary set  $\mathcal{X}\neq \emptyset$  and a Hilbert space  $(H,\langle \cdot ,\cdot \rangle_H)$  of functions on  $\mathcal{X}$ , we say that a map  $k:\mathcal{X}\times \mathcal{X}\rightarrow \mathbb{R}$  is a reproducing kernel for  $H$  if 1)  $k(\cdot ,x)\in H$  for all  $x\in \mathcal{X}$ ; 2) for all  $x\in \mathcal{X}$  and  $f\in H$  we have  $f(x) = \langle f,k(\cdot ,x)\rangle_{H}$ . Note that if a reproducing kernel exists, it is unique. If such a Hilbert space has a reproducing kernel, we call  $H$  a reproducing kernel Hilbert space (RKHS) and  $k$  its (reproducing) kernel. It is well-known that a reproducing kernel is
symmetric and positive semidefinite, and that every symmetric and positive semidefinite function has a unique RKHS for which it is the reproducing kernel. For brevity, if  $k$  is symmetric and positive semidefinite, or equivalently, if it is the reproducing kernel of an RKHS, we call  $k$  simply a kernel, and denote by  $(H_k, \langle \cdot, \cdot \rangle_k)$  its unique associated RKHS. Define also  $H_k^{\mathrm{pre}} = \operatorname{span}\{k(\cdot, x) \mid x \in \mathcal{X}\}$ , then for  $f = \sum_{n=1}^{N} \alpha_n k(\cdot, x_n) \in H_k^{\mathrm{pre}}$  and  $g = \sum_{m=1}^{M} \beta_m k(\cdot, y_m) \in H_k^{\mathrm{pre}}$  we have  $\langle f, g \rangle_k = \sum_{n=1}^{N} \sum_{m=1}^{M} \alpha_n \beta_m k(y_m, x_n)$ , and  $H_k^{\mathrm{pre}}$  is dense in  $H_k$ .
The mean field limit of functions and kernels Given  $f_{M}:X^{M}\to \mathbb{R},M\in \mathbb{N}_{+}$ , and  $f:\mathcal{P}(X)\to \mathbb{R}$ , we say that  $f_{M}$  converges in mean field to  $f$  and that  $f$  is the (or a) mean field limit of  $f_{M}$ , if  $\lim_{M\to \infty}\sup_{\vec{x}\in X^M}|f_M(\vec{x}) - f(\hat{\mu} [\vec{x}])| = 0$ . In this case, we write  $f_{M}\xrightarrow{\mathcal{P}_{1}}f$ . Let now  $(Y,d_Y)$  be another metric space and  $f_{M}:X^{M}\times Y\to \mathbb{R},M\in \mathbb{N}_{+}$ , and  $f:\mathcal{P}(X)\times Y\to \mathbb{R}$ , then we say that  $f_{M}$  converges in mean field to  $f$  and that  $f$  is the (or a) mean field limit of  $f_{M}$ , if for all compact  $K\subseteq Y$  we have
$$
\lim  _ {M \rightarrow \infty} \sup  _ {\vec {x} \in X ^ {M}, y \in K} | f _ {M} (\vec {x}, y) - f (\hat {\mu} [ \vec {x} ], y) | = 0. \tag {1}
$$
and also write  $f_{M} \xrightarrow{\mathcal{P}_{1}} f$ . The following existence results for mean field limits is slightly more general than what is available in the literature, and it is essentially a direct generalization of [25, Theorem 2.1], in the form of [26, Lemma 1.2].
Proposition 2.1. Let  $(X,d_X)$  be a compact metric space and  $(Z,d_Z)$  a metric space that has a countable basis  $(U_n)_n$  such that  $\bar{U}_n$  is compact for all  $n\in \mathbb{N}$ . Let  $f_{M}:X^{M}\times Z\to \mathbb{R}$ ,  $M\in \mathbb{N}_{+}$ , be a sequence of functions fulfilling the following conditions: 1) (Symmetry in  $\vec{x}$ )² For all  $M\in \mathbb{N}_{+}$ ,  $\vec{x}\in X^M$ ,  $z\in Z$  and permutations  $\sigma \in S_M$ , we have  $f_{M}(\sigma \vec{x},z) = f_{M}(\vec{x},z)$ ; 2) (Uniform boundedness) There exists  $B_{f}\in \mathbb{R}_{\geq 0}$  and a function  $b:Z\to \mathbb{R}_{\geq 0}$  such that  $\forall M\in \mathbb{N}_{+},\vec{x}\in X^{M},z\in z:|f_{M}(\vec{x},z)|\leq B_{f} + b(z)$ ; 3) (Uniform Lipschitz continuity) There exists some  $L_{f}\in \mathbb{R}_{>0}$  such that for all  $M\in \mathbb{N}_{+}$ ,  $\vec{x}_1,\vec{x}_2\in X^M$ ,  $z_1,z_2\in Z$  we have  $|f_{M}(\vec{x}_{1},z_{1}) - f_{M}(\vec{x}_{2},z_{2})|\leq L_{f}(d_{\mathrm{KR}}(\hat{\mu} [\vec{x}_1],\hat{\mu} [\vec{x}_2]) + d_Z(z_1,z_2))$ .
Then there exists a subsequence  $(f_{M_{\ell}})_{\ell}$  and a continuous function  $f:\mathcal{P}(X)\times Z\to \mathbb{R}$  such that  $f_{M_{\ell}}\xrightarrow{\mathcal{P}_1} f$  for  $\ell \to \infty$ . Furthermore,  $f$  is  $L_{f}$ -Lipschitz continuous and there exists  $B_F\in \mathbb{R}_{\geq 0}$  such that for all  $\mu \in \mathcal{P}(X),z\in Z$  we have  $|f(\mu ,z)|\leq B_F + b(z)$ .
We now turn to the mean field limit of kernels as introduced in [22]: Given  $k_{M}:X^{M}\times X^{M}\to \mathbb{R}$  and  $k:\mathcal{P}(X)\times \mathcal{P}(X)\to \mathbb{R}$ , we say that  $k_{M}$  converges in mean field to  $k$  and that  $k$  is the (or a) mean field limit of  $k_{M}$ , if
$$
\lim  _ {M \rightarrow \infty} \sup  _ {\vec {x}, \vec {x} ^ {\prime} \in X ^ {M}} | k _ {M} (\vec {x}, \vec {x} ^ {\prime}) - k (\hat {\mu} [ \vec {x} ], \hat {\mu} [ \vec {x} ^ {\prime} ]) | = 0. \tag {2}
$$
In this case we write  $k_{M}\xrightarrow{\mathcal{P}_{1}}k$
For convenience, we recall [22, Theorem 2.1], which ensures the existence of a mean field limit of a sequence of kernels.
Proposition 2.2. Let  $k_{M}:X^{M}\times X^{M}\to \mathbb{R}$  be a sequence of kernels fulfilling the following conditions. 1) (Symmetry in  $\vec{x}$ ) For all  $M\in \mathbb{N}_{+}$ ,  $\vec{x},\vec{x}^{\prime}\in X^{M}$  and permutations  $\sigma \in S_M$  we have  $k_{M}(\sigma \vec{x},\vec{x}^{\prime}) = k_{M}(\vec{x},\vec{x}^{\prime})$ ; 2) (Uniform boundedness) There exists  $C_k\in \mathbb{R}_{\geq 0}$  such that  $\forall M\in \mathbb{N}_{+},\vec{x},\vec{x}^{\prime}\in X^{M}:|k_{M}(\vec{x},\vec{x}^{\prime})|\leq C_{k}$ ; 3) (Uniform Lipschitz continuity) There exists some  $L_{k}\in \mathbb{R}_{>0}$  such that for all  $M\in \mathbb{N}_{+}$ ,  $\vec{x}_1,\vec{x}_1',\vec{x}_2,\vec{x}_2'\in X^M$  we have  $|k_{M}(\vec{x}_{1},\vec{x}_{1}') - k_{M}(\vec{x}_{2},\vec{x}_{2}')|\leq L_{k}d_{\mathrm{KR}}^{2}[(\hat{\mu} [\vec{x}_{1}],\hat{\mu} [\vec{x}_{1}'],(\hat{\mu} [\vec{x}_{2}],\hat{\mu} [\vec{x}_{2}'])]$ .
Then there exists a subsequence  $(k_{M_{\ell}})_{\ell}$  and a continuous kernel  $k:\mathcal{P}(X)\times \mathcal{P}(X)\to \mathbb{R}$  such that  $k_{M_{\ell}}\xrightarrow{\mathcal{P}_1}k$ , and  $k$  is also bounded by  $C_k$ .
Let  $k_{M}:X^{M}\times X^{M}\to \mathbb{R}$  be a given sequence of kernels fulfilling the conditions of Proposition 2.2. Then there exists a subsequence  $(k_{M_{\ell}})_{\ell}$  converging in mean field to a kernel  $k:\mathcal{P}(X)\times \mathcal{P}(X)\to \mathbb{R}$ .
<sup>2</sup>As is well-known, cf. [26, Remark 1.1.3], this condition is actually implied by the next condition. However, as usual in the kinetic theory literature, we kept this condition for emphasis.
Figure 1: The kernel  $k$  arises as the mean field limit (MFL) of the kernels  $k_{M}$  (Proposition 2.2). Every uniformly norm-bounded sequence  $f_{M} \in H_{M}$ ,  $M \in \mathbb{N}_{+}$ , has an MFL in  $H_{k}$ , and every function  $f \in H_{k}$  arises as such an MFL (Theorem 2.3). Based on [22, Figure 1].
From now on, we only consider this subsequence and denote it again by  $(k_{M})_{M}$ , i.e.,  $k_{M} \xrightarrow{\mathcal{P}_{1}} k$ . Unless noted otherwise, every time we need a further subsequence, we will make this explicit.
The RKHS of the mean field limit kernel Denote by  $H_{M} \coloneqq H_{k_{M}}$  the (unique) RKHS corresponding to kernel  $k_{M}$  and denote by  $H_{k}$  the unique RKHS of  $k$ . For basic properties of these objects as well as classes of suitable kernels we refer to [22].
We clarify the relation between  $H_{M}$  and  $H_{k}$  in the next result.
Theorem 2.3. 1) For every  $f \in H_k$ , there exists a sequence  $f_M \in H_M$ ,  $M \in \mathbb{N}_+$ , such that  $f_M \xrightarrow{\mathcal{P}_1} f$ . 2) Let  $f_M \in H_M$  be sequence such that there exists  $B \in \mathbb{R}_{\geq 0}$  with  $\| f_M \|_M \leq B$  for all  $M \in \mathbb{N}_+$ . Then there exists a subsequence  $(f_{M_\ell})_\ell$  and  $f \in H_k$  with  $f_{M_\ell} \xrightarrow{\mathcal{P}_1} f$  and  $\| f \|_k \leq B$ .
In other words, on the one hand, every RKHS function from  $H_{k}$  arises as a mean field limit of RKHS functions from  $H_{M}$ . On the other hand, every uniformly norm-bounded sequence of RKHS functions  $(f_{M})_{M}$  has a mean field limit in  $H_{k}$ .
Note that the preceding result is considerably stronger than the corresponding results in [22]: In contrast to [22, Theorem 4.4] we do not need to go to another subsequence in the first item, and we ensure that the mean field limit  $f$  is contained in  $H_{k}$  (and norm-bounded by the same uniform bound), which was missing from Corollary 4.3 in the same reference.
The relation between the kernels  $k_{M}$  and their RKHSs  $H_{M}$ , and the mean field limit kernel  $k$  and its RKHS  $H_{k}$  is illustrated as a commutative diagram in Figure 1. In order to arrive at the mean field RKHS  $H_{k}$ , on the one hand, we consider the mean field limit  $k$  of the  $k_{M}$ , and then form the corresponding RKHS  $H_{k}$ . This is essentially the content of Proposition 2.2. On the other hand, we can first go from the kernel  $k_{M}$  to the associated unique RKHS  $H_{M}$  (for each  $M \in \mathbb{N}_{+}$ ). Theorem 2.3 then says that  $H_{k}$  can be interpreted as a mean field limit of the RKHSs  $H_{M}$ , since every function in  $H_{k}$  arises as a mean field limit of a sequence of functions from the  $H_{M}$ , and every uniformly norm-bounded sequence of such functions has a mean field limit that is in  $H_{k}$ .
Next, we state two technical results that will play an important role in the following developments, and which might be of independent interest. They describe lim inf and lim sup inequalities required for  $\Gamma$ -convergence arguments used later on.
Lemma 2.4. Let  $f_{M} \in H_{M}$ ,  $M \in \mathbb{N}_{+}$ , and  $f \in H_{k}$  such that  $f_{M} \xrightarrow{\mathcal{P}_{1}} f$ , then
$$
\| f \| _ {k} \leq \lim  _ {M \rightarrow \infty} \inf  _ {M} \| f _ {M} \| _ {M}. \tag {3}
$$
Lemma 2.5. Let  $f \in H_k$ . Then there exist  $f_M \in H_M$ ,  $M \in \mathbb{N}_+$ , such that  $\lim_{M \to \infty} \sup_{\vec{x} \in X^M} |f_M(\vec{x}) - f(\hat{\mu}[\vec{x}])| = 0$ , and
$$
\lim  _ {M \rightarrow \infty} \| f _ {M} \| _ {M} \leq \| f \| _ {k}. \tag {4}
$$
3 Approximation with kernels in the mean field limit