SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} R _ {2} = \left| \sum_ {\ell , \ell^ {\prime} = 1} ^ {L _ {n (M)}} \alpha_ {\ell} ^ {(n (M))} \alpha_ {\ell^ {\prime}} ^ {(n (M))} k (\hat {\mu} [ \vec {x} _ {M} ^ {(n (M), \ell^ {\prime})} ], \hat {\mu} [ \vec {x} _ {M} ^ {(n (M), \ell^ {\prime})} ]) - \sum_ {\ell , \ell^ {\prime} = 1} ^ {L _ {n (M)}} \alpha_ {\ell} ^ {(n (M))} \alpha_ {\ell^ {\prime}} ^ {(n (M))} k (\mu_ {\ell^ {\prime}} ^ {(n (M))}, \mu_ {\ell} ^ {(n (M))}) \right| \\ \leq \sum_ {\ell , \ell^ {\prime} = 1} ^ {L _ {n (M)}} | \alpha_ {\ell} ^ {(n (M))} | | \alpha_ {\ell^ {\prime}} ^ {(n (M))} | | k (\hat {\mu} [ \vec {x} _ {M} ^ {(n (M), \ell^ {\prime})} ], \hat {\mu} [ \vec {x} _ {M} ^ {(n (M), \ell^ {\prime})} ]) - k (\mu_ {\ell^ {\prime}} ^ {(n (M))}, \mu_ {\ell} ^ {(n (M))}) | \\ \leq L _ {k} \sum_ {\ell , \ell^ {\prime} = 1} ^ {L _ {n (M)}} | \alpha_ {\ell} ^ {(n (M))} | | \alpha_ {\ell^ {\prime}} ^ {(n (M))} | \left(d _ {\mathrm {K R}} (\hat {\mu} [ \vec {x} _ {M} ^ {(n (M), \ell)} ], \mu_ {\ell} ^ {(n (M))}) + d _ {\mathrm {K R}} (\hat {\mu} [ \vec {x} _ {M} ^ {(n (M), \ell^ {\prime})} ], \mu_ {\ell^ {\prime}} ^ {(n (M))})\right) \\ \leq \frac {\epsilon_ {n (M)} ^ {2}}{2}. \\ \end{array}
$$
$$
\begin{array}{l} \left\| f _ {M} \right\| _ {M} ^ {2} \leq \left\| f \right\| _ {k} ^ {2} + | R _ {1} | + | R _ {2} | \\ \leq \| f \| _ {k} ^ {2} + \frac {\epsilon_ {n (M)} ^ {2}}{2} + \frac {\epsilon_ {n (M)} ^ {2}}{2} \\ \leq \| f \| _ {k} ^ {2} + \epsilon^ {2}, \\ \end{array}
$$
so  $\| f_M\| _M\leq \| f\| _k + \epsilon$  for all  $M\geq M_{n_{\epsilon}}$ , and since  $\epsilon >0$  was arbitrary, we finally get  $\lim \sup_{M\to \infty}\| f_M\| _M\leq \| f\| _k.$
Finally, we can now provide the proof for the central Theorem 2.3.
Proof. of Theorem 2.3 The first statement is part of Lemma 2.5. Let us turn to the second statement: The existence of the subsequence  $(f_{M_{\ell}})_{\ell}$  and the continuous function  $f:\mathcal{P}(X)\to \mathbb{R}$  with  $f_{M_{\ell}}\xrightarrow{\mathcal{P}_1}f$  was shown in [22, Corollary 4.3], so we only have to ensure that  $f\in H_k$  with  $\| f\| _k\leq B$ . For this, we use the characterization of RKHS functions from Theorem B.1. In particular, we will utilize the notation introduced there.
Step 1 Let  $(\vec{\mu},\vec{\alpha})\in \mathcal{P}(X)^N\times \mathbb{R}^N$ . We show that if  $\mathcal{W}(\vec{\mu},\vec{\alpha},k) = 0$ , then  $\mathcal{E}(\vec{\mu},\vec{\alpha},f) = 0$ .
Assume that  $\mathcal{W}(\vec{\mu},\vec{\alpha},k) = 0$ . If  $B = 0$ , then  $f_{M}\equiv 0$  and  $f_{M_{\ell}}\xrightarrow{\mathcal{P}_1}f$  implies that  $f\equiv 0$ , so the claim is clear in this case. Assume now  $B > 0$ , let  $\epsilon >0$  be arbitrary and for  $n = 1,\ldots ,N$ , choose sequences  $\vec{x}_n^{[M]}\in X^M$  such that  $\vec{x}_n^{[M]}\xrightarrow{d_{\mathrm{KR}}}\mu_n$  for  $M\to \infty$ . For convenience, define  $\vec{X}^{[M]} = \left(\vec{x}_1^{[M]}\dots \vec{x}_N^{[M]}\right)$ . Choose now  $\ell_{\epsilon}\in \mathbb{N}$  such that for all  $M\geq M_{\ell_{\epsilon}}$  we get  $\mathcal{W}(\vec{X}^{[M]},\vec{\alpha},k_M)\leq \epsilon /B$ . This is possible since  $k_{M}\xrightarrow{\mathcal{P}_{1}}k$  together with the continuity of  $k_{M}$  and  $k$  as well as  $\vec{x}_n^{[M]}\xrightarrow{d_{\mathrm{KR}}}\mu_n$  for  $M\to \infty$  and all  $n = 1,\ldots ,N$  implies that  $\mathcal{W}(\vec{X}^{[M]},\vec{\alpha},k_M)\to \mathcal{W}(\vec{\mu},\vec{\alpha},k) = 0$ . Let now  $\ell \geq \ell_{\epsilon}$  be arbitrary and observe that  $f_{M}\in H_{M}$  implies  $\mathcal{N}(f_M,k_M) < \infty$  according to Theorem B.1, so in particular  $\mathcal{D}(\vec{X}^{[M_\ell ]},\vec{\alpha},f_{M_\ell},k_{M_\ell}) < \infty$ .
If  $\mathcal{W}(\vec{X}^{[M_\ell]},\vec{\alpha},k_{M_\ell}) = 0$ , then we get that  $\mathcal{E}(\vec{X}^{[M_\ell]},\vec{\alpha},f_{M_\ell}) = 0 \leq \epsilon$  since  $\mathcal{D}(\vec{X}^{[M_\ell]},\vec{\alpha},f_{M_\ell},k_{M_\ell}) < \infty$ , which implies by definition that  $\mathcal{E}(\vec{X}^{[M_\ell]},\vec{\alpha},f_{M_\ell}) = 0$ .
If  $\mathcal{W}(\vec{X}^{[M_\ell]},\vec{\alpha},k_{M_\ell}) > 0$  , then we have
$$
\frac {\mathcal {E} (\vec {X} ^ {[ M _ {\ell} ]} , \vec {\alpha} , f _ {M _ {\ell}})}{\mathcal {W} (\vec {X} ^ {[ M _ {\ell} ]} , \vec {\alpha} , k _ {M _ {\ell}})} = \mathcal {D} (\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, f _ {M _ {\ell}}, k _ {M _ {\ell}}) \leq \mathcal {N} (f _ {M _ {\ell}}, k _ {M _ {\ell}}) = \| f _ {M _ {\ell}} \| _ {M _ {\ell}} \leq B,
$$
$$
\mathcal {E} (\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, f _ {M _ {\ell}}) \leq B \mathcal {W} (\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, k _ {M _ {\ell}}) \leq \epsilon .
$$
Since  $f_{M_{\ell}} \xrightarrow{\mathcal{P}_{1}} f$  together with the continuity of  $f_{M}$  and  $f$  as well as  $\vec{x}_n^{[M]}\xrightarrow{d_{\mathrm{KR}}}\mu_n$  implies that  $\mathcal{E}(\vec{X}^{[M_{\ell}]},\vec{\alpha},f_{M_{\ell}})\to \mathcal{E}(\vec{\mu},\vec{\alpha},f)$ , we get that  $\mathcal{E}(\vec{\mu},\vec{\alpha},f) \leq \epsilon$ , and since  $\epsilon >0$  was arbitrary we arrive at  $\mathcal{E}(\vec{\mu},\vec{\alpha},f) \leq 0$ .
Assume now that  $\mathcal{E}(\vec{\mu},\vec{\alpha},f) < 0$ . This implies that there exist  $\delta >0$  and  $\ell_{\delta}\in \mathbb{N}$  such that for all  $\ell \geq \ell_{\delta}$  we have  $\mathcal{E}(\vec{X}^{[M_{\ell}]},\vec{\alpha},f_{M_{\ell}})\leq -\delta < 0$ , since  $\mathcal{E}(\vec{X}^{[M_{\ell}]},\vec{\alpha},f_{M_{\ell}})\to \mathcal{E}(\vec{\mu},\vec{\alpha},f)$ . Let  $\ell \geq \ell_{\delta}$ , then we get that  $\mathcal{E}(\vec{X}^{[M_{\ell}]}, - \vec{\alpha},f_{M_{\ell}})\geq \delta >0$  and we have  $\mathcal{W}(\vec{X}^{[M_{\ell}]}, - \vec{\alpha},k_{M_{\ell}}) = \mathcal{W}(\vec{X}^{[M_{\ell}]},\vec{\alpha},k_{M_{\ell}}) > 0$ . We can then continue with
$$
\begin{array}{l} \frac {\delta}{\mathcal {W} \left(\vec {X} ^ {[ M _ {\ell} ]} , \vec {\alpha} , k _ {M _ {\ell}}\right)} \leq \frac {\mathcal {E} \left(\vec {X} ^ {[ M _ {\ell} ]} , - \vec {\alpha} , f _ {M _ {\ell}}\right)}{\mathcal {W} \left(\vec {X} ^ {[ M _ {\ell} ]} , - \vec {\alpha} , k _ {M _ {\ell}}\right)} \\ \leq \mathcal {D} \left(\vec {X} ^ {[ M _ {\ell} ]}, - \vec {\alpha}, f _ {M _ {\ell}}, k _ {M _ {\ell}}\right) \\ \leq \mathcal {N} \left(f _ {M _ {\ell}}, k _ {M _ {\ell}}\right) \\ = \| f _ {M _ {\ell}} \| _ {M _ {\ell}} \leq B, \\ \end{array}
$$
which implies that  $\mathcal{W}(\vec{X}^{[M_\ell]}, -\vec{\alpha}, k_{M_\ell}) = \mathcal{W}(\vec{X}^{[M_\ell]}, \vec{\alpha}, k_{M_\ell}) \geq \delta / B$ . But since  $\mathcal{W}(\vec{X}^{[M_\ell]}, \vec{\alpha}, k_{M_\ell}) \to \mathcal{W}(\vec{\mu}, \vec{\alpha}, k)$ , this implies that  $\mathcal{W}(\vec{\mu}, \vec{\alpha}, k) \geq \delta / B > 0$ , a contradiction. Altogether,  $\mathcal{E}(\vec{\mu}, \vec{\alpha}, f) = 0$ .
Step 2 Let  $(\vec{\mu},\vec{\alpha})\in \mathcal{P}(X)^N\times \mathbb{R}^N$  . If  $\mathcal{W}(\vec{\mu},\vec{\alpha},k) > 0$  and  $\mathcal{E}(\vec{\mu},\vec{\alpha},f) > 0$  , then
$$
\frac {\mathcal {E} (\vec {\mu} , \vec {\alpha} , f)}{\mathcal {W} (\vec {\mu} , \vec {\alpha} , k)} \leq B.
$$
To show this, let  $\alpha > 1$  and  $\beta \in (0,1)$  be arbitrary. Define
and observe that  $\epsilon_{\alpha}, \epsilon_{\beta} > 0$ . Furthermore, for all  $n = 1, \ldots, N$  choose a sequence  $\vec{x}_n^{[M]} \in X^M$  such that  $\vec{x}_n^{[M]} \xrightarrow{d_{\mathrm{KR}}} \mu_n$  for  $M \to \infty$ , and define  $\vec{X}^{[M]} = \left( \vec{x}_1^{[M]} \dots \vec{x}_N^{[M]} \right)$ . Choose  $\ell_{\epsilon} \in \mathbb{N}_{+}$  such that for all  $\ell \geq \ell_{\epsilon}$  we have
$$
\begin{array}{l} \left| \mathcal {E} \left(\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, f _ {M _ {\ell}}\right) - \mathcal {E} (\vec {\mu}, \vec {\alpha}, f) \right| \leq \epsilon_ {\alpha} \\ \left| \mathcal {W} \left(\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, k _ {M _ {\ell}}\right) - \mathcal {W} (\vec {\mu}, \vec {\alpha}, k) \right| \leq \epsilon_ {\beta} \\ \end{array}
$$
and  $\mathcal{W}(\vec{X}^{[M_\ell]},\vec{\alpha},k_{M_\ell}) > 0$ . Such an  $\ell_{\epsilon}$  exists because  $k_{M}\xrightarrow{\mathcal{P}_{1}}k$  together with the continuity of  $k_{M}$  and  $k$  as well as the convergence of  $\vec{x}_n^{[M]}$  to  $\mu_{n}$  imply that  $\mathcal{W}(\vec{X}^{[M_\ell]},\vec{\alpha},k_{M_\ell})\to \mathcal{W}(\vec{\mu},\vec{\alpha},k)$ , and  $f_{M_{\ell}}\xrightarrow{\mathcal{P}_{1}}f$  together with the continuity of  $f_{M}$  and  $f$  imply that  $\mathcal{E}(\vec{X}^{[M_\ell]},\vec{\alpha},f_{M_\ell})\to \mathcal{E}(\vec{\mu},\vec{\alpha},f)$ .
Let now  $\ell \geq \ell_{\epsilon}$  be arbitrary. By definition of  $\epsilon_{\alpha}$  we get  $\alpha \epsilon_{\alpha} \leq (\alpha - 1)\mathcal{E}(\vec{\mu}, \vec{\alpha}, f)$ , which in turn leads to
$$
\begin{array}{l} \epsilon_ {\alpha} \leq \epsilon_ {\alpha} - \alpha \epsilon_ {\alpha} + (\alpha - 1) \mathcal {E} (\vec {\mu}, \vec {\alpha}, f) \\ = - (\alpha - 1) \epsilon_ {\alpha} + (\alpha - 1) \mathcal {E} (\vec {\mu}, \vec {\alpha}, f) \\ = (\alpha - 1) \left(\mathcal {E} \left(\vec {\mu}, \vec {\alpha}, f\right) - \epsilon_ {\alpha}\right) \\ \leq (\alpha - 1) \mathcal {E} \left(\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, f _ {M _ {\ell}}\right), \\ \end{array}
$$
where we used in the last inequality that  $\alpha - 1 > 0$  and by choice of  $\ell_{\epsilon}$  we have  $\mathcal{E}(\vec{\mu}, \vec{\alpha}, f) \leq \mathcal{E}(\vec{X}^{[M_{\varepsilon}]}, \vec{\alpha}, f_{M_{\varepsilon}}) + \epsilon_{\alpha}$ . We can then continue with
$$
\begin{array}{l} \mathcal {E} (\vec {\mu}, \vec {\alpha}, f) \leq \mathcal {E} \left(\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, f _ {M _ {\ell}}\right) + \epsilon_ {\alpha} \\ \leq \mathcal {E} \left(\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, f _ {M _ {\ell}}\right) + (\alpha - 1) \mathcal {E} \left(\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, f _ {M _ {\ell}}\right) \\ = \alpha \mathcal {E} \left(\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, f _ {M _ {\ell}}\right). \\ \end{array}
$$
Next, by definition of  $\epsilon_{\beta}$  and choice of  $\ell_{\epsilon}$  we find that
$$
\begin{array}{l} \mathcal {W} \left(\vec {X} ^ {[ M _ {\ell} ]}, \vec {\alpha}, k _ {M _ {\ell}}\right) \leq \mathcal {W} (\vec {\mu}, \vec {\alpha}, k) + \epsilon_ {\beta} \\ = \mathcal {W} (\vec {\mu}, \vec {\alpha}, k) + (1 / \beta - 1) \mathcal {W} (\vec {\mu}, \vec {\alpha}, k) \\ = (1 / \beta) \mathcal {W} (\vec {\mu}, \vec {\alpha}, k), \\ \end{array}
$$
$$
\frac {1}{\mathcal {W} (\vec {\mu} , \vec {\alpha} , k)} \leq \frac {1}{\beta \mathcal {W} (\vec {X} ^ {[ M _ {\ell} ]} , \vec {\alpha} , k _ {M _ {\ell}})}.
$$
Combining these results, we get that for all  $\ell \geq \ell_{\epsilon}$
$$
\frac {\mathcal {E} (\vec {\mu} , \vec {\alpha} , f)}{\mathcal {W} (\vec {\mu} , \vec {\alpha} , k)} \leq \frac {\alpha}{\beta} \frac {\mathcal {E} \left(\vec {X} ^ {[ M _ {\ell} ]} , \vec {\alpha} , f _ {M _ {\ell}}\right)}{\mathcal {W} \left(\vec {X} ^ {[ M _ {\ell} ]} , \vec {\alpha} , k _ {M _ {\ell}}\right)} \leq \frac {\alpha}{\beta} \mathcal {N} \left(f _ {M _ {\ell}}, k _ {M _ {\ell}}\right) = \frac {\alpha}{\beta} \| f _ {M _ {\ell}} \| _ {M _ {\ell}} \leq \frac {\alpha}{\beta} B.
$$
Since  $\alpha > 1$  and  $\beta \in (0,1)$  were arbitrary, this shows that
Step 3 Let  $(\vec{\mu},\vec{\alpha})\in \mathcal{P}(X)^N\times \mathbb{R}^N$  be arbitrary. If  $\mathcal{W}(\vec{\mu},\vec{\alpha},k) = 0$ , then we get from Step 1 that  $\mathcal{E}(\vec{\mu},\vec{\alpha},f) = 0\leq B$ . Assume now  $\mathcal{W}(\vec{\mu},\vec{\alpha},k) > 0$ . If  $\mathcal{E}(\vec{\mu},\vec{\alpha},f) = 0$ , then again  $\mathcal{E}(\vec{\mu},\vec{\alpha},f) = 0\leq B$ . If  $\mathcal{E}(\vec{\mu},\vec{\alpha},f) > 0$ , then Step 2 ensures that
$$
\frac {\mathcal {E} (\vec {\mu} , \vec {\alpha} , f)}{\mathcal {W} (\vec {\mu} , \vec {\alpha} , k)} = \mathcal {D} (\vec {\mu}, \vec {\alpha}, f, k) \leq B.
$$
Finally, if  $\mathcal{E}(\vec{\mu},\vec{\alpha},f) < 0$  , then again
$$
\frac {\mathcal {E} (\vec {\mu} , \vec {\alpha} , f)}{\mathcal {W} (\vec {\mu} , \vec {\alpha} , k)} = \mathcal {D} (\vec {\mu}, \vec {\alpha}, f, k) <   0 \leq B.
$$
Altogether, we get that  $\mathcal{D}(\vec{\mu},\vec{\alpha},f,k)\leq B$ . Since  $(\vec{\mu},\vec{\alpha})$  was arbitrary, maximization leads to  $\mathcal{N}(f,k)\leq B < \infty$ , hence  $f\in H_k$  and  $\| f\| _k = \mathcal{N}(f,k)\leq B$ .
A.2 Proofs for Section 3
In this section we provide the proofs for the results relating to approximation with kernels in the mean field limit.
Proof. of Proposition 3.1 Let  $f \in \mathcal{F}$  and  $\epsilon > 0$  be arbitrary. Let  $B \in \mathbb{R}_{\geq 0}$  and  $f_{M} \in \mathcal{F}_{M}$ ,  $\hat{f}_{M} \in H_{M}$ ,  $M \in \mathbb{N}_{+}$ , such that  $f_{M} \xrightarrow{\mathcal{P}_{1}} f$ ,  $\| f_{M} - \hat{f}_{M} \| \leq \frac{\epsilon}{5}$  and  $\| \hat{f}_{M} \|_{M} \leq B$  for all  $M \in \mathbb{N}_{+}$  (exist by definition of  $\mathcal{F}$ ). Theorem 2.3 ensures that there exists a subsequence  $(f_{M_{\ell}})_{\ell}$  and  $\hat{f} \in H_{k}$  with  $\| \hat{f} \|_{k} \leq B$  such that  $\hat{f}_{M_{\ell}} \xrightarrow{\mathcal{P}_{1}} \hat{f}$  for  $\ell \to \infty$ . Choose now  $L_{1} \in \mathbb{N}_{+}$  such that for all  $\ell \geq L_{1}$  we have
$$
\sup _ {\vec {x} \in X ^ {M _ {\ell}}} | \hat {f} _ {M _ {\ell}} (\vec {x}) - \hat {f} (\hat {\mu} [ \vec {x} ]) | \leq \frac {\epsilon}{5}
$$
$$
\sup _ {\vec {x} \in X ^ {M} _ {\ell}} | f _ {M _ {\ell}} (\vec {x}) - f (\hat {\mu} [ \vec {x} ]) | \leq \frac {\epsilon}{5}.
$$
Let now  $\mu \in \mathcal{P}(X)$  be arbitrary and choose a sequence  $\vec{x}_M \in X^M$  with  $\hat{\mu}[\vec{x}_M] \xrightarrow{d_{\mathrm{KR}}}\mu$ . Finally, let  $L_2 \in \mathbb{N}_+$  such that for all  $\ell \geq L_2$  we have
$$
| f (\mu) - f (\hat {\mu} [ \vec {x} _ {M _ {\ell}} ]) | \leq \frac {\epsilon}{5}
$$
$$
| \hat {f} (\mu) - \hat {f} (\hat {\mu} [ \vec {x} _ {M _ {\ell}} ]) | \leq \frac {\epsilon}{5}
$$
(such an  $L_{2}$  exists due to the continuity of  $f$  and  $\hat{f}$ ).
We now have for  $\ell \geq \max \{L_1, L_2\}$  that
$$
\begin{array}{l} | f (\mu) - \hat {f} (\mu) | \leq | f (\mu) - f (\hat {\mu} [ \vec {x} _ {M _ {\ell}} ]) | + | f (\hat {\mu} [ \vec {x} _ {M _ {\ell}} ]) - f _ {M _ {\ell}} (\vec {x} _ {M _ {\ell}}) | + | f _ {M _ {\ell}} (\vec {x} _ {M _ {\ell}}) - \hat {f} _ {M _ {\ell}} (\vec {x} _ {M _ {\ell}}) | \\ + \left| \hat {f} _ {M _ {\ell}} \left(\vec {x} _ {M _ {\ell}}\right) - \hat {f} \left(\hat {\mu} \left[ \vec {x} _ {M _ {\ell}} \right]\right) \right| + \left| \hat {f} \left(\hat {\mu} \left[ \vec {x} _ {M _ {\ell}} \right]\right) - \hat {f} (\mu) \right| \\ \leq \frac {\epsilon}{5} + \frac {\epsilon}{5} + \frac {\epsilon}{5} + \frac {\epsilon}{5} + \frac {\epsilon}{5} = \epsilon . \\ \end{array}
$$
Since  $\mu$  was arbitrary, the result follows.
Proof. of Remark 3.2 We first show that  $\mathcal{F}$  is a subvectorspace. Let  $f, g \in \mathcal{F}$  and  $\lambda \in \mathbb{R}$ ,  $\epsilon > 0$  be arbitrary. W.l.o.g. we can assume  $\lambda \neq 0$ . Choose sequences  $f_{M}, g_{M} \in \mathcal{F}_{M}, \hat{f}_{M}, \hat{g}_{M} \in H_{M}, M \in \mathbb{N}_{+}$ , and constants  $B_{f}, B_{g} \in \mathbb{R}_{\geq 0}$  from the definition of  $\mathcal{F}$  for  $f, \frac{\epsilon}{2|\lambda|}$ , and  $g, \frac{\epsilon}{2}$ , respectively. Let  $M \in \mathbb{N}_{+}$ ,  $\vec{x} \in X^{M}$  be arbitrary, then
$$
| \lambda f _ {M} (\vec {x}) + g (\vec {x}) - (\lambda f (\hat {\mu} [ \vec {x} ]) - g (\hat {\mu} [ \vec {x} ])) | \leq | \lambda | | f _ {M} (\vec {x}) - f (\hat {\mu} [ \vec {x} ]) | + | g _ {M} (\vec {x}) - g (\hat {\mu} [ \vec {x} ]) |
$$
together with  $f_{M} \xrightarrow{\mathcal{P}_{1}} f$ ,  $g_{M} \xrightarrow{\mathcal{P}_{1}} g$  shows that  $\lambda f_{M} + g_{M} \xrightarrow{\mathcal{P}_{1}} \lambda f + g$ .
Next, we have for all  $M \in \mathbb{N}_{+}$  that
$$
\left\| \left(\lambda f _ {M} + g _ {M}\right) - \left(\lambda \hat {f} _ {M} + \hat {g} _ {M}\right) \right\| _ {\infty} \leq | \lambda | \| f _ {M} - \hat {f} _ {M} \| _ {\infty} + \| g _ {M} - \hat {g} _ {M} \| _ {\infty} \leq | \lambda | \frac {\epsilon}{2 | \lambda |} + \frac {\epsilon}{2} = \epsilon .
$$
$$
\left\| \lambda \hat {f} _ {M} + \hat {g} _ {M} \right\| _ {M} \leq | \lambda | \| \hat {f} _ {M} \| _ {M} + \| \hat {g} _ {M} \| _ {M} \leq | \lambda | B _ {f} + B _ {g},
$$
establishing that  $(\lambda \hat{f}_M + \hat{g}_M)_M$  is uniformly norm-bounded. Altogether, we have that  $\lambda f + g \in \mathcal{F}$ . We now turn to the second claim. Let  $(f^{(n)})_n \subseteq \mathcal{F}$  such that  $f^{(n)} \to f$  for some  $f \in C^0(\mathcal{P}(X), \mathbb{R})$  and for all  $\bar{\epsilon} > 0$  there exist  $f_M^{(n)} \in \mathcal{F}_M$ ,  $\hat{f}_M^{(n)} \in H_M$ ,  $(\rho_M)_M \subseteq \mathbb{R}_{\geq 0}$  and  $B^{(n)} \in \mathbb{R}_{\geq 0}$  with  $\rho_M \searrow 0$ ,  $\|f_M^{(n)} - \hat{f}_M^{(n)}\|_{\infty} \leq \bar{\epsilon}$  and  $\|f_M^{(n)}\|_M \leq B^{(n)}$  for all  $n$ ,  $M \in \mathbb{N}_+$ , and
$$
\sup  _ {\vec {x} \in X ^ {M}} | f _ {M} ^ {(n)} (\vec {x}) - f ^ {(n)} (\hat {\mu} [ \vec {x} ]) | \leq \rho_ {M}
$$
for all  $n, M \in \mathbb{N}_+$ . We now show that  $f \in \mathcal{F}$ . For this, let  $\epsilon > 0$  be arbitrary and choose  $f_M^{(n)} \in \mathcal{F}_M$ ,  $\hat{f}_M^{(n)} \in H_M$ ,  $(\rho_M)_M \subseteq \mathbb{R}_{\geq 0}$  and  $B^{(n)} \in \mathbb{R}_{\geq 0}$  as above with  $\bar{\epsilon} = \frac{\epsilon}{4}$ . Let  $N \in \mathbb{N}_+$  be such that  $\| f^{(m)} - f^{(n)} \|_{\infty} \leq \frac{\epsilon}{4}$  for all  $m, n \geq N$  (such an  $N$  exists since  $(f^{(n)})_n$  converges in  $C^0(\mathcal{P}(X), \mathbb{R})$  and hence is a Cauchy sequence). Furthermore, let  $M_\rho \in \mathbb{N}_+$  be such that for all  $M \geq M_\rho$  we have  $\rho_M \leq \frac{\epsilon}{4}$ . Define now  $f_M = f_M^{(M)}$  and  $\hat{f}_M = \hat{f}_M^{(M)}$  for  $M = 1, \ldots, M_\rho - 1$ , and  $f_M = f_M^{(M + N)}$ ,  $\hat{f}_M = \hat{f}_M^{(N)}$  for  $M \geq M_\rho$ .
Step 1 Let  $M \geq M_{\rho}$  and  $\vec{x} \in X^{M}$  be arbitrary. We have
$$
\begin{array}{l} \left| f _ {M} (\vec {x}) - f \left(\hat {\mu} [ \vec {x} ]\right) \right| = \left| f _ {M} ^ {(N + M)} (\vec {x}) - f \left(\hat {\mu} [ \vec {x} ]\right) \right| \\ \leq \left| f _ {M} ^ {(N + M)} (\vec {x}) - f ^ {(N + M)} (\hat {\mu} [ \vec {x} ]) \right| + \left| f ^ {(N + M)} (\hat {\mu} [ \vec {x} ]) - f (\hat {\mu} [ \vec {x} ]) \right| \\ \leq \rho_ {M} + \| f ^ {(N + M)} - f \| _ {\infty}, \\ \end{array}
$$
and since the right hand side (which is independent of  $\vec{x}$ ) converges to 0 for  $M \to \infty$ , we get  $f_{M} \xrightarrow{\mathcal{P}_{1}} f$ .
Step 2 For  $M = 1,\dots ,M_{\rho}$  we get
$$
\| f _ {M} - \hat {f} _ {M} \| _ {\infty} = \| f _ {M} ^ {(M)} - \hat {f} _ {M} ^ {(M)} \| _ {\infty} \leq \bar {\epsilon} \leq \epsilon .
$$
Let now  $M \geq M_{\rho}$  and  $\vec{x} \in X^{M}$  be arbitrary. We have
$$
\begin{array}{l} \left| f _ {M} (\vec {x}) - \hat {f} _ {M} (\vec {x}) \right| = \left| f _ {M} ^ {(M + N)} (\vec {x}) - \hat {f} _ {M} ^ {(N)} (\vec {x}) \right| \\ \leq \left| f _ {M} ^ {(M + N)} (\vec {x}) - f ^ {(N + M)} (\hat {\mu} [ \vec {x} ]) \right| + \left| f ^ {(N + M)} (\hat {\mu} [ \vec {x} ]) - f ^ {(N)} (\hat {\mu} [ \vec {x} ]) \right| \\ + \left| f ^ {(N)} (\hat {\mu} [ \vec {x} ]) - f _ {M} ^ {(N)} (\vec {x}) \right| + \left| f _ {M} ^ {(N)} (\vec {x}) - \hat {f} _ {M} ^ {(N)} (\vec {x}) \right| \\ \leq \sup  _ {\vec {x} ^ {\prime} \in X ^ {M}} | f _ {M} ^ {(M + N)} (\vec {x} ^ {\prime}) - f ^ {(M + N)} (\hat {\mu} [ \vec {x} ^ {\prime} ]) | + \| f ^ {(M + N)} - f ^ {(N)} \| _ {\infty} \\ + \sup  _ {\vec {x} ^ {\prime} \in X ^ {M}} | f ^ {(N)} (\hat {\mu} [ \vec {x} ^ {\prime} ]) - f _ {M} ^ {(N)} (\vec {x} ^ {\prime}) | + \| f _ {M} ^ {(N)} - \hat {f} _ {M} ^ {(N)} \| _ {\infty} \\ \leq \rho_ {M} + \frac {\epsilon}{4} + \rho_ {M} + \bar {\epsilon} \\ \leq 4 \frac {\epsilon}{4} = \epsilon , \\ \end{array}
$$
and since  $\vec{x}\in X^M$  was arbitrary, we get  $\| f_M - \hat{f}_M\|_{\infty}\leq \epsilon$
Step 3 For  $M = 1, \ldots, M_{\rho} - 1$  we get by construction that  $\| \hat{f}_M \|_M = \| \hat{f}_M^{(M)} \|_M \leq B^{(M)}$ , and for  $M \geq M_{\rho}$  we find  $\| \hat{f}_M \|_M = \| \hat{f}_M^{(N)} \|_M \leq B^{(N)}$ . Altogether, we get for  $M \in \mathbb{N}_+$  that
$$
\| \hat {f} _ {M} \| _ {M} \leq \max  \{B ^ {(1)}, \dots , B ^ {\left(M _ {\rho} - 1\right)}, B ^ {(N)} \}.
$$
Combining the three steps establishes that  $f\in \mathcal{F}$
Finally, here is the proof of the representer theorem in the mean field limit.
Proof. of Theorem 3.3 The existence and uniqueness of  $f_{M}$  and  $f$  follows from the well-known representer theorem (applied to all  $k_{M}$  and  $k$ ).
We now turn to the convergence of the minimizers. For all  $M \in \mathbb{N}_{+}$  we have
$$
\lambda \| f _ {M} ^ {*} \| _ {M} \leq L \left(f _ {M} ^ {*} \left(\vec {x} _ {1} ^ {[ M ]}\right), \dots , f _ {M} ^ {*} \left(\vec {x} _ {N} ^ {[ M ]}\right)\right) + \lambda \| f \| _ {M} \leq L (0, \dots , 0),
$$
i.e.,  $\| f_M^*\| _M\leq L(0,\dots ,0) / \lambda$  . Define
$$
\begin{array}{l} \mathcal {L} _ {M}: H _ {M} \to \mathbb {R} _ {\geq 0}, f \mapsto L (f (\vec {x} _ {1} ^ {[ M ]}), \dots , f (\vec {x} _ {N} ^ {[ M ]})) + \lambda \| f \| _ {M} \\ \mathcal {L}: H _ {k} \rightarrow \mathbb {R} _ {\geq 0}, f \mapsto L (f (\mu_ {1}), \dots , f (\mu_ {N})) + \lambda \| f \| _ {k}, \\ \end{array}
$$
and let  $f_{M} \in H_{M}$  with  $f_{M} \xrightarrow{\mathcal{P}_{1}} f$  for some  $f \in H_{k}$ . The continuity of  $f_{M}$ ,  $f$  and  $L$  as well as  $\vec{x}_n^{[M]} \xrightarrow{d_{\mathrm{KR}}} \mu_n$  for  $M \to \infty$  and all  $n = 1, \ldots, N$ , imply then that
$\lim_{M\to \infty}L(f_M(\vec{x}_1^{[M]}),\ldots ,f_M(\vec{x}_N^{[M]}) = L(f(\mu_1),\ldots ,f(\mu_N)).$  Combining this with Lemma 2.4 leads to
$$
\mathcal{L}(f)\leq \liminf_{M\to \infty}\mathcal{L}_{M}(f).
$$
Let now  $f \in H_{k}$  be arbitrary and let  $f_{M} \in H_{M}$  be the sequence from Lemma 2.5. Using the same arguments as above we find that
$$
\limsup_{M\to \infty}\mathcal{L}_{M}(f_{M})\leq \| f\|_{k}.
$$
We have shown that  $\mathcal{L}_M \xrightarrow{\Gamma} \mathcal{L}$  and hence Proposition B.3 ensures that there exists a subsequence  $(f_{M_\ell}^*)_\ell$  such that  $f_{M_\ell}^* \xrightarrow{\mathcal{P}_1} f^*$  and  $\mathcal{L}_{M_\ell}(f_{M_\ell}^*) \to \mathcal{L}(f^*)$ .
A.3 Proofs for Section 4
Proof. of Lemma 4.2 That  $\ell$  is nonnegative is clear from the proof of Proposition 2.1. Let now all  $\ell_M$  be convex and let  $\mu \in \mathcal{P}(X)$ ,  $y \in Y, t_1, t_2 \in \mathbb{R}$  and  $\lambda \in (0,1)$  be arbitrary, and define  $I = [\min\{t_1, t_2\}, \max\{t_1, t_2\}]$ . Furthermore, let  $\vec{x}_M \in X^M$  with  $\vec{x}_M \xrightarrow{d_{\mathrm{KR}}} \mu$  for  $M \to \infty$  and  $\epsilon > 0$  be arbitrary. Choose now  $M$  so large that
$$
\begin{array}{l} |\ell (\mu ,y,\lambda t_{1} + (1 - \lambda)t_{2}) - \ell (\hat{\mu} [\vec{x}_{M}],y,\lambda t_{1} + (1 - \lambda)t_{2})|\\ \leq \frac{\epsilon}{6}\sup_{\substack{\vec{x}\in X^{M}\\ y^{\prime}\in Y,t\in I}}|\ell_{M}(\vec{x},y^{\prime},t^{\prime}) - \ell (\hat{\mu} [\vec{x} ],y^{\prime},t^{\prime})| \\ \leq \frac {\epsilon}{6}. \\ \end{array}
$$
This is possible due to the continuity of  $\ell$ , as well as  $\ell_M \xrightarrow{\mathcal{P}_1} \ell$ . We then have
$$
\begin{array}{l} \ell (\mu , y, \lambda t _ {1} + (1 - \lambda) t _ {2}) \leq \ell (\hat {\mu} [ \vec {x} ], y, \lambda t _ {1} + (1 - \lambda) t _ {2}) + \frac {\epsilon}{6} \\ \leq \ell_ {M} (\vec {x} _ {M}, y, \lambda t _ {1} + (1 - \lambda) t _ {2}) + \frac {\epsilon}{3} \\ \leq \lambda \ell_ {M} (\vec {x} _ {M}, y, t _ {1}) + (1 - \lambda) \ell_ {M} (\vec {x} _ {M}, y, t _ {2}) + \frac {\epsilon}{3} \\ \leq \lambda \ell (\hat {\mu} [ \vec {x} _ {M} ], y, t _ {1}) + (1 - \lambda) \ell (\hat {\mu} [ \vec {x} _ {M} ], y, t _ {2}) + \frac {\epsilon}{3} + (\lambda + 1 - \lambda) \frac {\epsilon}{6} \\ \leq \lambda \ell (\mu , y, t _ {1}) + (1 - \lambda) \ell (\mu , y, t _ {2}) + \epsilon , \\ \end{array}
$$
and since  $\epsilon > 0$  was arbitrary, this establishes