SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \left\| \mathbf {h} _ {\alpha} ^ {(L)} - \mathbf {h} _ {\beta} ^ {(L)} \right\| _ {2} = \left\| \varphi_ {L} \left(\int_ {\mathbb {M} _ {L - 1}} \mathbf {h} _ {-} ^ {(L - 1)} d \alpha\right) - \varphi_ {L} \left(\int_ {\mathbb {M} _ {L - 1}} \mathbf {h} _ {-} ^ {(L - 1)} d \beta\right) \right\| _ {2} \\ \leq \| \varphi_ {L} \| _ {L} \cdot \left\| \int_ {\mathbb {M} _ {L - 1}} \mathbf {h} _ {-} ^ {(L - 1)} d \alpha - \int_ {\mathbb {M} _ {L - 1}} \mathbf {h} _ {-} ^ {(L - 1)} d \beta \right\| _ {2} \\ \leq \| \varphi_ {L} \| _ {L} \cdot \| \mathbf {h} _ {-} ^ {(L - 1)} \| _ {B L} \cdot \mathrm {W} (\alpha , \beta) \\ = \| \varphi_ {L} \| _ {L} \cdot \left(\| \mathbf {h} _ {-} ^ {(L - 1)} \| _ {\infty} + \| \mathbf {h} _ {-} ^ {(L - 1)} \| _ {L}\right) \cdot d _ {\mathbb {W}, L} (\alpha , \beta) \\ \leq \| \varphi_ {L} \| _ {L} \cdot \left(\| \mathbf {h} _ {-} ^ {(L - 1)} \| _ {\infty} + C _ {\boldsymbol {\varphi} _ {L - 1}}\right) \cdot d _ {\mathbb {W}, L} (\alpha , \beta) \\ = C _ {\varphi} \cdot d _ {\mathbb {W}, L} (\alpha , \beta) \\ \end{array}
$$
for all  $\alpha, \beta \in \mathbb{M}_h$  by the induction hypothesis. Hence, we get the first inequality. The second equality then follows from the first by the same reasoning as in the inductive step above.
It remains to prove the inequalities for  $d_{\mathbb{W},\infty}$  and  $\delta_{\mathbb{W},\infty}$ . These follow from the two just-proven inequalities. First, for all  $\alpha, \beta \in \mathcal{M}_{\leq 1}(\mathbb{M})$ , we have
$$
\begin{array}{l} \left\| \mathbf {h} _ {\alpha} ^ {(L)} - \mathbf {h} _ {\beta} ^ {(L)} \right\| _ {2} = \left\| \mathbf {h} _ {p _ {\infty}, L (\alpha)} ^ {(L)} - \mathbf {h} _ {p _ {\infty}, L (\beta)} ^ {(L)} \right\| _ {2} \leq C _ {\varphi} \cdot d _ {\mathbb {W}, L} \left(p _ {\infty , L} (\alpha), p _ {\infty , L} (\beta)\right) \\ \leq C _ {\varphi} \cdot L \cdot d _ {\mathbb {W}, \infty} (\alpha , \beta). \\ \end{array}
$$
Second, for all  $\mu ,\nu \in \mathcal{P}(\mathbb{M})$  , we have
$$
\begin{array}{l} \left\| \mathbf {h} _ {\mu} - \mathbf {h} _ {\nu} \right\| _ {2} = \left\| \mathbf {h} _ {(p _ {\infty}, L) * \mu} - \mathbf {h} _ {(p _ {\infty}, L) * \nu} \right\| _ {2} \leq C _ {(\boldsymbol {\varphi}, \psi)} \cdot \delta_ {\mathbb {W}, L} ((p _ {\infty}, L) * \mu , (p _ {\infty}, L) * \nu) \\ \leq C _ {(\varphi , \psi)} \cdot L \cdot \delta_ {\mathbb {W}, \infty} (\mu , \nu). \\ \end{array}
$$
C.3 Universality of message-passing graph neural networks
Here, we prove our universal approximation theorem for MPNNs on iterated degree measures, which states that the sets  $\mathcal{N}_t^1$  and  $\mathcal{N}\mathcal{N}_t^1$  are dense in  $C(\mathbb{M}_t,\mathbb{R})$  and  $C(\mathcal{P}(\mathbb{M}_t),\mathbb{R})$ , respectively. It follows, by simple inductive applications of the Stone-Weierstrass theorem, cf. Appendix A.4, to the set  $\mathcal{N}_t^1$ . Essentially, we show that  $\mathcal{N}_t^1$  satisfies all requirements of the Stone-Weierstrass theorem. Then, an application of the Stone-Weierstrass theorem combined with the definition of iterated degree measures
yields that  $\mathcal{N}_{t+1}^{1}$  separates points, which allows us to show that  $\mathcal{N}_{t+1}^{1}$  again satisfies all requirements of the Stone-Weierstrass theorem. In the following, to stress the dependence of  $\mathbf{h}_{-}^{(t)}$  and  $\mathbf{h}_{-}$  on the MPNN model  $\varphi$  and on the MPNN model  $\varphi$  and the Lipschitz function  $\psi$ , we sometimes write  $\varphi_{-}^{(t)}$  and  $(\varphi, \psi)$  instead, respectively.
Lemma 25. Let  $0 \leq t \leq \infty$ . The set  $\mathcal{N}_t^1$  is closed under multiplication and linear combinations, contains  $\mathbf{1}_{\mathbb{M}_t}$ , and separates points of  $\mathbb{M}_t$ .
Proof. Base of the induction. For  $t = 0$ , the claim trivially holds as  $\mathcal{N}_0^1$  contains precisely the constant functions and  $\mathbb{M}_0 = \{1\}$  is the one-point space.
Induction step. Let  $t > 0$ . Clearly,  $\mathcal{N}_t^1$  contains the all-one function  $\mathbf{1}_{\mathbb{M}_t}$  since we can always choose  $\varphi_t$  in an MPNN model to be the all-one function. Next, consider two functions  $\varphi_{-}^{(t)}$  and  $\varphi_{-}^{\prime (t)}$  from  $\mathcal{N}_t^1$  for  $t$ -layer MPNN models  $\varphi$  and  $\varphi'$ . Define  $\psi_{\mathrm{mul}}((x,y)^T)\coloneqq x\cdot y$  and  $\psi_{\mathrm{add}}((x,y)^T)\coloneqq x + c\cdot y$  for all  $x,y\in \mathbb{R}$ , where  $c\in \mathbb{R}$  is fixed. Then, define the MPNN model
$$
\varphi_ {\mathrm {m u l}} := \left(\varphi_ {0} \times \varphi_ {0} ^ {\prime}, \dots , \varphi_ {t - 1} \times \varphi_ {t - 1} ^ {\prime}, \psi_ {\mathrm {m u l}} \circ \left(\varphi_ {t} \times \varphi_ {t} ^ {\prime}\right)\right)
$$
and define  $\varphi_{\mathrm{add}}$  analogously via  $\psi_{\mathrm{add}}$ . Note that  $\varphi_{\mathrm{mul}}$  is in fact an MPNN model since multiplication on a compact, and hence, a bounded subset of  $\mathbb{R}^2$  is Lipschitz continuous. Then,
$$
\varphi_ {-} ^ {(t)} \cdot \varphi_ {-} ^ {\prime} ^ {(t)} = \varphi_ {\mathrm {m u l} -} ^ {(t)}
$$
$$
\varphi_ {-} ^ {(t)} + c \cdot \varphi_ {-} ^ {\prime} ^ {(t)} = \varphi_ {\mathrm {a d d} -} ^ {(t)},
$$
which implies that  $\mathcal{N}_t^1$  is closed under multiplication and linear combinations.
Finally, let  $\alpha, \beta \in \mathbb{M}_t = \mathcal{M}_{\leq 1}(\mathbb{M}_{t-1})$  with  $\alpha \neq \beta$ . By the induction hypothesis, the set  $\mathcal{N}_{t-1}^1$  is closed under multiplication and linear combinations, contains  $\mathbf{1}_{\mathbb{M}_{t-1}}$ , and separates points of  $\mathbb{M}_{t-1}$ . Hence, it is a subalgebra of  $C(\mathbb{M}_{t-1})$  that separates points and contains the constants. By the Stone-Weierstrass theorem,  $\mathcal{N}_{t-1}^1$  is dense in  $C(\mathbb{M}_{t-1})$ , which means that there is a  $(t-1)$ -layer MPNN model  $\varphi$  with output dimension one such that
$$
\int_ {\mathbb {M} _ {t - 1}} \varphi_ {-} ^ {(t - 1)} d \alpha \neq \int_ {\mathbb {M} _ {t - 1}} \varphi_ {-} ^ {(t - 1)} d \beta .
$$
Define the  $t$ -layer MPNN model  $\varphi' \coloneqq (\varphi_0, \dots, \varphi_{t-1}, \psi_{\mathrm{id}})$ , where  $\psi_{\mathrm{id}}(x) \coloneqq x$  for every  $x \in \mathbb{R}$ . Then,  $\varphi_{-}^{(t)} \in \mathcal{N}_t^1$ , separates  $\alpha$  and  $\beta$  since
$$
\boldsymbol {\varphi} _ {\alpha} ^ {\prime} ^ {(t)} = \int_ {\mathbb {M} _ {t - 1}} \boldsymbol {\varphi} _ {-} ^ {(t - 1)} d \alpha \neq \int_ {\mathbb {M} _ {t - 1}} \boldsymbol {\varphi} _ {-} ^ {(t - 1)} d \beta = \boldsymbol {\varphi} _ {\beta} ^ {\prime} ^ {(t)},
$$
cf. Section 2.
The claim for  $\mathcal{N}_{\infty}^{1}$  now easily follows:  $\mathcal{N}_{\infty}^{1}$  contains the all-one function  $\mathbf{1}_{\mathbb{M}} = \mathbf{1}_{\mathbb{M}_0} \circ p_{\infty,0} \in \mathcal{N}_{\infty}^{1}$  and separates points of  $\mathbb{M}$  since, if we have  $\alpha, \beta \in \mathbb{M}$  with  $\alpha \neq \beta$ , there is some  $t \in \mathbb{N}$  such that  $\alpha_t \neq \beta_t$ . By the already proven claim for  $\mathcal{N}_t^1$ , there is a  $t$ -layer MPNN model  $\varphi$  with output dimension one such that  $\varphi_{\alpha_t}^{(t)} \neq \varphi_{\beta_t}^{(t)}$ . Hence,  $\varphi_{-}^{(t)} \circ p_{\infty,t}(\alpha) = \varphi_{\alpha_t}^{(t)} \neq \varphi_{\beta_t}^{(t)} = \varphi_{-}^{(t)} \circ p_{\infty,t}(\beta)$  for  $\varphi_{-}^{(t)} \circ p_{\infty,t} \in \mathcal{N}_{\infty}^{1}$ .
To see that  $\mathcal{N}_{\infty}^{1}$  is closed under multiplication and linear combinations, let  $\varphi_{-}^{(t)} \circ p_{\infty, t}$  and  $\varphi_{-}^{\prime}(t^{\prime}) \circ p_{\infty, t^{\prime}}$  be two functions from  $\mathcal{N}_{\infty}^{1}$ , where  $\varphi$  and  $\varphi^{\prime}$  are  $t$ -layer and  $t^{\prime}$ -layer MPNN models with output dimension one, respectively. We assume that  $t \geq t^{\prime}$ ; the other case is analogous. Then,
$$
\varphi_ {-} ^ {\prime} \left(t ^ {\prime}\right) \circ p _ {\infty , t ^ {\prime}} = \varphi_ {-} ^ {\prime} \left(t ^ {\prime}\right) \circ p _ {t, t ^ {\prime}} \circ p _ {\infty , t}
$$
by the definition of  $\mathbb{M}_{\infty}$ . For now, assume that  $\varphi_{-}^{\prime (t^{\prime})} \circ p_{t,t^{\prime}} \in \mathcal{N}_t^1$ . Then, by the already proven claim for  $\mathcal{N}_t^1$ , we have  $\varphi_{-}^{(t)} \cdot (\varphi_{-}^{\prime (t^{\prime})} \circ p_{t,t^{\prime}}) \in \mathcal{N}_t^1$ , which means that
$$
\left(\boldsymbol {\varphi} _ {-} ^ {(t)} \circ p _ {\infty , t}\right) \cdot \left(\boldsymbol {\varphi} _ {-} ^ {\prime} \quad^ {(t ^ {\prime})} \circ p _ {\infty , t ^ {\prime}}\right) = \left(\boldsymbol {\varphi} _ {-} ^ {(t)} \cdot \left(\boldsymbol {\varphi} _ {-} ^ {\prime} \quad^ {(t ^ {\prime})} \circ p _ {t, t ^ {\prime}}\right)\right) \circ p _ {\infty , t} \in \mathcal {N} _ {\infty} ^ {1}.
$$
This implies that  $\mathcal{N}_{\infty}^{1}$  is closed under multiplication, and by analogous reasoning, we get that it is closed under linear combinations. To finish the proof, it remains to show that  $\varphi_{-}^{\prime}(t^{\prime})\circ p_{t,t^{\prime}}\in \mathcal{N}_{t}^{1}$ , which follows from the following claim.
Claim 26. Let  $n, t \geq 0$ . Let  $\varphi$  be a  $t$ -layer MPNN-model with  $d_t = n$ . Then,  $\mathbf{h}_{-}^{(t)} \circ p_{t+1,t} \in \mathcal{N}_{t+1}^n$ .
Base of the induction. For  $t = 0$ , we have  $\mathbf{h}_{\alpha}^{(0)} = \varphi_0$  for every  $\alpha \in \mathbb{M}_0$ , i.e., the only element of  $\mathbb{M}_0$  is mapped to  $\varphi_0 \in \mathbb{R}^n$ . Let  $\varphi_1 \colon \mathbb{R}^n \to \mathbb{R}^n$  be the constant function that maps all inputs to  $\varphi_0$  and define the 1-layer MPNN model  $\varphi' := (\varphi_0, \varphi_1)$ . Then,
$$
\boldsymbol {\varphi} _ {-} ^ {(0)} \circ p _ {1, 0} (\alpha) = \varphi_ {0} = \varphi_ {1} \left(\int_ {\mathbb {M} _ {0}} \varphi_ {0} d \alpha\right) = \boldsymbol {\varphi} _ {\alpha} ^ {(1)}
$$
for every  $\alpha \in \mathbb{M}_1$ , i.e.,  $\varphi_{-}^{(0)} \circ p_{1,0} = \varphi_{-}^{\prime (1)} \in \mathcal{N}_{1}^{n}$ .
Induction step. Let  $t > 0$  and  $\varphi$  be a  $t$ -layer MPNN model. Then, for every  $\alpha \in \mathbb{M}_{t + 1}$ , we have
$$
\begin{array}{l} \left(\varphi_ {-} ^ {(t)} \circ p _ {t + 1, t}\right) (\alpha) = \varphi_ {t} \left(\int_ {\mathbb {M} _ {t - 1}} \varphi_ {-} ^ {(t - 1)} d p _ {t + 1, t} (\alpha)\right) = \varphi_ {t} \left(\int_ {\mathbb {M} _ {t - 1}} \varphi_ {-} ^ {(t - 1)} d \left(p _ {t, t - 1}\right) _ {*} (\alpha)\right) \\ = \varphi_ {t} \left(\int_ {\mathbb {M} _ {t}} \varphi_ {-} ^ {(t - 1)} \circ p _ {t, t - 1} d \alpha\right). \\ \end{array}
$$
By the induction hypothesis, we have  $\varphi_{-}^{(t - 1)}\circ p_{t,t - 1}\in \mathcal{N}_{t}^{d_{t}}$ , i.e.,  $\varphi_{-}^{(t - 1)}\circ p_{t,t - 1} = \varphi_{-}^{\prime}(t)$  for a  $t$ -layer MPNN model  $\varphi^{\prime}$ . Hence, we have
$$
(\varphi_ {-} ^ {(t)} \circ p _ {t + 1, t}) (\alpha) = \varphi_ {t} \left(\int_ {\mathbb {M} _ {t}} \varphi_ {-} ^ {\prime} ^ {(t)} d \alpha\right) = \varphi_ {\alpha} ^ {\prime \prime} ^ {(t)}
$$
for every  $\alpha \in \mathbb{M}_{t + 1}$ , where  $\varphi'' \coloneqq (\varphi_0', \ldots, \varphi_t', \varphi_t)$  is an MPNN model with  $t + 1$  layers. Hence,  $\varphi_{-}^{(t)} \circ p_{t + 1,t} \in \mathcal{N}_{t + 1}^n$ .
With Lemma 25, we immediately obtain Theorem 4, which we restate here for better readability.
Theorem 4. Let  $0 \leq L \leq \infty$ . Then,  $\mathcal{N}_L^1$  is dense in  $C(\mathbb{M}_L, \mathbb{R})$  and  $\mathcal{N}\mathcal{N}_L^1$  is dense in  $C(\mathcal{P}(\mathbb{M}_L), \mathbb{R})$ .
Proof. By Lemma 25, the Stone-Weierstrass theorem is applicable to  $\mathcal{N}_L^1$ , and hence,  $\mathcal{N}_L^1$  is dense in  $C(\mathbb{M}_L,\mathbb{R})$ . We can then use this to show that  $\mathcal{N}\mathcal{N}_L^1$  is dense in  $C(\mathcal{P}(\mathbb{M}_L),\mathbb{R})$ . By the same arguments as in the inductive step in the proof of Lemma 25,  $\mathcal{N}\mathcal{N}_L^1$  is closed under multiplication and linear combinations, contains the all-one function, and separates points of  $\mathcal{P}(\mathbb{M}_L)$ . Hence, an application of the Stone-Weierstrass theorem yields that  $\mathcal{N}\mathcal{N}_L^1$  is dense in  $C(\mathcal{P}(\mathbb{M}_L),\mathbb{R})$ .
Theorem 4 then yields Corollary 5.
Corollary 5. Let  $0 \leq L \leq \infty$  and  $n > 0$ . Let  $\nu \in \mathcal{P}(\mathbb{M}_L)$  and  $(\nu_i)_i$  be a sequence with  $\nu_i \in \mathcal{P}(\mathbb{M}_L)$ . Then,  $\nu_i \to \nu$  if and only if  $\mathbf{h}_{\nu_i} \to \mathbf{h}_\nu$  for all  $L$ -layer MPNN models  $\varphi$  and Lipschitz  $\psi \colon \mathbb{R}^{d_L} \to \mathbb{R}^n$ .
Proof. First, let  $n = 1$ . When restricted to functions  $\mathbf{h}_{-} \in \mathcal{NN}_L^n$  of the form  $\mathbf{h}_{\nu} = \int_{\mathbb{M}_L} \mathbf{h}_{-}^{(L)} d\nu$ , i.e., the function  $\psi$  is the identity, the claim follows since  $\mathcal{N}_L^1$  is dense in  $C(\mathbb{M}_L, \mathbb{R})$  by Theorem 4 and the definition of the weak topology on  $\mathcal{P}(\mathbb{M}_L)$ , cf. Section 2. Since the function  $\psi$  in the definition of  $\mathbf{h}_{-} \in \mathcal{NN}_L^n$ , which, in general, is of the form  $\mathbf{h}_{\nu} = \psi \left( \int_{\mathbb{M}_L} \mathbf{h}_{-}^{(L)} d\nu \right)$ , is continuous, the equivalence also holds when considering all functions in the set  $\mathcal{NN}_L^n$ . Finally, since one can always consider the projection to a single component and conversely map a single real number to a vector of these numbers, the equivalence also holds in the case  $n > 1$ .
C.4 Equivalence of distances
Here, we formally state and prove Theorem 1. Let us first recall the informal statement from the main body of the paper.
Let us give an overview of the proof. The implication  $(1) \Rightarrow (3)$  is just Lemma 3, and its converse is Theorem 6. Convergence of measures in  $\mathcal{P}(\mathbb{M})$  in the weak topology is equivalent to convergence in  $\delta_{\mathbb{P}}$  (or  $\delta_{\mathbb{W}}$ ) by Theorem 2, convergence of all MPNN outputs by Corollary 5, convergence of tree homomorphism densities by Grebik and Rocha [52], cf. Appendix D, and thus, convergence in the tree distance by Böker [18]. Therefore, the remaining equivalences all follow with a compactness argument similar to the one in Theorem 6; this was also used by Böker [18] to prove the equivalence of (2) and (4). Let us formally state the equivalence of all mentioned notions of convergence in the following theorem.
Theorem 27. Let  $(W_{i})_{i}$  be a sequence of graphons  $W_{i}:[0,1]^{2}\to [0,1]$ , and let  $W:[0,1]^2\to [0,1]$  be a graphon. Then, the following are equivalent:
1.  $\delta_{\mathsf{P}}(W_i,W)\to 0$  (or equivalently  $\delta_{\mathsf{W}}(W_i,W)\to 0$
2.  $\delta_{\square}^{\mathcal{T}}(W_i,W)\to 0$
3.  $\mathbf{h}_{W_i}\to \mathbf{h}_W$  for every MPNN model  $\varphi$  and Lipschitz  $\psi \colon \mathbb{R}^{d_L}\to \mathbb{R}^n$ , where  $n > 0$ .
4.  $t(T,W_i)\to t(T,W)$  for every tree  $T$
5.  $\nu_{W_i}\rightarrow \nu_W$
Proof. (1) and (3) are equivalent to (5) by Theorem 2 and Corollary 5. (2) is equivalent to (4) by [18], which in turn is equivalent to (5) by the result of Grebik and Rocha [52], cf. Appendix D.  $\square$
We note that we only state Theorem 27 for graphons and not elements of  $\mathcal{P}(\mathbb{M})$  since the tree distance of [18] is only defined for graphons and not for probability measures. We further note that the following non-approximative variant of Theorem 27 also holds.
Theorem 28. Let  $U$  and  $W$  be graphons. Then, the following are equivalent:
1.  $\delta_{\mathbb{P}}(U,W) = 0$  (or equivalently  $\delta_{\mathbb{W}}(U,W) = 0$ ).
2.  $\delta_{\square}^{\mathcal{T}}(U,W) = 0$
3.  $\mathbf{h}_U = \mathbf{h}_W$  for every MPNN model  $\varphi$  and Lipschitz  $\psi \colon \mathbb{R}^{d_L}\to \mathbb{R}^n$ , where  $n > 0$ .
4.  $t(T,U) = t(T,W)$  for every tree  $T$ .
5.  $\nu_{U} = \nu_{W}$
Proof. (2) is equivalent to (4) by [18]. The other equivalences follow as in Theorem 27 since  $\mathcal{P}(\mathbb{M})$  is Hausdorff.
Let us now demonstrate how compactness can be used to deduce Theorem 1 from Theorem 27. For the direction  $(1) \Rightarrow (3)$ , we actually do not need compactness since Lemma 3 already is a slightly stronger statement. For the sake of completeness, we nevertheless state it as an epsilon-delta statement.
Theorem 29. Let  $n > 0$  be fixed. For every  $L \in \mathbb{N}$ ,  $C > 0$ , and  $\varepsilon > 0$ , there is a  $\delta > 0$  such that, for all graphons  $U$  and  $W$ , if  $\delta_{\mathsf{P}}(U,W) \leq \delta$ , then  $\| \mathbf{h}_U - \mathbf{h}_W \|_2 \leq \epsilon$  for every  $L'$ -layer MPNN model  $\varphi$  and Lipschitz  $\psi: \mathbb{R}^{d_{L'}} \to \mathbb{R}^n$  with  $L' \leq L$  and  $C_{(\varphi, \psi)} \leq C$ .
Proof. Follows immediately from Lemma 3.  $\square$
Let us formally prove the converse direction  $(3) \Rightarrow (1)$ .
Proof. Assume that there is an  $\varepsilon > 0$  such that such  $L \in \mathbb{N}$ ,  $C > 0$ , and  $\delta > 0$  do not exist. Then, for every  $k \geq 0$ , there are graphons  $U_k$  and  $W_k$  with  $\|\mathbf{h}_{U_k} - \mathbf{h}_{W_k}\|_2 \leq 1/k$  for every MPNN model  $\varphi$  and Lipschitz  $\psi$  with output dimension  $n$ , at most  $k$  layers and  $C_{(\varphi, \psi)} \leq k$  but also  $\delta_{\mathsf{P}}(U_k, W_k) > \varepsilon$ . By the compactness of the graphon space, there are subsequences  $(U_{i_k})_k$  and  $(W_{i_k})_k$  of  $(U_k)_k$  and  $(W_k)_k$  converging to graphons  $\widetilde{U}$  and  $\widetilde{W}$ , respectively, in the cut distance and, hence, also in the tree distance. Let  $\varphi$  be an  $L$ -layer MPNN model and  $\psi: \mathbb{R}^{d_L} \to \mathbb{R}^n$  be Lipschitz. Then, by Theorem 27, also  $(\mathbf{h}_{U_{i_k}})_k$  and  $(\mathbf{h}_{W_{i_k}})_k$  converge to  $\mathbf{h}_{\widetilde{U}}$  and  $\mathbf{h}_{\widetilde{W}}$ , respectively. Hence,
$$
\left\| \mathbf {h} _ {\widetilde {U}} - \mathbf {h} _ {\widetilde {W}} \right\| _ {2} \leq \left\| \mathbf {h} _ {\widetilde {U}} - \mathbf {h} _ {U _ {i _ {k}}} \right\| _ {2} + \left\| \mathbf {h} _ {U _ {i _ {k}}} - \mathbf {h} _ {W _ {i _ {k}}} \right\| _ {2} + \left\| \mathbf {h} _ {W _ {i _ {k}}} - \mathbf {h} _ {\widetilde {W}} \right\| _ {2} \xrightarrow {k \to \infty} 0
$$
by the assumption, i.e.,  $\mathbf{h}_{\widetilde{U}} = \mathbf{h}_{\widetilde{W}}$ . Since this holds for every MPNN model and Lipschitz  $\psi$ , we have  $\delta_{\mathbb{P}}(\widetilde{U},\widetilde{W}) = 0$  by Theorem 28. Then, however
$$
\delta_ {\mathsf {P}} \left(U _ {i _ {k}}, W _ {i _ {k}}\right) \leq \delta_ {\mathsf {P}} \left(U _ {i _ {k}}, \widetilde {U}\right) + \delta_ {\mathsf {P}} \left(\widetilde {U}, \widetilde {W}\right) + \delta_ {\mathsf {P}} \left(\widetilde {W}, W _ {i _ {k}}\right) \xrightarrow {k \rightarrow \infty} 0
$$
since  $(U_{i_k})_k$  and  $(W_{i_k})_k$  converge to  $\widetilde{U}$  and  $\widetilde{W}$ , respectively, also in  $\delta_{\mathsf{P}}$  by Theorem 27. This contradicts the assumption that  $\delta_{\mathsf{P}}(U_k,W_k) > \varepsilon$  for every  $k\geq 0$ .
In the above theorem, we could simply replace  $\delta_{\mathsf{P}}$  by  $\delta_{\mathsf{W}}$  or  $\delta_{\square}^{\mathcal{T}}$ ; the proof is analogous. We can also state the equivalence of these metrics explicitly, e.g., graphons that are close in  $\delta_{\mathsf{P}}$  are close in  $\delta_{\square}^{\mathcal{T}}$  and vice versa.
Theorem 30. For every  $\varepsilon > 0$ , there is a  $\delta > 0$  such that, for all graphons  $U$  and  $W$ , if  $\delta_{\mathsf{P}}(U,W) \leq \delta$ , then  $\delta_{\square}^{\mathcal{T}}(U,W) \leq \varepsilon$ .
Proof. Assume that there is a  $\varepsilon > 0$  such that such a  $\delta > 0$  does not exist. Then, for every  $k \geq 0$ , there are graphons  $U_k$  and  $W_k$  with  $\delta_{\mathsf{P}}(U_k, W_k) \leq 1 / k$  but  $\delta_{\square}^{\mathcal{T}}(U_k, W_k) > \varepsilon$ . By the compactness of the graphon space, there are subsequences  $(U_{i_k})_k$  and  $(W_{i_k})_k$  of  $(U_k)_k$  and  $(W_k)_k$  weakly converging to graphons  $\widetilde{U}$  and  $\widetilde{W}$ , respectively, in the cut distance and, hence, also in the tree distance, which in turn by Theorem 27 means that they also converge in  $\delta_{\mathsf{P}}$ . Combining this with the assumption yields that  $\delta_{\mathsf{P}}(\widetilde{U}, \widetilde{W}) = 0$ , which implies  $\delta_{\square}^{\mathcal{T}}(\widetilde{U}, \widetilde{W}) = 0$  by Theorem 28. This, however, is a contradiction to  $\delta_{\square}^{\mathcal{T}}(U_k, W_k) > \varepsilon$  since  $(U_{i_k})_k$  and  $(W_{i_k})_k$  converge to  $\widetilde{U}$  and  $\widetilde{W}$ , respectively, in  $\delta_{\square}^{\mathcal{T}}$ .
Theorem 31. For every  $\varepsilon >0$ , there is an  $\delta >0$  such that, for all graphons  $U$  and  $W$ , if  $\delta_{\square}^{\mathcal{T}}(U,W)\leq \delta$ , then  $\delta_{\mathbb{P}}(U,W)\leq \varepsilon$ .
Proof. Analogous to Theorem 30.
The equivalence between the tree distance and tree homomorphism densities was proven in [18]. Let us state them here for the sake of completeness. Note that the statements are essentially the same as Theorem 29 and Theorem 6, where the constant  $C$  and number of layers  $L$  is replaced by the order  $k$  of the trees.
Theorem 32 ([18, Corollary 20]). For every tree  $T$  and every  $\varepsilon > 0$ , there is a  $\delta > 0$  such that, for all graphons  $U$  and  $W$ , if  $\delta_{\square}^{T}(U, W) \leq \delta$ , then  $|t(T, U) - t(T, W)| \leq \varepsilon$ .
Theorem 33 ([18, Corollary 21]). For every  $\varepsilon > 0$ , there are  $k > 0$  and  $\delta > 0$  such that, for all graphons  $U$  and  $W$ , if  $|t(T,U) - t(T,W)|$  for every tree  $T$  on  $k$  vertices, then  $\delta_{\square}^{T}(U,W) \leq \varepsilon$ .
D Tree homomorphism densities
We briefly repeat the proof of Grebik and Rocha [52] for a universal approximation theorem for (linear combinations) of tree homomorphism density functions. The purpose of this is twofold. First, it serves as a reference since we use this result for a part of the proof of Theorem 1. Secondly, however, it illustrates the striking similarities between tree homomorphism densities and MPNNs. Not only do both tree homomorphism densities and MPNNs satisfy a universal approximation theorem, but the theorem statements and proofs in this section for tree homomorphism densities and in Appendix C.3 for MPNNs are nearly identical. Hence, tree homomorphism densities and MPNNs are two different ways of arriving at the same end goal, a set of functions that has certain properties and, in particular, describes the similarity of graphons modulo the 1-WL test.
The following definition is implicit in Grebik and Rocha [52]. For a rooted tree  $T$  of height at most  $h \in \mathbb{N}$ , we inductively define the homomorphism density function  $t_h(T, \cdot) \colon \mathbb{M}_h \to [0,1]$  by letting  $t_h(T, \cdot) \coloneqq \mathbf{1}_{\mathbb{M}_h}$  if the height of  $T$  is zero and
$$
t _ {h} (T, \alpha) := \left(\int_ {\mathbb {M} _ {h - 1}} t _ {h - 1} \left(T _ {1}, \cdot\right) d \alpha\right) \cdot \dots \cdot \left(\int_ {\mathbb {M} _ {h - 1}} t _ {h - 1} \left(T _ {k}, \cdot\right) d \alpha\right)
$$
for every  $\alpha \in \mathbb{M}_h$  if  $h > 0$ , where  $T_1, \ldots, T_k$  are the trees rooted in the children of the root of  $T$ . Intuitively,  $t_h(T, \alpha)$  is the homomorphism density of  $T$  when the root of  $T$  is mapped to a point of color  $\alpha$ . Additionally, we define  $t_{\infty}(T, \cdot) \colon \mathbb{M} \to [0, 1]$  by  $t_{\infty}(T, \cdot) \coloneqq t_h(T, \cdot) \circ p_{\infty, h}$ . One can show that this is well defined, i.e., that the definition is independent of  $h$ , by a simple induction.
Lemma 34. Let  $T$  be a rooted tree of height at most  $h$ . Then,  $t_{h + 1}(T,\cdot) = t_h(T,\cdot)\circ p_{h + 1,h}$ .