SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Recently, connections between GNNs and Weisfeiler-Leman type algorithms have been shown [12, 49, 89, 115]. Specifically, Morris et al. [89] and Xu et al. [115] showed that the 1-WL limits the expressive power of any possible GNN architecture in distinguishing non-isomorphic graphs. In turn, these results have been generalized to the  $k$ -WL, see, e.g., Azizian and Lelarge [7], Geerts [47], Maron et al. [81], Morris et al. [89, 91, 93], and connected to permutation-equivariant functions approximation over graphs, see, e.g., Azizian and Lelarge [7], Chen et al. [28], Geerts and Reutter [48], Maehara and NT [79]. Further, Aamand et al. [1] devised an improved analysis using randomization. Recent works have extended the expressive power of GNNs, e.g., by using random features [2, 32, 104], equivariant graph polynomials [100], or homomorphism and subgraph counts [13, 22, 30, 95], see Morris et al. [92] for a thorough survey. Recently, Grohe [56] showed tight connections between GNNs' expressivity and circuit complexity. Moreover, Rosenbluth et al. [102] investigated the expressive power of different aggregation functions beyond sum aggregation.
B.2 Graph metrics beyond edit distance
As discussed before, the edit distance does not capture the structural similarity of graphs well. A metric that arguably captures the similarity of graphs well is the cut distance, which was introduced by Frieze and Kannan [41] and provides the central notion of convergence in the theory of dense graph limits. Graph similarity can also be defined via graph homomorphism densities, and as part of the aforementioned theory, Borgs et al. [20], Lovász and Szegedy [77] proved that this is equivalent to the cut distance. Böker [18] introduced a variant of the cut distance based on the well-known system of linear equations defining fractional isomorphism, the tree distance, which is equivalent to similarity defined via tree homomorphism densities. To study MPNNs' stability properties, Chuang and Jegelka [31] introduced Tree Mover's Distance to compare two graphs via hierarchical optimal transport between their computation trees. Similarly, Chen et al. [26] introduced the Weisfeiler-Leman distance, a polynomial-time computable pseudometric on graphs by combining techniques from optimal transport with the 1-WL, and gave an interpretation in terms of stochastic processes [27].
Here, we collect proofs that we omitted from the main body of the paper.
C.1 Equivalence of message-passing graph neural networks
Here, we show that, first, for a graph  $G$ , the output of an MPNN on  $G$  equals the output of the MPNN on the corresponding graphon  $W_{G}$  and, second, the output of an MPNN on a graphon  $W$  equals the output of the corresponding MPNN on the corresponding DIDM  $\nu_{W}$  of  $W$ . Hence, it suffices to consider MPNNs on DIDMs. We note that, while  $\varphi$  is an MPNN model and  $\psi$  is Lipschitz in the following theorems, we actually do not need Lipschitz continuity and continuous functions would suffice.
Theorem 8. Let  $G$  be a graph and  $\varphi$  be an  $L$ -layer MPNN model. Let  $(I_v)_{v \in V(G)}$  be the partition of [0, 1] used in the construction of  $W_G$  from  $G$ . Then, for all  $t \in [L]$ ,  $v \in V(G)$ , and  $x \in I_v$ ,
$$
\mathbf {h} _ {v} ^ {(t)} = \mathbf {h} _ {x} ^ {(t)}.
$$
Proof. We prove the claim by induction on  $t$ .
Base of the induction. The first space of colors is  $\mathbb{M}_0 = \{1\}$ . Then, for all  $v \in V(G)$  and  $x \in I_v$
$$
\mathbf {h} _ {v} ^ {(0)} = \varphi_ {0} = \mathbf {h} _ {x} ^ {(0)}.
$$
Induction step. The induction assumption is that  $\mathbf{h}_v^{(t - 1)} = \mathbf{h}_x^{(t - 1)}$  for all  $v\in V(G)$  and  $x\in I_v$ . Then, for all  $v\in V(G)$  and  $x\in I_v$ , we have
$$
\begin{array}{l} \mathbf {h} _ {x} ^ {(t)} = \varphi_ {t} \left(\int_ {[ 0, 1 ]} W _ {G} (x, y) \mathbf {h} _ {y} ^ {(t - 1)} d \lambda (y)\right) = \varphi_ {t} \left(\sum_ {u \in V (G)} \int_ {I _ {u}} W _ {G} (x, y) \mathbf {h} _ {y} ^ {(t - 1)} d \lambda (y)\right) \\ = \varphi_ {t} \left(\sum_ {u \in V (G)} \int_ {I _ {u}} W _ {G} (x, y) \mathbf {h} _ {u} ^ {(t - 1)} d \lambda (y)\right) \\ = \varphi_ {t} \left(\frac {1}{| V (G) |} \sum_ {u \in N (v)} \mathbf {h} _ {u} ^ {(t - 1)}\right) = \mathbf {h} _ {v} ^ {(t)}. \\ \end{array}
$$
Theorem 9. Let  $G$  be a graph, let  $\varphi$  be an  $L$ -layer MPNN model, and let  $\psi$  be Lipschitz. Let  $(I_v)_{v \in V(G)}$  be the partition of  $[0,1]$  used in the construction  $W_G$  from  $G$ . Then,
$$
\mathbf {h} _ {G} = \mathbf {h} _ {W _ {G}}.
$$
Proof. With Theorem 8, we get
$$
\begin{array}{l} \mathbf {h} _ {W _ {G}} = \psi \left(\int_ {[ 0, 1 ]} \mathbf {h} _ {x} ^ {(L)} d \lambda (x)\right) = \psi \left(\sum_ {v \in V (G)} \int_ {I _ {v}} \mathbf {h} _ {x} ^ {(L)} d \lambda (x)\right) \\ = \psi \left(\sum_ {v \in V (G)} \int_ {I _ {v}} \mathbf {h} _ {v} ^ {(L)} d \lambda (x)\right) \\ = \psi \bigg (\frac {1}{| V (G) |} \sum_ {v \in V (G)} \mathbf {h} _ {v} ^ {(L)} \bigg) = \mathbf {h} _ {G}. \\ \end{array}
$$
To prove the equivalence of an MPNN on a graphon  $W$  and the corresponding DIDM  $\nu_{W}$ , we need the following definition. Let  $\delta \colon [0,1] \to \mathbb{R}$  be a nonnegative measurable function. Then, define the measure  $\nu_{\delta}$  by
$$
\nu_ {\delta} (A) := \int_ {A} \delta d \lambda
$$
for measurable  $A \subseteq [0,1]$ . The following lemma, which can be interpreted as the "well-definedness of the change of variable  $z = \delta(y)$ ", is taken from [16, Theorem 16.11].
Lemma 10. Let  $\delta :[0,1]\to \mathbb{R}$  be a nonnegative measurable function. Then, a measurable function  $f\colon [0,1]\to \mathbb{R}$  is integrable with respect to  $\nu_{\delta}$  if and only if  $f\delta$  is integrable with respect to  $\lambda$ , in which case
$$
\int_ {A} f d \nu_ {\delta} = \int_ {A} f \delta d \lambda
$$
holds for every measurable  $A$ .
Theorem 11. Let  $W \in \mathcal{W}$  and  $\varphi$  be an  $L$ -layer MPNN model. Then, for every  $t \in [L]$  and almost every  $x \in [0,1]$ ,
$$
\mathbf {h} _ {x} ^ {(t)} = \mathbf {h} _ {\mathrm {i} _ {W}, t} ^ {(t)} (x) = \mathbf {h} _ {\mathrm {i} _ {W}} ^ {(t)} (x).
$$
Proof. We only have to show the first equality. Then, the second follows directly from the definitions of  $\mathbf{h}_{-}^{(t)}: \mathbb{M} \to \mathbb{R}^{d_t}$  and  $\mathrm{i}_W$ .
Base of the induction. The first space of colors is  $\mathbb{M}_0 = \{1\}$ . Then,
$$
\mathbf {h} _ {x} ^ {(0)} = \varphi_ {0} = \mathbf {h} _ {\mathrm {i} _ {W, t} (x)} ^ {(0)}.
$$
Induction step. The induction assumption is that  $\mathbf{h}_x^{(t - 1)} = \mathbf{h}_{i_{W,t - 1}(x)}^{(t - 1)}$  for almost every  $x\in [0,1]$ . Let us prove the induction step for  $t$ . We have
$$
\mathbf {h} _ {x} ^ {(t)} = \varphi_ {t} \left(\int_ {[ 0, 1 ]} W (x, y) \mathbf {h} _ {y} ^ {(t - 1)} d \lambda (y)\right).
$$
Recall that for any  $A\in \mathcal{B}(\mathbb{M}_{h - 1})$
$$
\bigl(\mathrm{i}_{W,t}(x)\bigr)(A) = \int_{\mathrm{i}_{W,t - 1}^{-1}(A)}W(x,y)d\lambda (y),
$$
so, by the notation  $\nu_{W(x, - )}$  introduced before Lemma 10,
$$
\mathrm {i} _ {W, t} (x) = \left(\mathrm {i} _ {W, t - 1}\right) _ {*} \nu_ {W (x, -)}.
$$
Hence, by Lemma 10 with  $\delta = W(x, -)$  and  $f = \mathbf{h}_{-}^{(t - 1)}$ , we have
$$
\begin{array}{l} \mathbf {h} _ {\mathrm {i} _ {W, t} (x)} ^ {(t)} = \varphi_ {t} \left(\int_ {\mathbb {M} _ {t - 1}} \mathbf {h} _ {-} ^ {(t - 1)} d \mathrm {i} _ {W, t} (x)\right) = \varphi_ {t} \left(\int_ {\mathbb {M} _ {t - 1}} \mathbf {h} _ {-} ^ {(t - 1)} d (\mathrm {i} _ {W, t - 1}) * \nu_ {W (x, -)}\right) \\ = \varphi_ {t} \left(\int_ {[ 0, 1 ]} \mathbf {h} _ {-} ^ {(t - 1)} \circ \mathrm {i} _ {W, t - 1} d \nu_ {W (x, -)}\right) \\ = \varphi_ {t} \left(\int_ {[ 0, 1 ]} W (x, y) \mathbf {h} _ {\mathrm {i} _ {W, t - 1} (y)} ^ {(t - 1)} d \lambda (y)\right). \\ \end{array}
$$
Hence, by the induction assumption,  $\mathbf{h}_x^{(t)} = \mathbf{h}_{\mathrm{i}_{W,t}(x)}^{(t)}$
Theorem 12. Let  $W \in \mathcal{W}$ , let  $\varphi$  be an  $L$ -layer MPNN model, and let  $\psi$  be Lipschitz. Then,
$$
\mathbf {h} _ {W} = \mathbf {h} _ {\nu_ {W, L}} = \mathbf {h} _ {\nu_ {W}}.
$$
Proof. We only have to prove the first inequality; the second follows from the first and the definition of  $\nu_{W}$ . The first inequality follows from Theorem 11:
$$
\begin{array}{l} \mathbf {h} _ {\nu_ {W, L}} = \psi \left(\int_ {\mathbb {M} _ {L}} \mathbf {h} _ {-} ^ {(L)} d \nu_ {W, L}\right) = \psi \left(\int_ {\mathbb {M} _ {L}} \mathbf {h} _ {-} ^ {(L)} d (\mathrm {i} _ {W, L}) _ {*} \lambda\right) \\ = \psi \Big (\int_ {[ 0, 1 ]} \mathbf {h} _ {i _ {W, L} (x)} ^ {(L)} d \lambda (x) \Big) \\ = \psi \left(\int_ {[ 0, 1 ]} \mathbf {h} _ {x} ^ {(L)} d \lambda (x)\right) \\ = \mathbf {h} _ {W}. \\ \end{array}
$$
C.2 Metrics on iterated degree measures
We first prove Theorem 2 for the metrics based on the Prokhorov metric. Then, we prove the inequalities between the Prokhorov metric  $\mathsf{P}$  and the unbalanced Wasserstein metric  $\mathsf{W}$  from Section 3 and deduce Theorem 2 for the unbalanced Wasserstein metric from it. After that, we prove Lemma 3.
C.2.1 The Prokhorov metric
To begin, let us restate Theorem 2 for better readability.
Let  $(S,d)$  be a complete separable metric space with Borel  $\sigma$ -algebra  $\mathcal{B}$ . Recall that, for a subset  $A \subseteq S$  and  $\varepsilon \geq 0$ , one defines  $A^{\varepsilon} \coloneqq \{y \in S \mid d(x,y) < \varepsilon \text{ for some } x \in A\}$ . Then, the Prokhorov metric  $\mathsf{P}$  on  $\mathcal{M}_{\leq 1}(S,\mathcal{B})$  is given by
It is not hard to see that we can replace  $A^{\varepsilon}$  by  $A^{\varepsilon]} \coloneqq \{y \in S \mid d(x, y) \leq \varepsilon$  for some  $x \in A\}$  in the above definition without changing the value  $\mathsf{P}(\mu, \nu)$ . Moreover, if  $\| \mu \| \geq \| \nu \|$ , then the definition simplifies to
$$
\mathrm {P} (\mu , \nu) = \inf  \left\{\varepsilon > 0 \mid \mu (A) \leq \nu \left(A ^ {\varepsilon}\right) + \varepsilon \text {f o r e v e r y} A \in \mathcal {B} \right\}
$$
since, if  $\mu(A) \leq \nu(A^{\varepsilon}) + \varepsilon$  for every  $A \in \mathcal{B}$ , then  $\nu(B) \leq \mu(B^{\varepsilon}) + \varepsilon + \| \nu \| - \| \mu \| \leq \mu(B^{\varepsilon}) + \varepsilon$  for every  $B \in \mathcal{B}$  [40, Lemma 4.30].
As the name suggests,  $\mathsf{P}$  is a metric on  $\mathcal{M}(S)$  [99, Section 1.4], and Prokhorov also proved that convergence in  $\mathsf{P}$  is equivalent to convergence in the weak topology.
Theorem 13 ([99, Theorem 1.11]). Let  $(S,d)$  be a complete separable metric space. Then,  $(\mathcal{M}(S),\mathsf{P})$  is a complete separable metric space, and convergence in  $\mathsf{P}$  is equivalent to weak convergence of measures.
The well-definedness of  $d_{\mathbb{P},h}$  and  $\delta_{\mathbb{P},h}$  follows from an inductive application of this theorem.
Lemma 14. Let  $0 \leq h \leq \infty$ . The metric  $d_{\mathbb{P},h}$  is well-defined and metrizes the topology of  $\mathbb{M}_h$ . The metric  $\delta_{\mathbb{P},h}$  is well-defined and metrizes the topology of  $\mathcal{P}(\mathbb{M}_h)$ .
Proof. First, we consider  $d_{\mathbb{P},h}$ . Let  $h \in \mathbb{N}$ . To see why we even have to prove that  $d_{\mathbb{P},h}$  is well-defined, note that the measures in  $\mathbb{M}_{h+1} = \mathcal{M}_{\leq 1}(\mathbb{M}_h) = \mathcal{M}_{\leq 1}(\mathbb{M}_h, \mathcal{B}(\mathbb{M}_h))$  by definition are functions  $\mu \colon \mathcal{B}(\mathbb{M}_h) \to \mathbb{R}$ , where  $\mathcal{B}(\mathbb{M}_h)$  depends on the topology of  $\mathbb{M}_h$ , which is the weak topology in our case. Hence, it is crucial that  $d_{\mathbb{P},h}$  metrizes the weak topology on  $\mathbb{M}_h$ ; otherwise the sets  $\mathcal{M}_{\leq 1}(\mathbb{M}_h, \mathcal{B}(\mathbb{M}_h))$  and  $\mathcal{M}_{\leq 1}(\mathbb{M}_h, d_{\mathbb{P},h})$  might not even be the same.
For the induction basis  $h = 0$ , this claim trivially holds. Then, the induction hypothesis yields that  $d_{\mathbb{P},h}$  is well-defined and metrizes the weak topology on  $\mathbb{M}_h$ . Then,  $d_{\mathbb{P},h + 1}$  is well-defined as the Borel  $\sigma$ -algebra of  $(\mathbb{M}_h,d_{\mathbb{P},h})$  equals  $\mathcal{B}(\mathbb{M}_h)$  and, by Theorem 13, convergence in  $d_{\mathbb{P},h + 1}$  is equivalent to weak convergence on  $\mathcal{M}_{\leq 1}(\mathbb{M}_h,d_{\mathbb{P},h})$ , which is just weak convergence on  $\mathcal{M}_{\leq 1}(\mathbb{M}_h,\mathcal{B}(\mathbb{M}_h)) = \mathbb{M}_{h + 1}$ . Hence, the topology induced by  $d_{\mathbb{P},h + 1}$  is equal to the weak topology on  $\mathbb{M}_{h + 1}$  as both spaces are metrizable. For  $d_{\mathbb{P},\infty}$ , the claim directly follows since  $d_{\mathbb{P},\infty}$  is just a variant of the product metric, which metrizes the product topology.
The claim for  $\delta_{\mathbb{P},h}$  then follows from the just proven claim for  $d_{\mathbb{P},h}$  by the same reasoning as in the inductive step.
The following lemma states that the distances  $d_{\mathsf{P},h}$  behave as expected, i.e., the distance of IDMs can only increase when  $h$  increases.
Lemma 15. Let  $0 \leq h \leq h' < \infty$ . Then,  $d_{\mathsf{P},h}(p_{h',h}(\alpha), p_{h',h}(\beta)) \leq d_{\mathsf{P},h'}(\alpha, \beta)$  for all  $\alpha, \beta \in \mathbb{M}_h$ .
$$
d _ {\mathsf {P}, h} \left(p _ {h + 1, h} (\alpha), p _ {h + 1, h} (\beta)\right) \leq d _ {\mathsf {P}, h + 1} (\alpha , \beta),
$$
for all  $\alpha, \beta \in \mathbb{M}_{h+1}$ , where  $h \geq 0$ . We prove this by induction on  $h$ , where for  $h = 0$ , the statement trivially holds. For the inductive step, we assume without loss of generality that  $\| \alpha \| \geq \| \beta \|$ , which directly implies  $\| p_{h+1,h}(\alpha) \| \geq \| p_{h+1,h}(\beta) \|$ . We show that if  $\varepsilon > 0$  such that  $\alpha(A) \leq \beta(A^{\varepsilon}) + \varepsilon$  for every  $A \in \mathcal{B}(\mathbb{M}_{h+1})$ , then also  $p_{h+1,h}(\alpha)(A) \leq p_{h+1,h}(\beta)(A^{\varepsilon}) + \varepsilon$ . For such an  $\varepsilon$ , we have
$$
\begin{array}{l} p _ {h + 1, h} (\alpha) (A) = \left(p _ {h, h - 1}\right) _ {*} \alpha (A) = \alpha \left(p _ {h, h - 1} ^ {- 1} (A)\right) \\ \leq \beta ((p _ {h, h - 1} ^ {- 1} (A)) ^ {\varepsilon}) + \varepsilon \\ \leq \beta \left(p _ {h, h - 1} ^ {- 1} \left(A ^ {\varepsilon}\right)\right) + \varepsilon = p _ {h + 1, h} (\beta) \left(A ^ {\varepsilon}\right) + \varepsilon , \\ \end{array}
$$
where  $\beta((p_{h,h-1}^{-1}(A))^{\varepsilon}) \leq \beta(p_{h,h-1}^{-1}(A^{\varepsilon}))$  holds by the induction hypothesis: We show that  $(p_{h,h-1}^{-1}(A))^{\varepsilon} \subseteq p_{h,h-1}^{-1}(A^{\varepsilon})$ . Let  $x \in (p_{h,h-1}^{-1}(A))^{\varepsilon}$ . Then, there is a  $y \in (p_{h,h-1}^{-1}(A))$  such that  $d_{\mathbb{P},h}(x,y) < \varepsilon$ . By the inductive hypothesis, we have
$$
d _ {\mathsf {P}, h - 1} \left(p _ {h, h - 1} (x), p _ {h, h - 1} (y)\right) \leq d _ {\mathsf {P}, h} (x, y) <   \varepsilon .
$$
Since  $p_{h,h-1}(y) \in A$ , we have  $p_{h,h-1}(x) \in A^{\varepsilon}$ , and thus,  $x \in p_{h,h-1}^{-1}(A)$ .
C.2.2 Computability of the Prokhorov metric
It remains to prove that  $d_{\mathbb{P},h}$  and  $\delta_{\mathbb{P},h}$  are polynomial-time computable. A paper of Garel and Massé [46] is concerned with computing the Prokhorov metric of (possibly non-discrete) probability distributions. To this end, they first quantize the distributions and then show that the Prokhorov metric of finitely-supported probability distributions can be computed exactly by solving a series of maximum-flow problems; this latter part is based on results by Schay [106] and García-Palomares and Evarist Giné [45], which essentially shows how the Prokhorov metric can be formulated as a linear-optimization problem. By generalizing this linear program to finite measures, we can adapt the algorithm of Garel and Massé [46] to obtain an algorithm for computing the Prokhorov distance of finite measures.
Theorem 16. Let  $\mu, \nu \in \mathcal{M}(S)$ , where  $(S,d)$  is a finite metric space with  $S = \{x_{1},\ldots,x_{n}\}$ . Then, the Prokhorov metric  $\mathsf{P}(\mu,\nu)$  can be computed in time polynomial in  $n$  and the number of bits needed to encode  $d$ ,  $\mu$ , and  $\nu$ .
Proof. Without loss of generality, we assume that  $\| \mu \| \geq \| \nu \|$ . Then,
$$
\begin{array}{l} \mathrm {P} (\mu , \nu) = \inf  \left\{\varepsilon > 0 \mid \mu (A) \leq \nu \left(A ^ {\varepsilon}\right) + \varepsilon \text {f o r e v e r y} A \subseteq S \right\} \\ = \inf  \left\{\varepsilon > 0 \mid \mu (A) \leq \nu \left(A ^ {\varepsilon ]}\right) + \varepsilon \text {f o r e v e r y} A \subseteq S \right\} \\ = \inf  \left\{\varepsilon \geq 0 \mid \varepsilon \geq \rho (\varepsilon) \right\}, \\ \end{array}
$$
$$
\rho (\varepsilon) := \inf  \left\{\eta > 0 \mid \mu (A) \leq \nu \left(A ^ {\varepsilon ]}\right) + \eta \text {f o r e v e r y} A \subseteq S \right\}
$$
for  $\varepsilon > 0$ . The following claim generalizes an observation of Schay [106, Theorem 1] and García-Palomares and Evarist Giné [45, Lemma] to finite measures and shows that the value of  $\rho(\varepsilon)$  can be expressed as a linear program.
Claim 17. Let  $\varepsilon >0$ . Then,
$$
\rho (\varepsilon) = \| \mu \| - \max  \sum_ {i, j = 1} ^ {n} \mathbb {1} [ d (x _ {i}, x _ {j}) \leq \varepsilon ] \cdot x _ {i j},
$$
where the maximum is taken over variables  $x_{ij}$  such that  $\sum_{i=1}^{n} x_{ij} = \nu(\{x_j\})$  for every  $j \in [n]$ ,  $\sum_{j=1}^{n} x_{ij} \leq \mu(\{x_i\})$  for every  $i \in [n]$ , and  $x_{ij} \geq 0$  for all  $i, j \in [n]$ .
Proof. For the sake of brevity, define  $p_i \coloneqq \mu(\{x_i\})$  and  $q_i \coloneqq \nu(\{x_i\})$  for  $i \in [n]$  and  $d_{ij}^{\varepsilon} \coloneqq \mathbb{1}[d(x_i, x_j) \leq \varepsilon]$  for all  $i, j \in [n]$ . We have
$$
\begin{array}{l} \rho (\varepsilon) = \max_{A\subseteq S}\bigl(\mu (A) - \nu (A^{\varepsilon ]})\bigr) = \max_{\substack{v_{j}\in \{0,1\} ,\\ w_{i}\in \{0,1\} ,\\ v_{j}\geq d_{ij}^{\varepsilon} - 1 + w_{i}}}\sum_{i\in [n]}(p_{i}w_{i} - q_{i}v_{i}) \\ = \max_{\substack{0\leq v_{j}\leq 1,\\ 0\leq w_{i}\leq 1,\\ v_{j}\geq d_{ij}^{\varepsilon} - 1 + w_{i}}}\sum_{i\in [n]}(p_{i}w_{i} - q_{i}v_{i}) \\ \end{array}
$$
where the first equality holds by definition of  $\rho(\varepsilon)$ , the second since one can view the sets  $A$  and  $A^{\varepsilon}$  as vectors  $\bar{v}$  and  $\bar{w}$ , respectively, and the third, since the optimum value of a linear program is attained on an extreme point of the convex polytope specifying the feasible solutions, which are 0-1-vectors for this specific linear program. Moreover, since values  $v_{j} > 1$  and  $w_{i} < 0$  can always be set to 1 and 0, respectively, without decreasing the value of the target function, this further equals to
$$
\max_{\substack{v_{j}\geq 0,\\ w_{i}\leq 1,\\ v_{j}\geq d_{ij}^{\varepsilon} - 1 + w_{i}}}\sum_{i\in [n]}(p_{i}w_{i} - q_{i}v_{i}) = \| \mu \| -\min_{\substack{u_{i}\geq 0,\\ v_{j}\geq 0,\\ u_{i} + v_{j}\geq d_{ij}^{\varepsilon}}} \sum_{i\in [n]}(p_{i}u_{i} + q_{i}v_{i}),
$$
where the second equality results from the substitution  $u_{i} \coloneqq 1 - w_{i}$ . Then, considering the dual to the linear minimization problem yields that this further equals to
$$
\rho (\varepsilon) = \| \mu \| - \max  \sum_ {i, j = 1} ^ {n} d _ {i j} ^ {\varepsilon} \cdot x _ {i j},
$$
where the maximum is taken over variables  $x_{ij}$  such that  $\sum_{i=1}^{n} x_{ij} \leq q_j$  for every  $j \in [n]$ ,  $\sum_{j=1}^{n} x_{ij} \leq p_i$  for every  $i \in [n]$ , and  $x_{ij} \geq 0$  for all  $i, j \in [n]$ . Since we can always increase the variables  $x_{ij}$  such that  $\sum_{i=1}^{n} x_{ij} = q_j$  for every  $j \in [n]$  holds without decreasing the target function, we obtain the claim.
With this claim, we can use a polynomial-time method for solving linear programs to compute  $\rho(\varepsilon)$  in time polynomial in  $n$  and the number of bits needed to encode  $d$ ,  $\mu$ , and  $\nu$ . Alternatively, it is straightforward to express this linear program as a maximum-flow problem, which can also be solved in polynomial time. The mapping  $\varepsilon \mapsto \rho(\varepsilon)$  is a non-increasing step function whose jumps can only occur at points in  $D := \{d(x_i, x_j) \mid i, j \in [n]\}$ . Let  $D' := D \cup \{0\}$ . Then, we have
$$
\begin{array}{l} \mathrm {P} (\mu , \nu) = \inf  \left\{\varepsilon \geq 0 \mid \rho (\varepsilon) \leq \varepsilon \right\} \\ = \min  \left(\left\{\varepsilon \in D ^ {\prime} \mid \rho (\varepsilon) \leq \varepsilon \right\} \cup \left\{\rho (\varepsilon) \mid \varepsilon \in D ^ {\prime} \text {w i t h} \rho (\varepsilon) > \varepsilon \right\}\right), \\ \end{array}
$$
which can be computed in time polynomial in  $n$  and the number of bits needed to encode  $d, \mu$ , and  $\nu$  since  $D$  contains at most  $n^2$  values.
Theorem 16 can then be used to compute the distance  $\delta_{\mathsf{P},h}$  of two given graphs. Intuitively, we run the 1-WL on these graphs in parallel and additionally, after the  $i$ th refinement round, compute a matrix  $D_{i}$  containing the distances  $d_{\mathsf{P},i}$  between any pair of colors. This matrix can be computed from the new colors and  $D_{i-1}$ . In the end, we use  $D_{h}$  to compute  $\delta_{\mathsf{P},h}$ . This is essentially the same approach that Chen et al. [26] use for showing that their Weisfeiler-Leman distance is polynomial-time computable.
Theorem 18. Let  $0 \leq h < \infty$ , and let  $G$  and  $H$  be graphs. Then,  $\delta_{\mathbb{P},h}(G,H)$  can be computed in time polynomial in  $h$  and the size of  $G$  and  $H$ .
Proof. We inductively compute  $D_h \in \mathbb{R}^{V(G) \times V(H)}$  with  $D_{h,uv} = d_{\mathsf{P},h}(\mathrm{i}_{G,h}(u),\mathrm{i}_{H,h}(v))$ . In a practical setting, it makes sense to index  $D_h$  by the colors/IDMs of the vertices instead in a practical setting to reduce the size of the matrix and avoid unnecessary computations of the same value. However, for the sake of simplicity, we stick to this definition of  $D_h$  here. For  $h = 0$ , this is just the all-zero matrix. After having computed  $D_n$ , for all pairs of vertices  $u \in V(G), v \in V(H)$ , we use the matrix  $D_h$  to compute the measures  $\mathrm{i}_{G,h+1}(u)$  and  $\mathrm{i}_{H,h+1}(v)$  and then use Theorem 16 to compute their distance w.r.t.  $d_{\mathsf{P},h+1}$ . We note that we do not need the distances between the IDMs of two vertices  $u, v \in V(G)$  or  $u, v \in V(H)$  for this. Hence, the matrix  $D_h \in \mathbb{R}^{V(G) \times V(H)}$  suffices to compute  $D_{h+1} \in \mathbb{R}^{V(G) \times V(H)}$ . Furthermore, we note that we actually do not need to know the precise element of  $\mathbb{M}_h$  assigned to a vertex to do so: we basically run color refinement in parallel and use its renamed colors after  $h$  rounds as our space  $S$ . From the coloring after  $h$  refinement rounds and the matrix  $D_h$ , another application of Theorem 16 yields the distance  $\delta_{\mathsf{P},h}(G,H) = \delta_{\mathsf{P},h}(\nu_{G,h},\nu_{H,h})$ .
Let us give a rough analysis of the running time of the algorithm from Theorem 18. Let  $n$  be the maximum number of vertices in the two graphs  $G$  and  $H$ . Then, we have to compute the Prokhorov metric of two IDMs  $h \cdot n^2$  times and once for the final distance  $\delta_{\mathsf{P},h}(G,H)$ . To compute a single such Prokhorov metric, we have to solve  $n^2$  maximum-flow problems on a graph with  $n$  vertices and  $n^2$
edges, where a single maximum-flow problem takes time  $\mathcal{O}(n^3)$  [96]. Hence, the overall running time is  $\mathcal{O}(h\cdot n^2\cdot n^2\cdot n^3) = \mathcal{O}(h\cdot n^7)$ ; We will see that the distance  $\delta_{\mathrm{W},h}$  can be computed in time  $\mathcal{O}(h\cdot n^5\cdot \log n)$ , which is much better, but nevertheless not ideal for practical purposes. This shows why we need a different way of approximating these distances for practical purposes and leads to our experiments in Section 6.