SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For all other vertices the neighborhood functions return the empty set. We prove that  $h_v^{l+1}$  performs gAMP it inside out, starting with  $m_v^{l+1,k}$ . We represent  $m_v^{l+1,k}$  as AMP  $\phi(v_k, (W, \mathcal{N}_1, \mathcal{N}_2, \mathcal{N}_3, F), h^l)$ . Let  $\rho(v_k, h_{v_k}^l, u, h_u^l) = (h_u^l, e_{uv_k})$  where  $e_{uv_k}$  is an edge constant depending on  $u, v_k$ . By definition  $\rho$  is an atom that is constructed by combining a color atom (A3) with a constant atom (A1) into a tuple atom (A4). Note that  $\mathrm{MES}_k^{l,\mathrm{normal}}\left(\{\{(h_u^l, e_{uv}) \mid u \in Q_{v,G}^{k,t}\}\}\right)$  is AMP that applies the function  $\mathrm{MES}_k^{l,\mathrm{normal}}$  (S4) to an AMP that aggregates (S2) the atom  $\rho(v, h_v^l, u, h_u^l)$  over all  $u \in \mathcal{N}_3(v_k)$ . Next,  $f(G_{v,G}^{k,t})$  can be seen as a constant atom (A1)  $k(v_k, v_k) = f(G_{v,G}^{k,t})$ . Let  $\varphi, \phi$  be two AMPs, then we define the addition AMP as  $f_+((\varphi, \phi)) = \varphi + \phi$ . Hence, we can represent  $m_v^{l+1,k}$  as an AMP
$$
\phi (v _ {k}, (W, \mathcal {N} _ {1}, \mathcal {N} _ {2}, \mathcal {N} _ {3}, F), h ^ {l}) = f _ {+} \left(\mathrm {M E S} _ {k} ^ {l, \mathrm {n o r m a l}} \left(\left\{\left\{(h _ {u} ^ {l}, e _ {u v}) \mid u \in Q _ {v, G} ^ {k, t} \right\} \right\}\right), f (G _ {v, G} ^ {k, t})\right).
$$
Next, we argue that  $h_v^{l+1,k}$  is AMP represented as  $\varphi(v_k, (W, \mathcal{N}_1, \mathcal{N}_2, \mathcal{N}_3, F), h^l)$ . Observe, that we can write  $\varphi$  as a function application of  $\mathrm{UPD}_k^l$  to a tuple
$$
\left(\phi (v _ {k}, (W, \mathcal {N} _ {1}, \mathcal {N} _ {2}, \mathcal {N} _ {3}, F), \left\{h _ {x} ^ {l} \mid x \in \mathcal {N} _ {2} (v _ {k}) \right\}\right)
$$
where  $\{h_x^l\mid x\in \mathcal{N}_2(v_k)\}$  is AMP (S2) that aggregates over the only  $\mathcal{N}_2$  neighbor. As  $\mathcal{N}_2(v_k) = \{v\}$  it holds that  $\varphi (v_{k},(W,\mathcal{N}_{1},\mathcal{N}_{2},\mathcal{N}_{3},F)$  corresponds to  $h_v^{l + 1,k}$ . Finally, we argue that  $h_v^{l + 1}$  is gAMP  $\psi (v,(W,\mathcal{N}_1,\mathcal{N}_2,\mathcal{N}_3,F)$ . We write  $\psi$  as a function application (W5) of COMBINE to the nested aggregation (W3)
$$
\left\{\left. \varphi (v _ {k}, h ^ {l}, W, \mathcal {N} _ {1}, \mathcal {N} _ {2}, \mathcal {N} _ {3}) \mid v _ {k} \in \mathcal {N} _ {1} (v) \right\}. \right.
$$
Thus,  $\psi(v, (W, \mathcal{N}_1, \mathcal{N}_2, \mathcal{N}_3, F)$  corresponds to  $h_v^{l+1}$ . As  $h_v^{l+1}$  is a gAMP it follows by Theorem 5.2 that KP-GNN can be weakly simulated.
We obtain a proof that  $K$ -hop message passing [Feng et al., 2022] can be weakly simulated as special case of the above proof by removing the term  $f(G_{v,G}^{k,t})$  from the definition of  $m_v^{l + 1,k}$ .
Note that atoms, AMP and gAMP are generally defined over all objects. However, by construction this AMP will only be used to aggregate colors of objects  $v_{k}$  with  $v \in V(G), k \in [[K]]$ . Thus, we denote it only for these objects despite it being defined over all objects. We use the similar simplifications throughout this proof.
F Handcrafted Efficient Graph Transformations
F.1 CWN: Efficient Weak Simulation of CW Networks
We define a transformation  $\overline{\mathrm{CWN}}$  that allows for more efficient weak simulation of CW Networks. We prove the following theorem:
Theorem F.1. Message Passing Simplicial Networks [Bodnar et al., 2021b] and CW Networks [Bodnar et al., 2021a] can be weakly simulated by CWN with a simulation factor of 2 such that the number of vertices is equal to the number of cells.
Proof. We prove this theorem for CW Networks as they a generalization of Message Passing Simplicial Networks. We begin with definitions required for the proof. A regular cell complexes consists of different cells each with a fixed dimension. We have a boundary relation  $\prec$  given on the cells. For a cell  $\sigma$  we call  $\mathcal{B}(\sigma) = \{\tau \mid \tau \prec \sigma\}$  its boundary adjacent cells and  $\mathcal{C}(\sigma) = \{\tau \mid \sigma \prec \tau\}$  its co-boundary adjacent cells. We define  $\mathcal{C}(\sigma, \mu) = \mathcal{C}_{\sigma} \cap \mathcal{C}_{\mu}$ . For a cell  $\sigma$  we call  $\mathcal{N}_{\uparrow}(\sigma) = \{\tau \mid \exists \delta$  such that  $\sigma \prec \delta$  and  $\tau \prec \delta\}$  its upper adjacent cells. For a color  $c^t$  at iteration  $t$  we use the notation  $c_{\mathcal{F}}^t(\sigma)$  to denote the color of all objects in  $\mathcal{F}(\sigma)$  at iteration  $t$  meaning  $c_{\mathcal{F}}^t(\sigma) = \{\{c_x^t \mid x \in \mathcal{F}(\sigma)\}\}$ . For a given cell  $\sigma$  in a regular cell complex  $X$  a CW Network computes its color in iteration  $t + 1$  with the color update function
$$
c _ {\sigma} ^ {t + 1} = \operatorname {U P D} _ {1} (\sigma , X, c ^ {t}) = \operatorname {H A S H} \left(c _ {\sigma} ^ {t}, c _ {\mathcal {B}} ^ {t} (\sigma), c _ {\uparrow} ^ {t} (\sigma)\right).
$$
where  $c_{\uparrow}^{t}(\sigma)$  performs the non pairwise message passing
$$
c _ {\uparrow} ^ {t} (\sigma) = \left\{\left\{\left(c _ {\mu} ^ {t}, c _ {\delta} ^ {t}\right) \mid \mu \in \mathcal {N} _ {\uparrow} (\sigma) \text {a n d} \delta \in \mathcal {C} (\sigma , \mu) \right\} \right\}.
$$
It is straightforward to show that the color update function  $\mathrm{UPD}_1$  performs generalized augmented message passing and that CWN can thus be weakly simulated with a simulation factor of 2. However, for this Algorithm 6 would generate more vertices than cells. Instead, we present a hand-crafted graph transformation that requires no additional vertices. We define a color update function that replaces the non pairwise message passing into separate pairwise message aggregations:
$$
c _ {\sigma} ^ {t + 1} = \mathrm {U P D} _ {2} (\sigma , X, c ^ {t}) = \mathrm {H A S H} \bigg (\pi_ {\sigma} ^ {t}, \pi_ {\mathcal {B}} ^ {t} (\sigma), \pi_ {\mathcal {N} _ {\uparrow}} ^ {t} (\sigma), \pi_ {\mathcal {N} _ {\uparrow}} ^ {t} (\sigma) \bigg),
$$
$$
\mathcal {N} _ {\uparrow} (\sigma) = \{\delta \mid \mu \in \mathcal {N} _ {\uparrow} (\sigma) \text {a n d} \delta \in \mathcal {C} (\sigma , \mu) \}.
$$
We require the dimension of cells to be encoded in the initial coloring. Note that  $\mathrm{UPD}_2$  performs augmented message passing:  $\pi_{\sigma}^{t}$  is a color atom (A3) and  $\pi_{\mathcal{B}}^{t}(\sigma), \pi_{\mathcal{N}_{\uparrow}}^{t}(\sigma), \pi_{\mathcal{N}_{\uparrow}}^{t}(\sigma)$  correspond to a AMPs (S2) that aggregates a color over a neighborhood from  $\mathcal{B}(\sigma), \mathcal{N}_{\uparrow}(\sigma), \mathcal{N}_{\uparrow}(\sigma)$ . Hence, we can strongly simulate  $\mathrm{UPD}_2$  by Theorem 3.5. Next, we show that for every  $t \geq 0$  the coloring produced in iteration  $2t$  of  $\mathrm{UPD}_2$  refines the coloring produced by  $t$  iterations of  $\mathrm{UPD}_1$  (CWN). Combining this with the fact that we can strongly simulated  $\mathrm{UPD}_2$  means that the coloring produced by  $2t$  iterations of strongly simulating  $\mathrm{UPD}_2$  refines the coloring from  $t$  iterations of CWN. This proves that we can weakly simulated CWN with a simulation factor of 2. We use  $\overline{\mathrm{CWN}}$  to denote the transformation  $\mathrm{AME}_{\mathrm{UPD}_2}$ .
Let  $t \geq 0$  be an integer. Let  $X, X'$  be two arbitrary regular cell complexes. Let  $\sigma, \nu$  be two arbitrary cells from these cell complexes. We show by induction on  $t$  that if  $\mathrm{UPD}_2$  is unable to distinguish  $\sigma$  from  $\nu$  in iteration  $2t$ , then  $\mathrm{UPD}_1$  is unable to distinguish cell  $\sigma$  from cell  $\nu$  in iteration  $t$ . We use  $c_{\sigma}^t$  to denote the color assigned to cell  $\sigma$  by the coloring from iteration  $t$  of  $\mathrm{UPD}_1$ . We use  $\pi_{\sigma}^t$  to denote the color assigned to vertex  $\sigma$  in from iteration  $t$  of  $\mathrm{UPD}_2$ . To prove the theorem, we show that  $\pi_{\sigma}^{2t} = \pi_{\nu}^{2t} \Rightarrow c_{\sigma}^t = c_{\nu}^t$ . The base case ( $t = 0$ ) holds trivially, as the features of the cells are encoded in the vertex features. Next, we assume the statement holds for  $t = T$  and prove that it holds for  $t = T + 1$ . This means we assume  $\pi_{\sigma}^{2(T + 1)} = \pi_{\nu}^{2(T + 1)}$  and have to show:
$$
c _ {\sigma} ^ {T + 1} = \left(c _ {\sigma} ^ {T}, c _ {\mathcal {B}} ^ {T} (\sigma), c _ {\uparrow} ^ {T} (\sigma)\right) \stackrel {!} {=} \left(c _ {\nu} ^ {T}, c _ {\mathcal {B}} ^ {T} (\nu), c _ {\uparrow} ^ {T} (\nu)\right) = c _ {\nu} ^ {T + 1}.
$$
From  $\pi_{\sigma}^{2(T + 1)} = \pi_{\nu}^{2(T + 1)}$  it follows that  $\pi_{\sigma}^{2T + 2} = \pi_{\nu}^{2T + 2}$  and  $\pi_{\sigma}^{2T} = \pi_{\nu}^{2T}$ . By the induction hypothesis it then follows that  $c_{\sigma}^{T} = c_{\nu}^{T}$ . From  $\pi_{\sigma}^{2T + 2} = \pi_{\nu}^{2T + 2}$  it follows that  $\pi_{\mathcal{B}}(\sigma)^{2T + 1} =$
Figure 4: A graph (left) transformed into bags of subgraphs (center) by a policy and the result of applying DSS to the graph and bags of subgraphs (right). Yellow vertices represent the original graph and red vertices the subgraphs.
$\pi_{\mathcal{B}}(\nu)^{2T + 1}$ . Hence, there exists a bijective function  $\alpha : \mathcal{B}(\sigma) \to \mathcal{B}(\nu)$  that maps vertices with the same color assigned by  $\pi^{2T + 1}$  to another, meaning for all  $x \in \mathcal{B}(\sigma)$  it holds that  $\pi_x^{2T + 1} = \pi_{\alpha(x)}^{2T + 1}$  and  $\pi_x^{2T} = \pi_{\alpha(x)}^{2T}$ . The existence of  $\alpha$  and the induction hypothesis imply that  $c_{\mathcal{B}}^T(\sigma) = c_{\mathcal{B}}^T(\nu)$ . Similarly, there exist a function  $\beta : \mathcal{N}_{\uparrow}(\sigma) \to \mathcal{N}_{\uparrow}(\nu)$  that bijectively map vertices assigned the same color by  $\pi^{2T + 1}$  to another. Thus, for every  $x \in \mathcal{N}_{\uparrow}(\sigma)$  it holds that  $\pi_x^{2T + 1} = \pi_{\beta(x)}^{2T + 1}$  and  $\pi_x^{2T} = \pi_{\beta(x)}^{2T}$ . Furthermore, it holds that for every  $x \in \mathcal{N}_{\uparrow}(\sigma)$  there exists a bijective function  $\gamma_x : \mathcal{B}(x) \to \mathcal{B}(\alpha(x))$  that maps cells to another that get assigned the same color by  $\pi^{2T}$ . Combining this yields
$$
\left\{\left\{\left\{\pi_ {x} ^ {2 T}, \pi_ {y} ^ {2 T} \right\} \right\} \mid y \in \mathcal {B} (x), x \in N _ {\uparrow} (\sigma) \right\} = \left\{\left\{\left\{\pi_ {x} ^ {2 T}, \pi_ {y} ^ {2 T} \right\} \right\} \mid y \in \mathcal {B} (x), x \in N _ {\uparrow} (\nu) \right\}.
$$
By construction each  $y$  has a higher dimension than  $x$ . This is encoded in the features, thus it follows that
$$
\left\{\left\{\left(\pi_ {x} ^ {2 T}, \pi_ {y} ^ {2 T}\right) \mid y \in \mathcal {B} (x), x \in N _ {\uparrow} (\sigma) \right\} \right\} = \left\{\left\{\left(\pi_ {x} ^ {2 T}, \pi_ {y} ^ {2 T}\right) \mid y \in \mathcal {B} (x), x \in N _ {\uparrow} (\nu) \right\} \right\}.
$$
From the induction hypothesis it follows that
$$
\left\{\left\{\left(c _ {x} ^ {T}, c _ {y} ^ {T}\right) \mid y \in \mathcal {B} (x), x \in N _ {\uparrow} (\sigma) \right\} \right\} = \left\{\left\{\left(c _ {x} ^ {T}, c _ {y} ^ {T}\right) \mid y \in \mathcal {B} (x), x \in N _ {\uparrow} (\nu) \right\} \right\}. \tag {1}
$$
We prove that this is equivalent to showing that  $c_{\uparrow}^{T}(\sigma) = c_{\uparrow}^{T}(\nu)$ . For this, we show that for every cell  $\rho$  it holds that
$$
\left\{(\mu , \delta) \mid \mu \in \mathcal {B} (\delta), \delta \in \mathcal {N} _ {\uparrow} (\rho) \right\} \stackrel {!} {=} \left\{(\mu , \delta) \mid \mu \in \mathcal {N} _ {\uparrow} (\rho) \text {a n d} \delta \in \mathcal {C} (\rho , \mu) \right\}.
$$
First, we show that the set on the left is a subset of the set on the right. Suppose an arbitrary element  $(\mu, \delta)$  with  $\mu \in \mathcal{B}(\delta)$  and  $\delta \in \mathcal{N}_{\uparrow}(\rho)$  from the left set. Note that  $\mu \in \mathcal{B}(\delta)$  implies  $\mu \prec \delta$ . Furthermore,  $\delta \in \mathcal{N}_{\uparrow}(\rho)$  means  $\rho \prec \delta$ . Thus,  $\delta \in \mathcal{C}(\rho, \mu)$ . Finally, this also implies that  $\mu \in \mathcal{N}_{\uparrow}(\rho)$ . Hence,  $(\mu, \delta)$  is an element of the right set. Next, we show that the set on the right is a subset of the left set. Let  $(\mu, \delta)$  with  $\mu \in \mathcal{N}_{\uparrow}(\rho)$  and  $\delta \in \mathcal{C}(\rho, \mu)$  be an arbitrary element from the set on the right. From  $\delta \in \mathcal{C}(\rho, \mu)$  it follows that  $\rho \prec \delta$  and  $\mu \prec \delta$ . By the definition of  $\mathcal{B}$ , it follows that  $\rho \in \mathcal{B}(\delta)$  and  $\mu \in \mathcal{B}(\delta)$ . It remains to show that  $\delta \in \mathcal{N}_{\uparrow}(\rho)$  meaning we need to show that  $\delta \in \{y | x \in \mathcal{N}_{\uparrow}(\delta)$  and  $y \in \mathcal{C}(\delta, x)\}$ . By definition we know that  $\mu \in \mathcal{N}_{\uparrow}(\delta)$  and that  $\delta \in \mathcal{C}(\delta, \mu)$ . Hence,  $\delta \in \mathcal{N}_{\uparrow}(\rho)$ . This implies that  $(\mu, \delta)$  is an element of the set on the left, meaning that the set on the right is a subset of the set on the left. Thus, the two sets are equal. We use this to rewrite Equation 1 into
$$
\left\{\left\{\left(c _ {x} ^ {T}, c _ {y} ^ {T}\right) \mid x \in \mathcal {N} _ {\uparrow} (\sigma) \text {a n d} y \in \mathcal {C} (\sigma , x) \right\} \right\} = \left\{\left\{\left(c _ {x} ^ {T}, c _ {y} ^ {T}\right) \mid x \in \mathcal {N} _ {\uparrow} (\nu) \text {a n d} y \in \mathcal {C} (\nu , x) \right\} \right\}. \tag {2}
$$
Hence,  $c_{\uparrow}^{T}(\sigma) = c_{\uparrow}^{T}(\nu)$ . This concludes the proof.
F.2 DSS: Weakly Simulating DSS Efficiently
Recall that the color update function of DSS WL is defined as
where  $\mathcal{N}_S(v)$  is the set of all neighbors of vertex  $v$  in subgraph  $S$ , and  $C_v^t$  contains the colors of  $v$  across different subgraphs
Let  $v \in V(G)$ . According to the graph transformation from Algorithm 6 with simulation factor 2 we would create a separate helper vertex for each vertex from  $X = \{x \mid x \in \mathcal{N}_G(v)\}$  where each  $x \in X$  is connected to all instances of  $x$  across the different subgraphs. A straightforward way to do this, is to add an additional copy of the original graph. A drawback of the graph transformation from Algorithm 6 is that because of  $C_v^t = \left\{\left\{c_{v,S'}^t \mid S' \in \pi(G) \right\} + v \in V(S')\right\}$  we need to create edges between every copy of  $v$  among all subgraphs, that means we create up to  $|V| \cdot |\pi(G)|^2$  extra edges. We can remedy this by adding edges between  $v$  in the original graph and all instances of  $v$  in the other subgraphs. Thus, it suffices to create at most  $|E(G)| + |V(G)| \cdot |\pi(G)|$  edges. Similarly, for every  $v \in V(G)$  and all subgraphs  $S \in \pi(G)$  containing  $v$  we need to add edges to the corresponding helper node for all neighbors  $\mathcal{N}_G(v)$ . Interestingly, it is not necessary to add these edges and instead it is possible to rely on the edges generated by adding an additional copy of the original graph. However, this means that for a vertex  $v$  it takes three steps to obtain the information  $\{\{C_x^t \mid x \in \mathcal{N}_G(v)\}\}$ . Thus, this comes at the cost of increasing the simulation factor to 3. We call the resulting graph transformation DSS. Figure 4 shows an example of DSS.
Definition F.2 (DSS). Given a graph  $G$  and policy  $\pi$  we define  $G_{\pi} = (V_{\pi}, E_{\pi}, X_{\pi}, Y_{\pi})$  to be the graph obtained by DSS. Where
$$
\begin{array}{l} V _ {\pi} = \bigcup_ {S \in \pi (G) \cup G} V (S), \\ E _ {\pi} = \bigcup_ {S \in \pi (G) \cup G} E (S) \cup \left\{\left\{v _ {G}, v _ {S} \right\} \mid S \in \pi (G) \text {a n d} v \in V (S) \right\}. \\ \end{array}
$$
For unlabeled graphs we introduce a vertex feature that encodes whether it was created from a subgraph  $S \in \pi(G)$  or was created from  $G$ . For labeled graphs we extend the features correspondingly. We add edge features that allow us to distinguish whether an edge was created due from  $E(S)$  or from  $\{\{v_G, v_S\} \mid S \in \pi(G) \text{ and } v \in V(S)\}$ .
We show that for arbitrary policies and graphs WL on the transformed graphs is at least as expressive as DSS-WL on the original graphs.
Theorem F.3.  $\overline{DSS}$  weakly simulates DSS-WL with a simulation factor of 3.
Proof. Let  $G$  and  $H$  be two graphs and  $\pi$  a policy. We use  $\tau_{v,S}^{t}$  to denote the coloring of vertex  $v_{S}$  obtained by iteration  $t$  of WL on  $G_{\pi}$  or  $H_{\pi}$ , and  $\tau_{v,G}^{t}$  to denote the color in iteration  $t$  of WL of vertex  $v$  that was created from the original graph  $G$  (respectively  $\tau_{v,H}^{t}$ ). We use  $c_{v,S}^{t}$  to denote the color of vertex  $v$  in subgraph  $S$  obtained in the  $t$ -th iteration of DSS-WL. We prove that for every  $t \geq 0$  it holds that
$$
\forall v, w \in V (G), S, T \in \pi (G): \tau_ {v, S} ^ {3 t} = \tau_ {w, T} ^ {3 t} \Rightarrow c _ {v, S} ^ {t} = c _ {w, T} ^ {t}
$$
by induction on the iteration  $t$  of DSS-WL.
Base case: All vertices  $v_{S} \in V(G_{\pi}), w_{T} \in V(H_{\pi})$  are assigned the same initial color (except the indicator that they were created from a subgraph) as their counterpart from DSS-WL. Thus from  $\tau_{v,S} = \tau_{w,T}$  it follows that  $c_{v,S}^{0} = c_{w,T}^{0}$ .
Induction hypothesis: We assume the statement holds for all  $t < n$  where  $n$  is an arbitrary non-negative integer.
Induction step: Let  $v \in V(G), w \in V(H), S \in \pi(G), T \in \pi(H)$ . We assume that  $\tau_{v,S}^{3(n+1)} = \tau_{w,T}^{3(n+1)}$  holds and want to show  $c_{v,S}^{n+1} = c_{w,T}^{n+1}$ . It is equivalent to the assumption that  $\tau_{v,S}^{3n+3} = \tau_{w,T}^{3n+3}$  which also implies  $\tau_{v,S}^{3n} = \tau_{w,T}^{3n}$ . We need to show that
$$
1. c _ {v, S} ^ {n} = c _ {w, T} ^ {n},
$$
2.  $\{\{c_{x,S}^n\mid x\in \mathcal{N}_S(v)\} \} = \{\{c_{y,T}^n\mid y\in \mathcal{N}_T(w)\} \}$  , and
3.  $C_v^n = C_w^n$ , and  $\{\{C_x^n \mid x \in \mathcal{N}_G(v)\}\} = \{\{C_y^n \mid y \in \mathcal{N}_H(w)\}\}$ .
From the induction hypothesis and  $\tau_{v,S}^{3n} = \tau_{w,T}^{3n}$  it immediately follows that  $c_{v,S}^n = c_{w,T}^n$ .
We want to show that  $\{\{c_{x,S}^n\mid x\in \mathcal{N}_S(v)\} \} = \{\{c_{y,T}^n\mid y\in \mathcal{N}_T(w)\} \}$ . From  $\tau_{v,S}^{3n + 3} = \tau_{w,T}^{3n + 3}$  it follows that there exists a bijection  $\beta :\mathcal{N}_{G_{\pi}}(v_S)\to \mathcal{N}_{H_{\pi}}(w_T)$  where for every  $x\in \mathcal{N}_{G_{\pi}}(v_S)$  it holds that  $\tau_x^{3n + 2} = \tau_{\beta (x)}^{3n + 2}$  which implies  $\tau_x^{3n} = \tau_{\beta (x)}^{3n}$ . Additionally, all neighbors of  $v_{S}$  are from the subgraph  $S$  or the original graph  $G$  and the vertex features allow us to distinguish between those two types. Thus,  $\beta$  maps vertices from  $V(S)$  to vertices from  $V(T)$ . Note that these are exactly the vertices whose colors are in  $\{\{c_{x,S}^n\mid x\in \mathcal{N}_S(v)\} \}$ ,  $\{\{c_{y,T}^n\mid y\in \mathcal{N}_T(w)\} \}$ . Thus, by combining this with the induction hypothesis we obtain that  $\{\{c_{x,S}^n\mid x\in \mathcal{N}_S(v)\} \} = \{\{c_{y,T}^n\mid y\in \mathcal{N}_T(w)\} \}$ .
We want to show that  $C_v^n = C_w^n$ . We use the function  $\beta$  defined in the previous paragraph. Due to the vertex features we know that  $\beta$  must map  $v_G$  to  $w_H$  as there exists only one neighbor with the "created from the original graph" feature among  $v_S'$  and  $w_T$ 's neighbors. Hence,  $\tau_{v,G}^{3n + 2} = \tau_{w,H}^{3n + 2}$  which implies  $\tau_{v,G}^{3n + 1} = \tau_{w,H}^{3n + 1}$ . This implies that there exists a bijective function  $\gamma$  mapping neighbors of  $v_G$  to neighbors of  $w_G$  such that they are assigned the same color by  $\tau^{3n}$ . The vertex  $v_G$  and  $w_G$  in the copy of  $H$  have two types of neighbors: neighbors  $\mathcal{N}_G(v),\mathcal{N}_H(w)$  from the original graph and copies of  $v,w$  over different subgraphs. By the vertex features we can distinguish these two types of neighbors. Thus,  $\gamma$  maps copies of  $v$  from the different subgraphs to copies of  $w$  over different subgraphs. As these vertices correspond to  $v,S'$  and  $w,T'$  for all  $S' \in \pi (G),T' \in \pi (G)$ , we can combine this with the induction hypothesis to obtain that  $C_v^n = \left\{\left|c_{v,S'}^n\mid S'\in \pi (G)\right.\right\} = \left\{\left|c_{w,T'}^n\mid T'\in \pi (H)\right.\right.$  and  $w\in V(T')\} = C_w^n$ .
Finally, we want to show  $\{\{C_x^n | \in \mathcal{N}_G(v)\} = \{\{C_y^n | y \in \mathcal{N}_H(w)\}\}$ . Since the copy of  $v$  and  $w$  created from the original graph can be uniquely identified by its vertex feature from  $\mathcal{N}_{G_\pi}(v), \mathcal{N}_{H_\pi}(w)$  we know that  $\beta$  maps these two vertices together and thus that  $\tau_{v,G}^{3n+2} = \tau_{w,H}^{3n+2}$ . Hence, there exists a bijective function  $\sigma: \mathcal{N}_G(v) \to \mathcal{N}_H(w)$  such that for every neighbor  $x \in \mathcal{N}_G(v)$  it holds that  $\tau_{x,G}^{3n+1} = \tau_{\sigma(x),H}^{3n+1}$ . We want to show that for every such neighbor  $x \in N_G(v)$  it holds that  $C_x^n = C_{\sigma(x)}^n$ . Let  $x$  be one such vertex. As  $\tau_{x,G}^{3n+1} = \tau_{\sigma(x),H}^{3n+1}$  we can use the same argument as in the previous paragraph to obtain that  $C_x^n = C_{\sigma(x)}^n$ . This concludes the proof of the induction hypothesis and shows that the theorem holds.
F.3 Strong Simulation of  $K$ -hop GNNs and KP-GNNs
We prove the following Theorem:
Theorem F.4.  $K$ -hop message passing and KP-GNNs [Feng et al., 2022] can be strongly simulated.
We prove the theorem only for KP-GNNs as  $K$ -hop message passing is a special case of KP-GNNs. In the proof we define a new message passing algorithm we call simple KP-GNN (sKP-GNN). We prove that (1) sKP-GNNs are as expressives as KP-GNNs in every iteration; and (2) that sKP-GNN is strongly simulatable. The theorem then follows from the combination of (1) and (2). We begin with the definition of sKP-GNN.
Definition F.5. (sKP-GNN) The color update function of sKP-GNNs is defined as
$$
m _ {v} ^ {l + 1, k} = \operatorname {M E S} _ {k} ^ {l, \text {n o r m a l}} \left(\left\{\left\{\left(h _ {u} ^ {l}, e _ {u v}\right) \mid u \in Q _ {v, G} ^ {k, t} \right\} \right\}\right) + f \left(G _ {v, G} ^ {k, t}\right),
$$
$$
f (G _ {v, G} ^ {k, t}) = \mathrm {E M B} ((E (Q _ {v, G} ^ {k, t}), C _ {k} ^ {k ^ {\prime}})),
$$
$$
h _ {v} ^ {l + 1} = \operatorname {C O M B I N E} ^ {l} \left(h _ {v} ^ {l + 1, 1}, \dots , h _ {v} ^ {l + 1, K}\right).
$$
Note that sKP-GNN only differs from KP-GNN by the definition of  $h_v^{l+1}$ :
- sKP-GNN:  $h_v^{l+1} = \text{COMBINE}^l(h_v^{l+1,1}, \ldots, h_v^{l+1,K})$  and
KP-GNN:  $h_v^{l+1} = \mathrm{COMBINE}^l\left(\left\{\left\{h_v^{l+1,k} \mid k = 1,2,\ldots,K\right\}\right\}\right)$ .
It is important to note that this transformation from a nested-aggregation to a tuple is only possible because there is a natural order on the multisets aggregated (the hop size  $k$ ). We prove that sKP-GNN is as expressive as KP-GNN in every iteration.
Lemma F.6. Suppose both  $sKP$ -GNN and  $KP$ -GNN use an injective COMBINE function. Then, for every graph  $G = (V, E, F)$  and integer  $l \geq 0$  it holds that the coloring  $sKP$ - $GNN^l(G, F|_V)$  refines the coloring  $KP$ - $GNN^l(G, F|_V)$ .
Proof. Let  $G = (V, E, F)$  and  $G' = (V', E', F')$  be two graphs,  $v, w \in (V \cup V')$ , and  $l \geq 0$ ,  $K \geq 1$  be integers. We furthermore assume that the COMBINE function is injective. We prove that if
$$
\operatorname {C O M B I N E} ^ {l} \left(h _ {v} ^ {l + 1, 1}, \dots , h _ {v} ^ {l + 1, K}\right) = \operatorname {C O M B I N E} ^ {l} \left(h _ {w} ^ {l + 1, 1}, \dots , h _ {w} ^ {l + 1, K}\right) \tag {3}
$$
$$
\operatorname {C O M B I N E} ^ {l} \left(\left\{\left\{h _ {v} ^ {l + 1, k} \mid k = 1, 2, \dots , K \right\} \right\}\right) = \operatorname {C O M B I N E} ^ {l} \left(\left\{\left\{h _ {w} ^ {l + 1, k} \mid k = 1, 2, \dots , K \right\} \right\}\right). \tag {4}
$$