SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. In Theorem 3.2 we will prove that  $(d + 1)$ -DRFWL(2) is strictly more powerful than  $d$ -DRFWL(2), for any  $d \geqslant 1$ . Therefore, we only need to show that 1-DRFWL(2) is strictly more powerful than WL(1). We will first prove that 1-DRFWL(2) can give an implementation of the WL(1) test. Actually, let
$$
W ^ {(0)} (u, v) = \left\{ \begin{array}{l l} W _ {\mathrm {W L} (1)} ^ {(0)} (u), & d (u, v) = 0, \\ \text {N U L L}, & d (u, v) = 1, \end{array} \right. \tag {15}
$$
be the initial 1-DRFWL(2) colors of  $(u,v)\in \mathcal{V}_G^2$  with  $0\leqslant d(u,v)\leqslant 1$ . Here  $W_{\mathrm{WL}(1)}^{(0)}(u)$  is the initial WL(1) color of node  $u$  (which is identical for all  $u\in \mathcal{V}_G$ ). It is obvious that  $W^{(0)}(u,v)$  does only depend on  $d(u,v)$ . As for the update rule, at the  $(2t - 1)$ -th iteration with  $t\geqslant 1$ , we ask
$$
\operatorname {P O O L} _ {0 1} ^ {1 (2 t - 1)} \left(\{\{(W ^ {(2 t - 2)} (u, v), W ^ {(2 t - 2)} (u, u)) \} \}\right) = W ^ {(2 t - 2)} (u, u), \tag {16}
$$
$$
\mathrm {P O O L} _ {1 0} ^ {1 (2 t - 1)} \left(\{\{(W ^ {(2 t - 2)} (v, v), W ^ {(2 t - 2)} (u, v)) \} \}\right) = W ^ {(2 t - 2)} (v, v), \tag {17}
$$
for those  $(u,v)$  with distance 1. Here, both  $\mathcal{N}_0(u)\cap \mathcal{N}_1(v)$  and  $\mathcal{N}_1(u)\cap \mathcal{N}_0(v)$  have only one element, so  $\mathrm{POOL}_{01}^{1(2t - 1)}$  and  $\mathrm{POOL}_{10}^{1(2t - 1)}$  simply select the second and the first component from the unique 2-tuple in the corresponding multisets, respectively. The  $\mathrm{HASH}_k^{(2t - 1)}$  functions for  $k = 0$  or 1 are chosen as
$$
\mathrm {H A S H} _ {1} ^ {(2 t - 1)} \left(W ^ {(2 t - 2)} (u, v), \left(M _ {i j} ^ {1 (2 t - 1)}\right) _ {0 \leqslant i, j \leqslant 1}\right) = \text {C O N C A T} \left(M _ {0 1} ^ {1 (2 t - 1)}, M _ {1 0} ^ {1 (2 t - 1)}\right), \tag {18}
$$
$$
\text {f o r} d (u, v) = 1,
$$
$$
\mathrm {H A S H} _ {0} ^ {(2 t - 1)} \left(W ^ {(2 t - 2)} (u, u), \left(M _ {i j} ^ {0 (2 t - 1)}\right) _ {0 \leqslant i, j \leqslant 1}\right) = W ^ {(2 t - 2)} (u, u), \tag {19}
$$
$$
\mathrm {f o r} d (u, v) = 0.
$$
What we did is to ask the  $(2t - 1)$ -th iteration of 1-DRFWL(2) to record the WL(1) colors  $\left(W_{\mathbf{WL}(1)}^{(t - 1)}(u), W_{\mathbf{WL}(1)}^{(t - 1)}(v)\right)$  in the 1-DRFWL(2) color of node pair  $(u, v)$ . In the  $2t$ -th iteration, 1-DRFWL(2) then uses this record to update the colors of  $(u, u)$  and  $(v, v)$ :
$$
\mathrm {H A S H} _ {1} ^ {(2 t)} \left(W ^ {(2 t - 1)} (u, v), \left(M _ {i j} ^ {1 (2 t)}\right) _ {0 \leqslant i, j \leqslant 1}\right) = W ^ {(2 t - 1)} (u, v), \tag {20}
$$
$$
\mathrm {H A S H} _ {0} ^ {(2 t)} \left(W ^ {(2 t - 1)} (u, u), \left(M _ {i j} ^ {0 (2 t)}\right) _ {0 \leqslant i, j \leqslant 1}\right) = M _ {1 1} ^ {0 (2 t)}, \tag {21}
$$
$$
\text {f o r} d (u, v) = 0.
$$
And we ask
$$
M _ {1 1} ^ {0 (2 t)} (u, v) = \operatorname {H A S H} _ {\mathrm {W L} (1)} ^ {(t)} \left(W ^ {(2 t - 2)} (u, u), \operatorname {P O O L} _ {\mathrm {W L} (1)} ^ {(t)} \left(\{\{W ^ {(2 t - 2)} (v, v): v \in \mathcal {N} (u) \}\}\right)\right). \tag {22}
$$
Here  $W^{(2t - 2)}(u,u)$  and  $W^{(2t - 2)}(v,v)$  are stored in  $W^{(2t - 1)}(u,v)$  in the last iteration (in the first and the second components respectively).  $\mathrm{HASH}_{\mathrm{WL}(1)}^{(t)}$  and  $\mathrm{POOL}_{\mathrm{WL}(1)}^{(t)}$  are the hashing functions used by WL(1) test, as in (1). It is easy to see the above implementation uses 2 iterations of 1-DRFWL(2) update to simulate 1 iteration of WL(1) update. Therefore, 1-DRFWL(2) is at least as powerful as WL(1) in terms of the ability to distinguish between non-isomorphic graphs.
To see why 1-DRFWL(2) is strictly more powerful than WL(1), we only need to find out a pair of graphs  $G$  and  $H$  such that WL(1) cannot distinguish between them while 1-DRFWL(2) can. Let  $G$  be two 3-cycles and  $H$  be one 6-cycle. Of course WL(1) cannot distinguish between  $G$  and  $H$ . However, 1-DRFWL(2) can distinguish between them because there are no triangles in  $H$  but two in  $G$ , as is made clear in the following. In the first iteration of 1-DRFWL(2), the  $M_{11}^{1(1)}$  term collects common neighbors of every node pair  $(u, v)$  with distance 1. In  $G$  this results in non-empty common neighbor multisetts for all distance-1 tuples  $(u, v)$ , while in  $H$  all such multisets are empty. 1-DRFWL(2) then makes use of this discrepancy by properly choosing the  $\mathrm{HASH}_1^{(1)}$  function that assigns different colors for distance-1 tuples in  $G$  and in  $H$ . Therefore, 1-DRFWL(2) can distinguish between  $G$  and  $H$ .
We restate Theorem 3.2 as following,
Theorem B.2. In terms of the ability to distinguish between non-isomorphic graphs,  $FWL(2)$  is strictly more powerful than  $d$ -DRFWL(2), for any  $d \geqslant 1$ . Moreover,  $(d + 1)$ -DRFWL(2) is strictly more powerful than  $d$ -DRFWL(2).
Proof. First we show that  $\mathrm{FWL}(2)$  can give an implementation of  $d$ -DRFWL(2) for any  $d \geqslant 1$ . The implementation is in three steps:
Distance calculation. For any 2-tuple  $(u,v)\in \mathcal{V}_G^2$ , let its FWL(2) color  $W^{(0)}(u,v)$  be 0 if  $u = v$ , 1 if  $\{u,v\} \in \mathcal{E}_G$ , and  $\infty$  otherwise. In the first  $(n - 1)$  iterations, FWL(2) can use the following update rule to calculate distance between any pair of nodes,
$$
W ^ {(t)} (u, v) = \min  \left(W ^ {(t - 1)} (u, v), \min  _ {w \in \mathcal {V} _ {G}} \left(W ^ {(t - 1)} (w, v) + W ^ {(t - 1)} (u, w)\right)\right). \tag {23}
$$
When  $t = n - 1$ , the FWL(2) color for any  $(u,v)\in \mathcal{V}_G^2$  gets stable and becomes  $d(u,v)$ .
Initial color generation. At the  $n$ -th iteration, FWL(2) transforms  $W^{(n-1)}(u,v) = d(u,v)$  into
$$
W ^ {(n)} (u, v) = \left\{ \begin{array}{l l} \operatorname {C O N C A T} \left(d (u, v), W _ {d - \mathrm {D R F W L} (2)} ^ {(0)} (u, v)\right), & \text {i f} 0 \leqslant d (u, v) \leqslant d, \\ \operatorname {C O N C A T} \left(d (u, v), \text {N U L L}\right), & \text {o t h e r w i s e .} \end{array} \right. \tag {24}
$$
Here  $W_{d\text{-DRFWL}(2)}^{(0)}(u,v)$  is the initial  $d$ -DRFWL(2) color of 2-tuple  $(u,v)$ . Because  $W_{d\text{-DRFWL}(2)}^{(0)}(u,v)$  only depends on  $d(u,v)$ , the RHS in (24) is a function of  $W^{(n-1)}(u,v) = d(u,v)$ , and thus complies with the general form of FWL(2) as in (3).
$d$ -DRFWL(2) Update. For the  $(n + t)$ -th iteration with  $t \geqslant 1$ , the FWL(2) test uses the following POOL $^{(n + t)}$  and  $\mathrm{HASH}^{(n + t)}$  functions to simulate the  $t$ -th iteration of  $d$ -DRFWL(2):
-  $\mathrm{HASH}^{(n + t)}$  reads the first field of  $W^{(n + t - 1)}(u,v)$  to get  $d(u,v) = k$ , and decides whether to update the color for  $(u,v)$  (if  $0 \leqslant k \leqslant d$ ) or not (otherwise).
- For every  $w \in \mathcal{V}_G$ , POOL $^{(n + t)}$  reads the first field of  $W^{(n + t - 1)}(w, v)$  and  $W^{(n + t - 1)}(u, w)$  to find that  $d(w, v) = i$  and  $d(u, w) = j$ , and then either selects  $W_{d\text{-DRFWL}(2)}^{(t - 1)}(w, v)$  and  $W_{d\text{-DRFWL}(2)}^{(t - 1)}(u, w)$  from the second field of  $W^{(n + t - 1)}(w, v)$  and  $W^{(n + t - 1)}(u, w)$ , and applies (7) to get  $M_{ij}^{k(t)}(u, v)$ , if it finds  $0 \leqslant i, j \leqslant d$ , or ignores the contribution from  $\left(W^{(n + t - 1)}(w, v), W^{(n + t - 1)}(u, w)\right)$  otherwise.
-  $\mathrm{HASH}^{(n + t)}$  calculates  $W_{d\text{-DRFWL}(2)}^{(t)}(u,v)$  using (6), and assigns it to the second field of  $W^{(n + t)}(u,v)$ , if it has decided that the color for  $(u,v)$  should be updated.
For the READOUT part, FWL(2) simply ignores the tuples  $(u,v)$  with  $d(u,v) > d$ . It is easy to see the above construction does provide an implementation for  $d$ -DRFWL(2).
Similarly, one can prove that  $(d + 1)$ -DRFWL(2) also gives an implementation of  $d$ -DRFWL(2), for any  $d \geqslant 1$ , by only executing the "initial color generation" and "d-DRFWL(2) update" steps stated above. Since the colors for  $(u, v)$  with  $d(u, v) > d$  never update, it is sufficient only keeping track of pairs with mutual distance  $\leqslant d + 1$ , which  $(d + 1)$ -DRFWL(2) is capable of.
Now we will turn to prove the separation result: for any  $d \geqslant 1$ , there exist graphs  $G$  and  $H$  that FWL(2) (or  $(d + 1)$ -DRFWL(2)) can distinguish, but  $d$ -DRFWL(2) cannot. We ask  $G$  to be two  $(3d + 1)$ -cycles, and  $H$  a single  $(6d + 2)$ -cycle. FWL(2) can distinguish between  $G$  and  $H$  by (i) calculating distance between every pair of nodes in  $G$  and  $H$ ; (ii) check if there is  $(u, v) \in \mathcal{V}^2$  such that  $d(u, v) = \infty$  at the READOUT step. The above procedure will give  $G$  a true label and  $H$  a false label.
To see why  $(d + 1)$ -DRFWL(2) can also distinguish between  $G$  and  $H$ , we designate that  $\mathrm{POOL}_{ij}^{d + 1(1)}$  be a multiset function that counts the elements in the multiset, for all  $0 \leqslant i,j \leqslant d + 1$ , and that  $\mathrm{HASH}_{d + 1}^{(1)}$  be the sum of all items in  $\left(M_{ij}^{d + 1(1)}(u,v)\right)_{0 \leqslant i,j \leqslant d + 1}$ . Briefly speaking, we are asking the first iteration of  $(d + 1)$ -DRFWL(2) to count how many nodes  $w$  contribute to the update of a distance- $(d + 1)$  tuple  $(u,v)$ , via the form  $\left(W^{(0)}(w,v), W^{(0)}(u,w)\right)$ . For any distance- $(d + 1)$  tuple in  $G$ , the count is 4 for  $d = 1$  and  $d + 5$  for  $d > 1$  (actually when  $d > 1$ , a distance- $(d + 1)$  tuple  $(u,v)$  splits a  $(3d + 1)$ -cycle into an inferior arc of length  $(d + 1)$  and a superior arc of length  $2d$ ;  $d$
nodes on the inferior arc and 3 nodes on the superior arc, along with  $u$  and  $v$ , sum up to the number; Figure 3a illustrates the case of  $d = 3$ . For a distance-  $(d + 1)$  tuple in  $H$ , the count becomes  $d + 2$ , which is always different from the count in  $G$ . The  $(d + 1)$ -DRFWL(2) test then uses this discrepancy to render different colors for a distance-  $(d + 1)$  tuple in  $G$  and in  $H$ , thus telling  $G$  from  $H$ .
The above proof also reveals why  $d$ -DRFWL(2) fails to distinguish between  $G$  and  $H$ . Actually, let  $(u,v)$  be a distance- $k$  tuple in  $\mathcal{V}_G^2$  or  $\mathcal{V}_H^2$  with  $0 \leqslant k \leqslant d$ . In the following we use  $W^{(t)}(u,v)$  to denote the  $d$ -DRFWL(2) color of  $(u,v)$  at the  $t$ -th iteration, both for  $(u,v) \in \mathcal{V}_G^2$  and  $(u,v) \in \mathcal{V}_H^2$ . We will now use induction to prove that for all  $t$ ,  $W^{(t)}(u,v)$  only depends on  $d(u,v)$ , and does not depend on whether  $(u,v)$  is in  $G$  or  $H$ . This implies that for  $(u,v) \in \mathcal{V}_G^2$  and  $(u',v') \in \mathcal{V}_H^2$ ,  $W^{(t)}(u,v) = W^{(t)}(u',v')$  holds for all iterations  $t$  as long as  $d(u,v) = d(u',v')$ .
Base case. Since the initial  $d$ -DRFWL(2) color of a distance- $k$  tuple  $(0 \leqslant k \leqslant d)$  only depends on  $k$ , the  $t = 0$  case is trivial.
Induction step. Now we assume  $W^{(t - 1)}(u,v)$  only depends on  $d(u,v)$  no matter which graph  $(u,v)$  is in, for some  $t\geqslant 1$ . We will then prove that no matter what  $\mathrm{HASH}_k^{(t)}$  and  $\mathrm{POOL}_{ij}^{k(t)}$  functions we choose,  $W^{(t)}(u,v)$  only depends on  $d(u,v)$  no matter which graph  $(u,v)$  is in. Actually, it is sufficient to prove that the multisets
$$
\{\{(i, j): w \in \mathcal {N} _ {i} (u) \cap \mathcal {N} _ {j} (v), 0 \leqslant i, j \leqslant d \} \} \tag {25}
$$
are equal, for any  $(u,v) \in \mathcal{V}_G^2 \cup \mathcal{V}_H^2$  with  $d(u,v) = k$  and  $0 \leqslant k \leqslant d$ . This is because the inductive hypothesis leads us to the fact that
$$
M _ {i j} ^ {k (t)} (u, v) = \mathrm {P O O L} _ {i j} ^ {k (t)} \left(\left\{\left\{\left(W ^ {(t - 1)} (w, v), W ^ {(t - 1)} (u, w)\right): w \in \mathcal {N} _ {i} (u) \cap \mathcal {N} _ {j} (v) \right\} \right\}\right).
$$
can only depend on the number of elements of the multiset on the RHS, because all elements in the RHS multiset must be equal. Therefore, the aforementioned condition guarantees that  $M_{ij}^{k(t)}(u,v)$  are all the same for any distance-  $k$  tuple  $(u,v)\in \mathcal{V}_G^2\cup \mathcal{V}_H^2$  with any fixed  $i,j,k$  and any  $t$ . This, along with the update rule (6), makes the induction step.
Now we prove that multisets defined by (25) are equal for all  $(u,v)\in \mathcal{V}_G^2\cup \mathcal{V}_H^2$  with any given  $d(u,v) = k,0\leqslant k\leqslant d$ . The intuition of the proof can be obtained from Figure 3b, which shows that for any distance-  $d$  tuple  $(u,v)$  in a cycle not shorter than  $(3d + 1)$ , the nodes that contribute to (25) are exactly those nodes on the inferior arc cut out by  $(u,v)$ , plus  $u$  and  $v$ . This means that the distance-  $d$  tuple is completely agnostic about the length of the cycle in which it lies, as long as the cycle length is  $\geqslant 3d + 1$ . Therefore, any distance-  $d$  tuple cannot tell whether it is in  $G$  or in  $H$ , resulting in the conclusion that the multisets in (25) are equal for all distance-  $d$  tuples  $(u,v)$ . Similar arguments apply to all distance-  $k$  tuples in  $G$  and  $H$ , as long as  $0\leqslant k\leqslant d$ . Therefore, we assert that  $W^{(t)}(u,v)$  does only depend on  $d(u,v)$ , no matter which graph  $(u,v)$  is in, at the  $t$ -th iteration. This finishes the inductive proof.
Notice that there are  $(6d + 2)$  distance- $k$  tuples in either  $G$  or  $H$ , for any  $0 \leqslant k \leqslant d$ ; moreover, each of those distance- $k$  tuples have identical colors. We then assert that  $G$  and  $H$  must get identical representations after running  $d$ -DRFWL(2) on them. Therefore,  $d$ -DRFWL(2) fails to distinguish between  $G$  and  $H$ .
We remark that for any  $d \geqslant 1$ ,  $d$ -DRFWL(2) tests cannot distinguish between two  $k$ -cycles and a  $2k$ -cycle, as long as  $k \geqslant 3d + 1$ . The proof for this fact is similar to the one elaborated above.
B.3 Proof of the equivalence in representation power between  $d$ -DRFWL(2) tests and  $d$ -DRFWL(2) GNNs
At the end of 3.2, we informally state the fact that  $d$ -DRFWL(2) GNNs have equal representation power to  $d$ -DRFWL(2) tests under certain assumptions. We now restate the fact as the following
Proposition B.3. Let  $q: \mathcal{G} \to$  colors be a  $d$ -DRFWL(2) test whose  $\mathrm{HASH}_k^{(t)}$ ,  $\mathrm{POOL}_{ij}^{k(t)}$  and READOUT functions are injective,  $\forall 0 \leqslant i,j,k \leqslant d$  and  $\forall t \geqslant 1$ ; in addition,  $q$  assigns different initial colors to 2-tuples  $(u,v)$  with different  $u,v$  distances. Let  $f: \mathcal{G} \to \mathbb{R}^p$  be a  $d$ -DRFWL(2) GNN
with  $T$ $d$ -DRFWL(2) GNN layers; the initial representations  $h_{uv}^{(0)}$  are assumed to depend only on  $d(u,v)$ .
If two graphs  $G$  and  $H$  get different representations under  $f$ , i.e.  $f(G) \neq f(H)$ , then  $q$  assigns different colors for  $G$  and  $H$ . Moreover, if  $G$  and  $H$  are graphs such that  $q$  assigns different colors for  $G$  and  $H$ , there exists a d-DRFWL(2) GNN  $f$  such that  $f(G) \neq f(H)$ .
Proof. For simplicity, we denote
$$
\mathcal {V} _ {G, \leqslant d} ^ {2} = \{(u, v) \in \mathcal {V} _ {G} ^ {2}: 0 \leqslant d (u, v) \leqslant d \}. \tag {26}
$$
We prove the first part by proving that, if two tuples  $(u,v)$  and  $(u^{\prime},v^{\prime})$  get different representations  $h_{uv}^{(T)}$  and  $h_{u^{\prime}v^{\prime}}^{(T)}$  after applying  $L_{T}\circ \sigma_{T - 1}\circ \dots \circ \sigma_{1}\circ L_{1}$ , then
$$
W ^ {(T)} (u, v) \neq W ^ {(T)} \left(u ^ {\prime}, v ^ {\prime}\right), \tag {27}
$$
after we apply  $q$  for  $T$  iterations. Since  $W^{(\infty)}(u,v)$  is a refinement of  $W^{(T)}(u,v)$ , (27) implies that  $W^{(\infty)}(u,v) \neq W^{(\infty)}(u',v')$ . Now,  $f(G) \neq f(H)$  means for all bijections  $b$  from  $\mathcal{V}_{G,\leqslant d}^2$  to  $\mathcal{V}_{H,\leqslant d}^2$ , there exists a pair  $(u,v) \in \mathcal{V}_{G,\leqslant d}^2$  such that  $h_{uv}^{(T)} \neq h_{u'v'}^{(T)}$  with  $(u',v') = b(u,v)$ . If the above statement holds true, this means for  $(u,v)$  and  $(u',v')$  we have  $W^{(\infty)}(u,v) \neq W^{(\infty)}(u',v')$ . Since this is the case for any bijection  $b$ , we assert
$$
\begin{array}{l} \text {R E A D O U T} \left(\left\{\left\{W ^ {(\infty)} (u, v): (u, v) \in \mathcal {V} _ {G, \leqslant d} ^ {2} \right\} \right\}\right) \\ \neq \operatorname {R E A D O U T} \left(\left\{\left\{W ^ {(\infty)} (u, v): (u, v) \in \mathcal {V} _ {H, \leqslant d} ^ {2} \right\} \right\}\right), \\ \end{array}
$$
or simply  $W(G) \neq W(H)$ , and the first part is proved.
Proof for the above statement can be conducted inductively. When  $T = 0$ ,  $h_{uv}^{(0)} \neq h_{u'v'}^{(0)}$  means  $d(u,v) \neq d(u',v')$ , since the initial representation  $h_{uv}^{(0)}$  of  $(u,v)$  only depends on the distance  $d(u,v)$ . By the second condition of  $q$ , the above fact further implies  $W^{(0)}(u,v) \neq W^{(0)}(u',v')$ , and the base case is proved.
Now, assuming that the above statement is true for  $T = \ell - 1$  with  $\ell \geqslant 1$ , we now prove the  $T = \ell$  case. Given  $h_{uv}^{(\ell)} \neq h_{u'v'}^{(\ell)}$ , there are two possibilities: (a)  $h_{uv}^{(\ell - 1)} \neq h_{u'v'}^{(\ell - 1)}$ ; (b) for some  $i, j$ ,  $a_{uv}^{ijk(\ell)} \neq a_{u'v'}^{ijk(\ell)}$ , where  $k = d(u,v) = d(u',v')$  (we can safely assume  $d(u,v) = d(u',v')$ , since otherwise (27) trivially holds).
For possibility (a), the inductive hypothesis tells us  $W^{(\ell -1)}(u,v) \neq W^{(\ell -1)}(u',v')$ , thus  $W^{(\ell)}(u,v) \neq W^{(\ell)}(u',v')$ . For possibility (b), notice that  $a_{uv}^{ijk(\ell)}$  is a function of the multiset
$$
\{\{(h _ {w v} ^ {(\ell - 1)}, h _ {u w} ^ {(\ell - 1)}): w \in \mathcal {N} _ {i} (u) \cap \mathcal {N} _ {j} (v) \} \}.
$$
Therefore,  $a_{uv}^{ijk(\ell)} \neq a_{u'v'}^{ijk(\ell)}$  means that for all bijection  $b'$  from  $\mathcal{N}_i(u) \cap \mathcal{N}_j(v)$  to  $\mathcal{N}_i(u') \cap \mathcal{N}_j(v')$ , there exists  $w \in \mathcal{N}_i(u) \cap \mathcal{N}_j(v)$  such that  $(h_{uv}^{(\ell-1)}, h_{uw}^{(\ell-1)}) \neq (h_{w'v'}^{(\ell-1)}, h_{u'w'}^{(\ell-1)})$ , where  $w' = b'(w)$ . Without loss of generality, let us discuss the case where  $h_{uv}^{(\ell-1)} \neq h_{w'v'}^{(\ell-1)}$ . In this case, the inductive hypothesis tells us  $W^{(\ell-1)}(w,v) \neq W^{(\ell-1)}(w',v')$ , thus  $(W^{(\ell-1)}(w,v), W^{(\ell-1)}(u,w)) \neq (W^{(\ell-1)}(w',v'), W^{(\ell-1)}(u',w'))$ . The other case  $h_{uw}^{(\ell-1)} \neq h_{u'w'}^{(\ell-1)}$  also leads to this result. Since the bijection  $b'$  is arbitrary, we assert that the multiset
$$
\{\{(W ^ {(\ell - 1)} (w, v), W ^ {(\ell - 1)} (u, w)): w \in \mathcal {N} _ {i} (u) \cap \mathcal {N} _ {j} (v) \} \}
$$
$$
\{\{(W ^ {(\ell - 1)} \left(w ^ {\prime}, v ^ {\prime}\right), W ^ {(\ell - 1)} \left(u ^ {\prime}, w ^ {\prime}\right)): w ^ {\prime} \in \mathcal {N} _ {i} \left(u ^ {\prime}\right) \cap \mathcal {N} _ {j} \left(v ^ {\prime}\right) \} \}
$$
are not equal. Since the functions  $\mathrm{POOL}_{ij}^{k(\ell)}$  and  $\mathrm{HASH}_k^{(\ell)}$  are injective, the above result implies  $W^{(\ell)}(u,v)\neq W^{(\ell)}(u',v')$ . So far, we have finished the induction step. Therefore, the first part of the theorem is true.
For the second part of the theorem, we quote the well-known result of Xu et al. [55] (Lemma 5 of the paper), that there exists a function  $f: \mathcal{X} \to \mathbb{R}^n$  on a countable input space  $\mathcal{X}$  such that any multiset function  $g$  can be written as
$$
g (X) = \phi \left(\sum_ {x \in X} f (x)\right) \tag {28}
$$
for some function  $\phi$ , where  $X$  is an arbitrary multiset whose elements are in  $\mathcal{X}$ .
Now, let  $G$  and  $H$  be graphs that both obtain their  $d$ -DRFWL(2) stable colorings under  $q$  after  $T$  iterations, and that  $q$  assigns different colors for them. We will now prove that there exists a  $d$ -DRFWL(2) GNN with  $T$ $d$ -DRFWL(2) GNN layers that gives  $G$  and  $H$  different representations.
Since the set of  $d$ -DRFWL(2) colors generated by  $q$  is countable, we can designate a mapping  $\nu : \mathrm{colors} \to \mathbb{N}$  that assigns a unique integer for every kind of  $d$ -DRFWL(2) color. Under mapping  $\nu$ , all  $\mathrm{HASH}_k^{(t)}$ ,  $\mathrm{POOL}_{ij}^{k(t)}$  and READOUT functions can be seen as functions with codomain  $\mathbb{N}$ . Then, the initial representation for  $(u,v)$  with  $0 \leqslant d(u,v) \leqslant d$  is  $h_{uv}^{(0)} = \nu(W^{(0)}(u,v))$ . For the update rules, the lemma in [55] tells us that by choosing proper  $m_{ijk}^{(t)}$  and  $\phi_{ijk}^{(t)}$  functions, the following equality can hold,
$$
\mathrm {P O O L} _ {i j} ^ {k (t)} (X) = \phi_ {i j k} ^ {(t)} \left(\sum_ {x \in X} m _ {i j k} ^ {(t)} (\nu (x))\right),
$$
with  $X$  being any multiset of 2-tuples of  $d$ -DRFWL(2) colors. Therefore, we can choose  $m_{ijk}^{(t)}$  in (10) following the instructions of the above lemma, and choose  $\bigoplus$  as  $\sum$ . We then choose the functions  $f_k^{(t)}$  in (11) as
$$
f _ {k} ^ {(t)} \left(h _ {u v} ^ {(t - 1)}, \left(a _ {u v} ^ {i j k (t)}\right) _ {0 \leqslant i, j \leqslant d}\right) = \mathrm {H A S H} _ {k} ^ {(t)} \left(\nu^ {- 1} \left(h _ {u v} ^ {(t - 1)}\right), \left(\phi_ {i j k} ^ {(t)} \left(a _ {u v} ^ {i j k (t)}\right)\right) _ {0 \leqslant i, j \leqslant d}\right). \tag {29}
$$
It is now easy to iteratively prove that  $h_{uv}^{(t)}$  is exactly  $\nu \left(W^{(t)}(u,v)\right)$ . For the pooling layer, we again leverage the lemma in [55] to assert that there exist functions  $r$  and  $M'$  such that
$$
\operatorname {R E A D O U T} (X) = M ^ {\prime} \left(\sum_ {x \in X} r (\nu (x))\right), \tag {30}
$$
where  $X$  is any multiset of  $d$ -DRFWL(2) colors. We then choose
$$
R (X) = \sum_ {x \in X} r (\nu (x)), \tag {31}
$$
and choose  $M = \nu \circ M'$ . Now the  $d$ -DRFWL(2) GNN constructed above with  $T$ $d$ -DRFWL(2) GNN layers produces exactly the same output as  $q$  (except that the output is converted to integers via  $\nu$ ). Because  $q$  assigns  $G$  and  $H$  different colors, the  $d$ -DRFWL(2) GNN gives different representations for  $G$  and  $H$ .
C Proofs of theorems in Section 4
We first give a few definitions that will be useful in the proofs.
Definition C.1 (Pair-wise cycle counts). Let  $u, v \in \mathcal{V}_G$ , we denote  $C_{k,l}(u,v)$  as the number of  $(k + l)$ -cycles  $S$  that satisfy:
-  $S$  passes  $u$  and  $v$ .
- There exists a  $k$ -path and an  $l$ -path (distinct from one another) from  $u$  to  $v$ , such that every edge in either path is an edge included in  $S$ .
For instance, the substructures counted by  $C_{2,3}(u,v)$  and  $C_{3,4}(u,v)$  are depicted in Figures 4a and 4b respectively.
Definition C.2 (Tailed triangle counts). Let  $u, v \in \mathcal{V}_G$ ,  $T(u, v)$  is defined as the number of tailed triangles in which  $u$  and  $v$  are at positions shown in Figure 4c.
Definition C.3 (Chordal cycle counts). Let  $u, v \in \mathcal{V}_G$ ,  $CC_1(u, v)$  and  $CC_2(u, v)$  are defined as the numbers of chordal cycles in which  $u$  and  $v$  are at positions shown in Figures 4d and 4e, respectively. Moreover, we use  $CC_1(u)$  and  $CC_2(u)$  to denote the node-level chordal cycle counts with the node  $u$  located at positions shown in Figures 4f and 4g, respectively.
The notations  $CC_{1}$  and  $CC_{2}$  are overloaded in the above definition, which shall not cause ambiguities with a check on the number of arguments.
Definition C.4 (Triangle-rectangle counts). Let  $u, v \in \mathcal{V}_G$ ,  $TR_1(u, v)$  and  $TR_2(u, v)$  are defined as the numbers of triangle-rectangles in which  $u$  and  $v$  are at positions shown in Figures 4h and 4i, respectively. We also define three types of node-level triangle-rectangle counts, namely  $TR_1(u)$ ,  $TR_2(u)$  and  $TR_3(u)$ . They are the numbers of triangle-rectangles where node  $u$  is located at positions shown in Figures 4j, 4k and 4l respectively.
(d)  $CC_{1}(u,v)$
(a)  $C_{2,3}(u,v)$
(b)  $C_{3,4}(u,v)$
(c)  $T(u,v)$
(e)  $CC_2(u, v)$
(i)  $TR_{2}(u,v)$