Title: Weisfeiler and Leman Go Loopy: A New Hierarchy for Graph Representational Learning
Venue: NeurIPS 2024
Accepted: yes
Keywords: Graph Neural Networks

Weisfeiler-Leman (WL) Test

Homomorphism Counting

Theory and Expressivity in GNNs

Cactus Graphs
Abstract: We introduce $r$-loopy Weisfeiler-Leman ($r$-$\ell$WL), a novel hierarchy of graph isomorphism tests and a corresponding GNN framework, $r$-$\ell$MPNN, that can count cycles up to length $r{+}2$. Most notably, we show that $r$-$\ell$WL can count homomorphisms of cactus graphs. This extends 1-WL, which can only count homomorphisms of trees and, in fact, is incomparable to $k$-WL for any fixed $k$. We empirically validate the expressive and counting power of $r$-$\ell$MPNN on several synthetic datasets and demonstrate the scalability and strong performance on various real-world datasets, particularly on sparse graphs.

Weisfeiler and Leman Go Loopy: A New Hierarchy for Graph Representational Learning

Abstract

We introduce r -loopy Weisfeiler-Leman ( r -glyph[lscript] WL), a novel hierarchy of graph isomorphism tests and a corresponding GNN framework, r -glyph[lscript] MPNN, that can count cycles up to length r +2 . Most notably, we show that r -glyph[lscript] WL can count homomorphisms of cactus graphs. This extends 1-WL, which can only count homomorphisms of trees and, in fact, we prove that r -glyph[lscript] WL is incomparable to k -WL for any fixed k . We empirically validate the expressive and counting power of r -glyph[lscript] MPNN on several synthetic datasets and demonstrate the scalability and strong performance on various real-world datasets, particularly on sparse graphs. Our code is available on GitHub.

1 Introduction

Graph Neural Networks (GNNs) (Scarselli et al., 2009; Bronstein et al., 2017) have become a prevalent architecture for processing graph-structured data, contributing significantly to various applied sciences, such as drug discovery (Stokes et al., 2020), recommender systems (Fan et al., 2019), and fake news detection (Monti et al., 2019).

Among various architectures, Message Passing Neural Networks (MPNNs) (Gilmer et al., 2017) are widely used in practice, as they encompass only local computation, leading to fast and scalable models. Despite their success, the representational power of MPNNs is bounded by the WeisfeilerLeman (WL) test, a classical algorithm for graph isomorphism testing (Xu et al., 2019; Morris et al., 2019). This limitation hinders MPNNs from recognizing basic substructures like cycles (Chen et al., 2020). However, specific substructures can be crucial in many applications. For example, in organic chemistry, the presence of cycles can impact various chemical properties of the underlying molecules (Deshpande et al., 2002; Koyutürk et al., 2004). Therefore, it is crucial to investigate whether GNNs can count certain substructures and to design architectures that surpass the limited power of MPNNs.

Several models have been proposed to surpass the limitations of WL. Many of these models draw inspiration from higher-order WL variants (Morris et al., 2019), enabling them to count a broader range of substructures. For instance, GNNs emulating 3 -WL can count cycles up to length 7 . However, this increased expressivity comes at a high computational cost, as 3 -WL does not respect the sparsity of real-world graphs, posing serious scalability issues. Hence, there is a critical need to design expressive GNNs that respect the inherent sparsity of real-world graphs (Morris et al., 2023).

*Equal contribution.

0

1

2

3

4

5

Input graph

1

3

4

5

1

3

4

5

1

2

3

v

= 0

. . .

. . .

. . .

0

4

0

4

v

= 5

Preprocessing: extracting

N

r

(

v

)

Pooling direct neighbors

Path-wise

GNN

Path-wise

GNN

Pooling path

embeddings

Pooling path

embeddings

Graph

Pooling

Graph

Embedding

Training: paths-to-graph embedding

Main Contributions. We introduce a novel class of color refinement algorithms called r -loopy Weisfeiler-Leman test ( r -glyph[lscript] WL) and a corresponding class of GNNs named r -loopy Graph Isomorphism Networks ( r -glyph[lscript] GIN) . The key idea is to collect messages not only from neighboring nodes but also from the paths connecting any two distinct neighboring nodes, as illustrated in Figure 1. This approach enhances the resulting GNNs' expressivity beyond 1-WL. In particular, r -glyph[lscript] WLcan count cycles up to length r +2 , even surpassing the k -WL hierarchy.

Furthermore, we prove that r -glyph[lscript] WL can homomorphism-count any cactus graph with cycles up to length r +2 . Cactus graphs are valuable due to their structural properties and simplicity, making them useful for modeling in areas such as electrical engineering (Nishi et al., 1986) and computational biology (Paten et al., 2011). For instance, aromatic compounds often form cactus graphs, where the molecular core, usually a cycle, is coonected to functional groups (e.g., carboxyl groups) that can significantly impact the properties of the molecule. Thus, the ability to homomorphism-count cactus graphs can enhance model performance, and it allows us to compare the expressive power of r -glyph[lscript] WLwith other popular GNNs in a quantitative manner (Barceló et al., 2021; B. Zhang et al., 2024). Specifically, we show that r -glyph[lscript] WLis more expressive than GNNs that include explicit homomorphism counts of cycle graphs, known as F -Hom-GNNs (Barceló et al., 2021). Additionally, 1 -glyph[lscript] WL can already separate infinitely many graphs that Subgraph k -GNNs (Frasca et al., 2022; Qian et al., 2022) cannot (see, e.g., Figure 7). The higher expressivity, paired with the local computations, highlights the enhanced potential of r -glyph[lscript] GIN, showing its competitive performance and the efficiency of its forward pass on real-world datasets, see Section 7.

2 Related Work

The notion of expressivity in standard neural networks is linked to the ability to approximate any continuous function (Cybenko, 1989; Hornik et al., 1989). In contrast, GNN expressivity is measured by the ability to distinguish non-isomorphic graphs. According to the Stone-Weierstrass theorem, these criteria are equivalent (Chen et al., 2019; Dasoulas et al., 2021): a network that can distinguish all graphs can approximate any continuous function. Therefore, research often focuses on determining which graphs a GNN can distinguish (Morris et al., 2023).

Xu et al. (2019) and Morris et al. (2019) proved that the expressive power of MPNNs is bounded by 1 -WL. Subsequent works (Maron et al., 2018; Morris et al., 2019, 2020) introduced higher-order GNNs that have the same expressive power as k -WL or its local variants (Geerts et al., 2022). Although these networks are universal (Maron et al., 2019b; Keriven et al., 2019), their exponential time and space complexity in k renders them impractical. Abboud et al. (2022) proposed k -hop GNNs which aggregate information from k -hop neighbors, thus, enhancing expressivity beyond 1 -WL but within 3 -WL (Feng et al., 2022). Michel et al. (2023) and Graziani et al. (2024) construct GNNs that process paths emanating from each node to overcome 1 -WL. Subgraph GNNs (Bevilacqua et al., 2021; You et al., 2021; Frasca et al., 2022; Huang et al., 2022) surpass 1 -WL by decomposing the initial input

graph into a bag of subgraphs. However, subgraph GNNs are upper-bounded by 3-WL (Frasca et al., 2022). A different line of work leverages positional encoding through unique node identifiers (Vignac et al., 2020), random features (Abboud et al., 2021; Sato et al., 2021) or eigenvectors (Lim et al., 2022; Maskey et al., 2022) to augment the expressive power of MPNNs.

While the predominant approach for gauging the expressive power of GNNs is within the k -WL hierarchy, such a measure is inherently qualitative, as it cannot shed light on substructures a particular GNN can encode. Lovász (1967) showed that homomorphism counts is a complete graph invariant , meaning two graphs are isomorphic if and only if their homomorphism counts are identical. Building on this result, B. Zhang et al. (2024) advocate for homomorphism-count as a quantitative measure of expressivity, as GNN architectures can homomorphism-count particular families of motifs. Tinhofer (1986, 1991) established that 1 -WL is equivalent to counting homomorphisms from graphs with tree-width one, while Dell et al. (2018) proved the equivalence between k -WL and the ability to count homomorphisms from graphs with tree-width k . Nguyen et al. (2020), Barceló et al. (2021), Welke et al. (2023), and Jin et al. (2024) used homomorphism counts to develop expressive GNNs.

Manually augmenting node features with homomorphism counts can be disadvantageous as performance depends on the chosen substructures. This can be alleviated by designing domain-agnostic GNNs that can learn structural information suitable for the task at hand. For instance, higher-order GNNs can count a large class of substructures as homomorphisms (B. Zhang et al., 2024), but they suffer from scalability issues. We propose r -glyph[lscript] WLand r -glyph[lscript] GIN, which can count homomorphisms of cactus graphs without adding explicit substructure counts. Our method is scalable to large datasets, particularly when the graphs in these datasets are sparse.

3 Preliminaries

Let G be the set of all simple and undirected graphs, and let G ∈ G . We denote the set of nodes by V ( G ) and the set of edges by E ( G ) . The direct neighborhood of a node v ∈ V ( G ) is defined as N ( v ) := { u ∈ V ( G ) | { v, u } ∈ E ( G ) } .

Definition 1. Let F, G ∈ G . A homomorphism from F to G is a map h : V ( F ) → V ( G ) such that { u, v } ∈ E ( F ) implies { h ( u ) , h ( v ) } ∈ E ( G ) . A subgraph isomorphism is an injective homomorphism.

Intuitively, a homomorphism from F to G is an edge-preserving map. A subgraph isomorphism ensures that F actually occurs as a subgraph of G . Consequently, it also maps distinct edges to distinct edges. A visual explanation can be found in Figure 5. We denote by Hom( F, G ) the set of homomorphisms from F to G and by hom( F, G ) its cardinality. Similarly, we denote by Sub( F, G ) the set of subgraph isomorphisms from F to G and by and sub( F, G ) its cardinality.

3.1 Graph Invariants

In order to unify different expressivity measures, we recall the definition of graph invariants.

glyph[negationslash]

Definition 2. Let P be a designated set, referred to as the palette . A graph invariant is a function ζ : G → P such that ζ ( G ) = ζ ( H ) for all isomorphic pairs G,H ∈ G . ζ is a complete graph invariant if ζ ( G ) = ζ ( F ) for all non-isomorphic pairs G,F ∈ G .

Complete graph invariants have maximal expressive power. However, no polynomial-time algorithm to compute a complete graph invariant is known. To compare the expressive power of different graph invariants, such as graph colorings and GNN architectures, we introduce the following definition.

glyph[negationslash]

Definition 3. Let γ, ζ be two graph invariants. We say that γ is more powerful than ζ ( γ glyph[subsetsqequal] ζ ) if for every pair G,H ∈ G , γ ( G ) = γ ( H ) implies ζ ( G ) = ζ ( H ) . We say that γ is strictly more powerful than ζ if γ glyph[subsetsqequal] ζ and there exists a pair F, G ∈ G such that γ ( G ) = γ ( H ) and ζ ( G ) = ζ ( H ) .

3.2 Message Passing Neural Networks and Weisfeiler-Leman

Message passing is an iterative algorithm that updates the colors of each node v ∈ V ( G ) as

The graph output after t iterations is given by

Here, g ( t ) , h are functions on the domain of multisets and f ( t ) is a function on the domain of tuples. For each t , the colorings c ( t ) are graph invariants. When the subsets of nodes with the same colors cannot be further split into different color groups, the algorithm terminates; the stable coloring after convergence is denoted by c ( G ) .

Choosing injective functions for all f ( t ) and setting g ( t ) and h as the identity function results in 1-WL (Weisfeiler et al., 1968). If f ( t ) , g ( t ) , h are chosen as suitable neural networks, one obtains a Message Passing Neural Network (MPNN). Xu et al. (2019) proved that MPNNs are as powerful as 1 -WL if the functions f ( t ) , g ( t ) , and h are injective on their respective domains. The k -WL algorithms uplift the expressive power of 1 -WL by considering interactions between k -tuples of nodes. This results in a hierarchy of strictly more powerful graph invariants (see Appendix B.1 for a formal definition).

3.3 Homomorphism and Subgraph Counting Expressivity

A more nuanced graph invariant can be built by considering the occurrences of a motif F .

Definition 4. Let F ∈ G . A graph invariant ζ can homomorphism-count F if for all pairs G,H ∈ G ζ ( G ) = ζ ( H ) implies hom( F, G ) = hom( F, H ) . By analogy, ζ can subgraph-count F if for all pairs G,H ∈ G , ζ ( G ) = ζ ( H ) implies sub( F, G ) = sub( F, H ) .

If F is a family of graphs, we say that ζ can homomorphism-count F if ζ can homomorphism-count every F ∈ F ; we denote the vector of homomorphism-count by hom( F , G ) := (hom( F, G )) F ∈F . Interpreting hom( F , · ) as a graph invariant, given by G ↦→ hom( F , G ) , another graph invariant ζ can homomorphism-count F if and only if ζ glyph[subsetsqequal] hom( F , · ) .

The ability of a graph invariant to count homomorphisms is highly relevant because hom( G , · ) is a complete graph invariant. Conversely, if ζ is a complete graph invariant, then ζ can homomorphismcount all graphs (Lovász, 1967). Additionally, homomorphism-counting serves as a quantitative expressivity measure to compare different WL variants and GNNs, such as k -WL, Subgraph GNNs, and other methods (Lanzinger et al., 2024; B. Zhang et al., 2024), and allows for relating them to our proposed r -glyph[lscript] WLvariant, as detailed in Corollary 2.

4 Loopy Weisfeiler-Leman Algorithm

In this section, we introduce a new graph invariant by enhancing the direct neighborhood of nodes with simple paths between neighbors.

glyph[negationslash]

Definition 5. Let G ∈ G . A simple path of length r is a collection p = { p i } r +1 i =1 of r +1 nodes such that { p i , p i +1 } ∈ E ( G ) and i = j = ⇒ p i = p j for every i, j ∈ { 1 , . . . , r } ,.

glyph[negationslash]

Simple paths are the building blocks of r -neighborhoods, which in turn are the backbone of our r -glyph[lscript] WLalgorithm. The following definition is inspired by (Cantwell et al., 2019; Kirkley et al., 2021).

Definition 6. Let G ∈ G and r ∈ N \ { 0 } , we define the r -neighborhood N r ( v ) of v ∈ V ( G ) as N r ( v ) := { p | p simple path of length r , p 1 , p r +1 ∈ N ( v ) , v / ∈ p } .

For consistency, we set N 0 ( v ) := N ( v ) . An example of the construction of r -neighborhood is shown in Figure 2, where different r -neighborhoods of node v are represented with different colors.

We generalize 1 -WL in (1) as follows.

v

N

0

(

v

)

N

1

(

v

)

N

2

(

v

)

Definition 7. We define the r -loopy Weisfeiler-Leman ( r -glyph[lscript] WL) test by the following color update:

where c ( t ) r ( p ) := ( c ( t ) r ( p 1 ) , c ( t ) r ( p 2 ) , . . . , c ( t ) r ( p r +1 ) ) is the sequence of colors of nodes in the path.

We denote by c ( t ) r ( G ) the final graph output after t iterations of r -glyph[lscript] WL, i.e.,

and by c r ( G ) the stable coloring after convergence. The stable coloring c r serves as graph invariant and will be referred to as r -glyph[lscript] WL.

5 Expressivity of r -glyph[lscript] WL

We analyze the expressivity of r -glyph[lscript] WLin terms of its ability to distinguish non-isomorphic graphs, subgraph-count, and homomorphism-count motifs. The proofs for all statements are in Appendix D.

5.1 Isomorphism Expressivity

It is straightforward to check that 0 -glyph[lscript] WLcorresponds to 1 -WL, since N 0 ( v ) = N ( v ) for all nodes v . However, increasing r leads to a strict increase in expressivity.

Proposition 1. Let 0 ≤ q < r . Then, r -glyph[lscript] WL is strictly more powerful than q -glyph[lscript] WL. In particular, every r -glyph[lscript] WL is strictly more powerful than 1 -WL.

This shows that the number of graphs we can distinguish monotonically increases with r . We empirically verify this fact on several synthetic datasets in Section 7.

5.2 Subgraph Expressivity

Recent studies highlight limitations in the ability of certain graph invariants to subgraph-count cycles. For instance, 1 -WL cannot subgraph-count cycles (Chen et al., 2020, Theorem 3.3), while 3 -WL can only subgraph-count cycles of length up to 7 (Arvind et al., 2020, Theorem 3.5). Similarly, Subgraph GNNs have limited cycle-counting ability (Huang et al., 2022, Proposition 3.1). In contrast, r -glyph[lscript] WL can count cycles of arbitrary length, as shown in the following statement.

Theorem 1. For any r ≥ 1 , r -glyph[lscript] WL can subgraph-count all cycles with at most r +2 nodes.

Since 3 -WL cannot subgraph-count any cycle with more than 7 nodes, Theorem 1 implies that 6 -glyph[lscript] WL is not less powerful than 3 -WL. This observation generalizes to any k -WL, as shown next.

Corollary 1. Let k ∈ N . There exists r ∈ N , such that r -glyph[lscript] WL is not less powerful than k -WL. Specifically, r ∈ O ( k 2 ) , with r ≤ k ( k +1) 2 -2 for even k and r ≤ ( k +1) 2 2 -2 for odd k .

The r -glyph[lscript] WLcolor refinement algorithm surpasses the limits of the k -WL hierarchy while only using local computation. This is particularly important since already 3 -WL is computationally infeasible, whereas our method can scale efficiently to higher orders if the graphs are sparse, which is commonly the case in real-world applications.

5.3 Homomorphism Expressivity

The following section unveils a close connection between the expressivity of r -glyph[lscript] WL and cactus graphs (Harary et al., 1953), a significant class between trees and graphs with tree-width 2.

Definition 8. A cactus graph is a graph where every edge lies on at most one simple cycle. For r ≥ 2 , an r -cactus graph is a cactus where every simple cycle has at most r vertices. We denote by M the set of all cactus graphs, and by M r the set of all q -cactus graphs for q ≤ r .

Figure 6 shows two examples of cactus graphs. From the expressivity perspective, the ability to homomorphism-count cactus graphs establishes a lower bound strictly between the homomorphismcounting capabilities of 1-WL and 3-WL (Neuen, 2024), as cactus graphs are a strict superset of all trees and a strict subset of all graphs of treewidth two. With this in mind, we are now ready to present our significant result on the homomorphism expressivity of our r -glyph[lscript] WLalgorithm.

Theorem 2. Let r ≥ 0 . Then, r -glyph[lscript] WL can homomorphism-count M r +2 .

We refer to Appendix G for a detailed proof of Theorem 2, which is fairly involved and requires defining canonical tree decompositions of cactus graphs and unfolding trees of r -glyph[lscript] WL. Demonstrating their strong connection, we then follow the approach in (Dell et al., 2018; B. Zhang et al., 2024) to decompose homomorphism counts of cactus graphs. In fact, we prove a more general result, showing that r -glyph[lscript] WLcan count all fan-cactus graphs , see Appendix G for more details.

The class M 2 contains only forests; hence, Theorem 2 implies the standard results on the ability of 1 -WL to count forests. Since forests are the only class of graphs 1 -WL can count, Theorem 2 implies that r -glyph[lscript] WLis always strictly more powerful than 1 -WL, corroborating the claim in Proposition 1.

The implications of Theorem 2 are profound: it establishes that r -glyph[lscript] WLcan homomorphism-count a large class of graphs. Specifically, Theorem 2 provides a quantitative expressivity measure that enables comparison of r -glyph[lscript] WL's expressivity with other WL variants and GNNs. This comparison is achieved by examining the range of graphs that r -glyph[lscript] WLcan homomorphism-count against those countable by other models, as detailed in works by Barceló et al. (2021) and B. Zhang et al. (2024). For instance, B. Zhang et al. (2024) showed that Subgraph GNNs (Bevilacqua et al., 2021; You et al., 2021; Frasca et al., 2022; Huang et al., 2022) are limited to homomorphism-count graphs with end-point shared NED. Hence, Subgraph GNNs can not homomorphism-count F = { } , while 1 -glyph[lscript] WLcan. Based on this, we can identify pairs of graphs that 1 -glyph[lscript] WLcan distinguish but Subgraph GNNs cannot. We summarize these and other implications of Theorem 2 in the following corollary.

Corollary 2. Let r ∈ N \ { 0 } . Then,

r -glyph[lscript] WL is more powerful than F -Hom-GNNs, where F = { C 3 , . . . , C r +2 } .

ii) 1 -glyph[lscript] WL is not less powerful than Subgraph GNNs. In particular, any r -glyph[lscript] WL can separate infinitely many graphs that Subgraph GNNs fail to distinguish.

iii) For any k > 0 , 1 -glyph[lscript] WL is not less powerful than Subgraph k -GNNs. In particular, any r -glyph[lscript] WL can separate infinitely many graphs that Subgraph k -GNNs fail to distinguish.

iv) r -glyph[lscript] WL can subgraph-count all graphs F such that spasm( F ) ⊂ M r +2 , where spasm( F ) := { H ∈ G | ∃ surjective h ∈ Hom( F, H ) } . In particular, if 1 ≤ r ≤ 4 , then r -glyph[lscript] WL can subgraph-count all paths up to length r +3 .

A detailed explanation of Subgraph ( k -)GNNs, F -Hom-GNNs, along with the proofs of Corollary 2, can be found in Appendix H. Finally, we note that Theorem 2 states a loose lower bound on the homomorphism expressivity of r -glyph[lscript] WL. This observation opens the avenue for future research to explore tight lower bounds, or upper bounds, on the homomorphism expressivity of r -glyph[lscript] WL.

6 Loopy Message Passing

In this section, we build a GNN emulating r -glyph[lscript] WL.

Definition 9. For t ∈ { 0 , . . . , T -1 } and k ∈ { 0 , . . . , r } , r -glyph[lscript] MPNN applies the following message, update and readout functions:

In the following statement, we link the expressive power of r -glyph[lscript] MPNN and r -glyph[lscript] WL.

Theorem 3. For fixed t, r ≥ 0 , t iterations of r -glyph[lscript] WL are more powerful than r -glyph[lscript] MPNN with t layers. Conversely, r -glyph[lscript] MPNN is more powerful than r -glyph[lscript] WL if the functions f ( t ) , g ( t ) in (3) are injective.

The previous result derives conditions under which r -glyph[lscript] MPNN is as expressive as r -glyph[lscript] WL. To implement r -glyph[lscript] MPNN in practice, we choose suitable neural layers for f ( t ) k , g ( t ) , and h in Definition 9. As a consequence of (Xu et al., 2019, Lemma 5), the aggregation function in (3) can be written as

for suitable functions f, g . Since 1-WL is injective on forests (Arvind et al., 2015), hence on paths, and since GIN can approximate 1-WL (Xu et al., 2019), we choose f = MLP and g = GIN . Hence, r -glyph[lscript] GIN is defined as an r -glyph[lscript] MPNN that updates node features via

To reduce the number of learnable parameters in (4), the GIN k can be shared among all k . Nothing prevents from choosing a different path-processing layer; we opted for GIN because it is simple yet maximally expressive on paths. We refer to Figure 1 for a visual depiction of r -glyph[lscript] GIN.

Computational Complexity The complexity of r -glyph[lscript] GIN is O ( | E | + ∑ v ∈ V ( G ) ∑ r k =1 2 k |N k ( v ) | ) . The former addend is the standard message complexity, while the latter arises from applying GIN to paths of length k ≤ r . This implies that our model's complexity scales linearly with the number of edges, and with the number of paths within N k ( v ) . The number of such paths is typically less than the number of edges. For example, ZINC12K has overall 598 K edges while only containing 374K paths in N r ( v ) for 1 ≤ r ≤ 5 . Hence, the runtime overhead is small in practice. Compared to 3-WLGNN (Dwivedi et al., 2022a), which has the same cycle-counting expressivity, our model requires ca. 10 seconds/epoch while 3-WLGNN takes ca. 329 . 49 seconds/epoch on ZINC12K. Our runtime is comparable to that of GAT, MoNet, or GatedGCN (see Table 10 for a thorough comparison).

Comparison with (Michel et al., 2023) PathNN updates node features by computing all possible paths starting from each node. In contrast, our approach selects paths between distinct neighbors, potentially resulting in fewer paths. For instance, a tree's r -neighborhoods ( r ≥ 1 ) are empty, while counts of paths between nodes are quadratic. Notably, Michel et al. (2023) do not explore the impact of increasing the path length on architecture expressiveness, a consideration we address (see, e.g., Proposition 1 and Corollary 1). Another significant contribution of our work, which we assert does not hold (at least not trivially) for PathNN, is the provable ability to subgraph-count cycle graphs (see, e.g., Theorem 1) and homomorphism-count cactus graphs (see, e.g., Theorem 2).

7 Experiments

All instructions to reproduce the experiments are available on GitHub (MIT license). Additional information on the training and test details can be found in Appendix C.

Expressive Power. We showcase the expressive power of r -glyph[lscript] GIN on synthetic datasets:

GRAPH8C (Balcilar et al., 2021) comprises 11 117 connected non-isomorphic simple graphs on 8 nodes; 312 pairs are 1 -WL equivalent but none is 3 -WL equivalent.

EXP_ISO (Abboud et al., 2022) comprises 600 pairs of 1 -WL equivalent graphs.

0

6

.

4 10

-

2

1

0

1

2

3

r

GRAPH8C

0

1

0

1

2

3

r

EXP_ISO

Proportion of indistinguishable pairs (

)

0 10

-

3

10

-

2

10

-

1

0

1

2

3

r

COSPECTRAL10

010

-

5

1

10

2

1

2

3

4

r

SR16622

L

1

distance (

→

)

r

Model

Basic (60)

Regular (140)

Extension (100)

CFI (100)

3-WL

60

50

100

60

PPGN

60

50

100

23

NestedGNN

59

48

59

0

GSN

60

99

95

0

OSAN

52

41

82

2

4

-

glyph[lscript]

GIN

60

100

95

2

0

1

2

3

4

5

6

1

2

1

acc.

EXP

CEXP

CSL

0

1

2

3

4

5

6

1

2

1

acc.

EXP

CEXP

CSL

r

COSPECTRAL10 (van Dam et al., 2003): the dataset comprises two cospectral 4 -regular non-isomorphic graphs on 10 nodes which are 1 -WL equivalent (see, e.g., Figure 8a).

SR16622 (Michel et al., 2023) comprises two strongly regular graphs on 16 nodes, namely the Shrikhande and the 4 × 4 rook graph, which are 3 -WL equivalent (see, e.g., Figure 8b).

The goal is to check whether the model can distinguish non-isomorphic pairs at initialization. The results are shown in Figure 3.

Additionally, Table 1 shows the performance on BREC (Wang et al., 2024), which includes 400 pairs of non-isomorphic graphs ranging from 1 -WL to 4 -WL equivalent. The baselines include PPGN, which is 3-WL equivalent and can count up to 7-cycles and homomorphism-count all graphs of tree-width 2; NestedGNN which is between 1-WL and 3-WL; GSN which is more powerful than 1-WL but whose expressive power depends on the chosen pattern.

Finally, Figure 4 reports the performance on synthetic classification tasks:

EXP, CEXP (Abboud et al., 2021) require expressive power beyond 1-WL.

CSL (Murphy et al., 2019) comprises 150 cycle graphs with skip links (see, e.g., Figure 8c). The task is to predict the length of the skip link.

Counting Power. Following (B. Zhang et al., 2024), we use the SUBGRAPHCOUNT dataset (Chen et al., 2020) to test the ability to homomorphism- and subgraphs-count exemplary motifs.

There is a strict hierarchy in the expressive power of the baselines: MPNN glyph[subsetsqequal] Subgraph GNN glyph[subsetsqequal] local 2-GNN glyph[subsetsqequal] local 2-FGNN. These variants, apart from MPNNs, are more expressive than 1-WL and can subgraph-count up to 7-cycles.

Real-World Datasets. We experimented with three benchmark datasets: ZINC250K (Irwin et al., 2012), ZINC12K (Dwivedi et al., 2022a), and QM9 (Wu et al., 2018) which consist of 250 000 , 12 000 , and 130 831 molecular graphs, respectively. We report the mean and standard deviation over 4 random seeds.

For ZINC250K and ZINC12K, we selected as baseline models standard MPNNs (GIN, GCN, GAT), Subgraph GNNs (NestedGNN, GNNAK+, SUN), domain-agnostic GNNs fed with substructure counts (GSN, CIN), a GNN processing paths (PathNN), and expressive GNNs with provable cycle counting power (HIMP, SignNet, I2-GNN, DRFWL). Following the standard procedure, we kept the number of parameters under 500K (Dwivedi et al., 2022a) for ZINC12K. The results are detailed in Table 3.

For the QM9 dataset, we followed the setup of (Huang et al., 2022; Zhou et al., 2023). Specifically, the test MAE is multiplied by the standard deviation of the target and divided by the corresponding conversion unit. The baseline results and models were obtained from (Zhou et al., 2023), including expressive GNNs with provable cycle counting power. We omit methods that use additional geometric features to focus on the model's expressive power. The results are presented in Table 4.

Discussion of Results The results in Figures 3 and 4 and Table 1 constitute a strong empirical validation of our theory: increasing r leads to more expressive r -glyph[lscript] MPNN. Albeit 6 -glyph[lscript] WLis not less powerful than 3 -WL (see, e.g., Section 5.2), in practice, smaller values of r can already distinguish pair of graphs that are 3 -WL equivalent, such as the Shrikhande and the (4 × 4) rook graphs. In the BREC dataset, 4 -glyph[lscript] GIN distinguishes all pairs of strongly regular graphs, significantly outperforming 3 -WL (0/50 graphs). Notably, 4 -glyph[lscript] GIN can already distinguish 257 out of 400 total pairs of graphs, surpassing other expressive GNNs like PPGN (233/400), theoretically equivalent to 3 -WL, and NestedGNN (166/400). Refer to (Wang et al., 2024, Table 2) for detailed baseline results.

The results in Table 2 further substantiate our theory, as r -glyph[lscript] WLcan effectively count cycles of length r +2 (see, e.g., Theorem 1).

On molecular datasets, we observe that r -glyph[lscript] GIN, although designed for subgraph-counting cycles and homomorphism-counting cactus graphs, is highly competitive. Notably, we outperform the baseline 0 -glyph[lscript] GIN by 226% on ZINC12K and 400% on ZINC250K and surpass domain-agnostic methods such as CIN or GSN. We conjecture that this is attributed to straightforward optimization, driven by the simplicity of the architecture (see, e.g., Figure 1) and its inductive bias towards counting cycles.

Limitations Path calculations can become infeasible for dense graphs due to O ( N d r ) complexity, where N is the number of nodes and d is the average degree. However, for sparse graphs, the runtime remains reasonably low. For instance, preprocessing ZINC12K for r = 5 takes just over a minute.

8 Conclusion

In this paper,we introduce a novel hierarchy of color refinement algorithms, denoted as r -glyph[lscript] WL, which incorporates an augmented neighborhood mechanism accounting for nearby paths. We establish connections between r -glyph[lscript] WLand the classical k -WL. We construct a GNN ( r -glyph[lscript] MPNN) designed to emulate and match the expressive powerof r -glyph[lscript] WL. Theoretical and empirical evidence support the claim that r -glyph[lscript] MPNN can effectively subgraph-count cycles and homomorphism-count cactus graphs.

Future research could focus on precisely characterizing the expressivity of r -glyph[lscript] WL tests by identifying the maximal class of graphs that r -glyph[lscript] WL can homomorphism-count. This would facilitate comparisons by constructing pairs of graphs that r -glyph[lscript] WLcannot separate, but other WL variants can. Another promising direction involves exploring the generalization capabilities of GNNs with provable homomorphism-counting properties. The ability to homomorphism-count certain motifs could provide a mathematical framework to support the intuitive notion that the capacity to count relevant features may improve generalization. We observed this improved generalization experimentally in our ablation study on ZINC12K (see, e.g., Table 8).

References

Abboud, R., Ceylan, I. I., Grohe, M., and Lukasiewicz, T. (2021). 'The Surprising Power of Graph Neural Networks with Random Node Initialization'. In: International Joint Conference on Artificial Intelligence (IJCAI) , pp. 2112-2118.

Abboud, R., Dimitrov, R., and Ceylan, I. I. (2022). 'Shortest Path Networks for Graph Property Prediction'. In: Learning on Graphs Conference (LoG) , 5:1-5:25.

Arvind, V., Fuhlbrück, F., Köbler, J., and Verbitsky, O. (2020). 'On Weisfeiler-Leman Invariance: Subgraph Counts and Related Graph Properties'. In: Journal of Computer and System Sciences 113, pp. 42-59.

Arvind, V., Köbler, J., Rattan, G., and Verbitsky, O. (2015). 'On the Power of Color Refinement'. In: International Symposium Fundamentals of Computation Theory (FCT) . Vol. 9210. Lecture Notes in Computer Science, pp. 339-350.

Balcilar, M., Heroux, P., Gauzere, B., Vasseur, P., Adam, S., and Honeine, P. (2021). 'Breaking the Limits of Message Passing Graph Neural Networks'. In: International Conference on Machine Learning (ICML) , pp. 599-608.

Barceló, P., Geerts, F., Reutter, J., and Ryschkov, M. (2021). 'Graph neural networks with local graph parameters'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 25280-25293.

Bevilacqua, B., Frasca, F., Lim, D., Srinivasan, B., Cai, C., Balamurugan, G., Bronstein, M. M., and Maron, H. (2021). 'Equivariant Subgraph Aggregation Networks'. In: International Conference on Learning Representations (ICLR) .

Bodnar, C., Frasca, F., Otter, N., Wang, Y ., Liò, P., Montufar, G. F., and Bronstein, M. (2021). 'Weisfeiler and Lehman Go Cellular: CW Networks'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 2625-2640.

Bouritsas, G., Frasca, F., Zafeiriou, S., and Bronstein, M. M. (2023). 'Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting'. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 45.1, pp. 657-668.

Bresson, X. and Laurent, T. (2017). 'Residual gated graph convnets'. In: arXiv preprint arXiv:1711.07553 .

Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., and Vandergheynst, P. (2017). 'Geometric Deep Learning: Going beyond Euclidean Data'. In: IEEE Signal Processing Magazine 34.4, pp. 18-42.

Cantwell, G. T. and Newman, M. E. J. (2019). 'Message Passing on Networks with Loops'. In: Proceedings of the National Academy of Sciences 116.47, pp. 23398-23403.

Chen, Z., Chen, L., Villar, S., and Bruna, J. (2020). 'Can Graph Neural Networks Count Substructures?' In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 10383-10395.

Chen, Z., Villar, S., Chen, L., and Bruna, J. (2019). 'On the equivalence between graph isomorphism testing and function approximation with gnns'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 15868-15876.

Colbourn, C. J. and Booth, K. S. (1981). 'Linear Time Automorphism Algorithms for Trees, Interval Graphs, and Planar Graphs'. In: SIAM Journal on Computing 10.1, pp. 203-225.

Curticapean, R., Dell, H., and Marx, D. (2017). 'Homomorphisms are a good basis for counting small subgraphs'. In: ACM SIGACT Symposium on Theory of Computing (STOC) , pp. 210-223.

Cybenko, G. (1989). 'Approximation by superpositions of a sigmoidal function'. In: Mathematics of control, signals and systems 2.4, pp. 303-314.

Dasoulas, G., Santos, L. D., Scaman, K., and Virmaux, A. (Jan. 7, 2021). 'Coloring Graph Neural Networks for Node Disambiguation'. In: International Joint Conference on Artificial Intelligence (IJCAI) , pp. 2126-2132.

Dell, H., Grohe, M., and Rattan, G. (2018). 'Lovász Meets Weisfeiler and Leman'. In: International Colloquium on Automata, Languages, and Programming (ICALP) . Vol. 107. LIPIcs, 40:1-40:14.

Deshpande, M., Kuramochi, M., and Karypis, G. (2002). 'Automated Approaches for Classifying Structures'. In: ACM SIGKDD Workshop on Data Mining in Bioinformatics (BIOKDD) , pp. 11-18.

Dwivedi, V. P., Joshi, C. K., Luu, A. T., Laurent, T., Bengio, Y., and Bresson, X. (2022a). 'Benchmarking Graph Neural Networks'. In: Journal of Machine Learning Research 24.43, pp. 1-48.

Dwivedi, V. P., Rampáek, L., Galkin, M., Parviz, A., Wolf, G., Luu, A. T., and Beaini, D. (2022b). 'Long Range Graph Benchmark'. In: Advances in Neural Information Processing Systems . Ed. by S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh. Vol. 35. Curran Associates, Inc., pp. 22326-22340.

Fan, W., Ma, Y., Li, Q., He, Y., Zhao, Y. E., Tang, J., and Yin, D. (2019). 'Graph Neural Networks for Social Recommendation'. In: The World Wide Web Conference (WWW) , pp. 417-426.

Feng, J., Chen, Y., Li, F., Sarkar, A., and Zhang, M. (2022). 'How Powerful are K-hop Message Passing Graph Neural Networks'. In: Advances in Neural Information Processing Systems (NeurIPS) .

Fey, M., Yuen, J. G., and Weichert, F. (2020). 'Hierarchical Inter-Message Passing for Learning on Molecular Graphs'. In: ICML Graph Representation Learning and Beyond (GRL+) Workhop .

Fey, M. and Lenssen, J. E. (2019). 'Fast Graph Representation Learning with PyTorch Geometric'. In: ICLR Workshop on Representation Learning on Graphs and Manifolds .

Frasca, F., Bevilacqua, B., Bronstein, M., and Maron, H. (2022). 'Understanding and Extending Subgraph GNNs by Rethinking Their Symmetries'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 31376-31390.

Fürer, M. (2001). 'Weisfeiler-Lehman Refinement Requires at Least a Linear Number of Iterations'. In: Automata, Languages and Programming . Ed. by F. Orejas, P. G. Spirakis, and J. van Leeuwen, pp. 322-333.

Geerts, F. and Reutter, J. L. (2022). 'Expressiveness and Approximation Properties of Graph Neural Networks'. In: International Conference on Learning Representations (ICLR) .

Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. (2017). 'Neural Message Passing for Quantum Chemistry'. In: International Conference on Machine Learning (ICML) , pp. 1263-1272.

Graziani, C., Drucks, T., Jogl, F., Bianchini, M., scarselli, franco, and Gärtner, T. (2024). 'The Expressive Power of Path-Based Graph Neural Networks'. In: Forty-first International Conference on Machine Learning .

Hagberg, A. A., Schult, D. A., Swart, P., and Hagberg, J. (2008). 'Exploring Network Structure, Dynamics, and Function using NetworkX'. In: Python in Science Conference (SciPy),

Harary, F. and Uhlenbeck, G. E. (1953). 'On the Number of Husimi Trees, I'. In: Proceedings of the National Academy of Sciences of the United States of America 39.4, pp. 315-322.

Hornik, K., Stinchcombe, M., and White, H. (1989). 'Multilayer feedforward networks are universal approximators'. In: Neural networks 2.5, pp. 359-366.

Hu*, W., Liu*, B., Gomes, J., Zitnik, M., Liang, P., Pande, V ., and Leskovec, J. (2020a). 'Strategies for Pre-training Graph Neural Networks'. In: International Conference on Learning Representations (ICLR) .

-(2020b). 'Strategies for Pre-training Graph Neural Networks'. In: International Conference on Learning Representations .

Huang, Y., Peng, X., Ma, J., and Zhang, M. (2022). 'Boosting the Cycle Counting Power of Graph Neural Networks with I 2 -GNNs'. In: International Conference on Learning Representations (ICLR) .

Ioffe, S. and Szegedy, C. (2015). 'Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift'. In: International Conference on Machine Learning, (ICML) . Vol. 37, pp. 448-456.

Irwin, J. J., Sterling, T., Mysinger, M. M., Bolstad, E. S., and Coleman, R. G. (2012). 'ZINC: A Free Tool to Discover Chemistry for Biology'. In: Journal of Chemical Information and Modeling 52, pp. 1757-1768.

Jin, E., Bronstein, M., Ceylan, I. I., and Lanzinger, M. (2024). 'Homomorphism Counts for Graph Neural Networks: All About That Basis'. In: International Conference on Machine Learning (ICML) .

Keriven, N. and Peyré, G. (2019). 'Universal invariant and equivariant graph neural networks'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 7090-7099.

Kingma, D. P. and Ba, J. (2015). 'Adam: A Method for Stochastic Optimization'. In: International Conference on Learning Representations (ICLR) .

Kipf, T. N. and Welling, M. (2017). 'Semi-Supervised Classification with Graph Convolutional Networks'. In: International Conference on Learning Representations (ICLR) .

Kirkley, A., Cantwell, G. T., and Newman, M. E. J. (2021). 'Belief Propagation for Networks with Loops'. In: Science Advances 7.17, eabf1211.

Korte, B. and Vygen, J. (2018). Combinatorial Optimization . Springer.

Koyutürk, M., Grama, A. Y., and Szpankowski, W. (2004). 'An efficient algorithm for detecting frequent subgraphs in biological networks'. In: Bioinformatics 20 Suppl 1, pp. i200-7.

Lanzinger, M. and Barcelo, P. (2024). 'On the Power of the Weisfeiler-Leman Test for Graph Motif Parameters'. In: International Conference on Learning Representations (ICLR) .

Lim, D., Robinson, J. D., Zhao, L., Smidt, T., Sra, S., Maron, H., and Jegelka, S. (2022). 'Sign and Basis Invariant Networks for Spectral Graph Representation Learning'. In: International Conference on Learning Representations (ICLR) .

Lovász, L. M. (1967). 'Operations with structures'. In: Acta Mathematica Academiae Scientiarum Hungarica 18, pp. 321-328.

Maron, H., Ben-Hamu, H., Serviansky, H., and Lipman, Y. (2019a). 'Provably Powerful Graph Networks'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 2153-2164.

Maron, H., Ben-Hamu, H., Shamir, N., and Lipman, Y. (2018). 'Invariant and Equivariant Graph Networks'. In: International Conference on Learning Representations (ICLR) .

Maron, H., Fetaya, E., Segol, N., and Lipman, Y. (2019b). 'On the Universality of Invariant Networks'. In: International Conference on Machine Learning (ICML) , pp. 4363-4371.

Maskey, S., Parviz, A., Thiessen, M., Stärk, H., Sadikaj, Y., and Maron, H. (2022). 'Generalized Laplacian Positional Encoding for Graph Representation Learning'. In: NeurIPS 2022 Workshop on Symmetry and Geometry in Neural Representations .

Michel, G., Nikolentzos, G., Lutzeyer, J. F., and Vazirgiannis, M. (2023). 'Path Neural Networks: Expressive and Accurate Graph Neural Networks'. In: International Conference on Machine Learning (ICML) , pp. 24737-24755.

Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J., and Bronstein, M. M. (2017). 'Geometric deep learning on graphs and manifolds using mixture model cnns'. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 5115-5124.

Monti, F., Frasca, F., Eynard, D., Mannion, D., and Bronstein, M. M. (2019). Fake News Detection on Social Media using Geometric Deep Learning . arXiv: 1902.06673 .

Morris, C., Lipman, Y., Maron, H., Rieck, B., Kriege, N. M., Grohe, M., Fey, M., and Borgwardt, K. (2023). 'Weisfeiler and Leman Go Machine Learning: The Story so Far'. In: Journal of Machine Learning Research 24, 333:1-333:59.

Morris, C., Rattan, G., and Mutzel, P. (2020). 'Weisfeiler and Leman go sparse: Towards scalable higher-order graph embeddings'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 21824-21840.

Morris, C., Ritzert, M., Fey, M., Hamilton, W. L., Lenssen, J. E., Rattan, G., and Grohe, M. (2019). 'Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks'. In: AAAI Conference on Artificial Intelligence (AAAI) , pp. 4602-4609.

Murphy, R., Srinivasan, B., Rao, V., and Ribeiro, B. (2019). 'Relational Pooling for Graph Representations'. In: International Conference on Machine Learning (ICML) , pp. 4663-4673.

Neuen, D. (2024). 'Homomorphism-Distinguishing Closedness for Graphs of Bounded Tree-Width'. In: International Symposium on Theoretical Aspects of Computer Science (STACS) , 53:1-53:12.

Nguyen, H. and Maehara, T. (2020). 'Graph Homomorphism Convolution'. In: International Conference on Machine Learning (ICML) , pp. 7306-7316.

Nishi, T. and Chua, L. (1986). 'Uniqueness of solution for nonlinear resistive circuits containing CCCS's or VCVS's whose controlling coefficients are finite'. In: IEEE Transactions on Circuits and Systems 33.4, pp. 381-397.

Paszke, A. et al. (2019). 'PyTorch: An Imperative Style, High-Performance Deep Learning Library'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 8024-8035.

Paten, B., Diekhans, M., Earl, D., John, J. S., Ma, J., Suh, B., and Haussler, D. (2011). 'Cactus graphs for genome comparisons'. In: Journal of Computational Biology 18.3, pp. 469-481.

Qian, C., Rattan, G., Geerts, F., Niepert, M., and Morris, C. (2022). 'Ordered subgraph aggregation networks'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 21030-21045.

Sato, R., Yamada, M., and Kashima, H. (2021). 'Random Features Strengthen Graph Neural Networks'. In: SIAM International Conference on Data Mining (SDM) , pp. 333-341.

Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. (2009). 'The Graph Neural Network Model'. In: IEEE Transactions on Neural Networks 20.1, pp. 61-80.

Stokes, J. M. et al. (2020). 'A deep learning approach to antibiotic discovery'. In: Cell 180.4, 688702.e13.

Tinhofer, G. (1986). 'Graph isomorphism and theorems of Birkhoff type'. In: Computing 36, pp. 285300.

-(1991). 'A note on compact graphs'. In: Discrete Applied Mathematics 30, pp. 253-264.

van Dam, E. R. and Haemers, W. H. (2003). 'Which Graphs Are Determined by Their Spectrum?' In: Linear Algebra and its Applications . Vol. 373. Combinatorial Matrix Theory Conference, pp. 241-272.

Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. (2018). 'Graph Attention Networks'. In: International Conference on Learning Representations (ICLR) .

Vignac, C., Loukas, A., and Frossard, P. (2020). 'Building powerful and equivariant graph neural networks with structural message-passing'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 14143-14155.

Wang, Y. and Zhang, M. (2024). 'An Empirical Study of Realized GNN Expressiveness'. In: International Conference on Machine Learning (ICML) .

Weisfeiler, B. and Lehman., A. (1968). 'The reduction of a graph to canonical form and the algebra which appears therein'. In: Nauchno-Technicheskaya Informatsia 9.

Welke, P., Thiessen, M., Jogl, F., and Gärtner, T. (2023). 'Expectation-Complete Graph Representations with Homomorphisms'. In: International Conference on Machine Learning (ICML) , pp. 36910-36925.

Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Geniesse, C., Pappu, A. S., Leswing, K., and Pande, V. (2018). MoleculeNet: A Benchmark for Molecular Machine Learning . arXiv: 1703.00564 .

Xu, K., Hu, W., Leskovec, J., and Jegelka, S. (2019). 'How Powerful Are Graph Neural Networks?' In: International Conference on Learning Representations (ICLR) .

You, J., Gomes-Selman, J. M., Ying, R., and Leskovec, J. (2021). 'Identity-Aware Graph Neural Networks'. In: AAAI Conference on Artificial Intelligence (AAAI) , pp. 10737-10745.

Zhang, B., Gai, J., Du, Y., Ye, Q., He, D., and Wang, L. (2024). 'Beyond Weisfeiler-Lehman: A Quantitative Framework for GNN Expressiveness'. In: International Conference on Learning Representations (ICLR) .

Zhang, M. and Li, P. (2021). 'Nested Graph Neural Networks'. In: Advances in Neural Information Processing Systems (NeurIPS) , pp. 15734-15747.

Zhao, L., Jin, W., Akoglu, L., and Shah, N. (2022). 'From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness'. In: International Conference on Learning Representations (ICLR) .

Zhou, J., Feng, J., Wang, X., and Zhang, M. (2023). 'Distance-Restricted Folklore Weisfeiler-Leman GNNs with Provable Cycle Counting Power'. In: Advances in Neural Information Processing Systems (NeurIPS) .

Appendix

Table of Contents

A Additional Figures

Homomorphism

Subgraph

Isomorphism

Bijective

Homomorphism

Isomorphism

v

v

N

0

(

v

)

N

1

(

v

)

(a)

r

= 0

v

v

N

0

(

v

)

N

2

(

v

)

(b)

r

= 1

5

6

4

3

2

1

(a) Input graph

F

.

6

,

{

4

,

5

}

4

,

{

6

,

3

}

3

,

{

2

,

4

}

2

,

{

1

,

3

}

1

,

{

2

,

3

}

3

,

{

1

,

4

}

4

,

{

5

,

3

}

5

,

{

4

,

6

}

3

,

{

1

,

2

}

4

,

∅

3

,

∅

4

,

{

5

,

6

}

6

,

∅

5

,

∅

1

,

∅

2

,

∅

(b) Fürer graph

G

(

F

)

:

hom(

F, G

(

F

)) = 68

.

6

,

{

4

,

5

}

4

,

{

6

,

3

}

3

,

{

2

,

4

}

2

,

{

1

,

3

}

1

,

{

2

,

3

}

3

,

{

1

,

4

}

4

,

{

5

,

3

}

5

,

{

4

,

6

}

3

,

{

1

,

2

}

4

,

∅

3

,

∅

4

,

{

5

,

6

}

6

,

∅

5

,

∅

1

,

∅

2

,

∅

(c) Twisted Fürer graph

H

(

F

)

:

hom(

F, H

(

F

)) = 34

.

glyph[negationslash]

v

v

(a) COSPECTRAL10.

v

v

(b) SR16622.

v

v

(c) CSL example, skip length

2

(left) and

3

(right).

B Additional Notions

B.1 Higher-Order Weisfeiler-Leman Tests

It is possible to uplift the expressive power of WL by considering higher-order interactions. The simplest higher-order variant of WL is the k -dimensional Weisfeiler-Leman test, denoted by k -WL. Given a graph G with nodes V ( G ) and edges E ( G ) , the algorithm generates a new graph H where each node is a k -tuple of elements of V ( G )

and edges E ( H ) are built among those k -tuples that differ in one entry only

where d H is the Hamming distance. The algorithm assigns to each node v ∈ V ( H ) an initial color depending on the isomorphic type of the induced subgraph G [ v ] . The color refinements scheme is

exactly (1) applied to H . While H can be generated by a simple algorithm, the approach quickly becomes impractical as the number of nodes and edges grows exponentially in k .

(a) Input graphs.

(c)

3

-WL at initialization.

(d)

1

-

glyph[lscript]

WLafter one iteration.

C Experimental Details

Our model is implemented in PyTorch (BSD-3 license) (Paszke et al., 2019), using PyTorch Geometric (MIT license) (Fey et al., 2019). The r -neighborhoods are computed with NetworkX (Creative Commons Zero v1.0 Universal) (Hagberg et al., 2008) as preprocessing. Hyperparameters

on real-world datasets were tuned using grid search; for synthetic experiments, we fixed one configuration of hyperparameters. All experiments were run on an internal cluster with Intel Xeon CPUs (28 cores, 192GB RAM) and GeForce RTX 3090 Ti GPUs (4 units, 24GB memory each), as well as Intel Xeon CPUs (32 cores, 192GB RAM) and NVIDIA RTX A6000 GPUs (3 units, 48GB memory each). All models are trained with Adam optimizer (Kingma et al., 2015).

C.1 Synthetic Datasets

The SR16622 dataset is retrieved from the official PATHNN repository (MIT license) (Michel et al., 2023). The GRAPH8C dataset is downloaded from Australian National University webpage (Creative Commons Attribution 4.0 International (CC BY 4.0) license). The EXP, EXP_ISO, and CEXP datasets are downloaded from GNN-RNI official repository (GPL-3.0 license) (Abboud et al., 2021), while the corresponding splits are generated via Stratified 5-fold cross-validation. The CSL dataset is provided by torch_geometric , while the corresponding splits are taken from PathNN official repository. The SUBGRAPHCOUNT dataset is taken from the official repository (MIT license) of (Zhao et al., 2022). The BREC dataset is downloaded from its official repository (MIT license) (Wang et al., 2024). The configuration of hyperparameters can be found in Table 5. For the synthetic datasets, we fixed one configuration and studied the effect of increasing r on the expressive and counting power of the architecture.

For the SR16622, GRAPH8C, EXP_ISO, and COSPECTRAL10 datasets, we report the mean and standard deviation over 100 random seeds. For the EXP, CEXP, and CSL datasets, we report the mean and standard deviation of 5-fold cross-validation. For BREC, we follow the original setup and perform an α -level Hotellings T-square test; see (Wang et al., 2024) for more details. For the SUBGRAPHCOUNT dataset, we report the mean and standard deviation over 4 random seeds, using the original splits from (Zhao et al., 2022).

C.2 Real-World Datasets

All real-world datasets are provided by torch_geometric . The splits for both ZINC datasets are also provided by torch_geometric . For QM9, we follow the set-up of (Zhou et al., 2023) and use random 80 / 10 / 10 splits. Details for the datasets are provided in Table 6.

Hyperparameters were tuned using grid search. For ZINC12K, the grid was defined by Hidden Size ∈ { 64 , 128 } and Num. Layers ∈ { 3 , 4 , 5 } . For ZINC250K, the grid was defined by Hidden Size ∈ { 128 , 256 } and Num. Layers ∈ { 4 } . For the QM9 tasks, the grid was defined by Hidden Size ∈ { 64 , 128 } and Num. Layers ∈ { 3 , 4 , 5 } . For the QM9 tasks, we followed the training set-up of (Zhou et al., 2023), training for 400 epochs with a ReduceLROnPlateau scheduler, reducing the learning rate by a factor of 0 . 9 if the validation metric did not decrease for 10 epochs. The exact hyperparameters are given in Table 7.

All real-world datasets come with edge features. We use an encoder layer, followed by a linear layer to encode node, edge features, and atomic types before passing them to the r -glyph[lscript] GIN. Within the r -glyph[lscript] GIN layers, we process the edge features via a 2 -layered learnable MLP, and replace the GIN in (4) by GINE layers (Hu* et al., 2020a). After t rounds of r -glyph[lscript] GIN layer, we apply a two-layered MLP as decoder layer. In all experiments, BatchNorm1D (Ioffe et al., 2015) is used in the MLP layers. We refer to Figure 1 for a depiction of the architecture.

D Preparation for Proofs

We begin by recalling some core concepts that are relevant for Section 5 and the proofs therein. Definition 10. A node invariant ζ ( · ) is a mapping that assigns to each graph G ∈ G a function ζ G : V ( G ) → P , which satisfies

where H is any graph isomorphic to G and h is the corresponding isomorphism from H to G .

The following definition enables us to compare the expressive power of different node invariants.

Definition 11 (Node Invariant Refinement) . Given two node invariants γ and ζ . We say that ζ refines γ if for every fixed graph G and nodes u, v ∈ V ( G ) , it holds ζ G ( u ) = ζ G ( v ) ⇒ γ G ( u ) = γ G ( v ) . We write ζ glyph[subsetsqequal] γ .

We emphasize that every node invariant ζ induces a graph invariant A [ γ ] by collecting the multiset, i.e., G ↦→{{ ζ G ( v ) }} v ∈ V ( G ) . We denote the induced graph invariant of a node invariant γ as A [ γ ] .

The following lemma establishes a connection between the expressive power of two node invariants (see Definition 11) and that of their induced graph invariants (see Definition 3).

Lemma 1. Let ζ, γ be node invariant. If ζ glyph[subsetsqequal] γ , then A [ ζ ] is more powerful than A [ γ ] .

Proof. Let G,H be two graphs, and let P be the underlying palette of ζ, γ . Consider the function

As a consequence of ζ glyph[subsetsqequal] γ , φ is well-defined, since

Assume that A [ ζ ]( G ) = A [ ζ ]( H ) , i.e,

As φ is well-defined, we have

which leads to A [ γ ]( G ) = A [ γ ]( H ) .

E Appendix for Section 5.1

In this section, we provide the proof of Proposition 1 from the main paper.

Proposition 1. Let 0 ≤ q < r . Then, r -glyph[lscript] WL is strictly more powerful than q -glyph[lscript] WL. In particular, every r -glyph[lscript] WL is strictly more powerful than 1 -WL.

Proof of Proposition 1. Let r ≥ 0 . We aim to prove that ( r +1) -glyph[lscript] WLis strictly more powerful than r -glyph[lscript] WL. We begin by demonstrating that ( r +1) -glyph[lscript] WLis more powerful than r -glyph[lscript] WL.

To establish this, we rely on Lemma 1. Specifically, we demonstrate that the underlying ( r +1) -glyph[lscript] WL node invariant c r +1 refines c r . Moreover, we go beyond and show that the node invariant c ( t ) r +1 refines c ( t ) r at every iteration t ≥ 0 , which shows that t iterations of ( r +1) -glyph[lscript] WLare more powerful than t iterations of r -glyph[lscript] WL.

For this purpose, let G be a graph with node set V ( G ) . For t = 0 , c (0) r +1 glyph[subsetsqequal] c (0) r since both algorithms start with the same labels. By induction, we assume that

holds; we need to prove that (5) implies

Since HASH in Definition 7 is injective, c ( t ) r +1 ( u ) = c ( t ) r +1 ( v ) in (5) leads to

for all q ∈ { 0 , . . . , r } . The assumption c ( t ) r ( u q l,k ) = c ( t ) r ( v q l,k ) in (5) is satisfied for every path u q l = { u q l,k } ∈ N q ( u ) and v q l = { v q l,k } ∈ N q ( v ) for q = 0 , . . . , r , l = 1 , . . . , |N v | and k = 1 , . . . , q +1 . Hence,

Inputting this into Definition 7, we get (6), i.e, c ( t +1) r +1 glyph[subsetsqequal] c ( t +1) r .

The 'strictly' can be deduced as follows. The cycle graph on (2 r + 6) nodes equipped with a chord between nodes 1 and r +4 is r -glyph[lscript] WLequivalent to the graph consisting of two ( r +3) -cycles connected by one edge; however, they are not ( r +1) -glyph[lscript] WLequivalent (see, e.g., Figure 6).

F Appendix for Section 5.2

The goal of this subsection is to provide a proof for Theorem 1 and Corollary 1. In fact, we present and prove a more general statement. Specifically, for a graph G and v ∈ V ( G ) , we introduce the node invariant sub( F x , G v ) , defined as the count of subgraph isomorphisms from F to G that are rooted, meaning that x is mapped to v . Let us denote this node invariant as sub( F x , · ) . Our result establishes that c (1) r ( · ) refines sub( C x , · ) for every cycle graph C with at most r nodes. In simpler terms, c (1) r can determine how often node v appears in a cycle C .

Lemma 2. Let r ≥ 1 . For every cycle graph C with at most r +2 nodes and x ∈ V ( C ) , it holds c (1) r ( · ) glyph[subsetsqequal] sub( C x , · ) .

Proof of Lemma 2. Let G be any graph, u, v ∈ V ( G ) , and q = 1 , . . . , r + 2 . Let C be a cycle graph with q nodes. It is important to note that for every x 1 , x 2 ∈ C , we have sub( C x 1 , G v ) = sub( C x 2 , G v ) since every node in C is automorphic to each other. Therefore, we can arbitrarily choose any x ∈ V ( C ) .

We show that

glyph[negationslash]

glyph[negationslash]

The number of injective homomorphisms from the q -long cycles C x to G v , i.e., sub( C a q , G v ) , is equal to the number of paths of length ( q -2) between distinct neighbors of v .

The neighborhood N ( q -2) ( v ) comprises exactly all paths of length ( q -2) between any two distinct neighbors of v . Therefore,

glyph[negationslash]

Thus which implies

glyph[negationslash]

Finally, as HASH in Definition 7 is injective, we get the thesis c (1) q -2 ( u ) = c (1) q -2 ( v ) .

Now, Theorem 1 from the main paper is a simple corollary of Lemma 2.

Theorem 1. For any r ≥ 1 , r -glyph[lscript] WL can subgraph-count all cycles with at most r +2 nodes.

Proof of Theorem 1. Combining Lemma 2 and Lemma 1, we get that c (1) r (as a graph invariant) is stronger than the induced graph invariant A [sub( C x , · )] . Now, consider graphs G,H , and assume without loss of generality that | V ( G ) | = n = | V ( H ) | .

glyph[negationslash]

glyph[negationslash]

If c (1) r ( G ) = c (1) r ( H ) , we have A [sub( C x , · )] ( G ) = A [sub( C x , · )] ( H ) . Hence, by definition of induced graph invariants,

Hence,

which is equivalent to sub( C,G ) = sub( C,H ) .

We proceed to restate Corollary 1 and provide its proof.

Corollary 1. Let k ∈ N . There exists r ∈ N , such that r -glyph[lscript] WL is not less powerful than k -WL. Specifically, r ∈ O ( k 2 ) , with r ≤ k ( k +1) 2 -2 for even k and r ≤ ( k +1) 2 2 -2 for odd k .

glyph[negationslash]

Proof of Corollary 1. Let k > 0 . We need to show that there exist r k ∈ N and a pair of graphs G,H , such that k -WL( G ) = k -WL( H ) and r k -glyph[lscript] WL ( G ) = r k -glyph[lscript] WL ( H ) .

glyph[negationslash]

The hereditary treewidth hdtw( F ) of a graph F is the maximum treewidth of ϕ ( F ) where ϕ is an edge surjective homomorphism. Neuen (2024) has shown that k -WL can subgraph-count a graph F if and only if hdtw( F ) ≤ k . This directly implies that for F with hereditary treewidth larger than k , there exist graphs G F , H F with k -WL( G F ) = k -WL( H F ) and sub( F, G ) = sub( F, H ) .

Since the hereditary tree-width of cycle graphs is not uniformly bounded (Arvind et al., 2020), for every k > 0 there exists a cycle C c k of length c k ∈ N with hereditary treewidth larger than k . Setting F = C c k concludes the existence proof with r k = c k -2 .

To see that r k ∈ O ( k 2 ) , note that the complete graph K n on n vertices has treewidth n -1 and exactly ( n 2 ) edges. For odd n , K n is Eulerian, i.e., there exists an edge surjective homomorphism from a cycle to K n which uses each edge exactly once, i.e., from C ( n 2 ) . If n is odd, the minimum T -join which makes K n Eulerian contains exactly n 2 edges (see, e.g., Korte et al., 2018). As a result, there exists an edge surjective homomorphism from C ( n 2 ) to K n if n is odd, and an edge surjective homomorphism from C ( n 2 ) + n 2 to K n if n is even. This implies that hdtw( C c k ) > k for c k = ( k +1 2 ) + glyph[ceilingleft] k +1 2 glyph[ceilingright] . Hence, r k := c k -2 ∈ O ( k 2 ) .

G Appendix on Homomorphism Counting and Section 5.3

In this section, we provide background information and all proofs related to homomorphism counts. We begin by introducing additional definitions and notation.

Definition 12 (Induced Subgraph) . Let G = ( V ( G ) , E ( G )) and S ⊂ V ( G ) . The induced subgraph G [ S ] of G over S is defined as the graph G [ S ] with vertices V ( G [ S ]) = S and edges E ( G [ S ]) = {{ u, v } ∈ E ( G ) | u, v ∈ S } .

The following definition indicates whether a pair of nodes is connected by an edge or not.

Definition 13 (Atomic Type) . For a tuple of nodes ( u 1 , u 2 ) , the atomic type atp G (( u 1 , u 2 )) of G over ( u 1 , u 2 ) indicates where { u 1 , u 2 } ∈ E ( G ) , i.e., atp G (( u 1 , u 2 )) = 1 if { u 1 , u 2 } ∈ E ( G ) and zero otherwise.

We continue by defining tree graphs , an important class of graphs closely related to the 1 -WL test.

Definition 14 (Tree Graph) . A graph T is called a tree (graph) if it is connected and does not contain cycles. A rooted tree T s = ( V ( T s ) , E ( T s )) is a tree in which a node s ∈ V ( T s ) is singled out. This node is called the root of the tree. For each vertex t ∈ V ( T s ) , we define its depth dep T s ( t ) := dist T s ( t, s ) , where dist denotes the shortest path distance between t and s . The depth of T s is then the maximum depth among all nodes t ∈ V ( T ) . We define Desc T s ( t ) the set of descendants of t , i.e., Desc T s ( t ) = { t ′ ∈ T s | dep T s ( t ′ ) = dep T s ( t ) + dist T s ( t, t ′ ) } . For each t ∈ V ( T s ) \ { s } , we define the parent node pa T s ( t ) of t as the unique node t ′ ∈ N ( t ) such that dep T s ( t ) = dep T s ( t ′ ) + 1 . We define the subtree of T s rooted at node t by T s [ t ] , i.e., T s [ t ] := T s [Desc T s ( t )] .

The remainder of this section is structured as follows. Appendix G.1 introduces the basics of tree decompositions. In Appendix G.2, we present the class of fan cactus graphs, encompassing all cactus graphs, and develop its canonical tree decomposition. We present an alternative formulation of r -glyph[lscript] WL in Appendix G.3 for technical reasons. Subsequently, in Appendix G.4, we define the unfolding tree of r -glyph[lscript] WL and illustrate its relation to the r -glyph[lscript] WL colors and canonical tree decompositions of fan cactus graphs. Finally, in Appendix G.4.1, we establish the groundwork to conclude the proof of Theorem 2, a simple corollary of all the results in this section.

G.1 Tree Decomposition Preliminaries

Along with its notation, this subsection closely adheres to the conventions outlined by B. Zhang et al. (2024, Section C). We start with a formal definition of a tree decomposition for a graph.

Definition 15 (Tree Decomposition) . Let G = ( V ( G ) , E ( G )) . A tree decomposition of G is a tree T = ( V ( T ) , E ( T )) together with a function β T : V ( T ) → 2 V ( G ) satisfying the following conditions:

Each tree node t ∈ V ( T ) is mapped to a non-empty subset of vertices β T ( t ) ⊂ V ( G ) in G , referred to as a bag. We say tree node t contains vertex u if u ∈ β T ( t ) .

For each edge { u, v } ∈ E ( G ) , there exists at least one tree node t ∈ V ( T ) such that { u, v } ⊂ β T ( t ) .

For each vertex u ∈ V ( G ) , all tree nodes t containing u form a connected subtree, i.e., the induced subgraph T [ { t ∈ V ( T ) : u ∈ β T ( t ) } ] is connected.

If ( T, β T ) is a tree decomposition of G , we refer to the tuple ( G,T,β T ) as a tree-decomposed graph. The width of the tree decomposition T of G is defined as

If T has root s , we also denote it as ( G,T s , β T ) .

Definition 16 (Treewidth) . The treewidth of a graph G , denoted as tw( G ) , is the minimum positive integer k such that there exists a tree decomposition of width k .

G.2 Cactus Graphs and their Canonical Tree Decomposition

Cactus graphs play a crucial role in graph theory due to their unique structural properties. Before delving into their canonical tree decomposition, we define the concept of a rooted r -cactus graph. To simplify the notation, we assume that graphs in this section are connected and that V ( G ) ⊆ N for all graphs G . Further, we assume that r ∈ N throughout this section.

Definition 17 (Rooted r -Cactus Graph) . A cactus graph is a graph where every edge lies on at most one simple cycle. An r -cactus graph is a cactus graph where every simple cycle has at most r vertices. A rooted cactus (graph) G s is a cactus graph G with a root node s ∈ V ( G ) .

Now, we introduce the notion of a fan cactus, which is an essential concept for our subsequent discussions on the canonical tree decomposition of these graphs.

Definition 18 (Fan Cactus) . Let G s be a rooted r -cactus. For every simple cycle C in G let v C be the unique vertex in C that is closest to s . We obtain a fan r -cactus F s from a rooted r -cactus G s by adding an arbitrary number of edges { v C , w } to any cycle C with w ∈ V ( C ) . Let M r +2 be the class of graphs F with s ∈ V ( F ) such that F s is a fan r -cactus.

Remark 1. Every r -cactus is a fan r -cactus. Every fan r -cactus is outerplanar. Every outerplanar graph has tree-width at most 2.

Figure 10 shows an example of a fan 6 -cactus. As fan cacti are outerplanar, graph isomorphism can be decided in linear time. One way to do so is to use a canonicalization function, that maps graphs to a unique representative of each set of isomorphic graphs. We denote the set of all such representatives as M r +2 / ∼ = ⊆ M r +2 .

Lemma 3 (Colbourn et al. (1981)) . There exists a function canon : M r +2 →M r +2 / ∼ = such that

G ∼ = canon( G )

G ∼ = H ⇐⇒ V (canon( G )) = V (canon( H )) ∧ E (canon( G )) = E (canon( H )) .

Moreover, given G ∈ M r +2 , canon( G ) can be computed in linear time.

For each G ∈ M r +2 we denote the isomorphism between G and canon( G ) as canon G . Colbourn et al. (1981) describe a bottom-up algorithm to obtain canon( G ) of a fan r -cactus G . We will implicitly use the results of this canonicalization to define a canonical tree decomposition of fan r -cacti. The crucial point in the algorithm is a simple way to decide which 'direction' to use when dealing with a cycle in the underlying cactus graph. Each undirected, rooted cycle allows for a choice between two directions when building a tree decomposition. We will first define a tree decomposition for a rooted cycle which depends on a choice of direction and then define a canonical direction of cycles in G based on canon G .

Definition 19 (Tree Decomposition of Rooted Cycle) . Let C n be a cycle graph on n nodes v 0 to v n -1 . The path T on nodes w 1 , . . . , w 2 n -3 with bags β ( w 1 ) = { v 0 , v 1 } and for i ≥ 2

is a tree decomposition of C n . We say that v 0 and v 1 correspond to w 1 and v i corresponds to w 2 i -1 for i ≥ 2 .

A depiction of the tree decomposition T 0 (right) of C 6 (left) is shown below. Note that we have to choose one of two possible orientations of the undirected cycle to construct T 0 . We address this choice in the next definition.

0

1

2

3

4

5

{

0

,

1

}

{

0

,

1

,

2

}

{

0

,

2

}

{

5

,

0

}

{

0

,

4

,

5

}

{

0

,

2

,

3

}

{

0

,

3

}

{

0

,

3

,

4

}

{

0

,

4

}

The choice of 'smaller' does not matter as long as it defines a total order. One can, for example, use a lexicographical order. Based on Definition 20, we now define a canonical tree decomposition of fan cactus graphs, in the sense that any two isomorphic fan cactus graphs will have isomorphic tree decompositions.

Definition 21 (Canonical Tree Decomposition of Fan r -Cactus Graphs) . Let F s be a fan r -cactus and G s its underlying r -cactus. We define the canonical tree decomposition T ˜ s of F rooted at ˜ s as follows

Node Gadget: For all v ∈ V ( F ) add a node t to V ( T ) and set β ( t ) = { v } . We choose ˜ s such that β (˜ s ) = { s } .

Tree Edge Gadget: For all { v, w } ∈ E ( G ) that are not on a simple cycle in F add a node x { v,w } to V ( T ) with β ( x { v,w } ) = { v, w } and edges { v, x { v,w } } and { w,x { v,w } } to E ( T )

Cycle Gadget: For each (undirected) cycle C in the underlying cactus G , add a copy of its canonical tree decomposition T v C C of C rooted at v C to T and connect nodes in it to the corresponding node gadgets.

See Figure 10 for an illustration. For the discussions in subsequent sections, we introduce the following definition.

12

7

8

9

10

11

C

3

13

4

1

2

3

C

1

6

5

C

2

v

C

1

v

C

2

v

C

3

{

7

,

12

}

{

12

}

{

7

}

{

7

,

8

}

{

8

}

{

7

,

8

,

9

}

{

7

,

9

}

{

9

}

{

7

,

9

,

10

}

{

7

,

10

}

{

10

}

{

7

,

10

,

11

}

{

7

,

11

}

{

11

}

{

7

,

11

,

12

}

{

13

}

{

12

,

13

}

{

4

}

{

1

}

{

2

}

{

1

,

4

,

3

}

{

1

,

4

}

{

1

,

2

}

{

1

,

3

,

2

}

{

1

,

3

}

{

3

}

{

4

,

7

}

{

2

,

5

}

{

2

,

6

}

{

2

,

5

,

6

}

{

6

}

{

5

}

Definition 22 (Depth in the Canonical Tree Decomposition of Fan r -Cactus Graphs) . Let ( F, T s ) be a canonical tree decomposition of a fan r -cactus. We define the depth dep( t ) of t ∈ V ( T ) recursively as follows:

The depth of ( F, T s ) is then the maximum depth of any node t ∈ V ( T s ) .

Intuitively, for a given fan r -cactus graph F with its canonical tree decomposition T s , Definition 22 captures the depth (see Definition 15) of the tree T s , if cycles in F and the corresponding bags in T s were replaced by single edges.

Lemma 4. Let F s be a fan r -cactus. The canonical tree decomposition ( F, T ˜ s ) is a tree decomposition of F s .

Proof. We need to show that (1) T is a tree, (2) for every edge e ∈ E ( F ) there exists some bag β ( v ) with e ⊆ β ( v ) , and (3) T [ { t ∈ V ( T ) : u ∈ β ( t ) } ] is connected.

To see that T does not contain cycles, note that we replace each cycle with its cycle gadget, which is a path. It is easy to see that T is connected as G is connected.

For (2), note that tree edges e ∈ V ( F ) have their own gadget node in x e with β ( x e ) = e . Similarly, each edge e on a simple cycle C of the underlying cactus F of G is contained in some bag within the cycle gadget of C . Finally, for diagonal edges { v C , w } ∈ E ( F ) \ E ( G ) , v C is contained in any bag of the cycle gadget of C . As a result, { v C , v } is contained in the bag of the corresponding node of v .

glyph[negationslash]

For (3), note that in the tree edge gadget, nodes t with v ∈ β ( t ) are connected to the node gadget of v . In the cycle gadget, any node t with w ∈ β ( t ) is either directly or via its neighbor connected to the node gadget of w if w = v C . As the cycle gadget is connected and v C is in any bag of the gadget, a path to the node gadget of v C exists where every bag contains v C .

We conclude this subsection with a formal definition of when two canonical tree decompositions are isomorphic and prove the main result of this section, i.e., that canonical tree decompositions of fan r -cacti G s , H t are isomorphic whenever G s , H t are isomorphic.

Definition 23 (Isomorphism between canonical tree-decomposed graphs) . Given two canonical tree-decomposed graphs ( G,T s ) and ( ˜ G, ˜ T ˜ s ) , a pair of mappings ( ρ, τ ) is called an isomorphism between ( G,T s ) and ( ˜ G, ˜ T ˜ s ) , denoted by ( G,T s ) ∼ = ( ˜ G, ˜ T ˜ s ) , if the following holds:

ρ is an isomorphism between G and ˜ G ,

τ is an isomorphism between T s and ˜ T ˜ s ,

For any t ∈ T s , we have ρ ( β T ( t )) = β ˜ T ( τ ( t )) .

Lemma 5. Let G s ∼ = H t be rooted r -fan cacti. Then ( G s , T [ G s ]) ∼ = ( H t , T [ H t ]) .

Proof. Let ρ be a root preserving isomorphism between G s and H t . According to Lemma 3 then there exist isomorphisms canon G and canon H with ρ = canon G ◦ canon -1 H . We construct τ : V ( T [ G s ]) → V ( T [ H t ]) from ρ as follows: It is easy to see that ρ induces a bijective mapping τ between the nodes of T [ G s ] and T [ H t ] that assigns each gadget node v ∈ V ( T [ G s ]) to the unique gadget node τ ( v ) ∈ V ( T [ H t ]) with β ( τ ( v )) = ρ ( β ( v )) . By the same argument, τ maps the root of T [ G s ] to the root of T [ H t ] .

Now assume by contradiction that τ is not an isomorphism between T [ G s ] and T [ H t ] . That means that w.l.o.g. there exists { v, w } ∈ E ( T [ G s ]) with { τ ( v ) , τ ( w ) } / ∈ E ( T [ H t ]) . However, for the bags of v, w it holds canon G ( β ( v )) = canon H ( β ( τ ( v ))) and canon G ( β ( w )) = canon H ( β ( τ ( w ))) . This cannot happen, as the addition of edges in Definition 21 depends only on the images of the bags under canon .

G.3 Alternative r -glyph[lscript] WL

In this subsection, we define slightly modified versions of 1 -WL and r -glyph[lscript] WLthat we consider in the subsequent sections.

Definition 24 (Alternative 1 -WL and r -glyph[lscript] WL) . The alternative 1 -WL test refines vertices' colors as

Equivalently, we define the alternative r -glyph[lscript] WL via

It is well-known that both the alternative 1 -WL test and the standard 1 -WL test are equally powerful (in terms of their expressive power). Similarly, the alternative k -WL test and the standard k -WL test are equally powerful. For the sake of simplicity in the subsequent discussion, we will refer to both the alternative 1 -WL and k -WL tests simply as the 1 -WL and k -WL tests, respectively. Although this practice may seem like a slight abuse of notation, it is justified because the expressive power of these tests remains unaffected.

Finally, as noted in Section 6, we alter the r -glyph[lscript] WLalgorithm slightly by incorporating atomic types into the path representation. Hence, we update node features according to

where ( atp( v, p ) , c ( t ) r ( p ) ) := (( atp( v, p 1 ) , c ( t ) r ( p 1 ) ) , . . . , ( atp( v, p q +1 ) , c ( t ) r ( p q +1 ) )) for p = { p i } q +1 i =1 ∈ N q ( v ) . The definition of atomic types atp( · , · ) is given in Definition 13. Clearly this version of r -glyph[lscript] WLis more powerful than the standard version, according to Definition 3. However, it is unclear whether r -glyph[lscript] WLwith atomic types is strictly more powerful than standard r -glyph[lscript] WL.

G.4 The Unfolding Tree of r -glyph[lscript] WL

Given Definition 20, we assume, for the remainder of this appendix, that every fan cactus graph has a unique labeling function, allowing us to select a unique orientation for every cycle in the graph. We call this orientation the canonical orientation . If not otherwise mentioned, we consider the canonical orientation of cycle graphs.

We begin this section by introducing a critical concept known as bag isomorphism (Dell et al., 2018; B. Zhang et al., 2024).

Definition 25 (Bag Isomorphism) . Let ( F, T s ) be a tree-decomposed graph, and G be a graph. A homomorphism f from F to G is called a bag isomorphism from ( F, T s ) to G if, for all t ∈ V ( T s ) , the mapping f is an isomorphism from F [ β T s ( t )] to G [ f ( β T s ( t ))] . We denote by BIso(( F, T s ) , G ) the set of all bag isomorphisms from ( F, T s ) to G , and set bIso(( F, T s ) , G ) = | BIso(( F, T s ) , G ) | .

Moving forward, we proceed to define r -glyph[lscript] WLunfolding trees, which intuitively construct, for a given graph and a node in the graph, the computational graph of the r -glyph[lscript] WLalgorithm and its canonical tree decomposition.

Definition 26 (Unfolding tree of r -glyph[lscript] WL) . Given a graph G , vertex v ∈ V ( G ) and a non-negative D ∈ Z , the depth2 D r -glyph[lscript] WL unfolding tree of a graph G ∈ M r +2 at node v , denoted as ( F ( D ) ( v ) , T ( D ) ( v ) ) , is a tree-decomposition ( F, T s ) constructed in the following way:

Initialization : V ( F ) = { v } without edges, and T s only has a root node s with β T s ( s ) = { v } . Define a mapping V ( F ) → V ( G ) as π ( v ) = v .

Introduce nodes : For each leaf node t with | β T s ( t ) | = 1 in T s , do the following procedure:

Let β T ( t ) = { g } . For each w ∈ V ( G ) do the following:

Add a fresh child t w to t in T s .

Add a fresh vertex f to F and extend π with [ f ↦→ w ] .

Define the bag of t w by β T s ( t w ) = β T s ( t ) ∪ { f } .

Add an edge between f and g if { π ( f ) , π ( g ) } ∈ E ( G ) .

3. Introduce paths : For each q = 1 , . . . , r , do: For each length q path with canonical orientation p = { p i } q +1 i =1 ∈ N q ( g ) , do the following:

Add a fresh path t p = { t { p 1 } , t { p 1 ,p 2 } , t { p 2 } , . . . , t { p q } , t { p q ,p q +1 } , t { p q +1 } } to t in T s .

Add q + 1 fresh vertices f 1 , . . . , f q +1 to F and extend π with [ f i ↦→ p i ] for every i = 1 , . . . , q +1 .

7

1

2

3

8

4

5

6

G

7'

1

2'

3'

8'

2'

3'

4'

5'

6'

F

(2)

(1)

1

1, 2'

1, 3'

1, 4'

2'

1,2'

1,2',8'

1,8'

1,8',3'

1,3'

2'

8'

3'

3'

4'

T

(2)

(1)

1, 8'

8'

. . .

For i = 1 , . . . , q , let the bag of t { p i ,p i +1 } be defined via β T s ( t { p i ,p i +1 } ) = β T s ( t ) ∪ { f i , f i +1 } .

d) For

i

= 1

, . . . , q

+1

, let the bag of

t

{

p

i

}

be defined via

For i = 1 , . . . , q +1 , add edges between f i and f i +1 .

Add edges between g and f 1 , . . . , f q +1 such that for every i = 1 , . . . , q , we have F [ β T s ( t { p i ,p i +1 } )] = F [ { f i , f i +1 , g } ] ∼ = G [ π ( β T s ( t { p i ,p i +1 } ))] , i.e., add edges between g and f i if and only if there is an edge between { π ( g ) , π ( f i ) } ∈ E ( G ) .

Forget nodes : If t is a leaf node of T s with | β T s ( t ) | = 2 and parent t ′ with | β T s ( t ) | = 1 , do the following:

Add a fresh child t 1 of t to T s .

Let f be that vertex introduced at t , that is, we have β T s ( t ) \ β T s ( t ′ ) = { f } .

We set β T s ( t 1 ) = { f } .

Forget paths : If t p = { t { p 1 } , t { p 1 ,p 2 } , t { p 2 } , . . . , t { p q } , t { p q ,p q +1 } , t { p q +1 } } is a leaf path of T s with parent t ′ of t { p 1 } , do the following:

For i = 2 , . . . , q +1 , add a fresh child ˜ t { p i } to t { p i } .

Let f 2 , . . . , f q +1 be the vertices introduced at t p , that is, we have β T s ( t { p i } ) \ β T s ( t ′ ) = { f i } .

For i = 2 , . . . , q +1 , we set β T s ( ˜ t { p i } ) = { f i } .

We refer to Figure 11 for the depth-2 2 -glyph[lscript] WLunfolding tree of an example graph.

Theorem 4. Let r ≥ 1 . For any graph G , any vertex v ∈ V ( G ) , and any non-negative integer D , let ( F ( D ) ( v ) , T ( D ) ( v ) ) be its depth2 D r -glyph[lscript] WL unfolding tree at node v . Then, F ( D ) ( v ) is a fan r -cactus graph, and T ( D ) ( v ) is an r -canonical tree decomposition of F ( D ) ( v ) . Moreover, the constructed mapping π in Definition 26 is a bag isomorphism from ( F ( D ) ( v ) , T ( D ) ( v ) ) to the graph G .

Proof. Clear by the definition of the depth2 D unfolding tree of r -glyph[lscript] WL.

We present the following results that fully characterize when two graphs and their respective nodes have the same r -glyph[lscript] WLcolors in terms of their r -glyph[lscript] WLunfolding trees.

Theorem 5. Let r ∈ N . For any two connected graphs G,H , any vertices v ∈ V ( G ) and x ∈ V ( H ) and any D ∈ N , it holds: c ( D ) r ( v ) = c ( D ) r ( x ) if and only if there exists a root preserving isomorphism between ( F ( D ) ( v ) , T ( D ) ( v ) ) and ( F ( D ) ( x ) , T ( D ) ( x ) ) .

Proof of ' = ⇒ '. The proof is based on induction over D . When D = 0 , the theorem obviously holds. Assume that the theorem holds for D ≤ d , and consider D = d + 1 . We show that if c ( d +1) r ( v ) = c ( d +1) r ( x ) , then there exists an isomorphism ( ρ, τ ) from ( F ( d +1) ( v ) , T ( d +1) ( v ) ) to ( F ( d +1) ( x ) , T ( d +1) ( x ) ) such that ρ ( v ) = x .

β

T

s

(

t

{

p

i

}

) =

β

T

s

(

t

)

∪ {

f

i

}

.

i.e., | V ( G ) | = | V ( H ) | , and we set n = | V ( G ) | . We enumerate V ( G ) = { w 1 , . . . , w n } and V ( H ) = { z 1 , . . . , z n } such that

for all i = 1 , . . . , n . Also, again since c ( d +1) r ( v ) = c ( d +1) r ( x ) , we have for every q = 1 , . . . , r ,

In particular, |N q ( v ) | = |N q ( x ) | and we can enumerate the paths in N q ( v ) and N q ( x ) such that

for every l = 1 , . . . , |N q ( v ) | .

Now, by definition of the r -glyph[lscript] WLunfolding tree, the graph F ( d +1) ( v ) is isomorphic to the union of: a) all graphs F ( d ) ( w i ) for i = 1 , . . . , n , plus additional edges between w i to v if { w i , v } ∈ E ( G ) , and b) all graphs F ( d ) ( p q l,k ) for q = 1 , . . . , r , l = 1 , . . . , |N q ( v ) | for any path p q l = { p q l, 1 , . . . , p q l,q +1 } ∈ N q ( v ) . And adding, for k = 1 , . . . q , edges between p q l,k and p q l,k +1 . And adding, for k = 1 , . . . q +1 , edges p q l,k and v if there is one in G , i.e., if { p q l,k , v } ∈ E ( G ) .

Similarly, the tree T ( d +1) ( v ) is isomorphic to the disjoint union of all trees T ( d ) ( w i ) (for i = 1 , . . . , n ) and T ( d ) ( p q l,k ) (for q = 1 , . . . , r , k = 1 , . . . , q +1 and l = 1 , . . . , |N ( v ) | ). Plus adding the following fresh tree nodes and edges: a root node s , nodes t w i (for i = 1 , . . . , n ) that connects to s and the root of T ( d ) ( w i ) . And for q = 1 , . . . , r , l = 1 , . . . , |N q ( v ) | for any path p q l ∈ N q ( v ) a path of length 2 q ,

By (8) and induction, there exist isomorphisms ( ρ i , τ i ) from ( F ( d ) ( w i ) , T ( d ) ( w i )) to ( F ( d ) ( z i ) , T ( d ) ( z i )) such that ρ i ( w i ) = z i for i = 1 , . . . , n . By (9) and induction, there exist isomorphisms ( ρ q l,k , τ q l,k ) from ( F ( d ) ( u q l,k ) , T ( d ) ( u q l,k )) to ( F ( d ) ( y q l,k ) , T ( d ) ( y q l,k )) such ρ i ( u q l,k ) = y q l,k for q = 1 , . . . , r , k = 1 , . . . , q +1 and l = 1 , . . . , |N q ( v ) | .

We now construct ρ by merging all ρ i and ρ q l,k , and construct τ by merging all τ i and τ q l,k . We finally specify an appropriate mapping for the tree root, its direct children and the paths attached to the tree root. Then, it is easy to see that ( ρ, τ ) is well-defined and an isomorphism between ( F ( d +1) ( v ) , T ( d +1) ( v ) ) and ( F ( d +1) ( x ) , T ( d +1) ( x ) ) such that ρ ( v ) = x .

Proof of ' ⇐ = '. We now prove the other direction, again via induction over D . When D = 0 the assertion obviously holds. Assume that the assertion holds for D ≤ d . Now, assume that there exists an isomorphism ( ρ, τ ) between ( F ( d +1) ( v ) , T ( d +1) ( v ) ) and ( F ( d +1) ( x ) , T ( d +1) ( x ) ) such that ρ ( v ) = x . We show that c ( d +1) r ( v ) = c ( d +1) r ( x ) .

We begin our proof by establishing the equality of two multisets: {{ ( c ( d ) r ( w ) , atp( v, w )) | w ∈ V ( G ) }} and {{ ( c ( d ) r ( z ) , atp( v, z )) | z ∈ V ( H ) }} . The proof of this equivalence closely mirrors the argument presented in the proof of B. Zhang et al. (2024, Lemma C.14). Since τ is an isomorphism it maps all tree nodes T ( d +1) ( v ) of depth 2 with 1 element in their bag to the corresponding tree nodes in T ( d +1) ( x ) . Let s 1 , . . . , s n and t 1 , . . . , t n be the nodes in T ( d +1) ( v ) and T ( d +1) ( x ) of depth 2 with 1 element in their bag, respectively. For i = 1 , . . . , n , let s ′ i and t ′ i the parents of s i and t i , respectively. We then choose the order such that the following holds for all i = 1 , . . . , n

Let β T ( d +1) ( v ) ( s ′ i ) = { v, ˜ w i } and β T ( d +1) ( x ) ( t ′ i ) = { x, ˜ z i } . Then, ρ ( v ) = x and ρ ( ˜ w i ) = ˜ z i and thus, per assumption, { v, ˜ w i } ∈ E ( F ( d +1) ( v )) if and only if { x, ˜ z i } ∈ E ( F ( d +1) ( x )) .

τ is an isomorphism from the subtree rooted at s i in T ( d +1) ( v ) , i.e., T ( d +1) ( v )[ s i ] , the subtree rooted at t i in T ( d +1) ( x ) , i.e., T ( d +1) ( v )[ t i ] .

For all s ∈ Desc T ( d +1) ( v ) ( s i ) , it holds ρ ( β T ( d +1) ( v ) ( s )) = β T ( d +1) ( x ) ( τ ( s )) .

By the definition of the unfolding tree, ρ is an isomorphism from the induced subgraph F ( d +1) ( v ) [ T ( d +1) ( v )[ s i ] ] and the induced subgraph F ( d +1) ( x ) [ T ( d +1) ( x )[ t i ] ] .

By the last three items, we get that ( F ( d +1) ( v ) [ T ( d +1) ( v )[ s i ] ] , T ( d +1) ( v )[ s i ] ) and ( F ( d +1) ( x ) [ T ( d +1) ( x )[ t i ] ] , T ( d +1) ( x )[ t i ] ) are isomorphic. By definition of the r -glyph[lscript] WL unfolding tree, ( F ( d +1) ( v ) [ T ( d +1) ( v )[ s i ] ] , T ( d +1) ( v )[ s i ] ) is isomorphic to ( F ( d ) ( w i ) , T ( d ) ( w i )) for some w i ∈ V ( G ) that satisfies { ˜ w i , v } ∈ E F ( d +1) ( v ) if and only if { w i , v } ∈ E ( G ) . And ( F ( d +1) ( x ) [ T ( d +1) ( x )[ t i ] ] , T ( d +1) ( x )[ t i ] ) is isomorphic to ( F ( d ) ( z i ) , F ( d ) ( z i )) for some z i ∈ V ( H ) that satisfies { ˜ z i , x } ∈ E ( F ( d +1) ( x )) if and only if { z i , x } ∈ E ( G ) . Hence, by induction, we have atp ( v, w i ) = atp ( x, z i ) and c ( d ) r ( w i ) = c ( d ) r ( z i ) for all i = 1 , . . . , n .

It remains to show that, for every q = 1 , . . . , r ,

Fix q = 1 , . . . , r . Since τ is an isomorphism it maps all paths of length q in T ( d +1) ( v ) connected to v to paths of length q in T ( d +1) ( x ) connected to x .

By construction of the r -glyph[lscript] WLunfolding tree and since ( ρ, τ ) is an isomorphism, it holds |N q ( v ) | = |N q ( x ) | . Denote the relevant bags at depth 2 by s ′ q l,k for l = 1 , . . . , |N q ( v ) | and k = 1 , . . . , q +1 . Denote by s q l,k and t q l,k the parents of s ′ q l,k and t ′ q l,k , respectively. We then choose the order l = 1 , . . . , |N q ( v ) | and k = 1 , . . . , q +1 such that it holds

Let β T ( d +1) ( s ′ q l,k ) = { v, ˜ w q l,k } and β T ( d +1) ( t ′ q l,k ) = { x, ˜ z q l,k } . Then, ρ ( ˜ w q l,k ) = ˜ z q l,k and thus, per assumption, { v, ˜ w q l,k } ∈ E ( F ( d +1) ( v )) if and only if { x, ˜ z q l,k } ∈ E ( F ( d +1) ( x ))

τ is an isomorphism from the subtree rooted at s q l,k in T ( d +1) ( v ) , i.e., T ( d +1) ( v )[ s q l,k ] , to the subtree rooted at t q l,k in T ( d +1) ( x ) , i.e., T ( d +1) ( x )[ t q l,k ] .

For all s ∈ Desc T ( d +1) ( v ) ( s q l,k ) , it holds ρ ( β T ( d +1) ( v ) ( s )) = β T ( d +1) ( x ) ( τ ( s )) .

By the definition of the unfolding tree, ρ is an isomorphism from the induced subgraph F ( d +1) ( v ) [ T ( d +1) ( v )[ s q l,k ] ] and the induced subgraph F ( d +1) ( x ) [ T ( d +1) ( x )[ t q l,k ] ] .

By the last three items, we get that ( F ( d +1) ( v ) [ T ( d +1) ( v )[ s q l,k ] ] , T ( d +1) ( v )[ s q l,k ] ) and ( F ( d +1) ( x ) [ T ( d +1) ( x )[ t q l,k ] ] , T ( d +1) ( x )[ t q l,k ] ) are isomorphic. By definition of the r -glyph[lscript] WLunfolding tree, ( F ( d +1) ( v ) [ T ( d +1) ( v )[ s q l,k ] ] , T ( d +1) ( v )[ s q l,k ] ) is isomorphic to ( F ( d ) ( w q l,k ) , T ( d ) ( w q l,k )) for w q l,k ∈ V ( G ) that satisfies { ˜ w q l,k , v } ∈ E F ( d +1) ( v ) if and only if { w q l,k , v } ∈ E ( G ) .

We introduce the following definition that provides a similarity measure between a graph and a tree-decomposed graph.

Definition 27. Given a graph G and a tree-decomposed graph ( F, T s ) , define

where ( F ( D ) ( v ) , T ( D ) ( v )) is the depth2 D r -glyph[lscript] WL unfolding tree of G at v .

The counting function cnt (( F, T s ) , G ) serves as a key metric, allowing us to draw connections between r -glyph[lscript] WLcolorings of two different graphs.

Corollary 3. Let r ∈ N . Let G and H be two graphs. Then, c r ( G ) = c r ( H ) if and only if cnt (( F, T s ) , G ) = cnt (( F, T s ) , H ) holds for all graphs ( F, T s ) ∈ M r +2 .

Proof of ' = ⇒ '. Let c r ( G ) = c r ( H ) , i.e.,

Assume, by contradiction, that there exists a tuple ( F, T s ) ∈ M r +2 such that cnt (( F, T s ) , G ) = cnt (( F, T s ) , H ) . Let c 1 , . . . , c k be the final colors of nodes in V ( G ) and V ( H ) . Then, define for i = 1 , . . . , k glyph[negationslash]

We have and

glyph[negationslash]

Since cnt (( F, T s ) , G ) = cnt (( F, T s ) , H ) , there exist an index i = 1 , . . . , k such that glyph[negationslash]

Furthermore, there exists i n ∈ N such that there are exactly n nodes v 1 , . . . , v n and x 1 , . . . , x n such that

Hence, as c r refines c ( D ) r , we have

By (10), there exists some D ∈ N such that (without loss of generality) ( F ( D ) ( v 1 ) , T ( D ) ( v 1 )) ∼ = ( F, T s ) . Then, by Theorem 5, we have

There does not exist any other node w with c r ( w ) = c 1 such that the corresponding unfolding tree is isomorphic to ( F ( D ) ( v 1 ) , T ( D ) ( v 1 )) . Hence, cnt(( F, T s ) , G [ c i ]) = cnt(( F, T s ) , H [ c i ]) , which is a contradiction.

Proof of ' ⇐ = '. Suppose that cnt(( F, T s ) , G ) = cnt(( F, T s ) , H ) for all ( F, T s ) ∈ M r +2 . Let c 1 , . . . , c k G with multiplicities m 1 , . . . , m k G and ˜ c 1 , . . . , ˜ c k H with multiplicities ˜ m 1 , . . . , ˜ m k G be the final colors of r -glyph[lscript] WL applied to G and H , respectively. Consider some v ∈ V ( G ) such that c r ( v ) = c 1 . Let D be sufficiently large (any D after convergence of r -glyph[lscript] WL), then cnt(( F ( D ) ( v ) , T ( D ) ( v )) , G ) = cnt(( F ( D ) ( v ) , T ( D ) ( v )) , H ) since ( F ( D ) ( v ) , T ( D ) ( v )) ∈ M r +2 . Hence, without loss of generality, c 1 = ˜ c 1 and m 1 = ˜ m 1 . Repeating this argument for all colors finishes the proof.

G.4.1 Proof of Theorem 2

In this section, we employ techniques adapted from the works of Dell et al. (2018) and B. Zhang et al. (2024) to derive a proof for Theorem 2 from the established result in Corollary 3.

Definition 28 (Definition 20 in (Dell et al., 2018)) . Let ( F, T t ) and ( ˜ F, ˜ T s ) be two tree-decomposed graphs. A pair of mappings ( ρ, τ ) is said to be a bag isomorphism homomorphism from ( F, T t ) to ( ˜ F, ˜ T s ) if it satisfies the following conditions

ρ is a homomorphism from F to ˜ F .

τ is a homomorphism from T t to ˜ T s .

τ is depth-surjective, i.e., the image of T t under τ contains vertices at every depth present in ˜ T s .

For all t ′ ∈ T t , we have dep T t ( t ′ ) = dep ˜ T s ( τ ( t ′ )) and F [ β T t ( t ′ )] ∼ = ˜ F [ β ˜ T s ( τ ( t ′ ))] .

For all t ′ ∈ T t , the set equality ρ ( β T t ( t ′ )) = β ˜ T s ( τ ( t ′ )) holds.

The depth of T t and ˜ T s is equal.

We denote the set of bag isomorphism homomorphisms from ( F, T t ) to ( ˜ F, ˜ T s ) by BIsoHom ( ( F, T t ) , ( ˜ F, ˜ T s ) ) and set bIsoHom ( ( F, T t ) , ( ˜ F, ˜ T s ) ) = | BIsoHom ( ( F, T t ) , ( ˜ F, ˜ T s ) ) | .

We continue with the following lemma that shows a linear relation between the number of bag isomorphisms and the output of the counting function in Definition 27.

Lemma 6. Let r ∈ N . For any tree-decomposed graph ( F, T s ) ∈ M r +2 and any graph G , it holds

Proof. Let ( F, T s ) be a tree-decomposed graph such that T s has depth 2 D . The sum is over all isomorphism types ( ˜ F, ˜ T t ) of tree-decomposed graphs. This sum is finite and thus well-defined as bIsoHom ( ( F, T s ) , ( ˜ F, ˜ T t )) = 0 holds if ˜ T t has depth unequal to 2 D or nodes with at least ( r +1) · ( | V ( G ) | -1) children.

Assume that for the root bag of ( F, T s ) it holds β T s ( s ) = { v } . Let x ∈ V ( G ) be any vertex in G , and denote by ( F ( D ) ( x ) , T ( D ) ( x )) the depth2 D r -glyph[lscript] WL-unfolding tree at node x . Define the following two sets,

We prove that | S 1 ( x ) | = | S 2 ( x ) | for every x ∈ V ( G ) , which is equivalent to (11). For this, we show for any bag isomorphism h from ( F, T s ) to G with h ( v ) = x , there exists a unique bag isomorphism homomorphism σ from ( F, T s ) to ( F ( D ) ( x ) , T ( D ) ( x )) with σ ( v ) = x such that h = π ◦ σ , where π is the bag isomorphism from ( F ( D ) ( x ) , T ( D ) ( x )) to G , defined in Definition 26 and Theorem 4, respectively. To visualize this proof idea, see Figure 12.

First, define ρ ( v ) := x . Let v 1 , . . . , v n ∈ V ( F ) be nodes that correspond to bags in T s of depth 2 with one element inside the bag and their parents having two elements in their bag, i.e., { v i } are the corresponding bags. Similarly, set x 1 , . . . , x m ∈ V ( F ( D ) ( x )) nodes that correspond to bags of depth 2 in T ( D ) ( x ) , with one element inside the bag and their parents having two elements in their bag. Since h is a bag isomorphism and π as well, for every i = 1 , . . . , n there exists a j i such that h ( v i ) = ˜ x j i = π ( x j i ) , where ˜ x j i ∈ V ( G ) and x j i ∈ V ( F ( D ) ( x ) ) . Since π and h are bag isomorphisms, we have

Now, set ρ ( v i ) = x j i for every i = 1 , . . . , n . Based on (12), we can easily define τ such that τ satisfies Definition 28 with respect to bags that are of depth 1 and 2 .

For q = 1 , . . . , r and l = 1 , . . . , |N q ( v ) | , let p q l be a path of length 2 q starting from the root node s in T s . Every such path p q l in T s corresponds to unique path v q l , that is in N q ( x ) , of length q in F . We represent the path by { v q l, 1 , v q l, 2 , . . . , v q l,q +1 } , where every consecutive node is connected to each other and for k = 1 , . . . q +1 , we have { v, v q l,k } ∈ E ( F ) iff { h ( v ) , h ( v q l,k ) } ∈ E ( G ) as h is a bag isomorphism. Further for every node k = 1 , . . . , q +1 there exists a j q l,k such that h ( v q l,k ) = ˜ x j q l,k = π ( x j q l,k ) , where ˜ x j q l,k ∈ V ( G ) , { ˜ x j q l, 1 , . . . , ˜ x j q l,q +1 } ∈ N q ( x ) and { x j q l, 1 , . . . , x j q l,q +1 } ∈ N q ( x ) . We set σ ( v q l,k ) = x j q l,k for every k = 1 , . . . , q +1 . Clearly, we have F [ {{ v, v i , v i +1 }} ] ∼ = G [{{ x, ˜ x j i , ˜ x j i +1 }}] ∼ = F ( D ) ( x ) [{{ x, x j i , x j i +1 }}] . (13)

Now, based on (13), we can easily define τ such that τ satisfies Definition 28 with respect to bags that correspond to paths in N q ( v ) for q = 1 , . . . , r . Now, following this construction recursively leads to a bag isomorphism ρ such that h = π ◦ ρ .

It remains to show that ( ρ, τ ) is unique (up to isomorphism). For this, let ( ρ 1 , τ 1 ) be another bag isomorphism homomorphism between ( F, T s ) and ( F ( D ) ( x ) , T ( D ) ( x )) such that ρ 1 ( v ) = x and h = π ◦ ρ 1 . We show that ρ = ρ 1 .

We begin by showing that ρ ( v ) = ρ 1 ( v ) for every v that is not in a cycle. Adopting the previous notations, consider v 1 , . . . , v n ∈ V ( F ) and x 1 , . . . , x m ∈ V ( F ( D ) ( x )) . For each i = 1 , . . . , n , let k i and l i be the indices such that ρ ( v i ) = x k i and ρ 1 ( v i ) = x l i . Consequently, π ( x k i ) = π ( x l i ) . We note that the image of h ( v i ) is not contained in a cycle in G , as otherwise, h would not be a bag isomorphism. Similarly, x k i and x l i are not contained in a cycle; otherwise, ρ and ρ 1 would not be bag isomorphisms. Now, π is an injective mapping if the domain is restricted to nodes that are of depth 1 and 2 , and not contained in a cycle. Hence, x k i = x l i .

Wecontinue by showing that for every w ∈ V ( F ) , that is contained in a cycle, we have ρ ( w ) = ρ 1 ( w ) . This follows a similar argument as the nodes that are not included in any cycle. We summarize the argument shortly: It must hold that ρ ( w ) and ρ 1 ( w ) are contained in a cycle, and π ( ρ ( w )) and π ( ρ 1 ( w )) as well. Now, π is injective if the domain is restricted to nodes that are only contained in cycles. Hence, ρ 1 = ρ .

We continue this subsection by introducing the concept of a bag extension in the context of treedecomposed graphs. This definition formalizes the notion of one tree-decomposed graph being an extension of another.

Definition 29 (Definition 20 in (Dell et al., 2018)) . Let ( F, T t ) be a tree-decomposed graph. A bag extension of ( F, T t ) is a graph ( H,T t ) with V ( H ) = V ( F ) such that for every t ∈ V ( T t ) the induced subgraph H [ β T t ( t )] is an extension of F [ β T t ( T )] , i.e., if e ∈ E ( F [ β T t ( T )]) , then e ∈ E ( H [ β T t ( T )]) . We define bExt ( ( F, T t ) , ( ˜ F, ˜ T s ) ) as the number of bag extensions of ( F, T t ) that are isomorphic to ( ˜ F, ˜ T s ) .

Intuitively, a bag extension of a tree-decomposed graph ( F, T s ) can be achieved by adding an arbitrary number of edges to F . Each added edge must be contained within a bag that corresponds to a node in the tree T s .

Definition 30 (Definition C.28 in (B. Zhang et al., 2024)) . Given a tree-decomposed graph ( F, T r ) and a graph G , a bag-strong homomorphism from ( F, T s ) to G is a homomorphism f from F to G such that, for all t ∈ V ( T r ) , f is a strong homomorphism from F [ β T s ( t )] to G [ f ( β T s ( t ))] , i.e., { u, v } ∈ E ( F [ β T s ( t )]) iff { f ( u ) , f ( v ) } ∈ E ( G [ f ( β T s ( t ))]) . Denote BStrHom(( F, T s ) , G ) to be the set of all bag-strong homomorphisms from ( F, T s ) to G , and denote bStrHom(( F, T s ) , G ) = | BStrHom(( F, T s ) , G ) | .

We continue with decomposing the number of homomorphism from a fan cactus graph to any graph. Lemma 7. Let r ∈ N . For any tree-decomposed graph ( F, T s ) ∈ M r +2 and any graph G , it holds

F

F

(1)

G

σ

π

h

Proof. The proof follows the lines of Lemma C.29. in (B. Zhang et al., 2024). First, (14) is well-defined as T s is finite, hence, there can only be finitely many bag extensions of ( F, T s ) . Further, consider the set

We consider the mapping σ from S to hom( F, G ) via (( ρ, τ ) , g ) ↦→ g ◦ ρ . We show that for every homomorphism h there exists a unique, up to automorphisms, ( ˜ F, ˜ T t ) ∈ M r +2 , ( ρ, τ ) and g such that h = g ◦ ρ .

We begin with the existence part. For h ∈ hom( F, G ) , we define ( ˜ F, ˜ T t ) ∈ M r +2 , ( ρ, τ ) and g as follows.

We define ˜ T t := T s . Clearly, ( ˜ F, ˜ T t ) ∈ M r +2 and it is a bag extension as only edges are added that are contained within a bag that corresponds to a node in T s .

We define ρ and τ as the identity mappings on their respective domain, leading to ( ρ, τ ) ∈ BExt ( ( F, T s ) , ( ˜ F, ˜ T t )) .

We define g = h . For x ∈ ˜ T t , we show that g is a strong homomorphism from ˜ F [ β ˜ T t ( x )] to G [ g ( β ˜ T t ( x ))] . Let { u, v } ∈ E ( ˜ F [ β ˜ T t ( x )] ) , then { g ( u ) , g ( v ) } ∈ E ( G [ g ( β ˜ T t ( x ))]) as h is a homomorphism with respect to the edges E ( F ) and in (15) only edge { u, v } were added that satisfy { h ( u ) , h ( v ) } ∈ E ( G ) . On the other hand { g ( u ) , g ( v ) } ∈ E ( G [ g ( β ˜ T t ( x ))]) , but { u, v } glyph[negationslash]∈ E ( ˜ F [ β ˜ T t ( x )] ) would contradict (15) as u, v are contained in the same bag β ˜ T t ( x ) . Hence, g ∈ BstrHom (( ˜ F, ˜ T t ) , G ) .

We finally prove the uniqueness part, i.e., that σ ( ( ˜ F 1 , ˜ T t 1 1 ) , ( ρ 1 , τ 1 ) , g 1 ) = h implies that there exists an isomorphism (˜ ρ, ˜ τ ) from ( ˜ F 1 , ˜ T t 1 1 ) to ( ˜ F, ˜ T t ) such that ˜ ρ ◦ ρ 1 = ρ , ˜ τ ◦ τ 1 = τ . We first prove that ˜ F 1 ∼ = ˜ F and ˜ T t 1 1 ∼ = ˜ T t .

For any u, v ∈ V ( F ) , we obviously have ρ ( u ) = ρ ( v ) iff u = v iff ρ 1 ( u ) = ρ 1 ( v ) as ρ and ρ 1 are injective mappings.

Let u, v ∈ V ( F ) . Consider { ρ 1 ( u ) , ρ 1 ( v ) } ∈ E ( ˜ F 1 ) , we show that { ρ ( u ) , ρ ( v ) } ∈ E ( ˜ F ) . If { u, v } ∈ E ( F ) , then clearly { ρ ( u ) , ρ ( v ) } ∈ E ( ˜ F ) as ρ is a homomorphism. Hence, assume that { u, v } glyph[negationslash]∈ E ( F ) . Then, u, v must be contained in the same bag of T s as ρ 1 is a bag extension and only node pairs are added if they are in the same bag. Hence, ρ ( u ) and ρ ( v ) are contained in the same bag. As g 1 is a homomorphism, we have { g 1 ( ρ 1 ( u )) , g 1 ( ρ 1 ( v )) } ∈ E ( G ) . But, then also { g ( ρ ( u )) , g ( ρ ( v )) } ∈ E ( G ) , and as g is a strong homomorphism (with respect to the bag in which ρ ( u ) and ρ ( v ) are contained), we have { ρ ( u ) , ρ ( v ) } ∈ E ( ˜ F ) . By symmetry of the argument, we have { ρ 1 ( u ) , ρ 1 ( v ) } ∈ E ( ˜ F 1 ) iff { ρ ( u ) , ρ ( v ) } ∈ E ( ˜ F ) .

Since ρ 1 and ρ are bag extension, they are bijective on their respective domain. Hence, ˜ ρ = ρ ◦ ρ -1 1 defines an isomorphism from ˜ F 1 to ˜ F . On the other hand, ˜ T t 1 1 ∼ = ˜ T t trivially holds, again with ˜ τ = τ ◦ τ -1 1 .

We have ˜ ρ ◦ ρ 1 = ρ , ˜ τ ◦ τ 1 = τ . We show that the tuple (˜ ρ, ˜ τ ) is an isomorphism, i.e., it remains to show that for any b ∈ ˜ T t 1 1 , we have ˜ ρ ( β ˜ T t 1 1 ( b )) = β ˜ T t (˜ τ ( b )) . Since τ 1 is surjective, we can choose a such that τ 1 ( a ) = b . Then,

The first and third equalities hold since ( ρ 1 , τ 1 ) and ( ρ, τ ) are bag extensions.

Definition 31 (Definition 30 in (B. Zhang et al., 2024)) . Given two tree-decomposed graphs ( F, T s ) and ( ˜ F, ˜ T t ) , a homomorphism ( ρ, τ ) from ( F, T s ) to ( ˜ F, ˜ T t ) is called bag-strong surjective if ρ is a bag-strong homomorphism from ( F, T s ) to ˜ F and is surjective on both vertices and edges, and τ is an isomorphism from T s to ˜ T t such that for all x ∈ V ( T s ) , we have ρ ( β T s ( x )) = β ˜ T t ( τ ( x )) . Denote BStrSurj(( F, T s ) , ( ˜ F, ˜ T t )) to be the set of all bag-strong subjective homomorphisms from ( F, T s ) to ( ˜ F, ˜ T t ) , and denote bStrSurj(( F, T s ) , ( ˜ F, ˜ T t )) = | BStrSurj(( F, T s ) , ( ˜ F, ˜ T t )) | .

Lemma 8. Let r ∈ N . For any tree-decomposed graph ( F, T s ) ∈ M r +2 and any graph G , it holds

where aut( ˜ F, ˜ T t ) counts the number of automorphisms of ( ˜ F, ˜ T t ) .

Proof. The proof follows the lines of Lemma C.31. in (B. Zhang et al., 2024).

Consider the set

We consider the mapping σ from S to BStrHom(( F, T s ) , G ) via (( ρ, τ ) , g ) ↦→ g ◦ ρ . We show that for every bag-strong homomorphism h there exists a unique, up to automorphisms, ( ˜ F, ˜ T t ) ∈ M r +2 , bag-strong surjective homomorphism ( ρ, τ ) and g such that h = g ◦ ρ .

We begin with the existence part. For h ∈ BStrHom(( F, T s ) , G ) , we define ( ˜ F, ˜ T t ) ∈ M r +2 , ( ρ, τ ) and g as follows.

We define ˜ F by defining an equivalence relation ∼ on V ( F ) : u ∼ v if h ( u ) = h ( v ) and there exists a path P in T s with endpoints t 1 , t 2 ∈ V ( T s ) such that u ∈ β T s ( t 1 ) , v ∈ β T s ( t 2 ) , and all nodes t on the path P satisfies that h ( u ) = h ( v ) ∈ h ( β T s ( t )) . We then define ρ as the quotient map with respect to ∼ and set ˜ F = F/ ∼ , i.e.,

glyph[negationslash]

which is well-defined as { u, v } ∈ E ( F ) imples ρ ( u ) = ρ ( v ) since h is a homomorphism. Then, ρ is surjective per construction.

Wedefine the mapping g : V ( ˜ F ) → V ( G ) such that g ( ρ ( u )) = h ( u ) for all u ∈ V ( F ) . This mapping g is well-defined since ρ ( u ) = ρ ( v ) implies h ( u ) = h ( v ) , and ρ : V ( F ) → V ( ˜ F ) is surjective. This leads to the equality h = g ◦ ρ . To demonstrate that g is a homomorphism, consider any edge ( x, y ) ∈ E ( ˜ F ) . There exists an edge ( u, v ) ∈ E ( F ) such that ρ ( u ) = x and ρ ( v ) = y , which implies ( h ( u ) , h ( v )) ∈ E ( G ) , since h is a homomorphism. Consequently, this means ( g ( x ) , g ( y )) ∈ E ( G ) .

We continue by defining the tree ˜ T t := ( V ( T ) , E ( T ) , β ˜ T t ) . We set t = s , and define τ to be the identity. Furthermore, we have β ˜ T t ( x ) = ρ ( β T s ( x )) for all x ∈ V ( T ) . It remains to prove that ( ˜ F, ˜ T t ) ∈ M r +2 is a valid tree decomposition. For this, it suffices to prove that for any vertex x ∈ V ( ˜ F ) the subgraph B ˜ T t ( x ) is connected. For this, let x ∈ V ( ˜ F ) and t 1 , t 2 ∈ B ˜ T t ( x ) . Then, there exists u ∈ β T s ( t 1 ) , v ∈ β T s ( t 2 ) such that ρ ( u ) = x, ρ ( v ) = x . Therefore, u ∼ v . As such, there exists a path P ∈ T s such that all nodes b on P satisfy h ( u ) ∈ h ( β T s ( b )) . Hence, for every b ∈ P there exists some w b ∈ β T s ( b ) such that h ( w b ) = h ( u ) , and consequently w b ∼ u . Finally, x = ρ ( u ) = ρ ( w b ) ∈ ρ ( β T s ( b )) = β ˜ T t ( b ) for all b in the path P . Hence, ( ˜ F, ˜ T t ) ∈ M r +2 .

It remains to prove that ρ is a bag-strong surjective homomorphism and g is a bag isomorphism. We begin by showing that ρ is a bag-strong surjective homomorphism. For this, let t ∈ V ( T s ) and u, v ∈ β T s ( t ) . If { u, v } glyph[negationslash]∈ E ( F ) , then { h ( u ) , h ( v ) } glyph[negationslash]∈ E ( G ) (since h is a bag-strong homomorphism). Therefore, { ρ ( u ) , ρ ( v ) } glyph[negationslash]∈ E ( ˜ F ) since g is a homomorphism. Hence, ρ is a bag-strong surjective homomorphism.

We show that g is a bag isomorphism. Let x ∈ V ( ˜ T t ) , and consider ˜ u, ˜ v ∈ β ˜ T t ( x ) . Since ρ is surjective, there exist u, v ∈ β T s ( x ) such that ρ ( u ) = ˜ u and ρ ( v ) = ˜ v . We have { ρ ( u ) , ρ ( v ) } glyph[negationslash]∈ E ( ˜ F ) iff { h ( u ) , h ( v ) } glyph[negationslash]∈ E ( G ) , since both ρ and h are bag-strong homomorphisms. Therefore, g is a bag isomorphism.

We finally prove that σ ( ( ˜ F 1 , ˜ T t 1 ) , ( ρ 1 , τ 1 ) , g 1 ) = σ ( ( ˜ F, ˜ T t ) , ( ρ, τ ) , g ) implies there exists an isomorphism (˜ ρ, ˜ τ ) from ( ˜ F 1 , ˜ T t 1 1 ) to ( ˜ F, ˜ T t ) such that ˜ ρ ◦ ρ 1 = ρ, ˜ τ ◦ τ 1 = τ, g 1 = g ◦ ˜ ρ . Let h = g 1 ◦ ρ 1 = g ◦ ρ . We will only show that ˜ F 1 ∼ = ˜ F since the remaining procedure is almost the same as in previous proofs. It suffices to prove that, for all u, v ∈ V ( F ) , ρ 1 ( u ) = ρ 1 ( v ) iff

h ( u ) = h ( v ) , and

There exists a path P in T s with endpoints t 1 , t 2 ∈ V ( T ) such that u ∈ β T s ( t 1 ) , v ∈ β T s ( t 2 ) , and all node x on path P satisfies that h ( u ) ∈ h ( β T s ( x )) .

We begin by showing the first direction, i.e., ρ 1 ( u ) = ρ 1 ( v ) implies Items a) and b). If ρ 1 ( u ) = ρ 1 ( v ) , we clearly have h ( u ) = h ( v ) as g 1 is well-defined. Also, there exists x 1 ∈ B T s ( u ) , x 2 ∈ B T s ( v ) , i.e., u ∈ β T s ( x 1 ) and v ∈ β T s ( x 2 ) . Hence, ρ ( u ) ∈ ρ ( β T s ( x 1 )) ⊂ β ˜ T t 1 1 ( τ 1 ( x 1 )) and ρ 1 ( u ) = ρ 1 ( v ) ∈ ρ 1 ( β T s ( x 2 )) ⊂ β ˜ T t 1 1 ( τ 1 ( x 2 )) since ( ρ 1 , τ 1 ) is a homomorphism. Hence, τ 1 ( x 1 ) , τ 1 ( x 2 ) ∈ B ˜ T t 1 1 ( ρ 1 ( u )) . Since ˜ T t 1 1 [ B ˜ T t 1 1 ( ρ 1 ( u ))] is connected, there is a path P in ˜ T t 1 1 [ B ˜ T t 1 1 ( ρ 1 ( u ))] with endpoints τ 1 ( x 1 ) , τ 1 ( x 2 ) such that all nodes x on P satisfies ρ 1 ( u ) ∈ β ˜ T t 1 1 ( x ) = β ˜ T t 1 1 ( τ ◦ τ -1 ( x )) = ρ 1 ( β T s ( τ -1 1 ( x )) ) . We conclude h ( u ) = g 1 ( ρ 1 ( u )) ∈ g 1 ( ρ 1 ( β T s ( τ -1 1 ( x )))) = h ( β T s ( τ -1 1 ( x ))) .

glyph[negationslash]

We continue by showing the second direction, i.e., ρ 1 ( u ) = ρ 1 ( v ) if Items a) and b). We prove this by contradiction, i.e., assume ρ 1 ( u ) = ρ 1 ( v ) but the above items (a) and (b) hold. We consider two cases.First, assume that u and v are in the same bag of T s . Then, as ( ρ 1 , τ 1 ) is a homomorphism, the nodes ρ 1 ( u ) and ρ 1 ( v ) are in the same bag of ˜ T t 1 1 . Since g 1 is a bag isomorphism, we have g 1 ( ρ 1 ( u )) = g 1 ( ρ 1 ( v )) . This contradicts Item (a) above.

glyph[negationslash]

glyph[negationslash]

Now, consider the second case. For this, assume that u and v are not in the same bag of T s . Then, there exist two adjacent nodes x 1 , x 2 on path P such that u ∈ β T s ( x 1 ) , u glyph[negationslash]∈ β T s ( x 2 ) . We have β T s ( x 2 ) ⊂ β T s ( x 1 ) as for every pair of nodes t 1 , t 2 in a canonical tree decomposition with { t 1 , t 2 } ∈ E ( T s ) we have either β T s ( t 1 ) ⊂ β T s ( t 2 ) or β T s ( t 2 ) ⊂ β T s ( t 1 ) . Now, item (b) implies that there exists w ∈ β T s ( x 2 ) such that w = u and h ( w ) = h ( u ) . Then, ρ 1 ( w ) ∈ ρ 1 ( β T s ( x 2 )) ⊂ ρ 1 ( β T s ( x 1 )) ⊂ β ˜ T t 1 1 ( τ 1 ( x 1 )) . Therefore, ρ 1 ( u ) and ρ 1 ( w ) are two different nodes in β ˜ T t 1 1 ( τ 1 ( x 1 )) with g 1 ( ρ 1 ( u )) = h ( u ) = h ( w ) = g 1 ( ρ 1 ( w )) . This contradicts the condition that g 1 is a bag isomorphism. This yields the desired result that ˜ F ∼ = ˜ F 1 .

Finally, we restate Theorem 2, with its proof now being a straightforward corollary of the preceding results in this section.

Theorem 2. Let r ≥ 0 . Then, r -glyph[lscript] WL can homomorphism-count M r +2 .

Proof. According to Corollary 3, if c r ( G ) = c r ( H ) , then cnt( F, G ) = cnt( F, H ) for every F ∈ M r +2 . Utilizing Lemma 6, we extend this result to bag isomorphism counts: bIso( F, G ) = bIso( F, H ) holds for every F ∈ M r +2 . Finally, invoking Lemma 7 and Lemma 8, we conclude that hom( F, G ) = hom( F, H ) for all F ∈ M r +2 .

H Implications of Theorem 2

In this section, we discuss important implications of Theorem 2 and provide proofs for the results in Corollary 2.

H.1 Appendix on F -Hom-GNNs and Proof of Corollary 2 i)

Recent work in the domain of MPNNs has explored enhancing the initial node features by incorporating homomorphism counts (Barceló et al., 2021). We summarize this approach in this section and compare it to our r -glyph[lscript] WLalgorithm.

Define F = { P s 1 , . . . , P s l } as a collection of rooted graphs, termed as patterns. In F -Hom-MPNNs, the initial feature vector of a vertex v in a graph G combines a one-hot encoding of the label χ G ( v ) with homomorphism counts corresponding to each pattern in F . The feature vector for each vertex v is recursively defined over rounds of message passing as follows:

Here, g ( t ) and f ( t ) represent the update and aggregation functions at depth t , respectively.

H.1.1 Expressivity of F -Hom-MPNNs

In this section, we summarize known results about the expressivity of F -Hom-MPNNs. The main result from Barceló et al., 2021 can be summarized as follows.

Theorem 6. For any two graphs G and H , it holds F -Hom-MPNNs can separate G and H if and only if hom ( T, G ) = hom ( T, H ) , for every F -pattern tree.

To understand the above theorem, we need to define the concept of F -pattern trees. For this, we define the graph join operator ∗ as follows. Given two rooted graphs G v and H w , the join graph ( G ∗ H ) v is obtained by taking the disjoint union of G v and H w , followed by identifying w with v . The root of the join graph is v . Further, if G is a graph and P r is a rooted graph, then joining a vertex v in G with P r results in the disjoint union of G and P r , where r is identified with v .

Let F = { P 1 , . . . , P l } . An F -pattern tree T r is constructed from a standard rooted tree S r = ( V, E, χ ) , which serves as the core structure, or the "backbone", of T r . To form T r , each vertex s ∈ V of the backbone may be joined to any number of duplicates of any patterns from F . Conceptually, an F -pattern tree is a tree graph enhanced by attaching multiple instances of any pattern from F to the nodes of the backbone tree. However, it is important to note that additional patterns may not be attached to any node that already derives from a pattern in F . Our method can homomorphism-count graphs where this is allowed, see Appendix H.1.2 .

Examples of F -pattern trees for F = { } are

where grey vertices are part of the backbones of the F -pattern trees, black vertices are the joined node and white vertices are part of the attached patterns. We define the set of F -pattern trees by F Tr .

H.1.2 Comparison with r -glyph[lscript] WL

We compare our proposed r -glyph[lscript] GIN against F r -Hom-MPNNs, where F r = { C 3 , . . . , C r +2 } consists of cycle graphs up to length r +2 . Both MPNN variants exhibit equivalent preprocessing complexity. However, after the initial layer, the computational complexity of our method is marginally higher, yet it increases linearly with the number of cycles present in the underlying graph.

According to Theorem 2, our method r -glyph[lscript] GIN can homomorphism-count all fan ( r +2) -cactus graphs. In particular, r -glyph[lscript] GIN can homomorphism-count all F r -pattern trees. For example, there are infinitely many fan r -cactus graphs that cannot be represented as F r -pattern trees, e.g., for r = 1

We restate Corollary 2 ii) and give a short proof.

Corollary 4. Let r ∈ N \ { 0 } . Then, r -glyph[lscript] WL is more powerful than F -Hom-MPNNs, where F = { C 3 , . . . , C r +2 } .

Proof. The proof of Corollary 2 ii) can be stated as a summary of all finding of the previous subsection: By Theorem 2, we have r -glyph[lscript] WL glyph[subsetsqequal] hom( M r +2 , · ) . By Theorem 6, we have F -Hom-MPNNs glyph[subsetsqequal] hom( F Tr , · ) and hom( F Tr , · ) glyph[subsetsqequal] F -Hom-MPNNs. Clearly, F Tr ⊂ M r +2 . Hence, r -glyph[lscript] WL glyph[subsetsqequal] hom( M r +2 , · ) glyph[subsetsqequal] hom( F Tr , · ) . Hence, r -glyph[lscript] WLis more powerful than F -Hom-MPNNs.

H.2 Appendix on Subgraph GNNs and Proof of Corollary 2 ii)

Subgraph GNNs treat a graph as a collection of graphs { G u | u ∈ N ( v ) } , where G u is a graph obtained by marking the corresponding node u . For every graph G u it runs an independent WL-

algorithm, i.e.,

Here, g ( t ) and f ( t ) represent the update and aggregation functions at depth t , respectively. The final node representations after t rounds are then calculated by

H.2.1 Expressivity of Subgraph GNNs and Comparison with r -glyph[lscript] WL

The expressivity of subgraph GNNs is fully characterized by the class

i.e., Subgraph GNNs can separate a pair of graphs G,H if and only if hom( F sub , G ) = hom( F sub , H ) . Furthermore, the set F sub is the maximal set that satisfies this property (B. Zhang et al., 2024, Theorem 3.4). We restate Corollary 2 ii) and provide a proof.

glyph[negationslash]

Corollary 5. 1 -glyph[lscript] WLis not less powerful than Subgraph GNNs. In particular, any r -glyph[lscript] WLcan separate infinitely many graphs that Subgraph GNNs fail to distinguish.

Proof. We show that already 1 -glyph[lscript] WLcan separate infinitely many graphs that Subgraph GNNs fail to distinguish. The other statements then follow as a simple corollary of Proposition 1.

glyph[negationslash]

For clarity, we begin by demonstrating that there exists a pair of graphs that 1 -glyph[lscript] GIN can separate, but Subgraph GNNs cannot distinguish. Consider the graph F defined as follows: F = { } . It holds that F ∈ M 3 \ F sub , where F sub is the maximal set that Subgraph GNNs can homomorphismcount. Then, by (B. Zhang et al., 2024, Theorem 3.4), there exists a pair of graphs G ( F ) and H ( F ) such that hom( F, G ( F )) = hom( F, H ( F )) and hom( F sub , G ( F )) = hom( F sub , H ( F )) . Hence, Subgraph GNNs cannot separate G ( F ) and H ( F ) . Since F ∈ M 3 , by hom( F, G ( F )) = hom( F, H ( F )) and Theorem 2, 1 -glyph[lscript] WLcan separate G ( F ) and H ( F ) .

glyph[negationslash]

This argument can be repeated for every F ∈ M 3 \ F sub . Since there are infinitely many graphs in M 3 \ F sub , the corollary follows.

We mention that the construction of the pair of graphs G ( F ) and H ( F ) in the previous proof is based on (twisted) Fürer graphs and is largely motivated by the constructions by Fürer (2001) and B. Zhang et al. (2024). More precisely, we can define G ( F ) as the Fürer graph of F and H ( F ) as the corresponding twisted Fürer graph. See Figure 7c for a visualization of F, G ( F ) , and H ( F ) .

H.3 Appendix on Subgraph k -GNNs and Proof of Corollary 2 iii)

Qian et al. (2022) introduced a higher-order version of Subgraph GNNs that compute representations for subgraphs made of tuples of nodes. Specifically, Subgraph k -GNNs - referred to as vertexsubgraph k -OSANs in the original work (Qian et al., 2022) - treat a graph as a collection of graphs { G u | u ∈ V ( G ) k } , where G u is a graph obtained by marking the corresponding nodes u . For every graph G u it runs an independent WL-algorithm, i.e.,

Here, g ( t ) and f ( t ) represent the update and aggregation functions at depth t , respectively. For every k -tuple u , the final representations after t rounds are then calculated by

The final graph representation is then given by

The homomorphism-expressivity of these GNNs is characterized as follows:

Theorem 7 (B. Zhang et al., 2024) . The homomorphism-expressivity of Subgraph k -GNN is given by F sub( k ) = { F : ∃ U ⊂ V F s.t. | U | ≤ k and F \ U is a forest } .

Since the set F sub( k ) is the maximal set of graphs that Subgraph k -GNNs can homomorphism-count, we can derive the following corollary, which restates Corollary 2 iii) and provides a proof.

Corollary 6. For any k ≥ 1 , 1 -glyph[lscript] WL is not less powerful than Subgraph k -GNNs. In particular, any r -glyph[lscript] WL can separate infinitely many graphs that Subgraph k -GNNs fail to distinguish.

Proof. The proof parallels the proof of Corollary 2. We present it here for completeness.

Let k ≥ 1 . We will show that there exists a pair of graphs that 1 -glyph[lscript] GIN can distinguish, but Subgraph k -GNNs cannot. Consider the graph F , defined as the unique graph with k triangle graphs, all connected by an edge to a single node.

glyph[negationslash]

It holds that F ∈ M 3 \ F sub( k ) , where F sub( k ) is the maximal set that Subgraph k -GNNs can homomorphism-count. Then, by (B. Zhang et al., 2024, Theorem 3.8), there exists a pair of graphs G ( F ) and H ( F ) such that hom( F, G ( F )) = hom( F, H ( F )) and hom( F sub( k ) , G ( F )) = hom( F sub( k ) , H ( F )) . Hence, Subgraph k -GNNs cannot separate G ( F ) and H ( F ) . Since F ∈ M 3 , by hom( F, G ( F )) = hom( F, H ( F )) and Theorem 2, 1 -glyph[lscript] WLcan separate G ( F ) and H ( F ) .

glyph[negationslash]

This argument can be repeated for every F ∈ M 3 \ F sub( k ) . Since there are infinitely many non-isomorphic graphs in M 3 \ F sub( k ) , the corollary follows.

H.4 Proof of Corollary 2 iv)

Proof of Corollary 2 iv). Given graphs F and G , it is well-known (see, e.g., (Neuen, 2024; Curticapean et al., 2017)) that sub( F, G ) can be decomposed as:

Here, the sum ranges over all non-isomorphic graphs in spasm( F ) . The sum in (20) is finite since the homomorphic image of F has at most | V ( F ) | nodes. Per assumption, we have spasm( F ) ⊂ M r +2 , i.e., by Theorem 2, r -glyph[lscript] WL can homomorphism-count spasm( F ) . In particular, if r -glyph[lscript] WL cannot separate two graphs G and H , we have hom(spasm( F ) , G ) = hom(spasm( F ) , H ) , and hence, sub( F, G ) = sub( F, H ) .

The result on subgraph-counting paths follows directly as the homomorphic image of a path P r +3 of length r +3 lies in M r .

I Appendix for Section 6

Theorem 3. For fixed t, r ≥ 0 , t iterations of r -glyph[lscript] WL are more powerful than r -glyph[lscript] MPNN with t layers. Conversely, r -glyph[lscript] MPNN is more powerful than r -glyph[lscript] WL if the functions f ( t ) , g ( t ) in (3) are injective.

Proof of Theorem 3. We begin by proving that c ( t ) r glyph[subsetsqequal] h ( t ) r . We argue by induction over t for any fixed r ≥ 0 .

Initially, c (0) r = h (0) r as both labeling functions start with the same base labels. Now assume c ( t +1) r ( u ) = c ( t +1) r ( v ) for some u, v ∈ V ( G ) . By definition,

This implies c ( t ) r ( u ) = c ( t ) r ( v ) and

as HASH is an injective function.

By induction hypothesis, we hence have h ( t ) r ( u ) = h ( t ) r ( v ) and

which implies that any function, in particular f ( t +1) k and g ( t +1) have to return the same result. Therefore, we have h ( t +1) r ( u ) = h ( t +1) r ( v ) .

We proceed to prove h ( t ) r glyph[subsetsqequal] c ( t ) r if all message, update, and readout functions are injective in Definition 9. For this, we show that for each t ≥ 0 there exists an injective function φ such that h ( t ) r = φ ◦ c ( t ) r . For t = 0 , we can choose φ to be the identity function. Assume that for t -1 there exists an injective function φ such that h ( t -1) r ( v ) = φ ◦ c ( t -1) r ( v ) . Then, we can write

where for every q = 0 , . . . , r , we set φ ◦ m ( t ) q ( v ) := {{ ( φ ◦ c ( t -1) r ( p )) | p ∈ N q ( v ) }} and ( φ ◦ c ( t -1) r ( p )) = ( φ ◦ c ( t -1) r ( p 1 ) , . . . , φ ◦ c ( t -1) r ( p q +1 )) for p = { p i } q +1 i =1 ∈ N q ( v ) . By assumption, all message, update, and readout functions are injective in Definition 9. Since the concatenation of injective functions is injective, there exists an injective function ψ such that

As HASH in Definition 7 is injective, the inverse HASH -1 exists and is also injective. Hence,

Choosing φ = ψ ◦ HASH -1 finishes the proof.

We conclude this section with the following lemma that justifies our architectural choice in (4). Lemma 9. Let x ∈ Q r . Then there exist ε ∈ R r such that

glyph[negationslash]

is an injective function.

Proof. We prove this claim by induction. For r = 0 , any x = 0 ∈ R fulfills the claim. Now, let ε ∈ R r such that ϕ ( x ) : Q r → R is injective. The set Q [ ε 1 , . . . , ε r ] = { ∑ r k =0 ε k x k | x ∈ Q r } is

glyph[negationslash]

glyph[negationslash]

countable and hence a proper subset of R . It follows that there exists ε r +1 ∈ R with ε r +1 / ∈ Q [ ε ] . Note that 0 ∈ Q and hence ε r +1 = 0 . We now prove our claim by contradiction.

glyph[negationslash]

glyph[negationslash]

Assume there exist x = x ′ ∈ Q r +1 with ∑ r +1 k =0 ε k x k = ∑ r +1 k =0 ε k x ′ k . We distinguish two cases: x i = x ′ i for all i ≤ r and x r +1 = x ′ r +1 : Then immediately glyph[negationslash]

∑ r k =0 ε k x k = ∑ r k =0 ε k x ′ k : But then

The left hand side is an element of Q [ ε 1 , . . . , ε r ] . However, ε r +1 ( x ′ r +1 -x r +1 ) / ∈ Q [ ε 1 , . . . , ε r ] by choice of ε r +1 , leading to a contradiction.

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We mention our main results in the abstract and introduction (see Section 1), and provide proofs for all claims (see Appendix E ff.) and code for all experiments (see our GitHub repository). We clearly state the scope of our theoretical results and emphasize that our method provides an expressive architecture suitable for sparse real-world graphs.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We describe the computational complexity of our method in a separate paragraph (see Section 6). We specifically address its preprocessing complexity on dense graphs in a separate paragraph (see Section 7). We also mention important future work that would complement our theoretical contributions in the conclusion section (see Section 8).

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All theorems are numbered, and we refer to the formal proofs in the appendix (see Appendix E ff.).

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: All hyperparameters, hyperparameter ranges, and used resources are given in Appendix C. Our GitHub repository provides instructions on how to reproduce all experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All hyperparameters, hyperparameter ranges, and used resources are given in the appendix (see Appendix C). Our GitHub repository contains instructions to reproduce all experiments.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Data splits, hyperparameters, hyperparameter grids (if used), optimizer, etc. are detailed in the main paper (see Section 7) and appendix (see Appendix C), with additional instructions provided in the anonymous GitHub link (see our GitHub repository).

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We report error bars and confidence intervals for our experimental results (see Section 7), capturing the variability due to different random seeds and initialization. Detailed information about statistical significance tests is provided in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: All information on the compute resources is given in the appendix (see Appendix C). For the real-world experiments, we provide run times, and for the synthetic experiments, the run times are negligible.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We respect the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: While our work has numerous potential societal consequences, none of which we feel must be explicitly emphasized here, its impact lies in providing a theoretical foundation for improved graph representation learning methodologies.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper does not release data or models with a high risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All used assets are properly cited, and the licenses are mentioned.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We introduce new assets in the form of code. All details about training, datasets, and models are given in the submitted paper and/or in our GitHub repository, including detailed instructions to run the code. The license is provided under the MIT license.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing or research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces the $r$-loopy Weisfeiler-Leman ($r$-$l$WL) test, an innovative hierarchy of graph isomorphism tests, and the corresponding GNN framework, $r$-$l$MPNN. This new approach extends the counting capabilities of previous algorithms, specifically allowing the counting of cycles up to length $r+2$ and homomorphisms of cactus graphs. Empirical validation demonstrates the expressiveness and performance of $r$-$l$MPNN on both synthetic and real-world datasets.\n\nSTRENGTHS:\nThe paper's strengths are rooted in its originality, introducing a novel algorithm ($r$-$l$WL) and corresponding GNNs ($r$-$l$GIN) that significantly enhance the expressivity of graph neural networks. These contributions are supported by rigorous theoretical proofs and empirical validation. Specifically, $r$-$l$WL demonstrates the ability to count cycles up to length $r+2$ and homomorphisms of cactus graphs, substantiated with detailed mathematical proofs. The experiments use several synthetic datasets to validate the counting power and expressiveness of $r$-$l$MPNN effectively. Furthermore, the paper contextualizes its contributions within prior work, highlighting the limitations of existing methods and demonstrating how $r$-$l$WL and $r$-$l$MPNN address these gaps. Overall, the claims are well-supported by theoretical proofs and empirical results, indicating a clear improvement over existing methods.\n\nWEAKNESSES:\nSome of the mathematical proofs are complex and may be difficult for readers without a strong background in graph theory and GNNs. Providing additional intuitive explanations or examples could improve accessibility. While the empirical validation is strong, it could be expanded to include a broader range of real-world datasets to further demonstrate the robustness and generalizability of the approach.\n\nQUESTIONS:\nThe paper is generally well-written and clear, although some sections could benefit from additional explanations or examples to aid understanding. I have a few questions:\n\n1. How does the computational complexity of $r$-$l$WL compared to existing higher-order WL variants like $3$-WL in practice, especially when dealing with large and dense graphs?\n\n2. What are the limitations of $r$-$l$WL in terms of scalability and memory usage, particularly when applied to real-world datasets with varying degrees of sparsity?\n\n3. In Table 1, why didn't your method perform well on the Extension (100) and CFI (100) datasets compared to 3-WL and PPGN?\n\nLIMITATIONS:\nThe authors have addressed the limitations related to the complexity of higher-order GNNs and the scalability issues associated with $k$-WL. However, a more detailed discussion on the limitations of $r$-$l$WL in terms of computational overhead and potential impact on large-scale applications would be beneficial.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-22"}

comments: SUMMARY:
The paper proposed a hierarchy of graph isomorphism tests and a corresponding GNN framework, $r$-$\ell$-MPNN while showing the ability to count homomorphisms of cactus graphs.

STRENGTHS:
The strengths of the paper are:
* The ability to count homomorphisms of cactus graphs without any additional explicit substructure counts.
* Scalability towards large datasets, especially when the graphs are sparse for these datasets.
* The paper is well-written and easy to understand.

WEAKNESSES:
The weaknesses of the paper is that while the paper is well written, there are some terms undefined, or unexplained such as HASH; see the first question in the following section.

QUESTIONS:
1) What is the HASH function? do you have an example of such a function?
2) Can you add tables concerning the times of your method compared to other methods?
3)  What is the motivation behind using $r=5$ in the experiments? would any $r>5$ result in worse results or simply better results by small significance on the expense of might higher time?

LIMITATIONS:
The authors have adequately addressed the limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose a loopy version of the Weisfeiler-Lehman (WL) algorithm. This version utilizes an extended notion of neighborhood, incorporating paths between standard neighboring vertices to update vertex coloring. By parameterizing the length $r$ of these paths, we obtain $r$-$l$WL. The results include a hierarchy for $r$-$l$WL based on $r$ and a comparison with $k$-WL. The most technical result is that $r$-$l$WL is expressive enough to determine homomorphism counts of $(r+2)$-cactus graphs. Experiments show that $r$-$l$WL is competitive in detecting substructures.\n\nSTRENGTHS:\n**S1 Interesting Idea:**\nThe inclusion of paths in the neighborhood is an elegant and innovative way of extracting local information around a vertex. This approach allows for rigorous theoretical analysis and captures cactus graphs, which is a significant advantage.\n\n**S2 Experiments:**\nThe authors empirically demonstrate that the neural version of $r$-$l$WL captures crucial information for graph learning tasks.\n\n**S3 Capturing Cacti:** The most technically involved proof shows that cactus graphs, or at least their homomorphism counts, can be captured. This is a noteworthy result.\n\nWEAKNESSES:\n**W1 Capturing Cacti:** \nThe motivation for focusing on cactus graphs is unclear. The authors should better justify why this class of graphs is interesting and relevant, and provide examples in the main paper.\n\n**W2 Insufficient Comparison with Subgraph GNNs:** \nThe paper mentions that subgraph GNNs are bounded by $3$-WL, but there are subgraph GNNs beyond $3$-WL, such as $k$-OSAN s (ordered subgraph aggregation networks, Qian et al.), which can capture features that $k$-WL cannot and are bounded by $(k+1)$-WL. The paper overlooks this line of work and lacks comparison with $k$-OSANs, both empirically and theoretically. For example, specifying paths of length $r$ and a central node requires special subgraphs of size $r+1$ (the path plus central node), suggesting that $r$-$l$WL may be included in $(r+1)$-OSAN? This could imply that some results stem from $k$-OSAN properties. A more detailed comparison is needed.\n\n**W3 Experimental Comparison:**\nThe experiments do not seem to include any subgraph GNNs that capture properties beyond 3-WL. The authors should make more clear what are the capabilities of the methods with which they compare.\n\n**W4 Unlabeled Graphs:**\nThe paper does not address vertex labels. Can the approach be generalized to vertex labels? Additionally, $c^{(0)}$ in section 3.2 is undefined.\n\nMinor Comments: The authors sometimes use obscure references. For instance:\n- line 71: Do you mean Tinhofer or Dvorak?\n- line 96: Why refer to Dimitrov 2023 for the notion of graph invariant, which has existed for ages?\n\nQUESTIONS:\nPlease comment on **W1**, **W2** and **W3**.\n\nLIMITATIONS:\nThis has been addressed in a satisfactory way by the authors.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
This introduce $r$-loopy Weisfeiler-Leman, a new hierarchy of graph isomorphism test and a corresponding GNN framework. It achieves good cycle counting power and surpuss $k$-WL in some cases. The power of r-lWL is examined in various synthetic and real-world datasets.

STRENGTHS:
1. Strong theoretic results with concrete proof.
2. The algorithm is local with good expressivity.

WEAKNESSES:
1. More datasets [1, 2] can be included to evaluate expressivity, especially long range expressivity. 


[1] Vijay Prakash Dwivedi, Ladislav Rampásek, Michael Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, Dominique Beaini: Long Range Graph Benchmark. NeurIPS 2022.
[2] Yanbo Wang, Muhan Zhang. Towards Better Evaluation of GNN Expressiveness with BREC Dataset. arxiv/abs/2304.07702

QUESTIONS:
1. For any pair of non-isomorphic graph, can $r$-lwl differentiate them with large enough $r$? (similar that $k$-WL can solve graph isomorphism problem for graph of node less than $k$).

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-17