SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Honghua Dong $^{1,2,3*}$  Jiawei Xu $^{1,4*}$  Yu Yang $^{1,4*}$  Rui Zhao $^{1}$  Shiwen Wu $^{5}$  Chun Yuan $^{4}$  Xiu Li $^{4}$  Chris Maddison $^{2,3}$  Lei Han $^{1\dagger}$ $^{1}$ Tencent Robotics X University of Toronto  $^{3}$ Vector Institute Tsinghua University Hong Kong University of Science and Technology
Graph neural networks, which typically exchange information between local neighbors, often struggle to capture long-range interactions (LRIs) within the graph. Building a graph hierarchy via graph pooling methods is a promising approach to address this challenge; however, hierarchical information propagation cannot entirely take over the role of local information aggregation. To balance locality and hierarchy, we integrate the local and hierarchical structures, represented by intra- and inter-graphs respectively, of a multi-scale graph hierarchy into a single mega graph. Our proposed MeGraph model consists of multiple layers alternating between local and hierarchical information aggregation on the mega graph. Each layer first performs local-aware message-passing on graphs of varied scales via the intra-graph edges, then fuses information across the entire hierarchy along the bidirectional pathways formed by inter-graph edges. By repeating this fusion process, local and hierarchical information could intertwine and complement each other. To evaluate our model, we establish a new Graph Theory Benchmark designed to assess LRI capture ability, in which MeGraph demonstrates dominant performance. Furthermore, MeGraph exhibits superior or equivalent performance to state-of-the-art models on the Long Range Graph Benchmark. The experimental results on commonly adopted real-world datasets further demonstrate the broad applicability of MeGraph.<sup>1</sup>
Graph-structured data, such as social networks, traffic networks, and biological data, are prevalent across a plethora of real-world applications. Recently, Graph Neural Networks (GNNs) have emerged as a powerful tool for modeling and understanding the intricate relationships and patterns present in such data. Most existing GNNs learn graph representations by iteratively aggregating information from individual nodes' local neighborhoods through the message-passing mechanism. Despite their effectiveness, these GNNs struggle to capture long-range interactions (LRIs) between nodes in the graph. For instance, when employing a 4-layer vanilla GNN on the 9-node ( $A$  to  $I$ ) graph (as shown in Fig. 1), the receptive field of node  $A$  is limited to 4-hop neighbors, making the aggregation of information from nodes  $G$ ,  $H$ , and  $I$  into node  $A$  quite challenging. While GNNs could theoretically incorporate information from nodes  $n$ -hops away with  $n$ -layers of message passing, this often leads to over-smoothing and over-squashing issues [17, 3] when  $n$  is large.
* Equal Contribution. Work done while HD, JX and YY are interns at Tencent Robotics X.
†Corresponding authors, contact honghuad@cs.toronto.edu and lxhan@tencent.com.
<sup>1</sup>Project website and open-source code can be found at https://sites.google.com/view/megraph.
Figure 1: Illustration of the graph pooling operation, graph pyramid, and mega graph. Graph pooling is a downsampling process comprising SELECT, CONNECT, and REDUCE steps. It begins by selecting subsets for grouping and each subset collapses into a new node in the pooled graph. Next, it forms new edges by merging the original ones, and finally calculates the pooled graph's features. In this graph, nodes  $A, B, C, D, E, F$ , and  $G, H, I$  are pooled into nodes  $1, 2$ , and  $3$  respectively, while the edges  $(B, E)$  and  $(C, D)$  are merged into  $(1, 2)$ . Graph Pyramid involves multi-scaled graphs derived from iterative graph pooling, with the height indicating different scales and  $h = 1$  symbolizing the original graph. Mega Graph is formed by connecting the graph pyramid using inter-graph edges, which are the by-products of graph pooling.
One mainstream solution to this problem involves constructing a multi-scale graph hierarchy through graph pooling methods. Previous efforts, such as Graph UNets [22] and HGNet [47], have attempted to broaden the receptive field using this strategy. They downsample and upsample the graph, aggregating information along the hierarchy. However, hierarchical information propagation cannot take over the role of local information aggregation. To illustrate, consider the graph hierarchy depicted in Fig. 1. The information propagated along the hierarchy from node  $B$  to nodes  $D, E$ , and  $F$  tends to be similar since they share the common path  $B-1-X-2$ . However, in the original graph, node  $B$  holds different degrees of importance to nodes  $D, E$ , and  $F$  as they are 2, 1, and 3 hops away respectively.
To balance the importance of locality and hierarchy, we amalgamate the local and hierarchical structures of a multi-scale graph hierarchy into a single mega graph as depicted in Fig. 1, where we refer to the local structure as intra-graph edges and hierarchical structure as inter-graph edges. Based on this mega graph, we introduce our MeGraph model consisting of  $n$  Mee layers. Each layer first performs local-aware message-passing on graphs of varied scales via the intra-graph edges and then fuses information across the whole hierarchy along the bidirectional pathways formed by inter-graph edges. This method enables hierarchically fused information to circulate within local structures and allows locally fused information to distribute across the hierarchy. By repeating this fusion process, local and hierarchical information could intertwine and complement each other. Moreover, to support flexible graph pooling ratios when constructing the multi-scale graph hierarchy, we propose a new graph pooling method S-EdgePool that improves from EdgePool [13].
In our experiments, We first evaluate MeGraph's capability to capture Long Range interactions (LRIs). We establish a Graph Theory Benchmark comprising four tasks related to shortest paths and one related to connected components. MeGraph demonstrates superior performance compared with many competitive baselines. MeGraph also achieves comparable or superior performance than the state-of-the-art on the Long Range Graph Benchmark (LRGB) [17]. In addition, we perform extensive experiments on widely-used real-world datasets that are not explicitly tailored for assessing the capacity to capture LRIs. These include the GNN benchmark [15] and OGB-G datasets [28]. In these tests, MeGraph demonstrates superior or equivalent performance compared to the baseline models, suggesting its broad applicability and effectiveness.
The main contributions of this work are summarized as follows: 1) Mega graph and novel architecture: we propose the mega graph, a multi-scale graph formed by intra- and inter-graph edges, where the message-passing over the mega graph naturally balances locality and hierarchy. On this basis, we introduce a novel architecture MeGraph, which alternates information aggregation along the intra- and inter-edges of the mega graph. This fusion process intertwines local and hierarchical information, leading to mutual benefits. 2) Hierarchical information fusion: we design a bidirectional pathway to facilitate information fusion among the hierarchies. 3) S-EdgePool: we enhance EdgePool into S-EdgePool, allowing an adjustable pooling ratio. 4) Benchmark and Evaluations: we establish a new graph theory benchmark to evaluate the ability of models to capture LIRIs. In these evaluations, MeGraph exhibits dominant performance. Additionally, MeGraph achieves new SOTA in one task of LRGB and shows better or comparable performance compared with baselines on popular real-world datasets.
2 Notations, Backgrounds and Preliminaries
Let  $\mathcal{G} = (\mathcal{V},\mathcal{E})$  be a graph with node set  $\mathcal{V}$  (of cardinality  $N^v$ ) and edge set  $\mathcal{E}$  (of cardinality  $N^e$ ). The edge set can be represented as  $\mathcal{E} = \{(s_k,t_k)\}_{k = 1:N^e}$ , where  $s_k$  and  $t_k$  are the indices of the source and target nodes connected by edge  $k$ . We define  $\mathbf{X}^{\mathcal{G}}$  as features of graph  $\mathcal{G}$ , which is a combination of global (graph-level) features  $\mathbf{u}^{\mathcal{G}}$ , node features  $\mathbf{V}^{\mathcal{G}}$ , and edge features  $\mathbf{E}^{\mathcal{G}}$ . Accordingly, we use  $\mathbf{V}_i^{\mathcal{G}}$  to represent the features of a specific node  $v_i$ , and  $\mathbf{E}_k^{\mathcal{G}}$  denotes the features of a specific edge  $(s_k,t_k)$ . We may abuse the notations by omitting the superscript  $\mathcal{G}$  when there is no context ambiguity.
2.1 Graph Network (GN) Block
We adopt the Graph Network (GN) block design in accordance with the GN framework [6]. In our notation, a GN block accepts a graph  $\mathcal{G}$  and features  $\mathbf{X} = (\mathbf{u},\mathbf{V},\mathbf{E})$  as inputs, and produces new features  $\mathbf{X}' = (\mathbf{u}',\mathbf{V}',\mathbf{E}')$ . A full GN block [6] includes the following update steps. In each of these steps,  $\phi$  denotes an update function, typically implemented as a neural network:
**Edge features:**  $\mathbf{E}_k' = \phi^e (\mathbf{E}_k, \mathbf{V}_{s_k}, \mathbf{V}_{t_k}, \mathbf{u}), \forall k \in [1, N^e]$ .
Node features:  $\mathbf{V}_i' = \phi^v (\rho^{e\to v}(\{\mathbf{E}_k'\}_{k\in [1,N^e ],t_k = i}),\mathbf{V}_i,\mathbf{u}),\forall i\in [1,N^v ]$  , where  $\rho^{e\rightarrow v}$  is an aggregation function taking the features of incoming edges as inputs.
Global features:  $\mathbf{u}' = \phi^u (\rho^{e\to u}(\mathbf{E}')$ $\rho^{v\to u}(\mathbf{V}')$  , u), where  $\rho^{e\rightarrow u}$  and  $\rho^{v\rightarrow u}$  are two global aggregation functions over edge and node features.
Given a fixed graph structure  $\mathcal{G}$  and the consistent input and output formats outlined above, GN blocks can be seamlessly integrated to construct complex, deep graph networks.
2.2 Graph Pooling
Graph pooling operation downsamples the graph structure and its associated features while ensuring the preservation of structural and semantic information inherent to the graph. Drawing from the SRC framework [23], we identify graph pooling as a category of functions,  $\mathsf{P00L}$ , that maps a graph  $\mathcal{G} = (\mathcal{V},\mathcal{E})$  with  $N^v$  nodes and features  $\mathbf{X}^{\mathcal{G}}$  to a reduced graph  $\tilde{\mathcal{G}} = (\tilde{\mathcal{V}},\tilde{\mathcal{E}})$  with  $N^{\tilde{v}}$  nodes and new features  $\mathbf{X}^{\tilde{\mathcal{G}}}$ . Here,  $N^{\tilde{v}}\leq N^{v}$  and  $(\tilde{\mathcal{G}},\mathbf{X}^{\tilde{\mathcal{G}}}) = \mathsf{P00L}(\mathcal{G},\mathbf{X}^{\mathcal{G}})$ .
The SRC framework deconstructs the POOL operation into SELECT, REDUCE, and CONNECT functions, which encompass most existing graph pooling techniques. We reinterpret these functions in our own notation as follows:
$$
(\hat {\mathcal {G}}, \mathbf {X} ^ {\hat {\mathcal {G}}}) = \operatorname {S E L E C T} (\mathcal {G}, \mathbf {X} ^ {\mathcal {G}}); \tilde {\mathcal {G}} = \operatorname {C O N N E C T} (\mathcal {G}, \hat {\mathcal {G}}, \mathbf {X} ^ {\hat {\mathcal {G}}}); \mathbf {X} ^ {\tilde {\mathcal {G}}} = \operatorname {R E D U C E} (\mathbf {X} ^ {\mathcal {G}}, \hat {\mathcal {G}}, \mathbf {X} ^ {\hat {\mathcal {G}}}). \tag {1}
$$
As shown in Fig. 1, the SELECT establishes  $N^{\tilde{v}}$  nodes for the pooled graph, and each node  $\tilde{v}$  corresponds to a subset of nodes  $S_{\tilde{v}} \subseteq \mathcal{V}$  in the input graph. This creates an undirected bipartite graph  $\hat{\mathcal{G}} = (\hat{\mathcal{V}}, \hat{\mathcal{E}})$ , with  $\hat{\mathcal{V}} = \mathcal{V} \cup \tilde{\mathcal{V}}$  and  $(v, \tilde{v}) \in \hat{\mathcal{E}}$  if and only if  $v \in S_{\tilde{v}}$ . We refer to this graph  $\hat{\mathcal{G}}$  as the inter-graph, a larger graph that links nodes in the input graph  $\mathcal{G}$  with nodes in the pooled graph  $\tilde{\mathcal{G}}$ . The SELECT function can be generalized to include inter-graph features  $\mathbf{X}^{\hat{\mathcal{G}}}$ . As an example, edge weights can be introduced for some edge  $(\hat{s}_k, \hat{t}_k)$  in graph  $\hat{\mathcal{G}}$  to gauge the importance of node  $\hat{s}_k$  from the input graph contributing to node  $\hat{t}_k$  in the pooled graph.
The CONNECT function reconstructs the edge set  $\tilde{\mathcal{E}}$  between the nodes in  $\tilde{\nu}$  of the pooled graph  $\tilde{\mathcal{G}}$  based on the original edges in  $\mathcal{E}$  and the inter-graph edges in  $\hat{\mathcal{E}}$ . The REDUCE function calculates the graph features  $\mathbf{X}^{\tilde{\mathcal{G}}}$  of graph  $\tilde{\mathcal{G}}$  by aggregating input graph features  $\mathbf{X}^{\mathcal{G}}$ , taking into account both the inter-graph  $\hat{\mathcal{G}}$  and features  $\mathbf{X}^{\hat{\mathcal{G}}}$ . In a similar vein to the relationship between graph lifting and coarsening, we define the EXPAND function for graph features, which serves as the inverse of the REDUCE function:  $\mathbf{X}^{\mathcal{G}} = \mathrm{EXPAND}(\mathbf{X}^{\tilde{\mathcal{G}}},\hat{\mathcal{G}},\mathbf{X}^{\hat{\mathcal{G}}})$ .
We begin with the introduction of the mega graph (Sec.3.1), which amalgamates the local (intraedges) and hierarchical (inter-edges) structures of a multi-scale graph hierarchy into a single graph.
Figure 2: Illustration of the MeGraph model, where  $n_{-}$  denotes  $n - 1$ . The blue and green circles represent features of intra- and inter-graphs, respectively. In this figure, the horizontal and vertical directions represent the interaction among the local structure (intra-graph) and graph hierarchy (inter-graph) respectively. The features of intra- and inter-graphs are represented by blue and green circles, respectively. In this figure, the horizontal and vertical directions denote the local structure and graph hierarchy respectively. During the encode stage, the mega graph is constructed using graph pooling. In the process stage, the Mee layer, which features bidirectional pathways across multiple scales, is stacked  $n$  times. In the decode stage, multi-scale features are read out. The golden inter GN blocks form bidirectional pathways across the whole hierarchy.
Following this, we present the MeGraph model (Sec.3.2), which alternates between the aggregation of local and hierarchical information along the intra- and inter-edges of the mega graph. We then discuss the specific choices made for the core modules of the MeGraph, along with the innovations (Sec.3.3). Finally, we delve into the computational complexity of the MeGraph model (Sec.3.4).
3.1 Connecting Multi-scale Graphs into a Mega Graph
Similar to the concept of an image pyramid [1], a graph pyramid is constructed by stacking multi-scale graphs, which are obtained through iterative downsampling of the graph using a graph pooling technique. Formally, in alignment with the definition of an image feature pyramid [39], we define a graph feature pyramid as a set of graphs  $\mathcal{G}_{1:h} := \{\mathcal{G}_i\}_{i=1,\dots,h}$  and their corresponding features  $\mathbf{X}^{\mathcal{G}_{1:h}} := \{\mathbf{X}^{\mathcal{G}_i}\}_{i=1,\dots,h}$ . Here,  $\mathcal{G}_1$  represents the original graph,  $\mathbf{X}^{\mathcal{G}_1}$  signifies the initial features,  $h$  stands for the height, and  $(\mathcal{G}_i, \mathbf{X}^{\mathcal{G}_i}) = \mathrm{P00L}(\mathcal{G}_{i-1}, \mathbf{X}^{\mathcal{G}_{i-1}})$  for  $i > 1$ .
By iteratively applying the POOL function, we can collect the inter-graphs  $\hat{\mathcal{G}}_{1:h} \coloneqq \{\hat{\mathcal{G}}_i\}_{i=1,\dots,h-1}$  and their features  $\mathbf{X}^{\hat{\mathcal{G}}_{1:h}} \coloneqq \{\mathbf{X}^{\hat{\mathcal{G}}_i}\}_{i=1,\dots,h-1}$  (since there are  $h-1$  inter-graphs for  $h$  intra-graphs), where  $(\hat{\mathcal{G}}_i, \mathbf{X}^{\hat{\mathcal{G}}_i}) = \mathrm{SELECT}(\mathcal{G}_i, \mathbf{X}^{\mathcal{G}_i})$  for  $i < h$ . The bipartite inter-graph  $\hat{\mathcal{G}}$  and its features  $\mathbf{X}^{\hat{\mathcal{G}}}$  essentially depict the relationships between the graphs before and after the pooling process (see Sec. 2.2).
Finally, as illustrated in Fig. 1, we wire the graph pyramid  $\mathcal{G}_{1:h}$  using the edges found in the bipartite graphs  $\hat{\mathcal{G}}_{1:h}$ . This results in a mega graph  $\mathcal{M}\mathcal{G} = (\mathcal{MV},\mathcal{ME})$ , where  $\mathcal{MV} = \bigcup_{i=1}^{h}\mathcal{V}_i$  and  $\mathcal{ME} = \bigcup_{i=1}^{h}\mathcal{E}_i \cup \bigcup_{i=1}^{h-1}\hat{\mathcal{E}}_i$ . The structure of the mega graph would vary as the graph pooling method trains. We denote  $\mathcal{M}\mathcal{G}_{\mathrm{intra}} = \bigcup_{i=1}^{h}\mathcal{G}_i$  as the intra-graph of  $\mathcal{M}\mathcal{G}$ , and refer to the edges therein as intra-edges. Correspondingly,  $\mathcal{M}\mathcal{G}_{\mathrm{inter}} = \bigcup_{i=1}^{h-1}\hat{\mathcal{G}}_i$  is referred to as the inter-graph of  $\mathcal{M}\mathcal{G}$ , with its corresponding edges termed as inter-edges. The features  $\mathbf{X}^{\mathcal{M}\mathcal{G}}$  of the mega graph  $\mathcal{M}\mathcal{G}$  is a combination of intra-graph features  $\mathbf{X}^{\mathcal{G}_{1:h}}$  and inter-graph features  $\mathbf{X}^{\hat{\mathcal{G}}_{1:h}}$ .
3.2 Mega Graph Message Passing
We introduce the MeGraph architecture, designed to perform local and hierarchical aggregations over the mega graph alternately. As shown in Fig.2, the architecture follows the encode-process-decode design [6, 25] and incorporates GN blocks (refer to Sec. 2.1) as fundamental building blocks.
During the encode stage, initial features are inputted into an intra-graph GN block, which is followed by a sequence of graph pooling operations to construct the mega graph  $\mathcal{M}\mathcal{G}$  and its associated features
Figure 3: Illustration of the Mee layer, where  $i_{-}$  denotes  $i - 1$  and  $j_{+}$  denotes for  $j + 1$ . The blue and green circles represent the features of intra- and inter-graphs, respectively. Grey and golden arrows represent the intra and inter GN blocks. The cross-update utilizes inter GN blocks to exchange information between consecutive heights, as elaborated in the main text. The Mee layer first aggregates information locally along inter-graph edges. It then applies cross-updates sequentially from lower to higher levels, accumulating information along the pathway to pass to the higher hierarchy. The process is reversed in the last step.
$(\mathbf{X}^0)^{\mathcal{M}\mathcal{G}}$ . In the process stage, the Mee layer, which performs both local and hierarchical information aggregation within the mega graph, is stacked  $n$  times. The  $i$ -th Mee layer receives  $(\mathbf{X}^{i - 1})^{\mathcal{M}\mathcal{G}}$  as input and outputs  $(\mathbf{X}^i)^{\mathcal{M}\mathcal{G}}$ . Through the stacking of Mee layers, a deeper architecture is created, enabling a more profound fusion of local and hierarchical information. Lastly, in the decode stage, the features  $(\mathbf{X}^n)^{\mathcal{M}\mathcal{G}}$  are transformed into task-specific representations using readout functions.
Mee Layer. The Mee layer is designed to aggregate local and hierarchical information within the mega graph. A detailed structure of the Mee layer is depicted in Fig. 3.
For the  $i$ -th Mee layer, we consider inputs denoted by  $(\mathbf{X}^{i-1})^{\mathcal{MG}} = \{(\mathbf{X}^{i-1})^{\mathcal{G}_{1:h}}, (\mathbf{X}^{i-1})^{\hat{\mathcal{G}}_{1:h}}\}$ . For simplicity, we omit the superscript and denote the features of intra- and inter-graphs as  $\{\mathbf{X}_j^{i-1}\}_{j=1,\dots,h} := (\mathbf{X}^{i-1})^{\mathcal{G}_{1:h}}$  and  $\{\hat{\mathbf{X}}_j^{i-1}\}_{j=1,\dots,h-1} := (\mathbf{X}^{i-1})^{\hat{\mathcal{G}}_{1:h}}$  respectively.
The first step applies GN blocks on intra-graph edges, performing message passing on the local structure of graphs at each scale:  $\mathbf{X}_j^{\prime} = \mathrm{GN}_{\mathrm{intra}}^{i,j}(\mathcal{G}_j,\mathbf{X}_j^{i - 1})$ . Here,  $\mathbf{X}_j^{\prime}$  represents the updated intra-graph  $\mathcal{G}_j$  features.
The second and third steps focus on multi-scale information fusion. The second step applies cross-updates across consecutive heights from 1 to  $h$ , while the third step reverses the process, forming a bidirectional pathway for the information flow across the hierarchy. The cross-update between consecutive heights  $j$  and  $j + 1$  is denoted by a function  $(\mathbf{X}_j', \hat{\mathbf{X}}_j', \mathbf{X}j + 1') = \mathbf{X} - \mathrm{UPD}(j, \mathbf{X}_j, \hat{\mathbf{X}}_j, \mathbf{X}j + 1)$ . The prime notation indicates the updated value, and residual links [26] are used in practice.
This cross-update can be implemented via an inter-graph convolution with  $\mathrm{GN}_{\mathrm{inter}}^{i,j}$ , referred to as  $X$ -Conv (detailed in App.C.1). Alternatively, it can be realized using the REDUCE and EXPAND operations of POOL (refer to Sec.2.2) by  $\mathbf{X}_{j+1}^{\prime} = \mathrm{REDUCE}(\hat{\mathcal{G}}_j, \hat{\mathbf{X}}_j^0, \mathbf{X}_j)$  and  $\mathbf{X}_j^{\prime} = \mathrm{EXPAND}(\hat{\mathcal{G}}_j, \hat{\mathbf{X}}_j^0, \mathbf{X}_{j+1})$ , where  $\hat{\mathcal{G}}_j$  is the  $j$ -th inter-graph. We denote this implementation as  $X$ -Pool.
The Mee layer outputs features  $\{\mathbf{X}_j^i\}_{j = 1,\dots ,h}$  and  $\{\hat{\mathbf{X}}_j^i\}_{j = 1,\dots ,h - 1}$ . Residual links [26] can be added from  $\mathbf{X}_j^{i - 1}$  to  $\mathbf{X}_j^i$  and from  $\hat{\mathbf{X}}_j^{i - 1}$  to  $\hat{\mathbf{X}}_j^i$  empirically, creating shortcuts that bypass GN blocks in the Mee layer. It's worth noting that the intra and inter GN blocks can share parameters across all heights  $j$  to accommodate varying heights, or across all Mee layers to handle varying layer numbers.
3.3 Module Choice and Innovation
MeGraph incorporates two fundamental modules: the graph pooling operator and the GN block. This architecture can accommodate any graph pooling method from the P00L function family (refer to Sec. 2.2). Furthermore, the GN block is not strictly confined to the graph convolution layer found in standard GCN, GIN, or GAT.
Graph Pooling. There are a number of commonly used graph pooling methods, including DiffPool [62], TopKPool [22], EdgePool [13], etc. We opt for EdgePool due to its simplicity, efficiency,
and ability to naturally preserve the graph's connectivity through edge contraction. However, edge contraction is applied only to the edges in a specific maximal matching of the graph's nodes [13], thereby setting a lower limit of  $50\%$  to the pooling ratio  $\eta_v$ . This constraint implies that a minimum of  $\log_2N$  pooling operations is required to reduce a graph of  $N$  nodes to a single node. To address this limitation, we propose the Stridden EdgePool (S-EdgePool), which allows for a variable pooling stride.
The principle behind S-EdgePool involves dynamically tracking the clusters of nodes created by the contraction of selected edges. Similar to EdgePool, edges are processed in descending order based on their scores. When an edge is contracted, if both ends do not belong to the same node cluster, the two clusters containing the endpoints of the edge merge. The current edge can be contracted if the resulting cluster contains no more than  $\tau_{c}$  nodes after this edge's contraction. The iteration stops prematurely once a pooling ratio,  $\eta_v$ , is achieved. During pooling, each node cluster is pooled as a new node. When  $\tau_{c} = 2$ , S-EdgePool reverts to the original EdgePool. The algorithm's details and pseudocode are available in App. C.2.
For efficiency, we employ the disjoint-set data structure to dynamically maintain the node clusters, which has a complexity of  $O(E\alpha(E))$ , where  $E$  is the number of edges and  $\alpha(E)$  is a function that grows slower than  $\log(E)$  [50]. The total time complexity of S-EdgePool is equivalent to EdgePool and is calculated as  $O(ED + E\log E)$ , where  $D$  is the embedding size,  $O(ED)$  from computing edge scores and  $O(E\log E)$  from sorting the edges.