A Comprehensive Graph Pooling Benchmark: Effectiveness, Robustness and Generalizability

## Abstract

Abstract Graph pooling has gained attention for its ability to obtain effective node and graph representations for various downstream tasks. Despite the recent surge in graph pooling approaches, there is a lack of standardized experimental settings and fair benchmarks to evaluate their performance. To address this issue, we have constructed a comprehensive benchmark that includes 17 graph pooling methods and 28 different graph datasets. This benchmark systematically assesses the performance of graph pooling methods in three dimensions, i.e., effectiveness, robustness, and generalizability. We first evaluate the performance of these graph pooling approaches across different tasks including graph classification, graph regression and node classification. Then, we investigate their performance under potential noise attacks and out-of-distribution shifts in real-world scenarios. We also involve detailed efficiency analysis, backbone analysis, parameter analysis and visualization to provide more evidence. Extensive experiments validate the strong capability and applicability of graph pooling approaches in various scenarios, which can provide valuable insights and guidance for deep geometric learning research. The source code of our benchmark is available at https://github.com/goose315/Graph_Pooling_Benchmark .

## 1 Introduction

Recently, graph neural networks (GNNs) have garnered significant attention with extensive benchmarks (Tan et al., ; Li et al., ; Hu et al., ) due to their remarkable ability to process graph-structured data across various domains including social networks (Wu et al., ; Yang et al., ; Zhang et al., ), rumor detection (Bian et al., ; Zhang et al., ), biological networks (Wu et al., ; Choi et al., ), recommender systems (Ma et al., ) and community detection (Alsentzer et al., ; Sun et al., ). Graph pooling approaches play a crucial role in GNNs by enabling the hierarchical reduction of graph representations, which is essential for capturing multi-scale structures and long-range dependencies (Liu et al., ; Wu et al., ; Dwivedi et al., ). They can preserve crucial topological semantics and relationships, which have shown effective for tasks including graph classification, node clustering, and graph generation (Liu et al., ; ; Grattarola et al., ; Li et al., ). In addition, by aggregating nodes and edges, graph pooling can also simplify large-scale graphs, facilitating the application of GNNs in real-world problems (Defferrard et al., ; Ying et al., ; Mesquita et al., ; Zhang et al., ; Tsitsulin et al., ). Therefore, understanding and enhancing graph pooling approaches is the key to increasing GNN performance, driving the progress of deep geometric learning.

In literature, existing graph pooling approaches can be roughly divided into two categories (Bianchi & Lachi, ; Liu et al., ), i.e., node dropping pooling (Knyazev et al., ; Lee et al., ; Ranjan et al., ; Ma et al., ; Zhang et al., ; Zhou et al., ; Pang et al., ; Bacciu et al., ; Zhang et al., ; ; Song et al., ) and node clustering pooling approaches (Ying et al., ; Bianchi et al., ; Duval & Malliaros, ; Wu et al., ; Hansen & Bianchi, ; Tsitsulin et al., ; Bianchi, ), based on the strategies used to simplify the graph. node dropping pooling utilizes a learnable scoring function to remove nodes with relatively low significance scores, resulting in lower computational costs, while node clustering pooling approaches typically treat graph pooling as a node clustering problem, where clusters are considered as new nodes for the coarsened graph (Liu et al., ; Bianchi & Lachi, ). Even though graph pooling research is becoming increasingly popular, there is still no standardized benchmark that allows for an impartial and consistent comparison of various graph pooling methods. Furthermore, due to the diversity and complexity of graph datasets, numerous experimental settings have been used in previous studies, such as varied proportions of training data and train/validation/test splits (Bian et al., ; Hansen & Bianchi, ; Dwivedi et al., ; Xu et al., ).
As a result, a comprehensive and publicly available benchmark of graph pooling approaches is highly expected that can facilitate the evaluation and comparison of different approaches, ensuring the reproducibility of results and further advancing the area of graph machine learning.

Towards this end, we present a comprehensive graph pooling benchmark, which includes 17 graph pooling methods and 28 datasets across different graph machine learning problems. In particular, we extensively investigate graph pooling approaches across three key perspectives, i.e., effectiveness, robustness, and generalizability. To begin, we provide a fair and thorough effectiveness comparison of existing graph pooling approaches across graph classification, graph regression and node classification. Then, we evaluate the robustness of graph pooling approaches under both noise attacks on graph structures and node attributes. In addition, we study the generalizability of different approaches under out-of-distribution shifts from both size and density levels. Finally, we include efficiency comparison, parameter analysis and backbone analysis for completeness.

Based on extensive experimental results, we have made the following observations: (1) Node clustering pooling methods outperform node dropping pooling methods in terms of robustness, generalizability, and performance on graph regression tasks. (2) Node clustering pooling methods incur higher computational costs, and both approaches exhibit comparable performance on graph classification tasks. (3) AsymCheegerCutPool and ParsPool demonstrate strong performance in graph classification tasks. (4) As the scale of graph data decreases, the performance gap between different graph pooling methods in node classification tasks increases, with KMISPool and ParsPool exhibiting outstanding performance. (5) Most graph pooling approaches experience significant performance degradation due to distribution shifts and are also challenged by class imbalance issues, but the extent of this impact varies across different datasets. (6) Node clustering pooling is relatively superior to node dropping pooling in terms of robustness and generalizability, while KMISPool demonstrates relatively better robustness and generalizability in node dropping pooling methods.

The main contributions of this paper are as follows:

- •
Comprehensive Benchmark. We present a comprehensive graph pooling benchmark, which incorporates 17 state-of-the-art graph pooling approaches and 28 diverse datasets across graph classification, graph regression, and node classification.
- •
Extensive Analysis. To investigate the pros and cons of graph pooling approaches, we thoroughly evaluate current approaches from three perspectives, i.e., effectiveness, robustness, and generalizability, which can serve as guidance for researchers in different applications.
- •
Open-source Material. We have made our benchmark of all these graph pooling approaches available and reproducible, and we believe our benchmark can benefit researchers in both graph machine learning and interdisciplinary fields.

## 2 Preliminaries

Notations. Consider a graph $G$ characterized by a vertex set $V$ and an edge set $E$. The features associated with each vertex are represented by the matrix $\bm{X}\in\mathbb{R}^{|V|\times d}$, where $|V|$ denotes the number of vertices, and $d$ signifies the dimensionality of the attribute vectors. The adjacency relationships within the graph are encapsulated by the adjacency matrix $\bm{A}\in\{0,1\}^{|V|\times|V|}$, where an entry $\bm{A}[i,j]=1$ indicates the presence of an edge between vertex $v_{i}$ and vertex $v_{j}$; otherwise, $\bm{A}[i,j]=0$.

Graph Pooling (Liu et al., ; Grattarola et al., ; Bianchi & Lachi, ). The aim of graph pooling is to reduce the spatial size of feature maps while preserving essential semantics, which thereby decreases computational complexity and memory usage. In this work, we focus on hierarchical pooling approaches (Ying et al., ). Let POOL
denote a graph pooling function which maps $G$ to a graph $G^{\prime}=(V^{\prime},E^{\prime})$ with the reduced size:

$$ ${G}^{\prime}=\texttt{POOL}({G})\,,$ (1) $$

where $|V^{\prime}|<|V|$. the process has two main principal components, i.e., reduction, and connection (Grattarola et al., ). In particular, the reduction component aims to generate pooled nodes and their attributes in ${G}^{\prime}$ during the connection. The connection component computes the edges $E^{\prime}$ among the $V^{\prime}$ nodes.

Figure: Figure 1: Overview of our hierarchical backbone for graph classification and graph regression.
Refer to caption: x1.png

Graph Classification and Regression (Knyazev et al., ; Chen et al., ; Grattarola et al., ). The two primary graph-level tasks are graph regression and graph classification. Here, a graph dataset $\mathcal{G}$ is provided as a set of graph-label pairs $(G_{i},y_{i})$, where $y_{i}$ denotes the label for graph $G_{i}$. The objective is to train a powerful discriminative model $f$ that predicts the correct label $y_{i}$ given an input graph $G_{i}$. In graph classification, $y_{i}$ are categorical labels ${1,\cdots,K}$ with $K$ as the number of classes, while in graph regression, $y_{i}$ are continuous values. A well-trained graph classification model should output labels that closely match the true labels, and similarly, a graph regression model should predict values that are nearly identical to the ground truth values. In these tasks, graph pooling always accompanies graph convolutional operators. In formulation, the basic updating rule is written as follows:

$$ $\bm{H}^{(l+1)}=\sigma(\tilde{\bm{D}}^{-\frac{1}{2}}\tilde{\bm{A}}\tilde{D}^{- \frac{1}{2}}\bm{H}^{(l)}\bm{W}^{(l)})\,,$ (2) $$

where $\bm{H}^{(l)}$ denotes the node feature matrix at layer $l$, $\bm{W}^{(l)}$ denotes the weight matrix at the corresponding layer, $\tilde{\bm{A}}=\bm{A}+\bm{I}$ is the adjacency matrix $\bm{A}$ plus the identity matrix $\bm{I}$, $\tilde{\bm{D}}$ is the degree matrix of $\tilde{\bm{A}}$, and $\sigma$ is a nonlinear activation function (Kipf & Welling, ). The pooling layers can be formulated as:

$$ $\bm{H}^{(pool)}=\texttt{POOL}(\bm{H}^{(L)})\,,$ (3) $$

where $\bm{H}^{(pool)}$ is the node feature matrix after pooling
We iteratively conduct graph convolution and graph pooling operators and adopt a readout function to output the graph representation for downstream tasks. The overview of the basic hierarchical backbone can be found in Figure 1.

Node Classification (Kipf & Welling, ; Veličković et al., ; Perozzi et al., ). The aim of node classification is to assign semantic labels to nodes in a graph according to their attributes and relationships with different nodes. Each dataset involves a graph $G$, consisting of nodes $v_{i}$ and their corresponding labels $y_{i}$. $|V|$ is divided into a labeled set $V^{l}$ and a unlabeled set $V^{u}$. We are required to train a graph neural network model that can predict the missing labels of nodes in $V^{u}$ using the attributes of other nodes. U-Net framework (Ronneberger et al., ) is widely used to incorporate pooling operations for node classification. In the encoder part, U-Net progressively applies pooling and graph convolution to downsample the graphs and extract multi-scale features. The decoder part of U-Net utilizes upsampling and graph convolution to gradually upsample the low-resolution feature maps back to the original graph size. Residual connections are employed to directly transfer the feature maps from the encoder to the decoder, facilitating the preservation of fine-grained semantics during upsampling (Ronneberger et al., ; Ibtehaz & Rahman, ; Leng et al., ). For more information about the U-Net, please refer to Appendix B.

## 3 Graph Pooling Benchmark

### 3.1 Graph Pooling Approaches

Our benchmark contains 17 state-of-the-art graph pooling approaches (detailed in Table 1): TopKPool (Knyazev et al., ), SAGPool (Lee et al., ), ASAPool (Ranjan et al., ), PANPool (Ma et al., ), COPool (Zhou et al., ), CGIPool (Pang et al., ), KMISPool (Bacciu et al., ), GSAPool (Zhang et al., ), HGPSLPool (Zhang et al., ), AsymCheegerCutPool (Hansen & Bianchi, ), DiffPool (Ying et al., ), MincutPool (Bianchi et al., ), DMoNPool (Tsitsulin et al., ), HoscPool (Duval & Malliaros, ), JustBalancePool (Bianchi, ), SEPool (Wu et al., ), and ParsPool (Song et al., ). More information related to the selected pooling methods can be found in the Appendix C.

**Table 1: Overview of experimental details of graph pooling research. These papers utilize different settings, which validates the necessity of building a comprehensive and fair benchmark.**
| Methods | Datasets | Tasks |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Node Dropping Pooling |  |  |  |  |  |  |  |  |  |
| TopKPool | NIPS’19 | MNIST, COLLAB, PROTEINS, D&D |  |  |  |  |  |  | Graph Classification |
| SAGPool | ICML’19 | D&D, PROTEINS, NCI1, NCI109, FRANKENSTEIN |  |  |  |  |  |  | Graph Classification |
| ASAPool | AAAI’20 | D&D, PROTEINS, NCI1, NCI109, FRANKENSTEIN |  |  |  |  |  |  | Graph Classification |
| PANPool | NIPS’20 | PROTEINS, PROTEINS_FULL, NCI1, AIDS, MUTAGENCITY |  |  |  |  |  |  | Graph Classification |
| COPool | ECMLPKDD’22 | BZR, AIDS, NCI1, NCI109, PROTEINS, QM7, IMDB-M |  |  |  |  |  |  | Graph Classification, Graph Regression |
| CGIPool | SIGIR’22 | NCI1, NCI109, MUTAG, IMDB-B, IMDB-M, COLLAB, PROTEINS |  |  |  |  |  |  | Graph Classification |
| KMISPool | AAAI’23 | D&D, REDDIT-B, REDDIT-5K, REDDIT-12K, Github |  |  |  |  |  |  | Graph Classification, Node Classification |
| GSAPool | WWW’20 | D&D, NCI1, NCI109, MUTAG |  |  |  |  |  |  | Graph Classification |
| HGPSLPool | Arxiv’19 | D&D, PROTEINS, NCI1, NCI109, ENZYMES, MUTAG |  |  |  |  |  |  | Graph Classification |
| Node Clustering Pooling |  |  |  |  |  |  |  |  |  |
| AsymCheegerCutPool | ICML’23 | Cora, Citeseer, Pubmed, DBLP |  |  |  |  |  |  | Node Classification |
| DiffPool | NIPS’18 | D&D, PROTEINS, COLLAB, ENZYMES, REDDIT-MULTI |  |  |  |  |  |  | Graph Classification |
| MincutPool | ICML’20 | D&D, PROTEINS, COLLAB, REDDIT-B, MUTAG, QM9 |  |  |  |  |  |  | Graph Classification, Graph Regression |
| DMoNPool | JMLR’23 | Cora, Citeseer Pubmed, Coauthor |  |  |  |  |  |  | Node Classification |
| HoscPool | CIKM’22 | Cora, Citeseer Pubmed, Coauthor, DBLP, Email-EU |  |  |  |  |  |  | Node Classification |
| JustBalancePool | Arxiv’22 | Cora, Citeseer, Pubmed, DBLP |  |  |  |  |  |  | Node Classification |
| SEPool | ICML’22 | IMDB-B, IMDB-M, COLLAB, MUTAG, Cora, Citeseer, Pubmed |  |  |  |  |  |  | Graph Classification, Node Classification |
| ParsPool | ICLR’24 | D&D, PROTEINS, NCI1, NCI109, Ogbg-molpcba, Cora, Citeseer, Pubmed |  |  |  |  |  |  | Graph Classification, Node Classification |

### 3.2 Datasets

To systematically evaluate graph pooling methods, we integrate 28 datasets from different domains. For graph classification, we select eleven publicly available datasets from TUDataset (Morris et al., ), including seven molecules datasets, i.e., MUTAG (Debnath et al., ), NCI1 (Wale et al., ), NCI109 (Wale et al., ), COX2 Sutherland et al. (), AIDS Riesen & Bunke (), FRANKENSTEIN Orsini et al. (), and Mutagenicity Debnath et al. (), four bioinformatics datasets, i.e. D&D (Shervashidze et al., ), PROTEINS (Borgwardt et al., ), PROTEINS_FULL (Borgwardt et al., ), and ENZYMES (Schomburg et al., ), three social network dataset, i.e., IMDB-BINARY (IMDB-B) (Cai & Wang, ), IMDB-MULTI (IMDB-M) (Cai & Wang, ), and COLLAB (Cai & Wang, ). We also include a large-scale graph classification dataset, Ogbg-molpcba, from the Open Graph Benchmark (OGB) (Hu et al., ). For graph regression, we choose six datasets from MoleculeNet (Wu et al., ) including QM7, QM8, BACE, ESOL, FreeSolv, and Lipophilicity. For node classification, we utilize three citation networks, i.e., Cora, Citeseer, and Pubmed (Yang et al., ), three website networks, i.e., Cornell, Texas, and Wisconsin (Pei et al., ), and the GitHub dataset (Rozemberczki et al., ).
We also obtain a large-scale dataset, Ogbn-arxiv, from OGB (Hu et al., ). More information about the summary statistics and description of the datasets are detailed in the Appendix D.

### 3.3 Evaluation Protocols

Our benchmark evaluation encompasses three key aspects, i.e., effectiveness, robustness, and generalizability. We perform a hyperparameter search for all pooling methods; detailed information can be found in Appendix E. Firstly, we conduct a performance comparison of graph pooling approaches across three tasks including graph classification, graph regression, and node classification. For graph and node classification tasks, we employ average precision for Ogbg-molpcba, and accuracy for remaining datasets as the evaluation metric. For graph regression, we use root mean square error (RMSE) for ESOL, FreeSolv, and Lipophilicity (Wu et al., ). Following previous research (Xu et al., ), we use the area under the receiver operating characteristic (AUROC) curve to evaluate BACE, and mean absolute error (MAE) for QM7 and QM8. Secondly, our benchmark evaluates the robustness of graph pooling approaches in both graph-level and node-level tasks across two perspectives: structural robustness and feature robustness (Li & Wang, ). In particular, we add and drop edges of graphs to study structural robustness and mask node features to investigate feature robustness. Thirdly, we employ size-based and density-based distribution shifts to evaluate the generalizability of different pooling methods in graph-level tasks under real-world scenarios (Gui et al., ). We also use degree-based and closeness-based distribution shifts to assess the generalizability of different pooling methods in node-level tasks. In addition to these three views, we conduct a further analysis of these graph pooling approaches including the comparison of efficiency, visualization, and different backbone parameter choices.

## 4 Experiment

**Table 2: Results of graph classification for different graph pooling methods. The mean and variance of average precision (Ogbg-molpcba) and accuracy (remaining datasets) are reported. The best and 2nd best are noted in bold font and underlined, respectively. OOM denotes out of GPU memory, and OOT denotes cannot be computed within 24 hours.**
| Methods | Ogbg-molpcba |  | PROTEINS |  | NCI1 |  | NCI109 |  | MUTAG |  | D&D |  | IMDB-B |  | IMDB-M |  | COLLAB | Avg. | Rank |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Node Drop Pooling |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| TopKPool | 15.79$\pm$0.42 |  | 70.83$\pm$1.25 |  | 70.34$\pm$1.80 |  | 69.65$\pm$1.61 |  | 82.76$\pm$4.88 |  | 69.07$\pm$5.52 |  | 74.44$\pm$3.71 |  | 48.44$\pm$3.46 |  | 75.38$\pm$1.13 | 70.11 | 12.44 |
| SAGPool | 21.08$\pm$2.19 |  | 74.64$\pm$1.53 |  | 73.10$\pm$1.21 |  | 71.29$\pm$0.82 |  | 81.61$\pm$5.86 |  | 73.27$\pm$1.12 |  | 75.33$\pm$3.31 |  | 48.74$\pm$3.09 |  | 77.91$\pm$2.22 | 71.99 | 8.06 |
| ASAPool | OOT |  | 73.69$\pm$1.48 |  | 73.48$\pm$1.03 |  | 70.45$\pm$0.84 |  | 72.41$\pm$10.15 |  | OOT |  | 71.56$\pm$3.46 |  | 46.96$\pm$3.72 |  | OOT | 68.09 | 13.56 |
| PANPool | 21.81$\pm$0.99 |  | 70.60$\pm$1.67 |  | 73.29$\pm$1.07 |  | 70.84$\pm$1.23 |  | 78.16$\pm$8.60 |  | 73.27$\pm$4.05 |  | 73.33$\pm$3.57 |  | 47.70$\pm$3.58 |  | 78.40$\pm$2.80 | 70.70 | 11.00 |
| COPool | 25.50$\pm$2.46 |  | 75.24$\pm$2.46 |  | 74.10$\pm$1.06 |  | 71.35$\pm$1.05 |  | 83.91$\pm$3.25 |  | 73.57$\pm$0.42 |  | 74.44$\pm$4.40 |  | 48.89$\pm$3.82 |  | 81.33$\pm$1.15 | 72.85 | 5.28 |
| CGIPool | 23.78$\pm$6.71 |  | 73.57$\pm$1.49 |  | 75.72$\pm$1.65 |  | 73.81$\pm$0.42 |  | 86.21$\pm$4.88 |  | 72.07$\pm$1.47 |  | 74.22$\pm$3.62 |  | 46.22$\pm$2.02 |  | 80.40$\pm$1.71 | 72.78 | 8.22 |
| KMISPool | 26.85$\pm$0.28 |  | 70.63$\pm$1.01 |  | 73.15$\pm$2.19 |  | 73.17$\pm$1.10 |  | 80.46$\pm$4.30 |  | 70.57$\pm$1.70 |  | 72.89$\pm$3.62 |  | 46.96$\pm$2.47 |  | 80.71$\pm$0.49 | 71.07 | 9.83 |
| GSAPool | 26.95$\pm$1.36 |  | 72.14$\pm$1.09 |  | 71.12$\pm$1.33 |  | 70.65$\pm$1.45 |  | 87.36$\pm$1.63 |  | 72.97$\pm$1.27 |  | 74.67$\pm$3.93 |  | 46.37$\pm$4.13 |  | 76.84$\pm$2.11 | 71.52 | 9.28 |
| HGPSLPool | 22.78$\pm$0.51 |  | 72.02$\pm$1.73 |  | 72.22$\pm$0.42 |  | 70.35$\pm$1.31 |  | 71.26$\pm$12.70 |  | 73.27$\pm$2.78 |  | 72.89$\pm$4.37 |  | 46.81$\pm$2.19 |  | 79.24$\pm$0.80 | 69.76 | 12.17 |
| Node Clustering Pooling |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| AsymCheegerCutPool | 24.82$\pm$0.60 |  | 74.60$\pm$1.96 |  | 75.90$\pm$1.69 |  | 73.98$\pm$1.88 |  | 89.66$\pm$2.82 |  | 74.47$\pm$2.36 |  | 74.89$\pm$3.14 |  | 48.30$\pm$3.63 |  | 80.62$\pm$1.06 | 74.05 | 4.72 |
| DiffPool | 25.21$\pm$0.42 |  | 74.80$\pm$1.71 |  | 74.72$\pm$1.82 |  | 75.16$\pm$0.35 |  | 80.46$\pm$5.86 |  | 73.57$\pm$1.85 |  | 74.44$\pm$0.63 |  | 47.70$\pm$4.01 |  | 78.89$\pm$0.55 | 72.47 | 6.50 |
| MincutPool | 24.97$\pm$0.41 |  | 72.42$\pm$1.71 |  | 75.53$\pm$1.15 |  | 74.30$\pm$1.33 |  | 85.06$\pm$1.63 |  | 71.77$\pm$2.12 |  | 73.78$\pm$3.94 |  | 45.93$\pm$2.55 |  | 76.53$\pm$1.60 | 71.91 | 9.56 |
| DMoNPool | 24.75$\pm$0.67 |  | 68.45$\pm$4.79 |  | 72.45$\pm$0.15 |  | 71.18$\pm$1.66 |  | 75.86$\pm$5.63 |  | 75.38$\pm$0.42 |  | 73.33$\pm$3.81 |  | 47.26$\pm$1.68 |  | 77.07$\pm$0.44 | 70.12 | 11.17 |
| HoscPool | 24.63$\pm$0.37 |  | 72.42$\pm$0.74 |  | 76.88$\pm$0.61 |  | 76.13$\pm$1.97 |  | 85.06$\pm$1.63 |  | 71.77$\pm$2.12 |  | 74.67$\pm$1.96 |  | 45.93$\pm$2.77 |  | 78.18$\pm$1.69 | 72.63 | 8.06 |
| JustBalancePool | 25.19$\pm$0.42 |  | 68.85$\pm$2.97 |  | 76.34$\pm$0.46 |  | 76.34$\pm$1.51 |  | 81.61$\pm$1.63 |  | 71.77$\pm$2.12 |  | 74.89$\pm$4.09 |  | 45.93$\pm$5.08 |  | 77.87$\pm$1.26 | 71.70 | 8.67 |
| SEPool | OOT |  | 62.25$\pm$4.57 |  | 62.77$\pm$2.25 |  | 63.74$\pm$2.30 |  | 67.22$\pm$8.41 |  | 80.26$\pm$3.04 |  | 77.00$\pm$4.05 |  | 54.13$\pm$3.71 |  | 75.64$\pm$2.04 | 67.88 | 11.39 |
| ParsPool | 26.63$\pm$0.30 |  | 75.02$\pm$0.64 |  | 77.07$\pm$0.23 |  | 76.20$\pm$0.44 |  | 79.31$\pm$5.63 |  | 76.10$\pm$0.80 |  | 75.11$\pm$2.20 |  | 49.48$\pm$0.91 |  | 83.60$\pm$0.50 | 73.99 | 3.11 |

### 4.1 Experimental Settings

All graph pooling methods in our benchmark are implemented by PyTorch (Paszke et al., ). Graph convolutional networks serve as the default encoders for all algorithms. The experimental setup includes a Linux server equipped with NVIDIA A100 GPUs, with an Intel Xeon Gold 6354 CPU. The software stack comprises PyTorch 1.11.0, PyTorch-geometric 2.1.0 (Fey & Lenssen, ), and Python 3.9.16. More details about experimental settings can be found in Appendix E.

### 4.2 Effectiveness Analysis

Performance on Graph Classification. To begin, we investigate the performance of different graph pooling approaches on graph classification. The results of compared approaches on seven popular datasets are recorded in Table 2. Firstly, in general, ParsPool, AsymCheegerCutPool, and COPool are the three best-performing pooling models, and the performance of all 17 pooling methods varies significantly across different datasets. No single pooling method consistently outperforms the others across all datasets. Secondly, it is noteworthy that SEPool achieves significant advantages on D&D, IMDB-B, and IMDB-M. This is because SEPool’s coding tree structure, and these datasets are characterized by high clustering coefficients Watts & Strogatz (), which benefits the coding tree method because the hierarchical nature of the tree can better capture and represent these localized, highly connected substructures (Wu et al., ). However, SEPool also implies greater computational resource overhead, which presents challenges when processing large-scale graph data such as Ogbg-molpcba. Thirdly, the methods with the highest average accuracy are ParsPool and AsymCheegerCutPool. ParsPool can capture a personalized pooling structure for each individual graph, while AsymCheegerCutPool calculates cluster assignments based on a tighter relaxation in terms of Graph Total Variation (GTV) (Song et al., ; Hansen & Bianchi, ). These methods are flexible, and the datasets differ significantly in diameter, degree, and clustering coefficients.

**Table 3: Results of graph regression for different pooling methods. The mean and variance of MAE (QM7, QM8), AUROC (BACE), RMSE (ESOL, FreeSolv, Lipophilicity) are reported. - denotes cannot converge.**
| Methods | QM7 |  | QM8 |  | BACE |  | ESOL |  | FreeSolv |  | Lipophilicity | Rank |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Node Drop Pooling |  |  |  |  |  |  |  |  |  |  |  |  |
| TopKPool | 63.39$\pm$9.66 |  | 0.021$\pm$0.001 |  | 0.85$\pm$0.02 |  | 0.96$\pm$0.06 |  | 1.92$\pm$0.37 |  | 0.80$\pm$0.02 | 8.1 |
| SAGPool | 97.69$\pm$11.19 |  | 0.023$\pm$0.001 |  | 0.84$\pm$0.01 |  | 1.16$\pm$0.07 |  | 2.31$\pm$0.66 |  | 0.93$\pm$0.06 | 13.1 |
| ASAPool | 56.79$\pm$6.17 |  | 0.029$\pm$0.008 |  | 0.85$\pm$0.02 |  | 0.92$\pm$0.03 |  | 1.92$\pm$0.37 |  | 0.78$\pm$0.05 | 8.2 |
| PANPool | 53.04$\pm$1.20 |  | 0.015$\pm$0.000 |  | 0.83$\pm$0.02 |  | 1.01$\pm$0.03 |  | 1.80$\pm$0.10 |  | 0.84$\pm$0.01 | 9.7 |
| COPool | 84.22$\pm$3.28 |  | 0.020$\pm$0.001 |  | 0.85$\pm$0.01 |  | 0.98$\pm$0.07 |  | 1.85$\pm$0.24 |  | 0.85$\pm$0.02 | 8.5 |
| CGIPool | 97.41$\pm$16.25 |  | 0.020$\pm$0.001 |  | 0.84$\pm$0.03 |  | 1.59$\pm$0.62 |  | 2.49$\pm$0.97 |  | 0.83$\pm$0.07 | 11.7 |
| KMISPool | 80.51$\pm$21.34 |  | 0.017$\pm$0.001 |  | 0.85$\pm$0.02 |  | 0.95$\pm$0.04 |  | 1.29$\pm$0.18 |  | 0.81$\pm$0.03 | 5.9 |
| GSAPool | 106.72$\pm$22.90 |  | 0.021$\pm$0.001 |  | 0.85$\pm$0.02 |  | 0.96$\pm$0.08 |  | 1.95$\pm$0.26 |  | 0.82$\pm$0.03 | 8.8 |
| HGPSLPool | 47.88$\pm$0.83 |  | 0.015$\pm$0.000 |  | 0.84$\pm$0.01 |  | 1.02$\pm$0.06 |  | 1.62$\pm$0.09 |  | 0.76$\pm$0.01 | 7.8 |
| Node Clustering Pooling |  |  |  |  |  |  |  |  |  |  |  |  |
| AsymCheegerCutPool | 64.91$\pm$8.30 |  | 0.031 $\pm$ 0.005 |  | 0.84 $\pm$ 0.01 |  | 0.99 $\pm$ 0.12 |  | 2.00 $\pm$ 0.18 |  | 0.95 $\pm$ 0.11 | 12.9 |
| DiffPool | 54.98$\pm$3.44 |  | 0.037 $\pm$ 0.010 |  | 0.84 $\pm$ 0.02 |  | 0.81 $\pm$ 0.05 |  | 1.20 $\pm$ 0.09 |  | 0.73 $\pm$ 0.03 | 8.0 |
| MincutPool | - |  | 0.020 $\pm$ 0.001 |  | 0.85 $\pm$ 0.02 |  | 0.76 $\pm$ 0.02 |  | 1.19 $\pm$ 0.18 |  | 0.73 $\pm$ 0.02 | 4.3 |
| DMoNPool | - |  | 0.021 $\pm$ 0.001 |  | 0.85 $\pm$ 0.02 |  | 0.68 $\pm$ 0.02 |  | 1.16 $\pm$ 0.15 |  | 0.69 $\pm$ 0.02 | 3.5 |
| HoscPool | 59.44$\pm$21.48 |  | 0.019 $\pm$ 0.002 |  | 0.84 $\pm$ 0.01 |  | 0.76 $\pm$ 0.02 |  | 1.14 $\pm$ 0.13 |  | 0.72 $\pm$ 0.02 | 4.6 |
| JustBalancePool | - |  | 0.022 $\pm$ 0.004 |  | 0.85 $\pm$ 0.02 |  | 0.74 $\pm$ 0.03 |  | 1.26 $\pm$ 0.16 |  | 0.70 $\pm$ 0.01 | 4.9 |

Performance on Graph Regression. We further explore the performance of different pooling methods through graph-level regression tasks. As shown in Table 3, we can observe that: Firstly, overall, node clustering pooling methods outperform node dropping pooling methods, with DMoNPool and MincutPool showing the best performance. The possible reason is that in graph regression tasks, the model’s objective is to predict a continuous numerical output. Such tasks typically require capturing global structural features and continuity within the graph. Compared to node clustering pooling, node dropping pooling tends to lose more global information (Tsitsulin et al., ; Bianchi et al., ). DMoNPool and MincutPool are more inclined to maintain the global characteristics of the graph rather than emphasizing the representation of locally important structures, which may result in their performance being inferior to that of ParsPool, AsymCheegerCutPool, and COPool in graph classification tasks. Secondly, in the BACE dataset, the performance of most pooling methods tends to be consistent, whereas in other datasets, there is a greater variance in performance. The possible reason is that although the graphs in the BACE dataset are relatively large, the average diameter is relatively small, so different pooling methods face fewer challenges in summarizing the global structural information of the graphs, which may lead to more consistent performance.

Performance on Node Classification.
Table 4 presents the performance of various pooling methods in node classification tasks. We observe the following: Firstly, KMISPool and ParsPool demonstrate the best overall performance, significantly outperforming other methods on small-scale datasets such as Cornell, Texas, and Wisconsin. Secondly, node classification models without pooling layers achieve comparable results to most pooling methods across the majority of datasets. A potential reason for this is that pooling operations tend to lose substantial node information, which consequently weakens performance in node classification tasks. Thirdly, the scalability of ASAPool, PANPool, HGPSLPool, SEPool, and ParsPool still requires improvement, as they face memory/runtime bottlenecks, making it cannot complete training on larger datasets such as Ogbn-arxiv or GitHub.

**Table 4: Results of node classification for different pooling methods. No Pooling denotes without pooling layers.**
| Methods | Ogbn-arxiv |  | Cora |  | Citeseer |  | Pubmed |  | Cornell |  | Texas |  | Wisconsin |  | Github | Avg. | Rank |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| TopKPool | 53.36$\pm$0.03 |  | 88.91$\pm$0.93 |  | 77.56$\pm$0.85 |  | 86.13$\pm$0.34 |  | 49.09$\pm$2.57 |  | 54.18$\pm$4.80 |  | 51.58$\pm$3.05 |  | 86.95$\pm$0.20 | 67.91 | 8.00 |
| SAGPool | 53.39$\pm$0.02 |  | 89.18$\pm$0.65 |  | 77.56$\pm$0.81 |  | 86.07$\pm$0.70 |  | 81.09$\pm$3.74 |  | 55.64$\pm$2.95 |  | 51.32$\pm$3.53 |  | 86.99$\pm$0.22 | 73.48 | 5.33 |
| ASAPool | OOM |  | 89.10$\pm$0.86 |  | 77.76$\pm$1.01 |  | 85.74$\pm$0.18 |  | 79.64$\pm$2.91 |  | 54.55$\pm$4.74 |  | 50.79$\pm$3.49 |  | OOM | 72.93 | 6.83 |
| PANPool | OOM |  | 89.05$\pm$0.96 |  | 77.20$\pm$0.98 |  | 85.88$\pm$0.11 |  | 78.91$\pm$2.47 |  | 56.36$\pm$5.14 |  | 50.53$\pm$3.18 |  | OOM | 72.99 | 8.00 |
| COPool | OOM |  | 89.00$\pm$0.70 |  | 77.26$\pm$0.89 |  | 85.27$\pm$0.27 |  | 77.82$\pm$3.13 |  | 56.36$\pm$5.98 |  | 52.37$\pm$2.68 |  | 86.68$\pm$0.20 | 73.01 | 7.50 |
| CGIPool | 53.60$\pm$0.39 |  | 89.15$\pm$0.84 |  | 77.40$\pm$0.81 |  | 85.92$\pm$0.66 |  | 81.82$\pm$4.30 |  | 54.55$\pm$4.15 |  | 51.05$\pm$3.85 |  | 86.88$\pm$0.22 | 73.32 | 6.33 |
| KMISPool | 53.38$\pm$0.08 |  | 89.74$\pm$0.02 |  | 77.75$\pm$0.01 |  | 87.80$\pm$0.01 |  | 79.56$\pm$1.31 |  | 81.42$\pm$1.64 |  | 82.31$\pm$0.50 |  | 87.09$\pm$0.04 | 83.10 | 2.50 |
| GSAPool | 53.76$\pm$0.11 |  | 89.05$\pm$0.77 |  | 77.16$\pm$0.92 |  | 86.21$\pm$0.73 |  | 80.36$\pm$3.71 |  | 54.18$\pm$3.88 |  | 51.84$\pm$4.29 |  | 87.12$\pm$0.09 | 73.13 | 6.67 |
| HGPSLPool | OOM |  | 89.08$\pm$0.83 |  | 77.84$\pm$0.67 |  | OOM |  | 58.55$\pm$3.71 |  | 55.27$\pm$3.37 |  | 51.58$\pm$1.75 |  | OOM | 69.71 | 6.33 |
| SEPool | OOT |  | 83.17$\pm$0.00 |  | 70.47$\pm$0.00 |  | 79.33$\pm$0.80 |  | 51.35$\pm$0.00 |  | 66.66$\pm$6.50 |  | 57.51$\pm$0.85 |  | OOT | 68.08 | 8.67 |
| ParsPool | OOM |  | 84.51$\pm$0.35 |  | 74.27$\pm$0.38 |  | 89.20$\pm$0.00 |  | 72.07$\pm$6.49 |  | 81.98$\pm$6.49 |  | 82.35$\pm$17.94 |  | OOM | 80.73 | 5.50 |
| No Pooling | 53.61$\pm$0.07 |  | 89.48$\pm$0.27 |  | 77.69$\pm$0.24 |  | 86.10$\pm$0.06 |  | 48.83$\pm$1.24 |  | 56.50$\pm$1.11 |  | 54.43$\pm$1.54 |  | 86.46$\pm$0.02 | 68.84 | 5.17 |

**Table 5: Results of graph classification under random noise attack for different pooling methods.**
| Dataset | Ptb Method | TopKPool | SAGPool | ASAPool | PANPool | KMISPool | DiffPool | MincutPool | JustBalancePool |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| PROTEINS | ADD | 73.58$\pm$5.77 | 72.76$\pm$3.87 | 74.59$\pm$3.74 | 39.63$\pm$42.27 | 71.14$\pm$0.57 | 73.78$\pm$2.59 | 72.36$\pm$2.92 | 75.00$\pm$1.49 |
| DROP | 71.95$\pm$2.28 | 73.58$\pm$2.55 | 72.76$\pm$1.88 | 38.62$\pm$42.97 | 71.34$\pm$1.32 | 71.34$\pm$1.80 | 71.75$\pm$2.07 | 73.37$\pm$1.25 |  |
| MASK | 72.56$\pm$3.11 | 73.78$\pm$3.45 | 71.14$\pm$2.55 | 72.88$\pm$4.71 | 72.97$\pm$2.24 | 72.97$\pm$0.29 | 72.36$\pm$3.24 | 70.12$\pm$2.28 |  |
| NCI1 | ADD | 65.91$\pm$0.65 | 67.53$\pm$2.25 | 71.42$\pm$1.53 | 66.29$\pm$0.40 | 72.66$\pm$1.40 | 72.66$\pm$0.20 | 70.77$\pm$1.80 | 73.96$\pm$0.08 |
| DROP | 63.16$\pm$1.52 | 61.64$\pm$1.46 | 64.07$\pm$0.73 | 65.53$\pm$1.31 | 73.58$\pm$3.00 | 66.18$\pm$2.22 | 65.05$\pm$0.73 | 64.18$\pm$1.61 |  |
| MASK | 63.86$\pm$1.39 | 63.16$\pm$2.10 | 66.94$\pm$0.87 | 66.18$\pm$0.61 | 65.15$\pm$2.30 | 68.23$\pm$2.73 | 67.10$\pm$2.07 | 67.91$\pm$1.98 |  |
| NCI109 | ADD | 66.18$\pm$0.73 | 68.55$\pm$1.13 | 69.62$\pm$2.11 | 64.84$\pm$2.74 | 75.32$\pm$0.99 | 73.33$\pm$0.97 | 71.34$\pm$2.89 | 71.29$\pm$2.79 |
| DROP | 63.59$\pm$1.40 | 64.61$\pm$1.49 | 64.24$\pm$1.86 | 65.64$\pm$1.26 | 73.85$\pm$2.95 | 66.18$\pm$2.22 | 65.05$\pm$0.73 | 64.18$\pm$1.61 |  |
| MASK | 65.22$\pm$1.97 | 66.61$\pm$0.92 | 65.65$\pm$1.08 | 66.29$\pm$0.91 | 66.34$\pm$1.85 | 66.99$\pm$2.71 | 68.23$\pm$0.47 | 66.61$\pm$3.43 |  |
| MUTAG | ADD | 86.21$\pm$5.63 | 79.31$\pm$2.82 | 75.86$\pm$10.15 | 68.97$\pm$4.88 | 80.46$\pm$4.30 | 72.41$\pm$9.75 | 78.16$\pm$4.30 | 68.97$\pm$4.88 |
| DROP | 87.36$\pm$4.30 | 63.22$\pm$13.9 | 72.41$\pm$14.90 | 68.97$\pm$2.82 | 80.46$\pm$4.30 | 78.16$\pm$1.63 | 74.71$\pm$9.89 | 75.86$\pm$11.26 |  |
| MASK | 78.16$\pm$16.26 | 64.37$\pm$9.05 | 60.92$\pm$7.09 | 71.26$\pm$3.25 | 83.91$\pm$8.13 | 78.16$\pm$4.30 | 77.01$\pm$3.25 | 70.11$\pm$4.30 |  |

**Table 6: Results of node classification under random noise attack for different pooling methods.**
| Dataset | Ptb Method | TopKPool | SAGPool | ASAPool | PANPool | COPool | CGIPool | KMISPool | GSAPool | HGPSLPool |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Cora | ADD | 73.90$\pm$0.24 | 74.41$\pm$0.12 | OOM | 75.75$\pm$0.14 | 69.52$\pm$0.58 | 70.01$\pm$0.24 | 75.64$\pm$0.03 | 74.29$\pm$0.15 | 75.58$\pm$0.14 |
| DROP | 85.01$\pm$0.09 | 85.45$\pm$0.36 | 85.40$\pm$0.19 | 85.11$\pm$0.20 | 85.06$\pm$0.13 | 85.04$\pm$0.23 | 85.83$\pm$0.21 | 85.30$\pm$0.33 | 85.60$\pm$0.19 |  |
| MASK | 87.70$\pm$0.16 | 87.75$\pm$0.18 | 87.94$\pm$0.12 | 87.48$\pm$0.24 | 86.88$\pm$0.26 | 87.42$\pm$0.03 | 87.81$\pm$0.17 | 87.83$\pm$0.37 | 87.59$\pm$0.24 |  |
| Citeseer | ADD | 62.64$\pm$0.19 | 62.47$\pm$0.41 | 63.43$\pm$0.14 | 63.38$\pm$0.37 | 62.62$\pm$0.21 | 61.94$\pm$0.29 | 63.52$\pm$0.20 | 62.69$\pm$0.23 | 63.42$\pm$0.21 |
| DROP | 75.31$\pm$0.26 | 75.50$\pm$0.19 | 75.81$\pm$0.04 | 75.52$\pm$0.10 | 75.18$\pm$0.46 | 75.34$\pm$0.12 | 76.54$\pm$0.32 | 75.32$\pm$0.29 | 76.00$\pm$0.21 |  |
| MASK | 73.29$\pm$0.27 | 73.41$\pm$0.25 | 73.57$\pm$0.17 | 73.42$\pm$0.20 | 73.54$\pm$0.44 | 73.28$\pm$0.49 | 73.63$\pm$0.10 | 73.45$\pm$0.20 | 73.30$\pm$0.24 |  |
| Pubmed | ADD | 71.06$\pm$0.25 | 70.75$\pm$0.41 | OOM | 70.62$\pm$0.12 | 68.21$\pm$0.11 | 67.92$\pm$0.45 | 71.59$\pm$0.01 | 70.83$\pm$0.17 | OOM |
| DROP | 85.46$\pm$0.09 | 86.03$\pm$0.12 | OOM | 85.55$\pm$0.04 | 85.68$\pm$0.04 | 85.59$\pm$0.13 | 85.30$\pm$0.06 | 85.59$\pm$0.06 | OOM |  |
| MASK | 84.24$\pm$0.04 | 84.34$\pm$0.06 | OOM | 83.75$\pm$0.06 | 83.31$\pm$0.07 | 83.78$\pm$0.17 | 83.83$\pm$0.02 | 84.36$\pm$0.11 | OOM |  |

Figure: Figure 2: Performance of different approaches w.r.t. different rates of random noise.
Refer to caption: extracted/5895684/image/noise_rate1.png

### 4.3 Robustness Analysis

The compared performance for three types of random noise on eight graph pooling methods on the PROTEINS, NCI1, NCI109, and MUTAG datasets are shown in Table 5. With a probability of 50%, edges of the graph are randomly removed or added, and node features are randomly masked. We can observe that: Firstly, overall, node clustering pooling demonstrates better robustness against three types of attacks compared to node dropping pooling. Secondly, among node dropping pooling methods, KMISPool generally performs the best. However, for small datasets such as MUTAG, TopKPool achieves the highest performance under noise attacks, because its node selection mechanism is less sensitive to local noise variations (Knyazev et al., ). Thirdly, noise attacking increases the performance fluctuations of pooling methods, making their prediction results more unstable. Fourthly, in larger datasets such as PROTEINS, NCI1, and NIC109, dropping edges has a greater impact on performance, whereas for MUTAG, masking node features has a more significant effect.

Table LABEL:noise_node presents the results of the robustness analysis for node-level tasks. From Table LABEL:noise_node, we observe the following: Firstly, random attacks on the graph lead to a decrease in performance on node classification tasks, with different types of attacks causing varying degrees of performance degradation. Randomly adding edges has the most negative impact on performance, while randomly deleting edges has the least impact. Secondly, for larger graphs such as Cora, Citeseer, and Pubmed, KMISPool performs the best, whereas for smaller graphs such as Cornell, Texas, and Wisconsin, ASAPool performs better. Appendix F.1 provides results for the robustness analysis of node-level tasks.

As depicted in As shown in Figure 2, the model’s performance generally declines as the noise intensity increases. It is observed that, at the same level of noise, the impact on accuracy is more pronounced on smaller datasets Cora and CiteSeer, while it is relatively minor on larger dataset Pubmed. Among the three types of noise, although the accuracy of nearly all methods decreases amidst fluctuations, KMISPool and PANPool exhibit the strongest robustness, while COPool performs relatively poorly, despite the fact that most pooling methods show very similar performance.

**Table 7: Results of graph classification under distribution shifts. Size and density denote two types of shifts across training and test datasets. Micro-F1 and Macro-F1 metrics are provided for each shift type.**
| Method | D&D | NCI1 |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Size | Density | Size | Density |  |  |  |  |  |
| Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 |  |
| Node Drop Pooling |  |  |  |  |  |  |  |  |
| TopKPool | 68.08$\pm$1.60 | 63.69$\pm$1.13 | 31.98$\pm$15.37 | 29.43$\pm$15.16 | 25.89$\pm$0.46 | 24.98$\pm$0.43 | 53.48$\pm$2.11 | 51.02$\pm$3.14 |
| SAGPool | 81.36$\pm$2.49 | 74.47$\pm$1.48 | 55.37$\pm$0.89 | 50.14$\pm$0.97 | 25.08$\pm$2.39 | 23.90$\pm$2.95 | 47.00$\pm$3.60 | 45.86$\pm$3.49 |
| ASAPool | OOT | OOT | OOT | OOT | 26.29$\pm$3.66 | 25.29$\pm$4.42 | 53.17$\pm$1.85 | 51.34$\pm$1.18 |
| PANPool | 77.68$\pm$8.37 | 71.44$\pm$6.50 | 41.92$\pm$10.72 | 40.36$\pm$10.05 | 25.00$\pm$0.00 | 23.74$\pm$0.08 | 52.08$\pm$2.24 | 49.14$\pm$0.66 |
| COPool | 64.41$\pm$5.66 | 60.37$\pm$3.74 | 47.91$\pm$2.51 | 44.30$\pm$1.81 | 27.99$\pm$3.86 | 27.17$\pm$4.48 | 54.67$\pm$2.22 | 53.09$\pm$2.35 |
| CGIPool | 75.99$\pm$6.65 | 69.62$\pm$5.58 | 56.38$\pm$1.76 | 51.10$\pm$1.45 | 28.16$\pm$5.18 | 27.26$\pm$5.81 | 56.20$\pm$0.86 | 53.93$\pm$1.12 |
| KMISPool | 80.23$\pm$5.24 | 73.30$\pm$4.04 | 54.58$\pm$5.26 | 49.81$\pm$3.62 | 50.97$\pm$9.51 | 49.02$\pm$7.87 | 55.42$\pm$1.47 | 51.27$\pm$0.30 |
| GSAPool | 58.19$\pm$26.99 | 53.06$\pm$25.69 | 33.79$\pm$20.93 | 29.39$\pm$19.01 | 26.21$\pm$1.24 | 25.19$\pm$1.56 | 50.31$\pm$3.39 | 49.32$\pm$2.65 |
| HGPSLPool | 85.59$\pm$1.20 | 78.34$\pm$1.66 | 52.43$\pm$2.25 | 49.59$\pm$1.89 | 19.66$\pm$0.52 | 17.52$\pm$0.66 | 56.95$\pm$1.64 | 51.93$\pm$1.14 |
| Node Clustering Pooling |  |  |  |  |  |  |  |  |
| AsymCheegerCutPool | 74.47$\pm$0.06 | 73.60$\pm$0.06 | 86.13$\pm$0.00 | 50.86$\pm$0.01 | 48.87$\pm$3.06 | 45.42$\pm$1.65 | 70.01$\pm$0.00 | 46.95$\pm$0.00 |
| DiffPool | 73.87$\pm$0.02 | 73.35$\pm$0.02 | 86.49$\pm$0.02 | 47.44$\pm$0.01 | 19.50$\pm$0.00 | 16.63$\pm$0.01 | 69.53$\pm$0.00 | 50.87$\pm$0.15 |
| MincutPool | 69.97$\pm$0.17 | 67.95$\pm$0.17 | 87.39$\pm$0.00 | 46.63$\pm$0.00 | 19.58$\pm$0.00 | 16.69$\pm$0.00 | 68.64$\pm$0.00 | 50.31$\pm$0.09 |
| DMoNPool | 72.67$\pm$0.11 | 72.25$\pm$0.13 | 82.52$\pm$0.00 | 54.21$\pm$0.00 | 79.29$\pm$0.04 | 64.30$\pm$0.00 | 68.92$\pm$0.00 | 48.76$\pm$0.02 |
| HoscPool | 70.27$\pm$0.01 | 69.20$\pm$0.00 | 87.39$\pm$0.00 | 46.63$\pm$0.00 | 24.60$\pm$0.27 | 23.39$\pm$0.36 | 70.48$\pm$0.01 | 56.55$\pm$0.35 |
| JustBalancePool | 68.77$\pm$0.00 | 67.82$\pm$0.02 | 87.39$\pm$0.00 | 46.63$\pm$0.00 | 19.98$\pm$0.00 | 17.28$\pm$0.01 | 68.64$\pm$0.00 | 50.31$\pm$0.09 |
| ParsPool | 68.36$\pm$1.60 | 62.50$\pm$1.76 | 63.06$\pm$4.84 | 48.35$\pm$0.86 | 52.59$\pm$4.46 | 49.95$\pm$3.32 | 56.27$\pm$1.97 | 52.96$\pm$0.88 |

**Table 8: Results of node classification under distribution shifts. Degree and closeness denote two types of shifts across training and test datasets.**
| Method | Cora | Citeseer |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Degree | Closeness | Degree | Closeness |  |  |  |  |  |
| Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 |  |
| TopKPool | 83.65$\pm$0.50 | 82.29$\pm$0.47 | 83.21$\pm$0.18 | 81.98$\pm$0.31 | 67.27$\pm$0.35 | 63.60$\pm$0.26 | 72.28$\pm$0.67 | 65.64$\pm$1.00 |
| SAGPool | 84.19$\pm$0.12 | 82.73$\pm$0.27 | 81.68$\pm$1.54 | 80.34$\pm$1.55 | 65.46$\pm$0.61 | 62.22$\pm$0.49 | 72.28$\pm$1.10 | 66.35$\pm$1.87 |
| ASAPool | 83.16$\pm$0.00 | 82.24$\pm$0.13 | 84.10$\pm$0.37 | 82.97$\pm$0.32 | 67.47$\pm$0.23 | 63.94$\pm$0.31 | 72.84$\pm$0.49 | 65.27$\pm$1.74 |
| PANPool | 84.00$\pm$0.37 | 82.83$\pm$0.37 | 84.44$\pm$0.30 | 83.44$\pm$0.36 | 67.63$\pm$0.26 | 63.85$\pm$0.28 | 73.45$\pm$0.86 | 67.48$\pm$1.75 |
| COPool | 83.90$\pm$0.12 | 82.62$\pm$0.13 | 81.14$\pm$1.72 | 80.00$\pm$1.01 | 66.43$\pm$0.59 | 62.93$\pm$0.42 | 72.80$\pm$0.00 | 67.14$\pm$0.42 |
| CGIPool | 83.21$\pm$0.77 | 82.24$\pm$0.82 | 82.27$\pm$0.55 | 80.91$\pm$0.80 | 65.66$\pm$1.34 | 62.14$\pm$1.46 | 72.36$\pm$0.46 | 66.80$\pm$0.91 |
| KMISPool | 84.00$\pm$0.18 | 82.57$\pm$0.15 | 83.55$\pm$0.39 | 82.36$\pm$0.41 | 67.35$\pm$0.15 | 63.69$\pm$0.14 | 72.72$\pm$0.30 | 66.51$\pm$0.39 |
| GSAPool | 83.70$\pm$0.07 | 82.19$\pm$0.27 | 83.16$\pm$0.32 | 81.89$\pm$0.37 | 67.07$\pm$0.49 | 63.56$\pm$0.42 | 72.84$\pm$0.44 | 66.33$\pm$0.37 |
| HGPSLPool | 84.19$\pm$0.00 | 82.83$\pm$0.04 | 83.46$\pm$0.79 | 82.32$\pm$0.89 | 67.67$\pm$0.15 | 64.13$\pm$0.16 | 73.20$\pm$0.46 | 67.95$\pm$0.40 |

Figure: Figure 3: Comparison of performance, training time, and memory usage across different approaches.
Refer to caption: extracted/5895684/image/time_space.png

### 4.4 Generalizability Analysis

Table 7 and Table 8 presents the performance of different graph pooling methods under out-of-distribution shifts. For the graph-level datasets D&D and NCI1, we implement two types of distribution shifts. The first type is based on the number of nodes, where the smallest 50% of graphs by node count are used as the training set, and the largest 20% as the test set, with the remainder serving as the validation set (Bevilacqua et al., ; Chen et al., ). Following the same criteria, the second type of out-of-distribution shifts are generated based on graph density (Chen et al., ). For the node-level datasets Cora and Citeseer, the first type of out-of-distribution shift is the top 50% of nodes with the highest degrees as the training set, the bottom 25% with the lowest degrees as the test set, and the remaining nodes as the validation set. The second type is based on closeness centrality (the reciprocal of the sum of the shortest path lengths from a node to all other nodes). We use the 50% of nodes with the lowest closeness as the training set, the 25% with the highest closeness as the test set, and the remaining nodes as the validation set. For further details and more experiments for the generalizability analysis, please refer to the Appendix D.4 and Appendix F.2.

From Tables 7 and 8, we have the following observations. Firstly, node-level out-of-distribution shifts also reduce the performance of pooling models, but the extent of this reduction is smaller compared to out-of-distribution shifts in graph classification tasks. The potential reason is that, in node-level tasks, the propagation
of information are usually confined to the local neighborhood of nodes, whereas graph-level tasks require handling information spread over a larger scope. Secondly, Macro-F1 is generally lower than Micro-F1, which indicates that the model has weaker recognition capabilities for minority classes. Thirdly, node clustering pooling exhibits better generalizability than node dropping pooling in graph classification tasks. Fourthly, HGPSLPool and PANPool exhibit the best performance, potentially due to the fact that HGPSLPool combines graph convolution with spectral clustering, enabling it to better capture higher-order relationships and local topological structures, which is advantageous in node-level tasks. Meanwhile, PANPool utilizes an adaptive pooling strategy that adjusts the pooling method to suit different node feature distributions, enhancing the model’s robustness and generalization capability under out-of-distribution conditions.

Figure: Figure 4: The t-SNE visualization w.r.t. different pooling ratios of TopKPool and SAGPool.
Refer to caption: x2.png

### 4.5 Further Analysis

Efficiency Comparison. In this part, we conduct an efficiency analysis of graph pooling methods on the MUTAG, IMDB-MULTI, and COLLAB datasets. We calculate the time of the algorithms by measuring the duration needed to complete 200 epochs of training with the 512 batch size. For space efficiency, we compute the GPU memory utilization during the training process. From Figure 3, it can be observed that ASAPool, DiffPool, MincutPool, and JustBalancePool exhibit significantly higher time and space costs. In contrast, node dropping pooling methods such as TopKPool, SAGPool, and KMISPool demonstrate lower time and space costs. The underlying reason is that node clustering pooling methods require converting graph data into an adjacency matrix form and simplifying the graph through clustering rather than directly removing nodes.

Visualization. Figure 4 shows the t-SNE visualization for TopKPool and SAGPool under different pooling ratios. From the results, we observe that as the pooling ratio increases from $0.1$ to $0.9$, the different classes form more distinct clusters in the t-SNE plot when the pooling ratio is low. As the pooling ratio increases, the model retains more nodes, leading to a greater overlap between nodes of different classes and a reduction in inter-class separability. Moreover, when the pooling ratio is $0.9$, SAGPool shows a higher degree of class separability compared to TopKPool.

Backbone Analysis. Figure 5 presents the performance of four pooling methods based on GCNConv (Kipf & Welling, ), GATConv (Veličković et al., ), SAGEConv (Hamilton et al., ), and GraphConv (Morris et al., ) on four datasets, NCI1, NCI109, PROTEINS, and PROTEINS_FULL. On average, as the backbone models change, most pooling methods exhibit significant performance fluctuations, and no single backbone model consistently maintains a leading position. Except for the PROTEINS_FULL, the performance of GraphConv is relatively better.

Figure: Figure 5: Performance w.r.t. graph convolution backbones for different pooling methods.
Refer to caption: extracted/5895684/image/backbone_change.png

Figure: Figure 6: Performance w.r.t. different pooling ratios for four pooling methods.
Refer to caption: extracted/5895684/image/pooling_ratio.png

Parameter Analysis. Figure 6 shows the performance of four pooling methods on the COX2, AIDS, FRANKENSTEIN, and Mutagenicity datasets. From the results, we observe that as the pooling rate increases from $0.1$ to $0.9$, the performance increases before reaching saturation in most cases. The performance variation among different pooling methods is significant as the pooling ratio changes, it is necessary to adjust the pooling ratio when employing pooling methods.

## 5 Conclusion

In this paper, we construct the first graph pooling benchmark that includes 17 state-of-the-art approaches and 28 different graph datasets across graph classification, graph regression, and node classification. We find that node clustering pooling methods outperform node dropping pooling methods in terms of robustness and generalizability, but at the cost of higher computational expenses. This benchmark systematically analyzes the effectiveness, robustness, and generalizability of graph pooling methods. We also make our benchmark publically available to advance the fields of graph machine learning and applications. One limitation of our benchmark is the lack of more complicated settings under label scarcity. In future works, we would extend our graph pooling benchmark to more realistic settings such as semi-supervised learning and few-shot learning.

## Appendix

## Appendix A Related Work

### A.1 Graph Classification and Graph Regression

Graphs provide an effective tool to represent interaction among different objects (Wu et al., ). Graph classification (Baek et al., ) is a fundamental graph machine learning problem, which aims to classify each graph into its corresponding category. The majority of current works adopt message passing mechanisms (Wu et al., ), where each node receives information from its neighbors in a recursive manner. Then, a graph readout function is adopted to summarize all node representations into a graph-level representation for downstream classification. Graph classification has extensive applications in various domains such as molecular property prediction (Wieder et al., ) and protein function analysis (Mills et al., ). Graph regression (Qin et al., ) is close to graph classification which maps graph-level data into continuous vectors. Researchers usually utilize graph regression to formulate molecular property predictions (Mqawass & Popov, ). Graph pooling has been an important topic in graph-level tasks (Knyazev et al., ; Lee et al., ; Ranjan et al., ; Ma et al., ; Zhou et al., ; Pang et al., ; Bacciu et al., ; Zhang et al., ; ; Hansen & Bianchi, ; Ying et al., ; Bianchi et al., ; Tsitsulin et al., ; Duval & Malliaros, ; Bianchi, ), which generally utilize a hierarchical way to refine the graph structures (Bianchi & Lachi, ; Liu et al., ). In this work, we generally study the performance of graph pooling on graph-level tasks and validate the effectiveness of graph pooling approaches in most cases.

### A.2 Node Classification

Node classification aims to classify each node in a graph based on its attributes and relationship with the other nodes (Xiao et al., ; Ju et al., ). Node classification has various applications in the real world, including social network analysis (Camacho et al., ), knowledge graphs (Ye et al., ), bioinformatics (Bhagat et al., ) and online commerce services (Yu et al., ). Graph neural networks have been widely utilized to solve the problem by learning semantics information across nodes by neighborhood propagation. Since graph pooling would reduce the number of nodes, recent works utilize a U-Net architecture (Gao & Ji, ), which involves down-sampling and up-sampling with residue connections. In this work, we systematically evaluate the performance of graph pooling on node classification and observe that graph pooling has limited improvement compared with basic graph neural network architectures.

### A.3 Previous Benchmark Research

Previous studies have built benchmark for graph-related tasks (Errica et al., ; Hu et al., ). In particular,  Errica et al. () is a benchmark including six different GNN models across nine commonly used TUDataset datasets. Open Graph Benchmark (OGB) (Hu et al., ) evaluate differnet graph neural network approaches experiments on graph classification, graph regression, and node classification.  Errica et al. () only involve one graph pooling method and Hu et al. () does not involve any graph pooling methods. In comparison, our method focuses on graph pooling techniques rather than graph neural networks. Moroever, our benchmark explores the robustness of these methods by introducing noise attacks in both graph classification and node classification tasks and investigate their generalizability through out-of-distribution shifts.

## Appendix B Details of U-Net based Node Classification

Most existing studies that combine node classification with pooling have utilized the U-Net architecture (Wu et al., ; Song et al., ; Zhang et al., ). Figure A.1 shows the overview of the U-Net framework. Pooling plays a crucial role in the U-Net structure, as it progressively reduces the graph size in the downsampling path to extract important global features while preserving essential local information. In the upsampling path, pooling facilitates the fusion of features from the downsampling path with those being progressively recovered in the upsampling path through skip connections, thereby assisting the model in accurately performing graph node classification or other tasks. Pooling not only simplifies the graph’s complexity but also provides the model with multi-scale feature representation capabilities.

In the downsampling path, the input feature matrix is first subjected to graph convolution, where the product of the adjacency matrix and the feature matrix, along with the weighted sum of the weight matrix and the bias term, yields the activated feature matrix $\mathbf{H}^{(l+1)}$. Next, a pooling operation is applied, reducing the number of nodes by selecting those with higher scores, thereby transforming the original feature matrix ($\mathbf{H}^{(l+1)}$ and adjacency matrix $\mathbf{A}^{(l)}$ into a smaller feature matrix $\mathbf{H^{\prime}}^{(l+1)}$ and a corresponding adjacency matrix $\mathbf{A^{\prime}}^{(l+1)}$. In the upsampling path, the pooled feature matrix $\mathbf{H^{\prime}}^{(l+1)}$ is first upsampled to restore the original number of nodes, generating a new feature matrix $\mathbf{H^{\prime\prime}}^{(l+1)}$. Then, the restored feature matrix is concatenated with the corresponding feature matrix $\mathbf{H}^{(l)}$ from the downsampling path, forming the merged feature matrix $\mathbf{H}^{(l+1)}_{\text{merged}}$. Finally, the merged feature matrix undergoes another graph convolution, resulting in the output feature matrix $\mathbf{H}^{(l+2)}$.

Figure: Figure A.1: Overview of the graph U-Net framework for node classification.
Refer to caption: x3.png

## Appendix C Details of Selected Pooling Methods

### C.1 Node Dropping Pooling

TopKPool (Knyazev et al., ). TopKPool utilizes the attention mechanism to learn the scores of different nodes and then selects the nodes with top scores, which can learn important local portions from original graphs.

SAGPool (Lee et al., ). SAGPool utilizes a different graph neural network to learn importance scores, which can guide the pooling process effectively.

ASAPool (Ranjan et al., ). ASAPool considers the neighboring subgraphs to represent nodes and then adopts the attention mechanism to generate subgraph representations. The importance nodes are selected by a graph neural networks with local extremum information.

PANPool (Ma et al., ). PANPool constructs the maximal entropy transition (MET) matrix based on graph Laplacian, which can generate importance scores for different nodes.

COPool (Zhou et al., ). COPool learns pooled representations from the complimentary edge and node views. The edge view comes from high-order semantics information while the node view stems from importance scores from the cut proximity matrix.

CGIPool (Pang et al., ). CGIPool incorporates mutual information optimization into graph pooling, which can enhance the graph-level relationships between the original graph and the pooled graph.

KMISPool (Bacciu et al., ). KMISPool incorporates the Maximal k-Independent Sets (k-MIS) into graph pooling, which can detect the important nodes in the graph with topological preserved.

GSAPool (Zhang et al., ). GSAPool integrates both structural and attribute information in different components. These scores from different components are then fused to guide the graph pooling process.

HGPSLPool (Zhang et al., ). HGPSLPool not only utilizes graph pooling to determine important nodes from the original graph, but also leverages graph structure learning to explore the topological information in the pooled graph.

### C.2 Node Clustering Pooling

AsymCheegerCutPool (Hansen & Bianchi, ). AsymCheegerCutPool conducts graph clustering to generate the assignment of each node according to graph total variation (GTV). Each cluster is aggregated in a hierarchical manner during graph pooling.

DiffPool (Ying et al., ). DiffPool introduces a learnable soft assignment of each node during graph clustering, and then maps each cluster into the coarsened nodes in the pooling graph.

MincutPool (Bianchi et al., ). MincutPool relaxes the classic normalized mincut problem into a continuous fashion, and then optimizes a graph neural network to achieve this. The graph clustering results are adopted to guide the graph pooling process.

DMoNPool (Tsitsulin et al., ). DMoNPool introduces an objective based on modularity for graph clustering and then adds a regularization term to avoid trivial solutions during optimization. Similarly, graph clustering results are leveraged for graph pooling.

HoscPool (Duval & Malliaros, ). HoscPool combines higher-order relationships in the graph with graph pooling based on motif conductance. It minimizes a relaxed motif spectral clustering objective and involves multiple motifs to learn hierarchical semantics.

JustBalancePool (Bianchi, ). JustBalancePool consists of two components. On the one hand, it aims to reduce the local quadratic variation during graph clustering. On the other hand, it involves a balanced term to reduce the risk of degenerate solutions.

SEPool (Wu et al., ). SEPool generates a clustering assignment matrix in one go through a global optimization algorithm, avoiding the suboptimality associated with layer-by-layer pooling.

ParsPool (Song et al., ). ParsPool is characterized by the introduction of a graph parsing algorithm that adaptively learns a personalized pooling structure for each graph. ParsPool is inspired by bottom-up grammar induction and can generate a flexible pooling tree structure for each graph.

## Appendix D Detailed Description of Datasets

### D.1 Graph Classification

Table A.1 provides descriptive statistics of the selected datasets, revealing that our chosen datasets encompass graph data of varying scales and features. This diversity establishes a robust foundation for benchmarking. The following are detailed descriptions of these datasets:

Ogbg-molpcba comprises a collection of 437,929 molecules, each represented as a graph where nodes are atoms and edges indicate chemical bonds between atoms. Each node is associated with features such as atom type, valence, and charge. The dataset involves 128 biological activity labels, each representing a binary classification task that indicates whether a molecule exhibits a specific biological activity (Hu et al., ).

PROTEINS represents protein structures; nodes denote secondary structure elements (SSEs) and the edges indicate the relationships between these SSEs that are in close proximity. The primary goal of this dataset is to assist in the classification of proteins into different structural classes based on their amino acid sequences and structure— structural characteristics. Each graph’s label is the protein class, so the dataset covers diverse protein structures (Borgwardt et al., ).

PROTEINS_FULL is an extended version of PROTEINS. Each graph directly represents a protein structure: nodes correspond to SSEs like alpha helices and beta sheets (Borgwardt et al., ).

NCI1 is a collection of chemical compound graphs. Originating from the National Cancer Institute (NCI) database, each graph sample is a compound in which nodes represent atoms and edges represent the bonds between them. The dataset is binary-class labeled, indicating biological activity via compounds’ anti-cancer activity against specific cell lines (Wale et al., ).

NCI109 is also a collection of chemical compound graphs derived from NCI. Similarly, each node in the graph denotes an atom and each edge denotes a bond. The two classes in NCI109 are about compounds’ ability to inhibit or interact with the specified cancer cell line (Wale et al., ).

MUTAG consists of 188 chemical molecule graphs, where each node represents an atom. The nodes have different atomic types, such as carbon, nitrogen, oxygen, etc. Edges represent chemical bonds between atoms, such as single or double bonds, indicating their connections in the molecule. The objective is to predict whether each molecule is mutagenic, with positive labels indicating mutagenic molecules and negative labels indicating non-mutagenic molecules (Morris et al., ).

D&D is a dataset of protein structure graphs for graph classification. Each graph in this dataset represents a protein, with nodes corresponding to amino acids and edges corresponding to the spatial or sequential proximity between these amino acids. The primary objective of the D&D dataset is to classify proteins into one of two categories: enzymes or non-enzymes (Shervashidze et al., ).

IMDB-B is a collection of social network graphs derived from the Internet Movie Database (IMDB). Each graph is about a collaboration network from movies whereby nodes stand for actors or actresses and edges indicate that the two actors appeared in the same movie — this dataset comprises two classes reflecting the movie genres (Cai & Wang, ).

IMDB-M, similar to IMDB-B, represents each movie as a graph where the nodes represent actors and the edges represent co-appearances of actors in the same movie. However, the nodes in IMDB-M are categorized into three classes, and it includes a larger number of actors (Cai & Wang, ).

COLLAB consists of 5,000 graphs, each representing a collaboration network of a group of authors in different research fields. In each graph, nodes represent authors, and edges represent collaborations between authors, indicating that the connected authors have co-authored at least one paper. The graphs have three classes, each corresponding to an academic research field. (Morris et al., ).

COX2 consists of 467 graphs, where each graph corresponds to a molecule. The nodes represent atoms, and the edges represent chemical bonds, and the graph label indicates whether the molecule is a COX-2 inhibitor (Sutherland et al., ).

AIDS consists of 2,000 graphs. Each graph corresponds to a molecule, where the nodes represent individual atoms and the edges represent chemical bonds between these atoms. The goal is to predict the inhibitory effect of molecules on HIV based on their structure. (Riesen & Bunke, ).

FRANKENSTEIN consists of 4,337 graphs. Each graph in this dataset represents a chemical compound, where the nodes correspond to atoms, and the edges represent the bonds between them. The graph labels indicate whether the molecule is considered an active compound (Orsini et al., ).

Mutagenicity contains 4,337 molecular graphs. In Mutagenicity, each graph represents a molecule, where the nodes are atoms and the edges denote chemical bonds between the nodes. The classification goal is to predict whether a molecule is mutagenic or not (Debnath et al., ).

**Table A.1: Summary statistics of datasets for graph classification. CC denotes the clustering coefficient, and Diameter representing the maximum value of the shortest path between any two nodes in the graph.**
| Datasets | Graphs | Classes | Avg. Nodes | Avg. Edges | Node Attr. | Avg. Diameter | Avg. Degree | Avg. CC |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Ogbg-molpcba | 437,929 | 2*128 | 26.00 | 27.50 | - | 12.00 | 2.20 | 0.00 |
| PROTEINS | 1,113 | 2 | 39.06 | 72.82 | + (1) | 11.14 | 3.73 | 0.51 |
| PROTEINS_full | 1,113 | 2 | 39.06 | 72.82 | + (29) | 11.14 | 3.73 | 0.51 |
| NCI1 | 4,110 | 2 | 29.87 | 32.30 | - | 11.45 | 2.16 | 0.00 |
| NCI109 | 4,127 | 2 | 29.68 | 32.13 | - | 11.21 | 2.16 | 0.00 |
| MUTAG | 188 | 2 | 17.90 | 39.60 | + (7) | 8.22 | 2.19 | 0.00 |
| D&D | 1,178 | 2 | 284.32 | 715.66 | - | 16.45 | 4.92 | 0.48 |
| IMDB-B | 1,000 | 2 | 19.77 | 96.53 | - | 1.86 | 8.89 | 0.95 |
| IMDB-M | 1,500 | 3 | 13.00 | 65.94 | - | 1.47 | 8.10 | 0.97 |
| COLLAB | 5,000 | 3 | 74.49 | 2457.22 | - | 1.86 | 37.36 | 0.89 |
| COX2 | 467 | 2 | 41.22 | 43.45 | + (3) | 13.79 | 2.11 | 0.00 |
| AIDS | 2,000 | 2 | 15.69 | 16.20 | + (4) | 6.56 | 2.01 | 0.01 |
| FRANKENSTEIN | 4,337 | 2 | 16.90 | 17.88 | + (780) | 7.86 | 2.06 | 0.01 |
| Mutagenicity | 4,337 | 2 | 30.32 | 30.77 | - | 9.10 | 2.04 | 0.00 |

### D.2 Graph Regression

Table A.2 provides an overview of the selected datasets in terms of their tasks, compounds and their features, recommended splits, and metrics. A more detailed description is provided below.

QM7 and QM8 are benchmark datasets in computational chemistry, designed to facilitate the development and evaluation of machine learning approaches for quantum mechanical property prediction. It contains approximately 7,165 (QM7) and 21,786 (QM8) molecular structures, each characterized by their calculated properties using quantum chemistry methods, specifically focusing on electronic spectra (Wu et al., ; Montavon et al., ).

BACE is a collection of biochemical data used to evaluate computational methods for drug discovery. The dataset includes a total of 1,522 compounds, each annotated with their binding affinities, as well as molecular descriptors and fingerprints to facilitate the development and assessment of machine learning modelsa (Wu et al., ; Ciordia et al., ).

ESOL is a prominent resource in cheminformatics, designed for evaluating machine learning models on the prediction of aqueous solubility of small molecules. The dataset, derived from the work of Delaney, encompasses a diverse range of chemical compounds with experimentally determined solubility values expressed in logS, where S is the solubility in mols per liter. It includes 1128 compounds, serving as a benchmark for solubility prediction tasks (Delaney, ; Wu et al., ).

FreeSolv is a dataset containing hydration-free energies for small molecules in an aqueous solution. It comprises data for a wide range of organic molecules, providing both experimental values and calculated predictions based on molecular simulations (Mobley & Guthrie, ; Wu et al., ).

Lipophilicity is primarily utilized for studying and evaluating molecular lipophilicity. This dataset comprises 4,200 compounds sourced from the ChEMBL database, with experimentally measured partition coefficient (logD) values that reflect the distribution behavior of compounds in a water-octanol system (Lukashina et al., ; Wu et al., ).

**Table A.2: Details of datasets for graph regression.**
| Datasets | Tasks | Compounds | Split | Avg. Nodes | Avg. Edges | Avg. Diameter | Avg. Degree | Avg. CC |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| QM7 | 1 | 7,165 | Scaffold | 6.79 | 6.44 | 4.21 | 1.89 | 0.06 |
| QM8 | 12 | 21,786 | Random | 7.77 | 8.09 | 4.35 | 2.08 | 0.09 |
| BACE | 1 | 1,522 | Scaffold | 34.09 | 36.86 | 4.35 | 2.08 | 0.01 |
| ESOL | 1 | 1,128 | Random | 13.30 | 13.69 | 7.02 | 1.98 | 0.00 |
| FreeSolv | 1 | 643 | Random | 8.76 | 8.43 | 5.06 | 1.84 | 0.00 |
| Lipophilicity | 1 | 4,200 | Random | 27.04 | 29.50 | 13.85 | 2.18 | 0.00 |

### D.3 Node Classification

Table A.3 presents descriptive statistics of the seven datasets used for node classification. It is evident that there is a significant variance in the scale of the selected datasets, each possessing distinct characteristics. Further background information and details about these datasets are provided below.

Ogbn-arxiv comprises a collection of 169,343 scientific publications classified into 40 distinct categories. Each paper is represented by a node with a 128-dimensional feature, which comes from the average of word embeddings in the corresponding title and abstract. Edges indicate citation relationships between papers (Hu et al., ).

Cora comprises a collection of 2,708 scientific publications classified into seven distinct categories. Each publication in the dataset is represented as a node in a citation network, where edges indicate citation relationships between papers (Yang et al., ).

CiteSeer is a widely used citation network dataset. It comprises scientific publications categorized into six classes, with each publication represented by a 3,327-dimensional binary vector recording the presence or absence of specific words (Yang et al., ).

PubMed consists of scientific publications from the PubMed database, categorized into three classes based on their Medical Subject Headings (MeSH) terms. Each node has a sparse bag-of-words vector derived from the content of the corresponding publication (Yang et al., ).

**Table A.3: Summary statistics of datasets for node classification.**
| Datasets | Number of Nodes | Number of Edges | Number of Features | Number of Classes | Diameter | Avg. Degree | Avg. CC |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Ogbn-arxiv | 169,343 | 1,166,243 | 128 | 40 | 23 | 13.72 | 0.23 |
| Cora | 2,708 | 10,556 | 1,433 | 7 | NA | 3.90 | 0.24 |
| CiteSeer | 3,327 | 9,104 | 3,703 | 6 | NA | 2.74 | 0.14 |
| PubMed | 19,717 | 88,648 | 500 | 3 | 18 | 4.50 | 0.06 |
| Cornell | 183 | 298 | 1,703 | 5 | 8 | 3.06 | 0.17 |
| Texas | 183 | 325 | 1,703 | 5 | 8 | 3.22 | 0.20 |
| Wisconsin | 251 | 515 | 1,703 | 5 | 8 | 3.71 | 0.21 |
| Github | 37,700 | 578,006 | 0 | 2 | 7 | 15.33 | 0.01 |

Cornell, Texas, and Wisconsin are made up of nodes that represent web pages and edges which denote hyperlinks between these pages. Each node has a class which denotes the topic of the web page; this allows tasks including node classification and link prediction to be performed. The datasets differ in size: Cornell and Texas each have 183 nodes while Wisconsin has 251 nodes (Pei et al., ).

Github includes node attributes representing the features of developers, such as their interests, skills, and contributions to various repositories. The edges within the network capture the interactions and collaborations of developers, creating a multi-faceted graph structure (Rozemberczki et al., ).

### D.4 Out-of-distribution shifts

Size shifts. For the selected datasets, including NCI1, D&D, NCI109, and IMDB-B, we utilized the data provided by the authors of size-invariant-GNNs (Bevilacqua et al., ). In this setup, the graphs with the smallest 50% of nodes are used as the training set, those with the largest 20% of nodes are used as the test set, and the remaining graphs were used as the validation set. The data can be downloaded from [https://www.dropbox.com/s/38eg3twe4dd1hbt/data.zip](https://www.dropbox.com/s/38eg3twe4dd1hbt/data.zip).

Density shifts. For the selected datasets, we divide the datasets based on graph density: the 50% of graphs with the lowest density are used as the training set, the 20% with the highest density are used as the test set, and the remaining graphs are used as the validation set. After applying density shifts, the following densities are observed: for D&D, the training set density is 0.0274, the validation set density is 0.0536, the test set density is 0.1142; for NCI1, the training set density is 0.1229, the validation set density is 0.1920, the test set density is 0.2786; for NCI109, the training set density is 0.1248, the validation set density is 0.1943, the test set density is 0.2770; for IMDB-B, the training set density is 0.6574, the validation set density is 1.1074, the test set density is 1.7427.

Degree shifts. For the selected datasets, we divide the datasets based on node degree: the 50% of nodes with the highest degree are used as the training set, the 25% with the lowest degree are used as the test set, and the remaining nodes are used as the validation set. After applying degree shifts, we can observe that: for Cora, the training set average degree is 5.9431, the validation set average degree is 2.4225, and the test set average degree is 1.2836; for Citeseer, the training set average degree is 4.3313, the validation set average degree is 1.3430, and the test set average degree is 0.9424; for Pubmed, the training set average degree is 7.9148, the validation set average degree is 1.1552, and the test set average degree is 1.0000; for Cornell, the training set average degree is 3.2198, the validation set average degree is 0.1111, and the test set average degree is 0.0000; for Texas, the training set average degree is 3.3626, the validation set average degree is 0.4222, and the test set average degree is 0.0000; for Wisconsin, the training set average degree is 3.7600, the validation set average degree is 0.7258, and the test set average degree is 0.0000.

Closeness shifts. For the selected datasets, we divide the datasets based on node closeness: the 50% of nodes with the lowest closeness are used as the training set, the 25% with the highest closeness are used as the test set, the remaining nodes used as the validation set. After applying closeness shifts, we can observe that: for Cora, the training set average closeness is 0.1076, the validation set average closeness is 0.1560, the test set average closeness is 0.1786; for Citeseer, the training set average closeness is 0.0150, the validation set average closeness is 0.0679, the test set average closeness is 0.0832; for Pubmed, the training set average closeness is 0.1448, the validation set average closeness is 0.1669, the test set average closeness is 0.1850; for Cornell, the training set average closeness is 0.2690, the validation set average closeness is 0.3754, the test set average closeness is 0.3896; for Texas, the training set average closeness is 0.2899, the validation set average closeness is 0.3887, the test set average closeness is 0.4047; for Wisconsin, the training set average closeness is 0.2630, the validation set average closeness is 0.3686, the test set average closeness is 0.3855.

**Table A.4: Details of hyperparameter tuning for different pooling methods**
| Methods | Hyperparameter space |  |
| --- | --- | --- |
| Node Dropping Pooling |  |  |
| TopKPool | NIPS’19 | Pooling ratio: 0.1, 0.3, 0.5, 0.7, 0.9 |
| SAGPool | ICML’19 | Pooling ratio: 0.1, 0.3, 0.5, 0.7, 0.9 |
| ASAPool | AAAI’20 | Pooling ratio: 0.1, 0.3, 0.5, 0.7, 0.9 |
| PANPool | NIPS’20 | Pooling ratio: 0.1, 0.3, 0.5, 0.7, 0.9 |
| COPool | ECMLPKDD’22 | Pooling ratio: 0.1, 0.3, 0.5, 0.7, 0.9; K: 1, 2, 3 |
| CGIPool | SIGIR’22 | Pooling ratio: 0.1, 0.3, 0.5, 0.7, 0.9 |
| KMISPool | AAAI’23 | The independent sets $K$: 1, 2, 3, 4, 5 |
| GSAPool | WWW’20 | Pooling ratio: 0.1, 0.3, 0.5, 0.7, 0.9; Alpha: 0.2, 0.4, 0.6, 0.8 |
| HGPSLPool | Arxiv’19 | Pooling ratio: 0.1, 0.3, 0.5, 0.7, 0.9 |
| ParsPool | ICLR’24 | Parsingnet layers: 1, 2, 3; Deepsets layers: 1, 2, 3 |
| Node Clustering Pooling |  |  |
| AsymCheegerCutPool | ICML’23 | MLP layers: 1, 2; MLP hidden channels: 64, 128, 256 |
| DiffPool | NIPS’18 | Not applicable |
| MincutPool | ICML’20 | Temperature: 1, 1.5, 1.8, 2.0 |
| DMoNPool | JMLR’23 | Clusters: 2, 4, 6, 8, 10, 12 |
| HoscPool | CIKM’22 | Mu: 0.2, 0.5, 0.8; Alpha: 0.2, 0.5, 0.8 |
| JustBalancePool | Arxiv’22 | Not applicable |
| SEPool | ICML’22 | Tree<br>depth: 1, 2, 3; Number of blocks: 1, 2, 3, 4 |

## Appendix E Additional Experimental Details

### E.1 Graph Classification

The classification model comprises three primary components: GCNConv layers, pooling methods, and a global average pooling layer. The hidden and output channels for this model are both set to 64. Initially, the data passes through three GCNConv layers with ReLU activation functions, followed by two pooling layers, before arriving at a global average pooling layer. The embedding output from this global layer undergoes further processing through a linear layer with ReLU activation, having dimensions (64, 32), followed by another linear layer without any activation function but with dimensions (32, number of classes). The final output can be available after applying softmax to the embedding output. All models use the Adam optimizer with a learning rate of 0.001 and are trained for 200 epochs by minimizing the negative log-likelihood loss function. For Ogbg-molpcba, the data is divided into training, validation, and test sets with an 80%, 10%, and 10% split, respectively. The remaining datasets are divided into training, validation, and test sets with a 70%, 15%, and 15% split. Each trial is repeated multiple times with different random seeds.

### E.2 Graph Regression

We use a backbone network inspired by MESPool (Xu et al., ) for graph regression. The model mainly consists of three GINConv layers with ReLU activation functions and BatchNorm, along with two pooling layers, followed by a global average pooling layer. All channels (both hidden and output) are set to 64. The embedding output from the global average pooling layer passes through another linear layer with ReLU activation, having dimensions (64, 32). All models use the Adam optimizer with a learning rate of 0.001 and are trained for 200 epochs by minimizing the negative log-likelihood loss function. All data are processed using a 5-fold cross-validation and are run on multiple different seeds.

### E.3 Node Classification

For node classification, we utilize a U-Net architecture, which we divide into a downsampling convolutional part and an upsampling convolutional part (Siddique et al., ). The downsampling convolutional section includes two GCNConv layers with ReLU activation functions, with pooling applied between these layers. In the upsampling convolutional section, we use the indices saved during pooling for upsampling, restoring features to their pre-pooling size. The upsampled features are then fused with the corresponding residual features from the downsampling path, either through summation or concatenation. Finally, the fused features are processed and activated through a GCNConv layer. All models employ the Adam optimizer with a learning rate set to 0.001 and are trained for 200 epochs using cross-entropy loss. All data are processed using a 5-fold cross-validation and are run on multiple different seeds.

### E.4 Hyperparameter Tuning

Details of hyperparameter tuning for different pooling methods can be found in Table A.4. We performed hyperparameter searches for each dataset in each task.

## Appendix F Additional Experiments

### F.1 Robustness Analysis

Table LABEL:noise_node_appendix shows the additional robustness analysis on more node-level datasets. From Table LABEL:noise_node_appendix, we observe the following: Firstly, for smaller node classification datasets such as Cornell, Texas, and Wisconsin, masking node features results in the greatest performance loss, while edge deletion leads to the smallest performance loss. The potential reason is that these smaller datasets inherently possess higher local characteristics and structural sparsity, making node features more critical for the model’s classification tasks. Secondly, consistent with the robustness analysis on Cora, Citeseer, and Pubmed, ASAPool and KMISPool demonstrate superior performance, indicating that these pooling methods exhibit stronger robustness in node classification tasks.

### F.2 Generalizability Analysis

Table A.6 presents the results of size-based and density-based distribution shifts on NCI109 and IMDB-B, respectively. From Table A.6, we obtain conclusions similar to those in the main text: for the NCI109 dataset, node dropping pooling methods perform worse than node clustering pooling methods, whereas on the IMDB-B dataset, node dropping pooling methods outperform node clustering pooling methods. Overall, AsymCheegerCutPool, MinCutPool, and DMoNPool outperform other pooling methods.

Table A.7 presents the results of degree-based and closeness-based distribution shifts on node classification tasks across four datasets: Pubmed, Cornell, Texas, and Wisconsin. We observe the following: Firstly, KMISPool and GSAPool generally perform the best, yet no single pooling method consistently leads across all datasets. Secondly, the issue of class imbalance persists, and it is more pronounced in smaller datasets such as Cornell, Texas, and Wisconsin. Thirdly, smaller datasets like Cornell, Texas, and Wisconsin are more sensitive to distribution shifts compared to the larger dataset Pubmed, resulting in more significant performance degradation.

**Table A.5: Results of node classification under random noise attack.**
| Dataset | Ptb Method | TopKPool | SAGPool | ASAPool | PANPool | COPool | CGIPool | KMISPool | GSAPool | HGPSLPool |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Cornell | ADD | 46.64$\pm$0.25 | 46.45$\pm$0.77 | 47.18$\pm$1.03 | 47.00$\pm$1.18 | 46.63$\pm$0.25 | 46.63$\pm$0.92 | 46.81$\pm$0.67 | 46.81$\pm$1.12 | 46.45$\pm$0.45 |
| DROP | 61.73$\pm$1.17 | 62.09$\pm$1.84 | 62.29$\pm$1.15 | 61.93$\pm$0.95 | 62.25$\pm$0.44 | 61.36$\pm$2.00 | 63.19$\pm$1.79 | 63.01$\pm$0.54 | 62.46$\pm$1.33 |  |
| MASK | 46.99$\pm$0.90 | 47.36$\pm$1.56 | 47.91$\pm$2.46 | 47.00$\pm$0.45 | 48.11$\pm$1.17 | 46.63$\pm$1.03 | 47.74$\pm$1.13 | 47.01$\pm$1.35 | 47.55$\pm$1.19 |  |
| Texas | ADD | 58.63$\pm$0.67 | 61.37$\pm$0.94 | 60.48$\pm$0.62 | 59.92$\pm$0.92 | 59.01$\pm$0.44 | 58.99$\pm$1.14 | 58.83$\pm$0.49 | 59.37$\pm$1.39 | 58.63$\pm$0.23 |
| DROP | 64.47$\pm$2.02 | 64.47$\pm$0.42 | 63.92$\pm$2.38 | 64.30$\pm$0.89 | 63.92$\pm$1.96 | 63.57$\pm$1.40 | 65.57$\pm$1.59 | 65.57$\pm$1.33 | 63.57$\pm$2.06 |  |
| MASK | 57.56$\pm$0.92 | 57.93$\pm$1.55 | 58.85$\pm$1.44 | 57.19$\pm$0.91 | 57.37$\pm$1.33 | 57.56$\pm$0.93 | 58.10$\pm$1.09 | 57.93$\pm$0.92 | 57.92$\pm$0.43 |  |
| Wisconsin | ADD | 54.46$\pm$0.99 | 53.66$\pm$0.98 | 56.18$\pm$1.49 | 55.52$\pm$0.21 | 55.78$\pm$0.98 | 55.78$\pm$1.30 | 54.99$\pm$0.34 | 54.73$\pm$0.48 | 53.65$\pm$0.37 |
| DROP | 61.23$\pm$1.23 | 60.55$\pm$1.15 | 60.29$\pm$1.68 | 59.76$\pm$1.72 | 60.43$\pm$1.79 | 59.36$\pm$0.87 | 60.16$\pm$0.66 | 60.43$\pm$0.95 | 61.36$\pm$1.17 |  |
| MASK | 47.02$\pm$0.55 | 47.94$\pm$0.18 | 49.80$\pm$0.86 | 48.60$\pm$1.14 | 52.59$\pm$0.01 | 48.35$\pm$0.49 | 49.53$\pm$1.15 | 48.20$\pm$1.11 | 48.08$\pm$0.96 |  |

**Table A.6: Results of graph classification under distribution shifts.**
| Method | NCI109 | IMDB-B |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Size | Density | Size | Density |  |  |  |  |  |
| Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 |  |
| Node Dropping Pooling |  |  |  |  |  |  |  |  |
| TopKPool | 25.10$\pm$0,81 | 22.90$\pm$1.01 | 55.92$\pm$2.57 | 54.88$\pm$1.42 | 56.00$\pm$1.22 | 53.34$\pm$2.26 | 72.00$\pm$5.94 | 68.13$\pm$5.78 |
| SAGPool | 24.07$\pm$1.29 | 21.64$\pm$1.58 | 53.31$\pm$1.74 | 51.46$\pm$0.98 | 67.67$\pm$5.10 | 67.44$\pm$4.97 | 74.27$\pm$3.92 | 67.13$\pm$5.78 |
| ASAPool | 22.57$\pm$0.70 | 19.42$\pm$1.04 | 58.42$\pm$2.73 | 57.09$\pm$2.02 | 73.83$\pm$12.43 | 73.70$\pm$12.36 | 80.80$\pm$4.94 | 78.70$\pm$4.15 |
| PANPool | 25.73$\pm$0.11 | 23.80$\pm$0.12 | 56.25$\pm$1.63 | 54.34$\pm$1.93 | 66.50$\pm$9.34 | 65.73$\pm$10.38 | 76.27$\pm$4.21 | 71.65$\pm$4.67 |
| COPool | 24.94$\pm$1.72 | 22.77$\pm$2.31 | 57.10$\pm$2.20 | 55.39$\pm$1.21 | 65.33$\pm$2.72 | 65.13$\pm$2.94 | 72.40$\pm$9.91 | 69.52$\pm$8.26 |
| CGIPool | 24.54$\pm$1.19 | 22.39$\pm$1.46 | 61.36$\pm$0.84 | 57.98$\pm$3.78 | 72.83$\pm$8.00 | 72.69$\pm$7.87 | 71.60$\pm$5.44 | 63.60$\pm$7.27 |
| KMISPool | 43.78$\pm$5.82 | 43.24$\pm$5.33 | 58.08$\pm$3.45 | 50.03$\pm$6.64 | 75.33$\pm$4.78 | 75.16$\pm$4.61 | 78.80$\pm$0.86 | 73.74$\pm$0.63 |
| GSAPool | 25.97$\pm$3.11 | 24.00$\pm$4.03 | 53.04$\pm$1.30 | 52.64$\pm$1.16 | 70.17$\pm$3.70 | 69.17$\pm$4.45 | 78.80$\pm$0.86 | 73.11$\pm$0.99 |
| HGPSLPool | 21.54$\pm$0.22 | 18.17$\pm$0.43 | 58.18$\pm$2.26 | 55.06$\pm$3.02 | 69.33$\pm$4.71 | 69.25$\pm$4.76 | 75.07$\pm$1.86 | 70.37$\pm$1.81 |
| ParsPool | 42.68$\pm$0.81 | 41.97$\pm$0.74 | 59.91$\pm$1.13 | 56.13$\pm$2.70 | 75.00$\pm$3.63 | 74.92$\pm$3.57 | 76.00$\pm$2.99 | 71.63$\pm$3.00 |
| Node Clustering Pooling |  |  |  |  |  |  |  |  |
| AsymCheegerCutPool | 79.18$\pm$0.00 | 49.92$\pm$0.12 | 68.53$\pm$0.00 | 44.29$\pm$0.00 | 71.50$\pm$0.60 | 71.45$\pm$0.60 | 78.80$\pm$0.00 | 73.75$\pm$0.00 |
| DiffPool | 21.38$\pm$0.00 | 17.61$\pm$0.00 | 69.47$\pm$0.00 | 48.65$\pm$0.02 | 69.17$\pm$0.70 | 67.31$\pm$0.83 | 65.20$\pm$1.10 | 62.72$\pm$0.76 |
| MinCutPool | 31.83$\pm$0.77 | 30.50$\pm$0.90 | 70.76$\pm$0.00 | 56.27$\pm$0.02 | 70.17$\pm$0.01 | 68.28$\pm$0.03 | 78.40$\pm$0.01 | 73.91$\pm$0.02 |
| DMoNPool | 79.41$\pm$0.01 | 55.84$\pm$0.01 | 67.44$\pm$0.05 | 55.80$\pm$0.13 | 74.33$\pm$1.59 | 73.85$\pm$1.61 | 77.33$\pm$0.13 | 72.56$\pm$0.20 |
| HoscPool | 33.73$\pm$2.35 | 30.58$\pm$2.13 | 69.54$\pm$0.05 | 54.41$\pm$0.03 | 72.83$\pm$0.09 | 72.37$\pm$0.10 | 77.20$\pm$0.04 | 73.43$\pm$0.05 |
| JustBalancePool | 58.43$\pm$4.14 | 44.15$\pm$1.17 | 71.26$\pm$0.00 | 58.08$\pm$0.01 | 76.17$\pm$0.17 | 74.69$\pm$0.35 | 78.40$\pm$0.01 | 73.91$\pm$0.02 |

**Table A.7: Results of node classification under distribution shifts.**
| Method | Pubmed | Cornell |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Degree | Closeness | Degree | Closeness |  |  |  |  |  |
| Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 | Micro-F1 | Macro-F1 |  |
| TopKPool | 81.66$\pm$0.32 | 81.11$\pm$0.38 | 85.66$\pm$0.09 | 82.36$\pm$0.24 | 34.07$\pm$2.10 | 22.35$\pm$2.92 | 57.78$\pm$4.80 | 25.56$\pm$8.37 |
| SAGPool | 83.07$\pm$1.02 | 82.57$\pm$1.07 | 85.96$\pm$0.34 | 82.73$\pm$0.38 | 36.30$\pm$4.19 | 24.39$\pm$4.03 | 61.48$\pm$4.57 | 31.54$\pm$8.24 |
| ASAPool | 82.11$\pm$0.12 | 81.94$\pm$0.55 | 85.02$\pm$0.54 | 82.12$\pm$0.77 | 34.07$\pm$1.05 | 20.94$\pm$3.40 | 60.74$\pm$2.77 | 26.33$\pm$2.21 |
| PANPool | 81.65$\pm$0.12 | 81.15$\pm$0.11 | 85.55$\pm$0.03 | 82.16$\pm$0.20 | 35.56$\pm$0.00 | 23.55$\pm$0.90 | 54.07$\pm$2.10 | 18.58$\pm$3.28 |
| COPool | 80.42$\pm$0.14 | 79.82$\pm$0.20 | 84.99$\pm$0.35 | 81.81$\pm$0.29 | 35.56$\pm$1.81 | 22.64$\pm$3.21 | 49.63$\pm$6.87 | 21.09$\pm$3.44 |
| CGIPool | 80.35$\pm$1.33 | 79.69$\pm$1.32 | 84.97$\pm$0.75 | 81.74$\pm$0.90 | 36.30$\pm$4.19 | 25.34$\pm$4.15 | 60.00$\pm$1.81 | 27.48$\pm$3.60 |
| KMISPool | 83.41$\pm$0.02 | 82.92$\pm$0.03 | 85.66$\pm$0.07 | 82.50$\pm$0.12 | 34.81$\pm$1.05 | 22.99$\pm$2.00 | 58.52$\pm$5.54 | 24.70$\pm$8.45 |
| GSAPool | 83.15$\pm$0.81 | 82.61$\pm$0.83 | 85.83$\pm$0.32 | 82.70$\pm$0.50 | 35.56$\pm$1.81 | 23.23$\pm$1.21 | 62.22$\pm$1.81 | 30.06$\pm$4.20 |
| HGPSLPool | OOM | OOM | OOM | OOM | 34.07$\pm$1.05 | 21.39$\pm$1.41 | 57.78$\pm$1.81 | 23.63$\pm$1.61 |

## Peer Reviews

confidence: 3
contribution: 1
initial_rating: 3
presentation: 3
questions: 1.	What does Rank in Tables 2, 3, and 4 mean? Please explain it in the paper.
2.	What does feature masking mean? Please explain it in the paper.
3.	KMISPool and ParsPool do not demonstrate the best overall performance on Cornell, please double-check the first claim made in section 4.2 Performance on Node Classification.
4.	For the visualization, would it be possible to add more numeric metrics to support the claim? E.g., the overlap ratio.
5.	Would it be possible to include experiments of node pooling methods on node classification tasks under distribution shifts to validate the findings of the paper?
6.	Would it be possible to address the concerns mentioned above? Please at least consider rewriting the findings in the introduction to make them consistent with the claims in the experiment sections and the conclusion.
soundness: 2
strengths: 1.	Comprehensive benchmark. The paper provides a novel and publicly available benchmark, which comprehensively includes 17 state-of-the-art graph pooling methods, 28 diverse datasets, and 3 graph learning tasks on both graph and node levels. The comprehensiveness helps set a new standard for evaluating graph pooling methods.

2.	Diverse evaluation criteria. The authors evaluate different graph pooling methods from three diverse dimensions: effectiveness, robustness, and generalizability, which provide a comprehensive view of the strengths and weaknesses of different methods.
summary: This paper presents a comprehensive benchmark for evaluating graph pooling methods. It covers 17 pooling techniques and tests their performance across 28 graph datasets in three tasks, including graph classification, graph regression, and node classification. The study focuses on three key areas: effectiveness, robustness under edge and feature noises, and generalizability under out-of-distribution shifts. Additionally, the paper provides analyses of computational efficiency, backbone variations, parameter sensitivity, and visualization of different baselines. The authors emphasize that their open-source benchmark is intended to standardize the evaluation of graph pooling techniques, facilitate reproducibility, and guide future research in deep geometric learning.
weaknesses: 1.	Lack of novelty. The findings, especially those presented in the introduction section, are straightforward and naïve, lacking novel insights into graph pooling methods that can guide future research. (1) The author lists six observations in the introduction part, some of which are too specific (e.g. the third point) and should only be mentioned in the corresponding experiment sections. (2) The findings in this paper are simple empirical observations of the experiment results but fail to provide valuable insights that can guide future research. For example, the authors simply observe that node clustering pooling methods outperform node dropping pooling methods in terms of robustness and generalizability, but no underlying explanations are given. (3) Some of the findings are also well-known, such as the high computational cost of node clustering pooling methods [1].

2.	Questionable or inconsistent claims. (1) The authors claim that node clustering pooling methods outperform node dropping pooling methods in terms of robustness, whereas Table 5 shows that node dropping pooling methods achieve the best results in 8 different settings, compared to node clustering methods that only achieve the best results in 5 settings. (2) The authors claim that node clustering pooling methods outperform node dropping pooling methods in terms of generalizability, but Table 5 (node classification under distribution shifts) only includes node dropping pooling methods. (3) The 4th finding in the introduction is not mentioned or verified in the corresponding experiment section. (4) The 1st finding in the introduction partially overlaps with the 6th finding.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: Please see weaknesses.
soundness: 3
strengths: - The paper provides a comprehensive literature overview for graph pooling.

- Open source codes reimplementing the existing methods with commonly used datasets are valuable assets, which might allow fairer performance comparisons in future works.

- The work provides remarkable observations for the performances of existing works with respect to their effectiveness, robustness, and generalizability.

- The paper is written well and easy to follow.
summary: The paper benchmarks 17 state-of-the-art approaches and 28 different graph datasets used in graph pooling for graph-level and node-level tasks. The existing methods are evaluated based on their effectiveness, robustness, and generalizability, where valuable discussions/observations are provided for their performance.
weaknesses: - For graph-level tasks, the Authors seem to only use GCN as the GNN backbone. More advanced GNN designs, like GIN, can also be incorporated here.

- Additional analyses for the effect of graph structure (homophilic vs. heterophilic) on existing works' performance can provide further insights. For such an analysis, synthetic graphs may be generated with different structural properties.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: Please check the weaknesses.
soundness: 2
strengths: - The paper makes a significant effort to evaluate these methods, offering a fair comparison for future studies.

- It includes the generalization and robustness aspects that have been overlooked in previous evaluations.
summary: In summary, this paper provides a comprehensive evaluation of 17 graph pooling methods using 28 different datasets across three tasks. This paper evaluates these methods from the perspectives of performance, robustness, and generalizability.
weaknesses: - This paper still lacks the additional suggestions and directions for the future development of graph learning methods. Which directions and challenges should be considered when designing new methods?

- Besides, it does not sufficiently explore the relationship between graph characteristics and the selection of graph learning methods, which is crucial for a comprehensive evaluation benchmark.

---

confidence: 4
contribution: 1
initial_rating: 3
presentation: 2
questions: 4.	The taxonomy of graph pooling methods in lines 46-47, is it originally proposed in this paper?
5.	What is the ‘Feature map’ in line 106?
6.	Line 107 ‘In this work, we focus on hierarchical pooling approaches’. Does it mean this benchmark only contains hierarchical pooling approaches? According to a recent survey, the graph pooling community is obviously more than hierarchical pooling approaches.
7.	Lines 125-126 ‘during the connection’ should be ‘during the pooling process’ ?
8.	Lines 125-126 Why is the ‘connection component’ necessary? Dropping and clustering nodes, according to your taxonomy, can affect the topology too.
9.	Lines 209-210 ‘We perform a hyperparameter search for all pooling methods; detailed information can be found in Appendix E’. However, Appendix E does not provide any details on how the hyperparameter search was conducted. Further experiments should be conducted for hyperparameter sensitivity to enable fair comparison, or at least performance reported in the original paper VS in your paper.
10.	Experiments on robustness against noise, specifically Tabel 5 and 6, were not conducted on all datasets. Why are these datasets selected while the others are not?
11.	The analysis of why method A performs well while method B does not on certain tasks and scenarios is quite disorganized. I suggest organizing it into a table like a survey paper and presenting each method’s design, along with an analysis of its strengths and weaknesses, in a separate section.
12.	See weakness 3. I suggest synthesizing the code you’ve collected and standardizing the input, output, and evaluation protocols.

[1] Liu C, Zhan Y, Wu J, et al. Graph Pooling for Graph Neural Networks: Progress, Challenges, and Opportunities. IJCAI 23
soundness: 3
strengths: 1.	This paper presents a collection of 17 graph pooling methods over 28 different graph datasets, and the authors claim to be the first work of benchmarking.
summary: This paper presents a comprehensive graph pooling benchmark, which incorporates 17 state-of-the-art graph pooling approaches and 28 diverse datasets across graph classification, graph regression, and node classification. Experiments from three perspectives, i.e., effectiveness, robustness, and generalizability were conducted to compare collected methods with each other.
weaknesses: 1.	Further experiments should be conducted for hyperparameter sensitivity to enable fair comparison.
2.	Experiments on robustness against noise, specifically Tabel 5 and 6, were not conducted on all datasets. The same problem exists in experiments on generalization ability.
3.	The biggest weakness of this paper is that it only presents baseline results of existing methods, lacking new insights for researchers who are already familiar with this field, or a code framework that newcomers can use to quickly develop their own insights.