Neural Pooling for Graph Neural Networks
Authors: Sai Sree Harsha
Deepak Mishra
Venue:  (2021)
Keywords: graph neural networks
graph pooling
representation learning

Abstract: Tasks such as graph classification, require graph pooling to learn graph-level representations from constituent node representations. In this work, we propose two novel methods using fully connected neural network layers for graph pooling, namely Neural Pooling Method 1 and 2. Our proposed methods have the ability to handle variable number of nodes in different graphs, and are also invariant to the isomorphic structures of graphs. In addition, compared to existing graph pooling methods, our proposed methods are able to capture information from all nodes, collect second-order statistics, and leverage the ability of neural networks to learn relationships among node representations, making them more powerful. We perform experiments on graph classification tasks in the bio-informatics and social network domains to determine the effectiveness of our proposed methods. Experimental results show that our methods lead to an absolute increase of upto 1.2% in classification accuracy over previous works and a general decrease in standard deviation across multiple runs indicating greater reliability. Experimental results also indicate that this improvement in performance is consistent across several datasets.


Under review as a conference paper at ICLR 2021
NEURAL POOLING FOR GRAPH NEURAL NETWORKS
Anonymous authors
Paper under double-blind review
ABSTRACT
Tasks such as graph classification, require graph pooling to learn graph-level rep- resentations from constituent node representations. In this work, we propose two novel methods using fully connected neural network layers for graph pooling, namely Neural Pooling Method I and 2. Our proposed methods have the ability to handle variable number of nodes in different graphs, and also invariant to the isomorphic structures of graphs. In addition, compared to existing graph pooling methods, our proposed methods are able to capture information from all nodes, collect second-order statistics, and leverage the ability of neural networks to learn relationships among node representations, making them more powerful. We per- form experiments on graph classification tasks in the bio-informatics and social network domains to determine the effectiveness our proposed methods. Exper- imental results show that our methods lead to an absolute increase of upto 1.2% in classification accuracy over previous works and a general decrease in standard deviation across multiple runs indicating greater reliability. Experimental results also indicate that this improvement in performance is consistent across several datasets.
INTRODUCTION
Over the past several years, there is growing number of applications where data is generated from non-Euclidean domains and is represented as graphs with complex relationships and interdepen- dency between entities. Deep learning generalised from grid-like data the graph domain has led
to the development of the remarkably successful Graph Neural Networks (GNNs) (Fan et al.) 2019
Gao et al. 2019 Ma et al., 2019a| Wang al., 2019b) and its numerous variants such the Graph
Convolutional Network (GCN) (Kipf & Welling 2017j, GraphSAGE (Hamilton et al.) (017). graph
attention network (GAT) (Veličković et al. 2018). jumping knowledge network (JK) (Xu et al. 2018), and graph isomorphism networks (GINs) (Xu et al., 2019), etc.
Pooling common operation in deep learning on grid-like data, such as images. Pooling lay- provide approach to down sampling feature maps by summarizing the presence of features in patches of the feature map. It reduces dimensionality and also provides local translational in- variance. In the case of graph data, pooling is used to obtain a representation of graph using its constituent node representations. However, it is challenging to develop graph pooling methods due the some special properties of graph data such as the variable number of nodes in different graphs and the isomorphic structures of graphs. Firstly, the number of nodes varies in different graphs, while the graph representations are usually required to have the same fixed size fit into other downstream machine learning models where they are used for tasks such as classification. There- fore, graph pooling should be capable of handling the variable number of node representations as inputs and producing fixed-sized graph representations. Secondly, unlike images and texts where we can order pixels and words according to the spatial structural information, there no inherent order- ing relationship among nodes in graphs. Therefore, isomorphic graphs should have the same graph representation, and hence graph pooling should give the same output by taking node representations in any order as inputs.
Our main contributions in this work are two novel graph pooling methods, Neural Pooling Method 1 and 2. These new pooling methods allow to do the following.i) produce the same dimensional graph representation for graphs with variable number of nodes, ii) remain invariant to the isomorphic structures of graphs, iii) collect second- order statistics, iv) leverage trainable parameters in the form

---

Under review as a conference paper at ICLR 2021
of fully connected neural networks to learn relationships between underlying node representations to generate high quality graph representations which are then used for graph classification tasks.
Experiments performed on four benchmark bio-informatics datasets and five popular social net- work datasets to demonstrate the effectiveness and superiority of our proposed graph pooling meth- ods. Experimental results show that our methods lead to improvement in classification accuracy over existing methods and are also more reliable as compared to previous works.
2 RELATED WORK
2.1 GRAPH NEURAL NETWORKS
A graph can be represented by its adjacency matrix and node features. Formally, for a graph G consisting of nodes, its topology information can be represented by an adjacency matrix A E {0. 10. and the node features can be represented as X E R assuming each node has d-dimensional feature vector. GNNs learn feature representations for different nodes using these matrices (Gilmer et al.) 2017). Several approaches are proposed to investigate deep GNNs, and they generally follow neighborhood information aggregation scheme (Gilmer et al., 2017; Xu (et al. 2019; Kipf & Welling 2017; Hamilton et al., 2017 Veličković et al. 2018). In each step, the representation of node is updated by aggregating the representations of its neighbors. Graph Convolutional Networks (GCNs) are popular variants of GNNs and inspired by the first order graph Laplacian methods (Kipf & Welling 2017). Graph pooling is used to connect embedded graphs outputted by GNN layers with classifiers for graph classification. Given a graph, GNN layers pro- duce node representations, where each node is embedded as a vector. Graph pooling is applied after GNN layers to process node representations into single feature vector as the graph representation. A classifier takes the graph representation and performs graph classification.
2.2
GRAPH POOLING
Early studies employ simple methods such as averaging and summation as graph pooling (Xu et al,,
2019
Duvenaud et al. 2015 Defferrard et al., 2016). However, averaging and summation do not
capture the feature correlation information, curtailing the overall model performance (Zhang et al.)
2018).
Other studies have proposed ad advanced graph pooling methods, including DIFFPOOL (Ying
SORT-POOL (Zhang et al.) 2018). TOPKPOOL (Gao & Ji 2019). SAGPOOL (Lee al. 2019 and EIGENPOOL (Ma et al. 2019b). and achieve great performance on multiple benchmark datasets. EIGENPOOL involves the computation of eigenvectors, which is slow and expensive. DIFFPOOL (Ying et al. 2018) treats the graph pooling as node clustering problem. A cluster of nodes from the original graph are merged to form a new node in the new graph. DIFF- POOL (Ying et al., 2018) proposes to perform the graph convolution operation on node features to obtain node clustering assignment matrix. Intuitively, the class assignment of a given node should depend on the class assignments of other neighbouring nodes. However, DIFFPOOL does not ex- plicitly consider high-order structural relationships, which that are important for graph pooling. SORTPOOL (Zhang et al. 2018). TOPKPOOL (Gao & Ji 2019), and SAGPOOL (Lee et al.) 2019) learn to select important nodes from the original graph and use these nodes build new graph. They share the similar idea to learn a sorting vector based on node representations, which indicates the importance of different nodes. Then only the top important nodes are selected form new graph while the other nodes are ignored. However, the ignored nodes may contain important fea- tures and this information is lost during pooling. is worth noting that all the graph pooling methods mentioned till now only collect first-order statistics (Boureau et al. 2010). A recent study has pro- posed second order graph pooling methods POooolsmmap and Sooolatttnttion (Wang & Ji 2020). In this work, we propose two novel methods using fully connected neural network layers for graph pooling, namely Neural Pooling Method I and 2. Compared to existing graph pooling methods, our proposed methods able capture information from all nodes, collect second-orde order statistics, and leverage the ability of neural networks to learn relationships among node representations, making them more powerful.
et al.
2018).
2

---

Under review as a conference paper at ICLR 2021
3 METHODOLOGY
3.1 PROPERTIES OF GRAPHI POOLING
Consider graph (A, X) represented by its adjacency matrix A {0, 1 1 and node feature matrix X E IRn where the number of nodes in and d the dimension of node features. The node features may come from node labels or node degrees. Graph neural networks are known be powerful in learning good node representation matrix H from A and X:
= [h1, h22
n GNN(A, X) E R" X
(1)
where rows H. h R 1, n, are representations of nodes, and is the dimension of the node representation obtained from the GNN and depends the architecture of the GNN. The task that we focus in this work to obtain a graph representation vector he from H, which is then fed into classifier to perform graph classification:
hg = g([A), H) E TR
where g(:) is the graph pooling function and is the dimension of ha. Here, [A] means that the information from A can be optionally used in graph pooling. For simplicity, we omit it in the following discussion.
Note that the function g() must satisfy two requirements to serve as graph pooling. First, g() should be able to take H with variable number of rows the inputs and produce fixed-sized outputs. Specifically, different graphs may have different number of nodes, which means that n is a variable. On the other hand, c, which is the dimension of the graph representation hg is supposed to be fixed to fit into the classifier. Second, g(-) should output the same he when the order of rows of H changes. This permutation invariance property is necessary concrete, two graph 9, = (A,, X,) and G2 = (A2, X2) are isomorphic, GNNs will output the same multi of node representations. That there exists permutation matrix P {0, 1]md x such that H, = PH2, for H, = GNN(A), X1) and H₂ = GNN(A2, X2). However, the graph representation computed by g(-) should be the same, i.e., g (H1 = g(2)) if H1 PH
handle isomorphic graphs. To be
3.2 NEURAL POOLING METHOD
Our first proposed method called Neural Pooling Method 1. Consider a node representation matrix H obtained following Equation 1 in Section 3.1
f'x8
H
H²
I
f'x
8xf
8xf
8x1
FC
f'x8
Graph Representation
Neural
Neural
G
Network
Network
Layer 1
Layer
Figure 1: Illustaaioon of our proposed Neural Pooling Method This an example for graph G with 8 nodes. GNNs can learn representations for each node and graph pooling processes node representations into graph representation vector.
H is passed through Fully Connected Neural Network Layer (FCT.) to obtain H' as:
H' = FCL (H) E R where f' <
(4)
After this H' is again passed through second Fully Connected Neural Network Layer (FCL?) to obtain

---

Under review as a conference paper at ICLR 2021
==F R"x1
(5)
Finally the graph representation hg obtained as:
= = H QE=
(6)
where HT denotes the transpose of H'.
Neural Pooling Method always outputs an aonn'"diinn f'-dimensional graph representation for H R x regardless of the value of is also invariant to permutation that it outputs the same graph representation, even when the order rows of H changes.
Intuition: The FCL* performs the role of reducing the dimensionality of the input node represen- tations. The trainable parameters of this FCL* can thought of as learning a mapping from the the f" f'-dimensional space. The FCL reduces the f' d'-dimensional node representations to dimensional representation, Q. H' R can be viewed H' [11, l2, E IR", 2=1,2...., f'. The vector 1j encodes the spatial distribution of the j-th feature in the graph. Based on this view, H Q able to capture the topology information and Q can be thought as roughly encoding the position of nodes by learning the weights according to which the j-th feature is aggregated across the nodes.
1 where 1
Neural Pooling Method hence, leverages the ability of neural networks to learn the topological structure as well as correlation among the node representations in H. It captures the essential features and connections between underlying data. It also reduces the dimensionality of H, and results in an accurate representation of the input graph.
3.3 NEURAL POOLING METHOD
Our second proposed method is called Neural Pooling Method 2. Consider a node representation matrix H obtained following Equation in Section 3.1
8x8
H²
fxx
Flatten
fx8
8xf
8 8x
xxf
FC
FC Neural
Gruph
Neural
Representation
G
Network
Network
Layer
Layer
Figure 2: Illustration of our proposed Neural Pooling Method 2. This is example for graph with 8 nodes. GNNs can learn representations for each node and graph pooling processes node representations into graph representation vector.
H is passed through Fully Connected Neural Network Layer (FCL') to obtain H" as:
(H) After this H" is again passed through second Fully Connected Neural Network Layer (FCL3)
H" = FCL (H) E RR" f" where f" <
(8)
obtain as:
H' = FCL' R where f < f"
(9)
Finally the graph representation hG obtained as:
hg = Flattent T H' E R x1
(10)
where HT denotes the transpose of H'
4

---

Under review as a conference paper at ICLR 2021
The FCL' performs the role of reducing the dimensionality of the imput node represen- tations. The trainable parameters of this FCL can be thought of as learning a mapping from the f to the f" di-dimensional space. The FCL further reduces the f" di-deennsion node representations to dimensional representation. H' E R can be viewed H' = [4,, l2,, ....... 1 where L E "R",j==,, 2, f'. The vector 1 encodes the spatial distribution of the j-th feature in the graph. Based on this view, H
Intuition:
T H' is able to capture the topology information.
Similar the previous method, Neural Pooling Method 2 satisfies both the conditions of graph pooling which is that it always outputs an - dim-iiinnio graph representation for E RR" x regardless of the value of It is also invariant to permutation so that it outputs the same graph representation when the order of rows of H changes.
EXPERIMENTAL SETUP
We perform experiments on graph classification tasks in the bio-informatics and social network domains to demonstrate the effectiveness and superiority of our proposed methods, namely Neural Pooling Methods 1 and Details of datasets and parameter settings are described below.
4.1 DATASETS
We use nine graph classification datasets from (Yanardag & Vishwanathan 2015), including four bioinformatics datasets and five social network datasets. Only bioinformaties datasets come with node labels. For the social network datasets, we use one-hot encoding of node degrees as features. The details of the datasets summarized in Tande[[]and Table
MUTAG (Debnath et al. 1991 is bioinformatics dataset of 188 graphs representing nitro compounds. The task is to classify each graph by determining whether the compound is mutagenic aromatic or heteroaromatic.
PTC (Toivonen et al. 2003 is a bioinformatics dataset 344 graphs representing chemical compounds. Each node comes with one of 19 discrete node labels. The task is to predict the rodent carcinogenicity for each graph.
PROTEINS (Borgwardt et al. 2005) is bioinformatics dataset of 1,113 graph structures of proteins. Nodes the graphs refer to secondary structure elements (SSEs) and have discrete node labels indicating whether they represent a helix, sheet or turn. And edges mean that two nodes are neighbors along the amino-acid sequence or in space. The task predict the protein function for each graph.
NCI1 (Wae al. 2008) is bioinformatics dataset of 4,1 10 graphs representing chemical compounds. The graph classification label is decided by anti-cancer screens for ability to suppress inhibit the growth of panel of human tumor cell lines.
COLLAB a scientific collaboration dataset of 5,000 graphs corresponding to ego- networks. The dataset is derived from 3 public collaboration datasets (Leskovec et al. 2005). Each ego-network contains different researchers from each field and is labeled by the corresponding field. The three fields are High Energy Physics, Condensed Matter Physics, and Astro Physics.
BOOB-B is a movie collaboration dataset of 1,000 graphs representing ego- networks for actors/actresses. The dataset derived from collaboration graphs on Action and Romance genres. In each graph, nodes represent actors/actresses and edges simply mean they collaborate the same movie. The graphs are labeled by the corresponding genre and the task is to identify the genre for each graph.
IMDB-MULT is multi-class version of and has three extra genres, namely, Comedy, Romance and Sci-Fi.
It contains 1,500 ego-networks
a dataset of 2,000 graphs where each graph represents an online dis- cussion thread. Nodes in graph correspond users appearing in the corresponding dis- cussion thread and an edge means that one user responded to another. TrolIXChromosomes and atheism are discussion-based subreddits, forming two classes to be classified.
REDDIT-BINARE
5

---

Under review as a conference paper at ICLR 2021
Table 1: Details of bioinformaties datasets
Name
MUTAG
PTC
PROTEINS
NCII
graphs
188
344
1113
4110
# classes
2
2
2
nodes(max)
28
109
620
111
nodes(avg.)
18.0
25.6
39.1
29.9
Table 2: Details of social network datasets
Name
COLLAB
IMDB-B
IMDB-M
RDT-B
RDT-M5K
graphs
5000
1000
1500
2000
5000
classes
2
3
5
# nodes(max)
492
136
89
3783
3783
# nodes(avg.)
74.5
19.8
13.0
429.6
508.5
REDDIT-MULTISK is similar dataset REDDIT- BINARY, which contains 5,000 graphs. The difference lies in that REDDIT-MULTISK crawled data from five different subreddits, namely, worldnews, videos, Animal,, aww and mildlyinteresting. And the task is to identify the subreddit of each graph instead of determining the type of sub- reddits.
4.2 TRAINING AND EVALUATION
Following YYanardag & Vishwanathan 2015 Niepert al., 2016), model performance is evaluated using 10-fold cross-validation
reported as the average and standard deviation of validation ac- curacies across the 10 folds. For GNNs, we follow the same training process in (Xu et al.) 2019). The GNN has layers. Each multi-layer perceptron (MLP) has layers with batch normalization [loff & Szegedy 2015). Dropout (Srivastava al. 2014) is applied in the classifiers. The Adam Kingma & Ba 2015) optimizer is used with the learning rate initialized as 0.01 and decayed by 0.5 every 50 epochs. The number of total epochs is selected according to the best rrossvaaliatiio accuracy. We tune the number of hidden units (16, 32, 64) and the batch size (32, 128) using grid search.
4.3 BASELINES
We compare methods with various other graph pooling methods the graph classification task, including DIFFPOOL (Ying et al. 2018), SORT-POOL (Zhang et al. 2018)) TOPKPOOL (Gao & Ji 2019), SAGPOOL (Lee al. 2019). and EIGEN-POOL, (Ma et al. 2019b). DIFFPOOL maps nodes to a pre-defined number of clusters but is hard to train. EIGENPOOL involves the computa- tion of cigenvectors, which slow and expensive. SORTPOOL. SAGPOOL and TOPKPOOL rely on the top-K sorting to select a fixed number (K) of nodes and order them, during which the infor- mation from unselected nodes discarded. We also compare with some recent methods including COVPOOL (Wang et al. 2019a), ATTNPOOL (Girdhar & Ramanan (017) as well as second order pooling methods SOPoolhimap and SOPool (Wang Ji 2020).
RESULTS & DISCUSSION
The results our experiments are summarized Table and Table FFrm the results we can that methods lead to improvement in classification accuracy over existing methods and

---

Under review as a conference paper at ICLR 2021
Table 3: Comparison results of our proposed methods with other graph pooling methods bioin- formatics datasets. Results shown are the average classification accuracy and standard deviation across 10-fold cross-validation.
PTC
PROTEINS
MUTAG
NCI1
SUM/AVG et al. 2019)
64.6 = 7.0
76.2 2.8
89.4 5.6
82.7 = 1.7
DIFFPOOL Ying et al.
2018
66.1 + 7.7
7.8
94.8 + 4.8
76.6 1.3
(ZTPOOOL(Zhang et al.
2018
69.5 + 6.3
79.2 + 3.0
95.2 + 3.9
78.9 + 2.7
TOPKPOOL/Gaa Ji 2019)
68.4 + 6.4
79.1 2.2
94.7 3.5
79.6 1.7
SAGPOOL(Lee et al.
2019
69.0 = 6.6
78.4 + 3.1
93.9 + 3.3
79.0 = 2.8
Girdar & Ramanan
2017)
71.2 = 8.0
77.5 3.3
93.2 + 5.8
80.6 2.1
EIGENPOOL Ma et al.
2019b
76.6 + 2.3
80.6 + 4.3
77.0 + 2.3
COVPOOL Wang et al.
2019a)
73.3 +: 5.1
80.1 + 2.2
95.3 + 3.7
83.5 1.9
SOPOOL Wang
2020
72.9 - 6.2
79.4 + 3.2
93.6 - 4.1
82.8 1.4
SOPOOLoimap Wang & Ji
2020)
75.0 +: 4.3
80.1 + 2.7
95.3 + 4.4
83.6 + 1.4
Neural Pooling ((urr))
74.5 +: 3.7
80.6 2.7
94.0 2.3
83.1 1.2
Neural Pooling 2(ours)
76.2 4: 4.2
79.6 + 3.0
95.5 + 2.4
83.4 + 1.9
Table 4: Comparison results of our proposed methods with other graph pooling methods on social network datsets. Results shown are the average classification accuracy and standard deviation across 10-fold cross-validation.
COLLAB
RDT-B
IMDB-B
IMDB-M
RDT-M5K
SUM/AVG 80.2 + 1.9
92.4 + 2.5
75.1 5.1
52.3 2.8
57.5 + 1.5
DIFFPOOL
75.3 2.2
74.4 + 4.0
50.1 = 3.2
SORTPOOL
78.2 + 1.6
81.6 = 4.6
77.5
53.1 2.9
48.4 + 4.8
TOPKPOOL
79.6 99.6
77.8 + 5.1
53.7 = 2.8
SAGPOOL
78.9 1.7
77.8 + 2.9
53.1 2.8
ATTNPOOL
81.8 2.2
92.5 2.3
77.1 + 4.4
53.8 2.5
57.9 1.7
COVPOOL
79.3 + 1.8
90.3 - 3.6
72.1 5.1
47.8 = 2.7
58.4 + 1.7
SOPOOL
81.1 1.8
91.7 2.7
78.1 + 4.0
54.3 2.6
58.3 1.4
SOPOOL -bimap
79.9 + 1.9
89.6 + 3.3
78.4 + 4.7
54.6 - 3.6
58.4 1.6
Neural Pooling ((ours)
80.5 - 1.5
90.6 + 2.3
79.0 + 2.3
55.1 2.2
58.5 + 1.8
Neural Pooling 2(ours)
81.0 - 1.7
91.5 = 3.0
78.5 + 2.4
54.4 1.9
59.1 1.4
are also more reliable as compared previous works observed from the lower values of standard deviation. This enhancement performance is consistent across all the datasets. The results may be attributed to the fact that compared to existing graph pooling methods, pooling methods are able to use information from all nodes, collect second-order statistics, and leverage the ability of neural networks learn from underlying data, making them powerful. The Neural Pooling methods utilize the ability of neural networks to learn the topological structure as well as correlation among the node representations in, capturing essential features and connections between underlying data.

---

Under review as a conference paper at ICLR 2021
6 COMPLEXITY
Consider graph G = (A, X) represented by its adjacency matrix A E {0, 1 and node feature matrix X E IR" d, where ni is the number of nodes in and d the dimension of node features. Consider, H [k], h2,
J² = GNN(A, X) E R" where rows of H. h; E R/, i = 2, nt are representations of n nodes, Consider a direct application of second-order graph
pooling obtain the graph representation hg as:
hg Flatten H) x1
(11)
where H denotes the transpose of H.
However, it causes explosion in the number of training parameters in the following classifier when f is large, making the learning process harder to converge and easier to overfit. While each layer in GNN usually has outputs with a small number of hidden units (e.g. 16. 32, 64), it has been pointed out that graph representation learning benefits from using information from outputs of all layers, obtaining better performance and generalization ability. outputs across all layers in a GNN. In this case. H has large final f. making direct use of second- order pooling infeasible. For example, if GNN has 5 layers and each layer's outputs have 32 hidden units, / becomes 32 x 5 160. Suppose hg is sent into a 1-layer fully-connected classifier for graph categories in a graph classification task. It results in 160 = 25, 600c training parameters, which is excessive. We omit the bias term for simplicity. On the other hand, both of proposed novel graph pooling methods significantly reduce the number of training parameters. In the case of Neural Pooling Method I, considering the previous example if f' is chosen to be 64, and f is 160, then the total number of trainable parameters in the 2 FCLs and a 1-layer fully-connected c class classifier will be (160 x 64) 64 + 64c 10, 304 + 64c notably reducing the number of parameters as compared to 25, 600c. In the case of Neural Pooling Method 2, if f is chosen to be 64, ʃ' as 32 and ʃ is 160, then the total number of trainable parameters in the 2 FC Ls and I-layer fully-connected c class classifier will be (160 x 64) + (64 x 32) + 322c = 12, 288 + 1024c again reducing the number of parameters when compared to 25, 600c.
is usually achieved by concatenating
7 CONCLUSION
In this work, we propose to perform graph representation learning with Neural Pooling, by pointing out that Neural Pooling can naturally solve the challenges of graph pooling. Neural Pooling is more powerful than existing graph pooling methods, since it is capable of using all node information, col- lecting second-order statisties that encode feature correlations and topology information and lever- age the ability of neural networks to learn from underlying data, making them more powerful. Our proposed methods solve the practical problems incurred by directly using second-order pooling with GNNs. To demonstrate the effectiveness and superiority of our methods, perform experiments on graph classification tasks in the bio-informatics and social network domains to demonstrate the effectiveness and superiority of our proposed methods. Experimental results show that our methods improve the performance significantly and consistently. An interesting future work direction could be extend our methods for hierarchical graph pooling, where the output is is pseudo graph with fewer nodes than the input graph. It is used to build hierarchical GNNs, where hierarchical graph pooling is used several times between GNN layers gradually decrease the number of nodes.
REFERENCES
KM Borgwardt, Stefan Schönauer, SVN Vishwanathan, Alexander Smola, and HP Kriegel. Protein function prediction via graph kernels. Bioinformatics, 21 Suppl 1:i47-56, 01 2005.
Y-Lan Boureau, Jean Ponce, and Yann LeCun. A theoretical analysis of feature pooling in visual recognition. In Johannes Fürnkranz and Thorsten Joachims (eds.), Proceedings of the 27th In- ternational Conference on Machine Learning (ICML-10) pp. 111-118, Haifa, Israel, June 2010. Omnipress. BoLLttt:/// wwww wtt.m/m1111..././.eeoomomo 638 pdff
Asim Kumar Debnath, Rosa L Lopez de Compadre, Gargi Debnath, Alan J Shusterman, and Cor- win Hansch. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro com-
8

---

Under review as a conference paper at ICLR 2021
pounds. correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry, 34(2):88-7777, 1991.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filterin. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.). Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5- 10, 2016, Barcelona, Spain, pp- 3837-3845, 2016.
David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alan Aspuru-Guzik and Ryan P Adams. Convolutional networks graphs for learning molecular fingerprints. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems 28, pp. 2224-2233 Curran Associates, Inc., 2015.
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. Graph neural net- works for social recommendation. In The World Wide Web Conference, WWW '19, pp. 417-426, New York, NY, USA. 2019. Association for Computing Machinery. ISBN 978 4503666448. doi: 10.1 10.1145/3308558.311133 134888. BtRp:ttps:///oi.ccro http:////..bboooo...............
1111/1101/51..... 001085558833313441 3313488
Hongchang Gao, Jian Pei, and Heng Huang. Conditional random field enhanced graph convolu-
tional neural networks. In Proceedings of the 25th ACM SIGKDD International Conference on
Knowledge Discovery & Duta Mining, KDD '19, pp. 276-284, New York, NY, USA, 2019. Asso- ciation for Computing Machinery. ISBN 4578645636216 doi: 10.1 10.1145/329250003333888 URL https://doo /10 1145/3292500 303092205000333383333
Hongyang Gao and Shuiwang Ji. Graph u-nets. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICMI. 2019, 9- 15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 2092-2092. PMLR, 2019. URL http:://proceedingg. mir press/v977 gaol 9a. html
Justin Gilmer, Samuel Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 1263-1272. PMLR, 2017. URL http:://proceeingg..rt www..ngs...r......mm.. www..r.pees//ll. sotess/vv0/gge lelggimmeerr lmerl 7a html
Rohit Girdhar and Deva Ramanan. Attentional pooling for action recognition. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vish- wanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA. USA. pp. 34 45, 2017. URL htttp://papezs. nips cc/papex/ 6609-attent ionetttttiionnnae ona -ooo 1 ing -ettfforrrttttoooo ion wwc.gniiooitiiiion
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 1024-1000 1034-1 Curran Associates, Inc., 2017. URL http:: 6703 inductive representation learning on large grwphs.apphp....o pdf
nips cc/paper/
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis R. Bach and David M. Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pp. 448 456. JMLR.org, 2015. URL mlr. press/
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference Learning Representations, ICLR 2015, San Diego CA, USA. May 7-9, 2015, Conference Track Proceedings. 2015. URL http: Tlarxiv.orgaaat
6980

---

Under review as a conference paper at ICLR 2021
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In 5th International Conference on Learning Representations, ICLR 2017. Toulon, France, April 24-26. 2017, Conference Track Proceedings. OpwnReviie..ett 2017 2017. //openreview. wwt. S
htth:::s:
Junhyun Lee, Inyeop Lee, and Jaewoo Kang. Self-attention graph pooling. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning. ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 3734-3743, PMLR, 2019. URL http:t/prrccceeingngg m www.prsss/////1. /eell 9c ht.
Jure Leskovec, Jon Kleinberg, and Christos Faloutsos. Graphs over time: densification laws, shrink- ing diameters and possible explanations. In Proceedings of the eleventh ACM SIGKDD interna- tional conference Knowledge discovery in data mining, pp. 177-187, 2005.
Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. Disentangled graph convolutional networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th Inter- national Conference on Machine Learning, ICML. 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 1212-42211. PMLR, 2019a. URL htkLhhtt::///roceeeee
ngs ml .r.press/vv 99 1 9a htm
Yao Ma, Suhang Wang. Charu C. Aggarwal, and Jiliang Tang. Graph convolutional networks with cigenpooling. KDD 11,, pp. 723-731, New York, NY, USA, 2019b. Association for Computing Machinery. ISBN 978 450362016.... doi: 10.1145/292200.333388. URL httpps://oii
org/10. 1 145 /2 3292500 3330982
Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural net- works for graphs. In Maria-Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of the 33nd International Conference on Machine Learning. ICML 2016, New York City, NY. USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pp. 2014-2023. .MLR.org, 2016. URL httt:://proceee///// eigeeeediingg.... ml prppsss/v48. htt.a....s...r..../e/ees./e/ee n epert html
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15(1): 1929-1958, 2014. (ttp::tt:://dt acm. www.m.orr//eeee wwwp/org//aaaeeee on. www.oocccco..mmmmmmmmmmmmmmmmmm. d=mmm:dd=27331311
Hannu Toivonen, Ashwin Srinivasan, Ross D. King, Stefan Kramer, and Christoph Helma. Sta- tistical evaluation of the Predictive Toxicology Challenge 1000-2001. Bioinformaties, 19(10): 1193-1193, 07 2003. ISSN 1367-4803. doi: 10.10%3bioinfoomattsstttgggg URL https:
100 1 0993/bir intioonnffrro forrat ittmmaattssttttte c
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018. URL
Nikil Wale, Ian A Watson, and George Karypis. Comparison of descriptor spaces for chemical com- pound retrieval and classification. Knowledge and Information Systems, 14(3):347-375 2008
Qilong Wang, Jiangtao Xie, Wangmeng Zuo, Lei Zhang, and Peihua Li. Deep cnns meet global covariance pooling: Better representation and generalization. CoRR, abs/1904.06836, 2019a. httgg//aab/9999453333333 UtRphttp://axxiv.oo
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. Heterogeneous graph attention network. In The World Wide Web Conference, WWw '19, pp. 2022-2032, New York, NY, USA, 2019b. Association for Computing Machinery. ISBN 9781450366748, doi: 3313562
00.1145/3308558331335522 URL httptttss///.oo.ooo http://o..orggg...o http://0..1153338853130
Z. Wang and S. Ji. Second-orde pooling for graph neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Ste- fanie Jegelka. Representation learning on graphs with jumping knowledge networks. In Jen- nifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on
10

---

Under review as a conference paper at ICLR 2021
Machine Learning. ICML 2018, Stockholmssässsan. Stockholm, Sweden, July 10-15, 2018, vol- ume 80 of Proceedings of Machine Learning Research, pp. 5458. PMLR, 2018. URL
http://proceeelnng. mmr wtt.:/ss.w...p..ssssssm http///000xxxlcccec
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful graph neural
networks? In International Conference on Learning Representations, 2019. URL https:
//openzeview. http://oooommea117GG.........
Pinar Yanardag and S.V.N. Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD International Conference Knowledge Discovery and Data Mining, KDD '15. pp. 1365-1374, New York, NY, USA. 2015. Association for Computing Machinery. ISBN 9781450336642. doi: 10.1447778823882784477. URL https:://ci.orgg/// 1145/ 2783258.278388
Zhitao Ying. Jiaxuan You, Christopher Morris, Xiang Ren, William L. Hamilton, and Jure Leskovec. Hierarchical graph representation learning with differentiable pooling. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa Bianchi and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Anmual Conference on Neural Infor- mation Processing Systems 2018, NeurlPS 2018, 3-8 December 2018, Montréal, Canada, pp- 4805-4815, 2018
Muhan Zhang. Zhicheng Cui, Marion Neumann, and Yixin Chen. An end-to-end deep learning architecture for graph classification. AAAI, 2018.


Reviews:
[{"review_id": "zJYI4MkySro", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1). The claim of “the variable number of node representations as inputs and producing fixed-sized graph representations” is weak. This can be easily addressed by simply using average pooling or sum pooling.\n\n\n\n2).. The writing needs to be improved. The quality is very low. For example, even in the introduction, where is the ending of the second paragraph? some  full stop mark is needed. “Neural Pooling Method 1 and 2”, it is better to give them specific names for better reference. “After this H0 is again passed through”, simply not readable..\n\n3). “Neural Pooling Method 2” lacks of intuition. Could you please provide more intuitions on the solution during the rebuttal?\n\n\n4). The results reported seems much worse than results reported in other paper. For example, in the paper below. We can easily see the best result in PTC, best result is 80.41±6.92 while the submission gives only 76.2 ± 4.2  From that perspective, I did not see any advantage in the submission.\n\nStructural Landmarking and Interaction Modelling: on Resolution Dilemmas in Graph Classification,\nhttps://arxiv.org/pdf/2006.15763.pdf\n\n5). The paper is lack of parameter sensertivity analysis, without which the robustness of the proposed algorithms is unknown to me. I would suggest the authors add additional section to discuss that.\n\n6). The draft includes an github link to share the code, however that link indicates the author affiliation information. Clearly I could not access the code on the github as I would have risked infringing anonymity.", "extra_scores": {}, "extra_text": {}}, {"review_id": "hvJrBGY_keL", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes two fully-connected layers based neural graph pooling methods for graph neural networks, named Neural Pooling Method 1 and Neural Pooling Method 2. The first method uses a first FC to reduce the feature dimension and then FC2 to compute the weights to do weighted-average over features for different nodes. The second method uses two FC to reduce the dimension and then compute second-order statistics by Flatten(H^{\\top}H). Experimental results on four datasets (PTC, PROTEINS, IMDB-BINARY, IMDB-MULTI) of two tasks (bioinformatics, social networks) show that the proposed graph pooling method can improve the performance by 0.5%-1.2% accuracy while decreasing the std.\n \nStrengths:\n- The proposed method is simple and motivated by several limitations of current graph pooling methods such as average and summation, DIFFPOOL, SORTPOOL, TOPKPOOL, SAGPOOL, and EIGENPOOL.\n \n- The proposed approach is simple and the experimental results can deliver improvements on several tasks and datasets.\n \nWeaknesses:\n- My biggest concern is that the proposed approach lacks originality and novelty because it is a simplification and variant of SOPOOL from Second-Order Pooling for Graph Neural Networks (Ji and Wang, 2020)\n \n- Based on the author's writing, it is unclear what is the second-order statistics for graph pooling, why it is important to have second-order pooling, and how the proposed method can capture the second-order statistics.\n \n- The proposed graph pooling method is only experimented with 1 underlying particular choice of GNN (Xu et al., 2019), so it is unclear how well the method can perform on other GNN architectures.\n \n- The four datasets only have 2 or 3 classes and upto 620 nodes. So it is clear how well the method can generalize to large-scale graph classification problems. \n \n- The improvement of the proposed methods compared with SOPpool is marginal. For example, On PROTEINS, the accuracy is improved by 0.5% with the same std. On other datasets, the improvements are only at most 1.2%. To show the proposed approach is better, more datasets or tasks should be used. For example, there are five bioinformatics datasets (MUTAG, PTC, PROTEINS, NCI1, DD) and five social network datasets (COLLAB, IMDB-BINARY, IMDB-MULTI, REDDIT-BINARY, REDDIT-MULTI5K).\n\n- There is not enough discussion and analysis of the results. Especially, there should be some analysis to compare the method 1 and method 2: For different datasets, when one method is better than the other? Some examples would be helpful, too. \n \n- While the author explains the proposed method has lower complexity, there is still no formal analysis or quantitative measures of running time from experiments.\n \n- The writing can be improved, In the abstract and introduction, the author should describe the approach briefly and explain its characteristics including why it can handle variable number of nodes, invariant to isomorphic graph structures, capture information of all nodes, and especially why it can collect second-order statistics.\n \n- Furthermore, there is a lot of repetition of problem statements. The problem and notation is introduced formally in section 3.1, but is repeated again and again at the beginning of section 3.2 and section 3.3\n \nQuestions:\n- Do both of your method 1 and method 2 capture second-order statistics? My understanding is that only method 2 captures second-order statistics by computing Flatten(H^{\\top}H). Is this correct?\n- How do you compare your method with SOPpool (Ji and Wang, 2020)?\n- Have you tried other datasets or other tasks?\n- Have you tried your graph pooling approaches on other underlying GNN models?\n- Is your standard deviation in Table 2 based on 1 run of 10 folds or multiple runs of 10-fold cross-validation?\n \nMinor:\n- Please give better names for your approaches and give a better title. \"Neural Pooling Method\" is too general and thus not particular enough to summarize your method.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "22CQLYPWbRD", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this manuscript, the authors propose two novel methods using fully connected neural network layers for graph pooling, namely Neural Pooling Method 1 and 2. compared to existing graph pooling methods, the authors think their methods are able to capture information from all nodes, collect second-order statistics, and leverage the ability of neural networks to learn relationships among node representations, making them more powerful.\n\nPros :\n1. This work studies an important topic but less explored topic, graph pooling.\n2. Propose to perform graph representation learning with Neural Pooling. \n3. Experimental results are interesting.\n\nCons:\n1. The main concern is the lack of novelty, and the technical contribution is very limited. The essential difference between the author’s manuscript and this article is unclear -- ‘Z. Wang and S. Ji. Second-order pooling for graph neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.’? Just one more layer? And there is a lot of overlap in content with this article.\n\n2. It is actually a stealth exchange of concepts that the authors attribute the success of the methods to the neural networks. On the one hand, the reason $H^{'^T}Q$ or  $H^{'^T}H^{'}$ can capture topology information is not that the neural networks can learn $H'$ that contains topology information, but that $H$ itself contains local topology information and $H^{T}H$ is capable of capturing second-order statistics. In fact, the neural networks don’t use topological structure. They play a vital role in reducing dimension and parameters, which is proven in Section 6. On the other hand, $Q$ can be thought of as the weight of the node, not as the correlation among the node representations. Therefore, the statement ‘Neural networks can learn the correlation among the node representation’ lack of further explanation.\n\n3. About the experiment of this manuscript:\na.\tSeveral advanced pooling methods are ignored, especially EigenPooling. Why does it appear in the method comparison in Section 4.3, but disappear in Table 2.\nb.\tDatasets are not enough. Just 4 of the 9 data sets in this article are used -- ‘Z. Wang and S. Ji. Second-order pooling for graph neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.’\n\n4. About the clarity of this manuscript:\na.\tFigures 1 and 2 are not beautiful and have minor issues. $H^{'^T}Q$ and $QH^{'^T}$ are not equal, which can be corrected by slightly changing the diagrams.\nb.\tRepeat a paragraph and an equation many times. For example, in Sections 3.1, 3.2, 3.3 and Section 6, there are redundant.\nc.\tThe lack of punctuation, such as the end of the second paragraph of Section 1 on the first page, and the end of the second last paragraph of Section 3.3 on page 5, etc.\nd.\tMany long sentences make understanding difficult. For example, in section 3.2, there is only one long sentence in a paragraph. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "-HH_TtSshvs", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nIn this paper, the authors proposed two graph pooling methods, i.e., Neural Pooling Method 1 and 2. Both of them are flat pooling strategies, which try to obtain a graph representation directly from its node representations without coarsening graphs step by step. Specifically, the major idea of Neural Pooling Method 1 is to use GCN layer to learn a score for each node. Then, the graph representation is obtained by weighted summing the node representations with the learned scores as weights.  Neural Pooling Method 2 follows a similar design. The difference is that, instead of a single score, it has multiple scores for each node, which leads to a matrix for graph representation. This matrix is then flattened into a vector to serve as the graph representation.\n\nIn general, the novelty of this paper is limited. Some other concerns are listed as follows:\nIt is not clearly motivated why the topology information can be preserved by the two proposed pooling method. It would be better if the authors could provide more explanation.\nThe process in Equation (6) can be viewed as a weighted summation. However, the values in $Q$ seem to be unbounded, which makes the magnitude of the graph representation h_G highly dependent on the size of graphs (i.e., number of nodes). Is it designed in this way on purpose to capture the node size information?  The same issue exists in the Neural Pooling Method 2.\nIt would be better if the users could adopt more datasets should for experiments.\nMinor comments:\nWhen analyzing the complexity of algorithms, it might be better to use general notations instead of concrete numbers.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All four knowledgeable referees have indicated reject due to many concerns. In particular, reviewers pointed out that the novelty of this paper is not clear because the difference from related work is very limited (i.e., the difference from Z. Wang and S. Ji is not clear, other than using one additional layer),  and they were concerned that the results of the experiment are not convincing (For example, the results reported in this paper are significantly inferior to those reported in other papers, the GNN architecture used is limited, and the performance difference especially in the additional experiments in the revision, is very marginal). No reviewers were convinced by the authors' claims even through the author's rebuttal and revision.

One important note: Reviewers have stated that they did not explicitly check the identity of the author and did not pose a problem on this, but if we follow the link specified in the original submission, we can see the identity of the author, which may be considered as a violation of the double-blind policy. This is a small and regrettable mistake, but it can be a serious problem in the review process. In this review process, reviewers unanimously suggested rejection even ignoring this issue, but it seems that you need to pay attention in your future submissions.

Decision: Reject