Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This work proposes a subgraph attention mechanism on graphs. Compared to the previous graph attention layer, the node in the graph attends to its subgraph. The subgraph is represented by an aggregated feature representation with a sampled fixed-size subgraph. The methods are evaluated on both node classification and graph classification problems.

I have major concerns about the novelty, and experiments in this work.

1. The motivation is not clear. Using a subgraph or neighborhood to represent a node is reasonable. However, this work samples a subset of nodes from the one-hop neighborhood and aggregates them for attention mechanism. It is very similar to a GCN + GAT. The sampling process even loses some neighborhood information in the graph.

2. The experimental setups are very strange. In Table 2, the methods are compared to GCN and GAT on node classification problems. The performance of GAT is too low and even lower than that reported in GAT. Can authors explain this? It is highly recommended to use the same experimental settings as in GCN and GAT. The same problem exists in Table 3. Can authors provide a performance comparison based on the same settings in GIN?

3. The performance improvements are very unstable and marginal. In Table 3, the proposed methods can not compete with previous methods especially on large datasets like IMDB-MULTI. I wonder how the proposed methods perform on very large datasets such as reddit.

4. Can authors provide comparisons with a simple GCN+GAT? 
review_title: Official Blind Review #4

final_score: rating: 3: Weak Reject
review_content: This paper introduces a subgraph attention method for graphs. Recently, many papers have shown that attention is a very important concept. However, there was no attention method for graph input structures, while a particular subset of nodes is very crucial to make the output.  

This paper first proposes the graph attention mechanism and hierarchical graph pooling idea. The attention basically subsamples subtrees so that each node can have the same number of attention candidates. Then, we can the attention network as many other papers. Experimental results show that the proposed attention based algorithm outperforms other algorithms.

I think this paper attacks a very important issue "graph attention" and have a very nice algorithm and results. Overall, my recommendation is "accept".

Cons.
It would better if the authors test some other different attention networks along with the current way.

================================================
I've read all discussions and changed my score. The novely of this work is not enough as R4 pointed out.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: The paper proposes a novel attention approach to graph neural networks which is applicable to both of node and graph classification. The proposed method gives an attention to a subgraph instead of a node by which importance can be controlled as a set of nodes. Further, the authors also introduce two types of attention in the hierarchical structure of the network called intra- and inter- level attention.

Considering subgraph attention would be novel and a reasonable idea. The sampling-based approach is a bit naive though it would be easy to implement. Reliability of some results are not clear for me because of the small training set.

The intra- and inter- attention approach would be a reasonable, but the relation with subgraph attention is not mentioned in my understanding. These two are independent approaches? Nothing is related to each other?

The MUTAG dataset has only 188 graphs, and so, in the smallest case in Table 3, the training data only contains 188*0.15 = 28.2 graphs. For me, learning an attention neural network with less than 30 sample is difficult to evaluate. Is there any rationale that the proposed method works on such a small dataset?

In sensitivity analysis in A.1, the performance on different max subgraph size (T) is shown, and the change of the performance is moderate. One of the main claims of the paper is that considering a subgraph (not a node) increases the performance. This results does not show the increase of the performance with the increase of the subgraph size T. Showing the performance with T = 1 can be informative to verify improvement brought by the subgraph attention.

Is Figure 3 training set or test set?
review_title: Official Blind Review #2

Meta-review: Initially, two reviewers gave high scores to this paper while they both admitted that they know little about this field. The other review raised significant concerns on novelty while claiming high confidence. During discussions, one of the high-scoring reviewers lowered his/her score. Thus a reject is recommended.
Decision: Reject