Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: Graph Neural Networks is a popular architecture for the analysis of chemical molecules. The authors propose an auxiliary module that can be attached to a GNN that can boost the representation power of GNNs. The auxiliary module has three building blocks: 1. a supernode, 2. a transmitter unit and 3. a warp gate unit. The authors show through carefully designed experiments that these additions can be attached to any type of GNN and that they are successful in reducing both the training error and test error. A variety of graph regression and graph classification tasks are chosen to show the efficacy of the method.

The paper is well written and easy to follow. The modification suggested by the authors is novel and useful. Experiments are well designed. An aspect that is not clear from the paper is the ability of these models to overfit the data as you increase the representation power of the network. While the authors claim that to be one of the shortcomings of existing GNNs, it is not clear whether the proposed method solves that problem. For example, in figure 4. the training loss hardly decreases as the number of layers are increased. It would be good if the authors can share any insights on this point. 

Overall, I think this is a good paper that the community will benefit from.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The paper proposes an auxiliary module for GNNs to boost the representation power. The new module consists of virtual supernode, attention unit, and gating unit, each of which is demonstrated useful in the experiments. The module can be applied to various types of GNNs. 

This work can be seen as an improvement to previous virtual supernode based methods. Adding the attention units and gating units is rational, and the effectiveness is also proved in the ablation studies. However, the claimed contribution of improving the representation power may mainly come from the idea of supernodes (instead of the attention and gating). This largely reduces the novelty of this paper and make it incremental, because using virtual supernodes is not this paper’s original idea.

The paper is generally well written. However, the comparison with previous supernode based models is not described clearly enough. The authors listed the difference from (Glimer et al. 2017) and (Li et al. 2017) in Table 1, but ignored (Pham et al. 2017) and (Battaglia et al. 2018), which were also cited in the related work. Moreover, (Li et al. 2017)’s method is actually different from the simple supernode baseline, in that it is not a bidirectional message passing between supernode and the main network. Table 1 does not contain this property.

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: = Summary
A Graph Neural Network extension integrating a global supernode explicitly into the message passing process. Concretely, message passing along the graph edges is alternated with message passing to/from a fresh super-node. Experiments show that this improves results of a number of common GNN architectures on four datasets.

= Strong/Weak Points
+ Simple but useful extension of the existing super-node idea
+ Experiments on a number of datasets and baseline GNN architectures, providing ample experimental evidence of the usefulness of the method.
- Writing is overcomplicated and uses a lot of jargon ("transmitter unit", "warp gate", "intermodule hyperspace"). I found the text entirely impenetrable and instead simply focused on Fig. 3 + the actual equations.

= Recommendation
This is a nice contribution of minor novelty, with empirical evidence of its usefulness. I believe the paper should be accepted to a large conference such as ICLR.

= Minor Comments
- Fig. 3: Inconsistent "intra-module" (top) vs. "intra module" (bottom) 
- Concurrent work in https://openreview.net/forum?id=B1lnbRNtwr discusses a "sandwich" model which alternates graph message passing with (essentially) a Transformer layer applied to all nodes. This idea seems related (in that it alternates local and global information exchange).
review_title: Official Blind Review #2

Meta-review: This paper presents an auxiliary module to boost the representation power of GNNs. The new module consists of virtual supernode, attention unit, and warp gate unit. The usefulness of each component is shown in well-organized experiments.
This is the very borderline paper with split scores. While all reviewers basically agree that the empirical findings in the paper are interesting and could be valuable to the community, one reviewer raised concern regarding the incremental novelty of the method, which is also understood by other reviewers. The impression was not changed through authors’ response and reviewer discussion, and there is no strong opinion to champion the paper. Therefore, I’d like to recommend rejection this time. 

Decision: Reject