Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: Summary

The paper proposes two graph smoothness metrics for measuring the usefulness of graph information. The feature smoothness indicates how much information can be gained by aggregating neighboring nodes while the label smoothness assesses the quality of this information. The authors show that Graph Neural Networks (GNNs) work best for tasks with high features smoothness and low label smoothness by utilizing information from surrounding nodes which also tends to have the same label. Based on these two metrics, the authors introduce a framework, called Context-Surrounding Graph Neural Network (CS-GNN), that utilizes important information from neighboring nodes of the same label while reduce the disturbance from neighboring nodes from different classes. The results demonstrate considerable improvement across 5 different tasks. 

Strength

The authors advocate for better understanding of the use of graph information in learning, which is both an important and interesting problem. Two graph smoothness metrics appears to be intuitive and reflect common situations in graph-based data (1) features from neighboring nodes contribute differently to target node representation (2) neighboring nodes information sometimes causing disturbance if node with different labels tend to be connected. The paper provides some theoretical analysis that supports this claim and thorough experiments that show the correlation between the two proposed metrics with the performance of GNNs. 

Weakness

While the paper is reasonably readable, there is certainly room for improvements in the clarity of the paper. First, I would suggest the authors to avoid too much word repetition as well as long, obscure sentences. For example, the first sentence of section 2.2 can  be rewritten as “GNNs usually contains an aggregation step to collect neighboring information and a combination step that merges this information with node features.” The flow of the paper is also hard to follow and need some rearrangement. For instance, paragraph 3 of section 2.1 can be pushed until section 3.3. Another suggestion about the flow is to separate section 2.2 into two subsections for features smoothness and label smoothness (also the title of this section need to be refined). Finally, the results would be more clear if separated into different tables or subsections/paragraphs.

Questions

* Is there a particular reason for using the KLD instead of mutual information?
* In 2nd sentence of section 2.2, what does “node’s own information” mean? If it is the individual node’s features then why it naturally the representation vector h_v (which is aggregated with neighboring nodes?)
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper proposes two smoothness metrics to measure the quantity and quality of the graph information that GNNs employ in the model. With the smoothness measures, this paper proposes a new GNN model to improve the use of graph information. Overall, the paper is well-organized and clearly written. The main concern is the novelty, since the proposed method is pretty close to the graph attention network (GAT), except using the two smoothness metrics and a slightly different way to compute the attention coefficients. Experimental results show marginal improvement over existing GNN methods on the node classification task. Given these aspects, it is not that convincing that the introduced smoothness metrics are necessary. I would like to recommend a weak reject for this paper.


Suggestions to improve the paper:

1) It would be better to provide more convincing evidence and motivation for the smoothness metrics, either in theory or empirical analysis.

2) When describing the proposed method, organize the key differences in a more clear way. For example, add the proposed method directly in Table 1 and summarize the key differences in bullet points. Currently, this important part is deferred to Section 3.3, which may cause confusion for the readers to understand the paper.

3) In Section 3.2, this paper claims that the proposed method can easily include side information on graphs to improve performance, by using the topology feature in the last fully connected layer for class prediction. However, this technique can also be used in existing GNN models, and it is not clear why this is described as something unique for the proposed method. Also, there is no corresponding ablation experiments to compare the performance of the proposed method with and without using local topology features.
review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: The authors study how neighbor information on graphs can be used in Graph Neural Networks. It proposes measures on whether the data in neighboring nodes are useful in terms of labels or features. It also provides a new Graph Neural Network algorithm that is a modification of attention-based models incorporating the derived label and feature smoothness measures. The paper demonstrates the usefulness of these measures and algorithms with several different baselines from different families. The writing is mostly smooth, and the authors seem to provide enough detail of the experiments performed.

The proposed measures look simple but effective (as demonstrated in Table 2). The paper compares different techniques and shows that when neighboring labels are not smooth, techniques such as label propagation does not help. I do recommend Table 4 Lambda f and Lambda l values to be included in the main paper (though text mentions). When incorporated into the attention-based GNNs, Figure 1 also shows that smoothness parameters of the techniques also improve. 
review_title: Official Blind Review #3

Meta-review: Two reviewers are positive about this paper while the other reviewer is negative. The low-scoring reviewer did not respond to discussions. I also read the paper and found it interesting. Thus an accept is recommended.
Decision: Accept (Poster)