Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors proposed a unified message passing model to make a graph neural network to be able to incorporate both label propagation and feature propagation. Compared to previous work, the proposed model can also make use of partial label information in both training and inference stages. Experiments on three OGBN datasets show that the proposed methods achieve promising performance.

Pros:

    The motivation of the paper is very clearly stated in the text and the experiments successfully show that by incorporate label propagation and feature propagation, the performance can be improved.

    The partial label information is included in both training and inference stages.

Cons:

    In the literature review, one very important work [A], which is very closely related to the proposed model is ignored. In [A], a unified model for neural message passing is already established, and the proposed model seems to be a special case of the neural message passing module in [A]. The authors should compare their model to the one in [A] and clearly state the difference between the proposed model and the one [A], and should justify why should not use [A] directly.

[A] Gilmer, Justin, et al. "Neural Message Passing for Quantum Chemistry." ICML. 2017.

review_title: Review ---- Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification 

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper presents a novel unified model that jointly harnesses the power of graph convolutional networks and label propagation algorithms based on the unified message passing framework. The UniMP first employs graph Transformer networks to jointly propagate both feature and label information. Then, to avoid label leakage, a masked label prediction strategy is employed.

Pros:
* The presented method shows strong empirical performance on the open graph benchmark dataset.
* The whole framework is simple and the idea is easy to follow.

Cons:
* What is the label leakage problem? It is not clear to me (1) why label will be leaked during the joint learning process and (2) what the outcome does label leakage bring.
* The writing of this paper is poor. The authors are suggested to polish their paper. Please see minor comments below.
* Although the proposed UniMP achieves state-of-the-art performance, the experiments are not convincing enough.
  * Experimental results are not consistent, c.f. Table 4~6 and Table 7. It seems that the standalone Transformer even surpasses UniMP on ogbn-products and ogbn-arxiv.
  * It seems that the hyper-parameter specifications vary greatly across the three datasets. To me the residual connection is helpful when stacking many layers (Li et al., 2019; Chen et al., 2020), while in this paper the number of layers is relatively low. A sensitivity analysis on the network depth is necessary to demonstrate the impact of the residual connection.
  * When compared with GAT, the main differences are (1) different implementations of self-attention mechanisms and (2) whether to adopt gated residual connections. However, no ablation studies are provided to demonstrate the impact of these two independent components. Especially, as shown in Table 7 and Figure 3, given $X$,$A$,$\hat{Y}$, transformer outperforms GAT. The authors are expected to elaborate on which component (self-attention implementation or gated residual connection) brings the improvement.

Minor comments:
* Abstract: we adopt a Graph Transformer jointly [using] label embedding?
* Abstract: UniMP ... and be empirical powerful -> is empirical powerful
* Page 2: there are different -> they are different
* Mathematical notations are severely abused; for example, hidden_size should be represented by $f$, and how $\hat{Y}_e$ is transformed from $\hat{Y}$ is not clear.

review_title: Official review

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposed a novel unified massage passing strategy for semi-supervised graph learning scenario. It combines the advantages of the GNN and label propagation algorithms.

In the model, the label information is projected into the feature space, then merged/added with the node feature for the GNN feature aggregation module. By this way, the label information could be more effectively utilized in the semi-supervised scenario. In addition, a masked label prediction strategy is proposed to reduce the negative influence of the label leakage problem. The extensive experimental results demonstrate the effectiveness of the proposed model.

Pros:

This paper proposed a novel framework to more effectively and explicitly utilize the label information by GNN in semi-supervised scenario, and the experiments illustrate its effectiveness in general benchmark datasets.

Cons:

The insight/motivation of this method is not very significant. Compared with other methods such as APPNP,  TPN and GCN-LPA, the major differences is another effective/unified network structure to merge feature and label information together. Compared with GCN-LPA, the improvements in the insight/motivation aspect is not very significant. This paper shows empirical discussion compared with GCN-LPA, while it could be better to show some insight/theoretical discussion/analysis.

The improvements compared with other baselines are not significant. If more parameters/weights are deployed in other baselines, could the performance be better? Please discuss this point.

review_title: More discussions could be added

final_score: rating: 7: Good paper, accept
review_content: Summary:

The paper proposes a new Graph Transformer (UniMP) based model with the motive of combining two powerful semi-supervised node classification techniques, GNN and LPA. Proposed Graph Transformer unifies feature and label propagation in conjunction to provide a better performance in semi-supervised node property classification task. UniMP also benefits from a masked label prediction strategy which is inspired by [Devlin et al.](https://arxiv.org/abs/1810.04805)
Proposed method is able to achieve SOTA performance on several datasets from Open Graph Benchmark. 

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Pros:

+ The method proposed for unifying feature and label propagation is simple yet sufficiently novel.

+ This paper is able to support all the claims with rigorous experimentation. The experimentation is done using Open Graph Benchmark datasets and the proposed method achieves SOTA on all the datasets used.
    
+ The paper is very well written and is easy to understand. I personally like the simple yet effective nature of the method.

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Questions: 

- How do you map label information to feature information space? Is it a linear model? Did you explore any other embedding approaches before deciding on a linear model?

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    
Overall, I vote for accepting the paper. This paper provides a novel method to unify GNN and LPA algorithms in order to take advantage of feature as well as label information. Proposed method is well supported by rigorous experimentation on various datasets and ablation studies on various inputs. 

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    
Minor Comments/Typos:
    
- Section 1: In addition, there are **different** between → In addition, there are **differences** between
    
- Section 4: To **verified** our model → To **verify** our model
    
- Section 4.2: Some of the **including** results → Some of the **included** results
    
- Section 4.3: instead of **itself** **feature** → instead of **it's features**
    
- Section 4.4: model still **be** uncertain → model still **remains** uncertain
review_title: Official Blind Review #2

Meta-review: This paper proposes a semi-supervised graph classification technique that unifies feature and label propagation techniques. The resulting algorithm is a simple extension that attains strong performance. Reviewers were divided on this submission. Some reviewers felt the proposed algorithm did not constitute a sufficient technical contribution given that it was a simple combination of existing techniques. I tend to agree with other reviewers that the simplicity is a benefit. However, despite the methods simplicity there was significant confusion about the details of the method and multiple reviewers flagged that the paper was difficult to read and understand. It further could benefit from additional discussion and some clarification/cleanup of the experimental results. Finally, multiple reviewers asked for better situating of the proposed method with respect to prior work. Given these concerns, I do not think the paper is ready for publication. I would recommend the reviewers do a thorough re-write of the paper to address these concerns and consider resubmitting.
Decision: Reject