Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents an Ordered GNN for handling the heterophily problem, which attempts to improve the updating process in the message passing scheme. The paper studies an important problem, i.e., how to extend current GNNs to heterophilic graphs.
strength_and_weaknesses: This paper presents an Ordered GNN for handling the heterophily problem, which attempts to improve the updating process in the message passing scheme. The paper studies an important problem, i.e., how to extend current GNNs to heterophilic graphs. Unfortunately, there are many confusions and presentation problems in this paper, which makes it hard to follow. Besides, I also have several concerns about its motivation, detailed method, and evaluations.

Comments
1. I am afraid that there are many confusions and grammar errors in the paper, which makes it hard to follow. For example, in Sec. 2.2, I am wondering what "one round of message passing", "the neurons used to represent the nodes’ ego information", "the neurons used to represent message passing within Tao^{(k-1)} should be a subset of those for Tao^{(k)}", and "the delta between k-1 and k" mean. Besides, the paper may benefit from revising certain presentations, e.g., "its k-th order rooted tree", "And since …", "Apparently here we have…", ". where Tao^{(0)} corresponds to…". I am afraid that the paper needs to be carefully revised.
2. I am afraid that some utilized mathematic symbols lack formal definitions and thus induce unnecessary confusions. For example, Tao^{(k)}_{v} is defined as the "k-th order rooted tree" of node v. However, it is not clear what its subset operation is (as shown in Eq. (3)) and why "Tao^{(0)}_v corresponds to the neurons used to represent the ego representations".
3. I am afraid that the reviews of related work in Sec. 1 are confusing. This paper focuses on two drawbacks of the message passing scheme, i.e., over-smoothing and heterophily. It is not clear which problems these reviewed literatures were designed to solve. It actually seems that this paper considers these two problems as the same problem.
4. In Sec. 2.5, I am afraid that the intuitive analysis for Fig. 2 is not convincing. The paper states "In Case 1, two root nodes share the same first-order neighborhood pattern, one GNN layer can capture this good heterophily and obtain good performance". This analysis only considers node 1 and node 2. However, the children of these two nodes possess different situations which are not considered in this analysis. The analysis for Case 2 also has a similar problem.
5. I am afraid that the results of the employed baselines in Table 2 are most borrowed from a preprint paper, which is not convincing. For example, I am just wondering why the results of GRP-GNN are much lower than its reported ones.
6. I am just wondering how the proposed method can specifically solve the over-smoothing and heterophily problems.

clarity,_quality,_novelty_and_reproducibility: Please refer to the Strength And Weaknesses above.
summary_of_the_review: The paper studies an important problem, i.e., how to extend current GNNs to heterophilic graphs. Unfortunately, there are many confusions and presentation problems in this paper, which makes it hard to follow. Besides, I also have several concerns about its motivation, detailed method, and evaluations.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents an ordering mechanism for the messages passing in GNN for node representation learning. In particular they are focused on the combining stage in the GNN procedure. The ordering mechanism is achieved by the rooted-tree hierarchy and a curated gating function. which can iteratively preserve the ego information and gradually absorb k-hop neighboring features. The authors claimed to address the problem of “over-smoothing” and “heterophily”. The experiment results show the performance boost in both settings above. And the large scale dataset experiment demonstrated the scalability of the proposed method.
strength_and_weaknesses: Strength:
1. Intuitive and effective design of the gating function to iteratively update the feature while preserving the ego node info by inductive bias using ordered neurons.
2. Sufficient number of experiment in both small and large scale dataset.
3. Good presentation, easy to follow.

To be clear:
1. For the experiment result in the Squirrel dataset, the performance surpass the baselines by a quite large margin. I'm wondering if there is an underline reason for such improvement.
2. Fig 3 shows some baselines which previously designed to deal with the over-smoothing problem are actually pretty good. Even though there overall performance doesn't beat the proposed Ordered GNN. A good question is that if the number of layers is further increased, what would happen? In another word, is such carefully designed gating procedure actually needed for solving the over-smoothing problem?

clarity,_quality,_novelty_and_reproducibility: The presentation is clear. The experiment result is convincing. But, the source code is not released which prevents me from the judgment of the reproducibility.

Although the ordered neuron trick was introduced in prior work, the author adapted it to literately preserving the ego info which is still considered as a good contribution in terms of novelty.
summary_of_the_review: The proposed Ordered GNN approach is clear and the experiments and the ablation studies demonstrated the effectiveness of each part of the design. The algorithm is tested in both small and large scale dataset. I recommend for accept.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed one ordered GNNs to mix the features calculated from different hops. This method can achieve the SOTA performance on both homophily and heterophily graphs, and could prevent the over-smoothing problem.
strength_and_weaknesses: ### Strengths:
It is interesting to introduce the ordered neurons into graph representation learning, and it is one straightforward solution for the over-smoothing problem. The experimental results demonstrate the effectiveness in preventing this problem.

### Weakness:

1. The motivation for separating the different hops is not clear. As mentioned in the introduction, what are the advantages of “model the information exactly at some orders”? 

2. The evaluations on the node-wise split points are lacking. Designing the personalized operation for each node helps to improve the model performance [1,2]. It seems that different nodes have different split points as mentioned in Section 2.3. This aspect is not evaluated in the experiments, either by comparing with some node-wise GNNs or showing the different split points. Besides, there lacks a detailed discussion about the ordered neuron methods to verify the contributions of this paper.

3. The information extraction on the ego features. The self-loop-free graphs are utilized in this paper. How to incorporate the node features from itself? 

### Refs:
[1] Graph Neural Networks with Node-wise Architecture. KDD 2022

[2] Policy-GNN: Aggregation Optimization for Graph Neural Networks. KDD 2020

clarity,_quality,_novelty_and_reproducibility: The motivation of the proposed method is kind of unclear and lack some evaluations.
summary_of_the_review: It is interesting to employ the ordered neurons in the graph representation learning and the experiments can demonstrate the effectiveness. However, the proposed method is not well motivated and some baselines are lacked.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new message passing method for GNN. It orders the message passing into the node representation with specific blocks of neurons targeted for message passing within specific hops. Extensive experiments show the effectiveness of the proposed method on homophily and heterophily data. It also alleviates the over-smoothing problem. 
strength_and_weaknesses: The strength of this paper includes:
1. The paper is well-motivated. 
2. The experiments are extensive.
3. The analysis is sufficient.
 The weaknesses are:
1. The experimental results are not impressive. The improvement is not significant.
2. The comparison methods are not STOA. Some recent works are ignored.
clarity,_quality,_novelty_and_reproducibility: This paper develops a new method for message passing. It aligns the hierarchy of the rooted-tree of a central node with the ordered neurons. The motivation is clear and the paper is well-organized. The experiments are also comprehensive to evaluate the effect of each component. 

summary_of_the_review: My worry about this paper is on the experimental part. Most of the comparison methods are published in or before 2020. Only GGCN, which was published in 2021, is compared. Much progress has been achieved in 2021 and 2022. Therefore, it is fairer to compare some of them. Furthermore, the improvement is also not significant in most cases.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents an interesting variant of message passing in GNNs in which the messages are ordered. This mechanism is shown to address the problems of GNNs on heterophilic graphs. The reviewers complained about insufficient SOTA baselines, which the authors addressed in the rebuttal. We believe the paper can be accepted. 
justification_for_why_not_higher_score: Not sufficiently novel/not sufficiently good results
justification_for_why_not_lower_score: interesting idea
Decision: Accept: poster