From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: This paper considers three types of conditioning to improve the performance of GNNs and apply the proposed method to computational chemistry tasks. While the method is intuitive and easy to understand, the technical contribution of conditioning is limited, as these conditioning methods exist in the literature. The experiments need to be expanded to incorporate models other than EGNN and the significance of the results needs to be provided in the results table. Given these concerns, I would recommend rejection of this paper.
justification_for_why_not_higher_score: limited technical contribution
justification_for_why_not_lower_score: n/a

From: ICLR.cc/2023/Conference/Paper5099/Reviewer_j1mL
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: In this paper, the authors summarize and analyze the existing ways (or called conditioning methods) that use edge attribute information together with node embeddings in GNN. The intuition of each conditioning method is explained for comparison. In experiments, the authors use a GNN model as the baseline to show the difference between these conditioning methods on benchmark datasets.
strength_and_weaknesses: Pros:
1. The existing ways of using additional information when updating node embeddings in GNNs are summarized and analyzed.
2. The paper is easy to follow and well-organized.

Cons:
1. The addressed methods are all based on existing works without having new conditioning methods.
2. The experiments are only done based on EGNN, which is not a well strong model in the related field. Besides, the empirically compared conditioning methods are only focusing on geometric distances.
3. The models except for EGNN and EGNN variants in Table 1 and Table 3 seem to be unrelated to the discussions. These models perform much better than the EGNN variants and the authors do not mention them at all.
clarity,_quality,_novelty_and_reproducibility: The clarity of the paper is good. However, the quality and novelty of the proposed method are limited. The code is not provided for reproducibility.
summary_of_the_review: Even though the existing conditioning methods in GNNs are summarized and analyzed, the authors only provide limited experiments. It would be better to use more powerful GNN as baseline and more comprehensively evaluate the conditioning methods.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper5099/Reviewer_xVZv
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: In this paper, the authors analyze the three different message-passing schemes of Graph Neural Networks conditional on the geometric property, namely weak, strong, and pure methods, and empirically study their effect.
strength_and_weaknesses: In this paper, the authors analyze the three message-passing schemes of Graph Neural Networks conditional on the geometric property, namely weak, strong, and pure methods, which relate to concatenation, gating, and transformations depending on the geometric attributes. To compare the effects of these methods, the authors perform experiments on molecular datasets and conclude that the strong method, i.e., the gating method, not only has higher computational efficiency but also achieves better performance in most cases.

Strength:
1. The paper is well-written and easy to read.
2. The paper focuses on an interesting problem, i.e., how to employ geometric attributes in GNNs properly.

Weaknesses:
1. The novelty and significance of this paper are concerned. As mentioned in the paper, the three methods have already been raised in previous work. And this paper only summarizes these methods and only gets some trivial results through experiments without theoretical analysis. The paper proposes an interesting question, but it doesn't lead to a meaningful conclusion.
2. The experiment is not sufficient. The authors only compare the three message-passing methods based on the EGNN model. In order to get a more reasonable conclusion, it is necessary to conduct experiments on several basic models.

clarity,_quality,_novelty_and_reproducibility: they are OK.
summary_of_the_review: This paper studies an interesting problem in geometric GNN design. However, it has limited novelty and has drawn few meaningful conclusions.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 1: The contributions are neither significant nor novel.
empirical_novelty_and_significance: 1: The contributions are neither significant nor novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper5099/Reviewer_QMHG
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The paper proposes three types of conditioning (weak, strong, and pure) to unify some conditioning methods in the literature. Two empirical testing studies show that strong conditioning can perform better than weak conditioning and more efficiently than pure conditioning.
strength_and_weaknesses: Strengths:

1. A general framework of conditioning methods is proposed.


Weaknesses:

1. In the experimental section, it would be great if the authors could show more statistics on the results, like standard deviation, so that it can further show that improvement made by strong conditioning is statistically significant. Also, some experimental setups are missing, like the number of runs for the two tasks.
2. Even though the authors empirically justify that for the EGNN instance, conditioning methods help boost performance, can any theory be developed to verify it?
3. Since the authors only show the advantage of strong conditioning over weak conditioning on the EGNN model, it is not convincing enough to assert that this will still hold for other models. Therefore, It is encouraged and interesting to see the applications of these conditioning methods on a few more models.


Questions:

1. Are these conditioning methods designed only for geometric GNNs, i.e., GNNs with graph data input with Euclidean embedding?
2. In Table 2(b), is there any reason why strong-EGNN is much worse than weak-EGNN in predicting $\mu$?
3. Is there any reason why EGNN is not compared on the MD17 dataset?




clarity,_quality,_novelty_and_reproducibility: 1. Notation is confused, for example, $\mathbf{W}(\mathbf{a})$, $\mathbf{W}_{\mathbf{a}}$, and $\mathbf{W}^{a}$. What do $\mathbf{f}^l_j$ and $\mathbf{f}_j$ mean respectively in Eqn. (4)? Also, $\phi_e$ has two vector inputs $\mathbf{h}_i^l$, $\mathbf{h}_J^l$ and conditional on a scalar at the bottom of page 2, but why it then changes to have one input $f_j^l$ and condition on a vector in Eqn. (4)? And how is the operation $\mathbf{W}(\mathbf{x}_j – \mathbf{x}_i) \mathbf{f}_j$ defined or performed?
2. Writing can be improved. Some sentences are hard to understand; for example, “Chollet (2017) … scales/gates the output” above Eqn. (11).
3. The code for the experiment part is not submitted/uploaded.
summary_of_the_review: Overall, it is insightful to unify current models into a framework. However, more numeric and theoretical supports are encouraged. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough