Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: summary: The authors use a generic algorithm to embed molecular graph into 3D space using a distance matrix computed from PageRank. Then they use this 3D information as additional features to do property prediction. Results show that a) this approach is better than using GNNs without this information b) the augmented structures can even be applied to on Euclidean GNNs such as DimNet and get better results compared to simple GNNs.
main_review: This paper attempts to construct a learning approach that is not meaningful for molecular setting. Specifically, the authors are trying to predict properties that are a function of chemical graph *and* configuration (3D info) from the chemical graph alone. This is done by adding pseudocoordinate information extracted from the chemical graph itself. However, since no real coordinate information is added this just seems to indicate that some inefficiency in the GNNs compared to was exploited - it must be possible to get similar performance with suitable GNNs because they use exactly the same input information eventually.

More fundamentally, it is simply meaningless to predict a property of the 3d structure independent of the 3d structure. What *would* be meaningful is to predict an ensemble average over 3d structures such as a solvation free energy as a function of the chemical graph alone, but the prediction of the QM9 properties certainly isn't. As a result, I am not convinced of any of the results.
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper focuses on the task of molecular property prediction and proposes novel approaches to build synthetic coordinates (geometric information) in molecules when atom positions are not available for advanced GNNs. Since the performance of those GNNs highly relies on atom coordinates, the applicable cases of advanced GNNs are limited without reliable and efficient approaches to generate coordinates before training.

The authors propose two ways to generate the synthetic coordinates: 1. Graph-based distance and angles via Personalized Page-Rank; 2. Molecular distance bounds and corresponding angles based on chemical knowledge. To embed the resulting distances and angles, the authors propose to convert the original molecular graphs to line graphs. With line graphs as input, regular GNNs work as directional MPNNs, which exhibit better performance in practice. Experiments on benchmarks validate the improvements achieved by using synthetic coordinates in molecular graphs with several baseline models. 
main_review: The proposed synthetic coordinates address the gap between the performance of GNNs on molecular property predictions when the atom coordinates are provided and not provided. The design of advanced GNNs with available atom positions is also different from regular GNNs when only molecular graphs are achievable. With synthetic coordinates and line graphs, regular GNNs can be converted to directional MPNNs, which have the potential to gain better performance with distance and angles as features. The proposed idea is novel and contributes to the related field especially molecular property prediction is an important subarea now. 

Overall the paper is well-written but can be better organized: I feel that the "Molecular configurations" section is a bit too long for discussing the background chemical knowledge and motivations. In contrast, the "Synthetic coordinates" section that describes the proposed methods is too concise without touching enough details. Some other suggestions for "Synthetic coordinates" section:

(1) The proposed methods can be explained with values computed on certain example molecules for better understanding. 

(2) It would be interesting to compare the synthetic coordinates achieved by the two methods to find any possible relations. 

(3) As for the implementation, although the authors mention combining both two synthetic coordinates, it is not clear how the two coordinates are implemented together in GNNs. 

The results in the experiment section clearly demonstrate the improvements achieved by introducing synthetic coordinates and line graphs. I would suggest that Table 1 and Table 5 can be better written since it seems that each ablated model only contains one modification (distance, line graph, or angle), while "+angle" actually means "+disrance & line graph & angle". By only reading the tables, it is unknown that the changes are done step-by-step.


limitations_and_societal_impact: The authors have addressed the limitations and potential negative social impact of their work in section 5.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: The paper proposes a method for estimating molecular properties without 3D information about the molecules. The main contribution of the paper is an approach to approximate the local distance and angular information. This may then be fed into other GNNs for calculating the various properties. The authors also explore improvements related to using line graphs.

Results are shown on QM9, ZINC and ogbg-molhiv datasets.

main_review: The authors propose an interesting approach to approximating local geometry information in molecules. It is a novel and potentially useful technique when geometry information cannot be obtained, as the authors describe in Section 3. However, the authors never compare against other techniques for computing approximate geometry information, such as force fields or direct approaches. Without these experiments the paper is below the acceptance threshold.

\+ Interesting approach that sidesteps the need for 3D geometry information when predicting molecular properties. 

\+ Initial results look promising.

\-- No comparison with other approximate 3D geometry estimation techniques. Comparison should be done with force field methods or direct methods (i.e., other computationally efficient approaches). Otherwise the usefulness of the author’s technique is unknown.

\- ML models are also trained to predict atomic forces (SchNet, DimeNet, etc.). These may then be used to predict relaxed structures for the molecules (potentially finding multiple conformers). These methods are also computationally very fast and may be more accurate than force fields, especially for small molecules.

\- The authors state that conformers are a significant problem. However, the models can be run on multiple potential conformers and the ones with lowest energy selected. This won’t work if the energy isn’t estimated, but may be true for many applications.

\- On line 193, the authors state “The angles in ring systems are handled separately based on a set of hand-crafted rules.” It’d be better is a principled method was used in all cases.

\- The results in Table 5 contradict the earlier results. It would greatly strengthen the paper if the discrepancy was explained.

Other comments:

* Move limitations after the results section. This will give the reader better context on the discussion.
	
* Recommend not emphasizing the 49% improvement with DeeperGCN since it’s not the SOTA approach (Table 1). Better to state the improvement on the best method, SMP.

* The labels in the tables are confusing as to which settings are used. E.g., in Table three, do PPR and bounds use angles (I assumed so)?

limitations_and_societal_impact: Good.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper aims to improve molecular property prediction by generating synthetic
3D coordinates. The method is based on the key observation that the true 3D
coordinates are difficult to obtain and oftentimes undecidable for molecular
structures. The paper proposes synthetic coordinates, which then enables using
GNNs without requiring the true molecular configuration. The experimental
results on several data sets have shown very promising results using this
method.

main_review: This work generates synthetic 3D coordinates to improve molecular property
prediction. The key observation is that many molecule properties are related the
their 3d structures, but the true 3D coordinates are 3D coordinates are
difficult to obtain and oftentimes undecidable. The authors thus propose to
generate "synthetic" coordinates, which enables using GNNs without requiring the
true molecular configurations.

The synthetic coordinates provide pairwise distances between atoms, which can
then be used to calculate distances and angles as edge features for the 2D
molecule graphs. This can be then used to enhance GNNs into directional MPNNs to
improve their predictive power. The authors propose two methods for generating
synthetic coordinates: 1) graph-based distances based on personalized PageRank
scores; and 2) molecular distance bounds based on the interacting atom types.

The overall idea of incorporating synthetic coordinates into GNNs is well
motivated, and has shown very good performance. However, the design of the two
proposed methods for obtaining synthetic coordinates are quite arbitrary and
heuristic. This is especially true for the personalized pagerank method for
generating distances. The authors first calculate PPR scores and then turn that
into distances using the arccos function. But why does this conversion makes
sense? How can the arccos of PPR scores approximate the true 3D distance?

The experimental results are quite positive. The author have shown that such
"synthetic" coordinates enhancements can improve the performance of regular GNNs
on multiple benchmarks.


limitations_and_societal_impact: Yes
review_title: null

Meta-review: The ratings were 3, 6, 4, 6.

The reviewer who gave the 3 wrote "it is simply meaningless to predict a property of the 3d structure independent of the 3d structure", however the authors gave a detailed response that suggests there might have been some confusion, and the reviewer didn't follow up. Moreover, this approach effectively uses carefully chosen inductive biases, which seems to go beyond "some inefficiency in the GNNs compared to was exploited" as the reviewer suggests.

For the reviewer who gave the 4, since the authors provide some new results in response, the score probably should be elevated.

While this is still a borderline paper, I recommend "accept" given that the results are pretty good, there doesn't seem to be any technical problem, and there is clear novelty.
Decision: Accept (Poster)