Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a model to generate molecules using graph and geometric representations. The method is tested against several relevant baselines on standard and custom datasets, with a primary focus on chemical validity.
main_review: Pros:
 - Problem is relevant and clearly defined
 - Method is intuitive and accessible - generate atom type, generate connections to existing atoms, generate positions, repeat
 - Some results are positive, generally better or similar to other methods. The model's tendency to construct valid molecules without checks is impressive.
 - Writing is clear, organized, and developed, discrepancies are discussed

Cons:
 - The result in Table 1 make me question the overall value of the technique.
   - The method w/o geometry has higher validity and uniqueness than the model w/ geometry.
   - All of the methods (w/ checks) produce valid molecules and have similar uniqueness and higher novelty. The paper discusses this apparent shortcoming and largely concludes that the comparison is misleading due to intentional limitations (number of atoms) in the proposed method vs the other methods. If this is true, is it possible to reproduce Table 1 where only "fair" comparisons are made (maybe by dropping large molecules from the other methods)?
 - The paper discusses how the conditional model could be used to retain features from one molecule when constructing new molecules but doesn't demonstrate this. It would strengthen the paper to see a simple demonstration of this application.
summary_of_the_review: The paper takes a positive step forward in constructing generative models over molecules by including graphical and geometrical information in a single model. However, some of the empirical evaluations muddle the overall story and could be improved/clarified.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose GEN3D, an ML-based approach for 3D molecular structure generation. GEN3D places atoms in space to build molecules in an atom-by-atom fashion and constructs a corresponding molecular graph that can be employed to enforce organic chemistry rules.
main_review: 1. The related work is pretty detailed but neglects some relevant related papers. 
   1) Simm, G. N. C.; Pinsler, R.; Hernández-Lobato, J. M. Reinforcement Learning for Molecular Design Guided by Quantum Mechanics. ICML 2020.
   1) Gebauer, N. W. A.; Gastegger, M.; Hessmann, S. S. P.; Müller, K.-R.; Schütt, K. T. Inverse Design of 3d Molecular Structures with Conditional Generative Neural Networks. arXiv:2109.04824 [physics, stat] 2021.
   1) Simm, G. N. C.; Pinsler, R.; Csányi, G.; Hernández-Lobato, J. M. Symmetry-Aware Actor-Critic for 3D Molecular Design. ICLR 2020.

2. It is not clear how working with a graph is beneficial. 1) The authors claim that organic chemistry rules can easily be enforced through a graph representation. However, it's unclear to what extent these relatively simple rules are holding ML models back from proposing more chemically meaningful molecules. 2) Working with a graph restricts the application of this model to organic chemistry, and metals or multi-molecular clusters become unattainable.
summary_of_the_review: GEN3D appears to be a competitive approach for molecular structure generation. The main weakness of this paper is, however, that it lacks methodological novelty. Methods, such as G-Schnet, MolGym, and ENF, are closely related to this work, and the originality of this work is, in my view, insufficient for this venue.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a generative model to sample a new molecule with its 3d coordinates. The generative process is divided into three different steps: 1) the atom network generates a new atom 2) the edge network connects the new atom to the existing atoms 3) the distance and angle network generates the pairwise distances and angles from existing atoms to the new atoms. EGNN is used for each of these sub-networks to impose the equivariance to translations and rotations. Experiments with QM9 datasets show the geometric correctness of the proposed generative process.

main_review: - Pros
    - Highlights the importance of generating 3d structure for molecule generation.
    - The paper provides new insight of the QM9 dataset.

- Cons
    - Insufficient description. The paper seems not-self contained. Difficult to understand some of the details.
        - Since ICLR is not chemistry focused conference, it would be good to provide some details of domain-specific information
    - The 3D structure generation is unclear.
        - To generate 3D coordinates, the model places a probability mass over the grids of distances and angles.
        - The provided process is likely to place a mass over the impossible configuration of distances and angles.
        - From the grid, the model sample a new location. However, sampling is not back-propagatable in general. It is unclear how the model handles this.
    - A mismatch between motivation and experiments
        - None of the experiments is related to the proposed downstream tasks in the introduction. Although generating 3D structure seems plausible, the benefit of it is unclear without direct application of 3D structure
            - JS divergence on distance is the only metric that is directly connected to the molecule geometry. However, it is not sufficient to show the usefulness of 3D structure.
        - For example, it would be more interesting to show that the generated graphs have relatively low-energy conformations than those from the other models.
    - A proper ablation study on model selection and training procedures would improve the strength of the proposed approach.

- Questions
    - What are the losses used to train the model? Please provide the detailed training procedure.
    - Is the equivariance of the subnetworks sufficient to impose the equivariance of the entire network?
        - Is permutation invariant unnecessary? Since the generation is autoregressive, to put the same distribution to the same molecule with different sequences, it needs to be permutation invariant.
    - If the description of QM9 dataset is correct, how do we understand the reported performance of some previous models which achieve 90%+ uniqueness and novelty such as GraphDF[1] and MoFlow[2]. This seems impossible based on the description provided in this paper.
    - In the Introduction, a significant portion has been used to criticize the approaches with SMILES. What problem can be arisen from converting SMILES string to 3D structure? and how the proposed solution can avoid this? - e.g., w. OpenBabel
    - What is the GEOM-QM9 dataset, and how does it differ from QM9?
        - In the last sentence of Section 3, a random uniform noise is added to each coordinate. Is this applied to all datasets?

1. Luo, Youzhi, Keqiang Yan, and Shuiwang Ji. "GraphDF: A discrete flow model for molecular graph generation." arXiv preprint arXiv:2102.01189 (2021).
1. Liu, Meng, et al. "GraphEBM: Molecular graph generation with energy-based models." arXiv preprint arXiv:2102.00546 (2021).
summary_of_the_review: Although the proposed method seems an important research direction for molecule generation, the experiments did not show the usefulness of such approaches to compare with the previously studied methods. Some of the descriptions are insufficient to understand the model details as well.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: * The paper proposes an autoregressive generative model for 3D molecular graph generation, by predicting the next atom type, next edge type, next atom distances, and next atom angles sequentially based on the existing graph.
* The authors claim that the proposed model can generate 3D molecules with higher rates of chemical validity, better atom-distance distributions, and is very cheap to train compared with previous models.
main_review: **Strengths**
* The proposed model is a nice extension of previous autoregressive graph generative models, e.g., GCPN. Inspired by the mechanism of distance geometry and internal coordinates (e.g., pairwise distance and angles), the authors propose to autoregressively predict the next atom/edge type, next atom distances/angles, thus enabling the model to concurrently generate molecular graphs and 3D geometries. The whole methodology is classic and easy to follow.
* They experimentally validate the effectiveness of the proposed model. The experiment design seems good as it covers several research topics ranging from 2D molecular graph generation, 3D molecule design, to molecular conformation generation.
* They plan to release the code in the future.

**Weaknesses**
* The novelty of the work is limited. Compared to the previous autoregressive graph generative models (e.g., GCPN, GraphAF, MolecularRNN[1,2,3]), the key difference is that they incorporate the atomic distance prediction and angle prediction into their model. However, distance geometry modeling and angle prediction are not something new. So the model is more like a combination of techniques in 2D graph generation and 3D structure modeling.
* Since the paper introduces a **new generative model** for 3D molecule design, I recommend authors introduce the generative model in a more formal (mathematical) way. For example, we can introduce the probabilistic distribution we want to model, the way we factorize the distribution (autoregressively), and how we estimate the parameters. This is a minor suggestion, but I think it will improve the quality of the draft. 

**Reference**
1. Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation
2. GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation
3. MolecularRNN: Generating realistic molecular graphs with optimized properties

Post rebuttal: The authors partially addressed my concerns and I would like to update the score to 5
summary_of_the_review: My main concerns of the paper are its novelty and the way it introduces the generative model.
review_title: null

Meta-review: This paper proposes to generate 3D molecules using a step by step approach. The reviewers raised major concerns on the experiments, novelty, writing and technical details. The authors also were not aware of many of the important references, part (but not all) of which have been included during discussions. It is clear that this work is not ready to be accepted by ICLR.
Decision: Reject