Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper proposes an actor-critic neural network architecture for autoregressive generation of 3D molecular structures with reinforcement learning (RL). It builds upon the RL approach by Simm et al. (2020) which makes use of internal coordinates in order to deal with the symmetries that occur when placing atoms in the molecular design process.
The main contribution is the introduction of a covariant state-action representation extracted with Cormorant (Anderson et al., 2019) and a covariant, spherical distribution for the orientation of newly placed atoms (w.r.t. a focal atom) replacing the internal coordinates of the former approach. It is shown that these architectural and conceptual changes allow to build highly symmetric structures (which were problematic with the former approach) and improve the generalization as well as the quality of generated molecules in several tasks from the RL environment MolGym.

The paper has a clear structure, is well-written and includes several experiments to verify the main claims. To the best of my knowledge, the proposed architecture is the first model for generation of 3d molecules that uses covariant features to leverage the rotational symmetries of molecules. This seems to improve the model by Simm et al. (2020), especially when it comes to symmetric structures such as $SF_6$. The proposed covariant formulation could be beneficial for other existing or future generative models for 3d molecules, as well.

The experimental section could be improved by additional experiments to allow for comparison between the proposed architecture and other approaches for generation of molecular graphs and 3d molecules (Liu et al , Gebauer et al. 2019).  In many real-world molecular design tasks, a variety of (possibly diverse) molecules that exhibit certain target properties are desirable. It would be interesting to see whether the proposed model can also be applied for these purposes instead of the rather limited task to generate stable molecule from a predefined bag.

Pros
----
- well motivated neural network design incorporating the necessary symmetries
- extended experiments show generalization performance

Cons
----
- It does not become quite clear, but as far as I understand only the most probable action is chosen during offline evaluation (Sec 3.3). An additional experiment on how the validity and diversity depend on the degree of exploration would be useful.
- The ms could be improved by additional experiments that allow for comparison with previous molecule generation models on different tasks (see Liu et al 2018, Mansimov et al 2019, Gebauer et al 2019).

Update:
I raised my score to reflect the added metric and experiments that improved the paper.
review_title: Well-motivated approach to 3d molecular structure generation

final_score: rating: 6: Marginally above acceptance threshold
review_content: ### Summary of the paper
The paper utilizes a actor-critic framework for 3D molecular design. The central part of the approach is a rotation equivariant network (Comorant). For each atom, Comorant learns a state representation so that it is equivariant under rotation. The method is evaluated on 9 different molecules and it outperforms Simm et al.'s method in terms of validity and diversity.

### Strength and weakness
1. The method adopts a better model architecture (Comorant) to learn the state representation. However, Comorant is developed by Anderson et al. Therefore it cannot be counted as the contribution of this paper. The actor-critic formulation is also standard in RL. The sequential decision process is also similar to Simm et al. The technical innovation is not original enough.

2. The evaluation protocol is problematic. In table 1, authors only report validity and diversity of generated compounds. The validity is defined as "successfully parsed by RDKit". To my knowledge, RDKit validity checking is based on 2D constraints such as valence, aromaticity and kekulization. It does not capture 3D information at all. Are the generated compound stable? What is the RMSD of generated compounds? Simm et al. 2020 reported RMSD to measure the structural stability. Why RMSD metric is missing? This is important because 2D graph generation models already satisfy validity quite well (100% validity for even large molecules). 

3. Motivation is not clear. In the paper, authors state that the choice of focal atom, element and distance have to be *invariant* to rotation. It seems like invariant representation is sufficient. Moreover, the advantage over Simm et al. is not clear in section 4. Why highly symmetric states are problematic for prior work? How does this method solves this issue? I am afraid that the experiment section does not address this, since most test cases are not "highly symmetric" to my knowledge.

### Overall evaluation
I vote for rejection. My major reason is the problematic evaluation protocol. RMSD must be added to evaluate the stability of compounds. Moreover, I would like to see evaluation on "highly symmetric" cases. I think it's important since authors state that this is the major limitation of prior work.

### Question
1. Is the method scalable to large molecules? How is the runtime of your model compared to Simm et al.?

### Post rebuttal feedback
It's good to see that RMSD experiments are added and the results are better than Simm et al. Therefore, I am raising my score to 6. I also realized that the validity calculation is different from standard graph generation methods. The validity results now look reasonable to me.
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: This work presents an approach for 3D molecular design using reinforcement learning that exploits rotational symmetries in molecular conformations. The formulation involves an MDP that selects atoms from a “bag” and positions them in 3D space. The reward function is based on PM6 energies to encourage the generation of low-energy structures. The manuscript is well-written and provides ample context and appropriate references.

The generative formulation directly builds on that of Simm et al. 2020 to choose a focal atom, choose an element to add, choose a distance, then choose an orientation. The consideration of rotational covariance during new atom placement and the use of Cormorant (Anderson et al. 2019) for embedding seem to be the primary novelties of this work. Thus, Simm et al.’s approach relying on an internal coordinate representation serves as the primary baseline in the empirical evaluations here; “Opt” is meant to serve as an essentially-random baseline and performs predictably poorly. One could even envision using a 2D generative model with a heuristic conformer generation scheme followed by geometry optimization using PM6 forces as a stronger baseline approach.

The benefit of Covariant over Internal in terms of actual optimization performance (Figure 5, Figure 6) appears marginal. Did the authors perform any experiments to optimize more than just PM6 energy (e.g., a computed property) to compare the two methods? It’s not clear how significant the observed differences are and to what extent achieving rotational covariance helps in this task. An ablation study that decouples the effects of the rotationally-invariant critic and the rotationally-covariant actor could be informative. How sensitive are either of these models to hyperparameter selection?

Covariant does appear to generate a greater number of valid structures (Table 1), although this evaluation maps the 3D coordinates back to a SMILES string to check for validity and uniqueness, which seems not to be the intended application of this work. For that evaluation, one could consider using purely graph- or SMILES-based approaches and more traditional generative methods. 

A primary advantage of this formulation seems to be the ability to generate molecular clusters (in contrast to 2D approaches) and periodic materials (in contrast to Simm et al.). The current set of experiments do not demonstrate this ability in my opinion. There also is not evidence that “building in this inductive bias enables [one] to generate geometrically complex molecular structures that were unattainable with previous approaches.”

The integration of rotational covariance with 3D molecular generation is elegant and, to my knowledge, novel. I don’t disagree with the premise that one *should* strive to have these invariances, but I don’t believe that the current set of experiments actually demonstrate the superiority. 

review_title: Elegant theoretical justification, but experimental performance is lacking

Meta-review: The paper provides a new covariant approach to 3D molecular generation motivated by the desire handle compounds with symmetries. To this end, the method uses equivariant state representations for autoregressive generation, built largely from recently proposed covariant molecular networks (comorant), and integrating such representations within an existing actor-critic RL generation framework (Simm et al). The selection of focal atom, element to add, and the distance are realized in an equivariant manner while the compound valuation remains invariant to rotation. The approach is clean and well-executed. The authors added additional experiments (e.g., RMSD demonstrating stability of generated compounds) to further reinforce the case for the method. 

Decision: Accept (Poster)