Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper describes PhiSNet, a model for predicting the Hamiltonians of systems of atoms in the context of quantum chemistry. The output Hamiltonian can then be used to derive the wavefunction using traditional methods, and the wavefunction can in turn be used to derive physically observable properties of interest. A core part of the design of this model is built-in equivariance to rotation in three dimensions. The paper defines the architecture of the PhiSNet and each of its equivariant component layers in detail, and results are presented from a comparison to the competing method SchNOrb.
main_review: In general the paper is well-written and well-structured, with about the right amount of space devoted to each part (e.g. the background in section 3 is pretty minimal, but I think this is the correct decision here, and I appreciated the longer-form background provided in the supplement). As with almost all papers, it would benefit from another proofreading pass to catch minor errors (e.g. "calculations and led to" on line 25 should be "calculations, and have led to"). The detailed description of PhiSNet and its component parts is clear and well laid out. Figure 1 is excellent, I found it extremely helpful in clarifying the problem setting and the architecture.

There are, as the authors note in section 2, a number of existing SE(3)-equivariant machine learning methods. In section 4.1, it is unclear which of the building blocks the authors are claiming as novel and which are taken from (or heavily based on) these earlier works. They say "we describe novel general-purpose operations" on line 118-119, but clearly not everything that follows is novel (e.g. the chosen activation function and the linear layers). I would appreciate more clarity on this.

A point about terminology: I'm unclear exactly what the authors mean by "irrep". Strictly an irrep is a representation, i.e. a homomorphism from SO(3) to GL(N) (which satisfies some additional properties to be "irreducible"), but the authors also seem to refer to feature vectors as "irreps" (e.g. in section 4 on line 152). Is the idea that SO(3) acts on these feature vectors by an irrep?

The results appear impressive, though since I am unfamiliar with this application domain I am taking it on good faith that the results in Table 1 represent a fair comparison against a state-of-the-art competitor. The results are the most compelling part of the paper, and they are the main thing that convinces me of the value of the model architecture described in section 4.
limitations_and_societal_impact: The authors adequately address some limitations of their work in section 5. They do not discuss potential societal impact - it is not clear to me that such a discussion is necessary for this paper.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: 
This contribution describes a method that approximates quantum mechanical total energies and wavefunctions with compute costs that are similar to those of Hartree Fock calculations. The paper describes a novel architecture, PhiSNet, that predicts wavefunctions from nuclear charges and positions: an equivariant featurization of the atomic information feeds a network that is built using a number SE(3) equivariant building blocks that in turn feeds to a block that builds the Hamiltonian matrix. The authors demonstrate that PhiSNet can train to predict DFT total energies to accuracies of about 0.11 kcal/mol for aspirin and better for smaller molecules. They further demonstrate that the network can "correct" a HF level wavefunction to an effective wavefunction that includes the effects of electron correlation.

main_review: 
The method is a clear improvement over SchNOrb: it incorporates SO3 symmetries in its core and it shows significantly better approximation accuracy, better data efficiency, and reduced model size. The paper is written in a clear and concise way, though the demonstrations are somewhat lacking compared to what I'd have expected for the NeurIPS conference. The possible combinations of the building blocks all make sense, but I'd have liked to see an ablation study or additional arguments in the supplement for the particular combinations of architectural choices. Importantly, the authors do not discuss how/if this architecture could generalize to predict results on a system not seen in the training set. They also don't show performance on total energy when using the previously established 1K MD17 training datasets, so it's unclear how much this contribution adds to practical applications. Although the prediction of DFT level wavefunctions for new molecular conformations is potentially interesting, I was more excited about the possibility to correct these wavefunction to an effective CCSD(T) wavefunction. Unfortunately, the example with a single illustration of a single water molecule is not very convincing that this architecture can provide useful practical tool in the near future.

Edit: following the additional work that demonstrates the performance gains on DFT-level calculations, I've updated my overall score to 6.

limitations_and_societal_impact: The authors discuss the scaling limitations of their work as they related to the cubic scaling with the number of atoms; there are no potential negative societal impacts of this work.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper describes a method to predict the wavefunctions and electron densities using a rotationally equivariant method. The method achieves sufficient accuracy for the model to be used to predict properties such as energies and forces directly from atom positions and charges. The approach is evaluated by comparing to an existing rotationally-invariant approach using prediction errors in Kohn-Sham matrices, energies and wavefunction cosine similarity as metrics to compare the methods. Background on relevance and significance as well as implementation details are provided in the supplementary information along with sample code.
main_review: **Main paper**

1. Related works: The following highly related references should be cited since they are part of very related threads of work:

   [1] "SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks" by Fabian B. Fuchs and Daniel E. Worrall and Volker Fischer and Max Welling

   [2] "OrbNet: Deep learning for quantum chemistry using symmetry-adapted atomic-orbital features" by Zhuoran Qiao, Matthew Welborn, Animashree Anandkumar, Frederick R. Manby, and  Thomas F. Miller

   At the very least some comments on how the proposed method differs from these approaches should be included. Reference [1] presents an equivariant network for property prediction and reference [2] includes a 1000x speedup over DFT by leveraging symmetry-aware features for orbital data. 
    
2. The abstract, Section 1 and Section 3 (theory development) all mention that the underlying geometries of molecules need to to rotation-equivariant to capture the various rotational symmetries without losing the information about the rotation. This suggests that an SO(3)-equivariant network would be sufficient. However, there is an abrupt jump to SE(3)-equivariance in Section 4 (Deep learning architecture). What is the reasoning behind that? Note that SE(3) expresses $[R | t]$ and SO(3) expresses $R$, where $R$ is the rotation matrix and $t$ is the translation vector in $\mathbb{R}^3$. We can ignore the homogeneous formulation's scale factor for now. How/When do translations need to be preserved and predicted? How do translations even manifest in a representation of a molecule? Do individual bonds within a molecule elongate or contract (in response to temperature or pressure, for example) and if so, then how is that data fed into the network? Or is SE(3)-equivariance being used as a superset of SO(3)? Or is it the case that properties of groups of atoms within a molecule are of interest, and the bonds attaching these groups to rest of the molecule can change length, thereby encoding some useful information in the translation components? 

3. Figure 1 (D). The HF-level function, the correction (amplified) and the CC-level function are presented. By eye, the HF-level function and CC-level function look the same to me. Is it correct to expect 'CC-level function = HF-level function + correction'? If not, how should we interpret Figure 1-D? A clarifying caption would be helpful. Alternatively, the differences in HF-level function and CC-level function could be highlighted. 

4. Evaluation [Table 1]: The paper clearly states that the approach is built on PhySNet, yet the proposed method (PhiSNet) is not compared to PhySNet. It is instead compared only to SchNOrb. The results presented in Table 1 for SchNOrb do correlate with the results in the supplementary material of the SchNOrb publication, but only partially. So I presume authors ran additional experiments. Great. 

5. Evaluation [Table 1]: It would be good to have a comparison to PhySNet here as it would allow comparison to a whole host of methods. OrbNet evaluates a number of methods against DFT including PhySNet and at a 1000x speedup on DFT. It would be good to get a rough estimate of speedup of the proposed method wrt DFT as a general guideline. 

6. Evaluation: Several other works in this area present results for property prediction (QM9, QM7, DrugBank etc). It is curious to not find these included in the paper as the paper explicitly claims to provide good transferability to property prediction. I do note that some results wrt HF to CCSD transfer learning are provided in the supplementary material but as these measure very different quantities from those in related works, it's unclear how to interpret the results. (Apologies, I am still getting my head round the level of theory business).

7. Section 4.1 is missing a sentence or two in way of introduction. It's quite unclear whether the layers defined in Section 4.1 refer to PhiSNet (this paper) or PhySNet (a previous work upon which this paper builds). I am assuming they refer to PhiSNet (this paper) based on the cited references inline and line 144-145, but just staing this upfront would be easier for the reader. 

8. Section 4.1 [Interaction blocks]: This is the first point where positions $\mathbf{r}$ of the atoms and the inter-atom distances $\mathbf{r}_{ij}$ are used. So coming back to my earlier point about SO(3) vs SE(3), there could be an argument here for needing SE(3) to encode the bond lengths between the atoms. Much as before, aren't these known and fixed quantities or do they vary substantially, and if they do, do datasets actually have that variation? Else, why not just assume some units of 'one C-H bond away'? (Of course, things get complicated when the bonds are not unique, i.e. valence allows either double or triple bond between C and N etc). But this can largely by disambiguated by using valence constraints.

9. In Eq (16), it might be good to add some suffix or words to clarify that the $\mathbf{f}$ in the definition of $\mathbf{c} = residual(\mathbf{f})$ and $\mathbf{n} = residual(\mathbf{f})$ operates on different atoms within the expressions itself. 

10. Network size: A comparison of the number of parameters is provided wrt SchNOrb in the supplementary information (17M for PhiSNet and 34M for SchNOrb). How many parameters does PhySNet have, seeing as that's the network this paper is most closely based on?

11. In the contributions section, the authors say the proposed method improves over state of the art by two orders of magnitude. It would be helpful to clarify whether this is the case when comparing to equivariant methods or non-equivariant methods? I presume it is the latter. Going back to the point on evaluation, as the main evaluation only compares against one other method, it would be good to add a comparison to PhySNet and perhaps also, where possible a partial/alternative comparison to other methods, such as the ones OrbNet uses for evaluation with a accuracy-effort tradeoff to highlight the benefits of the current approach (end-end trainable). Ballpark number could also suffice for such evaluation. 

12. There is a slight contradiction in the contributions section (Lines 49-53): Contribution 1 suggests the need for SE(3) whereas contribution 2 suggests only transformation under rotations are required to be preserved. 

13. Evaluation: PhySNet seems to improve with ensembles. Does it help with PhiSNet? If it's not needed, that could be a good thing to highlight. 

14. Would the method scale to work on proteins (larger molecules but more repeated structures) as it does for small molecules. Most of the examples included in the paper are for small molecules. There is a passing comment about computational complexity scaling with the number of atoms in Section 5, but my question is more around the repeated structures aspect.  


**Supplementary information:**

1. Thank you! The background information on quantum chemistry (Section A) leading up to the embedding details (Section C) along with the network architecture diagrams and Hamiltonian matrix construstion with minimal basis sets (Section D and Figure S5) are the clearest I have seen in this field. To make this even better, I would suggest using terminology from both ML/CV and Physics where relevant. Group equivariance is heavily studied in both physics and computer vision, for example, but different terminology is used. Perhaps for electronic dissemination, it would make sense to have a long version of this paper that interweaves the main paper and the supplementary info so as to give the reader an uninterrupted (if longer) and lucid flow of reading.

2. Why do the batch sizes used for different molecules (Table S4) vary so much? Is it in direct proportion to the number of atoms per molecule? Do you not have to change the learning rate or the number of epochs to adapt? In other words, are the number of atoms within a batch for the different molecules roughly constant? 

**Notes**

Writing style: At several points while reading the paper, I had questions along the lines of 'What do the different levels that the authors refer to mean?' or 'What is **S** matrix'? All these questions are answered nicely in the supplementary information, but then it's unclear what order to read the paper and supplementary info in? Given the topic of the paper, quite a bit of background from quantum chemistry/physics is required, I think it's fine for there to be a substantial supplementary information for those who may need it, but it would be nicer for all readers if the supplementary info was referred to in the main paper whenever the background from supplementary info is required for understanding the content in the main paper. For example, 'We predict **S** (see Supp. info Section .. for explanation of **S**)'. 

**Overall assessment**

This paper presents a very good motivation of DL methods for quantum chemistry,specifically learning wavefunctions from electronic orbital information. The method presented is novel even though it builds on previous work in the area. The method addresses a key challenge and demonstrates significant improvement in accuracy of wavefunction and energy prediction with an end-end approach. However, some comparisons and comments on differences wrt key existing works would be helpful in understanding the importance of this work. 
limitations_and_societal_impact: Yes, to a reasonable extent.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes PhiSNet, an SE3 equivariant NN whose inputs are distance vectors between atoms of a given molecule with nuclear charges, and outputs a prediction of the entries of the Hamiltonian matrix that is used to solve the Schrodinger equation to find the electronic wavefunction of the molecule. It is claimed that training PhiSNet on a molecular dataset reaches state of the art prediction accuracies on various molecules.

main_review: Without a background in quantum chemistry, it is difficult for me to assess the significance of the results in the paper, hence I will focus on assessing the proposed methodology.

The paper attributes the success of PhiSNet over the baseline SchNOrb to its SE3 equivariance, but it is unclear how much of the improvements over the baseline are due to its equivariance and how much is from design choices of the network. Of course these two come hand in hand, so it's difficult to separate their contributions to improving prediction accuracy, but a more detailed description of the baseline method along with a description of the difference between this and PhiSNet would be desirable. In addition, given the claims of SOTA, it would be useful to have ablations to explain the various design choices of PhiSNet, e.g. parameterisation of Interaction blocks (eq 13), using a different set of parameters for the central point features and the neighbouring point features (eq 15, 16), or using the other neighbours of atom i for computing the pair interaction features between atoms i & j (eq 18). In summary, a better breakdown of what design choices have led to how much improvement in prediction accuracy would be very helpful for explaining the success of PhiSNet and will encourage follow up work to make further improvements.

The paper introduces PhiSNet as a novel SE3 equivariant architecture for point cloud data, but it's not clear where the novelty is: it's not clear whether the authors are claiming some of the building blocks in Section 4.1 are novel or that the combination of these existing building blocks is novel. The building blocks described in Section 4.1 are lacking in citations e.g. Tensor product contractions have been used frequently in the SE3 equivariance literature, so relevant work should be cited e.g. Clebsch-Gordan Nets [Kondor et al, 2018].

I appreciate the paper's attempt to make the methodology clear, with details on notation and the methodology, but there remain some points of confusion 
- In line 207 it is said 'tilde{x} serves as input to the next module in the chain' but I don't see tilde{x} appearing subsequently.
- In line 209, what is the sum across?
- I find the use of Tensor product expansions to construct matrix predictions interesting, but the explanation in Section 4.2 is not so clear (e.g. what is used as 'a' in eq 15?). It would be helpful to include Figure S5 as part of the main paper for this.

limitations_and_societal_impact: n/a
review_title: null

Meta-review: The authors present an SE(3) equivariant architecture for predicting symmetric Hamiltonian matrices, which can then be used to derive physical properties of molecules. This architecture provides a dramatic improvement in accuracy over baseline methods. The majority of reviewers agreed it was an excellent paper deserving acceptance. The one holdout reviewer had concerns about the scale of the experiments and recommended that results be presented for the entirety of the MD17 dataset. However I am convinced by the author response that this would not be feasible with the computational resources at their disposal, and I am willing to cut them some slack on this given the quality of results on the systems presented. Therefore I recommend that the paper be accepted.
Decision: Accept (Poster)