Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The authors proposed PhysChem network for molecule representation. PhysChem consists of four modules, the initializer, the PhysNet, ChemNet and readout. Specifically, in PhysNet, the position, momentum and mass of an atom are modeled using rules in mechanics. The authors conduct a series of experiments to verify the proposed algorithm.
main_review: 1.	There are many previous work on QM9. You can find the results by searching 
(1)	https://paperswithcode.com/sota/formation-energy-on-qm9. 
(2)	https://paperswithcode.com/sota/drug-discovery-on-qm9
And QM9 has lots of (sub)tasks. I have two questions towards this point:
(i)	In Table 2, 3, 4, do you use the same way to get the training/validation/test data?
(ii)	A comparison with [ref1] should be discussed.

2.	Some ablation/discussion are required:
a.	What if you use Dihedral angle like [15]?
b.	In PhysNet, do we need both Positional Force and Torsion Forces? What if we use only one of them?

3.	I’m a little confused by your results. In Table 2, why not provide the results of PhysChem? In Table 3, w/ conf, what if we use PhysChem? More surprisingly, on QM7, ChemNet (s.a.) achieves the best results even without conf information.

4.	The RDKit generated conformations are not accurate but useful as illustrated in Table 3. What if we start from RDKit and then finetuning using your proposed method?

[ref1] Molecular Mechanics-Driven Graph Neural Network with Multiplex Graph for Molecular Structures, https://arxiv.org/pdf/2011.07457v1.pdf

limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper reports the use of a multi-task/model fusion approach to molecular representation learning: one NN learns to generate conformations through Hamiltonian dynamics with learned forces and another NN does supervised message-passing in an established way

main_review: PROs
The architecture fusion is novel and seems effective. By making 3D information a label rather than an input the models are more likely to be transferable, even when there are no existing geometries to train on, as is often the case in large chemical libraries or new chemical spaces

Learning forces directly makes energy not conserved, but it's fine for this application since there is no real "energy" being learned, and it's just a pseudo-energy (and pseudo-forces) to reach the desired geometries. I expect learning straightup pseudo-forces will become a standard in the field

The partitioning of pairwise interactions into the modified LJ and torsions coming from the Chemical Net is new and intereseting but it is not clear whether it's useful or important. A many-body MPNN model like MPNN (Schnet) could learn those terms without crosstalks between the models. I'd be intereseted in learning more about the role of the torsional terms, whether the can be ablated or replaced by learnable manybody MPNN terms in the potential and at what cost in performance / cost. 

CONs
The architectures themselves are not very innovative.

The RDKit RMSD for QMX data has me very worried. Most papers report around 0.3 RMSD between RDKit and DFT (and it has be low since the QMX geometries typically come from relaxing initial guesses produced with RDKit) (See arXiv:2105.07246v1 or any of the work from Jian Tang, for example) The values in table 2 and 3 are way too high, and raise concerns about how RMSD is being calculated (atom indexing issues?)

What is a "random" guess for a geometry? Just sprinkling atoms in space will sure give more error than 2 A. 

The justification for the low-order polynomial "Lennard Jones" is not great, but at the of the day it's an empirical choice, so it's fine. 

The baselines for property prediction are a bit old, so it is not clear how the models stack. I'd encourage the authors to refer to more recent architectures for property prediction, including modern equivariant ones. 


Small remarks:
"gravitations" for "attractions"
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper presents a novel method called PhysChem for molecular representation learning by fusing physical and chemical information of molecules. PhysChem consists of two cooperating neural networks, one called PhysNet and another called ChemNet. PhysNet learns 3D conformations by simulating molecular dynamics with parametrized forces, while ChemNet is a geometry-aware message passing network. The experiment results for MoleculeNet benchmark and SARS-Cov-2 datasets showed superior performance over the other state-of-the-art baselines.

GNNs or Geometric NNs are mostly separated to 3D-geometry-aware ones used for quantum-calculation surrogates and 2D-abstraction ones for QSARs predicting properties such as toxicity and solvability. Representation learning that integrates these two aspects is very interesting and promising.

main_review: This is a very interesting paper that integrates two different aspects of molecules that have been often studied separately. The structure of molecules are quite compositional/combinatorial (e.g. bonds, functional groups, fragments, scaffolds, etc), which motivates us to directly input graphs into neural networks as we seen in GNNs; On the other hand, a molecules exhibits its function in 3D physical space, and act as particle systems that follows the laws of (quantum) mechanics, which is physically constrained rather than being handled as a pure combinatorial object.

As shown in recent studies such as [16](ICLR2021) and [26](ICML2021), neural networks alone can potentially simulate molecular dynamics for conformation prediction. It is thus quite natural to expect that these networks capture physical aspects of molecules in 3D. So, the paper's idea is to learn PhysNet and ChemNet simultaneously: PhysNet uses ChemNet features to have torsion forces in chemical bonds, while ChemNet uses bond-length and bond-angle features from PhysNet 3D conformation. Doing like this, PhysChem is applicable to the situations where the task would be 3D-conformation related, but such conformation information is not directly available. (like a recent KDDCup2021 "PCQM4M-LSC" task to predict DFT-calculated properties from 2D molecular graphs?)

Overall I liked the idea, and but here are some comments.

- For "PhysChem (w/o conf.)" in Table 3 and Table 4, the paper uses UFF-based conformation generation of RDKit for supervised training. Are these conformations unique for each molecule?   
The RDKit document says that "Since the 2018.09 release of the RDKit, ETKDG is the default conformer generation method", and it looks to be able to generate multiple conformations by running the distance geometry calculation multiple times from different random start points. These multiplicities don't affect the performance of PhysChem?

- In Table 3, I'm not sure why ChemNet (standalone) can be better than or equal to ChemNet (multitask), PhysChem, and other baselines for QM7 and QM9. QM7 and QM9 is basically geometric-aware tasks, for example, energy is changed if the xyz coordinates are changed. In this sense, if we dropped 3d information as in ChemNet(standalone), it can be hard to predict these quantities. Do you have any rational explanations for this?

- Similarly, what makes PhysNet(s.a) better than HamEng? Without torsion forces from ChemNet, apparently PhysNet would have similar-level features, or even simpler and rough approximation of Lennard-Jones potentials. What brings us this improvement over HamEng?

- For ChemNet, the "triplet descriptors" are unique features, but the remaining would be within the scope of the standard customized MPNNs. So what makes this difference, for example, over MoleculeNet? The ChemNet architecture looks like a variant of gated graph neural networks (GGNNs), and how good if we just grab a standard GGNN here (for Table 3 and 4)?

- Is there any risk of leakage by intensive hyper-parameter tuning for PhysNet, ChemNet, PhysChem? How these results are dependent on careful tuning of hyperparameter tuning?

- The name "PhysNet" might be very confusing because there already exists a widely-cited model called "PhysNet" also applied to QM9 prediction.  
  
PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges (2019)
https://doi.org/10.1021/acs.jctc.9b00181  
For related work, check an ICML2021 paper https://arxiv.org/abs/2102.09844

limitations_and_societal_impact: The limitations are partly described in the Conclusion section.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors are interested in the linked tasks of predicting (i) 3D conformations and (ii) chemical/biomedical properties of molecules. They show that by designing a model (PhysChem) that tackles both of these tasks simultaneously, they can often do better on each individual task than when these tasks are considered separately. This is empirically demonstrated on the QM{7,8,9} datasets, the MoleculeNet  Physical Chemistry Benchmark [32], as well as a SARS-CoV-2 dataset [3], where the authors also compare to a variety of older approaches.




### Further Details on the Architecture
To be more specific about the architecture, PhysChem consists of two networks (which share information at each layer and both take in as input the graph representation of a molecule):
1. PhysNet, is used to predict the conformation of a molecule and it does this by first predicting the initial positions of the atoms (using a graph convolutional network (GCN) and RNN) before then updating these positions using the equations of motion under the action of interatomic forces output by a fully connected NN.
2. ChemNet†, is used to predict chemical/biomedical properties of a molecule, and is a new variant of a message passing neural network [9].

† This is a new network, not to be confused with previous networks also called "ChemNet", e.g.
Preuer, K., Renz, P., Unterthiner, T., Hochreiter, S. and Klambauer, G. (2018) ‘Fréchet ChemNet Distance: A Metric for Generative Models for Molecules in Drug Discovery’, Journal of chemical information and modeling, 58
main_review: **SUMMARY** 
I think the authors tackle an interesting problem, that of producing better representations of molecules by simultaneously training on different (but linked) tasks, and appear to achieve impressive empirical results. PhysNet alone is shown to outperform previous methods for predicting conformations. However, the proposed networks consist of several moving parts, and the architectural decisions seem fairly arbitrary at times; it would have been nice here to have some intuition or empirical/theoretical justification of the choices made (see limitations box below). Therefore, currently I have gone with a borderline acceptance score but am happy to raise this score if this point gets addressed in the rebuttal.



## Originality
i. The authors produce a novel architecture for predicting both molecular conformations and properties at the same time. Overall, the approach bears some similarities to HamNet [16], for instance the use of a neural physical engine to predict conformations along with the idea of a supplementary MPNN-like architecture to produce a molecular fingerprint. However, there are also some key differences (described in lines 81-91) such as the idea of parameterizing the forces directly or the idea of receiving information back from the MPNN into the physical engine. This ultimately results in better empirical results (Tables 1-3).
 
## Quality

ii. As far as I can tell the submission is technically sound. The claims are supported empirically and state of the art performance is obtained (compared to some relatively recent methods, e.g. [16] at ICLR 2021). However, this is evaluated in terms of MSE/MAE and it would be nice to also have a comparison in terms of number of parameters/compute costs. There is actually some work in this direction in Appendix H, which shows PhysChem is competitive in this regard, but it would be nice to have this extended to the other models and brought into the main paper.


## Clarity

iii. The approach is quite involved but, considering this, I thought that the authors did a good job at explaining the various steps involved. In particular, Figure 1 provides a helpful overall view of the approach and algorithms 1-3 (in the appendix) provide detailed descriptions of the individual steps involved. The authors provide code which I have glanced through. This is helpful but perhaps could be tidied up a little (e.g. there are a lot of top-level files and I am usure which are important, and also installation instructions should be provided in the README.md)

iv.One aspect relating to clarity that could be improved is better motivation of the choices made. Some of the architectural decisions seem a little ad-hoc and it would be nice to have some intuition for them. For instance, I'm confused as to the motivation behind using Lennard-Jones potentials (line 153) and then the subsequent simplification of these with lower-degree polynomials (line 158). It would be nice to have a more detailed background of this choice for a ML audience in the appendix. (I provide other examples of architectural decisions for which I think further explanation could be provided in the limitations box below.)

v. On a much more minor point, the grammar/choice of words could perhaps be improved in a few places. However, this should be easy to fix. Here are a few examples:  
    a. Line 66: "Models that leveraged 3D geometries of molecules generally performed better than those simply used graph notations" => "those _that_ simply used graph notations"?  
    b. Line 145-146: "and f is the force that the atom is compelled" => "the force that the atom _experiences_."...?  
    c. Line 35: "For example, torsions of bond lengths and angles greatly influence the dynamics the of particle systems" => I think the "the" is unnecessary...?  


## Significance
vi. The results improve upon state of the art and therefore I believe may be of interest to other researchers or practitioners. Ablation tests are performed to distinguish the performance improvements due to the PhysNet vs ChemNet but I still am a bit confused as to what components of these individual networks are important (see limitations box below).


## Other Miscellaneous Questions to the Authors

vii. For the molecular properties tasks (e.g. Table 3 and 4) it appears much of the gains come down to the architecture of the MPNN network used, rather than the use of conformational information inside this network. Is this fair? Why do you think this network works better than the previous ones compared to? 

viii. Could PhysNet be used as an addition to other networks in the MPNN framework?

ix. I was surprised that only 4 time steps were used in estimating the position/momenta of the atoms in PhysNet. This seems rather low (for instance, in HamNet [16] my understanding is that they use the equivalent of 20 such steps)? Do the positions and momenta converge after so few steps?

x. In the main paper (line 263) you state you set $d_f=128$, yet in the appendix (line 50) this is said to actually be 3? Which value did you actually use? I'm also having trouble reconciling the results in Figure S2 (of the Appendix) with Table 2 of the main paper. For instance, (assuming $d_f=3$) Figure S2 states the results on conformation learning on QM7 are ~0.52; however, in Table 2 they are stated at ~0.49. I would be grateful if you could point out where I am going wrong?

xi. In addition to reporting multi-MAE in the main paper it would be nice to have the individual MAEs for the separate tasks in QM9 in the appendix if possible (for consistency with previous work, e.g. [32, Table S9; 33, Table ST5]).

xii. In figure 2, what do the superscripts for the different PhysChem models represent (e.g. 0,1,2)? (sorry if I missed this).

xiii The LSTM (Eq 4) outputs specific atom coordinates, but I believe the ultimate loss is invariant to rotations/translations of these coordinates? Does this mean the LSTM obtains the same loss even if it arbitrarily translates all initial positions by the same amount? Why not fix the first atom's initial position always to the origin?

limitations_and_societal_impact: ## Limitations
A. My understanding is that the approach proposes several modifications to the approach used in HamNet [16] but it wasn't clear to me which of these are important to obtain the improved results/why these changes have been made. Examples include:
1. PhysNet predicts the forces directly rather than the energies  (as in HamNet [16]).
2. PhysNet obtains back from ChemNet MPNN information (in the form of "torsion forces") -- in HamNet the information only flows from the physical engine to the MPNN.
3. PhysNet uses a different conformation decoder loss (Eqn 13).
4. ChemNet uses a bespoke message passing scheme, relying on messages that are computed from triplets of graph nodes (as opposed to the ordinary use of pairs -- Section 3.4).  

It would be good if empirical evidence/better motivation of these choices was provided.

B. At a higher-level, it would be nice if the general idea of sharing information between conformational and chemical property prediction networks is shown to be useful in a way that is agnostic to the specific architectures used for the separate networks. At the moment this contribution seems somewhat muddled up with the contributions of providing new (and somewhat bespoke) network architectures for these tasks.

C. It actually seems (Table 3) that sharing the information from ChemNet into PhysNet actually degrades performance on the larger datasets QM8/QM9. I would be interested to hear why PhysNet cannot ignore this information if it is not helpful? This also seems to be in conflict with the central claim of the paper (that sharing information between these different tasks is helpful)?


## Societal impact
D. The authors include a Broader Impact subsection in section 5. Here the authors state: "With limited possibility, the abuse of PhysChem in drug discovery may violate some ethics of life science." However, I admit to not understanding the specific potential negative societal impacts that they envisage and would have found an explicit example helpful.

E. In this section the authors also state that their approach provides a more interpretable architecture. However, I would have liked more details on what they meant/some evidence backing up such claims.
review_title: null

Meta-review: In this paper, the authors proposed a new neural network model called PhysChem for molecule representation. In PhysChem, a PhysNet and a ChemNet interact with each other to facilitate the learning process. The authors conduct a series of experiments to verify the practical effectiveness proposed algorithm. Overall speaking, the reviewers are positive on this paper. They like the intention of the authors to integrate physical and chemical knowledge in the design of the neural network. But on the other hand, the reviewers also raise some concerns, including the experimental setting, the role of PhysNet, etc. The authors did a good job in addressing most of the concerns, and the consensus made among the reviewers is more leaning towards the positive side. Therefore my recommendation is ACCEPT as a poster.
Decision: Accept (Poster)