Conformation-Guided Molecular Representation with Hamiltonian Neural Networks
Authors: Ziyao Li
Shuwen Yang
Guojie Song
Lingsheng Cai
Venue: ICLR 2021 Poster (2021)
Keywords: Molecular Representation
Neural Physics Engines
Molecular Dynamics
Graph Neural Networks

Abstract: Well-designed molecular representations (fingerprints) are vital to combine medical chemistry and deep learning. Whereas incorporating 3D geometry of molecules (i.e. conformations) in their representations seems beneficial, current 3D algorithms are still in infancy. In this paper, we propose a novel molecular representation algorithm which preserves 3D conformations of molecules with a Molecular Hamiltonian Network (HamNet). In HamNet, implicit positions and momentums of atoms in a molecule interact in the Hamiltonian Engine following the discretized Hamiltonian equations. These implicit coordinations are supervised with real conformations with translation- & rotation-invariant losses, and further used as inputs to the Fingerprint Generator, a message-passing neural network. Experiments show that the Hamiltonian Engine can well preserve molecular conformations, and that the fingerprints generated by HamNet achieve state-of-the-art performances on MoleculeNet, a standard molecular machine learning benchmark.

Published as a conference paper at ICLR 2021
CONFORMATION-GUDDEL MOLECULAR REPRESENTA- TION WITH HAMILTONIAN NEURAL NETWORKS
Ziyao Li', Shuwen Guojie Song", Lingsheng Cai²
Center for Data Science, Peking University, Beijing, China
Key Laboratory of Machine Perception and Intelligence (MOE), Peking University, Beijing, China [1 (leeeezy, swyang, g gisong, cai inggheng) @ edu.en en
ABSTRACT
Well-designed molecular representations (fingerprints) are vital to combine med- ical chemistry and deep learning. Whereas incorporating 3D geometry of molecules (i.e. conformations) in their representations seems beneficial, current 3D algorithms are still in infancy. In this paper, we propose a novel molecular representation algorithm which preserves 3D conformations of molecules with a Molecular Hamiltonian Network (HamNet). In HamNet, implicit positions and momentums of atoms in a molecule interact in the Hamiltonian Engine following the discretized Hamiltonian equations. These implicit coordinations are super- vised with real conformations with translation- & rotation-invariant losses, and further used as inputs to the Fingerprint Generator, a message-passinn work. Experiments show that the Hamiltonian Engine can well preserve molec- ular conformations, and that the fingerprints generated by HamNet achieve state- of-the-art performances MoleculeNet, standard molecular machine learning benchmark.
neural net-
INTRODUCTION
The past several years have prevalence of the intersection between medical chemistry and deep learning. Remarkable progress has been made in various applications on small molecules. ranging from generation (Tin et al.] 2018 You et al.] 2018) and property prediction (Gilmer al., Klicpera et al. 2020) to protein-ligand interaction analysis (Lim et al. 2019 Wang al. 2020), yet all these tasks rely on well-designed numerical representations, or fingerprints, of molecules. These fingerprints encode molecular structures and serve as the indicators in downstream tasks. Early work of molecular fingerprints (Morgan 1965 Rogers & Hahn 2010) started from encoding the two-dimensional (2D) structures of molecules, l.e. the chemical bonds between atoms, often stored as atom-bond graphs. More recently, trend of incorporating molecular geometry into the representations arose (Axen et al. 2017 Cho & Choi 2019).
2017
Cho & Choi, 2019;
Molecular geometry refers to the conformation (the three-dimensional (3D) coordinations of atoms) of molecule, which contains widely interested chemical information such as bond lengths and angles, and thus stands vital for determining physical, chemical, and biomedical properties of the molecule. Whereas incorporating 3D geometry of molecules seems indeed beneficial, 3D finger- prints, especially in combination with deep learning, are still in infancy. The use of 3D fingerprints limited by pragmatic considerations including i) calculation costs, ii) translational & rotational invariances, and rii) the availability of conformations, especially considering the generated ligand candidates in drug discovery tasks. Furthermore, compared with current 3D algorithms, mature 2D fingerprints (Rogers& & Hahn 2010; Gilmer et al. 2017 Xiong et al.) (020)) are generally more popular with equivalent or even better performances practice. For example, as a 2D approach, Attentive Fingerprints (Attentive FP) (Xiong et al. 2020) have become the de facto state-of-the--a approach.
To push the boundaries of leveraging 3D geometries in molecular fingerprints, we propose HamNet (Molecular Hamiltonian Networks). HamNet simulates the process of molecular dynamics (MD)
Equal Contribution.
1 "Corrsponniig Author.

---

Published as a conference paper at ICLR 2021
to model the conformations of small molecules, based which final fingerprints calculated similarly to (Xiong et al., 2020). To address the potential lack labeled conformations, HamNet does not regard molecular conformations all all-time available inputs. Instead, A Hamiltonian engine is designed to reconstruct known conformations and generalize for unknown ones. Encoded from atom features, implicis positions and momentums of atoms interact in the engine following the dis- cretized Hamiltonian Equations with learnable energy and dissipation functions. Final positions are supervised with real conformations, and further used as inputs to Message- (MPNN) (Gilmer et al. (017) to generate the fingerprints. Novel loss functions with translational & rotational invariances are proposed supervise the Hamiltonian Engine, and the architecture of the Fingerprint Generator is elaborated to better incorporate the output quantities from the engine.
Neural Network
We show via our canforrmttion-rcconttuuiiiiiiiii experiments that the proposed Hamiltonian Engine is eligible to better predict molecular conformations than conventional geometric approaches as well as common neural structures (MPNNs). We also evaluate HamNet on several datasets with different targets collected in standard molecular machine learning benchmark, MoleculeNet (Wu et al. 2017), all following the same experimental setups. HamNet demonstrates state-of-the-art performances, outperforming baselines including both 2D and 3D approaches.
2 PRELIMINARIES
Notations. Given a molecule with atoms, we use v; to denote the features of atom and eij that the chemical bond between and j (if exists). Bold, upper-case letters denote matrices, and lower-case, vectors. All vectors in this paper column vectors, and stands for the transpose operation. We use + for the concatenation operation of vectors. The positions and momentums of atom are denoted qi and pi. and the set of all positions of atoms in molecule is denoted as Q = (qi, (dis""", qn))_ .W() refers to the neighhorhood of node e in some graph.
Graph Convolutional Networks (GCNs). Given attributed graph G AV X). where V = {vi, Un is the set of vertices, A E Raxa the (weighted) adjacency matrix, and X E Rund the attribute matrix, GCNs (Kipf & Welling 2017) calculate the hidden states of graph nodes
GCN (1 (X) (x) (L),
H
(AH"WW))
( X, 1=
L 1- (1)
Here, H .h...'... he.)) T are hidden representations of nodes, Á D + AD 1/2 is the nor- malized adjacency matrix, D with D;: = E; Aij is the diagonal matrix of node degrees, and Ws are network parameters.
Message-Passsing Neural Networks (MPNNs). MPNN (Gilmer al. 2017) introduced general framework of Graph Neural Networks (GNNs). In the f-th layer of typical MPNN, messages (m') are generated between two connected nodes (i, j) based on the hidden representations of both nodes (h') and the edge in-between. After that, nodes receive the messages and update their own hidden representations. readout function is then defined over final node representations (h²) to derive graph-level representations. Denoted in formula, the calculation follows
Σ (h,, him Cv,w), h+t1 Uh 1, wEN(v))
R((h"lve (2)
m
where M,, Ue, R. are the message, update and readout functions.
Hamiltonian Equations. The Hamiltonian Equations depict Newton's laws of motion in the form of first-order PDEs. Considering a system of n particles with positions (qi. qnx) and momentums (p......P .Pn), the dynamics of the system follow
dg. 0H 0p,
p. dp:
0H 0q.
q: dt
(3)
dt
where H is the Hamiltonian of the system, and equals the total system energy. Generally, the Hamiltonian is composed of the kinetic energy of all particles and the potential energy as
H Σ T +u. =
(4)
2

---

Published as a conference paper at ICLR 2021
Pariit
GCN+ LSTM
(ig)
Aqu
Hamiltonian Engine
Hamiltonian Equations
Siverstoon
Xinniez
GCN+ LSTM
AP's
Als 4p
Rost Anorgel Adlonony
Brvff Conternationn
Doupativ
Position & Momentum Intiialization
Featurization
Hamilloniaa Molccular Dyramics
A
A
Fingerprint Generator
Laver
p!
Shiiecsio Tosls
Busi Nol
Rekaate Foot
Atom-leve Attentive (Vss, Passing)
Global Attentive Layers (Rendout Func.)
Layers with leorrable parameters
Inputs/Labbeis
Interaction betwern two medules
Hidken Representations
Figure I: The overall structure of HamNet.
Mcanwhile, dissipation exists in the system, the Hamiltonian Equations shall be adapted as
OH d 0p:"
0H
- 07
0
p: =
m
(5)
0g.
0q:
3q:
Op:
where m. is the mass of the particle and d is the dissipation function which describes how the system energy is dissipated by the outer environment.
METHOD: HAMNET
Figure shows the overall architecture HamNet. HamNet consists of two modules: i) Hamil- tonian Engine, where molecular conformations are reconstructed; and ii) Fingerprint Generator, where final fingerprints are generated from atom bond features and outputs from the Hamiltonian Engine.
3.1 HAMILTONIAN ENGINE
Discretized Hamiltonian Equations. The Hamiltonian Engine is designed to simulate the physical interactions between atoms in molecule. To correctly incorporate the laws of motion, discretize the Hamiltonian Equations with dissipation (Bquation 5) and model the energy and dissipation with learnable functions. At the 1-th step 1,
the engine, for atom i,
(H(1)
(7(
Oe!1)
(1+1)
L.
(4)
p (t+1)
p.
+ my
(6)
0p,"
0q.
0p."
Here, v is hyperparameter of step size which controls the granularity of the discretization, m, is the (normalized) mass of atom i, and (()), 4(() are the learnable Hamiltonian and dissipation functions of (() and (t);; (superscripts skipped)
H Σ Ti(p:) + U(g. ... ,q.), +
p(p.).
It should be noted that in order to improve the expressional power of the network, extend the concept of positions and momentums into implicit ones in a generalized d ddddimensional space, i.e. q.p E Rdi,d, .d, > 3. We will discuss how to supervise these implicit quantities, and will show the in- fluences of the dimensionality in the experimental results. Nonetheless, we parameterize the energy
3

---

Published as a conference paper at ICLR 2021
and dissipation function in physically inspired manner: for the atom- wise kinetic energy, we gen- cralize the definition of kinetic energy (I' _ as the quadratic forms of the implicit momentums; fennuatiiiounnttoonn for the atom-wise dissipation, we adapt the Rayleigh's 's dissipation function ( = E Gzd:())) into the generalized space. Denoted in formula, we have
T.(p.) P. WT WTP:
P. W W p:
(b(p.)
(8)
2m,
2mg
For the potential energy, simplify the Lennard-Jone potential (U(r) c (x--12 (1-1)-1-9) with parameterized distances rs in the generalized space, that is,
U
Σ Wajs #
Mk = Taj" Tij
ri = r.q. q,) = (q: -;)) TW Wo - -g))
In both Equation and Equation Ws are network parameters. For the mass of the atoms, we empirically normalize the relative atomic mass M; of atom with my = M,/50. One would also note that the parameters in each step of the engine remain the same, and thus the scale of parameters in the engine depends only on d and is irrelevant to the depth T.
Initial positions and momentums. Graph- based neural networks are used initialize these spatial quantities. Molecules are essentially graphs, while bonds between atoms can be of different types (single, double, triple and aromatic). Instead of using channel-wise GCNs which assign different parameters for different types of edges (Schlichtkrull et al. 2018), we calculate bond-strength adjacency for the molecules: the strength of a bond depends on the atom-bond-atom tuple, i.e.
Aij = sigmoid (MLP ((V, @ Eij + vy))) if the bond exists, Aij = 0 otherwise.
(10)
With so-defined adjacency, we first encode the atoms with vanilla GCNs (Equation]) I and concate- nate all hidden layers following the DenseNer scheme (Huang al., 2017). Deep enough GCNs do capture entire molecular structures, however, atoms with identical chemical environment cannot be distinguished, for example, carbon atoms in the benzene ring. This may not be problem in conventional MPNNs, but atoms with coinciding positions are inacceptable in physics as well as the Hamiltonian engine. Therefore, we conduct an LSTM over the GCN outputs generate unique positions for atoms. The order that the atoms appear in the LSTM conforms with the SMILES rep- resentations (Weininger 1988) of the molecule, which display atoms in the molecule in a specific topological order. Denoted in formula, initial positions and momentums of atoms are calculated as
qi
(1)
0) LSSM,,, (qs) q=a: I
(11)
L p:
(0) P = LSTM, (p., Pay. .P..)
(12)
t=
9. are hidden representations of atom i in the I-th GCN layer and sy is the atom at k-th position in the SMILES. As unique orders are assigned for atoms, their initial positions are thus unique.
where f
Conformation preserving. After the dynamical process in the Hamiltonian Engine, positions in the generalized Rd space are transformed into real 3D space linearly, that is,
Q QW trans: Q
IR x3 Q = (q1, qq.) E gaxd,
(13)
Considering translational & rotational invariances, we do not require that Q approximates the la- beled 3D coordinations of atoms. Instead, three translational- and rotational-invariint metries are proposed and used to supervise Q:
Kabsch-RMss RMSD (K-RMSD). We use the Kabsch Algorithum (Kabsch 1976) to rotate and align the approximated atom positions (Q) to the real ones (Q R), and then calculate the Root of Mean Squared Deviations (RMSD) of two conformations using atom mass as weights:
En m, x
1 qth 1
Q Kabsch ((ibsc(((Q()), Lk-rmed (Q.Q))(Q)))))))
(14)
Σ mi
4

---

Published as a conference paper at ICLR 2021
One should note that the alignment Kabsch(::-) is calculated with SVD and is thus differentiable. II) Distance Loss. Pair-wise distances between atoms are vital quantities in describing molecular conformations and enjoy desired invariances. Therefore, propose the metric Liit as
Laisi Qt)
E (1la: 2:112 --112-1100 --11aa" af'l(a)
(15)
III) ADJ-k Loss. A drawback of the naive distance loss is that distances between far atoms are over-emphasized, leading to deformed local structures. Therefore, we further propose ADJ-k loss, where only distances between k-hop connected atoms are preserved under weights calculated from hop-distances. Denote the normalized, simple adjacency matrix A the ADJ-k loss is defined
L 1( QQ.QR)
Σ 1,==1
A (l2: 9 11 of T)((2)
(16)
In implementation, a linear combination of K-RMSD and ADJ-3 losses supervise the engine, iee. LIE Lx-rrmsd + X where a hyperparameter.
3.2 FINGERPRINT GENERATOR
After the dynamical process in the Hamiltonian Engine, the molecular fingerprints are generated with the outputs as well as atom & bond features. The architecture of the Fingerprint Generator can be seen as MPNN (Gimeer al. 2017) instance. Analogous that Attentive FP (Xiong al. 2020), messages are generated with Graph Attention Layers (GAT) (Velickovic et al. 2018). and hidden representations of nodes are updated with Gated Recurrent Units (GRU) (Cho et al. 2014). Nonetheless, the architecture of HamNet is further adapted as conformation-awwen calculation of messages and attentive energies to incorporate relative positions and momentums. Denoted in formula, the atom-level calculation in the Fingerprint Generator follows
we modify the
h ML,( fiz MLP((e)),
Tt (q. p.) (q, + @
(17)
(w (w!)) (fi θ mg)) a softmax (fd,l) < NN())).
(18)
1+1 m
it wh Orey JEN(()
Σ
N+I m 1 0,0, L. (19)
Atom representations in the last layer (hfss then serve inputs global attentive readout func- tion. virtual meta node (g) is established and connected to all atoms order to conduct M layers of attentive pooling. Similar to that in atom-level calculation, the positions and momentums of atoms are incorporated the calculation:
h
Σ
8TH = softmax ((1) V}),
(20)
m gg
Σ
W m qi e
p.
h = GRU s)). m = 0, ..., M. 1 01.
(21)
Here, 8y is the global message, and is the set all atoms. The final output, A.M is the desired fingerprints of the target molecules. The same current neural molecular fingerprints (Duvenaud et al. 2015 Xiong et al. 2020), the generated fingerprints are then supervised with molecular prop- erties, such as regression and classification tasks.
'The simple adjacency matrix refers to the indicator matrix bond existences, regardless bond types; the normalization conducted the same as Á in GCNs (See Section 2

---

Published as a conference paper at ICLR 2021
3.3 DISCUSSION
From a physical perspective, the Hamiltonian Engine is essentially inspired by and highly related to Molecular Dynamics (MD). The potential energy function in the engine can be regarded as a generalized while simplified molecular force field. Force fields are widely used tools in molecular simulation, where potential energies are modeled as a family of functions of conformations, whose parameters are calculated with quantum chemistry or determined by experiments. After a force field is established, conformations are optimized by minimizing the potential energy. Similarly, the dissipation we introduced in the Hamiltonian Engine serves an implicit optimization of potential energy, as the system energy is continuously dissipated through during the dynamical process. As result, after adequate steps, the molecular conformations always converge the potential energy, and the momentums of atoms converge to O. From deep learning perspective, the Hamiltonian Engine can be seen pair of dual, residual MPNNs operating on fully-connected graphs: at each step, messages calculated by q influence p and vice versa. Under this view, the most essential difference of introducing physical laws that the messages passed from q to p symmetric between any two given atoms, following the Newton's 's third law and conforming a conservative field (the potential force field). Speaking more detailly, q-messages sent between a pair of atoms (i.e. the forces, 07//Qq) are implicitly guaranteed as symmetric, yet the actual p-update (i.e. the accelerations, p/m) may not be: they are also related to properties of the receiver (the atom mass) and the outer environment (the dissipation).
local minimum of
RESULTS
4.1 EXPERIMENTAL SETUP
Datasets. Five molecular datasets used to evaluate HamNet, including a Quantum Mechan- ics dataset (QM9) and four biomedical datasets, namely Tox21, Lipop, FreeSolv and ESOL. QM9 (Ramakrishnan et al. 2014) contains calculated conformations and 12 quantitative quantum- chemical properties of 133k molecules; Tox21 contains 12 binary toxicological indices of 7, 831 molecules; Lipop contains quantitative liposolubility lipophilicity of 4, 200 molecules; FreeSolv contains quantitative hydration free energy of 642 molecules, and ESOL contains quantitative sol- ubility of 128 molecules. All datasets are referred in MoleculeNet, and the same metrics P data split ratios and multi-task scheme (for QM9 and Tox21) are used in our paper.
Featurization and Implementation. We use the identical featurization as Attentive FP (Xiong al. 2020). In total, 39-dimensiinal atom features (including atom types, atom degree, indicators of aromaticity and chirality et al) and 10-dimensional bond features (including bond types, indicator of conjugation et al) are derived from the molecules. One could refer to the Appendix for more details of featurization. As a default setup, we use a 20-step (T = 20) Hamiltonian Engine with d, = 32, and L 2, MM = 2 with 200-dimensional hidden representations (dim(h.) dim(h, 200) in the Fingerprint Generator. For the training of HamNet, we first train the Hamiltonian Engine with known conformations and use the output train the Fingerprint Generator, with mean-squared- error losses for regression tasks with RMSE metric, mear-absolute-eree losses for those with MAE metric, and cross-entropy losses for classification tasks, Other implementation details, including the choices of hyperparameters on different datasets and the training setup are available in the Appendix.
f
4.2 CONFORMATION PREDICTION
We evaluate the ability of the Hamiltonian Engine in predicting molecular conformations on QM9, where known conformations of molecules are available, and reported the Kabsch-RMSDD Lk-rmsd and distance loss Laist losses. Two baselines are compared against the Hamiltonian Engine (Ham. Eng.): i) an MPNN with the exact architecture proposed in (Gilmer et al. 2017), supervised in the
We use MAE for QM9, ROC for 1, and RMSE for Li pop, FreeSolv & ESOL.
3 Data are randomly split to 8: as training, validation and test sets.
Models are trained simultancously preserve all targets after standard normalization, and averaged per- formances are reported.
5On datasets without known conformations, generate labeled conformations with RDKit to train the engine

---

Published as a conference paper at ICLR 2021
Table 1: Quantitative results of conformation prediction on QM9.
METRIC
Kabsch-RMSD) (A)
Distance Loss (10-?A)
MPNN
1.708
8.620
RDKit
1.649
7.519
Ham. Eng- (wo LSTM)
2.039
10.871
Ham. Eng. (w/o dyn.)
1.442
5.519
Ham. Eng. (w/o
1.389
5,227
Ham. Eng. (w/o ADJ-3)
1.084
7,746
Ham. Eng. (as proposed)
1.384
5.186
(h)
(c)
Real Conf.
Step 0
Step 3
Step
Step 9
Step 12
Step 15
Step 20
Figure 2: Visualized conformations different steps of the Hamiltonian Engine.
same way we have introduced in Section 3.1 ii) a Distance Geometry (Blaney & Dixon 2007)
method tuned with the Universal Force Field (UFF), implemented the RDKit package (referred as RDKi) 6 We also conduct an ablation analysis of the Hamiltonian Engine by testing: i) engine with the LSTM removed (w/o LSTM); ii) an engine with no Hamiltonian dynamics, i.e. T = 0 (w/o dyn.); iii) an engine without the dissipation function (w/o Φ); and iv) an engine trained without the ADJ-3 loss (w/o ADJ-3).
Table shows the two losses the test sets. Our approach outperforms the Distance Geometry baseline (RDKit) by 16%, while the MPNN cannot. the ablation analysis, effectiveness of differ- components also varified: improvements both metries observed when LSTM, molecular dynamics. and dissipation function exist. Although training the Hamiltonian Engine simply with Kabsch-RMSDD leads to better performances the very metric, distance losses of these models unacceptably large. This indicates although atoms tend to appear closer to their labeled locations, the relative structures inside the molecules are compromised, which a particularly undesired result in molecular science. One could refer to Figure 2 for more intuitive understanding of the dynam- ics in the Hamiltonian Engine: atoms in the initial conformations (Q()) tend to gather around the molecular centers, and the repulsion forces derived from the potential energy stretch the molecules into the correct conformations (Q "). As dissipation exists in the system, conformations converge to the real ones after adequate steps.
4.3 MOLECULAR PROPERTY PREDICTION
We compare HamNet with five baselines on the molecular property prediction tasks. Molecu- leNer (Wu et al.] 2017) tested collection of molecular representation approaches the time, by which we present the best performances achieved. ii) 3DGCN (Cho & Choil (2019) augmented conventional GCN-based methods with input bond directions. iii) DimeNer (Klicpera et al., (020) proposed directional message passing where messages instead of atoms are embedded. iv) Atten- MPNN instance where local and global attentive layers
live FP (Xiong al. 2020) proposed are used. v) CMPNN Song et al. 2020) strengthened the message interactions between nodes and edges through communicative kernel. Two HamNet variants also tested: i) HamNet without known conformation, where all g, p-related components in the Fingerprint Generator are removed;
WW the 2020.03.1.0 version of the RDKit package. See http:htt://wet rok orrg

---

Published as a conference paper at ICLR 2021
Table 2: Quantitative results on various datasets of baselines, HamNet, and its variants. Baselines
using 3D conformations of test molecules are marked italic. For different metrics, "1" indicates that the higher is better, "L" contrarily. We directly take reported performances from corresponding references, and leave unreported entries blank ("-").
DATASET
QM9
Tox21 Multi-ROCT
Lipop RMSEL
FreeSolv RMSEL
ESOL RMSE,
METRIC
Multi-MAEL
MoleculeNet (2017)
2.350
0.829
0.655
1.150
0.580
3DGCN (2019)
0.82410.014
0.558110.069
DimeNet (2020)
1.920
Attentive FP (2020)
1.292
0.857
0.578
0.736
0.505
CMPNN (2020)
0.856 0.006
0.808
0.547=0.011 1
HamNet (w/o conf.)
1.237=0.030
0,868=0.012
0.572=0.011
0,840#0.025
0.547:+0.015
HamNet (real conf.)
1.199=00.017
0.864100006
0.566=0.015
0.811
0.584:0.012
HamNet (ours)
1.194:10.08
0,875 0.006
0.557=0.014
0.731
0.504::00016
per Mi
per
Dearea Lowe
S-
-00
(a) Engine depth (T').
(b) Dimensionality of q. (dy).
(c) Step size (1)).
Figure 3: Effects on conformation prediction of hyperparameters in the Hamiltonian Engine. Dis- tance losses and running time versus (a) the engine depth T; (b) the dimensionality of the general- ized space dy; and (c) the step size of discretization are plotted.
ii a HamNet with real conformations, where the Hamiltonian Engine is removed and all q, ps transformed from real coordinations, with MLPs, to the corresponding dimensionality. Five replicas of HamNet models are trained with means and standard deviations reported.
sahoes the quantitative results of the fingerprints. As HamNet and all chosen baselines follow the same evaluation scheme proposed in MoleculeNet, we directly present the reported performances and leave unreported ones blank. We also italicize baselines which leverage true 3D conformations of the test set. HamNet is able to significantly outperform all baselines all datasets. Moreover, compared with the HamNet variant using real conformations, HamNet with the Hamiltonian En- gine still performs better. We believe the reason is that as the Hamiltonian Engine trained with translation- & rotation- invariant losses, the generalized space enjoys more robustness compared with real coordinations of atoms used directly.
4.4 PARAMETER ANALYSIS
We conduct further analysis of the effects of several important hyperparameters on the conformation prediction performances in the Hamiltonian Engine, including the engine depth (I.), the dimension- ality of the generalized space (dy). and the step size (1). Figure demoontrate the distance loss and / or the running time. i) As the depth increases, the running time increases linearly, and the test loss gradually decreases until convergence. Empirically, 25-30 steps suffice. ii) The running time of the engine hardly influenced by the dimensionality dy. while the performance enjoys significant improvement by increasing the dimensionality when d, < 32. iii) An appropriate choice of the step size crucial. For example, with the fixed engine depth 20, an ideal choice of the step size would be 1) E [0.025, 0.050).
8

---

Published as a conference paper at ICLR 2021
5 DISCUSSION & FUTURE WORK
In this paper, we proposed a novel molecular representation approach, Molecular Hamiltonian Net- work (HamNet). Instead of directly using conformations as inputs, HamNet learns to predict real conformations with a physically inspired module, the Hamiltonian Engine. Novel loss functions with translational rotational invariances are proposed to train the engine. Final representations of molecules generated with Fingerprint Generator, whose architecture is based on MPNNs and considerately modified to incorporate generated implicit conformations. We discussed the rela- tionships between physics and deep learning inside the Hamiltonian Engine from both perspectives, and we believe that the physics-based model enjoys better interpretability than general MPNNs. We further demonstrated with our experiments that the proposed Hamiltonian Engine eligible to learn molecular conformations, and that HamNet achieves state-of-the-arl performances on molecular property prediction tasks in standard benchmark (MoleculeNet).
It should be noted that a recent trend in incorporating machine learning, especially deep learn- ing, into modeling molecular potentials emerged (Chmiela et al., 2017 2018 Zhang et al. 2018
Another related field of HamNet is neural physics engines Sranchez-Gonzale et al. 2018: 2019 Greydanus al. 2019), which learn conduct simulations that conform to physical laws. The de- sign of the tonian Engine in our paper is highly motivated by these works, while the ultimate goal of HamNet is to derive well-designed molecular representations, instead of to accurately model the molecular dynamics. Also, instead of using the structural optimization to derive stablized con- formations after a force-field (potentials in HamNet) is established, HamNet uses the Hamiltonian Engine to make the whole process differentiable.
For future work, a promising aspect would be to elaborate the learnable potential, kinetics and dis- sipation functions used in the Hamiltonian Engine. Work in using HamNet in more forward applications would also be useful, such as virtual screening, pootein-ligante binding prediction, etc. In addition, interesting attempt in further modifying HamNet would be change the current discretization approach of the Hamiltonian Equations to Neural ODEs (Chen et al. 20188] Sanchez- Gonzalez al. 2019), which may yield finer-grained simulation of the molecular dynamics.
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China (Grant No. 61876006). We would also like thank Dr. Chenbo Wang for his help in the physics theories of this paper.
REFERENCES
S. D. Axen, X. P. Huang, E. L. Cceres, L. Gendelev, B. L. Roth, and M. J. Keiser. A simple representation of fire-dimensionall molecular structure. Journal of medicinal chemistry, 60(17): 7393-7409, 2017.
Jeffrey M. Blaney and J. Scott Dixon. Distance Geometry in Molecular Modeling, pp. 299-335. John Wiley & Sons, Ltd, 2007.
Tian Qi Chen, Yulía Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018 (NeurIPS 2018), pp. 5573-6583, 2018
Stefan Chmiela, Alexandre Tkatchenko, Huziel E. Sauceda, Igor Poltavsky, Kristof T. Schütt, and Klaus-Robert Müller. Machine learning of accurate energy-conserving molecular force fields.
Science advances, 3(5), 2017.
Stefan Chmiela, Huziel E. Sauceda, Robur-Robert Miler, and Alexandre Tkatchenko. Towards exact molecular dynamies simulations with machine-learned foree fields. Nature communications, (1), 2018.
H. Cho and I. S. Choi. Enhanced deep-learnigg prediction of molecular properties via augmentation of bond topology. ChemMedChem, 14(17): 160--16990.0 2019.

---

Published as a conference paper at ICLR 2021
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülgehre. Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference Empirical Methods in Natural Language Processing (EMNLP 2014), pp. 1724-1734, 2014.
David Duvenaud, Dougal Maclaurin, Jorge Iparrauuurp,rrrguurrrre,e,, Rafael Hamezz-ommarrlli. Timo- thy Hirzel, Alán Aspuru-Guzik, and Ryan P. Adams. Convolutional networks on graphs for learn- ing molecular fingerprints. Advances Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015 (NeurIPS 2015), pp. 2224 2232, 2015.
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning (ICML 2017), pp. 1263-1272, 2017.
Samuel Greydanus. Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Ad- vances Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 (NeurIPS 2019), pp. 15353-15363, 2019.
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. In Proceedingy the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), pp. 2261-2269, 2017.
Wengong Jin, Regina Barzilay, and Tommi S. Jaakkola. Junction tree variational autoencoder for molecular graph generation. In Proceedings of the 35th International Conference on Machine Learning, (ICML 2018), pp. 2328-2337, 2337, 2018.
Wolfgang Kabsch. A solution of the best rotation relate two sets of vectors. Acta Crystallograph- ica A, 32:922-923, -2322292,,,11 1976.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In Proceedings of the 5th International Conference on Learning Representations (ICLR 2017), 2017.
Johannes Klicpera, Janek Groß, and Stephan Günnemann. Directional message passing for molec- ular graphs. In 8th International Conference on Learning Representations (ICLR 2020), 2020.
Lim, S. Ryu, K. Park, Y. J. Choe, J. Ham, and W. Y. Kim. Predicting drug-target interaction using a novel graph neural network with 3d drruture-embeddee graph representation. Journal of chemical information and modeling, 59(9):3981-3988 2019.
H. L. Morgan. The generation of unique machine description for chemical structures-a technique developed at chemical abstracts service. Journal of chemical documentation, 5(2):63-112, 1965.
Raghunathan Ramakrishnan, Pavlo O. Dral, Matthias Rupp, and O. Anatole Lilienfeld. Quan- tum chemistry structures and properties of 134 kilo molecules. Scientific Data, 1(140022), 2014.
D. Rogers and M. Hahn. Extended-coonectivtty fingerprints. Journal of chemical information and modeling, 50(5):742-754, 2010.
Alvaro Sanchez-Gonzzlezz, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin A. Ried- miller, Raia Hadsell, and Peter W. Battaglia. Graph networks as learnable physics engines for inference and control. In Proceedings of the 35th International Conference on Machine Learning (ICML 2018), pp. 4467-4476, 2018.
Alvaro
Victor Bapst, Kyle Cranmer, and Peter W. Battaglia. Hamiltonian graph networks with ODE integrators. CoRR, abs/1909, 12790, 2019.
Michael Sejr Schlichtkrull, Thomas N. Kipf. Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In Proceedings of the 15th Extended Semantic Web Conference (ESWC 2018), pp. 593-607, 2018.
10

---

Published as a conference paper at ICLR 2021
Ying Song, Shuangjia Zheng. Zhangming Niu, Zhang-Hup Hua Fu, Yutong Lu, and Yuedong Yang. Communicative representation learning on attributed molecular graphs. In Proceedings of the TwentyNinth International Joint Conference Artificial Intelligence, (IJCAI 2020), pp. 2831- 2838, 2020.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In Proceedings of the 6th International Conference on Learn- ing Representations (ICLR 2018), 2018.
Y. Wang. J. Hu, J. Lai, Y. Li, H. Jin, L. Zhang, L. R. Zhang, and Z. M. Liu. Tí3p: Three-dimensional force fields fingerprint learned by deep capsular network. Journal of chemical information and modeling, 6066):2754-27666 2020.
D. Weininger. Smiles 1. introduction and encoding rules. Journal of chemical information and computer sciences, 28(31), 1988.
Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V. Pande. Moleculenet: benchmark for molecular machine learning. Chemical science, 12::513-330, 2017.
Z. Xiong, D. Wang, X. Liu, F. Zhong, X. Wan, X. Li, Z. Li, X. Luo, K. Chen, H. Jiang, and M. Zheng. Pushing the boundaries molecular representation for drug discovery with the graph attention mechanism. Journal of medicinal chemistry, 63(16):8749 8760, 2020.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay S. Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in Neural Informa- tion Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018
(NeurIPS 2018), pp. 6412-6422, 2018
Linfeng Zhang, Jiequn Han, Han Wang, Roberto Car, and E. Weinan. Deep potential molecular dynamics: scalable model with the accuracy quantum mechanics. Physical review letters, 120(14), 2018.
11


Reviews:
[{"review_id": "WFogNcgqoMC", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**\nThe paper proposes a new method for generating fingerprints for small molecules. It is based on two components: a \"Hamiltonian Engine\" that runs a brief simulation, predicting the structure of the small molecule by minimizing a learned potential energy, and 2) a message passing algorithm that uses the predicted structure as input. The reported experimental results demonstrate state-of-the-art performance.\n\n**Strengths**\n o The manuscript addresses an important problem\n o The Hamiltonian Engine is a non-trivial contribution, which should be of broad interest.\n o The reported results seem to significantly outperform the current state of the art\n\n**Weaknesses**\nThe paper is information-dense and in some places difficult to read. \n\n**Recommendation**\nI recommend this paper be accepted. To my knowledge, the presented method is novel, and the results are convincing. Furthermore, it is an interesting contribution on the interface between physics and machine learning, which should be of interest even beyond the small-molecule ML subfield. The manuscript would however benefit from a slightly clearer presentation (see below).\n\n**Detailed feedback**\n\nAlthough I'm generally positive about the article, there are several places where I would recommend sentences could be improved to more clearly communicate the central ideas in the paper:\n\nPage 1. The last paragraph on page 1 should make it clearer what the role of the Hamiltonian Engine is. In my opinion, the choice of the word \"preserve\", is particularly confusing - it seems to me more like a \"prediction\" or a \"reconstruction\". The work \"preserve\" made me think that the positions from the original molecule were used to initialize the positions and momenta of the simulator (\"preserve\" thus meaning the ability of the forcefield to stabilize a given structure). Figure 2 cleared this up for me, but that does not appear until page 7. Also, the authors write that HamNet \"simulates the process of molecular dynamics\", but it is not clear why  a Hamiltonian Engine is needed at all - couldn't you simply have used a short MD simulation with an established molecular force field? \n\nFigure 1. Could anything be done visually to show how the Fingerprint Generator uses the output from the Hamiltonian Engine? Currently, the two methodological components of the paper seem a bit disconnected in the figure.\n\nPage 4. \"Graph-based neural networks are used to initialize these spatial quantities.\" It would be informative if you could add a few lines to motivate this choice. Couldn't you have used some idealized positions as a starting point for your simulation? Later on in the paper you mention that you use RDKit to create positions when they are not available in the data - couldn't you then just always use RDKit to create an initial structure - or would this be less accurate or too slow?\n\nPage 5. \"conformations always converge to a local minimum\" Perhaps you could comment on the nature of the learned energy landscape.  Do we expect it to be a highly multimodal landscape, with risk for convergence to suboptimal local optima? Is this why you place so much emphasis on the initialization of your simulations?\n\n**Clarifying questions to the authors**\nPage 4. \"we simplify the Lennard Jones potential\"\nWhy do you make the simplification to r^{-4} - r^{-2}? It seems to me that you could have calculated the standard r^{-12} - r{^-6} equally well from the r^2 that you have available.\n\nPage 4. \"we empirically normalize the relative atomic mass\".\nWhat is the reason for this normalization? It it just to improve the training behavior of the model? Is this a hyperparameter that needs to be optimized?\n\nFigure 2. Does the Step 0 correspond to an initialization produced by the GCN+LSTM? The positions look very compact - wouldn't you have expected them to look more realistic?\n\nPage 6. \"tuned with the Universal Force Field\" Could you have done a distance geometry initialization but using your learned forcefield for subsequent fine-tuning?. Would this be superior to using the Universal Force Field?\n\nPage 7, \"where the Hamiltonian Engine is removed and all q,ps are transformed from real coordinations\"\nIf you are only given structures, how to you obtain the momenta?\n\nPage 8. I found it interesting that it was necessary to map the positions and momenta to a higher dimensional space for optimal performance. Do you know if this overparamterization is necessary purely to facilitate gradient-based training, or whether there are more fundamental reasons for modelling them this way?\n\n**Minor comments**\nPage 1. \"For examples, Attentive Fingerprints have yet become the de facto state-of-the-art approach\".\nTo a reader not intimitely familiar with these references it is not clear what you mean by this sentence. Is Attentive FP a 2D method? I think there is also a \"to\" missing before \"become\".\n\nPage 4. \"but atoms with coincide positions\". \"coincide\" -> \"coinciding\"\n\nPage 4, \"we do not explicitly require \\hat Q to conform to labeled coordinations of atoms\". This was a bit difficult to read. I assume you mean that you only predict Q up to a rotation and a translation. Perhaps rephrase.\n\nPage 5. \"Speaking for detailly\". Rephrase.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "H1-zhj8otJi", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to use 3d conformations for learning molecular fingerprints by 1) training a generative model to predict the 3d coordinates and 2) use those to train a \"fingerprint generator\" to obtain fingerprints by learning to predict molecular properties. The paper is well structured and clearly written. The intuition behind the Hamiltonian engine is nicely explained in the discussion section. The idea of a two step procedure to include 3d structure information in the training even when it is not available at test time is interesting and original. \n\nUnfortunately, some of the design decisions did not became quite clear to me: Why is the relaxation modeled with an MD-like dynamics approach instead of structure optimization, i.e. following the gradient of the energy? Although, here is an ablation study for leaving out the dynamics that performs worse, this might rather indicate that the initial guess of the positions is not good enough to find the correct equilibrium structure. Given the construction of the network that generates the initial positions, this seems quite likely, since it uses some canonical ordering (no permutation invariance) and does not include some notion of rotational symmetry. In contrast, there is previous work that demonstrates that this is possible with autoregressive models (Mansimov et al, Sci Rep, 2019; Gebauer et al, NeurIPS 2019). These methods also achieve much lower RMSDs than the presented approach on QM9 data (both <0.5 A RMSD). Another baseline would be to find a good positional guess, e.g. with RDkit, and then optimize with an ML force field that has already been used for MD (Schuett et al, J Chem Phys, 2019) or relaxation in crystal structure prediction (Podryabinkin, Phys. Rev. B, 2019). \nMoreover, since the datasets without 3d positions are labeled with RDkit (which i guess uses the same method as in the RDKit baseline of Table 1?), it would be interesting to see whether these structures would also be sufficient for the property prediction of QM9 in Tab. 2. Currently, I am not convinced that the more accurate positions (w.r.t. DFT equilibrium) obtained by HamNet actually improve the prediction that much. In particular, since the improvement compared to prediction without conformations in Table 2 is not significant according to the reported error bars.\n\nPros\n------\n+ Interesting idea, clearly written paper\n+ Physically motivated approach\n+ Improvements against simple baselines and ablations\n\nCons\n-------\n- Missing of important baselines (see above)\n- Only \"multi-mae\" reported for QM9. This makes it hard to judge the quality of the prediction and compare to predictions of individual properties in many other papers (DimeNet, MPNN, SchNet, HIP-NN, DTNN, PhysNet, etc). Please list also individual metrics and compare to previous work.\n- Datasets without 3d positions are labeled with a computationally cheap method from RDKit.  This does not make much sense: Why then not directly use these positions?\n- Timings for the prediction should be added to judge computational cost compared to a fast force-field or semi-empirical method.\n\nUpdate:\nThe responses cleared up some aspects of the Hamiltonian engine and I adjusted my score accordingly. Still, additional baselines would improve the paper answer some open questions and validate some of the claims: Is the Ham. Eng. indeed faster than optimizing with a cheap force field? Is there then an advantage compared to featurization of the RDKit coordinates with one of the many MPNNs for 3d coordinates? This would be interesting to analyze.", "extra_scores": {}, "extra_text": {}}, {"review_id": "EDYZzkegBDH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper presents a novel neural network module called Hamiltonian Neural Networks to learn representation of molecules. The module consists of two main components: 1) Hamiltonian Engine (HE), and 2) Fingerprint Generator (FG). The HE 1) takes a molecular graph with atom and bond features as inputs, and first generate \"generalized\" positions p and momentums q via GNN+LSTM. These ps and qs are fed into a discrete Hamiltonian system with dissipation, and produces (generalized) \"conformations\" of molecules. The FG also takes a molecular graph with atom and bond features + the generalized positions and momentums from 1) as inputs to generate the final vector embedding of the input molecule. HE is trained to fit the input 3D conformation with a weighted combination of two types lossses (K-RMSD + 5 * ADJ-k loss). Posing this \"Hamiltonian\" inductive bias to the model, the paper demonstrated that the prediction performance over multiple molecular tasks from MoleculeNet are all improved. Also, the analysis of HE module or the hyperparamter sensitivity analysis are provided.\n\nComments:\nThis is a very interesting work in the light of molecular representation learning. The 3D geometries of molecules, i.e. conformations, are actually not rigid-body-like, rather has a degree of freedom varied according to the physical rule as we see them in molecular dynamics. It would be reasonable to see that this inductive bias explitly as \"Hamiltonian system with learnable parameters\" actually brings empirical performance improvements. \n\nI enjoyed the paper and core ideas, but the description and presentation of the (complicated) method's inner workings would need to be improved. In particular, it'll be nice to make clear the following points:\n\n- There are no description on the classifier or regressor head for downstream tasks of MoleculeNet, and how to train the entire network. This should be explicitly included in the paper. The paper only provides how to get the fingerprint, but seemingly there are no explanations or description on how to train this FG (or the entire module of HE+FG). We'll need to add some task-specifc heads after HE -> FG for MoleculeNet tasks. Then, is HE first trained with K-RMSD + 5 * ADJ-k loss, and the entire HE + FG fitted with the task-specific supervision loss? Or the entire HE + FG fitted with K-RMSD + 5 * ADJ-k loss + alpha * task-specific supervision loss (with some alpha)?\n\n- The model desciprion of Figure 1 is unclear and rough, and it was quite painstaking to get the information on what are learnable parameters and what are just states or intermediates. Also, the p and q from HE to FG would be also hard to capture at first glance. I understand that the module is quite complicated and not easy to depict it in one figure, but I'll appreciate if the authors can include all learnable parameters in Figure 1. (For example, like the model figure, Fig.4, of the DimeNet paper).\n\n- Related to the above problem, some \"ablation study\" would be quite unclear. For example, \"HamNet (real conf.)\" in Table 2 is to tweak HE somehow, or just feed the real 3D conformations into FG with momentums of 0s (with removing the entire HE processing)?? In the first place, the HE used 32-dimensional vector for ps and qs. Is it fair if this means to use 3-dimensional vector instead (at least, need to pass it to MLP for converting the same dimension?). What if we use 3-dimension for d_f??\n\n- \"HamNet (w/o conf.)\", \"Ham. Eng. (w/o LSTM)\", \"Ham. Eng. (w/o dyn.)\" are also quite unclear.\n\n- It is quite interesting to see \"HamNet (ours)\" was better than \"HamNet (real conf)\" even for QM9, and moreover, given that for most tasks, the 3D input conformation is generated by RDKit distance geometry with UFF. But the internal conformation of HamNet is actually \"d_f\"-dimensional (32-dimensional), and this difference could come from this expressiveness. To confirm this, it would be nice to include the performance of \"HamNet (ours) with d_f = 3\". (By the way, since the 2018.09 release of the RDKit, ETKDG is the default conformation generation method. What if we use this instead of the distance geometry...?)\n\n- It would be very nice to add some remarks on related work for readers. The name and method would definitely remind related work such as \"Hamiltonian Neural Networks (Greydanus et al, NeurIPS 2019)\" , Machine-learning potential for MD (see [a][b] below, for example), and and \"learning to simulate\" methods. See the “delta graph network” (DeltaGN) baseline of the cited workshop paper by Sanchez-Gonzalez et al, 2019. \n\n[a] Chmiela et al, Machine learning of accurate energy-conserving molecular force fields (Science Advances, 2017)\n[b] Chmiela et al, Towards exact molecular dynamics simulations with machine-learned force fields (Nature Communications, 2018)\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes to lear molecular descriptors that account for the 3D structure of molecules. This is done by using first a "Hamiltonian Engine" that runs a brief simulation, predicting the structure of the small molecule by minimizing a learned potential energy, and second, a message passing algorithm that uses the predicted structure as input. The reported experimental results show state-of-the-art performance.

Strengths:

1 - Relevant contribution through the Hamiltonian Engine.

2 - Strong empirical results.

Weaknesses:

3 - Some reviewers mentioned that the readability of the paper could be improved.

I recommend the authors to also take into account the concerns of AnonReviewer1 to
improve the paper.
Decision: Accept (Poster)