GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensembles
Authors: Octavian-Eugen Ganea
Lagnajit Pattanaik
Connor W. Coley
Regina Barzilay
Klavs Jensen
William Green
Tommi S. Jaakkola
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: molecular conformer generation
molecular geometry
molecular generative models
message passing neural networks
3D generation
molecular conformations

Abstract: Prediction of a molecule’s 3D conformer ensemble from the molecular graph holds a key role in areas of cheminformatics and drug discovery. Existing generative models have several drawbacks including lack of modeling important molecular geometry elements (e.g., torsion angles), separate optimization stages prone to error accumulation, and the need for structure fine-tuning based on approximate classical force-fields or computationally expensive methods. We propose GEOMOL --- an end-to-end, non-autoregressive, and SE(3)-invariant machine learning approach to generate distributions of low-energy molecular 3D conformers. Leveraging the power of message passing neural networks (MPNNs) to capture local and global graph information, we predict local atomic 3D structures and torsion angles, avoid- ing unnecessary over-parameterization of the geometric degrees of freedom (e.g., one angle per non-terminal bond). Such local predictions suffice both for both the training loss computation and for the full deterministic conformer assembly (at test time). We devise a non-adversarial optimal transport based loss function to promote diverse conformer generation. GEOMOL predominantly outperforms popular open-source, commercial, or state-of-the-art machine learning (ML) models, while achieving significant speed-ups. We expect such differentiable 3D structure generators to significantly impact molecular modeling and related applications.

GEOMOL: Torsional Geometric Generation of Molecular 3D Conformer Ensembles
Octavian-Eugen Ganea +,
Lagnajit Pattanaik +
Connor W. Coley?
Regina Barzilay
Klavs F. Jensen'
William Green
Tommi S. Jaakkola $
Abstract
Prediction of a molecule's 3D conformer ensemble from the molecular graph holds key role areas of cheminformaties and drug discovery. Existing generative models have several drawbacks including lack of modeling important molecular geometry elements (e.g., torsion angles), separate optimization stages prone to error accumulation, and the need for structure fine-tuning based approximate classical force-fields or computationally expensive methods. We propose GEOMOL - an end-to-end, non-auuoreerrssivve, and SE(3)-invariant invariaan machine learning approach to generate distributions of low-energy molecular 3D conformers. Leveraging the power of message passing neural networks (MPNNs) capture local and global graph information, we predict local atomic 3D structures and torsion angles, avoid- ing unnecessary wver-parameterization of the geometric degrees of freedom (e.g-- one angle per non-terminal bond). Such local predictions suffice both for both the training loss computation and for the full deterministic conformer assembly (at test time). We devise non-adversarial optimal transport based loss function to promote diverse conformer generation. GEOMOL predominantly outperforms pop- ular open-source, commercial, or state-of-the-art machine learning (ML) models, while achieving significant speed-ups. We expect such differentiable 3D structure generators to significantly impact molecular modeling and related applications.
Overview
GEOMOL
3D conformer
Problem & importance. We tackle the problem of molecular conformer generation (MCG), i.c., predict- ing the ensemble low-energy 3D conformations of a small molecule solely based on the molecular graph (fig. M. single conformation represented by the list of 3D coordinates for each atom in the respective molecule. this work, we assume that the low-energy states are implicitly defined by the given dataset, i.e., our training data consist of molecular graphs and corresponding sets of energetically favor- able 3D conformations. Low-energy structures are the most stable configurations and, thus, expected to be observed most often experimentally.
molocular
graph
Figure 1: We generate representative set of low-energy 3D conformers from the input molecular graph. This example molecule has both rigid (rings) and flexible parts. Conform- shown aligned and juxtaposed.
"qqual first authorship contribution. Correspondence (oct, {oct,lagnaj Smit. edu. DDppartment of Chemical Engineering, MIT, Cambridge, MA 02139
I CCmmpterr Science and Artificial Intelligence Laboratory, MIT. Cambridge, 02139 *Code is available at https::///// / http///gttttt com com/Patttanaikk//eooMoo
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Dealing with molecules in their natural 3D structure is of great importance in areas such as chemin- formatics or computational drug discovery because conformations determine biological, chemical. and physical properties [Guimaraes et al.
2012
Schütt et al.) 2018 Klicpera et al., 2019 Axelrod and
Bombar-Bombarelij 2020b] [Schütt et al. 202T; Liu et al.; 2021 such as charge distribution, potential
energy. docking poses McGaan]] 2011], shape similarity |uumar and Zhang 2018]. pharmacophore
searching [Schwab] 2010]. or descriptors for 3/4D QSAR [Verma et al. 2010 For instance, in drug
design is crucial to understand how molecule binds to specific target protein; this process heavily depends on the 3D structures of the two components, both in terms of geometric (shape matching)
and chemical (hydrophobichhyddoohiiio interactions (Gaizaa et al. 2020 Sverrisson al. 2020
Motivation & challenges of existing methods. The main challenge in MCG comes from the enormous size of the 3D structure space consisting of bond lengths, bond angles, and torsion angles. It is known that the molecular graph imposes specific constraints on possible 3D conformations, e.g.. bond length ranges depend the respective bond types, while tetrahedral centers dictate local spatial arrangement. However, the space of possible conformations grows exponentially with the graph size and number of rotatable bonds, thus hindering exhaustive brute force exploration even for relatively small molecules. Additionally, the number of plausibly-stable low-energy states is unknown a priori and can vary between one and several thousand conformations for single molecule (Chan al.) 202T]. Nevertheless, various facets of the curse of dimensionality have been favorably tackled by ML models in different contexts, and our goal is to build on the recent ML efforts for MCG MMansimov Xu et al.) [202]].
et al. 2019 Simm and Hrrnanndzz-Lbbttee 2020 Lemm et al. 202 1
Molecular conformations can be determined experimentally, but existing techniques are very ex- pensive. As a consequence, predictive computational models have been developed over the past few decades, traditionally being categorized as either stochastic systematic (rule-based) meth- ods [Hawkins] [017]. Stochastic approaches have traditionally been based on molecular dynamics (MD) or Markov chain Monte Carlo (MCMC) techniques, potentially combined with genetic algo- rithms (GAs). They can do extensive explorations of the energy landscape and accurately sample equilibrium structures, but quickly become prohibitively slow for larger molecules (Shim and MacK- erell 2011 Ballard et al. 2015 De Vivo et al. 2016 Hawkins 2017). e.g., they require several CPU minutes for a single tike molecule. Moreover, stochastic methods have difficulties sam- pling diverse and representative conformers, prioritizing quantity over quality. On the other hand, rule-based systematic methods achieve state-of-the-art
commercial software [Friedrich et al. 2017] with OMEGA [Hawkins et al.) 2010 Hawkins and Nicholls 2012) being a popular example. They usually process a single drug-like like molecule under a second. They address the aforementioned challenges of stochastic methods by relying on carefully curated torsion templates (torsion rules), rule-based generators, and knowledge bases of rigid 3D fragments, which are assembled together and combined with subsequent stability score ranking. However, torsion angles are mostly varied independently (based on their fragments), without explicitly capturing their global interactions, which results in difficulties for larger and more flexible molecules. Furthermore, the curated fragments and rules are inadequate for more challenging inputs (c.g., transition states or open-shell molccules).
Both types of methods can be combined with Distance Geometry (DG) techniques to generate the initial 3D conformation. First, the 3D atom distance matrix is generated based on a set of distance constraints or from a specialized model. Subsequently, the corresponding 3D atom coordinates are learned to approximately match these predicted distances [Havel al. 1983b Crippen et al.) 1988 Havel 1998 Lagorce al. 2009 Riniker and Landrum [015]. Indeed, modern stochastic algorithms are entirely based on DG methods [Riniker and Landrum| 2013). The inductive bias of rotational and translational invariance is guaranteed for DG, thus being appealing for MI. models (Simm and Mernandez-Lobaate 2020 Xu et al. 2021 Pattanaik et al. 2020b]. However, several drawbacks weaken this important direction: i) the distance matrix is overparameterized compared to the actual number of degrees of freedom, ii) it is difficult to enforce 3D Euclidean distance constraints as well as geometric graph constraints (e.g.. torsion angles rings [Riniker and Landrum 2015); ); iii)
important aspects of molecular geometry are not explicitly modeled, e.g., torsion angles of rotatable bonds or tetrahedral centers; iv) expensive force-field energy fine-tuning of the generated conformers is vital for a reasonable quality [Xu et al. 2021 Simm and Hernandez- HernandezzLobato 2020]; iv) the resulting multi-stage pipeline is prone error accumulation as opposed to an end-to-end model.
Previous methods often rely on a force field (FF) energy function minimization to fine-tune the conformers. These are hand designed energy models which use parameters estimated from experi-

---

ment and/or computed from quantum mechanics (e.g.. Universal Force Field [Rappé et al., 1992]. Merck Molecular Force Field [algren 1996)). However, FFs are crude approximations of the true molecular potential energy surface Kanal et al 2018). limited in the interactions they can capture in biomolecules due to their strong assumptions [Barman al. 2015]. addition, FF energy optimization is relatively slow and increases error accumulation in multi-pipeline method.
Relation to protein folding. There has been impressive recent progress on modeling protein folding dynamics [Ingraham et al., 2018, AlQuraishi 2019 Noč al. 2019 Senior al., 2020], where crystallized 3D structures are predicted solely from the amino-acid sequence using ML methods. However, molecules pose unique challenges, being highly branched graphs containing cycles, different types of bonds, and chirality information. This makes protein folding approaches not readily transferable general molecular data.
Our key contributions & model in nutshell. In this work, we investigate the question:
Can we design a fast and generalizable deep learning model to predict high-qualite representative, and diverse 3D conformational ensembles from input molecular graphs?
To tackle this question, we propose GEOMOL (shown in fig. 2 exhibiting the following merits:
It is end-to-end trainable, non-autoregressive. and does not rely on DG techniques (thus avoiding aforementioned drawbacks). More precisely, it outputs minimal set of geometric quantities (i.e.. angles and distances) sufficient for full deterministic reconstruction of the 3D conformer.
models conformers in an SE(3)-invariant (translation/rotttiin)) manner by design. This desirable inductive bias was previously either achieved using multi-step DG methods (Simm and Hernandez Lobato 2020 or not captured at all [Mansimov et al., 2019
explicitly models and predicts essential molecular geometry elements: torsion angles and local 3D structures (bond distances and bond angles adjacent to each atom). Together with the input molecular graph, these are used for k-hop distance computation at train time and full deterministic conformation assembly at test time. Crucially, a single torsion angle is computed per each non-terminal bond, irrespective of the number and permutation of the neighboring atoms at each end-point of the respective bond.
do not perpparameterize these predictions, i.e.,
The above geometric elements (torsion angles, local structures) are SE(3) invarriat (by definition or usage) and we jointly predict them using MPNNs [Gilmer et al. 2017 and self-attention networks. Thus, unlike [Mansimov et al. 2019] we are not affected by MPNNs pitfalls that obstruct direct predictions of 3D atom coordinates from node embeddings, e.g., symmetric or locally isomorphic nodes would always have identical MPNN embeddings [Xu al. 2019) Garg al. 2020
and, as a consequence, would be inappropriately assigned identical 3D coordinates.
To promote diverse conformer ensembles with good coverage, we devise tailored generative loss that does not use slow or difficult-to-ppiinn -o-otmmiie adversarial training techniques. Using optimal transport, GEOMOL finds the best matching between generated and ground truth conformers based on their pairwise log-likelihood loss, requiring only minimization.
It explicitly and deterministically distinguishes reflected structures (enantiomers) by solving tetrahedral stereocenters using oriented volumes and local chiral descriptors, bypassing the need for iterative optimization usually done in DG approaches.
Empirically, we conduct experiments on two benchmarks: GEOM-QM9 (smaller molecules relevant gas-phase chemistry) and GEOM-DRUGS (drug-like molecules) [Axerrd and Gomez Bombarelli 2020a]. Our method often outperforms previous ML and two popular open-source or commercial methods in different metrics. Moreover, show competitive quality even without the frequently-used compurationally-demanddi fine-tuning FF strategies.
GEOMOL processes drug-like molecules seconds or less, being orders of magnitude faster than popular baselines (e.g., KTKG//DDKKK Riniker and Landrum 2015)), without sacrificing quality.
3

---

Repeat N times to genorate conformers
ple -oo----
sam
die
Full conformer assembly
Message Passing Neural Network
(EE) SE
local neighboring structurce. angles 3-hop
Atom embeddinng
Preditt 1-hop 3D cpordinates. bond dirancces and ang sg.
molecular graph
Tast time (ony, Involves: Detarministc ring corractios
Training loasc 1,A,S- distaazze loses band azigle sorstion bangle Joes
Figure 2: Overview of the GEOMOL model, which is SE(3)-invarinn by design. Given a molecular graph, first compute MPNNs atom embeddings. Next, we predict the local 3D structures (LS) of each non-terminal atom a permutation invariant way, explicitly solving chirality. Third, for each bond connecting non-terminal vertices, we assemble the two LS by predicting single torsion angle, avoiding overparameterization. Finally, the full conformer is assembled (only) at test time.
2 Method
Problem setup notations. Our input is any molecular graph G = (V, with node and edge features, X E R Ve E V and Cu E TR', Y(u, v) E E representing atom types, formal charges, bond types, etc. For each molecular graph G, have variable-size set of low-energy ground truth 3D conformers (CT), that we predict with a model {C*Ja def ((G). A conformer is a map C: - I from graph nodes to 3D coordinates, but a simplified notation is C, E R& for V. We use additional notations: d(X, Y) ||ex Cy is the 3D distance between X and Y; LXYZ the counter-clockw of the 2D planes Cy and cycy Cr (formula is in appendix C We use the corresponding XYT) when manipulating a ground truth conformer.
(CCW) angle ZCxcy C2: Z(XYZ, XYT) is the CCW dihedral angle
c,
Any conformer is defined up to SE(3) transformation, i.e., any translation or rotation applied to the set {c. hve. classic conformer distance function that satisfies this constraint is root-mean- square deviation of atomic positions (RMSD), computed by the Kabsch alignment algorithm Kabsch 1976
2.1 GEOMOL high-level overview
Our approach, shown in fig. 2 comprises three steps. First, we predict the local 3D structure of each non-terminal atom, which we deem local structure (LS), by combining self-attention layers and MPNNs with deterministic corrections for tetrahedral centers. Bond distances and bond angles are computed from the predicted LS. Next, we assemble all neighboring pairs of LSs by predicting the torsion angles and aligning them. Importantly, since L.Ss are fixed, it suffices to only predict single value for the dihedral angle of each bond. Towards this goal, we develop canonical representation of torsion angles via a local coordinate system defined SE(3)-equivariantly w.r.t. the full structure, which allows us to predict exactly the number of degrees of freedom. Finally, test time, assemble all predicted pairs of neighboring LSs to construct the full conformer, applying deterministic ring append random Gaussian noise vectors to
corrections. In order to generate diverse conformers, each initial node feature vector and use an optimal transport-bast based loss function for training.
2.2 Message passing neural networks (MPNNs)
Given input graph G, an MPNN Gilmer et al. 2017 Battaglia al.) 2018 Yang et al. 2019 computes node embeddings h, Rd, Vo using layers iterative message passing:
h ((+1)
(C()) Σ
@(h h(0), ee.e)) (...e..))) where h. del concat(xy, zu), Z. N/ sLa)
VEA.
for each f E _0.. T 1]. where N, fu Vi v) E E}. while & and are generic functions, e.g., implemented using multilayer perceptrons (MLP) or attention [Veličković al. 2017]. Final node embeddings are obtained by the embedding of the last layer: h, h (2)) Yu V. Finally,

---

also compute molecular embedding: hym! def M h. We leave comparison with other
MPNN variants for future work, e.g.. Kipt and Welling (017). Veličković et al. [0177] Hamilton
al. 2017 Xu al. 2019].
2.3 Local structure (1-hop) prediction model
Following notations in fig. for each non-terminal graph vertex X E V having graph neighbors Nx = [E...n we predict its local 3D structure (LS), i.e, the relative 3D positions of all Ti when X is centered in the origin. The generic model a function f(hr, hy,, :x)) = (P1..... PP,) E _3*= that, additionally, should satisfy permutation equivariance w.r.t. T, s, namely, the 3D position of each neighbor T; should not change regardless of the ordering of the X 'ss neighbors:
f (bTTan) bT hx) = (Pa(1).-. Pa(n)); Vx Sn
(2)
Our choice is the encoder part of transformer [Vaswani et al.) 2017]. without any positional encoding, thus satisfying per- mutation equivariance. This model takes input the set {concat [hr, x E [I..n]] in any order and synchronously updates the n embeddings based on several transformer layers. The final layer projects the embeddings to 3 dimensions, result- ing list (P1.... .P.) E R3x" having the exact same node order as the input list.
X
For each non-terminal atom X, we predict the relative 3D position of each of its graph neigh- bors. {Te (t in permutation equivariant manner.
Figure 3:
Enforcing local consistency. We desire the LS model f() to be distance-consistens i... any bond distance d(X, Y) is
the same, no matter it is computed from the I.S of node or of node Y. To achieve this, we use the above transformer just to compute bond directions (which will be aligned using a separate approach described in section 2.4. while we obtain the bond distances with separate symmetric model. Concretely,
let the above transformer s') predict (PI....,,.. Pn) E Rax", while the final local 3D coordinates ||pil Ip dGNN (hx, hr;), Vi, where each bond distance is predicted with symmetric model N (hx, hy) del = softplus(1 (hx, hy) + (by, hx)), V(X,Y) E E, with the same shared vi (e.g.. an MLP). For notation simplicity, will just use P. instead of p/--
P def
Regarding SE(3) invariance. The above model is not SE(3)-invariant per se, but it used as such. Namely, on one hand we compute SE(3)-invariant quantities: 1- hop distances d(T,, X), 2-hop distances d(T),T)). and bending angles ZT.XT>. These their ground-truth counterparts in the final loss, see section 2.5
will be compared
On the other hand, the LS of adjacent graph nodes are assem- bled together for computing torsion angles or for building the full conformer at test time. This process explicitly defined to be SE(3)-invariant described in section 2.4
3D
H
Tetrahedral chiral corrections. When embedding the local neighborhood of a node in 3D space, has to carefully ac- count for tetrahedral stereocenters (fig. g Tetrahedral chirality common form of stereochemistry which restricts the 3D location of neighboring substituents of a central atom with four distinct neighbors; molecules which differ by single tetrahedral stereocenter, i.e., enantiomers, are mirror images of each other. Chirality heavily impacts some properties of small molecules-e.2 bioactivity. Existing MPNNs using only the molecular graph cannot distinguish chiral centers (fig. L.. but solutions exist [Pattanaik et al., 2020a]. Mathematically, enantiomers can be differentiated based on the oriented volume around the tetrahedral center. That is, given the ordered set of neighbor 3D coordinates around the center, namely P1: P2, P3, PA E R3, the
Figure 4: Chirality: even if the two shown graphs are isomorphic, they have distinct 3D structures that can be distinguished by the order of the carbon center's neighbors.
5

---

sign of the volume of the tetrahedron formed by the neighbors is
1
1
1
(P P2, P4)
det
sign
E1
1/2
D3
F4
1/
1/2
13
y4
21
23
Enantiomeric structures always have opposite signs for the oriented volume (Crippen et al. 1988 Since we generate local 3D structures directly, we can also use local 3D chiral descriptors to ensure the correct generation of tetrahedral stereocenters. RDKit internally keeps track of these local chiral labels, denoted by CW/CCW labels (detailed in e.g.. Pattanaik et al. (2020a]). Importantly, each local chiral label corresponds to certain oriented volume (CW +1 and CCW =1). Thus, when generating an LS for a tetrahedral center, we calculate the oriented volume and check against the internal RDKit label. If it results in the incorrect oriented volume (i.e., the incorrect stereocenter was generated), we simply reflect the structure by flipping against the z-axis. This ensures that all tetrahedral stereocenters centers are generated exactly, and no iterative optimization necessary as with traditional DG-based generators.
2.4 Torsion angle representation and local structure (LS) assembly
Once the LS of each atom/vertex is predicted, we assemble them in pairs corresponding to each non-terminal bond the molecular graph. We describe this process for a bond connecting atoms and Y, each having additional graph neighbors (Ti) and, resp., {2, jE[L..m] See fig. 5
Torsion angle over- paramttrrmmtiiioonnnn We first note that, for any assembled bond XY (fig right), and Vi, & E [1..n], Vj. E [1..m]. the dihedral angles Z(XYT, XYTT) and Z(XY XY XYZ,) are fully determined by the LS of nodes X and Y. respectively. they do not depend on the torsion angle of bond XY. Next, observe that there is exactly one torsion angle for any bond XY, given unique indexing of the neighbors. This happens because of the following constraint:
(XYT, XYZ, = (XYT,,
XYT)) + +(Y XY )) (mod 2x) (3)
Thus, in order to avoid unnecessary wver-prrameterization, we predict single torsion angle a per each bond XY connecting non-terminal atoms.
Torsion angle formulation. However, it is still unclear at this point how to define this unique angle in a canonical way that is: i) permutation invariant w.r.t. the nodes in the set (T; (ej)..)) and, respectively, in the set (Zy (Zj)eet..mm ii) SE(3)-invariant w.r.t. the full 3D conformer, and iil) agrees with eq.
hy.
by hes (FDD)
Tonsion angle
See.
Neural lire
def cos Aij) sin Aij)
T
Z
Let A
del XYZ,) and Sij
Let
7
Cyj IR be real coefficients such that Σi, CtySit E R2 not the null vector. Then, we define the torsion angle a def #atan2 [|] It is easy
that this formulation satisfies both invariances claimed above. We further state (and prove in appendix A that our proposed formulation gives torsion an- gle uniquely determined by all local angles Z(Y XT,, Y XT,), X X Zi) and by the true underlying torsion angle:
Figure 5: Assembly of the local
structures of bonded atoms X and Y based on the predicted torsion angle.
T,, 7, and fixed weights Cij E TR. such that Proposition 1. Given 3D coordinates of nodes X, K.Y.T.,
Ens CijSij E IR2 is not the null vector, then o = alam2( ( is unique, ie.,, if we change the torsion angle of bond XY, then will change. Formally, if we rotate the set of bonds (XT,), jointly around the line XY with the same angle 7, then a will be exactly shifted with 7/-
'e define atan2 slightly different than standard: atan2(r cos(a), sin(a)) del Va E [0, 2m), Ri.

---

How to set Cty? Breaking symmetries. A simple solution is to choose Cij = 1, Vi,j. However, in some important cases, local symmetries may result in O. For example, this happens if, for some j. we have A. 2im + cl., vi E [1..n]. One solution is to use different Cij to differentiate between the different subgraphs rooted at different T; (and similarly for Zz). This is reminiscent of traditional group priorities used for distinguishing E/Z. isomers. We devise flexible solution to distinguish these subgraphs: a differentiable real valued function computed from the MPNN node embeddings cij M LP(TT + hz,) R. with MLP being neural network shared across all bonds and molecules. Note that we constrain Cij = Cjis thus guaranteeing that the same a is obtained if we swap X and Y (and their neighbors, respectively).
Final LS assembly for single bond. We now describe the assembly process depicted in fig. 5 We
first predict the LS of node X as in section 2.3 obtaining several 3D coordinates: Px 0, py: P7;
RR, Vi E [1..n], as well as the LS of node Y: qy = q.x q2, E R vjj E [1..m]. By design, we
have that 4 I| = ||py|- These two sets are eurrently not aligned. To achieve this, we first rotate the LS of X such that py becomes [||py ||
0] while Px remains 0. Next, we rotate and translate the I.S of Y such that qy becomes py and qx becomes px = O. These two rotations have one degree of freedom each, which we set randomly. Exact formulas are in appendix B Thus, the bond XY now matched, but the torsional rotation is still arbitrary/random. The remaining step to rotate the LS of X with an angle y such that all dihedrals Z(XYT;, XY ZY) match their true counterparts. This
0
done by applying to all vectors Pr, the same rotation of type: H
o
cos(7) sin(7)
-sin(7)
cos(7)
How to compute ?? The current dihedrals Agr def Zeur (XYT,, XYZ)) depend on the random
torsional rotations from the initial assembly step of LS of X and of Y. After applying the H, rotation, we obtain the new dihedral angles:
[A - 7] mod 2T that should match the ground
truth dihedral angles A (XYX,; Xy This is equivalently written as S = A Sy.
where S.y
cos( (7) sin (7) and A our def [cos( A (er
(T) Lets Σis Cayst and A cur def - coss .j
sin cur)
sin
A cur 6)
Eig Cij of proposition
cur
The necessary condition for y becomes Sy (Arrr) T s*, which is also sufficient due T| and, in practice, we do
This implies it is enough predict only the normalized -
(oo((a) sin(a) using a function commutative in X and Y (i.e., swapping X and Y
that by predicting S.. does not change a)):
= [o(bx, by, hmot) o(hy, hx, hmot) ) |mod 2T
(4)
[cos(7) (Acur) soll (A our)
where ø is neural network (c.g., MLP). Finally, s., sin(?)
Sa.
2.5 An optimal transport (OT) loss function for diverse conformer generation
Loss per single conformer. Assume first that we predict single conformer C. Based on all LS and torsion angle predictions, we deterministically compute all 1/2/3-hop distances and bond/torsion angles. If the corresponding ground truth conformer C* is known, we feed those quantities into a negative log-likelihood loss, denote by C(C, C*) and detailed in appendix D Similar to [enior et al. [020]. we fit distances using normal distributions and angles using von Mises distributions. This a much faster approach compared to habitual RMSD losses that compare full conformers.
Figure 6: Before (left) and af- ter (right) introducing match- ing loss to distinguish symmet- ric graph nodes. Hydrogen predictions in both groups are visibly improved.
Node symmetries. Our current formulation has difficulties dis- tinguishing pairs of symmetric graph nodes that are less than 3 hops away, e.g., hydrogen groups. We address this using a tailored matching loss detailed in appendix D and exemplified in fig. G
7

---

Table 1: Results the GEOM-DRUGS dataset. All models without FF fine-tuning. "R" and
"P" denote Recall and Precision. Note: OMEGA is an established commercial (C) software.
COV R (%) f
AMR
R (Å)
COV
P (%) f
AMR P (A) &
Models
Mean
Median
Mean
Median
Mean
Median
Mean
Median
GraphDG (ML)
10.37
0.00
1.950
1.933
3.98
0.00
2.420
2.420
CGCF (ML)
54.35
56.74
1.248
1.224
24.48
15.00
1.837
1.829
RDKit/ETTDD ETKDG
68.78
76.04
1.042
0.982
71.06
88.24
1.036
0.943
OMEGA (C)
81.64
97.25
0.851
0.771
77.18
96.15
0.951
0.854
GEOMOL (s = 9.5)
86.07
98.06
0.846
0.820
71.78
83.77
1.039
0.982
GEOMOL (s = 5)
82.43
95.10
0.862
0.837
78.52
94.40
0.933
0.856
Table 2: Results on the GEOM-QM9 dataset. See caption of table
COV
R (%) f
AMR
R (A) +
COV
P (%) t
AMR
P (Å) 4
Models
Mean
Median
Mean
Median
Mean
Median
Mean
Median
GraphDG (ML)
74.66
100.00
0.373
0.337
63.03
77.60
0.450
0.404
CGCF (ML)
69.47
96.15
0.425
0.374
38.20
33.33
0.711
0.695
RDKiL/ETKDG ETKDG
85.13
100.00
0.235
0,199
86.80
100.00
0.232
0.205
OMEGA (C)
85.51
100.00
0.177
0.126
82.86
100.00
0.224
0.186
GEOMOL (s 5)
91.52
100.00
0.225
0.193
86.71
100.00
0.270
0.241
Total OT loss per ensemble of conformers. In practice, our model generates a set of conform- ers [C(]RE[L. that needs to match variable sized set of low-energy ground truth conformers, {Ci he](t..A) However, we do not know a priori the number of true conformers or the matching between generated and true conformers. We also wish to avoid expensive and problematic adversarial training. Our solution is an OT-based, minimization-only loss function:
Censemble EMD (cTh)= = min Σ ITaE(CK, k.1
TEQx.1
where EMD is the Earth Mover Distance, T is the transport plan satisfying QK,L def {T E RKXL T1, k k T = 111). The minimization w.r.t. T is computed quickly using Earth Mover
Distance and the POT library FFlamary and Courty) 2017
2.6 Full conformer assembly at test time
Knowing all true LSs and torsion angles is, in theory, enough for a deterministic unique SE(3)- invariant reconstruction of the full conformer. However, in practice, these predictions might have small errors that accumulate, e.g., in rings. To mitigate this issue, we deterministically build the full conformer (only at test time) by first predicting smoothed structure of (fused) rings separately, and then assembling the full conformer following any graph traversal order (any order gives the same conformer, this procedure does not break the non-autoregressivee behavior). We detail this step in appendix B
3 Experiments
We empirically evaluate GEOMOL on the task of low-energy conformer ensemble generation for small and drug-like molecules. We largely follow the evaluation protocols of recent methods [Simm and Hernandez-Looaan
2020 Xu et al. 2021 but also introduce new useful metrics.
Datasets splits. We use two popular datasets: GEOM-QM9
RRamakrishnan et al. 2014 2020a Statistics and other details are in
and BEOM-DRUGS [Axerrdd and
fig. T0 and Mansimov et al. [2019]. Datasets are preprocessed as described in appendix G We split them randomly based on molecules into train/validation/test
(0%/10))) At the end, for each dataset, we sample 1000 random test molecules as the final test set. Thus, the splits contain
106586/13323/100 and 243473/30433/1000 molecules for GEOM-QME and GEOM-DRUGS, resp.
8

---

Reference
GraphDG
CGCF
ETKDG
OMEGA
GeaMol
100
STIDIO
Stecall coverage (h
00
40
20
Number of tolatable bonds
XT030
conformet
10-1
10
10
Number of rotatable bonds
Figure 7: Left: Examples of generated structures. For every model, we show the best generated conformer, i.c., with the smallest RMSD to the shown ground truth. More examples are appendix Right/top: Number of rotatable bonds per DRUGS test molecule versus COV Recall (95% confidence intervals). Right/bottom: conformer generation times for each model.
We compare established recent baselines (discussed in section ETKDG/RDKit 2015 likely the most popular open-source software, a stochastic DG-based method developed the RDKit package. OMEGA (Hawkins et 2010 Hawkins and Nicholls 2012 Friedrich et al.) 2017]]. rule-based method, is one of the most established commercial software, with more than decade of continuous development. OMEGA and ETKDG some of the fastest and best scaling existing approaches. Finally, we compare with the recent ML models of highest reported quality: GraphDG [Simm and Hernandez-Lobate 2020 and CGCF Xt 2021
Baselines.
[Riniker and Landrum
Evaluation metrics. We follow prior work [Simm and Hernandez-Lobato) 2020 Xu et al. 2021
and use noot-mean-squant deviation of atomic positions (RMSD) compare any two conformers. This is defined as the normalized Frobenius norm of the two corresponding matrices of 3D coordinates after being SB(3)-aligned priori (using the Kabsch alignment algorithm [Kabsch] 1976)). Next, we introduce four types of metrics to compare two conformer ensembles, generated by method, {C& bee(I...).. and ground truth, {Ci he(....)) These metries follow the established classification metrics of Precision and Recall and are defined for given threshold 6 >> 0 as:
COV R (Recall)
H E [I..L:: 3K E [L..K], RM SDMSD CT) 6
AMR R (Recall) def 1
Σ min RMSD(C..C)) kE.I..K]
(5)
L
[E
where AMR "Average Minimum RMSD", COV "Coverage", and COV (Precision) and AMR P (Precision) defined in eq. ( but with the generated and ground truth conformer sets swapped. The recall metrics measure how many the ground truth conformers are correctly predicted, while the precision metrics indicate how many generated structures of high quality. Specifically, in terms of recall, COV measures the percentage of correct generated conformers from the ground truth set (where correct conformer is defined
within RMSD threshold of the true conformer), while AMR measures the average RMSD of each generated conformer with its closest groun truth match. Depending on the application, either of the metrics might be of greater interest. We follow Xu al. 2021 and set 0.5À for GEOM-QM9 and 8 1. 1.25 for GEOM-DRUGS.
Training and test details. For each input molecule having K ground truth conformers, we generate exactly 2K conformers using any of the considered methods. For GEOMOL, this is done by sampling

---

different random noise vectors that are appended to node and edge features before the MPNN (eq. ! At train time, our model uses a standard deviation (std) s (see eq. 4 of for both GEOM-QM9 and GEOM-DRUGS, At test time, GEOMOL. can use the same or different values, depending on the downstream application, i.e., higher & results in more diverse conformers, while lower 8 gives more quality (better precision). For OMEGA, it is not possible to specify a desired number of conformers. So, we tune the RMSD threshold (which decides how many conformers to keep) such that the total generated conformers by OMEGA are approximately 2K. For GEOM-QM9, this corresponds to no RMSD cutoff (i.e., OMEGA generates all possible conformers), and for GEOM-DRUGS, this corresponds to a cutoff of 0.7À (meaning no two generated conformers will have a distance smaller than this cutoff). We discuss
and additional training details in appendix H
Results & discussion. Results are shown in table D and table 2 and confidence intervals are in appendix As noted above, GEOMOL can be run with different noise std at test time, depending on which metric the user is interested in. Even though OMEGA an established commercial software with more than decade of continuous development, our model remarkably frequently outperforms it. Note that OMEGA fails to generate any conformers for 7% of the QM9 test set (many of which include fused rings). Moreover, (except for AMR-P on QM9) and very recent ML models such as GraphDG and CGCF, sometimes by large margin. For qualitative insight, we show generated examples in fig. 7 and appendix N
also outperform the popular RDKiVETKKG open-source model
Additionally, we show in fig. 7 how COV Recall results are affected by the increasing number of rotatable bonds in the test molecule. As expected, having more rotatable bonds makes the problem harder, and this affects all baselines, but GEOMOL maintains reasonable coverage even for more difficult molecules. Moreover, in appendix K and table we show energy calculations of the generated conformers to support their plausibility. Additionally, results with energy-based relaxations are given in appendix g
Running time. Fig. 7 shows conformer generation test running times. Our model is the fastest method from the considered baselines, being much faster than CGCF or ETKDG/RDKit. Moreover, GEOMOL scales favorably for molecules with increasing number of rotatable bonds.
Conclusion
We proposed GEOMOL, an end-to-end generative approach for molecular 3D conformation ensembles that explicitly models various molecular geometric aspects such as torsion angles or chirality. We expect that such differentiable structure generators will significantly impact small molecule conformer generation along with many related applications (e.g., protein-ligane ligand binding), thus speeding up areas such as drug discovery. GEOMoL's full source code will be made publicly available.
Limitations & future work. A few current limitations are highlighted and left for future exten- sions (see also discussion in appendix D). First, our model does not currently support disconnected molecular graphs, e.g., ionic salts, but it can be applied to each connected component, followed by 3D alignment. Next, our approach would benefit from explicit modeling of long distance interactions, especially for macrocycles or large molecules. This remains to be addressed in an efficient manner. Third, explicitly using ground truth energy values could further improve GEOMOL. Last, we look forward to fine-tune GROMOL on applications such as generating molecular docking poses or descriptors for 4D QSAR.
Acknowledgments and Disclosure of Funding
OEG thanks Bracha Laufer, Tian Xie, Xiang Fu, Peter Mikhael, and the rest of the RB and TJ group members for their helpful comments and suggestions. LP thanks Camille Bilodeau and the rest of the WHG, KFJ, and CWC research groups for their useful discussions. We also thank Pat Walters, Simon Axelrod, and Rafael BomeazlBombarelii for their insightful feedback as well as Minkai Xu and Shitong Luo for helping run the CGCF model. Both OEG and LP are funded by the Machine Learning for Pharmaceutical Discovery and Synthesis consortium.
10

---

References
Mohammed AlQuraishi. End-to-end differentiable learning protein structure. Cell systems, 8(4):
292-301, 2019.
Simon Axelrod and Rafael Gomez-Bombardie Bombbreombb...... Geom: Energy-annotar molecular conformations for property prediction and molecular generation. arXiv preprint arXiv:2006.05531, 2020a.
Simon Axelrod and Rafael Gomez- Molecular machine learning with conformer ensem- bles. arXiv preprint arXiv:2012.08452 2020b.
Andrew Ballard, Stefano Martiniani, Jacob D Stevenson, Sandeep Somani, and David Wales. Exploiting the potential energy landscape to sample free energy. Wiley Interdisciplinary Reviews: Computational Molecular Science, 5(3):273-22) 2015.
Arghya Barman, Bruce Batiste, and Donald Hamelberg. Pushing the limits of molecular mechan- ics force field to probe weak ch.. π interactions proteins. Journal of chemical theory and computation, ((4)):854-1863) 2015.
Peter W Battaglia, Jessica B Hamrick, Vietor Bapst, Alvaro Sanzaez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv: 1806.01261, 2018.
Lucian Chan, Garrett M Morris, and Geoffrey R Hutchison. Understanding conformational entropy in small molecules. Journal of Chemical Theory and Computation, 17(4):2099-2066, 2021.
Gordon M Crippen, Timothy F Havel, et al. Distance geometry and molecular conformation, volume 74, Research Studies Press Taunton, 1988.
Marco De Vivo, Matteo Masetti, Giovanni Bottegoni, and Andrea Cavalli. Role of molecular dynamics and related methods in drug discovery. Journal of medicinal chemistry, 590)):4355-4411 2016.
Rémi Flamary and Nicolas Courty. Pot python optimal transport library. GitHlu: hups://github con/rflumary/POT, 2017.
Nils-Ole Friedrich, Christina de Bruyn Kops, Florian Flachsenberg, Kai Sommer, Matthias Rarey, and Johannes Kirchmair. Benchmarking commercial conformer ensemble generators. Journal of chemical information and modeling, 57(11):2719-772) 2017.
Pablo Gainza, Freyr Sverrisson, Frederico Monti, Emanuele Rodola, D Boscaini, MM Bronstein, and BE Correia. Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning. Nature Methods, 17(2):184 1120)99999999110000 2020.
Vikas Garg, Stefanie Jegelka, and Tommi Jaakkola. Generalization and representational limits of graph neural networks. In International Conference on Machine Learning, pages 3419-3430. PMLR, 2020.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 1263-1272. JMLR. org, 2017.
Stefan Grimme. Exploration of chemical compound, conformer, and reaction space with meta- dynamies simulations based on tight-binding quantum chemical calculations. Journal of chemical theory and computation, 18857:28472286.. 2019.
Cristiano RW Guimaraes, Alan M Mathiowetz, Marina Shalaeva, Gilles Goetz, and Spiros Liras. Use of 3d properties to characterize beyond rule-of-5 property space for passive permeation. Journal of chemical information and modeling, 52(4):882-890 2012.
Thomas A Halgren. Merck molecular force field. i. basis, form, scope, parameterization, and performance of mmf194. Journal of computational chemistry, 17(5-6)490-519, 1996.
Thomas Halgren. Mmff mmff94s option for energy minimization studies. Journal of Computa- tional Chemistry, 20(7):720-729, 1999.

---

William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 1025-1035, 2017.
Timothy F Havel. Distance geometry: Theory, algorithms, and chemical applications. Encyclopedia of Computational Chemistry, 120:723-742, 1998.
Timothy F Havel, Gordon M Crippen, Irwin D Kuntz, and Jeffrey M Blaney. The combinatorial distance geometry method for the calculation of molecular conformation ii. sample problems and computational statistics. Journal of theoretical biology, 104(3):383-44) 1983a.
Timothy Havel, Irwin D Kuntz, and Gordon M Crippen. The combinatorial distance geometry method for the calculation of molecular conformation. i. new approach to old problem. Journal of theoretical biology, 104(3)355--811 1983b.
Paul CD Hawkins. Conformation generation: the state of the art. Journal of Chemical Information and Modeling, 57(9)11777-1756, 2017.
Paul CD Hawkins and Anthony Nicholls. Conformer generation with omega: learning from the data and the analysis of failures. Journal of chemical information and modeling. 12(11)62919-2936 2012.
Paul CD Hawkins, A Geoffrey Skillman, Gregory L Warren, Benjamin A Ellingson, and Matthew T Stahl. Conformer generation with omega: algorithm and validation using high quality structures from the protein databank and cambridge structural database. Journal of chemical information and modeling, 50(4):572-584, 2010.
John Ingraham, Adam Riesselman, Chris Sander, and Debora Marks. Learning protein structure with differentiable simulator. In International Conference on Learning Representations, 2018.
Wolfgang Kabsch. A solution for the best rotation to relate two sets of vectors, Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography, 32(5):922-923, 1976.
Ilana Y Kanal, John A Keith, and Geoffrey R Hutchison. A sobering assessment of force field methods for low energy conformer predictions. International Journal of Quantum Chemistry, e1555), 2018
Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference Learning Representations (ICLR), 2017. 2017.
Johannes Klicpera, Janek Groß, and Stephan Günnemann. Directional message passing for molecular graphs. In International Conference on Learning Representations, 2019.
Ashutosh Kumar and Kam YJ Zhang. Advances in the development of shape similarity methods and their application in drug discovery. Frontiers in chemistry, 6:315, 2018.
David Lagorce, Tania Pencheva, Bruno O Villoutreix, and Maria A Miteva. Dg-ammos: A new tool to generate 3d conformation of small molecules using istance g cometry and utomated olecular echanics ptimization for in silico creening. BMC Chemical Biology, 9(1):1-10, 2009.
Dominik Lemm, Guido Falk von Rudorff, and O Anatole von Lilienfeld. Energy-free machine learning predictions ab initio structures. arXiv preprint arXiv:2102.028806 2021.
Yi Liu, Limei Wang, Meng Liu, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d graph networks. arXiv preprint arXiv:2102.05013, 2021.
Elman Mansimov, Omar Mahmood, Seokho Kang, and Kyunghyun Cho. Molecular geometry prediction using a deep generative graph neural network. Scientific reports, 911) 2019.
Mark MeGann. Fred pose prediction and virtual screening accuracy. Journal of chemical information and modeling, 51(3):578-50) 1.1
12

---

Frank Noé, Simon Olsson, Jonas Köhler, and Hao Wu. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning. Science, 365(6457), 2019.
Lagnajit Pattanaik, Octavian E Ganea, Ian Coley, Klavs F Jensen, William H Green, and Con- nor W Coley. Message passing networks for molecules with tetrahedral chirality. arXiv preprint 2010:22012.00094, 2020a.
Lagnajit Pattanaik, John B Ingraham, Colin A Grambow, and William H Green. Generating transition states of isomerization reactions with deep learning. Physical Chemistry Chemical Physics, 22 (41):23618 1-26622, 2020b.
Raghunathan Ramakrishnan, Pavio O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):177, 2014.
Anthony K Rappé. Carla J Casewit, KS Colwell, William A Goddard III, and W Mason Skiff. Uff, full periodic table force field for molecular mechanics and molecular dynamics simulations. Journal of the American chemical sociery, 114225):00244100333 1992.
Sereina Riniker and Gregory A Landrum. Better informed distance geometry: using what we know to improve conformation generation. Journal of chemical information and modeling. 55(12): 2672, 2015.
Kristof T Schütt, Huziel Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. Schnet-a deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148(24):241722, 2018.
Kristof T Schütt, Oliver T Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. arXiv preprint arXiv:2102.03150, 2021.
Christof H Schwab. Conformations and 3d pharmacophore searching. Drug Discovery Today:
Technologies, 7(4):e245-e253, 2010.
Andrew W Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Zidek, Alexander WR Nelson, Alex Bridgland, et al. Improved protein structure prediction using potentials from deep learning. Nature, 577(7792):706-710, 2020.
Jihyun Shim and Alexander D MacKerell Jr. Computational ligand-hased rational design: role of conformational sampling and force fields in model development. MealChemComm, 2(5):356-370,
2011.
Gregor Simm and Jose Miguel Hernandez-Lobato. A generative model for molecular distance geometry. In International Conference on Machine Learning. pages
PMLR, 2020.
Freyr Sverrisson, Jean Feydy, Bruno Correia, and Michael Bronstein. Fast end-to-end learning on protein surfaces. bioRxiv, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Proceedings of the 31 International Conference Neural Information Processing Systems, pages 6000-6010, 2017.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: 2017.
Jitender Verma, Vijay M Khedkar, and Evans C Coutinho. 3d-qsar in drug design-a review. Current topics in medicinal chemistry, 10(1):95-111 115, 2010.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? International Conference on Learning Representations (ICLR), 2019.
Minkai Xu, Shitong Luo, Yoshua Bengio, Jian Peng, and Jian Tang, Learning neural generative dynamics for molecular conformation generation. International Conference on Learning Represen- tations (ICLR), 2021.
Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angel Guzman- Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, et al. Analyzing learned molecular representations for property prediction. Journal of chemical information and modeling, 59(8):
3370-3388, 2019.
13


Reviews:
[{"review_id": "eAJgJloxaVa", "reviewer": "Reviewer_2HaS", "summary": "The authors propose the generative model that produces 3d conformations from a molecular graph based on predictions of local structures. First, for each anchor atom, the model predicts relative 1-hop positions of neighboring atoms, then predicts torsion angles. Finally, assemble the molecule in the deterministic way given predicted components. The authors comprehensively evaluate the model on the recently proposed dataset GEOM and compare it with state-of-the-art neural-based conformer generators and special software.\n\nThe paper's main contributions are: the authors propose a principal novel scheme for conformation generation that avoids expensive molecular geometry optimization used in current state-of-the-art neural conformer generators, still being translation, rotation, and reflection equivariant. The proposed model outperforms other baselines in both speeds of conformation generation and diversity of generated objects.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality:** The idea proposed in the paper is quite novel and differs from the current state-of-the-art neural methods for conformation generation both in used conformation representation and algorithm of generation: model operates with bond lengths, dihedral, and torsion angles instead of distance matrices and avoids time-consuming iterative process to optimize the final coordinates of conformation.\n\n**Quality:** The authors carefully describe their motivation, goals and provide an overview of existing methods. However, the paper lacks a broad discussion of potential applications of the proposed model in the drug-discovery area.\n\n**Clarity:** The paper is easy to follow. The equations and essential details of the generative process are clearly stated. Still, the paper reading experience suffers from the abundance of inline equations and the absence of equations references. Also, some subsections (i.e., 'Final LS assembly for a single bond' and 'How to compute γ?') are quite technical and can be moved fully or partially to the Appendix.\n\n**Significance:** The experimental results show the gap in the coverage and diversity metrics between the proposed model and alternatives. Still, the metrics do not provide standard deviation for reported metrics, so it's hard to determine how significant this gap is.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The main limitation of the paper is the absence of generated conformations quality evaluations - how physically plausible the generated conformations in contrast with ground-truth conformations and sampled conformations from other baselines. The model could show the high diversity and coverage metrics, still producing conformations with physically implausible fragment structures. This evaluation can be easily done by computing the potential energies of conformations.\n\n**Update**: The authors clarified this point in their rebuttal and provided an additional experiment demonstrating potential energies of generated structures.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "QGQsrfO1ASn", "reviewer": "Reviewer_c4sf", "summary": "This paper introduces GeoMol, an end-to-end learning method for molecular conformation generation.  It is based on the construction and alignment of elements of local atomic structure, is efficient in its parameterization of geometric degrees of freedom, and is SE(3) invariant.  It achieves competitive performance on the GEOM-DRUGS and GEOM-QM9 tasks.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: To the best of my knowledge, the method is novel, and represents a fast and clean way of generating molecular conformations. In particular, the generation of elements of local structure and their subsequent assembly in an end-to-end manner allows for much more computationally efficient performance than previous methods of comparable performance.\n\nQuality: The submission is technically sounds and the claims are backed up through experimental results.  The evaluation metrics are standard and well justified.\n\nClarity: The paper is very well written and self-contained.\n\nSignificance: I believe the results to be significant, especially as they allow for the rapid generation of many conformers with good accuracy.  One concern I have is whether this method will maintain accuracy for much larger molecules such as proteins, as these often are driven by long-distance interactions, and any local errors can compound.  \n\n*** Post-response update ***\n\nThank you for the additional information regarding your attempts to more explicitly model in long-distance interactions.  Seems like exciting future work!", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors convincingly address the limitations of the work. \n\nThere are no negative societal impacts to be addressed here.", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "ClTUUddz6fB", "reviewer": "Reviewer_DEuZ", "summary": "This paper studies how to generate low-energy 3D molecular conformations based on 2D molecular graphs.  \nTo this end, they propose the GEOMOL, an end-to-end trainable model that models molecular conformations in a SE(3)-invariant fashion, which first predicts the 3D local structures and then assembles them into final conformations with the predicted torsional angles. \n\n", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The GEOMOL takes torsional angles as well as stereochemistry (Tetrahedral chiral) into consideration and bypasses the 2-stage distance geometry technique used in previous works.  \n\nGEOMOL consists of several sub-modules, e.g., local Structure (1-hop) Prediction, Tetrahedral chiral corrections, Local Structure (LS) Assembly.  \nI find these sub-modules are overall novel at least in the ML community. The local structure assembly algorithm is the most interesting idea of the work in my view, which seems quite general and can be applied to other 3D modeling tasks.   \n\nThe paper is generally well-written and the codes are provided.\nExperimental results on GEOM datasets show that the performance of GEOMOL is quite promising.\n\n**Questions**\n\n I wonder how well can each designed submodule perform on each task, especially on torsional angles prediction? (local structure prediction task and torsional angle prediction task). \n\n**Update**\nThe authors address my concerns and looking forward to see the updated version of this exciting work.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The main limitations of the work are two folds: \n\n* (1) It only models the local structures (covalent bonds and their angles) and overlooks the long-range non-bonded angles, which may be crucial in multi-molecular systems or large molecules.  \n\n     Eq.4 (Line 273-274) indicates that GEOMOL predicts the torsional angle for each pair of LS independently. The benefit is that all LS can be assembled in parallel. However, it fails to take other structures into consideration (h_mol is used but it is not good enough).\n\n* (2) The model relies on the torsional angles to assemble local structures, which means it fails to handle molecular systems with disconnected molecular graphs (e.g., molecular complexes). \nOne potential way is to combine GEOMOL with the previous distance-based models, i.e., first use GEOMOL to compute the structure of each connected component, and then use distance-based method (CGCF[1] and GraphDG[2]) or gradient-based (ConfGF[3], a related work on conformation generation, maybe it's better to cite it in paper) to assembly disconnected part.\n\n**reference**\n1. Xu, M., Luo, S., Bengio, Y., Peng, J., and Tang, J. Learning neural generative dynamics for molecular conformation generation.\n2. Simm, G. and Hernandez-Lobato, J. M. A generative model for molecular distance geometry\n3. Shi, Chence and Luo, Shitong and Xu, Minkai and Tang, Jian. Learning Gradient Fields for Molecular Conformation Generation", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces an end-to-end learning method called GeoMol for molecular conformation generation.  The algorithm is novel and provides  a fast and clean way of generating molecular conformations. It achieves competitive performance on the GEOM-DRUGS and GEOM-QM9 tasks. All the reviewers are excited about the work. 

Decision: Accept (Spotlight)