A Generative Model for Molecular Distance Geometry
Authors: Gregor N. C. Simm
José Miguel Hernández-Lobato
Venue:  (2020)
Keywords: graph neural networks
variational autoencoders
distance geometry
molecular conformation

Abstract: Computing equilibrium states for many-body systems, such as molecules, is a long-standing challenge. In the absence of methods for generating statistically independent samples, great computational effort is invested in simulating these systems using, for example, Markov chain Monte Carlo. We present a probabilistic model that generates such samples for molecules from their graph representations. Our model learns a low-dimensional manifold that preserves the geometry of local atomic neighborhoods through a principled learning representation that is based on Euclidean distance geometry. We create a new dataset for molecular conformation generation with which we show experimentally that our generative model achieves state-of-the-art accuracy. Finally, we show how to use our model as a proposal distribution in an importance sampling scheme to compute molecular properties.

Under review as a conference paper at ICLR 2020
A GENERATIVE MODEL FOR MOLECULAR DISTANCE GEOMETRY
Anonymous authors Paper under double-blind review
ABSTRACT
Computing equilibrium states for many-body systems, such as molecules, is a long- song-standing challenge. In the absence of methods for generating statistically in- dependent samples, great computational effort is invested in simulating these sys- tems using, for example, Markov chain Monte Carlo. We present a probabilistic model that generates such samples for molecules from their graph representations. Our model learns low-dimensional manifold that preserves the geometry local atomic neighborhoods through principled learning representation that is based on Euclidean distance geometry. In a new benchmark for molecular conformation generation, show experimentally that our generative model achieves state-of- the-art accuracy. Finally, we show how to use our model as a proposal distribution in an importance sampling scheme compute molecular properties.
INTRODUCTION
Over the last few years, many highly-effective deep learning methods generating small molecules with desired properties (e.g., novel drugs) have emerged Boanez--ommbarlli et al. 2018 Segler cet al. 2018 Dai et al.) 2018 Jin et al. 2018 Bradshaw et al. 2019a; Liu et al. 2018 You et al. 2018 Bradshaw et al. 2019b). These methods operate using graph representations of molecules in which nodes and edges represent atoms and bonds, respectively. A representation that is closer to the physical system is one in which a molecule is described by its geometry or conformation. A conformation x of a molecule is defined by a set of atoms r:) where N, is the number of atoms in the molecule, & E (H, O..... is the chemical element of the atom i, and r; R is its position in Cartesian coordinates. Importantly, the relative positions of the atoms are restricted by the bonds in the molecule and the angles between them. Due to thermal fluctuations resulting in stretching of and rotations around bonds, there exist infinitely many conformations of a molecule. molecule's graph representation and a set of its conformations are shown Fig. [ Under wide range of conditions, the probability p(x) of conformation x, is governed by the Boltzmann dis- tribution and is proportional to exp Exx) //gT), where E(x) E IR is the conformation's energy, kn is the Boltzmann constant, and T is the temperature.
To compute a molecular property for a molecule, one must sample from p(x). The main approach is to start with one conformation and make small changes to it over time, e.g., by using Markov chain Monte Carlo (MCMC) or molecular dynamics (MD). These methods can be used to accurately sample equilibrium states of molecules, but they become computationally expensive for larger ones (Shim & MacKerell 2011: Ballard et al. 2015 De Vivo et al. 2016). Other heuristic approaches ex- ist in which distances between atoms are set to fixed idealized values (Havel, 2002 Blaney & Dixon, 2007]. Several Several methods based on statistical learning have also recently been developed to tackle the issue of conformation generation. However, they are mainly geared towards studying proteins and their folding dynamics (AIQuraishi) 2019). Some of these models are not targeting a distribution over conformations but the most stable folded configuration (Evans et al. 2018 Ingraham et al.)
2019 while others are not transferable between different molecules (Lemke & Peter 2019 Noē
al. 2019).
This work includes the following key contributions:
We introduce a novel probabilistic model for learning conformational distributions of molecules with graph neural networks.

---

Under review as a conference paper at ICLR 2020
H
X6
X3
X5
X2
Figure I: Standard graph representation molecule (left) with a set of possible conformations {x} (right). Hydrogen (H), carbon (C), and oxygen (O) atoms are colored white, gray, and red, respectively. Conformations feature the same atom types and bonds but the atoms are arranged differently in space. These differences arise from rotations around and stretching of bonds in the molecule.
We create new, challenging benchmark for conformation generation, which is made pub- licly available. To the best of our knowledge, this is the first benchmark of this kind.
By combining conditional variational autoencoder (CVAE) with an Euclidean distance geometry (EDG) algorithm we present state-of-the-arr approach for generating one-shot samples of molecular conformations for unseen molecules that is independent their size and shape.
We develop rigorous experimental approach for evaluating and comparing the accuracy of conformation generation methods based on the mean maximum deviation distance metric.
We show how this generative model can be used as a proposal distribution in an importance sampling (IS) scheme to estimate molecular properties.
2
METHOD
Our goal is to build a statistical model that generates molecular conformations in one-shot fashion from molecule's 'ss graph representation. First, we describe how molecule's conformation can be represented by a set of pairwise distances between atoms and why this presentation is advantageous one in Cartesian coordinates (Section 2.1 Second, tion 2.2 that will generate sets of atomic distances for given molecular graph. Third, we explain in Section 2.3 how set of predicted distances can be transformed into molecular conformation and why this transformation is necessary. Finally, we detail in Section 2.4 how our generative model can be used a proposal distribution an IS scheme to estimate molecular properties.
present generative model in Sec-
2.1 EXTENDED MOLECULAR GRAPHS AND DISTANCE GEOMETRY
In this study, molecule is represented by undirected graph which is defined tuple G (V, V {u.),"r is the set of nodes representing atoms, where each e; E R. holds atomic attributes (e.g., the element type e,). E ((ER, Tk, ) is the set of edges, where each e E F. holds an edge's attributes (c.g.. the bond type), and тą and s are the nodes an edge is connecting. Here, E represents the molccular bonds (and the auxiliary edges which are explained below) in the molecule.
k=1 N-
We assume that, given molecular graph G, one can represent one of its conformations x by a set of atomic distances d 1= ) where dę Irx rssl the Euclidean distance between the positions of the atoms and $4 in this conformation. As the set of edges between the bonded atoms (Fbana) alone would not suffice to describe a conformation, we expand the traditional graph repre- sentation of molecule by adding auxiliary edges. Auxiliary edges between atoms that are second neighbors in the original graph fix angles between atoms, and those between third neighbors fix dihe- dral angles (denoted Eaggle and Eaitedmat respectively). In this work, Eangle consists of edges between all second neighbors the original graph. Edges between third neighbors are added according to always referring to this extended molecular
heuristic (see Appendix I.I). From now graph when talking about molecular graphs. In Fig, 2 the process of extending the molecular graph and the extraction of d from and & are illustrated.
2

---

Under review as a conference paper at ICLR 2020
A
B
d
d3
61
63
1/3
d2
X
d1
2/
d2
C1
dg
U5
d
e2
N
g
Figure 2: A) The structural formula of a molecule is converted to an extended molecular graph G consisting of nodes representing atoms (circles, e-g., v1) and edges representing molecular bonds (solid lines, e.g., é1 E EbonA) and auxiliary edges (dotted lines, e.g., e2 E Eangle and 63 Emieedaa)> B) The distances d are extracted from a conformation based on the edges E. C) Graphical model of the variational autoencoder: generative model po po 2. (z]G) (solid lines) and variational approximation 4o (2| 9) (dashed lines).
A key advantage of representation in terms of distances is its invariance to rotation and trans- lation; by contrast, Cartesian coordinates depend on the (arbitrary) choice of origin, for example. In addition, it reflects pair-wise physical interactions and their generally local nature. Auxiliary edges can be placed between higher-order neighbors depending on how far the physical interactions dominating the potential energy of the system reach.
We have set of Ng molecular graphs (G)) Further, for each G1, we have Si conformational samples {xt,) ) from the ground-trute truth distribution resulting St sets of distances {dr.s//=- With this data, we will train generative model which we detail in the following section.
2.2 GENERATIVE MODEL
We employ CVAE (Kingma & Welling 2014 Pagnoni et al. 2018) to model the distribution over distances d given molecular graph G. A CVAE first encodes together with d into latent space z E RRNN, where & N+, with an encoder (z)z,, Subsequently, the decoder Po(d (2 g) decodes back into set of distances. A graphical model shown Fig. C).
A conformation has, in general, 3.N,, - 6 spatial degrees of freedom (dofs): one dof per spacial dimension per atom minus three translational and three rotational dofs. Therefore, the latent space should be proportional to the number of atoms in the molecule. In addition, the latent space should be smaller than 3.N, as is the role of the encoder to project the conformation into a lower-dimensional space. As result, set k to avoid overfitting
Here, go (z(z)cL dl G) and po (d|z, G) are Gaussian distributions, the mean and variance of which are modeled by two artificial neural networks. At the center of this model are message- neural networks (MPNNs) (Gilmer et al., (017)) with multi-heat attention (Veličković et (0188). In short, MPNN convolutional neural network that allows end-to-end learning of prediction pipelines whose inputs are graphs of arbitrary size and shape. In a convolution, neighboring nodes exchange so-called messages between neighbors update their attributes. Edges update their attributes with the features of the nodes they are connecting. The MPNN is a well-studied technique that achieves
state-of-ther -fwtoff---e-rrtt performance in representation learning for molecules (Kipf Welling 2017 Duve-
naud et al. 2015 Kearnes et al. [2166] Schütt et al. 201 7b; Gilmer et al. 2017 Kusner et al. 2017
Bradshaw et al. 2019a).
In the following, we describe the details of the model
2 In Fig. an illustration of the model is shown. In the encoder qo (z dl 9), each de is concatenated with the respective edge feature e to give co T F.+1, Then, each e and each ea are passed to Femuy and Femu,e (two multilayer perceptrons,
EEpeermmnns showed that our model performs similarly with latent space of R.2N. and overfits with latent spaces of 3N.. 2The full model (including all parameters) is available online 1b420F886667844573355
https://fft gaha com/s/
3

---

Under review as a conference paper at ICLR 2020
0/
Gand
1/3
11
F
C3
U1
22
Renc
22
d2
d3
MP
Uiz
U5
28
2/A
Fanc,e
U/
Fdoc,
2/3
Hbd oād
MP
dec
Riice
Edec,
G
The molecular graph G together with the distances d are passed through the model con- sisting of an encoder 4s (z d. 9) and a decoder pe pollz. 9). See the main text for details.
Figure 3:
MLPs), respectively, give Gold where Geod (10 N. (1vi {(e Pk, sk)) o E R
and kkenc RR Then, T MPNNs of depth 1, {MP(1)T ene consecutively applied to obtain G Finally, the read-out function Rene (an MLP) takes each o to predict the mean Ms. R and the variance of, E of the Gaussian distribution for 2;. The so-called reparametrization trick is employed to draw a sample for z,. In summary.
Fi,enc
(0)
Feac,e (14),
k.enc Fenc,e(E)))
(0)
GLIl
MP (G(0))
g(++1) MP(() (g(()),
G(TT) MP
(CCT-")(C(T-1))
(I)
M2 o. Rene (u (1(2))
In the decoder (dl 9), each z; is concatenated with the respective node feature to give of E E. +1 Each v/ and each E₁ are passed to Face,v and F (two MLPs), respectively, give 9/0) where G( = ((1))) [ejee N ((eka rk; No == Vi,dec E TIRL_,, and éxuee E LR.,, Then, T MPNNs
(t)
of depth 1, [MP()) 1T
dec f==1- are consecutively applied to obtain G(22) Finally, the read-out function Race (an MLP) takes each 6k, dee to predict the mean Hd, E R and the variance o2 E IR the Gaussian distribution for dg. In summary,
Faee, (0)
k,dee (esc,a(es)s
dee
c (P(") (g(0) dec
g(t+1) MP(() (g(t)),
g(T) MP(T-")(c(T-)) (C(T-1))
dee (g
pda dde Rate (E )
(T)
The sets of parameters in the encoder and decoder, and O (i.e., parameters in Fenc, Fenc,es (MP(t)) end ) Renc- Face, Faoc,er (MP(0)7 dec 1t=1 Race), respectively, are optimized by maximizing
the evidence lower bound (ELBO):
L E Zzzga (logo
- ()
where the prior (z) consists of factorized Gaussians. The optimal values for the hyperpa- rameters for the network dimensions, number of message passes, batch size, and learning rate of the Adam optimizer (Kingma & Baj (2044) were tuned by maximizing the validation performance (ELB8 (ELBO) with Bayesian optimizer and reported in Appendix A.1.3

---

Under review as a conference paper at ICLR 2020
2.3 CONFORMATION GENERATION THROUGH EUCLIDEAN DISTANCE GEOMETRY
To compute molecular properties, quantum-chemical methods need to be employed which require the input, i.e., the molecule, to be in Cartesian coordinates Therefore, No to translate the set of distances {d&] No
an EDG algorithm
k= to set of atomic coordinates (r:)
EDG is the mathematical basis for a geometric theory of molecular conformation. In the field of machine learning, Weinberger & Saul (2006) used it for learning image manifolds, [Tenenbaum et al. (2000) for image understanding and handwriting recognition, Jain & Saul (2004) for speech and music, and Demaine et al.) (2009) for music and musical rhythms. An EDG description of a molecular system consists of list of lower and upper bounds on the distances between pairs of atoms mdd..mins ma ) k=r Here, Po (dlz, I G) is used to model these bounds, namely, we set the bounds to {(Ud, Oda Mdz + od.)) where Hd, and od, are the mean and standard deviation for each distance de given by the CVAE. Then, an EDG algorithm determines a set of Cartesian coordinates (r: (ri) so that these bounds are fulfilled (see Appendix A.2 for details) 5 Together with
the corresponding chemical elements {cz J
obtain conformation
2.4 CALCULATION OF MOLECULAR PROPERTIES
We can get an MC estimate of the expectation Eg([] of a property O (e.g., the dipole moment) for a molecule represented by 9 by drawing conformational samples x; p(x(G) and computing O(x;) E IR with quantum-chemical method (e.g., density functional theory). Since we cannot draw samples from p(x()) directly, we employ an IS integration scheme (Bishop) 2009) with our CVAE as the proposal distribution. We assume that we can readily evaluate the unnormalized probability of a conformation p(xlg) = (
where x must be a conformation of the molecule and the energy E(x) is determined with a quantum-chemical method. Since the EDG algorithm is map- ping the distribution po (d|2, 9) to a point mass in 3R,,, the MC estimate for the resulting distribu- tion Pprop Pprop(x (x( is given by mixture of delta functions, each of which is centered at the x; resulting
from mapping po z, 9) to R&N, where 2; ~ (z(2))), that is, Pprop (x(9) Ei= 8(x x:).
The IS estimator for the expectation of O w. r. p (() then reads
Ed
MC
N
O(x)
IS
Σ
N
Ox))
5(x 9
(4)
N
N
Perop
(x())*
1=1
where x, p(x,(g) and x version of fxx) is then
Perop (x( so that the expectation of O t. the normalized
1o 1 Eg[O] = Eg = Z [ox))(X11)
(5)
where Z = E, p(x²) and N is the number of samples. When dividing two delta functions have assumed that they take some arbitrarily large finite value.
3 RELATED WORKS
The standard approach for generating molecular conformations is to start with one, and make small changes to it over time, e.g., by using MCMC or MD. These methods are considered the gold standard for sampling equilibrium states, but they are computationally expensive, especially the molecule is large and the Hamiltonian is based quantum principles (Shim & MacK- crell 201T Ballard et al. 2015 De Vivo et al. 2016).
A much faster but more approximate approach for conformation generation is EDG (Havel 2002
Blaney & Dixon 2007
Lagorce et al. 2009 Riniker & Landrum 2015). Lower and upper distance
'Even though quantum-chemical methods require the input to he in Cartesian coordinates, calculated prop- crties, such as the energy, are invariant under translation and rotation.
'There are additional constraints due chirality. However, since they are given by and are fixed, they are not modeled by our method.
*Often there exist multiple solutions for the same set of bounds. As the bounds are generally tight, the solutions very similar. Therefore, we only generate one set of coordinates per set of bounds.
5

---

Under review as a conference paper at ICLR 2020
bounds for pairs of atoms in molecule are fixed values based on ideal bond lengths, bond angles, and torsional angles. These values are often extracted from crystal structure databases (Allen (2022). These methods aim to generate low-energy conformation, not to generate unbiased samples from the underlying distribution at a certain temperature.
There exist several machine learning approaches well, however, they mostly tailored towards studying protein dynamics. For example, Noć et al. (2019) trained Boltzmann generators on the energy function of proteins to provide unbiased, one-shot samples from their equilibrium states. This is achieved by training an invertible neural network to learn a coordinate transformation from system's 's configurations a latent space representation. Further, Lemke Peter (2019) proposed a dimensionality reduction algorithm that is based on neural network autoencoder in combination with a nonlinear distance metric to generate samples protein structures. Both models learn protein-specific coordinate transformations that cannot be transferred to other molecules.
AlQuraishi (219)) introduced an end-to-end differentiable recurrent geometric network for protein structure learning based on amino acid sequences. Also, Ingraham al. (2019) proposed neural energy simulator model for protein structure that makes use of protein sequence information. In contrast to amino acid sequences, molecular graphs are, in general, not linear but highly branched and often contain cycles. This makes them unsuitable for recurrent networks.
Finally, Mansimov al. (2019) presented conditional deep generative graph neural network to generate molecular conformations given a molecular graph. Their goal is to predict the most likely conformation and not distribution over conformations. Instead of encoding molecular environ- ments atomic distances, they work directly in Cartesian coordinates. As a result, the generated conformations showed significant structural differences compared to the ground-truth and required refinement through force field, which often employed in MD simulations.
We argue that our model has several advantages the approaches reviewed above:
It is fast alternative to resource-intensive approaches based on MCMC or MD.
Our principled representation based on pair-wise distances does not restrict our approach to any particular molecular structure.
Since model employs message-passine passing neural networks, it is transferable it can trapolate from only few graphs to unseen ones.
4 THE CONF17 BENCHMARK
The CONF17 benchmark is the first benchmark for molecular conformation sampling It is based on the ISO17 7 dataset (Schätt et al.] 2017a) which consists of conformations of various molecules with the atomic composition 12 drawn from the QM9 dataset (Ramakrishnan et al., 2014). These conformations were generated by ab initio molecular dynamics simulations at 500 Kelvin which generates trajectories of single molecule covering a large variety of conformations, The CONF17 benchmark consists of 127 distinct molecular graphs each with 3380 conformations on average. We split this dataset into multiple training and test splits, each consisting of 107 and 20 graphs, respectively (see Appendix A.1 for more details)
Fig. 4 (A), the structural formulae of a random selection of molecules from this benchmark are shown. Most molecules feature highly-strained, complex 3D structures such rings which are typical drug-like molecules. number of degrees of freedom, that makes this benchmark challenging. In Fig, 4 (B), the frequency of distances (in A) in the conformations are shown for each edge type. can be seen that the marginal distributions of the edge distances are multimodal and highly context dependent.
thus the structural complexity of the molecules, not their
'atasetts such one published by Kanal 2018 only include conformers, i.e., the stable confor- mations a molecule, not distribution over conformations.
"The CONF17 benchmark is publicly available online 1b420fa65bd78c45773
https: gahare. com/s/

---

Under review as a conference paper at ICLR 2020
A
CH2
HyC
HbC
HO
OH
HO
HBC
CH3
HyC
HO
CH3
HyC
OH
H₂C
OH
CH₃
HO
CHg
B
1e3
1e3
1e3
H-C
2000
H-H
H-C
1500
H-O
H-C
H-O
H-O
Count
1000
C-0
Count
1500
C-C
Count
60
C-0
C-O
0-0
1000
0-0
40
500
500
20
0
1.0
Distance [A]
Distance [A]
Distance [A]
Figure 4: Overview of the CONF17 benchmark. (A) Structural formulae of a random selection of molecules. (B) Distribution of distances (in Å) grouped by edge (from left right: Evond, Eangle and Editedra)) and vertex type (chemical element).
5 EXPERIMENTS
We assess the performance of our method, named Graph Distance Geometry (GRAPHDGG), by com- paring with two state-of-the-art methods for molecular conformation generation: RDKIT (Riikkrr & LLaddumm 2015). classical EDG approach, and DL4CHEM (Mansimov et al. 2019), machine learning approach. We trained GRAPHDG and DL4CHEM on three different training and test splits of the CONF17 benchmark using Adam (Kingma & Ba 2014). We generated 3000 conformations with each method for molecular graphs in a test set.
5.1 DISTRIBUTIONS OVER DISTANCES
We assessed the accuracy of the distance distributions of RDKIT, DL4CHEM, and GRAPHIDG by calculating maximum mean discrepancy (MMD) (Gretton et al. 2012) to ground-truth dis- tribution. We compute the MMD using a Gaussian kernel, where set the standard deviation to be the median distance between distances tances in the conformations from the ground-truth and those generated by RDKIT and DLACHEM. For each train-test split and each G in a test set, we compute the MMD of the joint distribution of distances between C and atoms (H atoms are usually ignored), the MMDs of pair-wise distances p(d,, dy (),, and the MMDs between the marginals of individual distances p(d: G). We aggregate the results of three train-test splits, and, finally, compute the median MMDs and average rankings.
the aggregate sample. For this, we determined the dis-
The results are summarized in Table It can be seen that the samples from GRAPHDG are signifi- cantly closer to the ground-trus distribution than the other methods. RDKIT is slightly worse than GRAPHDG while DLACHEM seems to struggle with the complexity of the molecules and the small number of graphs in the training set.
In Fig. we showcase the accuracy of our model by plotting the marginal distributions p(d.19 (d) for distances between and atoms given a molecular graph from a test set. It can be seen that RDKIT consistently underestimates the marginal variances. This is because this method aims to predict the most stable conformation, i.e., the distribution' 's mode. In contrast, DL4CHEM often fails to predict the correct mean. For this molecule, GRAPHDG is the most accurate predicting the right mean and variance in most cases. Additional figures can be found in the Appendix A.4, where we also show plots for the marginal distributions p(d, dz19).
7

---

Under review as a conference paper at ICLR 2020
Table 1: Assessment of the accuracy of the distributions over conformations generated by three models compared to the ground-truth. We compare the distributions with respect to the marginals p(dx ()) p(de, di [g), and the distribution over all edges between C and O atoms p( p(fd+)9)) Two different metrics are used: median MMD between ground-truth conformations and generated ones, and mean ranking (1 to 3) based on the MMD. Reported are the results for molecular graphs test set from three train-test splits. Standard errors are given in brackets.
Median MMD
Mean Ranking DL4CHEM
RDKIT
DL4CHEM
GRAPHDG
RDKIT
GRAPHDG
p(d+1g)
0.55 (0.01)
1.11 (0.01)
0.38 (0.02)
1.71 (0.03)
2.74 (0.02)
1.51 (0.03)
p(dr, dilg)
0.53 (0.01)
1.09 (0.01)
0.34 (0.01)
1.66 (0.02)
2.92 (0.01)
1.43 (0.02)
0.60 (0.01)
1.07 (0.03)
0.44 (0.05)
1.58 (0.05)
2.90 (0.05)
1.45 (0.02)
1000
Count
500
1.4
1.5
2.0
1.5
2.0
2.5
1.4
1.5
2.0
1000
3-4
3-6
4-5
4-7
9-6
Count
500
1000
Ground-truth
Count
500
RDKit
DL4Chem
GraphDG
1.5
Figure 5: Marginal distributions p(dx (9) of ground-truth and predicted bond distances (in Å) be- tween C and atoms given a molecular graph from the test set. The atoms connected by each edge dy are indicated in each subplot (sı-rR). In the 3D structure of the molecule, carbon and oxygen atoms are colored gray and red, respectively. H atoms are omitted for clarity.
5.2 GENERATION OF CONFORMATIONS
We passed the distances from our generative model to an EDG algorithm to obtain conformations. For 99.9% of the sets of distances, all triangle inequalities held. For 94% of the molecular graphs, the algorithm succeeded which is 8 higher than the success rate we observed for RDKIT. For each molecular graph in a test set, we generated 50 conformations with each method. This took DL4CHEM, RDKIT, and GRAPHDG on average around hundreds of milliseconds per molecule In contrast, single conformation the ISO17 dataset takes around minute to compute. In Fig 6 an overlay of these conformations of six molecules generated by the different methods is shown. can be seen that RDKIT's conformations show too little variance, while DI.4CHEM's structures mostly invalid, which is due in part to its failure predict the correct interatomic angles. Our method slightly overestimates the structural variance (see, for example, Fig. 6 top row, second column), but produces conformations that are the closest to the ground-truth.
5.3 CALCULATION OF MOLECULAR PROPERTIES
We estimate expected molecular properties for molecular graphs from the test set with N 50 conformational samples each. Due to their poor quality, we could not compute properties Oxx), including the energy E(x), for conformations generated with DL4CHEM, and thus, this method is excluded from this analysis. In Table 2 it can be seen that RDKIT and GRAPHDG perform
'll simulations were carried out computer equipped with an 17-3820 CPU and GeForee GTX 1080
Ti GPU.
8

---

Under review as a conference paper at ICLR 2020
Ground-truth
RDKit
DL4Chem
GraphDG
Ground-truth
DL4Chem
GraphDG
Figure 6: Overlay of 50 conformations from the ground -ruth and three models based six random molecular graphs from the test set. C, O, and H atoms are colored gray, red, and white, respectively.
Table 2: Median difference in average properties between ground-truth and RDKIT and GRAPHDG: total electronic energy Eetee (in kJ/mol), the energy of the HOMO and the LUMO ELUMO and ELUMO. respectively (in eV), and the dipole moment (in debyc). Reported are the results for molecular graphs from the test set, averaged over three train-test splits. Standard errors are given in brackets.
RDKIT
GRAPHDG
Éetec
42.7 (4.3)
58.0 (21.0)
0.08 (0.04)
0.10 (0.05)
ÉHOMO
CLUMO
0.15 (0.03)
0.09 (0.05)
0.29 (0.05)
0.33 (0.09)
similarly well (see Appendix A.2 for computational details). However, both methods are still highly inaccurate for Estee (in practice, an accuracy of less than 5 kJ/mol required). Close inspection of the conformations shows that, even though GRAPHDG predicts the most accurate distances overall, the variances of certain strongly constrained distances (e.g., triple bonds) are overestimated so that the energies of the conformations increase drastically.
LIMITATIONS
The first limitation of this work that the CVAE can sample (with low probability) invalid sets of distances for which there exists 3D structure. Second, the CoNF17 benchmark covers only small portion of chemical space. Finally, large set of auxiliary edges would be required to capture long-range correlations (e.g., in proteins). Future work will address these points.
CONCLUSIONS
We presented GRAPHDG, a transferable, generative model that allows sampling from a distribution over molecular conformations. We developed principled learning representation of conforma- tions that is based on distances between atoms. Then, we proposed challenging benchmark for comparing molecular conformation generators. With this benchmark, we show experimentally that conformations generated by GRAPHDG are closer to the ground-truth than those generated by other methods. Finally, we employ our model as a proposal distribution in an IS integration scheme to estimate molecular properties. While orbital energies and the dipole moments were predicted well, a larger and more diverse dataset will be necessary for meaningful estimates of electronic energies. Further, methods have be devised to estimate how many conformations need to be generated to ensure all important conformations have been sampled. Finally, our model could be trained on conformational distributions at different temperatures transfer learning-type setting.

---

Under review as a conference paper at ICLR 2020
REFERENCES
H. Allen. The Cambridge Structural Datahase: A quarter of a million crystal structures and rising. Acta Crystallogr, Sect. B: Struct. Sci, 58(3):380-388, 2002. doi: 10.1 7077//000077810000338887
Mohammed AlQuraishi. End-to-End Differentiable Learning of Protein Structure. Cell Systems, 8 (4):292-301.e3. 2019, doi: 10.1016/j.cels.2000 2019..0....20...0000 9.03.006.
Andrew Ballard, Stefano Martiniani, Jacob D. Stevenson, Sandeep Somani, and David J. Wales. Exploiting the potential energy landscape to sample free energy. WIREs Comput, Mol. Sci., 5(3): 273-289, 2015. doi: 10.1002/wcms.1211
Christopher M. Bishop. Patfern Recognition and Machine Learning. Information Science and Statistics. Springer, New York, 8 edition, 2009. ISBN 978-0-387-31077--2
Jeffrey M. Blaney and J. Scott Dixon. Distance Geometry in Molecular Modeling. In Reviews in Computational Chemistry, pp. 299-335. John Wiley & Sons, Ltd, 2007. ISBN 978-0-470-12582.
3. doi: 10. 000100229780447022233...oo
John Bradshaw, Matt J. Kusner, Brooks Paige, Marwin H. S. Segler, and José Miguel Hernández- Lobato. A generative model for electron paths. In International Conference Learning Repre- sentations, 2019a.
John Bradshaw, Brooks Paige, Matt J. Kusner, Marwin H. S. Segler, and José Miguel Hernández- Lobato. A Model to Search for Synthesizable Molecules.
Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax-directed variational autoen- coder for structured data. International Conference Learning Representations, 2018.
Marco De Vivo, Matteo Masetti, Giovanni Bottegoni, and Andrea Cavalli. Role of Molecular Dy- namics and Related Methods in Drug Discovery. J. Med. Chem., 59(9):4035 4061, 2016. doi: (m022//ass
Erik D. Demaine, Francisco Gomez-Martin, Henk Meijer, David Rappaport, Perouz Taslakian, God- fried T. Toussaint, Teryy Winograd, and David R. Wood. The distance geometry of music. Com- putational Geometry, 42(5)) 454, 2009,
doi: 10.1016fj.ooggeo20000 http://oooo.200000..0000 04.005.
David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alan Aspuru-Guzike and Ryan P Adams. Convolutional Networks on Graphs for Learning Molecular Fingerprints. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems 28, pp. 2232-2232. Curran Associates, Inc., 2015.
R. Evans, J. Jumper, J. Kirkpatrick, L. Sifre, T. F. G. Green, C. Qin, A. Zidek, A. Nelson, A. Bridgland, H. Penedones, S. Petersen, K. Simonyan, S. Crossan, D. Jones, D. Silver, K. Kavukcuoglu, D. Hassabis, and A. W. Senior. De novo structure prediction with deep-learning based scoring. In Thirteenth Critical Assessment of Techniques for Protein Structure Prediction, 2018.
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, pp. 1263-1272, 2017.
Rafael Gómez- Bombarloomaareli,,, Jennifer N. Wei. David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sänchez-Lengelingg Dennis Sheberla, Jorge Aguilera-Iparraguiree Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Guzik Automatic Chemical Design Using Data-Driven Continuous Representation of Molecules. ACS Cent. Sci., 4(2):268 276, 2018. doi: 10.1021/ acscentsei.7600572.
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A Kernel Two-Sample Test. J. Mach. Learn. Res., 13:723-773, 2012.
Wolfgang Guba, Agnes Meyder, Matthias Rarey, and Jérôme Hert. Torsion Library Reloaded: A New Version of Expiet-Derived SMARTS Rules for Assessing Conformations of Small Molecules. J. Chem. Inf. Model,, 56(1):1-5, 2016. doi: 10.102//asxjcimm55005222
10

---

Under review as a conference paper at ICLR 2020
Timothy F. Havel. Distance Geometry: Theory, Algorithms, and Chemical Application, In Encyclo- pedia of Computational Chemistry. American Cancer Society, 2002. ISBN 978-0-470-84501-11 doi: 10.100220470845515.ccahh.
John Ingraham, Adam Riesselman, Chris Sander, and Debora Marks. Learning Protein Structure with Differentiable Simulator. In International Conference on Learning Representations, 2019.
V. Jain and L. K. Saul. Exploratory analysis and visualization of speech and music by locally linear embedding. In 2004 IEEE International Conference Acoustics, Speech, and Signal Processing, volume 3, pp. ii--984, 2004. doi: 10. (0011090CCASSSP224...3330
Jan Jensen. XY22Mol. http://ggthbbbbmom//ssssgggppxpu 2019.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 2322-2333 Stockholm, Sweden, 2018. PMLR.
Iaaa Y. Kanal, John A. Keith, and Geoffrey R. Hutchison. sobering assessment of small-molecule force field methods for low energy conformer predictions. Int. J. Quantum Chem., 118(5):225512, 2018. doi: 10. 1511..
Steven Kearnes, Kevin McCloskey, Mare Berndl, Vijay Pande, and Patrick Riley. Molecular graph convolutions: Moving beyond fingerprints. J. Comput.-AAie Aided Mol. Des., 30(8):595-608, 2016.
doi: 10220097--
Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. arXiv: 69800, 2014.
Diederik P. Kingma and Max Welling, Auto-Encoding Variational Bayes. In International Confer- ence on Learning Representations, 2014.
Thomas N. Kipf and Max Welling, Semi-Supervised Classification with Graph Convolutional Net- works. International Conference on Learning Representations, 2017.
Matt J. Kusner, Brooks Paige, and José Miguel Hemmández-Lobato. Grammar Variational Autoen- coder. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Con- ference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp- 1945-1954, International Convention Centre, Sydney, Australia, 2017, PMLR.
David Lagorce, Tania Pencheva, Bruno O. Villoutreix, and Maria A. Miteva. DG-AMMOS: A New tool to generate 3D conformation of small molecules using Distance Geometry and Automated Molecular Mechanics Optimization for in silico Screening. BMC Chem. Biol., 9:6. 2009. doi: 10. 10.11886/472--669-9-4
Tobias Lemke and Christine Peter. EncoderMap: Dimensionality Reduction and Generation of Molecule Conformations. J. Chem. Theory Comput., 2019. doi: 10. N0.1022//acs.iccc.80009757
Qi Liu, Miltiadis Allamanis, Marc Brockschmidt, and Alexander Gaunt. Constrained Graph Varia- tional Autoencoders for Molecule Design. In Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Resa-Bianche and R. Garnett (eds.), Advances in Neural Information Processing Systems 31, pp. 7795-7804. Curran Associates, Inc., 2018
Elman Mansimov, Omar Mahmood, Seokho Kang. and Kyunghyun Cho. Molecular geometry pre- diction using a deep generative graph neural network. arXiv: 1904.00314 [physics, stat), 2019.
Frank Noé. Simon Olsson, Jonas Köhler, and Hao Wu. Boltzmann generators: Sampling equilibrium states mady-body systems with deep learning. Science, 365(6457) 2019. doi: 10. 10.1126/science.aav 147.
Artidoro Pagnoni, Kevin Liu, and Shangyan Li. Conditional Variational Autoencoder for Neural Machine Translation. arXiv:/8/2 04405, 2018.

---

Under review as a conference paper at ICLR 2020
John P. Perdew, Matthias Ernzerhof, and Kieron Burke. Rationale for mixing exact exchange with density functional approximations. Chem. Phys., 105(22):9882-9985 1996. doi: 10.1063/1. 472933.
Raghunathan Ramakrishnan, Pavlo O. Dral, Matthias Rupp, and O. Anatole von Lilienfeld. Quan- tum chemistry structures and properties of 134 kilo molecules. Sci. Data, 1:140022, 2014. doi: 10. 1..1088/data.2014.22..
Sereina Riniker and Gregory A. Landrum. Better Informed Distance Geometry: Using What We Know To Improve Conformation Generation. J. Chem. Inf. Model., 55(12):552222574, 2015, doi: 10.
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. SchNet: continuous filter convolutional neural net- work for modeling quantum interactions. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 991-1001. Curran Associates, Inc., 2017a.
Kristof T. Schütt, Farhad Arbabzadah, Stefan Chmicla, Klaus R. Müller, and Alexandre Tkatchenko. Quantum-chemick insights from deep tensor neural networks. Nat. Commun., 8: 8:13890, 2017b. doi: 10.1033//comms 13890.
Marwin H. S. Segler, Thierry Kogej. Christian Tyrchan, and Mark P. Waller. Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks. ACS Cent. Sci., 4(1): 120-131, 2018. doi: 10. aa.10//aaccentsci.bb00000
Jihyun Shim and Alexander D. MacKerell, Jr. Computational ligand-based rational design: Role of conformational sampling and force fields in model development. Med. Chem. Commun. 2(5):
356-370, 201 doi: 10. 00.1039//IM0000444
Qiming Sun, Timothy Berkelbach, Nick S. Blunt, George H. Booth, Sheng Guo, Zhendong Li, Junzi Liu, James D. McClain, Elvira R. Sayfutyarova, Sandeep Sharma, Sebastian Wouters, and Garnet Kin-Lic Chan. PySCF: The Python-based simulations of chemistry framework. WIREs Comput, Mol. Sci., 8(1), 2018. doi: 10. 10.1002/wcms.1330 1340.
Joshua B. Tenenbaum, Vin de Silva, and John C. Langford. work for Nonlinear Dimensionality Reduction. Science, 290(5500):3119-333, 2000. doi: 10. 2e.1126/scennce.205555000
Global Geometric Frame-
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. In International Conference Learning Representations, 2018.
Florian Weigend. Accurate Coulomb fittiin basis sets for H to Rn. Phys. Chem. Chem. Phys., 8(9): 1057-1065, 2006. doi: 10.1039
Florian Weigend and Reinhart Ahlrichs. Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy. Phys. Chem. Chem. Phys., 7(18):3297-3306 2005. doi: 10. 10.1039/B508541A.
Kilian Q. Weinberger and Lawrence K. Saul. Unsupervised Learning of Image Manifolds by Semidefinite Programming. Int. J. Comput. Vision, 70(1):77-90, 2006. doi: 10.1007/ si 1263-000-49939-z
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovee. Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. In S. Bengio, H. Wallach, and R. Gamett (eds.), Advances in Neural Infor-
H. Larochelle, K. Grauman, N. mation Processing Systems 31, pp. 6410-644 6421. Curran Associates, Inc., 2018.
12

---

Under review as a conference paper at ICLR 2020
A APPENDIX
A.1 CONF17 BENCIMARR
A.1.1 DATA GENERATION
The IS017 dataset (Schütt al. 2017a) was processed in the following way. First, conformations in which the molecular connectivity was modified (i.e., bonds were broken or new ones are formed) were discarded. For this, the tool XYZ2MOL (Jensen 2019) was employed. Second, the molecular graphs were augmented by adding auxiliary edges for reasons described in Section 2. Auxiliary edges between all second neighbors were added. This can lead system's s geometry, however, this did not pose problem in our experiments. In addition, auxiliary edges between third neighbors were added to fix dihedral angles. Since there are potentially many ways of specifying dihedral angle in molecular system, resorted to the works of Riniker & Landrum (2155) and Guba et al. (2016) to decide where to place edges between third neighbors.
slight over-spectfication of the
A.1.2 INPUT FEATURES
Below list the node and edges features in the CONF17 benchmark.
Table 3: Node features.
Feature
Data Type
Dimension
atomic number chiral tag
integer
one-hot (R, S, and N/A) 3
Table 4: Edge features.
Feature
Data Type
Dimension
kind
one-hot (indicating whether is in Exond- L'agle or Edihedmal one-hot (E, Z, Any, None, and N/A) integer (single, double, triple or N/A)
3
stereo chemistry
5
type
is aromatic
binary
1
is conjugated is in ring of size
binary
1
one-hot (3, ......9 9) and N/A
8
A.1.3 MODEL ARCHITECTURE
The full model is available In following, the hyperparameters of our model are specified:
Activations throughout this paper: ReLU; L.,, Le: 10; Fene,vi neural network with depth 2, width 20; Fenc,e: neural network with depth 3, width 60; Face.v Fdec,e: neural networks with depth 2, width
70; {MP(t) 12' [MP y7 MPNN width depth 1 and three multi-head attention heads, T 3, for node and edge updates neural networks with depth 2, and width 70 were used. Rene. Rassi neural networks with depth 2, width 70. Batch size; 16 (conformations);
A.2 COMPUTATIONAL DETAILS
A.2.1 QUANTUM-CHEMIN CALCULATIONS
All chemical caleulations carried out with the PySCF program package (version 1.5) (Sun et al. (018)) employing the exchange-correlatinn correlation density functional PBE (Perdew et al. 1996), and the deP2-SVP (Weigend & Ahlrichs 2005 Weigend 2006) basis set.
13

---

Under review as a conference paper at ICLR 2020
Conformations generated by DL4CHEM did not succeed some atoms too close to each other. Self-consistent field algorithms in quantum-chemical chemiccal software such as PySCF do not converge for such molecular structures.
With quantum-cheneee chemmmca methods, we calculate several properties that concern the states the elec- trons in the conformation. These are the total electronic energy Eetec- the energy of the electron in the highest occupied molecular orbital (HOMO in eV) <HOMO- the energy of the lowest unoccupied molecular orbital (LUMO in eV) ELUMO, and the norm the dipole moment (in debye).
A.2.2 EUCLIDEAN DISTANCE GEOMETRY
We refer the reader Havel (0022) for theory on EDG, algorithms, and chemical applications. In summary, the EDG procedure consists of the following three steps:
Bound smoothing: extrapolating a complete set of lower and upper limits on all the dis- tances from the sparse set of lower and upper bounds.
2. Embedding: choosing random distance matrix from within these limits, and computing coordinates that are certain best-fit to the distances.
3. Optimization: optimizing these coordinates versus an error function which measures the total violation of the distance (and chirality) constraints.
We the EDG implementation found in RDKIT Riniker & Landrum 2015) with default settings.
A.3 GENERATION OF CONFORMATIONS
Ground-truth
RDKit
DL4Chem
Graph
Figure 7: Overlay of 50 conformations from the ground-truth, RDKIT, DL4CHEM, and GRAPHIDG based on two random molecular graphs from the test set. C, O, and H atoms are colored gray, red, and white, respectively.
A.4 DISTRIBUTIONS OVER DISTANCES
Below, the marginal distributions of the distances for variety of molecular graphs shown.
14

---

Under review as a conference paper at ICLR 2020
1000
Count
500
1.25
1.50
1.751
1.0
1.5
1.0
1.5
2
1.0
1.5
1.5
2.0
2
1000
Count
500
1.0
1.5
2
2.25
2.252.502.755
2
1.5
2.0
2.5
1000
Count
500
1.5
2.0
2.5
2.0
2.5
1
3.1
2.252.502.75
2.0
2.5
1000
Count
Ground-truth
500
RDKit
DL4Chem
GraphDG
2.0
2.5
2.5
3.0
Figure 8: Marginal distributions p(dg) ()) of ground and predicted distances (in A) between C and O atoms given molecular graph from the test set. The atoms connected by each edge dg are indicated in each subplot (8t-)k) In the 3D structure of the molecule, carbon and oxygen atoms colored gray and red, respectively. H atoms are omitted for clarity.
15

---

Under review as a conference paper at ICLR 2020
1-2
1-6
1-8
2-3
3-4
4-5
4-7
7-8
100
15
14
10
15
15
2.0
10
150
150
125
125
1.0
15
10
10
Ground-truth
RDKit
DL4Chem
GraphDG
15
2.
70
70
5.6
10
15
Figure 9: Marginal distributions d,d;, of ground-truth and predicted distances for a molecular graph from the test set (in A). Here, d; and d, are restricted edges representing bonds between and O atoms. In the 3D structure of the molecule, carbon and oxygen atoms are colored gray and red, respectively. atoms are omitted for clarity.
16

---

Under review as a conference paper at ICLR 2020
1000
Count
500
1000
7-8
Count
500
1000
1-3
1-5
1-6
3-6
S
1-8
Count
500
2
2
2.002.25
1.52.2..5
1.5 2.0
2
2.5
3.0
1000
2-6
2-8
Count
500
Ground-truth
RDKit
DL4Chem
GraphDG
2
2
Figure 10: See caption of Fig.
g
17

---

Under review as a conference paper at ICLR 2020
1-2
1-4
1-7
2-3
2-7
3-4
4-5
6-7
7-8
15
20
155
20
15
20
2515
20
25
10
15
10
15
10
2.
10
15
16
16
14
14
20
20
15
20
25
20
20
Ground-truth
155
15
23
RDKit
2.0
2.
DL4Chem
GraphDG
1.0 15
48
8
7
1.
15
10
1.0
155
Figure 11: See caption of Fig.
9
18

---

Under review as a conference paper at ICLR 2020
3-4
3-7
4-5
Count
500
1.25
1.50 1.751
1.0
1.5
1.0
1.5
2
1.0
1.5
1.5
2.0
2
1000
4-7
0-2
Count
500
1.0
1.5
2
2.25
2.25
1
2
1.5
2.0
2.5
1000
Count
500
1.5
2.0
2.5
2.0
2.5
2
1
1
3 1
2
2.252.502.75
2.0
2.5
1000
3-8
Count
Ground-truth
500
RDKit
DL4Chem
GraphDG
2.0
2.5
2.5
3.0
2
Figure 2:: See caption of Fig.
g
19

---

Under review as a conference paper at ICLR 2020
1-2
1-6
1-7
2-3
3-4
3-8
4-5
5-6
6-7
7-B
150
25
Ground-truth
RDKit
DL4Chem
GraphDG
05
10
15
15
15
6
1
7
5
2
4
3
05 10
15
Figure 13: See caption of Fig.
9
20

---

Under review as a conference paper at ICLR 2020
3-4
3-7
4-5
Count
500
1.25
1.50 1.751
1.0
1.5
1.0
1.5
2
1.0
1.5
1.5
2.0
2
1000
4-7
0-2
Count
500
1.0
1.5
2
2.25
2.25..502.72
1
2
1.5
2.0
2.5
1000
Count
500
1.5
2.0
2.5
2.0
2.5
2
1
1
3 1
2
2.252.502.75
2.0
2.5
1000
3-8
Count
Ground-truth
500
RDKit
DL4Chem
GraphDG
2.0
2.5
2.5
3.0
2
Figure 14: See caption of Fig.
g
21

---

Under review as a conference paper at ICLR 2020
1-2
1-6
1-8
2-3
3-4
3-5
3-7
5-6
6-7
7-8
125153L
15
20
25
10
15
15
20
25
3:05
10 15
125 1300
10
175
175
150
150
1.5
125
2.0
20
Ground-truth
RDKit
15
15
DL4Chem
1.
10
GraphDG
15
29
85
2.0
70
65 1.
3
15
15
0,
05
150
150
150
125
125
Figure 15: See caption of Fig.
9
22


Reviews:
[{"review_id": "SkgzPOBuYB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nIn this paper, a new method for estimating the equilibrium measure of molecular compounds is proposed and tested on a new (enriched) version of an existing data set (CONF17 is derived from the pre-existing ISO17, itself extracted from QM9). \nThe method takes as input a collection of graph structures of a given stoichiometry mixture, and learns the corresponding conformations.\nThis results in a generative model, taking as input a graph structure (or actually a single conformation instance, thus setting the stoichiometry AND the isomer/graph structure), to generate (in a one-shot fashion) other conformations of that isomer.\nThe distribution of conformations sampled in this way compare well with those of an extensive training set (CONF17, set of isomers and conformations of a single compound), and in particular they perform better than 2 alternative methods published recently (2015 and 2019, the former performing globally much better than the latter!).\n\n\nConcretely, the model/method/algorithm consists in a combination of existing network architectures.  The main original contribution is to map the Cartesian coordinates of atoms into a pairwise distance space, which is naturally translationally and rotationally invariant.  Some of the second and third neighbors distances in the graph structure are considered, such that the conformation is completely set (this would not be the case if considering only 1st nearest neighbors distances).\nThe precise combination of architectures is explained in section 2.2. This explanation is a little fast for me (but I am new to this field). \nIn that section, it could be nice to precise that the job of the CVAE is to learn (among others) the parameters mu,sigma of q_phi(z|d,G), using a batch of conformations of a given isomer (and then generalize to other isomers, each time guessing the correct mu,sigma parameters, even for a new isomer).\nThis is not obvious for the inexperienced reader (I hope I \"guessed\" correctly).\n\nThe \"new dataset\" shared in this work is indeed useful for reproducing results and for later comparison with other works which aim at sampling conformations.\nHowever, it derives directly from ISO17, which is itself an extraction from QM9. ISO17 lists the Cartesian coordinates of 127 isomers of C7,H10,O2 (each having ~3000 conformations).  The \"new\" CONF17 is novel only in that it provides these conformations in distance-matrix space, i.e. it sets which pairs of atoms need to have their distances computed.  This is very useful because this choice of pairs is not unique, and so setting it once is necessary for comparison.  I would not stress too much that this is a new dataset: at first it actually sounds like authors chose a dataset such that their method outperforms others'!  Which turns out not to be the case (they chose C7,H10,O2, but this doesn't look like fine-tuning).  Indeed, their sharing of this augmented version of ISO17 is absolutely necessary: I would simply put it differently, stating from the beginning that they share an augmented version of a pre-existing data set.\n\nAbout the experimental protocol/methodology:\nthe work carried out seems quite impressive. Indeed, the paper presents a rather extensive comparison of the new model with 2 other ones, on a large data set.\nHowever sometimes the presentation rushes a bit towards results, in the sense that it is not easy to figure out over which ensemble the mean/median/max/min has been computed. This is related to the intrinsic complexity of estimating accuracy, given the encapsulation (compound-> isomer-> conformations-> each bond-> distribution over bond properties).  I think however it could be made clearer, and this would benefit the paper greatly.\nA deeper comment: in the field of conformation sampling, there are two issues. One is to correctly sample states in each local minimum of the free energy. This is addressed in the paper. The second issue is to hunt for all possible local minima of the isomer, i.e. sample even unlikely but crucial conformations. This is not addressed here.  For computing average energy, dipole moment etc, the most likely conformations will dominate, most of the time.  However given the topic I think it would be fair to mention this point, and explain why in this specific case, the second issue is not crucial.\nBecause of this, also, I think \n\nOverall, the paper is well written and is very well situated within the literature (including the very recent literature).\nIt states clearly its contributions and results, and provides detailed presentation of experimental results.\nI think it contributes to the field and provides an interesting new way of dealing with the problem of molecular conformations sampling.  The results obtained are much better than those of the compared ML method (RDkit).\nHowever, some claims (in introduction or in section 3) are stated more strongly than they are supported by direct experimental evidence.\n- ''... the first work on sampling molecular conformations for molecules of arbitrary size and shape based on deep learning.''\nHowever, the experiments have been performed only on isomers of  C7,H10,O2.\nFurthermore, there is no discussion on the computational cost of the method at all, so its extendibility to larger shapes is not obvious.\n- '' We create a new, challenging benchmark dataset CONF 17 for conformation generation, which is made publicly available.''\nI made remarks on this claim above.'' \n- '' We develop a rigorous experimental approach for evaluating and comparing the accuracy of conformation generation methods based on the mean maximum deviation distance metric.''\nHowever the paper's methodology doesn't address the issue of ''hunting for new local minima''. \nThe data set could be enriched also with a labeling of ''which conformation is in which local minimum of free energy''\n- ''It is a fast alternative to resource-intensive approaches based on MCMC or MD.''\nAs there was no mention of speed of computation in the paper (although we can guess it is faster), this is unsupported. However it is easy to remedy, simply give an order of magnitude of sampling time (for thousands of conformations of 1 isomer).\n- ''Our principled representation based on pair-wise distances does not restrict our approach to any type of system (e.g., proteins) or any particular graph structure. In addition, it is extendable in a systematic fashion.''\nThis suggests that the method would also work for dense packings. However this is not shown in the experiments.\nProbably I misunderstood the claim, and it only refers to the ability to tackle general molecular compounds, not just linear ones?  However proteins are not always linear, but can present cycles, and other branch-style patterns, so the claim is not very clear to me.\n\nBecause some of the important claims are only weakly supported, and because of the relatively small novelty of the method, I lean on rejecting the paper.\n\n\n\nAdditional comments (for improving the paper):\n\nThere is no discussion of hyper-parameter tuning, or any detail of the intermediate GNN that constitute the encoder and decoder in the CVAE. \nSo it was either not done at all, and then it should be mentioned; or it was done and it should be discussed quickly.\nIn particular, in section 2.2 there is no discussion on the impact of the latent space' dimension, here set to Nv, i.e. 7+2+10=19 (correct?).  I guess there is a balance between accuracy and generalizability, a short discussion on this point would be interesting.\n\nThe paper should clarify the hierarchy of molecular configurations:\nfor a given composition (%C, %O, %H, etc) (here, fixed throughout the paper at C7,H10,O2)\nthere are several structures possible (graphs or isomers),\nfor a given graph (isomer), there can be a couple of chiral variants (mirroring of (pieces of) the molecule,\nfor each chiral variant, there are conformations, which correspond to rather weak variations of the Cartesian coordinates, due to thermal fluctuations. Some of these variations can be important, e.g. when a bond can freely rotate. Some correspond to bending or torsion, etc, of the molecule.\nThe algorithm presented in the paper attempts to sample appropriately the equilibrium distribution of conformations for each given graph structure (at fixed chirality, if I understood correctly).\n\nAdd a comment on why (in your opinion) the distributions are not exactly correctly sampled, which after all is the goal (since the initial graph structure is given as input here).\n\n\nAdditional question, for curiosity. In section 2.3: only a single molecule output is taken from each distance matrix.\nCould you comment on the possibility to sample several Cartesian coordinates, for each distance matrix ?  It seems like several solutions should (often) exist. \nSampling continuously from that distribution would surely provide a more continuous (rich) output (am I correct) ?\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "HyxVxQy0FH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe authors propose a generative model designed for molecules, which is essentially a conditional variational auto-encoder. The model learns to generate, when conditioning on a molecule graph, the distribution of distances between each of the atoms and its second and third neighbour. Finally, using these information new molecules can be generated, which satisfy the generated distances. In the experiments the authors show the effectiveness of the method.\n\nGeneral Comments:\nThe paper is ok written, but as a non-expert of the field, I find it quite hard to follow. More specifically, I think that the main content aims to readers which are very familiar with the problem of molecule modeling, which is of course understandable. However, I find the technical (machine learning) content a bit unclear, because I think that the proposed method is not presented properly in details (see comments):\n\n1. I think that authors should specify the dimensionalities of each variable and function in the paper. Unfortunately, none of them is clearly defined, which makes the technical part a bit hard to follow.\n\n2. As a non-expert of the filed, I would like to know what exactly is a conformation x? Does it refer to the Euclidean coordinates of each node (atom) v of the graph? (Related to comment #1.)\n\n3. For each G, there are multiple possible conformations x_i, where each x_i specifies the Euclidean coordinates of all the nodes (atoms) v_j? Also, each node v_j has some attributes? So the goal is to represent each conformation x_i -implicitly- as a set of distances between the nodes v_j? I think that Sec 2.1. can be more clear about the definition of the data, such that to be accessible from non experts in molecule modeling. (Related to comment #1.)\n\n4. I think that the definition of the model in Sec 2.2. could have been better. As before, I believe that it is hard to understand explicitly what the functions input and output is, since the dimensionalities are not defined. In my opinion, it would have been better if instead of this high level text you could provide explicitly all the steps of the model e.g. the concatenations, the appends, etc.\n\n5. I could not understand if and how the model is able to handle graphs with different number of nodes, and different number of distances among them.\n\n6. The experimental section seems solid enough, and shows that the proposed approach works better than the other methods. Also, I like the fact that some of the limitations of the method are stated. Moreover, the related work seems to be properly included. However, since I am not an expert of the field, I am not able to provide precise feedback for these two parts.\n\nIn general, I think that the proposed model solves good enough the problem that is designed for. Also, in my opinion, the manuscript aims for specialized readers, which is definitely understandable. However, my main concern regarding the current version, is that the technical (machine learning) content is a bit unclear and probably too high level. In case the technical content was presented properly, I think that it would have been a much better fit for this community.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "HyxOGi_D5H", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a generative model for generating molecule with three dimensional structure. Given a graph, it leverages a variational autoencoder to embed the distance between two atoms into latent vectors (encoder) and then generate distance between two atoms based on the latent vector. Even if the problem is well motivated, I have several concerns:\nI. What is the variable \"x\"? Is it possible to define a bit formally? It appears that x=(G,d) i.e. the graph with d.\nII. It would be nice to develop the generative model of (G,d) i.e. both G and d. Simply generating d looks a bit restrictive. \nIII. How do you model/learn \\mathcal{O}(x)?\n\nI would be curious to know the answer of these questions.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a solution to generating molecule with three dimensional structure by learning a low-dimensional manifold that preserves the geometry of local atomic neighborhoods based on Euclidean distance geometry. 

The application is interesting and the proposed solution is reasonable. The authors did a good job at addressing most concerns raised in the reviews and updating the draft. 

Two main concerns were left unresolved: one is the lack of novelty in the proposed model, and the other is that some arguments in the paper are not fully supported. The paper could benefit from one more round of revision before being ready for publication. 


Decision: Reject