Title: A GENERATIVE MODEL FOR MOLECULAR DISTANCE GEOMETRY Anonymous authors Paper under double-blind review

Abstract: Computing equilibrium states for many-body systems, such as molecules, is a long-standing challenge. In the absence of methods for generating statistically independent samples, great computational effort is invested in simulating these systems using, for example, Markov chain Monte Carlo. We present a probabilistic model that generates such samples for molecules from their graph representations. Our model learns a low-dimensional manifold that preserves the geometry of local atomic neighborhoods through a principled learning representation that is based on Euclidean distance geometry. In a new benchmark for molecular conformation generation, we show experimentally that our generative model achieves state-ofthe-art accuracy. Finally, we show how to use our model as a proposal distribution in an importance sampling scheme to compute molecular properties.

1. INTRODUCTION:

Over the last few years, many highly-effective deep learning methods generating small molecules with desired properties (e.g., novel drugs) have emerged (Gómez-Bombarelli et al., 2018; Segler et al., 2018; Dai et al., 2018; Jin et al., 2018; Bradshaw et al., 2019a; Liu et al., 2018; You et al., 2018; Bradshaw et al., 2019b) . These methods operate using graph representations of molecules in which nodes and edges represent atoms and bonds, respectively. A representation that is closer to the physical system is one in which a molecule is described by its geometry or conformation. A conformation x of a molecule is defined by a set of atoms {( i , r i )} Nv i=1 , where N v is the number of atoms in the molecule, i ∈ {H, C, O, ...} is the chemical element of the atom i, and r i ∈ R 3 is its position in Cartesian coordinates. Importantly, the relative positions of the atoms are restricted by the bonds in the molecule and the angles between them. Due to thermal fluctuations resulting in stretching of and rotations around bonds, there exist infinitely many conformations of a molecule. A molecule's graph representation and a set of its conformations are shown in Fig. 1 . Under a wide range of conditions, the probability p(x) of a conformation x, is governed by the Boltzmann distribution and is proportional to exp{-E(x)/k B T }, where E(x) ∈ R is the conformation's energy, k B is the Boltzmann constant, and T is the temperature.

To compute a molecular property for a molecule, one must sample from p(x). The main approach is to start with one conformation and make small changes to it over time, e.g., by using Markov chain Monte Carlo (MCMC) or molecular dynamics (MD). These methods can be used to accurately sample equilibrium states of molecules, but they become computationally expensive for larger ones (Shim & MacKerell, 2011; Ballard et al., 2015; De Vivo et al., 2016) . Other heuristic approaches exist in which distances between atoms are set to fixed idealized values (Havel, 2002; Blaney & Dixon, 2007) . Several methods based on statistical learning have also recently been developed to tackle the issue of conformation generation. However, they are mainly geared towards studying proteins and their folding dynamics (AlQuraishi, 2019) . Some of these models are not targeting a distribution over conformations but the most stable folded configuration (Evans et al., 2018; Ingraham et al., 2019) , while others are not transferable between different molecules (Lemke & Peter, 2019; Noé et al., 2019) .

This work includes the following key contributions::

• We introduce a novel probabilistic model for learning conformational distributions of molecules with graph neural networks. • We create a new, challenging benchmark for conformation generation, which is made publicly available. To the best of our knowledge, this is the first benchmark of this kind. • By combining a conditional variational autoencoder (CVAE) with an Euclidean distance geometry (EDG) algorithm we present a state-of-the-art approach for generating one-shot samples of molecular conformations for unseen molecules that is independent of their size and shape. • We develop a rigorous experimental approach for evaluating and comparing the accuracy of conformation generation methods based on the mean maximum deviation distance metric. • We show how this generative model can be used as a proposal distribution in an importance sampling (IS) scheme to estimate molecular properties.

2. METHOD:

Our goal is to build a statistical model that generates molecular conformations in a one-shot fashion from a molecule's graph representation. First, we describe how a molecule's conformation can be represented by a set of pairwise distances between atoms and why this presentation is advantageous over one in Cartesian coordinates (Section 2.1). Second, we present a generative model in Section 2.2 that will generate sets of atomic distances for a given molecular graph. Third, we explain in Section 2.3 how a set of predicted distances can be transformed into a molecular conformation and why this transformation is necessary. Finally, we detail in Section 2.4 how our generative model can be used as a proposal distribution in an IS scheme to estimate molecular properties.

2.1. EXTENDED MOLECULAR GRAPHS AND DISTANCE GEOMETRY:

In this study, a molecule is represented by an undirected graph which is defined as a tuple G = (V, E). V = {v i } Nv i=1 is the set of nodes representing atoms, where each v i ∈ R Fv holds atomic attributes (e.g., the element type i ). E = {(e k , r k , s k )} Ne k=1 is the set of edges, where each e k ∈ R Fe holds an edge's attributes (e.g., the bond type), and r k and s k are the nodes an edge is connecting.

Here, E represents the molecular bonds (and the auxiliary edges which are explained below) in the molecule.

We assume that, given a molecular graph G, one can represent one of its conformations x by a set of atomic distances d = {d k } Ne k=1 , where d k = |r r k -r s k | is the Euclidean distance between the positions of the atoms r k and s k in this conformation. As the set of edges between the bonded atoms (E bond ) alone would not suffice to describe a conformation, we expand the traditional graph representation of a molecule by adding auxiliary edges. Auxiliary edges between atoms that are second neighbors in the original graph fix angles between atoms, and those between third neighbors fix dihedral angles (denoted E angle and E dihedral , respectively). In this work, E angle consists of edges between all second neighbors in the original graph. Edges between third neighbors are added according to a heuristic (see Appendix A.1). From now on we are always referring to this extended molecular graph when talking about molecular graphs. In Fig. 2 , the process of extending the molecular graph and the extraction of d from x and G are illustrated. A key advantage of a representation in terms of distances is its invariance to rotation and translation; by contrast, Cartesian coordinates depend on the (arbitrary) choice of origin, for example. In addition, it reflects pair-wise physical interactions and their generally local nature. Auxiliary edges can be placed between higher-order neighbors depending on how far the physical interactions dominating the potential energy of the system reach.

We have a set of N G molecular graphs {G l } N G l=1 . Further, for each G l , we have S l conformational samples {x l,j } S l j=1 from the ground-truth distribution resulting in S l sets of distances {d l,j } S l j=1 . With this data, we will train a generative model which we detail in the following section.

2.2. GENERATIVE MODEL:

We employ a CVAE (Kingma & Welling, 2014; Pagnoni et al., 2018) to model the distribution over distances d given a molecular graph G. A CVAE first encodes G together with d into a latent space z ∈ R kNv , where k ∈ N + , with an encoder q φ (z|d, G). Subsequently, the decoder p θ (d|z, G) decodes z back into a set of distances. A graphical model is shown in Fig. 2 

C).:

A conformation has, in general, 3N v -6 spatial degrees of freedom (dofs): one dof per spacial dimension per atom minus three translational and three rotational dofs. Therefore, the latent space should be proportional to the number of atoms in the molecule. In addition, the latent space should be smaller than 3N v as it is the role of the encoder to project the conformation into a lower-dimensional space. As a result, we set k = 1 to avoid overfitting. 1Here, q φ (z|d, G) and p θ (d|z, G) are Gaussian distributions, the mean and variance of which are modeled by two artificial neural networks. At the center of this model are message-passing neural networks (MPNNs) (Gilmer et al., 2017) with multi-head attention (Veličković et al., 2018) . In short, an MPNN is a convolutional neural network that allows end-to-end learning of prediction pipelines whose inputs are graphs of arbitrary size and shape. In a convolution, neighboring nodes exchange so-called messages between neighbors to update their attributes. Edges update their attributes with the features of the nodes they are connecting. The MPNN is a well-studied technique that achieves state-of-the-art performance in representation learning for molecules (Kipf & Welling, 2017; Duvenaud et al., 2015; Kearnes et al., 2016; Schütt et al., 2017b; Gilmer et al., 2017; Kusner et al., 2017; Bradshaw et al., 2019a) .

In the following, we describe the details of the model. 2 In Fig. 3 , an illustration of the model is shown. In the encoder q φ (z|d, G), each d k is concatenated with the respective edge feature e k to give e k ∈ R Fe+1 . Then, each v i and each e k are passed to F enc,v and F enc,e (two multilayer perceptrons, 

EQUATION

i,enc ∈ R Lv , and e

(t) k,enc ∈ R Le . Then, T MPNNs of depth 1, {MP (t) enc } T t=1 , are consecutively applied to obtain G (T ) enc . Finally, the read-out function R enc (an MLP) takes each v (T )

i, enc to predict the mean µ zi ∈ R and the variance σ 2 zi ∈ R of the Gaussian distribution for z i . The so-called reparametrization trick is employed to draw a sample for z i . In summary,

v (0) i,enc = F enc,v (v i ), e (0) k,enc = F enc,e (e i ), G (1) enc = MP (0) enc (G (0) enc ), G (t+1) enc = MP (t) enc (G (t) enc ), G (T ) enc = MP (T -1) enc (G (T -1) enc ), µ zi , σ 2 zi = R enc (v (T )

i,enc ).

(1)

In the decoder p θ (d|z, G), each z i is concatenated with the respective node feature v i to give v i ∈ R Fv+1 . Each v i and each e k are passed to F dec,v and F dec,e (two MLPs), respectively, to give

EQUATION

i,dec ∈ R Lv , and e 

v (0) i,dec = F dec,v (v i ), e (0) k,dec = F dec,e (e i ), G (1) dec = MP (0) dec (G (0) dec ), G (t+1) dec = MP (t) dec (G (t) dec ), G (T ) dec = MP (T -1) dec (G (T -1) dec ), µ d k , σ 2 d k = R dec (e (T )

k,dec ).

(

EQUATION

The sets of parameters in the encoder and decoder, φ and θ (i.e., parameters in F enc,v , F enc,e ,

EQUATION

dec } T t=1 , R dec ), respectively, are optimized by maximizing the evidence lower bound (ELBO):

EQUATION

where the prior p θ (z|G) consists of factorized Gaussians. The optimal values for the hyperparameters for the network dimensions, number of message passes, batch size, and learning rate of the Adam optimizer (Kingma & Ba, 2014) were tuned by maximizing the validation performance (ELBO) with a Bayesian optimizer and are reported in Appendix A.1.3.

2.3. CONFORMATION GENERATION THROUGH EUCLIDEAN DISTANCE GEOMETRY:

To compute molecular properties, quantum-chemical methods need to be employed which require the input, i.e., the molecule, to be in Cartesian coordinates. 3 Therefore, we use an EDG algorithm to translate the set of distances {d k } Ne k=1 to a set of atomic coordinates {r i } Nv i=1 .4 EDG is the mathematical basis for a geometric theory of molecular conformation. In the field of machine learning, Weinberger & Saul (2006) used it for learning image manifolds, Tenenbaum et al. (2000) for image understanding and handwriting recognition, Jain & Saul (2004) for speech and music, and Demaine et al. (2009) for music and musical rhythms. An EDG description of a molecular system consists of a list of lower and upper bounds on the distances between pairs of atoms {(d k,min , d k,max )} Ne k=1 . Here, p θ (d|z, G) is used to model these bounds, namely, we set the bounds to {(µ

d k -σ d k , µ d k + σ d k )}

, where µ d k and σ d k are the mean and standard deviation for each distance d k given by the CVAE. Then, an EDG algorithm determines a set of Cartesian coordinates {r i } Nv i=1 so that these bounds are fulfilled (see Appendix A.2 for details). 5 Together with the corresponding chemical elements { i } Nv i=1 , we obtain a conformation x.

2.4. CALCULATION OF MOLECULAR PROPERTIES:

We can get an MC estimate of the expectation E G [O] of a property O (e.g., the dipole moment) for a molecule represented by G by drawing conformational samples x i ∼ p(x|G) and computing O(x i ) ∈ R with a quantum-chemical method (e.g., density functional theory). Since we cannot draw samples from p(x|G) directly, we employ an IS integration scheme (Bishop, 2009) with our CVAE as the proposal distribution. We assume that we can readily evaluate the unnormalized probability of a conformation p(x|G) = exp{-E(x)/k B T }, where x must be a conformation of the molecule and the energy E(x) is determined with a quantum-chemical method. Since the EDG algorithm is mapping the distribution p θ (d|z, G) to a point mass in R 3Nv , the MC estimate for the resulting distribution p prop (x|G) is given by a mixture of delta functions, each of which is centered at the x i resulting from mapping p θ (d|z i , G) to R 3Nv , where

z i ∼ p θ (z|G), that is, p prop (x|G) ≈ 1 N N i=1 δ(x -x i ).

The IS estimator for the expectation of O w. r. t. p(x|G) then reads

EQUATION

where x i ∼ p(x i |G) and x i ∼ p prop (x i |G), so that the expectation of O w. r. t. the normalized version of p(x) is then

EQUATION

where Z ≈ N i=1 p(x i ) and N is the number of samples. When dividing two delta functions we have assumed that they take some arbitrarily large finite value.

3. RELATED WORKS:

The standard approach for generating molecular conformations is to start with one, and make small changes to it over time, e.g., by using MCMC or MD. These methods are considered the gold standard for sampling equilibrium states, but they are computationally expensive, especially if the molecule is large and the Hamiltonian is based on quantum-mechanical principles (Shim & MacKerell, 2011; Ballard et al., 2015; De Vivo et al., 2016) .

A much faster but more approximate approach for conformation generation is EDG (Havel, 2002; Blaney & Dixon, 2007; Lagorce et al., 2009; Riniker & Landrum, 2015) . Lower and upper distance bounds for pairs of atoms in a molecule are fixed values based on ideal bond lengths, bond angles, and torsional angles. These values are often extracted from crystal structure databases (Allen, 2002) . These methods aim to generate a low-energy conformation, not to generate unbiased samples from the underlying distribution at a certain temperature.

There exist several machine learning approaches as well, however, they are mostly tailored towards studying protein dynamics. For example, Noé et al. (2019) trained Boltzmann generators on the energy function of proteins to provide unbiased, one-shot samples from their equilibrium states. This is achieved by training an invertible neural network to learn a coordinate transformation from a system's configurations to a latent space representation. Further, Lemke & Peter (2019) proposed a dimensionality reduction algorithm that is based on a neural network autoencoder in combination with a nonlinear distance metric to generate samples for protein structures. Both models learn protein-specific coordinate transformations that cannot be transferred to other molecules. AlQuraishi ( 2019) introduced an end-to-end differentiable recurrent geometric network for protein structure learning based on amino acid sequences. Also, Ingraham et al. (2019) proposed a neural energy simulator model for protein structure that makes use of protein sequence information. In contrast to amino acid sequences, molecular graphs are, in general, not linear but highly branched and often contain cycles. This makes them unsuitable for recurrent networks.

Finally, Mansimov et al. (2019) presented a conditional deep generative graph neural network to generate molecular conformations given a molecular graph. Their goal is to predict the most likely conformation and not a distribution over conformations. Instead of encoding molecular environments in atomic distances, they work directly in Cartesian coordinates. As a result, the generated conformations showed significant structural differences compared to the ground-truth and required refinement through a force field, which is often employed in MD simulations.

We argue that our model has several advantages over the approaches reviewed above:

• It is a fast alternative to resource-intensive approaches based on MCMC or MD.

• Our principled representation based on pair-wise distances does not restrict our approach to any particular molecular structure.

• Since our model employs message-passing neural networks, it is transferable -it can extrapolate from only a few graphs to unseen ones.

4. THE CONF17 BENCHMARK:

The CONF17 benchmark is the first benchmark for molecular conformation sampling. 6 It is based on the ISO17 dataset (Schütt et al., 2017a) which consists of conformations of various molecules with the atomic composition C 7 H 10 O 2 drawn from the QM9 dataset (Ramakrishnan et al., 2014) . These conformations were generated by ab initio molecular dynamics simulations at 500 Kelvin which generates trajectories of a single molecule covering a large variety of conformations. The CONF17 benchmark consists of 127 distinct molecular graphs each with 3380 conformations on average. We split this dataset into multiple training and test splits, each consisting of 107 and 20 graphs, respectively (see Appendix A.1 for more details).7 

In Fig. 4 , (A), the structural formulae of a random selection of molecules from this benchmark are shown. Most molecules feature highly-strained, complex 3D structures such as rings which are typical of drug-like molecules. It is thus the structural complexity of the molecules, not their number of degrees of freedom, that makes this benchmark challenging. In 

5. EXPERIMENTS:

We assess the performance of our method, named Graph Distance Geometry (GRAPHDG), by comparing it with two state-of-the-art methods for molecular conformation generation: RDKIT (Riniker & Landrum, 2015) , a classical EDG approach, and DL4CHEM (Mansimov et al., 2019) , a machine learning approach. We trained GRAPHDG and DL4CHEM on three different training and test splits of the CONF17 benchmark using Adam (Kingma & Ba, 2014) . We generated 3000 conformations with each method for molecular graphs in a test set.

5.1. DISTRIBUTIONS OVER DISTANCES:

We assessed the accuracy of the distance distributions of RDKIT, DL4CHEM, and GRAPHDG by calculating the maximum mean discrepancy (MMD) (Gretton et al., 2012) to the ground-truth distribution. We compute the MMD using a Gaussian kernel, where we set the standard deviation to be the median distance between distances d in the aggregate sample. For this, we determined the distances in the conformations from the ground-truth and those generated by RDKIT and DL4CHEM.

For each train-test split and each G in a test set, we compute the MMD of the joint distribution of distances between C and O atoms (H atoms are usually ignored), the MMDs of pair-wise distances p(d i , d j |G), and the MMDs between the marginals of individual distances p(d i |G). We aggregate the results of three train-test splits, and, finally, compute the median MMDs and average rankings.

The results are summarized in Table 1 . It can be seen that the samples from GRAPHDG are significantly closer to the ground-truth distribution than the other methods. RDKIT is slightly worse than GRAPHDG while DL4CHEM seems to struggle with the complexity of the molecules and the small number of graphs in the training set.

In Fig. 5 , we showcase the accuracy of our model by plotting the marginal distributions p(d i |G) for distances between C and O atoms given a molecular graph from a test set. It can be seen that RDKIT consistently underestimates the marginal variances. This is because this method aims to predict the most stable conformation, i.e., the distribution's mode. In contrast, DL4CHEM often fails to predict the correct mean. For this molecule, GRAPHDG is the most accurate, predicting the right mean and variance in most cases. Additional figures can be found in the Appendix A.4, where we also show plots for the marginal distributions p(d i , d j |G). 

5.2. GENERATION OF CONFORMATIONS:

We passed the distances from our generative model to an EDG algorithm to obtain conformations. For 99.9% of the sets of distances, all triangle inequalities held. For 94% of the molecular graphs, the algorithm succeeded which is 8 pp higher than the success rate we observed for RDKIT. For each molecular graph in a test set, we generated 50 conformations with each method. This took DL4CHEM, RDKIT, and GRAPHDG on average around hundreds of milliseconds per molecule. 8In contrast, a single conformation in the ISO17 dataset takes around a minute to compute. In Fig. 6 , an overlay of these conformations of six molecules generated by the different methods is shown. It can be seen that RDKIT's conformations show too little variance, while DL4CHEM's structures are mostly invalid, which is due in part to its failure to predict the correct interatomic angles. Our method slightly overestimates the structural variance (see, for example, Fig. 6 , top row, second column), but produces conformations that are the closest to the ground-truth.

We estimate expected molecular properties for molecular graphs from the test set with N = 50 conformational samples each. Due to their poor quality, we could not compute properties O(x), including the energy E(x), for conformations generated with DL4CHEM, and thus, this method is excluded from this analysis. In Table 2 , it can be seen that RDKIT and GRAPHDG perform similarly well (see Appendix A.2 for computational details). However, both methods are still highly inaccurate for E elec (in practice, an accuracy of less than 5 kJ/mol is required). Close inspection of the conformations shows that, even though GRAPHDG predicts the most accurate distances overall, the variances of certain strongly constrained distances (e.g., triple bonds) are overestimated so that the energies of the conformations increase drastically.

6. LIMITATIONS:

The first limitation of this work is that the CVAE can sample (with low probability) invalid sets of distances for which there exists no 3D structure. Second, the CONF17 benchmark covers only a small portion of chemical space. Finally, a large set of auxiliary edges would be required to capture long-range correlations (e.g., in proteins). Future work will address these points.

7. CONCLUSIONS:

We presented GRAPHDG, a transferable, generative model that allows sampling from a distribution over molecular conformations. We developed a principled learning representation of conformations that is based on distances between atoms. Then, we proposed a challenging benchmark for comparing molecular conformation generators. With this benchmark, we show experimentally that conformations generated by GRAPHDG are closer to the ground-truth than those generated by other methods. Finally, we employ our model as a proposal distribution in an IS integration scheme to estimate molecular properties. While orbital energies and the dipole moments were predicted well, a larger and more diverse dataset will be necessary for meaningful estimates of electronic energies. Further, methods have to be devised to estimate how many conformations need to be generated to ensure all important conformations have been sampled. Finally, our model could be trained on conformational distributions at different temperatures in a transfer learning-type setting. 

:

Experiments showed that our model performs similarly with a latent space of R

2Nv and overfits with latent spaces of R 3Nv .2 The full model (including all parameters) is available online https://figshare.com/s/ 1b42bf865bd78c457354

Even though quantum-chemical methods require the input to be in Cartesian coordinates, calculated properties, such as the energy, are invariant under translation and rotation.

4 There are additional constraints due to chirality. However, since they are given by G and are fixed, they are not modeled by our

method.5 Often there exist multiple solutions for the same set of bounds. As the bounds are generally tight, the solutions are very similar. Therefore, we only generate one set of coordinates per set of bounds.

Datasets such as the one published byKanal et al. (2018) only include conformers, i.e., the stable conformations of a molecule, and not a distribution over conformations.

7 The CONF17 benchmark is publicly available online https://figshare.com/s/ 1b42bf865bd78c457354.

All simulations were carried out on a computer equipped with an i7-3820 CPU and a GeForce GTX 1080 Ti GPU.

Figures and Tables:

Figure 1: Standard graph representation of a molecule (left) with a set of possible conformations {x i } (right). Hydrogen (H), carbon (C), and oxygen (O) atoms are colored white, gray, and red, respectively. Conformations feature the same atom types and bonds but the atoms are arranged differently in space. These differences arise from rotations around and stretching of bonds in the molecule.

Figure 2: A) The structural formula of a molecule is converted to an extended molecular graph G consisting of nodes representing atoms (circles, e.g., v 1 ) and edges representing molecular bonds (solid lines, e.g., e 1 ∈ E bond ) and auxiliary edges (dotted lines, e.g., e 2 ∈ E angle and e 3 ∈ E dihedral ). B) The distances d are extracted from a conformation x based on the edges E. C) Graphical model of the variational autoencoder: generative model p θ (d|z, G)p θ (z|G) (solid lines) and variational approximation q φ (z|d, G) (dashed lines).

Figure3: The molecular graph G together with the distances d are passed through the model consisting of an encoder q φ (z|d, G) and a decoder p θ (d|z, G). See the main text for details.

∈ R Le . Then, T MPNNs of depth 1, {MP (t) dec } T t=1 , are consecutively applied to obtain G (T ) dec . Finally, the read-out function R dec (an MLP) takes each e (T ) k, dec to predict the mean µ d k ∈ R and the variance σ 2 d k ∈ R of the Gaussian distribution for d k . In summary,

Figure 4: Overview of the CONF17 benchmark. (A) Structural formulae of a random selection of molecules. (B) Distribution of distances (in Å) grouped by edge (from left to right: E bond , E angle , and E dihedral ) and vertex type (chemical element).

Figure 5: Marginal distributions p(d k |G) of ground-truth and predicted bond distances (in Å) between C and O atoms given a molecular graph from the test set. The atoms connected by each edge d k are indicated in each subplot (s k -r k ). In the 3D structure of the molecule, carbon and oxygen atoms are colored gray and red, respectively. H atoms are omitted for clarity.

Figure 6: Overlay of 50 conformations from the ground-truth and three models based on six random molecular graphs from the test set. C, O, and H atoms are colored gray, red, and white, respectively.

Figure 8: Marginal distributions p(d k |G) of ground-truth and predicted distances (in Å) between C and O atoms given a molecular graph from the test set. The atoms connected by each edge d k are indicated in each subplot (s k -r k ). In the 3D structure of the molecule, carbon and oxygen atoms are colored gray and red, respectively. H atoms are omitted for clarity.

Figure 9: Marginal distributions p(d i , d j |G) of ground-truth and predicted distances for a molecular graph from the test set (in Å). Here, d i and d j are restricted to edges representing bonds between C and O atoms. In the 3D structure of the molecule, carbon and oxygen atoms are colored gray and red, respectively. H atoms are omitted for clarity.

Assessment of the accuracy of the distributions over conformations generated by three models compared to the ground-truth. We compare the distributions with respect to the marginals p(d k |G), p(d k , d l |G), and the distribution over all edges between C and O atoms p({d k }|G). Two different metrics are used: median MMD between ground-truth conformations and generated ones, and mean ranking (1 to 3) based on the MMD. Reported are the results for molecular graphs in a test set from three train-test splits. Standard errors are given in brackets.

Table content: <table><tr><td/><td/><td>Median MMD</td><td/><td>Mean Ranking</td></tr><tr><td/><td>RDKIT</td><td>DL4CHEM GRAPHDG</td><td>RDKIT</td><td>DL4CHEM GRAPHDG</td></tr><tr><td>p(d k |G)</td><td colspan="4">0.55 (0.01) 1.11 (0.01) 0.38 (0.02) 1.71 (0.03) 2.74 (0.02) 1.51 (0.03)</td></tr><tr><td colspan="5">p(d k , d l |G) 0.53 (0.01) 1.09 (0.01) 0.34 (0.01) 1.66 (0.02) 2.92 (0.01) 1.43 (0.02)</td></tr><tr><td colspan="5">p({d k }|G) 0.60 (0.01) 1.07 (0.03) 0.44 (0.05) 1.58 (0.05) 2.90 (0.05) 1.45 (0.02)</td></tr></table>

Median difference in average properties between ground-truth and RDKIT and GRAPHDG: total electronic energy E elec (in kJ/mol), the energy of the HOMO and the LUMO LUMO and LUMO , respectively (in eV), and the dipole moment µ (in debye). Reported are the results for molecular graphs from the test set, averaged over three train-test splits. Standard errors are given in brackets.

Table content: <table><tr><td/><td>RDKIT</td><td>GRAPHDG</td></tr><tr><td>E elec</td><td colspan="2">42.7 (4.3) 58.0 (21.0)</td></tr><tr><td>HOMO</td><td colspan="2">0.08 (0.04) 0.10 (0.05)</td></tr><tr><td>LUMO</td><td colspan="2">0.15 (0.03) 0.09 (0.05)</td></tr><tr><td>µ</td><td colspan="2">0.29 (0.05) 0.33 (0.09)</td></tr></table>

Peer review (iclr2020):
In this paper, a new method for estimating the equilibrium measure of molecular compounds is proposed and tested on a new (enriched) version of an existing data set (CONF17 is derived from the pre-existing ISO17, itself extracted from QM9). The method takes as input a collection of graph structures of a given stoichiometry mixture, and learns the corresponding conformations. This results in a generative model, taking as input a graph structure (or actually a single conformation instance, thus setting the stoichiometry AND the isomer/graph structure), to generate (in a one-shot fashion) other conformations of that isomer. The distribution of conformations sampled in this way compare well with those of an extensive training set (CONF17, set of isomers and conformations of a single compound), and in particular they perform better than 2 alternative methods published recently (2015 and 2019, the former performing globally much better than the latter!). Concretely, the model/method/algorithm consists in a combination of existing network architectures. The main original contribution is to map the Cartesian coordinates of atoms into a pairwise distance space, which is naturally translationally and rotationally invariant. Some of the second and third neighbors distances in the graph structure are considered, such that the conformation is completely set (this would not be the case if considering only 1st nearest neighbors distances). The precise combination of architectures is explained in section 2.2. This explanation is a little fast for me (but I am new to this field). In that section, it could be nice to precise that the job of the CVAE is to learn (among others) the parameters mu,sigma of q_phi(z|d,G), using a batch of conformations of a given isomer (and then generalize to other isomers, each time guessing the correct mu,sigma parameters, even for a new isomer). This is not obvious for the inexperienced reader (I hope I *guessed* correctly). The *new dataset* shared in this work is indeed useful for reproducing results and for later comparison with other works which aim at sampling conformations. However, it derives directly from ISO17, which is itself an extraction from QM9. ISO17 lists the Cartesian coordinates of 127 isomers of C7,H10,O2 (each having ~3000 conformations). The *new* CONF17 is novel only in that it provides these conformations in distance-matrix space, i.e. it sets which pairs of atoms need to have their distances computed. This is very useful because this choice of pairs is not unique, and so setting it once is necessary for comparison. I would not stress too much that this is a new dataset: at first it actually sounds like authors chose a dataset such that their method outperforms others*! Which turns out not to be the case (they chose C7,H10,O2, but this doesn*t look like fine-tuning). Indeed, their sharing of this augmented version of ISO17 is absolutely necessary: I would simply put it differently, stating from the beginning that they share an augmented version of a pre-existing data set. About the experimental protocol/methodology: the work carried out seems quite impressive. Indeed, the paper presents a rather extensive comparison of the new model with 2 other ones, on a large data set. However sometimes the presentation rushes a bit towards results, in the sense that it is not easy to figure out over which ensemble the mean/median/max/min has been computed. This is related to the intrinsic complexity of estimating accuracy, given the encapsulation (compound-> isomer-> conformations-> each bond-> distribution over bond properties). I think however it could be made clearer, and this would benefit the paper greatly. A deeper comment: in the field of conformation sampling, there are two issues. One is to correctly sample states in each local minimum of the free energy. This is addressed in the paper. The second issue is to hunt for all possible local minima of the isomer, i.e. sample even unlikely but crucial conformations. This is not addressed here. For computing average energy, dipole moment etc, the most likely conformations will dominate, most of the time. However given the topic I think it would be fair to mention this point, and explain why in this specific case, the second issue is not crucial. Because of this, also, I think Overall, the paper is well written and is very well situated within the literature (including the very recent literature). It states clearly its contributions and results, and provides detailed presentation of experimental results. I think it contributes to the field and provides an interesting new way of dealing with the problem of molecular conformations sampling. The results obtained are much better than those of the compared ML method (RDkit). However, some claims (in introduction or in section 3) are stated more strongly than they are supported by direct experimental evidence. - **... the first work on sampling molecular conformations for molecules of arbitrary size and shape based on deep learning.** However, the experiments have been performed only on isomers of C7,H10,O2. Furthermore, there is no discussion on the computational cost of the method at all, so its extendibility to larger shapes is not obvious. - ** We create a new, challenging benchmark dataset CONF 17 for conformation generation, which is made publicly available.** I made remarks on this claim above.** - ** We develop a rigorous experimental approach for evaluating and comparing the accuracy of conformation generation methods based on the mean maximum deviation distance metric.** However the paper*s methodology doesn*t address the issue of **hunting for new local minima**. The data set could be enriched also with a labeling of **which conformation is in which local minimum of free energy** - **It is a fast alternative to resource-intensive approaches based on MCMC or MD.** As there was no mention of speed of computation in the paper (although we can guess it is faster), this is unsupported. However it is easy to remedy, simply give an order of magnitude of sampling time (for thousands of conformations of 1 isomer). - **Our principled representation based on pair-wise distances does not restrict our approach to any type of system (e.g., proteins) or any particular graph structure. In addition, it is extendable in a systematic fashion.** This suggests that the method would also work for dense packings. However this is not shown in the experiments. Probably I misunderstood the claim, and it only refers to the ability to tackle general molecular compounds, not just linear ones? However proteins are not always linear, but can present cycles, and other branch-style patterns, so the claim is not very clear to me. Because some of the important claims are only weakly supported, and because of the relatively small novelty of the method, I lean on rejecting the paper. Additional comments (for improving the paper): There is no discussion of hyper-parameter tuning, or any detail of the intermediate GNN that constitute the encoder and decoder in the CVAE. So it was either not done at all, and then it should be mentioned; or it was done and it should be discussed quickly. In particular, in section 2.2 there is no discussion on the impact of the latent space* dimension, here set to Nv, i.e. 7+2+10=19 (correct?). I guess there is a balance between accuracy and generalizability, a short discussion on this point would be interesting. The paper should clarify the hierarchy of molecular configurations: for a given composition (%C, %O, %H, etc) (here, fixed throughout the paper at C7,H10,O2) there are several structures possible (graphs or isomers), for a given graph (isomer), there can be a couple of chiral variants (mirroring of (pieces of) the molecule, for each chiral variant, there are conformations, which correspond to rather weak variations of the Cartesian coordinates, due to thermal fluctuations. Some of these variations can be important, e.g. when a bond can freely rotate. Some correspond to bending or torsion, etc, of the molecule. The algorithm presented in the paper attempts to sample appropriately the equilibrium distribution of conformations for each given graph structure (at fixed chirality, if I understood correctly). Add a comment on why (in your opinion) the distributions are not exactly correctly sampled, which after all is the goal (since the initial graph structure is given as input here). Additional question, for curiosity. In section 2.3: only a single molecule output is taken from each distance matrix. Could you comment on the possibility to sample several Cartesian coordinates, for each distance matrix ? It seems like several solutions should (often) exist. Sampling continuously from that distribution would surely provide a more continuous (rich) output (am I correct) ?