Title: A GENERATIVE MODEL FOR MOLECULAR DISTANCE GEOMETRY Anonymous authors Paper under double-blind review

Abstract: Computing equilibrium states for many-body systems, such as molecules, is a long-standing challenge. In the absence of methods for generating statistically independent samples, great computational effort is invested in simulating these systems using, for example, Markov chain Monte Carlo. We present a probabilistic model that generates such samples for molecules from their graph representations. Our model learns a low-dimensional manifold that preserves the geometry of local atomic neighborhoods through a principled learning representation that is based on Euclidean distance geometry. In a new benchmark for molecular conformation generation, we show experimentally that our generative model achieves state-ofthe-art accuracy. Finally, we show how to use our model as a proposal distribution in an importance sampling scheme to compute molecular properties.

1. INTRODUCTION:

Over the last few years, many highly-effective deep learning methods generating small molecules with desired properties (e.g., novel drugs) have emerged (Gómez-Bombarelli et al., 2018; Segler et al., 2018; Dai et al., 2018; Jin et al., 2018; Bradshaw et al., 2019a; Liu et al., 2018; You et al., 2018; Bradshaw et al., 2019b) . These methods operate using graph representations of molecules in which nodes and edges represent atoms and bonds, respectively. A representation that is closer to the physical system is one in which a molecule is described by its geometry or conformation. A conformation x of a molecule is defined by a set of atoms {( i , r i )} Nv i=1 , where N v is the number of atoms in the molecule, i ∈ {H, C, O, ...} is the chemical element of the atom i, and r i ∈ R 3 is its position in Cartesian coordinates. Importantly, the relative positions of the atoms are restricted by the bonds in the molecule and the angles between them. Due to thermal fluctuations resulting in stretching of and rotations around bonds, there exist infinitely many conformations of a molecule. A molecule's graph representation and a set of its conformations are shown in Fig. 1 . Under a wide range of conditions, the probability p(x) of a conformation x, is governed by the Boltzmann distribution and is proportional to exp{-E(x)/k B T }, where E(x) ∈ R is the conformation's energy, k B is the Boltzmann constant, and T is the temperature.

To compute a molecular property for a molecule, one must sample from p(x). The main approach is to start with one conformation and make small changes to it over time, e.g., by using Markov chain Monte Carlo (MCMC) or molecular dynamics (MD). These methods can be used to accurately sample equilibrium states of molecules, but they become computationally expensive for larger ones (Shim & MacKerell, 2011; Ballard et al., 2015; De Vivo et al., 2016) . Other heuristic approaches exist in which distances between atoms are set to fixed idealized values (Havel, 2002; Blaney & Dixon, 2007) . Several methods based on statistical learning have also recently been developed to tackle the issue of conformation generation. However, they are mainly geared towards studying proteins and their folding dynamics (AlQuraishi, 2019) . Some of these models are not targeting a distribution over conformations but the most stable folded configuration (Evans et al., 2018; Ingraham et al., 2019) , while others are not transferable between different molecules (Lemke & Peter, 2019; Noé et al., 2019) .

This work includes the following key contributions::

• We introduce a novel probabilistic model for learning conformational distributions of molecules with graph neural networks. • We create a new, challenging benchmark for conformation generation, which is made publicly available. To the best of our knowledge, this is the first benchmark of this kind. • By combining a conditional variational autoencoder (CVAE) with an Euclidean distance geometry (EDG) algorithm we present a state-of-the-art approach for generating one-shot samples of molecular conformations for unseen molecules that is independent of their size and shape. • We develop a rigorous experimental approach for evaluating and comparing the accuracy of conformation generation methods based on the mean maximum deviation distance metric. • We show how this generative model can be used as a proposal distribution in an importance sampling (IS) scheme to estimate molecular properties.

2. METHOD:

Our goal is to build a statistical model that generates molecular conformations in a one-shot fashion from a molecule's graph representation. First, we describe how a molecule's conformation can be represented by a set of pairwise distances between atoms and why this presentation is advantageous over one in Cartesian coordinates (Section 2.1). Second, we present a generative model in Section 2.2 that will generate sets of atomic distances for a given molecular graph. Third, we explain in Section 2.3 how a set of predicted distances can be transformed into a molecular conformation and why this transformation is necessary. Finally, we detail in Section 2.4 how our generative model can be used as a proposal distribution in an IS scheme to estimate molecular properties.

2.1. EXTENDED MOLECULAR GRAPHS AND DISTANCE GEOMETRY:

In this study, a molecule is represented by an undirected graph which is defined as a tuple G = (V, E). V = {v i } Nv i=1 is the set of nodes representing atoms, where each v i ∈ R Fv holds atomic attributes (e.g., the element type i ). E = {(e k , r k , s k )} Ne k=1 is the set of edges, where each e k ∈ R Fe holds an edge's attributes (e.g., the bond type), and r k and s k are the nodes an edge is connecting.

Here, E represents the molecular bonds (and the auxiliary edges which are explained below) in the molecule.

We assume that, given a molecular graph G, one can represent one of its conformations x by a set of atomic distances d = {d k } Ne k=1 , where d k = |r r k -r s k | is the Euclidean distance between the positions of the atoms r k and s k in this conformation. As the set of edges between the bonded atoms (E bond ) alone would not suffice to describe a conformation, we expand the traditional graph representation of a molecule by adding auxiliary edges. Auxiliary edges between atoms that are second neighbors in the original graph fix angles between atoms, and those between third neighbors fix dihedral angles (denoted E angle and E dihedral , respectively). In this work, E angle consists of edges between all second neighbors in the original graph. Edges between third neighbors are added according to a heuristic (see Appendix A.1). From now on we are always referring to this extended molecular graph when talking about molecular graphs. In Fig. 2 , the process of extending the molecular graph and the extraction of d from x and G are illustrated. A key advantage of a representation in terms of distances is its invariance to rotation and translation; by contrast, Cartesian coordinates depend on the (arbitrary) choice of origin, for example. In addition, it reflects pair-wise physical interactions and their generally local nature. Auxiliary edges can be placed between higher-order neighbors depending on how far the physical interactions dominating the potential energy of the system reach.

We have a set of N G molecular graphs {G l } N G l=1 . Further, for each G l , we have S l conformational samples {x l,j } S l j=1 from the ground-truth distribution resulting in S l sets of distances {d l,j } S l j=1 . With this data, we will train a generative model which we detail in the following section.

2.2. GENERATIVE MODEL:

We employ a CVAE (Kingma & Welling, 2014; Pagnoni et al., 2018) to model the distribution over distances d given a molecular graph G. A CVAE first encodes G together with d into a latent space z ∈ R kNv , where k ∈ N + , with an encoder q φ (z|d, G). Subsequently, the decoder p θ (d|z, G) decodes z back into a set of distances. A graphical model is shown in Fig. 2 

C).:

A conformation has, in general, 3N v -6 spatial degrees of freedom (dofs): one dof per spacial dimension per atom minus three translational and three rotational dofs. Therefore, the latent space should be proportional to the number of atoms in the molecule. In addition, the latent space should be smaller than 3N v as it is the role of the encoder to project the conformation into a lower-dimensional space. As a result, we set k = 1 to avoid overfitting. 1Here, q φ (z|d, G) and p θ (d|z, G) are Gaussian distributions, the mean and variance of which are modeled by two artificial neural networks. At the center of this model are message-passing neural networks (MPNNs) (Gilmer et al., 2017) with multi-head attention (Veličković et al., 2018) . In short, an MPNN is a convolutional neural network that allows end-to-end learning of prediction pipelines whose inputs are graphs of arbitrary size and shape. In a convolution, neighboring nodes exchange so-called messages between neighbors to update their attributes. Edges update their attributes with the features of the nodes they are connecting. The MPNN is a well-studied technique that achieves state-of-the-art performance in representation learning for molecules (Kipf & Welling, 2017; Duvenaud et al., 2015; Kearnes et al., 2016; Schütt et al., 2017b; Gilmer et al., 2017; Kusner et al., 2017; Bradshaw et al., 2019a) .

In the following, we describe the details of the model. 2 In Fig. 3 , an illustration of the model is shown. In the encoder q φ (z|d, G), each d k is concatenated with the respective edge feature e k to give e k ∈ R Fe+1 . Then, each v i and each e k are passed to F enc,v and F enc,e (two multilayer perceptrons, 

EQUATION

i,enc ∈ R Lv , and e

(t) k,enc ∈ R Le . Then, T MPNNs of depth 1, {MP (t) enc } T t=1 , are consecutively applied to obtain G (T ) enc . Finally, the read-out function R enc (an MLP) takes each v (T )

i, enc to predict the mean µ zi ∈ R and the variance σ 2 zi ∈ R of the Gaussian distribution for z i . The so-called reparametrization trick is employed to draw a sample for z i . In summary,

v (0) i,enc = F enc,v (v i ), e (0) k,enc = F enc,e (e i ), G (1) enc = MP (0) enc (G (0) enc ), G (t+1) enc = MP (t) enc (G (t) enc ), G (T ) enc = MP (T -1) enc (G (T -1) enc ), µ zi , σ 2 zi = R enc (v (T )

i,enc ).

(1)

In the decoder p θ (d|z, G), each z i is concatenated with the respective node feature v i to give v i ∈ R Fv+1 . Each v i and each e k are passed to F dec,v and F dec,e (two MLPs), respectively, to give

EQUATION

i,dec ∈ R Lv , and e 

v (0) i,dec = F dec,v (v i ), e (0) k,dec = F dec,e (e i ), G (1) dec = MP (0) dec (G (0) dec ), G (t+1) dec = MP (t) dec (G (t) dec ), G (T ) dec = MP (T -1) dec (G (T -1) dec ), µ d k , σ 2 d k = R dec (e (T )

k,dec ).

(

EQUATION

The sets of parameters in the encoder and decoder, φ and θ (i.e., parameters in F enc,v , F enc,e ,

EQUATION

dec } T t=1 , R dec ), respectively, are optimized by maximizing the evidence lower bound (ELBO):

EQUATION

where the prior p θ (z|G) consists of factorized Gaussians. The optimal values for the hyperparameters for the network dimensions, number of message passes, batch size, and learning rate of the Adam optimizer (Kingma & Ba, 2014) were tuned by maximizing the validation performance (ELBO) with a Bayesian optimizer and are reported in Appendix A.1.3.

2.3. CONFORMATION GENERATION THROUGH EUCLIDEAN DISTANCE GEOMETRY:

To compute molecular properties, quantum-chemical methods need to be employed which require the input, i.e., the molecule, to be in Cartesian coordinates. 3 Therefore, we use an EDG algorithm to translate the set of distances {d k } Ne k=1 to a set of atomic coordinates {r i } Nv i=1 .4 EDG is the mathematical basis for a geometric theory of molecular conformation. In the field of machine learning, Weinberger & Saul (2006) used it for learning image manifolds, Tenenbaum et al. (2000) for image understanding and handwriting recognition, Jain & Saul (2004) for speech and music, and Demaine et al. (2009) for music and musical rhythms. An EDG description of a molecular system consists of a list of lower and upper bounds on the distances between pairs of atoms {(d k,min , d k,max )} Ne k=1 . Here, p θ (d|z, G) is used to model these bounds, namely, we set the bounds to {(µ

d k -σ d k , µ d k + σ d k )}

, where µ d k and σ d k are the mean and standard deviation for each distance d k given by the CVAE. Then, an EDG algorithm determines a set of Cartesian coordinates {r i } Nv i=1 so that these bounds are fulfilled (see Appendix A.2 for details). 5 Together with the corresponding chemical elements { i } Nv i=1 , we obtain a conformation x.

2.4. CALCULATION OF MOLECULAR PROPERTIES:

We can get an MC estimate of the expectation E G [O] of a property O (e.g., the dipole moment) for a molecule represented by G by drawing conformational samples x i ∼ p(x|G) and computing O(x i ) ∈ R with a quantum-chemical method (e.g., density functional theory). Since we cannot draw samples from p(x|G) directly, we employ an IS integration scheme (Bishop, 2009) with our CVAE as the proposal distribution. We assume that we can readily evaluate the unnormalized probability of a conformation p(x|G) = exp{-E(x)/k B T }, where x must be a conformation of the molecule and the energy E(x) is determined with a quantum-chemical method. Since the EDG algorithm is mapping the distribution p θ (d|z, G) to a point mass in R 3Nv , the MC estimate for the resulting distribution p prop (x|G) is given by a mixture of delta functions, each of which is centered at the x i resulting from mapping p θ (d|z i , G) to R 3Nv , where

z i ∼ p θ (z|G), that is, p prop (x|G) ≈ 1 N N i=1 δ(x -x i ).

The IS estimator for the expectation of O w. r. t. p(x|G) then reads

EQUATION

where x i ∼ p(x i |G) and x i ∼ p prop (x i |G), so that the expectation of O w. r. t. the normalized version of p(x) is then

EQUATION

where Z ≈ N i=1 p(x i ) and N is the number of samples. When dividing two delta functions we have assumed that they take some arbitrarily large finite value.

3. RELATED WORKS:

The standard approach for generating molecular conformations is to start with one, and make small changes to it over time, e.g., by using MCMC or MD. These methods are considered the gold standard for sampling equilibrium states, but they are computationally expensive, especially if the molecule is large and the Hamiltonian is based on quantum-mechanical principles (Shim & MacKerell, 2011; Ballard et al., 2015; De Vivo et al., 2016) .

A much faster but more approximate approach for conformation generation is EDG (Havel, 2002; Blaney & Dixon, 2007; Lagorce et al., 2009; Riniker & Landrum, 2015) . Lower and upper distance bounds for pairs of atoms in a molecule are fixed values based on ideal bond lengths, bond angles, and torsional angles. These values are often extracted from crystal structure databases (Allen, 2002) . These methods aim to generate a low-energy conformation, not to generate unbiased samples from the underlying distribution at a certain temperature.

There exist several machine learning approaches as well, however, they are mostly tailored towards studying protein dynamics. For example, Noé et al. (2019) trained Boltzmann generators on the energy function of proteins to provide unbiased, one-shot samples from their equilibrium states. This is achieved by training an invertible neural network to learn a coordinate transformation from a system's configurations to a latent space representation. Further, Lemke & Peter (2019) proposed a dimensionality reduction algorithm that is based on a neural network autoencoder in combination with a nonlinear distance metric to generate samples for protein structures. Both models learn protein-specific coordinate transformations that cannot be transferred to other molecules. AlQuraishi ( 2019) introduced an end-to-end differentiable recurrent geometric network for protein structure learning based on amino acid sequences. Also, Ingraham et al. (2019) proposed a neural energy simulator model for protein structure that makes use of protein sequence information. In contrast to amino acid sequences, molecular graphs are, in general, not linear but highly branched and often contain cycles. This makes them unsuitable for recurrent networks.

Finally, Mansimov et al. (2019) presented a conditional deep generative graph neural network to generate molecular conformations given a molecular graph. Their goal is to predict the most likely conformation and not a distribution over conformations. Instead of encoding molecular environments in atomic distances, they work directly in Cartesian coordinates. As a result, the generated conformations showed significant structural differences compared to the ground-truth and required refinement through a force field, which is often employed in MD simulations.

We argue that our model has several advantages over the approaches reviewed above:

• It is a fast alternative to resource-intensive approaches based on MCMC or MD.

• Our principled representation based on pair-wise distances does not restrict our approach to any particular molecular structure.

• Since our model employs message-passing neural networks, it is transferable -it can extrapolate from only a few graphs to unseen ones.

4. THE CONF17 BENCHMARK:

The CONF17 benchmark is the first benchmark for molecular conformation sampling. 6 It is based on the ISO17 dataset (Schütt et al., 2017a) which consists of conformations of various molecules with the atomic composition C 7 H 10 O 2 drawn from the QM9 dataset (Ramakrishnan et al., 2014) . These conformations were generated by ab initio molecular dynamics simulations at 500 Kelvin which generates trajectories of a single molecule covering a large variety of conformations. The CONF17 benchmark consists of 127 distinct molecular graphs each with 3380 conformations on average. We split this dataset into multiple training and test splits, each consisting of 107 and 20 graphs, respectively (see Appendix A.1 for more details).7 

In Fig. 4 , (A), the structural formulae of a random selection of molecules from this benchmark are shown. Most molecules feature highly-strained, complex 3D structures such as rings which are typical of drug-like molecules. It is thus the structural complexity of the molecules, not their number of degrees of freedom, that makes this benchmark challenging. In 

5. EXPERIMENTS:

We assess the performance of our method, named Graph Distance Geometry (GRAPHDG), by comparing it with two state-of-the-art methods for molecular conformation generation: RDKIT (Riniker & Landrum, 2015) , a classical EDG approach, and DL4CHEM (Mansimov et al., 2019) , a machine learning approach. We trained GRAPHDG and DL4CHEM on three different training and test splits of the CONF17 benchmark using Adam (Kingma & Ba, 2014) . We generated 3000 conformations with each method for molecular graphs in a test set.

5.1. DISTRIBUTIONS OVER DISTANCES:

We assessed the accuracy of the distance distributions of RDKIT, DL4CHEM, and GRAPHDG by calculating the maximum mean discrepancy (MMD) (Gretton et al., 2012) to the ground-truth distribution. We compute the MMD using a Gaussian kernel, where we set the standard deviation to be the median distance between distances d in the aggregate sample. For this, we determined the distances in the conformations from the ground-truth and those generated by RDKIT and DL4CHEM.

For each train-test split and each G in a test set, we compute the MMD of the joint distribution of distances between C and O atoms (H atoms are usually ignored), the MMDs of pair-wise distances p(d i , d j |G), and the MMDs between the marginals of individual distances p(d i |G). We aggregate the results of three train-test splits, and, finally, compute the median MMDs and average rankings.

The results are summarized in Table 1 . It can be seen that the samples from GRAPHDG are significantly closer to the ground-truth distribution than the other methods. RDKIT is slightly worse than GRAPHDG while DL4CHEM seems to struggle with the complexity of the molecules and the small number of graphs in the training set.

In Fig. 5 , we showcase the accuracy of our model by plotting the marginal distributions p(d i |G) for distances between C and O atoms given a molecular graph from a test set. It can be seen that RDKIT consistently underestimates the marginal variances. This is because this method aims to predict the most stable conformation, i.e., the distribution's mode. In contrast, DL4CHEM often fails to predict the correct mean. For this molecule, GRAPHDG is the most accurate, predicting the right mean and variance in most cases. Additional figures can be found in the Appendix A.4, where we also show plots for the marginal distributions p(d i , d j |G). 

5.2. GENERATION OF CONFORMATIONS:

We passed the distances from our generative model to an EDG algorithm to obtain conformations. For 99.9% of the sets of distances, all triangle inequalities held. For 94% of the molecular graphs, the algorithm succeeded which is 8 pp higher than the success rate we observed for RDKIT. For each molecular graph in a test set, we generated 50 conformations with each method. This took DL4CHEM, RDKIT, and GRAPHDG on average around hundreds of milliseconds per molecule. 8In contrast, a single conformation in the ISO17 dataset takes around a minute to compute. In Fig. 6 , an overlay of these conformations of six molecules generated by the different methods is shown. It can be seen that RDKIT's conformations show too little variance, while DL4CHEM's structures are mostly invalid, which is due in part to its failure to predict the correct interatomic angles. Our method slightly overestimates the structural variance (see, for example, Fig. 6 , top row, second column), but produces conformations that are the closest to the ground-truth.

We estimate expected molecular properties for molecular graphs from the test set with N = 50 conformational samples each. Due to their poor quality, we could not compute properties O(x), including the energy E(x), for conformations generated with DL4CHEM, and thus, this method is excluded from this analysis. In Table 2 , it can be seen that RDKIT and GRAPHDG perform similarly well (see Appendix A.2 for computational details). However, both methods are still highly inaccurate for E elec (in practice, an accuracy of less than 5 kJ/mol is required). Close inspection of the conformations shows that, even though GRAPHDG predicts the most accurate distances overall, the variances of certain strongly constrained distances (e.g., triple bonds) are overestimated so that the energies of the conformations increase drastically.

6. LIMITATIONS:

The first limitation of this work is that the CVAE can sample (with low probability) invalid sets of distances for which there exists no 3D structure. Second, the CONF17 benchmark covers only a small portion of chemical space. Finally, a large set of auxiliary edges would be required to capture long-range correlations (e.g., in proteins). Future work will address these points.

7. CONCLUSIONS:

We presented GRAPHDG, a transferable, generative model that allows sampling from a distribution over molecular conformations. We developed a principled learning representation of conformations that is based on distances between atoms. Then, we proposed a challenging benchmark for comparing molecular conformation generators. With this benchmark, we show experimentally that conformations generated by GRAPHDG are closer to the ground-truth than those generated by other methods. Finally, we employ our model as a proposal distribution in an IS integration scheme to estimate molecular properties. While orbital energies and the dipole moments were predicted well, a larger and more diverse dataset will be necessary for meaningful estimates of electronic energies. Further, methods have to be devised to estimate how many conformations need to be generated to ensure all important conformations have been sampled. Finally, our model could be trained on conformational distributions at different temperatures in a transfer learning-type setting. 

:

Experiments showed that our model performs similarly with a latent space of R

2Nv and overfits with latent spaces of R 3Nv .2 The full model (including all parameters) is available online https://figshare.com/s/ 1b42bf865bd78c457354

Even though quantum-chemical methods require the input to be in Cartesian coordinates, calculated properties, such as the energy, are invariant under translation and rotation.

4 There are additional constraints due to chirality. However, since they are given by G and are fixed, they are not modeled by our

method.5 Often there exist multiple solutions for the same set of bounds. As the bounds are generally tight, the solutions are very similar. Therefore, we only generate one set of coordinates per set of bounds.

Datasets such as the one published byKanal et al. (2018) only include conformers, i.e., the stable conformations of a molecule, and not a distribution over conformations.

7 The CONF17 benchmark is publicly available online https://figshare.com/s/ 1b42bf865bd78c457354.

All simulations were carried out on a computer equipped with an i7-3820 CPU and a GeForce GTX 1080 Ti GPU.

Figures and Tables:

Figure 1: Standard graph representation of a molecule (left) with a set of possible conformations {x i } (right). Hydrogen (H), carbon (C), and oxygen (O) atoms are colored white, gray, and red, respectively. Conformations feature the same atom types and bonds but the atoms are arranged differently in space. These differences arise from rotations around and stretching of bonds in the molecule.

Figure 2: A) The structural formula of a molecule is converted to an extended molecular graph G consisting of nodes representing atoms (circles, e.g., v 1 ) and edges representing molecular bonds (solid lines, e.g., e 1 ∈ E bond ) and auxiliary edges (dotted lines, e.g., e 2 ∈ E angle and e 3 ∈ E dihedral ). B) The distances d are extracted from a conformation x based on the edges E. C) Graphical model of the variational autoencoder: generative model p θ (d|z, G)p θ (z|G) (solid lines) and variational approximation q φ (z|d, G) (dashed lines).

Figure3: The molecular graph G together with the distances d are passed through the model consisting of an encoder q φ (z|d, G) and a decoder p θ (d|z, G). See the main text for details.

∈ R Le . Then, T MPNNs of depth 1, {MP (t) dec } T t=1 , are consecutively applied to obtain G (T ) dec . Finally, the read-out function R dec (an MLP) takes each e (T ) k, dec to predict the mean µ d k ∈ R and the variance σ 2 d k ∈ R of the Gaussian distribution for d k . In summary,

Figure 4: Overview of the CONF17 benchmark. (A) Structural formulae of a random selection of molecules. (B) Distribution of distances (in Å) grouped by edge (from left to right: E bond , E angle , and E dihedral ) and vertex type (chemical element).

Figure 5: Marginal distributions p(d k |G) of ground-truth and predicted bond distances (in Å) between C and O atoms given a molecular graph from the test set. The atoms connected by each edge d k are indicated in each subplot (s k -r k ). In the 3D structure of the molecule, carbon and oxygen atoms are colored gray and red, respectively. H atoms are omitted for clarity.

Figure 6: Overlay of 50 conformations from the ground-truth and three models based on six random molecular graphs from the test set. C, O, and H atoms are colored gray, red, and white, respectively.

Figure 8: Marginal distributions p(d k |G) of ground-truth and predicted distances (in Å) between C and O atoms given a molecular graph from the test set. The atoms connected by each edge d k are indicated in each subplot (s k -r k ). In the 3D structure of the molecule, carbon and oxygen atoms are colored gray and red, respectively. H atoms are omitted for clarity.

Figure 9: Marginal distributions p(d i , d j |G) of ground-truth and predicted distances for a molecular graph from the test set (in Å). Here, d i and d j are restricted to edges representing bonds between C and O atoms. In the 3D structure of the molecule, carbon and oxygen atoms are colored gray and red, respectively. H atoms are omitted for clarity.

Assessment of the accuracy of the distributions over conformations generated by three models compared to the ground-truth. We compare the distributions with respect to the marginals p(d k |G), p(d k , d l |G), and the distribution over all edges between C and O atoms p({d k }|G). Two different metrics are used: median MMD between ground-truth conformations and generated ones, and mean ranking (1 to 3) based on the MMD. Reported are the results for molecular graphs in a test set from three train-test splits. Standard errors are given in brackets.

Table content: <table><tr><td/><td/><td>Median MMD</td><td/><td>Mean Ranking</td></tr><tr><td/><td>RDKIT</td><td>DL4CHEM GRAPHDG</td><td>RDKIT</td><td>DL4CHEM GRAPHDG</td></tr><tr><td>p(d k |G)</td><td colspan="4">0.55 (0.01) 1.11 (0.01) 0.38 (0.02) 1.71 (0.03) 2.74 (0.02) 1.51 (0.03)</td></tr><tr><td colspan="5">p(d k , d l |G) 0.53 (0.01) 1.09 (0.01) 0.34 (0.01) 1.66 (0.02) 2.92 (0.01) 1.43 (0.02)</td></tr><tr><td colspan="5">p({d k }|G) 0.60 (0.01) 1.07 (0.03) 0.44 (0.05) 1.58 (0.05) 2.90 (0.05) 1.45 (0.02)</td></tr></table>

Median difference in average properties between ground-truth and RDKIT and GRAPHDG: total electronic energy E elec (in kJ/mol), the energy of the HOMO and the LUMO LUMO and LUMO , respectively (in eV), and the dipole moment µ (in debye). Reported are the results for molecular graphs from the test set, averaged over three train-test splits. Standard errors are given in brackets.

Table content: <table><tr><td/><td>RDKIT</td><td>GRAPHDG</td></tr><tr><td>E elec</td><td colspan="2">42.7 (4.3) 58.0 (21.0)</td></tr><tr><td>HOMO</td><td colspan="2">0.08 (0.04) 0.10 (0.05)</td></tr><tr><td>LUMO</td><td colspan="2">0.15 (0.03) 0.09 (0.05)</td></tr><tr><td>µ</td><td colspan="2">0.29 (0.05) 0.33 (0.09)</td></tr></table>

Peer review (iclr2020):
The paper proposes a generative model for generating molecule with three dimensional structure. Given a graph, it leverages a variational autoencoder to embed the distance between two atoms into latent vectors (encoder) and then generate distance between two atoms based on the latent vector. Even if the problem is well motivated, I have several concerns: I. What is the variable *x*? Is it possible to define a bit formally? It appears that x=(G,d) i.e. the graph with d. II. It would be nice to develop the generative model of (G,d) i.e. both G and d. Simply generating d looks a bit restrictive. III. How do you model/learn mathcal{O}(x)? I would be curious to know the answer of these questions.