Year: 2019

Abstract:
Computing equilibrium states for many-body systems, such as molecules, is a long-standing challenge. In the absence of methods for generating statistically independent samples, great computational effort is invested in simulating these systems using, for example, Markov chain Monte Carlo. We present a probabilistic model that generates such samples for molecules from their graph representations. Our model learns a low-dimensional manifold that preserves the geometry of local atomic neighborhoods through a principled learning representation that is based on Euclidean distance geometry. In a new benchmark for molecular conformation generation, we show experimentally that our generative model achieves state-ofthe-art accuracy. Finally, we show how to use our model as a proposal distribution in an importance sampling scheme to compute molecular properties.

heading: 1 INTRODUCTION
text: Over the last few years, many highly-effective deep learning methods generating small molecules with desired properties (e.g., novel drugs) have emerged (Gómez-Bombarelli et al., 2018; Segler et al., 2018; Dai et al., 2018; Jin et al., 2018; Bradshaw et al., 2019a; Liu et al., 2018; You et al., 2018; Bradshaw et al., 2019b). These methods operate using graph representations of molecules in which nodes and edges represent atoms and bonds, respectively. A representation that is closer to the physical system is one in which a molecule is described by its geometry or conformation. A conformation x of a molecule is defined by a set of atoms {( i, ri)}Nvi=1, where Nv is the number of atoms in the molecule, i ∈ {H,C,O, ...} is the chemical element of the atom i, and ri ∈ R3 is its position in Cartesian coordinates. Importantly, the relative positions of the atoms are restricted by the bonds in the molecule and the angles between them. Due to thermal fluctuations resulting in stretching of and rotations around bonds, there exist infinitely many conformations of a molecule. A molecule’s graph representation and a set of its conformations are shown in Fig. 1. Under a wide range of conditions, the probability p(x) of a conformation x, is governed by the Boltzmann distribution and is proportional to exp{−E(x)/kBT}, where E(x) ∈ R is the conformation’s energy, kB is the Boltzmann constant, and T is the temperature.
To compute a molecular property for a molecule, one must sample from p(x). The main approach is to start with one conformation and make small changes to it over time, e.g., by using Markov chain Monte Carlo (MCMC) or molecular dynamics (MD). These methods can be used to accurately sample equilibrium states of molecules, but they become computationally expensive for larger ones (Shim & MacKerell, 2011; Ballard et al., 2015; De Vivo et al., 2016). Other heuristic approaches exist in which distances between atoms are set to fixed idealized values (Havel, 2002; Blaney & Dixon, 2007). Several methods based on statistical learning have also recently been developed to tackle the issue of conformation generation. However, they are mainly geared towards studying proteins and their folding dynamics (AlQuraishi, 2019). Some of these models are not targeting a distribution over conformations but the most stable folded configuration (Evans et al., 2018; Ingraham et al., 2019), while others are not transferable between different molecules (Lemke & Peter, 2019; Noé et al., 2019).
This work includes the following key contributions:
• We introduce a novel probabilistic model for learning conformational distributions of molecules with graph neural networks.
• We create a new, challenging benchmark for conformation generation, which is made publicly available. To the best of our knowledge, this is the first benchmark of this kind.
• By combining a conditional variational autoencoder (CVAE) with an Euclidean distance geometry (EDG) algorithm we present a state-of-the-art approach for generating one-shot samples of molecular conformations for unseen molecules that is independent of their size and shape.
• We develop a rigorous experimental approach for evaluating and comparing the accuracy of conformation generation methods based on the mean maximum deviation distance metric.
• We show how this generative model can be used as a proposal distribution in an importance sampling (IS) scheme to estimate molecular properties.

heading: 2 METHOD
text: Our goal is to build a statistical model that generates molecular conformations in a one-shot fashion from a molecule’s graph representation. First, we describe how a molecule’s conformation can be represented by a set of pairwise distances between atoms and why this presentation is advantageous over one in Cartesian coordinates (Section 2.1). Second, we present a generative model in Section 2.2 that will generate sets of atomic distances for a given molecular graph. Third, we explain in Section 2.3 how a set of predicted distances can be transformed into a molecular conformation and why this transformation is necessary. Finally, we detail in Section 2.4 how our generative model can be used as a proposal distribution in an IS scheme to estimate molecular properties.

heading: 2.1 EXTENDED MOLECULAR GRAPHS AND DISTANCE GEOMETRY
text: In this study, a molecule is represented by an undirected graph which is defined as a tuple G = (V,E). V = {vi}Nvi=1 is the set of nodes representing atoms, where each vi ∈ RFv holds atomic attributes (e.g., the element type i). E = {(ek, rk, sk)}Nek=1 is the set of edges, where each ek ∈ RFe holds an edge’s attributes (e.g., the bond type), and rk and sk are the nodes an edge is connecting. Here, E represents the molecular bonds (and the auxiliary edges which are explained below) in the molecule.
We assume that, given a molecular graph G, one can represent one of its conformations x by a set of atomic distances d = {dk}Nek=1, where dk = |rrk − rsk | is the Euclidean distance between the positions of the atoms rk and sk in this conformation. As the set of edges between the bonded atoms (Ebond) alone would not suffice to describe a conformation, we expand the traditional graph representation of a molecule by adding auxiliary edges. Auxiliary edges between atoms that are second neighbors in the original graph fix angles between atoms, and those between third neighbors fix dihedral angles (denoted Eangle and Edihedral, respectively). In this work, Eangle consists of edges between all second neighbors in the original graph. Edges between third neighbors are added according to a heuristic (see Appendix A.1). From now on we are always referring to this extended molecular graph when talking about molecular graphs. In Fig. 2, the process of extending the molecular graph and the extraction of d from x and G are illustrated.
A key advantage of a representation in terms of distances is its invariance to rotation and translation; by contrast, Cartesian coordinates depend on the (arbitrary) choice of origin, for example. In addition, it reflects pair-wise physical interactions and their generally local nature. Auxiliary edges can be placed between higher-order neighbors depending on how far the physical interactions dominating the potential energy of the system reach.
We have a set of NG molecular graphs {Gl}NGl=1. Further, for each Gl, we have Sl conformational samples {xl,j}Slj=1 from the ground-truth distribution resulting in Sl sets of distances {dl,j} Sl j=1. With this data, we will train a generative model which we detail in the following section.

heading: 2.2 GENERATIVE MODEL
text: We employ a CVAE (Kingma & Welling, 2014; Pagnoni et al., 2018) to model the distribution over distances d given a molecular graph G. A CVAE first encodes G together with d into a latent space z ∈ RkNv , where k ∈ N+, with an encoder qφ(z|d,G). Subsequently, the decoder pθ(d|z,G) decodes z back into a set of distances. A graphical model is shown in Fig. 2 C).
A conformation has, in general, 3Nv − 6 spatial degrees of freedom (dofs): one dof per spacial dimension per atom minus three translational and three rotational dofs. Therefore, the latent space should be proportional to the number of atoms in the molecule. In addition, the latent space should be smaller than 3Nv as it is the role of the encoder to project the conformation into a lower-dimensional space. As a result, we set k = 1 to avoid overfitting.1
Here, qφ(z|d,G) and pθ(d|z,G) are Gaussian distributions, the mean and variance of which are modeled by two artificial neural networks. At the center of this model are message-passing neural networks (MPNNs) (Gilmer et al., 2017) with multi-head attention (Veličković et al., 2018). In short, an MPNN is a convolutional neural network that allows end-to-end learning of prediction pipelines whose inputs are graphs of arbitrary size and shape. In a convolution, neighboring nodes exchange so-called messages between neighbors to update their attributes. Edges update their attributes with the features of the nodes they are connecting. The MPNN is a well-studied technique that achieves state-of-the-art performance in representation learning for molecules (Kipf & Welling, 2017; Duvenaud et al., 2015; Kearnes et al., 2016; Schütt et al., 2017b; Gilmer et al., 2017; Kusner et al., 2017; Bradshaw et al., 2019a).
In the following, we describe the details of the model.2 In Fig. 3, an illustration of the model is shown. In the encoder qφ(z|d,G), each dk is concatenated with the respective edge feature ek to give e′k ∈ RFe+1. Then, each vi and each e′k are passed to Fenc,v and Fenc,e (two multilayer perceptrons,
1Experiments showed that our model performs similarly with a latent space of R2Nv and overfits with latent spaces of R3Nv .
2The full model (including all parameters) is available online https://figshare.com/s/ 1b42bf865bd78c457354
MLPs), respectively, to give G(0)enc , where G(t)enc = ({v(t)i,enc} Nv i=1, {(e (t) k,enc, rk, sk)} Ne k=1), v (t) i,enc ∈ RLv , and e(t)k,enc ∈ RLe . Then, T MPNNs of depth 1, {MP (t) enc}Tt=1, are consecutively applied to obtain G(T )enc . Finally, the read-out function Renc (an MLP) takes each v(T )i, enc to predict the mean µzi ∈ R and the variance σ2zi ∈ R of the Gaussian distribution for zi. The so-called reparametrization trick is employed to draw a sample for zi. In summary,
v (0) i,enc = Fenc,v(vi), e (0) k,enc = Fenc,e(e ′ i),
G(1)enc = MP(0)enc(G(0)enc ), G(t+1)enc = MP(t)enc(G(t)enc), G(T )enc = MP(T−1)enc (G(T−1)enc ),
µzi , σ 2 zi = Renc(v (T ) i,enc).
(1)
In the decoder pθ(d|z,G), each zi is concatenated with the respective node feature vi to give v′i ∈ RFv+1. Each v′i and each ek are passed to Fdec,v and Fdec,e (two MLPs), respectively, to give G (0) dec , where G(t)dec = ({v (t) i,dec} Nv i=1, {(e (t) k,dec, rk, sk)} Ne k=1), v (t) i,dec ∈ RLv , and e (t) k,dec ∈ RLe . Then, T MPNNs of depth 1, {MP(t)dec}Tt=1, are consecutively applied to obtain G (T ) dec . Finally, the read-out functionRdec (an MLP) takes each e(T )k, dec to predict the mean µdk ∈ R and the variance σ2dk ∈ R of the Gaussian distribution for dk. In summary,
v (0) i,dec = Fdec,v(v ′ i), e (0) k,dec = Fdec,e(ei),
G(1)dec = MP (0) dec(G (0) dec ), G (t+1) dec = MP (t) dec(G (t) dec), G (T ) dec = MP (T−1) dec (G (T−1) dec ),
µdk , σ 2 dk = Rdec(e (T ) k,dec).
(2)
The sets of parameters in the encoder and decoder, φ and θ (i.e., parameters in Fenc,v , Fenc,e, {MP(t)enc}Tt=1, Renc, Fdec,v , Fdec,e, {MP (t) dec}Tt=1, Rdec), respectively, are optimized by maximizing the evidence lower bound (ELBO):
L = Ez∼qφ(z|d,G)[log pθ(d|z,G)]−DKL[qφ(z|d,G)||pθ(z|G)], (3)
where the prior pθ(z|G) consists of factorized Gaussians. The optimal values for the hyperparameters for the network dimensions, number of message passes, batch size, and learning rate of the Adam optimizer (Kingma & Ba, 2014) were tuned by maximizing the validation performance (ELBO) with a Bayesian optimizer and are reported in Appendix A.1.3.

heading: 2.3 CONFORMATION GENERATION THROUGH EUCLIDEAN DISTANCE GEOMETRY
text: To compute molecular properties, quantum-chemical methods need to be employed which require the input, i.e., the molecule, to be in Cartesian coordinates.3 Therefore, we use an EDG algorithm to translate the set of distances {dk}Nek=1 to a set of atomic coordinates {ri} Nv i=1. 4
EDG is the mathematical basis for a geometric theory of molecular conformation. In the field of machine learning, Weinberger & Saul (2006) used it for learning image manifolds, Tenenbaum et al. (2000) for image understanding and handwriting recognition, Jain & Saul (2004) for speech and music, and Demaine et al. (2009) for music and musical rhythms. An EDG description of a molecular system consists of a list of lower and upper bounds on the distances between pairs of atoms {(dk,min, dk,max)}Nek=1. Here, pθ(d|z,G) is used to model these bounds, namely, we set the bounds to {(µdk − σdk , µdk + σdk)}, where µdk and σdk are the mean and standard deviation for each distance dk given by the CVAE. Then, an EDG algorithm determines a set of Cartesian coordinates {ri}Nvi=1 so that these bounds are fulfilled (see Appendix A.2 for details).5 Together with the corresponding chemical elements { i}Nvi=1, we obtain a conformation x.

heading: 2.4 CALCULATION OF MOLECULAR PROPERTIES
text: We can get an MC estimate of the expectation EG [O] of a property O (e.g., the dipole moment) for a molecule represented by G by drawing conformational samples xi ∼ p(x|G) and computing O(xi) ∈ R with a quantum-chemical method (e.g., density functional theory). Since we cannot draw samples from p(x|G) directly, we employ an IS integration scheme (Bishop, 2009) with our CVAE as the proposal distribution. We assume that we can readily evaluate the unnormalized probability of a conformation p̃(x|G) = exp{−E(x)/kBT}, where x must be a conformation of the molecule and the energy E(x) is determined with a quantum-chemical method. Since the EDG algorithm is mapping the distribution pθ(d|z,G) to a point mass in R3Nv , the MC estimate for the resulting distribution pprop(x|G) is given by a mixture of delta functions, each of which is centered at the xi resulting from mapping pθ(d|zi,G) to R3Nv , where zi ∼ pθ(z|G), that is, pprop(x|G) ≈ 1N ∑N i=1 δ(x − xi). The IS estimator for the expectation of O w. r. t. p̃(x|G) then reads
ÊG [O] MC ≈ 1
N N∑ i=1 O(xi) IS = 1 N N∑ i=1 O(x′i) p̃(x′i|G) pprop(x′i|G) , (4)
where xi ∼ p̃(xi|G) and x′i ∼ pprop(x′i|G), so that the expectation of O w. r. t. the normalized version of p̃(x) is then
EG [O] = ÊG [O] ÊG [1] ≈ 1 Z N∑ i=1 O(xi)p̃(x′i|G), (5)
where Z ≈ ∑N i=1 p̃(x ′ i) and N is the number of samples. When dividing two delta functions we have assumed that they take some arbitrarily large finite value.

heading: 3 RELATED WORKS
text: The standard approach for generating molecular conformations is to start with one, and make small changes to it over time, e.g., by using MCMC or MD. These methods are considered the gold standard for sampling equilibrium states, but they are computationally expensive, especially if the molecule is large and the Hamiltonian is based on quantum-mechanical principles (Shim & MacKerell, 2011; Ballard et al., 2015; De Vivo et al., 2016).
A much faster but more approximate approach for conformation generation is EDG (Havel, 2002; Blaney & Dixon, 2007; Lagorce et al., 2009; Riniker & Landrum, 2015). Lower and upper distance
3Even though quantum-chemical methods require the input to be in Cartesian coordinates, calculated properties, such as the energy, are invariant under translation and rotation.
4There are additional constraints due to chirality. However, since they are given by G and are fixed, they are not modeled by our method.
5Often there exist multiple solutions for the same set of bounds. As the bounds are generally tight, the solutions are very similar. Therefore, we only generate one set of coordinates per set of bounds.
bounds for pairs of atoms in a molecule are fixed values based on ideal bond lengths, bond angles, and torsional angles. These values are often extracted from crystal structure databases (Allen, 2002). These methods aim to generate a low-energy conformation, not to generate unbiased samples from the underlying distribution at a certain temperature.
There exist several machine learning approaches as well, however, they are mostly tailored towards studying protein dynamics. For example, Noé et al. (2019) trained Boltzmann generators on the energy function of proteins to provide unbiased, one-shot samples from their equilibrium states. This is achieved by training an invertible neural network to learn a coordinate transformation from a system’s configurations to a latent space representation. Further, Lemke & Peter (2019) proposed a dimensionality reduction algorithm that is based on a neural network autoencoder in combination with a nonlinear distance metric to generate samples for protein structures. Both models learn protein-specific coordinate transformations that cannot be transferred to other molecules.
AlQuraishi (2019) introduced an end-to-end differentiable recurrent geometric network for protein structure learning based on amino acid sequences. Also, Ingraham et al. (2019) proposed a neural energy simulator model for protein structure that makes use of protein sequence information. In contrast to amino acid sequences, molecular graphs are, in general, not linear but highly branched and often contain cycles. This makes them unsuitable for recurrent networks.
Finally, Mansimov et al. (2019) presented a conditional deep generative graph neural network to generate molecular conformations given a molecular graph. Their goal is to predict the most likely conformation and not a distribution over conformations. Instead of encoding molecular environments in atomic distances, they work directly in Cartesian coordinates. As a result, the generated conformations showed significant structural differences compared to the ground-truth and required refinement through a force field, which is often employed in MD simulations.
We argue that our model has several advantages over the approaches reviewed above:
• It is a fast alternative to resource-intensive approaches based on MCMC or MD.
• Our principled representation based on pair-wise distances does not restrict our approach to any particular molecular structure.
• Since our model employs message-passing neural networks, it is transferable – it can extrapolate from only a few graphs to unseen ones.

heading: 4 THE CONF17 BENCHMARK
text: The CONF17 benchmark is the first benchmark for molecular conformation sampling.6 It is based on the ISO17 dataset (Schütt et al., 2017a) which consists of conformations of various molecules with the atomic composition C7H10O2 drawn from the QM9 dataset (Ramakrishnan et al., 2014). These conformations were generated by ab initio molecular dynamics simulations at 500 Kelvin which generates trajectories of a single molecule covering a large variety of conformations. The CONF17 benchmark consists of 127 distinct molecular graphs each with 3380 conformations on average. We split this dataset into multiple training and test splits, each consisting of 107 and 20 graphs, respectively (see Appendix A.1 for more details).7
In Fig. 4, (A), the structural formulae of a random selection of molecules from this benchmark are shown. Most molecules feature highly-strained, complex 3D structures such as rings which are typical of drug-like molecules. It is thus the structural complexity of the molecules, not their number of degrees of freedom, that makes this benchmark challenging. In Fig. 4, (B), the frequency of distances (in Å) in the conformations are shown for each edge type. It can be seen that the marginal distributions of the edge distances are multimodal and highly context dependent.
6Datasets such as the one published by Kanal et al. (2018) only include conformers, i.e., the stable conformations of a molecule, and not a distribution over conformations.
7The CONF17 benchmark is publicly available online https://figshare.com/s/ 1b42bf865bd78c457354.

heading: 5 EXPERIMENTS
text: We assess the performance of our method, named Graph Distance Geometry (GRAPHDG), by comparing it with two state-of-the-art methods for molecular conformation generation: RDKIT (Riniker & Landrum, 2015), a classical EDG approach, and DL4CHEM (Mansimov et al., 2019), a machine learning approach. We trained GRAPHDG and DL4CHEM on three different training and test splits of the CONF17 benchmark using Adam (Kingma & Ba, 2014). We generated 3000 conformations with each method for molecular graphs in a test set.

heading: 5.1 DISTRIBUTIONS OVER DISTANCES
text: We assessed the accuracy of the distance distributions of RDKIT, DL4CHEM, and GRAPHDG by calculating the maximum mean discrepancy (MMD) (Gretton et al., 2012) to the ground-truth distribution. We compute the MMD using a Gaussian kernel, where we set the standard deviation to be the median distance between distances d in the aggregate sample. For this, we determined the distances in the conformations from the ground-truth and those generated by RDKIT and DL4CHEM. For each train-test split and each G in a test set, we compute the MMD of the joint distribution of distances between C and O atoms (H atoms are usually ignored), the MMDs of pair-wise distances p(di, dj |G), and the MMDs between the marginals of individual distances p(di|G). We aggregate the results of three train-test splits, and, finally, compute the median MMDs and average rankings. The results are summarized in Table 1. It can be seen that the samples from GRAPHDG are significantly closer to the ground-truth distribution than the other methods. RDKIT is slightly worse than GRAPHDG while DL4CHEM seems to struggle with the complexity of the molecules and the small number of graphs in the training set.
In Fig. 5, we showcase the accuracy of our model by plotting the marginal distributions p(di|G) for distances between C and O atoms given a molecular graph from a test set. It can be seen that RDKIT consistently underestimates the marginal variances. This is because this method aims to predict the most stable conformation, i.e., the distribution’s mode. In contrast, DL4CHEM often fails to predict the correct mean. For this molecule, GRAPHDG is the most accurate, predicting the right mean and variance in most cases. Additional figures can be found in the Appendix A.4, where we also show plots for the marginal distributions p(di, dj |G).

heading: 5.2 GENERATION OF CONFORMATIONS
text: We passed the distances from our generative model to an EDG algorithm to obtain conformations. For 99.9% of the sets of distances, all triangle inequalities held. For 94% of the molecular graphs, the algorithm succeeded which is 8 pp higher than the success rate we observed for RDKIT. For each molecular graph in a test set, we generated 50 conformations with each method. This took DL4CHEM, RDKIT, and GRAPHDG on average around hundreds of milliseconds per molecule.8 In contrast, a single conformation in the ISO17 dataset takes around a minute to compute. In Fig. 6, an overlay of these conformations of six molecules generated by the different methods is shown. It can be seen that RDKIT’s conformations show too little variance, while DL4CHEM’s structures are mostly invalid, which is due in part to its failure to predict the correct interatomic angles. Our method slightly overestimates the structural variance (see, for example, Fig. 6, top row, second column), but produces conformations that are the closest to the ground-truth.

heading: 5.3 CALCULATION OF MOLECULAR PROPERTIES
text: We estimate expected molecular properties for molecular graphs from the test set with N = 50 conformational samples each. Due to their poor quality, we could not compute properties O(x), including the energy E(x), for conformations generated with DL4CHEM, and thus, this method is excluded from this analysis. In Table 2, it can be seen that RDKIT and GRAPHDG perform
8All simulations were carried out on a computer equipped with an i7-3820 CPU and a GeForce GTX 1080 Ti GPU.
similarly well (see Appendix A.2 for computational details). However, both methods are still highly inaccurate for Eelec (in practice, an accuracy of less than 5 kJ/mol is required). Close inspection of the conformations shows that, even though GRAPHDG predicts the most accurate distances overall, the variances of certain strongly constrained distances (e.g., triple bonds) are overestimated so that the energies of the conformations increase drastically.

heading: 6 LIMITATIONS
text: The first limitation of this work is that the CVAE can sample (with low probability) invalid sets of distances for which there exists no 3D structure. Second, the CONF17 benchmark covers only a small portion of chemical space. Finally, a large set of auxiliary edges would be required to capture long-range correlations (e.g., in proteins). Future work will address these points.

heading: 7 CONCLUSIONS
text: We presented GRAPHDG, a transferable, generative model that allows sampling from a distribution over molecular conformations. We developed a principled learning representation of conformations that is based on distances between atoms. Then, we proposed a challenging benchmark for comparing molecular conformation generators. With this benchmark, we show experimentally that conformations generated by GRAPHDG are closer to the ground-truth than those generated by other methods. Finally, we employ our model as a proposal distribution in an IS integration scheme to estimate molecular properties. While orbital energies and the dipole moments were predicted well, a larger and more diverse dataset will be necessary for meaningful estimates of electronic energies. Further, methods have to be devised to estimate how many conformations need to be generated to ensure all important conformations have been sampled. Finally, our model could be trained on conformational distributions at different temperatures in a transfer learning-type setting.

heading: A APPENDIX

heading: A.1 CONF17 BENCHMARK

heading: A.1.1 DATA GENERATION
text: The ISO17 dataset (Schütt et al., 2017a) was processed in the following way. First, conformations in which the molecular connectivity was modified (i.e., bonds were broken or new ones are formed) were discarded. For this, the tool XYZ2MOL (Jensen, 2019) was employed. Second, the molecular graphs were augmented by adding auxiliary edges for reasons described in Section 2.1. Auxiliary edges between all second neighbors were added. This can lead to a slight over-specification of the system’s geometry, however, this did not pose a problem in our experiments. In addition, auxiliary edges between third neighbors were added to fix dihedral angles. Since there are potentially many ways of specifying a dihedral angle in a molecular system, we resorted to the works of Riniker & Landrum (2015) and Guba et al. (2016) to decide where to place edges between third neighbors.
A.1.2 INPUT FEATURES
Below we list the node and edges features in the CONF17 benchmark.

heading: A.1.3 MODEL ARCHITECTURE
text: The full model is available online https://figshare.com/s/1b42bf865bd78c457354. In following, the hyperparameters of our model are specified:
Activations throughout this paper: ReLU; Lv , Le: 10; Fenc,v: neural network with depth 2, width 20; Fenc,e: neural network with depth 3, width 60; Fdec,v , Fdec,e: neural networks with depth 2, width 70; {MP(t)enc}Tt=1, {MP (t) dec}Tt=1: MPNN width depth 1 and three multi-head attention heads, T = 3, for node and edge updates neural networks with depth 2, and width 70 were used. Renc, Rdec: neural networks with depth 2, width 70. Batch size: 16 (conformations);

heading: A.2 COMPUTATIONAL DETAILS

heading: A.2.1 QUANTUM-CHEMICAL CALCULATIONS
text: All quantum-chemical calculations were carried out with the PySCF program package (version 1.5) (Sun et al., 2018) employing the exchange-correlation density functional PBE (Perdew et al., 1996), and the def2-SVP (Weigend & Ahlrichs, 2005; Weigend, 2006) basis set.
Conformations generated by DL4CHEM did not succeed as some atoms were too close to each other. Self-consistent field algorithms in quantum-chemical software such as PySCF do not converge for such molecular structures.
With quantum-chemical methods, we calculate several properties that concern the states of the electrons in the conformation. These are the total electronic energy Eelec, the energy of the electron in the highest occupied molecular orbital (HOMO in eV) HOMO, the energy of the lowest unoccupied molecular orbital (LUMO in eV) LUMO, and the norm of the dipole moment µ (in debye).

heading: A.2.2 EUCLIDEAN DISTANCE GEOMETRY
text: We refer the reader to Havel (2002) for theory on EDG, algorithms, and chemical applications. In summary, the EDG procedure consists of the following three steps:
1. Bound smoothing: extrapolating a complete set of lower and upper limits on all the distances from the sparse set of lower and upper bounds.
2. Embedding: choosing a random distance matrix from within these limits, and computing coordinates that are a certain best-fit to the distances.
3. Optimization: optimizing these coordinates versus an error function which measures the total violation of the distance (and chirality) constraints.
We use the EDG implementation found in RDKIT (Riniker & Landrum, 2015) with default settings.

heading: A.3 GENERATION OF CONFORMATIONS

heading: A.4 DISTRIBUTIONS OVER DISTANCES
text: Below, the marginal distributions of the distances for a variety of molecular graphs are shown.

Review summaries:
The authors propose a generative model designed for molecules, which is essentially a conditional variational auto-encoder. The model learns to generate, when conditioning on a molecule graph, the distribution of distances between each of the atoms and its second and third neighbour. Finally, using these information new molecules can be generated, which satisfy the generated distances. In the experiments the authors show the effectiveness of the method.