Predicting Molecular Conformation via Dynamic Graph Score Matching
Year: 2021

Abstract:
Predicting stable 3D conformations from 2D molecular graphs has been a longstanding challenge in computational chemistry. Recently, machine learning approaches have demonstrated very promising results compared to traditional experimental and physics-based simulation methods. These approaches mainly focus on modeling the local interactions between neighboring atoms on the molecular graphs and overlook the long-range interactions between non-bonded atoms. However, these non-bonded atoms may be proximal to each other in 3D space, and modeling their interactions is of crucial importance to accurately determine molecular conformations, especially for large molecules and multi-molecular complexes. In this paper, we propose a new approach called Dynamic Graph Score Matching (DGSM) for molecular conformation prediction, which models both the local and long-range interactions by dynamically constructing graph structures between atoms according to their spatial proximity during both training and inference. Specifically, the DGSM directly estimates the gradient fields of the logarithm density of atomic coordinates according to the dynamically constructed graphs using score matching methods. The whole framework can be efficiently trained in an end-to-end fashion. Experiments across multiple tasks show that the DGSM outperforms state-of-theart baselines by a large margin, and it is capable of generating conformations for a broader range of systems such as proteins and multi-molecular complexes.

heading: 1 Introduction
text: Graph-based representations of molecules has become prevalent in a variety of tasks such as property prediction [13, 31] and molecule generation [17, 33, 45]. However, a more natural and intrinsic representation of a molecule is its 3D geometry or conformation, which represents a molecule as a set of 3D coordinates. The 3D representation of molecules is central to many tasks, such as molecular properties prediction and virtual screening. Nevertheless, determining the conformation of a molecule remains a challenging task — both computational approaches, e.g., molecular dynamics (MD) [9], and experimental approaches, e.g., crystallography, are expensive and time-consuming.
Recently, machine learning approaches have demonstrated promising performance for molecular conformation generation. Pioneering methods such as GRAPHDG [36] and CGCF [43] first predict interatomic distances between bonded atoms and then solve 3D coordinates from the predicted distances via a post-processing algorithm. Very recently, Shi et al. proposed the CONFGF [34], which employs the score matching technique [38] to learn pseudo-forces between bonded atoms and iteratively applies the forces to a randomly initialized 3D structure until convergence. CONFGF gets
*Equal contribution. Work was done during Shitong’s internship at Mila.
35th Conference on Neural Information Processing Systems (NeurIPS 2021), virtual.
rid of the two-stage fashion in prior works and significantly improves the performance. Nevertheless, these approaches have a common major limitation — they mainly focus on modeling the local interactions between bonded atoms defined by the input molecular graphs but fail to capture long-range interactions between non-bonded atoms1, as they only model distances (or gradients) between bonded atoms. While in molecular mechanics, the potential energy of a molecule that alters conformations can be modeled as a sum of four parts [22]:
E = Ebond + Eangle + Etorsion + Enon-bonded, (1)
where Ebond, Eangle, and Etorsion model local interactions between bonded atoms, which are modeled in previous methods [34, 36, 43]. Long-range interactions between non-bonded atoms, denoted as Enon-bonded, are also non-trivial, which shape the molecular geometry via non-negligible electrostatic forces or van der Waals forces, etc. For multi-molecular complexes, non-bonded interactions dominate complexes’ geometry. An ideal solution to conformation generation should therefore capture both the local and long-range interactions. In Figure 1, we present three typical molecular systems where long-range interactions play a key role in determining their conformations.
To tackle the aforementioned challenge of modeling long-range interactions, in this paper, we propose the Dynamic Graph Score Matching (DGSM) for molecular conformation generation, following the principle of CONFGF [34] that learns the gradients of the logarithm density of atomic coordinates. Instead of relying on the static input molecular graph as existing work, the basic idea is to dynamically construct graph structures between atoms based on their spatial proximity during both training and inference. This allows the model to (1) dynamically learn molecular graph representations with evolved graph structures that take long-range interactions into consideration, and (2) dynamically determine a set of interatomic distances that contribute to gradients of the current atomic coordinates. Specifically, the edges in the dynamic graph consist of two parts. The first part of edges are determined by covalent bonds, which capture local interactions between atoms (Ebond, Eangle and Etorsion). The second part of edges are determined dynamically by spatial proximity between atoms at each training or sampling step, i.e., two atoms are connected as long as they are proximal, no matter whether they are bonded. Such a strategy is able to effectively capture non-local interactions (Enon-bonded) since the magnitude of long-range interactions is inversely correlated with distances between atoms [30]. It remains meanwhile scalable as we avoid connecting all the atom-pairs, which has quadratic complexity. In addition, modeling non-bonded interactions enable the model to sample conformations for multi-molecular complexes, which represent a broader range of problems.
We conduct extensive experiments and compare DGSM against previous state-of-the-art methods on both standard conformation generation and property prediction tasks. Numerical results show that DGSM outperforms previous methods by a clear margin, confirming the benefit of modeling long-range interactions. Besides, to further demonstrate the advantage of DGSM, we bring attention to two more challenging tasks — protein sidechain conformation prediction and multi-molecular complex structure prediction. These two new tasks represent two classes of practical challenges: predicting structures for macro-molecules and multi-molecular complexes.

heading: 2 Related Work
text: Prior works on conformation generation mainly rely on molecular dynamics (MD) [9], where new conformations are sequentially generated based on an initial conformation and a physical model for interatomic potentials [25, 27]. Although capable of accurately sampling equilibrium conformations, these methods are computationally intensive, especially for large molecular systems [2, 35], e.g.,
1Note that the receptive field of graph neural networks is much smaller than the diameter of molecular graphs in large systems, e.g., protein.
proteins. Another category of approaches leverage distance geometry [8] and fix distances between atoms to idealized values heuristically [4], which are much faster but less accurate.
Recently, a variety of deep generative models have been proposed for molecular conformation generation, which strike a good balance between computational efficiency and accuracy. Among these methods, Mansimov et al. [24] first propose a variational autoencoder to directly generate 3D atomic coordinates. Albeit simple, this method fails to model the roto-translation equivariance of molecular conformations, leading to unsatisfactory performance. To preserve roto-translation equivariance, Simm and Hernandez-Lobato [36] and Xu et al. [43] first model the molecular distance geometry and then reconstruct atomic coordinates from generated distances by solving an optimization problem. The state-of-the-art method CONFGF [34] estimates pseudo-forces acting on atoms and generates conformations via Langevin MCMC [42], which bypasses the two-stage fashion in previous works and enhances the performance significantly. Two concurrent works [12, 44] exist which generate conformations in end-to-end fashion via geometry elements assembly and bilevel programming respectively. Recently there has also been attempt to use reinforcement learning for conformation search [14]. Such a method is incapable of modeling bond lengths explicitly, and is fundamentally different from other approaches. To summarize, all of the previous methods focus mainly on modeling the local interactions based on the static input molecular graphs (or augmented graphs by adding auxiliary edges between atoms that are two- and three-hops away) and overlook the long-range non-bonded interactions between atoms. In contrast, our DGSM explicitly models both the local and long-range interactions via dynamic graph score matching and effectively addresses the above issue.

heading: 3 Preliminaries

heading: 3.1 Notations and Problem Formulation
text: Notations. Let G = 〈V, E〉 be a molecular graph, where V = {v1, v2, · · · , v|V|} is the set of nodes representing atoms, and E = {eij | (i, j) ⊆ V × V} is the set of edges representing inter-atomic bonds in the molecule. Each node vi ∈ V is labeled with atomic attributes, e.g., the element type Zi and the atomic coordinate ri ∈ R3. Each edge eij ∈ E is labeled with a bond type. The conformation of the molecular graph G can be represented as a matrixR ∈ R|V|×3. The distances between all pairs of atoms can be represented as a matrixD ∈ R|V|×|V|, whereDij := dij = ‖ri − rj‖2 denotes the Euclidean distance between the positions of vi and vj . Following previous work [34, 36, 43], we expand the original molecular graph by adding auxiliary edges between atoms that are second and third neighbors in G to reduce the degrees of freedom in 3D coordinates. Problem Formulation. Given a molecular graph G = 〈V, E〉, the task of molecular conformation generation is the conditional generation of conformationsR = [r1; r2, · · · ; r|V|] ∈ R|V|×3 based on G, while being able to capture long-range interactions between non-bonded atoms. Note that G may has multiple connected components, e.g., protein-ligand complexes and multi-molecular complexes.

heading: 3.2 Score-Based Generative Modeling
text: Score-based generative modeling [15, 38–40] is a class of generative models that has recently proven effective in a variety of tasks, ranging from image generation [15, 40], audio synthesis [7, 21] to shape generation [6, 23]. For any continuously differentiable probability density p(x), we define its score function s(x) as∇x log p(x), i.e., the direction where the logarithm data density grows most rapidly. Score-based generative modeling perturbs the data with different levels of Gaussian noise and jointly estimates the score function of p(x) using neural networks. Samples are then generated by sampling from a sequence of decreasing noise levels with Langevin dynamics [42].
Formally, given a data distribution pdata(x), let {σi}Li=1 be a sequence of noise levels that satisfies σ1 > σ2 > · · · > σL and σi/σi−1 = γ. Consider a series of noise distributions pσi(x̃ | x) := N (x̃;x, σ2i I), and denote the corresponding perturbed data distribution as pσi(x̃) := ∫ pσi(x̃ | x)pdata(x)dx. Song and Ermon [38] propose to jointly approximate the score function of each noise level, denoted by sθ(x, σi), with the following objective:
θ∗ = argminθ 1
2L
L∑
i=1
σ2i Epdata(x)Epσi (x̃|x) [∥∥∥sθ(x̃, σi)−∇x̃ log pσi(x̃ | x) ∥∥∥ 2
2
] . (2)
Assuming sufficient data and model capacity, the optimal noise conditional score network sθ∗(x, σi) matches∇x log pσi(x) almost everywhere [38]. After training score networks, Song and Ermon [38] run annealed Langevin dynamics for each pσi(x) sequentially, where samples from each noise level serve as initializations for Langevin dynamics of the next noise level. Given σL small enough, the final samples from pσL(x) approximate to samples from pdata under minor conditions.

heading: 4 Model
text: Our approach treats the conformation generation as sequentially moving atoms towards high-density regions guided by pseudo-forces, i.e., gradients of atoms. Following Shi et al. [34], we leverage the denoising score matching [38, 41] to approximate the gradients of the logarithm density of atomic coordinates, denoted as ∇R log p(R | G). To model atomic gradients that are sensitive to both the local and long-range interactions (Eq. 1) and inspired by the fact that long-range interactions decrease rapidly as distances increase, we propose to dynamically construct graph structures with non-bonded edges between atom pairs within a distance based on the current spatial proximity. In this way, we enable the model to effectively capture long-range non-bonded interactions while avoid connecting all atoms, which is computationally expensive. To ensure that the distribution of graph structures during training matches with that during generation, we devise a dynamic graph score matching algorithm, where graph structures are also dynamically determined during training depending on added perturbations. The whole framework is illustrated in Figure 2 and Figure 3. Below we describe the framework of score estimation for Cartesian coordinates in Section 4.1, the dynamic graph score matching algorithm in Section 4.2, and the generation procedure in Section 4.3.

heading: 4.1 Score Estimation for Cartesian Coordinates
text: Our goal is to learn the gradients of the logarithm density (score) of atomic coordinates, i.e., ∇R log p(R | G). Directly parameterizing score networks on absolute Cartesian coordinates with Graph Neural Networks (GNNs) [11, 13, 19, 29] relies on the arbitrary choice of rotation and translation [36, 43], which are non-essential degrees of freedom for effecting conformational changes in molecular systems. Therefore, we explicitly exclude them from the model, by first estimating scores for a set of dynamically determined interatomic distances, and then backpropagating gradients from distances to Cartesian coordinates via differentiation.
Given a molecular graph G = 〈V, E〉, the probability p(R | G) of a conformationR is subject to the Boltzmann distribution and is proportional to exp (−E(R)/kBT ), whereE(R) is the conformational energy, kB is the Boltzmann constant, and T is the temperature. We assume the logarithm density of a conformation, i.e., the negative conformational energy up to a constant, can be parameterized as a function of interatomic distances, conditional on molecular graph G:
log pθ(R | G) := fG(e1(R), e2(R), · · · , eK(R)), (3) where {ek : R|V|×3 → R}Kk=1 is a set of functions that calculate K interatomic distances, which are invariant under the rotation and translation of R. And fG : RK → R is a graph neural network that predicts the negative conformational energy based on distances and the 2D graph representation G. Using interatomic distances for energy prediction is favored in existing literature [16, 20, 31, 34], as it preserves the 3D rotation and translation symmetries of molecular systems. Since
{ek(R)}Kk=1 are continuously differentiable with respect to the Cartesian coordinatesR, the gradients of logarithm density of interest, i.e.,∇R log pθ(R | G), is interrelated with the logarithm density of each interatomic distance via chain rule:
∀i, sθ(R)i := ∂fG(e1(R), e2(R), · · · , eK(R))
∂ri
=
K∑
k=1
∂fG(e1(R), e2(R), · · · , eK(R)) ∂ek(R) · ∂ek(R) ∂ri
=
K∑
k=1
sθ(ek(R)) · ∂ek(R)
∂ri ,
(4)
where sθ(R)i denotes ∇ri log pθ(R | G), sθ(ek(R)) denotes ∇ek(R) log pθ(R | G), and ∂ek(R)∂ri can be calculated efficiently in closed form (see supplementary material for the full derivation).
Motivated by the above equation, we first train a noise conditional score network to jointly predict the score of interatomic distances, i.e., {sθ(ek(R), σ)}Kk=1. After training the noise conditional score network, the gradients of the logarithm density of atomic coordinates, i.e., sθ(R, σ), can be estimated via Eq. 4. We have the following proposition (proof in supplementary material): Proposition 1 (Roto-Translation Equivariance). With the assumption that we can parameterize log pθ(R | G) as a function of interatomic distances, conditional on molecular graph G (Eq. 3), the score function sθ(R) defined in Eq. 4 is roto-translation equivariant.
Remarkably, the choice of {ek(R)}Kk=1 is flexible under this framework and can be carefully designed for specific goals. An ideal set of interatomic distances should capture both the local and long-range interactions between atoms (Eq. 1). Proposition 2 (Connection with CONFGF). The recent proposed CONFGF [34] is a special case of our approach, where they only model local distances between the first-order, the second-order, and the third-order neighbors, i.e., {ek}Kk=1 map the conformation to a set of distances between bonded atoms. Therefore, CONFGF fails to capture long-range interactions between non-bonded atoms.

heading: 4.2 Dynamic Graph Score Matching with Noise Conditional Score Networks
text: In this section, we describe the proposed dynamic graph score matching for interatomic distances, with the goal of modeling both the local and long-range interactions. To ensure that the learned score functions cover all regions with different graph structures, we dynamically construct graph structures with non-bonded edges between atoms during training, based on added perturbations. Following Song and Ermon [38], we train a noise conditional score network to jointly estimate scores for perturbed distributions of a set of dynamically-determined interatomic distances, i.e., {sθ(ek(R), σ)}Kk=1, and parameterize the score network with the message passing neural network (MPNN) [13].
Dynamic Score Matching. To capture long-range interactions between non-bonded atoms in a molecular system, a naive way is to treat the molecular graph as a fully-connected graph and model the gradients of logarithm density of distances between all pairs of atoms. However, such a practice is computationally expensive especially for large systems, e.g., proteins, and is sometimes unnecessary, e.g., van der Waals interactions decay rapidly as distances increase. As a remedy, we set a cutoff distance and assume each atom only interacts with all atoms within the cutoff distance, ignoring all interactions out of the considered sphere. This is a very popular strategy in computational chemistry that strikes a good balance between efficiency and accuracy [26, 31].
Formally, consider a molecular graph G = 〈V, E〉 with distances between all pairs of atoms D ∈ R|V|×|V| computed from its conformation R ∈ R|V|×3. For a given noise level σ, we perturb the distances D with Gaussian noise at each training step on the fly, and then augment the original graph structure with non-bonded edges between atom pairs within a certain threshold distance:
∀(i, j), D̃ij ∼ N (Dij , σ2), Eσ = E ∪ {eij | D̃ij < δ}, dσ = {Dij | eij ∈ Eσ}, d̃σ = {D̃ij | eij ∈ Eσ},
(5)
where Eσ is the constructed graph structure for noise level σ, and δ is a hyper-parameter that controls the radius of long-range interactions. We empirically verify that a 10 Å cutoff is sufficient for systems
we are studying, which is consistent with some experimental results in molecular dynamics [26]. dσ and d̃σ denote the original and perturbed interatomic distances in augmented graph structure respectively. Hereafter, we omit the subscript for simplicity and use E , d, and d̃ instead, assuming all graphs are dynamically constructed during training and sampling.
With the above strategy, the graph structure of a specific molecular graph G is variadic depending on added perturbation, and all graph structures are possible as long as we sample sufficient enough noise. This will result in (1) a dynamically-determined graph structure for message passing and representation learning, which takes long-range interactions into consideration, and (2) a dynamicallydetermined set of interatomic distances, i.e., {ek(R)}Kk=1, for score estimation, which contributes to gradients of atomic coordinates according to Eq. 4. Note that the vanilla implementation of Eq. 5 requires computing all distances between atom pairs. In practice, to avoid quadratic complexity, we pre-filter distant neighbors before adding perturbations for each atom by constructing radius graph with 2δ threshold, and empirically verify that it performs efficiently and effectively.
Parameterizing with MPNNs. Let {σi}Li=1 be a sequence of noise levels. Our goal is to learn a noise conditional score network to jointly estimate the scores of all perturbed distance distributions, i.e., ∀σ ∈ {σi}Li=1 : sθ(d̃, σ) ≈ ∇d̃ log pσ(d̃ | G), where pσ(d̃ | G) = ∫ p(d | G)N (d̃ | d, σ2I). Following suggestions of Song and Ermon [38], we parameterize the score network with a MPNN as follows (see supplementary material for the full architecture):
hnodei = MPNN(G, E , D̃)i, ∀vi ∈ V hedgeij = Concat(h node i ,h node j ), ∀eij ∈ Eσ
sθ(d̃, σ)ij = sθ(d̃)ij/σ = MLP(h edge ij )/σ, ∀eij ∈ Eσ
(6)
where {hnodei } |V| i=1 are node embeddings computed by the MPNN based on the dynamically constructed molecular graph and the perturbed distances, {hedgeij } are embeddings for each edge in E , and sθ(d̃, σ)ij is the predicted score for interatomic distance D̃ij (eij ∈ E). The noise conditional network can be jointly optimized with the following objective [38]:
θ∗ = argminθ 1
2L
L∑
i=1
σ2i Ep(d|G)Epσi (d̃|d,G) [∥∥∥sθ(d̃) σi + d̃− d σ2i ∥∥∥ 2 2 ] , (7)
where all expectations can be efficiently estimated using empirical averages.

heading: 4.3 Generation
text: After training the noise conditional score network, molecular conformations can be generated via annealed Langevin dynamics [38], guided by the gradients of atomic coordinates. The gradients can be computed via Eq. 4 based on dynamically constructed graph structures at each step of stochastic update, which allows model to effectively capture both the local and long-range interactions that contribute to the atomic gradients. Formally, given a molecular graph G, we first sample an initial conformation R0 from a fixed prior distribution. We here take the prior distribution as a standard
Gaussian N (R0 | 0, I). Then, we update the conformation by running T steps of Langevin dynamic to get a sample from each noise conditional score network sθ(R, σi) sequentially with a special step size schedule αi = ε · σ2i /σ2L. Samples from each noise level are used to initialize Langevin dynamics for the next noise level. At each sampling step t, we first construct graph structures with non-bonded edges within a given distance δ based on the current pairwise distances Dt−1 computed fromRt−1, and then get a set of interatomic distances dt−1 for score estimation:
Et−1 = E ∪ {eij |Dt−1,ij < δ}, dt−1 = {Dt−1,ij | eij ∈ Et−1}. (8) The conformation is then updated using the gradient information from the score network (Eq. 4). We provide the pseudo-code in Algorithm 1.
5 Experiments Algorithm 1 Annealed Langevin dynamics [38] Require: G = 〈V, E〉, {σi}Li=1, δ, ε, T .
1: Initialize conformationR0 2: for i← 1 to L do 3: αi ← ε · σ2i /σ2L . αi is the step size. 4: for t← 1 to T do 5: Et−1,dt−1 ← aug(E ,Rt−1, δ) . Eq. 8 6: sθ(Rt−1, σi)← get(sθ(dt−1, σi)) . Eq. 4 7: Draw zt ∼ N (0, I) 8: Rt ← Rt−1 + αisθ(Rt−1, σi) + √ 2αizt
9: end for 10: R0 ← RT 11: end for Return: Generated conformationRT .
Following previous works [34, 36, 43] on conformation generation, we evaluate the proposed DGSM using the following two standard tasks: Conformation Generation (Section 5.1), and Property Prediction (Section 5.2). To further demonstrate DGSM’s capability of modeling longrange interactions, we evaluate it on two more challenging benchmark tasks: Protein Sidechain Conformation Generation and Multi-molecular Complex Conformation Generation (Section 5.3). We describe experimental setups in task-specific sections.

heading: 5.1 Conformation Generation
text: Setup. This task evaluates the model’s capability to generate stable molecular conformations by measuring both accuracy and diversity of generated conformations. Following previous works [34, 43], we use the GEOM-QM9 and GEOM-Drugs [1] datasets for this task. We use the train-test split provided by [34]. The train splits of GEOM-QM9 and GEOM-Drugs both contain 40,000 molecules, each with 5 conformations for training, or 200,000 conformations in total. The test split of GEOM-QM9 contains 200 molecules with 22,408 conformations, and the test split of GEOM-Drugs contains 200 molecules with 14,324 conformations.
We compare DGSM against 5 state-of-the-art baselines: RDKIT [28], CVGAE [24], GRAPHDG [36], CGCF [43] and CONFGF [34]. For each molecule in the test set, we sample twice as many conformations as its reference conformations. We use the matching score (MAT) to measure the accuracy of generated conformations, and coverage score (COV) to measure the diversity following [34, 43]. Both metrics are based on Root Mean Squared Deviations (RMSD) between molecules, taking symmetries into account (see supplementary material for the details of metrics).
Results. We report the mean and median COV and MAT scores over all the molecules in the test split on GEOM-QM9 and GEOM-Drugs datasets. As shown in Table 1, DGSM consistently outperforms
all the baselines. Notably, both DGSM and CONFGF are score-based models, but DGSM achieves better performance. The difference between them is that DGSM successfully takes long-range interactions into consideration via dynamic graph score matching. This confirms the significant benefit of modeling long-range interactions. We present several conformations generated by different approaches in Figure 4, which shows that DGSM successfully captures the long-range interactions in highlighted areas while the other baselines fail, resulting in distorted structures in those areas.
5.2 Property Prediction
Setup. This task demonstrates how generative models for molecular conformations can be applied to property prediction as a downstream task. It also provides an assessment on the quality of generated conformations in a different light. We estimate the ensemble properties [1] of a molecular graph by aggregating its conformational properties following [34]. In specific, we first use the models
to generate 50 conformations for each molecular graph in a subset of GEOM-QM9 [34], and use PSI4 [37], a quantum chemical toolkit, to calculate each conformation’s energy and HOMO-LUMO gap. Then, we calculate average energy E, lowest energy Emin, average gap ∆ , minimum gap ∆ min, and maximum gap ∆ max from the conformational energy and gap. We evaluate the accuracy of estimated ensemble property by measuring their mean absolute errors (MAE) to the ground truth values. CVGAE is excluded in this task as its performance is poor, which is also reported in [36, 34].
Results. Table 2 shows that DGSM outperforms other machine learning-based methods by a clear margin. DGSM’s estimation of average energyE and minimum gap ∆ min is close to RDKIT but still outperforms the most competitive ML-based method CONFGF. The calculation of conformational energy is highly sensitive to changes in geometry — even a subtle deviation in bond lengths leads to significant energy change [36]. Therefore, the superior performance of DGSM indicates that it generates much more accurate conformations than other methods, leading to more accurate property estimation. This validates again the effectiveness of modeling long-range interactions.

heading: 5.3 Large Molecule and Multi-molecular Modeling
text: Protein Sidechain Conformation This task is to predict protein sidechain conformations based on its backbone structures. Compared to conventional molecular conformations generation in previous sections, the main challenge of this task is two-fold: (1) large number of atoms, which prohibits constructing complete graphs that grow quadratically to model long-range interactions.
(2) covalent bonds are sparse, which limits the power of the edge augmentation techniques in previous work. DGSM tackles these two challenges via dynamic graph score matching as introduced.
We use the SidechainNet [18] dataset for this task and follow the official train-test splits. We compare DGSM with the state-of-the-art conformation generation model CONFGF. Despite that there are some machine learning-based methods specialized in protein sidechain structure prediction [10], they are built upon rotamer libraries [5], which incorporates a lot of domain knowledge. Thus, our method is not comparable with them. The main purpose of this task is to justify the effectiveness of DGSM for large molecules. For each protein, we generate 5 sidechain conformations with different initialization, and calculate the mean and min RMSD between the ground-truth conformation and the generated conformations. We report the overall mean and min RMSD scores by averaging scores of each protein in the test set in in Table 3, which shows that DGSM achieves better performance than previous state-of-the-art model. We also present an example in Figure 5(a), and we can see that the predicted conformation is consistent with the ground truth in major parts.
Multi-molecular Complex Conformation This task is to predict conformations for multi-molecular complexes. A multi-molecular complex is made up of multiple molecules and there is no covalent bonds between them. Long-range interactions dominate the structure of multi-molecular complexes. The purpose of this task is to demonstrate DGSM’s potential application to a broader range of problems and provide a novel benchmark for conformation generation. We use the quantum chemical software xtb [3] to construct a dataset consisting of 24 water-organic complexes each with several hundreds of conformations, and leave out 4 complexes for testing (see supplementary material for details). We do not report RMSD-based metrics such as COV and MAT because the structures of multi-molecular complexes are highly flexible. Two set of generated examples are presented in Figure 5(b). We observe that water molecules are placed
regularly around the solute organic molecule. Notably, hydrogen bonds (between water and the solute, and between water and water) are formed correctly. This can also be evidenced in the histogram of Hydrogen-Oxygen distances (Figure 6), where there is a peak between 1.5Å and 2.5Å, i.e., the range of hydrogen bond length between Hydrogen and Oxygen.

heading: 6 Conclusion and Future Work
text: We propose DGSM, a novel score-based approach for generating equilibrium molecular conformations. DGSM is capable of modeling both the local and long-range interactions in molecular systems, by dynamically constructing graph structures based on spatial proximity between atoms during both training and inference. We also devise a dynamic graph score matching algorithm to effectively estimate atomic gradients, where graph structures are dynamically determined depending on added perturbations. Extensive experiments over two standard tasks and two original tasks show that DGSM outperforms the state-of-the-art method by a large margin, confirming the significant benefit of modeling long-range interactions. In the future, we plan to apply our approach to the more challenging problem of protein structure prediction.
Acknowledgments and Disclosure of Funding
We would like to thank all the reviewers for the insightful comments. This project is supported by the Natural Sciences and Engineering Research Council (NSERC) Discovery Grant, the Canada CIFAR AI Chair Program, collaboration grants between Microsoft Research and Mila, Samsung Electronics Co., Ldt., Amazon Faculty Research Award, Tencent AI Lab Rhino-Bird Gift Fund and a NRC Collaborative R&D Project (AI4D-CORE-06). This project was also partially funded by IVADO Fundamental Research Project grant PRF-2019-3583139727.

Review summaries:
The authors develop an improved method for 3D molecular conformation generation from 2D molecular graphs that can more directly incorporate non-bonded, topologically long-range interactions into the generation process. Their starting point is a recent method that leverages score matching with a GNN based on the 2D molecular graph topology, or more specifically, a GNN that is trained to estimate the gradient of the noise-perturbed log density of interatomic distances on the graph edges. They follow the same framework, but add additional edges to the graph topology based on the current perturbed distances when they fall below a certain cutoff (10 Angstroms). Through their experiments, they show that this relatively modest modification signifcantly improves generation performance of small molecule conformers and can be applied to larger and more complex systems such as protein side chain packing and multi-molecular complexes.

This paper introduces a new approach for generating molecular conformations, using score-based generative modeling.  The key contribution over prior work is the dynamic addition (during both training and inference) of long-range interactions between non-bonded atoms that are not proximal in the molecular graph.  As these interactions are often critical to determining the underlying molecular conformations, the authors contend that their addition leads to improved performance on a range of tasks.  In particular, they demonstrate improved performance over previous state-of-the-art on the generation of conformations for small molecules, protein side chains, and multi-molecular complexes, as well as small-molecule ensemble property prediction.

This paper proposes to generate molecular conformation by score matching. The proposed method, called DGSM, learns a score network to predict gradient field with respect to a predicted conformation. The score network is parameterized as a message passing network over a dynamic graph between atoms. The graph is constructed based on pairwise distance between atoms in the current conformation. Finally, DGSM uses the learned gradient field along with Langevin dynamics to sample molecular conformations at test time. DGSM is evaluated on two conformation generation benchmarks and achieved new state-of-the-art results.

This paper addresses the task of molecular conformer generation, i.e. generate the 3D conformer ensemble conditioned on the molecular graph. The introduced method, Dynamic Graph Score Matching (DGSM), is using score matching to generate 3D structures by explicitly capturing both local and long-range interactions. This is done using dynamic graphs constructed via atom coordinate perturbations using different noise levels as required by the denoising score matching procedure.  The score is estimated using message passing neural networks which operate on these dynamic graphs. At inference time, Langevin dynamics is used to sample conformations from the generated distribution. Empirically, the authors show clear improvements over previous machine learning (ML) models and over one popular open-source non-ML method (RDKit) on molecular conformer generation on two different tasks, as well as improvements on experiments on property prediction based on molecular conformations, protein sidechain conformation generation, and multi-molecular complex conformation.  In short, I think this approach is quite compelling and would be very impactful for future work related to graph conditional 3D generative models of various data in biology, chemistry, etc. The paper is very well written and the method is clean, addressing nicely using dynamic graphs the very important problem of modeling long-range interactions in 3D generation.