Predicting Molecular Conformation via Dynamic Graph Score Matching
Authors: Shitong Luo
Chence Shi
Minkai Xu
Jian Tang
Venue: NeurIPS 2021 Poster (2021)
Keywords: Molecular conformation generation
score matching
generative model

Abstract: Predicting stable 3D conformations from 2D molecular graphs has been a long-standing challenge in computational chemistry. Recently, machine learning approaches have demonstrated very promising results compared to traditional experimental and physics-based simulation methods. These approaches mainly focus on modeling the local interactions between neighboring atoms on the molecular graphs and overlook the long-range interactions between non-bonded atoms. However, these non-bonded atoms may be proximal to each other in 3D space, and modeling their interactions is of crucial importance to accurately determine molecular conformations, especially for large molecules and multi-molecular complexes. In this paper, we propose a new approach called Dynamic Graph Score Matching (DGSM) for molecular conformation prediction, which models both the local and long-range interactions by dynamically constructing graph structures between atoms according to their spatial proximity during both training and inference. Specifically, the DGSM directly estimates the gradient fields of the logarithm density of atomic coordinates according to the dynamically constructed graphs using score matching methods. The whole framework can be efficiently trained in an end-to-end fashion. Experiments across multiple tasks show that the DGSM outperforms state-of-the-art baselines by a large margin, 
and it is capable of generating conformations for a broader range of systems such as proteins and multi-molecular complexes.

Predicting Molecular Conformation via Dynamic Graph Score Matching
Shitong Luo*!, Chence Shiw2.3, Minkai 2u JJan Tang2.4.
!Peking University Milla Québec Al Institute 3 Université de Montréal 'HEC Montréal SCIFAR Al Research Chair luostâpku edu. chence. shi @umontreal.....mmmmmmmm.a..a minkai. xudumontreal ca, jian. tang@hec.coo ca
Abstract
Predicting stable 3D conformations from 2D molecular graphs has been a long- standing challenge in computational chemistry. Recently, machine learning ap- proaches have demonstrated very promising results compared traditional experi- mental and physics- bassed simulation methods. These approaches mainly focus on modeling the local interactions between neighboring atoms the molecular graphs and overlook the long-range interactions between non-bonded atoms. However, these non-bonded atoms may be proximal to each other in 3D space, and modeling their interactions is of crucial importance to accurately determine molecular con- formations, especially for large molecules and paper, we propose a new approach called Dynamic Graph Score Matching (DGSM) for molecular conformation prediction, which models both the local and long-range interactions by dynamically constructing graph structures between atoms according their spatial proximity during both training and inference. Specifically, the DGSM directly estimates the gradient fields of the logarithm density of atomic coordinates according to the dynamically constructed graphs using score matching methods. The whole framework can be efficiently trained in an end-to-end fashion. Experiments across multiple tasks show that the DGSM outperforms state-of-the- art baselines by large margin, and it is capable of generating conformations for a broader range of systems such as proteins and malli-molecular complexes.
complexes. In this
Introduction
Graph-based representations of molecules has become prevalent variety of tasks such as property However, more natural and intrinsic representation molecule is its geometry or conformation, which represents molecule as set of 3D coordinates. The 3D representation of molecules is central to many tasks, such as molecular molecule
prediction 13 31 and molecule generation [17
33 45
properties prediction and virtual screening. Nevertheless, determining the conformation remains challenging task both computational approaches, e.g., molecular dynamics (MD) [ and experimental approaches, e.g., crystallography, are expensive and time-consuming.
Recently, machine learning approaches have demonstrated promising performance for molecular conformation generation. Pioneering methods such as GRAPHDG (36] and CGCF (43) first predict interatomic distances between bonded atoms and then solve 3D coordinates from the predicted distances via post-processing algorithm. Very recently, Shi et al. proposed the CONFGR [34], which employs the score matching technique (38 to learn poeudo-forces between bonded atoms and iteratively applies the forces randomly initialized 3D structure until convergence. CONFGF gets
"Equal contribution. Work was done during Shitong's internship at Mila.
35th Conference on Neural Information Processing Systems (NcurIPS 2021), virtual.

---

(a) Small Molecule
(b) Protein
(c) Multi-molecular Complex
Figure 1: Three molecular systems where long-range interactions crucial for their conformations.
rid of the two-stage fashion in prior works and significantly improves the performance. Nevertheless, these approaches have common major limitation they mainly focus on modeling the local inter- actions between bonded atoms defined by the input molecular graphs but fail to capture long-range interactions between non-bonded atoms as they only model distances (or gradients) between bonded atoms. While in molecular mechanies, the potential energy of a molecule that alters conformations can be modeled as a sum of four parts [22]:
E Ebonn + Eaagle + Exorsion + Eman-
where Ebond, Eangle. and Eiorsion model local interactions between bonded atoms, which are modeled in previous methods [34] 36 43 Long-range interactions between non-bonded atoms, denoted as Enon-bonded: are also non-trivial, which shape the molecular geometry via non-negligibe electrostatic forces or van der Waals forces, etc. For multi-molecular complexes, non-bonded interactions dominate complexes' geometry. An ideal solution to conformation generation should therefore capture both the local and range interactions. In Figure [ present three typical molecular systems where long-range interactions play a key role in determining their conformations.
To tackle the aforementioned challenge of modeling long-range interactions, in this paper, we propose the Dynamic Graph Score Matching (DGSM) for molecular conformation generation, following the principle of CONFGF [4 that learns the gradients of the logarithm density of atomic coordinates. Instead of relying on the static input molecular graph as existing work, the basic idea is to dynamically construct graph structures between atoms based on their spatial proximity during both training and inference. This allows the model to (1) dynamically learn molecular graph representations with evolved graph structures that take long-range interactions into consideration, and (2) dynamically determine a set of interatomic distances that contribute to gradients of the current atomic coordinates. Specifically, the edges in the dynamic graph consist of two parts. The first part of edges are determined by covalent bonds, which capture local interactions between atoms (Fbond, Fangle and Euuesion. ) The second part of edges are determined dynamically by spatial proximity between atoms at each training or sampling step, i.e., two atoms are connected as long as they are proximal, no matter whether they bonded. Such strategy is able effectively capture non-local interactions (Enon-bondea) since the magnitude of long-range interactions is inversely correlated with distances between atoms (30). It remains meanwhile scalable as avoid connecting all the atom-pairs, which has quadratic complexity. In addition, modeling non-bonded interactions enable the model to sample conformations for multi molellla complexes, which represent a broader range of problems.
We conduct extensive experiments and compare DGSM against previous statc-of-the ttthhhhrrt art methods on both standard conformation generation and property prediction tasks. Numerical results show that DGSM outperforms previous methods by a clear margin, confirming the benefit of modeling long-range interactions. Besides, further demonstrate the advantage of DGSM, we bring attention to two more challenging tasks protein sidechain conformation prediction and multi-molecular complex structure prediction. These two new tasks represent classes of practical challenges: predicting structures for macro-molecules and multi-molecular complexes.
2 Related Work
Prior works on conformation generation mainly rely on molecular dynamics (MD) 9 where new conformations are sequentially generated based on an initial conformation and a physical model for interatomic potentials (25) 27). Although capable of accurately sampling equilibrium conformations, these methods are computationally intensive, especially for large molecular systems 2 35 c.g..
Note that the receptive field of graph neural networks is much smaller than the diameter of molecular graphs in large systems, e.g., protein.

---

proteins. Another category of approaches leverage distance geometry 8 and fix distances between atoms to idealized values heuristically (4, which are much faster but less accurate.
Recently, a variety of deep generative models have been proposed for molecular conformation generation, which strike a good balance between computational efficiency and accuracy. Among these methods, Mansimov et al. [24] first propose a variational autoencoder to directly generate 3D atomic coordinates. Albeit simple, this method fails to model the roto-translation equivariance of molecular conformations, leading to unsatisfactory performance. To preserve roto-translation equivariance, Simm and Hernandez- Lobato [36 and Xu et al. first model the molecular distance geometry and then reconstruct atomic coordinates from generated distances by solving an optimization problem. The state-of of-the-rrt method CONFGF [34] estimates pseudo-forces acting on atoms and generates conformations via Langevin MCMC 42 which bypasses the two-stage fashion in previous works
and enhances the performance significantly. Two concurrent works [12 44 exist which generate conformations in end-to-end fashion via geometry elements assembly and bilevel programming respectively. Recently there has also been attempt to use reinforcement learning for conformation search [4.. Such method is incapable of modeling bond lengths explicitly, and is fundamentally different from other approaches. To summarize, all of the previous methods focus mainly on modeling the local interactions based on the static input molecular graphs (or augmented graphs by adding auxiliary edges between atoms that are two- and three-hops away) and overlook the long-range non-bonded interactions between atoms. In contrast, our DGSM explicitly models both the local and long-range interactions via dynamic graph score matching and effectively addresses the above issue.
3 Preliminaries
3.1 Notations and Problem Formulation
Notations. Let = (V, be a molecular graph, where V {v1, U.. fado is the set of nodes representing atoms, and & {er (i.5) S V x V} the set of edges representing inter-atomic bonds in the molecule. Each node us V is labeled with atomic attributes, e.g., the element type Z; and the atomic coordinate r; E RRS. Each edge Eij E is labeled with a bond type. The conformation of the molecular graph 9 can be represented as a matrix R E RIVX33 The distances between all pairs of atoms can be represented as a matrix E IK |V), where D.j == dij ||: - ryil2 denotes the Euclidean distance between the positions of and Uy. Following previous work [34] 36 43 we
expand the original molecular graph by adding auxiliary edges between atoms that are second and third neighbors in G to reduce the degrees of freedom in 3D coordinates.
Problem Formulation. Given a molecular graph G = (V,E), the task of molecular conformation generation is the conditional generation of conformations
[r1; 72, Tyvll R [V]x3 based on G, while being able to capture long-range interactions between non-bddded atoms. Note that G may has multiple connected components, e.g., protein-ligaan complexes and molei-moleeular complexes.
3.2 Score-Based Generative Modeling
Score-based generative modeling 11 38-40) is class of generative models that has recently proven
effective in a variety of tasks, ranging from image generation [15 40), audio synthesis 7 21 to shape generation 6 23 For any continuously differentiable probability density p(x), we define its score function s( as V, x l(z), i.e., the direction where the logarithm data density grows most rapidly. Score-based generative modeling perturbs the data with different levels of Gaussian noise and jointly estimates the score function of p(x) using neural networks. Samples are then generated by sampling from a sequence of decreasing noise levels with Langevin dynamics 42
Formally, given a data distribution Pdina (x). let {o be a sequence of noise levels that satisfies σi > 02 >
T and a,/0;- 1 7. Consider a series of noise distributions Po, x)
" and denote the corresponding perturbed data distribution as Po: (z) = f Pa. (
x) P)dam (x) dxx. Song and Ermon [38] propose to jointly approximate the score function of each noise level, denoted by so (2, o;), with the following objective:
1
0" argming 2T.
Σ
o E Epno Ep,, (z)z)
80 a o.)) - Va log xx)
(2)
2
3

---

Perturbed D
Dynamically Masked D
Masked D D
6/3
0.3
Q.
2
0:3 3.5
1.7
IDD 8)
Loss
Apl D)
2.7 2.9
1.7
122
3.9
0.8
0.9
1.8
0.8
0.5
H
Score Network
The training procedure of the proposed DGSM. To model long-range interactions, the graph structures are dynamically determined by adding non- non-bonded edges based on added perturbations at each step. The bonded edges are marked by black border.
Figure 2:
Assuming sufficient data and model capacity, the optimal noise conditional score network sø- (z, og matches v log Po, (x) almost everywhere [38]. After training score networks, Song and Ermon [38 annealed Langevin dynamics for each Po, (a:) sequentially, where samples from each noise level serve as initializations for Langevin dynamics of the next noise level. Given σr, small enough, the final samples from Po, (a)) approximate samples from pdat under minor conditions.
4
Model
Our approach treats the conformation generation sequentially moving atoms towards high-density regions guided by pseudo-forces. i.e., gradients of atoms. Following Shi al. (34, we leverage the denoising score matching [38 41 to approximate the gradients of the logarithm density of atomic coordinates, denoted as n log p 9). To model atomic gradients that are sensitive both the local and long-range interactions (Eq. D and inspired by the fact that long-range interactions decrease rapidly as distances increase, we propose to dynamically construct graph structures with non-bonded edges between atom pairs within a distance based the current spatial proximity. In this way, enable the model to effectively capture long-range non-bonded interactions while avoid connecting all atoms, which is computationally expensive. To ensure that the distribution of graph structures during training matches with that during generation, we devise a dynamic graph score matching algorithm, where graph structures are also dynamically determined during training depending added perturbations. The whole framework is illustrated in Figure and Figure 3 Below we describe the framework of score estimation for Cartesian coordinates in Section 4.1 the dynamic graph score matching algorithm in Section 4.2 and the generation procedure Section
4.3
4.1 Score Estimation for Cartesian Coordinates
Our goal is learn the gradients of the logarithm density (score) of atomic coordinates, i.c., V log p( R G). Directly parameterizing score networks on absolute Cartesian coordinates with Graph Neural Networks (GNNs) [1 19 29] relies the arbitrary choice of rotation and transla- tion 33 43,, which are non-essential degrees freedom for effecting conformational changes in molecular systems. Therefore, explicitly exclude them from the model, by first estimating scores for a set of dynamically determined interatomic distances, and then backpropagating gradients from distances Cartesian coordinates via differentiation.
Given a molecular graph G = EV, the probability p(R 9) of conformation Boltzmann distribution and is proportional to exp (-E(R)/kB))) where E(R)) is the conformational energy, k is the Boltzmann constant, and 7 is the temperature. We assume the logarithm density of conformation, i.c., the negative conformational energy up to constant, be parameterized function of interatomic distances, conditional on molecular graph G:
subject the
log po R () fg(e) (R), (R), ,ex eek (R)),
where {ex RIVXK + RJKK are invariant under the rotation and translation R. And fg I - R graph neural network that predicts the negative conformational energy based on distances and the 2D graph representation G. Using interatomic distances for energy prediction is favored in existing literature []6 20 31 34), preserves the 3D rotation and translation symmetries of molecular systems. Since
set of functions that calculate K interatomic distances, which

---

{e)(R) are continuously differentiable with respect to the Cartesian coordinates R. the gradients of logarithm density of interest, i.c., Vn log (R g), is interrelated with the logarithm density of each interatomic distance via chain rule:
(e: R), e2 RR) (ex(",) ,x((R)
Vi, 80 R);
Ōr:
K afg(es (R), e2 "., ex (R)) (
dec (
Ör,
k=
dea dex(R)
Σ so(ex(R))
Ör;
where se (R), denotes Vr log [R| g). (e RR) denotes V (R) log po R 9), and Rex(R) can be calculated efficiently in closed form (see supplementary material for the full derivation).
ör;
Motivated by the above equation, we first train noise conditional score network to jointly predict the score of interatomic distances, i.c., { (ex [R),o] ] k= After training the noise conditional score network, the gradients of the logarithm density of atomic coordinates, i.e., so( R, o). can be estimated via Eq. 4 We have the following proposition (proof in supplementary material):
Proposition 1 (Roto-Translation Equivariance). With the assumption that we can parameterize log pe R 9) as a function of interatomic distances, conditional on molecular graph G (Eq. 3) the score function se (R) defined in Eq. 4 is noto-translation equivariant.
Remarkably, the choice of {e& [ is flexible under this framework and can be carefully designed for specific goals. An ideal set of interatomic distances should capture both the local and long-range interactions between atoms (Eq. D.
Proposition 2 (Connection with CONFGF). The recent proposed CONFGF (34] is a special case of our approach, where they only model local distances between the first-order, the second-order, and
K the third-order neighbors, i.e., {ex] k=1 f== map the conformation to set of distances between bonded atoms. Therefore, CONFGF fails to capture long-range interactions between non-bonded atoms.
4.2 Dynamic Graph Score Matching with Noise Conditional Score Networks
In this section, we describe the proposed dynamic graph score matching for interatomic distances, with the goal of modeling both the local and long-range interactions. To ensure that the learned score functions cover all regions with different graph structures, we dynamically construct graph structures with non-bonned edges between atoms during training. based added perturbations. Following Song and Ermon [8), we train a noise conditional score network to jointly estimate scores for perturbed distributions of of caaamccall-ddttrmmmnn interatomic distances, i.e., {so ( R a) k= and parameterize the score network with the message passing neural network (MPNN) 13
Dynamic Score Matching. To capture long-range interactions between non-bonded atoms in molecular system, a naive way is to treat the molecular graph as a fully-connected graph and model the gradients of logarithm density of distances between all pairs of atoms. However, such a practice is computationally expensive especially for large systems, e.g., proteins, and sometimes unnecessary, e.g., van der Waals interactions decay rapidly distances increase. As a remedy, we set a cutoff distance and assume each atom only interacts with all atoms within the cutoff distance, ignoring all interactions out of the considered sphere. This is a very popular strategy in computational chemistry that strikes a good balance between efficiency and accuracy 226 31].
Formally, consider a molecular graph G (V, E) with distances between all pairs of atoms D E R x computed from its conformation R E R|Vx33 For a given noise level a, we perturb the distances with Gaussian noise at each training step on the fly, and then augment the original graph structure with non-bonded edges between atom pairs within a certain threshold distance:
Vi.,). ~ O') E EuEufee Dy <) d, = (D, eey Eg),
iD: eij Ec).
where E, is the constructed graph structure for noise level ar, and 8 is hyper-parameter that controls the radius of longerange interactions. We empirically verify that 10 Á cutoff is sufficient for systems
5

---

RO
R
R
R++1
Ry
I
Random
Initialization
R
Dynomic Graph Cor strusction
Scot W Distannces
Score was Coondirates
R
Dinaric Graph Construction
Figure 3: The generation procedure the proposed DGSM via Langevin dynamics. The graph struc- ture is dynamically constructed each step of stochastic update based the current conformation.
we are studying, which is consistent with some experimental results in molecular dynamics [26]. de and d denote the original and perturbed interatomic distances in augmented graph structure respectively. Hereafter, we omit the subscript for simplicity and use E, d, and d instead, assuming all graphs are dynamically constructed during training and sampling.
With the above strategy, the graph structure of a specific molecular graph G is variadic depending on added perturbation, and all graph structures possible long sample sufficient enough noise. This will result in (1) a Synnaiccall-ddtermiee representation learning, which takes long-range interactions into consideration, and (2) dynamically- determined set of interatomic distances, i.e., {ex R) gradients of atomic coordinates according to Eq. 4 requires computing all distances between atom pairs. In practice, to avoid quadratic complexity, we pre-filter distant neighbors before adding perturbations for each atom by constructing radius graph with 25 threshold, and empirically verify that it performs efficiently and effectively.
graph structure for message passing and
K k=1- for score estimation, which contributes to Note that the vanilla implementation of Eq 5
Parameterizing with MPNNs. Let {or) be sequence of noise levels. Our goal is to learn noise conditional score network to jointly estimate the scores of all perturbed distance distributions, i.e. Vo E {o.) se (d, o) & V Po
Po (d 9), where Pa (d 9) = f p(d G) d, o² I). Following suggestions of Song and Ermon (38), parameterize the score network with MPNN as follows (see supplementary material for the full architecture):
h MPNN(G,E, E, D):. Vu, E V
h
edge
Concat (haude hirdee) Vei Ea
(6)
se (d, a d) so (d))./// MLP )/o, Yeij Eg
where (hoode) 1== are node embeddings computed by the MPNN based on the dynamically constructed
molecular graph and the perturbed distances, (h.geee are embeddings for each edge in E, and so so(d, oy is the predicted score for interatomic distance D,, (Eig E E). The noise conditional
network can be jointly optimized with the following objective 38
s0
d
0- argming 2L
of (d
a; of
where all expectations can be efficiently estimated using empirical averages.
4.3 Generation
After training the noise conditional score network, molecular conformations can be generated via annealed Langevin dynamics (38), guided by the gradients of atomic coordinates. The gradients can be computed via Eq. based on dynamically constructed graph structures each step stochastic update, which allows model to effectively capture both the local and long-range interactions that contribute to the atomic gradients. Formally, given molecular graph G, we first sample an initial conformation Rg from a fixed prior distribution. We here take the prior distribution standard

---

Gaussian (R, O, .). Then, we update the conformation by running T steps of Langevin dynamic to get sample from each noise conditional score network sø R. o;) sequentially with special step size schedule
e of /ot. Samples from each noise level are used to initialize Langevin dynamies for the next noise level. At each sampling step I, we first construct graph structures with non-bonded edges within a given distance based on the current pairwise distances D computed from R and then get a set of interatomic distances d for score estimation:
E 3 U fei D;-1.ú5 < 6), d (D:-1,5
Csj E E(-1).
(8)
The conformation is then updated using the gradient information from the score network (Eq. provide the pseudo-code in Algorithm
4.
We
5 Experiments
Algorithm 1 Annealed Langevin dynamics [38]
Following previous works 34 36
43
Require: g (V, E), (= T.
on conformation generation, we evaluate
the proposed DGSM using the following
1: Initialize conformation Ro
two standard tasks: Conformation Gen-
2: for i + 1 to do
eration (Section 1 and Property Pre-
3:
a; + E o
an is the step size.
diction (Section 5.2 To further demon-
4: for to
strate DGSM's capability of modeling long- range interactions, evaluate on two more challenging benchmark tasks: Protein
5:
6-1,dd-11 t- ang(E, R
> Eq.
6: sø ,, + get d1
D Eq.
7: Draw 2, ~ ~N(,
Sidechain Conformation Generation and Multi-molecular Complex Conformation Generation (Section 5.3). We describe ex- perimental setups in task-specife sections.
8:
R² R 0,80 o;) + 22,2
9:
end for
10: Ro Rr
11: end for
Return: Generated conformation Rr.
5.1
Conformation Generation
Setup. This task evaluates the model's capability generate stable molecular conformations by measuring both accuracy and diversity of generated conformations. Following previous works
we use the GEOM-QM9 and GHOM--Druss [T] datasets for this task. We use the train-test split provided by (34]. The train splits of GEOM-QM9 and GEOM-Drugs both contain 40,000 molecules, each with conformations for training, or 200,000 conformations total. The test split of GEOM-QM9 contains 200 molecules with 22,408 conformations, and the test split of GEOM-Drugs contains 200 molecules with 14,324 conformations.
34 43
We compare DGSM against 33), CGCF 4 and CONFGF (3). For each molecule in the test set, sample twice as many conformations as its reference conformations. We use the matching score (MAT) to measure the accuracy of generated conformations, and coverage score (COV) to measure the diversity following Both metrics are based on Root Mean Squared Deviations (RMSD) between molecules, taking symmetries into account (see supplementary material for the details of metrics).
toitte-of-theartt baselines: RDKIT (28,, CVGAE 24), GRAPHDG
34 43
Results. We report the mean and median COV and MAT scores over all the molecules in the test split on GEOM-QM9 and GEOM-Drugs datasets. As shown in Table T DGSM consistently outperforms
Table 1: COV and MAT scores on GEOM-QM9 and GEOM-Drugs datasets. The threshold 5 of COV
score is 0.5A for GEOM-QM9 and 1.25Á for GEOM-Drugs following Xu et al. 43 (t): the higher the better. (Q): the lower the better.
GEOM-QM9
GEOM-Drug
COV (%. 7)
MAT (A, .)
COV
(%. 1)
MAT (A, J)
Method
Mean
Median
Mean
Median
Mean
Median
Mean
Median
RDKIT
28
83.26
90.78
0.3447
0,2935
60.91
65.70
1.2026
1.1252
CVGAE 24
0.09
0.00
1.6713
1.6088
0.00
0.00
3,0702
2.9937
GRAPHDG
36
73.33
84.21
0.4245
0.3973
8.27
0.00
1.9722
1.9845
CGCF
43
77.52
80.40
0.4206
0.3903
54.19
56.35
1.2575
1.2356
CONFGE
34
88.49
94.13
0.2673
0.2685
62.15
70.93
1.1629
1.1596
DGSM
91.49
95.92
0.2139
0.2137
78.73
94.39
1.0154
0.9980
7

---

Ref. Conf.
ConIIF
GraphDG
Figure 4: Examples of conformations generated by different models based on four random molecular graphs from the test set of BEOM--ruugs We present three reference conformations for each molecule, and visualize the best-aligned conformations generated by each method. Areas where long-range interactions should be modeled are highlighted in green.
all the baselines. Notably, both DGSM and CONFGF are coore-based models, but DGSM achieves better performance. The difference between them is that DGSM successfully takes long-range interactions into consideration via dynamic graph score matching. This confirms the significant benefit of modeling long-range interactions. We present several conformations generated by different approaches in Figure which shows that DGSM successfully captures the long-range interactions in highlighted areas while the other baselines fail, resulting in distorted structures in those areas.
5.2 Property Prediction
Setup. This task demonstrates how genera- tive models for molecular conformations can be applied to property prediction down- stream task. also provides an assessment on the quality of generated conformations in a different light. We estimate the ensemble properties [1 of molecular graph by aggre- gating its conformational properties follow- ing (34] In specific, first use the models
Table 2: Mean absolute errors (MAE) of predicted ensemble properties in eV.
Method
E
Emns
Armi
Armx
RDKIT
0.9233
0.6585
0.3698
0.8021
0.2359
GRAPHDG
9.1027
0.8882
1.7973
4.1743
0.4776
CGCF
28.9661
2.8410)
2.8356
10.6361
0.5954
CONEGF
2.7886
0.1765
0.4688
2.1843
0.1433
DGSM
L.0333
0.0761
0.1963
1.1811
0.1271
to generate 50 conformations for each molecular graph in a subset of GEOM-QM9 K4 and use Ps14 [37] quantum chemical toolkit, calculate each conformation's 's energy and HOMO-LUMO gap. Then, calculate average energy E., lowest energy Emmin average gap Åc, minimum gap Acmin- and maximum gap Acmu from the conformational energy and gap. We evaluate the accuracy of estimated ensemble property by measuring their mean absolute errors (MAE) to the ground truth
values. CVGAE is excluded in this task as its performance is poor, which is also reported in [36 34)
Results. Table shows that DGSM outperforms other machine leaming-based methods by clear margin. DGSM's 's estimation of average energy E and minimum gap Ae min is close to RDKIT but still outperforms the most competitive Ma.ebased method CONFGF. The calculation of conformational energy is highly sensitive to changes in geometry even subtle deviation in bond lengths leads to significant energy change (36). Therefore, the superior performance of DGSM indicates that generates much more accurate conformations than other methods, leading to more accurate property estimation. This validates again the effectiveness of modeling long-range interactions.
5.3 Large Molecule and Multi-molecular Modeling
Protein Sidechain Conformation This task is to pre- dict protein sidechain conformations based its back- bone structures. Compared to conventional molecular conformations generation in previous sections, the main challenge of this task is two-fold: (1) large number of atoms, which prohibits constructing complete graphs that grow quadratically to model long-range interactions. (2) covalent bonds are sparse, which limits the power of the edge augmentation techniques in previous work. DGSM tackles these two challenges via dynamic graph score matching as introduced.
Table RMSD of different approaches on sidechain conformation generation.
RMSD
Method
Mean (A)
Min (A)
CONEGF DGSM
3.38
3.11
2.85 (1 15.7%) 2.61 (1 16.1%)

---

Greeed
Rat
RAt
(a)
(b)
Figure 5: (a) An example of the generated protein sidechain conformation with atomic-level coor- dinates. The ground-truth sidechain (blue) and the generated sidechain (red) are highlighted. (b) Conformations of two multi multi-molecular complexes generated by DGSM.
We use the SidechainNet 18 dataset for this task and follow the official train-test splits. We compare DGSM with the -tate-of-the-art conformation generation model CONFGF. Despite that there are some machine learning-base based methods specialized in protein sidechain structure prediction [TO], they are built upon rotamer libraries [5]. which incorporates a lot of domain knowledge. Thus, our method is not comparable with them. The main purpose of this task to justify the effectiveness of DGSM for large molecules. For each protein, we generate 5 sidechain conformations with different initialization, and calculate the mean and min RMSD between the ground -ruth conformation and the generated conformations. We report the overall mean and min RMSD scores by averaging scores of each protein in the test set in in T which shows that DGSM achieves better performance than previous sate-offthheare model. We also present an example in Figure 5 a), and we can see that the predicted conformation is consistent with the ground truth in major parts.
Multi-molecular Complex Conformation This task is to predict conformations for multi-molecular complexes. A molti-moleeular complex is made up of multiple molecules and there is no covalent bonds between them. Long-range interactions dominate the structure of multi-molecular complexes. The purpose of this task is to demon- strate DGSM's 's potential application to broader range of problems and provide a novel benchmark for conformation generation. We the quantum chemical software xth 3 to construct a dataset con- sisting of 24 water-organic complexes each with several hundreds of conformations, and leave out 4 complexes for testing (see supple- mentary material for details). We do not report RMSD-bases metrics such as COV and MAT because the structures of complexes are highly flexible. Two set of generated examples are presented in Figure We observe that water molecules are placed regularly around the solute organic molecule. Notably, hydrogen bonds and between water and water) are formed correctly. This can also be evidenced in the histogram of Eatrogen-xxygen distances (Figure 6) where there is a peak between 1.5À and 2,5 i.e., the range of hydrogen bond length between Hydrogen and Oxygen.
Reference
Dunch
Hydregan
Banck
DGSM
0
2
Figure 6: The distribution of Hydrogen-Oxygen distances. The first peak from the left is covalent bonds and the second peak is hydrogen bonds.
(between water and the solute,
6 Conclusion and Future Work
We propose DGSM, a novel score-based approach for generating equilibrium molecular confor- mations. DGSM is capable of modeling both the local and long-range interactions in molecular systems, by dynamically constructing graph structures based on spatial proximity between atoms during both training and inference. We also devise a dynamic graph score matching algorithm to effectively estimate atomic gradients, where graph structures are dynamically determined depending on added perturbations. Extensive experiments over two standard tasks and two original tasks show that DGSM outperforms the state-of-the-art method by a large margin, confirming the significant benefit of modeling long-range interactions. the future, we plan to apply our approach to the more challenging problem of protein structure prediction.

---

Acknowledgments and Disclosure of Funding
We would like to thank all the reviewers for the insightful comments. This project supported by the Natural Sciences and Engineering Research Council (NSERC) Discovery Grant, the Canada CIFAR AI Chair Program, collaboration grants between Microsoft Research and Mila, Samsung Electronics Co., Ldt., Amazon Faculty Research Award, Tencent AI Lab Rhino-Bird Gift Fund and a NRC Collaborative R&D Project (AI4D-CORE-O) This project was also partially funded by IVADO Fundamental Research Project grant PRF-2019-3583111
References
[1] Simon Axelrod and Rafael mations for property prediction and molecular generation. arXiv preprint S.Kiv:2006.0553332 2020.
Geom: Energy-annotatee molecular confor-
[2] Andrew J Ballard, Stefano Martiniani, Jacob D Stevenson, Sandeep Somani, and David J Wales. Exploiting the potential energy landscape to sample free energy. Wiley Interdisciplinary Reviews: Computational Molecular Science, 5(3):273-289, 2015.
[3] Christoph Bannwarth, Eike Caldeweyher, Sebastian Ehlert, Andreas Hansen, Philipp Pracht, Jakob Seibert, Spicher Spicher, and Stefan Grimme. Extended tight-binding quantum chemistry methods. WIREs Comput. Mol. Sci., 11:e01493, 2020. doi: 10.1002/wems. 1493. URL
https: doi. org/10. 10. 2..000//cmms 1493
[4] Blaney and Dixon, Distance geometry in molecular modeling. Cheminform, 25, 2007.
[5] Michael Bower, Fred E Cohen, and Roland L Dunbrack Jr. Prediction of protein side-chain rotamers from a backbone- dependeen rotamer library: new homology modeling tool. Journal of molecular biology, 267(5): ( 10::::22822200 1997.
[6] Ruojin Cai, Guandao Yang. Hadar Averbuch-Eie Zekun Hao, Serge J. Belongie, Noah Snavely, and Bharath Hariharan. Learning gradient fields for shape generation. ArXiv, (at/22008.6552) 2020.
[7] Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. In International Conference on Learning Representations, 2021.
[8] Gordon M Crippen, Timothy F Havel, et Distance geometry and molecular conformation, volume 74. Research Studies Press Taunton, 1988.
[9] Marco De Vivo, Matteo Masetti, Giovanni Bottegoni, and Andrea Cavalli. Role of molecular dynamics and related methods in drug discovery. Journal of medicinal chemistry, 59(9): 4035 4061, 2016.
[10] Yilun Du, Joshua Meier, Jerry Ma, Rob Fergus, and Alexander Rives. Energy-based mod- els for atomiccresolution protein conformations. In International Conference on Learning Representations, 2020. URL http:p/rr.ppggeeeeeeeeeeee/eeeee
[11] David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru- Guzik, and Ryan P Adams. Convolutional networks on graphs for learning molecular fingerprints. Advances neurul information processing systems, pages 2224- 2232, 2015.
[12] Sctavian-Eugen Ganea, Lagnajit Pattanaik, Connor W. Coley, Regina Barzilay, Klavs Jensen, William H. Green, and T. Jaakkola. Geomol: Torsional geometric generation of molecular 3d conformer ensembles. ArXiv, 2021.
[13] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 1263-1272. JMLR. org, 2017.
10

---

[14] T. Gogineni, Ziping Xu, Exequiel Punzalan, Runxuan Jiang, Joshua A Kammeraad, Ambuj Tewari, and P. Zimmerman. Torsionnet: reinforcement learning approach sequential conformer search. ArXiv, abs/2006.07078, 2020.
[15] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, 2020.
[16] Weihua Hu. Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In International Conference on Learning Representations, 2020.
[17] Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. arXiv preprint wxxiv:/80..04366 2018.
[18] Jonathan E. King and D. Koes. Sidechainnet: An all-atom protein structure dataset for machine learning. arXiv, 2020.
[19] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv: 2016.
[20] Johannes Klicpera, Janek Groß, and Stephan Günnemann. Directional message passing for molecular graphs. In International Conference on Learming Representations, 2020.
[21] Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2021.
[22] A. Leach. Molecular modelling: Principles and applications. 1996.
[23] Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. ArXiv, abs/2103.01458, 2021.
[24] Elman Mansimov, Omar Mahmood, Scokho Kang, and Kyunghyun Cho. Molecular geometry prediction using deep generative graph neural network. arXiv preprint urXiv:1904.003/. 2019.
[25] R. Parr. Density-functional theory of atoms and molecules. 1989.
[26] S. Piana, K. Lindorff- Larsen,,, Robert M. Dirks, Salmon, R. Dror, and D. Shaw. Evaluating the effects of cutoffs and treatment of long-range electrostatics in protein folding simulations. PLoS ONE, 7, 2012.
[27] Anthony K Rappé, Carla J Casewit, KS Colwell, William A Goddard III, and W Mason Skiff. Uff, a full periodic table force field for molecular mechanics and molecular dynamics simulations. Journal of the American chemical society,
1992.
[28] Sereina Riniker and Gregory Landrum. Better informed distance geometry: Using what we know to improve conformation generation. Journal of Chemical Information and Modeling, 55 (12):2562-2574 2015.
[29] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling, E(n) equivariant graph neural networks, 2021.
[30] T. Schlick. Molecular modeling and simulation: An interdisciplinary guide. 2010.
[31] Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robel Müller. Schnet: A continuous- convolutional neural network for modeling quantum interactions. In Advances in Neural Information Processing Systems, pages 991-1001. Curran Associates, Inc., 2017.
[32] Kristof T Schütt, Farhad Arbabzadah, Stefan Chmiela, Klaus R Müller, and Alexandre Tkatchenko. Quantum-chemical insights from deep tensor neural networks. Nature com- munications, 8:13890, 2017.

---

[33] Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation. arXiv preprint arXiv:2001.0950 2020.
[34] Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular conformation generation. ArXiv, 2021.
[35] Jihyun Shim and Alexander D MacKerell Jr. Computational ligand- based rational design: role of conformational sampling and force fields in model development. MedChemCommn, 2(5):
356-370, 2011.
[36] Gregor Simm and Jose Miguel Hernandez-Lobato. A generative model for molecular distance geometry. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119, pages 8949 -999- PMLR, 2020.
[37] Daniel G. A. Smith, L. Burns, A. Simmonett, R. Parrish, M. C. Schieber, Raimondas Galvelis, Kraus, H. Kruse, Roberto Di Remigio, Asem Alenaizan, A. M. James, S. Lehtola, Jonathon Misiewicz, al. Psi4 1.4: Open-source software for high-throughput quantum chemistry. The Journal of chemical physics, 2020.
[38] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, volume 32, pages 1 11918- 1 11930. Curran Associates, Inc., 2019.
[39] Yang Song and Stefano Ermon. Improved techniques for training score-hased generative models. NeurIPS, 2020.
[40] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[41] Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Comput., 2011.
[42] Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML'11, 11, page 681-688, 2011.
[43] Minkai Xu, Shitong Luo, Yoshua Bengio, Jian Peng, and Jian Tang. Learning neural generative dynamics for molecular conformation generation. In International Conference on Learning Representations, 2021.
[44] Minkai Xu, Wujie Wang, Shitong Luo, Chence Shi, Yoshua Bengio, Rafael Kommez-Bombrrlle and Jian Tang. An end-to-end framework for molecular conformation generation via bilevel programming. In ICML, 2021.
[45] Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in neural information processing systems, pages 6410-6421, 2018
12


Reviews:
[{"review_id": "zPpd4L7uKPO", "reviewer": "Reviewer_S9Sr", "summary": "This paper introduces a new approach for generating molecular conformations, using score-based generative modeling.  The key contribution over prior work is the dynamic addition (during both training and inference) of long-range interactions between non-bonded atoms that are not proximal in the molecular graph.  As these interactions are often critical to determining the underlying molecular conformations, the authors contend that their addition leads to improved performance on a range of tasks.  In particular, they demonstrate improved performance over previous state-of-the-art on the generation of conformations for small molecules, protein side chains, and multi-molecular complexes, as well as small-molecule ensemble property prediction.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: While score-based generative modeling has been recently applied to molecular conformation generation (ref 32), and 3D information has been incorporated into many molecular learning algorithms, the dynamic addition of non-bonded atoms within the score matching paradigm is novel.  Most of the tasks are not new, with the exception of the multi-molecular complex conformation task (which does look quite interesting!).\n\nQuality: This is my main concern regarding this work.  I believe further experiments are required to conclusively support the claim that DGSM's improved performance derives from taking into account long-range interactions.  I realize that at a high-level CONFGF is DGSM without these non-bonded edges, though it is also unclear if there are other methodological differences that could be coming in to play.  In particular, I would like to see an ablation study comparing DGSM's performance with and without the augmentation of the original graph structure with non-bonded edges.  Even better would be plots demonstrating the change in performance of DGSM across different cutoff thresholds $\\delta$ (currently fixed to 10 Å).  I in fact would expect that these plots might look different across tasks involving small molecules, proteins, and multi-molecular complexes.\n\nClarity: The work is clearly written and well organized.\n\nSignificance: The results are overall quite compelling.  It is unclear if there will be much use (yet) of this method for larger molecular systems and complexes since specialized methods that incorporate a lot of domain knowledge are often state-of-the-art. \n\n*** Post-response update:\n\nThe reviewer addressed my concern regarding the non-bonded edge ablation, and the results support the overall conclusion, so I am raising my score.  I do think the decrease in performance as the cutoff distance increases is an interesting line of investigation to be pursued further.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors mostly do not discuss limitations of this work.  What size systems can currently be modeled?  The use of other computational methods (as opposed to direct experimental measurements) to provide ground truth data should also be addressed.\n\nI do not believe there are any potential negative societal impacts to be addressed here.", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "xIy5I5ZA-JM", "reviewer": "Reviewer_ssuS", "summary": "This paper proposes to generate molecular conformation by score matching. The proposed method, called DGSM, learns a score network to predict gradient field with respect to a predicted conformation. The score network is parameterized as a message passing network over a dynamic graph between atoms. The graph is constructed based on pairwise distance between atoms in the current conformation. Finally, DGSM uses the learned gradient field along with Langevin dynamics to sample molecular conformations at test time. DGSM is evaluated on two conformation generation benchmarks and achieved new state-of-the-art results.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, this paper is a nice (but small) extension to ConfGF (Shi et al., 2021), which first used score matching methods to generate molecular conformations. DGSM adopts the same score matching framework in ConfGF but modifies the score network architecture. The reviewer acknowledges the importance of dynamic graph construction and the experimental results show promising results.\n\n### Strength\n\n* An end-to-end framework for molecular conformation generation. The method is mostly data driven and does not use much domain-specific rules.\n* Experimental results indeed shows that DGSM outperforms ConfGF, supporting the main claim of the paper.\n* New experiments on macromolecules and multi-molecule complexes are interesting.\n\n### Weakness\n* DGSM is a small extension to ConfGF. The score matching framework has been proposed in prior literature, so the originality of DGSM is relatively small.\n* There is no quantitative results on multi-molecule complex conformation. What's the performance of ConfGF for this task? \n* The multi-molecule complex task seems artificial. Why not try your algorithm on protein-protein complex conformation prediction? ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This paper did not address its limitations. I have included constructive feedback in the review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5 hours"}, "extra_text": {}}, {"review_id": "eI7aqd8Iic6", "reviewer": "Reviewer_fHJN", "summary": "This paper addresses the task of molecular conformer generation, i.e. generate the 3D conformer ensemble conditioned on the molecular graph. The introduced method, Dynamic Graph Score Matching (DGSM), is using score matching to generate 3D structures by explicitly capturing both local and long-range interactions. This is done using dynamic graphs constructed via atom coordinate perturbations using different noise levels as required by the denoising score matching procedure.  The score is estimated using message passing neural networks which operate on these dynamic graphs. At inference time, Langevin dynamics is used to sample conformations from the generated distribution. Empirically, the authors show clear improvements over previous machine learning (ML) models and over one popular open-source non-ML method (RDKit) on molecular conformer generation on two different tasks, as well as improvements on experiments on property prediction based on molecular conformations, protein sidechain conformation generation, and multi-molecular complex conformation.\n\nIn short, I think this approach is quite compelling and would be very impactful for future work related to graph conditional 3D generative models of various data in biology, chemistry, etc. The paper is very well written and the method is clean, addressing nicely using dynamic graphs the very important problem of modeling long-range interactions in 3D generation.  ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. Originality:\n\nUsing denoising score matching for 3D conformer generation is a novel and non-trivial contribution.  The authors employ dynamic graphs for this problem and describe all challenges and solutions. Related work is well written, but it might be useful to discuss non-ML literature a bit more in depth, e.g. see [1].\n\n\n2. Quality:\n\nI already argued above that this is a good quality submission. However, the authors do not discuss any limitations of their work. It would be great if they could describe common failure cases of their conformer generation model. Also, how stable is the training of the denoising score matching procedure ?\n\n3. Clarity: paper is well written.\n\n4. Significance: already commented above. This paper provides a nice clean solution to a very important problem in graph conditional 3D generation.\n\n\n5. Improvement points: while I generally like this work, there are a few points where the authors could improve their paper.\n\n- experimental results in table 1: COV and MAT only show information about the recovered ground truth conformers, i.e. equivalent of recall. It would be also very useful to understand how many of the generated structures are close to ground truth conformers, i.e. to show the equivalent metrics of precision. Also, including a stronger non-ML baseline (e.g. OMEGA https://www.eyesopen.com/omega ) would be helpful in making this work more visible and useful in the chemistry community.\n\n- it is known that conformations of a single molecule are usually changing mostly in terms of their rotatable bonds. Can the authors comment on how torsion angles can be integrated in their framework ?\n\n- chirality is a very important issue for molecular conformer generation that was not explicitly tackled until very recently (see [2]). Can the authors comment on whether this can be integrated in DGSM?\n\n\n[1] Conformation Generation: The State of the Art, Hawkings, 2017\n[2] GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensembles, Ganea et al., 2021", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitation section is missing.", "review_text": "", "extra_scores": {"ethical_concerns": "n/a", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "86xe2xAcLG", "reviewer": "Reviewer_d6Qw", "summary": "The authors develop an improved method for 3D molecular conformation generation from 2D molecular graphs that can more directly incorporate non-bonded, topologically long-range interactions into the generation process. Their starting point is a recent method that leverages score matching with a GNN based on the 2D molecular graph topology, or more specifically, a GNN that is trained to estimate the gradient of the noise-perturbed log density of interatomic distances on the graph edges. They follow the same framework, but add additional edges to the graph topology based on the current perturbed distances when they fall below a certain cutoff (10 Angstroms). Through their experiments, they show that this relatively modest modification signifcantly improves generation performance of small molecule conformers and can be applied to larger and more complex systems such as protein side chain packing and multi-molecular complexes.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths \n\nMotivation and Significance. \nNon-bonded interactions are central to understanding larger molecular systems but have been a blind spot or weakness for many ML-based approaches to conformation generation, in part because explicitly modeling all possible pairwise interactions is expensive. This work offers a tractable solution to this by using the domain-motivated design decision of spatial neighborhood cutoffs. This very nicely combines with score matching based generative modeling, since with that framework, you can essentially train on static randomly sampled graphs with approximately linear scaling in the number of atoms, but you still retain the power to dynamically account for arbitrary ij interactions at sampling time.\n\nSmall molecule results. \nThe results on the GEOM datasets are quite strong and already show that non-bonded interactions can make a difference for small molecular systems such as QM9 and drug-like molecules. The improved typical coverage and accuracy seem like qualitative improvements that could make a meaningful impact on conformer generation in molecular property prediction or in-silico discovery efforts. Additionally the improved ensemble energetics from a QM method in Table 2 suggest that this framework is improving the ability of ML-based conformer methods to model these systems in a physically accurate way.\n\nClarity. \nI found the paper well written and easy to follow, with effective figures and notation.\n\nWeaknesses\n\nLarger system baselines. \nBoth the protein side chain and multi-molecular complex experiments suggest potential applications of the framework, but it would significantly strengthen the paper to add some kind of external baseline model performance to calibrate expectations. For example, although the authors mention that rotamer based side-chain predictors perform much better, it would be helpful to add a baseline such as Rosetta to understand how far the model is from being practically applicable in that domain.\n\nNovelty and ablation. \nThe primary methodological change here is simple, but I don't that think that is a big weakness because it has clear and meaningful impacts on the capability of the framework. Since that change is so specific, though, I think that it would benefit the paper for the authors to more thoroughly investigate the role of the cutoff distance. For example, does making the cutoff distance small add a useful inductive bias, or is it more a matter of a performance/speed tradeoff?\n\n---Update post-response---\nThank you for adding the additional results about the role of the cutoff distances and how it affects both accuracy and diversity. It is interesting that, at least in current form, there can be benefits to smaller neighborhoods. The authors' point about the method being a general framework is well taken and also the main reason I did and would recommend acceptance (and keep my score the same). I agree it will be quite interesting to see how it can compare to application-specific methods with additional domain-specific tuning. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I would suggest that the authors explicitly discuss limitations of the model with respect to chirality. Since the model is based purely on distance-based score matching, it will not be able to differentiate between stereoisomers. I imagine that this is likely already causing problems for protein side chain prediction, since some amino acid side chains, such as isoleucine and threonine, contain chiral carbons.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers all agreed this work should be accepted. They appreciated the clarity, the results (particularly the small molecule results), the motivation of the problem being addressed, the end-to-end aspect of the model, and the related work section. The authors should take into account the detailed suggestions of the reviewers for the camera ready version, to make an already nice paper even better.
Decision: Accept (Poster)