Torsional Diffusion for Molecular Conformer Generation
Authors: Bowen Jing
Gabriele Corso
Jeffrey Chang
Regina Barzilay
Tommi S. Jaakkola
Venue: NeurIPS 2022 Accept (2022)
Keywords: conformer generation
diffusion models
score-based models
molecular structure
equivariance
geometric deep learning
Boltzmann generator

Abstract: Molecular conformer generation is a fundamental task in computational chemistry. Several machine learning approaches have been developed, but none have outperformed state-of-the-art cheminformatics methods. We propose torsional diffusion, a novel diffusion framework that operates on the space of torsion angles via a diffusion process on the hypertorus and an extrinsic-to-intrinsic score model. On a standard benchmark of drug-like molecules, torsional diffusion generates superior conformer ensembles compared to machine learning and cheminformatics methods in terms of both RMSD and chemical properties, and is orders of magnitude faster than previous diffusion-based models. Moreover, our model provides exact likelihoods, which we employ to build the first generalizable Boltzmann generator. Code is available at https://github.com/gcorso/torsional-diffusion.

Torsional Diffusion for
Molecular Conformer Generation
Bowen Jing," Gabriele Corso," Jeffrey Chang, Regina Barzilay,' Tommi Jaakkola'
'CSAIL, Massachusetts Institute of Technology Dept. of Physics, Harvard University
Abstract
Molecular conformer generation is a fundamental task in computational chemistry. Several machine learning approaches have been developed, but none have outper- formed state-of-the-ar cheminformatics methods. We propose torsional diffusion, novel diffusion framework that operates the space of torsion angles via diffusion process on the hypertorus and an extrinsictto-intinsscc score model. On a standard benchmark of drug-lit like molecules, torsional diffusion generates superior conformer ensembles compared to machine learning and cheminformatics meth- ods in terms of both RMSD and chemical properties, and is orders of magnitude faster than previous diffusion-basee models. Moreover, our model provides exact likelihoods, which employ build the first generalizable Boltzmann generator. Code is available at https: /ttgt/gtttb/////////////teeeeh.. com/ http://coooooooooonooo//ooo// diffusion
1 Introduction
Many properties of a molecule are determined by the set of low-energy structures, called conformers, that it adopts in 3D space. Conformer generation therefore a fundamental problem in computational chemistry [Hawkins, 2017] and an area of increasing attention in machine learning. Traditional approaches to conformer generation consist of metadynamics-base but slow [Pracht al., 2020]; and cheminformaties- based methods, which are fast but less accurate [Hawkins et al., 2010, Riniker and Landrum, 2015]. Thus, there is growing interest in developing deep generative models to combine high accuracy with fast sampling.
methods, which are accurate
Diffusion score-based generative models [Ho et al., 2020, Song et al., 2021]-a promising class of generative models-have been applied to conformer generation under several different formulations. These have so far considered diffusion processes Fuclidean space, in which Gaussian noise is pairwise distances in distance matrix
injected independently into every data coordinate-eithee [Shi et al., 2021, Luo et al., 2021] or atomic coordinates in 3D [Xu et al., 2022]. However, these models require a large number of denoising steps and have far failed to outperform the best cheminformatics methods.
We instead propose torsional diffusion,, in which the diffusion process over conformers acts only on the torsion angles and leaves the other degrees of freedom fixed. This is possible and effective because the flexibility of molecule, and thus the difficulty of conformer generation, lies largely torsional degrees of freedom [Axelrod and Gömez-Bombarelli, 2022]; in particular, bond lengths and angles can already be determined quickly and accurately by standard cheminformatics methods. Leveraging this insight significantly reduces the dimensionality the sample space; drug-like molecules have, average, 44 atoms, corresponding to 3n-dimensional Euclidean space, but only m = 7.9 torsion angles of rotatable bonds.
Equual contribution. Correspondence to [bjing, georso]-mmit. edu.
2 As measured from standard dataset GEOM-DRUGS [Axelrod Bomberllommarlli,, 2022]
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

torsional diffusion
Conformer step i+1
intrinsic torsional update dr;
At
Intrinsic view
Extrinsic view
Reverse diffusion trajectory
AY
Σ Yir v rlr
foos
Reverse diffusion denoising step
Extrinsic- Extrinsie-to-intiiin to-Intrinscc score model
Pseudotorque layer
Extrinsic input
Interaction layers
Conformer at stop
Figure I: Overview torsional diffusion. Left: Extrinsic and intrinsic views of torsional diffusion (only 2 dimensions/bonds shown). Right: In step of reverse diffusion (A), the current conformer is provided as 3D structure (B) to the score model, which predicts intrinsic torsional updates (C). The final layer of the score model is constructed to resemble torque computation around each bond (D). Y refers to the spherical harmonics and V,, the learned atomic embeddings.
Torsion angle coordinates define not a Fuclidean space, but rather an m-dimensional torus T (Figure 1, left). However, the dimensionality and distribution over the torus vary between molecules and even between different ways of defining the torsional space for the same molecule. To resolve these difficulties, we develop
perrinsic-to-intrinns score model (Figure right) that takes input a 3D point cloud representation of the conformer in Euclidean space (extrinsic coordinates), and predicts output
torsional space specific to that molecule (intrinsic coordinates). To do so, we consider torsional score for a bond as geometric property of a 3D point cloud, and use (E(3)-equivarian networks predict them directly for each bond.
Unlike prior work, our model provides exact likelihoods of generated conformers, enabling training with the ground-truth energy function rather than samples alone. This connects with the literature on Boltzmann generators-ggnnerattvy generative models which aim to sample the Boltzmann distribution of physical systems without expensive molecular dynamics or MCMC simulations [Noé al., 2019, Köhler et al., 2021]. Thus, as variation on the torsional diffuusion framework, we develop torsional Bolrzmann generators that approximately sample the conditional Boltzmann distribution for unseen molecules. This starkly contrasts with existing Boltzmann generators, which are specific for the chemical system on which they trained.
Our main contributions are:
We formulate conformer generation in terms of diffusion modeling on the hypertorus the first demonstration of non- -on-Euulidean diffusion on complex datasets and develop an extrinsic-to-intrine score model that satisfies the required symmetries: SF) invariance, torsion definition invariance, and parity equivariance.
results the GEOM-DRUGS dataset [Axelrod and Gómez- Bombarelli, 2022] and are the first method to consistently outperform the established commercial software OMEGA [Hawkins, 2017]. We do using two orders of magnitude fewer denoising steps than GeoDiff [Xu et al., 2022), the best Fuclidean diffusion approach.
We obtain
We propose torsional Boltzmann generators the first Boltzmann generator based on diffu- sion models rather than normalizing flows and the first to be useful for class of molecules rather than a specific system.
Background
Diffusion generative models Consider the data distribution the starting distribution po po(x) of forward diffusion process described by an Ito stochastic differential equation (SDE):
dx f(x, t) dt + g(t) dw, t (0,T)
(I)

---

where w is the Wiener process and fxx,t) g(t are chosen functions. With sufficiently large T, the distribution pr(x)- the prior approaches simple Gaussian. Sampling from the prior and solving the reverse diffusion
dx = [f(x, (t) x po dt + g(t) dw
(2)
yields samples from the data distribution po (x) [Anderson, 1982, Song et al., 2021]. Diffusion, or score-based, generative models [Ho al., 2020. Song et al., 2021] learn the score Vx log (x(x) of the diffused data with a neural network and generate data by approximately solving the reverse diffusion. The score of the diffused data also defines probability flow ODE-a continuous normalizing flow that deterministically transforms the prior into the data distribution [Song et al., 2021]. We leverage the insight that, in many cases, this flow makes it possible to use diffusion models in place of normalizing flows and highlight one such case with the torsional Boltzmann generator.
Diffusion generative models have traditionally been used to model data on Euclidean spaces (such as images); however, De Bortoli et al. [2022] recently showed that the theoretical framework holds with relatively few modifications for data distributions on compact Riemannian manifolds. The hypertorus T", which Thich we use to define torsional diffusion, is a specific case of such manifold.
Several methods [Salimans and Ho, 2022, Vahdat et al., 202., Nichol and Dhariwal, 2021] have been proposed to improve and accelerate diffusion models in the domain of image generation. Among these, the most relevant to this work is subspace diffusion [Jing et al., 2022], in which the diffusion is progressively restricted to linear subspaces. Torsional diffusion can be viewed in a similar spirit, as it effectively restricts Euclidean diffusion to a nonlinear manifold given by fixing the non-torsional degrees of freedom.
Molecular conformer generation The conformers of a molecule are the set of its energetically favorable 3D structures, corresponding local minima of the potential energy surface." The gold standards for conformer generation are metadynamics- based methods such as CREST [Pracht et al., 2020], which explore the potential energy surface while filling local minima [Hawkins, 2017]. However, these require an average of 90 core-hours per drug-like molecule [Axelrod and Gómez- Bombarelli, 2022] and are not considered suitable for high-throughput applications. Cheminformatics methods instead leverage approximations from chemical heuristics, rules, and databases for signif- icantly faster generation [Lagorce et al., 2009, Cole et al., 2018, Miteva et al., 2010, Bolton et al., 2011, Li et al., 2007]; while these can readily model highly constrained degrees of freedom, they fail to capture the full energy landscape. The most well-regarded of such methods include the commercial software OMEGA [Hawkins et al., 2010] and the open-source RDKit ETKDG [Landrum et al., 2013, Riniker and Landrum, 2015].
A number of machine learning methods for conformer generation has been developed [Xu et al., 2021a.b, Shi et al., 2021, Luo al., 2021], the most recent and advanced of which are GeoMol [Ganea et al., 2021] and GeoDiff [Xu et al., 2022]. GeoDiff is Euclidean diffusion model that treats conformers as point clouds x E R33 and learns an SE(3) equivariant score. On the other hand, GeoMol employs graph neural network that, in single forward pass, predicts neighboring atomic coordinates and torsion angles from a stochastic seed.
Boltzmann generators An important problem in physics and chemistry is that of generating independent samples from a Boltzmann distribution pxx) xx e -Ex))/AT with known but unnormalized density. Generative models with exact likelihoods, such normalizing flows, can be trained to match such densitics [Noé et al., 2019] and thus provide independent samples from an approximation of the target distribution. Such Boltzmann generators have shown high fidelity on small organic molccules (Köhler et al., 2021] and utility on systems as large as proteins [Noé et al., 2019]. However, a separate model has be trained for every molecule, as the normalizing flows operate on intrinsic coordinates whose definitions are specific to that molecule. This limits the utility of existing Boltzmann generators for molecular screening applications.
1 Conformers are typically considered up to an energy cutoff above the global minimum.
"This is related to but distinct from conformer generation, as conformers are the local minima of the Boltzmann distribution rather than independent samples.
3

---

3 Torsional Diffusion
Consider a molecule as graph E) with atoms v V and bonds e E E,5 and denote the space of its possible conformers Co. A conformer C' Ca can be specified in terms of its intrinsic (or internal) coordinates: local structures L consisting of bond lengths, bond angles, and cycle conformations; and torsion angles consisting of dihedral angles around freely rotatable bonds (precise definitions in Appendix A). We consider bond freely rotatable severing the bond creates two connected components of G, each of which has at least two atoms." Thus, torsion angles in cycles (or rings), which cannot be rotated independently, are considered part of the local structure L.
Conformer generation consists of learning probability distributions PG T). However, the set of possible stable local structures for a particular molecule is very constrained and can be accurately predicted by fast cheminformatics methods, such as RDKit ETKDG [Riniker and Landrum, 2015] (see Appendix F. for verification). Thus, we use RDKit to provide approximate samples from Pa(L), and develop diffusion-based generative model to learn distributions ( L) over torsion angles-conditioned on given graph and local structure.
Our method is illustrated in Figure and detailed as follows. Section 3.1 formulates diffusion modeling on the torus defined by torsion angles. Section 3.2 describes the torsional score framework, Section 3.3 the required symmetries, and Section 3.4 our score model architecture. Section 3.5 discusses likelihoods, and Section 3.6 how likelihoods can be used for energy- -based training.
3.1 Diffusion modeling on T
Since each torsion angle coordinate lies in [0, 2x), the m torsion angles of conformer define hypertorus T". To learn a generative model over this space, we apply the continuous score-based framework of Song et al. [2021], which holds with minor modifications for data distributions compact Riemannian manifolds (such as T")) [De Bortoli et al., 2022]. Specifically, for Riemannian manifold M let x E M, let be the Brownian motion the manifold, and let the drift f(x, t), score Vx log Pt (x), and score model output s(x, t) be elements of the tangent space M. Then equation remains valid that is, discretizing and solving the reverse SDE on the manifold geodesic random walk starting with samples from px(x) approximately recovers the original data distribution po (x) [De Bortoli et al., 2022].
For the forward diffusion
rescaled Brownian motion given by f(x, t) 0.
d (t)
where (() is the noise scale. Specifically, Song and Ermon [2019], with miin 0.01 dmax = π, E (0, 1). Due to the compactness of the manifold, however, the prior pr(x) x) is no longer Gaussian, but a uniform distribution over M.
an exponential diffusion (4) o min of in
Training the score model with denoising score matching requires a procedure to sample from the perturbation kernel Pxjo(' x) of the forward diffusion and compute its score. We view the torus T = 20, as the quotient space IR" / with equivalence relations (T ~ (Ti + 2T, Tm
Tm.)... ... (TI,... Tm + 2m). Hence, the perturbation kernel for rescaled Brownian motion on Tm is the wrapped normal distribution on Rm, that is, for any T' [0, )nr"",, we have
PijoTT (T) x
Σ exp
T
T' + 2nd
20²(t)
deZ
where (f) is the noise scale of the perturbation kernel Pejo. We thus sample from the perturbation kernel by sampling from the corresponding unwrapped isotropic normal and taking elementwise mod 2x. The scores of the kernel are pre-computed using a numerical approximation. During training, we sample times t uniform and minimize the denoising score matching loss
Jusu = Ex
[]]s(T,, t) V log PHo(Te ro)][]]]
(4)
where the weight factors A(t) 1/E,-pppo V log (T 0)] are also precomputed. As the tangent space Tm is just R.. all the operations in the loss computation are the familiar ones.
5 CCirrality and other forms of stereoisomerism are discussed in Appendix F33 'Notably, this counts double bonds as rotatable. See Appendix F.3 for further discussion.

---

For inference, we first sample from a uniform prior over the torus. We then discretize and solve the reverse diffusion with geodesic random walk; however, since the exponential map on the torus (viewed as quotient space) is just (6) T + ö mod 2m, the geodesic random walk is equivalent to the wrapping of the random walk on R.
3.2 Torsional score framework
While we have defined the diffusion process over intrinsic coor- dinates, learning score model s(T.t) directly over intrinsic co- ordinates is potentially problematic for several reasons. First, the dimensionality m of the torsional space depends on the molecular graph G. Second, the mapping from torsional space to physically distinct conformers depends on G and local structures L. but it is unclear how to best provide these to model over T Third, there no canonical choice independent intrinsic coordinates (L, T): in particular, the torsion angle at rotatable bond can be defined any of the dihedral angles at that bond, depending on an arbitrary choice of reference neighbors (Figure 2 and Appendix A). Thus, even with fixed G' and L, the mapping from T to conformers is ill-defined. This posed significant challenge to prior works using intrinsic coordinates [Ganea et al., 2021].
A
B
At'
A
AT AT'
160
C
To circumvent these difficulties, we instead consider conformer C' E CG in terms of its extrinsic (or Cartesian) coordinates that is, as point cloud in 3D space, defined up to global roto-translation: Co = 3m ((3). Then, we construct the score model (c(C,t) a function over Cg rather than T The outputs remain in the tangent space of T"M, which is just RT.. Such a score model is simply an (3)-invariant model over point clouds in 3D space 8g 3R x [0, 7 + Rn conditioned on G. Thus, we have reduced the problem of learning a score the torus, conditioned on the molecular graph and local structure, to the much more familiar problem of predicting (3))--inrania scalar quantities-one for each bond-from 3D conformer.
T' = T"
Figure 2: A: The torsion around a bond depends on choice of neighbors. B: The change ÅT caused by a rela- tive rotation is the same for all choices. C: The sign of Ar is unambiguous because given the same neighbors, does not depend on bond direction.
It may appear that we still need to choose definition of each torsion angle T; so that we can sample from p- T) during training and solve the reverse SDE over during inference. However, we leverage the following insight: given fixed local structures, the action on C' of changing a single torsion angle T; by some Å can be applied without choosing definition (Figure 2). Geometrically, this action of the atoms on opposite sides of the bond and can be applied directly to the atomic coordinates in 3D. The geometrie intuition can be stated follows (proven in Appendix B and discussed further in Appendix F.2).
(signed) relative rotation
Proposition Let (bi, be a rotatable bond, let Xy(b,) be the positions of atoms on the by side of the molecule, and let R(O, xx.) E SE(3) be the rotation by Euler vector @ about Nee- Then for C, E CC. T; is any definition of the torsion angle around bond (bi, Gi),
(CC)=+((0)) +
Xy(b.) XV(b.)
if
E C..
(5)
((C")=T((0 Vi&i
Xx(()) R (0 Te. Xy(es)
where Poçc;
(Ie: Ib. )/1/=,, - Tb.ll
To apply torsion update AT (ATI.... ATm) involving all bonds, we apply AT; sequentially in any order. Then, since training and inference only make use of torsion updates AT, we work solely in terms of 3D point clouds and updates applied them. To draw local structures L from RDKit, we draw full 3D conformers C E CG and then randomize all torsion angles to sample uniformly over TmM To solve the reverse SDE, we repeatedly predict torsion updates directly from, and apply them directly to, the 3D point cloud. Therefore, since our method never requires choice of reference neighbors for any Tú it is manifestly invariant to such a choice. These procedures are detailed in Appendix C.
5

---

3.3 Parity equivariance
The torsional score framework presented thus far requires an SE(3) innarrian model. However, an additional symmetry requirement arises from the fact that the underlying physical energy is invariant, or extremely nearly so, under parity inversion [Quack, 2002]. Thus our learned density should respect p(C) P(-C) where -C = [-x E C). In terms of the conditional distribution over torsion angles, we require (CT [TC]) = p(T(-C) [(-C)). Then,
Proposition 2. P(T
= -(C0))pp((-C)) ( L(-C)), then for all diffusion times t, V (ologP((T(C) [(()) = = [(C))= V- lo C) I
(6)
Because the score model seeks to learn (C = V, (og L(C)), we must have (c( = sc(-C,t). Thus, the score model must be invariant under SE(3) but equivari- ant (change sign) under parity inversion of the input point cloud- ic.. it must output a set of pseudoscalars in Rm
3.4 Score network architecture
Based on sections 3.2 and 3.3, the desiderata for the score model are:
Predict a pseudoscalar θT, == alog Ör; E IR that is S (E(3) -invariant and parity equivariant for every rotatable bond in 3D point cloud representation of conformer.
While there exist several GNN architectures which et al., 2021], their $ E33 -invaariai outputs are also parity invariant and, therefore, cannot satisfy the desired symmetry. Instead, leverage the ability of equivariant networks based tensor products [Thomas et al., 2018, Geiger et al., 2022] to produce pseudoscalar outputs.
(33)-eqqivaarian [Jing al., 2021, Satorras
Our architecture, detailed in Appendix D. consists of an embedding layer, series of atomic convolu- tion layers, and final bond convolution layer. The first two closely follow the architecture of Tensor Field Networks [TThomas et al., 2018), and produce learned feature vectors for each atom. The final bond convolution layer constructs tensor product filters spatially centered on every rotatable bond and aggregates messages from neighboring atom features. We extract the pseudoscalar outputs of this filter to produce a single real-valued pseudoscalar prediction б7; for each rotatable bond.
Naively, the bond convolution layer could be constructed the same way as the atomic convolution layers, i... with spherical harmonic filters. However, to supply information about the orientation of the bond about which the torsion occurs, we construct filter from the product of the spherical harmonics with a representation the bond (Figure ID). Because the convolution conceptually resembles computing the torque, we call this final layer the pseudotorque layer.
3.5 Likelihood
By using the probability flow ODE, can compute the likelihood of any sample follows [Song et al., 2021, De Bortoli et al., 2022]:
/" (1) V ()) di
log Po (T)) log pr (rr))(T))
In Song et al. [2021], the divergence term is approximated via Hutchinson's 's method [Hutchinson, 1989], which gives unbiased estimate of log (T)).. However, this gives biased estimate of po po(T). which is unsuitable for our applications. Thus, we compute the divergence term directly, which is feasible here (unlike in Fuclidean diffusion) due to the reduced dimensionality of the torsional space.
The above likelihood in torsional space (G L), T E T"", but to enable compatibility with the Boltzmann measure e E(x))/TT is desirable to interconvert this with likelihood in Euclidean space p(x L), x E 3 A factor is necessary to convert between the volume element in torsional space and in Euclidean space (full derivation in Appendix B):
Proposition 3. Let x E C(T, L) be a centered conformer in Euclidean space. Then,
(x L)
PG(T L)
/dettett where grs
J( J( - (k)
Additional formalism is needed for translations, but it is independent of the conformer and can be ignored.

---

where the indices a, 3 are integers between I and m +3. For Sa m, Joh) is defined
EE V(bi,,
J = -J( j(h)
Σ
10
with J
(9)
x (xe xx)), E V(a),
n1
1=1
Xe,
for a E {m + 1.m + m,m + 3) as
J m xx &t,
J (8) m+2 D,
- - Xx. 2,
(10)
where (bi, 62) is the freely rotatable bond for torsion angle i, V(b.) is the of all nodes on the same side of the bond as bú, and 3t Ú, 2 are the unit vectors in the respective directions.
3.6 Energy- based training
By computing likelihoods, we can train torsional diffusion models to match the Boltzmann distribution over torsion angles using the energy function. At a high level, we minimize the usual score matching loss, but with simulated samples from the Boltzmann distribution rather than data samples. The procedure therefore consists of two stages: resampling and score matching, which are tightly coupled during training (Algorithm 1). In the resampling stage, we use the model as importance sampler for the Boltzmann distribution, where Proposition 3 is used to compute the (unnormalized) torsional Boltzmann density (T L). In the score-matching stage, the importance weights are used to approximate the denoising score-maten motth-nt loss with expectations taken over PG(T L). As the model learns the score, it improves an importance sampler.
This training procedure differs substantially from that of existing Boltzmann generators, which are trained as flows with a loss that directly depends
Algorithm 1: Energy- based training epoch
Input: Boltzmann density P. training pairs
the model density. In contrast, we train the model as a score-based model, but use it as ing and inference- to generate samples. The model density is needed only to reweight the samples to approximate the target density. Since in principle the model used for resam- pling does not need to be the same as the model being trained, can very few steps (a shallow flow) during resampling to accelerate training, and then increase the number of steps (a deeper flow) for better approximations during inference- an option unavailable to existing Boltzmann generators.
{(G,, L.) torsional diffusion model
for each (G,, [4) do
Sample Ti, TK ~ (G. Tx)):
during train-
for & < 1 to K do
L ih Approximate JnsM for pa x ÿ using {(ii, T:)) Minimize Jnsw:
(Tx T.)/ga, (Tx Lx);
Experiments
We evaluate torsional diffusion by comparing the generated and ground-truth conformers in terms of ensemble RMSD (Section 4.3) and properties (Section 4.4). Section 4.1 first discusses preprocessing procedure required to train conditional model ( L). Section 4.5 concludes with torsional
Boltzmann generators. See Appendix H for additional results, including ablation experiments.
4.1 Conformer matching
In focusing on (T L). we have assumed that we can sample local structures L ~ PG(L) with RDKit. While this assumption very good in terms of RMSD, the RDKit marginal Po is only an approximation of the ground truth PG(L). Thus, if we train on the denoising score-matching loss with ground truth conformers-i.e.. conditioned on ground truth local structures- will be distributional shift at test time, where only approximate local structures from Pa: (L. are available. We found that this shift significantly hurts performance.
We thus introduce preprocessing procedure called conformer matching. In brief, for the training split only, we substitute each ground truth conformer C' with synthetic conformer Ó' with local structures ~ (L) and made as similar as possible to C. That is, we use RDKit to generate
For example, if the resampler perfect, the procedure would reduce normal denoising score matching.
7

---

Table I: Quality of generated conformer ensembles for the GEOM-DRUGS test set in terms of Coverage (%) and Average Minimum RMSD (Å). We compute Coverage with a threshold of d = 0.75 A to better distinguish top methods. Note that this is different from most prior works, which used 8 = 1.25 A.
Recall
Precision
Coverage f Med
AMR +
Coverage 1
AMR
Method
Mean
Mean
Med
Mean
Med
Mean
Med
RDKit ETKDG
38.4
28.6
1.058
1.002
40.9
30.8
0.995
0.895
OMEGA
53.4
54.6
0.841
0.762
40.5
33.3
0.946
0.854
GeoMol
44.6
41.4
0.875
0.834
43.0
36.4
0.928
0.841
GeoDiff
42.1
37.8
0.835
0.809
24.9
14.5
1.136
1.090
Torsional Diffusion
72.7
80.0
0.582
0.565
55.2
56.9
0.778
0.729
and change torsion angles + to minimize RMSD(C, Ō). Naively, we could sample L -~ (L) independently for each conformer, but this eliminates any possible dependence between and T that could serve as training signal. Instead, we view the distributional shift as domain adaptation problem that can be solved by optimally aligning (L) and (c(L). See Appendix for details.
4.2 Experimental setup
Dataset We evaluate the GEOM dataset [Axelrod and Gómez-Bommarelli, 2022], which provides gold-standard conformer ensembles generated with metadynamics in CREST [Pracht et al., 2020]. We focus on GEOM-DRUGS- the largest and most pharmaceutically relevant part of the dataset- consisting of 304k drug-like molecules (average 44 atoms). To test the capacity to extrapolate to the largest molecules, we also collect from BEOM-MoleculeNett all species with more than 100 atoms into dataset we call GEOM-XL and use it to evaluate models trained on DRUGS. Finally, we train and evaluate models on GEOM CMMM,,QQ, more established dataset but with significantly smaller molecules (average 11 atoms). Results for GEOM-XL and GEOM-QNA are in Appendix H.
Evaluation We use the train/val/test splits from Ganea al. [2021] and use the same metrics to compare the generated and ground truth conformer ensembles: Average Minimum RMSD (AMR) and Coverage. These metries are reported both for Recall (Rich measures how well the generated ensemble covers the ground-truth ensemble- and Precision (P>-which measures the accuracy of the generated conformers. See Appendix G for exact definitions and further details. Following the literature, we generate 2K conformers for a molecule with K ground truth conformers.
We compare with the strongest existing methods from Section 2. Among cheminfor- matics methods, we evaluate RDKit ETKDG [Riniker and Landrum, 2015], the most established open-source package, and OMEGA [Hawkins al., 2010, Hawkins and Nicholls, 2012], a commer- cial software in continuous development. Among machine learning methods, we evaluate GeoMol [Ganea al., 2021] and GeoDiff [Xu et al., 2022], which have outperformed all previous models on the evaluation metrics. Note that GeoDiff originally used a small subset of the DRUGS dataset, so we retrained it using the splits from Ganea et al. [2021].
Baselines
4.3 Ensemble RMSD
Torsional diffusion significantly outperforms all previous methods on GHOM-DRUGS (Table 1 and Figure 3), reducing by 30% the average minimum recall RMSD and by 16% the precision RMSD relative to the previous ofate-of-the-ar method. Torsional diffusion is also the first ML method to consistently generate better ensembles than OMEGA. As OMEGA is well-established product used in industry, this represents essential step towards establishing the utility of conformer generation with machine learning.
Torsional diffusion offers specific advantages over both GeoDiff and GeoMol, the most advanced prior machine learning methods. GeoDiff, a Euclidean diffusion model, requires 5000 denoising steps to obtain the results shown, whereas our model-thane to the reduced degrees of freedom requires only 20 steps. In fact, our model outperforms GeoDiff with as few as denoising steps. As seen in Table 2, this translates to enormous runtime improvements.
8

---

Figure 3: Mean coverage for recall (left) and precision (right) when varying the threshold value ó on GEOM-DRUGS.
Coverage Recall
Coverage Precision
100
100
80
Coverage (%)
Coverage (%)
60
RDKit
RDKit
40
40
OMEGA
OMEGA
GeoMol
GeoMol
20
GeoDiff
20
GeoDiff
Tor. Diff.
Tor. Diff.
0
0
0.0
0.5
1.0
1.5
2.0
0.0
1.0
2.0
Threshold 6 (angstrom)
Threshold 5 (angstrom)
Table 2: Median AMR and runtime (core-secs per con- former) of machine learning methods, evaluated on CPU for comparison with RDKit.
Table 3: Median absolute error of gener-
ated v.s. ground truth ensemble properties.
E, Ae, Emin kcal/mol, μ debye.
Method
Steps
AMR-R
AMR-P
Runtime
Method
E
A
F Funin
RDKit
1.002
0.895
0.10
GeoMol
0.834
0.841
0.18
RDKit
0.81
0.52
0.75
1.16
GeoDiff
5000
0.809
1.090
305
OMEGA
0.68
0.66
0.68
0.69
GeoMol
0.42
0.34
0.59
0.40
Torsional
5
0.685
0.963
1.76
GeoDiff
0.31
0.35
0.89
0.39
Diffusion
10
0.580
0.791
2.82
Tor. Diff.
0.22
0.35
0.54
0.13
20
0.565
0.729
4.90
Compared to torsional diffusion, GeoMol similarly makes use of intrinsic coordinates. However, since GeoMol can only access the molecular graph, it less suited for reasoning about relationships that emerge only in spatial embedding, especially between regions of the molecule that are distant on the graph. Our extrinsictto-intriisic score framework-which gives direct access to spatial relationships-addeeee addresses precisely this issue. The empirical advantages are most evident for the large molecules GEOM-XL, on which GeoMol fails to improve consistently over RDKit (Appendix H). On the other hand, because GeoMol requires only single-forward pass, it retains the advantage of faster runtime compared to diffusion bassed methods.
4.4 Ensemble properties
While RMSD gives a geometric way to evaluate ensemble quality, we also consider the chemical similarity between generated and ground truth ensembles. For a random 100-molecule subset of DRUGS, we generate min(2K, conformers per molecule, relax the conformers with GFN2-xTB [Bannwarth et al., 2019]," and compare the Boltzmano-weighted properties of the generated and ground truth ensembles. Specifically, the following properties computed with xTB [Bannwarth et al., 2019]: energy E, dipole moment μ., HOMO-LUMO gap Ae, and the minimum energy Emin- The median errors for torsional diffusion and the basclines are shown in Table 4. Our method produces the most chemically accurate ensembles, especially in terms of energy. In particular, we significantly improve over GeoMol and GeoDiff in finding the lowest-energy conformers that only (on median) kcal/mol higher in energy than the global minimum.
Results without relaxation (which are less chemically meaningful) are in Appendix H.

---

4.5 Torsional Boltzmann generator
Finally, we evaluate how well torsional Boltzmann generator trained with MMFF [Halgren, 1996] ener- gies can sample the corresponding Boltzmann density over torsion angles. We train and test on GEOM- DRUGS molecules with 3-7 rotatable bonds and use the local structures of the first ground-truth conform- ers. For the baselines, we implement annealed impor- tance samplers (AIS) [Neal, 2001] with Metropolis- Hastings steps over the torsional space and tune the variance of the transition kernels.
Table 4: Effective sample size (out of 32) given by importance sampling weights over the torsional Boltzmann density.
Temp. (K)
Method Steps
1000
500
300
Uniform
1.71
1.21
1.02
5
2.20
1.36
1.18
AIS
20
3.12
1.76
1.30
100
6.72
3.12
2.06
Table 4 shows the quality of the samplers terms of the effective sample size (ESS) given by the weights of samples for each test molecule, which measures the a-divergence (with and Boltzmann distributions [Midgley et al., 2021]. Our method significantly outperforms the AIS baseline, and improves with increased step size despite being trained with only a 5-step resampler. Note that, since these evaluations are done on unseen molecules, they are beyond the capabilities of existing Boltzmann generators.
Torsional
5
7.28
3.60
3.04
BG
20
11.42
6.42
4.68
2) between the model
5
Conclusion
We presented torsional diffusion, a method for generating molecular conformers based on diffusion process restricted the most flexible degrees of freedom. Torsional diffusion is the first machine learning model to significantly outperform standard cheminformatics methods and is orders of magnitude faster than previous Euclidean diffusion models. Using the exact likelihoods provided by our model, we also train the first system-agnostic Boltzmann generator.
There are several exciting avenues for future work. A natural extension is to relax the rigid local structure assumption by developing an efficient diffusion- based model over the full space of intrinsic coordinates while still incorporating chemical constraints. Moreover, torsional diffusion-or similar ideas ideas-coule could be applicable to larger molecular systems, for which fast, parsimonious models of structural flexibility could benefit applications such drug discovery and protein design.
Acknowledgments
We pay tribute to Octavian-Eugen Ganea (1987-2022), dear colleague, mentor, and friend without whom this work would have never been possible.
We thank Hannes Stärk, Wenxian Shi, Xiang Fu, Felix Faltings, Jason Yim, Adam Fisch, Alex Wu, Jeremy Wohlwend, Peter Mikhael, and Saro Passaro for helpful feedback and discussions. We thank Lagnajit Pattanaik, Minkai Xu, and Simon Axelrod for their advice and support when working with, respectively, GeoMol, GeoDiff and the GEOM dataset. This work supported by the Machine Learning for Pharmaceutical Discovery and Synthesis (MLPDS) consortium, the Abdul Latif Jameel Clinic for Machine Learning in Health, the DTRRA Discovery of Medical Countermeasures Against New and Emerging (DOMANE) threats program, the DARPA Accelerated Molecular Discovery program and the Sanofi Computational Antibody Design grant. We acknowledge support from the Department of Energy Computational Science Graduate Fellowship (BJ), the Robert Shillman Fellowship (GC), and the NSF Graduate Research Fellowship (JC).
10

---

References
Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 1982.
Simon Axelrod and Rafael
Geom, energy annottted molecular conformations for property prediction and molecular generation. Scientific Data, 2022.
Christoph Bannwarth, Sebastian Ehlert, and Stefan Grimme, Gfn2-xtb-ai an accurate and broadly parametrized self-consistent tight-binding quantum chemical method with multipole electrostatics and mennsitt-deeedden dispersion contributions. Journal of chemical theory and computation, 2019.
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess Smidt, and Boris Kozinsky. E equviuirrii graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 2022.
Evan E Bolton, Sunghwan Kim, and Stephen H Bryant. Pubchem3d: conformer generation. Journal of cheminformatics, 2011.
Sean M Carroll. Spacetime and geometry. Cambridge University Press, 2019.
Gregory S Chirikjian. Stochastic models, information theory, and Lie groups, volume 2: Analytic methods and modern applications, volume 2. Springer Science & Business Media, 2011.
Jason C Cole, Oliver Korb, Patrick McCabe, Murray G Read, and Robin Taylor. Knowledge- based conformer generation using the cambridge structural database. Journal of Chemical Information and Modeling, 2018.
David F Crouse.
On implementing 2d rectangular assignment algorithms. IEEE Transactions on Aerospace and Electronic Systems, 2016.
Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton, Yee Whye Teh, and Arnaud Doucet. Riemannian score-based generative modeling. arXiv preprint, 2022.
Edward M Driggers, Stephen P Hale, Jinbo Lee, and Nicholas Terrett. The exploration of macro- cycles for drug discovery-an underexploited structural class. Nature Reviews Drug Discovery, 2008.
Octavian Ganea, Lagnajit Pattanaik, Connor Coley, Regina Barzilay, Klavs Jensen, William Green, and Tommi Jaakkola. Geomol: Torsional geometric generation of molecular 3d conformer ensembles. Advances in Neural Information Processing Systems, 2021.
Mario Geiger, Tess Smidt, Alby M., Benjamin Kurt Miller, Wouter Boomsma, Bradley Dice, Kostiantyn Lapchevskyi. Maurice Weiler, Michal Tyszkiewicz, Simon Batzner, Dylan Madis- etti, Martin Uhrin, Jes Frellsen, Nuri Jung, Sophia Sanborn, Mingjian Wen, Josh Rackers, Marcel Rød, and Michael Bailey. Euclidean neural networks: eănn, April 2022. URL https://dol org/10. 10. 5281/zenodo 6459381.
Thomas Halgren. Merck molecular force field. basis, form, scope, parameterization, and performance of mmff94. Journal of computational chemistry, 1996.
Paul CD Hawkins. Conformation generation: the state of the art. Journal of Chemical Information and Modeling, 2017.
Paul CD Hawkins and Anthony Nicholls. Conformer generation with omega: learning from the data set and the analysis of failures. Journal of chemical information and modeling, 2012.
Paul CD Hawkins, A Geoffrey Skillman, Gregory I. Warren, Benjamin A Ellingson, and Matthew T Stahl. Conformer generation with omega: algorithm and validation using high quality structures from the protein databank and cambridge structural database. Journal of chemical information and modeling, 2010.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 2020.

---

Michael F Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Communications in Simulation and Computation, 1989.
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. In International Conference Learning Reprexentations, 2021.
Bowen Jing, Gabriele Corso, Renato Berlinghieri, and Tommi Jaakkola. Subspace diffusion generative models. arXiv preprint arXIvv2205.01449, 2022.
Leslie Kish. Survey sampling. Number 04; HN29, K5. 1965.
Jonas Köhler, Andreas Krämer, and Frank Noé. Smooth normalizing flows. Advances in Neural Information Processing Systems, 2021.
David Lagorce, Tania Pencheva, Bruno O Villoutreix, and Maria A Miteva. Dg-ammos: A new tool to generate 3d conformation of small molecules using istance eometry and utomated m olecular m echanics ptimization for in silico creening. BMC Chemical Biology, 2009.
Greg Landrum et al. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling, 2013.
Jiabo Li, Tedman Ehlers, Jon Sutter, Shikha Varma-O Brien, and Johannes Kirchmair. Caesar: a conformer generation algorithm based recursive buildup and local rotational symmetry consideration. Journal of chemical information and modeling, 2007.
Shitong Luo, Chence Shi, Minkai Xu, and Jian Tang. Predicting molecular conformation via dynamic graph score matching. Advances in Neural Information Processing Systems, 2021.
Oscar Méndez- Lucio, Mazen Ahmad, Ehecatl Antonio del Kio-Chanona, and Jörg Kurt Wegner. geometric deep learning approach to predict binding conformations of bioactive molecules. Nature Machine Intelligence, 2021.
Laurence Illing Midgley, Vincent Stimper, Gregor NC Simm, and José Miguel Hernández-Lobato. Bootstrap your flow. arXiv preprini, 2021.
Maria A Miteva, Frederic Guyon, and Pierre Tuffery. Frog2: Efficient 3d conformation ensemble generator for small compounds. Nucleic acids research, 2010.
Radford M Neal. Annealed importance sampling. Statistics and computing, 2001.
Alex Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Interna- lional Conference on Machine Learning, 2021.
Frank Noé, Simon Olsson, Jonas Köhler, and Hao Wu. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning. Science, 2019.
Philipp Pracht, Fabian Bohle, and Stefan Grimme. Automated exploration of the low-energy chemical space with fast quantum chemical methods. Physical Chemistry Chemical Physics, 2020.
Martin Quack. How important is parity violation for molecular and biomolecular chirality? Ange- wandte Chemie International Edition, 2002.
Sereina Riniker and Gregory A Landrum. Better informed distance geometry: using what know to improve conformation generation. Journal of chemical information and modeling, 2015.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference Learning Representations, 2022.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. (n) equivariant graph neural networks. In International Conference on Machine Learning, 2021.
Kristof Schütr, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Robert Müller. Schnet: continuous-file convolutional neural network for modeling quantum interactions. Advances in neural information processing systems, 2017.
12

---

Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular conformation generation. International Conference on Machine Learning. 2021.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference Learning Representations, 2021.
Hannes Stärk, Octavian-Eugen Ganea, Lagnajit Pattanaik, Regina Barzilay, and Tommi Jaakkola. Equibind: Geometric deep learning for drug binding structure prediction. In International Confer- ence on Machine Learning. 2022.
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang. Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivariae neural networks for 3d point clouds. arXiv preprint, 2018.
Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. In Advances in Neural Information Processing Systems, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neurul information processing systems, 2017.
Minkai Xu, Shitong Luo, Yoshua Bengio, Jian Peng, and Jian Tang. Learning neural generative dynamics for molecular conformation generation. In International Conference on Learning Representations, 2021a.
Minkai Xu, Wujie Wang, Shitong Luo, Chence Shi, Yoshua Bengio, Rafael Komez-Bombarelli and Jian Tang. An end-to-end framework for molecular conformation generation via bilevel programming. International Conference on Machine Learning, 202
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. In International Conference Learning Representations, 2022.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Appendix
(c) Did you discuss any potential negative societal impacts of your work? [Yes] Conformer generation useful for many scientific applications, some of which could have negative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [Yes] See Ap- pendix B.
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix B.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material URL)? [Yes] The code can be found at https: htt//tgtttbbu///t//// com/
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix G.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Appendix H.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Approximately 2000 GPU-hours
on an internal cluster.
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] We used the code and data released
by GeoMol and GeoDiff released under MIT license and the GEOM datasets released
under CCO 1.0 license.
(c) Did you include any assets either in the supplemental material or URL? [Yes] See the instructions in our repository https: /ttp::///ttttttttte com/ wtt.ubb..o//cooro/// corsional-diffuse
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A] All data used is open-source.
(e) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [N/A] No personal data used.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "t7WDU3dgBcT", "reviewer": "Reviewer_QmNA", "summary": "This paper proposed a novel diffusion model for molecular conformer generation. The proposed diffusion model is operated in the space of torsion angles of the conformer. An extrinsic-to-intrinsic score model is learned for the diffusion process to predict the torsional scores directly from the 3D point cloud representation of the conformer. The exact likelihoods of the generated conformers can be computed to enable energy-based training with samples from the Boltzmann distribution. The proposed diffusion model outperforms existing machine learning and cheminformatic-based solutions on the GEOM benchmark. ", "questions": "I am very excited to see that the proposed method outperforms commercial software for molecular conformation prediction. I am interested in the potential of such a model in the future. What are the main practical limitations (e.g., sample efficiency, training time, inference time, ...) for deploying and using such a model in an industrial scenario? Can it be considered for high-throughput applications? What is the main challenge of adapting the model to larger molecules (e.g., proteins) use cases?", "limitations": "The authors have adequately addressed the limitations in the appendix.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper proposed a novel diffusion model that has a strong empirical impact on the fundamental task of molecular conformer generation. With comprehensive experiments, the authors showed that the model is able to generate more accurate conformers in less time compared with existing machine learning solutions. The model also consistently outperforms a commercial software OMEGA on the GEOM benchmark. The evaluations were conducted with high quality, with sufficient details, strong quantitative evidence, and adequate ablation analysis.\n\nTheoretically, the intuitions behind the important design decisions are well explained, i.e., learning a diffusion model in the torsional space instead of euclidean space, using a SE(3)-invariant model to parameterize the score function, etc. The theory behind the diffusion model and training algorithm is also sound. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "IxGFEGAsUTj", "reviewer": "Reviewer_hcv8", "summary": "This paper studies molecular conformation generation. Different from previous works that focus on coordinate prediction or distance prediction, this paper proposes to focus solely on torsional angles of rotatable bonds in molecules, leaving all other degrees of freedom fixed. \n\nInspired by diffusion models on Riemannian manifolds, the authors first lay the theoretical framework of torsional diffusion on hypertorus following the standard denoising diffusion models. Based on this framework, they show that such a diffusion process can also be defined on Cartesian coordinates, and design a SE(3)-invariant and parity equivariant score network operating on 3D point clouds.\n\nThey show the proposed method achieves state-of-the-art performance on GEOM-DRUGS, with much fewer denoising steps compared to previous models. The resulting method is also capable of exact likelihoods estimation, making it suitable for matching the Boltzmann distribution over torsion angles using the energy function.", "questions": "In line 111, the authors give the definition of the freely rotatable bond, which are those that serve as a bridge between two components. Is this definition the same as the definition provided by RDKit API? (As I remember we can get all rotatable bonds of a molecule by calling a certain function provided by RDKit).\n\nAs far as I understand, torsion angles in cycles are also the degrees of freedom that determine the conformation of a molecule. How are these rotatable bonds handled in the proposed method? Can we also do diffusion on these bonds?", "limitations": "The authors didn't discuss limitations and potential negative societal impact of their work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\nTorsional angles are the most important degrees of freedom that determine the conformations of a molecule. Focusing on torsional angle modeling for conformation prediction is a straightforward and novel idea. To the best of my knowledge, it is the first work that marries denoising diffusion models with torsional angle prediction, and I like this idea.\n\nThe resulting score functions for torsional angle diffusion is SE(3)-invariant and parity equivariant for every rotatable bond in a molecule, which is a nice property.\n\nThe proposed model is capable of exact likelihood estimation, which is another nice property thanks to the use of SDE.\n\n### Weaknesses\nThe model relies on an external algorithm, such as RDKit ETKDG, to generate the initial local structures for the target molecule, and the only degrees of freedom are torsional angles. This means the chirality of the conformation is entirely determined by the initial structure, or equivalently, by the external algorithm that provides the initial guess structure.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "B0duXulAweJ", "reviewer": "Reviewer_P55i", "summary": "This paper argues that instead of generating bond length, angle and torsion simultaneously, one should focus on the more difficult part, i.e. the torsion. And the authors further propose a diffusion model working on this Riemannian manifold of the torsion angles. A Bolzman generator is then introduced based on the likelihood calculated from the diffusion model. Experiments on molecules with normal size, large size and small size datasets show increased performance in terms of RMSD and convergence time.\n", "questions": "\nPlease see the above weaknesses.", "limitations": "The authors have adequately discussed the limitations of torsional diffusion, which I think is quite fair.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n-The paper is well motivated to work on the torsion angles only, which reasonably leads to advantages in speed and performance, under mild assumptions about the bond length and angle. The idea is novel. \n-The idea of using the diffusion model on the torsion-composed manifold is compatible with the Bolzman generator and leads to some good properties. \n-The paper is well written. The idea, the procedure, why symmetry properties are satisfied, how to solve the sample-wise intrinsic coordinates, are well elaborated. \n\nCons: \n-Ablation studies, e.g. about the depth of diffusion, conformed matching and any other hyperparameters would help to understand and are desired. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "38vhvK5vsOa", "reviewer": "Reviewer_7ogf", "summary": "This paper propose torsional diffusion for molecular conformer generation. Different from the previous conformer generation work which mainly predict the position of each atoms in the 3D space, this work view the conformer generation in a different view by generating the torsion. To achieve this, the paper also provides the exact likelihood of the generated conformers, which lead a better training pipeline comparing with the previous work. From the experiment side, the paper reaches a very good SOTA result comparing with GEOM-DRUGS dataset, which is very convincing. \n\nIn addition, the paper propose torsional Boltzmann generator that can be expand and adapt in generating various class of molecules.", "questions": "For GeoMol and GeoDiff's performance, it seems have a huge gap between the original paper. I am not sure if it is reasonable since the author uses a different data split. It may be fair to use their split for another comparison.", "limitations": "Same as the Questions Section. ", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is in a good writing, very easy to follow and the idea is natural. Rather than learning the Euclidian information in the previous task, learning the torsion to generate conformer is more reasonable in a physical natural. Although learning torsion was proposed in GeoMol using MPNN, enhancing it use Diffusion model is still soundable. The experiment result is solid with many details. \nFor the weakness, I know from the author's statement, they retrain the GeoDiff to adapt the larger dataset, but it still seems the GEOM-DRUGS and GEOM-QM9's performance for the baseline GeoDiff and GeoMol is much lower than their report number. Any suggestion towards this huge gap? Is it possible to use their split to have another comparison?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept