Sampling 3D Molecular Conformers with Diffusion Transformers
Authors: Thorben Frank
Winfried Ripken
Gregor Lied
Klaus Robert Muller
Oliver T. Unke
Stefan Chmiela
Venue: NeurIPS 2025 poster ()
Keywords: 3D Molecular Graphs
Molecular Modeling
Generative Modeling
Diffusion Transformers
Geometric Deep Learning
Flow Matching
Primary area: machine_learning_for_sciences
TL;DR: This papers introduces a Diffusion Transformer (DiT) for sampling 3D molecular conformers.

Abstract: Diffusion Transformers (DiTs) have demonstrated strong performance in generative modeling, particularly in image synthesis, making them a compelling choice for molecular conformer generation. However, applying DiTs to molecules introduces novel challenges, such as integrating discrete molecular graph information with continuous 3D geometry, handling Euclidean symmetries, and designing conditioning mechanisms that generalize across molecules of varying sizes and structures. We propose DiTMC, a framework that adapts DiTs to address these challenges through a modular architecture that separates the processing of 3D coordinates from conditioning on atomic connectivity. To this end, we introduce two complementary graph-based conditioning strategies that integrate seamlessly with the DiT architecture. These are combined with different attention mechanisms, including both standard non-equivariant and SO(3)-equivariant formulations, enabling flexible control over the trade-off between between accuracy and computational efficiency.
Experiments on standard conformer generation benchmarks (GEOM-QM9, -DRUGS, -XL) demonstrate that DiTMC achieves state-of-the-art precision and physical validity. Our results highlight how architectural choices and symmetry priors affect sample quality and efficiency, suggesting promising directions for large-scale generative modeling of molecular structures. Code is available at [https://github.com/ML4MolSim/dit_mc](https://github.com/ML4MolSim/dit_mc).

Sampling 3D Molecular Conformers with Diffusion Transformers
J. Thorben Frank 1.2+ Winfried Ripken 1.2-
Gregor Lied
Klaus-Robert Müller 1,,33,15
Oliver T. Unke
Stefan Chmiela
TTechicaal University Berlin BIIOLD Berlin
Google DeepMind
MMI for Informatics, Saarbrücken *Department of Artificial Intelligence, Korea University
thorbenjan. www..//gaaaaaaaaa..c..mom.mm com, oliverunke@google. com, stefan@chmiela. com
Abstract
Diffusion Transformers (DiTs) have demonstrated strong performance in generative modeling, particularly in image synthesis, making them compelling choice for molecular conformer generation. However, applying DiTs to molecules introduces novel challenges, such as integrating discrete molecular graph information with continuous 3D geometry, handling Euclidean symmetries, and designing condition- ing mechanisms that generalize across molecules of varying sizes and structures. We propose DiTMC, framework that adapts DìTs to address these challenges through a modular architecture that separates the processing of 3D coordinates from conditioning atomic connectivity. To this end, we introduce two com- plementary graph- based conditioning strategies that integrate seamlessly with the DiT architecture. These are combined with different attention mechanisms, includ- ing both standard non equivariant and sO(3)-equivarial formulations, enabling flexible control over the trade-off between between accuracy and computational efficiency. Experiments on standard conformer generation benchmarks (GEOM- QM9, -DRUGS, -XL) demonstrate that DiTMC achieves state-of-the-arr precision and physical validity. Our results highlight how architectural choices and sym- metry priors affect sample quality and efficiency, suggesting promising directions for large-scale generative modeling molecular structures. Code is available at https:://githiii hig.......... com/MLLMolSSimdaii dit_mc
Introduction
The Maree-dimensional arrangement of atoms in molecule, known conformation. determines its biological activity and physical properties, making it fundamental to applications such as computa- tional drug discovery and material design. Accurately predicting the most energetically favorable conformers (i.e., stable conformations) for large molecular systems is a highly non-trivial task. Tradi- tional techniques, such as Molecular Dynamics and Markov Chain Monte Carlo, attempt explore the conformational space by simulating physical movement or probabilistic sampling. However, these methods often require many simulation steps move from one conformer another, making them computationally expensive. Generative machine learning (ML) models offer a more targeted approach by allowing directly sample from the space of promising conformations.
Recent years have seen significant progress, enabled by the development of specialized architectures for the generation of molecules [119] and materials 12 This is in contrast to image and video synthesis, where the more generalized diffusion transformer (DiT) architecture [3] has
Equual contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

A
B
(1=
Scale
Transforer
x2
per
Algelet
1.4
Probabilly flw CDE
----
vix 1 G) 4r
per
Selvvtsciion hiok
K-
Pou. embod
Embed
MLL
Gruph embed
A.A.
pot
Prio (1=
Condihoiing
Atomic
Malicular
postions
numbers
graph
Figure 1:
(A) Diffusion transformer for molecular conformer generation (DiTMC), with interchange- able self-attention blocks and positional embeddings (PEs); we evaluate various combinations as detailed in the main text. (B) DiTMC predicts velocity per atom, used to model probability flow ODE, which samples from the probability distribution p( (9), where G is a molecular graph.
become a leading model, consistently delivering strong performance and efficiency across diverse applications 14 Adapting DiTs, which were originally developed for grid-structured image data, to continuous, irregular molecular geometries poses unique challenges, which need to be addressed unlock the potential of this powerful architecture for molecular conformer generation. Key design questions include how to encode molecular connectivity and incorporate Euclidean symmetries, such as translational and rotational invariance/eqiivariace.
In this work, we address these conceptual challenges and propose DiTMC, a new DiT-style archi- tecture for molecular conformer generation. We introduce novel conditioning strategies based on molecular graphs, enabling the generation of 3D structures. Our modular architectural design allows us systematically investigate the impact of different self-attention mechanisms within the DiT ar- chitecture on conformer generation quality and efficiency. We conduct a comparative study including standard (non-qquivariant) self-attentien with both absolute and relative positional embeddings and an explicitly (0(3))equivarin) variant. While exact equivariance can positively impact performance, also incurs significant computational costs. We find that simpler attention mechanisms are highly scalable and still perform competitively. Multiple of the tested DiTMC variants achieve state-of- art precision on established conformer generation benchmarks. Moreover, the molecular structure ensembles generated by our models align more closely with physical reality, as evidenced by the high accuracy of physical properties extracted from them. To summarize, our work contains the following main contributions:
We propose two complementary conditioning strategies based on trainable conditioning tokens for (pairs of) atoms extracted from molecular graphs, which are designed to align with architectural principles of DiTs. We propose to condition our self-attention formu- lation geodesic graph distances extracted from molecular graphs and demonstrate that it significantly increases performance of our model.
We investigate the impact different self-attention mechanisms, including standard (non- equivariant) and SO(3) -quuivraua formulations, on model accuracy and performance. We find that including symmetries can improve the fidelity of generated samples at the price of increased computational cost during training and inference.
Based on our insights, we present a simple, non-equivariant, yet expressive DiT architecture that achieves state-of-the-art precision and physical validity established benchmarks. Its performance improves with model scaling, making it promising candidate for large-scale molecular conformer generation.
2 Related Work
Generative Modeling Generative models create diverse, high-quality samples from unknown data
distribution. They are widely used in image generation [8 19). text synthesis (0) and the natural sciences 23
Most recent approaches learn probabilistic path from simple prior to the data

---

distribution, enabling both efficient sampling and likelihood estimation 22 This is achieved by different modeling paradigms, including flow matching [26] or denoising diffusion 24
(Diffusion) Transformers Originally introduced for natural language processing 29 transformer
architectures have become state-of-the-art in computer vision [8] and recently also found widespread
adoption in quantum chemistry, c.g., for protein prediction [30 31. 3D molecular generation [32
33 or molecular dynamics simulations (3) In the context of generative modeling, diffusion transformers [3 (DiTs) have emerged as powerful tools incorporating conditioning tokens, e.g., to
prompt image generation [3] or to design molecules and materials with desirable properties [37 This work applies prototypical DiT's to molecular conformer generation.
Molecular Conformer Generation Molecular conformer generation aims find atomic arrange- ments (Cartesian coordinates) consistent with a given molecular graph. Numerous MI. approaches have been proposed for sampling conformers, all aiming improve upon conventional methods.
While early approaches were based on RDKit
or variational auto encoders [39,4], more recent advances employ diffusion and flow-based models [42, 43), including E-NFs 29. CGCF 44
GeoDiff [45, DiffionDiff [46), MCF 447. ET-Flow [32), and DMT [48].
De-Novo Molecular Generation Instead of conditioning on given molecular graph, several existing approaches tackle the problem of finding novel and stable molecules given only set of atoms 8 49 50]. Bridging the gap between unconditional and conformer generation, universal models have been proposed that can solve multitude of tasks. In particular, models within the Uni-Mol model family [51 [2]. make use of graph geodesic distances. However, their design requires tracking high-dimensional pair-representations throughout the architecture, which makes less efficient.
Preliminaries
3.1 Molecular Conformers
A molecule can be represented as molecular graph 9 (V, where the nodes V correspond to atoms and the edges & represent chemical bonds between them. The nodes and edges contain information about their types, bond orders, and additional structural features such as branches, rings, and stereochemistry. The missing component is the exact spatial arrangement the N = | atoms, represented 3D point cloud E: Nx3 3 in Euclidean space. Only the relative distances between atoms are relevant, as translating or rotating the entire point cloud æ does not change the identity of the conformer. We frame conformer prediction sampling from the SE(3)-invariant conditional probability distribution p( 9), which will guide the design of our model architectures in Sec.
3.2 Conditional Flow Matching
Starting from an sasyll base distribution qo Rd + R20, a generative process creates samples from a target distribution Q1 Rd + Rzo (27) Here, Q1 models the molecular conformer data with d = N We aim to learn a time-dependent vector field u,(x) [0, 1] x N + R Nx3 which defines ordinary differential equation (ODE) whose solution pushes samples æo gx3 from the prior to samples 21 E g. x3 from the data distribution. We describe this transformation in terms of stochastic interpolant 2e
26 53
54]
A noisy sample at time & [0, 1] is defined as
3o + -..
where 6 E R3 drawn from the standard normal distribution A I) and scaled by a constant E R2o. We remark that represents progress along this interpolation path, not physical time. Notably, stochastic interpolants enable transformations between arbitrary distributions and allow us to assess the performance of the generative process under varying prior distributions go. This contrasts with, c.g., score based diffusion methods [24] (5), which typically assume an isotropic Gaussian prior.
The stochastic interpolant induces deterministic trajectory of densities (x(),, governed by an ODE known as the probability flow: dæ (x) dt.
(2)
3

---

If the vector field (x) was tractable to sample, the weights of a neural network (NN) (x, t)
[0, 1] x Th + Rd could be optimized directly by minimizing
(F) = E0-
[us(x) v (xx
The learned vector field rn could then be used to generate samples from the target distribution by starting from 2p ~ do and integrating the probability flow ODE (Eq. 2). for example, using a numerical scheme such Fuler's method, i.e., #t+ #; d (z,, A) for time step At.
However, for arbitrary distributions qo and 71, the objective in Eq. 3 computationally intractable 56 Instead, consider the expectation over interpolated point pairs from the two distributions. Eq.
defines a conditional probability distribution pr. (x <0, x1) = (z - t) 2o + t 31, o²), with
conditional vector field uz (x [D, x1) 21 Zo [27]. The ability to directly sample from the
conditional probability via Eq. allows formulating the conditional flow matching (CFM) objective
Uu (x ano an) v (ar, 1)l.
LCrM (0) (0.1)) 20000:00............. 21 (e.p.. kaa.
As shown in Ref. 26 the gradients of the two losses coincide, C VoLCrM. thereby recovering the vector field that defines the probability flow ODE in Eq 2 Following prior work [37] 577, we reparametrize the training objective to predict noise-free data æij directly. During inference, we invert the reparametrization to obtain 8 for sampling (see Appendix for details).
4 A New Diffusion Transformer for Molecular Conformer Sampling
We now describe the methodological advances of our work. We propose DiTMC, new DiT-style architecture for conformer generation by learning vector field using the loss in Eq. 4 As outlined in Sec. 3.1 this involves sampling from a conditional probability p(æ 9), where G is the molecular graph representing atomic connectivity. Therefore, we choose our model be a function (,,,,, 1,,t,,9 9). where the final output is a 3D velocity vector per atom, which is extracted from a readout layer.
Next, we outline the key components of the DiTMC architecture, with overview shown in Fig. Training and architectural details are provided in Appendix and Appendix D respectively.
4.1 Conditioning Tokens
We begin by defining the conditioning tokens in DITMC. Each DiTMC block receives time conditioning token c' E "R as well as atom-wise conditioning tokens C {ct E R lie [M]].
and pair-wise conditioning tokens C {clj E R i,j [N]} derived from G (V,
Time conditioning The current time t of the latent state æ, is encoded via a two-layer MLP as
= MLP(().
(5)
Atom-wise conditioning Atom-wise graph conditioning tokens are obtained from GNN inspired by the processor module of the MeshGraphNet (MGN) framework (58) as
c = GNN (V,E),
(6)
where of denotes the final node representation for atom i. See Appendix D.3 for details on the GNN.
Pair-wise conditioning We define pair-wise graph conditioning tokens inspired by the Graphormer architecture [59] as
Sf MLP (s(i.))
where (i.jj) denotes the graph geodesic, i.e., the shortest path between atoms and j in G. This formulation allows conditioning all atom pairs, including those not directly connected by bond.
4.2 Positional Embeddings
To encode the atomic positions = (f....,, F |n R33 of the latent state T,, we use positional embeddings (PEs). We examine representative range of positional embeddings that vary in the

---

number of Euclidean symmetries they respect by construction, which affects how the latent represen- tations transform under translations and rotations. We denote the set of positional embeddings by P. where P = (p. E } for the atom-wise (aPE) embeddings, and P {Pt i E [N], # j) for pair-wise (rPE or PE(3)) embeddings. Without loss of generality, we assume that the positions are centered such that the center of mass vanishes (see Appendix A).
Absolute Positional Embeddings Following Refs. dings (aPE) are calculated as
31 atom-wise absolute Positional Embed-
P PE MLP(F.),
such that p E TR... This kind of positional embedding does not preserve rotational nor translational invariance, and serves as baseline without any symmetry constraints.
Relative Positional Embeddings We use displacements vectors Fij = - -F- Fjj relative Positional Embeddings (rPE) as
build pairwise
PE Pij
MLP(Fgg)
such that Pi E R. This formulation ensures translational invariance but not rotational invariance.
Euclidean Positional Embeddings Adapting ideas from equivariant message passing neural networks like PaiNN [60 or NequlP [61], we construct -Quui-equivariant pairwise Euclidean Positional Embeddings (PE(3)) as a concatenation of L + components
1
PEC3 Pij
① 2=
(r(r)))) O Ye(f)))
(10)
where de R + RIxH is radial filter function, f F/r, and Y, (2+++1)x are spherical harmonics of degree 0, O,...,I. L. The element-wise multiplication "Q" between radial filters and spherical harmonics is understood to be "broadcasting" along axes with size 1, such that PE(3) R (L+1))xHH Under rotation of the input positions, these positional embeddings transform equivariantly (see Appendix E]. Moreover, because displacement vectors used as inputs, the embeddings are also invariant translations. As a result, they respect the full set of Euclidean symmetries relevant to molecular geometry.
(Ø) Y.) R(2+1)xxH and (after concatenation) Púj
4.3 DiTMC Block
we can define different DiTMC blocks that preserve the extent of Euclidean symmetries encoded in the embeddings throughout the model.
Based on the positional embedding strategies introduced in Sec. 4.2
Each DÍTMC block transforms set of input tokens H. into of output tokens H', which as input for the next block. For aPE and rPE, we have H = (hi,..., hy h, E TR. "R, and for PE(3),
we have H (hi.... h h; E (R(+))**HH)
In each DITMC block, we inject time-based, well graph-based atom-wise and pair-wise conditioning information via the conditioning tokens introduced in SScc. 4.1 We use the wiir-wise graph conditioning tokens Cg. during the self-attention update,
h: = h; + (H, P, P,
(1))
where we employ a standard self-attention mechanism for aPE and rPE and an -qqiiaaauaa self-attention mechanism for PE(3). Additionally, we use the time conditioning token ct and the atom-wise graph conditioning tokens CE (cf R l [N]} to obtain per-atom bias and scaling parameters,
0 Big Tiir Orgi 32i, 72: MLP(c* + cf),
(12)
and apply adaptive layer norm (AdaLN) and adaptive scale (AdaScale) 13 for conditioning (see Fig. TA similar to applications of DiTs image synthesis.
provide details on the non-equivariant DiTMC blocks based on aPE and rPE in Appendix D. and on the -Quiiaaqui DiTMC block based PE(3) in Appendix D.2
5

---

Table 1: Results on GEOM-QM9 for different generative models (parameter counts in parentheses). -R indicates Recall, indicates Precision. Best results in bold, second best underlined; our models are marked with an asterisk (+). Our results are averaged over three random seeds. See Appendix Tab. for results including standard deviations.
Method
COV-R [%]
AMR-R [
COV-P [%]t
AMR-P [A]
Mean
Median
Mean
Median
Mean
Median
Mean
Median
GeoMol (0.3M)
91.5
100.0
0.225
0.193
86.7
100.0
0.270
0.241
GeoDiff (1.6M)
76.5
100.0
0.299
0.229
50.0
33.5
0.524
0.510
Tors. Diff. (1.6M)
92.8
100.0
0.178
0.147
92.7
100.0
0.221
0.195
MCF-B (64M)
95.0
100.0
0.103
0.044
93.7
100.0
0.119
0.055
DMT-B (55M)
95.2
100.0
0.090
0.036
93.8
100.0
0.108
0.049
ET-Flow (8.3M)
96.5
100.0
0.073
0.030
94.1
100.0
0.098
0.039
+DITMC+aPE-B (9.5M)
96.1
100.0
0.073
0.030
95.4
100.0
0.085
0.037
DiTMMC+PP-B (9.6M)
96.3
100.0
0.070
0.027
95.7
100.0
0.080
0.035
PE3) (8.6M)
95.7
100.0
0.068
0.021
93.4
100.0
0.089
0.032
A
B
structural amilrlly peak
2.0
aPE
=PP
Close
10%-
1.8
the data distribttion
PADE
Maan [S]
86
(4]
10P
1.8
Cloce distibutioo
84
1.4
10
1.2
Large similarity batwvann genenated true conformers
78
Threshold (A]
Minimal RMSD prediction [Å]
Labent time
Figure 2: Analysis SO(3) S0(3)-equivarian (PE(3)) and non non- equivariant (aPE, rPE) model formulations on GEOM-QNE RMM.. (A) Mean Coverage Recall (COV-R) versus root mean square deviation (RMSD) threshold 8 to any reference conformer.
(B) Histogram of the minimal RMSD per generated sample. (C) Loss as function of latent time relative to PE(3) loss (see Appendixx for details).
5 Experiments
Datasets and Metrics We conduct our experiments on the GEOM dataset (62 comprising QM9 (133,258 small molecules) and AICures (304,466 drug-like molecules). Reference conformers generated using CREST 633 rotatable bonds and multiple stereocenters. The data splits are taken from Ref. 664
Drug-like molecules exhibit greater structural diversity, including more
We evaluate our models' ability to generate accurate and diverse conformers using average minimum RMSD (AMR) and coverage (COV), measuring recall (ground-truth coverage) and precision (genera- tion accuracy). A generated conformer is considered valid if it falls within a specified RMSD threshold of any reference conformer ing prior work, we generate 2K conformers per test molecule with K reference structures. Appendix G.3 provides further details on the calculation of metrics. Following ET-Flow [32 and GeomMol (64]
0.5A for GEOM-QM9 and 0. 7,7 for GEOM-DRUGS) Follow-
also apply chirality correction (see Appendix G.4).
Ablating Self-Attention and Positional Embedding Strategies The modular structure of DiTMC al- lows efficient exploration of the design space through variations in the positional embeddings and associated attention blocks (see|Sec. 4). We define three model variants, DiTMC+aPE, DiTMC++PE, and DiTMC+PE(3), which differ only in the choice of positional embedding and self-attention for- mulation. Architectural details can found in in/Appendix D On GEOM-QM9, our models produce diverse, high quality samples, outperforning the current state-of-the--n across all AMR-R, AMR-P and COV-P metrics, demonstrating the broad applicability of our modular design and conditioning
(sec Tab. D. We use the harmonic prior introduced in Ref. [65] throughout, which yields improved

---

Table 2: Ablation of conditioning strategies using DiTMC+aPE-B on GEOM-DRUGS. -R indicates Recall, -P indicates Precision. Best results in bold. Our results are averaged over three random seeds. See Appendix Tab. A14 for results on GEOM-QM9.
Method
COV-R [%]
AMR-R [A]4
COV-P [%]
AMR-P [Å]
Mean
Median
Mean
Median
Mean
Median
Mean
Median
No conditioning 16.6
4.7
1.132
1.110
5.15
1.2
1.849
1.845
Atom-wise
72.8
77.5
0.555
0.565
55.6
53.3
0.762
0.716
Atom-wise & Pair-wise
79.9
85.4
0.434
0.389
76.5
83.6
0.500
0.423
results compared to the Gaussian prior (see Appendix Tab. A13 and therefore adopt it in all sub- sequent experiments. Notably, our models maintain competitive performance even when using the Gaussian prior, contrary to the findings in Ref. [32]
Probing the effect of Euclidean symmetries Our PEs form hierarchy based on the extent of Euclidean symmetry incorporated by construction. This enables a systematic evaluation of how incorporating symmetry affects model behavior. We summarize findings GEOM-QM9 below.
Equivariance improves the fidelity of samples. We analyze COV-R Mean as a function of RMSD threshold 6 for all DITMC variants (see Fig. 2/). The S0(3)-equivariant DiTMC+PE(3) outperforms the non-equivariant DiTMC+aPE, and DiTMC+PPE at low ó, indicating that many of the generated conformers closely match the ground-truth structures. This appears as a leftward shift in the distribu- tion of the minimal RMSD found per generated structure (as shown in Fig. 2B) and aligns with the observation that DITMC+PE(3) achieves better AMR-R R and AMR-P values (as reported in Tab. I). To better understand this behavior, we examine the loss over time and find that the non-equivariant models exhibit higher error near the data distribution = 1) (see Fig. 2C). The increase in error towards the end of the generation trajectory results in noisier structures and reduced fidelity.
Equivariance increases the compulational cost for models of similar size. The benefit of higher fidelity comes at increased computational cost. During training, the equivariant DiTMC+PE(3) approximately 3.5 times slower than the non-equivariant DiTMC+aPE, and DrTMC+PPE and about 3 times slower inference (see Fig. 3). All models use the same number of layers and differ only in the number of heads per layer in order to match the total parameter count, discussed in Appenddix D
Conditioning Strategies To assess the impact of graph conditioning, we compare different condition- ing strategies. As further discussed in Appendix I22 we compare conditioning solely on atom-wise information (Eq. with an extended scheme that also incorporates pair-wise geodesic graph dis- tances (Eq. 7). In Tab. 2] we show ablation results on GEOM-DRUGS. We also ablate conditioning on pairwise information extracted from our conditioning GNN for each edge corresponding to a chemical bond (see Appendix (.2). We find all variants to be effective, but our proposed combination of geodesic distances and atom-wise information to perform best. This experiment underlines the importance of deriving conditioning tokens for all atom pairs, not just those connected by edges in the molccular graph (i.c. by chemical bonds), which lack global information about the graph structure.
Model Scaling To investigate model scaling, we define a small ("S"), base ("B"), and large ("L") DiTMC+aPE model variant (see Appendix DD. Each model trained with identical training hyperpa- rameters on the GEOM-DRUS to 54.1% for COV-P Mean) when scaling DiTMC+aPE-S to DiTMC+aPE-B (see Fig. A)) Scaling DiTMC+aPE-BB B to DiTMC+aPE-L yields relative improvements, which however might be smaller than expected given the strong increase performance from the small to the base model. Prior work has shown that to avoid diminishing returns, dataset and model sizes must be scaled together [66] [7]. Scaling only the model yields limited benefits as soon as model size saturates given the amount of data. Strong improvements of DiTMC+aPE-L over DiTMC+aPE-B are observed in terms of sampling fidelity (as indicated by higher COV-R and COV-P Mean values at small thresholds 8) but differences start to vanish at 8 = 0.75A (see Fig. 4A). Moreover, we obtain relative improvement terms of generalization to larger and previously unseen molecules from the GEOM-XL dataset [46) between 3. 1% (for AMR-P Mean) and 7.3% (for AMR-P Median), i.e., DiTMC+aPE-LL exhibits significantly
dataset. We find strong relative improvements for all metrics (up
stronger out-of-distribution performance compared to the smaller DiTMC+aPE-B (see Tab. 5)
7

---

A
Coverage (1)
(1)
B
80
70
Traniing
0.5
Inference
80-
60
0.8
50
AMR Meen (A)
CDV Mezn
60-
Meen
0.3-
0.4
Time
30-
Model
Mocel
30
0.2-
Embedding
Postional
20-
(1M)
0.2
(1M)
20
aPE-B (9.0M)
aPE: (9.5M)
0.1
rPE
10
2M)
PE(S)
Recal
Precision
Recall
Precision
aPE
FPE
PE(3)
Recall
Precision
Figure 3: (A) Coverage (COV) mean and Absolute Minimum RMSD (AMR) mean for DiTMC+aPE models of increasing model capacity on GEOM GEOM-DRUGS. (B) Training and inference time for different positional embedding (PE) and associated celf-attention strategies. (C) Recall and precision AMR mean for the different DiTMC models on GEOM-DRUGS, For panels (B) and (C) we use results from the base ("B") variant of each model.
Table 3: Results on GEOM-DRUGS for different generative models (parameter counts parentheses). -R indicates Recall, -P indicates Precision. Best results in marked with asterisk (+). Our results averaged over three random seeds. See Appendix Tab. A10 for results including standard deviations.
bold, second best underlined: our models
Method
COV-R [%]t
AMR-R [Å]
COV-P [%]t
AMR-P [Å]
Mean
Median
Mean
Median
Mean
Median
Mean
Median
GeoMol (0.3M)
44.6
41.4
0.875
0.834
43.0
36.4
0,928
0.841
GeoDiff (1.6M)
42.1
37.8
0.835
0.809
24.9
14.5
1.136
1.090
Tors. Diff. (1.6M)
72.7
80.0
0.582
0.565
55.2
56.9
0.778
0.729
MCF-L (242M)
84.7
92.2
0.390
0.247
66.8
71.3
0.618
0.530
DMT-L (150M)
85.8
92.3
0.375
0.346
67.9
72.5
0.598
0.527
ET-Flow SS (8.3M)
79.6
84.6
0.439
0.406
75.2
81.7
0.517
0.442
+DTMCC+aPE-B (9.5M)
79.9
85.4
0.434
0.389
76.5
83.6
0.500
0.423
DDTMC++PE-B (9.6M)
79.3
84.6
0.444
0,400
77.2
84.6
0.492
0.414
DDTMC+PP(3)-B (8.6M)
80.8
85.6
0.427
0.396
75.3
82.0
0.515
0.437
* DrTMC+aPE-L (28.2M)
79.2
84.4
0.432
0.386
77.8
85.7
0.470
0.387
* DiTMCH+PE-L (28.3M)
78.7
84.1
0.438
0.388
78.1
86.4
0.466
0.381
DDTMCC+EE3)L (31.1M)
80.8
85.6
0.415
0.376
76.4
82.6
0.491
0.414
Drug-like Molecules For the following experiments, we restrict our analysis to the base ("B") and large ("L") DiTMC variants. In contrast the smaller GEOM-QM9 dataset, the size of GEOM-DRUGS allows us to explore the design space of DiTMC under a more realistic setting, where training contstrained by fixed compute budget [67]. Our budget of 9 GPU days allows training DiTMC+aPE-L and DiTMC++PE-L for 50 epochs, and DiTMC+PE(3)-L for 10 epochs. For consistency, train the base models for the same number of epochs as their large counterparts. All DiTMC variants achieve state-of-the of--hh--ttt--tttt performance on GEOM-DRUGS for COV-P and AMR-P (see Tab. 3) Importantly, even the smaller DiTMC+aPE-B and DiTMC+PE(3)-B models outperform ET-Flow-SS of similar model size across all metrics, underlining the effectiveness of our approach.
Coverage vs. RMSD Threshold. We further analyse the COV-R Mean and COV-P Mean function of RMSD threshold 5 for DiTMC+aPE-B and DiTMC+aPE-L (see ( 4A). For small thresholds (8 < 0.44) both DiTMC+aPE models outperform all other methods for coverage recall and precision. For larger thresholds (p > 0. 0.4A) MCF-L starts to outperform aPE models in terms COV-R Mean. For COV-P Mean, DiTMC+aPE-B and DiTMC+aPE-L perform better than all other methods for all reasonably small thresholds (8 < 1.2. 1.2Á). In particular for the most relevant regime where thresholds are small, we see a strong benefit due to model scaling. We find similar results for DiTMC++PE. and DiTMC+PE(3) (see Appendix Fig. A9 and Fig. A10).

---

A
Coverage Recell (1)
Coverage Precision (1)
B
MGF-S MCF-L
aPEL
MCF-M MM
100
AMR PM Mean (A] AMFUR Moaan jkp
0.6
AMR Recall (-)
80
CCV Mear [S]
0.5
60-
0.4
MCF-8
MCF-$
0.9
MCF
MCF M
0.8
AMR Precision
ETFlow
0.7
Threshole
2PE-8
Thrushold
aPE-B
0.6
Tao.
0.8
1.00
1.25
1.50
0.25
16
10"
10
10"
Threshold (A)
Thresheld
[A]
Time [4]
C
MCF
ET-Flow
DITMC (ours)
Reference
Figure 4: (A) Coverage (COV) mean as function of Root Mean Square Deviation (RMSD) threshold 8 and (B) average minimum RMSD (AMR) mean vs. time per conformer for DiTMC+aPE and other ofate-of-the-ar models. Per model markers from left to right correspond to 5, 10, 20, and 50 sampler steps following Refs. [32 47. Note, that the original MCF paper reports results with two different samplers. Benchmark results (Tab. 3) are obtained with DDPM sampler (1000 steps) and AMR vs. time results are reported for DDIM sampler (5-50 steps). (C) Comparison conformers generated by MCF, ET-Flow, and DITMC against ground-truth reference conformers from GEOM-XI.. Generated conformers are rotationally aligned with their corresponding reference conformer.
Pareto Front. Additionally, we investigate the Pareto front of accuracy and computational efficiency, by plotting model accuracy as a function of wall clock time per generated conformer, following (see Appendix G.5 for details). As measure for accuracy we consider the average minimum RMSD (AMR), since it is independent of the RMSD threshold. We report results of DiTMC+aPE-B and DiTMC+aPE-L for AMR-R Mean and AMR-P Mean in Fig. 4B. For AMR-P Mean, both DiTMC+aPE models shift the whole Pareto front, yielding higher accuracy at lower computational cost. Even higher accuracies (at the cost of compute time) can be obtained by scaling DiTMC+aPE-B to DiTMC+aPEELL For recall, both DiTMC+aPE models shift the Pareto front for little compute times, but most accurate results at increased cost are obtained by MCF-L. Similar results are obtained for DiTMC+1PE (see Appendix Fig. A12), but benefits for DiTMC+PE(3) are limited due to high computational cost of equivariant operations (see Appendix Fig. A13).
Refs. 32 47
Ensemble Properties. To complement the RMSD-based geometric evaluation, we report the median absolute error (MAE) of ensemble properties between generated and reference conformers, following the protocol of MCF 47 and ET-Flow (32 (see Appendix G.6 for details). Our models predict ensemble properties more accurately than all baselines, highlighting the physical validity of our gen- erated structures (see Tab. 4). In particular, MCF, which shows better performance for recall metrics, outperformed by large margin (up to a factor of four for energy). The strong performance the aPE-B and aPF-L models underlines the potential of achieving high physical validity without any geometric priors, which have been hypothesized to one the reasons for ET-Flow outperforming MCF in the ensemble property task (32]
Generalization Performance Finally, we assess how well our models, trained on GEOM-DRUGS, generalize to larger and previously unseen molecules using the GEOM-XL dataset 46 It comprises 102 molecules with more than 100 atoms, whereas the molecules in the training set contain only 44 atoms on average. Following MCF [47] we report results for all 102 molecules and a subset of 77 molecules. ET-Flow uses a slightly different subset of 75 molecules. Our aPE models perform on par with or better than the previously best- best-performing method MCF-L (see (See 5), while using approximately times and 25 times fewer parameters for aPE-L. and aPE-B, respectively. The other baselines, such as ET-Flow, are outperformed by larger margin (see Appendix Tab. A12).

---

Table 4: Median absolute error of ensemble properties between generated and reference conformers for GEOM-DRUGS, Best results in bold, second best underlined; our models are marked with an asterisk (+). Results for MCF, ET-Flow, and ours are averaged over three random seeds. See Appendix for
Tab. for additional results including standard deviations.
Method
E [kcal/mol] + l [D]
Ac [kcal/mol] Emin [kcal/mol]
MCF-L (242M)
0.68
0.28
0.63
0.04
ET-Flow (8.3M)
0.18
0.18
0.35
0.02
+DiTMC+aPE-B (9.5M)
0.17
0.16
0.27
0.01
* +DITMC+aPE-L (28.2M)
0.16
0.14
0.27
0.01
Table 5: Out-of-distribution generalization results on GEOM-XI. for models trained on GEOM- DRUGS. -R indicates Recall, -P indicates Precision. Best results in bold, second best underlined; our models are marked with an asterisk (+). Our results are averaged over three random seeds. See Appendix Tab. A12 for additional results including standard deviations.
Method
75 77 mols
102 mols
AMR-R l
AMR-P [AI]
AMR-R [A14
AMR-P [A]
Mean
Median
Mean
Median
Mean
Median
Mean
Median
MCF-L (242M)
1.64
1.51
2.57
2.26
1.97
1.60
2.94
2.43
ET-Flow (8.3M)
2.00
1.80
2.96
2.63
2.31
1.93
3.31
2.84
* -DTTMC+aPE-B (9.5M)
1.68
1.47
2.59
2.24
1.96
1.60
2.90
2.48
-DTMCC+aPE-L (28.2M)
1.56
1.28
2.47
2.14
1.88
1.51
2.81
2.30
Summary and Limitations
We propose framework for molecular conformer generation that incorporates conditioning strategies tailored the architectural design principles of DiTs. This modular framework enables a rigorous exploration of different positional embedding and self-attention strategies, allowing to identify scalable generative architectures that perform competitively with prior methods on standard bench- marks. Our models achieve state-of-the-ee results GEOM-QM9 and GEOM-DRUGS excelling in both precision and physical validity, and show strong generalization to larger, previously unseen molecules from the GEOM-XL dataset. Exemplary generations for GEOM-XL are shown in Fig. with additional samples provided in Appendix M
Through ablation studies, we assess the impact of incorporating Euclidean symmetries into DITMC. While such symmetries improve performance, they also increase computational cost. Notably, simpler non-equivariant variants remain highly effective. These findings allow us to develop efficient, accurate, and scalable architecture suitable for large-scale conformer generation.
Nonetheless, some limitations persist. Our evaluation currently restricted to small and medium- sized molecules, with larger, more flexible compounds left for future work. Moreover, the training process depends on high-quality ground-truth conformers, which may be unavailable in some cases. An interesting direction for future work, is extending the scope of our model beyond conformer generation, for example to de-novo molecular generation 8 50 However, this would require re-design of our proposed conditioning strategies, as those are specifically tailored to molecular graphs as inputs. In terms of computational cost, DíTMC scales quadratic in the number of atoms, resulting in large computational cost for bigger systems. Therefore, integrating recently developed Euclidean fast attention mechanisms (88 poses an interesting direction for further research. Finally, while our analysis advances understanding of equivariance within DiT-based generative models, drawing broader conclusions would require further study.
10

---

7 Acknowledgements
JTF, WR, KRM, and SC acknowledge support by the German Ministry of Education and Research Further, this work was in part supported by the BMBF under Grants OLIS 14013A-E, 01GQ1115, 01GQ0850, 011518025A, 031L0207D, and 011S180374 KRM was partly supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grants funded by the Korea government (MSIT) (No.2019-0-00079, Artificial Intelligence Graduate School Program, Korea University and No. 2022-0-00984, Development of Artificial Intelligence Technology for Personalized Plug-and- Play Explanation and Verification of Explanation). We further want thank Romuald Elie, Michael Plainer, Adil Kabylda, Khaled Kahouli, Stefan Gugler, Martin Michajlow, Christoph Bornett, Johannes MaeB, Leon Werner and Maximilian Eißler for helpful discussion.
(BMBF) for BIFOLD

---

References
[1] Niklas Gebauer, Michael Gastegger, and Kristof Schüit. Symmetry-adapiee generation of 3d point sets for the targeted discovery of molecules. Advances in neural information processing systems, 32, 2019.
[2] Tomohide Masuda, Matthew Ragoza, and David Ryan Koes. Generating 3d molecular struc- tures conditional receptor binding site with deep generative models. arXiv preprint arXiv:2010.14442, 2020.
[3] Matthew Ragoza. Tomohide Masuda, and David Ryan Koes. Learning continuous representa- tion of 3d molecular structures with deep generative models. arXiv preprint arXiv:2010.08653 2020.
[4] Niklas WA Gebauer, Michael Gastegger, Stefaan SP Hessmann, Klaus-Robert Müller, and Kristof T Schütt. Inverse design of 3d molecular structures with conditional generative neural networks. Nature communications, 13(1):973, 2022.
[5] Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d generative model for structure-based drug design. Advances in Neural Information Processing Systems, 34:6229-6239, 2021.
[6] Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using variational autoencoders. In Artificial Neural Networks and Machine Learning-ICANN 2018: 27th International Conference on Artificial Neural Networks. Rhodes, Greece, October 4-7, 2018, Proceedings. Part 1 27, pages 412-422. Springer, 2018.
[7] Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs. arXiv preprint arXiv:1805.11973. 2018.
[8] Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning, pages 8867-8887. PMLR, 2022.
[9] Victor Garcia Satorras, Emiel Hoogeboom, Fabian Fuchs, Ingmar Posner, and Max Welling. E(n) equivariant normalizing flows. Advances in Neural Information Processing Systems, 34: 4181-415 -181-4192, 2021.
[10] Rui Jiao, Wenbing Huang, Peijia Lin, Jiaqi Han, Pin Chen, Yutong Lu, and Yang Liu. Crystal structure prediction by joint equivariant diffusion. Advances in Neural Information Processing Systems, 36:17464-17497, 2023.
[11] Giuseppe Vecchio, Renato Sortino. Simone Palazzo, and Concetto Spampinato. Matfuse: controllable material generation with diffusion models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 1429-4438, 2024.
[12] Claudio Zeni, Robert Pinsler, Daniel Zügner, Andrew Fowler, Matthew Horton, Xiang Fu, Zilong Wang, Aliaksandra Shysheya, Jonathan Crabbé, Shoko Ueda, et al. A generative model for inorganic materials design. Nature, pages 1-3, 2025.
[13] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4195 4205, October 2023.
[14] Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF conference computer vision and patlern recognition, pages 22669-22679, 2023.
[15] Shoufa Chen, Mengmeng Xu, Jiawei Ren, Yuren Cong, Sen He, Yanping Xie, Animesh Sinha, Ping Luo, Tao Xiang, and Juan-Manuel Perez-Rua. Gentron: Diffusion transformers for image and video generation. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 6441-6451, 2024.
12

---

[16] Yixin Liu, Kai Zhang. Yuan Li. Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, Lifang He, and Lichao Sun. Sora: A review on background. technology. limitations, and opportunities of large vision models. CoRR, abs/2402. 2024.
http:///oooorr/////// 10. 40w.1....888888 arXiv. 17177
[17] Xin Ma, Yaohui Wang. Xinyuan Chen, Gengyun Jia, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. Transactions on Machine Learning Research, 2025. ISSN 2835-8856.
[18] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021. URL httpps://pperrevlew..tt wtt.://www....eeee.ooooo/aaaaaaa
[19] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 10684-10695, 2022.
[20] Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10696-10706, 2022.
[21] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger. Kathryn Tunyasuvunakool, Russ Bates, Augustin Zídek. Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. nature, 59((787)))5883-5888 2021.
[22] Amira Alakhdar, Barnabas Poezos, and Newell Washburn. Diffusion models in de novo drug design. Journal of Chemical Information and Modeling, 64(119)7238-7255 2024.
[23] Tim Hsu, Babak Sadigh, Vasily Bulatov, and Fei Zhou. Score dynamics: Scaling molecular dynamics with picoseconds time steps via conditional diffusion model. Journal of Chemical Theory and Computation, 20(6):2335-2348, 2024.
[24] Jonathan Ho, Ajay Jain, and Pieter Abbecl. Denoising diffusion probabilistic models. In
H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6840-6851. Curran Associates, Inc., 2020. URL httphhtttts:/prrceedinnnee neurips. wttpccpppp... 4c5cctecs544af066777111111710613
cc/paper__. fraper_filee www.fllllapaaeee http://ppppp//////1a1/// /ttppppprr.iileepppppr/////////p
[25] Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 21696-21707. Curran Associates, Inc., 2021. URL https::/ wew.eeeeeeddiineneneeeeeeeeeeeee ww/paper.ec...c. fatperfffllepppppperpre .df
[26] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamin Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023. URL, hntttps:/oppnreviiee http::///pppreeeeeeeeeee///////a
[27] Michael S Albergo, Nicholas M Boffi, and Eric Vanden-Eijndenn Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint arXiv:2303.08797, 2023.
[28] Diederik P Kingma and Ruiqi Gao. Understanding diffusion objectives as the ELBO with simple data augmentation. In sevethh Conference on Neural Information Processing Systems, 2023. URL httpps:://oeerreviee
[29] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, & ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, ed- itors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, hwtpeer_iilee//pppe//1//////////
U. Von Luxburg,
Inc., 2017. URL http:/ttts///// htt.:ttttss/rroceedingge nww.ips....... cttccc/pppp.. wwtpcc//ppppeeeeeeeee
Paper pdf
13

---

[30] Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, 630(8016):493-5000 2024.
[31]
Tomas Geffner, Kieran Didi, Zuobai Zhang, Danny Reidenbach, Zhonglin Cao. Jason Yim, Mario Geiger, Christian Dallago, Emine Kucukbenli, Arash Vahdat, and Karsten Kreis. Pro- teina:
Scaling flow-based protein structure generative models. In The Thirteenth International Conference on Learning Representations, 2025. hattpss//openreviee wtwpeenrriiie..neforuull net/forum?
1d=TVQLu34bdw
[32] Majdi Hassan, Nikhil Shenoy, Jungyoon Lee, Hannes Stärk, Stephan Thaler, and Dominique Beaini. Et-flow: Equivariant flow-m fow-matching for molecular conformer generation. In A. Glober- son, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 128798- 128824. Curran Associates, files/ pptps//pp/p//////eeeeeee pdf
Inc., 2024. URL http:/ttts//ppoceedddiiee wwtp:/s...rrni.ccccce.eeeeoooe oat.aelter..aaaaaaaaaaaaaaaaaaa0
[33] Sarah Lewis, Tim Hempel, José Jiménez-Luna, Michael Gastegger, Yu Xie, Andrew YK Foong, Victor García Satorras, Osama Abdin, Bastiaan S Veeling, Iryna Zaporozhets, et al. Scalable emulation of protein equilibrium ensembles with generative deep learning. bioRxiv, pages 2024-12, 2024.
[34] Philipp Thölke and Gianni De Fabritiis. Equivariant transformers for neural network based molecular potentials. International Conference on Learning Representations, 2022. URL https://openrevies:
[35] J Thorben Frank, Oliver T Unke, Klaus-Robert Müller, and Stefan Chmiela. A euclidean transformer for fast and stable machine learned force fields. Nature Communications, 15(1): 6539, 2024.
[36] Yi-Lun Liao, Brandon M Wood, Abhishek Das. and Tess Smidt. Equiformerv2: Improved equivariant transformer for scaling higher-degree representations. The Twelfth International Conference on Learning Representations, 2024. URL httpps://opperreieeeeeee wwtpwiiwwwwwee.tteeeeeeee net/forum? id==mO0KZmrzDD
[37] Chaitanya K Joshi, Xiang Fu, Yi-Lun Liao, Vahe Gharakhanyan, Benjamin Kurt Miller, Anuroop Sriram, and Zachary W Ulissi. All-atom diffusion transformers: Unified generative modelling of molecules and materials. arXiv preprint 0tti::255000396653 2025.
[38] Sereina Riniker and Gregory A Landrum. Better informed distance geometry: using what we know to improve conformation generation. Journal of chemical information and modeling, 55 (12):2562-2574, 2015.
[39] Elman Mansimov, Omar Mahmood, Seokho Kang, and Kyunghyun Cho. Molecular geometry prediction using a deep generative graph neural network. Scientific reports, 911):20381, 2019.
[40] Gregor NC Simm and José Miguel Hermández-Lobato. A generative model for molccular distance geometry. arXiv preprint arXiv: /909. /459, 2019.
[41] Minkai Xu, Wujie Wang. Shitong Luo, Chence Shi, Yoshua Bengio, Rafael Bombarlli and Jian Tang. An end-to-end framework for molecular conformation generation via bilevel programming. In International conference machine learning, pages 11537-11547. PMLR, 2021.
[42] Shitong Luo, Chence Shi, Minkai Xu, and Jian Tang. Predicting molecular conformation via dynamic graph score matching. Advances in neural information processing systems, 34: 19784-19795. 2021.
[43] Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular conformation generation. In International conference machine learning, pages 9568..56. PMLR, 2021.
[44] Minkai Xu, Shitong Luo, Yoshua Bengio, Jian Peng, and Jian Tang. Learning neural generative dynamics for molecular conformation generation. arXiv preprint 102200.002.10240, 2021.
14

---

[45] Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geo- metric diffusion model for molecular conformation generation. arXiv preprint wrXiv2200.002223, 2022.
[46] Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional diffusion for molecular conformer generation. Advances in neural information processing systems, 35:24240-2427 2022.
[47] Yuyang Wang, Ahmed A Elhag, Navdeep Jaitly, Joshua M Susskind, and Miguel Angel Bautista. Swallowing the bitter pill: Simplified scalable conformer generation. arXiv preprint arXiv:2311. 17932, 2023.
[48] Zhiyuan Liu, Yanchen Luo, Han Huang, Enzhi Zhang, Sihang Li, Junfeng Fang, Yaorui Shi, Xiang Wang, Kenji Kawaguchi, and Tat-Seng Chua. NExt-mol: 3d diffusion meets 1d language modeling for 3d molecule generation. In The Thirteenth International Conference Learning Representations, 2025. URL httpps://oppereeeiieeee wtt//foruun.dddp60000..
[49] Lei Huang, Hengtong Zhang, Tingyang Xu, and Ka-Chun Wong. Mdm: Molecular diffusion model for 3d molecule generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 5105-5112, 2023.
[50] Minkai Xu, Alexander $ Powers, Ron O Dror, Stefano Ermon, and Jure Leskovec. Geometric latent diffusion models for 3d molecule generation. In International Conference Machine Learning, pages 38592-38610. PMLR, 2023.
[51] Gengmo Zhou, Zhifeng Gao, Qiankun Ding. Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. In The Eleventh International Conference on Learning Representations, 2023. URL https //openreview.o Cooperreieerr.ettfouriiiiiiieeee weet/foorum22d-KK2R666/066
[52] Xiaohong Ji, Zhen Wang, Zhifeng Gao, Hang Zheng, Linfeng Zhang, Guolin Ke, and Weinan E. Exploring molecular pretraining model at scale. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS '24, Red Hook, NY, USA, 2025. Curran Associates Inc. ISBN 9798331314385.
[53] Michael Samuel Albergo and Eric Vanden- Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations, 2023. URL httpp:ts////peerriiiii wwtp:eeent/tiiiae http://www...eetorooummaaaoaooo.
[54] Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023. URL htttps://openrevieew.t htt.:/ennnntttooommmdddd//ii.a1
[55] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021. URL http:ttt/////orrrrrii
het//oormm/dd5PXXIGGPRRR
[56] Alexander Tong, Kilian FATRAS, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Browd Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based genera- tive models with minibatch optimal transport. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL htttps://openreviee.. ott.:nneeoooooo/odd000000 Expert Certification.
[57] Jason Yim, Andrew Campbell, Andrew YK Foong, Michael Gastegger, José Jiménez-Luna, Sarah Lewis, Victor Garcia Satorras, Bastiaan Veeling, Regina Barzilay, Tommi Jaakkola, et al. Fast protein backbone generation with (3) flow matching. arXiv preprint arX0v:2310.05297, 2023.
[58] Tobias Pfaff, Meire Fortunato, Alvaro Sanzhez- and Peter Battaglia. Learning mesh- based simulation with graph networks. In International conference on learning representations, 2020.
15

---

[59] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie- Yan Liu. Do transformers really perform badly for graph representation? In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 28877-28888. Curran Associates, Inc., 2021. URL http/htttts//ppreeeediaa neurips. cappscc/pp wc/paper... files/ pptpp/ppppppp//////es/ pdf
[60] Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. International Conference on Machine Learning, pages 9377-9388. PMLR, 2021.
[61] Simon Batzner, Albert Musaclian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky, E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun., 13(1):2453, 2022.
[62] Simon Axelrod and Rafael KomezzBombareele Geom, energy-annotated molecular conforma- tions for property prediction and molecular generation. Scientific Data, 9(1):185, 2022.
[63] Philipp Pracht, Stefan Grimme, Christoph Bannwarth, Fabian Bohle, Sebastian Ehlert, Gereon Feldmann, Johannes Gorges, Marcel Müller, Tim Neudecker, Christoph Plett, et al. Crest-a program for the exploration of low-energy molecular chemical space. The Journal of Chemical Physics, 160(11), 1), 2024.
[64] Octavian Ganea, Lagnajit Pattanaik, Connor Coley, Regina Barzilay, Klavs Jensen, William Green, and Tommi Jaakkola. Geomol: Torsional geometric generation of molecular con- former ensembles. Advances in Neural Information Processing Systems, 34:13757-13769, 2021.
[65] Hannes Stark, Bowen Jing, Regina Barzilay, and Tommi Jaakkola. Harmonic prior self- conditioned flow matching for multi-ligand docking and binding site design. In NeurlPS 2023 Al for Science Workshop, 2023. URL httphttt/ppeereeeeeee
[66] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08300 2020.
[67] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbi, Aidan Clark, al. Training compute-optimet large language models. arXiv preprint 2rxXi:2203.155555 2022.
[68] J Thorben Frank, Stefan Chmiela, -oburt Müller, and Oliver T Unke. Euclidean fast attention: Machine learning global atomic representations at linear cost. arXiv preprint arXiv:2412.08541 2024.
[69] Jason Yim, Brian L. Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola. SE(3) diffusion model with application protein backbone generation. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedingy of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 40001- 40039. PMLR, 2023.
[70] Jonas Köhler, Leon Klein, and Frank Noc. Equivariant flows: Exact likelihood generative learning for symmetric densities. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 5361-5370. PMLR, 13-18 Jul 2020.
[71] Wolfgang Kabsch, A solution for the best rotation to relate two sets of vectors. Foundations of Crystallography, 32(5):922-9233 1976.
[72] Oliver Unke and Hartmut Maennel. E3x: equiivaiia deep learning made easy. CoRR,
abs/2401.07595, 2024. URL https: h ///oii org
orgg10 100 48550/550000 /arivv 2401. 07555
16

---

[73] Greg Landrum et al. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling. Greg Landrum, 8331.1)):521, 2013.
[74] Lagnajit Pattanaik, Octavian- Eugen Ganea, Ian Coley, Klavs Jensen, William H Green, and Connor W Coley. Message passing networks for molecules with tetrahedral chirality, arXiv preprint arXiv:2012.00094, 2020.
[75] Christoph Bannwarth, Sebastian Ehlert, and Stefan Grimme. Gfn2-xtb an accurate and broadly parametrized self-consistent fight-binding quantum chemical method with multipole electrostatics and density-dependent dispersion contributions. Journal of Chemical Theory doi: 10. 1a..02/acs.jct..BB0 176. URL https
and Computation, 15(3):1652-167) 2019.
1021/ acs. http://.....//a...cc...c.oooo 8b01.86
PMID: 30741547.
17

---

1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: [NA]
Guidelines:
18

---

The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: [NA]
19

---

Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
muuldes/CodeSubmisse
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: for more details.
/ //nips. wwtpcccppppcccccccccc wttp://ccggiide/eeeeeeeeeeeeeeee wwwoodeeumisssooooooioooo
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if' applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
20

---

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: While can be expected that general field of A14Science will have societal impact, our work is focused on the development of new deep learning architecture in this field. To have a direct societal impact, further studies under realistic applications, i.. wet-lab experiments would be required.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
21

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee whw.ree.wthhhe....c.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
22

---

Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
23

---

Justification: [NA]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy htteas////eerrpppoccc///o/rreees for what should or should not be described.
24

---

A From SE(3) to SO(3) Invariance
The larget data distribution of molecular conformers (æ) is SE(3) -in3a-invariant, i.c. does not change under translations and rotations of the input. Following Ref. [69], one can define SE(3)-invariant measure on SE(3)~ N by keeping the center of mass fixed at zero, which can be achieved via the centering operation from Eq. A15 This defines a subgroup SE(3)) called centered SE(3). It can then be shown, that one can define an SE(3) (533)-ivvariant measure on SE(3)) by constructing an SO(3) iinvrriant (rotationally invariant) measure on SE(3),
As a consequence, it is then sufficient to learn an (qqu)varuaa vector field on the space of centered input positions (see also Ref. (70). This is achieved by centering 2o, 21 and for the calculation of the interpolant. Moreover, the neural network output (predicted velocities) and the clean target Φ must be centered have zero center of mass.
discuss the implications for training Section B and for sampling in Section
B Training
Algorithm T describes the computation of the training loss for flow matching objective. We start by sampling from the prior 2n ~ po (2), the data distribution æ1 ~ (x:),, and a Gaussian distribution c (x;; 0, I). The interpolant is then defined as
32 = (1 t) xo + t æ1 + tO.
(A13)
where a E R> is a non-zero noise scaling parameter.
Rather than directly predicting the conditional vector field u, u, [o, 11). choose to reparametrize the network such that it predicts the clean sample 21- Similar to Ref. [57), we add a weighting term 1/(1 t)2 encourage the model accurately capture fine details close the data distribution. This gives rise to the following loss function
1 (I - -
L =
-1=1 (o 21112
(A14)
where t E (0, 1) denotes the timestep in the interpolant æ; E Nx3 I1 E R Nx3 is the clean geometry, and EV, denotes the molecular graph. The full algorithm is summarized in Algorithm
Geometry Alignment Given a set of vectors 44 = [w....., un 77 E R²) associated with a point cloud Nx3 we define a centering operation for the i-th row
N N
Center(x); i.
(A15)
which removes global drift in æ. Given two point clouds æ E NN3 and an E N23 with positions
R.A (Fia ..., FNA) and RB = (Fin ... FNn), we define rotational alignment operation
RotationAlign((.. A # BB)
(A16)
where Rop E R333 is the optimal rotation matrix, minimizing the root mean square deviation (RMSD) between the positions of point clouds A and B, which can obtained via the Kabsch algorithm [71). The full geometry alignment operation "GeometryAlign(zz" TB)" is given by
ΣA Center( (
(A17)
EB +- Center(Eg)
(A18)
EA <- Alggnti
(A19)
In words, both point clouds are first centered at the origin and then optimally aligned by rotation. This procedure minimizes the path length of linear interpolation between the point clouds A and B.
25

---

Algorithm 1 Conditional Flow Matching Training Loss
Require: Graph G, target 11, noise level Ø, Model x
1: Zo ~ Po. ~ (0,, ((()))
((,,), 1), R SO(3)
2: za, 21 <- Glometry-Align(aae
This centers zo and Φ1 and rotation-aligns 20 #1
3: 6 < Center(E)
4: 2t (1 (1-t)aro + 1m, de
5: f ApplyRotatiom/ at)
6: Φ t ApplyRotation R, x
7: ④1 < x (xx, 9.
8: 2 < Center(a1)
9: return
(1-1)²
4 - 21
Data Augmentation The target data distribution of molecular conformers p1 (a) is -inva-innariaan,, iee. it does not change under translations and rotations of the input. One can construct an SE(3)- invariant density by learning an SO(3) -qqiivarra vector field on centered SE(3) (see Section A). However, only DiTMC+PE(3) is SO(3) -0)3)eequil,rian,, whereas aPE and rPE violate SO(3)- equivariance. Therefore, we learn equivariance approximately during training, using data augmenta- tion. Specifically, we randomly sample rotation matrices R (orthogonal matrices with determinant +1) and apply them as
ApplyRotation RR, a);; = R
(A20)
where F; denotes the positions of the i-th atom, i.e., the i-th row in the point cloud æ E Nx3
Noise Scaling Parameter We ablated the noise scaling parameter o GEOM-QM9 and GEOM- DRUGS, comparing a larger value of 0.5 and a smaller value of 0.05. We set a to the value that empirically worked best for each dataset: 0.05 for GEOM-QM9 and 0.5 for GEOM-DRUGS.
Optimizer and Hyperparameters We use the Adam optimizer (weight decay 0.01) with batch size of 128 and learning rate of pmax GEOM-DRUGS. First, we increase the initial learning rate of po = 10 up to jmmax via a linear learning rate warmup over the first 1% of training steps. Afterwards, the learning rate decreased via a cosine decay schedule to Mmnin O for GEOM-QM9 and Mmin 1 x 10-5 for GEOM-DRUGE
10-4 for GEOM-QM9 and μmax
10 for
Compute Budget and Training Times All models on GEOM- QMM are trained for 250 epochs, which requires 2 days of training on Nvidia H100 GPU for aPE and rPE models and almost 4 days for PE(3). For
we fix the total compute budget per model to nine days on a single NVIDIA H100 GPU, due computational constraints. Within this budget, we can train aPE-L and rPE-L for 50 epochs and PE(3)-L for epochs. To ensure consistency within each strategy, the base ("B"" model are trained for the same number of epochs as the corresponding large ("I.") model.
C Sampling
For sampling, we use a simple Euler scheme with 50 steps to sample from the associated ordinary differential equation (ODE) as described in Algorithm [ Since during training we predict the clean sample 11 we re-parametrize the velocity required for the integration
2 (Ii,,, Dt
of (we, (,
(A21)
1-1
where ( 1,0,,,,9 1,9) is the original output of DITMC.
To ensure SF(3) invariance of the probability path from an (approximately) -equivaaqiaa velocity predictor, center the prior 2o po(x) and the prediction of DiTMC in each ODE step.
26

---

Algorithm 2 ODE Sampling
Require: Model zy. Graph G, steps N
1: ln + n/N for n E {O,...,N) N}
2: 20 Pprior(2))
Sample prior.
3: 2o < Center(an) (20)
4: for to N 1 do
5:
At - ta+1 - in
> Compute step size.
6:
31 f v (2t, in 9]
7:
σ1 + Center( (t1
8:
v v (51 1t, )/ (1 tn)
9:
Lba+: f ti, + V
Euler step.
end for
return 21
Table A6: Architectural details for MLPs used in the model. The feature dimension is given as
H headd 4head where Theaa is the number heads and dhend is the number of features per head.
Name
Layers
Hidden Dim
Out Dim
Activation
Input
DiTMC Block
2
4H
H
GELU
Tokens
gated
SO(3) DiTMC Block
2
4H
H
Tokens
GELU
Time and Atom Cond.
1
SiLU
è + cf C
Bond pair
2
H
H
SiLU
Time embedding
2
H
H
SiLU
E [0.
Shortest-hop embedding
2
H
H
SiLU
Hop distance
aPE embedding
2
SiLU
Abs. positions 7
rPE embedding
2
H
H
SiLU
Rel. positions Fij
GNN embedding
2
H
SiLU
Node/edge features
D Architectural Details
In this section, we describe the architecture of DiTMC and the conditioning GNN more detail.
All DiTMC models rely conditioning tokens C, for the time c' TR per-atom of E R and per atom-pair et E IR=H See main text Section 4.1 for more details. Following Ref. (13 adaptive layer norm (AdaLN) and adaptive scale (AdaScale) to include per-atom conditioning tokens based time and molecular graph information. To that end, construct conditioning tokens
C; c¹ cc.
(A22)
Tab. A6 contains details on the MI.ss used throughout our architecture, while Tab. A7 summarizes architectural details, as well as training and inference times for all DiTMC variants. Note that the number of attention heads in DiTMC+PE(3) is adjusted to match the total parameter count of the corresponding DiTMC+aPE and DiTMC+PPE variants, while all other hyperparameters are identical.
D.1 Non-Equivariant DITMC
In the non-equivariant DITMC formulations based aPE and rPE, we have the following set of tokens
H. = (h1,.... he h; E RR), Initial token representations are obtained via h; e(2,) + p for aPE and h, e(2;) for rPE, where e(z;) e R denotes a learnable embedding based on the atomic number N4 of atom i (35.
27

---

Architectural details for different PE strategies on GEOM-QM9 and GEOM-DRUGS. Times are measured on GEOM-QM9 with batch size 128 single Nvidia H100 GPU. T means number of transformer layers, Theads number of heads, dheud number of features per head in the attention update, and TMIGN number of layers for the conditioning mesh graph net. Thus, total feature dimension is given as H
Table A7:
Pheads dhead-
Model
T
Abeads
Ahend
TMGN
H
Train [ms]
Infer [ms]
DiTMC+aPE-S (IM)
2
32
1
128
5.8
2.0
DiTMC+aPE-B (9.5M)
32
2
256
19.2
8.1
DETMC++PE-B (9.6M)
6
32
2
256
19.7
8.3
D (8.6M)
32
2
192
70.0
25.9
DiTMC+aPE-L (28.2)) (28.2M)
32
3
384
32.7
9.5
DiTMC+1PE-L (28.3M)
8
12
32
3
384
33.5
10.1
DiTMC+PEG)-L
10
32
3
320
151.6
41.8
D.1.1 Self-Attention Operation
For ease of notation, we only describe self-attention with a single head, but employ multi-head attention [29] with Whends heads in our experiments. All self-attention blocks rely on query, key and value vectors, which are obtained from the input tokens H (h1,..... hn h2 E TR } as
= W hh,
kh, = h.
(A23)
where W, 91 WR. k. W, R #x are trainable weight matrices. We define slightly modified similar- ity kernel
q
(k C ④
sim(q. k u) = =xx
(A24)
VH
where H is used to inject additional information, e.g., conditioning signals and/or positional emheddings, and '() denotes element- -iise multiplication.
For absolute and relative PEs, slightly modify standard self-attention to allow injecting waie information into the values in addition to using our modified similarity kernel
ATT P.CP Ej= sim(g;; kj, Uny) (wy U.y))
(A25)
Σj= sim(q:; kj, usy)
where queries, keys, and values are obtained with Eq. A23 and the injected pair-wise information u. depends on the positional embedding strategy with
for absolute PEs, for relative PEs.
Mij
+
PE P
(A26)
Here of E. are pair-wise graph conditioning tokens (see Eq. and p. R and pPE. R are the absolute and relative PEs described above (see Eqs. and
D.1.2 Adaptive Layer Normalization and Adaptive Scale
In the standard, non-equivariant setting, we can follow the standard approach of other DiT architec- tures. We define adaptive layer norm as
AdaLN(h, LN(h) @ ()++))+++
(A27)
where LN is standard layer normalization without trainable scale and bias, and "O" denotes entry-wise product.
Adaptive Scale is defined as
AdaScale(h, >) h ④
(A28)
28

---

In each DiTMC block, we calculate
Ol Bis Yio 0.ii B20 728 = w (SiLU(c))),
(A29)
R and the output Ori, 1e Via: Orgi, P2: Y2i E RH The weight matrix W is initialized to all zeros, such that "AdaLN" behaves like identity at initialization. "AdaScale" damps all input tokens to zero at initialization such that the whole DiTMC block behaves like the identity function at initialization.
where W
split into equally sized vectors
D.1.3 Readout
Given final tokens h after performing updates via T DiTMC blocks, we use a readout layer to predict the atomic positions the clean data sample 21. As in the DiTMC blocks, we employ adaptive LN and therefore calculate
0; 3 W (SiLU(c:)),
(A30)
with weight matrix W 2 initialized all zeros and a Sr E ER and do
h4 Ad, ai, B,),
ã; - W readour
where W readout E RaxH is trainable weight matrix. Thus, we predict three-dimensional vector per-atom.
D.2 SO(3) Equivariant DİTMC
Following the notation in Ref. 72,, we denote (0)33-equivarial tokens as = (hi,.... hy h;
[[(+1)*xH), J, where L denotes the maximal degree of the spherical harmonics. We denote the features corresponding the b-th degree h(4) E RR((+1)xH, where the (2f 1) entries corresponds the orders m = - + per degree f. We refer the reader to Ref. [72] for an in-depth introduction
into equivariant features. For the initial token representation h; (1++1) we set h! e(2)).
where e(24) I denotes a learnable embedding based on the atomic number 2 N+ for
atom 35 All higher order features h with degree are initialized with zero. For our experiments maximal degree of L ==1. 1.
D.2.1 Self-Attention Operation
Our equivariant version of self-attention uses the same transformations for queries, keys and values like the non-equivariant counterpart, as well as the modified similarity kernel (see Appendix (sub subsection D.2.1) However, to preserve all Euclidean symmetries throughout the network, every token must transform equivariantly. One way to achieve this is by separating out the rotational degrees of freedom, encoding them with irreducible representations of the rotation group SO(3). This introduces a "degree-axis" of size (L + 1)2, which encodes angular components of increasing order. The maximum degree L is chosen to ensure high fidelity reasonable computational cost. For example, setting restricts the representation to scalars and vectors, as used models like PaiNN [60] or TorchMDNet (34) An (0(3)- equivariant formulation of self-attention is then given as
so (H); Ex=
21=1 sim(g,, K,, uhy) (fei) 69 v.))
(A31)
E sym/simmg sim(g, q,, ky, mhyy)
where equivariant queries, keys and values can calculated similarly to Eq. A23 and '8' denotes a Clebsch-Gordan (CG) tensor product contraction (72). The dot-product in the similarity measure is taken along both feature and degree axes, such that the overall update preserves equivariance (see Appendix Appendix for details). Tokens and scaling vectors are calculated as
Uij = ((ri) C
P.j
(A32)
where ((ri)) ((I--))xx is radial filter, and the element-wise products with the pair-wise conditioning tokens ci g broadcast along the degree axis. Importantly, the 26 + 1
29

---

subcomponents of the radial filter for degree l' are obtained by repeating per-degree filter functions ΦT(T,)) E RI* along the degree axis to preserve equivariance (see also Eq. 10
we use equivariant MLPs for the node- wise refinement after the self-attention calculation via equivariant formulation for dense layers and so-called gated non-linearities (see e.g. Ref. 772 for more details).
Since, standard MLPs do not preserve S0(3)-equivarian)
D.2.2 Adaptive Layer Normalization and Adaptive Scale
In the (033)-cquivariann case, we define an adapted version of AdaLN and AdaScale, preserving equivariance, Our equivariant formulation of AdaLN is given
(N( o (1+ ( +
) +
for é = 0, for l > 0,
EquivAdaLN(h,, a)
EquivLN(hh (9) (1 (1+
(A33)
where EquivLN is the equivariant formulation of layer normalization following Ref. [36 without trainable per-degree scales and LN is standard layer normalization without trainable scale and bias. Scaling vectors (*) gIx are defined per degree such that input scaling vectors are tensors ( Bias vectors only defined for the invariant (e 0) component of the tokens, since adding a non-zero bias to components with > would lead to non-equivariant operation (the bias does not transform under rotations). The element wise multiplication between (1 + (9) E 1g1x H
and tokens (e) E (222-1)x is "broadcasted" along the degree-axis. For = 0, Eq. A33 reduces to the standard adaptive layer normalization.
Equivariant adaptive scale is defined
Equiv AiiiiAdaSaaeeh.1. 7) (0) @ (+)
(A34)
As for "EquivAdaLN", define separate (0) IR I per degree l, such that y R(++1)xH Again, the element wise product "broadcasted" along the aegree-axis. Since no bias is involved, the invariant and equivariant parts in h can be treated equally.
Within each 50(3) eqquivarrt DiTMC block, we calculate
Or1i Yis O.2s, B2G 72: = W (SiLU(c)))
(A35)
where Qii, 02 Tua [[++))xH and 11 E R Thus, the weight matrix is given as W 2()(++))+2)xH and initialized all zeros, such that "EquivAdalyy behaves like identity initialization and "Equiv AdaScale" returns zeros. Thus, also the (equuquaaiaaiinn DiTMC block behaves like the identity function at initialization.
D.2.3 Readout
Given final equivariant features h4 [B(++))xxx x we use readout layer to predict the atomic positions of the clean data sample 31. We employ our equivariant formulation of adaptive layer normalization and calculate
a, B (SiLU(e))),
(A36)
with weight matrix W [2H((+1)xHB initialized to all zeros and o Bg E (L++1, We then do,
h + EquivAdaLN(hh, o;, B,),
y, < W readoon (t=1)
where W readout E IR | is trainable weight vector that is applied along the feature axis in h. Since
h (f=1) 13xH this produces per-atom vectors y, E IR3 As h; are rotationally equivariant so is 9,
R33
D.3
Conditioning GNN
Our conditioning GNN directly operates on the molecular graph 9 = (V,E) to obtain graph-based information for atom-wise conditioning. First, processes the molecular graph & to derive node
30

---

and edge input features, initially represented as one-hor vectors (a full list of features is provided in Lnf..b. A8p. These features then are projected into shared latent space using two-layer multilayer perceptrons (MLPs). Finally, we employ a GNN architecture inspired by the processor described in which refines the features via message passing.
the MeshGraphNet (MGN) framework [58
The conditioning GNN maintains and updates both node and edge representations across multiple layers. Each message passing block consists of two main steps: first, the edge representations updated based on the current edge representations and the representations of the connected nodes:
eig + felss, v3)
(A37)
where eij and v; denote the imput edge and node representations, and e are the updated edge repre- sentations. The learnable function fe is implemented two-layer MLP. Next, node representations v; are updated to v using aggregated messages from neighboring edges:
a/ + fo
Us.
Σ d
(A38)
where fo is also two-layer MLP, and the summation over all edges ending node
The final output of the described conditioning GNN is set of node embeddings per atom, and a set of edge embeddings per bond. We use the final node embeddings atom-wise graph conditioning tokens in DiTMC as detailed in [Sec. 4.1] We want to highlight that one can also use the final edge embedding pair-wise graph conditioning tokens in DITMC. We discuss this further Appendix I.2
E
Equivariance Proof for Euclidean Positional Embeddings
Given a set of transformations that act on a vector space A as Sg A + A to which we associate an abstract group G, function f + said be equivariant w.r.t. Gf
[(S,r)) (x),
(A39)
where T, B > BB is an equivalent transformation on the output space. Thus, in order to say that f is equivariant, must hold that under transformation of the input, the output transforms "in the same way".
Let us now recall our definition for the equivariant positional embeddings for single degree L
PEB3, (F:)) = (1FF.1))))) O Ye(f))),
Ps
(A40)
where Φ; RR - RIXH is radial filter function, f F/r, and Y, (((++1xx1 spherical harmonics of degree = = L. The element-wise multiplication '0' between radial filters and spherical harmonics understood to be "broadcasting" along axes with size such that (Øe C Y.) R(2(+1)xH We have also made the dependence of PE(3) the pairwise displacement vector Fi = - explicit.
Lets not consider single feature channel in PE(3), which is given as
P Pije
(F)
(Tty) = t ((1) @ Ye(f)).
(A41)
Rotating the input positions in Eq. A41 leads
Pije
(E)
(Rrr)
O Yi(R)))
(A42)
() O (6) (fa), ((0 (R)
(A43)
PBB) (F)))((F)))
(A44)
where D E
R (2 are the Wigner-D matrices for degree and R 3x33 is rotation
trix. According to Eq. A39 and Eq. A44 each channel transforms equivariant and thus Pij
(E) (Ft))
also equivariant.
31

---

The concatenation of different degrees up to some maximal degree L as given in the main body of the text
Pú
PEL
(Fi))
(1(r)) の Ye(fi))
(A45)
t=
transforms under rotation as
Púj
PRL)
R D(R)
PRO (Fa))
(A46)
with (R) D' (((0R)) IR [ being block-diagonal matrix with the Wigner-D D matrices degree D (R) (2(((1))(2++11) along the diagonal. Therefore, according Eq. A39 the proposed positional embeddings Pij are -quiivaaqi.
PE(3)
Invariance Proof for the Dot-Product
In the self-attention update, the dot-product between query and key computed as along the degree and the feature axis. Under rotation, the equivariant features behave as
h (RF)) D(R)h(F).
(A47)
where ( is the concatenation of Wigner-D matrices from above. The inner product along the degree axis for features h and g behaves under rotation
g g(R)) h(RF) =(F) D(R) D(R) = g() h(F),
(A48)
=ld
where made use of the fact that the Wigner-D matrices orthogonal matrices. Thus, the dot-product along the degree-axis is invariant and therefore taking the dot-product along the degree and then along the feature axis is also invariant.
G Implementation details
G.1 Data Preprocessing
For both GEOM-QM9 and GEOM-DRUGS, use the first 30 conformers for each molecule with the lowest energies, i.e., highest Boltzmann weights. We use the train/test/val split from Geomol [64]. using the same 1000 molecules for testing.
G.2 Input Featurization
Tab. A8 defines the features for each atom bond. Each feature computed using RDKit and one-hot encoded before being passed to the network.
G.3 Evaluation Metrics
During evaluation, we follow the same procedure as described in Refs. [32
46 47
64). The root-
mean-squar deviation (RMSD) metric measures the average distance between atoms generated conformer with respect to its reference, while taking into account all possible symmetries. For L= 2K let (Cr) .L and {C$)ke(L...... .y be the sets of generated conformers and reference conformers respectively. The average minimum RMSD (AMR) and coverage (COV) metrics for both recall (R) and precision (P) are defined as follows, where >> 0 is the coverage threshold:
32

---

Table A8: Atomic and bond features included in DiT-MC. All features are one-hot encoded.
Atom features
Options
Chirality
TETRAHEDRAL_CH, TETRAHEDRAL_CCH, UNSPECIFIED, OTHER
Number of hydrogens
0, I, 3,
Number of radical electrons
0, 1, 2,4
Atom type (QM9)
N,
Atom type (DRUGS)
Mg. AI, S, CI, K, Ca, Cr, Mn, Cu, Zn, Ga, Ge, As, Se, Br, Ag, In, Sb, I, Gd, Pt, Au, Hg, Bi
B,, C,
Aromaticity
true, false
Degree
1, 3, 4, other
Hybridization
sp, 8p" sp sp sp other
Implicit valence
0. 1, 4, other
Formal charge
-5, -4, 5, other
Presence in ring of size
x 4, 5, 6, other
Number of rings atom is in
0, I, 2, 3, other
Bond features
Options
Bond type
single, double, triple, aromatic
C,V- = 1
K k (I, Refn...,,,MSS(C,,C Ck) < <
(A49)
COV- E (1, (I,..., D} Rkee RMSCC C&) <6)
(A50)
AMR-R(C, Ō): == K ke kE.!....
1
Σ
min IE(I......
RMSD(ĆI, C+)
(A51)
AMR-P(C,,) = L
Σ
RMSD(Ć;, Cs)
(A52)
AE kE(I....K)
IE(I....,.)
G.4 Chirality Correction
Given the four 3D coordinates around chirality center denoted P1, P2. Ps; P4 R with Pi = (Zs, Mi, 2)) for i = 1,2, 4, we can compute the oriented volume OV of the tetrahedron via
1
1
OV 11.11.2,PP3 P3, P4)
1
det
32
33
34
6
1/1
13
(A53)
22
23
21_
1
(p1
-P4) ((P2
-P4) X (Ps -P4)).
6
Following GeomMol [64] and ET-Flow 32 we can then compare the orientation of the volume given by sign(OV) with the local chirality label produced by RDKit, which corresponds a certain orientation well (CW +1 and CCW -1) (74). If the orientation of the volume differs from the RDKit label, we correct the chirality of the conformer by reflecting its positions against the z-axis.
G.5 Pareto Front
For models, we generate conformers using 10. and sampling steps on single 100 GPU with a batch size of 128, following Refs. 32 47 The wall-clock time per generated sample obtained by measuring the average time per batch and dividing by the batch size. done in the original paper, adopt DDIM sampling for MCF-S, MCF-B and MCF-L.
33

---

Table A9; Results on GEOM-QM9 for different generative models (number of parameters in paren- theses). indicates Recall, indicates Precision. Best results bold, second best underlined; our models are marked with an asterisk "*". Our results are averaged over three random seeds with standard deviation reported below. Other works do not report standard deviations.
Method
COV-R [%lt
AMR-R [
COV-P 1%11
AMR-P [A]
Mean
Median
Mean
Median
Mean
Median
Mean
Median
GeoMol (0.3M)
91.5
100.0
0.225
0.193
86.7
100.0
0.270
0.241
GeoDiff (1.6M)
76.5
100.0
0.297
0.229
50.0
33.5
0.524
0.510
Tors. Diff. (1.6M)
92.8
100.0
0.178
0.147
92.7
100.0
0,221
0.195
MCF-B (64M)
95.0
100.0
0.103
0.044
93.7
100.0
0.119
0.055
DMT-B (55M)
95.2
100.0
0.090
0.036
93.8
100.0
0.108
0.049
ET-Flow (8.3M)
96.5
100.0
0.073
0.030
94.1
100.0
0.098
0.039
96.1
100.0
0,074
0.030
95.4
100.0
0.085
0.037
+DITMC+aPE-B (9.5M)
+0.3
#0.0
#00001
+0.001
#0.1
+:0.
+00001
+0.00
96.3
100.0
0.070
0.027
95.7
100.0
0.080
0.035
*DITMC++PE-B (9.6M)
+.00
=0.0
#0.001
+0.000
+=0.0
+=0.00
+0.000
95.7
100.0
0.068
0.021
93.4
100.0
0.089
0.032
-DTTMC+PE33)BB (8.6M)
+.033
+0.0
+0,002
+00001
+0.2
++0.0
+00002
0.001
G.6 Ensemble Properties
We adopt the property prediction task setup from MCF 47 and ET-Flow [32]. where we draw a subset of 100 randomly sampled molecules from the test set of GEOM-DRUGS and generate min(2K, 32) conformers for a molecule with K ground truth conformers. Afterwards we relax the conformers using GFN2-XTB [75] and compare the Boltzmann Boltznan--weighted properties of the generated and ground truth ensembles. More specifically, we employ xTB [75] to calculate the energy E, dipole moment μ., the HOMO-LUMO gap Ae and the minimum energy Emin- We repeat this procedure for three subsets each sampled with different random seed and report the averaged median absolute error and standard deviation of the different ensemble properties.
H Additional Experimental Results
H.1 Results on GEOM-QM9
We report the results on GEOM-QM9 including standard deviations for all DiTMC models in Tab. A9
H.2 Results on GEOM-DRUGS
We report the results on Tab. A10 The median absolute error of ensemble properties on GEOM-DRUGS is shown in Tab. All
including standard deviations for all DiTMC models in
Additional results for the coverage vs. RMSD threshold analysis, including results for DiTMC+PPE and DiTMC+PE(3), are provided in Fig. A8 Fig. A9 and Fig. A10
Additional results for the Pareto front analysis, including results for DiTMC++PE and DiTMC+PE(3),
provided in Fig. All Fig. A12
and Fig. A13
H.3 Results on GEOM-XL
We report the results GEOM-XL including standard deviations for all DITMC models in Tab. A12
34

---

Table A10: Results on GEOM-DRUGS for different generative models (number of parameters in parentheses). indicates Recall, -P indicates Precision. Best results in bold, second best underlined; our models are marked with an asterisk "*". Our results are averaged over three random seeds with standard deviation reported below. Other works do not report standard deviations.
Method
COV-R | f
AMR-R [A14
COV-P | 1%
AMR-P [Á]
Mean
Median
Mean
Median
Mean
Median
Mean
Median
GeoMol (0.3M)
44.6
41.4
0.875
0.834
43.0
36.4
0.928
0.841
GeoDiff (1.6M)
42.1
37.8
0.835
0.809
24.9
14.5
1.136
1.090
Tors. Diff. (1.6M)
72.7
80.0
0.582
0.565
55.2
56.9
0.778
0.729
MCF-S (13M)
79.4
87.5
0.512
0.492
57.4
57.6
0.761
0.715
MCF-B (64M)
84.0
91.5
0.427
0.402
64.0
66.2
0.667
0.605
MCF-L (242M)
84.7
92.2
0.390
0.247
66.8
71.3
0.618
0.530
DMT-L (150M)
85.8
92.3
0.375
0.346
67.9
72.5
0.598
0.527
ET-Flow SS (8.3M)
79.6
84.6
0.439
0.406
75.2
81.7
0.517
0.442
79.9
85.4
0.434
0.389
76.5
83.6
0.500
0.423
*DiTMC+aPE-B (9.5M)
10.1
+0.3
#0.002
=0.002
#0.11
10.3
+0.002
=0.004
79.3
84.6
0.444
0,400
77.2
84.6
0.492
0.414
*DiTMC+1PE-B (9.6M)
+0.1
+0.2
#0.002
#0.002
=0.1
#0.2
#0.001
#0.002
80.8
85.6
0.427
0.396
75.3
82.0
0.515
0.437
-DITMC+PE(3)-B (8.6M)
+0.1
+0.5
10.001
#0.001
#0.1
=0.2
=0.000
+0.003
79.2
84.4
0.432
0.386
77.8
85.7
0.470
0.387
+DTTMC+aPELLL (28.2M)
+-0.1
+0.2
+0.003
++0003
+.0.1
+0..5
+00000
+0.003
78.7
84.1
0.438
0.388
78.1
86.4
0.466
0.381
+-0.1
+0.4
+0.002
+00005
+.0.1
+0.33
+00.00
+00003
80.8
85.6
0.415
0.376
76.4
82.6
0.491
0.414
LDITMCC+PROOLL(31...
+-0.3
+-0.1
+00.03
++0.001
+-0.2
+0.33
+00.00
++0004
Additional Ablations
I.1 Gaussian vs. Harmonic Prior
As shown Tab. A13, using the harmonic prior improves metrics slightly our models on GEOM-QM9. Using the harmonic prior however doesn't 't seem to be crucial ingredient for the success of our method, as differences between Gaussian and Harmonic prior appear diminishing. the results in Tab. A13 verify, our method can also be used with a simple Gaussian prior effectively. For larger molecular graphs the expensive eigendecomposition of the graph Laplacian required for the Harmonic prior could therefore be avoided, which helps scaling our approach more easily.
1.2 Conditioning Strategies on GEOM-QM9
To evaluate the effectiveness of various graph conditioning strategies in DiTMC, compare the performance different conditioning methods against a baseline model without any conditioning. In addition conditioning strategies discussed in Sec. 4.1, we note that our conditioning GNN also produces edge-level representations, which can be used to define pair-wise graph conditioning tokens
GNNedge (V,E)
V(i,j) E
bond-pair:
(A54)
v(i,) & E.
These tokens only capture interactions between bonded atoms, i.e., when (i.j) E Conditioning tokens for non-bonded pairs set learnable vector e Self-attention still operates all atom pairs (i.j), even they are not connected by a chemical bond.
Specifically, we ablate the following conditioning strategies:
35

---

Table A11:
Median absolute error of ensemble properties between generated and reference conformers. Best results in bold, second best underlined; our models are marked with an asterisk "*". Results for MCF, ET-Flow, and ours are averaged over three random seeds with standard deviation reported below. Other works do not report standard deviations.
Method
E [kcal/mol] p [D]
Ae [kcal/mol] 4 Emin [kcal/mol]
GeoDiff (1.6M)
0.31
0.35
0.89
0.39
GeoMol (0.3M)
0.42
0.34
0.59
0.40
Torsional Diff. (1.6M)
0.22
0.35
0.54
0.13
0.68
0.28
0.63
0.04
MCF-L (242M)
-0.06
+0.05
+0.05
00.0
0.18
0.18
0.35
0.02
ET-Flow (8.3M)
+0.01
+0.01
+0.06
+0.00
0.17
0.16
0.27
0.01
*DiTMC+aPE-B (9.5M)
+0.00
+0.01
+0.01
+0.00
0.16
0.14
0.27
0.01
+0.02
+0.03
+0001
0.00
0.16
0.16
0,29
0.02
+DiTMC+PPE-B (9.6M)
10.03
+0.03
#0.02
1..0
0.16
0.15
0.28
0.01
-DITMCHPPE-L(283.)
+0.01
+0.02
+0.06
0..0
0.18
0.18
0.27
0.02
+DITMC+PE(3)-B (8.6M)
+0.01
+0.01
+0.03
+.000
0.17
0.14
0.25
0.01
-DITMC+PEO-LGLL.MM
10.01
10.01
#0.01
10.00
node only conditioning using only atom-wise graph conditioning tokens cg Fq.. 6
node & bond-pair conditioning using both atom-wise graph conditioning tokens ci (Eq. 6 and pair- wise graph conditioning tokens c derived from edge-level representations of the conditioning GNN as discussed above (Eq.A54) A54).
node & all-pair conditioning using both atom-wise graph conditioning tokens c, [Bq. 6 and pair-wise graph conditioning tokens et based on geodesic graph distances (Eq. 7).
As reported in Tab. A14, all our proposed conditioning strategies significantly reduce the average minimum RMSD (AMR) for both recall (AMR-R) and precision (AMR-P) using DiTMC+aPE-B, compared to the unconditioned baseline.
Notably, the "node all-pair" strategy achieves the best overall performance, with the lowest AMR values. These results highlight the strength of the all-pair conditioning strategy, which leverages graph geodesics to incorporate information from all atom pairs, rather than restricting conditioning to directly connected nodes or bonded pairs. This comprehensive approach captures more global structural information, thereby improving both precision and recall. See Appendix for a more in-depth analysis.
I.3 Index Positional Encoding (iPE)
Tab. A15 compares a variant including index positional encoding (iPE) from classic transformer architectures with DiTMC+aPE-B on GEOM-QM9. Specifically, we use the node index and encode via sinusoidal encodings into the tokens H before the first DiTMC block, similar to embedding the absolute positions via aPE. Since the molecular graphs are generated from SMILES strings via
36

---

Table A12:
Out-of-distribution generalization results on GEOM-XL for models trained on GEOM- DRUGS. -R indicates Recall, -P indicates Precision. Best results in bold, second best underlined; our models are marked with an asterisk "*". Our results are averaged over three random seeds with standard deviation reported below. Other works do not report standard deviations.
Method
75/77 mols
102 mols
AMR-R [A]
AMR-P
[A]
AMR-R [A14
AMR-P [Å]]
Mcan
Median
Mean
Median
Mean
Median
Mean
Median
Tor. Diff. (1.6M)
1.93
1.86
2.84
2.71
2.05
1.86
2.94
2.78
MCF-S (13M)
2.02
1.87
2.90
2.69
2.22
1.97
3.17
2.81
MCF-B (64M)
1.71
1.61
2.69
2.44
2.01
1.70
3.03
2.64
MCF-L (242M)
1.64
1.51
2.57
2.26
1.97
1.60
2.94
2.43
ET-Flow (8.3M)
2.00
1.80
2.96
2.63
2.31
1.93
3.31
2.84
1.68
1.47
2.59
2.24
1.96
1.60
2.90
2.48
*DiTMC+aPE-B (9.5M)
+0.00
=0.02
#0.00
#0.01
#0.00
10.03
#0.00
1..0
1.56
1.28
2.47
2.14
1.88
1.51
2.81
2.30
+DTTMC+aPE-L (28.22)
+0.01
+0.01
+0.00
++0.1
+0.01
+0.02
+0.00
+0002
1.69
1.41
2.52
2.11
1.97
1.61
2.86
2.33
+DTTMC+PE-B (9.6M)
#0.01
=0.33
1..00
=0.00
#0.01
#0.01
#0.00
10.01
1.66
1.37
2.47
2.18
1.96
1.61
2.82
2.42
*DiTMC++PE-L (28.3M)
+0.03
=0.01
#0,00
#0.02
#0.02
#0.02
+0.00
#0.02
1.73
1.55
2.71
2.35
1.98
1.67
3.03
2.60
=DITMC+PE(3)-B (8.6M)
#0.01
=0.01
#0.00
#0.01
#0.01
#0.02
+0.00
#0.01
1.57
1.46
2.60
2.27
1.85
1.58
2.93
2.53
+DITMC+PE@))L (31.M))
+-0.1
-0.01
+0.00
+-002
+00.2
+.00
+0.00
+0.03
Table A13: Ablation of PE strategies and Gaussian (G) and Harmonic (H) prior on GEOM-QM9. We report mean coverage (COV) at a threshold of 0.5Á, and mean average minimum RMSD (AMR)
for Recall -R and Precision -P. Best results in bold. All results averaged over three random seeds.
COV-R [%] 11
AMR-R IÅLL
COV-P [%]t
AMR-P [A]
Method
G
G
G
H
DiTMC+aPE-B
96.2
96.1
0.074
0.073
95.2
95.4
0.087
0.085
DET 96.0
96.3
0.073
0.070
95.2
95.7
0.084
0.080
DETMC+PE(3)-B
95.7
95.7
0.069
0.068
93.5
93.4
0.090
0.089
RDKit and RDKit has to some extend a canonical ordering, this information can used by the transformer architecture. However, index positional encoding breaks permutation equivariance (as we show in Tab. A15). This might be undesirable as permutation equivariance is one of the fundamental symmetries when learning graphs. Since the ordering of atoms in SMILES string is not uniquely defined, the trained network depends on the used framework for parsing the SMILES string or even a particular software version. We use RDKit (version 2024.9.5) for parsing SMILES strings to graphs.
Nevertheless, our DiTMC+aPE-B using iPE can effectively exploit the information contained in atom indices assigned by RDKit. A version of DiTMC+aPE-B without atom-pair conditioning but iPE achieves comparable performance to DiTMC+aPE-B using geodesic distances as atom-pair conditioning (pairwise conditioning). As our pairwise conditioning strategy is similarly or more effective than iPE additionally preserves permutation equivariance, should be preferred over iPE and we don't t use iPE in any of our other experiments.
37

---

Table A14: Ablation of conditioning strategies using DiTMC+aPE-B on GEOM-QM9. -R indicates Recall, -P indicates Precision. Best results in bold. Our results are averaged over three random seeds with standard deviation reported below.
Method
COV-R %
AMR-R [A14
COV-P [%lt
AMR-P [Å]
Mean
Median
Mean
Median
Mean
Median
Mean
Median
DiTMC+aPE-B
68.6
91.7
0.405
0.325
36.8
36.4
0.729
0.703
(no conditioning)
+1.0
#2.1
+0.005
+0.004
+0.5
+2.2
+0.006
10.007
DiTMC+aPE-B (node only)
96.3
100.0
0.079
0.037
93.2
100.0
0.112
0.051
+0.00
=0.0
+0.000
10.000
+0.2
=0.0
+0.001
#0.000
DiTMC+aPE-B
96.5
100.0
0.077
0.035
95.3
100.0
0.092
0.046
(node & bond-pair)
10..1
=0.0
+.0001
10.001
+0.2
=0.0
+0.001
#0.002
DiTMC+aPE-B (node & all-pair)
96.1
100.0
0.074
0.030
95.4
100.0
0.085
0.037
+0.33
=0.0
+.0001
0.001
=0.1
+0.0
+0.001
10.000
Table A15: Ablating index positional encoding (iPE) on GEOM-QM9 for different conditioning strategies (in brackets). To show the effect of atom permutations, we include results with randomly permuted atom indices (perm.). -R indicates Recall, -P indicates Precision. Best results in bold. Our results are averaged over three random seeds with standard deviation reported below.
Method
COV-R 1% 1
AMR-R [A]
COV-P [%] 1
AMR-P [Å]
Mean
Median
Mean
Median
Mean
Median
Mean
Median
DiTMC+aPE-B (node only)
96.3
100.0
0.079
0.037
93.2
100.0
0.112
0.051
+0.0
#0.0
+0.00
+0.00
+0.2
#0.0
+0.001
+0.000
DiTMC+aPE-B (node only), perm.
96.3
100.0
0.079
0.037
93.2
100.0
0.112
0.051
+0..0
#0.0
+0.00
+00000
#0.2
+0.0
+00001
+0.000
DiTMC+aPE+iPE-B (node only)
96.6
100.0
0.079
0.037
95.5
100.0
0.093
0.046
#00.4
#0.0
+0.002
#0.001
#.0.1
#0.0
+00.01
#0.001
DiTMC+aPE+iPE-B (node only), perm.
82.3
100.0
0.229
0.108
60.0
61.8
0.493
0.416
+1.1
+0.0
+0.008
+0.005
+0.9
#1.4
+00008
+0.011
DiTMC+aPE-B (node & pairwise)
96.1
100.0
0.074
0.030
95.4
100.0
0.085
0.037
#0.3
=0.0
10.001
#0.00
#0.1
#0.0
#0,001
#0.00
DiTMC+aPE-B
96.1
100.0
0.074
0.030
95.4
100.0
0.085
0.037
(node & pairwise), perm.
#0.3
=0.0
10.001
#0.001
#0.1
#0.0
#0.001
#0.00
J Analysis of training loss as a function of latent time
In this section, we provide details for the analysis Fig. 2C in the main part of the paper. We investigate the effect of the positional embeddings and self-attention formulations the accuracy of the model. We therefore take pre-trained models on GEOM-QM9 and compute the training loss (as detailed in Algorithm [ averaged over 1000 samples drawn randomly from the GEOM-QM9 validation set. We compute the loss for 30 logarithmically spaced values of t; = 1 10 where I; [-1.8, 0] with uniform spacing. We skip the stochastic term in the loss is done while sampling from the ODE.
As detailed in Fig. A5 we observe empirically that equivariance leads to a decreased loss close to the data distribution after training. This explains why our equivariant model more often succeeds to produce samples with increased fidelity, as depicted in figure Fig. 2B.
38

---

Loss(t), relative PE(3)
Loss(t), absolute values
harmonic prior
aPE, harmonic prior
rPE, harmonic prior
rPE, harmonic prior
1.8
PE(3), harmonic prior
0.5
PE(3), harmonic prior
0.4
loss
loss
1.4
0.3
0.2
1.0
0.1
0.0
0.8
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.4
0.6
0.8
Figure A5: Loss function of time comparing different PE strategies. Results averaged 1000 samples randomly drawn from the GEOM-QM9 validation set. Left: loss relative to PE(3) as baseline. In the important regime close the data distribution, the model PE(3) has lower loss, yielding higher sample fidelity. Right: absolute loss values for all PEs. The loss decreases close to the data distribution for all models.
We further note, that absolute loss values for models trained with all our PE strategies decrease as latent time increases (see (see A5). This is expected, conditional vector fields for each data sample will start to interact more strongly moving away from the data distribution. Our weighted loss (see Appendix B) effectively penalizes errors close the data distribution during training and helps with keeping the error low in this important regime.
K SMILES Classification Experiment via Conditioning GNN
The molecular graphs in dataset represented SMILES strings. A critical requirement of DiTMC is the ability to distinguish distinct molecular graphs or SMILES representations through conditioning. In this section, we investigate whether our conditioning GNN is capable of learning the necessary information to distinguish between different SMILES strings for the generation of matching molecular conformers.
As a proxy evaluation task, we assess whether the conditioning network alone can function as a classifier of SMILES strings. To this end, we construct two training datasets: toy dataset comprising three specific SMILES strings of a hydroxyl group moving along carbon chain (C) cccCcCcc, (D) CCCOCCC, CCC (0) CCCCCC), and larger set consisting of 1000 randomly sampled SMILES strings drawn from the GEOM-QM9 validation set. Each SMILES string becomes a seperate class, for each class there is exactly example in the training data. The classification task performed on the graph representations of the molecules, employing the same feature set and GNN architecture utilized in conditioning GNN (see Appendix D.3 and Appendix G.2 followed by simple classification head.
We train different models with batch size of 3 for 5000 epochs the curated toy dataset and batch size over 250 epochs on the GEOM-QM9 subset. We report classification accuracy on the training sets directly to evaluate the model's discriminative capacity. Furthermore, we explore whether conditioning weights obtained from an end-to-end trained model retain discriminatory power by freezing them and attaching linear classification head.
Our results, as shown Tab. A16, reveal that a simple linear classifier lacking message-passing capabilities fails to distinguish certain SMILES strings. Overall, our results indicate that a simple two-layer GNN effectively captures the necessary conditioning information through end-to-end training.
Fig. A6 A shows that without GNN layers, isomers will be misclassified.
39

---

Table A16: We measure the discriminative power of our conditioning graph network on training set of 1000 randomly sampled SMILES strings from the GEOM-QME CMM validation set, as well as a toy dataset of 3 different SMILES strings. We investigate the required number of message passing layers, as well using pre- mreitraieed weights from an end-to-end trained model.
GNN layers
Weight init
Trainable
Accuracy (GEOM-QM9) Accuracy (Toy Data)
random random trainable
trainable
0.887
0.333
1
0.999
0.666
random random pre-trained
trainable
1.000
1.000
2
frozen
0.980
1.000
2
frozen
1.000
1.000
HO
Predicted (idx 908)
(idx 911)
OH
OH
Predicted (idx 542)
True (idx 520)
HO
HO
OH
NH
Predicted (idx 931)
True (idx 932)
Figure A6: Misclassified exampled for SMILES classification experiment. We randomly pick examples, which are misclassified by classification head without any GNN layers. We show that GNN layers essential for correct classification of isomers.
L Analysis of Sampling Trajectories
Fig. A7 compares the generative performance of DÍTMC models trained on the GEOM-QM9 dataset
under two different conditioning strategies: (1) node-only conditioning and (2) node plus pairwise conditioning, where use the all-pair conditioning based geodesic graph distances. Each row in the figure corresponds to one example molecule selected from the test set. The molecules are chosen to maximize the root mean squared deviation (RMSD) between the final generated structures of the
40

---

prior
node only
node only
node and pair
node pair
bond
push
sample
bond
s0th
sample
pond
prior
node only
node only
node and pair
node and pair
otont
sample
bond
Smmpl
prior
node only
node only
nade pair
pair
-1
pash
sample
band
Zech
samplee
bond
prior
node only
node only
node and pair
node and pair
Ful Ozp
21_1
X
aaatnt
bond
path
ssmple
bond
89th
stmple
bond
prior
node
node
node and
node
-1
1
0
11
11
path
sample
Figure A7: Comparison of molecular generation with node-only versus node- and wise (all-pair) conditioning on GEOM-QM9, Each row shows prior sample, sampling trajectories, and final generated structure for both models; pairwise conditioning preserves bonds from the conditioning graph g.
two models. This selection highlights cases where the differences between the conditioning schemes are most pronounced.
Within each row, we display a sequence of images: the initial prior sample, followed by the interme- diate trajectory of the ODE sampling process over 50 sampling steps for the node-only conditioned model, and the resulting final structure (with predicted bonds rendered in yellow). This sequence is repeated for the node- and wise- conditioned model, allowing side-by-side visual comparison of the generation dynamics and final outputs.
41

---

The results reveal a consistent pattern: models trained with node-only conditioning fail to preserve bonding patterns from the conditioning graph G. This manifests as bond stretching or atom permuta- tion in the final structure. In contrast, the model, that is conditioned on pairwise geodesic distances, produces geometries that adhere more closely expected chemical structure and the given bonds. We note that if atoms are simply permuted by the model using only node conditioning, the generated structure might still be valid in terms the combination of generated 3D positions and atom types. The degraded performance of node conditioning versus node and pair-conditioning therefore in part be explained by the used RMSD and Coverage metrics, which are not invariant to permutations of atoms.
Our findings still underscore the importance of incorporating both node-level and pairwise features in molecular generative models, in particular when agreement with the given conditioning on bond graph essential.
M
Visualization
Fig. A14 Fig. A15 and Fig. Al a visual comparison of conformers generated by MCF, ET- Flow, and DiTMC against the corresponding ground-truth reference conformers for the GEOM-QM9, GEOM-DRUGS and GEOM-XL datasets, respectively. For each dataset, we randomly select six reference conformers from the test split and generate conformers using each method. Finally, we apply rotation alignment of the generated conformers with their corresponding reference conformer.
N
Code and Data Availability
The code and data to reproduce the main results of this paper, can be downloaded from here: https:// doi 5tt///000022//ooooo 15489212, via github: httpss//ggtaaabbcco/////obb////
mc
42

---

DITMC-aPE
Coverage Recall Mean vs. Threshold (1)
Coverage Recall Median Threshold (1)
80
COV-R Meen [%)
COV-R Median (%6
60
40
Threshold (A)
Threshold (A)
Coverage Precision Mean Threshold (t)
Coverage Precision Median Threshold (1)
80
COV-P Mean (%)
GOV-P Median 1%6]
60
40
0.25
0.50
0.75
1.00
1.25
1.50
0.25
0.50
0.75
1.00
1.25
1.50
(A)
(A)
MCF-S
MCF-B
MCF-L
ET-flow
DITMC+aPC-B
DITMC+aPE-L
Figure A8: Coverage (COV) for precision ("-P") and recall ("-R") function of RMSD threshold 5 for DiTMC+aPE and other state-of-the-ar methods on GEOM-DRUGS. The vertical dashed line denotes the RMSD threshold 0.75 commonly used for evaluation the GEOM-DRUGS dataset.
43

---

DITMC-rPE
Coverage Recall Mean vs. Threshold (f)
Coverage Recall Median vs. Threshold (1)
100
80
COVRR Mean [%6
COV-R Median
60
20
Threshold [A]
Threshald [A]
Coverage Precision Mean vs. Threshold (1)
Coverage Precision Median vs. Threshold (1)
100
80
CDV-P Mean [%]
COW- Median 188)
60
0.25
0.50
0.75
1.00
1.25
1.50
0.25
0.50
1.00
1.25
1.50
Threshold (A)
Threshold (A)
MCFS
MCF-8
MCF-L
ET-Flow
DITMC+IPE-B
DITMC+IPEL
Figure A9: Coverage (COV) for precision ("-P") and recall ("-R") function of RMSD threshold 5 for DiTMC++PE and other state-of-the-ar methods on GEOM-DRUGS, The vertical dashed line denotes the RMSD threshold 0.75 commonly used for evaluation the GEOM-DRUGS dataset.
44

---

DITMC-PE(3)
Coverage Recall Mean vs. Threshold (f)
Coverage Recall Median vs. Threshald (1)
100
80
COVAR Mean [%]
COV-R Median
60
Threshold (A)
Threshold (A)
Coverage Precision Mean vs. Threshold (1)
Coverage Precision Median vs. Threshold (t)
100
80
COV-P Mean [%)
COV-P Median 196]
60
0.25
0.50
0.75
1.00
1.25
1.50
0.25
0.50
0.75
1.00
1.25
1.50
Threshold (A)
Threshatd (A)
MCF-S
MCF-B
MCF-L
ET-Flow
DITMC+PE13)-
DITMC+
Figure A10: Coverage (COV) for precision ("-P") and recall ("-R") function of RMSD threshold 5 for DiTMC+PE(3) and other state-of-the-ar methods on GEOM-DRUGS, The vertical dashed line denotes the RMSD threshold 0.75 commonly used for evaluation the GEOM-DRUGS dataset.
45

---

DiTMC-aPE
Coverage Recall Mean vs. Time (1)
Coverage Precision Mean vs. Time (1)
75
70
80
COV-P Mean
Cov-R
60
75
55
45
10
10
10
1
10
10
10
10a2
Time [s]
Time [5]
Average Minimum RMSD Recall Mean vs. Time (4)
Average Minimum RMSD Precision Mean vs. Time (4)
0.65
0.90
0.85
0.60
0.80
AMR-3 (A)
0.75
AMR Mean
0,70
0.50
0.65
0.45
0.50
0.40
0.45
10
10-2
10
1
10a4
10"1
10
10
DiTMC+aPEB
DITMC+aPE-L
Figure A11: Average minimum RMSD (AMR) for precision ("-P") and recall ("-R") as a function of wall clock time per generated conformer. For each model, markers from left to right correspond to an increasing number of sampling steps during generation. Here, we follow Refs. [32 47 and use 5, 10, 20, and 50 sampler steps.
46

---

DiTMC-rPE
Coverage Recall Mean vs. Time (1)
Coverage Precision Mean vs. Time (?)
75
80
COI-P Mean [86
COVR
60
10
10
10
10"3
10-1
10-1
10
10-1
Time
Average Minimum RMSD Recall Mean Time
Average Minimum RMSD Precision Mean Time
0.65
0.90
0.85
0.50
AMR R Mean (A]
AMR P Mean (A]
0.55
0.70
0.65
0.60
0.55
10
1
10
10
10
10
10
10
Time
Time
MCF-L
DITMC+IPE
DITMCHIPELL
Figure A12: Average minimum RMSD (AMR) for precision ("-P") and recall ("-R") as a function of wall clock time per generated conformer. For each model, markers from left right correspond an increasing number of sampling steps during generation. Here, we follow Refs, [32 47 and use 5, 10, 20, and 50 sampler steps.
47

---

DiTMC-PE(3)
Coverage Recall Mean Time (1)
Coverage Precision Mean vs. Time (1)
85
80
75
70
COV-R (S]
80
CD4-P Mean [S]
65
55
50
45
10
10"3
10
10a1
10
10"3
10
10a1
Time [s]
Time [5]
Average Minimum RMSD Recall Mean Time (4)
Average Minimum RMSD Precision Mean Time
0.65
0.90
0.85
0.60
AMR Mean [A]
0.75
0.55
0.70
0.50
0.65
0.60
0.45
0.35
0.50
0.40
0.45
Time [s]
Time [6]
MCF-S
MCF-B
MCF-L
ET-Flow
DITMC+PEC)-8
Figure A13: Average minimum RMSD (AMR) for precision ("-P") and recall ("-R") as a function of wall clock time per generated conformer. For each model, markers from left to right correspond to an
increasing number of sampling steps during generation. Here, we follow Refs. [32 47 and use 5, 10, 20, and 50 sampler steps.
48

---

MCF
ET-Flow
DiTMC (ours)
Reference
Figure A14: Comparison conformers generated by MCF, ET-Flow, and DITMC against ground- truth reference conformers from GEOM-QM9. The generated conformers rotationally aligned with their corresponding reference conformer to facilitate comparison. From left to right: generated conformers from MCF, ET-Flow, DiTMC, and the corresponding ground-truth reference conformers.
49

---

MCF
ET-Flow
DiTMC (ours)
Reference
Figure A15: Comparison conformers generated by MCF, ET-Flow, and DiTMC against ground- truth reference conformers from GEOM-DRUGS. The generated conformers rotationally aligned with their corresponding reference conformer to facilitate comparison. From left to right: generated conformers from MCF, ET-Flow, DiTMC, and the corresponding ground-truth reference conformers.
50

---

MCF
ET-Flow
DiTMC (ours)
Reference
Figure A16: Comparison conformers generated by MCF, ET-Flow, and DITMC against ground- truth reference conformers from GEOM-XI. The generated conformers rotationally aligned with their corresponding reference conformer to facilitate comparison. From left to right: generated conformers from MCF, ET-Flow, DiTMC, and the corresponding ground-truth reference conformers.
51


Reviews:
[{"review_id": "uuw4i4Gd3n", "reviewer": "Reviewer_9sGw", "summary": "This paper aims to adapt the diffusion transformer, which has shown promise in general domains, to the task of molecular conformer prediction. To this end, several modifications are proposed within a new framework called DiTMC, including two types of molecular graph conditioning strategies. Moreover, the paper provides a detailed and informative discussion on the use of geometric priors (equivariant attention) versus standard self-attention for molecular conformation prediction. By combining conditioning strategies, attention mechanisms, and positional embedding, the proposed DiTMC has demonstrated effectiveness on conformation prediction tasks.", "questions": "1. Could the authors provide a more complete study over the scaling of the proposed approach by considering more model size? \n\n2. Could the authors provide a comparison between the base model and ET-Flow - SS baseline (8.3M)  over Geom-Drugs?", "limitations": "yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n1. Studying the inductive bias for molecular data is a very important topic. The paper aims to investigate the challenges of applying a widely adopted and proven framework, DiT, to molecular data, which should be encouraged.\n2. The paper provides a detailed comparison between equivariant and non-equivariant approaches. I appreciate the authors' effort to keep the methods simple and effective. I believe this discussion could inspire further exploration in model architecture design.\n3. Overall, the paper is clearly written, making the key elements easy to understand.\n\n**Weaknesses**\n\n1. The paper could be further improved by making the study more systematic. The scope of experiments and tasks is somewhat limited. As a new framework, many of the proposed elements—such as conditioning strategies, positional embedding, and attention—are not necessarily restricted to the task of conformation prediction. As the focus is on model architecture, I believe a more comprehensive evaluation would be beneficial, for example, on 3D molecule generation or molecular property prediction tasks.\n2. A key concern lies in the experimental results. While DiTMC-L, as claimed, achieves competitive performance with ~30M parameters (I highly recommend mentioning the parameter size in the main text instead of the appendix for clearer comparison), there is not a significant performance boost over the ET-Flow - SS baseline (8.3M) on Geom-Drugs. This raises questions about the effectiveness of the proposed approaches.\n3. The scalability of the proposed architecture is not clear. According to Fig. 3, increasing the number of parameters by four times does not yield the expected performance improvement, which may raise concerns about the model’s scalability.\n4. The technical novelty is limited. While I appreciate the efforts to adapt DiT to molecular graphs, from a methodological perspective, the contribution is relatively minor. This sets a high bar for assessing the completeness and effectiveness/performance of the method.", "was_revised": true, "final_justification": "empirically solid paper with slight concerns over novelty", "rebuttal": "We thank the reviewer for their careful reading and thorough review of our manuscript. Below, we respond in detail to each of the reviewers’ comments.\n\n> A key concern lies in the experimental results. While DiTMC-L, as claimed, achieves competitive performance with ~30M parameters (...), there is not a significant performance boost over the ET-Flow - SS baseline (8.3M) on Geom-Drugs. This raises questions about the effectiveness of the proposed approaches.”\n\n> 2. Could the authors provide a comparison between the base model and ET-Flow - SS baseline (8.3M) over Geom-Drugs?\n\nWe agree that all parameter counts should be clearly stated in the main document and will update the manuscript accordingly. However, we would like to clarify, that **we achieve better performance than ET-Flow-SS even with our base variants (e.g. DiTMC+aPE-B, 9.5M params)** on GEOM-DRUGS (see Table R1 based on Table A10 in the original manuscript) and also in out-of-distribution performance on GEOM-XL (see Table R2 based on Table A13 in the original manuscript), indicating that our improvements are not due to larger model size. **In fact, we observe an improvement of DiTMC+aPE-B over ET-Flow-SS for every single metric we investigate. Our relative improvements range up to 4.3% on GEOM-DRUGS and up to 17.1% on GEOM-XL.** \n\nFurthermore **our base models improve upon ET-Flow-SS in terms of sampling speed vs. accuracy.** Specifically, when using the same number of 50 ODE steps during sampling, DiTMC+aPE-B is almost 5 times faster than ET-Flow-SS and surpasses ET-Flow-SS in all metrics on GEOM-DRUGS (see Table R1).\n\n*Table R1: Comparison of DiTMC+aPE-B to ET-Flow-SS on GEOM-DRUGS. -R indicates Recall, -P indicates Precision. Number of parameters in brackets. Final row shows relative improvement of DiTMC+aPE-B over ET-Flow-SS. A positive percentage indicates an improvement by DiTMC+aPE-B over the ET-Flow-SS baseline.*\n\n|Method|COV-R Mean [%] ↑|COV-R Median [%] ↑|AMR-R Mean [Å] ↓|AMR-R Median [Å] ↓|COV-P Mean [%] ↑|COV-P Median [%] ↑|AMR-P Mean [Å] ↓|AMR-P Median [Å] ↓| Time per sample [ms] ↓|\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|ET-Flow-SS (8.3M)|79.6|84.6|0.439|0.406|75.2|81.7|0.517|0.442|62.5|\n|DiTMC+aPE-B (9.5M)|**79.9**|**85.4**|**0.434**|**0.389**|**76.5**|**83.6**|**0.500**|**0.423**|**12.7**|\n|Rel. Improvement (%)|+0.38|+0.95|+1.14|+4.19|+1.73|+2.33|+3.29|+4.30|+79.7|\n\n*Table R2: Out-of-distribution generalization results on GEOM-XL. The ET-Flow paper does not report ET-Flow-SS on GEOM-XL, such that we stick to ET-Flow. Final row shows relative improvement of DiTMC+aPE-B over the ET-Flow baseline.*\n\n|Method|AMR-R Mean [Å] ↓|AMR-R Median [Å] ↓|AMR-P Mean [Å] ↓|AMR-P Median [Å] ↓|# mols|\n|:---|:---:|:---:|:---:|:---:|:---:|\n|ET-Flow (8.3M)|2.31|1.93|3.31|2.84|102|\n|DiTMC+aPE-B (9.5M)|**1.96**|**1.60**|**2.90**|**2.48**|102|\n|Rel. Improvement (%)|+15.15|+17.10|+12.39|+12.68|-|\n\nMoreover, DiTMC+aPE-B also outperforms ET-Flow-SS for smaller acceptance thresholds by a significant margin (see Table R3), a regime that is particularly relevant for many real-world applications, as it reflects the model’s ability to generate highly accurate conformations. Notably, ET-Flow-SS has been long-regarded as the best-in-class model for small acceptance thresholds. We believe that surpassing it represents a significant advance.\n\n*Table R3: Comparison of coverage (COV) metrics for DiTMC+aPE-B to ET-Flow-SS with a RMSD threshold of 0.10Å on GEOM-DRUGS. Final row shows relative improvement of DiTMC+aPE-B over the ET-Flow-SS baseline.*\n\n|Method|COV-R Mean [%] ↑|COV-P Mean [%] ↑|\n|--|:--:|:--:|\n|ET-Flow-SS (8.3M)|17.03|13.19|\n|DiTMC+aPE-B (9.5M)|**27.16**|**24.72**|\n|Rel. Improvement (%)|+59.51|+87.42|\n\nOverall, these experiments clearly demonstrate a significant performance improvement compared to ET-Flow-SS even for our base model of equal size. We will update and extend our final manuscript accordingly. Further, we will add information about the number of parameters of the large models in the main text for better comparison. We would be grateful if these points could be taken into account in the final assessment of our work.\n\n> The scalability of the proposed architecture is not clear. According to Fig. 3, increasing the number of parameters by four times does not yield the expected performance improvement, which may raise concerns about the model’s scalability.\n\n> 1. Could the authors provide a more complete study over the scaling of the proposed approach by considering more model size?\n\n**Scaling our model from base to the large variant yields relative improvement in precision metrics between 3.5% (COV Mean) and 12.4% (AMR Median) (see Table A10 in the submitted manuscript). We obtain a relative improvement in terms of generalization to larger molecules by 3.1% to 7.3%**, i.e. our large model exhibits significantly stronger out-of-distribution performance compared to the base model (see Table R4). \n\n*Table R4: Out-of-distribution generalization results on GEOM-XL when scaling our model. Final row shows relative improvement of aPE-L over aPE-B.*\n\n|Method|AMR-R Mean [Å] ↓|AMR-R Median [Å] ↓|AMR-P Mean [Å] ↓|AMR-P Median [Å] ↓|# mols|\n|:---|:---:|:---:|:---:|:---:|:---:|\n|DiTMC+aPE-B (9.5M)|1.96|1.60|2.90|2.48|102|\n|DiTMC+aPE-L (28.2M)|**1.88**|**1.51**|**2.81**|**2.30**|102|\n|Rel. Improvement (%)|+4.1%|+5.6%|+3.1%|+7.3%|-|\n\nScaling the base (“B”) to the large model (“L”) improves performance, but less than might be expected. Ref. [1] shows that to avoid diminishing returns, dataset and model sizes must be scaled together. Scaling only the model yields limited benefits.\n\nTo investigate this effect, we have trained an additional smaller (1M params) variant of our aPE model (“S”) to compare its performance to our aPE-B model (9.5M params). **We obtain strong relative improvements up to 54.1% for coverage precision with aPE-B (see Table R5). This clearly shows that scaling is effective until model size saturates given the data.** Our base model appears close to saturation and more data is needed to fully leverage the larger model’s potential. We hope this highlights the strength of our scaling results and resolves the reviewer’s Q1.\n\n*Table R5. Benefit of scaling for DiTMC+aPE on GEOM-DRUGS. Final row shows relative improvement of aPE-B over aPE-S.*\n\n|Method|COV-R Mean [%] ↑|AMR-R Mean [Å] ↓|COV-P Mean [%] ↑|AMR-P Mean [Å] ↓| \n|:---|:---:|:---:|:---:|:---:|\n|DiTMC+aPE-S (1M)|68.9|0.630|49.6|0.828|\n|DiTMC+aPE-B (9.5M)|**79.9**|**0.434**|**76.5**|**0.500**| \n|Rel. Improvement (%)|+16.0|+31.1|+54.1|+39.6|\n\n[1] Kaplan, Jared, et al. \"Scaling laws for neural language models.\" arXiv preprint arXiv:2001.08361 (2020).\n\n> The paper could be further improved by making the study more systematic. “… I believe a more comprehensive evaluation would be beneficial, for example, on 3D molecule generation or molecular property prediction tasks”\n\n\nWe agree that assessing the broader applicability of our architecture is important. Our primary goal was to introduce and analyze a novel framework for conformer prediction, a task with distinct structural challenges. To ensure a systematic study, we ablated key components, including conditioning strategies, positional embeddings, attention variants, and prior distributions, within this domain. We also examined scaling behavior on in- and out-of-distribution tasks using diverse metrics, including ensemble property prediction. We chose this scope to provide controlled and interpretable insights. We hope this clarification helps convey the systematic nature of our study and addresses the raised concerns.\n\n\n> The technical novelty is limited. While I appreciate the efforts to adapt DiT to molecular graphs, from a methodological perspective, the contribution is relatively minor.\n\nWhile our approach builds upon a model architecture originally developed for the perceptual domain of computer vision, the successful adaptation to the domain of physically accurate molecular conformer generation required non-obvious and technically meaningful modifications, far from a straightforward transfer. Specifically, we solve the following, domain-specific challenges:\n- We introduce a novel **GNN-based conditioning mechanism** to incorporate molecular graph information, which we show is crucial for conformer generation (Appendix H). Our approach departs from the global or text-based conditioning typical of image-based DiTs by leveraging both **node-wise and pair-wise conditioning strategies** (applied to atoms, bonds, or all atom pairs). We find and highlight significant differences in effectiveness among these strategies in our ablation experiments (Appendix Table A8).\n- As part of this framework, we propose using **geodesic graph distances as conditioning information**. We are not aware of any other approach that uses geodesics (shortest paths) on the graph to condition the self-attention mechanism operating on the molecular geometry in 3D space. We have added a dedicated ablation (Table R6) to validate the impact of this component, showing that it is critical to the competitive performance of DiTMC.\n- Finally, we investigate the role of **incorporating physical symmetries** (translations and rotations) into the model. We discuss the trade-offs between fidelity and computational cost when dealing with the 3D nature of molecular data in accordance with geometric principles.\n\nOur ablation studies (Appendix E-H, K) and Table R6 demonstrate that removing any of the proposed modifications to the original DiT architecture leads to a substantial performance drop.\n\n*Table R6. Ablating the effects of our proposed conditioning strategy via graph geodesics on GEOM-DRUGS.*\n\n|Method|COV-R Mean [%] ↑|AMR-R Mean [Å] ↓|COV-P Mean [%] ↑|AMR-P Mean [Å] ↓| \n|:---|:---:|:---:|:---:|:---:|\n|DiTMC+aPE-B (no graph geodesics)|72.8|0.555|55.6|0.762|\n|DiTMC+aPE-B (9.5M)|**79.9**|**0.434**|**76.5**|**0.500**| \n|Rel. Improvement (%)|+9.8%|+21.8%|+37.6%|+34.4%|"}, {"review_id": "vkO2yBcrji", "reviewer": "Reviewer_akGf", "summary": "This paper proposes a diffusion transformer (DiT) learning system for generating ensembles of 3D conformers for small molecules. Algorithmic components are introduced to embed a full suite of molecular features within transformer architectures, including atomic and pairwise graph embeddings. Additionally, multiple 3D positional embedding schemes are tested, including absolute, relative, and equivariant-pairwise, which are incorporated into self-attention-based updates to the molecular structure along diffusion trajectories. Altogether, the method introduces graph-conditioned molecular structure optimization under a DiT framework. Results are presented for low-energy conformer generation in two standard geometry datasets and are compared with a handful of SOTA methods from the literature.", "questions": "1. Was a pairwise-$\\textit{distance}$ positional embedding considered? \n2. I must have missed somewhere, what is used as the prior ($t=0$) for each sample, i.e., where is the initial conformer obtained from?\n3. How are ensembles obtained? I.e., what does the rollout of parallel trajectories entail?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Generating ensembles with high realism is significant \n2. The formulation of the transformer architecture is elegant and extensively described (Figure 1 and Sec 4), w/ conditioning signals from full-graph embedding\n3. Including all-pair representations using the graph geodesic is nice, and seeing that this conditioning performs best is significant\n4. It is satisfying that many positional embedding schemes were tested\n5. Results for ensemble properties (Table 3) are compelling, though in E.3 it is described that this is after conformer relaxation with GFN2-xTB (this may be standard, but is still a limitation for practical use)\n\nWeaknesses:\n1. Motivation: I'm not sure that the “encod[ing of] molecular connectivity and incorporat[ing] Euclidean symmetries, such as translational and rotational invariance/equivariance” is such a major challenge as posed, though, again, the formulation of the architectural components is extensive. Equivariant transformer architectures, e.g., Equiformer have existed for several years specifically for molecular generation, which incorporate many of the components necessary. Equivariant diffusion mechanisms for 3D molecule generation, e.g., EDM, GeoLDM, GeoBFN and the like have also been well demonstrated for conformer generation, though not necessarily conditioned on graph structure as proposed or with transformers (that I know of), which is nice and, yes, challenging. Equivariant, graph-conditioned conformer generation architectures do exist, e.g., MoleCLUEs, though again not with DiTs.\n2. The formulation modeling a velocity, i.e., vector field looks very similar to neural force fields, which is not super novel, though again the utilization of DiTs for this task is interesting\n3. More details in the main text on how ensembles are generated, e.g., via rollout of different trajectories, would be appreciated, as would discussion or experimentation on the effects of noise scheduling in the diffusion process, as this is expected to play a significant role in model fidelity", "was_revised": true, "final_justification": "The authors' clarifications increased this reviewer's confidence in the work's acceptance, and I have increased my confidence rating accordingly.", "rebuttal": "We would like to thank the reviewer for taking the time to carefully read our submitted manuscript and the overall positive assessment of our work. In the following, we comment on the points and questions raised by the reviewer.\n\n> Results for ensemble properties (Table 3) are compelling, though in E.3 it is described that this is after conformer relaxation with GFN2-xTB (this may be standard, but is still a limitation for practical use).\n\nWe fully agree that this can pose a limitation in practical use cases, i.e. when the system size starts to introduce computational overhead due to the necessity of applying quantum mechanical methods. Within our experiment, we follow this procedure for consistency with prior work. To acknowledge this practical limitation we will add it to the “Discussion & Limitations” section of our final manuscript.\n\n\n> Motivation: I'm not sure that the “encod[ing of] molecular connectivity and incorporat[ing] Euclidean symmetries, such as translational and rotational invariance/equivariance” is such a major challenge as posed, though, again, the formulation of the architectural components is extensive. Equivariant transformer architectures, e.g., Equiformer have existed for several years specifically for molecular generation, which incorporate many of the components necessary. Equivariant diffusion mechanisms for 3D molecule generation, e.g., EDM, GeoLDM, GeoBFN and the like have also been well demonstrated for conformer generation, though not necessarily conditioned on graph structure as proposed or with transformers (that I know of), which is nice and, yes, challenging. Equivariant, graph-conditioned conformer generation architectures do exist, e.g., MoleCLUEs, though again not with DiTs.\n\n\nWe agree that the concept of equivariant transformers as well as ideas around equivariant diffusion mechanisms are already established within the community. As correctly pointed out by the reviewer, one of the technical novelties lies in the conditioning on graph structure via geodesics, enabling the clean separation of graph structure and the operation in Euclidean space.\n\n\nWe will clarify and change our formulation in the abstract and also add a section in the related work part, which relates our approach to equivariant transformer and diffusion models. This hopefully enables the reader to better locate our approach in the overall landscape of equivariant transformer architectures and diffusion models. \n\n> The formulation modeling a velocity, i.e., vector field looks very similar to neural force fields, which is not super novel, though again the utilization of DiTs for this task is interesting.\n\nIndeed, the output of our model (a 3-dimensional velocity vector per atom) resembles the structure of neural force fields which output atomic forces, which are also 3-dimensional vectors. In particular, the ET-Flow model takes great inspiration from the TorchMDNet model, which is an equivariant message passing neural network originally designed for neural force fields. Notably, our DiTMC models outperform the ET-Flow model even for the same number of parameters (see Table R1 and Table R2 below). This observation might hint to the fact that although input and output in both settings are structurally similar, generative models can benefit from additional architectural design choices as the ones made in our DiT architecture.\n\n*Table R1: Comparison of DiTMC+aPE-B to ET-Flow-SS on GEOM-DRUGS. -R indicates Recall, -P indicates Precision. Number of parameters in brackets. Final row shows relative improvement of DiTMC+aPE-B over ET-Flow-SS. A positive percentage indicates an improvement by DiTMC+aPE-B over the ET-Flow-SS baseline.*\n\n|Method|COV-R Mean [%] ↑|COV-R Median [%] ↑|AMR-R Mean [Å] ↓|AMR-R Median [Å] ↓|COV-P Mean  [%] ↑|COV-P Median [%] ↑|AMR-P Mean [Å] ↓|AMR-P Median [Å] ↓|\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|ET-Flow - SS (8.3M)|79.6|84.6|0.439|0.406|75.2|81.7|0.517|0.442|\n|DiTMC+aPE-B (9.5M)|**79.9**|**85.4**|**0.434**|**0.389**|**76.5**|**83.6**|**0.500**|**0.423**|\n|Rel. Improvement (%)|+0.38|+0.95|+1.14|+4.19|+1.73|+2.33|+3.29|+4.30|\n\n*Table R2: Out-of-distribution generalization results on GEOM-XL for models trained on GEOM-DRUGS. -R indicates Recall, -P indicates Precision. Number of parameters in brackets. Final row shows relative improvement of DiTMC+aPE-B over ET-Flow. A positive percentage indicates an improvement by DiTMC+aPE-B over the ET-Flow baseline.*\n\n| Method | AMR-R Mean [Å] ↓| AMR-R Median [Å] ↓| AMR-P Mean [Å] ↓| AMR-P Median [Å] ↓ | # mols |\n| :--- | :---: | :---: | :---: | :---: | :---: |\n| ET-Flow (8.3M) | 2.31 | 1.93 | 3.31 | 2.84 | 102 |\n| DiTMC+aPE-B (9.5M) | **1.96** | **1.60** | **2.90** | **2.48** | 102 |\n| Rel. Improvement (%) | +15.15 | +17.10 | +12.39 | +12.68 | - |\n\n> … discussion or experimentation on the effects of noise scheduling in the diffusion process, as this is expected to play a significant role in model fidelity\n\nThe choice of noise schedule mainly impacts the fidelity of generated samples via discretization error of the underlying ODE. It is therefore expected that most noise schedules converge to the same or very similar results in the limit of sufficiently small time steps, as is for example demonstrated in [1]. We find that evaluating our models with more steps doesn’t significantly change the results (see Table R3 below as an example), which likely means that we already hit that limit and won’t additionally benefit from a different schedule. \n\n*Table R3: Performance on GEOM-DRUGS for DiTMC+aPE-B with 50 and 100 ODE sampling steps. -R indicates Recall, -P indicates Precision.*\n|Method|COV-R Mean [%] ↑|COV-R Median [%] ↑|AMR-R Mean [Å] ↓|AMR-R Median [Å] ↓|COV-P Mean [%] ↑|COV-P Median [%] ↑|AMR-P Mean [Å] ↓|AMR-P Median [Å] ↓|\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|DiTMC+aPE-B (9.5M, 50 steps)|79.9|85.4|0.434|0.389|76.5|83.6|0.500|0.423|\n|DiTMC+aPE-B (9.5M, 100 steps)|79.7|84.2|0.434|0.393|76.7|84.1|0.498|0.417|\n\n[1] Karras, Tero, et al. \"Elucidating the design space of diffusion-based generative models.\" Advances in neural information processing systems 35 (2022): 26565-26577.\n\n\n> Was a pairwise-distance positional embedding considered? \n\nYes, we also investigated a relative positional encoding strategy. We analyzed its performance on the QM9 dataset and found the results to be in line with the results for aPE and PE(3) (see Table 1 in the original manuscript). For the following experiments on GEOM-DRUGS we only kept the simplest aPE and the most evolved PE(3) strategies. For consistency, we additionally trained an rPE-B model on GEOM-DRUGS and report the results in Table R4 below.\n\n*Table R4: Results on GEOM-DRUGS for different DiTMC variants. -R indicates Recall, -P indicates Precision.*\n\n|Method|COV-R Mean [%] ↑|COV-R Median [%] ↑|AMR-R Mean [Å] ↓|AMR-R Median [Å] ↓|COV-P Mean [%] ↑|COV-P Median [%] ↑|AMR-P Mean [Å] ↓|AMR-P Median [Å] ↓|\n|:---|:---|:---|:---|:---|:---|:---|:---|:---|\n|DiTMC+aPE-B|79.9|85.4|0.434|0.389|76.5|83.6|0.500|0.423|\n|DiTMC+rPE-B|79.2|84.6|0.444|0.400|77.2|84.6|0.492|0.414|\n|DiTMC+PE(3)-B|80.8|85.6|0.427|0.396|75.3|82.0|0.515|0.437|\n\n> I must have missed somewhere, what is used as the prior (t=0) for each sample, i.e., where is the initial conformer obtained from? \n\nWe employ a Harmonic prior, following Ref. [2], as described in Section 5 of the experiments. To assess the impact of this choice, we also evaluate a more conventional Gaussian prior as an alternative in Appendix F.2 (see Table A15). Notably, our models perform remarkably well even when using a Gaussian prior, which is in contrast to the findings of the authors of ET-Flow. We believe that this highlights the effectiveness of our proposed conditioning strategies and the use of geodesic graph distances as conditioning information.\n\n[2] Stärk, Hannes, et al. \"Harmonic Self-Conditioned Flow Matching for Multi-Ligand Docking and Binding Site Design.\" CoRR (2023).\n> More details in the main text on how ensembles are generated, e.g., via rollout of different trajectories, would be appreciated, …\n\n> How are ensembles obtained? I.e., what does the rollout of parallel trajectories entail?\n\nWe adopt the property prediction task setup from MCF and ET-Flow, where we draw a subset of 100 randomly sampled molecules from the test set of GEOM-DRUGS and generate min(2K, 32) conformers for a molecule with K ground truth conformers. Afterwards we relax the conformers using GFN2-xTB and compare the Boltzmann-weighted properties of the generated and ground truth ensembles. More specifically, we employ xTB to calculate the energy $E$, the dipole moment $\\mu$, the HOMO-LUMO gap $\\Delta \\epsilon$ and the minimum energy $E_{\\text{min}}$. We repeat this procedure for three subsets each sampled with a different random seed and report the averaged median absolute error and standard deviation of the different ensemble properties.\n\n\nIn our DiTMC framework, we employ deterministic ODE sampling within the Flow Matching formalism. In the absence of integration errors, the integration path for each sample is therefore uniquely defined by the prior sample. However, we can still generate multiple samples for the same or even different molecules in parallel, by drawing multiple samples from the prior and combining the corresponding graphs into a larger graph with many isolated partitions (batching). \n\n\nWe will add the additional information, discussing the generation of ensembles, to the main text of our manuscript."}, {"review_id": "RHttTAfiU0", "reviewer": "Reviewer_5UNN", "summary": "The authors extend the DiT architecture to irregular molecular geometries. Key moves are (i) two graph-based conditioning schemes (bond-pair vs. all-pair geodesic tokens) and (ii) interchangeable self-attention/positional-embedding blocks that range from standard (non-equivariant) to full SO(3)-equivariant forms. On GEOM-QM9, -DRUGS and -XL, DiTMC obtain state-of-the-art precision and comparable recall results while remaining computationally tractable, especially when using simple absolute PEs. Ablations quantify the fidelity/cost trade-off of enforcing Euclidean symmetries. Limitations include slower training for equivariant variants and restricted assessment to small/medium molecules.", "questions": "1. **Data Quality Sensitivity**: The paper demonstrates strong performance on high-quality datasets like GEOM. Could the authors discuss how the model's performance might degrade when trained on lower-quality or noisy conformer data? Are there any built-in robustness mechanisms?\n2. **Computational Cost Comparison**: While the inference times for DiTMC variants are provided, how do these compare to other state-of-the-art methods (e.g., MCF, ET-Flow) in terms of FLOPs, memory, or wall-clock time? A direct comparison would help assess practical trade-offs.\n3. **Scalability to Large Systems**: The experiments focus on small to medium-sized molecules. How does DiTMC scale to very large systems (e.g., >400 atoms), such as proteins or polymers? Are there architectural or sampling adjustments needed to maintain efficiency?", "limitations": "1. **Incremental Conceptual Advance**\n    \n    The work mainly repackages existing diffusion/flow-matching ideas within a transformer framework. While the engineering is solid, it introduces no fundamentally new generative principle, limiting its conceptual impact.\n    \n2. **Heavy Reliance on High-Quality Conformer Labels**\n    \n    Training hinges on well-curated datasets such as GEOM-QM9 and GEOM-DRUGS. In domains where reference conformers are scarce, noisy, or biased, model fidelity and transferability can degrade sharply.\n    \n3. **Restricted Molecular Scope**\n    \n    All experiments target small- to medium-sized, neutral organic molecules (≤ ~100 atoms). Performance on larger, highly flexible, charged, or metal-containing species—including peptides, macrocycles, and organometallics—remains untested.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n1. **Clear, modular architecture**\n    \n    By cleanly separating coordinate processing from graph conditioning, the authors can tease apart how positional encodings and symmetry priors influence results, turning these architectural knobs into concrete design guidelines.\n    \n2. **High accuracy with compact models**\n    \n    A 9.6 M-parameter DiTMC reaches 0.080 Å AMR-P and 95.7 % COV-P on GEOM-QM9, while a 77 M-parameter aPE-L variant secures state-of-the-art precision on GEOM-DRUGS—matching or outpacing far larger baselines.\n    \n3. **Comprehensive ablation suite**\n    \n    Systematic experiments on conditioning schemes, positional-encoding types, and model capacity lend strong empirical backing to the paper’s claims.\n    \n4. **Pragmatic symmetry–cost trade-offs**\n    \n    The study shows full equivariance incurs a 3–3.5× compute overhead yet yields only modest accuracy gains beyond ~0.5 Å, arming practitioners with data to balance fidelity against resources.\n\nWeakness\n\n1. **Incremental novelty**\n    \n    The core idea—plugging flow-matching/diffusion into a transformer with graph conditioning—extends recent DiT/Flow-Matching work\n    \n2. **Dependence on Training Data Quality**:\n    \n    DITMC’s performance relies heavily on the quality and diversity of training datasets (e.g., GEOM-QM9, GEOM-DRUGS). Limited or biased datasets may lead to poor generalization, especially for rare or complex molecular structures.\n    \n3. **Generalization to Large and Complex Molecules**:\n    \n    While effective on benchmark datasets, DITMC’s performance on larger, more complex molecules (e.g., proteins or macromolecules) is not fully validated due to dataset constraints. The modular architecture may struggle with the increased structural diversity and conformational flexibility of such systems.", "was_revised": true, "final_justification": "The novelty issues remain, so I keep the borderline accept score.", "rebuttal": "We thank the reviewer for their thoughtful comments. We have carefully addressed all the points raised, and hope that the revisions resolve the primary concerns.  We hope that these improvements will be viewed positively and assist in the reviewers’ final evaluation.\n\n\n> Incremental novelty. The core idea—plugging flow-matching/diffusion into a transformer with graph conditioning—extends recent DiT/Flow-Matching work.\n\n> Incremental Conceptual Advance. The work mainly repackages existing diffusion/flow-matching ideas within a transformer framework. While the engineering is solid, it introduces no fundamentally new generative principle, limiting its conceptual impact.\n\nWe respectfully disagree with the assessment that the conceptual advance of our work is incremental. While we build upon established foundations, namely developments from the perceptual domain of computer vision, a successful adaptation to the domain of physically accurate molecular conformer generation required non-obvious and technically meaningful modifications. We believe our work introduces domain-specific conceptual advances in the following ways:\n\n- We introduce a novel **GNN-based conditioning mechanism** to incorporate molecular graph information, which we show is crucial for conformer generation (Appendix H). Our approach departs from the global or text-based conditioning typical of image-based DiTs by leveraging both **node-wise and pair-wise conditioning strategies** (applied to atoms, bonds, or all atom pairs). We find and highlight significant differences in effectiveness among these strategies in our ablation experiments (Appendix Table A8).\n- As part of this framework, we propose using **geodesic graph distances as conditioning information**. We are not aware of any other approach that uses geodesics (shortest paths) on the graph to condition the self-attention mechanism operating on the molecular geometry in 3D space. We have added a dedicated ablation (Table R1) to validate the impact of this component, showing that it is critical to the competitive performance of DiTMC.\n- Finally, we investigate the role of **incorporating physical symmetries** (translations and rotations) into the model. We discuss the trade-offs between fidelity and computational cost when dealing with the 3D nature of molecular data in accordance with geometric principles.\n\nOur ablation studies (Appendix E-H, K) and Table R1 demonstrate that removing any of the proposed modifications to the original DiT architecture leads to a substantial performance drop.\n\nCollectively, our contributions provide a comprehensive toolkit for adapting DiTs to molecular domains, with a particular focus on conformer generation, including substantive technical innovations that extend beyond naive architectural transfer. These novelties ultimately enable DiTMC to improve upon all other state-of-the-art methods in terms of precision metrics, size extrapolation, and precision-speed Pareto front. We think the strong empirical evidence should not be overlooked when judging the novelty of our model. We hope this clarification goes some way toward addressing the reviewer’s concern.\n\n\n*Table R1. Ablating the effects of our proposed conditioning strategy via graph geodesics on the GEOM-DRUGS dataset. Relative improvement indicates how much the regular model benefits from using graph geodesics.*\n\n|Method|COV-R Mean [%] ↑|AMR-R Mean [Å] ↓|COV-P Mean [%] ↑|AMR-P Mean [Å] ↓| \n|:---|:---:|:---:|:---:|:---:|\n|DiTMC+aPE-B (no graph geodesics)|72.8|0.555|55.6|0.762|\n|DiTMC+aPE-B (9.5M)|**79.9**|**0.434**|**76.5**|**0.500**| \n|Rel. Improvement (%)|+9.8|+21.8|+37.6|+34.4|\n\n\n> Computational Cost Comparison: While the inference times for DiTMC variants are provided, how do these compare to other state-of-the-art methods (e.g., MCF, ET-Flow) in terms of FLOPs, memory, or wall-clock time? A direct comparison would help assess practical trade-offs.\n\nET-Flow and MCF do report the sampling time per conformers on the GEOM-DRUGS dataset. We find that our DiTMC+aPE-L is the fastest model, requiring 26 ms per generated conformer, whereas MCF-B needs 28 ms and ET-Flow 63 ms. For a fair comparison, we compare models with identical numbers of sampling steps (50 steps). **Thus, DiTMC+aPE-L outperforms MCF and ET-Flow in terms of all precision metrics and is faster at the same time, showing that we are able to shift the speed-precision Pareto front of current approaches.**\n\nWe agree that this is an important comparison which allows us to highlight the strengths of our proposed framework. Therefore, we will add this experiment to the final version of our manuscript.\n\n> Restricted Molecular Scope. All experiments target small- to medium-sized, neutral organic molecules (≤ ~100 atoms). Performance on larger, highly flexible, charged, or metal-containing species—including peptides, macrocycles, and organometallics—remains untested.\n\nWe have applied our DiTMC model to the GEOM-DRUGS and to the GEOM-XL dataset, which contain molecules with **up to 181 atoms and 232 atoms**, respectively. Both datasets actually contain **highly flexible, charged molecules as well as molecules with macrocycles and metals** [1]. As we are able to show (Table 2 and Table 4 in the original manuscript), DiTMC yields state-of-the-art performance on GEOM-DRUGS and is the best model on GEOM-XL. We hope that this clarification properly addresses the reviewers' concerns.\n\n[1] Axelrod, Simon, and Rafael Gomez-Bombarelli. \"GEOM, energy-annotated molecular conformations for property prediction and molecular generation.\" Scientific Data 9.1 (2022): 185.\n\n> Generalization to Large and Complex Molecules: While effective on benchmark datasets, DITMC’s performance on larger, more complex molecules (e.g., proteins or macromolecules) is not fully validated due to dataset constraints. The modular architecture may struggle with the increased structural diversity and conformational flexibility of such systems.\n\nWe tested the generalizability of our GEOM-DRUGS-trained DiTMC+aPE-B model on large, flexible molecules: a 512-atom polyalanine peptide and a 502-atom PET polymer. Both yielded accurate conformations, with RMSDs of 2.6 Å and 2.9 Å respectively, against their MMFF94-relaxed structures, confirming the robustness of our model.\n\n> Dependence on Training Data Quality: DITMC’s performance relies heavily on the quality and diversity of training datasets (e.g., GEOM-QM9, GEOM-DRUGS). Limited or biased datasets may lead to poor generalization, especially for rare or complex molecular structures.\n\n> Heavy Reliance on High-Quality Conformer Labels. Training hinges on well-curated datasets such as GEOM-QM9 and GEOM-DRUGS. In domains where reference conformers are scarce, noisy, or biased, model fidelity and transferability can degrade sharply.\n\nWe agree that generative models as a whole strongly rely on the availability of sufficient and also representative training data. To that end, we believe it is a particularly nice property of our presented framework, that it fully supports going from full SO(3) equivariant models to its non-equivariant counterpart. Equivariant models, while being computationally more expensive, have been shown to drastically improve the data efficiency [2], making such models particularly well suited in the low-data regime. We will add a corresponding discussion to the “Discussion & Limitations\" section of our manuscript.\n\n[2] Batzner, Simon, et al. \"E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials.\" Nature communications 13.1 (2022): 2453.\n\n> Data Quality Sensitivity: The paper demonstrates strong performance on high-quality datasets like GEOM. Could the authors discuss how the model's performance might degrade when trained on lower-quality or noisy conformer data? Are there any built-in robustness mechanisms?\n\nWe agree that this is an important point, as the accuracy of the generated conformers is bounded by the accuracy of the training data. This problem has been successfully tackled in other domains of quantum chemistry, by using lower quality data for pre-training and fine-tuning on little but high quality reference data [3]. Adapting such approaches to molecular conformer generation, poses an interesting direction for future research.\n\n[3] Cui, Mengnan et. al. \"Multi-fidelity transfer learning for quantum chemical data using a robust density functional tight binding baseline.\" Machine Learning: Science and Technology 6.1 (2025): 015071.\n\n> Scalability to Large Systems: The experiments focus on small to medium-sized molecules. How does DiTMC scale to very large systems (e.g., >400 atoms), such as proteins or polymers? Are there architectural or sampling adjustments needed to maintain efficiency?\n\nTo test the computational performance of DiTMC+aPE-B trained on GEOM-DRUGS, we analyse the inference time as a function of the number of atoms on the example of a capped polyalanine peptide with an increasing number of repeating units. Interestingly, we did not need to adjust the number of ODE steps during sampling of larger structures to obtain accurate results, suggesting that the learned velocity field maintains stability and accuracy across different system sizes. Table R2 reports the number of atoms and the time per sample.\n\n*Table R2. Time per sample for the capped polyalanine peptide using batch size of 1.*\n|Num. atoms|212|312|412|512|\n|-------------|:--:|:--:|:--:|:--:|\n|Time [s]|0.387|0.374|0.539|0.780|"}, {"review_id": "m8vqYGA4ok", "reviewer": "Reviewer_3qPx", "summary": "This paper presents DiTMC, a novel framework for adapting the powerful Diffusion Transformer (DiT) architecture to the task of 3D molecular conformer generation. The authors identify and address key challenges in this adaptation, namely the integration of discrete molecular graph information with continuous 3D geometry and the handling of Euclidean symmetries. The core of their contribution lies in a modular DiT architecture combined with two new graph-based conditioning strategies (bond-pair and all-pair) that effectively inject atomic connectivity information into the self-attention mechanism. The framework is designed to systematically explore the impact of different symmetry priors by comparing standard non-equivariant attention mechanisms with a fully SO(3)-equivariant formulation. Experiments on the GEOM-QM9 and GEOM-DRUGS benchmarks show that DiTMC variants achieve state-of-the-art results in both precision and physical validity, providing valuable insights into the trade-off between architectural choices, computational efficiency, and sample quality.", "questions": "The paper uses two variants of model size, \"Base\" and \"Large\", demonstrating a clear trade-off: the non-equivariant aPE model scales efficiently in terms of computation and achieves state-of-the-art precision, while the fully-equivariant PE(3) model offers higher fidelity at a significant computational cost. Which direction is more promising when scaling this framework up to much larger and more complex systems? Please comment.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1/ Systematic and Rigorous Investigation: A key strength of this paper is its systematic and rigorous approach. Instead of merely presenting a single new model, the authors design a modular framework that allows them to ablate and compare different architectural choices, particularly regarding symmetry handling (aPE vs. rPE vs. PE(3)). This comparative study provides a clear and insightful analysis of the trade-offs between model accuracy, computational cost, and the explicit enforcement of physical priors.\n\n2/ Novel and Effective Conditioning: The proposed conditioning strategies are well-designed for the DiT architecture. The \"all-pair\" conditioning, which uses graph geodesic distances to inform the attention mechanism about global connectivity, is particularly smart and is shown empirically to be the most effective strategy. This represents a non-trivial and successful adaptation for incorporating graph structure into a Transformer.\n\n3/ Strong Empirical Performance: The paper demonstrates state-of-the-art results on established and challenging benchmarks.\n\nWeaknesses\n\n1/ Computational Cost of Equivariance: While the analysis of the cost-fidelity trade-off is a strength, the high computational cost of the fully equivariant model is a practical weakness. The paper reports that the PE(3) model is approximately 3-3.5x slower than its non-equivariant counterparts, even with a smaller model size. The authors give a dicussion about this in the manuscript, and it sounds reasonable.", "was_revised": true, "final_justification": "I appreciate the efforts the authors made during RB and keep my acceptance rating.", "rebuttal": "We thank the reviewer for reading and reviewing our submitted manuscript and the overall positive assessment of our work. In the following we address questions and comments of the reviewer in a point-by-point manner.\n\n> 1/ Computational Cost of Equivariance: While the analysis of the cost-fidelity trade-off is a strength, the high computational cost of the fully equivariant model is a practical weakness. The paper reports that the PE(3) model is approximately 3-3.5x slower than its non-equivariant counterparts, even with a smaller model size. The authors give a dicussion about this in the manuscript, and it sounds reasonable.\n\nAs correctly pointed out by the reviewer, equivariant models introduce additional computational cost which can make “small” models computationally costly already. For this reason, reducing the computational cost of equivariant models is an ongoing topic of active research, which can be broadly categorized into two directions: The first class of approaches reduces computational cost by new architectural designs [1, 2] whereas the second class “sparsifies” the cost-dominating SO(3)-convolutions [3, 4]. We hope that our results motivate the adaptation of such approaches in the setting of 3D molecular conformer generation and we will add a short discussion about this within the “Discussion & Limitations” section of our manuscript.\n\n[1] Cen, Jiacheng, et al. \"Are high-degree representations really unnecessary in equivariant graph neural networks?.\" Advances in Neural Information Processing Systems 37 (2024): 26238-26266.\n\n[2] Frank, J. Thorben, et al. \"A Euclidean transformer for fast and stable machine learned force fields.\" Nature Communications 15.1 (2024): 6539. \n\n[3] Luo, Shengjie, Tianlang Chen, and Aditi S. Krishnapriyan. \"Enabling efficient equivariant operations in the fourier basis via gaunt tensor products.\" arXiv preprint arXiv:2401.10216 (2024).\n\n[4] Maennel, Hartmut, Oliver T. Unke, and Klaus-Robert Müller. \"Complete and efficient covariants for three-dimensional point configurations with application to learning molecular quantum properties.\" The Journal of Physical Chemistry Letters 15.51 (2024): 12513-12519.\n\n\n> The paper uses two variants of model size, \"Base\" and \"Large\", demonstrating a clear trade-off: the non-equivariant aPE model scales efficiently in terms of computation and achieves state-of-the-art precision, while the fully-equivariant PE(3) model offers higher fidelity at a significant computational cost. Which direction is more promising when scaling this framework up to much larger and more complex systems? Please comment.\n\nIn general, our equivariant models require significantly more compute for the same number of parameters, due to the presence of equivariant representations and SO(3)-convolutions. Under limited hardware resources, scaling the PE(3) model quickly becomes a bottleneck; for example, we observe OOM on a 40GB GPU with our PE(3)-L model for a batch size of 128 on GEOM-DRUGS. On the other hand, the equivariant model converges faster during training, as it does not require learning the equivariance of the target vector field through data augmentation during training. Nevertheless, under the same compute budget, aPE outperforms PE(3) for most metrics on the larger GEOM-DRUGS benchmark. During inference, the aPE-L model is a factor of 3 faster than the PE(3)-L, but the PE(3)-model produces higher fidelity structures (Fig. 2).\n\nScaling this framework to larger and more complex systems requires significantly larger and more diverse training datasets, as well as increasing the number of model parameters. The computational cost per additional parameter grows much faster in the PE(3) model than in the aPE model, such that we expect the aPE model to be more scalable in the limit of large dataset sizes. Our findings on GEOM-DRUGS already indicate this, since we find that the aPE model outperforms the PE(3) model in most metrics when trained with the same compute budget.\n\nDuring inference, the larger the systems, the faster the aPE model becomes compared to the PE(3) model, due to the substantially smaller pre-factor in the computational complexity. Thus, the benefit of using aPE compared to PE(3) in terms of inference speed grows in system size, and we consider this to be the more promising approach for large-scale training and inference. An exception might be applications, for which only very little training data is available (e.g., data can only be obtained from experimental measurements) and high fidelity is more important than computational speed (e.g., one requires only a few generated structures, but they must be as accurate as possible)."}, {"review_id": "5lrDdPqLv5", "reviewer": "Reviewer_E3hT", "summary": "This work modifies and applies the diffusion transformers architecture to the task of molecular conformer generation, i.e., generating low-energy geometric configurations for a fixed molecular structure formula. In order to translate the DiT architecture to this task, the authors introduce a molecular graph-based conditioning scheme as well as three positional encodings that are ablated. The authors claim state-of-the-art accuracy for their resulting DiTMC model on GEOMQM9 and GEOMDRUGS.", "questions": "Why is rPE frequently omitted?", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 1, "strengths_and_weaknesses": "Strengths:\n* The authors did a great job of communicating their contributions and documenting their methods and experiments.\n* The downstream application is straightforward and a valuable one to solve. \n* The proposed DiTMC seems to perform well on the selected tasks.\n\nWeaknesses:\n* The main weakness of this work is its limited novelty. Transferring a model with minimal adjustments from one modality to another does not, in my humble opinion, clear the bar for NeurIPS. Though I reckon that this point is subjective, and acknowledge if other reviewers disagree.\n* The experiment sections are inconsistent in that while rPE in all experiments within the main body and the appendix is the best performing method, it does not occur in most tables.\n* The empirical gains compared to previous works are present but limited.\n* The related work section is rudimentary and misses the field of Boltzmann generators. \n\nThis work touches many interesting discussion points within the AI4Science community, but leaves them without clear conclusions. I highly encourage the authors to take the opportunity and reframe their work as a thorough investigation of these topics, such as equivariance, positional encoding, cutoffs, etc., by performing detailed empirical ablation studies.", "was_revised": true, "final_justification": "I find the technical novelty and empirical gains too limited for a NeurIPS publication. But, I reckon that these are subjective interpretations and do not object a publication if the other reviewers think different.", "rebuttal": "We are grateful for the reviewer's careful evaluation and constructive suggestions, which have helped us improve the manuscript. Below, we respond to each comment in detail. We hope our clarifications and corresponding revisions have resolved the key issues, and we would appreciate it if this could be reflected in the final evaluation.\n\n>The main weakness of this work is its limited novelty. Transferring a model with minimal adjustments from one modality to another does not, in my humble opinion, clear the bar for NeurIPS. Though I reckon that this point is subjective, and acknowledge if other reviewers disagree.\n\nWe thank the reviewer for acknowledging that the evaluation of novelty can be subjective, and we would like to respectfully offer a different perspective on the novelty and significance of our work. While our approach builds upon an architecture originally developed for the perceptual domain of computer vision, the successful adaptation to the domain of physically accurate molecular conformer generation required non-obvious and technically meaningful modifications, far from a straightforward transfer. Specifically, we solve the following, domain-specific challenges:\n- We introduce a novel **GNN-based conditioning mechanism** to incorporate molecular graph information, which we show is crucial for conformer generation (Appendix H). Our approach departs from the global or text-based conditioning typical of image-based DiTs by leveraging both **node-wise and pair-wise conditioning strategies** (applied to atoms, bonds, or all atom pairs). We find and highlight significant differences in effectiveness among these strategies in our ablation experiments (Appendix Table A8).\n- As part of this framework, we propose using **geodesic graph distances as conditioning information**. We are not aware of any other approach that uses geodesics (shortest paths) on the graph to condition the self-attention mechanism operating on the molecular geometry in 3D space. We have added a dedicated ablation (Table R1) of this component, validating that it is critical to the competitive performance of DiTMC.\n- We investigate the role of **incorporating physical symmetries** into the model. We discuss the trade-offs between fidelity and computational cost when dealing with the 3D nature of molecular data in accordance with geometric principles.\n\nOur ablation studies (Appendix E-H, K) and Table R1 demonstrate that removing any of the proposed architectural innovations leads to a substantial performance drop.\nCollectively, our contributions provide a comprehensive toolkit for adapting DiTs to the molecular domain, with a particular focus on conformer generation, including substantive technical innovations that extend beyond naive architectural transfer. These novelties enable DiTMC to improve upon all other state-of-the-art methods in terms of precision metrics, size extrapolation, and precision-speed Pareto front. We think the strong empirical evidence should not be overlooked when judging the novelty of our model.\n\n*Table R1. Ablating the effects of our proposed conditioning strategy via graph geodesics on GEOM-DRUGS. Relative improvement indicates how much the regular model benefits from using graph geodesics.*\n\n|Method|COV-R Mean [%] ↑|AMR-R Mean [Å] ↓|COV-P Mean [%] ↑|AMR-P Mean [Å] ↓|\n|:---|:---:|:---:|:---:|:---:|\n|DiTMC+aPE-B (no graph geodesics)|72.8|0.555|55.6|0.762|\n|DiTMC+aPE-B|**79.9**|**0.434**|**76.5**|**0.500**|\n|Rel. Improvement (%)|+9.8|+21.8|+37.6|+34.4|\n\n>The experiment sections are inconsistent in that while rPE in all experiments within the main body and the appendix is the best performing method, it does not occur in most tables.\n\n>Why is rPE frequently omitted?\n\nWe thank the reviewer for pointing out this inconsistency in the presentation of our results. As stated in the original manuscript, we initially focused on the models using aPE and PE(3) on GEOM-DRUGS, as they represent the extreme cases of not including any, and including all Euclidean symmetries in the architecture design. We agree that this focus might be misleading and added a study on the performance of rPE models on GEOM-DRUGS to address the reviewer’s concern.\n\nWe have now included DiTMC+rPE-B on GEOM-DRUGS in all relevant tables (see Tables R2, R3 and R4 based on Tables A10, A11 and A13 in the original manuscript). The training for DiTMC+rPE-L on GEOM-DRUGS is still in progress. We will be happy to share the results during the discussion phase, and we will include all results in the final version of the paper.\n\n\n*Table R2: Results on GEOM-DRUGS for different DiTMC variants. -R indicates Recall, -P indicates Precision.*\n\n|Method|COV-R Mean [%] ↑|COV-R Median [%] ↑|AMR-R Mean [Å] ↓|AMR-R Median [Å] ↓|COV-P Mean [%] ↑|COV-P Median [%] ↑|AMR-P Mean [Å] ↓|AMR-P Median [Å] ↓|\n|:---|:---|:---|:---|:---|:---|:---|:---|:---|\n|DiTMC+aPE-B|79.9|85.4|0.434|0.389|76.5|83.6|0.500|0.423|\n|DiTMC+rPE-B|79.2|84.6|0.444|0.400|77.2|84.6|0.492|0.414|\n|DiTMC+PE(3)-B|80.8|85.6|0.427|0.396|75.3|82.0|0.515|0.437|\n\n*Table R3: Median absolute error of ensemble properties between generated and reference conformers for different DiTMC variants on GEOM-DRUGS.*\n\n|Method|$E$ [kcal/mol] ↓|$\\Delta \\epsilon$ [kcal/mol] ↓|$E_{min}$ [kcal/mol] ↓|$\\mu$ [D] ↓|\n|:---|:---|:---|:---|:---|\n|DiTMC+aPE-B|0.17|0.16|0.27|0.01|\n|DiTMC+rPE-B|0.16|0.16|0.29|0.02|\n|DiTMC+PE(3)-B|0.18|0.18|0.27|0.02|\n\n*Table R4: Out-of-distribution generalization results on GEOM-XL for different DiTMC variants trained on GEOM-DRUGS. -R indicates Recall, -P indicates Precision.*\n\n|Method|AMR-R Mean [Å] ↓|AMR-R Median [Å] ↓|AMR-P Mean [Å] ↓|AMR-P Median [Å] ↓|# mols|\n|:---|:---:|:---:|:---:|:---:|:---:|\n|DiTMC+aPE-B|1.96|1.60|2.90|2.48|102|\n|DiTMC+rPE-B|1.97|1.61|2.86|2.33|102|\n|DiTMC+PE(3)-B|1.98|1.67|3.03|2.60|102|\n\n> The empirical gains compared to previous works are present but limited.\n\nWhile the absolute numerical improvements may appear modest in some cases, they are consistent across a wide range of datasets and evaluation metrics (see Tables 1-4 in the original manuscript). Notably, *no other method in our comparison achieves this level of consistency* (across precision, out-of-distribution performance and accuracy of ensemble properties), which we believe is a key strength of our approach. In real-world applications, such robustness and generalization are often even more valuable than isolated gains on individual benchmarks. We compare against strong, well-established baselines that are already highly optimized. In such a setting, even moderate improvements reflect meaningful technical progress and nontrivial methodological innovation.\n\nSpecifically, our DiTMC+aPE-L model shows a **relative improvement from 3.3% up to 12.4% in precision metrics compared to existing state-of-the-art methods** (see Table 1 in the original manuscript). \n\nIn terms of out-of-distribution performance in GEOM-XL we show **relative improvements over the current state-of-the-art model MCF-L ranging from 3.9% up to 15.2%** (see Table R5 based on Table 4 in the original manuscript).\n\nFollowing the experiments performed in ET-Flow and MCF, **DiTMC outperforms existing methods for a tighter acceptance threshold. To that end, DiTMC yields relative improvements up to 38.4% for a threshold of 0.1Å** (see Table R6). This shows that our generated conformers are significantly closer to the ground truth conformers than the ones generated by other methods.\n\nCompared to the current state-of-the-art models ET-Flow and MCF, we find that our DiTMC+aPE-L is the fastest model, requiring 26 ms per generated conformer, whereas MCF-B requires 28 ms and ET-Flow requires 63 ms on the GEOM-DRUGS dataset. For a fair comparison, we compare models with identical numbers of sampling steps (50 steps). **Thus, DiTMC+aPE-L is faster and simultaneously outperforms MCF and ET-Flow in terms of all precision metrics, showing that we can shift the speed-precision Pareto front of current approaches**.\n\nWe hope our systematic improvements over state-of-the-art methods make it clear that our contributions are meaningful and not limited in scope. \n\n*Table R5: Out-of-distribution generalization results on GEOM-XL for models trained on GEOM-DRUGS. -R indicates Recall, -P indicates Precision. We report relative improvements of DiTMC+aPE-L to the best-performing model to date as a positive percentage.*\n\n|Method|AMR-R Mean [Å] ↓|AMR-R Median [Å] ↓|AMR-P Mean [Å] ↓|AMR-P Median [Å] ↓|# mols|\n|:---|:---:|:---:|:---:|:---:|:---:|\n|ET-Flow|2.00|1.80|2.96|2.63|75|\n|MCF - L|1.64|1.51|2.57|2.26|77|\n|DiTMC+aPE-L|**1.56**|**1.28**|**2.47**|**2.14**|77|\n|Rel. Improvement (%)|+4.9|+15.2|+3.9|+5.3|---|\n\n*Table R6: Comparison of coverage (COV) metrics for DiTMC+aPE-B to ET-Flow-SS and MCF-L with a RMSD threshold of 0.10Å on GEOM-DRUGS. We report relative improvements of DiTMC+aPE-L to the best-performing model to date as a positive percentage.*\n\n|Method|COV-R Mean [%] ↑|COV-P Mean [%] ↑|\n|:---|:---:|:---:|\n|ET-Flow-SS|17.03|13.19|\n|MCF-L|31.22|22.77|\n|DiTMC+aPE-L|**32.81**|**31.51**|\n|Rel. Improvement (%)|+5.1|+38.4|\n\n> The related work section is rudimentary and misses the field of Boltzmann generators.\n\nWe appreciate the reviewer’s suggestion and will expand the related work section to better contextualize our work. In short, our approach differs in two key ways from Boltzmann Generators: it explicitly conditions on molecular graph structure to enable transferability, an emerging challenge for BGs [1-3], and employs uniform weighting of conformers to focus on accurate 3D structure prediction over the prediction of thermodynamic ensembles.\n\n[1] Swanson, K., et al., (2023). Von Mises mixture distributions for molecular conformation generation. ICML.\n\n[2] Klein, L., et al., (2023). Timewarp: Transferable acceleration of molecular dynamics by learning time-coarsened dynamics. NeurIPS 37.\n\n[3] Klein, L., & Noé, F. (2024). Transferable Boltzmann generators. NeurIPS 37."}]

Author rebuttal:

Meta-review: This paper proposes several modifications to the Diffusion Transformer (DiT) to better adapt it to molecular conformer generation, resulting in an architecture the authors term DiTMC. Experiments on conformer generation benchmarks show favourable performance compared to baselines. 

 

On the positive side, reviewers considered this work to be a good study of inductive biases for molecular data. Their praised its' elegant modeling, and strong empirical results. 

 

On the other hand, some reviewers considered the performance boost over baselines not significant enough, and noted limited novelty. One reviewer also wanted to dig into scaling laws, which authors made possible by including several new results with a model smaller than our regular base model. Finally, some reviewers noted that the technological novelty seems to be somewhat limited. 

 

After rebuttal, three reviewers were in favour of accepting this work, whereas two leaned towards rejection. 

 

Taking all of the infromation into account, I recommend acceptance. I however encourage the authors to take reviewer discussion into account for the camera-ready version.
Decision: Accept (poster)