Efficient and Unbiased Sampling from Boltzmann Distributions via Variance-Tuned Diffusion Models
Authors: Fengzhe Zhang
Laurence Illing Midgley
José Miguel Hernández-Lobato
Venue: Accepted by TMLR ()

Abstract: Score-based diffusion models (SBDMs) are powerful amortized samplers for Boltzmann distributions; however, imperfect score estimates bias downstream Monte Carlo estimates. Classical importance sampling (IS) can correct this bias, but computing exact likelihoods requires solving the probability-flow ordinary differential equation (PF–ODE), a procedure that is prohibitively costly and scales poorly with dimensionality. We introduce Variance-Tuned Diffusion Importance Sampling (VT-DIS), a lightweight post-training method that adapts the per-step noise covariance of a pretrained SBDM by minimizing the $\alpha$-divergence $(\alpha=2)$ between its forward diffusion and reverse denoising trajectories. VT-DIS assigns a single trajectory-wise importance weight to the joint forward–reverse process, yielding unbiased expectation estimates at test time with negligible inference-time overhead compared to standard sampling. On the DW-4, LJ-13, and alanine-dipeptide benchmarks, VT-DIS achieves effective sample sizes of approximately 80%, 35%, and 3.5%, respectively, while using only a fraction of the computational budget required by vanilla diffusion + IS or PF-ODE–based IS.

Published in Transactions on Machine Learning Research (10/2025)
Efficient and Unbiased Sampling from Boltzmann Distributions via Variance- Susracce-Tuned Diffusion Models
Fengzhe Zhang University of Cambridge
f=2870com.ac..o uk
Laurence I. Midgley
Angström AT University of Cambridge
José Miguel Loraande
jmh233@com.ac. uk
Angström Al University of Cambridge
Reviewed on OpenReview: https: // openreview. net/ forum? id= Jq2ccMC55R
Abstract
Score-based diffusion models (SBDMs) are powerful amortized samplers for Boltzmann dis- tributions; however, imperfect score estimates bias downstream Monte Carlo estimates. Classical importance sampling (IS) can correct this bias, but computing exact likelihoods requires solving the probability-fow ordinary differential equation (PF-ODE), a procedure that is prohibitively costly and scales poorly with dimensionality. We introduce Varianco- Tuned Diffusion Importance Sampling (VT-DIS), post-training method that adapts the per-step noise covariance of a pretrained SBDM by minimizing the a-divergence (a = 2) between its forward diffusion and reverse denoising trajectories. VT-DIS assigns single wrajectory-wise importance weight to the joint forward-roverse process, yickding unbiased expectation estimales at lest. time with negligible inference-time overhead compared to stan- dard sampling. On the DW-4, LJ-13, and alanine-dipeptide benchmarks, VT-DIS achieves effective sample sizes of approximately 80%, 35%, and 3.5%, respectively, while using only fraction of the computational budget required by vanilla diffusion IS or PF-ODE-bass IS. Our code is available at httpss:/gitee com/f2920/cor tuned, diddddiffuoooo
1 Introduction
Sampling from Boltzmann distributions is central to many applications in statistical physics, chemistry, and materials science. Accurate equilibrium samples enable in silico prediction of thermodynamic observables, accelerating the discovery of catalysts, drugs, and novel materials and reducing reliance on costly wet-lab experiments. However, the high dimensionality and multimodal nature of realistic energy landscapes render traditional Markov chain Monte Carlo (MCMC) (Metropolis et al. 1953 Hastings 1970 and molecular dy-
namics (MD) simulations Frenkel & Smit 2023) prohibitively time-consuming for many practical problems.
Normalizing-ftox (NF) Boltzmann generators NNóé et al. 2019 Tan et al.) 2025 Köhler et al. 2020
amortize sampling by learning an expressive, invertible mapping from simple base distribution to the target, and exact density evaluation of generated samples permits unbiased importance sampling (IS). However, the requirement. of tractable Jacobian determinants constrains architectural flexibility, becoming a bottleneck in high dimensions. Contimuous normalizing flows (CNFs) (Chen et al. 2018 Grathwohl et al.) 2019 mitigate this issuc by parameterizing the transformation as an ordinary differential equation (ODE) and have been combined with E33vvqquia architectures for molecular systems (Garcia Satorras al. 2021 Klein al. 2023). Yet even CNFs demand expensive Jacobian computations for log-density evaluation.

---

Published in Transactions on Machine Learning Research (10/2025)
Score-based diffusion models (SBDMs) (Sohl-Dickstein et al. 2015 Ho et al. 2020 Song et al. 2020
offer an expressive alternative that does not require invertibility. As with any learned sampler, imperfect score estimates bias downstream Monte Carlo estimates. Two IS- based corrections exist for a pretrained score network: (i) solve the probability flow ODE to obtain exact likelihoods- an approach that is computationally intensive, and whose approxinations reintroduce bias; or (ii) solve the reverse stochastic differential equation (SDE) and perform IS over the joint aeward-reverss trajectory, whose density factorizes into Gaussian terms; however, naively applying this approach typically yields a very low effective sample size (ESS) because the joint proposal distribution cannot fully cover the target distribution.
We adopt the second approach and show that a simple post-training covariance adjustment makes it practical. We introduce variance-tuned diffusion importance sampling (VT-DIS), which freezes the score network and optimizes only the per-step noise covariance by minimizing the a-divergence (a = 2) between the forward diffusion and reverse sampling processes. VT-DIS yields an unbiased trajectory-wise estimator and achieves high ESS at an orders-of -faggnitude lower computational cost than vanilla diffusion + IS or NF-basod samplers. Our contributions are:
Algorithm. We propose VT-DIS, a post-training procodure that endows diffusion-based Boltzmann samplers with high-ESS, trajectory-wise importance sampling while incurring negligible inference-time overhead relative to sample generation. Furthermore, we extend VT-DIS to learn both diagonal and full covariance structures, moving beyond isotropic noise.
Equivariant extension. We demonstrate how VT-DIS integrates with E(3)-qquivariant diffusion mod- els, enabling the unbiased generation of molecular samples that respect desired symmetry constraints.
Empirical validation. On Gaussian mixtures, toy many-particle systems, and molecular benchmarks (DW-4, IJ-133, and alanine dipeptide), VT-DIS significantly increases ESS relative to naive diffusson + IS with the same number of sampling steps, while incurring only a fraction of the computational cost required for exact likelihood compulation via the probability-fow ODE.
2 Related Work
Normalizing-flow (NF) Boltzmann generators learn an invertible mapping from a simple base distribution the Boltzmann target, enabling exact evaluation of sample densities and unbiased importance sampling via
the chango-of-variable formula. Noóé et al. 2019). Subsequent work improved expressiveness and scalability:
Continuous Normalizing Flows (CNFs) based on neural ODEs (Chen et al., 2018 Grathwohl et al.) 2018)
symmetry-aware flows introduce -quivvariae for molecular systems (Klein et al. 2023). All CNF
variants, however, still require computationally expensive Jacobian determinants or ODE integration for exact likelihood evaluation, making unbiased sampling prohibitive for large biomolecules.
Score-based diffusion models learn the noise-perturbed score and generate samples either by reversing the stochastic differential equation (SDE) (Sohl-Dickstein et al.) 2015 Ho et al. 2020) or by integrating the
probability-flow ODE, which yields exact likelihoods (Song & Ermon 2019 Although diffusion methods
have proven elfective for molecular data (Hoogeboom et al. 2022 evaluating proposal log densities via the PF ODE still requires Jacobian-tras trace computation and becomes expensive in high dimensions.
Beyond the two popular approaches considered above, neural samplers (Vargas et al. 2023a Richter & Berner 2023 Vargas et al.] 2023bb ZZhang & Chen 2021 can, in principle, generate unbiased samples directly from a target energy function without requiring training data. However, He et al. (2025) show that these methods typically incur substantial computational overhead and are often less efficient than generating training data via MCMC and subsequently fitting diffusion model. Scalability is also concern: to the best of our knowledge, no neural sampler currently performs well on molecules in 3D Cartesian coordinates-thi focus of this work. We therefore omit neural-sampler baselines from our experiments.
VT-DIS: Bridging the Gap. VT-DIS retains the efficient stochastic reverse trajectory of standard diffu- sion models while enabling unbiased reweighting. Instcad of integrating the ODE, VT-DIS learns per-step Gaussian covariances by minimizing the a-divergence (a = 2) between the forward diffusion and reverse sam- pling processes, thereby maximizing ESS. Concurrent work has tuned covariances under the Hallback-Teibble
2

---

Published in Transactions on Machine Learning Research (10/2025)
divergence to improve sample quality (Ou et al. 2024 Bao et al. 2022 VT-DIS differs by explicitly opti- mizing the a-divergence objective for importance-sampea efficiency.
3 Background
Before presenting our method, we review three key ingredients: (i) score-based diffusion models (DMs), (ii) their eq3vvvauvva extensions, and (ii) the role of importance sampling (IS) obtaining unbiasod estimators at inference time.
3.1 Score-Based Diffusion Models
Score-based difflusion models (SBDMs) (Sohl-Dickstein et al. 2015 Ho al. 2020 Song al. 2020) generate
data by first diffusing sample from the data distribution datta (#) with progressively increasing Gaussian noise and then denoising it in reverse time. The forward process is described by the Itō stochastic differential equation (SDE)
dæ, = M(v..)) di + g(l) dwr,
(1)
where E [0, [ with T > O, f is the drift, the diffusion coefficient, and we a standard Wiener process, Throughout this work we follow the commonly used variance-erploding setting of Karras et al.(Karras et al. 2022 and set f = and g(t) = /2t. The forward SDE admits two equivalent reverse-time formulations:
v (()) dt + dui, dæ, V log (xx, di,
da,
(reverse SDE)
(2)
(probability-flow ODE),
(3)
where (a))) is the marginal density at time t and w. is Wiener process rumning backwards from T to 0. In practice we approximate the (intractable) score Va Po with neural network s0( trained by score matching (Hyvärinen & Dayan 2005 Song & Ermon 2019).
To draw a sample, we integrate cither or backward from t = T to a small cutoff & z 0, and take Øe as an approximate draw from Tdata- Solving the reverse SDE with the discrete DDPM sampler (Ho et al. 2020) amounts to the update rule po (1 vn) = pm Fn (m In; ()),, where to is the network's prediction of the clean signal step Given time grid ( we obtain the full trajectory density pe (xo.N) = p(œx) II Pe (In-1 xx,). Both the learned reverse-time transition pe(xn-1 wn) and the reference forward kernel q(E,, mn-1) are Gaussian; their closod-form expressions are provided Appendix A.3 Alternatively, one may solve the probability-fow ODE (3) with any off-the-shelf ODE solver to obtain samples at t
3.2 Equivvaaaa Based Diffusion Models
Standard diffusion models ignore the geometric symmetries that characterise many structured datasets. In molecular systems, for instance, the potential-energy surface is invariant under global trunslations, rotations, and reflections-the Euclidean group E(3) well as permutations of identical atoms. Encoding these symmetries explicitly is cralical for producing physically valid and statistically efficient samplers.
Equivariant backbone. Hoogeboom et al. (Hoogeboom et al. 2022
introduced E(3) equivariant diffusion models by replacing the standard score network with an Equivariant Graph Neural Network (EGNN) (Sator-
ras et. al. 2021). Following the analysis of Xu el al. (Xu al. 2022 generative process that is equivariant to E(3) transformations must satisfy two conditions:
1. Rotation and reflection invariance. For any orthogonal transformation REO(3),
p(Nx) = (RRN)) p(#n-1 xx)) = p( Ra Ræ,);
which implies that the marginal p(x.) is invariant al every time step.
2. Translation invariance. Perfect translation invariance cannot hold for non-degenerate continuous densities; instead, it is imposed by restricting all configurations to the centre-of-mass (CoM) subspace
Σ, x; 0, i.e. every sample is translated that its CoM is at the origin.
3

---

Published in Transactions on Machine Learning Research (10/2025)
Practical implementation. These constraints are enforced by replacing ordinary Gaussians N(() with Gaussians projected onto the Prro-CoM subspace, denoted (-) (sampling and density evaluation details are given in Appendix B Sampling proceeds exactly as in Sec. 3.1 we integrate either the reverse SDE or the probability-flow ODE, but all intermodiate states now live in the zero-CoM subspace and all conditional kernels are the projected Gaussians Ng. The resulting pipeline produces molecular configurations that are equivvaair by construction.
3.3 Diffusion Models with Importance Sampling
Assume that the (unnormalized) target density T-) can be evaluated point-wise, whereas drawing i.i.d. samples is infeasible. Directly using raw outputs from a pretrained diffusion model therefore yields biased estimates of expectations Exama [Ø(2z)] IŞ removes this bias through the weight (atta( where Po denotes the diffusion sampler's marginal at 0. Unfortunately, Po is typically intractable, and any approximation re-introduces bias. Because we may generate samples either by integrating the reverse SDE or by solving the probability-flow ODE, two corresponding IS strategies arise.
Reverse SDE pathway. When sampling with a DDPM-type discretization of the reverse SDE, define
N-1
((zo:N) datta (au)
πI ( xn), (e( p(æN) n=0
po ( Øn41), n=0
II
(4)
where both kernels are Gaussian and therefore tractable. The trajectory weight w po (ma.x) yields unbiased estimator (see Appendix A..3) In practice, however, the mismatch introduced by a finite time
x)
discretization causes the effective sample size (ESS) to collapse.
PF ODE pathway. If instcad solve the probability-ftow ODE can be written as (Chen el al. 2018
3 the likelihood of the terminal sample
Song et. al.) 2020
lo Po (z(0) = log logpr(x(TT)) + f V V (-tso(((((())) dt.
(5)
where 8o the score model. Computing the divergence term exactly is prohibitively expensive and is typically approximated with the Hutchinson Skilling trace estimator (Skilling 1989; [Hutchinson] 1989). Such stochastic estimates, together with timo-discretisation error, re-introduce bias and defeat the purpose of IS.
Limitations. Although both pathways are theoretically sound, each faces practical obstacles: the reverse- SDE approach yields vanishing ESS, whereas the ODE approach incurs excessive compute or bias. In the next section we address the first limitation by introducing covariance-adjusted scheme that dramatically increases ESS practice.
4 Method
In this section, we introduce the post-training method VT-DIS to improve ESS for diffusion models with importance sampling. We will first give an overview of the whole algorithm and explain each part of the algorithm in more details and the rationale behind them, then we will explain how our method can also be applied E(3)-Equivariant diffusion models for molccular gencrations.
4.1 Overview
We build on a pretrained score-based diffusion model, freezing its score network 8g, and introduce a post- training step that adjusts only the per-step noise covariance (n) N with & being learnable parameters. The objective to make the learned reverso-time process better agree with the true forward (noising) process by minimizing the the ESS. In practice, we: (1) Simulate target trajectories (mo:N) ) from the forward SDE; (2) Compute the
divergence, which directly controls the variance of importance weights and hence
4

---

Published in Transactions on Machine Learning Research (10/2025)
Algorithm Post- fost-trainnng Covariance Tuning (VT-DIS)
Require:
Score network 80, initial covariances { time grid {4a J batch size M
Ensure:
Tuned covariances (ΣΦ-}
1: repeat
I.M ~ ((<O.N) 2: Sample fz
(x)) -- log q( (ooo)) log POg)
M
4: I(Ø) -
1
Σ
expl
e (Φ))
M
5: (Ex) + {ΣΦ} ==1
VL
6: until convergence
[E } + {EΦ)
log density ratio between the forward and reverse joint distributions given current (E >==1: (3) Update the covariance parameters by minimizing the Monte Carlo estimate of the a 2 divergence; (4) At inference, sample with the tuned covariances and assign cach trajectory single importance weight. We summarize the whole process in Algorithm 1
4.2 Covariance Parameterization
We explore three complementary ways to parameterize the per-step covariance Eạ(n).
Isotropic Covariance The standard DDPM proposal
Po - (xx))) uses an isotropic variance En =
dddpm I I (where ddppm( (n) is defined in Appendix A.3
21 ). To tune this, for each n = 1, N
we introduce a nonnegative scalar lh. so that
En(n) nn adpm (n)
(6)
with 7h 2 enforced (e.g. via SoftPlus) and initialized at 7n 1 to recover the standard DDPM baseline. We use Φ to represent all tunable parameters for each n.
Structured covariances To capture per-coordinate anisotropy or correlations, we replace the scalar mul- tiplier with vector or matrix parameters. A diagonal form
En(n) o ddp (n) diag(T7..tn.....h...d Tm.d),
with Mh, 0 and initialized with each Mn. 1 that baseline is recovered. A fully correlated covariance uses learnable matrix Ln E Rdxd; (n) = ad (n²) Ln LT, with Ln initialized to the identity to recover the baseline. To limit parameter growth which makes optimization hard especially when number of time steps is large, we further propose a practical low-rank plus isotropic decomposition:
(n(n) = (n)
(n) (An + an.),
(8)
where An E Bddk where k < d is chosen depending on the computing budget and an 2 0 are learned, and A, + a is initialized to I by initializing A, to be zero matrix and an 1.
Sampleede models In the case that the number of time steps large and even the low-rank ncural network with input t, predict the optimal covariance matrix each step. Additionally, for maximal flexibility, (n(n) can be made function of the current state an (and (n) via neural network. That is:
version (8) becomes infcasible,
(n) = NN&(t.) or Ex(n) = NN (n)
Although this state-dependent form remains valid, our experiments did not show significant ESS gains over the timc-only or time-and-sample dependent paramotorizations and the additional function evaluation get covariance increases the computational cost during inference of diffusion models. Therefore, we defer its exploration to future work.
5

---

Published in Transactions on Machine Learning Research (10/2025)
4.3 Extension to Equi-aquivar Diffusion
Our VT-DIS covariance tuning readily exlends to equiivaaurr dilfusion models: the only modification is that all trajectories lie in the zero center of mass (CoM) subspace. Isotropic covariances require no change, since it commutes with rotations, reflections, and CoM projection.
For diagonal or fully correlated covariances, the lollowing form of covariance matrix will satisfy E(3)- equivariance and permutation equivariance: En(n) (n) I3,
(9)
where (n) RMX acts the M particles, Ia the three Cartesian coordinates and denotes the Kronecker product. The matrices Bœ(n) are learned per step. This form automatically satisfies E(3)- equivariance (proof in Appendix B To enforce permutation symmetry, Bø(n) must respect the specific particle-identities of the dataset. We illustrate two representative cases below.
Many-partiche systems All particles are identical, so diagonal Bn(n) reduces to an isotropic covariance and adds no flexibility. Instead, we parameterize the full covariance
Bx(n)= = (bn an a, 1,
where an and bn are learnable parameters with bn > and - < an bn ensure Ba(n) 0. Appendix B shows that this choice satisfies both (33))iinuva and full permutation invariance, needed for systems such as n-body systems.
Molecular distributions. Real-world molecules introduce chemical symmetrics that are more subtle than the "all particles indistinguishable" setting of the above section. For instance, in alanine dipeptide only the three hydrogens bound to the same carbon atom may be permuted freely; the remaining atoms are fixed. To encode such partial symmetries we assign every atom a symmetry label L; (I,...,K' i= I,..., M, where atoms sharing the same label are interchangeable while atoms with different labels are not. The covariance (T)n) al sampling slep is then constrained to depend on the pair (i,j) only through the labels (Li, Lg). Proof in Appendix B.3
1. Diagonal (per class variance). Let Th, 2 be learnable variance for class
(n)n) diag( (0,4, In.. (.,,,,...,7,,,L.
Positive definiteness is automatic when every n,, > 0.
2. Block-structured (full class class covariance). For any learnable matrix An RRKK and a learn- able scalar an O. Define
[m)( (An on dig.
where dig is the Kronecker delta. The added diagonal term a, guarantecs B²(n) x 0.
4.4 Objective Function
Our aim is align the reverse-process joint proposal (o( with the forward-diffusion joint q(FoN)) (see (4), in manner that directly improves importanco-sampling performance. Although one could minimize the Kullback Leibler (KL) divergence DKI(glpp>) (and indeed closed-form optimal covariances under this criterion have been derived (Ou et al., )024)), such minimization does not control tail mismatch. Poor alignment in the tails leads to high variance of importance weights and thus low ESS when sample size is large. Instead, we minimize the a-divergence with a = - which proportional to the variance of importance weights:
arg min Da=2 (gllps) arg min @
[(()== : /
q( g(<o.N) ((20.N) dæoiN R P& (x
1
((
(5)
D ~ q (10)
(6)
P& (x)
(7)
We use the a-divergence De (ullp)) m) (J (())a po(a))- do do- -1 (a # 0, In our experiments we minimize the case 2; the optimized loss (()) is the integral above.

---

Published in Transactions on Machine Learning Research (10/2025)
In practice, we draw M Irajectories ( from the forward process q to form the unbiased Monte Carlo estimate and minimize log ((Ø) for numerical stability (see Appendix C.Ip. Optimizing this loss by gradient descent yiclds covariance parameters that minimize importance weight variance and thereby maximize ESS.
4.5
Choice of Sampling Schedule
Karras ct al.
2022 propose family of contimooustimme schedules controlled by parameter p:
(N 1/0 + x 11000 - to = c,= IN=-T
In the limit p -> 00, this reduces to a fixed geometric grid t, to (tw/t)) n/(N-1) which we found consistently yields lower 2 divergence and higher ESS across all benchmarks. We also experimented with jointly tuning the schedule and per-step covariances during VT-DIS. While joint tuning improves performance on simple Gaussian mixtures, it proved unstable on molecular datasets. We defer a robust treatment of schedule learning to future work.
4.6 Complexity & Inference- Time Cost
Relative to a standard DDPM equipped with importance sampling, VT-DIS adds only a single, offline optimization step: tuning the per-step noise covariances (n)). After this one-time cost, generate samples and importance weights only adds negligible overhead to standard sampling from diffusion models-no extra function evaluations. Unlike flow-based Boltzmann generators diffuusion generate samples by solving PF ODE, VT-DIS never has integrate an ODE to evaluate marginal proposal log-densities, procedure that becomes prohibitively expensive in high-dimensional settings.
5 Experiments
Experimental setup. We assume access to both the energy function and the training data used to fit the diffusion model (same assumption as in, e.g., Klein et al. (2023): : Tan et al. (2025)). We first train a diffusion model with an FGNN backbone (Satorras et al. 2021 and EDM preconditioning (Karras et al., 2022) using a standard training procedure. We then apply VT-DIS to the same training set to optimize the proposal covariances. For evaluation, we compare VT-DIS with a DDPM+IS baseline under fixed compute budget measured by the number of function evaluations (NFEs; i.e., sampling steps). We also compare against importance sampling with exact likelihoods computed via the probability-fow ODE (PF-ODE). Performance is reported as both forward and reverse elfective sample size (ESS). We additionally report the Evidence Lower Bound (ELBO) and Evidence Upper Bound (EUBO) for the DDPM baseline and VT-DIS, which serve as complementary likelihood-baase metrics to assess the quality of the generated samples.
We consider three representative classes of Boltzmann systems: (i) Synthetic: high-dimensional two- mode Gaussian mixture (GMM-2); (ii) Many-particle: double-well (DW-4) and Tennard-Jones-13 (LJ-13) clusters (Köhler et al.) 2020); and (iii) Small-molecule: alanine dipeptide (ALDP). For all targets, the (unnormalized) density daata is available, enabling exact importance weights. We use the atomic coordinate representation for each sample and train our models on these coordinates. Further implementation details are provided in Appendix
5.1 GMM-2
We begin with a two-comppnent isotropic Gaussian mixture in Rdd.
Teata (2:) wi N(a; o + w2 N(a: u2 I)
where #1 = (1,...,1) 1)T E R4, 12 = (-2. ... - E Rd, of oh = 0.15, and w = 3. w2 3. We evaluate two ambient dimensions, E (50, 1003. Figure shows forward and reverse ESS as function of the number of
Empirically found that attempting estimate D2 with samples from the learned reverse process pa often induces mode collapse and degrades forward FSS.
7

---

Published in Transactions on Machine Learning Research (10/2025)
ESS for 50D GMM
ESS for 100D GMM
ESS (%6
40-
20
30-
10-
10
30
25
45
50
10
15
20
25
Number of Steps
Number Steps
DDPM
Isotropic Forward
Diagonal Forward
Full trank 10) Forward
DDPM Reverse
Isotropic Reverse
Diagonal Reverse
Full (rank 10) Reverse
Figure 1: GMM-2: ESS versus NFEs for dimensions 50 and = 100, comparing VT-DIS with the
DDPM + IS baseline. ESS estimates are computed using 10 samples.
ESS Dev)
ESS (Mean Std Dev) for Alanine Dipeptide
ESS (%)
ESS (96)
Number Steps
Number Steps
Number of Steps
tld Rovens
oenticteed hamaro dun jub senas
Dcem -orwer
Gas jsarna chons
T
Famard (Dageras
(a) DW-4 (d=8) 8)
(b) LJ-13 (d = 39)
(c) ALDP (d = 66)
Figure 2: ESS versus NFEs. VT-DIS consistently outperforms the standard DDPM + IS baseline across all tasks under the same computational budget. ESS values are estimated using 10 samples.
function evaluations (NFE) for VT-DIS with three noise covariance parameterizattons- -istrropic diagonal, and full rank- together with the DDPM + ISS baseline.
The plots reveal three key trends: (i) VT-DIS consistently outperforms the baseline in forward ESS, reverse ESS, or both, for every covariance parameterization. (fi) The full-rank covariance yields the highest forward and reverse ESS for both values of d. For d = 100, however, the forward-ESS error bars are wide when d = 100, indicating residual mismatch between the forward and reverse processes at very high dimension. (iii) With isotropic and diagonal covariances, reverse ESS improves as NFE grows, whereas forward ESS stays low, underscoring the importance modeling cross-dimensional correlations in this setting.
5.2 part-pa systems: DW-4 and LJ-13
We next evaluate VT-DIS on two prototypical many-body systems with pair-wise interactions: DW-4 (double-well well potential, four particles in RR, = =8 8) and LJ-18 (Lennard-Jones cluster, thirteen particles
in IR3 d Köhler et al. 2020).
ESS versus EsEs. Figure reports forward and reverse ESS as a function of the number of NFEs. On the low-dimensional DW-4 system, VT-DIS raises both ESS measures from 1% to 40% using only 100 NFEs. The improvement is even IS bascline yiclds virtually 0% ESS up to 400 NFEs, whercas VT-DIS achieves v 35% ESS on both forward and reverse measures with the same budget. Due lo the permulationn -on-aaiiaannaiina constraint of the many- particle system, the full covariance matrix has only one additional degree of freedom compared to an isotropic covariance. In this setting, we observe no significant ESS gain from using the full covariance, although both forward and reverse ESS exhibit lower standard deviations than with isotropic noise. Across both tasks, ESS increases monotonically with the number of NFFs, facilitating clear accuracy cost trade-off.
pronounced the higher-dimensions LJJ-33 cluster: the standard DDPM +
8

---

Published in Transactions on Machine Learning Research (10/2025)
DWW-d ELBO EUBO diffusion
LJ13 EL30 and EUBO diffusion
Alanina dipeptido ELBO EUBO
eaperisect 1 2100
80
sipi 2160)
jasss
denn S1A0
1 SEEm:
Boured velue (rett))
botaak 15.00
Boure value (maas)
-20
dag ELCO
Beteaii (3.00
30-
60
28
10
-40
100
150
200
250
300
350
400
100
150
200
250
300
350
400
200
300
400 Number of ciffusion steps
500
600
Number of diffusion steps
Number of diffusion steps
(a) DW-4 (d - 8)
(b) LJ-13 (d = 39)
(c) ALDP (d = 66)
Figure 3: ELBO SUBO/EUS EUBO vs. diffusion steps. ELBO (higher is better) and EUBO (lower is better) for the DDPM baseline and VT-DIS (isotropic, diagonal and full covariances) as function of the number of sampling steps. Error bars denote + one standard deviation across 3 repeated MC evaluations.
DW-4 Reweighted Histogram
U-13 - Reweighted Histogram
Alarine Dipeptide- Reweighted Histogram
Normalzzs Denety
Normalized Dersity
Normalized Density
8:0
Enargy
Energy
Energy
(a) DW-1, 100 NFEs
(b) IJ-13,, 100 NFEs
(c) ALDP, 600 NFEs
Figure 4: Energy distributions. Unweighted samples generated by VT-DIS with isotropic covariance (or- ange) are biased relative to ground-truth samples (blue), but weportence-weighheed samples (red) effectively correct the discrepancy. Histograms are plotted using 5 x 10y samples.
ELBO and EUBO. We also report the standard likelihood bounds (ELBO I log S EUBO), where smaller gap indicates better agreement between the forward noising path and the learned reverse process. Figure 3 shows that across DW-4 and LJ-13, VT-DIS consistently raises ELBO and lowers EUBO relative to the DDPM baseline, yielding markedly lighter "sandwich" at every step budget. Overall, these trends mirror the ESS gains, indicating better forward-reverse reverse matching under fixed compute.
Bias correction. To illustrate bias correction we compare three energy histograms for each task: (i) ground- Iruth samples from the test dataset, (ii) unwwiighte VT-DIS generated samples, and (iii) reweighted samples. Figure 4 showss that the raw proposals are biased, but importance reweighting realigns the distributions with the target, corroborating the ESS improvements.
5.3 Small Molecule: Alanine Dipeptide (ALDP)
We evaluate VT-DIS on the 66-dimensional Boltzmann distribution of alanine dipeptide (ALDP) in implicit solvent at temperature T 800 K. The unnormalized Boltzmann density is defined by a classical force-field potential, and the availability of its exact log density enables unbiased importance-sampling evaluation. Training and implementation details are provided in Appendix
Results. Figure 2c shows the forward and reverse ESS over a range of 200-600 NFEs. VT-DIS consistently improves the reverse ESS, reaching approximately 2.5% at 400 NFEs, compared to under 0.5% for the baseline. Among the covariance parameterizations, the full covariance version achieves slightly higher reverse ESS compared to diagonal and isotropic paramoterizations. Figure 3c further illustrates that VT-DIS with all three parameterizations yields a smaller gap between ELBO and EUBO relative to the DDPM baseline. Notably, the full covariance parameterization exhibits an even smaller ELBO-EEBO gap in the low diffusion- step regime, which aligns with the observed ESS performance.
Figure 4c displays the energy histogram, and Figure 3 presents the Ramachandran plot. Both demonstrate that importance sampling effectively corrects bias in the generated samples. Notably, the molecular dynamics
9

---

Published in Transactions on Machine Learning Research (10/2025)
Ground Truth
Generated (Unwelghted)
Generated (Reweighted)
02
02
Rtzd Demsity llog
.2
/2
/2
Figure 5: Ramachandran plots from left to right: ground-truth samples generated by MD, VT-DIS (isotropic) samples before and after reweighting. All plots are generated using 10 samples.
(MD) "ground-truth" samples themselves exhibit slight bias, as the reweighted energy histogram shifts toward lower energies than the MD samples.
For both VT-DIS and the baseline, the forward ESS is nearly zero due to a small number of extreme density- ratio outliers. This indicates that both methods' joint proposal distributions assign extremely low probability to regions containing true samples from the test dataset. We hypothesize that the score model is missing certain modes, supported by three empirical observations: (i) in the unreweighted samples (middle plot of Figure 5). the mode ncar o x/2 is underrepresented; (ii) running the DDPM bascline with very large numbers of time steps (1000 and 2000) still resulted in forward ESS & O, indicating a support-coverage mismatch rather than iistronization issues; (iii) subsequent experiments solving the PF ODE for importance sampling (Table similarly yielded forward ESS v on Alanine Dipeptide.
Since the score model defines the mean of the per-step proposal distributions and VT-DIS only adjusts the covariance, its capacity to improve forward ESS is limited if the score model inadequately trained. Therefore, VT-DIS yields substantial improvement of forward ESS only when the score model sufficiently captures all modes. Two promising future directions include: (i) enhancing the training of the score model to better capture all relevant modes direction complementary to our current work, as VT-DIS can enhance any trained diffusion model; and (ii) extending VT-DIS to adjust both the mean and covariance of the per-step proposal distributions, particularly useful when the score model is missing certain modes.
5.4 Computational Cost versus Probability-Flow ODE
While VT-DIS and PF-ODE + TS both yield unbiased estimators, they differ substantially in computational efficiency. Here, we compare the ESS and computational cost of each method.
Experimental setup. To isolate inference cost, we reuse the same pretrained score network and employ an identical number of integration steps for both methods. Computational effort is measured in giga floating- point operations (GFLOPs) per sample, and accuracy quantified by forward and reverse ESS. Results for DW-4, LJ-13, and ALDP are summarized in Table I We also include an extended discussion and results on solving the PF-ODE using the trace estimator in Appendix D.I
Results. PF-ODE + IS attains higher ESS but incurs a steep computational penalty: its GFLOPs per sample are one to two orders of magnilude greater than VT-DIS's and this gap widens with dimensionality. In practice, PF-ODE requires a minimum number of ODE steps to control prepssal-density bias; each additional step further increases cost. VT-DIS, in contrast, trades some ESS for dramatically lower compute. Its estimator is asymptotically unbiased as the sample number increases, regardless of the time-step budget, and each additional reverse-SDFF step directly boosts ESS.
10

---

Published in Transactions on Machine Learning Research (10/2025)
Table 1: Performance on DW-4, LJ-13 and ALDP with 10a samples on a single A100. The number in parentheses after each dataset denotes the sampling steps. f higher better, + lower is better.
DW-4 (100)
LJ-13 (200)
ALDP (200)
Rev. SDE
PF ODE
Rev. SDE
PF ODE
Rev. SDE
PF ODE
Forward ESS (%) 1
44.32
94.29
15.87
79.21
R
R
Reverse ESS (%)
46.76
83.60
17.69
77.42
1.05
35.39
GFlops / sample 0.81
7.70
26.2
2173
33.7
45967
6 Conclusion and Limitations
Contributions. We have introduced VT-DIS, post-training procedure that be applicd to any pre- trained diffusion model, and demonstrated its iveness in molecular generation. By computing trajectory- wise importance weights, VT-DIS achieves high effective sample size (ESS) across multiple benchmarks while using only a fraction of the computational budget required to solve the PF ODE for exact likelihood es- timation. Furthermore, VT-DIS provides asymptotically unbiased expectation estimates as the mumber of samples grows- inddesedended of the number of sampling steps. In contrast, PF-ODE solvers guarantec asymp- totically unbiased sampling only when both the number of samples and the number of discretization steps increase, which limits their practical utility. These findings highlight. VT-DIS's polential for diffusion-based Boltzmann generators.
Limitations and Future Work. Despite its advantages, VT-DIS has several limitations. First, the post- training procedure must be repeated for each sampling schedule, which can incur significant computations ional and memory costs. Figure o in Appendix D.2 shows a clear pattern in optimal hyperparameters across different step counts; future work could leverage this pattern to develop a unifiod post-training algorithm that supports arbitrary schedules in one stage. Second, optimizing the sampling schedule itself to minimize the a-divergence remains open question; existing methods from (Sabour et al., 2024 Williams et al. 2024) may offer promising starting point. Third, VT-DIS requires access to ground-trus -ruth samples to tune the per-step proposal covariances. When such samples are unavailable, a dataset must first be generated- e.g.. via MCMC or molecular dynamics (MD). A useful direction for future work is to adapt VT-DIS to operate with only a small set of approximate reference samples, which are casier to obtain.
Acknowledgments
We thank Jiajun He and Javier Antorán for valuable discussions during the development of this work, and Ruikang Ouyang and Zhuoyue Huang for their insightful feedback on carlier drafts. F7 and JMHL. acknowledge support from the Turing Al Fellowship under UKRI grant EP/ EP/V023756/L. LIM acknowledges support from Google's TPU Research Cloud program and the EPSRC through the SynTech CDT.
References
Tara Akhound-Sadegh, Jarrid Rector-Brond Brooks, Avishek Joey Bose, Sarthak Mittal, Pablo Lemos, Cheng-Hao Liu, Marcin Sendera, Siamak Ravanbakhsh, Gauthier Gidel, Yoshua Bengio, al. Iterated denoising energy matching for sampling from boltzmann densities. arXiv preprint 0611012402.0121,, 2024.
Fan Bao, Chongxuan Li, Jiacheng Sun, Jun Zhu, and Bo Zhang, Estimating the optimal covariance with imperfect mean in diffusion probabilistic models. arXiv preprint arXiv:2206.07300 2022.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Daan Frenkel and Berend Smit. Understanding molecular simulation: from algorithms to applications. Flsevier, 2023.
11

---

Published in Transactions on Machine Learning Research (10/2025)
Victor Garcia Satorras, Emiel Hoogeboom, Fabian Fuchs, Ingmar Posner, and Max Welling. E (n) equivariant normalizing flows. Advances in Newral Information Processing Systems, 34:4181-411 2021.
Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Πya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. arXiv preprint arXio:1810.01367, 2018.
Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, and David Duvenaud. Scalable reversible generative models with free-form continuous dynamics. In International Conference on Learning Representations, volume 3, 2019.
W Keith Hastings. Monte carlo sampling methods using markov chains and their applications. 1970.
Jiajun Hc, Yuanqi Du, Francisco Vargas, Dinghuai Zhang, Shreyas Padhy, RuiKang OuYang, Carla Gomes, and José Miguel Hernández-Lobato No trick, no treat: Pursuits and challenges lowards simulation-Iree training of neural samplers. arXiv preprint arXic:2502.0663. 06685, 2025.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851 6851, 2020.
Emiel Hoogchoom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning, pp. 8867 8887. PMLR, 2022.
Michael F Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Communicatiins in CndiislicsSimmnattion and Compulation, 18(3):1059 1076, 1989.
Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching, Journal of Machine Learning Research, 6(4), 2005.
Tero Karras, Miia Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Aduances in neural information processing systems, 35:26565-26577 2022.
Leon Klein, Andreas Krämer, and Frank Noé. Equivariant flow matching. Advances in Neural Information Processing Systems, 36:59886 59910, 2023.
Jonas Köhler, Leon Klein, and Frank Noé. Equivariant flows: exact likelihood generative learning for
symmetric densities. In International conference on machine learning, pp. 5361 5370. PMLR, 2020.
Nicholas Metropolis, Arianna W Rosenbluth, Marshall Rosenbluth, Augusta H Teller, and Edward Teller. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087- 1092, 1953.
Laurence Midgley, Vincent Stimper, Javier Antorán, Emile Mathieu, Bernhard Schölkopf, and José Miguel Monddndez-Lbbata. Se (3) equivariant augmented coupling flows. Advances in Neural Information Pro- cessing Systems, 36:79200-79225, 2023.
Laurence Illing Midgley, Vincent Stimper, Gregor NC Simm, Bernhard Schölkopf, and José Miguel Lobaao. Flow annealed importance sampling bootstrap. arXiv preprint arXiv:2208.01 011993 2022.
Frank Noé, Simon Olsson, Jonas Köhler, and Hao Wu. Boltzmann generators: Sampling equilibrium states 2019.
of many-body systems with deep learning. Science,
Zijing Ou, Mingtian Zhang, Andi Zhang, Tim Xiao, Yingzhen Li, and David Barber. Improving proba- bilistic diffusion models with optimal covariance matching. arXiv preprint arXi::2406.10800 2024.
Lorenz Richter and Julius Berner. Improved sampling via learned diffusions. arXiv:2307.01198 2023.
arXiv preprint
Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your steps: Optimizing sampling schedules in diffusion models. arXiv preprint orxi::2404. 14507, 2024.
12

---

Published in Transactions on Machine Learning Research (10/2025)
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference on machine learning, pp. 9323 9332. PMLR, 2021.
John Skilling. The eigenvalues of mega-dimensional matrices. Mazimam Entropy and Bayesian Methods:
Cambridge, England, 1988, pp- 455-466, 1989.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp. 2256 2265. PMLR, 2015.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural informalion processing systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishck Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXio preprinl arXiv:2011.13450, 2020.
Charlie B Tan, Avishek Joey Bose, Chen Lin, Leon Klein, Michael M Bronstein, and Alexander Tong. Scalable equilibrium sampling with sequential boltzmann generators. arXiv preprint arXiv:2502.1844 18462. 2025.
Francisco Vargas, Will Grathwohl, and Arnaud Doucet. Denoising diffusion samplers. ar preprint
arXiv:2302. 13834, 2023a.
Francisco Vargas, Shreyas Padhy, Denis Blessing, and Nikolas Nüsken. Transport meets variational inference: Controlled monte carlo diffusions. arXiv preprint wxXiv:2307.10666 2023b.
Christopher Williams, Andrew Campbell, Arnaud Doucet, and Saifuddin Syed. Score-optimal diffusion schedules. arXiv preprint arXim:2412.07877, 2024.
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint vrXim:2203.02922 2022.
Yilun Xu, Shangynan Tong, and Tommi Jaakkola. Stable target field for reduced variance score estimation in diffusion models. Xiv preprint arXiv:2302.00670 2023.
Qinsheng Zhang and Yongxin Chen. Path integral sampler: stochastic control approach for sampling. arXiv preprint 1x1:::111.11141, 2021.
13

---

Published in Transactions on Machine Learning Research (10/2025)
A Score-based Diffusion Models with Importance Sampling
In this appendix, we present the mathematical derivations for combining score- diffusion models with importance sampling. We first review the fundamentals of importance sampling and then show how it can be integrated into score-based diffusion to correct bias in generated samples.
A.1 Review of Importance Sampling
Importance sampling (IS) provides an estimator for expectations of the form
E [0(x)) f )(xx) Tdata (x) dæ,
where Taata is the (possibly unnormalized) target density and is test function. Directly sampling from ddtta may be infeasible, but we assume that we can evaluate its density up to normalization constant.
Instead, we draw samples (w!") j from proposal distribution p(æ) and form the self-normalized estimator
Ex Tansa [d(x)) Σ wn of (x))
where
Edata (x(1))
W
W/
p ((xx
Wn
Wmm
This self-normalized importance sampling (SNIS) estimator is consistent and asymptotically unbiased, with its variance decreasing as N increases.
A.2 Effective Sample Size
The quality of an IS estimator is often measured by its effective sample size (ESS). Two common definitions are:
Forward ESS, which evaluates the overlap with the target distribution:
N2
ESStwd
(1)
Taaaa
(11)
Z EN w(x())
where Z-1 = v EN 1 m. is the inverse normalizing constant.
Reverse ESS, which measures the concentration of weights under the proposal:
(EN (EN و)) Ei=1 w(y6))2
ESS
(12)
While reverse ESS is widely used for evaluation, can overestimate performance if p suffers from mode collapse. Forward ESS, by contrast, tends to provide a more reliable measure of overlap between the target and proposal.
A.3 Combining Score-based Diffusion Models with Importance Sampling
We now consider task of estimating the integral Eg [o(x)] where normalized or unnormalized Aaata the target density which we can evaluate, and & is the test function of interest. We aim train a diffusion models act the proposal distribution which we can draw samples from. Suppose we have time interval [c, T] and discretize the time interval into N sections with N+1 time steps such that to < la <... < <iN
14

---

Published in Transactions on Machine Learning Research (10/2025)
Assume the conditional proposal distribution of the diffusion model is p0(2n-112n)) and the conditional noise distribution is g( (a) for -,..... N. Using SNIS, the integral can be estimated as
E Exwa [o(x)] z Exion [d(x))]
(13)
/ ((2r0) (270)
(14)
/ o(2))T(Z)))
II can xn-1)
dxo:N
(15)
= f o(z0))
N xxx0))
T== ( (n-
po po( )da.::
(16)
(o(
I po (In (20m.)
K
(k)
n=1 g (k) (k)
in (ta-s) ((xa)
(17)
K
II n= po a (k)
in- tn
where (b))) for k: I,..., K are samples from the joint proposal distribution po and wy for k = .,..., are the importance weights. Note that is chosen such that the error for the first approximation is negligible.
We now need to specify the exact forms of g( xn-1) and po (Z) Recall the forward SDE with zero drift (f = 0) and dilfusion coefficient. g(l) = /21:
dat = 21 dw.
Given a time discretization =o < < IN T, the conditional target distribution of the forward process is Gaussian:
p( | xn-11) = -N In (h (2-1))
I): tn > 11-
(18)
By Bayes' rule, the DDPM-style reverse (proposal) distribution conditional on on and Do
p( Fn Fo) p( xn-1)) p( xoo)
(19)
= N(nnnni 11 20), a dd (n) I),
where
+2
(1
12
(1 Fo)
In
+
It.
(20)
1--1
12
addp (n)
n n
(21)
t2
In practice, wo will be predicted via a neural network and thus the reverse conditional defined as po pa x = p(z,-1|znn @o(#n, ty; 0)). Since both forward and reverse conditionals are Gaussian with closed-form parameters, the joint density ratio over trajectories be evaluated exactly with minimal overhead.
B VT-DIS for E(3)-Equivariant Diffusion Models
In this appendix we detail how the proposed velocity-tempered diffusion importance sampling (VT-DIS) post-training procedure can be integrated with E(3)-equivariaan diffusion models.
Notation. Let M 2 denote the number of particles and let n be the spatial dimension. Concatenate the coordinates of all particles into the column vector
(2. ..... on (d0) TR....
a E R..
15

---

Published in Transactions on Machine Learning Research (10/2025)
Unless stated otherwise restrict our attention to the zero crro--cettr-of-mass (CoM) subspace
M Σ x = } M
No = { E R
Mn
1
Chango-of-basis matrix. To remove the CoM degree of froedom, we construct changc-of-basis matrix P that maps any Xo from the Mn-dimensional ambient space to (M - -1)n-dimensional Euclidean space. Choose orthonormal matrix VM [MM-I)XM whose rows span the subspace {z E RM :1 x 0): one explicit choice is obtained by applying QR factorization to IM NI 11 Define
P= VM In RMM-Ijaxaa
where @ denotes the Kronecker product. For any No we sel 5 = Px E R(M-11)n The identities
PPT- 1(M--))
PTP= (INs M7 11 @ Ia,
(22)
follow immediately from VM VM = JM-- and VM = IM M 11
The first identity shows that P is an
isometry between Xo and (MM-1)))
Evaluating and Sampling from N₂ with full covariance For E No and positive-definite E e R MMx Mn whose image lies in Xy, define the density
Na μb E) = N( ft PEPT)
(28)
exp [ - - m)]-
(24)
(2x) dett(PEPT)
To get samples from this distribution, we have to first sample from the normal distribution with dimension (M-1) 1)n and then we map it back to the Mn dimensional ambient space so that its center of gravity equals zero using the change-of-basic matrix P. This the general way to get samples for N₂ with full covariance. Things will simplify if the covariance becomes isotropic.
If E I2I is isotropic,
24 will becomes:
Na u1 02
expl
202 T -w)]
(2TT2) (M-
because of the fact that PPT IM-- This is the Gaussian distribution constrained zero CoM subspace
with isotropic covariance (Hoogeboom et al. 2022 Xu el al. 2022). To sample from this distribution, we could sample in the Mn dimensional ambient space directly, and subtract its center of mass (Xu et al. 2022
Satorras et al. 2021
B.1 Proposed Covariance Structure Satisfies E(3)- [33--qquivariance
A diffusion model is E(3)-cqquvariant if, for every orthogonal transformation R E O(Mm) E (i) the forward- process endpoint distribution p(æT) is invariant, and (ii) the reverse-process conditionals p(æ,-1 xx)) are equivariant:
PR Ro,)) = p( xx) Vt fI....,T).
Invariance of the isotropic Gaussian prior p(ær)) No.. (0, (0,T IMra) is immediate. We therefore focus the conditional law.
Lemma 1. Let the reverse transition be
p(x x1) = N E),
B 8 In.
where B R is positive definite. Then the reverse kernel is E(3) -equivariant.
TTranslations are excluded because we restrict to the zero zero-CoM CoM subspace Xa; Sec.
16

---

(B Tn)-1 Published in Transactions on Machine Learning Research (10/2025)
Proof. Every spatial rotation acts identically on all M particles and can therefore be written the block- orthogonal matrix
R = IM & Ra; Rn E O(n).
Recall that the reverse kernel is defined on the (M - 1)n-dimensional coordinate ă == Pæ via
p xx) = N((x Pæt, PEP"),
E =B In B 0.
For arbitrary æ. y E No have
logp(Ry Ræ) = (PRy PEPT)
[(RRy PRx) PE-LPT (PRy PRx)
const
fy - x) R R p PL PR(y- - x)
Yu x) P PR RP P(y - x)
Hv x) PT PE- PT P - x)
[Py - Px:) P.- PT (Pt - Pa:)
= Nog. Pæ, PEPT)
logp(y x),
where step (a) uses the fact that PTP commutes with R:
RP P= (Tn 09 (VM I.) VM @ RS P PR
and step (b) uses the fact that
R (R ER)-
((IM & RB @ (Is & RR))
(B R2 Rm))
---
Hence we have proved that our proposed covariance form satisfies the E(3)--quivariance required.
B.2 Permutation Equivariance for Many-Particle Systems
Dalasets such DW-4 and LJ-13 treat all particles as indistinguishable; hence the reverse kernel must. be equivariant under any permutation of the M particles.
Lemma 2 (Permutation equivariance). Let the covariance in the reduced coordinate space be
Σ B
B (b-a) IM- a11
with parameters b and chosen such that b- a and b+ (M 2)a 0, ensuring B 0. For any permutation matrir SM MxM and the block permudation S = SM Jn, the reverse transition satisfies p(Sy) Sx) p(y æ) for all v, y Xn.
Proof. (i) Commutation with the CoM projector. Using PTP (Vyr (n) (Sec. B and the fact that SM 1 and 1 SM =T
SPT P (SM (IM 11 8 I, = (SM 11 & I, = PTPS.
M
M
17

---

Published in Transactions on Machine Learning Research (10/2025)
S commutes with the projector onto the zero-CoM subspace.
iii) Invariance of the covariance. Because SM merely reorders rows and columns,
SM B (b 10-a IM-1 Sw Shr (b a) IM-1 all" B,
whence SES =. (iit) Equality of densities. With the same quadratic-form argument used in the E(3) ((3)-qquivariance proof,
yy - xx) / pT---1 pJ Ply x) (Sy - Sx) PT P(Sy - S
establishing p(Sy Sa) = ply xt).
B.3 Permutation Equivariance for General Molecular Distributions
Again, aim to show that choice of B& satisfies the permuutation-cquiiariinc constraint under the parametrization proposed in Section 4.3
Permutation equivariance. Let SM denote any permutation matrix that reorders atoms within each label class (i.c., Lsu(() = L4). Analogous to the proof Scction B.2 because any permutation matrix commutes with the CoM projector, this condition automatically satistied. All that remains is to prove the invariance of the covariance. Because the two parameterizations above depend on the indices only through the labels, we have
SM (n(n) She (n)
Consequently, the lifted covariance
En(n) = (n @ In
satisfics
S (E>(n) = (n)
for every block permutation S = SM & Ins so the reverse kernel remains permutation-cquivariiat by the argument of the above section.
C Implementation Details
All experiments wall-clock hours on that device.
single NVIDIA A100 GPU (80 GB); hence every "GPU hour" refers to
C.1 Implementation of the objective function
Recall the objective (the a-divergence with a = 2):
C(Φ) = Ey ((xoNN)
M
gl
(m)
M M
mm))
P&s (xx:N)
M
(m))
P& X
m=
(m) where Nu.N g. w.(m)
(mm) Pa (1n)))
and M is the mumber of samples used to estimate the objective.
In practice, optimize the log of the a-divergence for numerical stability, which can be computed efficiently using the log-importance weights via the logsumexp trick. Specifically,
I(Φ) log L(Φ) log
Σ
exp(log (m))) logsumexpl (log (m)) M ( llog M.
(25)
M
Hence, work entirely in log space, which yields a numerically stable estimator of the loss.
18

---

Published in Transactions on Machine Learning Research (10/2025)
Table 2: Score-modes training time on a single A100.
Dataset
Architecture
Layers
Hidden dim
Batch Size
Iterations
GPU-h
GMM-2 (d-50)
MLP
512
5000
100 000
0.6
GMM-2 (d=100)
MLP
512
5000
100 000
1.0
DW-4
EGNN
4
128
512
100 000
0.5
LJ-13
EGNN
6
128
512
100 000
8.0
Alanine dipeptide
EGNN
256
512
100 000
8.0
C.2 Datasets
GMM-2. two-component Gaussian mixture used as a toy benchmark. Training and evaluation samples are drawn directly from the target distribution.
DW-4 and LJ-13. We use the 10 configurations released by Klein et al. 2023)) subset of 10 configurations trains the score model and VT-DIS; the remainder forms the test set.
Alanine dipeptide. We adopt the molecular dataset of Midgley et al. 2023 which provides 10 training samples and 10 test samples.
C.3 Training the Score Model
Our training pipeline-inchieee parameterization and preconditioning-fnn follows Karras et al. 2022
GMM-2. seven-layer MLP (hidden width 512), identical the network used by Alhhuud-Sadeee et al. (2024) for the GMM-40 task. Training runs for 100,000 iterations with batch size of 1,024 for both 50 and d= 100.
DW-4. EGNN backbone with 4 layers and hidden width of 128.
LJ-13. EGNN backbone with layers and hidden width of 128.
Alanine Dipeptide. EGNN backbone with 7 layers and a hidden width 256; node features include atom types and bond information. Scaling atomic coordinates by 10 during training improves sample quality; network outputs are divided by 10 inference.
All models are trained for 100,000 iterations with batch size of 512 using the Adam optimizer, learning rate of 0.001, and cosine anncaling schedule with minimum learning rate of x 10-6, Table 2 these details.
C.4 VT-DIS Post-training
We follow Algorithm exxactly. With the score network frozen, optimize the per-step covariance matrices by minimizing the a-divergence (a 2) between the forward and reverse processes using the same training set as for score learning. Each run comprises 5,000 optimization steps with batch size of 512 (256 for alanine dipeptide). All runs use the Adam optimizer with a learning rate of 0.01 and a cosine annealing schedule with a minimum learning rate of x 10-6 Table summarizes these settings.
The reported training time corresponds to tuning the isotropic covariance; diagonal and full covariance forms incur similar runtimes. We fix the number of time steps to 100; empirically, with fixed epochs and batch sizes, training time increases linearly with the number of sampling steps.
19

---

Published in Transactions on Machine Learning Research (10/2025)
Table VT-DIS post-training time on a single A100.
Dataset
Batch Size
Iterations
Steps K
GPU-h
GMM-2 (d 50)
512
5 000
100
1.8
GMM-2 (d 100)
512
5 000
100
3.5
DW-4
512
5 000
100
0.9
LJ-13
512
5 000
100
2.7
Alanine dipeptide
128
5 000
100
4.3
Table 4: VT-DIS sampling time on a single A100.
Dataset
Batch Size
Steps K
GPU-min
GMM-2 (d 50)
100,000
100
0.25
GMM-2 (d 100)
100,000
100
0.5
DW-4
100,000
100
1.3
LJ-13
100,000
100
4.3
Alanine dipeptide
10,000
100
3
C.5
Evaluation Metrics
We measure efficiency using forward and reverse effective sample sizes (ESS), defined in Eqs. and 12 The number of Monte Carlo draws for each estimate is specified in the corresponding figure or table caption. Table 4 summarizes the sampling time for each larget distribution. Empirically, sampling time increases linearly with the number of sampling steps.
D Additional Experimental Results
D.1 Computational Cost for Solving Probability Flow ODE
In this section, we compare effective sample size (ESS) and inference-time computational cost for an addi- tional bascline: PF-ODE with JVP-based trace estimation. Table summarizes the results. Using Hutchin- son's trace estimator reduces computational cost; however, with single probe vector the ESS drops sharply, whereas using 10 probe vectors increases both cost and ESS. VT-DIS retains the lowest inference-time time cost among all baselines considered. We emphasize that although Hutchinson's estimator is unbiased for the in- stantancous divergence, using stochastic divergence estimates to compute importance weights can introduce bias in self-normalized IS estimates. Consistent with prior work on these datasets (e.g., (Klein et al. 20233 Tan et al. 2025g), ) our main-text comparison computes exact divergences when applying IS to avoid this bias. Accordingly, we caution against using stochastic trace estimators for IS and explicitly flag this caveat in the trrce results.
D.2 Optimized Covariance Matrices
Figure i contrasts the per-step variance-scaling factor of VT-DIS, 7h (see G with the unit baseline of the original DDPM. Although we display the isotropic case, the same qualitative pattern appears for diagonal and tonk covariances.
Three regimes. Reading the (-axis from left. (L> 10-3) to right (1> 102) follows the reverse SDE from data space to pure noise. The learned profile is not monotone but exhibits three characteristic regimes:
1. Data dominated regime S 10-1). Here the signal-to-noise to-no ratio (SNR) high, the score network is the main source of model error, and under-dispersion relative to the true forward kernel results in heavy
20

---

of the Published in Transactions on Machine Learning Research (10/2025)
Table 5: Performance of importance sampling with diffusion on DW-4, LJ-13 and ALDP with 10 samples on a single A100 GPU. The number in parentheses after each dataset denotes the sampling steps. Methods: reverse SDE (VT-DIS) or PF ODE (exact or Hutchinson with probe vectors). higher better, lower is better.
Dataset (steps)
Method
Forward ESS (%) f Reverse ESS (%) t GFLOP sample +
SDE (Ours)
44.32
46.76
0.81
ODE (Exact)
94.29
83.60
7.70
DW-4 (100)
ODE EEtt. n=1)1
37.70
0.95
1.56
ODE Esst n=10)
80.60
82.67
8.27
SDE (Ours)
15.87
17.69
26.20
ODE (Exact)
79.21
77.42
2173.00
LJ-13 (200)
ODE (Est. n=1)
12.35
9.50
52.26
ODE (Est. n=10)
61.12
66.10
285.62
SDE (Ours)
0.00
1.05
33.70
ODE (Exact)
0.00
35.39
45 967.00
ALDP (200)
ODE (Est. n=1)t
0.00
0.74
702.78
ODE (Est. 1=10)
0.00
22.31
3850.39
Hutchinson trace estimator with probe vectors; introduces bias in importance sampling and can misstate ESS.
VT-DIS Optimal Parameters (DW-4)
VT-DIS Optimal Parameters (LJ-13)
GO000000000oo
Tuned Variance
Tuned Variance
1.5
16
DDPM Variance
DDPM Variance
1.4
1.3
1.2
1.2
1.0
3000
1.0
10
10
10"
10
10
10
10
10"
10
10a
Time Step
Time Step
(a) DW-4 (40 steps)
(b) LJ-13 (50 steps)
Figure 6: Optimal variance scaling The VT-DIS inflates early-time variance to improve coverage, adapts non-monotonically across the transition regime, and converges to the DDPM value (7,, = 1, red dashed line) in the pure-noise limit.
importanco-woightt tails. VT-DIS compensates by inflating the variance (7n ss 1.6-1.7), which widens the proposal and improves coverage.
Transition regime (10-1 10t). As the SNR drops, the target conditional gradually morphs from highly Gan-Gaussian to approximately Gaussian. The mismatch between the analytic DDPM variance (optimal for the Gaussian limit) and the true kernel therefore decreases. VT-DIS first reduces the scaling (a local minimum at t xz 0.3), then overshoots to a peak at =z 1, before descending. The intermediate bump arises where the lcarned score Do tends to over-estimate the gradient magnitude (a common artefact of score matching when the SNR is intermediate, e.g. O(1)) (Xu et al. 2023). Increasing the proposal variance temporarily counter- counter-balances this over-confidence, reducing the importance weights.
Noise regime (t 2 10!). The state distribution now close to (N0 for which the DDPM variance is eractly optimal. Accordingly, 7n relaxes 1, confirming that VT-DIS does not perturb steps that are already statistically efficient.
21

---

Published in Transactions on Machine Learning Research (10/2025)
ESS (Mean StdDev) for DW-4
30
25
ESS (%)
20
15
10
10a
10'
10²
Number of Samples
Figure 7: KL (a = 1) vs. a= divergence on DW-4 a tuning (red) attains higher, size-independent ESS, whereas KL tuning (blue) collapses for large M.
Implications. The variance inflations at early steps crucial: without them, trajectories started near the data manifold place negligible mass in regions where the forward kernel has significant probability, causing weight explosion and the near-zero forward ESS observed for the vanilla baseline. Conversely, keeping %, in the noise regime ensures that VT-DIS retains the favourable properties of the original DDPM sampler at large time steps, adding no unnccessary dispersion..
D.3 KL versus a-divergence
The VT-DIS objective in 10 minimiscs the a-divergence with a = 2, which is proportional to the variance of importance weights and therefore maximizes ESS directly (Midgley el al. 2022). For completeness we retrain the covariance parameters on DW-4 using the forward KL divergence (a = 1) and compare the resulting samplers.
Figure 7 plots forward (solid) and reverse (dashed) ESS functions of the sample size M. Tuning with 2 yields consistently higher and stable ESS in both directions. In contrast, KL-tuned parameters give (i) low forward ESS and (ii) reverse ESS that decreases with M, a classic symptom of heavy weight tails caused by poor proposal coverage. These observations confirm that controlling the second moment of weights (a = 2) is crucial for robust importance sampling, whereas KT. divergence alone fails to regularise the tails.
E Broader Impacts
Our method enhances the efficiency and accessibility of unbiased molecular sampling, potentially speeding up the discovery of catalysts, drugs, and new materials while reducing reliance on costly experiments. At the same time, stronger samplers could be misused to design toxic or environmentally harmful compounds more rapidly. There is also risk that institutions with advanced computing resources will widen the gap with those lacking such infrastructure, reinforcing inequities in scientific research.
22


Reviews:
[{"review_id": "5NT2OySUEv", "reviewer": "Reviewer_rUzh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper addresses bias from imperfect score estimates in diffusion models and proposes a debiasing method to recover equilibrium samples from a Boltzmann distribution. They introduce VT-DIS, a lightweight post-training procedure that freezes the score network and adapts per-step noise covariances. By minimizing an α-divergence (α = 2) between path measures, VT-DIS achieves a higher ESS. The method incorporates E(3)-equivariant diffusion models, allowing the modeling of molecular samples. On standard benchmarks (DW-4, LJ-13, ALDP), VT-DIS demonstrates markedly higher ESS than naïve diffusion+IS under a similar compute budget.\n\n**Strengths:**\n1. This paper tackles an important and practical problem of equilibrium sample generation with clear applications in molecular modeling.\n2. The setting considered is practical and realistic: both data and energy functions are available, but the score learning may be biased, necessitating debiasing at the post-training or inference stage.\n3. The use of $\\alpha=2$ divergence is well motivated, as it directly relates to the variance of importance weights and thereby promotes high ESS.\n4. The approach is practical, as it keeps the score network fixed and adapts only the noise covariance, allowing for easy adoption by practitioners with existing models.\n\n**Weaknesses:**\n1. Incomplete empirical validation: (i) ESS alone does not fully capture sample quality or mode coverage, and (ii) VT-DIS should be compared against PF-ODE + IS + trace estimation.\n2. Lacks clear rationale for noise covariance tuning, leaving open questions about alternatives.\n3. The time required for post-training seems substantial, so the claim that it is \"lightweight\" may be overstated.", "claims_and_evidence_yn": "No", "claims_explanation": "Although this paper addresses important problems in statistical physics and the proposed method appears promising, more empirical analysis is required to demonstrate both the efficiency and unbiasedness of VT-DIS. The reported metric is insufficient to support the claims fully, and the lack of comparison with PF-ODE + IS + trace estimation obscures the true efficiency of the method. Furthermore, the post-training stage appears computationally demanding (comparable to the original training time), so the claim that the procedure is \"lightweight\" may be overstated.", "audience_yn": "Yes", "audience_explanation": "The paper would likely be of interest to certain members of the TMLR audience. Generating equilibrium samples from Boltzmann distributions is central to in-silico thermodynamic property prediction for drugs and materials. The audience focused on ML for molecular systems, or the Boltzmann generator, would find this paper interesting.", "requested_changes": "1. **Additional evaluation metrics (especially for ALDP)**: For most tasks, only ESS is reported. However, as noted by Blessing et al. (2024), forward ESS tends to approach zero in higher-dimensional problems, making it a weak evaluation metric, while reverse ESS does not capture mode collapse. While ESS is certainly a representative metric directly related to the bias and variance of importance sampling, incorporating additional metrics (such as ELBO, EUBO, sample-based quality metrics, or energy distribution distances) would provide a more well-rounded evaluation. This could be especially useful for ALDP, where the gain in forward ESS is not evident.\n\n2. **Comparison with PF-ODE + IS + trace estimation**: To better assess the efficiency of VT-DIS, it would be valuable to include a comparison with PF-ODE using trace estimation, as this method may achieve a significantly lower cost while still maintaining high ESS. In practice, trace estimation via JVP typically adds only ~1.2× the cost of a forward pass, so such a comparison could give readers a clearer understanding of the trade-offs.\n\n3. **Clarify the rationale for noise covariance tuning**: The motivation for tuning the noise covariance is not fully clear. For example, instead of adjusting the noise covariance, one could fine-tune the score model directly with an $\\alpha=2$ divergence objective. Furthermore, when adapting the noise covariance of the forward process, the marginal density and score of the forward process also change. Should the marginal score be adapted as well?\n\n4. **Clarify the claim of \"lightweight\" post-training**: The current results suggest that post-training requires substantial compute, comparable to the original training time. It may therefore strengthen the paper to either provide additional evidence supporting the “lightweight” claim, or to clearly distinguish between the low inference overhead and the cost of post-training.\n\nReference: \n[1] Blessing et al., 2024. Beyond ELBOs: A large-scale evaluatoin of variational methods for sampling", "broader_impact_concerns": "I do not see additional broader-impact risks beyond those already identified by the authors.", "additional_comments": "1. The isotropic noise tuning results appear promising (except on the synthetic GMM task). Could the noise schedule itself be treated as a hyperparameter to be tuned? While I understand that learning the noise covariance may reduce hyperparameter search time, it is not clear to me whether similar ESS gains could be achieved simply by selecting an appropriate schedule in advance.\n\n2. The $\\alpha=2$ divergence involves density ratios, which are often high-variance and can cause numerical instability. Is minimizing $\\alpha=2$ divergence numerically stable in practice?\n\n3. The description of the toy dataset GMM-2 is unclear. Is it defined as a product of 1D distributions? In addition, there is a typo: $\\mu_1, \\mu_2 \\in \\mathbb{R}^d$ should be vectors rather than a scalar 1 or -2.\n\n4. The abbreviation GFLOPs is used without first introducing the full term.", "extra_scores": {}, "extra_text": {}}, {"review_id": "UnRajicVJW", "reviewer": "Reviewer_MrQi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Variance-Tuned Diffusion Importance Sampling (VT-DIS), a practical and lightweight post-training method to enable unbiased sampling from Boltzmann distributions using pretrained score-based diffusion models (SBDMs). The core contribution is a procedure to correct a SBDM sampler by freezing the score network and optimizing the per-step noise covariances of the reverse SDE process. The optimization objective is uniquely chosen as the \\alpha=2 divergence between the forward and reverse trajectory distributions, which can be shown to be equivalent to minimizing the variance of the importance sampling weights, thereby directly maximizing the Effective Sample Size (ESS). In other words, the proposed post-training correct directly optimizes the variance of the importance sampling weights, thereby allowing a more efficient IS.\n\n1) Key Strengths:\n- Practical approach: the method is a simple, post-training step that makes unbiased estimation from powerful diffusion models computationally feasible, offering a compelling trade-off between statistical efficiency and cost.\n- Sound methodology: the choice of the \\alpha=2 divergence is theoretically well-justified and empirically validated against the more common KL divergence (\\alpha=1), showing superior stability and performance for importance sampling.\n- Good empirical results: the paper demonstrates ESS improvements across various benchmarks, from synthetic data to certain molecular systems, with a empirical cost evaluation against PF-ODE approach.\n\n2) Key Weaknesses:\n- Limited Performance on certain complex scenarios: for the most complex benchmark, alanine dipeptide, the forward ESS remains near zero, suggesting limitations in the method's ability to ensure full coverage of the target distribution for challenging systems.\n- Practical dependencies: the method requires access to the original training data for the post-training step,", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper’s central claims are well-supported by a combination of theoretical reasoning and strong empirical evidence.\n\n- Claim: VT-DIS yields unbiased expectation estimates with high ESS. The \"unbiased\" property rests on the established theory of importance sampling, which is asymptotically unbiased. The \"high ESS\" claim is demonstrated in Figures 1 and 2, where VT-DIS outperforms the baseline DDPM+IS method, often by an order of magnitude.\n\n- Claim: negligible overhead compared to alternatives. This is supported by Table 1, which shows that VT-DIS requires 1-2 orders of magnitude fewer GFLOPs per sample than the PF-ODE approach for exact likelihoods. The overhead claim specifically about inference time is accurate.\n\n- Claim: \\alpha=2 objective is crucial for performance. This is justified both theoretically (by its equivalence to minimizing importance weight variance)  and empirically. The ablation study in Appendix D.2 and Figure 6 clearly shows that tuning with  \\alpha=1 (KL divergence) leads to a collapsing ESS, while \\alpha=2 provides stable and robust results, directly validating this key methodological choice.\n\n- Claim: the method works for equivariant models. The authors provide a sound theoretical framework for extending the covariance tuning to respect E(3) and permutation symmetries , and then demonstrate it successfully on relevant molecular benchmarks (DW-4, LJ-13, ALDP).", "audience_yn": "Yes", "audience_explanation": "The findings of this paper would be of high interest to several groups within the TMLR audience:\n\n1) Researchers in generative modeling: this paper provides a novel and practical solution to the bias of learned samplers in diffusion models. The insight of tuning noise covariances via the $\\alpha-$divergence (with $\\alpha=2$) to improve importance sampling is a valuable contribution to the broader literature on diffusion models and their properties.\n\n2) ML practitioners in scientific domains: the primary application is sampling from Boltzmann distributions, a central task in computational chemistry, physics, and materials science. The ability to generate unbiased samples from complex, high-dimensional energy surfaces efficiently makes this work relevant to anyone applying ML to scientific discovery.\n\n3) Researchers in monte carlo methods: the paper sits at the intersection of generative modeling and statistical computation. Its focus on improving the efficiency of importance sampling by leveraging a learned proposal will be of interest to statisticians and computer scientists working on sampling algorithms.", "requested_changes": "Here is a list of proposed adjustments. I have marked them as \"critical\" for acceptance or as suggestions that would \"strengthen\" the work:\n1) Refine the \"negligible overhead\" claim (strengthen): the abstract and introduction claim \"negligible overhead\". This is true for inference , but the post-training step itself has a non-trivial computational cost (Table 3 shows several GPU hours). Please clarify this by specifying \"negligible inference-time overhead\" to more accurately represent the method's cost profile.\n2) Expand the discussion on low forward ESS for ALDP (critical): the near-zero forward ESS on alanine dipeptide is a significant negative result that is currently understated. Please add a more thorough discussion in the limitations section. Does this point to a fundamental limitation of the score model itself, or is it a failure of the covariance tuning? Acknowledging and interpreting this result more deeply is crucial for understanding the method's boundaries.\n3) Explicitly state data dependency as a limitation (strengthen): the method requires the original training data for the post-training step. Please add a sentence to the limitations section explicitly stating this dependency, as practitioners may sometimes only have access to a pretrained model without its training set.\n4) Clarify the objective function's relation to formal divergences (Strengthen): while not strictly necessary, the paper could be slightly clearer that the objective function \\int (q^2/p) dx is the core term of the formal definition of $\\alpha-$divergence (\\alpha=2), and minimizing it is equivalent to minimizing the formal divergence due to their linear relationship. A brief footnote clarifying this could help readers who are experts in information geometry.", "broader_impact_concerns": "The authors have included a Broader Impact Statement in Appendix E, which is sufficient. The statement is balanced and addresses the key ethical considerations adequately.", "additional_comments": "This is an excellent paper that presents a simple, clever, and highly effective method for a significant problem. The work is well-executed, the experiments are convincing, and the writing is clear. The connection made between the \\alpha=2 divergence and the variance of importance weights is a particularly insightful contribution. After addressing the critical point about the low forward ESS on ALDP, I believe this paper will be a strong and valuable addition to the literature.", "extra_scores": {}, "extra_text": {}}, {"review_id": "QqlqHcTPTy", "reviewer": "Reviewer_JGyc", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The main contribution of this paper is a connection of SBDMs with sampling from Boltzmann densities. It shows that naïve use of this connection can yield low ESS due to approximation biases (e.g, time discretization and biased divergence estimates). To address this, the authors propose VT-PIS, a post-training procedure that performs trajectory-wise importance sampling with negligible overhead and integrate VT-PIS into E(3)-equivariant DMs to enable effective molecular generation. Experiments on toy problem and molecular benchmarks demonstrate that VT-PIS effectively increases ESS over naïve diffusion baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The derivations clearly motivate why bias arises in practice, and the proposed trajectory-wise importance sampling method is described precisely enough to assess its correctness. Empirically, experiments on various datasets consistently show higher ESS than naïve baseline  aligning with the method’s design intent.", "audience_yn": "Yes", "audience_explanation": "The paper is likely to attract interest within TMLR, as it bridges SBDMs and Boltzmann sampling, communities spanning generative modeling, probabilistic inference, and molecular ML. While the empirical comparisons are limited, the ideas presented are concrete enough to motivate follow-up studies and provide actionable guidance for practitioners.", "requested_changes": "* The paper claims that no neural samplers perform well on molecular data, yet several works [1,2] report results on systems such as DW-4 and LJ-13. Compared to this paper would strengthen the empirical support. \n\n* The current formulation appears to require access to samples from the very Boltzmann distribution one aims to sample from. In regimes where such samples are hard or infeasible to obtain, this assumption limits practical utility. It would better the proposed VT-PIS might remain effective even when only approximate reference samples are available. \n\n\n----\n    [1] Havens et al., Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matching\n    [2] Akhound-Sadegh et al., Iterated Denoising Energy Matching for Sampling from Boltzmann Densities", "broader_impact_concerns": "N/A", "additional_comments": "* Score-based diffusion models” are abbreviated as SBDMs in the abstract but as DMs in Section 3.1.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)