Progressive Tempering Sampler with Diffusion
Authors: Severi Rissanen
RuiKang OuYang
Jiajun He
Wenlin Chen
Markus Heinonen
Arno Solin
José Miguel Hernández-Lobato
Venue: ICML 2025 poster (2025)
Keywords: parallel tempering
neural samplers
sampling from unnormalised densities
Primary area: probabilistic_methods->monte_carlo_and_sampling_methods
TL;DR: We design a diffusion based neural sampler by progressively reducing the temperature with a guidance term.

Abstract: Recent research has focused on designing neural samplers that amortize the process of sampling from unnormalized densities. 
However, despite significant advancements, they still fall short of the state-of-the-art MCMC approach, Parallel Tempering (PT), when it comes to the efficiency of target evaluations. On the other hand, unlike a well-trained neural sampler, PT yields only dependent samples and needs to be rerun---at considerable computational cost---whenever new samples are required. To address these weaknesses, we propose the Progressive Tempering Sampler with Diffusion (PTSD), which trains diffusion models sequentially across temperatures, leveraging the advantages of PT to improve the training of neural samplers. We also introduce a novel method to combine high-temperature diffusion models to generate approximate lower-temperature samples, which are minimally refined using MCMC and used to train the next diffusion model. PTSD enables efficient reuse of sample information across temperature levels while generating well-mixed, uncorrelated samples. Our method significantly improves target evaluation efficiency,
outperforming diffusion-based neural samplers.

Progressive Tempering Sampler with Diffusion
Severi Rissanen"
RuiKang OuYang" 2 Jiajun He Wenlin Chen
Markus Heinonen'
Arno Solin'
José Miguel Hernández-Lobato
Abstract
BNEM IDEM
Recent research has focused on designing neu- ral samplers that amortize the process of sam- pling from unnormalized densities. However, de- spite significant advancements, they still fall short of the state-of-the-art MCMC approach, Parallel Tempering (PT), when it comes to the efficiency of target evaluations. On the other hand, unlike a well-trained neural sampler, PT yields only depen- dent samples and needs to be rerun- at consider- able computational cost-whenever new samples are required. To address these weaknesses, we propose the Progressive Tempering Sampler with Diffusion (PTSD), which trains diffusion models sequentially across temperatures, leveraging the advantages of PT improve the training of neu- ral samplers. We also introduce novel method to combine high-temperature diffusion models to generate approximate wwwr-teeperatuur samples, which are minimally refined using MCMC and used to train the next diffusion model. PTSD en- ables efficient reuse of sample information across temperature levels while generating well-mixed, uncorrelated samples. Our method significantly improves target evaluation efficiency, outperform- ing diffusion- based neural samplers.
8
DDS X CMCD
Sample emor (Wa)
7
Improved error and eval. count
6
PT+DM (baseline)
5
PTSD
(ours)
10"
107
10a
10
1020
10a
Target evaluation count
Figure 1. Sample error (W2 distance) and target evaluation times for several diffusion (and control) bbssed neural samplers on Many- Well-32 target, including DDS, iDEM, BNEM, and CMCD, with our proposed approach. We include the results obtained by first rumning PT and fit diffusion model post hoc for comparison.
(Noé al., 2019). We aim to draw independent samples from probability distribution with density:
p(x) = p(z) Z
s.t. Z z p(x) dz,
(I)
where we only assume access to the unnormalized density function p(x) without any ground-truit observations.
1. Introduction
The classical way to sample from the target density p Markov chain Monte Carlo (MCMC), where we design Markov chain whose invariant density is p, with the state- of-the-ar art method being Parallel Tempering (PT, Swend- sen & Wang, 1986; Geyer, 1991; Hukushima & Nemoto, 1996), also known as replica exchange. The key to PT's success lies running parallel MCMC chains at tem- tempering
Sampling from probability distributions is a fundamental problem in many fields of science, including Bayesian infer- ence (Gelman et al., 2013; Welling & Teh, 2011), statistical physics (Von Toussaint, 2011) and molecular simulations
"qqual contribution 'Deaarmeett of Computer Science, Aalto University, Finland 'Department of Engineering, Univer- sity of Cambridge, United Kingdom 'Department of Empir- ical Inference, Max Planck Institute for Intelligent Systems, Tübingen, Germany. Correspondence to: Severi Rissanen <sev- eri. www.aissnnnneeee aalto.foi RuiKang Ou Yang <ro352 cam.ac.uk."" Jiajun He <jh2383 @ww..ac.uu...............
peratures Ti..... TK. each corresponding version PT, x p p'/Tx of the original target. The chains at higher temperatures
easier to traverse across the entire support, and samples in those chains are swapped, facili- tating better mixing in low temperatures (Woodard et al., 2009). We also note that other choices of path exist, for example, geometric interpolant path pk xc "** 1-8 with a tractable reference Pa and 3g ranging from 0 to 1, or other
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Progressive Tempering Sampler with Diffusion
more flexible designs (Syed et al., 2021; Surjanovic al., 2022). In this paper, we follow the convention in most practical applications to use the annealing path.
through PT and defer model fitting to the final stage.
Therefore, a natural question is to integrate PT and neural cumplers--psitioon in the middle of the methodological geeetrum-leveragiin the strengths of both to build more efficient sampling approaches. In this paper, we formalize one attempt towards this direction. Concretely, we propose Progressive Tempering Sampler with Diffusion (PTSD)'. Our contributions are as follows:
However, despite significant advancements in PT. obtaining a new, independent sample requires an independent sam- ple from the highest temperature to be propagated to the lowest temperature (Surjanovic al.. 2024). Consequently, whenever new samples are needed, we need run PT until uncorrelated samples have been obtained at the highest tem- perature and transferred to the target, which might require considerable cost. Therefore, a growing trend in research has focused on learned neural samplers, which aim to amor- tize the sampling process and generate uncorrelated samples directly. Early approaches involved fitting normalizing flows to data generated by MCMC (Noé et al., 2019), while more recent efforts typically focus training generative models directly using only the target unnormalized density. Of par- ticular recent interest are diffusion- based neural samplers, driven by the remarkable success of diffusion models (Ho et al., 2020; Song et al., 2021) in generation tasks. How- ever, a key factor behind the success of diffusion models in these fields lies in their simple and stable denoising score matching (DSM) objective (Vincent, 201 I), which relies on access to ground truth data from the target distribution. As result, while diffusion models demonstrate impressive performance in generation tasks, translating this success to the problem of learning neural samplers directly from unnormalized densities remains challenging.
1.
We introduce novel guidance mechanism that enables diffusion models, trained to generate samples that approximate those from a lower-temperature distribution.
data,
2. Using this guidance term, we propose Progressive Tem- pering Sampler with Diffusion (PTSD). It fits diffusion models sequentially on higher temperatures and gener- ates samples for lower temperatures using our proposed guidance, allowing the information at higher tempera- tures to transfer lower ones efficiently.
We evaluate PTSD on a variety of targets, demonstrat- ing its effectiveness. Our approach achieves orders-of- magninude improvement in target density evaluation efficiency compared to other diffusion-based neural samplers, demonstrating great potential in integrating standard sampling algorithms and neural samplers.
2. Background
To illustrate this challenge, in Fig. 1, we compare several recent diffusion and control-based neural samplers, includ- ing DDS ( (Vargas et al., 2023), iDEM (Akhound-Sadegh et al., 2024), BNEM (Ou Yang et al.. 2024), and CMCD (Vargas et al., 2024), on the Many-Well-S3 target (Midg- ley et al., 2023), alongside the results by running PT and fitting a diffusion model post hoc to PT-generated data. Sadly, while showing promising sample qualities, these ap- proaches present significantly lower efficiency compared to directly fitting the diffusion to PT results. This inefficiency arises either from running importance sampling (IS) or an- nealed importance sampling (AIS) to estimate the objectives (Akhound- -Sauud et al., 2024; Ou Yang et al., 2024), or from the extensive number of target evaluations in simulating the trajectory (Vargas et al., 2023; 2024) due to the prevalent Langevin preconditioning in the network parameterization (He et al., 2025).
Markov chain Monte Carlo and Parallel Tempering Markov Chain Monte Carlo (MCMC) is a family of al- gorithms designed sample from an unnormalized density function by constructing Markov chain whose station- ary distribution matches the target. Representative MCMC methods include Gibbs Sampling (Geman & Geman, 1984), the Metropolis-Hasting (MH) algorithm (Metropolis et al., 1953; Hastings, 1970), the MetropollisAAjjusee Langevin Algorithm (MALA, Grenander & Miller, 1994), and Hamil- tonian Monte Carlo (HMC, Duane et al., 1987).
However, for multi-modal distributions, MCMC can easily get stuck in single mode, failing to explore the entire sup- port effectively in practice (Neal, 1993). Parallel Tempering (PT, Swendsen & Wang, 1986; Geyer, 1991; Hukushima & Nemoto, 1996) addresses this issue by introducing a se- quence of temperatures, TK > Tk-1 > ... > Ti, where Ti corresponds to the original target distribution. PT runs multiple Markov chains in parallel, each sampling from tempered version of the target distribution, defined as PT & p/TT,, as illustrated in Fig. 2. MCMC at high temper- atures can traverse between modes more efficiently, while
Does this imply that using PT followed by fitting diffusion model is the ultimate solution,or neural samplers? We posit that this is not the case. In fact, the methods shown in Fig. represent two ends of the methodological spectrum. On one end, approaches such as DDS, iDEM, BNEM, CMCD, aim to train neural samplers without utilizing any data, while on the other, methods depend exclusively on data generated
'The code for the paper will be available http:///www...oooe///////eeeeeee Sampler-with-Diffee Difffusiii.......
2

---

Progressive Tempering Sampler with Diffusion
Tempered distributions (2)
matching (Vincent, 2011). During sampling, we start with samples from (I |0, ) and follow Eq. (4).
Density
=
samples
Additionally, the score function is related to the denoising mean through Tweedie's formula (Efron, 2011; 1; Roberts & Tweedie, 1996):
Donsity
swaps
T 10
Vx, log = (E [o ma] =x)/
(5)
Therefore, for numerical stability, rather than directly ap- proximating the score function with a neural network, a common choice is regress the denoising mean using a denoiser network (x, t), done by Karras et al. (2022).
Density
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
1.2
PF-ODE and Hutchinson's trace estimator Besides the reverse SDE defined in Eq. (4), diffusion models (DMs) can also generate samples by the probabilistic flow (PF) ODE:
Figure 2. Illustration of parallel tempering with three temperatures.
da, = (()) logp:(x;) dl,
Piuss (6)
MCMC at lower temperatures explores local modes and provides unbiased samples from the target distribution. In- formation between different temperatures is shared by swap- ping samples. Specifically, for samples x; and z, at adjacent temperatures T; and Tj. PT swaps them with probability
This formulation not only provides an alternative method for sample generation but also offers a principled approach to estimating the log density of the generated samples. Con- cretely, by employing the instantaneous change-of-variables formula (Chen et al., 2018),
obtain
p= p=min min (1 ((e)))
d log p((x)) = (1)) tr(V?, log),(zz)) dt
and the Jacobian can be approximated by Hutchinson's 's trace estimator (Hutchinson, 1989; Grathwohl et al., 2018)
Diffuusion models DMs Soch--Dickstein et al., 2015; Ho al., 2020; Song et al., 2021) define a forward process cor- rupting the original data distribution with Gaussian noise, which corresponds to a diffusion SDE. We can generate samples by denoising progressively, corresponding to the time-reversal of the forward SDE.
tr( rv( m1 log Pe (x+)) E, VV (x:)c (8)
where follows a Rademacher distribution (Hutchinson, 1989). We can approximate the expectation using Monte Carlo integration, with vector-Jacobian products (VJP) en- abling efficient computation.
Assume target density po(x)) We define a noising pro- cess?
dr, = (t) dw,, Io ~ Po
(3)
3. Methods
where W, is a Wiener process. Its time reversal is given by
In this section, we describe our approach, Progressive Tem- pering Sampler with Diffusion (PTSD). Unlike other meth- ods shown in Fig. 1, integrates Parallel Tempering (PT) and neural samplers to achieve more efficient utilization of target energy evaluations.
(4) dz,= (o)=-2o(to(tt (x dt (t (2 du,
where Fi.. ~ Pr and V., loggt((2) (1:)) is the score function at diffusion time I, and W. is a reverse time Wiener process. The (a(e)) is the target density convolved with Gaussian kernel (x())) = [z; I (t)² 1) po dro.
In spite of the advances by PT, it can only generate inde- pendent samples when (1) uncorrected samples are drawn at the highest temperature, and (2) the uncorrected samples are propagated to the lowest temperature. Therefore, the efficiency of PT will highly rely on the quality of local ex- ploration and the swapping. On the other hand, although the MCMC chain can mix faster at higher temperatures, ob- taining an uncorrected sample still requires a considerable number of steps. Also, while the (unnormalized) densi- ties at different temperatures differ only their exponents
For sufficiently large o (1max). ). Ptasx is well approximated by Gaussian (xx [0, ( and hence is tractable. During training, we approximate the score Vz, log (rre with a -emendepnnd neural network by denoising score
For simplicity, we only discuss the exipoiceexploding process introduced by Song al. (2021): Karras et al. (2022) in this paper. However, we note that our proposed approach is also compatible with the variance-preserving process.
3

---

Progressive Tempering Sampler with Diffusion
the score function Va, l (r T) for L E [0, max and T E [Ti, T2). Now, we want to generate samples at lower temperature, To < Ti, which requires knowing the score function Vx, log Pt (
Tw
Tg-1
Tx-1
TR-2
init
0k
0 0K-1
0x
0k-1
0x-2
At t 0, this is simple be- cause, by definition, the scores at different temperatures are related by V.n log Pu( To) To = V Vao log Po Ti) -Ti. However, for > 0, this relationship no longer holds gen- eral due to the complex nature of Gaussian convolution.
DSM
DSM
DSM
DSM
guade
Ex
Bx-1
fig-1
EL-S
Instead, we apply Taylor expansion on score around T1:
Zx,
T) = Va.
T2) Va, logpi 8T
Stage 1: initialize buffer and model highest two temperatures
Stage K 1: sample with guidance and train model temperature T--2
+ (T- - Ti)
T-
Figure 3.
The training process of PTSD. We unroll the training process into a sequence of target temperatures. We first initialize buffers and models at the highest two temperatures and generate samples from lower temperatures sequentially using the tempera- ture guidance to extrapolate lower temperatures.
If the trained model is conditioned on a continuum of tem- peratures T, we could calculate the partial derivative by au- tomatic differentiation. However, this requires training the temperature conditiione diffusion model on sufficiently diverse set of temperatures to ensure robust estimation of the partial derivative.
and hence share global similarity, the swapping is only performed by "copying and pasting" the sample between temperatures, failing to leverage such prior knowledge.
Fortunately, we can approximate the derivative with finite differences:
2 ØT Vn, log (Ie: T2)
Therefore, a natural question is: can we find an algorithm to produce uncorrelated samples for higher temperatures, and more efficiently propagate the samples for higher tem- peratures to lower ones?
log T)
T-T
Vz, logpe(ar, Ti)
(10)
T2 15-11 Tı
Plugging Eq. (10) into Eq. (9), we have
We answer this question affirmatively with learned neural sampler. A well-trained neural sampler one tempera- ture can produce independent samples at cheaper cost. Moreover, the neural sampler can be viewed as a "func- tional representation" of the target density, and sharing the weights of the neural sampler across temperatures offers a more efficient mechanism for transferring information compared to traditional sample swapping.
T/ T
V (x;,
V
Io,,
T2 Ti
Ti T
V., log (ar. T2) (11)
T2 Ti
(1 + V., po Ti) 2,, log (xt, (2),
where (TTi-T/(T2-17) We highlight the similarity between Eq. (11) and the guidance in diffusion models (Ho & Salimans, 2021; Karras et al., 2024), offering an intuitive perspective on this estimator: contrasting a "better", lower-temperature model by guiding the model at current temperature Tj with its "worse", geggeer-empprature version. For this reason, term Eq. (11) temperature guidance.
With these motivations, now turn the details of our approach. We begin by introducing temperature guidance in Sec. 3.1, which is more efficient way to share information between temperatures by enabling the neural sampler trained at higher temperatures to generate approximate samples for a lower temperature. Then, we integrate this guidance term into the full pipeline of PTSD in Sec. 3.2.
One concern on this guidance is that when diffusion time t = 0, the known relation V., xa log (zo, T) T Vz, log Po Ti) Ti = Vzo log T2) T2 does not hold, indicating this approximate guidance is inaccurate when / - O. Fortunately, in diffusion models, the accuracy of the score at small time steps typically has minimal impact on the quality of the generation.
3.1. Temperature guidance
To avoid overloading notation, we use uppercase T to de- note temperatures while reserving lowercase for diffusion time steps. Given the target unnormalized density the normalized density function at temperature T is defined by /1/T, For simplicity, we will use (To, T) to represent the target distribution at temperature T. i.e. po( T' x p1/T" and pr T)= = [Nxz Fo. (t) (no, T) dzg-
Finally, similar to standard guidance methods, we note that temperature guidance is independent of the specific parame- terization of diffusion models. In our experiments, instead of estimating the score function, we follow Karras et al. (2022) to learn the denoising mean using a denoiser.
Assume we have trained diffusion model that approximates

---

Progressive Tempering Sampler with Diffusion
3.2. Progressive Tempering Sampler with Diffusion
Algorithm Training for PTSD
We now explore the use of temperature guidance con- struct a pipeline that integrates neural samplers with parallel tempering (PT), Analogous to PT. we define decreasing sequence of temperatures [Tx. T TıL, where Ti corresponds to the temperature of our target distribution.
Input: Target density ñ. Temperatures (T&))==) Empty
Buffers B = {Bx} jk R=1 Initial parallel tempering (PT)
steps L. Refinement PT steps 1, Truncate quantile т, Train- ing iterations M, Buffer size B.
Output: Model 01.
# Initialize at two highest temperatures: Initialize buffers Bk-1, BR with L steps PT; Train models x-11 0x for M iterations;
With the proposed temperature guidance formulation, we structure our algorithm as follows:
# Progressively decrease the temperature: for & from K do
1. Run PT at the highest two temperatures: To ini- tialize, we run MCMC (specifically, PT with two tem- peratures) at the two highest temperatures, Tk and Tk-1, and collect samples into buffers 3x and Bx-1- The Markov chain at these high temperatures gener- ally easier to explore the support and less likely to get trapped local mode (Earl & Deem, 2005).
# Sample with temperature-guudanee Draw B samples {z,, | for Tk-2 by PF ODE with temperature semperature-guidance, guidance, using models 0k- Op:
# Calculate Truncated IS Weights:
Calculate the IS weights {w, B fa=1 by Eq. (12);
Wmax + 7 quuuntt le ([wn)
2. Fit initial diffusion model: After obtaining sufficient samples at the two highest temperatures, we fit two diffusion models 0g and 0 each temperature".
For 1,... B, set Wn < (Wn, mma)); Renormalize {w,, Jm
# Importance Resample: for i from 1 to B do
3. Draw lower-temperature samples by temperature guidance: We then draw samples T using the temperature guidance method proposed in Sec. 3.1 and store these samples in the buffer Bx
n < Category (f (me, )
Append H, to Bx- end for
4. Fine-tune diffusion model for lower temperature: We initialize 0x -2 +- 0K-1, and fine-tune 0K-1 and 0k-2 using samples in buffer Bx-1 and Bx-2-
# Local PT Refinement:
Refine samples by 1-step PT in BB-2 and BB-1;
# Fine-tune models:
Initialize 0x-2 +- 0g-1;
We repeat steps 3 and 4 until we obtain diffusion models 0,.
Train 0x-2 on Bx-2 for M iterations;
Train 0 on Bx-1 for M iterations;
3.3. Improving Techniques for Training PTSD
end for
While the temperature guidance provides efficient way to guide higher temperature models to sample from lower tem- perature distributions, we should note that it only generates approximate samples. This approximation error accumu- lates when running PTSD with multiple temperature levels, ultimately leading to highly biased samples and models. Luckily, we have access the marginal (unnormalizedd) density of each temperature intermediate steps. This ables us to incorporate importance resampling or MCMC steps to refine the quality of the buffer. We now introduce two techniques in detail.
self-normalized importance weights as
p(Tn) /g(Øn) Σn=1 (( 1/To /q(xw)
(12)
Finally, we resample B instances from these B samples with replacement, where the selection probabilities are pro- portional to W...
This method is generally guaranteed to be unbiased as B + oo and is commonly adopted in Sequential Monte Carlo (SMC, Liu & Chen, 1998). When applying the method with diffusion model proposals, we must be careful, however: the discretization of the PF ODE and the approxi- mation error in Hutchinson's 's trace estimator (Hutchinson, 1989; Grathwohl et al., 2018) introduce unwanted variance in the normalized importance weights. To prevent the sam- pling process from becoming unstable, we employ truncated importance sampling (Ionides, 2008), where the unnormal- ized importance weights are clipped maximum value to
Truncated importance resampling We consider apply- ing importance resampling to intermediate steps at tempera- ture Ti. Specificallyy, we replace the score function in Eq. (6) with the temperature guidance Eq. (11) and generate sam- obtain samples ........ BB along with their corresponding densi- ties q( ( (in log space). We then compute the
ples by following the PF ODE. This allows
FFrmm now on, denote 0g as the parameter of the diffusion model trained at temperature Tg.
5

---

Progressive Tempering Sampler with Diffusion
reduce variance. In practice, we set the truncation threshold at predefined quantile of the importance weights.
sure between the sampling process and a target process. On the other hand, Iterated Denoising Energy Matching (iDEM, Akhound-Sadegh et al., 2024) estimates the score function directly using the target score identity (De Bor- toli et al., 2024) combined with self-normalized importance sampling. Bootstrapped Noised Energy Matching (BNEM, Yu'ang et al., 2024) generalizes this estimator to energy- parameterized diffusion models, while Diffusive KL (DiKL, He et al., 2024) integrates this estimator with variational score distillation techniques (Poole et al., 2023; Luo et al., 2024) to train a one-step generator as the neural sampler.
Local parallel tempering refinement While the trun- cated IS significantly improves sample quality, biases re- main due to the approximation error in Hutchinson's 's trace estimator, the self-normalize importance weights, and the truncation of weights. To address these biases, in addition to IS resampling, we refine the samples by performing several MCMC steps after collecting a buffer. Similar strategies were employed by Sendera et al. (2024); Chen et al. (2024a), where few steps of MCMC were applied to improve sam- ple quality in buffers.
A key property of neu- ral samplers is seif- bootstrapping behavior, as illustrated in Fig. 4. At each training step, the sampler generates samples from itself. Some approaches, such as FAB, iDEM, and BNEM, collect these samples in buffer. We then use these samples to compute the loss and improve the neural sampler. As the sampler improves, it generates samples that are closer to the target distribution. In turn, these higher-quality samples can provide a stronger training signal, further enhancing the neural sampler in self-reinforcing cycle.
Additionally, to improve mixing, rather than running MCMC within a single temperature, we can apply PT be- tween pairs of adjacent temperatures. Specifically, assume a buffer size of B, after collecting buffer B at temper- ature 2 we randomly pair samples B and B to form B pairs. Then, we run B PT processes in parallel, each initialized with pair of samples containing two chains at temperatures 7 and T improving sample qual- ity in both buffers. This approach integrates well with our framework, as we will use both B5-2 and Bx-1 to further fine-tune the diffusion models Ox-2 and 0k-1, by which we extrapolate further to T3
θ
sample
train
B
Figure 4. "Self-bootstrappingg training of neural samplers.
Optionally, we can optimize energy evaluation usage by only running the PT chains from subset of IS samples. The results from the PT chains are then added to the original IS results instead of entirely replacing them. Effectively we augment the IS results with the MCMC samples, which can be enough to get around the issue that IS by itself may result in low effective sample size.
However, this self-bootstraappin behavior can also intro- duce inefficiencies. The improvement occurs only through the training objective. If the current samples fail to provide strong enough signal for the objective to refine the neu- ral sampler, the sampler will repeatedly generate similar samples, leading to wasted computation.
We illustrate the training pipeline in Fig. 3, and detail the pseudo-code in Alg. 1. After training, we sample using the model at the lowest temperature 0 by either the reverse- SDE in Eq. (4) or the PF ODE in Eq. (4).
In contrast, our approach can be seen as bootstrapping across temperatures. We first train models higher tem- peratures, where sampling is easier. Then, we use the high- temperature model to generate samples at lower temper- atures and leverage these samples to fine-tune the model toward lower temperatures. Additionally, we incorporate importance sampling and PT to refine buffers at each in- termediate step. As a result, improvement occurs not only through the training objective but also through the proposed temperature guidance and refinement strategies.
4. Connection with Related Works
Training neural samplers for unnormalized density There are many approaches aiming to learn a network to sample from the target density without getting access to data. Flow Annealed Importance Sampling Bootstrap (FAB, Midgley et al., 2023) trains a normalizing flow using the a-2 divergence, estimated by Annealed Importance Sam- pling (Neal, 2001), and incorporates a replay buffer (Mnih et al., 2015; Schaul et al., 2016) to reduce computational cost and mitigate forgetting. Recently, due to the success of diffusion models, studies have focused on SDE and control- based neural samplers, such as Path Integral Sampler (PIS, Zhang& & Chen, 2022), Denoising Diffusson Samplers (DDS, Vargas et al., 2023), and Controlled Monte Carlo Diffusion (CMCD, Vargas et al., 2024), which match the path mea-
Integrating MCMC algorithms with neural samplers Recent line of research has explored integrating MCMC algorithms to improve the performance of neural samplers. They either introduce Sequential Monte Carlo (SMC) to correct bias (Arbel et al., 2021; Albergo & Vanden-Eijnden, 2024; Phillips et al., 2024; Chen et al., 2024a) or employ MCMC to improve the buffer quality (Sendera et al., 2024). However, while these works primarily focus on ensuring
6

---

Progressive Tempering Sampler with Diffusion
asymptotic correctness or improving the sample quality of neural samplers, they still incur a substantial overhead in evaluating the target energy compared to vanilla PT. In concurrent work, Zhang et al. (2025) proposed generalized parallel tempering, employing a neural network to transport samples between adjacent temperatures and thus boost the swap rate of vanilla PT. In contrast, our work takes a dual perspective: rather than embedding neural samplers into PT to improve PT, we incorporate PT's cemperature-exchange mechanism into neural samplers to enhance both sampling efficiency and sample quality of the neural sampler.
T=2.0
T=1.5
T=1.0
0.6
0.6
0.6
0.5
0.5
0.5
0.4
0.4
04
Density
0.3
0.3
0.3
0.0
0.0
0.0
Interatomic Distance
Interatomic Distance
Interatomic Distance
Figure 5. First two panels: Marginal density of the interatomic dis- tance on buffers at two temperatures, along with the marginal den- sity of diffusion model fit to the buffer. Right: The temperature- guided extrapolation based on the higher-temperature models. AD: auto-diff extrapolation: ME: model extrapolation; RS: score rescal- ing heuristic; GT: ground truth: PTSD: our temperature guidance.
Guidance and inference-time control in diffusion models Many tools exist for modifying and aggregating pretrained diffusion models at inference time. For instance, is possi- ble to combine multiple diffusion models to get their inter- section (Liu et al., 2022; Du et al., 2023) and add additional constraints and controls to the final distribution (e.g., for inverse problem solving) (Chung et al., 2023; Song et al., 2023; Rissanen et al., 2024). Of particular interest to us are so-called classifier-free guidance (Ho & Salimans, 2021) and its recent development (Karras et al., 2024), where two diffusion model denoisers, one closely fit to the data, and one less closely fit, are contrasted to form diffusion models with even better fits, and hence, better sample quality:
5.1. Extrapolation with Temperature guidance
To showcase the capability of proposed guidance. we con- duct experiment on the LJ-55 potential, where we train DMs at temperature 2.0 and 1.5 and aim to extrapolate to a target temperature 1.0. Fig. visualizes the histograms of inter- atomic distance of samples generated by the trained models at high temperatures, as well as the one of extrapolated samples at the target temperature.
D Duuided un (1xx = (1 + Do, (29) - wD, (2x),
(13)
The extrapolation is closer to the target temperature than either model at temperature 2.0 or 1.5, showing a large overlap with the ground truth. We also compare our ex- ploration approach with several other strategies. (1) model extrapolation (ME): as we train single model on both temperatures 2.0 and 1.5, and by conditioning on the tem- perature label, we can directly input temperature 1.0 and rely on the generalisation in deep neural networks, an ap- proach that was demonstrated useful for interpolation in Moqvist et al. (2025). (2) auto-diff extrapolation (AD): we directly use Equation (9) instead of finite difference approximation for the time-derivative; (3) score rescaling heuristic (RS): we simply anneal the score with the tem- perature V., log (r 1.0) v 1.5 - Va, lo 1.5), an approach considered by Skreta et al. (2025). Our proposed temperature guidance provides a better extrapolation than the other choices. Even though the extrapolation is not per- feet, it serves as an initialization for the local PT refinement, as proposed in Sec. 3.3.
Do, and Do. represent the well fitteittd and poorly-fiite fitted mod- els respectively (Karras et al., 2024). Guidance strength w > improves sample quality. When applied to condi- tional and unconditional models as the 'better' and 'worse" models, the method is called classifier-free guidance (Ho & Salimans, 2021), which has been crucial to the success of text-to-image diffusion models (Saharia et al., 2022: Ramesh et al., 2022; Rombach et al., 2022). This ap- proach closely parallels our temperature guidance method in Eq. (11) and Eq. (13).
5. Experiments and Results
In this section, we evaluate our proposed approach and compare with other baselines. In Sec. 5.1, we first test our temperature guidance method on the Lennard-Jones potential with 55 partieles (LJ-55), as introduced in (Köhler et al., 2020; Klein et al., 2024). As we will show, this guidance enables effective extrapolation.
5.2. Comparison with Baselines
In Sec. 5.2, we compare PTSD on two distinct multi-modal distributions, Mixture of 40 Gaussians (MoG-40) and Many- Well-32 (MW-32), with other neural samplers, including FAB (Midgley et al., 2023), iDEM (Akhound-Sadegh et al., 2024), BNEM (OuYang et al., 2024). DiKL (He et al., 2024), DDS (Vargas et al., 2023), and CMCD (Vargas et al., 2024). We also evaluate the performance of a diffusion model trained directly on PT-generated data (PT+DM).
We now measure the sample quality of PTSD and compare it with other baselines. We report Wasserstein-2 (W:)) dis- tance, Total Variation distance (TVD), and Maximum Mean Discrepancy (MMD). For all tasks, TVD and MMD are mea- sured over the energy histograms of samples, while W2 is measured in the samples. The calculation W2 in LJ-55 task takes the SE(()--quivariance into account as (Ou Yang et al.,
7

---

Progressive Tempering Sampler with Diffusion
Table Comparing PTSD with other neural sampler baselines. We measure (best, second best) the TVD and MMD between Energy histograms, and W2 distance between data samples. We note that as the energy histograms project the entire data space into one dimension, can be sensitive outliers but insensitive to mode coverage. '-' indicates that the method diverges or significantly worse than others.
GMM (d = 2)
MW32 (d = 32)
LJ55 (d = 165)
TVD
MMD +
W2
TVD 4
MMD +
W2 4
TVD
MMD +
W2 +
FAB
0.23 =0.01
0.30 =0.02
2.50 19
0.32 10.01
0.16 00.02
5.70 10.01
CMCD
00.01
0.08 10.01
3.36 20222
0.69 10.01
0.62 00.01
10.03
DDS
0.22
0.10 00.03
4.52 10.30
0.77 10000
00.02
7.60 10.02
DEM
0.09 #0.01
0.01 +0.01
3.26 #0.42
0.87 #0.01
0.87 =0.01
8.30 =0.02
0.49 #0.01
1.95 +0.00
BNEM
0.12 =0.00
0.07 00.00
2.16
0.31 +0.01
0.05 =0.01
8.41 +0.09
0.18 =0.01
0.01 +0.00
1.76 =0.00
DiKL
0.10
0.03
3.23 #0.24
0.31
0.16 =0.02
6.59 +0.03
PT+DM
0.13 00.05
0,06
3.01 1:0.6
0.13
0.03 1:0.02
5.04 1:000
0.64 20.02
0.22 a 01
1.82 40.00
PTSD
0.08 10.02
0.02 10001
1.93 10.41
0.14 10.05
0.04 1:0.03
4.99 20007
0. 0.79 1.001
0.19 10.00
1881 10.00
number of energy evaluations. In fact, LJ-55 is a relatively simple task for MCMC: a standard MALA requires only 4000 steps to mix. The challenge of LJ-55 for neural sam- plers arises because the target contains inhibitory regions and has large gradients, leading to instabilities. BNEM addresses this by first smoothing the target density and ex- plicitly regressing the target energy. On the other hand, our approach relies only on samples, making it stable and effi- cient but less sensitive to inhibitory regions. This explains why we fall slightly behind.
Table 2. Number of target density calls for different approaches to achieve the performance reported in Table 1.
Algorithm
GMM (d = 2) MW32 (d- = 32) LJSS (d = 165)
FAB
6.6 x 10
7.2 X 10
CMCD
4.4 x 10
1.6 10a
DDS
2.6 x 109
1.1 X 10
IDEM
5.0 x 1010
1.8 X 10th
1.3 1040
BNEM
7.5 x 10
1.8 x 10
6.4 x 10
DiKL
1.2 x 10
8.0 10
PT+DM
1.2 x 106
8.0 10c
1.65 x 10
PTSD (ours)
1.0 x 10
5.3 x 10
1.5 x 10
Additionally, to provide a more detailed comparison with PT+DM, we plot the sample quality of PT+DM using sam- ples obtained by PT with different numbers of target evalua- tions in Fig. 6. As we can see, on these targets, PTSD out- performs PT+DM using the same setting. This gain might come from the fact that PTSD can provide uncorrelated samples for high temperatures easily after being trained on these temperatures, and the temperature guidance serves as approximate yet more informative "swap" mechanism.
PieDo (0)
-B- PT-CH(331 PTSD (u01
No Ckataaa
ai Distance
PTSD 110
5.0
0.5
4.9
4
Trrget Evolustion Times
(m155 (x109)
Target Evallsstio Times (XI0
(a) MoG-40
(b) MW-32
5.3. Ablation Study
Figure 6. Comparing PTSD with the results obtained by fitting diffusion model post hoc to PT gengeeeate data. For MoG-40, we compare two different settings, using 4 or 10 temperature levels.
We conduct ablation studies to verify the effectiveness of temperature guidance and the improvement techniques pro- posed in Sec. 3.3 in Table on MW-32. As the guidance and the truncated IS enhance the performance of our algorithm. Additionally, we note that even without IS, PTSD remains competitive among all neural samplers in Table 1. This further demonstrates the effectiveness of temperature guidance on its own.
2024). For LJ-55, we optimize the energy evaluation usage by running PT from a subset of IS samples, as explained in Sec. 3. We note that the energy histogram is sensitive to outliers while generally being robust to mode collapse. In He et al. (2024, Fig. 5), the authors showed that the energy histogram can closely approximate the ground truth, even when significant mode collapse occurs. On the other hand, W2 distance may provide a more comprehensive assessment of sample quality. We also report the computational cost (in terms of target evaluations) in Table 2.
see, both
5.4. Scaling PTSD on Alanine Dipeptide
We also apply the method to the alanine dipeptide molecule in Cartesian coordinates. To maximise the benefits of PTSD, we run local PT refinement only for a subset of IS outputs, as detailed in Sec. 3.3. We evaluate both PTSD and PTDM with approximately the same amount of energy function evaluations (2.6 x 107), and show the resulting Ramachan- dran plots in Fig. 7. This example demonstrates the poten-
On both MoG-40 and MW-32, our approach achieves state-of-the-art sample quality and demonstrates orders- of oa-ggniie improvement in efficiency over other neural samplers. Our algorithm falls slightly behind on BNEM on sample quality. However, it requires significantly smaller
8

---

Progressive Tempering Sampler with Diffusion
Table 3. Mean log-likelihood of real data under our PTDM and PTSD models for the alanine dipeptide molecule, and the KL divergence between the ground truth Ramachandran histogram and the model-generated one from 10" samples.
Table 4. Ablations on the temperature guidance in Sec. 3.1 and the truncated IS in Sec. 3.3. We do not provide ablation on the local PT refinement as running MCMC clearly helpful.
TVD 4 W24
PTSD
PT+DM
PTSD w/o temp-guide
0.34
24.59
Eog po (x)] KLD
213.32 + 0.06 6.9e-2
212.38 + 0.05 3.2e-02
PTSD w/o IS PTSD
0.23
5.84
0.14
4.99
Ground truth
FTOM
PT50
Network training cost: while our approach improves efficiency in terms of target evaluations, it remains slower than parallel tempering in terms of wall-clock time in our experiments. This is because we need to fine-tune the diffusion model for each temperature level. For more complex target distributions, we expect that training the diffusion model will be more compu- tationally efficient than directly evaluating the target density and its gradient. Therefore, an important direc- tion for future work is extending our pipeline to more difficult target distributions.
1/2
-1
-1/2
0
1/2
11
-0/2
112
-0
-1/2
0
n/2
Figure 7. Ramachandran plots of ground truth, PT+DM, and PTSD. PT+DM and PTSD were trained with 2.6c7 energy evaluations.
tial scalability and efficiency of PTSD in realistic sampling problems. We also evaluate the log-likelihood of ground- truth data under our PTDM and PTSD models, and the KL divergence between the ground-truth Ramachandran his- togram and the model-generated one in Table 3. We show the projection of Ramachandran plots along two axes (ø and e in App. D.4. As we can see, while PT+DM appears slightly better performance on generate the large metastable sates, resulting to be more plausible than PTSD along the é axis, PTSD achieves better log- likelihood than PT+DM and also captures the small metastable state more effectively, as shown in Fig. 7. This showcases the great potential of PTSD when applied more complex systems.
2. Non-paralee execution: our approach relies on decreasing the temperature progressively, which is different from PT, where different chains can be distributed multiple devices and run in parallel.
3. Sensitivity to temperature schedule and network learning quality: our approach's 's performance can become fragile when temperature levels differ signifi- cantly or when the target distribution is too complex for the network to learn accurately. In contrast, vanilla PT is more robust to suboptimal temperature schedule and also offers additional flexibility in path selection. The sensitivity also appears in hyperparameters: the network choice, learning rate, and the truncation thresh- old for truncated IS can also impact the final sample.
6. Conclusions and Limitations
In this paper, we proposed a heuristic temperature guidance that allows us to generate samples at lower temperatures with pretrained diffiusion models at two higher temperatures. Based on this, we formulated the Progressive Tempering Sampler with Diffusion (PTSD). PTSD achieved competi- tive sample quality, demonstrated orders-of-man offers-of-magniuude im- provement in efficiency over other neural samplers, and showed promising direction in combining parallel temper- ing to enhance neural samplers.
Another line of future work is to test the amortizability of our proposed approach. In a recent study, Havens et al. (2025) introduced a conformer -ennerrttnnnntiionnn dataset comprising many targets with similar properties. Using PT to obtain samples for each target separately and then fitting a diffusion model would be expensive. In contrast, PTSD framework may have the potential to substantially reduce this cost.
Specifically, we could first collect data for each target at the two highest temperatures and fit a diffusion model condi- tioned on the target label. When performing temperature guidance, we could randomly sample targets to build buffer with mixed targets. This buffer could be used (with its asso- ciated target labels) to train the conditional diffusion model at lower temperatures. Thus, PTSD only requires running separate MCMC during initialization, and it can share infor- mation across targets as temperature decreases, resulting in a potential efficiency gain.
While training without data is appealing, this pursuit may be inefficient with current approaches. Instead, designing meth- ods that effectively integrate neural samplers with available data could offer greater practical benefits. While our work may not represent the optimal solution for this direction, it opens a promising avenue for future works, potentially advancing the practicality of neural samplers.
However, several key limitations still remain:

---

Progressive Tempering Sampler with Diffusion
Acknowledgments
Chen, J., Richter, L.., Berner, J., Blessing, D., Neumann, G., and Anandkumar, A. Sequential controlled Langevin diffusions. arXiv preprint arXiv:2412.0708/ 2024a.
We acknowledge Saifuddin Syed for discussions on par- allel tempering, and Yuanqi Du Mingtian Zhang, Louis Grenioux, Laurence Midgley and Javier Antorán for dis- cussions neural samplers. JH acknowledges support from the University of Cambridge Harding Distinguished Postgraduate Scholars Programme. JMHL and RKOY ac- knowledge the support of a Turing AI Fellowship under grant EP/V023756/1 SR, MH, and AS acknowledge fund- ing from the Research Council of Finland (grants 339730, 362408, 334600). This project acknowledges the resources Discov-
Chen, R. T., Rubanova, Y., Bettencourt, J., and Duvenaud, D. K. Neural ordinary differential equations. In Advances in Neural Information Processing Systems (NeurIPS), volume 31. Curran Associates, Inc., 2018.
Chen, W., Zhang, M., Paige, B., Hernández-Lobato, J. M., and Barber, D. Diffusive Gibbs sampling. In Proceedings of the International Conference on Machine Learning (ICML), pp. 7731-7747, PMLR, 2024b.
provided by the Cambridge Service for ery (CSD3) operated by the University of Cambridge Re- search Computing Service (www.csd3,co www.csd3.cam..ccukk cca.aac.kk. provided by Dell EMC and Intel using Tier-2 funding from the En- gineering and Physical Sciences Research Council (capital grant EP/T022159/1) and DIRAC funding from the Scienee and Technology Facilities Council (www.dirac.ac ..ck......ac.kk..... We acknowledge CSC- IT Center for Science, Finland, for awarding this project access to the LUMI supercomputer, owned by the EuroHPC Joint Undertaking, hosted by CSC (Finland) and the LUMI consortium through CSC. We ac- knowledge the computational resources provided by the Aalto Science-IT project.
Chung, H., Kim, J., Mccann, M. T., Klasky, M. L.., and Ye, J. C. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Confer- ence Learning Representations (ICLR), 2023.
De Bortoli, V., Hutchinson, M., Wirnsherger, P., and Doucet, A. Target score matching. arXiv preprint arXiv:2402.cm 08667, 2024.
Du, Y., Durkan, C., Strudel, R., Tenenbaum, J. B., Diele- man, S., Fergus, R., Sohl-Dickstein, J., Doucet, A., and Grathwohl, W. S. Reduce, reuse, recycle: Composi- tional generation with energy-based diffusion models and meme. In Proceedings of the International Conference on Machine Learning (ICML), pp. 8489-8510. PMLR, 2023.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Duane, S., Kennedy, A., Pendleton, B. J., and Roweth, D. Hybrid Monte Carlo. Physics Letters B, 195(2):216-225 1987.
Earl, D. J. and Deem, M. W. Parallel tempering: Theory, applications, and new perspectives. Physical Chemistry Chemical Physics, (223)3910039111 2005.
References
Shhound-Sadeghh T., Rector-Brooks, J., Bose, J., Mittal, S., Lemos, P., Liu, C.-H., Sendera, M., Ravanbakhsh, S., Gidel, G., Bengio. Y., et al. Iterated denoising energy matching for sampling from boltzmann densities. In Proceedings of the Forty-first International Conference on Machine Learning (ICMI), 2024.
Efron, B. Tweedie's formula and selection bias. Journal of the American Statistical Association, 106(496): 1614, 2011.
Flamary, R., Courty, N., Gramfort, A., Alaya, M. Z., Bois- bumon, A., Chambon, S., Chapel, L., Corenflos, A., Fatras, K., Fournier, N., Gautheron, I.., Gayraud, N. T., Janati, H., Rakotomamonjy, A., Redko, I., Rolet, A., Schutz, A., Seguy, V., Sutherland, D. J., Tavenard, R., Tong, A., and Vayer, T. Pot: Python optimal transport. Journal of Machine Learning Research, 22(78):1 1-8, 2021.
Albergo, M. S. and Vanden-Bijnden, E. Nets: A non-equilibrium transport sampler. arXiv preprint arXiv=2410.00211,, 2024.
Arbel, M., Matthews, A., and Doucet, A. Annealed flow transport monte carlo. In International Conference on Machine Learning, pp. 318-3301 PMLR, 2021.
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B. Bayesian Data Analysis. CRC Press, 2013.
Blessing, D., Jia, X., Esslinger, J., Vargas, F., and Neu- mann, G. Beyond ELBOs: A large-scale evaluation of variational methods for sampling. In Proceedings of the Forty-first International Conference on Machine Learning (ICML), 2024.
Geman, S. and Geman, D. Stochastic relaxation, Gibbs dis- tributions, and the Bayesian restoration images. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, PAMI-666):211-74) 1984.
10

---

and Computation, Progressive Tempering Sampler with Diffusion
Geyer, C. J. Markov chain Monte Carlo maximum likeli- hood. In Computing Science and Statistics: Proceedings of the 23rd Symposium the Interface, 1991.
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion- generative models. In Advances in Neural Information Processing Systems 35 (NeurlPS), pp. 26565 25557726577. Curran Associates, Inc., 2022.
Grathwohl, W., Chen, R. T., Bettencourt, J., Sutskever, I,, and Duvenaud, D. FFJORD: Free-form continuous dy- namics for scalable reversible generative models. In Inter- national Conference on Learning Representations (ICLR), 2018.
Karras, T., Aittala, M., Kynkäänniemi, T., Lehtinen, J., Aila, T., and Laine, S. Guiding diffusion model with a bad version of itself. In Advances in Neural Information Processing Systems (NeurIPS). Curran Associates, Inc., 2024.
Grenander, U. and Miller, M. I. Representations of knowl- edge in complex systems. Journal of the Royal Statistical Society: Series (Methodologicul), 56:549-581, 1994.
Klein, L.,, Krämer, A., and Noé, Equivariant flow match- ing. In Advances in Neural Information Processing Systems (NeurIPS), volume 36. Curran Associates, Inc., 2024.
Hastings, W. K. Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1): 97-109, 1970.
Köhler, J., Klein, L., and Noé, F. Equivariant flows: exact likelihood generative learning for symmetric densities. In Proceedings of the International Conference on Machine Learning (ICML), pp. 5361-5370 PMLR, 2020.
Havens, A., Miller, B. K., Yan, B., Domingo-Enrich, C., Sriram, A., Wood, B., Levine, D., Hu, B., Amos, B., Karrer, B., et al. Adjoint sampling: Highly scalable diffuuson samplers via adjoint matching. arXiv preprint arXiv:2504. 1713, 2025.
Liu, J. and Chen, R. Sequential Monte Carlo methods for dynamic systems. Journal of the American Statistical Association, 93(443):
He, J., Chen, W., Zhang, M., Barber, D., and Hernández- Lobato, J. M. Training neural samplers with reverse dif- fusive KL divergence. arXiv preprint arXiv:2410.12456, 2024.
1998.
Liu, N., Li, S., Du, Y., Torralba, A., and Tenenbaum, J. B. Compositional visual generation with composable dif- fusion models. In European Conference on Computer Vision, pp. 423-439. Springer, 2022.
He, J., Du, Y., Vargas, F., Zhang, D., Padhy, S., Ou Yungn R., Gomes, C., and Hermández-Lobato J. M. No trick, no treat:
Pursuits and challenges towards simulation- free training of neural samplers. arXiv preprint arXiv:2502.06688, 2025.
Luo, W. Hu, T. Zhang, S., Sun, J., Li, Z.,, and Zhang, Z. iiff-instruct: A universal approach for transfer- ring knowledge from pre-trained diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), volume 36. Curran Associates, Inc., 2024.
Ho, J. and Salimans, Classifier-fres free diffusion guidance. In NeurlPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E. Equation of state calculations by fast computing machines. The Journal of Chemical Physies, 21(6):
Ho. J., Jain, A. and Abbeel, P. Denoising diffusion proba- bilistic models. In Advances in Neural Information Pro- cessing Systems (NeurIPS), volume 33, pp. 6840-6851. Curran Associates, Inc., 2020.
06 1953.
Midgley, L. I., Stimper, V., Simm, G. N., Schölkopf, B., and Hernández- J. M. Flow annealed importance sampling bootstrap. In The Eleventh International Con- ference on Learning Representations (ICLR), 2023.
Hukushima, K. and Nemoto, K. Exchange Monte Carlo method and application to spin glass simulations. Journal of the Physical Society of Japan, 65(6):160--1608 1996.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidje- land, A. K., Ostrovski, G., et al. Human-level control through deep reinforcement learning. Nature, 518(7540): 529-533, 2015.
Hutchinson, M. F. A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Com- munications in 1859:1055- 1989.
lonides, E. L. Truncated importance sampling. Journal of Computational and Graphical Statistics, 17(2):295-31 2008.
Moore, J. H., Cole, D. J., and Csanyi, G. Computing hydra- tion free energies of small molecules with first principles accuracy. arXiv preprint arxiv:2405. 18/71, 2024.

---

Progressive Tempering Sampler with Diffusion
Moqvist, S., Chen, W., Schreiner, M., Nüske, F., and Olsson, S. Thermodynamic interpolation: generative approach to molecular thermodynamies and kinetics. Journal of Chemical Theory and Computation, 2025.
Satorras, V. G., Hoogeboom, E., and Welling, M. E(n) equivariant graph neural networks. arXiv preprint arxiv:2102.0984,, 2021.
Schaul, T., Quan, J., Antonoglou, I., and Silver, D. Priori- tized experience replay. arXiv preprint arxv:1511.08952. 2016.
Neal, R. M. Probabilistic inference using Markov chain Monte Carlo methods. Technical report, Department of Computer Science, University of Toronto, 1993.
Sendera, M., Kim, M., Mittal, S., Lemos, P., Scimeca, L., Rector-Brooks, J., Adam, A., Bengio, Y., and Malkin, N. Improved off-policy training of diffusion samplers. Advances in Neural Information Processing Systems (NeurIPS). Curran Associates, Inc., 2024.
Neal, R. M. Annealed importance sampling. Statistics and Computing, 11:125-139, 2001.
Noé, F., Olsson, S., Köhler, J., and Wu, H. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning. Science, 365(6457), 2019.
Skreta, M., Akhound-Sadegh, T., Ohanesian, V., Bondesan, R., Aspuru-Guzik, A., Doucet, A., Brekelmans, R., Tong, A., and Neklyudov, K. Feynman- correctors in diffu- sion: Annealing, guidance, and product of experts. arXiv preprint arXiv:2503 02800, 2025.
Ou Yang R., Qiang, B., and Heráández-Lobato, J. M. Bnem: A boltzmann sampler based on bootstrapped noised en- crgy matching. arXiv preprint arXiv:2409.09787 2024.
Phillips, A., Dau, H.-D., Hutchinson, M. J., De Bortoli, V., Deligiannidis, G., and Doucet, A. Particle denois- ing diffusion sampler. In Proceedings of the 41st Inter- national Conference on Machine Learning, pp. 40688- 40724, 2024.
Sohl-Dickstein, J., Weiss, A., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequilib- rium thermodynamics. arXiv preprint arxiv: 1503.03585, 2015.
Song, J., Vahdat, A., Mardani, M., and Kautz, J. souddiinvers-guudee diffusion models for inverse prob- lems. In International Conference on Learning Represen- tations (ICLR), 2023.
Poole, B., Jain, A., Barron, J. T., and Mildenhall, B. Dream- fusion: Text-to-3d using 2d diffusion. In The Eleventh International Conference Learning Representations (ICLR), 2023.
Song, Y., Sohl-Dickstein, J., Kingma, D). P., Kumar, A., Fr- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations (ICLR), 2021.
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.066 06155, 2022.
Rissanen, S., Heinonen, M.. and Solin, A. Free hunch: De- noiser covariance estimation for diffusion models without extra costs. arXiv preprint arXiv:2410.11144 2024.
Surjanovic, N., Syed, S., Bouchard- A., and Camp- bell, T. Parallel tempering with variational reference. Advances in Neural Information Processing Systems, 35: 565-577, 2022.
Roberts, G. O. and Tweedie, R. L. Exponential convergence of langevin distributions and their discrete approxima- tions. Bernoulli, pp. 341-363, 1996.
Surjanovic, N., Syed, S., Bouchard-Côté, A., and Campbell, T. Uniform ergodicity of parallel tempering with effi- cient local exploration. arXiv preprint arXiv:2405. 11384 2024.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with la- tent diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, 2022.
Swendsen, R. H. and Wang, J.-S. Replica Monte Carlo simulation of spin-glasses. Physical Review Letters, 57 (21):2607, 1986.
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E. L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al. Photorealistic text-to-image diffusion models with deep language under- standing. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pp. Associates, Inc., 2022.
Syed, S., Romaniello, V., Campbell, T., and Bouchard-Côré. A. Parallel tempering on optimized paths. In Inter- national Conference on Machine Learning, pp. 10033- 10042. PMLR, 2021.
Curran
Vargas, F., Grathwohl, W., and Doucet, A. Denoising diffu- sion samplers, arXiv preprint arXiv:2302. 13834, 2023.
12

---

Progressive Tempering Sampler with Diffusion
Vargas, F., Padhy, S., Blessing, D., and Nüsken, N. Trans- port meets variational inference: Controlled Monte Carlo diffusions. In The Twelfth International Conference Learning Representations (ICLR), 2024.
Vincent, A connection between score matching and de- noising autoencoders. Neural Computation, 23(7):1661- 1674, 2011.
Von Toussaint, U. Bayesian inference physics. Reviews of Modern Physies, 83(3):943-999, 2011.
Welling, M. and Teh, Y. W. Bayesian learning via stochastic gradient Langevin dynamics. Proceedings of the 28th International Conference on Machine Learning (ICML), pp. 681-688, 2011.
Woodard, D. B., Schmidler, S. C.,, and Huber, M. Conditions for rapid mixing of parallel and simulated tempering on multimodal distributions. 2009.
Wu, H., Köhler, J., and Noe, F. Stochastic normalizing flows. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pp. 5933-5944. Curran Associates, Inc., 2020.
Zhang, L.,, Potaptchik, P., Doucet, A., Dau, H.-D., and Syed, S. Generalised parallel tempering: Flexible replica exchange via flows and diffusions. arXiv preprint arXIv:2502.10328, 2025.
Zhang, Q and Chen, Y. Path integral sampler: A stochastic control approach for sampling. In International Confer- ence on Learning Representations (ICLR), 2022.
13

---

Progressive Tempering Sampler with Diffusion
Appendices
A. Target Distributions
Mixture of Gaussians (MoG-40) is a mixture of Gaussians with 40 components in 2-dimensional space, proposed by Midgley et al. (2023). Each component of the MoG-40 has mean within [-40, 40) [-40, 40] and an diagonal covariance of softplus(1).
Well-32 (MW-32) is multi-modal distribution in 32-dimensional space with 2 modes, proposed by Midgley et al. (2023). It is constructed by concatenating 16 independent samples from the Double-Well distribution (DW-2, Noé et al., 2019; Wu et al., 2020) in 2-dimensional space. While sampling 16 samples independently from DW-2 and stacking them to construct a MW-32 sample is easy, directly drawing samples from the MW-32 distribution is challenging as a consequence of its multi-modal nature.
Lennard-Jones-n (I.J-) describes a n-particle system, where the energy between two particles described by the Lennard-Jones potential and the system energy is given by the sum of pairwise 2-particle energy, i.e.
(r) = 4e (()))=((9)
12 - (
67
and U(X) = ΣΣ (ris), iE(M] ES
(14)
where X = (2), (.., F.), Paj is the distance between zy and zj. and and physical constants. While LJ-n can be challenging in terms of numerical stability since the energy of system can be problematically large when particles are too close to each others, it remains a relatively simple target for MCMC with proper initialization or numerically stable implementation. Therefore, we only employ LJ-55 for showcasing the capability of our proposed extrapolation. To ensure numerical stability, we employ cubic spline interpolation introduced by Moore et al. (2024) to smooth the extreme energy values when two particles are close to each other. In our main experiment, we consider LJ-55, While for the ablation study, we conduct experiments on both LJ-13 and LJ-55.
Alanine Dipeptide (ALDP) is 22- parpariiee system formed by two Alanine amino acids. We consider an implicit solvent with temperature of 300K, which is used by Midgley al. (2023). We use the implementation in Midgley et al. (2023) to calculate the energy. In contrast to FAB, which generates samples in the internal coordinate system, we consider generating samples in the Cartesian coordinate system.
B. Evaluation Metrics
Wasserstein-2 distance (W2) measures the difference between two probability distributions in an optimal transport framework. Given empirical samples μ from the sampler and ground truth samples v, the W22 distance is defined as:
Wz(u,v) = ( inf E, ((((x)))) 1/2 (ETl(())))
(15)
where ll is the transport plan with marginals distributions μ and v respectively, and is distance measure. To calculate the W2 in practice, we use the Hungarian algorithm implemented in the Python optimal transport package (POT, Flamary et al., 2021), where we employ the Fuclidean distance as our distance measure. We measure the W2 over samples for all tasks. For LJ-55, we calculate the W2 by taking SE(3), i.e. rotation and translation equivariance, into account. In particular, we calculate the distance between two set of samples by dKatsch(X Y) = min (,ESE(() | (YR + +1]ll, where Kabsch algorithm is applied to find the optimal rotation and translation.
Total Variation distance (TVD) measures the dissimilarity between two probability distributions, which quantifies the absolute differences between two densities over the entire sample space. Given two distribution P and Q defined on a space Ω with density functions and q, TVD is defined as
1 - 1p(x) TV {(x)) dr.
TVD(P:Q) =
(16)
We measure the TVD over energy histograms of samples for all tasks.
14

---

Progressive Tempering Sampler with Diffusion
Maximum Mean Discrepancy (MMD) measures the difference between distributions using functions in Reproduc- ing Kernel Hilbert Space (RKHS). Given two distributions P and Q, the MMD is defined
1/2
MMD(P,Q) -
sup
Ep[f(X) - Bolf(r)))
(17)
SEHIIIIISS
where f is a function in the RKHS associated with a kernel k(r, y) and || || a norm defined in this RKHS. The MMD can be computed by using the kernel trick as follows.
MMD(P, Q) (Ex.x'=>\XXX
Byyr=ol((XX"")) - 2Ex-PY==
(18)
We measure the MMD over energy histograms of samples for all tasks.
The error intervals in Table for the GMM and MW32 tasks for our method and PTDM were obtained by running the method with multiple seeds, and estimating the standard deviation. Any runs where the metrics calculation resulted in undefined values due generated outliers in MW32 were discarded. For GMM and MW32 with the baselines and LJ55 in general, we estimated the error by sampling multiple times from trained model. The error intervals for the log- oggliikedhood evaluations were obtained by estimating the standard deviation of log (I) evaluations and calculating the standard error by dividing by the square root of the amount of samples taken.
C. Experimental Settings
C.1. Diffusion Models for PTSD and PT+DM
Network Architecture. For non- non-particle system, i.e. GMM-40 and MW-32, employ a 5-layer MLP. For particle systems, Le. LJ-55 and ALDP, we employ the EGNN implemented by Satorras et al. (2021) with 3 layers.
Parameterization. We parameterize the DMs denoiser network to approximate the denoising mean following Karras et al. (2022), where the data is approximated by the standard deviation of the data used for training, i.e. the buffer in PTSD and the drawn samples from PT PT+DM.
Noise Schedule. Our noise schedule follows Karras et al. (2022), with the maximum time fmax 40.
Sampling Process.
We employ the Euler solver for sampling, where we discretize the time following Karras et al. (2022).
Parallel Tempering.
Our implementation of parallel tempering is based on the codebase of DiGS (Chen et al., 2024b).
Energy evaluation optimization. On both LJ55 and ALDP, we use the method of running PT at the temperature extrapolation steps only from a subset of IS filtered samples, and concatenate the PT samples to the original IS samples. the GMM and MW32 tasks, we instead run chain from each diffusion model generated (and resampled) sample, and take only the final output of the chain.
Temperature levels. For all experiments, we range of geometrically spaced temperatures from maximum temperature to the desired minimum temperature.
C.2. Hyperparameters
PTSD. Hyperparameter settings for PTSD on each task is illustrated in 5. The key hyperparameters the temperature range, number of temperatures, buffer size (amount of samples to train the diffusion models on), the number of initial PT steps at the highest two temperatures, the number of steps in PT while following the extrapolation steps, and the number of PT chains to use at the extrapolation step. For MW32, we did not perform the importance resampling at the last extrapolation step.
PT+DM. Hyperparameter settings for running PT on different targets are illustrated in Table 6. The batch size for DM
training the same as PTSD, and we train the DM until converged.
15

---

Progressive Tempering Sampler with Diffusion
Table 5. Hyperparameter settings for PTSD on different targets.
Hyperparameters! Target+
MoG-40
MW-32
LJ-55
LJ-55 (illustrative)
Temperature range
[1, 100]
[1,10]
[1,3]
[1.2]
Number of temperatures
10
10
3
Temperature schedule
geom
geom
geom
linear
Buffer Size
10000
12000
20000
20000
Batch size
1000
1000
1000
1000
Number of initial PT chains
100
20
1 40000
Number of initial PT steps PT swap interval
1000
20000
40000
5
5 5000
Burn-in at the initial PT
100
10000
5000
Interval for subsampling the initial PT chain
9
10
1 20000
Number of generated samples extrapolation
10000
12000
20000
Number of PT chains at extrapolation Number of PT steps after extrapolation
10000
12000
10
10
5
25
2500
2500
Number of training iterations
10000
10000
120000
120000
Importance resampling at last step
Yes
No
Yes
Yes
Table 6. Hyperparameter settings for PT on different targets.
hyperparams, target-
MoG-40
MW-32
LJ-55
LJ-55 (illustrative)
Temperature range
[1,2
[1, 10]
[1,3]
[1,2]
Num. of temperatures
4
10
3
3
Temperature schedule
geom
geom
geom
linear
FAB. For both MoG-40 and WM-32, we run FAB following exactly the setting by Midgley et al. (2023), with the code https:t//// /github.... com/ wwwllol//aaatttoreeeeeaea As they use buffer which can influence the target density evaluation time, we directly count the time when rumning the code.
DDS. We evaluate DDS using the implementation by Blessing al. (2024) with KL divergence following Vargas et al. (2023). For MoG-40, we train DDS for 10000 iterations with batch size of 2000, using Euler-Maruyama discretization with 128 steps. Note that DDS's network takes score term the input, and hence, the total number of target evaluations is 10000 x 2000 x 128. For WM-32, we apply early stopping at 3200 iterations, and hence, the total number of target evaluations is 3200 x 2000 x 128.
CMCD. We evaluate CMCD using the implementation by Blessing et al. (2024) with KI. divergence following Vargas et al. (2024). For MoG-40, we train CMCD with a batch size of 2000, using Euler-Maruyama discretization with 128 steps. It takes 17002 iterations to achieve good performance, and hence the total number of target evaluations is 17002 x 2000 x 128. For WM-32, we train CMCD with a batch size of 2000, using Murer-Maruyama discretization with 256 steps. We found training for more steps may lead to mode collapsing, and hence we early stop at 3200 iterations. The total number of target evaluations is 3200 2000 256.
iDEM. We evaluate iDEM using the implementation by Akhound-Sadeen Sadegh et al. (2024). For all tasks, we use Eyler- Maruyama discretization with 1000 steps and 100 inner-loops. For MoG-40, we train iDEM for 1000 outer-loops to ensure convergence, with batch size of 1000. The marginal scores estimated by 500 MC samples, which is clipped to maximum norm of 70. The total number of target evaluations is 1000 x 100 x 1000 x 5000. For MW-32, we train iDEM for 180 outer-loops, increase the number of MC samples to 1000, clip the score to a maximum norm of 1000, and keep the other hyperparameter the same. The total number of target evaluations is 180 x 100 x 1000 x 1000. For LJ-55, we further clip the score to a maximum norm of 20, decrease the batch size to 128, and train iDEM for 1000 outer-loops. The total number of target evaluations is 1000 100 x 128 x 1000.
16

---

Progressive Tempering Sampler with Diffusion
BNEM. We evaluate BNEM using the implementation by Ou Yunang al. (2024). For all tasks, we Ealer-Maruyama discretization with 1000 steps and 100 inner-loops, we also clip the score maximum norm of 1000 during sampling. For MoG-40, we train BNEM for 150 outer-loops to ensure convergence, with batch size of 1000. The noised energies are estimated by 500 MC samples. The total number of target evaluations is 150 x 100 x 1000 x 5000. For MW-32, we train BNEM with 180 outer-loops and change the number of MC samples to 1000, resulting in the total number of target evaluations to be 180 x 100 x 1000 x 1000. For LJ-5,, we use the almost the same hyperparameters as MW-32, except for decreasing the batch size to 128. We train BNEM for 500 outer-loops. The total number of target evaluations 500 100x x 128* x 1000.
DiKL. We evaluate DiKL. with the implementation by He et al. (2024). For Mog-40, we train for 75000 iterations using a batch size of 1024, and we take 15 AIS steps with 10 samples, with an extra MALA of 5 steps. Therefore, the total number of target evaluations is 75000 x 1025 x (15 x 10 + 5). Similarly, for MW-32, we train it for 50000 iterations, leading to a total number of target evaluation of 50000 x 1025 x (15 x 10 5).
C.3. Training DM for ALDP in Cartesian Space
ALDP are chiral molecules, which exist in two indistinguishable forms (L-form and D-form) that are mirror images of each other. While in nature, the ALDPs are found in L-form only, therefore we are interested in generating L-form samples. However, the EGNN implemented by Satorras al. (2021) cannot distinguish these two forms, we reflect the D-form samples generated by PT+DM make them into L-form. For PTSD, since the difference in forms doesn't t influence model training, we leave the D-form samples in all buffers while reflecting them at the last stage only for evaluation.
D. Supplementary Experiments and Results
D.1. Ablation Study for Extrapolation
In this section, we ablate different methods for extrapolating from higher- reeperrtemperrture DMs to lower-temperature one. In particular, we train two DMs on two temperatures T2 < T3, then extrapolate a lower temperature Ti < <T2. We consider the following ways:
1. NN Generalization: train temperatuur--condiiooned DM Do 1,T) on T2 and T3, then directly generalize to Ti by sampling from D9( Ti).
2. Auto-diff TE: train the same temperrauure-condiiioned DM, but extrapolate through the first-order Taylor expansion by auto-differentiation, i.e. Eq. (9).
3. Finite-differenee TE: train two distinct standard DMs, then extrapolate by Eq. (11).
We conduct experiments on GMM-40, DW-4 LJ-13, and LJ-55 benchmarks. For all experiments, we choose Ti = 0.1, To = 1, and T3 = 1.3, where T2 = is the same as the target distributions introduced in App. A. We use the same architecture introduced in App. C.I, and the training of each model is long enough to ensure convergence. The ground truth of each benchmark at Ti 0.1 is obtained by running MCMC on top of the T2 = samples for long enough time to ensure mixing.
Fig. 8 showcases the advantage of Finite-difference TE, where the extrapolated samples can be much closer to the ground- truth ones, compared to the other methods.
D.2. Measuring Performance through Observables
Samples from the cquilibrium (i.e. target) distribution are used for estimating the observables in many physical problems, which are mathematically the expectation of an observable function over the target distribution, i.e. (0)x == Ep(w) (Ox)) To measure the quality of generated samples by different models, we design toy observable function, which is a quadratic
*DW4 is a 4-particle system in 2-dimensional space introduced by Köhler et al. (2020). To clarify, we follow the notation used in (Köhler et al., 2020; Akhound-Sadege al., 2024) where the number indicates the number of particles. While the number 2 in DW-2 indicates the potential is a 2-dimensional space.
17

---

Progressive Tempering Sampler with Diffusion
100
Semo-43
030-
NN Generalization
(b) Auto-diff TE
(c) Finite-difference TE
(d) Ground Truth
Figure 8. Different ways for extrapolation temperature 0.1. In (d), oranges are Diffe--Difeeence TE: blues are ground-truth, which can also be obtained by running steps. 10 steps, steps, and 200 steps Langevin MCMC on top of Finite-Differne Difference TE samples, respectively.
18

---

Progressive Tempering Sampler with Diffusion
Table 7. Comparing PTSD with other neural sampler baselines. We measure toy observable O(X) Bp(tr) 2). We report Mean Absolute Error (MAE) corresponding to ground truth observable, which is estimated by 10000 samples from the target distribution. For each model, we use 10000 samples for estimating O(X). * indicates that the method diverges or is significantly worse than others.
GMM (d
MW32 (d 32)
LJ55 (d 165)
FAB
1.91
4.09
CMCD
7.03 20.83
3.76=0.1
DDS
0.85:1.50
9.89 =0.13
iDEM
6.57
9.28 =0.06
28.72
BNEM
7.42 +0.63
11.09 #0.18
9.48 +0.01
DiKL
7.59 #0.76
3.24 +0.17
PT+DM
12.09 #0.81
4.31 10.10
0.34 00.03
PTSD (ours)
1,35 10.69
1.49 10.16
1.86 10.04
Langevin
(d=2)
UJ5 (d=165)
sleps (d=165)
Pareto Frontier
64.25-
f
CMCD
FAB
0.3
-100
100
-64.0
0.2
150
-150
64.75-
IDEM
DIKL
BNEM
-200
200
65.25-
f
PT+DM
0.0
-250
(ours)
250
-65.50-
10a
10"
10a
10P
16th
10"
10a
10a
10t2"
10
10e
16k0
16
10a
10
Target Density Evaluations
Target Density Evaluations
Target Density Evaluaatons
Target Density Evaluations
Figure 9. Log-likelihood Ex(=) [log 9modet (z)) and energy evaluations for all the models. To ensure consistency, the log-likelihood calculated by fitting diffusion model samples generated from given model.
function ((x) (x a) - o). To make the observable function be SE(3)-invariant for IJ system, remove the mean of samples and ensure rotation-invarian for Ozz), i.e. O(Rx) Oz)), by taking and C' I.
Table 7 reports the MAE of the observable estimated by 10000 samples from different models, where the ground truth is obtained by Monte Carlo estimation through 10000 samples from the target distribution.
D.3. Measuring Performance through Log-Likelihood
Negative Log-Likelihod (NLL) is a statistical metric that measures the distance between the target distribution and the probabilistic model po. which is computed as follows
NLL(Ps: p) EE, Po()).
(19)
Notice that computing NLL requires access model density, which is intractable in both method and most of baselines. Only FAB implemented with Normalizing Flow has tractable model density. To remedy this, we train additional diffusion model to 10000 samples from each model and evaluate the log model density of generated sample zg through the probability-flow ODE (PF-ODE) as follows
logro(zo) = (og))((1) + ( ('v. f(a,,, f)dt, with I, =0 1011// S, [( ujdu.
(20)
Note that, while the above model density is exact and guaranteed by the instantaneou--chang--oowaa introduced in two ways: (1) the discretization; and (2) the variance of the estimation for the divergence term V using Hutchinson's 's estimator.
bias can be
The results are shown in Fig. 9. PTSD is consistently in the Pareto Frontier with respect to energy function evaluations and log- log-likelhoodd on all of the data sets.
19

---

Progressive Tempering Sampler with Diffusion
D.4. Alanine Dipeptide Experiment Setup and Results
We used 5 temperature geometrically spaced temperature levels for both the PTDM and PTSD models, spaced geometrically from 300K to 1500K. We initialised sampling by running single PT chain on the top two temperatures for 1 107 steps (total 2 x 102 energy evaluations), and obtained buffer of 200000 samples by subsampling. We trained the model for 100000 epochs on those 200000 samples with a learning rate of 2 x 10-3. At each step, we then generate 100000 samples on the next level and do IS resampling with the truncation quantile set to 0.8. We pick a random subset of 1000 samples, and run PT for 1000 chains for each, and pick every 10th sample from these chains, resulting in 100000 new samples that we mix with the original IS resampled results. This requires 2 x 1000 x 1000 = 2 x 10 energy evaluations at each extrapolation step. We have such steps, and the IS requires 100000 x 3 = 3 x 10y energy evaluations, resulting a total of x 102 +2 10 3+33 3 x 10a- 2.63 x 10 energy evaluations. We optimised the learning rate and truncation quantile hyperparameters to get to our final results. We chose the number 1000 for the amount of PT chains at each extrapolation such that we get a small enough energy evaluation budget, and early experiments showing that this was enough not to destabilize the training process.
For PTDM, we ran parallel tempering for 5,2c6 steps on the same 5 temperature levels, resulting in 5.2 x 10 x 2.6 x 107 energy evaluations. We trained the model for twice as long as we use to train each individual diffusion model our PTSD implementation with a learning rate of 4 10-3 We optimized over the learning rate to get to the final results. The setup of the 5 temperature levels was chosen after noticing in initial experiments that it works well for parallel tempering in particular, and we did not tune it for PTSD.
Fig. 10 shows the marginals of Fig. 7. Although PTDM does have less bias in the sigh-probbbility regions, the relatively short running time for PT has left it unable to effectively model the small mode on the right side of the ø plot. Table 3 shows the log-likelssoods as estimated with the probability flow ODE. We also evaluated the effective samples sizes (ESS), but the values for both diffusion models were very low (less than 1%) and showed high variance. We hypothesize that this due noise in the probability flow ODE causing some outliers with unusually low log-likelihood values, which causes these samples to dominate the importance weight distribution. In the experiments, to counter this. Fig. 11 shows the energy histogram of samples generated by the model, compared to the ground-truth distribution. The energies are in general higher for PTSD, indicating that the distribution is more spread out than the ground-truh distribution.
use truncated importance sampling
The reason we run MCMC from a subset of samples is that running it from all of the samples entangles the amount of samples we use for importance sampling with the amount of energy evaluations needed for MCMC. IS and MCMC have complimentary effects in the model: The benefit of importance sampling is that it is highly energy function evaluation efficient, using only one energy function per generated sample. The natural tradcoff is that it requires more compute for diffusion sampling and liglliiel estimation. On the other hand, benefit of MCMC sampling is that it can be used to increase the diversity of the IS outputs and obtain better coverage of the distribution for training the next diffusion model. Another advantage is that using correlated samples from the MCMC chains is also energy evaluation efficient. Accordingly, we noticed in our experiments that running a relatively small amount of longer MCMC chains from a large amount of IS generated samples results in a set of samples that has good distribution coverage while keeping the energy evaluation count low.
D.5. Additional Visualizations for Generated Samples
Plots Fig. 12 and Fig. 13 visualise the GMM and MW32 samples from the different baseline models.
D.6. Results on LJ55 with additional Langevin dynamics
The original evaluation scheme for the LJ55 task in (Akhound-Sadeg)) et al., 2024) involved running additional steps of Langevin dynamics on top of the pure neural sampler outputs. our main table, we chose to follow (OuYang et al., 2024) and show the results for the pure neural sampler outputs, more directly compare the performance of the base generative models. Here we provide results for the MCMC refined evaluation scheme, where we ran 1000 Langevin dynamics steps for each model generated sample. The sample evaluation results are shown in Table 8. The overall pattern is similar to the one in Table 1, where BNEM is the best, followed by PTDM, PTSD and iDEM. Fig. 9 shows the logelikelihood results with respect to energy function evaluations, placing PTSD and PTDM at the Pareto frontier.
20

---

Progressive Tempering Sampler with Diffusion
1.0
Ground Truth
0.8
PT+DM
Marginal density
PTSD
0.6
0.4
0.2
0.0
-T1
-T/2
T1/2
11
-11
-1/2
n/2
π
Φ
Figure 10. Marginal distributions the and ab angles in the alanine dipeptide molecule. While PTDM captures the lower part of the values accurately, the small mode the right side of the plot is better represented by PTSD.
Ground truth
0.06
PTSD
Density
0.04
0.02
0.00
-50
0
50
100
Energy / KgT
Figure 11. The energy histogram of samples generated by model on the alanine dipeptide molecule, vs. the energy histogram of ground-truth samples. The PTSD samples tend slightly higher energies, indicating that the distribution slightly more spread out than the ground-truth distribution.
21

---

Progressive Tempering Sampler with Diffusion
-0
-20
-30
-00
-30
-32
C40
-30
-20
-25
-0
BMW
-10
-10
-10
Figure 12. Visualising the samples generated by PTSD and other baselines on MoG-40.
Table LJ55 results for neural sampler methods using the evaluation scheme where additional MCMC steps top of the generated samples. We measure (best, second hest) the TVD and MMD between Energy histograms, and W2 distance hetween data samples.
LJ55 165) MMD $
TVD L
W2
iDEM
0.42 20.00G
0.10 =0.001
1.83 #00001
BNEM
0.09 1000001
0.00 10.000
1.81 1:0.000
PT+DM
0.15 10.004
0.01 10.001
1.82 10.001
PTSD
0.29 10.001
0.05 =0.001
1.86 100.001
22

---

Progressive Tempering Sampler with Diffusion
Ground Tith
DEV
16
DNCD
F150
Figure 13. Visualising 2-dimensional slices of samples generated by PTSD and other baselines on MW-32.
23


Reviews:
[{"review_id": "56H8rb2lv3", "reviewer": "Reviewer_G8RJ", "summary": "The article describes a method based on diffusion model in order to sample from unnormalized densities, such as $p(x) P \\frac{\\tilde{p}(x)}{Z}$. The approach is based on parallel tempering: many diffusion models are trained to match the distribution $p$ at various temperature. The general idea being that at large temperature, it is easy both to produce samples from $p$ and to reproduce faithfully the distribution's density using the samples. Therefore, first models are trained at large temperature: samples are easily generated using MCMC and the diffusion model are trained upon them. Then, lower temperature diffusion model are trained using eq. (11) starting from the weights learned with a previously trained machine (at a slightly higher temperature). The samples are corrected using importance sampling to correct the bias.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed feedback and your constructive comments. We hope that our detailed responses would solve your concerns. \n\n# Questions:\n## Q1: Metrics related to physics\n\nWe follow the metrics used in previous literatures, e.g., iDEM, NEM, DiKL, FAB, etc. In these baselines, they do not evaluate on any physical metrics. \n\nOne potential metric to use is to estimate the expectation of some manually crafted function using samples generated by our approach, and compare it with the value estimated using ground truth samples. We can view this as a “toy”  physical property. In practice, we use a quadratic function of the configuration, i.e., estimate $\\mathbb E_{p(x)}[ x^T x ]$. We compare the estimated integral with the ground-truth using the Mean Absolute Error (MAE) and normalise the error as a percentage of the ground-truth integral value. We estimate 90% confidence intervals with bootstrap resampling. As we can see, our method can still deliver good results consistently. \nWe will include this result in the camera-ready version.\n\n| Method | MW32         | LJ55           |\n|--------|--------------|----------------|\n| IDEM   | 8.97 ± 0.61  | 28.7251 ± 0.02 |\n| FAB    | 4.08 ± 0.14  | -              |\n| DIKL   | 3.19 ± 0.26  | -              |\n| DDS    | 9.89 ± 0.12  | -              |\n| CMCD   | 6.71 ± 0.26  | -              |\n| BNEM   | 10.61 ± 0.27 | 11.61 ± 0.05   |\n| PTSD   | 1.49 ± 0.16  | 1.87 ± 0.07   |\n| PTDM   | 4.32 ± 0.16  | 0.34 ± 0.04            |\n\n\n## Q2: How does the method depend on the number of temperatures?\n\nMore temperatures can (1) reduce the burden of training models for each temperature, and (2) can make our temperature guidance approximation more accurate. \n(1) is because, in practice, we always **fine-tune** our new diffusion model from the model trained on the last temperature. Using more temperature levels can reduce the difference between levels.\nFor (2), we can look at Eq (11) in our manuscript to see this. When T1 -> T2, the guided score will become accurate. However, similar to PT, running our approach with too many temperatures can make the pipeline less efficient. In practice, we found matching the temperature schedule with that of standard PT can usually lead to a good performance. We provide an additional ablation study for it, by differing number of temperatures for training PTSD on the GMM problem: https://anonymous.4open.science/r/PTSD-rebuttal-icml2025/num_temp_ablation.png.\n\n## Q3: Why is the W2 distance better, but the other metrics are not as different between different methods?\n\nThis is an interesting observation. We suspect that this is because W2 directly measures the difference between sets of samples, while TVD and MMD are computed  on the energy values for the samples, which loses more information. \nWe can also see our approach achieves better results in the above, reflecting a consistent trend with W2.\n\n\n## Q4: Convergence of the proposed method:\n> one weakness of the method is the lack of \"proof\" that the algorithm should converge toward the correct probability distribution in some limit. \n\nIn fact, we can always get better results by either increasing the number of temperatures or increasing the local parallel tempering refinement steps. \nWe can view these limits as just running annealed Langevin with a model trained along the sampling procedure. In this case, the only error will be the intrinsic model imperfection. Therefore, the convergence is in fact guaranteed.\n\n# Other concerns: \n## Q5: Potential failure case for target distribution with well-separated modes\nThank you for raising this insightful concern. We agree that our algorithm could potentially require many temperature levels for good performance if standard PT suffers due to some phase transition / separated modes. However, we emphasize that this limitation is not unique to our approach -  standard PT can also require more chains and require task-specific design (for example, the trajectory PT in the case you mentioned). \nWe will include this discussion in our camera-ready version, and we believe this discussion could make our paper stronger.\n\n## Q6: Computational cost\n>The main limitation might be the burden to train a large amount of diffusion models, when the number of temperatures levels become large [...]\n\nWe agree that this can be slow. However, in practice, we always **fine-tune** our new diffusion model from the model trained on the last temperature. Using more temperature levels can reduce the difference between levels, and hence, the training cost per level will also be reduced.", "claims_and_evidence": "The paper does not claim any particular theoretical or practical results. It is mostly based on an expansion of the score function as a function of a temperature parameter in order to learn diffusion model for the unnormalized density at various temperature. It shows that this approach seems to work in a set of benchmarks.", "methods_and_evaluation_criteria": "The method is based on the training of various diffusion models to adjust on a probability density that is annealed from a \"high temperature\" regime where it is easy to sample from and from which samples are first generated with Monte Carlo to train the first stage of the diffusion models. Then, using both a Taylor expansion of the score function and importance sampling, it progressively learn the set of diffusion models to adjust toward the target distribution reaching $T=1$ for the annealing parameters, without the need to sample from the target distribution. \nThe method is evaluated on a set of benchmarks and compare to other methods. Three different measures are used to compare the different models. \n\nOverall the criteria seems adequate for the considered setting.\n\nThere is no comment about how hard these benchmarks are in the text, such as a measured Monte Carlo relaxation time or any quantitative criteria.", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "The experimental design is sound.", "relation_to_broader_scientific_literature": "The method is related to parallel tempering. An aspect which limits the use of PT is the presence of first order transition along the temperature path, which typically occurs when the data distribution has well separated modes which is quite common in ML. This requires then extensive number of temperatures. This is discussed for instance in Béreux et al. \"Fast, accurate training and sampling of Restricted Boltzmann Machines\"  (ICLR 2025). I would imagine that the temperature expansion which is used in the diffusion model is going to break down as well, rendering the method inoperative in this case?", "essential_references_not_discussed": "I did not find that an important reference was missing.", "other_strengths_and_weaknesses": "- The proposed method combining PT and diffusion models is original, interesting and promising.\n- The main limitation might be the burden to train a large amount of diffusion models, when the number of temperatures levels become large \nthe computational cost  and how this compares to other methods is not clearly discussed", "other_comments_or_suggestions": "no additional suggestions.", "questions_for_authors": "This work shows how to use tempering to learn unnormalized probability distribution. I have some questions:\n* the authors use the method for instance on Lennard-Jones particular. Is there any metric related to the physics of the probability distribution that can be used in such case to attest the \"goodness\" of the generated samples ?\n* How does the method depend on the number of temperatures ?\n* The method seems to have a better W2 distance in general than the other methods, is there a reason why it should be the case ? (the other indicators are not as different between the different methods).\n* I have the impression that one weakness of the method is the lack of \"proof\" that the algorithm should converge toward the correct probability distribution in some limit. For instance, with MCMC, in theory we can always perform a good sampling by increasing the number of MC steps. Would there be some similar results for the setting used in this work ?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VAeChw7zeh", "reviewer": "Reviewer_v828", "summary": "This paper focuses on the problem of sampling from the unnormalized densities. The authors recognize the drawbacks of MCMC with Parallel Tempering as well as the ones of recent neural samplers (mostly based on the diffusion process). In particular, they proposes a novel sampling method  — Progressive Tempering Sampler with Diffusion — that trains diffusion models sequentially across temperatures. The main idea is to start from training the high-temperature diffusion models, which are used to generate a lower-temperature samples, slightly refined with MCMC approach. Such samples are then used to train the lower-temperature diffusion model. The method is compared against other sampler on standard benchmarks, where the authors focuses on the number of target evaluations during training and sample-based metrics.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed feedback and your constructive comments. We now reply to your concerns one by one. Should you find our reply satisfactory, we kindly encourage you to raise your score.\n\n## Q1: Limited empirical evidence & lack of scaling PTSD into more complex problems\n\nTo showcase the superiority and better scalability of our method, we conduct experiments on a more complex system-Alanine Dipeptide in Cartesian coordinates, a task that most of the neural samplers failed on. The results of experiments are provided in https://anonymous.4open.science/r/PTSD-rebuttal-icml2025/ramachandran.png. As we can see, our algorithm provides high-quality samples on this challenging task, with very few energy function evaluations. (uses almost an order of magnitude less evals than FAB)\n\nWe will include this results in our camera-ready version.\n\n## Q2: lack of comparison against PIS, DIS, or GFlowNets\nThank you for suggesting these baselines. However, we highlight that PIS, DIS and GFN essentially have the same property as DDS: they mostly use the same model architecture as DDS (parameter the network with a score term); and they require simulating the entire trajectory, which requires hundreds of energy calls to calculate the loss once. Therefore, we use DDS as a representation for this family of approaches.\n\n## Q3: Lack of results for FAB on LJ55\n\nThe result for FAB on LJ55 is not shown because its training diverged in our experiments. Similar results were also observed in iDEM: From iDEM Fig 9, we can see FBA is significantly worse than iDEM.\n\n\n\n## Q4: Different results for iDEM on LJ55\nWe note that in iDEM, they obtain their results on LJ-55 by running a few steps of Langevin dynamics on top of the generated samples. This can be seen from their codes: \n https://github.com/jarridrb/DEM/blob/main/configs/experiment/lj55_idem.yaml where they set “num_negative_time_steps: 10”. This means they run 10 steps of Langevin dynamics in https://github.com/jarridrb/DEM/blob/main/dem/models/components/sde_integration.py\n\nAs this trick is applicable to our approach and all the baselines, we present the results without it.\n\n## Q5: lack of logZ, NLL, etc.\nComputing logZ and NLL requires model density. Different baselines will have different ways to calculate logZ and NLL: For CMCD and DDS, we need to discretize the SDE into sequence of Gaussians; for diffusion models (ours and PT+DM), we can either use this approach, or estimate it with change of variable on PF-ODE. As different approaches have different discretizations or different underlying processes, this would make a fair comparison more difficult.\n\nIn the end of the day, one cares ultimately about the quality of the samples. Therefore, we believe that sample-based metrics are more straightforward and comparable across methods. \n\n## Q5: Lack of interatomic distances for LJ \nwe have provided it in https://anonymous.4open.science/r/PTSD-rebuttal-icml2025/LJ55_interatomic.png. We see that PT+DM is a strong baseline for LJ55, but PTSD is a close second in terms of visually matching with the ground-truth distribution.\nWe will add this visualization in our camera-ready version for a more comprehensive evaluation.\n\n\n## Q6: Ablation studies\nFollowing your suggestion, we studied the effect of the amount of temperatures in the GMM task, and report the results in https://anonymous.4open.science/r/PTSD-rebuttal-icml2025/num_temp_ablation.png. \n\nWe can see the trend is monotonic: all of the metrics improve with more temperature levels. This indicates we can always use more compute to obtain better results in our method.\n\n\n## Q7: Lack of comparison of the memory cost of training samplers\nOur approach encounters a larger memory cost in two places. We now analyze these two costs one by one:\n\n1. When we run MCMC on the samples from the buffer, if we parallelize them, we will encounter a larger memory cost compared to PT.\nIn fact, we view this as an advantage rather than a limitation as this indicates that our method enables efficient parallelization of MCMC into multiple independent chains, with the number of parallel chains determined by the buffer size. On the contrary, standard PT can only parallelize over as many chains as the number of temperatures. The buffer size is normally much larger than the number of temperatures; our approach can, in fact, allow for more massive parallelization.\n\n\n2. When we sample using our proposed temperature-guided score, we will need to save and call two models together in memory; we may also need to backprop through these two models. In total, they will increase the memory cost to ~4x of standard diffusion model sampling.\nWe agree that this a potential limitation for our approach, and we will include a discussion on this in our camera-ready version. \n\nThank you for this insightful question. This allows us to realize an additional advantage and limitation of our algorithm, which we believe have made our paper more solid and stronger.", "claims_and_evidence": "The claims regarding findings the advantages and drawbacks of neural samplers and Parallel Tempering, as well as introducing a novel method (being more efficient in terms of number of target evaluations during training) are well justified. Same, regarding the theoretical findings.\n\nHowever, I think that the claimed superiority to neural samplers in terms of generated samples and better scalability is not well justified, mostly due to the limited empirical evidence.", "methods_and_evaluation_criteria": "Yes, but they are limited. I would like to see the comparison against other neural samplers like PIS, DIS, or GFlowNets. Moreover, I’m concerned by the lack of standard metrics for sampling problems (logZ, NLL, etc.), which are missed in the experiments. \n\nIn particular, the authors do not provide the results for FAB on LJ55, which was one of the first methods checked in this setting and the results for TVD for iDEM are various from the ones from the original paper.", "theoretical_claims": "Yes, I've checked and haven't found any obvious drawbacks.", "experimental_designs_or_analyses": "Yes, for part of my concerns, please see the Methods And Evaluation Criteria section. Moreover, I generally think that the samples quality evaluation is missing, e.g., plot of interatomic distances for LJ potentials (comparing other methods), logZ or NLL metric, etc.\n\n**Ablation studies:** I think that the influence of used sequences of temperatures during training, and the influence of a buffer size might be beneficial for this work. Moreover, I would like to veto see the comparison between models on their memory costs.", "relation_to_broader_scientific_literature": "The key findings are based on existing knowledge and methods (PT and diffusion), but the proposed method is novel.", "essential_references_not_discussed": "The references are discussed in general.", "other_strengths_and_weaknesses": "**Strengths:**\n\n[1] Proposing a novel method of combining the PT with diffusion for more efficient training of samplers.\n\n[2] The method seems to be theoretical justified, and significantly lowered the number of needed target energy evaluation.\n\n\n**Weaknesses:**\n\n[1] Missing results for some benchmarks (like FAB).\n\n[2] Missing significant metrics for sampling quality.\n\n[3] Lack of comparison of memory cost of training samplers, and lack of scaling PTSD into more complex problems (since it requires lower number of target energy evaluations).\n\n[4] Regarding all of the mentioned weaknesses, I think that this paper (in the current form) has limited significance.", "other_comments_or_suggestions": "For comments and suggestions, please see the previous sections.", "questions_for_authors": "For questions, please see the previous sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rQU2PuxvgL", "reviewer": "Reviewer_WWs4", "summary": "The paper introduces a new sampling algorithm—Progressive Tempering Sampler with Diffusion (PTSD)—which aims to efficiently sample from unnormalized densities by combining ideas from traditional parallel tempering (PT) and modern diffusion-based neural samplers. The method is evaluated on several multimodal targets, where it demonstrates significant improvements in sample quality and target density evaluation efficiency over existing neural samplers.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We hope that our detailed responses would solve your concerns. If any questions or concerns arise, please do not hesitate to let us know, and we will address them properly. However, should you find our reply satisfactory, we kindly encourage you to raise your score.\n\n\n## Q1: the paper is hard to read in general and is applied only to a niche audience\nThank you for your comment. We will be happy to update our paper if you could kindly provide more information in detail about the part that you found hard to read.\nAdditionally,we respectfully disagree with the argument that this paper is niche. Sampling from unnormalized densities is a long-standing task in many areas, including Bayesian inference, Molecular simulation, physics, statistical chemistry, etc. There are many approaches trying to address this problem. Here is a non-exhaustive list of recent advancements:  [1, 2, 3, 4, 5].\n\n## Q2: The proposed PTSD method is kind of slow in the experiments conducted. Any specific reason for that?\nThis seems to be a misunderstanding. As we can see from Fig 6 and Tab 2 in our manuscript, our method is actually the most efficient approach in terms of energy function evaluations. It is order-of-magnitude faster than other neural sampler baselines. Could you specify where you found our method slow?\n\n## Q3: Proposed method is limited to simple models\nOur method does not rely on any specific architecture. One can apply any architecture to our proposed pipeline.\n\nAlso, for the problem we are addressing, we believe the model (e.g., EGNN) we use is indeed “SOTA” as it balances well running cost and performance. The architecture we use is a standard choice for the n-body system and small molecules [4, 6, 7]. Therefore, it would be very helpful if you could clarify what kind of model you expect us to evaluate our approach on.\n\n## Q4: The experiments in general seem kind of simplistic, and the results in Table 1 are also not that good. Is there a reason for that?\n \nThank you for your comment. We used the wrong hyperparameter (model size) when we ran the results for Table 1, and hence LJ55 is slightly worse than expected. We have fixed this issue, and we can see the updated results in https://anonymous.4open.science/r/PTSD-rebuttal-icml2025/updated_main_table.png, where both the energy-MMD and data-W2 are improved. We also see that number  of energy function evaluations of our approach is **less** than PT+DM across all 3 tasks. The visualization of the interatomic distance in https://anonymous.4open.science/r/PTSD-rebuttal-icml2025/LJ55_interatomic.png also reveals that our method does have advantages over BNEM in this regard, and is a strong competitor to PT+DM. \n\n\nAdditionally, we provide results on **alanine dipeptide** in Cartesian coordinates, which is a challenging task where most of the baselines failed. This significantly highlights the capability of our approach. The experimental results can be found in https://anonymous.4open.science/r/PTSD-rebuttal-icml2025/ramachandran.png.\n\n\n## Q5: Paper lacks proof\nOur approach relies on Taylor expansion and finite difference approximation for first-order derivatives. There are standard results in calculus. \nCould you please explain which part you suggest us to include a proof?\n\n\n[1] Noé, Frank, et al. \"Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning.\" Science 2019.\n\n[2] Doucet, Arnaud, et al. \"Score-based diffusion meets annealed importance sampling.\" NeurIPS 2022. \n\n[3] Midgley, Laurence Illing, et al. \"Flow Annealed Importance Sampling Bootstrap.\" ICLR 2023. \n\n[4] Akhound-Sadegh, Tara, et al. \"Iterated denoising energy matching for sampling from boltzmann densities.\" ICML 2024.\n\n[5] Vargas, Francisco, et al. \"Transport meets Variational Inference: Controlled Monte Carlo Diffusions.\" ICLR 2024.\n\n[6] Hoogeboom, Emiel, et al. \"Equivariant diffusion for molecule generation in 3d.\"  ICML 2022.\n\n[7] Klein, Leon, Andreas Krämer, and Frank Noé. \"Equivariant flow matching.\" NeurIPS 2023.", "claims_and_evidence": "Yes the claims look clear and are supported by basic math and some limited experiments.", "methods_and_evaluation_criteria": "Yes they do.", "theoretical_claims": "Yes I checked the math but the paper lacks proofs.", "experimental_designs_or_analyses": "Yes they look valid.", "relation_to_broader_scientific_literature": "I would say the findings are kind of limited and not very interesting to the broader scientific audience.", "essential_references_not_discussed": "Everything is cited.", "other_strengths_and_weaknesses": "Weaknesses: \n* the paper is hard to read in general and is applied only to a niche audience.\n* The proposed PTSD method is kind of slow in the experiments conducted. Any specific reason for that?\n* I would also say that the method is very limited to simple models. The authors have to try it to bigger and more state of the art models.\n* The experiments in general seem kind of simplistic and the results in Table 1 are also not that good. Is there a reason for that? Cause in its current form the paper is not strong either mathematically or experimentally.", "other_comments_or_suggestions": "No other comments.", "questions_for_authors": "Please check the Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IQ2CRolSQV", "reviewer": "Reviewer_cnuL", "summary": "This paper proposes learning diffusion models at various temperatures using MCMC data at high temperatures. These models are then used to sample from unnormalized probability distributions. The authors employ a Taylor expansion of the target distribution over temperature to derive a temperature-dependent drift term, which is sequentially applied to generate samples at lower temperatures. Biases are corrected using importance resampling. The method is evaluated on three different target distributions commonly referenced in the literature.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed feedback and your constructive comments. We hope that our detailed responses would solve your concerns.Should you find our reply satisfactory, we kindly encourage you to raise your score.\n\n## Q1: More details of the experiments\n\nThank you for pointing out this. We will add them in our camera-ready version.  We now address your concern related to method details you mentioned.\n\n\n### no documentation of the attempts made on the LJ-55 problem\nWe did not save the results for LJ-55 for some baselines as it is significantly worse than others. A similar pattern has already been observed by iDEM (Fig 9).\nWe note that these baselines will anyway require significantly more energy evaluation times than ours (and PT+DM).\n\n### Details about additional hyperparameters in Tab 4\nFor most of the targets, we tune PT+DM so that the swap rate is ~30%, and we use the same schedule for PT+DM and our approach. We also specifically tune PT+DM on GMM40 as it requires fewer levels. So we in fact put similar effort into tuning the temperatures for both approaches.\nHowever, we agree that our approach also has many other hyperparameters to tune, and this can make the tuning more involved. We will update our camera-ready paper to reflect this limitation.\nWe also want to emphysize that more hyperparameters also means a **larger design space**, which has the potential to improve the performance of our approach further.\n\n### More details about the diffusion model used for PT+DM\nTo ensure fair comparison, we employ exactly the **same** neural network architecture and training hyperparameters for PT+DM and PTSD.\n\n## Q2: The experiments are limited to three tasks\n Following your suggestion, we evaluate our model on a more complicated task. As our main focus for the application is molecules, we evaluate our method on Alanine dipeptide (ALDP)  in Cartesian coordinates. This task is highly challenging for neural samplers.As we can see in https://anonymous.4open.science/r/PTSD-rebuttal-icml2025/ramachandran.png, our algorithm still obtain reasonable performance on this task, which strongly support the effectiveness of our approach.\n\n## Q3: The Taylor expansion seems to be a crude approximation when $\\Delta T$ not close to 0\nWe totally agree. However, we found that this crude approximation provides a better exploration performance compared to using the derivative (with auto-diff) directly. We have provide a plot in https://anonymous.4open.science/r/PTSD-rebuttal-icml2025/extrapolation_ablation.png to illustrate this observation.\n\nThis is possibly due to 2 reasons:\n1. The derivative may not be robust, as we only train the model with a finite grid of temperatures;\n2. Our approximation eventually forms a auto-guidance, which may have the potential to reduce the network imperfection.\n## Q4: Figure 1 suggests that PTSD performs better than PT+DM, but Table 1 indicates that it is often outperformed by other methods\nWe made a mistake in the previous LJ55 results. We now have updated the table in https://anonymous.4open.science/r/PTSD-rebuttal-icml2025/updated_main_table.png to reflect this. The energy function evaluation count decreased significantly, while also improving the energy-MMD. Note also the interatomic distance plot, which shows that our method beats BNEM in this regard. \n\nWe can see that our approach delivers a performance that is better or on par with other baselines. \nWe also note that even though sometimes the baselines outperform our approach, the improvement is very small. On the other hand, our method consistently delivers good results on these targets, and even works well on the challenging ALDP target.\n\n## Q5: no error bars are reported\nThank you for mentioning the missing or error bars, we will update them in our camera-ready version.\n\n\n## Reply to Other Comments Or Suggestions\nThank you for pointing out the inconsistency of the PF-ODE writing, the missing references, and clarity of the VE SDE writing. We will fix them. For the other questions:\n\n>How are swaps done?\n\nWe use standard PT swap; the acceptance rate is in eq(2) in our paper.\n\n>How do we sample MW32 ground-truth? \n\nWe follow FAB to first run iid samples from Double Well, and then stack 16 samples together.\n\n\n## Responses to the Questions For Authors\n1. We train our diffusion model by standard denoising score matching\n\n2. For CMCD, and other diffusion samplers, it cannot directly train from data without increasing energy evaluation times. In fact, every time we calculate the loss, hundreds of energy evaluations. On the other hand, our pipeline does not need an extra energy evaluation and hence will be significantly more efficient compared to others.\n\n3. We use 10000 samples", "claims_and_evidence": "The authors claim to achieve significant improvements in target density evaluation efficiency compared to previous diffusion-based neural samplers. However, upon examining Tables 1 and 2, I am not convinced that this claim is fully substantiated. To me it looks like PT+DM performs better in many cases and possibly insignificantly worse in some cases.", "methods_and_evaluation_criteria": "The evaluation criteria are sensible, but it would be beneficial if the authors also reported the Evidence Lower Bound (ELBO). The method is evaluated on only a few problem types, which is less comprehensive than typical evaluations in related literature. Additionally, the 2-D Gaussian Mixture Model (GMM) problem is quite simplistic.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experimental design is logical, and some ablation studies are conducted.", "relation_to_broader_scientific_literature": "The broader relation to scientific literature is established.", "essential_references_not_discussed": "Essential references are discussed.", "other_strengths_and_weaknesses": "**Strengths:**\n\n- The idea of combining diffusion models with parallel tempering is compelling.\n\n**Weaknesses:**\n\n- Important experimental details are missing. For instance, for the LJ-55 problem, it is unclear what architectures are used for each method, how equivariance is ensured, and how many diffusion steps are employed.\n- There is no documentation of the attempts made for methods that diverge on the LJ-55 problem.\n- The experiments are limited to three tasks, one of which (GMM-2D) is overly simplistic. I suggest using more challenging alternatives like GMM-50D and Mixture of Students in 50D.\n- The Taylor expansion with respect to temperature, where $\\Delta T$ is significantly greater than 0, seems to be a crude approximation.\n- The method introduces many additional hyperparameters (Tab. 4), and it is unclear how much tuning is required. It is also uncertain whether the same level of hyperparameter tuning was applied to other methods, particularly PT+DM.\n- It is unclear what type of diffusion model was used for PT+DM. Overall, the experiments should be described in more detail.\n- Figure 1 suggests that PTSD performs better than PT+DM, but Table 1 indicates that it is often outperformed by other methods.\n- no error bars are reported", "other_comments_or_suggestions": "- \"PF-ODE\" is inconsistently written in three different ways in the paper.\n- Line 288: Consider providing a reference to the appendix and reporting the used truncation threshold.\n- Lines 120 ff.: Although not part of the proposed method, could the authors elaborate on whether the swaps are performed with a certain probability after each MCMC update step?\n- From where is the MW32 ground truth data obtained?\n- L.127 ff. please make clear that you are using the Variance Exploding SDE. The notation with $\\dot{\\sigma}$ is rather unusual. so please explain that you are using the same notation as in Song 2021.", "questions_for_authors": "- do you train your diffusion model using Hutchinson’s trace estimator or using score matching?\n- How does your procedure compare to training CMCD and other diffusion samplers on data from parallel tempering and then fine-tuning it on the target distribution?\n- How many samples are used for evaluation in Table 1?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a method for training diffusion samplers of unnormalised densities by iteratively training diffusion models on buffers of samples obtained by parallel tempering (PT) at decreasing temperatures. Notably, this is a simulation-free algorithm, as it involves only data-based (score matching) training of diffusion models, unlike the most successful prior methods; however, it is asymptotically unbiased only in the limit of converged PT at each iteration.

There was disagreement among the reviewers about the fairness of the comparisons (to iDEM with/without Langevin postprocessing and to methods that diverged when run out of the box). The responses largely addressed these concerns, and the authors are expected to include the new results in the final version.

Note: Reviewer cnuL initially did not respond to the authors' rebuttal and AC's requests for comment. While some of their comments have been substantially answered, in subsequent discussion the reviewer was still concerned about reproducibility / experiment details and rigour, and the authors are asked to take this feedback into account (reporting all experiment details and giving error estimates as promised). The review by Reviewer WWs4 was discounted, as it made a number of inaccurate comments, while others were answered in the response.
Decision: Accept (poster)