CREPE: Controlling diffusion with REPlica Exchange
Authors: Jiajun He
Paul Jeha
Peter Potaptchik
Leo Zhang
José Miguel Hernández-Lobato
Yuanqi Du
Saifuddin Syed
Francisco Vargas
Venue: ICLR 2026 Poster (2026)
Keywords: parallel tempering
diffusion model
inference-time control
replica exchange
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Inference-time control of diffusion models aims to steer model outputs to satisfy new constraints without retraining.
Previous approaches have mostly relied on heuristic guidance or have been coupled with Sequential Monte Carlo (SMC) for bias correction.
In this paper, we propose a flexible alternative based on replica exchange, an algorithm designed initially for sampling problems.
We refer to this method as CREPE (Controlling with REPlica Exchange). Unlike SMC, CREPE:
(i) generates particles sequentially, (ii) maintains high diversity in the generated samples after a burn-in period,
and
(iii) enables online refinement or early termination.
We demonstrate its versatility across various tasks, including temperature annealing, reward tilting, model composition and classifier-free guidance debiasing, with competitive performance compared to prior SMC methods.

Published as a conference paper at ICLR 2026
CREPE:
CONTROLLING DIFFUSION WITH REPLICA EXCHANGE
Jiajun He Paul Jeha
Peter Potaptchik
Leo Zhang
José Miguel Hernández-Lobato
Yuanqi Du Saifuddin Syed Francisco Vargas
"Corresponding to jh23836cam ac uk. 'qqual contributions. *Last authors.
'University of Cambridge, Technical University of Denmark, University of Oxford, *Cornell University, University of British Columbia, "Xaira Therapeutics.
CODE
WEBSITE
ABSTRACT
Inference- timennce--meeeeeee control of diffusion models aims steer model outputs to satisfy new constraints without retraining. Previous approaches have mostly relied on heuristic guidance or have been coupled with Sequential Monte Carlo (SMC) for bias correction. In this paper, we propose flexible alternative based on replica exchange, an algorithm designed initially for sampling problems. We refer to this method as CREPE (Controlling with REPlica Exchange). Unlike SMC, CREPE: (1) generates particles sequentially, (2) maintains high diversity in the generated samples after burn-in period, and (3) enables online refinmment or early termi- nation. We demonstrate its versatility across various tasks, including temperature annealing, reward-tilting. tilting, model composition and classifier- frree guidance debiasing, with competitive performance compared to prior SMC methods.
INTRODUCTION
Diffusion models (Ho et al., 2020; Song et al., 2021b;a) have revolutionised generative modelling with their ability produce high-quality samples across diverse modalities, including images (Rombach et al., 2022; Karras et al., 2022), videos (Ho et al., 2022), text (Austin et al., 2021), among others (Watson et al., 2023; Duan et al., 2023). It typically formalised as stochastic process initialised at tractable distribution (e.g., a Gaussian distribution recover the data distribution. This progressive nature not only enables diffusion models to excel at modelling complex distributions but also provides flexible approaches for steering the generation.
fully masked distribution) and evolves to
Inference-time control leverages this flexibility steer the generation of diffusion models, enabling tasks such posterior sampling (Dou & Song, 2024), reward-tilting (Wu et al., 2023; Singhal et al., 2025), tempering (Akhound- -adehh et al., 2025), or model composition (Du et al., 2023). This
class condition: balloon: prompt: a blue balloon
condition: pimwheel; prompt: colorfid pinwheel
class condition: Christmas stocking: prompt: green Christmas stocking
class condition: cab: prompt: yellow cab with dark background
CREPE iteration,
Figure 1: Trajectory of images generated using CREPE for prompted reward-tilting on ImageNet-512,, thinned every iterations. After burn-in, the samples align closely with the prompt.

---

Published as a conference paper at ICLR 2026
N particles pengrated in parallel
particles genemneed sequensilly
Tep
Iqpraed uj Supaigg
N1y 1
m 1
Reweiiht and roompl
Accepted
Rejected
commmization
communization
Figure 2: Comparison between diffusion inference-time control with SMC and CREPE. We visualise the diffusion process using colour shading: darker colours correspond to higher noise/mask levels (large t), while brighter colours indicate states closer to the data distribution (small t). SMC (Left): particles are initialised at = and progressively denoised towards lower noise levels. During denoising, importance resampling is applied to select particles that better satisfy the imposed constraints. CREPE (Right): particles are initialised several different diffusion steps, and they undergo local exploration and communication in parallel, evolving them towards desired constraints. The example shows using SMC and CREPE to debias classifier-free guidance.
was first explored through classifier (and classifier-free) guidance (Dhariwal & Nichol, 2021; Ho & Salimans, 2022), and has since been extended with a variety of approximation or fine-tuning approaches (Song et al., 2023a; Chung et al., 2023; Song et al., 2023b; Schneuing et al., 2024; Ye et al., 2024; Kong et al., 2025; Denker et al., 2024; Liu et al., 2024; Domingo-Enrich et al., 2024; Zhang et al., 2024). However, these methods often rely on heuristic approximations and typically suffer from inaccuracies, while fine-tuning approaches require additional training on data and may still suffer from imperfect optimisation. This has motivated studies to debias the errors (Wu et al., 2023; Skreta et al., 2025; Lee et al., 2025; Singhal et al., 2025; Thornton et al., 2025).
commonly used framework to debias is Sequential Monte Carlo (SMC, Del Moral et al., 2006), where one jointly evolves a set of weighted interacting particles along the generation path towards the desired distribution. SMC debiases the trajectories by importance sampling with potential resampling moves. However, SMC-baser debiasing methods typically suffer from several limitations: (1) it requires maintaining& a large number of particles throughout the denoising trajectory, which can be memory-intensive: (2) SMC tends to suffer from poor sample diversity, observed in several recent works (I.i et al., 2024; Young& & Akyildiz, 2024; Lee et al., 2025), a problem that is especially severe when the number of particles is small; (3) once the sampling process is complete, SMC cannot refine the generated samples. If the outcome is unsatisfactory or new constraints are added, one needs to regenerate new samples rather than iterate on the existing ones.
On the other hand, replica exchange, also known as Parallel Tempering (PT, Swendsen & Wang, 1986; Geyer, 1991: Hukushima & Nemoto, 1996), is a Markov Chain Monte Carlo (MCMC) algorithm providing a computationally dual framework to SMC. PT reverses the roles of parallelism and time in SMC samplers (Syed et al., 2024): instead of propagating batch of particles parallel along the denoising direction sequentially, PT runs chain at different denoising steps in parallel and generates particles sequentially. This reduces the burden of parallelism over large number of particles and enables the continual refinement of the generated samples. However, standard PT and its extensions (Ballard & Jarzynski, 2009; Zhang et al., 2025) were designed for sampling from unnormalised densities, where the target distribution is explicitly known. This highlights key challenge: in inference-time control, we only have access to a pretrained diffusion model. Can PT still be adapted to this selting to harness its desirable properties?
In the following, we answer this question affirmatively. In summary:
We formulate inference-time control with parallel tempering (PT) for diffusion models. Dubbed as Control with REPlica Exchange (CREPE), it shows how PT can be applied directly from pretrained diffusion models without explicit target densities.
We derive PT swap rates for several inference-tin time control applications, including tempering, reward tilting, debiasing classifier-fre free guidance, and model composition, for both the Gaussian diffusion model and discrete mask diffusions (Lou et al., 2023).
We validate our approach across various tasks and modalities, demonstrating improved performance and better inference-time scaling property compared to SMC-based approaches.
2

---

Published as a conference paper at ICLR 2026
2 BACKGROUND
We begin with a discussion of path measures for continuous-time Markov processes, followed by an introduction to diffusion models and their discrete counterparts (Song et al., 2021b; Lou et al., 2023; Shi al., 2023). Finally, we review replica exchange, aka parallel tempering (PT), particularly focusing on its accelerated variants (APT, Zhang et al., 2025).
2.1 PATH
DERIVATIVE AND RADON- MADON-NIKODYA NIKODYM ESTIMATOR
Let X, and X be continuous-time Markov process on state space X within the time interval s E [t, t']. In later sections, we will instantiate these processes as either diffusion or jump processes; for now, we keep the discussion general. Let IF f. denote the path measures defined as the law of the forward process X [a.04] = (X, s)set.. obtained by evolving X, forward from time / to t'. Similarly let Bi, denote the path measure for the backward process X ( (X'))stt.a1 obtained by evolving X μ backward from time time For more intuitive introduction, let's first consider the forward and backward processes at a given collection of time points t = to < t <.... < tk = t'.
The laws can be factored in terms of the forward transition kernels F Fugltu- of the forward process and the backward transition kernels Bic- of the backward process, i.e..
K
K
F. = (((2)))) II Faltto (0 Br. (onn )= = 1 (x II Bea
(1)
By taking ratios and a formal limit as maxe |/e 4--1| 0 (Berner et al., 2025, Proposition B.7.), we obtain the Radon-Nikodym derivative between F Fe, and By, e in the form of the density ratio between the marginals y and p', and a term Re, (Vargas et al., 2024; Berner et al., 2025).
d
(xe) Mo(x))
RI, (x(011))
(2)
dF, f.
Formally, Re, (2)),e) is defined as the ratio of the backward transition dynamics initialised terminat- ing zy and the forward transition dynamics initialised at 2, in the limit max; Its -/--1/ + 0,
I Bie_slee (Ita i (2te)) R (Fit,e1) max, M 1o1-0 I Freleeri (Ftt, (it, .)
lim
We note that Re, depends only on the transition dynamics of forward and backward processes, independent of the marginals μ and p'. When X, and X/ are constructed via stochastic differential equation (SDE) or continuous- time Markov chain (CTMC), we can express (2 analytically in terms of path integral of the drift and rate matrices respectively over æfr,)). which we describe in Appendix A.2. Going forward we will refer to R2, as the Radon- BakonnNikodym Estimator (RNE) between the forward and backward process over the interval [t, following He et al. (2025c).
2.2 DIFFUSION MODELS
Diffusion models (Ho al., 2020; Song al., 20bla;b) construct continuous time Markov process X, over a state space X with marginal distribution Pt at time interpolating between given target data distribution Po at time 0 and a reference noise distribution P1 at time
Let Pi, be the path-measure defining the law of paths X [1,4r] = (Xx) on the time interval [t, t'). We can equivalently express X [1.101 as forward process X evolving pt forward in time from to t', or as a backward process X, evolving Pr backward in time from t' to f. Since X s and X are constructed to fime-reversals of each other, we have forward and backward path measures coincide with Pr,e and hence the Radon-Nikodym derivative between them equals for any path F(t, It follows from Equation (2), the marginal densities ratio between Pa (Ix) and pe (mzr can be expressed in terms of R,,,, the RNE for the diffusion model over [t. t').
Pe (xv ((te)//p((ee (Pp( = R.,y (x
We describe two classes of diffusion models used in the literature when X = RRd
and and when X is finite.
3

---

Published as a conference paper at ICLR 2026
Gaussian diffusions Gaussian diffusion models (Song et al., 2021b; Albergo et al., 2023) construct Markov processes over X = IR. We define the forward process X, obtained by integrating the forward SDE with drift ft and diffusion coefficient ot initialised at the data distribution Po running forward in time, terminating at Gaussian noise X1 Pl.
Xo Pos X Pt. ax, f((x)) dt + or dw.,
(5)
Similarly, the backward process X₂ is obtained by integrating the backward SDE terminating at Gaussian noise Xi pı backward in time:
X ~L, X, Serpo Pt, =x of + or dW.
(6)
where 9t- = fc of V log Pt, and V log pt is the score function learned by a neural network.
Discrete diffusions When X is finite, discrete diffusion models (Campbell et al., 2022; Lou et al., 2023; Shi et al., 2024) define pt as a probability vector of size [X] representing the law of X The process is defined by integrating the forward CTMC initialised at the data distribution po with rate matrix A E |X| terminating at fully masked distribution P1:
Xo Pa. X, Pt. d.pt A, Pt-
The reverse process is encoded by the backward equation terminating at the fully masked distribution:
X PL, X. ~e,
0.p: A T Pt.
(8)
p=ly) Here, Mg is the backward rate matrix defined as Nz, v) = (v 2) p(ts)
)
for for y # x and A, a))
Σytz (1 y), where pily) is known the concrete score and is learned using neural network. (a(
2.3 REPLICA EXCHANGE AND ACCELERATED PT (APT)
Replica exchange, also known as parallel tempering (PT), was originally developed for sampling from multimodal distributions To over X. To do this, we first introduce annealing path of distributions (Ta)EEjo.j over X. interpolating between the target xo and reference πi chosen to be easy to sample from. PT construct a Markov chain X, = (XD
X.M over X++ invariant with respect to an annealing schedule discretising
the product tt ... TtM where to the annealing path. Given X,, at time we generate Xn+1 using communication step and local exploration step. The communication step performs sequence of Metropolised swaps moves between adjacent component of X.. It is advantageous apply a non-reversible communication (Okabe et al., 2001; Syed et al., 2022): the swap between components m and m of X,, is proposed only at iterations with matching parity, m mod 2. Then, the local exploration step updates each m-th component with a MCMC move targeting 7, Both local exploration and communication can be carried out in parallel for each m.
< tM
The standard formulation of PT proposes swapping neighbouring samples directly. This becomes inefficient when the neighbouring distributions of the annealing sequence have little overlap. To address this issue, Ballard & Jarzynski (2009; 2012); Zhang et al. (2025) proposed APT, extending the communication step the path space of stochastic processes. Concretely, an accelerated PT (APT) swap move between states (z, x²) targetting गe and respectively simulates (I) a forward proposal Markov process X., and (2) backward proposal Markov process X', over some time interval ['). The forward proposal X. propagates X backward proposal propagates X x' backward in time from t' to f. We then replace (æ, x²) with the terminate states (x), with probability as, (X
forward in time from / L', and the
[r,e] X ((.ee) equal to,
{
{ de d
d (ar)) de
(9) Here Que denotes the law of the forward paths X [t.e9 (X initialised at X, ne and Lur denotes the law of the backward paths X f/e. (Xx))s terminating at X my. The swap move remains valid when the paths are discretised long as the simulation of the proposal and the calculation of the Radon-Nikodym derivative follow the same discretisation.
ane (Tp.e, = min
4

---

Published as a conference paper at ICLR 2026
Algorithm 1 CREPE: Control with REPlica Exchange
Inputs: J pretrained diffusions; annealing path (TT))e[o.)). discretisation schedule (t., m PT iterations Output: target samples {X.,) In=1
Initialise X, (X3- ......... X6 by running diffusion model.
for 1.... N do
Run PT.
X, = (X 2..... ) +- X
for m = n mod 2 do
Communication (parallelise)
1 <- mm-1 and L' <- 4m
b Diffusion time interval
X, < X and X &- X
D. Generate proposal paths
Simulate proposal paths X, and X; for a [
b Equations (10) and (11)
Compute Ri R......,R,..
X p.n and X l.1
See Appendix A.2
01 Dipe( [1,0], X1 (t.r))
b. See Equation (13)
(X X +- (X, x)) with probability Or,e'
b. Swap move b Local Exploration (parallelise)
(Optionally) Update X m O..... . with local exploration.
3
METHODS
Given diffusion models p for the data distributions 16 for 1,....,J aim generate samples from a new distribution my related to ph without retraining the model. Some common examples of such tasks include tempering, reward-tilting wiwaadd-lliing/osteriorr sampling, and model composition.
tempering:
<n(a)) (x)) with inverse-temperature 3 0:
reward-iilting/pssterio sampling:
<a(a) (x) exp(ro(1)) with reward/iikllhowwl ro(2))
model composition:
To(2) II, (x) composing J diffusions ph. 1, J.
These are not the only options. In fact, one can also combine these tasks. For example, debiased classifier-free guidance aiming sample from (x) x po(x>)=" pa(: cc) can be achieved by combining tempering with composition. We will also demonstrate other combinations in Section
This section shows how this can be achieved using the accelerated parallel tempering framework outlined in Section 2.3. We can adapt this to obtain the Control REPlica Exchange (CREPE) algorithm summarised in Algorithm 1. We will outline the ingredients for CREPE, i.e. (1) annealing path, (2) a communication move, and (3) local exploration move.
3.1 ANNEALING PATH, COMMUNICATION AND LOCAL EXPLORATION
Annealing path We first introduce an annealing path of distributions (TT)tefo.1] interpolating between the target distribution no when = ( and a reference distribution where inference is tractable Ti when f 1. For example, we can assume mi is Gaussian the case of Gaussian diffusion or fully masked distribution in the discrete diffusion case. We additionally assume the marginal density ratio of the annealing path मe as functions of the marginal density ratio for the pre-trained diffusion model pd so that one can plug in the RNE relation Equation (4). Some common examples of annealing path include for inference time control include:
express
tempering:
(x) pf (x)2 with inverse-temperatuue 3 0; =e(x) pl (a) exp(r.(=)) with reward/likelihodd r=(x);; (((x) II, pl (z) composing diffusions pl. j 1,.
reward-tiltingppostee sampling:
model composition:
We now introduce the forward and backward proposal processes in the APT framework and show how to compute the acceptance probability. Here we focus on our discussion on the communication between two distributions n, and ne, and the same formula applies to any pair.
Communication
We first introduce the proposal processes for the communication. Precisely, we introduce Markov processes X and X 3. so that we can simulate the dynamics forward and backward in time, respec- tively. Concretely, in the of Gaussian diffusions defined in Equations (5) and (6). introduce the forward and backward SDEs driven by the same noise o, but with user-specified drift a, and b, In the case of discrete diffusion, introduce forward and backward CTMC with user-specified
5

---

Published as a conference paper at ICLR 2026
rate matrices A, and Be,
dx;= - ax(.)d + o.dW., Backward proposal: =x,=,,(X;)) + a,
Forward proposal:
deg: - A 0: 0.q. - B, dk
(10)
(11)
Here we use the arrow to indicate that the forward and backward processes correspond to different random variables, each with its own marginal density, and they are not necessarily time-reversal of each other. fact, there considerable flexibility in choosing a and by (or, in the discrete case, A, and B2). A natural choice for the inference-tine control task is to modify the diffusion dynamics so that X2, and X, approximate at time f provided when X, To and X Tj. However, in many cases, this choice is intractable or doesn't 't exist. We instead apply an approximation as exemplified in Appendix A.3. The misalignment with म. is corrected through the acceptance probability.
We now focus on the proposal processes within the time interval [4, l') We consider we have samples n. and n.' and perform communication steps between them using the APT framework. We define Qu as the law of the path X [t.t] obtained by evolving samples from πe time time t' using the forward proposal process. Similarly we define T as the law of the path X X evolving samples from me at time /' backward time using the backward proposal process. We highlight again that we do not require X [e.11] is time reversal of X (1.r9' we also do not assume the simulated forward and backward trajectories terminate at me or m respectively.
[t.e]] obtained by
We can use Equation (2) to express the Badon-Nikodye derivative between Th and Q in terms of the marginal density ratio of Th (xx)) and me (3:) and Ri (IT(,))) the RNE for the proposal processes over [L, ('), for any path F(e generated by the proposals,
a (x)(,11) adu
(#(e)) (x)..1)))) (e(e))
(12)
We substitute Equation (12) into Equation (9) obtain the acceptance probability,
Te (xe) (x(x
(((x)) Ty (x())
R (Z
min
(13)
Provided we can express the marginal density ratio of the in terms of the marginal density ratio of the pretrained diffusion models p² ..... .P...,,, we can compute Equation (13) in terms of the RNE's for the pre-trained diffusion model, R ".......... P2f..
using the RNE relation Equation (4), For ex- ample in the case of tempering with (x()) x pl (x)),, we have Te ( (m)) x RE (xpp,))) See Appendix A.5 for explicit expressions of the acceptance probability for tempering, reward- tilting/posterior sampling, and model composition. We can tractably compute the RNE terms in Equation (13) via the transition-kerne product in Equation (3) or via the path-integral Equa- tions (15) and (19) in Appendix A.2.
Optionally, can apply local exploration after each communication step'. For Gaussian diffusion, we adopt the corrector step from the predictor-corrector algorithm of Song et al. (2021b), using the score function of n, instead of pt.
Local exploration
Additionally, for the highest time step f = in Gaussian diffusions, the target marginal mi is a Gaussian distribution. To accelerate mixing, performing Langevin step following Syed et al. (2021; 2022); Zhang et al. (2025).
resample directly from this Gaussian instead of
For CTMC, the concrete score defines the density ratio between two states, allowing for the direct application of the Metropolis- Haatopolis-Hastingss algorithm (Metropolis et al., 1953; Hastings, 1970). We include detailed discussion on the local move in Appendix A.6.
Note that in standard parallel tempering, this local exploration is essential because its communication step only involves deterministic proposal. In contrast, accelerated PT, and thus our framework, uses stochastic communication proposal, which already introduces randomness. The local move only provides additional refinement and hence can be omitted when the scores of म, prohibitively expensive.

---

Published as a conference paper at ICLR 2026
-0.03
0.59
-0.03
0:50
0.06
9.09
2.00
0.
TQ
1K41
TK2
TK3
gounnd
SW sumplas
g g South
coepl sencles
(a) MD
(b) SMC
(c) CREPE
(a) SMC
(b) CREPE
Figure 4: TICA of Alanine Hexapeptide Figure 3: Histogram of Alanine Hexapeptide annealed to annealed to 600K by SMC and CREPE.
600K by SMC and CREPE, projected to two TICA axes. CREPE maintains more diversity.
3.2 CONTROL WITH REPLICA EXCHANGE (CREPE)
Now, we put the ingredients together in Algorithm 1. Letting the schedule times = to < ....
tM = 1, we generate Markov chain X,, (Xa ( x in x targeting Tr. X ... X Tax using the accelerated PT algorithm described in Section 2, with the annealing path, communication step, and local exploration move described above.
In practice, we can stabilise PT by running it only up to small time step to > O, instead of across the entire diffusion process. After to. will directly continue sampling until 0 with the diffusion model using drift 0₁- This is because tiny time steps often introduce numerical instability and yield low acceptance rates, especially in high-dimensional spaces. By truncating PT early, avoid these issues while retaining effectiveness, as the denoising steps, after sufficiently small to. will have no semantic change to the sample. This strategy was also applied in inference-time control with SMC, where resampling is restricted to a limited time interval (Skreta et al., 2025).
3.3 RELATED WORKS
SMC has been extensively applied to steer the generation process (Wu et al., 2023; Dou & Song, 2024; Singhal et al., 2025; Skreta et al., 2025; Lee al., 2025; Pani et al., 2025; He et al., 2025c; Hasan al.). SMC based-methodd simulate the discretised annealing path sequentially and generates particles in parallel. CREPE introduces a related, but computationally dual perspective on inference time control, by simulating the discretised annealing path in parallel and generating particles sequentially via MCMC. We illustrate their difference and connection in Figure
Trade-offs between SMC and CREPE SMC typically requires a large batch of particles to run in parallel, can suffer from low sample diversity or even mode collapse when the batch size is small, (Li et al., 2024; Young & Akyildiz, 2024; Lee et al., 2025). A common strategy is to run multiple batches. While this can improve diversity, it does not address the bias introduced by a small batch size. We illustrate and discuss this in detail in Appendix A. CREPE, by contrast, only requires parallelisation across different diffusion times (4m (4m) but generates new samples sequentially. It tends to produce more diverse samples naturally, as demonstrate in experiments. Another advantage of CREPE compared to SMC-based methods is that it supports online refinement: new constraints can be introduced on the fly, or samples can be further refined if their quality is insufficient. It is also an anytime inference algorithm: unlike SMC, which returns target samples only after the final iteration, CREPE can terminate at any iteration. However, a burn-in period is required: the samples from the first several iterations may not follow the desired target To and may be discarded.
We also highlight that when the number of PT iterations equals the number of SMC particles. and both methods use the same discretisation steps, controlling with PT and SMC incur the same number of network function evaluations (NFEs). We provide a detailed discussion in Appendix A.7.
4 EXPERIMENTS
We evaluate our proposed algorithm through comprehensive experiments across various domains, including molecules, images, trajectories, and discrete data. Please refer Appendix B for details.
Inference-time Tempering for Boltzmann Sampling We first consider the tempering task for sampling from Boltzmann distributions. Concretely, assuming have pretrained diffusion model trained on samples from (=() x (xx(kux we aim to generate samples from To(x)) exp(-U(xx) (z////// T This setting was considered in Skreta et al. (2025); Rissanen et al.
7

---

Published as a conference paper at ICLR 2026
Table 1: Inference-time tempering performance for Alanine Dipeptide, Tetrapeptide and Hexapeptide.
FKC
CREPE
RNF
Anneal Score
Anneul Noise
(Ours)
Alanine Dipeptide
Energy TVD
0.345
9:000
0.894 1:
0.391
0.224 1.000
ALA Dipeptide (800K 300K)
Distance TVD
0.023
4.000
0.036
1 boon
0.024
_0.0
0.019
1.0000
Sample W2
0.293
-000:
0.282
0,282
0.264
0.000
TICA MMD
0.116
2.00
0.108
boes
0,168
0.096
0.04
Energy TVD
0.122
0.436
2: ooon
0.154
0.00
0.122
20000
Alanino Tetrapeptide
ALA Tetrapeptide (800K - S00K)
Distance TVD
0.014
0.000
0.015
0.013
0.00
0.013
0.000
Sample W2
0.923
0000
0.892
0000
0.893
0.856
1.0004
TICA MMD
0.183
00000
0.138
0.155
0.000
0.035
3,000
Energy TVD
0.091
000
0.206
0006
0.087
-090
0.398
0.0
ALA Hexapeptide (800K - 600K)
Distance TVD
0.018
0.020
0.010
=0000
0.009
4.0.00
Sample W2
1.585
0:000
1.652
1.618
-000
1.299
2.000
Alanine Hcxapeptidc
TICA MMD
0.088
0.068
0.042
0.009
Table 2: Debias ImageNet-64 CFG. We do not discard burn-in samples in CREPE to ensure fair comparison.
30
0.0
20-
-2.5
Method
#Samples
IR (*)
CLIP (1)
FID (4)
200
400
200
400
2.5
-0,29
24.17
1.85
30-
32
-0.14
23,98
1.84
0.0
WW
FKC
128
-0.03
24.04
1.89
IR:512
CLIP-512
20
512
-0.08
24.31
1.96
50
100
150
200
50
100
150
200
-0.30
24.10
1.92
PT Iteration
PT Iteration
CREPE
-0.21
24.21
1.88
Figure 5: Prompted reward-tilting on ImageNet-64 (top)
-0.09
24.37
1.86
and 5 512 (bottom). We report mean and std across five different classes and prompts.
512
0.09
24.28
1.79
(2025); Akhound-Sadegh et al. (2025) to accelerate Boltzmann sampling tasks. In Table 1, we evaluate our proposed algorithm on three molecules with different sizes: Alanine Dipeptide, Tetrapeptide and Hexapeptide. We report the total variant distances (TVD) of energy and distance histograms, W2 distance of samples, as well as the maximum mean discrepancy (MMD) for the sample projected to 2D space with time-lagged independent component analysis (TICA, Molgedey & Schuster, 1994). We also show the TICA plot for Hexapeptide samples generated by SMC (w. RNE) and CREPE, along with molecular dynamics (MD) samples in Figures 3 and 4. We see CREPE achieves superior performance on three of the targets across most metrics. In particular, Figure 3 shows that CREPE achieves lower bias compared to SMC, echoing our discussion on mini-batch SMC in Appendix A. Figure 4 shows that CREPE maintains better diversity and avoids missed modes. The only exception the energy of the alanine hexapeptide. A likely explanation is that the pretrained model incurs higher error on this molecule, which is amplified by PT through repeated iterations. We also note that the energy histogram alone may not always reflect overall performance. It may appear favourable even when mode collapsing, as observed by Blessing et al. (2024); He et al. (2025a).
Debiasing Classifier-Free Guidance for Image Generation We now consider applying CREPE to debias classifier-free guidance (CFG, Ho & Salimans, 2022), a setting also explored by Skreta et al. (2025) with SMC. Concretely, given an unconditional diffusion (pt) and conditional diffusion (po(-)c)), we aim to sample from mn(-) x po(-)=- (o (o(-lc) In Table 2, we evaluate the ImageReward (IR, Xu et al., 2023), CLIP score (Hessel al., 2021) and FID (Heusel al., 2017) for images generated by CREPE. The IR and CLIP are conditioning on the class label. We also report results by FKC (Skreta et al., 2025), which is based on SMC for debiasing the CFG. We can see that when the number of samples is small (e.g., 8), the SMC-based FKC outperforms CREPE as expected, since PT typically requires a burn-in in period. However, as the number of generated samples increases, CREPE empirically outperforms FKC, particularly in terms of FID. Additionally, in the example images shown in Figures 11 to 14, we can see that FKC tends to produce visually similar samples within a batch, whereas CREPE maintains higher diversity.
Reward-tilting for Image Generation We now turn to reward-tilting in the context of image generation. Specifically, generate class-conditioned images using CFG with debiasing, and further steer the generation with more detailed instructions provided by ImageReward through a prompt. This also serves as example of combining multiple tasks (debiasing CFG and reward-tilting.
8

---

Published as a conference paper at ICLR 2026
Exsmple trsining Injjecterios
Trajectory iteration.
Trijectory Trajectory atter
Trajectory PT lientooss
Trajectory after 10Lk Trjeetory after PT iteration.
ficeations
PT iteration.
PT licraioo..
Figure 6: Stitched trajectory by CREPE with online refinement. We also visualise the training dataset (leftmost plot). In the first 100k iterations, the trajectories navigate from the initial @ to the final target @ Starting from 100k iteration, additional intermediate point ① is introduced. We observe that this new constraint is quickly satisfied (only 1 iterations after the new reward added).
1.0
2 2.
Success Rate
CompDiffuser. 68%
2 2
2
2
2
2
2
2.
2
0.5
4 4
4
4
4
4
4
#
4
4
4
4 4
44
4
4
#
4
4
4 4
4 4
4
4
4
#
14
#
4
4
4
4
4
4
0.0
0.0
0.5
1.0
1.5
7 7
7
7 7
7
PT iterations
x10$
7
7
7
7
7
7
7
7
Figure 7: Success rates V.s. PT iterations. Grey line shows the average success rate across 5 tasks. Black is the moving average. We also report values in Table 3.
CFG
CFG W. SMC
CFG W. CREPE
Figure 8: MNIST samples generated by CFG, and debiased by SMC and CREPE.
We visualise the samples obtained along the PT chains (thinned every 8 iterations) with their class label and prompt in Figure 1. We also plot IR and CLIP scores along PT iterations across five different classes and prompts in Figure 5. The IR and CLIP are conditional on the prompt. After the first burn-in period, CREPE effectively produces diverse images that closely align with the prompt.
Model Composition with Reward for Maze Navigation We now consider model composition. Following Luo al. (2025), we compose diffusion models trained short trajectories synthesise a coherent long-horizon path through the maze. Unlike Luo et al. (2025), who train diffusion models conditioned both ends and stitch segments via conditioning, and stitch segments using reward function. This task can be viewed as combination of reward- tilting and model composition, where we aim to generate samples from Ty ((x((())) ,,)(2, ..)))............. ,,("))
train unconditional model
exp(r) II, (z( This soward-based composition affords flexible constraints on the trajectory.
We use the po short trajectories of length 64 in large 2D maze. We train an unconditional diffusion model these short trajectories. To generate stitched trajectories, we impose the following rewards: (1) the starting point of the first trajectory is sufficiently close the initial state, (2) the endpoint of the last trajectory is sufficiently close to the target state, and (3) consecutive trajectories are connected in a tail-to-head manner. We include detailed forms the reward Appendix B.4. We first evaluate our approach on the five different initial-gon pairs considered by Luo et al. (2025). We report the success rate in Table and visualise the corresponding trace plots in Figure We include the results by (Luo see, combining an unconditional diffusion model with CREPE
dataset from Park et al. (2024), which consists of
al., 2025) reference. As achieves comparable or even better performance than directly training a conditional model, at the cost of more computing resources.
CREPE with Online Refinement An advantage of training an unconditional model and stitching with CREPE that it offers greater flexibility in specifying the reward function, and it naturally extends to online settings where new constraints may be introduced on the fly. To show this, we first run CREPE to navigate from the initial to the final target, and then add an additional reward that requires the trajectory to pass through intermediate point. In Figure 6, visualise stitched trajectory samples produced by CREPE at different PT iterations, where the intermediate point is introduced at 100k iterations. We can the new constraint is quickly satisfied.

---

Published as a conference paper at ICLR 2026
(propenal stre
500
(proposal =12)
(propsss 21e
0.92
(proporo shenghh-11
Perplexity
400
(propose) strength-1.00
(propcoe)) stanggtim1.2
Accuracy
0.50
CAEPE jpropseal steeghh 101
pproponnl stweghh-112
400
SMC (pmposs) stroogthee)
jprpponal strengh-1A)
CMG
Janassoal strengiie1
200
stongth-..0
100
0.75
1.0
1.1
1.4
1.5
1.6
1.8
1.0
1.1
1.2
1.5
1.7
1.8
Target Strength
Target Strengtt
(a) Perplexity
(b) Sentiment accuracy
Figure 9: Debiasing CFG for CTMC on text with SMC and CREPE. Perplexity (left) and sentiment accuracy (right) as function of target strength for SMC and CREPE across different proposal strengths. As we can see, CREPE achieves a significant reduction in perplexity while maintaining high sentiment accuracy across most settings.
We now consider applying CREPE to discrete diffusion. In the first example, we consider debiasing CFG for mask diffusion, as considered by Lee et al. (2025). In Figure visualise samples obtained by SMC (Lee et al., 2025) and CREPE. We can see that both algorithm achieves more plausible samples, with CREPE presenting slightly more sample diversity.
CREPE on CTMC
In the second example, we consider debiasing the CFG for sentiment-controlle text generation using discrete diffusion. The goal to generate text with a specified sentiment (positive or negative) by conditioning on the prompt "The sentiment of the text is nnegative ppssitive)", We sweep over target CFG strengths E {1.0, 1.2, 1.4, 1.8). Also, since our framework allows arbitrary proposal processes, we sweep over different proposal CFG strengths {1.0, 1.2, 1.4, 1.8).
Figure compares SMC and CREPE across the target strength sweep. CREPE maintains substan- tially better perplexity throughout the range, improving by up to factor of 5 at target strength 1.8. Both methods achieve similar sentiment accuracy, except for CREPE at (target strength = 1.4, proposal strength 1.0) where accuracy drops. A possible explanation is that CREPE itera- tively refines the current samples, so a poor sample can destabilize performance, whereas mini-batch SMC runs independent batches and is more robust to occasional failures. This suggests that CREPE is not immune to instabilities, and further investigation is needed, especially for its application to text. To better understand the gain from CREPE, in Appendix D, visualize the running average of perplexity and accuracy over CREPE iterations. We observe that perplexity steadily decreases. This indicates that standard CFG can distort the text distribution, and CREPE effectively debiases the CFG-induced distortion, producing higher-quality samples with substantially lower perplexity.
CONCLUSIONS AND FUTURE WORKS
In this work, we propose CREPE, a new framework for controlling diffusion models using replica exchange. CREPE offers an alternative to the widely employed SMC-based approaches for broad range of inference-time control tasks for diffusion models across different modalities. It demonstrates comparable efficiency with SMC, while additionally supporting online refinement and maintaining better sample diversity, opening new avenue for further exploration. One potential direction is investigate how advanced schedule-tuning and path selection techniques (Masrani al., 2021; Syed et al., 2021; Surjanovic et al., 2022; Máté & Fleuret, 2023; York, 2023) from classical parallel tempering for sampling can be adapted to our setting to provide better control over diffusion models.
The main limitations of CREPE are the presence of burn-in period and approximation errors introduced in the communication acceptance rate: CREPE typically requires a burn-in period to reach optimal performance. For large systems and expensive networks, this can result in high computational cost. Additionally, CREPE relies on the assumption of perfect diffusion model without discretisation error, which often does not hold in practice. While did not observe major failures in our experiments, this assumption may break in other settings. Besides, these approximation emors can accumulate over iterations, leading to deviations from the desired target.
10

---

Published as a conference paper at ICLR 2026
LLM USAGE DISCLOSURE
LLM was used at the sentence level to correct grammar.
ACKNOWLEDGEMENTS
JH acknowledges support from the University of Cambridge Harding Distinguished Postgraduate Scholars Programme. PP and LZ are supported by the EPSRC CDT in Modern Statistics and Statistical Machine Learning (EP/S023151/1) JMHL acknowledge support from EPSRC funding under grant EP/Y028805/1. JMHL. also acknowledges support from Turing Al Fellowship under grant EP/V023756/1. SS acknowledges support from the NSERC Postdoctoral Fellowship. On different note, JH and FV gratefully acknowledge PJ for the excellent food recommendations.
REFERENCES
Tara Akhound-Sadegh Jarrid Rector-Brooks, Avishek Joey Bose, Sarthak Mittal, Pablo Lemos, Cheng-Hao Liu, Marcin Sendera, Siamak Ravanbakhsh, Gauthier Gidel, Yoshua Bengio, Nicolas Malkin, and Alexander Tong. Iterated denoising energy matching for sampling from Boltzmann densities. In International Conference on Machine Learning. 2024.
Tara Akhound-Sadepp Jungyoon Lee, Avishek Joey Bose, Valentin De Bortoli, Arnaud Doucet, Michael M Bronstein, Dominique Beaini, Siamak Ravanbakhsh, Kirill Neklyudov, and Alexander Tong. Progressive inference-time annealing of diffusion models for sampling from boltzmann densities. arXiv preprint arXiv:2506. 16471, 2025.
Michael Albergo, Nicholas M Boffi, and Eric Vanden Vanden-Eijnden. Stochastic interpolants: unifying framework for flows and diffusions. arXiv preprini (3t.::2303.08797, 2023.
Afra Amini, Li Du, and Ryan Cotterell. Structured voronoi sampling, 2024. URL https://
arxiv. org/ abs 2200. 03061.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34:1 799771999959 2021.
Andrew J Ballard and Christopher Jarzynski. Replica exchange with nonequilibrium switches. Proceedings of the National Academy of Sciences, 106(30); 2009.
Andrew J Ballard and Christopher Jarzynski. Replica exchange with nonequilibrium switches: Enhancing equilibrium sampling by increasing replica overlap. The Journal of Chemical Physics, 136(19), 2012.
Julius Berner, Lorenz Richter, Marcin Sendera, Jarrid Rector- Brooos, and Nikolay Malkin. From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training. arXiv preprint arXiv:2501.06148 2025.
Denis Blessing, Xiaogang Jia, Johannes Esslinger, Francisco Vargas, and Gerhard Neumann. Be- yond elbos: A large-scale evaluation of variational methods for sampling. arXiv preprint arXiv:2406.017 074433 2024.
Andrew Campbell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. Advances in Neural Information Processing Systems, 35:28266-28277 28279, 2022.
Wenlin Chen, Mingtian Zhang. Brooks Paige José Miguel Hernández-Lobato and David Barber. Diffusive gibbs sampling. arXiv preprint 2400::2000000000 2024.
John Chodera, Andrea Rizzi, Levi Naden, Kyle Beauchamp, Patrick Grinaway, Mike Henry, Iván Pulido, Josh Fass, Alex Wade, Gregory A. Ross, Andreas Kraemer, Hannah Bruce Macdonald, jaimergp, Bas Rustenburg, David W.H. Swenson, Ivy Zhang, Dominie Rufa, Andy Simmonett,
11

---

Published as a conference paper at ICLR 2026
Mark J. Williamson, hb0402, Jake Fennick, Sander Roet, Benjamin Ries, lan Kenney, Irfan Alibay, Richard Gowers, and SimonBoothroyd. choderalab/ggemmmtoo::: 0.24.1, January 2025. URL https: /tt.//oi..a/ / 5281/2enooo 14782825.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mecann, Mare Louis Klasky, and Jong Chul Ye.
Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023.
Pierre Del Moral, Arnad Doucet, and Ajay Jasra. Sequential Monte Carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, (8(3):411-430) 2006.
Alexander Denker, Francisco Vargas, Shreyas Padhy, Kieran Didi, Simon Mathis, Riccardo Barbano, Vincent Dutordoir, Emile Mathieu, Urszula Julia Komorowska, and Pietro Lio. Deft: Efficient fine-tuning of diffusion models by learning the generalised h-transform. Advances in Neural Information Processing Systems, 11119666668822 2024.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794 2021.
Carles Domingo-Enrich, Michal Drozdzal, Brian Karrer, and Ricky TQ Chen. Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. arXiv preprint ar wrXhv:2409008866 2024.
Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In The Twelfth International Conference on Learning Representations, 2024.
Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Composi- tional generation with energy-based diffusion models and memc. In International conference on machine learning. pp. 8489-8510. PMLR, 2023.
Chenru Duan, Yuanqi Du, Haojun Jia, and Heather Kulik. Accurate transition state generation with an object-aware equivariant elementary reaction diffusion model. Nature computational science, 3 (12):1045-1055, 2023.
Peter Eastman, Raimondas Galvelis, Raál P Peláez, Charlles RA Abreu, Stephen Farr, Emilio Gallicchio, Anton Gorenko, Michael M Henry, Frank Hu, Jing Huang, et al. Openmm 8: molecular dynamics simulation with machine learning potentials. The Journal of Physical Chemistry B, 128 (1):109-116, 2023.
Bradley Efron. Tweedie's 's formula and selection bias. Journal of the American Statistical Association, 1664496):60021666 2011.
Charles J Geyer. Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface, 1991.
Mohsin Hasan, Marta Skreta, Alan Aspuru-Guziky Yoshua Bengio, and Kirill Neklyudov. Discrete feynman-kac correctors. In 2nd Al for Math Workshop@ ICML 2025.
W. K. Hastings. Monte carlo sampling methods using markov chains and their applications. Biometrika, 57(1):97-109, 1970. ISSN 00063444, 14643510. URL org/stabll stee/taable2233444
jstor.
Jiajun He, Wenlin Chen, Mingtian Zhang, David Barber, and José Miguel Hernández-Looato. Training neural samplers with reverse diffusive kl divergence. In The 28th International Conference on Artificial Intelligence and Statistics, 2025a.
Jiajun He, Yuanqi Du, Francisco Vargas, Yuanqing Wang, Carla P Gomes, José Miguel Hernández- Lobato, and Eric Vanden- Eijnden. Feat: Free energy estimators with adaptive transport. arXiv preprint ar arXiv:2504.115/0 2025b.
Jajun He, José Miguel Hemnández-Lobato Yuanqi Du, and Francisco Vargas. Rne: plug-and- play framework for diffusion density estimation and inference-time control. arXiv preprint arXiv:2506. 05668, 2025c.
12

---

Published as a conference paper at ICLR 2026
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference- free evaluation metric for image captioning. arXiv preprint arXi::2004.8778, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho and Tim Salimans. Classifier- diffusion guidance. arXiv preprint arXiv:2207.12590 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Jonathan Ho. Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8644 2022.
Peter Holderrieth, Michael S Albergo, and Tommi Jaakkola. Leaps: A discrete neural sampler via locally equivariant networks. arXiv preprint arXiv:2502. 10843, 2025.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning, pp. 8867-8887. PMLR. 2022.
Koji Hukushima and Koji Nemoto. Exchange Monte Carlo method and application to spin glass simulations. Journal of the Physical Society of Japan, 65(6):1604-1608, 1996.
W. Kabsch. A solution for the best rotation to relate two sets of vectors. Acta Crystallographcca Section A, 32(5):922-923, 1976. doi: https///bb.oog/00011/////67//// wiley. com/ /toi/a///////////o//////oo///aaa http:/aas......1//////6/777779/0
URL https:
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-26577, 2022.
Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24174-241 2417444484, 2024.
Lingkai Kong, Yuanqi Du, Wenhao Mu, Kirill Neklyudov, Valentin Bortoli, Dongxia Wu, Haorui Wang, Aaron M Ferber, Yian Ma, Carla P Gomes, et al. Diffusion models constrained samplers for optimization with unknown constraints. The 28th International Conference on Artificial Intelligence and Statistics, 2025.
Cheuk Kit Lee, Paul Jeha, Jes Frellsen, Pietro Lio, Michael Samuel Albergo, and Francisco Var- gas. Debiasing guidance for discrete diffusion with sequential monte carlo. arXiv preprint arXiv:2502.06079, 2025.
Xiner Li, Yulai Zhao, Chenyu Wang, Gabriele Scalia, Gokcen Eraslan, Surag Nair, Tommaso Biancalani, Shuiwang Ji, Aviv Regev, Sergey Levine, et al. Derivative-free guidance continuous and discrete diffusion models with soft based decoding. arXiv preprint arXiv:2408.0822. 2024.
Zhen Liu, Tim Z Xiao, Weiyang Liu, Yoshua Bengio. and Dinghuai Zhang. Efficient diversity- preserving diffusion alignment via gradient-in informed gflownets. arXiv preprint arXiv:2412.07775, 2024.
Aaron Lou, Chenlin Meng. and Stefano Ermon. Discrete diffusion language modeling by estimating the ratios of the data distribution. 2023.
Yunhao Luo, Utkarsh A Mishra, Yilun Du and Danfei Xu. Generative trajectory stitching through diffusion composition. arXiv preprint 2rX.::2503.05/33 2025.
13

---

Moritz Published as a conference paper at ICLR 2026
Vaden Masrani, Rob Brekelmans, Thang Bui, Frank Nielsen, Aram Galstyan, Greg Ver Steeg, and Frank Wood. q-paths: Generalizing the geometric annealing path using power means. In Uncertainty in Artificial Intelligence. 2021.
Bálint Mäté and Francois Fleuret. Learning interpolations between Boltzmann densities. Transactions on Machine Learning Research, 2023.
Nicholas Metropolis, Arianna W. Rosenbluth, Marshall N. Rosenbluth, Augusta H. Teller, and Edward Teller. Equation of state calculations by fast computing machines. The Journal of Chemical Physics, 2166):0887-1022, 06 1953. ISSN 0021-9606. doi: 10.1063/1.1699111 14. URL
https: //tp:://doi.. org/10 1003/1111..10 1.1.
L. Molgedey and H. G. Schuster. Separation of mixture of independent signals using time delayed correlations, Phys. Rev. Lett. 72:3634-3633 Jun 1994. doi: 10. 10.1109/PtysReeeLctt2.....0 URL
https: /tnk/iin....../ aps org/ wttp///g//oooooooo /tw////oo111///// 110. 11110////yyeeeeeett.ee 72 72.3634.
Tsuneyasu Okabe, Masaaki Kawata, Yuko Okamoto, and Masuhiro Mikami. Replica-exchange Monte Carlo method for the isobaric-isotherall ensemble. Chemical Physics Letters, 335(5-6): 435-439, 2001.
Chinmay Pani, Zijing Ou, and Yingzhen Li. Test-time alignment of discrete diffusion models with sequential monte carlo. arXiv preprini arXiv:2505.22524 2025.
Seohong Park, Kevin Frans, Benjamin Eysenbach, and Sergey Levine. Ogbench: Benchmarking offline goal-conditioned rl. arXiv preprint arXiv:2410.20092,, 2024.
Severi Rissanen, RuiKang YuYang, Jiajun He, Wenlin Chen, Markus Heinonen, Arno Solin, and José Miguel Hernández-oobato. Progressive tempering sampler with diffusion. arXiv preprint arXiv:2506.0522 05231, 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 00684-10695, 2022.
Martin K Scherer, Benjamin Trendelkamp-Schreee Fabian Paul, Guillermo
Hoffmann, Nuria Plattner, Christoph Wehmeyer, Jan- nen-Herrrik Prinz, and Frank Noé. Pyemma 2: software package for estimation, validation, and analysis of markov models. Journal of chemical theory and computation, 11(11):5525-544) 2015.
Arne Schneuing, Charles Harris, Yuanqi Du, Kieran Didi, Arian Jamasb, Ilia Igashov, Weitao Du, Carla Gomes, Tom L Blundell, Pietro Lio, et al. Structure-basee drug design with equivariant diffusion models. Nature Computational Science, 4(12):899- 000,9999999,9... 2024.
Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and generalized masked diffusion for discrete data. Advances in neural information processing systems, 37: 103131-103167, 2024.
Yuyang Shi, Valentin De Bortoli, Andrew Campbell, and Arnaud Doucet. Diffusion schr\"" odinger bridge matching, arXiv preprint arXiv:2303. 2023.
Raghav Singhal, Zachary Horvitz, Ryan Teehan, Mengye Ren, Zhou Yu, Kathleen McKeown, and Rajesh Ranganath. A general framework for inference-time scaling and steering of diffusion models. arXiv preprint arXiv:2501.068/ 2025.
Marta Skreta, Tara Akhound-Sadenn Viktor Ohanesian, Roberto Bondesan, Alán Aspuru- Guzik, Arnaud Doucet, Rob Brekelmans, Alexander Tong, and Kirill Neklyudov. Feynman-kac correctors in diffusion: Annealing, guidance, and product of experts. arXiv preprint arXiv:2503.02819, 2025.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations, 2021a.
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz, Pseudoinverse-guider diffusion models for inverse problems. In ICLR, 2023a.
14

---

Published as a conference paper at ICLR 2026
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for -ong-an--pay controllable generation. In International Conference on Machine Learning, pp. 32483 02983---48... PMLR, 2023b.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021b.
Nikola Surjanovie, Saifuddin Syed, Alexandre Bouchard-Côre and Trevor Campbell. Parallel tempering with variational reference. In Advances in Neural Information Processing Systems, 2022.
Robert H. Swendsen and Jian-Sheng Wang. Replica Monte Carlo simulation of spin-glasses. Physical Review Letters, 57:2607-2609. 1986.
Saifuddin Syed, Vittorio Romanicllo, Trevor Campbell, and Alexandre Bouchard-Côtt Parallel
tempering on optimized paths. In International Conference on Machine Learning, 2021.
Saifuddin Syed, Alexandre Bouchard-Côré, George Deligiannidis, and Arnaud Doucet. Non- reversible parallel tempering: scalable highly parallel MCMC scheme. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(2):321-33 -2-350, 2022.
Saifuddin Syed, Alexandre Bouchard-CGin -ötéé, Kevin Chem,, and Arnaud Doucet. Optimised annealed sequential Monte Carlo samplers. arXiv preprint arXiv:2408. 12057, 2024.
James Thornton, Louis Béthune, Ruixiang Zhang, Arwen Bradley, Preetum Nakkiran, and Shuangfei Zhai Composition and control with distilled energy diffusion models and sequential monte carlo. arXiv preprint (rxXix25022.27888 2025.
Francisco Vargas, Nikolas Nüsken, Shreyas Padhy, and Denis Blessing, Transport meets varia- tional inference: Controlled Monte Carlo diffusions. In International Conference on Learning Representations, 2024.
Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E Eisenach, Woody Ahern, Andrew Borst, Robert J Ragotte, Lukas F Milles, et al. De novo design of protein structure and function with rfdiffusion. Nature,
1099-11000 2023.
Luhuan Wu, Brian Trippe, Christian Naesseth, David Blei, and John P Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems, 36:3 36:31372-31403, 2023.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 01115933-15333 2023.
Haotian Ye, Haowei Lin, Jiaqi Han, Minkai Xu, Sheng Liu, Yitao Liang, Jianzhu Ma, James Y Zou, and Stefano Ermon. Tfg: Unified training-fit free guidance for diffusion models. Advances Neural Information Processing Systems, 17:22370-22411, 2024.
Darrin M York. Modern alchemical free energy methods for drug discovery explained. ACS Physical Chemistry Au, 3(6))478-49) 2023.
James Matthew Young and O Deniz Akyildiz. On diffusion posterior sampling via sequential monte carlo for zero-shot scaffolding of protein motifs. arXiv preprint arXiv:2412.05788 2024.
Dinghuai Zhang, Yizhe Zhang, Jiatao Gu, Ruixiang Zhang, Josh Susskind, Navdeep Jaitly, and Shuangfei Zhai. Improving gflownets for text-to-imane diffusion alignment. arXiv preprint
arXiv:2406.00633 2024.
Leo Zhang, Peter Potaptchik, Jiajun He, Yuanqi Du, Arnaud Doucet, Francisco Vargas, Hai-Dang Dau, and Saifuddin Syed. Accelerated parallel tempering via neural transports. arXiv preprint arXiv:2502. 10328, 2025.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
A Supplementary Methods and Discussion
17
A.1 Why Mini-batch SMC Is Not Favourable?
17
A.2
Continuous Form for R
18
A.3 Choice of Q Processes
18
A.4
Discretised Formula for PT Control
19
A.4.1 Gaussian Diffusion
19
A.4.2
CTMC
20
A.5
Acceptance Rate for Reward, CFG and Composition
21
A.5.1
Tempering
21
A.5.2
newaaddiilting/Poserrii sampling
21
A.5.3
Model composition
................................
22
A.6
Details on Local Exploration
22
A.6.1 Gaussian Diffusion
22
A.6.2 CTMC
................................
22
A.7 Computational Comparison between SMC and CREPE
23
B Experimental Details
23
B.1 Tempering
23
B.2 Image CFG Debiasing
24
B.3 Image Reward-tilting
24
B.4 Maze
25
B.5 CTMC
26
B.6 Text CTMC
26
C Additional Results
27
C.I Samples of Debiasing CFG on ImageNet
27
C.2 Samples of Reward-tilting on ImageNet
28
C33 Failure Mode of Reward-tilinng on ImageNet
29
C,4 Success Rate of Trajectory Stitching
29
C.5 More Results Trajectory Stitching with Online Refinement
30
................................
Debiasing CFG For CTMC Text
30
D.I Debiasing CFG for CTMC on MNIST
31
16

---

Published as a conference paper at ICLR 2026
A
SUPPLEMENTARY METHODS AND DISCUSSION
A.1
WIIY MINI-BATCII SMC IS NoT FAVOURABLE?
3.5
gmm density
exp reward
3.0
target density
2.5
1.5
1.0
0.5
0.0
-1.0
-0.5
0.0
0.5
1.0
1.5
(a) Illustration of GMM, reward and the ground truth target.
samples (bsz=4) ground truth density
3.5
samples
3.0
ground truth density
2.5
2.5
2.0
2.0
15
1.0
1.0
0.5
1.0
0.0
(b) Samples obtained by mini-batch SMC (bsz 4),
(c) Samples obtained by mini-batch SMC (bsz 10).
SMC samples (bsz=100)
35
CREPE samples
ground truth density
ground density
3.0
3
2.0
2-
1.5
0.0
-1.0
-0.5
0.0
0.5
1.0
1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
(d) Samples obtained by SMC (bsz 100).
(e) Samples obtained by CREPE.
Figure 10: Illustration of reward-tilted samples obtained with mini-batch SMC and CREPE. For SMC, collect 150,000 samples with batch size of 4/10/100; while for CREPE, collect 150,000 samples in single run.
In our main experiments, we observe that CREPE produces samples with higher diversity than SMC. natural question is: why not simply many mini-batches of SMC recover diversiry?
In fact. our SMC baseline already uses this strategy: following the convention in the cnffisiioncconreee SMC in hiii-batches and aggregate the resulting
with-SMC literature (e.g. Skreta et al., 2025), samples. Even in this setting, SMC still underperforms CREPE in terms of diversity. One might further ask why do not reduce the batch size more and increase the number of SMC runs.
17

---

Published as a conference paper at ICLR 2026
The reason is that this comes at the cost of a large bias. Running many mini-batch SMCs is conceptually equivalent to repeatedly restarting a local sampling algorithm. To see why this is problematic, consider multimodal target distribution where each SMC run tends to collapse onto a single mode due to resampling. If we restart SMC many times, we may indeed obtain samples from different modes, but the relative mode weights now reflect the probability that a single SMC run collapses into each mode, rather than the true target mass of each mode. Thus, while the aggregated samples may appear diverse, the relative weights between modes are incorrect, and the resulting empirical distribution is systematically biased away from the true target.
To illustrate this bias, we visualise reward-tilting on ID Gaussian mixture in Figure 10. As shown in the figure, while mini-batch SMC can produce samples from all modes, the relative proportions of these modes are noticeably distorted, whereas CREPE preserves the true mode weights much more faithfully. We note that this is only a 1D toy example; in higher dimensions, samples from SMC in each batch will collapse further, and the misweighting becomes even more severe.
A.2 CONTINUOUS FORM FOR R
We can express the RNE defined in Equation (3) analytically for SDE or CTMC (Berner et al., 2025; Holderrieth et al., 2025): Consider the following forward and backward SDEs:
ax, n. a) ds + a. ax; ve (x.) ds a, dw.
(14)
Then for any valid trajectory Xe. within time-interval [t. t',, under mild condition, we have
(err(1)))) exp (J"" 1 ( di, - J,"
1 (uller) dz, + 1/2 S. o3 (1Im,(+)))1 (c.(()
(15)
where the first and second terms on the RHS represent the Itô forward and backward integral. Consider discretisation with
to < l1 < ...
< lx = 1', they are defined as:
l as (x.) dF
lim
Σ ary (Ita) (1te Itt):
(16)
max, M811 -t 100
J. (x,))) dr.-
lim M241
Σ atxer (Kit4s) (Itens - -rt)).
(17)
max-
1-0
Note that this is different from the Riemann Integral, where the "direction" of summation does not matter and needs to converge to the same value.
Similarly, for the following forward and backward CTMCs:
aep: = M, Pt, d.p. = -N,
(18)
we have
(e = exp (1/" [N.( (T(
log Na(
Σ
(19)
(z
s:X, tz.
A.3 CHOICE OF Q PROCESSES
In this section, we exemplify some choices for the proposal process C and Q' Note that none of these choices is unique. In fact, one enjoys large flexibility in choosing these dynamics. Also, for fair comparison, we use the same proposal for SMC with RNE (He et al., 2025c).
Tempering: in the Gaussian diffusion (with forward and backward drifts fi and gi as defined in Equations (5) and (6)), one may choose the backward drift to be the standard noising kernel b, = ft. and the forward drift a, f 3o V log Pa following (He et al., 2025c; Skreta al., 2025).
Reward-tilting we can also choose the backward drift to be the standard noising kernel b, = fi- and the forward drift with reward- guidance a f: of(V log Pt Vr:) (Dou & Song, 2024). When the reward is non-differentiable or expensive to compute, we can also set a, In this case, will rely entirely on the SMC/PT correction.
= f. of log pr.
18

---

Published as a conference paper at ICLR 2026
Composition: can choose the backward drift to be the standard noising kernel b, fis and the forward drift using the summation of scores a, fa of E V Vlog p\) similar to He et al. (2025c); Skreta et al. (2025).
CFG Debiasing: CFG debiasing a simple combination annealing and composition. However, as it's commonly adopted in diffusion models, also discuss it individual case. For CFG debiasing, we can choose the backward drift to be the standard noising kernel b, = ft. and the forward drift using the standard CFG dynamics as at = ft - of (w V log p3 (-) + (1 w V log p (-1c)). We can also use a different CFG strength w' for the proposal drift = ft - of w v log pi (-) + (1
w/) V log pi (-Ic)), as considered by Lee al. (2025).
CTMC CFG Debiasing: for CFG debiasing in CTMC, we can also follow the heuristics proposed by Lee et al. (2025). Where we set A, = A₁ as the simple masking process; and we set (x,,,) Ae(y, x (e(vle) pe(a) y1-w where the pretrained conditional and unconditional model provide us (x)
(((())) (
the conditional and unconditional concrete score Pe(Mle) and Pe(x)
(z(e) p.()) Similar to Gaussian diffusion, we
can choose different strength w/ for the proposal process, adopted by Lee al. (2025).
A.4 DISCRETISED FORMULA FOR PT CONTROL.
In the main paper, we focus our discussion on the continuous-time formula. In practice, both the simulation of the processes and the calculation of the acceptance rate need to be discretised in time. In this section, we will discuss this discrete form.
A.4.1 GAUSSIAN DIFFUSION
Simulation with Marur-Ma discretisation For the diffusion models, we will apply EM discretisation for the forward and backward dynamics. Note that this is not the only choice. We can also use the DDPM transition kernel (Ho et al., 2020) or the exponential integrator.
Discretised Forward proposal:
X x = X, + a (X 6tt + o. Vōte, e~ N0,1),
(20)
Discretised Backward proposal: X X - by (X +o Vol', d'~,N(011)
(21)
Acceptance rate with discrete kernel With the same discretisation, we can calculate the R with Gaussian densities: concretely, consider the forward and backward processes discretised into K steps: t = to < t < < tk t'.
i4-1/te ltee (Iir I (atee) I Be
(22)
K Fo inlta- (Ite (ee-a))
where
DC
te-n t (Ite- N W(Ite-ii2 byg (Itz )tx 1bb11,, ts-1l)
(23)
70 tatzlte (Dizei (21,) N N( + ar (I ( t4+1 tel. 0f tul)
(24)
RP follows the same calculation. For the diffusion model in Equations (5) and (6), have
T T2 Bi llke (I (xtea)
Re (F(1,)) = I FP
(25)
with (Ite (
where
Bp 1lth (Ite- [1t2) N(11211: It. (21x))ex tk-1l, tx-1l) AP tets Ih (Ftaps (122.) -N : + Sta (Itz)) M4441 1x). Aal.0 (x))
(26)
(27)
Reference process to stabilise the calculation (He et al., 2025c) Using the above discretisation directly can lead to numerical issues. This issue was observed and analysed in He al. (2025c), arising from the misalignment of variance between forward and backward kernels. To address this,
19

---

Published as a conference paper at ICLR 2026
He al. (2025c) introduce analytical reference convert forward-backe kernel ratios into forward-forward and backward -aaadbbakwward kernel ratios.
Concretely, we introduce reference diffusion process:
Yo To:
V Tss d². - Is(P.) dw
(28)
where "o is chosen be Gaussian. Therefore, all " Gaussian with tractable mean and variance. We can hence write down its time-reversal easily:
Y1 71, Y dÝ. h1 (Y,)) o dW..
(29)
where h, f. V 7. We can also calculate the RNE for this reference process, which denote by R and following Equation (4), we know
(<(x)) R (x(0,1)= =I. (xx))
(30)
In practice we calculate R. (11)) as:
11 Big
Dr
alse (Itz (20))
R (x
(31)
I Fi.
iulen-s (Tita (Tte 1)
By 1lth (Ita (z1r)) N N 1i3t he (I ) )lt 16-11,,1000 of tx-1l) Fr fstalte (Fizes (21x) = (( + fox (F ) tul of th+1 txl)
(32)
(33)
We can see both Re and Re (or R...)) have the form of cawaadd-bccwaand kernel ratios; hence, viding them yields conversion from kernel ratios. More precisely, we calculate RE and R.C, follows:
di- to forward-forward and backward-backwer
DQ T--1 (Ite (Tt) I Fr
wile-- (Its (Fta-1) 7
(F
(34)
k= 1/4 1ta (Ttz- (21,) I===1 fulta-1 (Itz (Tee_1) Voe (xe)
FQ
In fact, emperically observed that CREPE remains robust than SMC even without the of reference. This is because, unlike the SMC weights, the PT acceptance ratio always involves ratios of the R's, which naturally mitigate the variance misalignment. Therefore, we only employ the reference in inference- time tempering experiments, where we found it yields better results.
A.4.2 СТмC
Simulation with Euler discretisation For CTMCs, we will apply Euler discretisation for the forward and backward dynamics. More precisely, for each token in the sample, we have
Discretised Forward proposal: ( (x,) = ò + II Ix + o(8t),
(35)
Discretised Backward proposal: (el())) 5 Sn + Bi xx 8t + o(tt)
(36)
where 8211 az denotes the delta function, which equals 1 when T2 x, and otherwise. This defines a categorical distribution. In practice, we ignore the o(ot) term. After evaluating the probabilities for all categories, we clip them be non-negative and then renormalise.
Acceptance rate with discrete kernel We calculate R-s with the same formula Equations (22) and (25). The only difference is that the transition kernels are defined by Categorical probability instead of Gaussian densities. Assuming the codebook size is V, for each token in the mask diffusion defined in Equations (7) and (8), we have
Bi 1/44 (Fea- 12t,) Cat(F 11 P2, (xx1))
(37)
FP tatilte (Degey (T2,)) = Cat(Fre [71, F2, (v
(38)
20

---

Published as a conference paper at ICLR 2026
where 181. P2.... pvl"(x represents the probability vector for Zic+1 element given by
[01 101.-0 eyl, with each
Sui + Ara (3res v1)8t
Óvzndey + Are (Tee v2)ôt
P1, P2.... (x))
(39)
Oovinee Aeu (Ires Uv )06
and:
Sor + M4, (Iee )8t
Spg.aan N/ (D U2) )6t
17,, ..... (xw)
(40)
Gov + A1, (T uv )ot
We also remove nan and inf values, and clip all values to be larger than 1e-8. 8. This makes the sum of all elements deviate from 1, but we found it work well in practice, when 5t is small, the deviation is negligible. This setup was also adopted in previous SMC works (Lee et al., 2025).
Similarly,
B2 Be allele (Ita- 12te)) ) Cat(Crt || F2, Vv)
(41)
7. rees (Dree (Tte)) )= Cal(Ere 11 #2.... Zv (
(42)
and
801 + (rez: U1
Ar, (Fea, 122 12)6t
11, P2.... (v))
(43)
+ Ary (Ftte Vv )81
and:
Be, (3ta: : U1 v1)8t
Pi D2..., (112
dez
Brg (Iea U2)
(44)
Brg (1tz uv )8t
Also, we do not use the reference process as we do not observe an instability issue.
A.5 ACCEPTANCE RATE FOR REWARD, CFG AND COMPOSITION
A.5.1 TEMPERING
Suppose To(x) & p)()) for some 3 > 0, with annealing path TT(x) x (x)) the maringal density
ratio satisfies,
Te' (x') (x( x
(x))
RB
(45)
pl(x))
and acceptance function equals,
1. ( (2 R (1
or 2 p/ee)) = min
(46)
Ro (2)
A.5.2 REW.RD-THI.TNN/POSTERRO SAMPLING
Suppose TT(r)= = (x) exp(ro(x)) given a reward/likihood function (z). We can construct an annealing path n. (z) = p/ (x) (x)( where is user specified reward function such that coincides with no at = heuristic choice (Wu al., 2023)
rz(x)= = 7670
E [Ex;~ 1X =0))
(47)
21

---

Published as a conference paper at ICLR 2026
with boundary conditions "/1 0 and T = 1, where the expectation is calculated with Tweedie's formula (Efron, 2011) with the pretrained diffusion. The marginal density ratio satisfies,
ne (xt) Tel (x)e
(xe)
exp(re(sx)) exp(re re(()e)) pi (21) exp(ri(xx))
(48)
(21101)))))
and hence the acceptance probability equals.
{
exp(rr(xx) r((e)) R Re (4 (owee))))) exp(re(x()) re(a()) Re (mpgg) (1
(.e min
(49)
A.5.3 MODEL. COMPOSITION
When To = 115-1 (x) with annealing path To(x) = I1 1 pl (z), the marginal density ratio satisfies,
Tr' (ITe) (xze
ohe (1r) (x))
II j=1
(50)
j=1
and the acceptance probability equals,
R (F
Rly (F(ee) (F(ee)
asp (ITe,))) min
1, II
(51)
j=1 P (1)e)))
A.6 DETAILS ON LOCAL EXPLORATION
In this section, we discuss our choice for local exploration.
A. GAUSSIAN DIFFUSION
For Gaussian diffusion, we modify the corrector step in the predictor-corrector algorithm by Song et al. (2021b). More concretely, we apply the Unadjusted I. Langevin Algorithm (ULAA) using the score of mr- This is, in most cases, available and is simply the combination of the pretrained diffusion's 's score Pt- One exception the reward-tilting case. this case, if the reward is cheap and differentiable, we can simply take the gradient of it to calculate the score of me. In our experiments we apply this local move. On the other hand, when the reward is non-diffent non-differentiable or expensive to take gradient, experiment on prompted reward-tilting, we omit this local move.
trajectory stitching,
omit the local move step. discussed in the footnote in Section 3.1. For our
We also find that using the step size proposed by Song al. (2021b) leads to instability in case. Therefore, we choose to use the step size aligned with the "step size" the denoising process. Precisely, the standard deviation of the Gaussian noise in the EM step (with discretisation size ôt) for Equation (6) step l is vol. In ULA I, we hence the step size jff so that the standard deviation of the Gaussian noise added in the ULA aligns with that in the denoising kernel.
A.6.2 СТмC
For CTMC, the concrete score approximates the density ratio (Lou et al., 2023)
Pe(x)
(x)
(52)
pv(y)
Using this relation, define rate matrix based the concrete score that satisfies detailed balance (DB). One choice is a
rate matrix:
Q(x, y) r(x, y) min(1, s (x),, Latt - Lazy Σ ((x,))
(53)
22

---

Published as a conference paper at ICLR 2026
where r(z, 2 0 is the proposal kernel. For example, could uniform proposal:
r(z,)) y)
(54)
X
Another option is to set the proposal kernel to the noising/masking process r(z, y) - (x, v) directly. However, for tasks such as CFG, we cannot use the predictor-correttrr algorithm proposed in Campbell et al. (2022), as we do not have access to the concrete score for the marginal of the CFG dynamics. Therefore, we need to resort to these MH-style correctors. In our experiments, we also found that CTMC also performs well without local moves.
A.7 COMPUTATIONAL COMPARISON BETWEEN SMC AND CREPE
Here, we include a discussion on the computation cost comparison between SMC and CREPE. To keep the discussion simple, will use the Gaussian diffusion model as example.
Assume in SMC, we resample every K steps, and in total we resample M times with a batch size of N. In CREPE, we run PT at M diffusion times, and diserete K steps for communication, and in total collect samples with N iterations. In both cases, the number of network evaluations will be aligned.
For SMC, it's easy see we need M K N NFES in total.
For the CREPE communication step, at each iteration, we only swap half of the levels, but we need to propose and calculate the RND for both directions, contributing to M K/2 x 2 NFEs. The local move will not need new NFEs, as we already evaluate the score for the newly accepted sample when we calculate the RND. Therefore, in total, CREPE also need M K -V NFEs.
B EXPERIMENTAL DETAILS
B. TEMPERING
Model and Training Details. For all three molecules, we EGNN network following (Hoogeboom et al., 2022). We use the VE (EDM) schedule following Karras et al. (2022), and adopt their preconditioning (Can; Cout; Cskip) as well. For Dipeptide and Tetrapeptide, the network has layers, each with 256 hidden units. For Hexapeptide, we increase the network size to 5 layers and 512 hidden units. Networks are trained with Adam with a learning rate 1e-4 until convergence. We also apply an EMA with rate of 0.999.
Data. The samples were gathered following He al. (2025b) from a 5-microsecond simulation with Generalised Born implicit solvent implemented in openmtt ools (Chodera et al., 2025) with AMBER ff96 classical force field. The Langevin middle integrator is implemented in Eastman et al. (2023) with a friction of 1/picosecond and a step size of 2 femtoseconds.
Metrics. We evaluate energy and interatomic distance TVD following Akhound-Sadege et al. (2024). We use the implementation of W2 distance by Akhound-Sadegh et al. (2024) as well to evaluate the sample W2. However, our system invariant to rotation and translation. Therefore, we align all samples a reference sample by the Kabsch algorithm (Kabsch, 1976) before evaluating W2 distance. We use PYEMMA (Scherer et al., 2015) to project samples into 2D using TICA with lag= 8, The TICA is fitted the ground truth trajectory. We first transfer the sample from Cartesian coordinates to backbone torsion angles before applying TICA. We then employ the MMD to the 2D TICA plot, using the implementation by Chen et al. (2024), with fixed bandwidth chosen to be the mid-distance between ground truth samples data and maintained throughout the evaluation of different methods.
Since some of the metrics are expensive to evaluate on large datasets, we randomly select 5,000 samples from both the ground truth and our generated samples by SMC or CREPE when evaluating all these metrics. We repeat this procedure three times and report the mean along with error bars.
CREPE Hyperparameters Our SDE follows EDM (Karras et al., 2022) with forward SDE defined as dX2 = W2tdW, We choose t E [tmin 0.001, mmx = 10). and discretised into 201 steps following Karras et al. (2022) by [t step ids (t (t/p = with p 7. We then select one PT level every 4 diffusion steps, resulting in M 51 levels, with each level containing K = 4 steps. We run CREPE for 50,000 iterations, collecting all samples after iteration 1000.
23

---

Published as a conference paper at ICLR 2026
We run CREPE with both communication and local move. We also use the reference process when calculating R as described in Appendix 1.4.1.
SMC (FKC and RNE) Hyperparameters We adopt the same schedule and discretisation CREPE. We resample every 4 diffusion denoising steps to align with the setup of CREPE. Both FKC and RNE are run in batches, a common strategy to mitigate the diversity loss in SMC (He et al., 2025e; Lee et al., 2025). Specifically, for each target, we run 50 batches of size 1,000, so that the overall computational budget for SMC and CREPE is aligned.
B.2 IMAGE CFG DEBIASING
Model Details We use the EDM2-S model on ImageNet-64 and the EDM2-XS model on ImageNet- 512 (Karras al., 2024). The ImageNet-64 model is in pixel-space, while the ImageNet-512 model is a latent diffusion model.
CREPE Hyperparameters We aim to debias CFG with 1.7. Our SDE follows EDM (Karras al., 2022) with forward SDE defined as dX, = 22dWW, We choose t [t-min 0.002, fmx 80), and discretised into 128 steps by [tmax + slep.ick (t ) with p = 7. However, we only apply Maruuama updates with
127
PT within the first 100 steps, after which proceed using standard CFG dynamics. We select one PT level every diffusion step, resulting in A/ 100 levels, with each level containing K 1 step. We omit local exploration steps.
SMC (FKC) Hyperparameters We use the same SDE, together with systematic resampling, adopting the same 128 128-step EDM schedule with FKC applied until timestep 100. After this, we proceed using standard Euler-Maruyama updates, same as CREPE.
Evaluation details for Figure 5 In Figure particles N used in CREPE and FKC. For each setup. we run B independent runs with randomly selected B classes. The selection is fixed between CREPE and FKC. We then gather NB samples to evaluate the metrics. To ensure the number of samples is roughly the same, for each different choice of N, we set B ceil(5000/N).
report IR, CLII and FID for different numbers of
B.3 IMAGE REWARD-TILTING
We first debias CFG with w = 1.3, together with reward-tilting. The reward is defined with ImageReward with the prompt we provide.
Model Details We use the same model as CFG debiasing: the EDM2-S model on ImageNet-64 and the EDM2-XS model ImageNet-512 (Karras et al., 2024).
Reward Details The final reward r is given by ImageReward (Xu et al., 2023) with the prompt we provide. We also multiply the reward value by 100 the original magnitude is small. The intermediate reward is defined by rz(zz) = 3,10[zo|az]] where the expectation is calculated by Tweedie's formula using the pretrained score network. The 3g is selected to be smooth interpolant between 31 = 0 and 30 = 1. We use St... = [ + m (3) - with = 5 for the PT level M corresponding m. The correspondence between m and tm is described in the following paragraph.
CREPE Hyperparameters Our SDE follows EDM (Karras et al., 2022) with forward SDE defined as d.X, = / 2f We choose t E [tmin = 0.002, tmax = 80], and discretised into 64 steps by [el/e + step idx (11// )) with - 7. However, we only apply PT within the first 32 steps, after which we proceed using standard E updates with CFG dynamics. We select one PT level every diffusion step, resulting in M 32 levels, with each level containing K step. We omit local exploration steps as the IeageReward is expensive and not implemented in a differentiable way.
Prompt Details The prompts and corresponding class indices are as follows:
"a blue balloon". idx 417
"a colorful pinwheel", idx 723
"a green Christmas stocking", idx 496
24

---

Published as a conference paper at ICLR 2026
"a yellow cab with dark background", idx 468
"an empty shopping cart", idx 791
B.4
MAZE
Model and Training Details We use an MLP with 5 layers and 512 hidden units to parameterise the diffusion model. We use the VE (EDM) schedule following Karras et al. (2022), and adopt their preconditioning (Cins Couty Cstipp as well. Networks are trained with Adam with learning rate 1 le-4 until convergence. We also apply an EMA with a rate of 0.999.
Data We the poi reminntazzed eeninnzz---ae antiinttttttttn tchtt dataset from Park al. (2024), which consists of short trajectories of length 64 in a large 2D maze. We visualise some training trajectories in Figure (Left). We follow Luo al. (2025) to first normalise the data to [-1, 1] for training. When evaluating and visualising, we unnormalise the trajectory back to its original scale.
Metrics We evaluate the success rate from the initial point to the target point. (Luo et al., 2025) considers a trajectory successfully navigating through the maze when the distance between the first point in the trajectory and the initial point smaller than 0.45 (unnormalised scale). However, in our case, we impose a harsher criterion: we additionally require the distance between the tail of the last short trajectory and the head of the next short trajectory to smaller than 0.45. We impose this criterion as our stitching is performed via a reward function instead of directly training the diffusion model conditional on both ends. Additionally, we also require that any points along the entire stitched trajectory have no overlap with the walls.
Choice of Reward Function We first define the reward r for the data in the clean space (1 = 0), and then provide the formula for the reward Tt when > 0. Recall we want to define reward function so that: (1) the starting point of the first trajectory is sufficiently close to the initial state, (2) the endpoint of the last trajectory is sufficiently close to the target state, and (3) consecutive trajectories are connected in a tail-to-head manner. We can impose the L distance for each of the constraints. But it known that 2 distance is making an implicit Gaussian noise assumption, which typically not "sharp" enough. Instead, we can also impose the L distance. However, the L distance can be weak when two points are far apart, as it implicitly assumes a Laplacian noise, which may lead to heavy-tailed behaviour. Therefore, to make use of both advantages of L' and L we take the summation of both. We use X represent the i-th point in the j-th short trajectory. Also, we use -1 1 to represent the last element, following the index convention of Python. We use O and P to represent the initial and target points.
Reward for initial point: , = - (o x - -0133 + A || - -01H)
(55)
Reward for target point: of = - Ar(Aea 10AA==1111100 1-11111-- -
PI +
(56)
Reward for neighboring trajectories:
(57)
y.
Σ Aw( x X x 1
(58)
where we set do Ap 100
J.AN 100, A,2 and Ay 10. and the final reward is:
r=r0 + + - N
(59)
For the case where we introduce an intermediate point I, we additionally impose
pl ΣΣ Aress(Arel - 7112 + A - -711
(60)
where we A, 100 J and an is "attention" defined by
au = cxp /113 T - T)1
(61)
Σ. exp(-AMrT -> X - 111 Tb X - III
and the temperature T = 10. The final results of CREPE will not be strongly influenced by the hyperparameter choices of the reward. However, there are two main principles to tune these values: (I) the reward strength should not be too small, as the trajectory will not be well-connected;; (2) the reward strength should not be too large, otherwise the PT swap rate will be 0 at some PT levels.
25

---

Published as a conference paper at ICLR 2026
Therefore, when tuning these hyperparameters, one may not need to run the algorithm for long time. If the short trajectories do not form a tail-to-head manner quickly, then the strength needs to be increased. On the other hand, if the communication rate is always 0 at some PT levels, the strength needs to be reduced.
We then consider how to choose the intermediate reward rt at diffusion time step t. We define it following the same principle as Chung et al. (2023):
Te (x X1 x 3. r. (E B[Xo|X/]] -M ,,XX//X)))
(62)
where we use X to represent the j-th short trajectory and E[Xo| X?] are calculated by Tweedie's formula using the learned score network. The Be is selected to be a smooth interpolant between
31 and 3g- 1. We use f... [s M (3 s//21e,, with p 10 for the PT level corresponding to tm- In the next paragraph, we describe the correspondence between m and tm-
CREPE Hyperparameetes Our SDE follows EDM (Karras et al., 2022) with forward SDE defined as dX; ,2, We choose imin 0.001, &max 20). and discretised into 601 steps following Karras et al. (2022) by [mmae + step.idk (1 //2/P - [ with 7. We then select one PT level every diffusion step, resulting M 601 levels, with each level containing K steps. We run CREPE with both communication and local move.
600
B.5 СТмC
Model Details We follow the experimental setup of Lee al. (2025) for CTMC experiments MNIST. We use Campbell et al. (2022)'s UNet with num_scales=] bum.res.blocks-- ch=64.
ch-mult=11, 2, 4], class_embed.cimm-33
SMC Hyperparameters We aim to sample from the target with CFG strength w = 1.2. We follow (Lee et al., 2025) perform partial resampling, using resampling fraction of 80% and an effective size threshold of 0.2 (i.e., trigger resampling when ESS < 0.2). We discretise the diffusion time horizon with 200 steps, and perform SMC with a batch size of 128. When collecting data for evaluating FID, we repeat 4 batches for each class.
CREPE Hyperparameters We use the same setup for w and discretisation steps as SMC. We treat each diffusion step as one PT level, resulting in M 200 levels, with each level containing K = 1 steps. We run 512 steps for each class to align the budget with SMC. We did not perform the local move and found it works well.
B.6 TEXT CTмC
Task We consider sentiment-controoledd text generation using discrete diffusion. The goal is to generate text with a specified sentiment (positive or negative) by conditioning on the prompt "The sentiment of the text (posative
Model Details We finetune the SEDD-Medium model (Lou et al., 2023) for 5 epochs on the SST-2 sentiment control dataset from Amini al. (2024). The finetuned model serves the conditional diffusion model for both SMC and CREPE.
Sampling Setup We discretise the CTMC process into 100 steps and generate 1000 samples length 1024. For SMC, we use batch size of 50 particles. For CREPE, we run 150 burn-in steps followed by 1,000 sampling steps, yielding 1000 samples after burning in. We sweep over target strength E (1.0, 1.2, 1.4, 1.8} and proposal strength E (1.0, 1.2, 1.4,1
Evaluation We evaluate sample quality along two axes:
Perplexity: We finetune a GPT2-XL model with LoRA on the SST-2 dataset and use it to compute perplexity of the generated samples.
Sentiment accuracy: We classify generated samples using pretrained DistilBERT classifier finetuned for sentiment analysis.'
hhttps://huggiingffcc..oo 4/disti 1bert- netuned
26

---

Published as a conference paper at ICLR 2026
C ADDITIONAL RESULTS
C.I SAMPLES OF DEBIASING CFG ON IMAGENET
Here provide more examples for CFG Debiasing with FKC and CREPE in Figures 11 to 14. The x-axis is the number of particles: SMC (e.g., FKC), they are running in parallel, while in CREPE, they running sequentially. For clear visualisation, y-axis corresponds to the diffusion steps: in SMC, they are running sequentially along one direction; while CREPE, they running parallel and undergo mutual communication steps. For clear visualisation, we thin along this axis by factor of 2. We can see that SMC tend to have low sample diversity due repeatedly resampling, while CREPE maintains higher sample diversity after burn-in.
thin along this axis by factor of 8. The
(a) FKC
(b) CREPE
Figure 11: CFG Debiasing with FKC and CREPE for class "slot" (idx 800).
(a) FKC
(b) CREPE
Figure 12: CFG Debiasing with FKC and CREPE for class "tick" (idx 78).
27

---

Published as a conference paper at ICLR 2026
FKC
(b) CREPE
Figure 13: CFG Debiasing with FKC and CREPE for class "spiny lobster" (idx 123).
(a) FKC
(b) CREPE
Figure 14: CFG Debiasing with FKC and CREPE for class "toilet tissue" (idx 999).
C.2 SAMPLES OF REWARD-TILTING ON IMAGENET
In Figure 1, we visualise chains of thinned samples for different prompts. Here, we present the results for the entire PT iteration. As each task has 200 images of 512 512, we downsize them here. We observe that CREPE produces diverse samples across iterations, aligning with the prompt after the initial burn-in in period. We also notice that neighbouring iterations often yield similar images, which arises from the use non-reversible PT: the last PT level is updated only in alternating (odd or even) iterations.
28

---

Published as a conference paper at ICLR 2026
(a) class: balloon; prompt: a blue balloon.
(b) class: pinwheel: prompt: a colorful pinwheel.
(c) class: shopping cart,
(d) class: Christmas stocking;
prompt: empty shopping cart.
prompt: green Christmas stocking.
(e) class: cab:
(f) class: unbrella; prompt: closed umbrella.
prompt: a yellow cab with dark background.
ImageNet-512. From left to right and top to bottom, each image corresponds to one CREPE iteration. We visualise the entire image trajectory in the first 200 PT iterations. The last example 15f is failure mode. Please refer to Appendix C.3 for discussion on the failure mode.
Figure 15: Prompted reward-tilting
C.3 FAILURE MODE OF REWARD-TILTING ON IMAGENET
While we demonstrated that prompted reward-tilting can be used to control image content in finer detail, it does not always succeed. One failure arises when the class contains samples that satisfy the prompt. For example, in Figure 15f, the algorithm fails to generate a closed umbrella.
C.4 SUCCESS RATE OF TRAJECTORY STITCHING
Table Success rates of CREPE across 5 tasks. The CompDiffuser results are taken from Luo et al. (2025), while the CREPE results averaged over 250 samples within each iteration range.
Method
Success rate (%)
CompDiffuser (Luo al., 2025)
68
iteration 0-50k
8.5
CREPE
iteration 50-100k
59.7
iteration 100-150k
84.6
29

---

Published as a conference paper at ICLR 2026
C.5 MORE RESULTS ON TRAJECTORY STITCHING WITH ONLINE REFINEMENT
In Figure 6, we select representative PT iterations to visualise the trajectory samples. To further quantify the performance of online refinement, evaluate the success rate of reaching the final target and the pass rate through the intermediate point, shown in Figure 16. At iteration 100k, we introduce additional reward corresponding the intermediate point. As be seen, the pass rate through the intermediate point quickly incrasses after the new reward is added, while the success rate slightly drops but remains high. This demonstrates that CREPE is capable of flexibly incorporating new constraints during sampling and adapting the trajectories online without retraining.
1.0
success rate
0.5
redd intermmeiaaee point
1
x10
pass rate at Intermedlate point
1.0
0.5
add
intermediate
point
0.0
PT Iterations
x10
Figure 16: Success rate and pass rate through intermediate point in the trajectory stitching task with
online refinement.
D DEBIASING CFG FOR CTMC ON TEXT
Larget strangchmm... proposallllll.o...o.ooooo...... strungthw1.c
larget strenggh proposal strengthm1.4
target strength proposal strengthmi
1.0
Bunning Accuracy
1.01
1.01
0.9
1.00
1.00
0.8
0.99
0.99
boundary
0.98
110
Running Perplesity
100
90
400
90
60
80
70
1000
1000
Samale Index
Sample Indox
Samplc Indox
Figure 17: Running average perplexity and sentiment accuracy as a function of CREPE sampling step for three temperature combinations of proposal and target CFG strength.
Figure 17 shows the running average of perplexity and accuracy CREPE generates ples, for three temperature configurations: (target strength = 1.4, proposal strength = 1.0). (target strength 1.4, proposal strength 1.4), and (target strength 1.8, proposal strength
30

---

Published as a conference paper at ICLR 2026
1.8). Perplexity decreases more samples generated, and accuracy is maintained close to 1.0, indicating effective sentiment control. This demonstrates that CREPE successfully debiases the bias introduced by CFG, leading to better text sample with lower perplexity. The exception is the (1.4, 1.0) configuration, where accuracy drops around step 1200. possible explanation is that a bad sample was proposed and accepted at this iteration, revealing potential weakness of CREPE that requires future exploration.
D.1 DEBIASING CFG FOR CTMC ON MNIST
Figure 18: We visualise the PT trajectory along both the annealing path and PT iterations. For clarity, we thin the annealing path by factor of 4 and record only every 8th PT iteration.
31


Reviews:
[{"review_id": "jnH4soqHSe", "reviewer": "Reviewer_MkR6", "summary": "The paper proposes CREPE, an inference-time control framework for diffusion models based on replica exchange (accelerated parallel tempering), offering an alternative to SMC. Across molecules, images (ImageNet-64/512), and trajectory stitching, CREPE achieves competitive or better quality and diversity than SMC with similar NFE cost, while supporting online refinement and anytime sampling.", "questions": "- Why is generating particles sequentially better than generating them in parallel? Can you provide any theoretic background of this?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Principled and general formulation.\n- Practical advantages over SMC: better sample diversity after burn-in, lower memory (no large particle batches), online refinement and early stopping; compute parity analysis clarifies similar NFE cost.\n- Broad, convincing experiments across modalities and tasks showing improved metrics (e.g., FID/IR, TICA diversity) and flexible compositions (e.g., CFG debiasing + reward tilting, trajectory stitching).", "weaknesses": "Novelty\n- Important recent related works are missing [1,2,3]. Especially, DAS [1] avoids reward over-optimization by the tempering technique, thereby maintaining output diversity, and also deals with online reward optimization, which significantly affects the novelty of this work. The author should contrast how their proposed APT are better than DAS in terms of design.\n- Some of the work uses sequential particle generation, rather than solely using parallel generation [3]\n\n\nExperiment Setup\n- Guidance results with recent T2I backbones (stable diffusions) are missing, which are widely used to compare the efficacy of inference-time guidance methods for diffusion [1,2,3].\n\n\n---\n\n[1] Test-time alignment of diffusion models without reward over-optimization, ICLR, 2025\n\n[2] Inference-time scaling for diffusion models beyond scaling denoising steps, CVPR, 2025\n\n[3] Inference-time Scaling of Diffusion Models through Classical Search, ArXiv, 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DHsEd0VtYG", "reviewer": "Reviewer_LYC3", "summary": "This paper focuses on inference-time control of diffusion models. It proposes a method based on parallel tempering as an alternative to guidance and Sequential Monte Carlo (SMC) methods. The method is applied across annealing, reward tilting, model composition, and classifier-free guidance debiasing tasks.", "questions": "1. Regarding Limitation 2 of SMC: It's confusing whether 'sample diversity' in line 77 refers to particle diversity during or at the end of SMC, or diversity of the sample distribution generated via SMC. In either case, it would be worth justifying the reason why it happens. Also, if the authors are referring to the first, one can run separate SMC samplers multiple times to sample diverse samples.\n2. Related to Question 1, the claim that SMC requires a large batch of particles (Section 3.3, line 353-354) doesn't seem to hold for SMC-based inference-time control of diffusion, as recent works ([1], [2]) show that few as 4 particles are enough to sample high-quality samples. That said, running separate SMC samplers multiple times seems like a (computationally) viable strategy. Superiority of PT over SMC would be more convincing if the authors included this as a baseline.\n3. Regarding Limitation 1 of SMC: SMC is memory-intensive due to saving particles. However, isn't CREPE also (perhaps even more) memory-intensive due to saving all timesteps? (This relates to the claim that SMC requires a large batch of particles in Question 2)\n4. Regarding Limitation 3 of SMC: Indeed, basic SMC can't refine samples, but it can be naturally extended via Nested SMC where the outer sampling adds noise and denoise back, like how recent works ([3], [4]) do.\n\n[1] Singhal, Raghav, et al. \"A General Framework for Inference-time Scaling and Steering of Diffusion Models.\" Forty-second International Conference on Machine Learning.\n[2] Kim, Sunwoo, Minkyu Kim, and Dongmin Park. \"Test-time Alignment of Diffusion Models without Reward Over-optimization.\" The Thirteenth International Conference on Learning Representations.\n[3] Uehara, Masatoshi, et al. \"Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design.\" arXiv preprint arXiv:2502.14944 (2025).\n[4] Ma, Nanye, et al. \"Inference-time scaling for diffusion models beyond scaling denoising steps.\" arXiv preprint arXiv:2501.09732 (2025).", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper investigates a new direction of applying Parallel Tempering (PT; Replica Exchange MCMC), a widely used sampling algorithm, as an alternative to SMC for inference-time control of diffusion models.\n2. The paper proposes a concrete algorithm tailored for diffusion models that can directly benefit from the properties of PT.\n3. Experiments are conducted across diverse domains, including molecule sampling, image generation, and navigation, to demonstrate its effectiveness and versatility.", "weaknesses": "1. The limitations of SMC-based inference-time control methods, which are the key motivation of this paper, aren't convincing enough. It should be supported by more concrete statements or additional experiments. SMC is currently the most widely used method for inference-time control, and I hope the authors can provide more evidence on why the community should explore directions using PT. (See Question 1, 2, 3, 4)\n2. The motivation of why PT is a promising alternative is not presented thoroughly in the introduction, and the explanation of why it could increase sample diversity isn't presented throughout the paper (or at least in the introduction or Section 3.3, where it should be).\n3. Introduction claims that derivation of PT swap rates is their key contribution, but it's not presented in the main text.\n4. It would be more interesting and convincing if there were applications of CREPE for more modern pre-trained models, such as the Stable Diffusion model family.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ukTBhzjjCv", "reviewer": "Reviewer_2CNP", "summary": "The paper introduces CREPE, a new inference-time control method for diffusion models. Unlike common approaches based on Sequential Monte Carlo (SMC), CREPE sequentially generates samples using accelerated parallel tempering (APT; Zhang et al. 2025), a variant of parallel tempering MCMC that runs parallel chains at different diffusion timesteps and swaps samples between adjacent levels. The proposed method is empirically shown to approximate the target distribution more accurately and exhibits higher sample diversity compared to SMC-steering.", "questions": "1. Line 96, \"without explicit target densities\": What does this mean? This claim does not seem valid for a reward-tilted setting.\n2. How robust is the algorithm to the initialization, particularly when the number of iterations is limited?\n3. Do SMC baselines use the same proposals as in Appendix A.2?\n4. Do SMC baselines use standard techniques, like adaptive resampling (resampling when ESS below a certain threshold) and systematic sampling [1]?\n5. Line 1040, \"we run 50 batches of size 1000\": Does this mean running SMC with 50,000 particles?\n\n---\n\n### References\n[1] Douc, Randal, and Olivier Cappé. \"Comparison of resampling schemes for particle filtering.\" ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005. IEEE, 2005.\n\n---\n\n### LLM usage disclosure\nI used LLM only to check grammar.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is clearly written and easy to follow. Their derivations are mathematically rigorous.\n2. The proposed algorithm is versatile and has broad practical potential for any application requiring inference-time control of diffusion models.\n3. The experiments are promising and directly compare against strong SMC baselines. The results show that CREPE achieves competitive performance while maintaining higher sample diversity.", "weaknesses": "1. A major concern is that this work doesn't seem to provide methodological improvement over accelerated parallel tempering (APT). To me, it seems a direct application of APT to inference-time control with pretrained diffusion (with an assumption that the pretrained model perfectly models the time-reversal of a noising process). An algorithmic difference with APT, or an explanation on why applying APT to inference-time control is not trivial, would be helpful to understand the contribution.\n2. Some details in the experiments section are missing, e.g., the number of random seeds used for each experiment, and the standard deviation for Table 2, runtime compared to SMC, etc. Also, the source code is not included.\n\nI'm open to raising my score if those two points are resolved.\n\n(Minor)  \n3. References need to be added in the appendix A.2., since some of them are studied in previous works.  \n4. Analysis of the acceptance probability would be valuable (e.g., how the choice of proposals affects the acceptance rates)  \n5. Typo, Line 198, the arrow for the backward process Markov  \n6. Type, Line 1088,\"For the CREPE communication step.\" needs to be removed", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g5s2RAnEkE", "reviewer": "Reviewer_AFgB", "summary": "This paper introduces CREPE (Controlling Diffusion with Replica Exchange), a framework for inference-time control of diffusion models via replica exchange (parallel tempering). The method adapts ideas from nonequilibrium statistical mechanics to generative inference by allowing multiple diffusion chains, each at different noise levels, to exchange information using accelerated parallel tempering (APT).\n\nThe authors derive swap acceptance ratios through Radon–Nikodym estimators between forward and backward path distributions, ensuring theoretically consistent swaps. Empirical evaluations on image and molecular generation tasks demonstrate that CREPE achieves comparable or slightly improved performance to SMC-based debiasing methods while enhancing sample diversity and supporting online refinement.", "questions": "1. How sensitive is CREPE’s performance to the number of replicas and swap frequency? \n\n2. What is the observed swap acceptance rate and average traversal time for a full temperature cycle in your experiments?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "1. Applying replica exchange to diffusion inference is novel. It draws a meaningful connection between thermodynamic sampling methods and modern generative modeling.\n\n2. The experiments cover several relevant applications—tempering, reward-tilting, model composition, and discrete diffusion.\n\n3. Replica exchange could improve performance and sample diversity, which is well-studied in the MCMC area.\n\n4. The paper is generally well-written, with clear motivation and good presentation.", "weaknesses": "My main concerns revolve around the practical overhead and complexity inherent in the Replica Exchange framework when applied to generative tasks, despite the theoretical benefits.\n\n1. In my opinion, the advantages of replica exchange/parallel tempering are (1) potential performance improvement in some metrics and (2) increased sample diversity. However, the disadvantage of this method is also very clear: each sampling step requires a larger Number of Function Evaluations (NFEs). For instance, if you consider N chains, a standard step requires N steps + additional swap calculations (N+ NFEs), whereas a single-chain method only requires 1 NFE. While I understand that some replica exchange design in MCMC try to improve efficiency by optimizing chain swap rates and tuning various hyperparameters to outperform single-chain methods under the same NFEs, the problem remains significant: (1) You have a large number of hyperparameters that require fine-tuning, such as the temperature and step size for each chain, and the hyperparameters related to the swap design. This complexity is very challenging in generative tasks. I am not questioning the contribution of this paper, but rather, I do not believe that the replica exchange approach can be broadly effective in generative models. Although this method provides an elegant analytical framework, I do not believe it can both significantly improve performance and be simple to use for generative tasks.\n\n2. MCMC-based sampling methods like CREPE necessitate a warm-up (burn-in samples) phase to achieve the target distribution, which requires potentially hundreds or even thousands of NFEs to reach low FIDs. This contrasts sharply with recent advancements in image generation, where distilled diffusion models [1-3] achieve competitive performance (e.g., FID below 2) on larger-resolution images using only a few NFEs or even one NFE. This fundamental requirement for a long burn-in period and high NFEs limits the applicability and competitive efficiency of MCMC sampling schemes like CREPE in real-time or resource-constrained engineering domains.\n\n3. There is no discussion on the related work of replica exchange / parallel tempering. Since these techniques have been thoroughly studied in MCMC, it would strengthen the paper to discuss which insights from the MCMC community could transfer to generative settings (e.g., ladder design, replica spacing, or swap design).\n\n\n4. The paper would benefit from diversity metrics (entropy, coverage, or mode count) to support this claim quantitatively, related to Figure 7 (replica exchange benefits sample diversity).\n\n5. Too frequent swaps can hinder exploitation in low-temperature chains. The paper mentions the use of non-reversible even/odd communication but provides no statistics (e.g., acceptance rate, cycle time from the lowest to the highest temperature). \n\n6. The swap acceptance rule appears to rely on exact energy or log-probability evaluations. It is unclear how this is handled under mini-batch training or high-dimensional data, where computing full log-likelihoods may be infeasible. Some clarification or approximation discussion would be valuable.\n\n**References:**\n\n[1] One-step Diffusion with Distribution Matching Distillation. CVPR 2024\n\n[2] Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion. ICLR 2024\n\n[3] Consistency Models Made Easy. ICLR 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)