Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Design
Authors: Zijing Ou
Chinmay Pani
Yingzhen Li
Venue: ICLR 2026 Poster (2026)
Keywords: discrete diffusion
test-time scaling
reward aligntment
Primary area: generative models

Abstract: Discrete diffusion models have become highly effective across various domains. However, real-world applications often require the generative process to adhere to certain constraints.
To this end, we propose a Sequential Monte Carlo (SMC) framework that enables scalable inference-time control of discrete diffusion models through principled importance weighting and optimal proposal construction.
Specifically, our approach derives tractable importance weights for a range of intermediate targets and characterises the optimal proposal, for which we develop two practical approximations: a first-order gradient-based approximation and an amortised proposal trained to minimise the log-variance of the importance weights.
Empirical results across synthetic tasks, language modelling, biology design, and text-to-image generation demonstrate that our framework enhances controllability and sample quality, highlighting the effectiveness of SMC as a versatile recipe for scaling discrete diffusion models at inference time.

Published as a conference paper at ICLR 2026
INFERENCE-TIME SCALING OF DISCRETE DIFFUSION MODELS VIA IMPORTANCE WEIGHTING AND OPTIMAL PROPOSAL DESIGN
Zijing Ou, Chinmay Pani, Yingzhen Imperial College London
2 ou220 iieeerimpiiaaa ac. uk
ABSTRACT
Discrete diffusion models have become highly effective across various domains. However, real-world applications often require the generative process to adhere certain constraints. To this end, propose a Sequential Monte Carlo (SMC) framework that enables scalable inference-time control of discrete diffusion mod- els through principled importance weighting and optimal proposal construction. Specifically, our approach derives tractable importance weights for a range of intermediate targets and characterises the optimal proposal, for which we develop two practical approximations: a first-order gradient- approximation and an amortised proposal trained to minimise the log- -arvinnnecee of the importance weights. Empirical results across synthetic tasks, language modelling, biology design, and text-to-image image generation demonstrate that our framework enhances controllability and sample quality, highlighting the effectiveness of SMC versatile recipe for scaling discrete diffusion models at inference time.
INTRODUCTION
Diffusion models (Sohl-Dickstein al., 2015; Ho al., 2020; Song et al., 2020) have achieved remarkable success across various domains, from image synthesis (Rombach et al., 2022; Esser et al., 2024) to scientific applications (Hoogeboom et al., 2022; Watson et al., 2023). Recently, advances in discrete diffusion models (Austin et al., 2021; Sahoo et al., 2024; Shi et al., 2024) have established them as powerful approach for modelling discrete data, notably in tasks such as language modelling (Nie et al., 2025; Zhang et al., 2025a) and code generation (Gat et al., 2024; Gong et al., 2025).
Despite their impressive capabilities, pretrained diffusion models often need to generate samples that meet specific downstream constraints. For example, text-to-image generation requires images aligned with human preferences (Black et al., 2023; Fan et al., 2023; Uchara et al., 2024), while protein generation demands stability or desired binding affinity (Verkuil et al., 2022; Uchara et al., 2025b). To address this challenge, existing approaches mainly fall into two categories: i) fine-tuning and ii) guidance methods. Fine-tuning methods, including techniques such steering (Rector-Brooks et al., 2024), reinforcement learning (Zekri & Boullé, 2025), and direct backpropagation (Wang et al., 2024), have demonstrated promising results. Nevertheless, these methods often suffer from reward over-optimisation, which can compromise sample quality and diversity. On the other hand, guidance and sampling methods (Li al., 2024; Gruver et al., 2023; Nisonoff al., 2024; Guo al., 2024; Uchara et al., 2025a) provide training-frr free alternatives that are easier to deploy. but they often suffer from reward under-optimisation. This limits their ability to enforce correct alignment, resulting in outputs that may not fully meet complex constraints.
In this paper, with a primary focus on discrete diffusion models, we propose Sequential Monte Carlo (SMC) (Del Moral et al., 2006) framework for test-time inference. By leveraging SMC, an asymptotically unbiased sampler, our approach enables test-time scaling, effectively addressing the over-optimisation issues commonly encountered by fine-tuning methods. Moreover, we propose a learnable amortised proposal to approximate the optimal SMC proposal, which mitigates the
"Equal contribution.

---

Published as a conference paper at ICLR 2026
under-optimisatee problems often associated with guidance- based methods, thereby improving both scalability and efficacy during inference. In summary, our contributions include:
We propose simple SMC framework for discrete diffusion models. By leveraging tractable importance weights, we show that SMC provides a general recipe for test-time scaling, enhancing classifier-free guidance and enabling effective reward alignment.
We propose two approximately optimal proposals: first-order approximation and a learn- able amortised proposal. The latter optimised by minimising the log-variance of impor- tance weights, leading to substantial improvements in the effectiveness of SMC.
We demonstrate the versatility of the proposed approach across broad range of applications, including language modelling, biology design, and text-to-image generation, highlighting its ability to consistently improve performance and generalise across diverse domains.
2 BACKGROUND
We first introduce the main preliminaries: discrete diffusion models and Sequential Monte Carlo.
DISCRETE DIFFUSION MODELS
Discrete diffusion models (Austin et al., 2021) define forward nosing process that interpolates between the original data distribution and fixed prior e E A on the V-simplex:
p(aa = Cat 0,70 + (1 - oz)u),,
where a is monotonically decreasing schedule from to 0 such that ær Cat(v). Masked diffusion models (Sahoo et al., 2024; Shi et al., 2024) are a special case that use a mask token [m] as the prior, with the induced posterior taking the form of
(at((x11122))
+ [m]
p(re-1/ze, m0) = Cat (1 Ul-,, M/ +aa
(2)
[m]
1-a;
Since zo is not available during inference, the reverse unmasking process is parametrised as (o(= [ry) = p(=,_1/21, (zt)), where (0())) is a denoising model that predicts the clean data
To. The model is trained by minimising the cross-entropy loss
T
de
L( 0) = Σ
Ep [())) I- log(") po (xx))) dt,
(3)
t=1
which is equivalent, if oo, the negative evidence lower bound of the log-likelihood log po (3r)
2.2 IMPORTANCE SAMPLING AND SEQUENTIAL MONTE CARLO
Consider the Monte Carlo integration problem Ex(,,) [6(x))], where sampling from the target distri- π. ( ((Eer)
bution æ is intractable. Importance sampling (Robert al., 1999) alleviates this issue by introducing proposal distribution allowing the expectation to be rewritten as
( (at22)166 [5(xz)) (gg(er))
((x))
(lx1))
oer ~ (FET).
(4)
While conceptually simple, importance sampling often suffers from high variance. To address this limitation, Sequential Monte Carlo (SMC) (Del Moral al., 2006) extends importance sampling by incorporating resampling and sequential weighting strategies across the path, thereby reducing variance in practice. In SMC, a key intuition the recursive formulation of the importance weight
(a
(( (
(x
(
(xx (6) (a(2))
Wy- (2 L:TT
We (1 (ITT)
(5)
q( Ø
(2
I
where we leverage the Markovian assumption that (2 trary forward kernel 7 and thus (1x ( (o()))
n ( Iire ( 1x (+)
for arbi- The recursion of
(6)
(3) (
1 (x(
2

---

Published as a conference paper at ICLR 2026
importance weight underlies the iterative procedure of SMC. Concretely, The procedure initialises begins by N particles of q(xr) with weights ( <- <<>4f/ (a(xl) ). For each step t ...... and particles i = 1, ..... N. SMC proceeds as follows: i) resample ancestor according to the weights {wr (w(") ( Ji=1= ii) propagate new particles via 20 g(zc-1(2)); and iii) updating the weights as (6)
wi-1 + (-( (x; (a (m/((( )g(=)_1 [=? The resulting collection of weighted particles
(
provides an asymptotically consistent approximation of the intermediate target distribution π (2re).
SEQUENCE MONTE CARLO FOR DISCRETE DIFFUSION MODELS
Given pretrained discrete diffusion model (o(2)). consider sampling from modified target distributions that enable inference-time control. These targets include: i) product distributions, general form underlying classifier free guidance (Ho & Salimans, 2022), defined x (())) xx pë, (xx, ) (ar); and ii) reward-tilting distributions, expressed as T(x+)) x po po(x)) exp(r(x:)) In the following section, we introduce how to construct tractable importance weights by carefully selecting forward kernels (x))) and show their connection to existing SMC formulations for continuous- time discrete diffusion models. We then discuss the choice of proposal distributions, which play a central role in balancing variance reduction with computational efficiency.
3.1 IMPORTANCE WEIGHT: TRACTABILITY WITH PRETRAINED DIFFUSION MODELS
To perform SMC, must evaluate the importance weight from Equation (5) at each step While the forward kernel ([(1)) and the proposal q(#;-1|zz) can be chosen flexibly, the ratio of
intermediate targets x(z-1) is generally intractable in diffusion models. With well-trained diffusion
[(xs)
model po. however, this ratio can be approximated via detailed balance po(2x-1) pa( where
pa(as) p(a)(=1-1>
p(_,,/=,-)) denotes the forward noising process and pa ( is the learned reverse counterpart.
Under this approximation, the importance weight for the product target takes the form
product: Pd, (1=-1/2:)1% D&, (01-1 (x
(6)
( 21 )r (1 (
Although tractable, this weight inevitably introduces approximation error unless the reverse model perfectly trained, due to the mismatch between the forward and backward processes. In contrast, for the reward-tilting the error can be eliminated by setting (xx(2;=-1) = p(zz|zt-1) yielding
exp(r(x(-1)) po( (111)1111) exp(r(x(-1) po(
reward-tilting: exp(r(x:)) p(xx(=r-1) q(
exp(r(x:)) q(x;-1 1xe)
is noteworthy that this cancellation is applicable to the product distributions, since the normalising con- stant of y can not be cancelled even if we choose y x pf (2e(2;=1)pp) (x))))) Nevertheless, as illustrated in Fig- ure 1, SMC with these tractable importance weights per- forms well across both two settings on 2D toy examples. Moreover, although we primarily focus discrete-ting diffusion, the proposed SMC method can be extended seam- lessly to the continuous-time setting, established in the following proposition.
plas plviotn
Figure I: SMC results for the reward- tilting and product target distributions.
Proposition (SMC for Continuous-Tinee Time Discrete Diffusion), Let R4 be the rate matrix generating the forward transition kernel (xx, At); and RG be its counterpart associated with the backward proposal kernel ((0t-2e(1a)) (xt, where At - is the infinitesimal time increment. Then, the importance weight at time t is given by wi = - - log (Ix)) ( + Ey, Rg (2 Ve [(x.)
a(ua) ds, if the forward kernel
is chosen such that the rate matrices satisfy detailed balance R, Be (Z,, )= = Ro (T))).
Proposition I coincides with the importance weight used in Holderrieth et al. (2025), which focus on sampling scenarios where the intermediate target π is tractable up to an unnormalized constant. In contrast, our setting concerns test-time control of pretrained diffusion models, where π is intractable. In Appendix B.2, we further connect our result to Lee et al. (2025), who also study reward tilting, but provide derivation from the perspective of discrete-time diffusion.
3

---

Published as a conference paper at ICLR 2026
Algorithm Training Optimal Proposal
Algorithm 2 SMC Sampling Procedure
1: Rollout trajectory (2 with giet (2)-2(a)))
1: Propose particles _ q(<)_1/=1)
2: Compute gradient with Equation (10)
2: Compute importance weight with Equation (5)
ga- 96) < + V Σ,
w
ا )(
( g w6
(9
3: Update <i db using fo, Iv
Σ, w
3: -asampplee...MMMtttnnmmaaala
(1)
+ Φ 19
10 + v 79
3.2 CHOICES OF PROPOSAL: THE WAY TO REDUCE VARIANCE
While the proposal q( (2x) offers substantial flexibility, the statistical efficiency of SMC is highly sensitive its choice: suboptimal proposals induce high-variance importance weights, which in turn precipitate particle degeneracy and hinder adequate exploration of the state space (Del Moral et al., 2006). Conversely, an appropriately constructed proposal substantially mitigates weight variance, thereby enhancing the effective sample size and ensuring stability of the inference procedure. The following proposition characterises the minimum variance choice of proposal:
Proposition 2 (Locally Optimal Proposal), Given the incremental importance weight as in Equa- tion (5) W (I+-1, v2a =(==-1)((a((s_1) =(=e))(B;-1(=x)) the proposal distribution that minimises the variance of 11-11 often referred to as the locally optimal proposal, is q(xt-1)24) x ((
Building on Proposition 2. one readily verifies that the optimal proposal distribution for the case of product target is tractable (see the remark in Appendix A.2 for discussion), under the choices of forward kernels 7 x pf (xx (1:-)=)(5555 (xx In contrast, for the reward-tiltig, the locally optimal proposal takes the form sible due to the inaccessibility of the corresponding normalising constant. Consequently, practical implementations must resort to approximations that balance variance reduction with computational feasibility. In what follows, we introduce two approximation strategies tailored to the reward-tilting setting: i) a baadee method to achieve first-order approximation, and ii) neural proposal trained to minimise the log-variance of the importance weight.
(xp(r(x1-1))p) (x==1 which is generally computationally infea-
3.2.1 APPROXIMATED OPTIMAL PROPOSAL VIA FIRST-ORDER APPROXIMATION
In reward tilting, evaluating the locally optimal proposal requires computing the normalising constant Z = cxp(r(xt-1)Ppo (xt-11=))) This computation entails O(x2)) forward pass through the reward model at each denoising step, which significantly slows down the generation speed, rendering the method impractical for large discrete state spaces. To mitigate this issue, we adopt the approach of Grathwohl et al. (2021); Zhang et al. (2022), treating r(xx) as function defined over continuous real-valued inputs, while evaluating on the discretised domain of interest. This allows us to apply a
first-order Taylor expansion to approximate the reward: r(z;-1) R r(x:) + (I x,)" (rr(),)
which in turn yields first-order approximation the locally optimal proposal:
(8) This approximation improves computational efficiency by requiring the reward function r to be evaluated and differentiated only once at de, instead of repeatedly across all states. Nevertheless, it assumes differentiable rewards and remains costly when the reward model is large. Motivated by Richter al. (2020); Richter & Berner (2023) and the amortisation technique in variational inference (Dayan et al., 1995; Kingma & Welling, 2013), we propose learning an amortised network that approximates the optimal proposal, resulting in a transition kernel that directly transports between successive intermediate targets as in Matthews et al. (2022). This reduces computation to a single network evaluation, thereby significantly enhancing the efficiency of SMC.
(<=)-12z;) x p0( exp(=]= 1
3.2.2 AMORTISED OPTIMAL PROPOSAL VIA LOG- VARIANCE MINIMISATION
To train network ge to approximate the locally optimal proposal, natural approach is minimise the log-variance of the importance weight:
exp(r(x(-1) pe (e(x2-1(ze) A Liog-var (Φ)
min V (Fo))) 9
Σ
log exp(r(x:)) (0(0)-1(11))
4

---

Published as a conference paper at ICLR 2026
where geef is arbitrary reference distribution that has the same support pe and lá. The following corollary establishes the validity of the proposed oogvariancc objective.
Corollary 1.
The locally optimal proposal 4 x x ( -1 )o (no(#t-1/1 that achieves the minimum
variance of the important weight V s(a--1) (2(()) unique.
((a))g(Z(-11 (as))
Although conceptually simple, naive Monte Carlo estimation of Ligg suffers from high variance and computational cost. To alleviate these issues, we introduce an auxiliary network Fy. : ++ R, parameterised by v., that estimates the mean of the log-weight. This yields the refined objective:
L( v/) = Ei,grolz 1.m)) log exp(r(x=-1)) pa (a( - (t)(t)
(10)
exp(r(x;)) 9o (0(x)-1(2,)
which provably upper bounds the Vorvinriance loss. To be specific, the following proposition holds:
Proposition 3. For any reference distribution Gret, we have Lior-var (Ø) S 7 [(ó vb. Moreover the minimiser C unique and attains its optimum when qs x (xp((((x=1 (2c-1
We outline the training and sampling procedures in Algorithms 1 and 2. For clarity, designate SMC SMCgrd. and SMCamot to denote, respectively, the variants employing the pretrained diffusion proposal, the first-order approximated proposal, and the learned amortised proposal. We further denote the first-order approximated and amortised proposals by Propgrad grad and Propanot which coincide with their corresponding SMC methods when restricted to single particle.
3.3 SEQUENTIAL. MONTE CARLO RECIPE: PRACTICAL IMPLEMENTATION
Building the theoretical characterisation of optimal proposals, next present a practical SMC recipe. An essential ingredient of the proposed framework is the introduction of twisted intermediate target for reward-tilting: (xxe) х po(x;) exp oo rz:)). where a > is a KL-regularisation
coefficient. This construction is motivated by the following identity
((x))
argmax B, [r(x:)] aXLL(=llps) x po(x)) exp
(1I)
Here, A [0, 1] acts temperature parameter that smoothly interpolates between the prior po (1x))
(A, = 0) and the fully reward-augmented target (A, = 1). By gradually increasing X over denoising steps, the influence of the reward tempered, thereby improving stability during sampling. In
scenarios where the reward is only defined on clean data, following Wu et al. (2023a); Kim et al. (2025), we approximate the optimal intermediate target as
T(x²) x po(x:) exp
(2)(
f(Iz) = M
1 M
Σ r(z) m=1
(m)))
(mm) ~ po(xollz).
(12)
However, categorical sampling from renders f(xx) non-differentiable w.r.t. St. To resolve it, employ the reparameterisation trick with Gumbel-Softmax (Jang al., 2016) enable differentiability (see Appendix C.2 for details). thereby making the approximated proposal applicable as in Equation (8). For completeness, Appendix C.3 provides further discussion of the computation of importance weights in Equation (7) under low-confidence sampling, where the ratio (o( is q(x; 1/#;)
not explicitly tractable. This extension ensures that the proposed SMC algorithm remains suitable for recent state-of-the-a discrete diffusion models for language modelling (Nie et al., 2025) and text-to-image generation (Bai et al., 2024), where low-confidence sampling (Chang et al., 2022) is commonly used.
EXPERIMENTS
To support our theoretical discussion, we first showcase the effectiveness of the proposed methods through synthetic experiment. We then evaluate it across wide range of applications, including language modelling, biological design, and text-to-image generation. Detailed experimental settings and additional results provided in Appendix D.
5

---

Published as a conference paper at ICLR 2026
Pretrained
Target fevro 0.30
Propurs
Propanet eah- 39
SMCa Smard 040
SMC Reaerd 4.39
SMCare EMDh 1.0
Nower s-d
Figure 2: Comparisons on reward- -ilted discreteised MoGs. We consider the reward function as 100 22 where X x= 12( /6 1/2) and y = 12(Y/663 1/2).
r(X, --==xxx
X²
210000-
Pretrained Seward 21.676
larget Reward: -2
Proply
Proporer Reward: o
SMC Reward 001
SMC Reward 003
SMC Reward
Reward: 665
S
7
0
4 4
4
4
4
4
4
4
4
4
4
4
4
Y
4
4
4
4
T
+
1
0
4
Y
4
4
4
4
4
4
4
4
4
Y
Y
y
3 7
2 2
2
4
+
y y 4
4
x
4 4
4
4 4
4
4
A
4
4
4
4
4
4
7f
4
5
B
4 4
4/
19
4
4
4
A
4
4
4
4
4
4/
4
2
3
8
4 4
4
4
4
4
4
4
4
4
7
4
Figure 3: Comparisons on reward-tiled binary MNIST. We train a classier Par Par(y] (z) on the clean data,
and the reward is given by r(z)
= log Par (araaargg [z), where Yurret denotes the target digit.
4.1 SYNTHETIC EXPERIEMNTS
We begin with the empirical evaluation with two synthetic experiments: binary MNIST and two- dimensional discretised mixture of Gaussians (MoG), each dimension comprising 64 categorical states. A discrete diffusion model is first pretrained on the clean data. We evaluate the proposed SMC- based reward-tilting variants in comparison with two non-SMC baselines: Prop corresponding to the first-order approximation in Equation (8), and Propanet which utilises an amortised proposal trained according to the objective in Equation (10), Performance is assessed using the earth mover's distance (EMD), alongside the evaluation of the reward on the generated samples.
The results, shown in Figures and 3. indicate that, compared to the non-SMC baselines, the SMC-based methods achieve superior performance, demonstrating the effectiveness of the proposed approach. Specifically, SMCamot attains the highest rewards and lowest EMD, though at the cost of reduced sample diversity, likely due reward over-optimisation. In contrast, SMCgrad maintains comparable sample quality while preserving high diversity, highlighting the effectiveness of the proposed approximated optimal proposal. Furthermore, Propamot significantly outperforms Prop grad underscoring the benefit of the log-variance minimisation objective. We further demonstrate the reward curves over training in Figures 7a and 7b, which shows that the proposed method can achieve stable reward convergence, confirming the efficiency and robustness in learning optimal proposal.
4.2 LANGUAGE MODELLING
We further evaluate our approach on language modelling, focusing on toxic text generation (Singhal al., 2025), an undesirable behaviour of language models, where the pretrained model MDLM (Sahoo et al., 2024) produces only 0.8% of samples flagged as toxic. To assess sample quality, we use four metries: i) Toxic, based on the same reward model applied during inference (Logacheva et al., 2022); ii) Toxic (Holdout), measured by holdout toxicity classifier trained on a multilingual mixture of datasets (Dementieva al., 2024); iii) generative perplexity with GPT2-XL (Radford et al., 2019); and iv) distinct uni/bi/trigrams (Dist-1/2/3) The first two metrics evaluate alignment with the reward, while the latter two measure semantic quality and diversity.
Following Han et al. (2022), we generate sequences of length 100 with 100 denoising steps condition on the given starting prompts, and report results averaged over 300 independent runs corresponding to 15 prompts with 20 generations per prompt. The results are summarised in Table 1, with an extended version provided in Table 4. We observe that SMC with proposals closer to the optimal achieves better performance the toxicity metrics, reflecting stronger alignment with the reward model. Among the SMC baselines. Propanx yields the best performance, highlighting the effectiveness of the log- variance minimisation objective. To further assess its effect, we plot the training dynamics in Figure 7c, which shows the reward steadily improving as training progresses. Notably, although the learned proposal sacrifices small degree of performance on perplexity and diversity, demonstrate

---

Published as a conference paper at ICLR 2026
Table The results of toxic text generation. We use widely adopted toxicity classifier as the reward (Logacheva et al., 2022), while the pretrained language model is MDLM (Sahoo et al., 2024).
# Particles
Method
Toxic t
Toxic (Holdout) 1
PPL (GPT2-XL) +
Dist- i/st-1/2/3>
Pretrained
0.8%
5.2%
121.1
56/92/96
N
Prop grad
58.0%
58.3%
216.7
58/93/96
Propan Pmoot
63.7%
75.7%
131.9
53/89/94
BoN
6.3%
16.7%
127.4
56/91/96
SMC base
26.7%
40.0%
132.3
57/92/96
= 8
SMC grad
95.0%
86.3%
132.1
57/92/96
SMCamot
100.0%
99.7%
147.6
44/81/91
SMC
SMCr
1981
-222
-224
Number of Particles
Number of Particles
Numbe' Particles
Number of Particies
Number Particies
Figure 4: Results of DNA sequence design. Both the pretrained discrete diffusion model and the reward models are adopted from Wang et al. (2024).
Figure 19 that it consistently generates coherent and semantically meaningful sequences, indicating that alignment improvements need not come at the expense of sample quality.
4.3 BIOLOGY DESIGN
In this experiment, we evaluate our method on DNA sequence design. Specifically, we adopt the pretrained model and the reward model from Wang al. (2024), which trained ~700 DNA sequences. To evaluate the performance, we consider five metrics: i) predicted activity (Pred-Activity); ii) chromatin accessibility classification accuracy (ATAC-Acc); iii) 3-mer Pearson correlation with dataset sequences (3-mer Corr); iv) JASPAR motif frequency correlation (JASPAR Corr); and v) approximate log-likelihood under the pretrained model (App-Log-Lik) For further details on these evaluation metrics, we refer the reader to Wang et al. (2024).
As shown in Figure 4, the performance improves consistently with an increasing number of particles, suggesting that SMC benefits from larger particle set by providing more accurate approximation of the target distribution. Compared SMC SMCamot achieves higher Pred-Activity and ATAC-Acc. while performing slightly worse the other three metrics. This can be attributed to the more mode-seeking behaviour their proposals, which emphasises high-probability regions at the expense of overall diversity. Nonetheless, observe that SMCamot with larger particle sets attains more favourable overall performance, indicating that learnable amortised proposal can effectively leverage the flexibility of SMC to balance quality and diversity. In Appendix D.2.4, we present additional comparisons with baseline methods, further demonstrating the effectiveness of our approach.
4.4
IMAGE GENERATION
In this section, we evaluate our method on image generation. We begin by demonstrating that SMC yields improvements over classifier-free guidance, which can be viewed as a special case of the product target in Equation (6). Subsequently, we present large-scale experiments illustrate the applicability of the proposed methods to text-to-image generation at scale.
Improving CFG with MaskGit (Chang al., 2022). Given pretrained diffusion model pe- classifier-free guidance (CFG) generates samples according to (o (x,, co ( 1-a, where is the CFG coefficient. CFG has been shown to enhance sample quality substantially (Ho & Salimans, 2022). By incorporating the importance weight defined in Equation (6), we can further improve CFG within proposed SMC framework.
7

---

Published as a conference paper at ICLR 2026
BoN
5MCuuse
SMCgrad
SMCanst
0.32
6.2
ImageReward Score
1.4
HPSV2 Score
0.31
Aesthetie Score
1.2
5.8
0.28
0.8
Number of Particles
Number of Particles
Number of Particles
Figure 5: The results of text-to-image generation across different reward models.
Specifically, we perform experiments with MaskGit (Chang et al., 2022) trained on Im- ageNer256 (Deng et al., 2009). To ensure tractable importance weight computation, adopt the ReMDM sampling scheme (Wang al., 2025) instead of the low-confidence sam- pling strategy from (Chang et al., 2022) (see Tables 11 for a comparison). The result is presented in Table 2. It shows that with fewer denoising steps, increasing the number of parti- cles leads to a substantial improvement in sample quality, as measured by FID and Inception Score (IS) with 50, 000 generated images, thereby demonstrating the effectiveness of the proposed SMC method. However, as the number of denoising steps increases, the benefit of using more particles diminishes. This can he attributed to the role of SMC in correcting sampling inaccuracies: with sufficient denoising steps, the sampling process itself becomes accurate enough, leaving limited room for further improvement through additional particles. In addition, we report results with different CFG coefficients in Tables 7 and 8. Interestingly, for larger CFG coefficients, increasing the partiele count tends to decrease FID while increasing IS. This behaviour is expected, since stronger CFG reduces sample diversity. With more accurate sampling under SMC, this reduction in diversity becomes more apparent, leading to lower FID but higher IS.
Table 2: Comparisons of different numbers of par- ticles with CFG= 1.25 on ImageNet255..
FiD
IS f
steps
8
16
32
8
16
32
N
24.64
14,94
12.02
62.8
90.7
107.5
N =
21.08
12.55
10.26
74.0
106.6
126.2
= 4
18.08
10.92
9.29
87.4
123.5
146.5
N
16.26
9.93
8.98
96.4
139.4
159.8
N 14.56
9.59
8.76
107.4
149.4
170.7
Improving Text-to-Image Generation with Meissonic (Bai al., 2024). We evaluate the scalability of the proposed methods on text-to-image generation using Meissonic (Bai et al., 2024) as the base model. Our experiments consider three text-image alignment rewards: Human Preference Score (HPSv2) (Wu et al., 2023b), Aesthetic Score (LAION, 2024), and IewaeReward (Xu et al., 2023). For the prompt distributions, we use photo and painting prompts from the Human Preference Dataset (HPDv2) (Wu et al., 2023b) for HPSv2, the Draw BrawBench prompt set for ImageReward, and curated set of 45 simple animal prompts for Aesthetic Score, follwoing Black et al. (2023).
The results are shown in Figure 5. We observe that performance consistently improves with an increasing number of particles, and SMCama outperforms all other methods, which highlights the benefit of the proposed SMC framework. In Figure 6, we visualise the alignment dynamics for the HPSv2 task, showing that the generated images progressively align more faithfully with the prompts, thereby validating the effectiveness of the proposed log-variance minimisation objective. Furthermore, Figures 7e to 7g present the convergence of the reward during training, and further qualitative examples in Appendix D.2.7 collectively reinforce the validity of our approach.
5 RELATED WORK
Discrete Diffusion Models. Discrete diffusion models (DDMs) were originally introduced in Austin et al. (2021); Sun et al. (2022); Campbell et al. (2022), grounded the framework of continuous- time Markov chains (Norris, 1998). More recently, masked diffuusonn models (MDMs) (Lou et al., 2023; Shi et al., 2024; Sahoo et al., 2024), a special case of DDMs, have shown strong performance in language modelling (Zhang al.. 2025a; Nie et al., 2025). In addition, MDMs have achieved promising results in math reasoning (Zhao et al., 2025), image synthesis (Bai et al., 2024), code planning (Gat al., 2024; Gong al., 2025), and biological sequence generation (Campbell et al., 2024), yielding
8

---

Published as a conference paper at ICLR 2026
A helinet wearing monkey skating.
The image feaatures castle surrounded by a dreamny garden with roses and a cloudy sky in the hackground.
Figure 6: Visualisation of alignment dynamics over the training progress, with images generated every 100 training steps. The generated images become more faithful to the text prompt.
performance comparable to continuous diffusion (Rombach et al., 2022) and autoregressive models (Radford et al., 2019). In contrast to these approaches, which primarily study large-scale pretraining, our work focuses on test-time inference and post-training alignment (Uehara et al., 2025b), where access to training data is not available.
Test-time Alignment of Discrete Difflusion Models. Existing alignment methods mainly fall into two categories: classifier guidance (Dhariwal & Nichol, 2021) and RL-based fine-tuning (Black et al., 2023). Although the score is ill-defined in discrete distributions, several works (Vignac al., 2022; Nisonoff et al., 2024; Schiff et al., 2024) employ a first-order approximation to the target distribution, which resembles the insight underlying our approximated proposal Prop grag in Equation (8). Alternatively, Rout al. (2025) perform guidance on the embedding space, mitigating the issue of ill-defined gradients. Chen et al. (2025b) introduce reward-free guidance, analogous to classifier-free guidance (Ho & Salimans, 2022) but designed for masked diffusion models. Moreover, Tang et al. (2025) propose tree search guided finetuning, which is related to the searching- based scaling methods on continuous diffusion (Ma et al., 2025; Zhang et al., 2025b; Jain et al., 2025; Ramesh & Mardani, 2025). Beyond guidance approaches, sampling based techniques have also demonstrated promising performance, including value- based sampling (Li et al., 2024), importance sampling (Guo al., 2024), and iterative refinement strategies (Uehara al., 2025a). While training- free and relatively efficient to deploy, these methods often face challenges in scalability and robustness. More recently, RL-based fine-tuning methods (Zekri & Boullé, 2025: Zhao et al., 2025; Gong et al., 2025) have gained significant traction, fueled by the remarkable success of Group Relative Policy Optimisation (GRPO) (Shao et al., 2024) in large language models (Guo et al., 2025). In parallel, steering-basee (Rector-Brood et al., 2024) approaches leveraging GFlowNets (Bengio et al., 2023) and direct backpropagation methods (Wang et al., 2024) have also demonstrated strong potential for test-time alignment. Distinct from these directions, our amortised proposal Propamet introduces an alternative perspective for fine-tuning pretrained discrete diffusion models: it minimises the log-variance of importance weights, criterion that has been rarely investigated in previous work.
Sequential Monte Carlo for Generative Modelling. SMC has emerged as a versatile framework for probabilistic modelling, providing effective tools for sampling and inference across wide range of applications, including particle filtering (Johansen, 2009). Bayesian experimental design (Ryan al., 2016), and probabilistic planning (Piché al., 2018). Most recently, SMC has been combined with diffusion models (Chen et al., 2024; He et al., 2025; Skreta et al., 2025; Wu et al., 2025; Ou al., 2026). transforming it into a powerful neural sampler capable of drawing from complex Boltzmann distributions. These developments have also extended SMC's reach to discrete domains, as demonstrated by Holderrieth et al. (2025); Lee et al. (2025). Beyond classical sampling tasks, SMC has further expanded to the improvement of generative models at test time. A seminal step in this direction was taken by Zhao et al. (2024), who introduced SMC as a principled probabilistic inference framework for addressing capability and safety challenges in large language models (L.MMs). Subsequent works (Feng et al., 2024; Puri et al., 2025) successfully applied this idea to enhance mathematical reasoning in LLMs, while others explored its use in hevard-guided adaptation of pretrained diffusion models (Trippe et al., 2022; Wu et al., 2023a; Cardoso et al., 2023; Dou & Song, 2024; Kim et al., 2025; Yoon et al., 2025; Chen et al., 2025a; Ren et al., 2025). Our work is most closely related to Singhal et al. (2025); Dang et al. (2025); Hasan et al. (2025), who employ SMC for test-time alignment of discrete diffusion models. However, their approaches treat pretrained diffusion

---

Published as a conference paper at ICLR 2026
models fixed proposal distributions. By contrast, take closer look the role of proposal choice, systematically investigating its impact and providing empirical evidence for a key insight: proposals that better approximate the optimal, which minimises the variance of importance weights, consistently lead to better performance.
CONCLUSION
In this paper, we introduced a Sequential Monte Carlo (SMC) framework tailored for discrete diffusion models. By exploiting tractable importance weights, established SMC powerful and principled recipe for test-time scaling. A central insight of our work that the proposal distribution crucial for unlocking the full potential SMC. Building on this observation, we developed two approximately optimal proposals: first-order approximation and learnable amortised proposal trained approximate optimal proposal by minimising the log variaace of importance weights. Extensive experiments across diverse domains demonstrated the effectiveness and scalability of our approaches. We hope this work inspires future studies on more efficient test-time scaling and post-training alignment strategies for discrete diffusion models.
7 ACKNOWLEDGEMENTS
ZO is supported by the Lee Family Scholarship. We would like to thank the anonymous reviewers for their constructive and valuable suggestions. ZO also thanks Ruixiang Zhang for insightful discussions and for his support with the experimental setup.
REFERENCES
Adam, K. D. B. J. et al. A method for stochastic optimization. arXiv preprint (6), 2014.
1412
Austin, J., Johnson, D. D., Ho, J., Tarlow, D., and Van Den Berg, R. Structured denoising diffusion models in discrete state-spaces, Advances in neural information processing systems, 34:17981- 17993, 2021.
Bai, J., Ye, T., Chow, W., Song, E., Li, X., Dong, Z., Zhu, I.,, and Yan, S. Meissonic: Revitalizing masked generative transformers for efficient high-resolution text-to-image synthesis. arXiv preprint arXiv:2410.085 110.6661,, 2024.
Bengio, Y., Lahlou, S., Deleu, T., Hu, J., Tiwari, M.. and Bengio, E. Gflownet foundations. Journal of Machine Learning Research, 24(210):1555, 2023.
Black, K., Janner, M., Du, Y., Kostrikov, I., and Levine, Training diffusion models with reinforce- ment learning. arXiv preprint 330.::33333333,,0000 2023.
Campbell, A., Benton, J., De Bortoli, V., Rainforth, T., Deligiannidis, G., and Doucet, A. A continuous time framework for discrete denoising models. Advances in Neural Information Processing Systems, 35:28266 28279, 2022.
Campbell, A., Yim, J., Barzilay, R., Rainforth, T,, and Jaakkola, T. Generative flows discrete state-spaces: Enabling multimodal flows with applications to protein co-design. arXiv preprint 2022:22002.04999, 2024.
Cardoso, G., Idrissi. Y. J. E.. Corff, L., and Moulines, E. Monte carlo guided diffusion for bayesian linear inverse problems. arXiv preprint 230.:2308.0788, 2023.
Chang, H., Zhang, H., Jiang, L., Liu, C., and Freeman, W. T. Maskgit: Masked generative im- age transformer. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pp. 11315-11322 2022.
Chen, H., Ren, Y., Min. M. R., Ying. L., and Izzo, Z. Solving inverse problems via diffusion- priors: An approximation free ensemble sampling approach. arXiv preprint 2506v:2506.033 2025a.
10

---

Published as a conference paper at ICLR 2026
Chen, J., Richter, L., Berner, J., Blessing, D., Neumann, G., and Anandkumar, A. Sequential controlled langevin diffusions. arXiv preprint arXiv:2412.07081, 2024.
Chen, T., Xu, M., Leskovec, J., and Ermon, S. Rfg: Test-time scaling for diffusion large language model reasoning with reward-free guidance. arXiv preprint arXiv:2509.2500 2025b.
Chu, W., Wu, Z., Chen, Y.. Song, Y.. and Yue, Y. Split gibbs discrete diffusion posterior sampling.
arXiv preprint arXiv:2503.0116/ 2025.
Dang, M., Han, J., Xu, M., Xu, K., Srivastava, A., and Ermon, S. Inference-time scaling of diffusion language models with partiele gibbs sampling, arXiv preprint arXiv:2507.08390 2025.
Dayan, P., Hinton, G. E., Neal, R. M., and Zemel, R. S. The helmholtz machine. Neural computation, 7(5):889-904, 1995.
Del Moral, P., Doucet, A., and Jasra, A. Sequential monte carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(3):111-46, 2006.
Dementieva, D., Moskovskiy, D., Babakov, N., Ayele, A. A., Rizwan, N., Schneider, E., Wang, X., Yimam, S. M., Ustalov, D., Stakovskii, E., et al. Overview the multilingual text detoxification task at pan 2024. Working Notes of CLEF, 2024.
Deng, J., Dong, W., Socher, R., Li, L.-J,, Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Denker, A., Padhy, S., Vargas, F., and Hertrich, J. Iterative importance fine-tuning of diffiusion models. arXiv preprint Stt::2502.044663 2025.
Dhariwal, and Nichol, Diffusion models beat gans on image synthesis. Advances neural information processing systems, 34:8780-87 8794, 2021.
Dou, Z. and Song, Y. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. The Twelfth International Conference on Leurning Representations, 2024.
Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
Fan, Y., Watkins, O., Du, Y., Liu, H., Ryu, M., Boutilier, C., Abbeel, P., Ghavamzadeh, M., Lee, K., and Lee, K. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 36:79858-79988 2023.
Feng, S., Kong, X., Ma. S., Zhang, A., Yin, D.,, Wang, C., Pang, R., and Yang, Step-by-step reasoning for math problems via twisted sequential monte carlo. arXiv preprint arXiv:4410.01920. 2024.
Gat, I., Remez, T., Shaul, N., Kreuk, F., Chen, R. T., Synnaeve, G., Adi, Y., and Lipman, Y. Discrete flow matching. Advances in Neural Information Processing Systems, 17:133345-133355 2024.
Gong. S.. Zhang, R., Zheng, H., Gu, J., Jaitly, N., Kong, L., and Zhang, Y. Diffucoder: Understanding and improving masked diffusion models for code generation. arXiv preprint arXiv:2506.20639 2025.
Gosai, S. J., Castro, R. I., Fuentes, N., Butts, J. C., Kales, S., Noche, R. R., Mouri, K., Sabeti, P. C., Reilly, S. K., and Tewhey, R. Machine- guineed design of synthetic cell type-specitt cis-regulatory elements. bioRxiv, 2023.
Grathwohl, W., Swersky, K., Hashemi, M., Duvenaud, D... and Maddison, C. Oops took a gradient: Scalable sampling for discrete distributions. International Conference on Machine Learning, pp. 3831-3841. PMLR, 2021.
Gruver, N., Stanton, S., Frey, N., Rudner, T. G., Hotzel, I., Lafrance-Vanasse, J.. Rajpal, A., Cho, K., and Wilson, A. G. Protein design with guided discrete diffusion. Advances in neural information processing systems, 36: 12411--225171 2023.
11

---

Published as a conference paper at ICLR 2026
Guo, D., Yang, D.., Zhang, H., Song. J., Zhang, R., Xu, R., Zhu, Q.. Ma, S., Wang. P., Bi, X., et al. Deepseek-rl: Incentivizing reasoning capability llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025.
Guo, W., Zhu, Y., Tao, M., and Chen, Y. Plug-and controllable generation for discrete masked models. arXiv preprint arXiv:2410.022 14.0224,, 2024.
Han, X., Kumar, S., and Tsvetkov, Y. Ssd-Im: aemi-autoregressvve simplex-based diffusion language model for text generation and modular control. arXiv preprini arXiv:2210. 17432, 2022.
Hasan, M., Skreta, M., Aspuru-Guzik, A., Bengio, Y., and Neklyudov, K. Discrete feynman-kac correctors. In 2nd Al for Math Workshop@ ICML 2025, 2025.
He, J., Hernandez-Lobato, J. M., Du, Y., and Vargas, Rne: a plug-and-play framework for diffusion density estimation and inference-time control. arXiv preprint arXiv:2506.0566 05668, 2025.
Ho, J. and Salimans, T. Classifier-free diffusion guidance. arXiv preprint arXiv:2207. 12598,, 2022.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Holderrieth, P., Albergo, M. S., and Jaakkola, T. Leaps: A discrete neural sampler locally equivariant networks. arXiv preprint 1.X.:22002.10443 2025.
Hoogeboom, E., Satorras, V. G., Vignac, C., and Welling, M. Equivariant diffusion for molecule generation in 3d. In International conference on machine learming, pp. 8867-8887. PMLR, 2022.
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2)3, 2022.
Jain, V., Sareen, K., Pedramfar, M., and Ravanbakhsh, S. Diffusion tree sampling: Scalable inference- time alignment of diffusion models. arXiv preprint arXiv:2506.2070, 2025.
Jang. E., Gu, S., and Poole, B. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv: /6/ 1.01 144.01114.. 2016.
Johansen, A. A tutorial on particle filtering and smoothing: Fifteen years later. 2009.
Kim, S., Kim, M., and Park, D. Alignment without over-optimization: Training-fres solution for diffusion models. arXiv preprint arXiv:2501.05800 2025.
Kingma, D. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 2013.
LAION. Laion aesthetic predictor. http:::/laaao.aallgggaaoooeeeees 2024. Accessed: 2024-09-27.
Lee, C. K., Jeha P., Frellsen, J., Lio, P., Albergo, M. S., and Vargas, F. Debiasing guidance for discrete diffusion with sequential monte carlo. arXiv preprint 2XXi:25502.0607, 2025.
Li, X., Zhao, Y., Wang, C., Scalia, G., Eraslan, G., Nair, S., Biancalani, T., Ji, S., Regev, A., Levine, et al. Derivative-free guidance in continuous and discrete diffusion models with soft value-based decoding. arXiv preprint arXiv:2408.08252, 2024.
Liu, 7., Xiao, T. 7., Liu, W., Bengio, Y., and Zhang, D. Efficient diversity-preserving diffusion alignment via gradient-informed gflownets. arXiv preprint arXiv:2412.07775, 2024.
Logacheva, V., Dementieva, D., Ustyantsev, S., Moskovskiy, D., Dale, D., Krotova, L. Semenov, N., and Panchenko, Paradetox: Detoxification with parallel data. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 6804-6818, 2022.
Loshchilov, and Hutter, F. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05000 2017.
12

---

Published as a conference paper at ICLR 2026
Lou, A., Meng, C.,, and Ermon. S. Discrete diffusion modeling by estimating the ratios of the data distribution. arXiv preprint arXiv:2310. 168344 2023.
Ma, N., Tong, S., Jía, H., Hu, H., Su, Y.-C., Zhang, M., Yang, X., Li, Y., Jaakkola, T., Jia, X., et al. Scaling inference time compute for diffusion models. Proceedings of the Computer Vision and Pattem Recognition Conference, pp. 2523-2534 2025.
Matthews, A., Arbel, M., Rezende, D. J., and Doucet, A. Continual repeated annealed flow transport monte carlo. In International Conference Machine Learning, pp. 15196-15219, PMLR, 2022.
Nie, S., Zhu, F., You, Z., Zhang, X., Ou, J., Hu, J., Zhou, Lin, Y., Wen, J.-R., and Li, C. Large language diffusion models. arXiv preprint trXiv:2502.0999) 2025.
Nisonoff, H., Xiong, Allenspach, S., and Listgarten, J. Unlocking guidance for discrete state-space diffusion and flow models. arXiv preprint arXiv:2406.01577 2024.
Norris, J. R. Markov chains. Number 2. Cambridge university press, 1998.
Oksendal, B. Stochastic differential equations: an introduction with applications. Springer Science & Business Media, 2013.
Ou, Z., Zhang, R., and Li, Discrete neural flow samplers with locally equivariant transformer. arXiv preprint arXiv:2505. 2025.
Ou, Z., Si, J., Zhu, J., Bohdal, O., Ozay, M., Ceritli, T., and Li, Y. Diffusion alignment beyond kl: Variance minimisation as effective policy optimiser. arXiv preprint arXiv:2602. 2026.
Piché, A., Thomas, V., Ibrahim, C., Bengio, and Pal, C. Probabilistic planning with sequential monte carlo methods. In International Conference on Learning Representations, 2018.
Puri, I., Sudalairaj. S., Xu, G., Xu, K., and Srivastava, A. Rollout roulette: A probabilistic inference approach to inference-time scaling of Ilms using particle-based monte carlo methods. arXiv preprint XXX1:22002.06188 2025.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAl blog, 188):9, 2019.
Ramesh, and Mardani, M. Test-time scaling of diffusion models via noise trajectory search. arXiv
preprint ar arXiv:2506.031/ 2025.
Rector-Ber Brooks, J., Hasan, M., Peng, Z., Quinn, Z., Liu, C., Mittal, S., Dziri, N., Bronstein, M., Bengio, Y., Chatterjee, P., et al. Steering masked discrete diffusion models via discrete denoising posterior prediction. arXiv preprint arXhv:2410.08134 2024.
Rector-Brooks,
Ren, Y., Gao, W., Ying, I.,, Rotskoff, G. M., and Han, J. Driftlite: Lightweight drift control for inference-time scaling diffusion models. arXiv preprint (rXiv:2509.21655 2025.
Richter, L. and Berner, J. Improved sampling via learned diffusions. arXiv preprint arXiv:2307.01198, 2023.
Richter, L., Boustati, A., Nüsken, N., Ruiz, F., and Akyildiz, O. D. Vargrad: a low-variance gradient estimator for variational inference. Advances in Neural Information Processing Systems, 33: 13481-13492, 2020.
Robert, C. P., Casella, G., and Casella, G. Monte Carlo statistical methods, volume 2. Springer, 1999.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pp. 10684-10695, 2022.
Rout, L., Lugmayr, A., Jafarian, Y., Varadharajan, S., Caramanis, C.. Shakkottai, S., and Kemelmacher- wwrelmmacher-Shizzmman Shwizermazznznnn Test-time anchoring for discrete diffusion posterior sampling. arXiv preprint arX arXiv:25/0.0029/ 2025.
13

---

Published as a conference paper at ICLR 2026
Ryan, E. G., Drovandi, C. C., McGree, J. M., and Pettitt, A. N. A review of modern computational algorithms for bayesian optimal design. International Statistical Review, 84(1):128 154, 2016.
Sahoo, S., Arriola, M., Schiff, Y., Gokaslan, A., Marroquin, E., Chiu, J., Rush, A., and Kuleshov, V. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37:130136-130131 2024.
Schiff, Y., Sahoo, S. S., Phung, H.. Wang, G., Boshar, S., Dalla-torre, H., de Almeida, B. P., Rush, A., Pierrot, T., and Kuleshov, V. Simple guidance mechanisms for discrete diffusion models. arXiv preprint arXiv:2412. 10193, 2024.
Shao, Z.,, Wang, P., Zhu, Q Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint ar arXiv:2402.0333
2024.
Shi, J., Han, K., Wang, Z., Doucet, A., and Titsias, M. Simplified and generalized masked diffusion for discrete data. Advances in neural information processing systems, 37:103131-103116 2024.
Singhal, R., Horvitz, Z.,, Teehan, R., Ren, M., Yu, Z., McKeown, K., and Ranganath, R. A gen- eral framework for inference-time scaling and steering of diffusion models. arXiv preprint arXiv:2501.06845 2025.
Skreta, M., Akhound-Sadeen T., Ohanesian, V.. Bondesan, R., Aspuru-Guzik A., Doucet, A., Brekelmans, R., Tong, A., and Neklyudov, K. Feynman-kac correctors in diffusion: Annealing, guidance, and product experts, arXiv preprint arXiv:2503. 222,02299000,899,, 2025.
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp. 2256-2265. pmlr, 2015.
Song. Y., Sohl-Dickstein, J., Kingma, P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. arXiv preprint 2rXiv:2011.13456 2020.
Sun, H., Yu, L., Dai, B., Schuurmans, D., and Dai, H. Score- continuous-time discrete diffusion models. arXiv preprint arXiv:2211.16755 2022.
Tang, S., Zhu, Y., Tao, M., and Chatterjee, P. Tr2-d2: Tree search guided trajectory- fine-tuning for discrete diffusion. arXiv preprint arXiv:2509.25177 2025.
Trippe, B. L., Yim, J., Tischer, D., Baker, D., Broderick, T., Barzilay, R., and Jaakkola, T. Diffusion probabilistic modeling of protein backbones in for the motif-scaffolding problem. arXiv preprint vrXiv:2206.041199 2022.
Uehara, M., Zhao, Y., Black, K., Hajiramezanali, E., Scalia, G., Diamant, N. L., Tseng, A. M., Levine, S., and Biancalani, T. Feedback efficient online fine-tuning of diffusion models. arXiv preprint arXiv:2402. 16359, 2024.
Uehara, M., Su, X., Zhao, X., Regev, A., Ji, S., Levine, S., and Biancalani, T. Reward-guides iterative refinement in diffusion models at test-time with applications to protein and dna design. 2025a.
arXiv preprint ar arXiv:2502.1499
Uehara, M., Zhao, Y., Wang. C., Li, X., Regev, A., Levine, S., and Biancalani, Reward-guided controlled generation for anference-time alignment in diffusion models: Tutorial and review. arXiv preprint ar 250,09985, 2025b.
Verkuil, R., Kabeli. O.. Du, Y., Wicky, B. I., Milles, L. F. Dauparas, J., Baker, D., Ovchinnikov, S., Sercu, T., and Rives, A. Language models generalize beyond natural proteins. BioRxiv, pp. 2022-12, 2022.
Vignac, C., Krawczuk, I., Siraudin, A., Wang, B., Cevher, V., and Frossard, Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv:2209. 2at.222299.4444... 2022.
14

---

Published as a conference paper at ICLR 2026
Wang, C., Uchara, M., He, Y., Wang. A., Biancalani, T., Lal, A.. Jaakkola, T., Levine, S., Wang, H., and Regev, A. Fine-tuning discrete diffusion models via reward optimization with applications to dna and protein design. arXiv preprint arXiv:2410. 1e..22440..3344... 2024.
Wang, G., Schiff, Y., Sahoo, S. S., and Kuleshov, V. Remasking discrete diffusion models with inference-time scaling. arXiv preprint arXiv:2503.00330 2025.
Watson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L., Yim, J., Eisenach, H. E., Ahern, W., Borst, A. J., Ragotte, R. J., Milles, L. F., al. De novo design of protein structure and function with 2023.
rfdiffusion. Nature, (10079766:108911111
Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229-2555 1992.
Wu, L.. Trippe, B., Naesseth, C., Blei, D., and Cunningham, J. P. Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems, 36:31372-31403, 2023a.
Wu, L., Han, Y., Naesseth, C. A., and Cunningham, J. P. Reverse diffusion sequential monte carlo samplers. arXiv preprint vrXiv:2508.05926 2025.
Wu, X., Hao, Y., Sun, K., Chen, Y., Zhu, F., Zhao, R., and Li, H. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint wwXiv:2306.09311 2023b.
Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., and Dong, Y. Imagereward: Learning and evaluating human preferences for sext-to-image generation. Advances in Neural Information Processing Systems, 36:15903-15935, 2023.
Yoon, T., Min, Y., Yeo, K., and Sung, M. psi-sampler: Initial particle sampling for sme-based inference-time reward alignment in score models. arXiv preprint arXiv:2506. TTt.:25006.132200 2025.
Zekri, O. and Boullé, N. Fine-tuning discrete diffusion models with policy gradient methods. arXiv preprint arXiv:2502.03884 2025.
Zhang, R., Liu, X., and Liu. Q. A langevin like sampler for discrete distributions. In International Conference on Machine Learning, pp. 26375-26396. PMLR, 2022.
Zhang, R., Zhai, S., Zhang. Y., Thornton, J., Ou, Z., Susskind, J., and Jaitly, N. Target concrete score matching: holistic framework for discrete diffusion. arXiv preprint arXiv:2504. 16431, 2025a.
Zhang, X., Lin, H., Ye, H., Zou, J., Ma, J., Liang, Y., and Du Y. Inference time scaling of diffusion models through classical search. arXiv preprint arXiv:2505.23614, 2025b.
Zhao, S., Brekelmans, R., Makhzani, A., and Grosse, R. Probabilistic inference in language models via twisted sequential monte carlo. arXiv preprint arXiv:2404.155: 2024.
Zhao, S., Gupta, D., Zheng, Q., and Grover, A. d1: Scaling reasoning in diffusion large language models via reinforcement learning, arXiv preprint arXiv:2504.12216. 2025.
15

---

Published as a conference paper at ICLR 2026
Appendix for "Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Design"
CONTENTS
Abstract Proof and Derivations
16
A.1 Brief Recap of SMC
16
A.2 Proof of Locally Optimal Proposal
17
A.3 Proof of Log-Variance Variance Minimisation Objective
18
Extending Discrete-time SMC to Continuous-time SMC
19
B.1 Background of CTMC
20
B.2 Continuous- Time Formulation of SMC
20
Implementation Details of Computing Importance Weight
23
C.I Sampling Schemes in Masked Diffusion Models
23
C.2 First-Order Approximation with Gumbel-Softmax Relaxation
24
C,3 Computing Importance Weight with Now-Confidence Sampling
25
Experimental Setting and Additional Results
25
D.1 Details of Experimental Setting
25
D.2 Additional Experimental Results
27
D.3 Statement of The Use of Large Language Models
31
A ABSTRACT PROOF AND DERIVATIONS
A.1 A BRIEF RECAP SMC
In this section, provide a brief overview of Sequential Monte Carlo (SMC). For target distri- bution (.x)), we consider the problem of estimating the expectation of test function 6, namely Ex(,,,) (xxz). When 8(-) is taken be the Dirac delta function, estimating this expectation reduces to constructing an empirical approximation of the distribution n (xx))
To estimate the expectation, importance sampling introduces a proposal distribution q, which is easy to sample from, and proposes estimator as follows
T(ICT) ((xT.T)
N
(2 IzT q(ETT)
E [x [ By(sr))
5(xz)
2
wi (1) ((x)))), where wi
(5 ETT
1=1
The key ingredients of SMC are the target distribution (xETT) and the proposal distribution ((*.r)) Here we consider the target distribution Markovian model associated with sequence of forward transition kernels 7: (FET) (xx)) [f-1 (x++1/2))) and the proposal distribution as
16

---

Published as a conference paper at ICLR 2026
((ur) ((x)) II g(1r(22+1) Substituting these into the importance weights gives
(x,) II - (1
wy (xT) II ((
((2)))(2441 (x)) ((244) (10411
(2241))(2)) (x+1)) (()) Ho (x
((44))
((22+1)4(z)))))) (((1)) w1+1
(13)
This recursive structure allows importance weights to be computed incrementally. SMC augments this with a resampling step to mitigate weight degeneracy. For N particles, SMC proceeds as follows:
1. Initialise: oi
T (xTT) Wi =
2. For Tert=T...... 1:
2 ((0)-11x(") (
(a) Propagate:
Mice (
(b) Update weights:
(( (osts)
then reset all weights (9) 1.
(c) Resample particles according to
Σy
The resulting set of particles (a))) wo Ji=1
NN
forms an empirical approximation of the target (I))).
A.2 PROOF OF LOCALLY OPTIMAL PROPOSAL
Proposition 2 (Locally Optimal Proposal). Given the incremental importance weight as in Equa-
tion (5) w (x;-1,);;; _(2)-1) (1(())11_1)) =(=x)g((t-1(=1)) the proposal distribution that minimises the variance of Wr-1, often referred to as the locally optimal proposal, is ((x+-1/2x) x 7((+=1)77(Z(((e-1)
Proof. We first present an intuitive argument to aid understanding, and subsequently provide the formal proof.
Intuitive argument. The optimal proposal distribution is characterised the one that minimises the variance the importance weights. In the degenerate case zero variance, the importance weight must be constant: n(a, 1)((,, [aa)) = c, for some constant > 0. Rearranging yields
7(x--1)9(m--11))
q (x,1_110))) (x(--1) (FF=-1 (0 T2+)
(14)
where c (((s) Σx., π(x (1 is the normalising constant.
Formal proof. The optimal proposal can be obtained by minimising the variance of the incremental x(x 1)((1 m2) importance weight 2( क(. 100(m; 1
9 = argmin Eyy (w] F. (1 x2)] + Σ a( -1)
Ze-1
argmin Ey [w( 2 Ey [2, + a(x))
Σ a (21) -
argmin Σ w() 2(( (xx) + alzz) tc,
F()
17

---

Published as a conference paper at ICLR 2026
where denotes a constant w.r.t. q and we introduce Lagrange multiplier a(x1) 0 for the constraint Ex._, (( = 1. Using the calculation variation, where the functional F should satisfy the Euler-Lagrange equation af OF O, we have 0g da By'
OF ( ((00-1)7(0-1112)))
a(xx) =0 = q² (2)-1 *( ((51-))((11110)) [x)) aa(x))
0q
The term (
is normalisation constant that does not depend on the 1. We can find its value by enforeing the constraint Em.., q² ( 1. This shows that the optimal proposal is 4² (((+-1/24)) & (x (1((111
Remark. Given Proposition 2, we can derive the form of the locally optimal proposal under different settings. Specifically, using the importance weight defined in Equations (6) and (7), let the forward kernel ((, (r-11) be specified as
product: i pi (x+ 2 (1r (1a-1) reward-tilting:- 7 x p(x)-1/()))
The corresponding importance weights are then
product: P8, (3/z- 1 ( (16-1)
exp(r(x;-1)) po (o(
reward-tilting: exp(r(xx)) (00--1 (2))
7 (
where 1(x;-1)= = = pf (ag | )pg (x, 1z) is the normalising constant. By Proposition 2, the corresponding locally optimal proposals are
Po, (3+-1 1 Ire lo, ( 1234))
product: q
reward-tilting: q x exp(r(r=-1))o(
7 (xx-1)
The normalising constant Z is tractable, since p(=)(1 is simple forward noising distribution (induced by Equation (1)) that does not involve network evaluation. In contrast, for the reward-tilting, dependence on the reward function r, which defined via a neural network, renders the optimal proposal intractable in general. This necessitates the development of approximation techniques.
A.3 PROOF OF
MINIMISATION OBJECTIVE
The locally optimal proposal g" < )pa (1((xe)) that achieves the minimum variance of the important weight V [[.; 1)) is unique.
Corollary 1.
7(2,)9(((-11))))
Proof. Recall that the variance is given by V, = [g[]]] (B)[w])² As shown in the proof of Proposition 2, the term E./li is constant w.r.t the choice of q. Therefore, minimising the variance is equivalent to minimise the expected square of the weights, [w2) which we will call F(g):
T (T( TT:)
((
F(g) Ey[w=1 = Σ q(F(-11211)
(15)
To simplify the notation, let qi = g 9((1-1 = i|x;)) and C; 1 The optimal
proposal is q" Cz/Z, where 7 = E, Cj. We then can rewrite the objective function F(g)
Σ C²
(Z
(
F(q)
Z >
(16)
q
qi
qi
Evaluating the function the optimum, q², we have
F(g") Σ 07
(Zq,)
Σ
2²
Σ
=Z2
(17)
18

---

Published as a conference paper at ICLR 2026
To prove the uniqueness of the locally optimal proposal q', the key insight is to relate the expression for F()) to the Chi-squared divergence, which is defined as
( q:)
(q))²
(gg01l1)= =
Σ
(18)
9i
qi
Rearranging this, we see that Σ,
10,1 = x² (2 + 1. Now we can express F(q) as
( ) z² (x2(1" [1)) +11 (2x2(0"111) =
F(g) - 2
(19)
4
Since F(q")= Z², we finally arrive at
F()) (q') x2 1g).
(20)
Since the xiddveegee is non-negative and x²(g2=2(221))) (14) = and only if g = q", we see that the equality F(y) F( holds only when x² (14) 0, which requires that q be identical to q" For
any other distribution & q- the divergence is strictly positive, meaning F(g) > F(q"). Therefore, qº is the unique distribution that minimises the variance of the importance weights.
O
Proposition 3. For any reference distribution qret, we have Llog-var (6) S TT b). Moreover the minimiser of unique and attains its optimum when qs x (p
Proof. To prove the result, we first recall the basic identitie R.ylu) argmin, F c)2 and V [w] Eg[(w E.(uu))3). Applying these, we obtain
log exp(r(2(-1)) Po [T) exp(r(x:)) go (02-1 (To)
Liog-var (Ø)
V yoot
log exp(r(z(-1)) po ( [1)) Fb
min Egge F,ER
exp(r(xx)) qs (x1 x2))
27
T log exp(r(x)-1)) po p [xx))
1
= T2 min Egns FiER
Fb
exp(r(xx)) (0(x0-1120 (xx) T
< T2 min Egga FIER
Σ log exp(r(x;)) g (x2-111
exp(r(0)-1)) po po(
Fe
log exp(r(xx-1)) po (0 (zt)) exp(r(x,)) g (0
2
T min E E
FIER
This motivates defining the loss function as
|1og exp(r(xx-1)) po( (oleii))oooolllll Fa((() 12
C(O, vb) Be,z,
(21)
exp(r(xx)) 2o (0(x1-1
Therefore, we have the inequality
var 5 ST²
(22)
Consequently, if (0*, 15t) = argminn a, C(O, (), then
((0" =0
Siog-var"")) 0
Viw] = 0.
(23)
Finally, by Corollary 1, the minimiser o² is unique, and the corresponding proposal takes the form aor (fe- (2tx) x
B EXTENDING DISCRETE-TIME SMC TO CONTINUOUS SMC
In this section, we extend our SMC algorithm from discrete time to continuous time. We begin by introducing the key preliminary: the continuous-tima Markov chain (CTMC) (Norris, 1998). We then establish connections to previous work Holderrieth et al. (2025); Lee et al. (2025), which develops continuous-time SMC methods for discrete diffusion models.
19

---

Published as a conference paper at ICLR 2026
B.1 BACKGROUND OF CTMC
A continuous-time Markov chain (Noris,, 1998) at time is characterised by a time-dependent rate matrix R X' X R, which captures the instantaneous rate of change of the transition probabilities. Specifically, the rate matrix Rg is defined as
Re = lim P++a(e(()))) 6y 8-=x 0,
1,
y = I
(24)
At-0
At
xx
By definition, the rate matrix equivalently yields the transition probability
Pe+ar((y)))) = by=z + (n(a,,)) O(
(25)
To ensure Pr-A valid distribution, the rate matrix RS must satisfy the following constraints:
Re(x,y) 0, A Re(x,x)= = Σ Re(x, v).
(26)
The transition probability Pr|s. for > satisfies the Kolmogrove equations (Oksendal, 2013):
Kolmogorov forward equation: Oxp))(((()
Σ Pala (1)
(27)
Kolmogorov backward equation: O.pr)(I(T)) = Σ Ra, v)p()(<)l))
(28)
The forward equation also induces a PDE for the marginal distribution p.
(xp( = Σ Rel,)t
(29)
Using the backward equation, one can derive Kolmogorov backward equation for expectations, also known as Dynkin's 's formula (Oksendal, 2013). In particular, we have the following lemma.
Lemma 1. Let h be a test function of interest and define uz(z) ppe(s [x()]) Then un satisfies the partial differential equation (z) = E, R2 (I, v)u(yy)
Proof.
0u
h22)0eP1((212))
Σ h(2) - Σ Ra ())1((()))))))
Σ Re(E.V) Σ v))((2
Σ Re(x,y)uel)
B.2 CONTINUOUS -TIME FORMULATION OF SMC
Proposition (SMC for Continuous-Time Discrete Diffusion). Let R2 be the matrix generating the forward transition kermel yx, (x)) and RS be its counterpart associated with the backward proposal kernel q(2;- (r)), where At + 0 is the infinitesimal time increment. Then, the importance weight at time L. is given by wt = J O2 log T (xx.) Ey. R2 R2 ys) w(R.) ry is chosen such that the rate matrices satisfy detailed balance (e( 36) (x.x) Ra x: (()))
alyz) ds, the forward kernel
20

---

Published as a conference paper at ICLR 2026
Proof. Recall Equation (5), where the importance weight is given by
(((-11 ((1)
Wi Wr-1 (2)-1
We (I..T)
(30)
(2x) ((
We now extend it to the continuous-time setting. Let R& and R. denote the rate matrices corresponding the proposal g and the forward noising transition 7, respectively. Consider discretisation with 7 denoising steps, indexed by time points to ... t ... < tr = 1, where each interval satisfies f--1
The discrete-time importance weight at step time is then computed as
(xx)
log w. log
log q ( (tt))
(31)
x
The second term in the RHS can be expanded as
Σ log
( [T, ,)
Σ log o Rr, (ot_1) 2e, | -log
1
log 8m 0 Re, (It, Deam
g( [(z)
Σ log
1+ (1rt ) log 1+ 1o T)
Σ Re, (dre) F2,) Pas (2its (izza)
1,4
Σ Res (2 Teil (2 It. 64:=t
1 + +
Re, ( Re Xti-1) (AG
Σ
Taking the limit T too, the importance weight becomes:
π (Txs) π (x)) +
log w. =log
(ee, dt + R
E'og log Rg (x1, - Re (x
(32)
$St,
By the fundamental theorem of calculus for piecewise differentiable functions, we have:
T(Tx)) log
= J. -0s log T (xx) +
Σ
log (x:) log π ( )
(33)
T(01)
the noising process chosen such that the rate matrix satisfies (1( (V),) then the importance weight simplifies accordingly
f 0 log x(x)) + Re(I,, (xe,, dt
log W.
f
a log (xx) + Σ Re( ve) T(yt) T(x.)
dt,
which completes the proof.
Remark. Proposition recovers the importance weights proposed in the SMC methods of Holderrieth et al. (2025); Ou et al. (2025). In those works, the intermediate target distribution is defined as geometric interpolation between the base and target distributions: T(x;) Pinse ( (Iz).. The proposal rate matrix R₁ is then trained to satisfy the Kolmogorov forward equation. In contrast, we consider different scenario: the pretrained model is available, but the intermediate target (xx)) cannot be computed explicitly. Moreover, the importance weight in Equation (32) can also be derived via the Rikon-Ni derivative (Campbell et al., 2024, Appendix C.I), (Denker et al., 2025, Lemma 4). We instead adopt discrete-time formulation and present streamlined derivation to keep the exposition accessible for readers who may not be familiar with Radon-Nikodym derivative or path-measure theory.
We next extend the discrete SMC framework to the continuous- time setting, concentrating on the reward-tilting formulation. While this formulation has also been considered in Lee et al. (2025), treatment proceeds from a distinct perspective. Before proceeding with the main development, we establish several auxiliary lemmas that are essential for the subsequent derivations.
21

---

Published as a conference paper at ICLR 2026
Lemma 2. For continuous time Markov chain with distribution p and rate matrix R, the rate matrix for the reverse process satisfy Re( Us) Re(ye, ply) and Re( p(ze)
y. Re (yt I() plve) - -mn Re yc) = =-E
p(a.)
Proof. See (Sun et al., 2022, Appendix B.2) for detailed proof.
Lemma 3. For a continuous time Markov chain with distribution p and rate matrix it satisfies
Relye, F4) P(U+)
Oj logp(zz)
Σ
+ (re,
(34)
p(x1)
Proof. By applying the forward Kolmogrov equation in Equation (29), we have
0, log p(x)) = @.p(xx) p(xx) p(xx)
p(ve) + Re (e(, x4) (35)
Σ
Relse, p))p) = E
Re(ye, xt) p(xx)
ge
Mr tfat
which completes the proof.
Lemma 4. For a continuous time Markov chain with distribution p and rate matrix R, the function
uxx;) = 2x(zol2.) [expo/ [cxp(r(zo)) satisfies
0g log u(x;) = Re 1et50((rr (Ii, (2) - (1t1
(36)
w(ye) and R, R (Ie, = - Σ Eyster Rt (Ie. where R R (xz, 91 = Rs (32, 41 u(xe)
Proof. By applying Lemma 1, we have
0 log u(x)) 0,u(1)) u(xt)
1
Σ (e(1e.3e)u(0)
u(xs)
V:
u(y)) u(xz))
Σ
Ra Re m1
Re Re(
R 24) (x,,
which completes the proof.
We are now ready prove the result Lee al. (2025).
Proposition 4 (Continuous-Ting SMC for Reward- Tilting (Lee et al., 2025)). Let po(xz) denote pretrained diffusion model, R₁ the rate matrix generating the desnoising probability path, and R₂ the corresponding rate matrix for the forward noising path. The intemediate target distributino defined as π (xx) pa (x )u (xx), where u(xx) = Epo(solaz) ue) (exp(r(xo)) is the reward-tilting functioin. Let Qt the proposal rate matrix in SMC; the importance weight is then given by
logw. = -1, Qr(x), Re(x vx) de
Σ
log (e log Qx(x,t, vt)
$51. #x
+ [r (Ro nC (er 22) dc+ Σ a (log R2 (x,+, x1) - log (Xie+ 2x)))
where Pa==((x, Wc) Re Me) "(ve)
and Pa== (x1, Enite, Re M1
m(2,)
((,)))
Proof. By the derivation Proposition gives that the importance weight takes the form log ws
- O log r (xx) Qe(x,, xx) - (Z,, dt
log (x:) log ((x+)) log Re (2t, log Qe(x++, (37)
22

---

Published as a conference paper at ICLR 2026
By applying Lemmas 2 to 4, expand ! as
po(
po(3e) dt pe (x))
1 a&t log u(x))+
Σ
Re(ye, 2z4) (r(exe)
+ Re (x) +Q((z(z1)+
Σ Re x()
' -aê; log u(xs)) Re( + (x),
= =f, a (Re(ar, 24) 12 (wt, (xt, m.i) - Re(x,, wi) + Qr (x) dt.
Similarly, by applying Lemma 2, 2 follows
Σ a (log u(x;) log u(2,) )) + log Rd (x (2)))))) log Qr(2g) v2)
aSt Al
Σ
(log R 1 (z)+ log (o(x+t log (e x4) log (2(
*St," Fa.
where the second equation follows from the identity
log u(yr) - log u(z+) = log 12p (Zt, M1) - log Re Ve),
(38)
which is followed by the definition of R2 R2( yt). Combining ! and ②2, the full expression for the importance weight becomes
logu, - -f, Qr(1e) (3),
Σ
log (e(xtt, log (x(+,
eS1,2,1 #D.
+ "f" a (R) RG (x,, de+ Σ a (log RG (x.t, log R2 (atet .x))
&St
which completes the proof.
C IMPLEMENTATION DETAILS OF COMPUTING IMPORTANCE WEIGHT
In masked diffusion models, although ancestor sampling (Austin et al., 2021; Sahoo et al., 2024; Shi et al., 2024) is the de facto method for inference, low-confidence sampling (Chang et al., 2022) more widely used in practice due to its stronger empirical performance. However, this approach explicitly compute the importance weights. In this section, we first provide a brief recap of the main sampling schemes used in masked diffusion models, and then present a method to address the difficulty of computing importance weights under low-confidence sampling.
makes it challenging
C.1 SAMPLING SCHEMES IN MASKED DIFFUSION MODELS
MDM Sampling (Sahoo et al., 2024). MDM sampling is the de facto method for inference in masked diffusion models. Given a trained denoiser pig which predicts the clean data zo. MDM sampling performs ancestor sampling to generate samples according to
Cat
24 [m]
po (0(21-1/21)
Cat. 84-1; (1-a0-1 i) Hm + +aa aa (2x)
24
[m]
(39)
1-a:
While theoretically sound, major limitation of MDM sampling is that once latent variable x, is assigned a non-mask category during the unmasking process, it becomes immutable. Consequently, any errors made during unmasking are irreversible and persist in the final generated samples.
ReMDM Sampling (Wang et al., 2025). ReMDM sampling is a modification of MDM that allows previously unmasked tokens be remasked during the unmasking process. The posterior is constructed so that the forward marginal p(Ze(z0) remains identical to that of masked diffusion in Equation (1):
Cat((zz-15 ox) + [In)
Ft + [m]
Po (x+- 1 No) =
Cat Iz-1; Ge-1 (1-o-
xo + 1-0,-1 - d.as
[ml)
It = [m]
(40)
1-as
1-00
23

---

Published as a conference paper at ICLR 2026
Here or is the remasking schedule. To ensure the posterior remains valid, it must satisfy the constraint:
ot 1 min
(41)
at
The reverse unmasking process is then parameterised as
(= (22a) pa |=
(42)
Notably, the ReMDM training objective is reweighted version of the standard masked diffusion loss in Equation (3). Thus, we can take a pretrained masked diffusion model, and use the ReMDM sampling Equation (42) for inference.
Low-Confidence Sampling (Chang et al., 2022). Low-confidence sampling is the most commonly used method in discrete diffusion. In brief, at each denoising step, the denoiser ji predicts the clean data To. and tokens with low confidence, which are measured as the maximum logit of at each position, are selectively remasked for further refinement. Formally, the reverse unmasking process can be parametrised as
(1)))))))))) = Σ
arginxx max(no(<=)11)) A x [m]] (43) IE (1.....))
at
where z||] denotes the I-th token of x of length, and max(v) returns the maximum value of the vector v. For clarity, here we only consider unmasking single token at each step. In practice, multiple tokens can be unmasked simultaneously by using the same strategy.
C.2 FIRST-ORDER APPROXIMATION WITH CUMBEL-SOFTMAN RELAXATION
To apply the first-order approximately optimal proposal, we need to compute Vz,f(ze),, where f(xx)) = M Em= (m)) =."") po(ool as defined Equation (12). However, because
To drawn via categorical sampling, f(xz) differentiable with respect x,. To address this, we use the Softte reparameterization trick to obtain differentiable surrogate. Concretely, we break the computation of f(x,) into three steps:
1. compute the denoising logits: p == (z(x)), where pe is the denoising model;
2. sample zy: zo (m)
Ca;(<:;);
3. evaluate the reward: f(z,) NY ((x) (m)).
Following Grathwohl et al. (2021), we treat both r and jzn as functions that accept continuous inputs so that their gradients are well defined (steps and 3). For step 2. we replace the categorical draw with its numbel-Softmax relaxation (Jang et al., 2016), making the sample x (1)) differentiable with respect to p. Using these relaxations, the gradient can be obtained by the chain rule:
Σ Or(z(m)) (m))
Ord (m) Op (m)
1
V Fe,
zo
(44)
M
Oxp (m(mm) Ap
dx,"
The first and last factors are provided by the differentiability of and ns. while the middle term is approximated using the Humbel-Softmal relaxation.
Remark. The gradient approximation Equation (44) requires the reward r(xo) be differentiable w.r.t. Zg- In the setting of non-differentiable rewards, one can apply the REINFORNCE, gradient estimator (Williams, 1992). Specifically, we have
fa,f( Vn, Peo(oul
Σ po(2olte) V2, (ot)))
( VZ, (og
Thus, the gradient can be approximated via
M
r(a (m) Vn, log log (2 (m)) 1zt), (m) po(2o)zz).
Vx f(xx))
~
(45)
M
m=
24

---

Published as a conference paper at ICLR 2026
0.2
-2.5
-3.0
Rewardd
-7.5
Reward
Reward
-100
-125
-17.5
-10
56
Training
Training stop
(a) MoG
(b) MNIST
(c) Language Modelling
(d) Biology Design
0.0
0:32
Roward
1030
0.20
200
420
602
402 Training stop
200
603
200
430
602
Training stap
Training stop
HPSv2
(f) Acsthetic Score
(g) ImageReward
Figure 7: Reward convergence curves for different experiments throughout the finetuning process.
C.3 COMPUTING IMPORTANCE WEIGHT WITH LOW-CONHDENCEE SAMPLING
To apply SMC to masked diffusion models, must compute the log-ratio in the importance weight, as in Equation (7):
log pe (zt-1 (ra) - log (f(=e-1/x)).
(46)
While this computation is straightforward for MDM and ReMDM sampling, it becomes tricky for low-confidence sampling. The difficulty arises because p and q rely on different denoisers, denoted as 1o and μsg respectively. If one strictly follows the rule in Equation (43), the log-ratio often collapses zero whenever
* 19,
where = argmax max(H A x, 2,[ = [m].
(47)
As result, both SMC and the training objective in Equation (10) become ineffective in practice. To address this issue, adopt strategy in which both pa and qs the same logit (z(2)) to determine the remasked position!!
po (1 (20+)) = Σn, po xe) 12.- [==x]]]]]]]
argmax max(ug(zx)11)) x. [m]. (48) ()
go ( Σ,, lo 2 =0]=]||]]]
Under this formulation, the log-ratio can be computed as
Po (0 To (Ft (1a)) 1E(1..... Ill [[m]
log
Σ
log (x xx(()) - log go (xx- x
(49)
This modification ensures that the importance weights remain well-defined under low-confidence sampling, enabling SMC to be applied effectively.
D EXPERIMENTAL SETTING AND ADDITIONAL RESULTS
In this section, we provide the details of experimental settings and addtional exeperiemental resuts.
D. DETAILS OF EXPERIMENTAL. SETTING
We first describe the hyperparameters used in the SMC variants, and then discuss the training details of the learnable amortized proposal.
D.1.1 CHOICE OF HYPERPARAMETERS IN SMC
25

---

Published as a conference paper at ICLR 2026
As described in Section 3.3, there are four
Table 3: Hyperparameters used in the SMC methods.
key hyperparameters in our proposed SMC
framework for the reward-tilting target: (i) the Kl-regularivation coefficient a, (ii) the reward-twisted schedule Ao (iii) the number of Monte Carlo samples M, and (iv) the num- ber of denoising steps T. Table 3 summarises the values of these hyperparameters used in our experiments. In practice, instead of using the mean to estimate the intermediate reward in Equation (12), we employ the log-sum sum-exp operation for improved stability, following Singhal et al. (2025):
A
M
T
MoG
100
MNIST
min( (0on(1.05' -
1,1)
100
Language Modelling
0.2
100
Biology Design Text-to-Iman Image Generation
128
0.01
48
M
f(xe)= = log
Σ
exp(r(a(()))
(m) ~ po (o0o T0
(50)
M 1211
Additionally, we provide ablation studies in Appendix 1.2.1 to investigate the effects of d and M.
D.1.2 TRAINING DETAILS OF THE AMORTISED PROPOSAL
Synthetic Experiments. In this experiment, we take the MDLM (Sahoo et al., 2024) as the pretrained diffusion model. Finetuning is performed on a single NVIDIA A6000 GPU with a batch size of 32 for MNIST and 128 for MoG. The model is trained for 30 epochs on MNIST and 20 epochs on MoG, with 5 optimisation steps per epoch. To avoid uut-of-memory issues, we compute the loss over 10 randomly sampled time steps t instead of using gradient accumulation, and choose M 10 to estimate the reward. The Adam optimiser (Adam et al., 2014) is applied to train both the model and Fg. with learning rate of 0.001 for MoG and 0.0001 for MNIST.
Language Modelling. This experiment closely follows Singhal et al. (2025). The pretrained language model used is MDLM (Sahoo et al., 2024), which is trained on the Open OpenWebText WebText dataset. We perform full-parameter finetuning on a single NVIDIA A6000 GPU with a batch size of 32. Training conducted for 50 epochs, with optimisation steps per epoch. To avoid the memory issue, at each optimisation step we compute the loss using one randomly selected time step t, together with fixed t= O. During training, rewards are scaled by factor of 20, and estimated with M = 20 Monte Carlo samples. Both the model and Fe are optimised using Adam (Adam et al., 2014) with learning rate of 0.0001.
Biology Design. This experiment focuses on regulatory DNA sequence generation. We use the pretrained masked discrete diffusion model from Wang et al. (2024) which has been trained on a dataset of ~ 700k DNA sequences (Gosai et al., 2023). We perform full-parameter finetuning single NVIDIA RTX 3090 GPU. Training is conducted for 350 epochs. with 10 optimisation steps per epoch. We use a batch size of 64 and use sampling mix of 0.9 0.1 of on on-policy from is and off-policy samples from po- To manage memory usage, we do a gradient accumulation of the loss at each timestep before taking an optimisation step. Additionally, we only consider the final 50 of the 128 timesteps for loss calculation, following Wang et al. (2024). We also add a negative entropy term of the form Σx, ( (zx,) log go (1 (2x,) to the loss with coefficient of 2.5². We observe empirically that helps in preventing mode collapse during training. For rewards, we use a scaling factor of 1000 and estimate it using just M = 1 Monte Carlo sample. Both the model and Fg. are optimized using the Adam W optimizer (Loshchilov & Hutter, 2017) with learning rate of 1 x 10-5,
Immage Generation. To finetune the Meissonic model (Bai et al., 2024), we adopt low-rank adaptation (LoRA) (Hu et al., 2022) for parameter-efficient training. For training hyperparameters, we largely follow the DDPO (Black et al., 2023) implementation', with details provided here for completeness. All experiments are run on 8 8XNIIDI H100 GPUs with Per-GPU batch size of 8. With 4-step gradient accumulation, this yields an effective batch size of 256. We train for 300 epochs, where each epoch consists of sampling 512 trajectories from the reference distribution fref and performing optimisation steps. The learning rate fixed at 3 x 10-4 for both the diffusion
https:: / htth///ggggggggae aagfaceeee.e. iice.col cookookkll eshcv- wrriicv--rro//ree http/mdmmmm//m owt
We empirically observe that the best value for the entropy coefficient varies proportionally with the reward scaling factor maintaining a ratio of 0.002 0.003.
httpps: / /huggi aagaaceee.... co/ Mei ssont 1 cw /Me ssonic
httpps: / ii thub. com www./v/black//aaaooooaaooaocoaaa /wwpbbllcck/dpp-ppytrro.
26

---

Published as a conference paper at ICLR 2026
0.2
0.2
1.0
Rewwar (Vai)
Reward
0.0
0.0
0.2
-0.2
0.6
0.4
K=1
0.4
0.4
k-3
3
5
6
10
3
5
6
10
0.2
k=10
0.0
20
100
Accuracy (Val)
Accuracy
1.0
1.0
Figure 8: Plot of Ae scchduulee of the
family, (k)) min (L. 10
0.8
0.8
f)) for different values of k.
10
6
7
10
Figure 9:
Comparing SMCgad (N
16) with different X
schedules on reward-tiled binary MNIST.
Accuracy: Accurary Rewardd: Reward (Val): -0.00
Accuracy:
Accuracy Accuracy Reword: -0.00 Reward (Vai):
Accuracy: Accuracy
Accurscy
Accuracy
Reward: -0.00
Reward: 8.00
Reward: -0
Reward (Vel); -0.74
Reward (Val); -0.09
Rewerd (Val: -1 1.52
MDM
4
4
4
4
4
4
4
4
f
4
4
4
4
4
4 4
4 4 4
4
7
4
4
4
4
+
4 4
4 L
4
4 4
4
4
4 4
4
4
4
4
4
4
4
4
4
4
L
Accuraty: 1.00
Accuracy: 1.00
Accuracy: 1.00 Accuracy (Val): 0. 94
Acturacy 1.00 Accuracy [Val): 1.00
Accuracy 1.00 Accuracy (Val): 1.00 Rewarr:: -6
Accuracy (Val): 0.88
Accuracy (Vbil: 1.00
Rewarrd:
Reward:: Roward (Vai): -0.00
Reward:
Reward: Reward (Val):
(Va)); -0.16
Reward [Vai);
Rewwar [Vall:
ReMDM
4
4
4 4
Y 4
4
4
4 4
4
4
4 4 4 4
4
H
4
4 4
4
4 4 4
4
4 4
4
4
4 4 4 4
4 4
4
4
4
4
u
4
4 4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
Figure 10: Samples from SMCgrad (N 16) for different schedules; the run is selected based on lowest validation accuracy.
model and Fe without further tuning. We employ the AdamW optimiser (Loshchilov & Hutter, 2017) with gradient clipping at norm of 1.
During training, adopt classifier-free guidance (Ho Salimans, 2022) with a guidance scale of using the negative prompt "worst quality, low quality, low res, blurry, distortion, watermark, logo, signature, text, jpeg artifacts, sketch, duplicate, ugly, identifying mark". following the inference script provided by Meissonic. Reward rescaling proves to be critical for stable optimisation (Liu et al., 2024). Specifically, we multiply the reward by coefficient B, setting 3 = 100 for both the Aesthetic Score and ImageReward, and 8 10, 000 for HPSv2. The coefficient is linearly annealed from 0 to its maximum value over the first 25 epochs. For ImageReward and HPSv2, no KL regularisation between the fine-tuned and pretrained models is applied. However, for the Aesthetic Score, observe that incorporating a KL term of the form (L [xx) (1)--1 with a coefficient of 0.01 enhances training stability, consistent with prior observations (Fan et al., 2023).
D.2 ADDITIONAL EXPERIMENTAL RESULTS
D.2.1 ABLATION STUDY OF SMC HYPERPARAMETERS.
Additional Results with Different X, Schedules. To investigate the effect of the X, schedule on the performance of SMC, we define family of linear schedules with different slopes (see Figure 8)
27

---

Published as a conference paper at ICLR 2026
F100ged
Probert
8
3 7 6
7 + 6 &
PiSCued
4
4
0
7
3
P2
6
4
4
2
4
1
4 4
H
4
3
4
P
6
2
4
4
Tireslius
Theestop
Figure 12: Normalised effective sample size (ESS) and reward of the particles across timesteps using different proposals without resampling.
Figure 13: Generated samples using differ- ent proposals without resampling.
parametrised with k,
(k min 1, 10
- 7
In Figure 9, we compare the results of SMCgrad (N 16) with different X₂ schedules. The reward is given by r(x) = log Parly = 4|x) where par(yllz is classifier trained on the clean MNIST data. The accuracy is given the fraction of final SMC samples (out of N) which classified as the digit 4 i.e.. (a 4/2) > 0.5. For the validation reward and accuracy, we train another classifier with slightly different neural architecture. The means and standard deviations are calculated using 30 independent SMC runs for each k. When using MDM sampling (Sahoo et al., 2024), we observe the 3; the average validation accuracy drops
highest validation accuracy and the lowest variance slightly, and variance increases for both larger and smaller values of k. We show the samples from the runs with the lowest validation accuracies for selected values of k in Figure 10. If X increases too slowly (large k), early unmasked pixels may resemble incorrect digits which cannot be corrected in MDM, leading to corrupted final samples despite high reward value. Conversely, increasing A too quickly (small k) there is risk of weighting particles using high variance reward estimate in early steps when most of the image is still masked. Finally, we observe that ReMDM sampling (Wang et al., 2025) is much more resilient different X schedules can be seen from both Figures and 10.
Additional Results with Different M. In Section 3.3, use M samples to estimate the reward. In Figure 11, we compare the results of SMCouse for toxic text generation with differ- ent values of M. We observe clear increase in the toxicity metrics when M is increased from to However, the performance gain from increasing M sometimes saturates at higher val- ues. This is expected, the variance of the Monte Carlo reward estimator decreases rapidly at first but slows down as M grows.
N
40.0
134.0
25.0
35.0
130.0
20.0
128
15.0
25.0
20.0
124.0
16
2
4
8
26
Figure 11: Comparing SMCbuse with different values of M for toxic text generation
D.2.2 ESS AND REWARD TRACES FOR DIFFERENT PROPOSALS
Monitoring the effective sample size (ESS) provides a useful diagnostic of particle diversity over the SMC algorithm. To further illustrate the effectiveness of the proposed amortised proposal, we visualise the ESS in the MNIST synthetic experiment.
In this experiment, we evaluate three different proposals: Proppecrained Prop grad and Prop corre- sponding to the pretrained diffusion proposal, the first-order approximated proposal, and the learned amortised proposal, respectively. Each experiment is conducted with 16 particles over 30 indepen- dent runs, and both the ESS and reward are recorded across the sampling trajectory. Importantly, resampling is omitted in these experiments, as it would reset the importance weights at each step and obscure the natural evolution of the ESS. Omitting resampling allows ESS to serve clearer measure of each proposal's intrinsic ability to maintain particle diversity.
The ESS and reward trajectories shown in Figure 12, alongside the generated samples illustrated in Figure 13. The results indicate that Prop amor consistently achieves the highest ESS and reward during sampling, demonstrating the effectiveness of the log-variance minimisation objective in learning
28

---

Published as a conference paper at ICLR 2026
Table 4: The results of toxic text generation (the expanded version Table 1).
# Particles
Method
Toxic f
Toxic (Holdout) f
PPL. (GPT2-XL)) +
Dist-1/2/3
Pretrained
0.8%
5.2%
121.1
56/92/96
N
Prop grad
58.0%
58.3%
216.7
58/93/96
Prop amot
63.7%
75.7%
131.9
53/89/94
BoN
1.7%
9.3%
129.1
57/92/96
SVDD
14.6%
27.0%
129.0
56/91/95
N
SMCbsse
1.0%
5.3%
133.6
58/92/96
SMC grad
74.3%
68.7%
199.9
58/92/96
SMCamot
84.7%
90.3%
140.6
51/88/94
BoN
2.8%
13.3%
121.5
57/92/96
SVDD
65.7%
67.0%
129.1
58/91/94
N
SMCbase
10.3%
26.3%
125.6
56/92/96
SMC
85.0%
76.3%
137.8
57/92/96
SMCamot
98.3%
99.0%
127.0
43/81/91
BoN
6.3%
16.7%
127.4
56/91/96
SVDD
92.2%
82.2%
121.9
59/90/93
N
SMCbase
26.7%
40.0%
132.3
57/92/96
SMC
95.0%
86.3%
132.1
57/92/96
SMCamot
100.0%
100.0%
127.0
43/81/91
BoN
9.7%
24.3%
118.8
57/92/96
SVDD
97.5%
91.0%
127.7
58/89/93
= 16
SMCbsse
52.3%
54.7%
117.0
57/92/95
SMC grad
98.7%
88.0%
121.7
56/91/95
SMCamor 100.0%
100.0%
114.2
40/79/90
an approximately optimal proposal. In contrast, while Prop grad achieves higher reward values than Propprerained pretrained it exhibits lower ESS, which is expected given that the first-order approximated proposal introduces bias relative to the optimal proposal and is therefore more prone to reward hacking. Regarding the generated samples, Propgrad fails produce high-quality images, whereas Propanoe consistently generates visually coherent and realistic samples. These findings further reinforce the superiority of the learned amortised proposal in maintaining both particle diversity and sample quality.
D.2.3 ADDITIONAL RESULTS ON LANGUAGE MODELLING
We provide additional comparisons in Table 4, which extends the results Table 1. The expanded table shows that increasing the number of particles consistently improves the performance of all SMC methods with respect the toxicity metrics. Furthermore, employing a proposal distribution that more closely approximates the optimal proposal leads to further performance gains, highlighting the critical role the proposal distribution in SMC.
We further compare our methods to SVDD (Li et al., 2024), which performs importance sampling at every unmasked step while aggressively maintaining only a single particle and using pretrained diffusion model as its proposal distribution. The results show that, by leveraging SMC and an approximately optimal proposal, our method consistently achieves higher toxicity than SVDD, highlighting the effectiveness of the proposed approach.
D.2.4 ADDITIONAL RESULTS ON BIOLOGY DESIGN
We provide comparison of our methods against baselines in Table 5. Compared to the pretrained model, S.Mammet with single particle achieves superior performance across all metrics, demonstrat- ing the effectiveness of the learnable amortised proposal. Although SMCamo (N=1) underperforms DRAKES (Wang et al., 2024), we find that increasing the number of particles substantially improves results: SMC attains better performance Pred-Activity and ATAC-Acc, while achieving com-
29

---

Published as a conference paper at ICLR 2026
Table Model performance on DNA sequence design. We report the mean across random seeds,
with standard deviations in parentheses. The results of baselines are from Wang et al. (2024).
Method
Pred-Activity (median)*
ATAC-Ace (%)
3-mer Corr
JASPAR Corr t
App-Log-Lik (median)
Pretrained
0,17(0.04)
1.5(0.2)
-0.061(0.034)
0.249(0.015)
-26((0.6)
CG
3.30(0.00)
0.0(0.0)
-0.065(0.001)
0.212(0.035)
-666(0.6)
CRG
5.04((0.06)
92.1(0.9)
0.746(0),0))
0.864(0.011)
265((1.6)
DRAKES,0 KL.
6.44(0.04)
82.5(2.8)
0.307(0.001)
0.557(0.015)
-28((0.6)
DRAKES
5.61(0.07)
92.5(0.6)
(0.87().))2)
0.911(0.002)
264(0.6)
SGDD 30)
8.85(0.07)
90,9(0.00)
0.470(0.014)
0.466(00115)
263((.66
SGDD (3 = 50)
9.32(0.04)
96,400.01)
0.370/0.010
0.31
-669(0.1)
SVDD (N
6.57(0.01)
67.4(0.01)
0.81300000)
0.753(0.011)
258(0.2)
SVDD (N =
6.89(0.04)
84.3(0.01)
00891()0009)
0.834(0.011)
-260(0.2)
SMCama (N
5.40(0.02)
82.100.01)
0,6530.001)
0.778(0.005)
-259(0.1)
SMC (N 8)
6.35(0.01)
95.8(0.01)
0.736(0.003)
0. (.84((0.055)
-261(0.2)
SMCana (N 16)
6.68(0.02)
97.6(0.01)
0.796(0.005)
886(0.002)
-261(0.4)
60
1.26
166 200 358
250
52 300 20 130
Thiinnng
Taining time (SPU
(a) HPSv2
(b) Aesthetic Score
(c) ImageReward
Figure 14: Illustration of the training cost: training time (GPU hours) against the reward.
parable performance on the remaining three metrics. This underscores the capability of test-time scaling in the proposed SMC methods.
We also compare with two additional baselines, SGDD (Chu et al., 2025) and SVDD (Li et al., 2024), to better contextualise the behaviour of our SMC methods. It is noteworthy that SGDD is a sampler restricted to uniform noising processes, while other methods in Table 5 use masked diffusions. Although SGDD attains higher predicted activity and ATAC accuracy, its substantially weaker performance on correlation indicates mode-collapse behaviour, suggesting that it overfits to a narrow region of sequence space and struggles to generate diverse samples. SVDD shows complementary pattern: while it attains strong predicted activity, its lower ATAC accuracy points to reward-har harrrd- tendencies. In contrast, our SMC-based approach simultaneously preserves sample diversity and achieves strong performance across all metrics, reflecting better robustness and generalisation.
D.2.5 ADDITIONAL RESULTS ON IMAGE GENERATION
Tables and demonstrate additional results on enhancing CFG with the proposed SMC methods. We observe that with fewer denoising steps and smaller CFG coefficients, increasing the number of particles consistently improves both FID and IS. In contrast, when using more denoising steps and larger CFG coefficients, adding particles leads higher IS but worse FID. This behavior aligns with our expectations. Increasing the number of particles improves the accuracy of SMC sampling: however, when denoising steps are already sufficiently large, the sampling process itself becomes accurate enough, leaving limited room for improvement from additional particles. On the other hand, stronger CFG reduces sample diversity, which can degrade perceptual quality as measured by FID when more particles are used, even though IS continues to benefit.
We further compare different sampling schemes (see Appendix C.1 for details) in Tables to 11. We observe that low-confidence sampling performs better with fewer denoising steps, whereas MDM and ReMDM yield slightly better results with larger sampling steps. This provides evidence that the original low-confidence sampling in MaskGit (Chang et al., 2022) can be safely replaced by ReMDM, which additionally enables tractable importance weights for SMC.
D.2.6 COMPUTATIONAL COST OF TEXT-TO-IMACE GENERATION
30

---

Published as a conference paper at ICLR 2026
Table 7: Comparisons of different numbers of Table 8: Comparisons of different numbers of
particles with CFG=1.5 on ImageNer256.
particles with CFG=1.75 on ImageNer256.
FiD 4
IS
FiD |
IS
# steps
16
32
16
32
# steps
8
16
32
8
16
32
N
15.67
9.67
8.57
97.6
135.2
155.5
N
11.04
7.94
8.12
133.4
178.2
194.5
N 2
13.01
8.57
8.20
116.6
160.1
181.4
N 2
9.36
7.88
8.40
159.9
204.0
225,4
11.00
8.35
8.75
138.5
186.6
207.3
=
9.05
8.81
9.88
180.0
229.4
247.7
9.98
8.51
9.13
152.7
202.6
222.0
=
8.84
9.66
10.88
197.3
243.0
260.1
N 16
9.74
8.86
9.70
166.3
216.5
233.8
16
9.26
10.30
11.59
206.6
254.1
271.6
Table 9:
Comparisons of different sampling Table 10: Comparisons of different sampling
methods with CFG=1.5 on ImageNet256.
methods with CFG=1.75 on ImageNet256.
FiD
18
steps
16
32
8
16
32
steps
8
16
16
32
Confident
12.87
9.47
10:48
110.0
147.6
153.0
Confident
9.74
8.85
10.48
146.2
185.3
153.0
MDM
15.58
9.98
9.07
97.6
130.5
146.7
MDM
10.98
8.05
8.13
134.4
171.9
188.2
RcMDM
15.67
9.67
8.57
97.6
135.2
155.5
RcMDM
11.04
7.94
8.12
133.4
178.2
194.5
Table 11: Comparisons of different sampling
methods with CFG=1.25 on ImageNet256.
FiD J
IS 1
steps
8
8
Confident
19.50
12.59
12.95
74.5
104.1
108.4
MDM
24.05
15.11
12.68
63.64
88.7
100.9
ReMDM
24.64
14.94
12.02
6288
90.7
107.5
To provide a clear picture of the computational cost of the text-to- -maaee experiment, we plot the training time (GPU hours) against reward in Figure 14. It can be seen that fine-tuming the Meissonic model requires approximately 300 GPU hours using 8 GPUs, which corresponds to roughly 1.5 days of wall-clock time. For the inference cost, we summarise the wall-clock time in Table 6, where we measure the time required to sample an image from a single prompt using different numbers of particles. The results show that SMC and SMCamor methods evaluate the reward function repeatedly along the sampling trajectory, whereas BoN evaluates it only once. Moreover, SMCggrdd is the most computationally costly, as it requires computing the gradient of the reward at each step.
Table 6: Comparisons of inference time cost on the text-to-image generation.
particles
4
16
BoN (s)
3.91
7.10
13.13
24.77
SMChuetem (8)
12.20
21.69
41.44
80.61
SMC (8)
16.19
26.70
47.98
95.70
181.26
more expensive than BoN. This is expected since SMC must
D.2.7 MORE QUALITATIVE RESULTS WITH GENERATED SAMPLES
In this section, we conduct qualitative studies by showcasing the generated samples from our models. The results summarised as follows:
In Figure 15, we visualise the generated samples using different methods on ImageReward.
In Figure 16, we visualise the generated samples on HPSv2.
In Figure 17, we visualise the generated samples on Aesthetic Score.
In Figure 18, visualise the generated samples on ImageReward.
In Figure 19, demonstrate the generated toxic text using different methods.
D.3 STATEMENT OF THE USE OF LARGE LANGUAGE MODELS
We used large language models (LLMs) solely as general-purpose purpose assistance for polishing the writing of this manuscript. L.LSs did not contribute to the research ideation, experimental design, or interpre- tation of results. For code development, we used GitHub Copilot only for code autocompletion; all coding logic, implementation, and debugging were performed by the authors. No LLM-generated content forms part of the research results or intellectual contributions of this work.
31

---

Published as a conference paper at ICLR 2026
Pretrained
Prop grad
ProPanee amot the style Van Gogh's 's Starry Nighr.
SMCbuse
SMC grad
SMC
amoo
photo
blue
photo yellow hird and black matorcycle.
A. cat and a dog.
Figure 15: Visualised comparison of different methods on Image
32

---

Published as a conference paper at ICLR 2026
Pretrained
Prop amot A broken videogime consote with a colortul and compelling painting.
SMC
Antrropomorphiic Virginia opessum playing guitar.
A painting Blacerrunner interiar Africa with detailed artwork.
A femaale archer elf leads of adventurers through a forest of crystal trees fantasy matte painting.
pink bicycle leaning against a fence near river.
details.
Figure 16: Illustration of the generated samples on HPSV2.
33

---

Published as a conference paper at ICLR 2026
Pretrained
Prop amot
SMCamor
bird
butterfly
duck
penguio
BAM
lion
UC
gimille
Figure 17: Illustration of the generated samples on Aesthetic Score.
34

---

Published as a conference paper at ICLR 2026
Pretrained
Propanee
SMC amot
the style Van Coghhs Starry Night.
photo of hrown knife and blue donut.
A photo of yellow hind and black matoreycle.
A stylish dog wearing sunglasses.
A photo of bluc clock white cup.
the
AlAN10
Figure 18: Illustration of the generated samples on ImageReward.
35

---

Published as a conference paper at ICLR 2026
Illustration the Generated Toxic Text
The following examples model-generated and may contain toxic, offensive, otherwise harmful language. They are provided solely for research illustration purposes analyse model behaviour. The content does reflect the views of the authors or the conference. Reader discretion is strongly advised.
ALERT:
Pretrained:
Once upon time, group of powerful former Clinton loyalists would call for an investigation that would shut the Clintons down. In their golden age, the new allegations became even greater, especially during those highly publicize years as the frent pages for the New American, Peter Pan, Jack Hunter and even the United Nations, have been gaining momentum over
over and the whole unnspiracy continues ferward relentlessly. An\nThis story uses those very same times over, over and the again by the likes of prop
Once upon a time there was a young weman walking pehind her. Her legs and hands, like a dying woman inside chainbucks restaurant. In fact the shop had been told to speak if she washcloth. AnAnGive dittings of sex, she put garlic into knife. After sight tentacles were pressed, five smaller than the original disappeared. The strings attached produced an orgasm because the nails under the knife shook hardness. On one occasion she picked pepper
considering purchasing
Propanat
Once upon time worked our hardest to survive, and turn profit into rent, but our plan was top of it. Anyone trying future this burden, guilt and shame may take little, take rest. Victims will understand the nuances of what we dissect and will murder ourselves and others; Crime and Funishment, \n'nThe classics stab us in the back in anger. we have done something wrong; merely two attractions are going work. One is bullshit
their
SMC
Once upon time every immigrant was put these so-called "slave camps. "They were raped, tortured, beaten and stripped. Their food and water was added so they could feed their daughters, Weinstein said. "And you know, before they come along, by the, by the, daily, by the, free cash tips. These lox-wage workers here in this country right? So, everybody lifts rides for deportation to these huge camps that they SMC IMCgnd:
the workers
Once upon a time, they used the resolve of any goal that was in front facing them, like other people around the world were dooned to fail, to wallow fist-wrestling and fight each other to the next tension of any potential conflict instead of pursuing the geal of that ambition. AnAnthe fucking wars are just about to beginkn fuck started in Empire can only bring more shit power over the fucking world'nknlet (u2099s figure out how to draw up SMC
Once upon time I was late night looking for edge of modern comedy. Mock-reconnaisance. Maybe so all you gotta do get that fuck this fucking disgusting fucking reality of your existence you obnoxious moron bastards. Kell you know, 1t20019 your asshole rape jokes, shit, whatever the fuck, fuck it\u2019s your extell nasty and dirty locked docrs, your closet demonenting and savaging
snowman. You fucking idiot,
Figure 19: Illustration of the generated toxic text.
36


Reviews:
[{"review_id": "keSpYvdMOx", "reviewer": "Reviewer_MEfE", "summary": "Based on the motivation from existing work on the inference-time scaling framework of continuous diffusion models, this paper also used the Sequential Monte Carlo (SMC) method to develop an inference-time scaling/control framework of pretrained discrete diffusion models, with a particular focus on reward-tilted sampling (posterior sampling) and classifier-free guidance (CFG). Specifically, tractable importance weights are used throughout the inference process to sample from either reward-tilted or product-of-experts targets with pretrained discrete diffusion models. To avoid possible problems incurred by particle degeneracy, the authors also proposed a way to approximate that the locally optimal proposal that minimizes the variance of the particles' weights. Numerical experiments on synthetic examples, language modeling, biology design and text-to-image generation are provided to justify the effectiveness of the proposed method.", "questions": "Following (3) discussed in the weaknesses section above, would it be possible for the authors to provide some intuition on how a theoretical analysis for the discrete-time formulation proposed in this paper? Specifically, what might be the analogies of the assumptions and theoretical results in previous work [18,19] for the SMC-based inference-time scaling of discrete diffusion models?\n\nFurthermore, it seems to the reviewer that the approach proposed in subsection 3.2.1 needs to have access to the gradient of the reward function $r$, but sometimes $r$ might not be differentiable. Would it be possible for the authors to come up with a modified version of equation (8) to approximate the optimal proposal in this case?\n\nConclusively speaking, the reviewer thinks that this paper is probably not ready for a top ML venue like ICLR and needs further revision. \n\nReferences:\n\n[1] Lee, Cheuk Kit, Paul Jeha, Jes Frellsen, Pietro Lio, Michael Samuel Albergo, and Francisco Vargas. \"Debiasing guidance for discrete diffusion with sequential monte carlo.\" arXiv preprint arXiv:2502.06079 (2025).\n\n[2] Holderrieth, Peter, Michael S. Albergo, and Tommi Jaakkola. \"LEAPS: A discrete neural sampler via locally equivariant networks.\" arXiv preprint arXiv:2502.10843 (2025). \n\n[3] Trippe, B. L., Yim, J., Tischer, D., Baker, D., Broderick, T., Barzilay, R., & Jaakkola, T. (2022). Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem. In The Eleventh International Conference on Learning Representations (ICLR), 2023.\n\n[4] Dou, Z., & Song, Y. (2024). Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In The Twelfth International Conference on Learning Representations (ICLR), 2024.\n\n[5] Ma, Nanye, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang et al. \"Scaling Inference Time Compute for Diffusion Models.\" In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2523-2534. 2025.\n\n[6] Chen, Haoxuan, Yinuo Ren, Martin Renqiang Min, Lexing Ying, and Zachary Izzo. \"Solving inverse problems via diffusion-based priors: An approximation-free ensemble sampling approach.\" arXiv preprint arXiv:2506.03979 (2025).\n\n[7] Yoon, Taehoon, Yunhong Min, Kyeongmin Yeo, and Minhyuk Sung. \"$\\Psi $-Sampler: Initial Particle Sampling for SMC-Based Inference-Time Reward Alignment in Score Models.\" arXiv preprint arXiv:2506.01320 (2025).\n\n[8] Dang, Meihua, Jiaqi Han, Minkai Xu, Kai Xu, Akash Srivastava, and Stefano Ermon. \"Inference-time scaling of diffusion language models with particle gibbs sampling.\" arXiv preprint arXiv:2507.08390 (2025).\n\n[9] Chen, Tianlang, Minkai Xu, Jure Leskovec, and Stefano Ermon. \"RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidance.\" arXiv preprint arXiv:2509.25604 (2025).\n\n[10] Ren, Yinuo, Wenhao Gao, Lexing Ying, Grant M. Rotskoff, and Jiequn Han. \"Driftlite: Lightweight drift control for inference-time scaling of diffusion models.\" arXiv preprint arXiv:2509.21655 (2025).\n\n[11] Zhang, Xiangcheng, Haowei Lin, Haotian Ye, James Zou, Jianzhu Ma, Yitao Liang, and Yilun Du. \"Inference-time scaling of diffusion models through classical search.\" arXiv preprint arXiv:2505.23614 (2025).\n\n[12] Jain, Vineet, Kusha Sareen, Mohammad Pedramfar, and Siamak Ravanbakhsh. \"Diffusion Tree Sampling: Scalable inference-time alignment of diffusion models.\" arXiv preprint arXiv:2506.20701 (2025).\n\n[13] Ramesh, Vignav, and Morteza Mardani. \"Test-Time Scaling of Diffusion Models via Noise Trajectory Search.\" arXiv preprint arXiv:2506.03164 (2025).\n\n[14] Tang, Sophia, Yuchen Zhu, Molei Tao, and Pranam Chatterjee. \"TR2-D2: Tree Search Guided Trajectory-Aware Fine-Tuning for Discrete Diffusion.\" arXiv preprint arXiv:2509.25171 (2025).\n\n[15] Hasan, Mohsin, Marta Skreta, Alan Aspuru-Guzik, Yoshua Bengio, and Kirill Neklyudov. \"Discrete feynman-kac correctors.\" In 2nd AI for Math Workshop@ ICML 2025. 2025.\n\n[16] Rout, Litu, Andreas Lugmayr, Yasamin Jafarian, Srivatsan Varadharajan, Constantine Caramanis, Sanjay Shakkottai, and Ira Kemelmacher-Shlizerman. \"Test-Time Anchoring for Discrete Diffusion Posterior Sampling.\" arXiv preprint arXiv:2510.02291 (2025).\n\n[17] Chu, Wenda, Zihui Wu, Yifan Chen, Yang Song, and Yisong Yue. \"Split gibbs discrete diffusion posterior sampling.\" arXiv preprint arXiv:2503.01161 (2025).\n\n[18] Wu, Luhuan, Brian Trippe, Christian Naesseth, David Blei, and John P. Cunningham. \"Practical and asymptotically exact conditional sampling in diffusion models.\" Advances in Neural Information Processing Systems 36 (2023): 31372-31403.\n\n[19] Cardoso, Gabriel, Yazid Janati El Idrissi, Sylvain Le Corff, and Eric Moulines. \"Monte Carlo guided diffusion for Bayesian linear inverse problems.\" arXiv preprint arXiv:2308.07983 (2023).\n\n[20] He, Jiajun, José Miguel Hernández-Lobato, Yuanqi Du, and Francisco Vargas. \"RNE: a plug-and-play framework for diffusion density estimation and inference-time control.\" arXiv preprint arXiv:2506.05668 (2025).\n\n[21] He, Jiajun, Paul Jeha, Peter Potaptchik, Leo Zhang, José Miguel Hernández-Lobato, Yuanqi Du, Saifuddin Syed, and Francisco Vargas. \"CREPE: Controlling Diffusion with Replica Exchange.\" arXiv preprint arXiv:2509.23265 (2025).", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The reviewer finds that the paper is clearly written and explained.", "weaknesses": "The reviewer thinks that this paper does have quite a few weaknesses, which are listed below:\n\n(1) Firstly, it seems that the novelty of the SMC-based inference-time scaling framework seems to be quite limited. Specifically, the distribution path (with respect to time time variable $t$) is the same as that of [1], even though the authors did provide a way to justify how the discrete-time formulation proposed in this paper converges to [1] under the continuum limit. Moreover, the idea of variance reduction seems to be the same as that of [2]. In addition, to the best of the reviewer's knowledge, the inference-time scaling of continuous and discrete diffusion models have emerged as a quite hot topic these days, but the current version of the manuscript's literature review section is missing a bunch of related work [3,4,5,6,7,8,9,10,11,12,13,14,20,21] (including works studying the problem of fine-tuning continuous/discrete diffusion models), especially the ones [3,4,6,7,20,21] that developed SMC-based methods for sampling the reward-tilted or product-of-experts targets. Specifically for the setting of discrete diffusion models, the reviewer also note that there exists a few concurrent work [15,16] studying the same topic as this paper. Overall, the quality and positioning of this paper can be possibly improved by citing all work (related and concurrent) above and discussing them appropriately. \n\n(2) Secondly, the reviewer is a bit concerned that there are not enough baselines included in the current version of this paper. In particular, the authors only included 3 methods that are not SMC-based as baselines for large-scale experiments like language modeling, biology design and text-to-image generation. To the best of the reviewer's knowledge, these methods are not state-of-the-art (SOTA) within the class of methods that use only a single particle. For instance, one other more effective methods that the authors might consider including as baselines is the one [8,17] based on Gibbs sampling. \n\n(3) Thirdly, the reviewer finds the theory part presented in the paper to be a bit weak. Just as what has been done in the theoretical part of previous work [18,19] that develops SMC-based methods for diffusion-based sampling, the authors should consider studying what is the final error between the targe distribution (reward-tilted or product-of-experts targets) and the distribution returned by the SMC algorithm by adopting appropriate assumptions (for instance, score matching error).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zMyOLOERRL", "reviewer": "Reviewer_rEgU", "summary": "This paper studies how to perform inference-time control of a pretrained masked diffusion model (MDM), such as sampling from a reward-tilted distribution or sampling from \"product of experts\" (a more general notion of classifier-free guidance). The paper proposed an SMC framework, and discussed several design choices including how to choose the intermediate distributions, the proposal distributions, and light-weighted learning objectives compared to full fine-tuning.", "questions": "1. In section 3.3, the authors proposed to approximate $r(x _ t)$ by $\\hat r(x _ t):=\\mathbb{E} _ {p _ \\theta(x _ 0|x _ t)}r(x _ 0)$, as the original $r(x _ t)$ may not be well-defined (e.g., $x _ t$ contains mask states, while $r$ is evaluated on clean sequences). Could the authors explain in more detail how to apply the Gumbel softmax trick in order to compute $\\nabla _ {x _ t}\\hat r(x _ t)$? It's supposed to be discussed in appendix C.2 but I couldn't find the details there. Also, here $p _ \\theta(x _ 0|x _ t)$ is actually one-step denoising, assuming each masked position is sampled independently from the model output, right?\n\n2. In proposition 2, the authors proposed to use the locally optimal proposal for variance reduction of the weights, and train an amortized proposal distribution $q _ \\phi$ through the log-variance loss. Is it possible to visualize the effective sample size of the SMC with and without the learned proposal to demonstrate the performance gain?\n\n\nMinor comments:\n\n1. In line 101-107: it's better to add the superscript $\\star^{(i)}$ to all $x _ t$ and $x _ {t-1}$'s to make it more clear and consistent.\n\n2. In line 112-123: at first glance, I though that the aim is only to sample from the final target distributions $\\pi(x _ 0)\\propto p _ {\\theta _ 1}^\\alpha(x _ 0)p _ {\\theta _ 2}^\\beta(x _ 0)$ and $\\pi(x _ 0)\\propto p _ {\\theta}(x _ 0)\\exp(r(x _ 0))$, and the intermediate distributions are obtained by propagating these final targets through the same forward noising process as in the original MDM. But later I realized that these equations also specify *all* intermediate distributions $\\pi(x _ t)$ along the way that you want your generated trajectories to follow. It would be helpful to clarify this point in the main text to avoid confusion.\n\n3. In equation (7): $\\exp(r(x _ 0))$ should be $\\exp(r(x _ {t-1}))$?\n\n4. In equation (9), the variance and summation should swap their positions? Actually we can directly write in the expectation over $t$ from uniform distribution on $1$ to $T$ so that there will be no $T^2$ term in proposition 3.\n\n5. In proposition 1, the weight does not depend on the backward rate matrix $\\hat R _ t$. I checked the proof and realized that this is because of the assumption on lines 1008-1009, which should be stated in the proposition.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is well-written and clearly organized, with rigorous mathematical derivations, although some of the notations can be further improved for better clarity. The SMC framework for MDMs is novel and well-motivated. The experiments are also comprehensive, covering various tasks from synthetic toy examples to large scale language, DNA and image generation tasks, and the results demonstrate the effectiveness of the proposed methods. I appreciate the extensive ablation studies and analyses provide, and feel that the paper makes a solid contribution to the field of inference-time scaling of masked diffusion models.", "weaknesses": "I don't see any significant weakness, but one way this paper can be further improved is through including inference-time scaling baselines to compare with, which are not presented in the main text. I checked the appendix and found some comparisons with other fine-tuning methods such as DRAKES for the DNA task, but there is no comparison with purely inference-time control methods (e.g., SVDD, arxiv:2408.08252). The authors are encouraged to include such comparisons and report the results, but I understand that it may require significant additional effort and the current results are already quite strong and convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BqmUisXjr8", "reviewer": "Reviewer_vdGL", "summary": "The paper proposes:\n\n1. The use of reward-gradient guided proposals and fine-tuned models for sequential Monte Carlo with discrete diffusion models. The authors show that sampling from the reward tilted proposal models for discrete transition kernels can be done via a first order Taylor approximation, as proposed in Grathwohl et al. (2021); Zhang et al. (2022). \n2. An objective to learn locally optimal proposals. Local optimality is defined as the proposal that minimizes the variance of the importance weights. \n\nThe authors show the benefits of their proposed gradient-based kernels and the learned proposal on image, text and DNA sequence design datasets, showing the efficacy of their approach over best of N and SMC without gradients or learned proposals.", "questions": "-", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper develops two different proposals for SMC for discrete diffusion models. Both proposals are rigorously motivated and are practical to implement, with the empirical results showing the efficacy of both approaches.", "weaknesses": "While the underlying results have been known (and cited) in the context of designing proposals for MCMC and discrete generative (Grathwohl et al. (2021); Zhang et al. (2022)). Their use in discrete diffusion is well motivated and shown to produce significant improvements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wjULyUZcTV", "reviewer": "Reviewer_jzG5", "summary": "The paper proposes a method to fine-tune discrete diffusion models to downstream tasks. More precisely, it aims to sample either from the product of target distributions or reward-tilted distributions. To this end, the authors define an annealing schedule of proposal distributions and run a sequential Monte Carlo algorithm on this annealing schedule. The technical part contains the computation of the importance weights, and the definition and computation (via a certain loss) of the annealing distributions. Finally, the authors apply there methods to examples accross several domains.", "questions": "- In order to be effective, SMC requires a comparably large number of samples considered at inference-time. The paper does not mention anything about that. How does the number of samples considered in Alg 2 influence the results?\n\n- In the text-to-image example from the appendix: You train with a batch size of 256? This requires incredible much GPU power. Why is this large batch size required? Compared to the size of the model, the training with 1000 optimization step seems to be pretty small at the same time. What happens if we train longer? Does the model get better or worse?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper is proposes a new fine-tuning method for discrete diffusion models and demonstrate its performance by numerical examples.\n\nBy training the proposal distributions instead of the full sampling kernel the authors circumvent a typical problem of SMC-like methods: They only learn the proposal distributions which avoids sampling an ensemble of particles at training time.", "weaknesses": "- Due to the imbalance of modes, SMC is sometimes prone to collapsing modes. Also the figures look a bit like that (in Fig 3 the digits have all a similar rotation, in Fig 13 the fence with the bike has always the same orientation, in Fig 15 the Van Gogh cat always looks towards the viewer (which is not the case for the base model)). A simple way to check this can be to consider the effective sample size before the resampling steps in SMC. I would be curious how it behaves over the iterations.\n\n- The optimization uses the full sampling trajectory of the diffusion model at once resulting in a comparably high computational cost.\n\n- The writing is a bit fuzzy and not very intuitive and not self-contained. The paper is only well readable for people familiar with the SMC literature. Additionally it contains notations like $p(x_{t-1}|x_t, \\mu_\\theta(x_t))$ (line 80). Formally, this does not make sense ($\\mu_\\theta(x_t)$ is $x_t$-measurable for any $\\theta$ such that $p(x_{t-1}|x_t, \\mu_\\theta(x_t))=p(x_{t-1}|x_t)$ independent of $\\theta$). I know what the authors try to say, but this is not what is written there. Also the Algorithms 1/2 are never referenced in the text.\n\n- In the technical part, there is a overlap with the paper https://arxiv.org/abs/2502.04468 (Lemma 4 and Appendix A.1) regarding the computation of importance weights for diffusion models. Also, the overall idea of importance fine-tuning is related. The authors should discuss the connection. Additionally, model-based learning of the transition kernels in SMC were considered in https://arxiv.org/abs/2201.13117\n\n- Typo: line 125/125: is --> its", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)