$\Psi$-Sampler: Initial Particle Sampling for SMC-Based Inference-Time Reward Alignment in Score Models
Authors: TaeHoon Yoon
Yunhong Min
Kyeongmin Yeo
Minhyuk Sung
Venue: NeurIPS 2025 spotlight ()
Keywords: Inference-time reward alignment
Score-based generative models
Primary area: deep_learning

Abstract: We introduce $\Psi$-Sampler, an SMC-based framework incorporating pCNL-based initial particle sampling for effective inference-time reward alignment with a score-based model. Inference-time reward alignment with score-based generative models has recently gained significant traction, following a broader paradigm shift from pre-training to post-training optimization. At the core of this trend is the application of Sequential Monte Carlo (SMC) to the denoising process. However, existing methods typically initialize particles from the Gaussian prior, which inadequately captures reward-relevant regions and results in reduced sampling efficiency. We demonstrate that initializing from the reward-aware posterior significantly improves alignment performance. To enable posterior sampling in high-dimensional latent spaces, we introduce the preconditioned Crank–Nicolson Langevin (pCNL) algorithm, which combines dimension-robust proposals with gradient-informed dynamics. This approach enables efficient and scalable posterior sampling and consistently improves performance across various reward alignment tasks, including layout-to-image generation, quantity-aware generation, and aesthetic-preference generation, as demonstrated in our experiments.

V-Sampler: Initial Particle Sampling for SMC-Based Inference-Time Reward Alignment in Score Models
Tachoon Yoon'
Yunhong Min
Kyeongmin Yeo"
Minhyuk Sung
KAIST
dbsohdd3
Abstract
We introduce V-SAMPLER, SMC-based framework incorporating PCNL-based initial particle sampling for effective inference-time reward alignment with a score-based generative model. Inference-time reward alignment with score-based generative models has recently gained significant traction, following a broader paradigm shift from pre-training to post-training optimization. At the core of this trend is the application of Sequential Monte Carlo (SMC) to the denoising process. However, existing methods typically initialize particles from the Gaussian prior, which inadequately captures reward-relevan regions and results in reduced sam- pling efficiency. We demonstrate that initializing from the reward-aware posterior significantly improves alignment performance. To enable posterior sampling in high- nigh-dimensional latent spaces, we introduce the preconditioned Crank Crank-Nicolson Nicolson Langevin (pCNL) algorithm, which combines dimension- robustt proposals with gradient-informed dynamics. This approach enables efficient and scalable posterior sampling and consistently improves performance across various reward align- ment tasks, including layout-to-image generation, quantity-aware generation, and aesthetic-preferncee generation, demonstrated in our experiments. Project Webpage: https::/pssisaampp: github..o
Introduction
Recently, shift in the scaling law paradigm from pre-training to post-training has opened new possibilities for achieving another leap in Al model performance, as exemplified by the unprecedented AGI score of GPT-o3 M and DeepSeek's s "Aha moment"
(2). Breakthroughs in LLMs have also extended to ccore-based generative models 777 resulting in significant improvements in user preference alignment [8]. Similar to the autoregressive generation process in LLMs, the denoising process in score- based generative models can be interpreted as a Sequential Monte Carlo (SMC) [ 11 process with a single particle at each step. This perspective allows inference- alignment to be applied analogously to LLMs by populating multiple particles at each step and selecting those that score highly under given reward function [8] 17. key distinction is that score- based generative models enable direct estimation of the final output from any noisy intermediate point via Tweedie's formula [18, facilitating accurate approximation the optimal value function through expected reward estimation.
20
However, previous SMC-based approaches
where each SMC step is coupled
with the denoising process of score-based generative models, are limited in their ability to effectively explore high-reward regions, the influence of the reward signal diminishes over time due to vanishing diffusion coefficient. Thus, rather than relying on particle exploration during later stages, it more critical to identify effective initial latents that are well-aligned with the reward model
Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

from the outset. In this work, we address this problem and propose an MCMC-based initial particle population method that generates strong starting points for the subsequent SMC process. This direction is particularly timely given recent advances in distillation techniques for score-based
generative models (73) now widely adopted in state-of-the-are models [28 29]. These methods
yield straighter generative trajectories and clearer Tweedie estimates [88] from early steps, enabling
more effective exploration from the reward-informed initial distribution.
A straightforward baseline for generating initial particles is the Top-K strategy: drawing multiple samples from the standard Gaussian prior and selecting those with the highest reward scores. Though effective, this naive approach offers limited improvement in subsequent SMC due to its reliance on brute-force sampling. Motivated by these limitations. we explore Markov Chain Monte Carlo (MCMC) 33 methods based on Langevin dynamics, which are particularly well-suited to our setting since we sample from the initial posterior distribution, whose form is known. Nevertheless, applying MCMC in our problem presents unique challenges: the exploration space is extremely high- dimensional (c.g., 65,536 for FLUU 28 ) posing significant challenges for conventional MCMC methods. In particular, the Hetropolis-Hastinge (MH) accept-reject mechanism, when used with standard Langevin-based samplers, becomes ineffective in such high-dimensional regimes, as the acceptance probability rapidly diminishes and most proposals are rejected.
Our key idea for enabling effective particle population from the initial reward informed distribution is to leverage the Preconditioned Crank Nickl-No (pCN) algorithm [36 which is designed for function spaces or algorithm (yielding PCNL.), its semi-implicit Fuler formulation allows for efficient exploration in a high-dimensional space. Furthermore, when augmented with the MH correction, the acceptance rate is significantly improved compared to vanilla MALA. We therefore propose performing pCNL over the initial posterior distribution and selecting samples at uniform intervals along the resulting Markov chain. These samples are then used as initial particles for the subsequent SMC process across the denoising steps. We refer to the entire Preeinne-PNLL-bbasee SMC-based Inference-time reward alignment-an as PSI (V) VV--aprler. To the best of our knowledge, this is the first work to apply the pCN algorithm in the context of generative modeling.
Hilbert spaces. When combined with the Langevin
initial particle sampling followed by
In our experiments, we evaluate three reward alignment tasks: layout-to-image generation (placing objects in designated bounding boxes within the image), muantity-aware generation (aligning the number of objects in the image with the specified count), and pesthetic-prefereen (enhancing visual appeal). We compare our V-S AMPLER against the base SMC method 14] with random initial particle sampling. SMC combined with initial particle sampling via Top-K -ff--, .. ULA, and MALA, as well as single-particle methods [39 40). Across all tasks, V-SAMPLER consistently achieves the best performance in terms of the given reward and generalizes well to the held-out reward, matching or surpassing existing baselines. Its improvement over the base SMC method highlights the importance of posterior- based initialization, while its outperformance over ULA and MALA further confirms the limitations of these methods in extremely high- gigh-dimensional spaces.
generation
2 Related Work
2.1 Inference-Time Reward Alignment
Sequential Monte Carlo (SMC) [1 has proven effective in guiding the generation process of score-based generative models for inference-time reward alignment [22] 21
12]. Prior SMC-based methods differ in their assumptions and applicability. For instance, FPS
15 14
155 and MCGdiff [2 are specifically designed for linear inverse problems and thus cannot generalize to arbitrary reward functions. SMC-Diff 21 depends on the idcalized assumption that the Icarned reverse process exactly matches the forward noising process an assumption that rarely holds in practice. TDS [4] and DAS [12] employ twisting and tempering strategies respectively to improve approximation accuracy while reducing the number of required particles. Despite these variations, all aforementioned SMC- based approaches share a common limitation that they initialize particles from the standard Gaussian prior, which is agnostic to the reward function. This mismatch can result in poor coverage of high-reward regions and reduced sampling efficiency.
In addition to multi-particle systems like SMC, single-partiche approaches have also been explored for inference-time reward alignment [39 These methods guide generation by applying reward gradients along single sampling trajectory. However, they are inherently limited in inference-time

---

reward alignment, as simply increasing the number of denoising steps does not consistently lead to better sample quality. In contrast, SMC-based methods allow users trade computational cost for improved reward alignment, making them more flexible and scalable in practice.
2.2 Fine-Tuning-Bee Reward Alignment
Beyond inference-timeg methods, another line of work focuses on fine-tuning score-based generative models for reward alignment. Some approaches perform supervised fine-tuning by weighting generated samples according their reward scores and updating the model favor high-reward outputs 43 while others frame the denoising process as a Markov Decision Process (MDPs) and apply reinforcement learning techniques such policy gradients 45 entropy-regulariva objectives to mitigate overoptimization [46.48]. These RL-based methods are especially useful when the reward model is non-differentiable but may miss gradient signals when available. More recent methods enable direct backpropagation of reward gradients through the generative process 49 [50].
Alternatively, several works (20, 5T 52 adopt stochastic optimal control (SOC) perspective, deriving closed-form optimal drift and initial distributions using pathwise KL objectives. While tinettuning -ased methods are an appealing approach, they have practical limitations in that they necessitate costly retraining whenever changes are made to the reward function or the pretrained model. Further, has been shown that fine-tuning-hal maaeng methods exhibit mode-seeking behavior [12]
which leads to low diversity in the generated samples.
3 Problem Definition & Background
3.1 Background: Score-Based Generative Models
Given a standard Gaussian distribution p1 = =N I) and data distribution po, score-based generative models trained to estimate the score function, which is the gradient of log-density, at intermediate distributions along probability path connecting In score-based generative models 3777 the data generation process is typically described by reverse-time stochastic differential equation (SDE) 4
to po-
g(t)² dx; + f(x,,t) u(x,, t) - log (xg);( ~ PL
+
(I)
where f(x,,t) and gt) denote the drift and diffusion coefficients, respectively, and W is a d- dimensional standard Brownian motion. The term u(xe, t) corresponds to the velocity field in flow-based model [53]27) 54] and also corresponds to the drift term of the probability flow ODE (PF-ODE) in diffusion models [4]. We assume that the generation process proceeds in decreasing to
time, i.e., from generative modeling literature 4 5
following the convention commonly adopted the basre-based
The deterministic flow-based generative model can be recovered by setting the diffusion coefficient g(t) = 0, thereby reducing the SDE to an ODE. Note that flow-based models [53] 27 originally formulated as an ODE, can be extended to an SDE formulation that shares the same intermediate
distributions Pt, thereby allowing stochasticity be introduced during generation 7 6 1 Moreover, the velocity field u(x, t) can be readily transformed into a score function 7 54] For these reasons, categorize both diffusion and flow-based models the score-based generative models.
3.2 Inference- Reward Alignment Using Score-Based Generative Models
aims generate high-reward samples x, E 12d without fine-tuning the pretrained score-base bosed generative model. The reward associated with each sample evaluated using a task-specific reward function r : Rdd IR, which may quantify aspects such as aesthetic quality or the degree to which generated image satisfies user-specified conditions.
Inference-time reward alignment 8
with respect to the reward function, which may lead to severe distributional drift or adversarial artifacts, regularization term is introduced to encourage the generated samples to remain close to the prior of the pre-trained generative model. This trade-off
But to avoid over-optimization [46] 20 [2
captured by defining a target distribution pe that balances reward maximization with prior adherence,
3

---

formally expressed as:
Po = arg max Rxong (r(x)) D Pkllallpo)
(6)
Here, term (a) in Eq. encourages the generation of high-reward samples, while term (b), the KL-diivergence, enforces proximity the pre-trained model's 's prior distribution po- The parameter a E R4 controls the strength of this regularization: larger values of a lead to stronger adherence to the prior, typically resulting in lower reward but higher proximity to the support of the generative model.
The target distribution Pū has closed-form expression, given by:
1 Zo
r(xo)
px(x))
Po (xn) exp
where Zu is normalizing constant. Detailed derivation using calculus of variations can be found in Kim al. [13. This reward-aware target distribution has been widely studied in the reinforcement learning Analogous ideas have also been adopted to fine-tuning score- based generative
literature (55 59]
20). in our case, this target distribution also serves as the objective from which one aims to sample in inference-time reward alignment task (8 12 13
models 66 51 52 48
Since sample generation in score-based models proceeds progressively through sequence of timesteps, it becomes important to maintain proximity with the pretrained model not just at the endpoint, but throughout the entire generative trajectory. To account for this, the original objective in Eq. is extended to trajectory-level formulation. Although there are some works [50] 46 45) that frame this problem as catropy-rrgulariize Markov Decision Process (MDPs), where each denoising step of score-based generative model corresponds to a policy in RL, we adopt a stochastic optimal control (SOC) perspective [20] 51 (52) which naturally aligns with the continuous- totttttuuueeeuueeeee structure of score-based generative models and yields principled expressions for both the optimal drift and the optimal initial distribution.
Building on this, the entropy-regularized SOC framework proposed by Uehara et al. (20 provides closed-form approximations for the optimal initial distribution, optimal control function, and optimal transition kernel that together enable sampling from the reward-aligned target distribution defined in Eq. using score-based generative models.
The optimal initial distribution can be derived using the Feynman Kac formula and approximated via Tweedie's formula [18] as:
(r(8011 a
(x1) Z1
(X(x) exp
where Xoje : Exp [xo] denotes Tweedie's formula [8] representing the conditional expectation
under Poje := p(xo xx). Under the same approximation, the transition kernel satisfying the optimality
condition is approximated by:
po (xx- exp(r(x0=-ar//0
exp(r(xuj:)/a) Pe (x(xx-Aelx:).
(5)
where (xx-)x)(x,) a transition kernel of the pretrained score- based generative model. Further details on the SOC framework and its theoretical foundations in the context of reward alignment are provided in the Appendix
3.3 Sequential Monte Carlo (SMC) with Denoising Process
21 22 have demonstrated that Sequential Monte Carlo (SMC) can efficiently generate samples from the target distribution in Eq. When applied to booee generative models, the denoising process is coupled with the sequential structure of SMC. Specifically, several prior works transition kernel for sampling from Eq. 3
For reward-alignnen tasks, recent works [12
15| 14
88] [2 adopt Eq. 5 as the intermediate target
In general, SMC methods 91 are a class of algorithms for sampling from sequences of probability distributions. Starting from K particles sampled i.i.d. from the initial distribution, SMC approximates a target distribution by maintaining population of K weighted particles, which are repeatedly updated through sequence of propagation, reweighting, and resampling steps. The weights are

---

updated over time according to the following rule:
Pur Pur (x Xx) (8)
Wi
(6)
((x
where Pur is an intermediate target kernel we want to sample from, and q(x1-11(xx) is proposal kernel used during propagation. As the number of particles increases, the approximation improves due to the asymptotic consistency of the SMC framework 62].
Following 8 12 14,
which derives both the intermediate target transition kernel and the associated proposal for reward-guided SMC, we compute the weight at each time step as:
axppr( arlx)) (6)
where Pur is set as Eq. The proposal distribution (xxx-x)) xx) is obtained by discretizing the reverse-time SDE with an optimal control. This yields the following proposal with the Tweedie's formula [8]
((x:-r( = (Nxx - f(x,, t) + ( "(
(()
(8)
Details on SMC and its connection reward-guided sampling are provided in the Appendix
3.4 Limitations of Previous SMC-Based Reward Alignment Methods
While prior work [22 [5] 14 21 22 has demonstrated the effectiveness of SMC in inference- reward alignment, these approaches typically rely on sampling initial particles from the standard Gaussian prior. We argue that sampling particles directly from the posterior Eq. 4 rather than the prior, is essential for better high-reward region coverage and efficiency in SMC. First, the effectiveness of the SMC proposal distribution Eq. [ diminishss over time making increasingly difficult to guide particles toward high-reward regions in later steps. As the diffusion coefficient g()) - 0 as & + 0, weakens the influence of the reward signal Vr(xo .). since scaled by g² (t) in the proposal. Second, the initial position of particles becomes particularly critical when the reward function is highly non-convex and multi-modal. While the denoising process may, principle, help particles escape local modes and explore better regions, this becomes increasingly difficult over time, not only due to the vanishing diffusion coefficient, but also because the intermediate distribution becomes less perturbed and more sharply concentrated, reducing connectivity between modes [63]. In contrast, at early time steps (e.g.. f = 1), the posterior distribution is more diffuse and better connected across modes, enabling more effective exploration. Furthermore, recent score-based generative models distilled for trajectory straightening have made the approximation of the optimal initial distribution in Eq. 4 sufficiently precise. These observations jointly motivate allocating computational effort to obtaining high-quality initial particles that are better aligned with the reward signal.
V-Sampler: pCNL-Based Initial Particle Sampling
In this work, we propose y V-Sampeer, a framework that combines efficient initial particle samping with SMC-based inference-time reward alignment for score-based generative models. The initial particles are sampled using the Preconditioned Crank-Nicolsse Langevin (pCNL) algorithm, hence the name PCNL-based initial particle sampling followed by SMC-based Inference-ting reward alignment. The key idea is to allocate computational effort to the initial particle selection by sampling directly from the posterior distribution defined Eq. 4 This reward-informed initialization ensures that the particle set is better aligned with the target distribution from the outset, resulting in improved sampling efficiency and estimation accuracy in the subsequent SMC process.
While the unnormalized density of the posterior distribution in Eq. 4 has an analytical form, drawing exact samples from remains challenging. A practical workaround to approximate posterior sampling via a Top-K-of-N strategy: generate N samples from the prior, and retain the top K resembles rejection
highest-scoring samples as initial particles. This variant of Best-of-N 664
66
We find that even this simple selection-basee approximation leads to meaningful improvements. But considering that sampling space is high-dimensional,, one can adopt Markov Chain Monte Carlo (MCMC) (00355 which is known to be effective at sampling from high-dimensional space. what follows, we briefly introduce Langevin-basse based MCMC algorithms that we adopt for posterior sampling.
sampling and serves as a crude approximation to posterior sampling 67
68
5

---

4.1 Background: Langevin-Based Markov Chain Monte Carlo Methods
Langevin-basd based MCMC refers to class of samplers that generate proposals by discretizing the Langevin dynamics, represented as stochastic differential equation (SDE),
dx 1 V log Pur (x) dt dW,
(9)
2
whose stationary distribution is the target density Pur. A single Suler-Maruyane discretization of the Langevin dynamics with step size > produces the proposal
= V log (x) + Vcz,
(0,
(10)
2
Accepting every proposal yields the Unadjusted Langevin Algorithm (ULA) B1. As a result, the Markov chain induced by ULA converges from the discretization error. In particular, since ULA does not include a correction mechanism, it does not guarantee convergence the target distribution Pur- EettrpoolisAAdjsste Langevin Algorithm (MALA) [31 combines the Langevin proposal Eq. [0] with the Metropolis-HHasting (MH) 669 30 correction, general accept-reject mechanism that eliminates discretization bias. Given the current state x and proposal x' ~ (xx'xx), the move is accepted with probability:
biased distribution whose discrepancy arises
@m x') = min (1, Puar (x') g(x[x')
(11)
Pear (x)) g(x((x)
This rule enforces detailed balance, so Pur is an invariant distribution of the resulting Markov chain. While MALA is commonly used in practice due to its simplicity and gradient-based efficiency, it becomes increasingly inefficient in extremely high-dimensional settings, is typical in image generative models (e-g., 65,536 for FLUX (28). With a fixed step size, its acceptance probability degenerates as d oo. Theoretically, to maintain reasonable acceptance rate, the step size must shrink with dimension, typically the optimal rate of O(d-1/3) [70], which leads to extremely slow mixing and inefficient exploration in extremely high-dimension space.
4.2 Preconditioned Crank-Nicolson Langevin (pCNL) Algorithm
To address high-dimensional sampling challenges (Sec. infinite-dimensional MCMC meth- ods (36 were developed, particularly for PDE-constrained Bayesian inverse problems. These methods remain well-posed even when dimensionality increases. Among them, the preconditioned Crank-Nicolson (pCN) algorithm offers simple, dimension-robus alternative Random Walk Metropolis (RWM), though it fails to leverage the potential function, limiting its efficiency.
To overcome this limitation, the preconditioned Crank-Nicole Langevin (pCNL) algorithm has been proposed (36] 37,, which augments the dimension-robustres of pCN with the gradient-infomma dynamics of Langevin methods (Eq, P). thereby improving sampling efficiency in high-dimensionall settings. The pCNL algorithm employs a semi-implicit Buler (Crank-Nicolson-ttpe) discretization of Langevin dynamics as follows:
(-xx
x xx
r(x011) + Vez, 2~ 2~010, I).
x'
x 2
(12)
a
assuming prior is (N, I) as in our case. This solution, and hence retains the dinension-robusttess of pCN, only when the drift induced by the prior is linear, as with standard Gaussian prior. Therefore, in our setting, it is applicable only at = 1, making it a particularly useful method that aligns with our proposal to sample particles from the posterior distribution Eq. where the prior is the standard Gaussian. With (1 </4)/(1 + c/4), we can rewrite above equation as:
update admits an explicit closed-form
r(Xo11)
x =x=xxx px +
12
2
NO,
(13)
Note that PCNL also adopts MH correction in Eq. 11 to guarantee convergence to the correct target distribution. The pCN algorithm maintains a well-defined, non-zero acceptance probability even in the
limit, allowing the use of fixed step sizes regardless of the dimension [36 37.. This property stems from its prior-preserving proposal, which ensures that the Gaussian reference measure is invariant under the proposal mechanism. This robustness carries over to PCNL, whose proposal inherits PCN's ability handle Gaussian priors in dimension-indepeedeen manner. We include the detailed acceptance probability formulas for MALA and pCNL in the Appendix C

---

(A) Original
Ground
(C) SMC
MMC
-AMMPLER
Figure I: Toy sampling-methie comparison. Each panel visualizes both the initial samples (blue) and their corresponding clean data samples (red). From left to right: (A) samples from the original score- based generative model: (B) the target distribution defined by Eq. (C) results from SMC; (D) results from MALA+SMC; and (E) results from our proposed V-Sampler.
4.3 Initial Particle Sampling
To sample initial particles using MCMC for the subsequent SMC process, we follow standard practices to ensure effective mixing and reduce sample autocorrelation. Specifically, we discard the initial portion of each chain as burn-in [7]] and apply thinning by subsampling at fixed intervals to mitigate high correlation between successive samples. constant step size used across iterations. Although adaptive step size schemes may improve convergence, we opt for fixed-step approach for simplicity. Once the initial particles are sampled, we apply the existing SMC-based method [14] g
Comparison of SMC Initialization in Toy Experiment. In Fig. 1 we present 2D toy experi- ment comparing SMC performance when initializing particles from the prior versus the posterior. We train simple few-step score-based generative model synthetic dataset where the clean data distribution Po is a 6-mode Gaussian Mixture Model (GMM), shown as red dots in Fig (A). The prior distribution is shown in blue, and the gray lines depict sampling trajectories during generation. We define a reward function that assigns high scores to samples from only a subset of the GMM modes, yielding target distribution corresponding optimal initial distribution the posterior at t = (Eq. is shown as blue dots in Fig. (B). We compare (C) standard SMC with prior sampled particles, (D) SMC with posterior samples from MALA, and (E) our wammpleer. All settings use the same total number of function evaluations (NFE). Prior-based SMC (C) uses 100 NFE; MALA+SMC and V-Sampler allocate 50 NFE for MCMC and use fewer particles for SMC. While MALA- initialization method (D) significantly improves alignment with the target distribution (red dots (B)) over prior- based method (C), some modes remain underrepresented. In contrast, V-Sampler (E) provides tighter alignment with the target distribution and the posterior distribution, illustrating its effectiveness in sampling high-quality samples. Full experimental details are provided in Appendix
(Eq. 3 as illustrated in Fig. (B) (red dots). The
5 Experiments
5.1 Experiment Setup
We validate our approach across three applications: layout-to-image generation, quantity-aware generation, and aesthetic-preferent refers to an evaluation metric that is not accessible during generation and is used solely to assess the generalization of the method. Full details for each application are provided in Appendix [D]
image generation. In experiments, the held-out reward
For the layout-to-image generation task, where the goal is to place user-specified objects within designated bounding boxes, use predicted bounding box information from detection model (72 and define the reward as the mean intersection-ove-Unneee bounding boxes. For the quantity-awand image generation task, which involves generating a user- specified object in specified quantity, we use the predicted count from a counting model [73] and detine the reward as the negative smooth LI loss between the predicted and target counts. In both tasks, we include evaluations using held-out reward models to assess generalization. Specifically, for layout-to-ine -o-iaae generation, we report mloU evaluated with different detection model [74] (held-out reward model); for quantity- aware image generation, we report mean absolute error (MAE) and counting accuracy using an alternative counting model 775 (held-out reward model). For aesthetic- prefeeene image generation task, which aims to produce visually appealing images, we use an aesthetic score prediction model (76 as the reward model and use its predicted score as the reward. Across all applications, further evaluate the generated images using ImageReward [77]
(mloU) between the predicted and target
7

---

SMC-Based Methods
Single Particle
Tasks
Metrics
Sumpling from Prior
Sumpling from Posterior
DPS 1
FreeDoM 10
TDS [4]
DAS [2]
Top-K-of-W
ULA
MALA
P-SMMPLER
GroundingDINO
18
0.166
0.177
0.417
0.363
0.425
0.370
0.401
0.467
Layout Image
mioU [4]
0.215
0,229
0,402
0.342
0.427
0.374
0.401
0,471
ImageRewani tN
0,705
0,713
0.962
0,938
0.957
0,388
0.65
1,035
VQA
0.681
0.650
0.794
0.784
0.855
0.783
0.789
0.810
T2u-Count
14.187
15.214
1.804
1,51
1,077
3.085
1,601
0.850
Quantity Aware
MAE [5
15.675
5.3
4,175
3,675
4.825
3.525
2.925
15
0.0
12.5
22.5
25.0
32.5
ImageReward
0.665
0.752
0.743
0.742
0.7%
VQA
0,933
0.941
Aestbetic'
6.310
6,869
6,909
Aesthctic
Preference
ImageReward
1,132
1.100
1,171
VQA
0.959
Table I: Quantitative comparison of AMSLER and baselines across three task domains. Bold indicates the best performance, while underline denotes the second-bos bood-best result for cach metric. Metrics marked with are used as seen reward during reward-guided sampling, where others are held-out reward. Higher values indicate better performance (1), unless otherwise noted ()).
and VQAScore 78). which assess overall image quality and text-image alignment. The baselines and our methods are categorized into three groups:
Single-Particle: DPS [39] and FreeDoMM E0 methods not based on SMC but instead use
single particle trajectory and perform gradient ascent. They are limited in scaling up the search space due to the use of a single particle.
SMC & Initial Particles from Prior: TDS 14 is the SMC-based method we take as the base for our methods. DAS 12 is variant introducing tempering strategy.
SMC & Initial Particles from Posterior: We evaluate four posterior- bosterior-based initialization strate- gies: -ff-K,, ULA, MALA, and V-SAMPILER. ULA and MALA use a small step size (0.05) ensure non-zero acceptance, while V-SAMPLER employs larger step size (0.5) for improved performance (See Sec.5.44. (.4).
We use 25 denoising steps for SMC--ased methods and 50 for single-partice methods compensate for their limited exploration. For SMC-hased methods, we match the total number of function evaluations (NFR) across all methods, allocating half of the budget initial particle sampling for posterior-base methods. We use FLUX (28 as the pretrained score-based generative model. Full experimental details are provided Appendix D
5.2 Quantitative Results
We present quantitative results in Tab Across all tasks, V-SAMPLER consistently achieves the best performance the given reward and strong generalization to held-out rewards. For SMC-based methods, sampling particles from the posterior distribution yields sigmificant improvements over those that sample directly from the prior, highlighting the importance of insforio-inffrmmed initialization. This improvement is particularly notable in complex tasks where high-reward outputs are rare, such layout-to-iman to-iaae generation and quantity- generation. For example, in quantity-aware generation, negative smooth L.1 loss improves from 1.804 with TDS (base SMC) to 1.077 with Tof-K-of-N and further 0.850 with our A/SAMPLER. Similarly, for layout-to-image generation, mloU increases from 0.417 (TDS) to 0.425 with Top-K-off and 0.467 with V/SAMPLER In contrast, initializing with ULA or MALA yields only marginal gains even degraded performance. due to the lack of Metropolis-Hastings correction in ULA and the limited exploration capacity of MALA in high-dimensional spaces. Single-particle methods consistently underperform compared to SMC-based methods.
Ablation Study. We conduct an ablation study that examines how performance varies under different allocations of fixed total NFE between the initial particle sampling stage (via Top-K K-of-K or MCMC) and the subsequent SMC stage; full results and analysis are provided Appendix G
Additional Results Conducted with Other Score-Based Generative Models. We additionally provide quantitative and qualitative results on three applications using another score-based generative model, SANA-Sprint 29 in Appendix
8

---

FreeDoM A8
TDS
DAS PP
Top-K-of-a
ULA
MALA
V-SAMPLER
Layout-to- Image
"A person is sitting on a chair and a bird is on top of a horse while horse is on the top of car."
airplane a balloon are on the ground with a and a car the sky."
82 blueberries"
Quantiyy
21 (A61)
66 (A16)
62 (..20)
73 (A9)
71
63
82 (A0)
"33 coins"
16 (A17)
39 (336
38 (d5)
30 (A3)
22 (&11)
35
33 (20)
Aesthetic
6.351
7,030
6.815
6.925
6.974
7.012
7.423
5.836
7.093
7.161
7.029
7.161
7.098
7.279
Figure 2: Qualitative results for each application demonstrate that V-SAMPLER consistently generates images aligned with the given conditions. Detailed analysis of each case provided in Sec. 5.3

---

Acceptance
GroundinDDNN 72
Salience DETR
74
LPIPS MPD
0:45
Accepance Prob.
PCHL MALA
mlool
PUN MALA
0.30
PCLL NALA
0.51
PCNL
0.21
0.52
MALA
800
00
0.5
12 Step size
1.5
0.37
0.0
06
1.3
135
n.24
0.
36
1,3
13
0.90
06
3n
Step size
Step size
Step size
Figure 3: Performance comparison of MALA and PCNL across different evaluation metries with varying step sizes. Conducted on reyout-to-immas generation application.
5.3 Qualitative Results
We additionally present qualitative results for each application in Fig. 2 generation task, we display the input bounding box locations alongside the corresponding phrases from the text prompt, using matching colors for each phrase and its associated bounding box. In the quantity-awan image generation task, we overlay the predicted object centroids obtained from a held-out counting model [75 to facilitate visual comparison. Below each image, we display the predicted count along with the absolute difference from the target quantity, formatted as (A-). The best-performing case is highlighted in blue. For the aesthetic preference task, we display the generated images alongside their predicted aesthetic scores. The first row corresponds to the prompt "Tiger", and the second to "Rabbit". As shown, V/SAMPLER produces high-quality results across all applications, matching the trends observed in the quantitative evaluations.
For the layout-to-image
From the first and second rows of Fig. [ we observe that baseline methods often fail to place objects correctly within the specified bounding boxes or generate them in entirely wrong locations. For instance, in the first row, most baselines fail to position the bird accurately, and in the second row, none correctly place the car. For quantity-awarn generation, the fourth row shows the counted results corresponding the third row. While U-SAMPLER successfully generates the target number of blueberries in an image, the baselines exhibit large errors-Top-- K-of-N comes closest but still misses some. In rows 5 and 6, only V-SAMPLER correctly generates the target number of coins. In the aesthetic preference task, although all methods produce realistic images, V/-S AMPI.R generates the most visually appealing image with the highest aesthetic score. Additional qualitative examples are provided in the Appendix [
5.4 Evaluation of Initial Partieles
In Fig we compare MALA and pCNL on the layout-to-image generation task across varying step sizes using four metrics: acceptance probability, reward (mloU via GroundingDINO (72), held-out reward (Salience DETR (74) ) and sample diversity (LPIPS MPD 77 All metries directly computed from the Tweedie estimates [18] of MCMC samples, before the SMC stage. As the step size increases, MALA's acceptance probability rapidly drops near-zero, while pCNL maintains stable acceptance probability, Larger step sizes generally improve reward scores, with performance tapering off at excessively large steps. Held-out reward trends mirror this pattern, suggesting that the improvements stem from genuinely higher-quality samples rather than reward overfitting [60] 20 Although LPIPS MPD slightly declines with increasing step size due to reduced acceptance, pCNL at step size 2.0 maintains diversity on par with MALA at 0.05. Additional results for other tasks are included the Appendix
Conclusion and Limitation
We present novel approach for inference-time reward alignment in score-based generative models by initializing SMC particles from the reward-aware posterior distribution. To address the challenge of high-dimensional sampling, we leverage the preconditioned Crank-Nicolson Langevin (pCNL) algorithm. Our method consistently outperforms existing baselines across tasks and reward models, demonstrating the effectiveness of posterior-guides initialization in enhancing sample quality under fixed compute budgets.
Limitations and Societal Impact. A limitation of our approach is that it assumes access to differentiable reward models and depends on accurate Tweedie approximations at early denoising steps. Also, while our method improves fine-grained control in generative modeling, it may also be misused to produce misleading or harmful content, such as hyper-realistic fake imagery. These risks highlights the importance of responsible development and deployment practices, including transparency, content verification, and appropriate use guidelines.
10

---

7 Acknowledgements
This work was supported by the NRF of Korea (RS-2023-002097223) ITTP grants (RS-2019- 11190075, 11--2022-11220094 RS-2023-00227552, RS-2022-000399817, 2025-25443318, R8-2005-026531133;; and the Technology Innovation Program (RS-2025-02317326) all funded by the Korean government (MSIT and MOTIE), well as by the DRB-KAIST SketchThe- Future Research Center.
RS-
References
[1] OpenAL.
Introducing OpenAl 03 and 04-mini.
https://openaa..oom/neee
introducing- o3- and- 04- mini/ 2025.
[2] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et DeepScek-RI: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
[3] Jascha Sohl-Dicke Dickkstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICMI,, 2015.
[4] Yang Song, Jascha bohl-Diccstten, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poolc. Score-based generative modcling through stochastic differential equations. In ICLR, 2020.
[5] Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurlPS, 2020.
[6] Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: unifying framework for flows and diffusions. arXiv preprint arXiv:2303.0875 2023.
[7] Nanye Ma. Mark Goldstein, Michael Albergo, Nicholas M. Boffi, Eric Vanden- Bijden--ijnden, and Saining Xie. SiT: Exploring flow and diffusion- generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.0877 2024.
[8] Masatoshi Uehara. Yulai Zhao, Chenyu Wang, Xiner Li, Aviv Regev. Sergey Levine, and Tommaso Biancalani. Inference-time alignment in diffusion models with reward-guided generation: Tutorial and review. arXiv preprint arXiv:2501.088 009685, 2025.
[9] Amaud Doucet, Nando De Freitas, Neil James Gordon, et al. Sequential Monte Carlo methods in practice, volume 1. Springer, 2001.
[10] Pierre Del Moral, Arnaud Doucet, and Ajay Jasra. Sequential monte carlo samplers. Journal of the Royal Statistical Sociery Series B: Statistical Methodology, 68(3):411-436, 2006.
[11] Nicolas Chopin, Omiros Papaspiliopoulos, et al. An introduction to sequential Monte Carlo, volume 4. Springer, 2020.
[12] Sunwoo Kim, Minkyu Kim, and Dongmin Park. Test-time alignment of diffusion models without reward over-optimizaaioon In ICI,R 2025.
[13] Jaihoon Kim, Tachoon Yoon, Jisung Hwang, and Minhyuk Sung. Inference-time scaling for flow models via stochastic generation and rollover budget forcing. arXiv preprint MxXiv:2503319336 2025.
[14] Luhuan Wu, Brian L. Trippe, Christian A Naesseth, John Patrick Cunningham, and David Blei. Practical and asymptotically exact conditional sampling in diffusion models. In NeurlPS, 2023.
[15] Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In ICLR, 2024.
[16] Anuj Singh. Sayak Mukherjee, Ahmad Beirami, and Hadi J. Rad. CoDe: Blockwise control for denoising diffusion models. Transactions on Machine Learning Research, 2025.
[17] Xiner Li. Yulai Zhao, Chenyu Wang, Gabriele Scalia, Gokcen Eraslan. Surag Nair. Tommaso Biancalani, Shuiwang Ji, Aviv Regev, Sergey Levine, and Masatoshi Uchara. Derivative-Fo free guidance in continuous and discrete diffusion models with soft salue-based decoding, arXiv preprint 240.:2408.08255 2024.
[18] Bradley Efron. Tweedie's 's formula and selection bias. Journal of the American Statistical Association, 106(496):1602 1604-116,66 2011.

---

[19] Masatoshi Uehara, Yulai Zhao. Tommaso Biancalani, and Sergey Levine. Understanding reinforvement learning-based fine-tuning of diffusion models: A tutorial and review. arXiv preprint arXiv:2407.13734 2024.
[20] Masatoshi Uchara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M Tseng, Tommaso Biancalani, and Sergey Levine. Fine-tuning of continuous-time diffusion models as entropy-regularized control. arXiv preprint arxiv:2402. 15194, 2024.
[21] Brian L.. Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi S. Jaakkola. Diffusion probabilistic modeling protein backbones in 3D for the scaffoliai problem. In ICLR, 2023.
[22] Gabriel Cardoso, Yazid Janati el idrissi, Sylvain Le Corff, and Eric Moulines, Monte Carlo guided denoising diffusion models for Bayesian linear inverse problems. ICLR, 2024.
[23] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency Models. In JCML, 2023.
[24] Dongjun Kim, Chich-HSse Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency Trajectory Models: Learning probability flow ODE trajectory of diffusion. In ICI.R, 2024.
[25] Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. In ICLR, 2025.
[26] Axel Sauer. Frederic Boesel, Tim Dockhorn. Andreas Blattmann, Patrick Esser, and Robin Rom- bach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. arXiv preprint arXiv:2403. 12015, 2024.
[27] Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023.
[28] Black Forest Labs. FLUX. https://gi
cwwcccoc//////// hoop///aaa-ooootttooooao labs/flux 2024.
[29] Junsong Chen, Shuchen Xue, Yuyang Zhao, Jincheng Yu, Sayak Paul, Junyu Chen, Han Cai, Enze Xic, and Song Han. SANA-Sprint: One-step diffusion with continuous-time consistency distillation. arXiv preprint arXiv:2503. 09641, 2025.
[30] W. K. Hastings. Monte carlo sampling methods using markov chains and their applications. Biometrika, 57(1):97-109, 1970.
[31] Gareth O. Roberts and Richard L. Tweedie. Exponential convergence of langevin distributions and their discrete approximations. Bernoulli, 2(4):341 - -363, 1996.
[32] Gareth O. Roberts and Osnat Stramer. Langevin diffusions and metropolis-hastings algorithms. Methodol- ogy and computing in applied probability, 4:337-357, 2002.
[33] Simon Duane, Anthony Kennedy, Brian Pendleton, and Duncan Roweth. Hybrid monte carlo. Physics letters B, 195(2):216-222 16222 1987.
[34] Mark Girolami and Ben Calderhead. Riemann manifold langevin and hamiltonian monte carlo methods. Journal of the Royal Statistical Society Series B: Statistical Methodology, 13(2):223-144, 2011.
[35] Steve Brooks, Andrew Gelman, Galin Jones, and Xiao-Li Meng. Handbook of Markov Chain Monte Carlo. Chapman and HaI//CRC, May 2011.
[36] S. L.. Cotter, G. O. Roberts, A. M. Stuart, and D. White. MCMC methods for functions: Modifying old algorithms to make them faster. Statistical Science, 28(3), August 2013.
[37] Alexandros Beskos, Mark Girolami, Shiwei Lan, Patrick E. Farrell, and Andrew M. Stuart. Geometric MCMC for nifintee-immensional inverse problems. Journal of Computational Physics, 335:327-351, April
2017.
[38] Alexandros Beskos, Gareth Roberts, Andrew Stuart, and Jochen Voss. MCMC methods for diffusion bridges. Stochastics and Dynamics, 08(03):319 9500 2008
[39] Hyungjin Chung, Jeongsol Kim. Michael Thompson Mecann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In ICLR, 2023.
[40] Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang, FreeDoM: Training-free free energy- guided conditional diffusion model. In ICCV, 2023.
12

---

[41] Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Roni Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In ICLR, 2024.
[42] Haotian Ye, Haowei Lin, Jiaqi Han, Minkai Xu, Sheng Liu, Yitao Liang, Jianzhu Ma, James Zou, and Stefano Ermon. TFG: Unified training- free guidance for diffusion models. In NeurIPS, 2024.
[43] Kimin Lee. Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text-to-image models using human feedback. arXiv preprint arXiv:2302.122 1212, 2023.
[44] Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: solid benchmark for evaluating human preferences text-to-image synthesis. arXiv preprint 23X.::333300034.44 2023.
[45] Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. In ICLR, 2024.
[46] Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Moham- mad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Reinforcement learning for fine-tuning text-to-image diffusion models. In NeurlPS, 2023.
[47] Bram Wallace, Meihua Dang. Rafael Rafailov, Lingi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong. Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In CVPR, 2024.
[48] Kai Yang. Jian Tao, Jiafei Lyu, Chunjiang Ge. Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. In CVPR, 2024.
[49] Kevin Clark, Paul Vicol, Kevin Swersky, and David J. Fleet. Directly fine-tuning diffusion models on differentiable rewards. In ICLR, 2024.
[50] Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to-image diffusion models with reward backpropagation. arXiv preprint arXiv:2310.03739, 2024.
[51] Wenpin Tang. Fine-tuning of diffusion models via stochastic control: entropy regularization and beyond. arXiv preprint (tl.::24030062299 2024.
[52] Carles Domingo-Enrin Michal Drozdzal, Brian Karrer, and Ricky T. Q. Chen. Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. In ICLR, 2025.
[53] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Lee. Flow matching for generative modeling. In ICLR, 2023.
[54] Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky Q. Chen, David Lopez-Paz, Heli Ben-Hamu, and Itai Gat. Flow matching guide and code. arXiv preprint arXiv:4412.06664, 2024.
[55] Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, and Sergey Levine. Reinforcement learning with deep cncrgy-based policies. In ICMI,, 2017.
[56] Matthieu Geist, Bruno Scherrer, and Olivier Pietquin. A theory of regularized markov decision processes. In ICMI, 2019.
[57] John Schulman, Xi Chen, and Pieter Abbcel. Equivalence between policy gradients and soft q g-leamiig. arXiv preprint arXiv: 1704.064004 2017.
[58] Gergely Neu, Anders Jonsson, and Vicenç Gómez. A unitied view of regroiy-regulaiized markov decision processes. orXiv preprint / 705.07798, 2017.
[59] Sergey Levine, Reinforcement learning and control as probabilistic inference: Tutorial and review, arXiv preprint arXiv: 00909, 2018
[60] Masatoshi Uehara, Yulai Zhao, Ehsan Hajiramezanall, Gabriele Scalia, Gökcen Eraslan, Avantika Lal, Sergey Levine, and Tommaso Biancalani. Bridging model- based optimization and generative modeling via conservative tnc-tuning of diffusion models. In NewrIPS, 2024.
[61] Nicolas Chopin. Central limit theorem for sequential monte carlo methods and its application to bayesian inference. The Annals of Statistics, 32(6), December 2004.
13

---

[62] Pierre Moral. Feynmun- Kuc formuuaaae:: genealogical and interacting purticle systems with applications. Springer, 2004.
[63] Beomsu Kim and Jong Chul Ye. Denoising MCMC for accelerating diffusion-based generative models. arXiv preprint wwXiv:2209.44588 2022.
[64] Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. In ICML, 2023.
[65] Lin Gui, Cristina Garbacea, and Victor Veitch. BoNBon alignment for large language models and the sweetness of best-of-n sampling. In NeurlPSS 2024.
[66] Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. WebGPT: Browser-assisted auestion-answering with human feedback. arXiv preprint arXiv:2112.09332 2022.
[67] Ahmad Beirami, Alekh Agarwal, Jonathan Berant, Alexander Amour, Jacob Eisenstein, Chirag Nagpal, and Ananda Theertha Suresh. Theoretical guarantees on the best-of-n alignment policy. arXiv preprint arXiv:2401.0rr 2024.
[68] Joy Qiping Yang. Salman Salamatian, Ziteng Sun, Ananda Theertha Suresh, and Ahmad Beirami. Asymp- totics of language model alignment. In 2024 IEEE International Symposium on Information Theory (ISIT). pages 2027-2032, 2024.
[69] Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087-1092 1953.
[70] Gareth O. Roberts and Jeffrey S. Rosenthal. Optimal scaling of discrete approximations langevin diffusions. Journal of the Royal Statistical Society. Series B (Statistical Methodology). 60(1):255-268, 1998.
[71] Charles Geyer. Practical markov chain monte carlo. Statistical Science, 7(4):473-483, 1992.
[72] Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang. Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding DINo: Marrying DINO with grounded pre-training for open-set object detection. In ECCV, 2024.
[73] Yifei Qian, Zhongliang Guo, Bowen Deng, Chun Tong Lei, Shuai Zhao, Chun Pong Lau, Xiaopeng Hong, and Michael P Pound. T21Count: Enhancing cross-modal understanding for zero-shot counting. In CVPR, 2025.
[74] Xiuquan Hou, Meigin Liu, Senlin Zhang, Ping Wei, and Badong Chen. Salience DETR: Enhancing detection transformer with hierarchical salience filtering refinement. In CVPR, 2024.
[75] N. Amini-Naieni, T. Han, and A. Zisserman. CountGD: Multi-modal open-world counting. In Neur!PS,
2024.
[76] Christoph Schuhmann. Laion aesthctic predictor. http://aaa...aaabbo///laooaaaaas aesthetiis
2022. Accessed: 2024-09-29.
[77] Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. ImageReward: learning and evaluating human preferences for text-to-image generation. In NeurlPS, 2023.
[78] Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. In ECCV, 2024.
[79] Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In CVPR, 2018
[80] Eslam Mohamed Bakr, Pengzhan Sun, Xiaoqian Shen, Faizan Farooq Khan, Li Erran Li, and Mohamed Elhoseiny. HRS-Bench: Holistic, reliable and scalable benchmark for text-to-image models. In ICCV, 2023.
[81] OpenAL GPT-4o system card. arXiv preprint arXiv:2110.22222220 2024.
[82] Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang. Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. In CVPR, 2023.
14

---

[83] Jiayu Xiao, Liang Li, Henglei Lv, Shuhui Wang. and Qingming Huang. R&b: Region and boundary aware zero-shot grounded lext-to-image generation. In ICLR, 2024.
[84] Quynh Phung. Songwei Ge, and Jia-Bin Huang. Grounded text-to-image synthesis with attention refocusing. arXiv preprint arXiv:2306.05422 2023. 2023.
[85] Yuseung Lee, Toelooo Yoon, and Minhyuk Sung. Groundit: Grounding diffusion transformers via noisy patch transplantation. In NeurIPS, 2024.
[86] Wonjun Kang, Kevin Galim, Hyung II Koo, and Nam Ik Cho. Counting guidance for high fidelity text-to-image synthesis. In WACV, 2025.
[87] Lital Binyamin, Yoad Tewel, Hilit Segev, Eran Hirsch, Royi Rassin, and Gal Chechik. Make it count: Text-to-image generation with an accurate number of objects. arXiv preprint arXiv:2406.10210, 2024.
[88] Oz Zafar, Lior Wolf, and Idan Schwartz. Iterative object count optimization for text-to-image diffusion models. arXiv preprint arXiv:2408.I / 1/721, 2024.
15

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
16

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: We do not contain new theorems rather provide toy experiments results to support our claim as well as main experiment results.
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibilityy which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Our datasets are based on combination of existing resources, including subset of the HRS-Spatial dataset [80]. GPT-4o-generated samples [8], and the animal dataset from 45
We plan to release both the code and data future revision.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g.. for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7 7/nips. wwwpcccccccc......c...cco
hwublicgguiids/// guwdge/eee/eeee (oodeSubisssii sssssnooiiiiyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
18

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting//etails
Question: Does the paper specify all the training and test details (e-g.. data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Due to limited time and computational resources, we were unable to conduct a full statistical significance analysis prior to submission. However, plan incorporate appropriate statistical evaluations, such as standard deviations and confidence intervals, as well additional ablation studies future revision.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emorr bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
show in tables
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
19

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:////waarpppsccccppp///teeia
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
20

---

Justification: Our work does not involve the release of any models or datasets that pose high risk of misuse. The models used our experiments are based on publicly available architectures, and our dataset construction process avoids the inclusion of any sensitive or harmful content.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version URL.
asset used and, possible, include
The name of the license (c.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, cettasets.paperswttt paperswi thcode. com/ datasets
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Although our dataset construction includes set of GPT--o-ggeeraiie samples, which may be considered new asset, this portion has not been publicly released the time of submission. As such, we do not consider any part of our submission to constitute released new asset for the purposes of this checklist item. We plan to release new data in future revision.
Guidelines:
The answer NA means that the paper does release new assets. Researchers should communicate the details the dataset/codeemoodel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
21

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve any crowdsourcing tasks or experiments with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This research does not involve human participants and therefore does not require IRB or equivalent ethical review.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: We used LLMs such as GPT-4o 81 for auxiliary purposes including dataset generation, but they were not involved in the development of the core methodology or experiments.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy /ttp::///baarppppcccc///oeerrree for what should or should not be described.
22

---

Appendix
A Reward Alignment with Stochastic Optimal Control
which our method builds upon, Uehara et al. (20) introduce both an additional drift term e (often referred to as control vector field) and modified initial distribution P1- Then the goal is to find & and P1 such that the resulting
In the reward alignment task for continuous- generative models 22 52
final distribution at time t = 0 matches the target distribution Pn defined in Eq. 3 Accordingly, the original temerse-time SDE used for generation (Eq. D is replaced by controlled SDE:
dx, (f(x4. t) (x,, t)) dt g())dW,
X1 ~ P1-
(14)
This entropy-regularized stochastic optimal control framework adopts pathwise optimization that integrates KL divergence penalties over trajectories and thus optimization formulation of Eq. changes accordingly:
Pi arg max Epw.rr [r(xo)] o:Dia. (PV. [pchii]]
(15)
where pela1 is measure trajectories induced by the controlled SDE in Eq. 14 and Pddata is measure over trajectories induced by the pre-trained SDE in Eq. [
| (²(1)
KL-divergence term in Eq. 15 can be expressed as the sum of affine control cost
and log
(x(x)) Radon-Nikodym derivative at 1, i.e., log (x(1)
1 f(' t==
[xb(,,,))) t)l? g²(()
P1 (x1)
3 Pi
arg max Ep [r(x))] - a.ppp..
dt + log pi (x1)
(16)
8.P1
which can be proved 22 [2) using Girsanov theorem and martingale property Itō integral. The optimal control n and the optimal initial distribution pi can be derived by introducing the optimal value function, defined as:
a [
[Ibb(x,, s)1l == g2(s)
(x+) max Ep r(xo)
ds Xe
(17)
2
where the expectation is taken over trajectories induced by the controlled SDE in Eq. 14 with current X₃ is given.
From the optimal value function at = 1, we can derive explicit formulation of the optimal initial distribution pi in terms of (xx) by plugging the definition of optimal value function
(Eq. [7) into Eq. 16
max Ep, (V:) - aDDL 17 lP
(18)
F
Solving this yields the following closed-form expression for the optimal initial distribution (derivable via calculus of variations 13). similarly to Eq. 3
Vi (x()
pi(x)) = Zi Pi (xi) exp
(19)
The optimal control g can be obtained from the Hamilton-Jacee Saan--Jaaoi---Bellaaae Bellman (HJB) equation and is expressed in terms of the gradient of the optimal value function:
(x1, = (1) (x.)
(20)
Moreover, the optimal value function itself admits an interpretable closed-form expression via the Feynman-Kac formula:
r(xo)
(xx) log Eptr
exp
(21)
23

---

Importantly, Uehara et al. (20 further proved that the marginal distribution pi (xx) induced by the controlled SDE (with optimal control 1p" and optimal initial distribution pi) is given by:
Vi (xx(
(x.) 7
(x.) exp
(22)
where (x() is the marginal distribution of the pretrained score-based generative model at time t. Similarly, the optimal transition kernel under the controlled dynamics is:
Po(x exp(V2-A(xx pe Po
(23)
exp(V exp(V((x.//a
where po (xxx-Ax( denotes the transition kernel of the pretrained model, i.c., corresponding to the discretization of the reverse-time SDE defined in Eq. [ For detail derivation, see Theorem I and Lemma in Uehara et al. [20]. Notably, Eq. [2 implies that by following the controlled dynamics defined by Eq. [4] initialized with the optimal distribution pi and guided by the optimal control s², the resulting distribution time = will match the target distribution p defined in Eq
Note that the optimal control, optimal initial distribution, and optimal transition kernel are all expressed in terms of the optimal value function. However, despite their interpretable forms, these expressions are not directly computable in practice due to the intractability of the posterior p(xo (x)).. This motivates the use of approximation techniques, most notably Tweedie's 's formula (18), which widely adopted in the literature [39 42
14 12 to make such expressions tractable. Under this approximation, the posterior is approximated by a Dirac-delta distribution centered at the posterior mean denoted by Xoje == Exo-po [xo] representing the conditional expectation under Poja = xx)) Consequently, the optimal value function simplifies to:
(x,) log / exp r(x0)
(x(x)) 8(x0 xojo)xxo r((oj)), p(xolx))dxx a log / exp (-(x))
(24)
where Xnje is a deterministic function of x4. Using this approximation, we have following approxima- tions for the optimal initial distribution Pi. the optimal control v. and the optimal transition kernel po which are used throughout the paper:
r(x0
p = 7, Pı (x1) exp
(25)
3 (x+) = 9² r r(Xo)e)
(26)
pd (x exp(r(xn
exp(r(xo::\/a) po (x(xxxx(
(27)
worth noting that sampling from the optimal initial distribution is essential theoretically guarantee convergence to the target distribution Eq. 3 Simply following the optimal control alone does not suffice and can in fact bias away from the target, phenomenon known as the value function
bias problem [52]. To the best of our knowledge, this is the first work to explicitly address this problem in the context of inference-time reward aligmmmet with score-based generative models.
B Sequential Monte Carlo and Reward-Guided Sampling
Sequential Monte Carlo (SMC) methods 9-111 also known as particle filter, are class of algo- rithms for sampling from sequences of probability distributions. Beginning with K particles drawn independently from an initial distribution, SMC maintains a weighted particle population, (x(") ( Ji=1+ and iteratively updates it through propagation, reweighting, and resampling steps to approximate the target distribution. During propagation, particles are moved using a proposal distribution; in the reweighting step, their importance weights are adjusted to reflect the discrepancy between the target and proposal distributions; and resampling preferentially retains high-weight particles while

---

eliminating low-weight ones (this is performed only conditionally, see below). The weights are updated over time according to the following rule:
(1) Pur (x Xx) (8)
q(x(_e))X) Xx) W/
(28)
where Puar is an intermediate target kernel we want to sample from, and q(x, is a proposal kernel used during propagation.
At each time t, if the effective sample size (ESS), defined as (Σ w Σ
falls
below predefined threshold, resampling is performed. Specifically, a set of ancestor indices {az 1 K ==
is drawn from a multinomial distribution based on the normalized weights. These indices
then used form the resampled particle {x(" (ol ) RV=1- If resampling not triggered,
simply set dz = i.
In the propagation stage, particle set fx f HK is generated via sampling from proposal distribution,
Ar (xx At x (al"). When resampling is applied, the weights are reset to uniform values, i.e., Wy =1 for all i. Regardless of whether resampling occurred, new weights (w 1.K are then computed using Eq. 28
In the context of reward-alignmen target distribution defined in Eq. 3 As the number of particles K grows, the approximation becomes increasingly accurate due to the consistency of the SMC framework [1 62). To make this effective,
tasks, SMC can be employed to approximately sample from the
the proposal kernel should ideally match the optimal transition kernel given in Eq. 23 However, as discussed in Appendix A this kernel is computationally intractable. Therefore, prior work [14 12
typically resorts its approximated form, as expressed in Eq. 27 This leads the weight each time being computed as:
16 Xx) (1) expp w tt q(x1-11 xx) W,
(29)
cxp(r(xon))//)g((xxx
where Pe(Xe-á:|x) (x) denotes the transition kernel of the pretrained score-based generative model. The pretrained model follows the SDE given in Eq. which upon discretization yields a Gaussian transition kernel, (x xx,x = Nxxx f(x,, t) At. (() AtI). On the other hand, for reward- guided sampling. i.c., to sample from the target distribution in Eq. 3 we follow controlled SDE in Eq. [4] At each intermediate time, the SOC framework (Appendix A) prescribes the use of the optimal control defined in Eq. 20 However, due to its intractability, the approximation in Eq. 26 is typically adopted in practice. Discretizing the controlled SDE under this approximation leads the following proposal distribution at each time:
(xx=-x( = -Xx - f(x,, 1) + (() ((OV((X)) T(Xole) AI, (()))
(30)
similar proposal has also been used in 12 8), where Taylor expansion was applied the context of entropy reguua-regul Markov Decision Process.
C Acceptance Probability of MALA and PCNL
30 acceptance rule that under- and the preconditioned Crank-Nicolson Langevin algorithm (pCNL) (36 37). Metropolis- Haaropolis-Hastings. algorithms form a class of MCMC methods that generate samples from target distribution by accepting rejecting proposed moves according to specific acceptance function. Let denote density proportional to the target distribution. Given the current state x and proposal x' g(x'Ix), with probability:
In this section we provide the Metropolis- CitopoolssHHastigss Hastings (MH)
69
pins both the Metropolis-Adjustee Langevin Algorithm (MALA) 31
32
(x). the MH step accepts the move
axx,x) = min (1,
( Perr (x') (x(x(xx)
(31)
Pear
the proposal kernel q(x') (x) taken to be the one-step Marraaaa discretization Langevin dynamics then it becomes the MALA, and Eq. 31 corresponds to the acceptance probability of
25

---

MALA. Choosing instead the semi- impiiii Siink--Niolson-typo discretization yields the proposal used in the pCNL, and Eq. 31 becomes the corresponding pCNL acceptance probability.
We first show that preconditioned Crank-Nicolson (pCN), which is modification of the Random- Walk Metropolis (RWM), preserves the Gaussian prior. pCN can be viewed as special case of pCNL obtained when the underlying Langevin dynamics is chosen so that the Gaussian prior N I) its invariant distribution. This leads to the proposal mechanism 36 371
= px
p²z,
2 ~ (O,
(32)
where p= (1 c/4)/(1///+ c/44 with e > 0 corresponding to the step size of the Langevin dynamics. Assume that the prior is the standard Gaussian (O, I) and let (O, I), then Eq. expresses x' as linear combination of two independent Gaussian random variables with unit covariance. Hence, by the closure of the Gaussian family under affine transformations, x' (O, well, thus preserving the Gaussian prior.
symmetric, i.e., p1 (x') (u (x(x') p1 (x) (x' x). where P1 (-) denotes Gaussian prior and (-(-))) denotes the proposal kernel of the pCN, i.e., Eq. 32
Next, in the case of pCN, Pı (xx))o
Remark 1. Let (.(.) as Gaussian prior (0,, and (o(-1-) as the proposal kernel of the pCN, i.e., N(x, (1 I²),,, then pı (xxx (xx(x) = (x))
Proof. Apart from normalization constants, p1 (xx))x((((())) can be calculated as:
x²2 exp
(x (x-)x')²
x²2 + x² 2pxxx
2
exp
2(1 -1-p5 p2)
exp
2(1 - p²)
Repeating the same calculation with p1 (x)90 (x' (x|xx) merely swaps x and x', leaving the numerator unchanged.
Hence the two products are identical.
We provide additional remark for ease of calculation.
Remark 2. Let (x:; C) be the density of a multivariate Gaussian with mean and positive- definite covariance C. For fixed C, the ratio of two such densities that differ only in the mean is
N(x; nt C)
exp (-aluelé: + (1,x):))
N(x; C)
where |11. := 1r
and (p, x)c == 1i' x- -C C-2
In our case, C corresponds to the identity matrix.
Acceptance Probability of pCNL. As before, let be proposal kernel of the pCN (Eq. 32) and dp be proposal kernel of the pCNL:
(0(x((x)= px +
p2z,
z N(O.I)
(33)
Ve v r(xo
(x(xx)) px
p²
z
z ~ N(01).
(34)
2
Let x = x'-xx and fo, Tp be the distributions of x under qo and gp, respectively. Then we obtain: VI-p²
(x(xx = (Xx; 0,I)
(35)
VE r(Xo11) r 2
(x|x) = NX;;
I).
(36)
Note that Xnj1 a function x. Then by Remark
(x xx)
(x(x)
e
r (Xoj1)
VV
r(Xo11)
)
9n [x'xx xx)
To (x
exp
2
(37)
26

---

For the fraction part of the acceptance probability (Eq. of pCNL, we have:
(pi (x
(38)
(x(x) (xf()) (xxxxxxxx))))pxxxx)x))
(39)
(x)())g x)) (2)
(x,xxx))
where the target distribution is set Eq. 25 In Eq.
38
divide both numerator and denominator
by a common term, and in Eq. 39 Denominator can be calculated utilizing Eq.
we uti ized Remark
pi pi(x) ()) (x (xl
xx)(xx))=
(40)
pi())qu(x111 (x/xx)
((xo11) exp
r(oo11)
Ve
r(xo)1)
V1 p² ),)
x' px
exp
8 V
(41)
a
a
2
a
with numerator being simply interchanging x and x'. The acceptance probability of pCNL is
min
[1 (,(x'xx)
(,(xxx))
Acceptance Probability of MALA. In the case of MALA, the proposal given as:
x x C V Par Plar
+ Vez,
z z~ (
(42)
2
c - 2
x (-x+
r(xo 1)
+ Vez,
~~000
(43)
where in PCNL we set the target distribution as Eq. can be expressed as:
25
Thus the proposal kernel of MALA QM
(x'lx) = N (x': x x(1 2 2
2 +
< r(xo
el
(44)
The fraction part of the acceptance probability (Eq. of MALA is given as:
pi('' JMM N(x'; 0, exp(r(22) x'(1 c/2) + c/2 Vr(x, /a
(45)
(x'
N(x; 0, I, (_I])xppr(xx x(1 - c/2) + c/2 (r(xo )/a,c)
PM (M'xx'x x)
(46)
PM x, x')
where we denote X := x (x'), i.e., caleulate Tweedie's formula with x'. Thus the denomina- tor, which is AM (x, x'), is proportional to the following expression:
X
((Xo)))
|X
{x (1
+
exp
(47)
2c
After simplifying the expression-specifieee canceling out the cross terms involving x and x' that will appear symmetrically in the numerator and denominator-the expression for (x,(x, (') becomes:
PM D,x(xx,) x')
(48)
(r(xo11)
r (xxo 1)
V r(xo11) (x' (x'-(1- 2
) (49)
|x|| +
= exp
exp
a
a
The numerator (x x) can be obtained by simply interchanging x and x'. The acceptance (x'xx probability of MALA is then, min (1, Par (x,x))
27

---

D Experimental Setup and Details
In this section, we provide comprehensive details for each application: bayout-to-image generation, quantity-aware image generation, and resthetic-preference image generation. We also include full experimental details.
Layout-to-Image Generation. This task involves placing user-specified objects within designated bounding boxes 882 [5]. We evaluate performance on 50 randomly sampled cases from the HRS- Spatial [0] dataset. As the reward model, we use GroundingDINO [72), and measure the alignment between predicted and target boxes using mean Intersection-over- Union (mloU). For the held-out reward, we compute mloU using a different object detector, Salience DETR
74
Quantity-Aware Image Generation. This task involves generating a user-specified object in specified quantity 88-888 We evaluate methods on custom dataset constructed via GPT-4o [8]] comprising 20 object categories with randomly assigned counts up to 90, totaling 40 evaluation cases. As the reward model, we used T21Count (73.. which takes a generated image and the corresponding text prompt as input and returns density map. Summing over this density map yiclds a differentiable estimate of the object count Rpred- The reward is defined as the negative smooth LI loss:
0..5 T2
[mrrd ngr < 1,
Poount
[Mpred Ngal + 0.5 [Mpred "gal 2 1.
where fgg denotes the input object quantity. For the held-out reward, we used an alternative counting model, CountGD (75]. This model returns integer-valued object counts. We apply confidence threshold of 0.3 and evaluate using mean absolute error (MAE) and counting accuracy, where a prediction is considered correct if Upred
Tltt-
Aesthetic-Prefeeee Preference Image Generation This task involves generating visually appealing images. We evaluate performance using 45 prompts consisting of animal names, provided in (45). As the reward model, we use the LAION Aesthetic Predictor V2 (76), which estimates the aesthetic quality of an image and is commonly used in reward-alignment literature [12 [1
[2] 60 45).
Common Held-Out Reward Models. For all applications. we additionally evaluate the gener- ated images using widely adopted held-out reward models for image quality and text alignment. Specifically, we use IeageReward (77). fine-tuned on human feedback, and VQAScore (78), which leverages a visual question answering (VQA) model. Both are based on vision- langugge models and assess how well the generated image aligns with the input text prompt.
Experimental Details. We use LLUX-Schnnll 28 the score- baser- generative model for our method and all baselines. Although FLUX is a flow-based model, SMC-based inference-time reward alignment can be applied by reformulating the generative ODE as an SDE [7]. as described in Sec. 3.1 3.T Further to ensure diversity between samples during SMC, we applied Variance Preserving (VP) interpolant conversion [3 [54] Apart from FLUX, we additionally report quantitative and
qualitative results using another score- score-based generative model, SANA-Sprint (29), in Appendix [H] These results demonstrate that the our claims are not tied to a specific model architecture, and additionally highlighting the robustness of V-SAMPLER.
We use 25 denoising steps for all SMC-based methods and 50 for single-partichl methods to compen- sate for their reduced exploration capacity. To ensure fair comparison, we fix the total number of function evaluations (NFE) across all SMC variants- generation tasks, and 500 for cesthetiic-refereee generation, we used half the NFE compared to other tasks because we found it sufficient for perfor- mance convergence. For methods that sample from the posterior, half of the NFE is allocated the initial sampling stage, resulting in 20 particles during SMC, while prior-based methods use all NFE for SMC with particles. For the besthetic-prefereen
for layout-to-image and quantity-awand aawae image
image generation. For resthetic-prefereee
image generation, we use half the number of particles in both settings to reflect the halved NFE. The ablation study comparing the performance on varying NFE allocation is provided in Appendix G For all experiments, we used NVIDIA A6000 GPUs with 48GB VRAM.
E
Toy Experiment Setup and Details
In this section, we describe the setup for the toy experiment presented in the main paper. Specifi- cally, the data distribution Po is six-mode Gaussian mixture with covariance 0.3I, consisting of
28

---

six equally weighted components uniformly arranged on a circle of radius 6. The reward func- tion defined as the sum of three Gaussian components centered points evenly spaced on the circle: r(x)= = Σ exp (-0.5)lz - p:ll²), where pl = (6.5, 0), pl = (-3.25, 3), and Ps - (-3.25, 33.2533). We trained a few-step score-based generative model using 4-layer MLP with hidden dimension 128. For sampling, we fixed the total NFE to 100 across all methods and visualized the results using 2,000 generated samples. We used step size 0.1 for MALA and 0.2 for PCNL. We deliberately used smaller step size for MALA than for pCNL to better reflect the actual experimental settings used in the main experiments (Sec. 5.1.
F Additional Evaluation Results of MALA and pCNL Initializations under Varying Step Size
We presented a comparison between MALA and pCNL under varying MCMC step sizes for the layout-to-image generation task in the Sec. [.4] Here, extend this analysis to the remaining two tasks. In Fig. 4 we report results for quantity-awan generation (top row) and besthetic-preferene generation (bottom row), comparing MALA and pCNL across range of step sizes.
image generation task, we report acceptance probability, reward (negative smooth LI loss via (33) held-out reward (mean absolute error (MAE) via CountGD (75), and LPIPS Mean Pairwise Distance (MPD) (79 (which measures the sample diversity). For the aesthetic-peeference task, report acceptance probability, reward (aesthetic score (76)), and LPIPS
For the quantity-
MPD [79]. All metrics are directly computed from the Tweedie estimates [8] of MCMC samples,
before the SMC stage.
As the step size increases, MALA's acceptance probability quickly falls to near zero, whereas PCNL retains a stable acceptance rate across wider range. In quantity- image generation, PCNL achieves its best T2I-Count reward and lowest MAE at moderate step sizes (approximately 0.5-1.0), while MALA's performance deteriorates sharply beyond 0.05. Although pCNL's LPIPS MPD decreases at larger step sizes, it consistently outperforms MALA in diversity across the same settings. In aesthetic-preference generation, pCNL generally achieves higher aesthetic scores, with only a slight drop at the smallest step size, and maintains higher LPIPS MPD across all settings. In contrast, MALA's aesthetic scores decline rapidly once its acceptance rate vanishes.
These results demonstrate that pCNL can effectively leverage larger step sizes to improve sample quality, with only minimal tradc-offss in diversity.
Quantity-aware Generation
Acceplance
T2I-Count 7A
MAE 75
I.PIS MPD 79
Accoptmne Prob. (1)
12.5
T21 Count (T)
LIWS MPDD (1)
MAE (3)
2.10
0.00
15
0.5%
0.
0.1
06
1.2
18
00
12
Step size
Step size
Step size
Step size
Aesthetic-prfferncce Generation
Acceptance
Aesthetic Score 76
LPIPS MPD 79
Acceptance Prob. (1)
0.45
Aesthetic Score (1)
6.60
LPIPS P MPD (1)
0.540
0.30
PCNL
6.45
PCNL
PNNL
MALA
0.525
0.15
0.510
0.00
0.0
0.6
1.2 Step size
1.8
0.0
0.6
1.8
Step size
Step size
Figure 4: Performance comparison of MALA and pCNL across different evaluation metrics under two generation settings: quantity-aware generation (top row) and aesthetic-preferencee generation (bottom row). Each graph illustrates the performance trend with varying step sizes.
29

---

G
Ablation Study: Stage
Varying NFE Allocation for Initial Particle Sampling
In this section, we present an ablation study examining how performance varies with different allocations of the total number function evaluations (NFE) between the initial particle sampling stage (via Vop-K-offNN or MCMC) and the subsequent SMC stage. In our main experiments, we adopt a balanced allocation, with 50% of the NFE budget used for initial particle sampling and the remaining 50% for SMC (denoted as 50%/50%)
To assess the effect of this design choice, we evaluate alternative NFE splits: 25% for initial particle sampling and 75% for SMC (denoted as 25%/75%) and 75% for initial particle sampling with only 25% for SMC (denoted 75%/25%) We conduct this analysis on both Top-K-of-N> and V SAMPER. For consistency, we fix the number of SMC steps to 25, adjusting the number of particles accordingly: 30 particles for the % setting and 10 particles for the 75%/25% setting, image generation, we use half the number of particles in both settings to reflect the halved NFE. Tab. 2 summarizes the results.
For the aesthetic- cesthetie-preference preference
The 7 split overinvests compute to initial particle sampling, resulting in inefficiency due to the significantly reduced number of particles used during the SMC phase. As result, consistently showed the worst performance across all reward metrics. Conversely, the 25%/75% split dedicates too little budget to initial particle sampling, limiting exploration despite using 1.5 times more particles than the 50%/50% split. This leads to weaker performance, particularly under held-out reward evaluations. In contrast, the balanced 50%/50% split consistently yields the most robust performance across both seen and held-out rewards across all tasks.
Top- K-o-N
W SAMPLER
Tasks
Metries
25%6/75%
50%/50%
75%/25%
25%
50%/550%
75%/25%
GroundingDINOf
12
0.424
0,425
0.390
0.454
0.467
0.433
Layout
0.401
0.463
0.471
0,426
Image
ImageReward
1.142
0,957
0.913
1.128
1.035
0.884
VQA
B
0.822
0.855
0.770
0.825
0.810
0.766
T21-Coun
1.021
1.077
2.934
0.804
0.850
1.893
MAE
15
3.675
5.65
3.6
2.925
3.7
Quantity
22.5
25.0
25.0
32.5
30.0
ImageReward
27
0.739
0.752
0.714
0.693
0.796
0.694
VQA
0.937
0.932
0.951
0.943
Aesthetie
6.958
6,879
6.853
7.015
7.012
6.868
Aesthetic
Preference
ImageReward
1.040
1.171
1.062
VQA
18
0.964
0.961
0.969
0.968
0.963
0.952
Table Ablation study results on varying NFE allocation conducted with V-SAMPLER and Top-K- of-N across three task domains. Bold indicates the best performance, while underline denotes the second-best result for each metric. Metries marked with 1 are used seen reward during reward- guided sampling, where others are held-out reward. Higher values indicate better performance (1), unless otherwise noted (L).
30

---

H U-SAMPLER with Other Score-Based Generative Model
To evaluate the generality of Y-SAMPLER and further support our claims, we additionally conduct experiments using SANA-Sprint (291, which is few-step flow-based generative models.
Despite architectural differences, SANA-Sprint integrates seamlessly with V V-SAMPLER. As shown in Tab. 3 it demonstrates reward and quality improvements consistent with those observed using FLUX [28 in Tab. [ indicating that our method generalizes beyond specific backbone. Notably, V-SAMPLER delivers the highest performance across all seen reward models and maintains strong generalization to held-out metrics. Moreover, among SMC-based methods, those that initialize particles from the posterior consistently outperform prior-based variants (except for few case), highlighting the benefit of posterior-ine iesfrrior-nformeed initialization.
These results further highlight the robustness of V-SAMPLER and its applicability to a various few-step score-based generative models.
We further provide qualitative results for each applications. conducted with SANA-Sprint, Fig. For the layout-to-image generation task, each example shows the input layout with color-coded phrases and corresponding bounding boxes for clarity. the quantity-aware image generation task, we overlay the predicted object centroids from a held-out counting model 77 on each image to facilitate comparison. The predicted count along with its absolute difference from the target quantity shown beneath each image in the format (A.), with the best best-performing result highlighted in blue. Note that for the aesthetic-prefet prefeeerrreeeeeeeeee generation task, the first row corresponds to the prompt "Dog" and the second to "Turkey". We display the generated images alongside their predicted aesthetic scores [76].
Single Particle
Tasks
Metries
Sampling Prior
Sampling from Posterior
DPS
FreeDoM
40
TDS 14
DAS
12
Top-
ULA
MALA
0-SAMPL.ER
GroundingDDNO
0.144
0.159
0.403
0.338
0.406
0.388
0.392
0.429
Layout
mloU 74]
0.229
0.242
0.405
0.343
0.406
0.393
0.394
0.432
ImageReward
1,263
VQA
22n-Co
23
11.290
12,839
0.110
0.122
0.0628
0.220
0.148
0.027
MAE 35
12.3
13.8
3.475
4.55
2,825
3.025
2.375
2.175
Quantity Aware
Ace (%) 18
0.0
0.0
30.0
22.5
27.5
22.5
30.0
32.5
InageReward 27
0.680
0,526
0.954
0.889
0.803
0.789
0.840
0.845
VQA 78
0.920
0.859
0.934
0.920
0.916
0.928
0.922
0.930
Aesthetic
7.324
Aesthetic Preference
ImageRewant
1.106
1.045
1.233
1,258
1.144
1.367
1.217
1.262
VQA
0.891
0.902
0.894
0.907
0.888
0.905
0.904
0.909
Table 3: Quantitative comparison of V-SAMPPLER and baselines across three task domains, conducted on SANA-Sprint [29]. Bold indicates the best performance, while underline denotes the second-best result for each metrie, Metries marked with are used seen reward during reward-guided sampling, where others are held-out reward. Higher values indicate better performance (1), unless otherwise noted ()).
31

---

FreeDoM 3D1
TDS CC
DAS PP
Top-K-of-N
ULA
MALA
V-SAMPLER
Layout -to Image
"A photo of a bear sitting between surfboard and choir with hird flying in the sky."
"A person sitting chair and bird sitting horse while horse is the top car."
"63 olives" "
Quantity Aware
19 (A44)
51 (A12)
59 (A4)
59 (A4)
54 (A99
54 (A9)
64 (A1)
"42 grapes"
31 (A11)
(26)
(A5)
45 (A3)
46 (
48 (46)
(40)
Aestbetic
6.333
7.599
7.627
7.621
7.559
7.660
7.958
6.054
7.197
6.979
7.339
7.427
7.112
7.498
Figure 5: Qualitative results for each application SANA-Sprint
29
32

---

I Additional Qualitative Results
In this section, we present additional qualitative results that extend the examples shown in Fig. of the main paper. Consistent with the Fig. 2 all results here are generated using FLUX 28
Layout-to-Image Generation. We present additional qualitative results for the sorouttto-image generation task in Fig. 6 Each example visualizes the imput layout with color-coded phrases and their corresponding bounding boxes for clarity. V-SAMPLER consistently respects both the spatial constraints and object presence specified in the layout. The first four rows (Row 1-4) illustrate failure cases where baseline methods generate objects in incorrect locations either misaligned with the bounding boxes or placed in unrelated regions. For instance, in Row 1, baseline methods fail to accurately place objects within the designated bounding boxes-both the dog and the eagle appear misaligned. Similarly, in Row 4, objects such as the cat and skateboard do not conform to the specified spatial constraints, spilling outside their intended regions in the baseline outputs. In contrast, V-SAMPLER successfully generates all objects within their designated bounding boxes. The last four rows (Rows 5-8) illustrate more severe failure cases by baselines, where not only is spatial alignment severly violated, but some objects entirely missing. For example, in Row 5, DAS 12 fails to generate the apple altogether, while the other baselines exhibit significant spatial misalignment. In Row 7, some baselines produce unrealistic object combinations -ucch the apple and red cup being merged-and and misinterpret the layout, placing the apple inside the red cup instead of correctly positioning it in the wooden bowl. V-SAMPLER not only positions each object correctly but also ensures that all described entities are present and visually distinct.
Quantity-Aware Image Generation. We provide additional qualitative results for the quantity- aware image generation task in 7 and Fig. g The examples cover a variety of object categories and target counts, showing that V-SAMPLER works reliably across different scenarios. For each image, we overlay the predicted object centroids from a held-out counting model [75] for easier comparison. Additionally, difference from the target quantity in the format (A-). We highlight best case with blue color. V- SAMPLER consistently generates the correct number of objects, even in more challenging cases like cluttered scenes or small, overlapping items. On the other hand, baseline methods often produce too many or too few objects, and sometimes include misleading objects. This trend holds across all categories, from food to everyday objects.
display the predicted count below each image, along with the absolute
Aesthetic-Preference Image Generation. Further qualitative results for besthetic--refereee image generation are presented in Fig. [ For each prompt (e.g., "Horse", "Bird"), ") we show the predicted aesthetic score (76) below each image. While all methods generate visually plausible outputs, V- SAMPLER consistently produces images with higher aesthetic appeal, as reflected in both qualitative impressions and the predicted aesthetic scores.
33

---

Layout
FreeDoM 3D
Top-
"A dog and a bird sitting branch while an eagle is flying in the sky."
"A banana and an apple are beneath a book and flower is lying on the book a room."
IY
photo of bear sitting between a surfboard and a chuir with bird flying in sky."
"A cat is sitting top skateboard, dog standing and airplane is flying in the sky."
"A photo of an apple and vase and hamburger and a pizza."
"A banana and an apple and an elephant and a backpack in the meadow with bird flying in the sky."
"A realistic photo of wooden bowl with an apple and red cup and yellow rulip blue vase and lemon."
"A realistic photo, hamburger and donut and couch and a bus and surfboard in the beach."
Figure 6: Qualitative results for layout-to-image generation. Examples show how different methods place objects based on input layouts. V-SAMPLER aligns well with the given boxes, while baselines often misplace or miss objects.
34

---

TP
Top- K-of-N
V-SAMPLER
17 acorms"
(A9)
(010)
(a.3)
12 (A5))
(A1)
(a9)
(A0)
STAP
6IRE
"59 bottle-cups'
TAR
6IEE 9
2 (A57)
35 (A24)
34 (A25)
50 (A9)
(A55
61 (A2)
58 (A1)
1
"26 breads" "
6 (0)
15 (A11)
17
31 (A5)
23 (A.3)
16 (A10)
26 (A0)
DLIVS
"42 olives" T
DEIVS
15 (A27)
(29)
46 (A4)
39 (A3)
48 (a6)
45 (a3)
42 (A0)
Figure 7: Qualitative results for quantity-awaid image generation. Across various object types and target counts, V-SAMPLER generates the right number of instances more reliably than baseline methods, which tend to over- or under-count.
35

---

HP
Top- K-o-N
TIM
"30 chocolates'
15 (A15)
(244
13 (A17)
32 (42)
21 (A9)
(A2)
29 (AI)
"18 potatoes"
16 (62)
20 (A2)
23 (A5)
(A3)
(A3)
20 (A2)
(A0)
"35 macarons'
7 (828)
29 (A6)
23 (A12)
31 (A1)
21 (A14)
31 (A4)
37 (A2)
Mannessorting
"48 wooden-cubes"
arting
8 (A)00)
27 (A21)
34 (A14)
35 (a13)
34 (A14)
46 (a2)
48 (A.0
Figure 8: Qualitative results for quantity image generation. Across various object types and target counts, V-SAMPLER generates the right number of instances more reliably than baseline methods, which tend to over- or under-count.
36

---

FreeDoM
TDS CP
DAS L2
Top-K -oP-Koff-N
UTA
MALA
V-SAMPLER
"Horse"
5.671
6.844
6.818
6,580 "Bird"
6,755
6.586
7.213
6.516
7.251
7.489
7.272 "Dolphin" "
6.482
7.494
7.555
6.701
6.827
6.889
6.949 "Duck"
6, .90
6.892
7,213
6.036
6.222
6.958
6.773
6.557
7.041
7.117
Figure 9: Qualitative results for aesthetic- image generation. V-SAMPLER produces images that are not only realistic but also more visually appealing, with better focus, balance, and overall look compared to baseline outputs.
37


Reviews:
[{"review_id": "cxY068AkLj", "reviewer": "Reviewer_P4YQ", "summary": "This paper investigates the limitations of the existing SMC framework, i.e. the initial particles have to be coming from the standard Gaussian prior. This limitation will give very poor coverage over the high-probability region. Also, it can significantly impeded the sampling efficiency. Therefore, this paper proposes a new approach to put the initial particles in more likely regions through pCNL. This reward-informed initialization ensures that the particle set is better aligned with the target distribution from the outset, which will make the sampling more efficient and accurate. \nThe paper shows a crude approximation of Top-K-of-N method, which works surprisingly well. Therefore, the paper adopts this crude and simple approach to do SMC sampling on layout-to-image generation tasks. Empirical results show that the model is able to outperform the baseline posterior-based sampling methods like ULA, MALA, etc significantly.", "questions": "N/A", "limitations": "The paper is well written. The main limitation is that the bridge between the theorem and the application. The authors have not stated clearly why they chose the three specific evaluation tasks.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strength:\n1. The paper proposes a very solid approach, which has a significant technical contribution.\n2. The paper's theorem is highly sound.\n3. The paper's results also reflect the advantages of the proposed method.\n4. The paper is written quite well. The motivation and background has been written in details to help the readers understand the limitations of the existing methods.\n\nWeakness:\n1. It's not particularly clear why the three specific applications are chosen. Why don't the authors use widely adopted benchmarks like COCO-image generation or Flickr-image generation? Is there a reason to not do normal text-to-image generation evaluation and focus on the more specific ones?", "was_revised": true, "final_justification": "I have read the rebuttal and decide to maintain my score.", "rebuttal": "We greatly appreciate your thoughtful review, especially your recognition of the paper’s strong technical contribution and clear writing. Below, we address the specific points raised.\n\n**(1) Text-to-image generation application.**\n\nWe conducted text-to-image generation experiment using the COCO-2014 dataset. Specifically, we randomly sampled 100 prompts and employed PickScore [1] as the reward model (which measures prompt alignment of the image). To evaluate generalization, we used VQAScore [2] as a held-out reward model (an evaluation metric that is not accessible during generation) which is also used in the main paper to assess prompt alignment. For score-based generative model, we used FLUX-Schnell [3] as in main experiment in the paper.\n\n|   | DPS [4] | FreeDoM [5] | TDS [6] | DAS [7] | Top-K-of-N | ULA+SMC| MALA+SMC | $\\Psi$-Sampler| \n|:------:|:------:|:------:|:------:|:------:|:------:|:------:|:------:|:-------:|\n| PickScore [1] (↑) | 0.224 | 0.226| 0.239| 0.239 | 0.239 | 0.238| 0.239| 0.240 |\n| VQA [2] (↑) | 0.864| 0.897 | 0.908 | 0.913 | 0.915| 0.893| 0.911 | 0.918 |\n\nAs shown in the table above, SMC-based methods (TDS, DAS, Top-K-of-N, ULA+SMC, MALA+SMC, $\\Psi$-Sampler) outperform single-particle methods such as DPS and FreeDoM. However, the differences among the SMC-based methods themselves are minor. The reason is that text-to-image generation has become relatively easy for current large-scale text-to-image generation models. As a result, when applying reward-guided sampling to this task, performance quickly saturates regardless of the specific method used, making it difficult to observe meaningful differences between approaches. \n\nTherefore, in the main experiment, we used alternative applications that are both practically relevant and more challenging. Aesthetic preference image generation is one such example, as it is a commonly used application in several recent works [7,8,9]. To evaluate our approach under more challenging reward conditions, we selected layout-to-image and quantity-aware image generation tasks, which are significantly harder for conventional text-to-image models to handle because they require precise control and fine-grained alignment with the input specification.\n\nIn addition to the three applications presented, we plan to enrich the paper with a broader set of experiments to explore the versatility and potential of our approach. Specifically, we aim to include additional applications such as style-guided image generation and segmentation map-guided image generation, which would offer a more comprehensive analysis and demonstrate the broader applicability of our method.\n\n[1] Kirstain et al., Pick-a-pic: An open dataset of user preferences for text-to-image generation, NeurIPS 2023\\\n[2] Lin et al., Evaluating text-to-visual generation with image-to-text generation, ECCV 2024\\\n[3] Black Forest Labs, FLUX, 2024\\\n[4] Chung et al., Diffusion posterior sampling for general noisy inverse problems, ICLR 2023\\\n[5] Yu et al., FreeDoM:Training-free energy-guided conditional diffusion model, ICCV 2023\\\n[6] Wu et al., Practical and asymptotically exact conditional sampling in diffusion models, NeurIPS 2023\\\n[7] Kim et al., Test-time alignment of diffusion models without reward over-optimization, ICLR 2025\\\n[8] Uehara et al., Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models, NeurIPS 2024\\\n[9] Black et al., Training Diffusion Models with Reinforcement Learning, ICLR 2024"}, {"review_id": "yOuUti5aU3", "reviewer": "Reviewer_UMRC", "summary": "The paper proposes the $\\Psi$-sampler, a SMC-based framework for accurately sampling the posterior distribution formed by a pre-trained diffusion model and a reward function. The algorithm contains two components: a pCNL-based MCMC sampling for the initial, non-Gaussian prior distribution, and an SMC-based sampling scheme. $\\Psi$-sampler is numerically evaluated on various image generation tasks and demonstrates convincing performance.", "questions": "I have the following questions that I would appreciate your answers to. Answers to them are very important for me to maintain a positive support of this paper. Clarifications are certainly welcome if I have some misunderstandings.\n\n1. Related to my major concern: how different is the initial distribution used in $\\Psi$ sampler from pure Gaussian, and can you visualize them in some toy examples? \n\n2. Related to my major concern: how would the algorithm perform if we just start from a Gaussian prior and do the SMC resampling only? Ablations on this part are essential to understanding the core contributions of this work. While I think that doing SMC for reward alignment itself is already a significant contribution to the field, it's even more critical to determine what truly works. \n\n3. Related to my minor concern: how much bias would there be by using Tweedie's formula for approximation instead of learning a real value function? I understand that this is a somewhat standard practice in the literature, but I am curious about how much bias would be mitigated when combining it with SMC. It would be beneficial to have some numerical experiments discussing this, as SMC is known to be an asymptotically exact sampling algorithm.", "limitations": "yes, limitations are addressed.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength:**\n1. The paper is very readable with a clear motivation and easy-to-follow logic.\n2. The proposed algorithm is novel and aims at an important application: post training of diffusion models.\n3. The numerical experiments are quite extensive and demonstrate the advantage of the proposed algorithms.\n\n**Weakness:**\n \n**I have a major concern regarding the validity of this idea of sampling from \"optimal initial distribution\" for technical reasons**. As is mentioned in [1], as well as eq (9) in the appendix, the optimal initial distribution is indeed proportional to $p_1(x_1)\\exp(V_1(x_1))$, where $v_t$ is the value function. From the expression of value function in eq (11),  the value function is given by\n$V_1(x_1)) = \\alpha \\log \\mathbb{E}[\\exp(r(x_0)/\\alpha) | x_1]$ where this expectation is taken with respect to the coupling between $x_0$ and $x_1$ produced by the path measure given by pretrained diffusion model. This creates an issue, as $x_0$ and $x_1$ are independent when the forward noising process add enough noise (which is usually the case in practice), making the conditional expectation just a regular expectation. Therefore, $V_1(x_1)$ is just a constant function, and the optimal initial distribution is just the Gaussian prior. This is also discussed in [2]. \n\nTherefore, I am concerned about the actual significance of starting from a non-Gaussian prior and whether SMC does all the heavy lifting for the performance of $\\Psi$-sampler. This creates additional computation overhead due to the use of MCMC (and one of the main novelties the paper claims is built upon this). This would certainly need more clarification and ablations. \n\nSome other minor issues are listed below:\n1. Missing some references in related work. I note that continuous time SMC-based diffusion posterior sampling algorithms have been proposed in [3] for image inverse problems, as well as in [4]. While this does not affect my evaluation of the contribution of this paper as these are concurrent, I think it's important to discuss connections to these works for a more comprehensive overview of the field to further improve readability.\n\n2. The introduction of eq (3), (4) might confuse readers. I think it's not emphasized enough that with eq (3), (4), the method **does not converge to the correct posterior distribution** even asymptotically when SMC has large number of particles. This is in contrast to the finetuning-based approach, which indeed recovers the correct distribution. The paragraph in appendix is much better for setting up the problem, and the approximation is more evident. I suggest moving that into the main text.\n\n**References**\n\n[1] Uehara, Masatoshi, et al. \"Fine-tuning of continuous-time diffusion models as entropy-regularized control.\"\n\n[2] Domingo-Enrich, Carles, et al. \"Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control.\"\n\n[3] Chen, Haoxuan, et al. \"Solving inverse problems via diffusion-based priors: An approximation-free ensemble sampling approach.\"\n\n[4] Skreta, Marta, et al. \"Feynman-kac correctors in diffusion: Annealing, guidance, and product of experts.\"", "was_revised": true, "final_justification": "The paper proposes an empirically effective algorithm, and my technical concern about its motivation is well addressed through the author's response. Based on the good empirical results of the method and the novelty, I maintain my recommendation of weak acceptance.", "rebuttal": "We greatly appreciate your review, especially your recognition of the paper’s clarity, the novelty of our method, and the importance of the application. Our detailed responses to your concerns are provided below. Please note that the references you mentioned appears to be missing. If you could provide the relevant citations, we would be happy to review it.\n\n**(1) Validity of the idea of sampling from \"optimal initial distribution\"**\n\nFirst of all, thank you for insightful question. However, since no reference was provided in the review, we were unable to verify what [2] refers to. If you could share the reference, we would be happy to review the relevant work and provide a more detailed response.\n\nRegarding your comment, in general, $\\mathbf{x}_0$ and $\\mathbf{x}_1$ are **not** independent unless a very specific diffusion coefficient is chosen—referred to as a *memoryless noise schedule*; for details, please refer to [1].\n\nLet the stochastic interpolant be defined as $\\mathbf{x}_t = a_t \\mathbf{x}_0 + b_t \\mathbf{x}_1$, which bridges the data distribution $\\mathbf{x}_0$ and the prior distribution $\\mathbf{x}_1$. The stochastic differential equation (SDE) that shares the same marginal distribution takes the form: $d\\mathbf{x}_t=\\mathbf{f}(\\mathbf{x}_t,t)dt+g(t)d\\mathbf{W}$ (see our supplementary material, Section A for further details). For $\\mathbf{x}_0$ and $\\mathbf{x}_1$ to be independent, the diffusion coefficient **should be** $g(t)=\\sqrt{2 b_t \\left( \\dot{b}_t - \\frac{\\dot{a}_t}{a_t}b_t\\right) }$. This condition defines the memoryless noise schedule [1]. Therefore, unless this specific diffusion coefficient is used, $\\mathbf{x}_0$ and $\\mathbf{x}_1$ are generally not independent, and the value function is not constant. We will include a clear explanation and its implications in the revision.\n\nA natural question that may arise is whether one could simply use the memoryless noise schedule instead. The issue is that its magnitude over time is quite large, which results in significantly more noise being added at each step. Consequently, this requires many more denoising steps to recover a clean image. For example, in our experiments, using a diffusion coefficient of $g(t) = 1 - (t - 1)^2$ enabled the generation of clean images within approximately 10 steps. In contrast, using the memoryless noise schedule ($g(t) = \\sqrt{\\frac{2t}{1 - t}}$) required several hundred denoising steps to achieve comparable visual quality.\n\nBeyond this, there are several practical reasons why $\\mathbf{x}_0$ and $\\mathbf{x}_1$ cannot be independent. Since training time and model capacity are finite, the learned model inevitably captures some correlation between noise and the image. This is often reflected in text-to-image generation, where sampling with specific noise inputs can yield better results for a given prompt [2]. Furthermore, many prior [3,4] works attempt to improve generation by selecting or optimizing noise vectors that better match the conditioning input. These studies provide practical evidence that they are in fact dependent in real-world models.\n\n**(2) Significance of starting from a non-Gaussian prior.**\n\nWe wish to bring to your attention that Table 1 of the main paper already covers the comparison related to your comment. Specifically, the baselines TDS [5] and DAS [6] are designed to sample initial particles for the subsequent SMC steps from the prior distribution, i.e., a standard Gaussian. To be more specific, TDS is a method that samples initial particles from prior and perform SMC, while DAS builds on this by incorporating a tempering strategy that scales the reward based on the time step. In contrast, our method employs posterior-based initialization before performing SMC. The empirical results show that this difference in initialization leads to a noticeable performance improvement. This indicates that the improvement is not solely due to SMC itself, but also to the quality of the initial particles.\n\nNote that, to ensure a fair comparison, we carefully matched the NFE across all SMC-based methods. We ensured that our method does not consume more NFE than the baselines. \nWe will revise the text to more clearly state that we have conducted such experiments.\n\n**(3) Missing references.**\n\nThank you for pointing this out. However, the references [3] and [4] mentioned in your review were not included, so we were unable to identify which specific works you are referring to. If you could kindly provide the references, we would be happy to review them in detail. We will make sure to incorporate those works in the revision for a more comprehensive overview of the field.\n\n**(4) Clarification of Eq.(3) and Eq.(4).**\n\nDue to space constraints, we were unable to include all related details in the main paper and thus placed them in the supplementary. We agree that this point could be made clearer, and we will revise the main text to better highlight this in the next version.\n\n**(5) Comparing posterior distributions to Gaussian with toy examples.**\n\nConsidering the high dimensionality used in our experiments, it is challenging to provide an intuitive illustration of the difference between the prior and posterior distributions. This is precisely why we included the toy experiment shown in Figure 1 of the main paper. In Figure 1(B), the blue dots represent samples from the posterior distribution, which exhibit a clear deviation from the Gaussian prior illustrated as blue dots in Figure 1(A). \n\n**(6) Ablation on initialization strategies for SMC.**\n\nAs mentioned in **(2) Significance of starting from a non-Gaussian prior.**, both TDS [5] and DAS [6] initialize particles from a Gaussian prior and apply SMC. These baselines are included in Table 1 of the main paper, and the results show that the method that samples initial particles from the posterior (labeled as **Sampling from Posterior** in Table 1) achieves better performance under the same computational budget. To clarify this point, we will revise the text accordingly.\n\n**(7) Bias of Tweedie's formula approximation.**\n\nFirst of all, regarding the mitigation of bias arising from the Tweedie approximation using SMC, the two are not directly related. The bias stems from the approximation itself where the posterior is approximated by a Dirac-delta distribution centered at the posterior mean, and SMC cannot eliminate this bias. \n\nNext, regarding the extent of bias introduced by Tweedie’s formula approximation, the resulting bias depends on how much the true posterior deviates from this point-mass approximation. When the posterior is sharp, the approximation becomes more accurate. \nCurrent few-step score-based generative models typically shows strong correlation between $\\mathbf{x}_t$ and $\\mathbf{x}_0$, leading to a sharper posterior and thus a better approximation.\n\nIn contrast, learning a value function introduces significant additional complexity, such as the need for reward data collection, architectural choices, and stability challenges during training. Moreover, value function training is highly sensitive to reward scale, distribution shift, and task-specific tuning, which can make generalization to new tasks difficult.\nTweedie’s formula approximation, on the other hand, offers a simple yet effective alternative that is easy to compute, requires no additional training, and generalizes readily across tasks. \n\n[1] Dormingo-Enrich et al., Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control, ICLR 2025\\\n[2] Zhou et al., Golden Noise for Diffusion Models: A Learning Framework, ICCV 2025\\\n[3] Guo et al., INITNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization, CVPR 2024\\\n[4] Wang et al., The Silent Assistant: NoiseQuery as Implicit Guidance for Goal-Driven Image Generation, ICCV 2025\\\n[5] Wu et al., Practical and asymptotically exact conditional sampling in diffusion models, NeurIPS 2023\\\n[6] Kim et al., Test-time alignment of diffusion models without reward over-optimization, ICLR 2025"}, {"review_id": "M4AYt7slsC", "reviewer": "Reviewer_TP7L", "summary": "The paper explores Sequential Monte Carlo (SMC) for inference-time reward alignment, which becomes popular since it does not require fine-tuning. However, existing methods assume a Gaussian prior for initialization. This ignores two issues: (1) the reward signal tends to weaken in later steps, reducing the effectiveness of SMC and (2) modern diffusion models often benefit from better initial points due to mode connectivity in early steps and trajectory straightening.\n\nTo address this, the authors propose using pCNL to initialize samples with reward information. Since pCNL keeps the Gaussian prior even in high dimensions, it is suitable for image generation tasks that require high-dimensional noise sampling. The paper compares standard SMC (prior-based), MALA+SMC (posterior-based), and the proposed psi-sampler (pCNL + SMC), and shows that the proposed method performs better.", "questions": "- How much does inference time differ between the proposed method and the baselines?\n- Could you add one more simple toy example (like Figure 1)? While the three tasks are convincing, such intuitive toy examples would help.\n- The method seems closely related to Uehara et al. [20]. Is the main difference that their method requires training, while yours works as inference-time initialization? Could you elaborate more on this difference?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n- The motivation is clear and well-explained with literatures of the recent works. (such as SMC-based methods and Uehara et al.) \n- It is practically useful since the method improves performance at inference time without extra training.\n- The paper shows that pCNL is more effective than other sampling methods such as Top-K-of-N or MALA.\n- The method works well across three different practical tasks: layout-to-image, quantity preference, and aesthetic preference.\n\nWeakness\n- The main concern is the trade-off between sampling cost and the quality improvement. Since each sample requires a reward query on the predicted x0, the cost may be high.\n- It is a bit surprising that the simple Top-K-of-N approach achieves comparable results in some cases, which may reduce the overall impact of the proposed method.\n- As the authors mention, the method depends on a reliable reward model.", "was_revised": true, "final_justification": "The paper is well-written and demonstrates a thorough review of prior work. It also presents a practically useful technique. The rebuttal also solved the raised questions and concerns such as time comparison, additional data and comparison with the previous work. Therefore, the reviewer raised the final score to accept.", "rebuttal": "We greatly appreciate your review, especially your recognition of the clear motivation, practical utility of inference-time alignment without additional training, and the effectiveness of our proposed method across multiple tasks. Below, we address the points raised in your review.\n\n**(1) Trade-off between sampling cost and the quality improvement.**\n\nIt is true that there is a trade-off between cost and quality. Inference-time scaling refers to enhancing performance by allocating more computational resources during inference rather than training. This enables users to adjust the computational budget at inference time to improve reward alignment, offering greater flexibility and scalability in practical settings. To explicitly demonstrate this trade-off, we conducted an additional experiment where we increased the computational cost and measured the resulting quality. This experiment was conducted using our method on the layout-to-image generation application, with the same evaluation metrics as those used in the main paper.\n\n| NFE | 1000 | 1500 | 2000 | 3000 |\n|:---:|:---:|:---:|:---:|:---:|\n| **GroundingDINO [1] (↑)**   | 0.467  | 0.486  | 0.495  |0.527|\n| **mIoU [2] (↑)**  | 0.471  | 0.468  | 0.476  |0.519|\n| **ImageReward [3] (↑)**  | 1.035  | 1.076  | 1.127  |1.124|\n| **VQA [4] (↑)**  | 0.810  | 0.808  | 0.855  | 0.861 |\n\nAs shown in the table above, we observe that the metric improves as the computational budget increases.\n\n**(2) Performance of Top-K-of-N approach.**\n\nAcross all applications, our method consistently outperforms Top-K-of-N when evaluated with known rewards. More importantly, even when assessed with task-specific held-out reward metrics that best reflect each application's core objective, our method shows superior performance.\n\nFor example, in the layout-to-image generation task, we evaluate using mean Intersection-over-Union (mIoU) measured with an alternative object detector (Salience-DETR [2]), and in quantity-aware image generation task, we use a different counting model (Count-GD [5]) to report MSE and Accuracy (%). In both cases, which serve as strong held-out evaluations, our method surpasses Top-K-of-N.\n\nWhile the performance is in some case comparable on general-purpose held-out reward metrics like VQA [4] and ImageReward [3], these evaluate image quality and text-prompt alignment more broadly. In contrast, the task-specific held-out metrics more directly reflect the ability to meet the unique conditions of each application.\nThus, the consistent gains on these application-relevant held-out metrics strongly demonstrate the effectiveness of our approach in task-specific alignment and control.\n\nFurthermore, we would like to emphasize that the Top-K-of-N approach is part of our contribution whioch serve as a novel framework that instantiates our core idea: sampling initial particles for SMC from the reward-informed posterior rather than from the prior. The essence of our contribution lies in demonstrating the importance of posterior-based initialization, and Top-K-of-N offers a simple yet effective approximation of this principle. That even such a rough approximation leads to improved performance strongly supports our central claim.\n\n**(3) Time comparison between proposed method and the baselines.**\n\nWe measured wall-clock time using the aesthetic-preference image generation application as an example to compare between different methods. All numbers in the table are reported in seconds, measured using an NVIDIA A100 GPU.\n\n| DPS [6]| FreeDoM [7]| TDS [8]| DAS [9] | Top-K-of-N | ULA+SMC | MALA+SMC | $\\Psi$-Sampler|\n|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|\n| 25 | 41| 242 | 240 | 235 | 238| 243 | 241 |\n\nTo summarize, all SMC-based methods, including TDS, DAS, Top-K-of-N, ULA+SMC, MALA+SMC, and $\\Psi$-Sampler, took approximately 4 minutes, while DPS took 25 seconds and FreeDoM took 41 seconds.\n\n**(4) Additional toy experiment.**\n\nDue to the rebuttal policy, we cannot include any image so we cannot visualize it and thus report quantitative metric for the additional toy experiment. We closely followed the Gaussian Mixture (GM) toy experiment setup described in MCGdiff [10]. \n\nBriefly, the experiment considers a $d$-dimensional Gaussian mixture model with 25 components, which serves as the data prior $p^{data}$. The mixture component weights are randomly sampled. To emulate a linear inverse problem, the setting includes a measurement model defined by $(y,A,\\sigma_y)\\in \\mathbb{R}^{d_y}\\times \\mathbb{R}^{d_y\\times d}\\times [0,1]$, where $y$ is the measurement, $A$ is a linear operator, and $\\sigma_y$ is measurement noise. And the measurement model is randomly sampled. The likelihood is given by $\\mathcal{N}(y|Ax, \\sigma^2_y I), x\\sim p^{data}$. \n\nMCGdiff [10] then compare generated samples with samples drawn from the analytically tractable posterior $\\propto p^{data}\\mathcal{N}(y|Ax, \\sigma^2_y I)$ by computing the sliced Wasserstein (SW) distance (lower values indicate closer alignment between the two distributions ). We trained a few-step score-based generative model to sample from $p^{data}$.\n\nWe evaluate SMC, MALA+SMC, and $\\Psi$-Sampler under a fixed computational budget of NFE = 200. We used the same MCMC step size for both MALA+SMC and $\\Psi$-Sampler. The SW distance is computed for each method, and we report the mean and standard deviation over 20 runs with different random seeds. We conducted experiments with $d\\in \\\\{80, 800\\\\}$, varying the measurement dimension $d_y \\in \\\\{1, 2, 4\\\\}$.\n\n- $d=80$\n| $d_y$| SMC| MALA+SMC| $\\Psi$-Sampler|\n|:----:|:----:|:----:|:----:|\n| 1| 1.9652 ± 0.3285| 1.6669 ± 0.2639| 1.6504 ± 0.2294|\n| 2| 1.3230 ± 0.2755| 1.1045 ± 0.2509| 1.1438 ± 0.2574|\n| 4| 0.4327 ± 0.0741| 0.3447 ± 0.0408| 0.3293 ± 0.0451|\n\n- $d=800$\n| $d_y$| SMC| MALA+SMC| $\\Psi$-Sampler|\n|:----:|:----:|:----:|:----:|\n| 1| 2.6379 ± 0.2657| 2.2552 ± 0.2351| 2.1168 ± 0.2275|\n| 2| 1.8778 ± 0.2702| 1.4101 ± 0.2031| 1.3067 ± 0.2285|\n| 4| 1.1826 ± 0.2327| 0.5704 ± 0.0274| 0.5417 ± 0.0400|\n\nThese results demonstrate that both posterior-based initialization methods (MALA+SMC and $\\Psi$-Sampler) consistently outperform standard SMC, which initializes particles from a Gaussian prior, even under the same NFE constraint. This highlights the importance of high-quality initial particles in improving SMC performance and achieving overall gains. Among the two, our method achieves better results than MALA+SMC.\n\nAdditionally, the advantage of our method over MALA+SMC becomes evident as the dimensionality grows (e.g., $d=800$ vs. $d=80$), illustrating the strength of pCNL in high-dimensional regimes. This further supports the use of pCNL in extremely high-dimension settings like our main experiment, which involves a 65,536-dimensional latent space.\n\n**(5) Difference between Uehara et al. [11] and ours.**\n\nAs you correctly pointed out, the main difference is that our method is completely training-free, whereas Uehara et al. [11] does require training. In fact, their method involves training several components. To be more specific, they fine-tune the generative model using a neural SDE solver. In terms of initialization, while we adopt an MCMC method to directly sample from a reward-informed posterior distribution (an approximation of the optimal initial distribution using Tweedie’s formula), they instead train an additional SDE-based model to approximately sample from the optimal initial distribution using neural SDE solver. This also requires learning a value function at time $t=1$, which incurs additional cost for data collection and training. These steps comprise a three-stage training process.\n\nSuch a pipeline can be costly and restrictive, especially when applying the method to a new generative model or a different reward function, as all components must be retrained. In contrast, our approach operates entirely at inference time and can be readily applied to new models or rewards without any additional training.\n\n[1] Liu et al., GroundingDINO: Marrying DINO with grounded pre-training for open-set object detection, ECCV 2024\\\n[2] Hou et al., Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement, CVPR 2024\\\n[3] Xu et al., ImageReward: learning and evaluating human preferences for text-to-image generation, NeurIPS 2023\\\n[4] Lin et al., Evaluating text-to-visual generation with image-to-text generation, ECCV 2024\\\n[5] Amini-Naieni et al., CountGD: Multi-Modal Open-World Counting, NeurIPS 2024\\\n[6] Chung et al., Diffusion posterior sampling for general noisy inverse problems, ICLR 2023\\\n[7] Yu et al., FreeDoM:Training-free energy-guided conditional diffusion model, ICCV 2023\\\n[8] Wu et al., Practical and asymptotically exact conditional sampling in diffusion models, NeurIPS 2023\\\n[9] Kim et al., Test-time alignment of diffusion models without reward over-optimization, ICLR 2025\\\n[10] Cardoso et al., Monte Carlo guided Diffusion for Bayesian linear inverse problems, ICLR 2024\\\n[11] Uehara et al., Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Control, arXiv"}, {"review_id": "e3dGp5SEED", "reviewer": "Reviewer_bsVN", "summary": "The paper proposes a method to better initialize particle samples for Sequential Monte Carlo samplers used to solve the problem of training-free guidance of diffusion methods. It does so by introducing a MCMC algorithm that targets the tractable version of the posterior distribution at time t=1, obtained through Tweedie's formula. It proposes in particular the pCNL MCMC algorithm to sample from the aforementioned distribution. The numerical experiments show both that there is a real gain of doing so in one toy problem and in several applications using several of the most recent generative models. It also shows that amognst the several candidates samplers that could in theory be used to initialize the distributions, pCNL seems to be the most efficient one.", "questions": "1. What happens to MALA in the mixture of Gaussian case?  Seems like an initialization problem, are those points stuck in local Minima? If so, why the pCNL is not stuck in the same local minima? Here the acceptance rate should not have a big importance, as the dimension is low. \n2. It should be stated somewhere in the text that the target distribution for the initialization is not exactly the posterior, but it's tractable version (obtained through Tweedie's) somewhere close to equation 11. It is stated that the approximations will be used in equation 3 but still can be misleading by the way equation 11 is written. I suggest the authors either define a notation for the Tweedie approximation that is distinct of the real potential or explicitly say in the text that they will use the tractable counterpart.\n3. The part of the toy problem of better mode coverage than MALA does not seem relevant, as both cover all the modes. The problem with MALA seems to be that it adds other modes to the prior due to the initialization.\n4. A detailled description of the setting used to produce the toy problem is lacking. It is not currently given in the appendix.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe paper proposes an original idea to enhance considerably the performance of Sequential Monte Carlo methods for training-free conditional generation in diffusion models. It does so by proposing a suitable MCMC sampler to target the Tweedie's approximation of the posterior distribution at t=1. It evaluates the proposed method in several relevant and nowadays used problems, namely with the latest generative models used. \n\nWeaknesses:\nI think that while the evaluation of the current method in industry relevant problems, SMC is a fundamentally probabilistic tool. In this sense, I think that the evaluation in \"toy problems\" could be further enhanced and clarified. Namely, by running systematic evaluation for several mixtures and several different conditioning problems with a statistical like metric (such as the sliced Wasserstein), one would be more confident in the performance of the proposed fix (Namely, are any modes not recovered in some situations?). I would recommend reproducing the mixture of Gaussians example presented in [1].\nA second point is that there are some training-free guidance algorithms that could be included in the comparison to better reflect the state of the art, such as [2] and [3].\n\n[1] Cardoso, G. V., El Idrissi, Y. J., Le Corff, S., & Moulines, E. (2024, January). Monte Carlo guided Diffusion for Bayesian linear inverse problems. In ICLR International Conference on Learning Representations.\n\n[2] Janati, Y., Moufad, B., Qassime, M. A. E., Durmus, A., Moulines, E., & Olsson, J. (2025). A Mixture-Based Framework for Guiding Diffusion Models. arXiv preprint arXiv:2502.03332.\n\n[3] Qu, Q., & Shen, L. (2024). Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency.", "was_revised": true, "final_justification": "The authors addressed my concerns during the rebuttal phase. As they were mostly related to clarity of the original text and the authors are engaged into adding further explanation and experiments performed during the rebuttal to the paper, I think the paper should be accepted.", "rebuttal": "We sincerely appreciate your positive feedback, particularly recognizing our contribution as an “original idea to enhance the performance of SMC methods” and for acknowledging the practical relevance of our evaluations using modern generative models. Below, we address the concerns and questions you have raised.\n\n**(1) Additional toy experiment.**\n\nBased on your suggestion, we conducted an additional toy experiment following the Gaussian Mixture (GM) setup described in MCGdiff [1].\n\nFor brief summary, the Gaussian mixture setting in MCGdiff [1] defines the data prior $p^{data}$ as a $d$-dimensional Gaussian mixture with 25 components, where the mixture weights are randomly sampled. To simulate a linear inverse problem, a measurement model is introduced with $(y, A, \\sigma_y) \\in \\mathbb{R}^{d_y} \\times \\mathbb{R}^{d_y \\times d} \\times [0, 1]$, where $y$ is the observation, $A$ is a linear operator, and $\\sigma_y$ is the measurement noise. And the measurement model is randomly sampled. The likelihood is given by $\\mathcal{N}(y \\mid Ax, \\sigma_y^2 I)$ with $x \\sim p^{data}$.\n\nWe compare the generated samples against those drawn from the analytically tractable posterior $\\propto p^{data} \\mathcal{N}(y \\mid Ax, \\sigma_y^2 I)$ using the sliced Wasserstein (SW) distance, where lower values indicate better alignment. We trained a few-step score-based generative model to sample from $p^{data}$.\n\nWe evaluate SMC, MALA+SMC, and $\\Psi$-Sampler under a fixed computational budget of NFE = 200. We used the same MCMC step size for both MALA+SMC and $\\Psi$-Sampler. The sliced Wasserstein (SW) distance is computed for each method, averaged over 20 trials with different random seeds. Experiments are conducted with $d \\in \\\\{80, 800\\\\}$, while the measurement dimension $d_y$ is varied across $\\\\{1, 2, 4\\\\}$.\n\n- $d=80$\n| $d_y$| SMC| MALA+SMC| $\\Psi$-Sampler|\n|:----:|:----:|:----:|:----:|\n| 1| 1.9652 ± 0.3285| 1.6669 ± 0.2639| 1.6504 ± 0.2294|\n| 2| 1.3230 ± 0.2755| 1.1045 ± 0.2509| 1.1438 ± 0.2574|\n| 4| 0.4327 ± 0.0741| 0.3447 ± 0.0408| 0.3293 ± 0.0451|\n\n- $d=800$\n| $d_y$| SMC| MALA+SMC| $\\Psi$-Sampler|\n|:----:|:----:|:----:|:----:|\n| 1| 2.6379 ± 0.2657| 2.2552 ± 0.2351| 2.1168 ± 0.2275|\n| 2| 1.8778 ± 0.2702| 1.4101 ± 0.2031| 1.3067 ± 0.2285|\n| 4| 1.1826 ± 0.2327| 0.5704 ± 0.0274| 0.5417 ± 0.0400|\n\nThe above results show that both posterior-initialization methods (MALA+SMC and $\\Psi$-Sampler) outperform standard SMC, which uses particles initialized from a Gaussian prior, even under the same NFE constraint. This indicates that the quality of initial particles influences the performance of SMC, leading to overall improved performance. Notably, our method further outperforms MALA+SMC. \n\nMoreover, the performance gap between MALA+SMC and $\\Psi$-Sampler becomes more evident in higher dimensions (i.e., $d=800$) compared to lower ones (i.e., $d=80$). This highlights the effectiveness of pCNL in high-dimensional settings and suggests its importance for extremely high-dimension scenarios such as our main experiment, which operates in a 65,536 dimensional space.\n\nWe sincerely thank the reviewer for suggesting this insightful toy experiment. We plan to include a more systematic evaluation on a broader range of toy settings, including this one, in the revision. \n\n**(2) Additional baselines comparison.**\n\nAs you pointed out, we reviewed relevant training-free guidance methods [2,3] and conducted additional experiments based on their algorithms. Below are the results on the three applications featured in our main paper, using the same evaluation metrics for consistency. For ease of comparison, we also include the results of DPS [4] and $\\Psi$-Sampler as reported in the main paper.\n\n- Layout-to-Image generation\n|  | GroundingDINO [5] (↑) | mIoU [6] (↑)| ImageReward [7] (↑) | VQA [8] (↑) |\n|:----:|:----:|:----:|:----:|:----:|\n| MGDM [2] | 0.204| 0.250| 0.960| 0.720|\n| ReSample [3] | 0.168 | 0.233 | 0.854 | 0.709 |\n| DPS [4] | 0.166| 0.215| 0.705| 0.684|\n| $\\Psi$-Sampler| 0.467 | 0.471| 1.035| 0.810 |\n\n- Quantity aware image generation\n|  | T2I-Count [9] (↓) | MAE [10] (↓)| Acc(%) [10] (↑) | ImageReward [7] (↑) | VQA [8] (↑) |\n|:----:|:----:|:----:|:----:|:----:|:----:|\n| MGDM [2] | 12.842 | 14.250 | 2.5 | 0.381|0.933|\n| ReSample [3] | 12.639| 15.175| 0.0| 0.827|0.961|\n| DPS [4]| 14.187| 15.7| 0.0| 0.746 |0.957|\n| $\\Psi$-Sampler| 0.850| 2.925| 32.5| 0.796 | 0.951|\n\n- Aesthetic preference image generation\n|  | Aesthetic [11] (↑) | ImageReward [7] (↑) | VQA [8] (↑) |\n|:----:|:----:|:----:|:----:|\n| MGDM [2] | 6.488| 1.078| 0.968| \n| ReSample [3] | 6.193 | 1.179 | 0.957 | \n| DPS [4]| 6.139| 1.116| 0.968| \n| $\\Psi$-Sampler| 7.012| 1.171 | 0.963 | \n\nAs can be seen from the results, MGDM and ReSample outperform DPS as they are more tailored methods. However, due to the inherent limitations of single-particle methods, their performance still falls short compared to SMC-based methods such as $\\Psi$-Sampler. As suggested, we will conduct additional experiments with other training-free guidance methods to further strengthen our analysis and include the results in the revision.\n\n**(3) Clarification on toy experiment.**\n\nWe would like to clarify the behavior of MALA+SMC in the toy experiment. The observed inferior performance and its tendency to get stuck in local minima were due to our deliberate choice of using a smaller step size for MALA compared to pCNL. The main motivation behind this decision was to better reflect the actual experimental settings used in the main paper, where MALA requires a smaller step size to avoid extremely low acceptance rates (see Figure 3 in our main paper).\nIndeed, if we were to use the same step size as pCNL in the toy setup, MALA+SMC would also avoid local minima and achieve performance comparable to $\\Psi$-Sampler. However, we intentionally matched the toy experiment’s configuration to our main experiments to ensure consistency in illustrating the practical behavior of each sampler.\n\nWhile MALA+SMC and $\\Psi$-Sampler yield similar results in low-dimensional settings—such as the toy experiment in the main paper—when their step sizes are matched, **(1) Additional toy experiment.** demonstrates that $\\Psi$-Sampler outperforms MALA+SMC in higher dimensions $d = 800$, even with identical step sizes. This suggests that pCNL offers a distinct advantage in high-dimensional regimes. In extremely high-dimensional cases like our main experiment, MALA suffers from a significantly lower acceptance rate, which necessitates using much smaller step sizes. Attempting to match the step size with pCNL leads to even worse performance for MALA. Therefore, regardless of whether MALA uses a smaller or matched step size, it consistently underperforms compared to $\\Psi$-Sampler, resulting in a clear performance gap in the main experiment.\n\nWe acknowledge that this point is unclear in the current version of the paper and will revise the text to better clarify the motivation and ensure there is no confusion regarding the toy experiment setup.\n\n**(4) Clarification on the use of Tweedie’s formula approximation.**\n\nWe appreciate your suggestion. As you noted, we will revise the discussion around Eq. (11) to explicitly emphasize that the target distribution used for initialization is an approximation derived from Tweedie’s formula.\n\n**(5) Clarification on mode coverage in toy experiment.**\n\nOur claim that $\\Psi$-Sampler exhibits “better posterior coverage” was intended to highlight that MALA+SMC samples in the toy experiment tend to deviate from the modes of the ground truth posterior distribution (blue dots in Figure 1(B) in our main paper). As discussed in **(3) Clarification on toy experiment**, we will carefully revise the content related to both the step size and the wording regarding posterior coverage to avoid potential misunderstandings.\n\n**(6) Specification of toy experiment.**\n\nWe provide detailed settings for the toy experiment used in the main paper. Specifically, we used a 2D Gaussian mixture with covariance $0.3I$, consisting of six equally weighted components uniformly arranged on a circle of radius 6. The reward function is defined as the sum of three Gaussians centered at points evenly spaced on the circle: $r(x) = e^{-0.5\\|x - p_1\\|^2} + e^{-0.5\\|x - p_2\\|^2} + e^{-0.5\\|x - p_3\\|^2}$, where $p_1=(6.5,0)$, $p_2 = (-3.25,3.25\\sqrt{3})$, and $p_3 = (-3.25,-3.25\\sqrt{3})$. We trained a few-step score-based generative model using 4-layer MLP with hidden dimension 128. For sampling, we fixed the total NFE to 100 across all methods and visualized the results using 2,000 generated samples. We used step size 0.1 for MALA and 0.2 for pCNL to reflect the step size difference in the main experiment. We will include these details in the appendix for our revised version to ensure reproducibility.\n\n[1] Cardoso et al., Monte Carlo guided Diffusion for Bayesian linear inverse problems, ICLR 2024\\\n[2] Janati et al., A Mixture-Based Framework for Guiding Diffusion Models, ICML 2025\\\n[3] Song et al., Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency, ICLR 2024\\\n[4] Chung et al., Diffusion posterior sampling for general noisy inverse problems, ICLR 2023\\\n[5] Liu et al., GroundingDINO: Marrying DINO with grounded pre-training for open-set object detection, ECCV 2024\\\n[6] Hou et al., Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement, CVPR 2024\\\n[7] Xu et al., ImageReward: learning and evaluating human preferences for text-to-image generation, NeurIPS 2023\\\n[8] Lin et al., Evaluating text-to-visual generation with image-to-text generation, ECCV 2024\\\n[9] Qian et al., T2ICount: Enhancing cross-modal understanding for zero-shot counting, CVPR 2025\\\n[10] Amini-Naieni et al., CountGD: Multi-Modal Open-World Counting, NeurIPS 2024\\\n[11] Schuhmann, Laion aesthetic predictor, 2022"}]

Author rebuttal:

Meta-review: The paper improves inference-time reward alignment by better initilisation of particles in SMC. The main conceptual contribution is using a reward-aware posterior and the pCNL sampling algorithm. Extensive numerical experiments support the claim of improvement performance compared to SOTA for a number of different tasks.
The paper is well written, cites appropriate literature and present a systematic analysis of the method. During the rebuttal, the reviewers identified some weaknesses related to the experiments, tradeoffs and novelty, and the authors provided convincing responses that address all the main concerns. 3 reviewers proposed a score of 5 and one of 4. I agree on the soundness of the paper and its impact for applications of score based models.
Decision: Accept (spotlight)