Inference-time scaling of diffusion models through classical search
Authors: XiangCheng Zhang
Haowei Lin
Haotian Ye
James Zou
Jianzhu Ma
Yitao Liang
Yilun Du
Venue: ICLR 2026 Poster (2026)
Keywords: diffusion models
inference-time scaling
compositional generation
search algorithms
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We propose a principled framework for inference-time scaling of diffusion models through classical search principles, with experiments in planning, RL and image generation.

Abstract: Classical search algorithms have long underpinned modern artificial intelligence. In this work, we tackle the challenge of inference-time control in diffusion models—adapting generated outputs to meet diverse test-time objectives—using principles from classical search. We propose a general framework that orchestrates local and global search to efficiently navigate the generative space. It performs compute-efficient global exploration using breadth-first and depth-first tree search and employs a theoretically grounded, scalable local search via annealed Langevin MCMC. We evaluate our approach on a range of challenging domains, including planning, offline reinforcement learning, and image generation, and observe significant gains in both performance and efficiency over baseline methods. These results demonstrate that classical search offers a principled and practical foundation for inference-time scaling in diffusion models. By jointly scaling local and global search for the first time, our framework establishes a new Pareto frontier across challenging decision-making domains.

Published as a conference paper at ICLR 2026
INFERENCE-TIME SCALING OF DIFFUSION MODELS THROUGH CLASSICAL SEARCH
Xiangeheng Zhang', Haowei Lin', Haotian James Zou
Jianzhu Ma', Yitao Liang', Yilun Du 3*
Helixon US Inc.
2Stanford University
Harvard University
ABSTRACT
Classical search algorithms have long underpinned modern artificial intelligence. In this work, we tackle the challenge of inference-time control in diffusion mod- els-adapting generated outputs to meet diverse test-time objectives- using prin- ciples from classical search. We propose a general framework that orchestrates local and global search to efficiently navigate the generative space. It performs compute-efficient global exploration using breadth-first and depth-first tree search and employs theoretically grounded, scalable local search via annealed Langevin MCMC. We evaluate our approach on a range of challenging domains, including planning, offline reinforcement learning, and image generation, and observe signifi- cant gains in both performance and efficiency over bascline methods. These results demonstrate that classical search offers principled and practical foundation for inference-time scaling in diffusion models. By jointly scaling local and global search for the first time, our framework establishes a new Pareto frontier across challenging decision-making domains.
INTRODUCTION
Classical search algorithms have laid the foundation for modern artificial intelligence (Russell & Norvig, 2009). In discrete settings, graph search algorithms are widely used to explore the state space. Breadth-firs ffrst search (BFS) (Moore, 1959) and depth-first search (DFS) (Tarjan, 1972) traverse the search tree in a fixed order. To better leverage problem-specific information, best-first search methods (Pearl, 1984), such as A (Hart et al., 1968), use heuristic to evaluate and prioritize states. Alternatively, local search methods, such as hill-climbing (Russell & Norvig, 2009, Sec. 4.1,, explore neighboring states. More recent techniques like gradient descent and Markov Chain Monte Carlo (MCMC) have become widely adopted in optimization and probabilistic inference, underpinning many modern Al models.
Diffusion models (Ho et al., 2020) have shown impressive performance in generative modeling for continuous domains such as images (Dhariwal & Nichol, 2021), videos (Ho et al., 2022). They are (Liu et al., 2024; Black et al., 2024; Team
also increasingly used in robotics and decision- et al., 2024) to generate diverse actions (Chi et al., 2023). However, generated samples may not always align with physical laws (Song et al., 2023) or human intent (Wallace et al., 2024), and the vast generative space often necessitates multiple trials to produce satisfactory outputs (Xic et al., 2025). To address this, we scale up inference-time compute using strategic search methods that navigate the generative manifold for high-quality samples. We formalize sample evaluation using a verifier function (ar.) defined on ground truth samples, which measures the quality of the sample. Such verifiers could be reward functions (Xu et al., 2023a), Q-functions (Lu et al., 2023), classifier conditions p(c|z0) (Ye et al., 2024; Dhariwal & Nichol, 2021), and VLMs (Huang al., 2023).
To efficiently search the generative space of diffusion models, we revisit classical search principles. To capture diverse modes in the complex distributions generated by diffusion models, we view sampling as traversing a search tree, employing BFS and DFS to progressively explore states during denoising. Similar to best-first search, we evaluate intermediate states 2, with the verifier ff ( prioritizing high-quality paths and efficiently allocating compute via branching and backtracking.
"Corssooning author. Contact ydu@seas.... hhwvard..aaaddeee

---

Published as a conference paper at ICLR 2026
Keys:
samples from the base diffission model
samples selected in global scarch
sumples after optimization in local search
Base distribution Po
Verifier score f
Composed distribution Pu
Lical refinemunt for boller aumplex
local search steps
base distribution
Selected modes
Direct sampling
Global search
Local global seurch
Illustration of search framework. Bottom left: direct sampling results in samples with low verifier scores. Bottom middle: global search identifies high score modes within the base distribution. Bottom right: local search further optimizes the samples for higher quality, driven by the gradient signal.
Figure I:
To go refine the base distribution and obtain higher-quality samples, we perform local search via Langevin MCMC, exploring the neighborhood of current samples under guidance from both the verifier gradient and the diffusion model's "score function" (Vincent, 201 1). By jointly optimizing the compositional objective of the diffusion model and the verifier (Du al., 2023), our local search surpasses the capabilities of the base model. An overview of our framework is shown in Fig.
Recent works scale diffusion model inference via particle-based SMC (Kim et al., 2025; Singhal et al., 2025a; Wu et al., 2023a; Li et al., 2024) and tree-based methods (Guo et al., 2025; Li et al., 2025b), which explores the diffusion sampling process with fixed schedule. We generalize the particle-based methods (Kim et al., 2025; Singhal et al., 2025a; Li et al., 2024) with BFS-based framework, clarifying prior design choices and providing an improved BFS baseline. Inspired by DFS, we add adaptive backtracking to allocate compute adaptively, surpassing BFS in efficiency and adaptivity. The adaptive backtrack schedule also outperforms the fixed noise injection in Ma et al. (2025), highlighting the importance of adaptivity in inference scaling. While global search remains limited to base-distribution modes, scaling local search with Langevin MCMC explores high-reward regions in neighborhoods of the base model samples, proving effective in challenging decision-makingg tasks. Finally, we jointly scale local and global search with the classical AI search principles, demonstrating superior performance over prior methods, which scale local and global search in isolation.
Our key contributions summarized as follows:
i) For global tree search, we elucidate the design space of prior BFS-style methods and provide improved BFS design. We further present the first adaptive DFS algorithm for diffusion inference scaling, offering superior efficiency and adaptivity.
ii We introduce theoretically grounded local search method using annealed Langevin MCMC, demonstrating superior performance in challenging domains.
iii) We propose unified framework for inference scaling that integrates both global and local search within the classical AI search paradigm. By jointly scaling local and global search for the first time, advance the Pareto frontier of inference-time scaling across diverse domains.
RELATED WORKS
Here, we provide a brief overview of inference-timg time scaling with diffusion models. For a more comprehensive literature review and discussion of concurrent works, Appendix B.
Particle-based scaling methods. Recent works (Kim et al., 2025: Singhal et al., 2025a) propose SMC-style particle filtering methods, scaling inference compute by increasing the number of particles
2

---

Published as a conference paper at ICLR 2026
and improving efficiency via resampling. Kim et al. (2025) propose biasing the transition kernel using verifier gradients and incorporating both the proposal and base transition probabilities during resampling. Similarly, rree-search-ber based methods (Li et al., 2024; 2025b; Guo et al., 2025) evaluate intermediate nodes and select promising candidates. Both approaches can be seen as special cases of our BFS tree search framework, which denoise set of particles in parallel, applying branching and filtering based on intermediate reward signals. SoP (Ma et al., 2025) proposes searching over sampling paths by adding M noise samples to each noisy particle and then denoising, thereby exploring neighboring sampling paths. Although this procedure modifies the denoising process, all particles still follow the same fixed denoising schedule. In addition to linear BFS- algorithms,, we also propose DFS-style algorithms with non-linear adaptive backtracking through noise injection and a score-dependent backtracking schedule, demonstrating superior performance over prior methods.
Concurrent with our work, Jain et al. (2025) additionally incorporate value backup from MCTS into the tree search for diffusion models, leveraging information from previous sampling paths, and enabling adaptive compute allocation with the design of anytime algorithm. Lee al. (2025) propose backtracking by sending fully denoised particles to all noise levels. The adaptive termination condition is designed based on the reward distribution of the denoised particles to ensure sufficient exploration. In contrast, our DFS determines backtrack noise level by the score of the particle, thus reducing excessive compute consumption on easy instances. In discrete diffusion models, Dang et al. (2025) introduce Partiele Gibbs Sampling for inference scaling, which outperforms SMC-based approaches (Singhal al., 2025a). However, their method is not directly applicable to our setting.
Gradient-basee guidance methods. To sample from a conditional distribution, classifier guidance (Dhariwal & Nichol, 2021) utilizes the gradient from trained noise-dependent classifier compute the conditional score function (Song et al., 2020b). However, such noise-dependent classifier requires additional training and data collection. To use classifiers defined on clean samples, training- free guidance methods (Ye et al., 2024; Chung & Ye, 2022; Song et al., 2023; Yu et al., 2023; He et al., 2023) approximates noisy conditional probability using the denoised output Zojee Such methods are inheritability biased due to their first order approximation. Different from prior methods that rely on the conditional diffusion process, we sample from the compositional distribution via annealed Langevin MCMC, which provides asymptotically exact sampling without any training.
3 BACKGROUNDS
3.1 DIFFUSION PROBABILISTIC MODELS
Suppose we have D-dimensional random variable To ED with distribution po())). Diffusson models (Ho et al., 2020; Song et al., 2020a) and the more general flow models (Lipman et al., 2022; Albergo & Vanden-Eijnden, 2022) are generative models that turn noise into data via a stochastic process {x The forward "noising" process with t > s can be defined as:
o
((<r|x,) = N Tz ar
2
1
Of
where ar, de are referred as the noise schedule with ay 04 = 1, oT og = 0. We can thus write the random variables 2; as an interpolation between data and noise (Ma et al., 2024):
3t 0, O,E,
and denote (xx) as the marginal distribution of x;, To model the reverse "denoising" process, we train the model using the denoising objective (Ho et al., 2020):
L(0)- Er, [ee t) E]]
which is equivalent of learning the score function of qx(xz) (Vincent, 2011), 1), as the ground truth of co(2,, t) is -a Va,, log qi (xx). To generate samples, we transform noise into data via the reverse
transition kernel po (x In practice, we either sample 2;-1 using deterministic samplers like DDIM (Song et al., 2020a):
Pt-1 04-1 (z, E t)) + 00-166 (Ii, t)
at
or stochastic samplers like DDPM (Ho al., 2020; Nichol & Dhariwal, 2021):
pa(#; (x)) = N (.:-15 15 mn ( 1), Eg ()))
3

---

Published as a conference paper at ICLR 2026
3.2 COMPOSITIONAL AND CONTROLLABLE GENERATION OF DPMS
Given a base diffusion model with data distribution (=:))), one may wish to sample 2n with some constraints or conditions (aro). Exact diffusion sampling from the composed distribution Po())) x Po (fro) would require training a time-dependent f on data generated by Po (Dhariwal & Nichol, 2021; Lu al., 2023), which may not be applicable in practice. Thus, we adopt optimization based methods to approximate the target distribution.
Compositional generation via annealed Langevin MCMC. When sampling from a compositional distribution composed of multiple probability distributions, (o x pu( Du et al. (2023) proposes annealed Langevin MCMC sampling. In this approach, sequence of annealed distributions Ze (atx) xx qr (=(z)) )r: (axa) is constructed with ér corresponds to the diffusion process of po. and samples are drawn using Langevin dynamies (Welling & Teh, 2011):
Ft xt + V le (x{) + 22n'', é'~ e'~N(0011)
Since the distribution of æ; converges to (2,, asymptotically as i + 00, - O, we can sample from (2o) following the annealing path {q: { with qo fn- Moreover, since the score of d can be directly computed by composing the score of two distributions:
Ve log qe (xx)) = Va log (x:) + Va, log (xz))
thus do not require extra training. We point out that the product distribution q (2,) does not represent a valid diffusion process, thus inverse diffusion sampling via V. log q is impossible.
Controllable generation through training-free guidance. During the sampling process, training- free guidance propose to update D; using gradient ascent
ã. = Iz + A A
æ. log (aap) + aa V #o11 log (
where Zote E[ x
2, 2.-0,e0(0... This method approximates the intractable posterior with the posterior mean: Eazl [(xo) [(E[zn (zz]). To enhance the guidance strength, Yu et al. (2023)
propose apply recurrence strategy, which first samples 0,-1 via po( (z,)) add the guidance gradient, then add noise back to di through the forward process (: a
a ~ po(()=)), . a
C1-1 t att1 4o(-10t_1), i i= 1,2, Nuocur
(4)
at
with Nreour being the total number of recurrence steps.
4 METHODS
Problem Formulation. Given pretrained diffusion model co( t) with base distribution po po())) test-time, we often wish optimize the generation process satisfy task-specific objectives. For example, RL may require generating high-value actions, image synthesis may seek constraint- satisfying images, and trajectory generation may demand physically valid outputs. this paper, we interested in how to scale test-time inference to follow such objectives.
We consider an inference- time scaling strategy that adjusts the sampling process based on a verifier function. Specifically, we define verifier f(x) IRD - R+ which specifies the degree to which samples optimize a specified objective. We then aim to bias sampling toward regions of the sample space where f (2xg is high. This leads to the objective of sampling from compositional distribution that combines the original model distribution with the verifier:
No (20) x po (o(
where X controls the weight of verifier scores.
Since exact sampling from the distribution is often impractical, we aim to search the manifold for the target samples at inference time, both globally and locally. First, we explore the diverse modes in the complex generative landscape of diffusion models through global graph search algorithms. However, global search alone can not generate samples beyond the pretrained model. We then propose to search the vicinity of the sample using hill-climbing style local search methods, guided by the verifier gradient. By combining local search with global search, we can avoid being stuck in local optima, advancing the performance of the base model by sampling beyond the pretrained modes.
4

---

Published as a conference paper at ICLR 2026
Particles
Particles
Backtracking
Keys:
x
accepted by verifier
Denoise
rejected by verifier
denoise
add noise
Best-of-N
BFS
DFS
Figure 2: Illustration of global tree search algorithms.
4.1 GLOBAL SEARCH FOR MODE IDENTIFICATION
To efficiently explore the modes of a diffusion model, we represent the Markov chain of its denoising process a fixed-depth tree, where the transition kernel - (x,)) may correspond either deterministic or stochastic samplers. This abstraction enables the use of classical tree-search heuristics design compute-efficien exploration methods.
A simple baseline is best-of-N (BoN) sampling: generate N independent trajectories and select the one with the highest verifier score at the final step. While straightforward, this approach discards valuable information from intermediate stages. To improve efficiency, inspired by best-first search (Pearl, 1984), we evaluate intermediate particles using the denoised estimate Zojr- This allows dynamic allocation of computation based on particle quality, using techniques such as branching and backtracking. We next describe two representative efficient search (BFS) and depth-first search DFS))adapted for diffusion model sampling. By prioritizing the expansion of nodes with higher score estimates and backtracking from low-quality regions of the tree, we can more effectively navigate the generative space and identify high- modes. An illustration of the tree search methods is provided in Fig. 2.
search
4.1.1 UNIFIED BFS- STYLE LINEAR SEARCH
Inspired by breadth-first search (BFS), which expands nodes level by level, we denoise set of N particles in parallel at each noise level. We score each intermediate particle { (xt k== using estimates of its verifier score fz and dynamically reallocate computation by sampling more children n for high-scoring nodes. We provide a general design space for schedules, scoring functions, and resampling strategies that unifies previous tree-search-basee and particle-based baselines such as SVDD (Li et al., 2024), DAS (Kim et al., 2025), and FK-stcering (Singhal et al., 2025b). The pseudocode is shown in Alg. 3, and more details can be found in Sec. D.I.
Tempering. To mitigate score estimation bias in early steps, Kim et al. (2025) increases weights on smaller time steps with Ty < 7r-1-... To, re-weighting scores with Te (x)))) Li et al. (2024) samples only from the top-scoring particle, i.e., T. X. We consider three different tempering design choices: Constant = Increase : = ((1 + +)) - 1) Inf : T. xc
Scoring, Following Kim al. (2025); Singhal et al. (2025b), we propose evaluate intermediate par- ticles/ fa+) using functions of the estimated rewards / ( along the sampling path s E [t, T.. This approach accounts for reward variation during sampling by considering not only the current reward but also its trajectory. Specifically, we define the following scoring functions [(=f) based on J (( for Current (=f())))) Difference =ef((=h)) f Max max,21 Txf (x .)..
Resampling. Given )(=?), we allocate particles as (n). ... ,n ) = Resample (N; w. ... w.
where nf is the number of children for at and w softmax (axf) is the softmax score of particle k. Multinomial resampling (Singhal et al., 2025b) samples n independently from the multinomial dis- tribution wx but suffers from large variance. Kim et al. (2025) adopts the Srinivasan sampling process (SSP) resampling (Alg. 4) for reduced variance. We compare the baseline Multinomial resampling and the variance-reduced SSP (Kim al., 2025); see Gerber al. (2020) for other methods.
5

---

Published as a conference paper at ICLR 2026
Prior methods can be seen special cases of BFS: SVDD (Li et al., 2024)= = BFS (Inf, Current, Multinomial); DAS (w/o grad) (Kim et al., 2025)= = BES (Increase, Difference, SSP) ; FK-steering (Singhal et al., 2025a) BES (Constant, Max, Multinomial). Ablations (Sec. 5.1) show SSP
resampling is key for performance, and our improved BFS (Increase, Max, SSP) consistently outperforms prior methods in global search efficiency.
4.1.2 DFS-STYLE NON-LINEAR SEARCH
Depth-first search (DFS) explores one branch of the search tree deeply as possible before back- tracking. In our setting, this corresponds to iteratively denoising a single particle until its verifier score drops below a predefined threshold: ((z0)) S òc, where 6, is threshold representing the quality requirement of the users.
Once the constraint is violated, the algorithm backtracks by reintroducing noise, moving to higher noise level fnex: = + AT using the forward diffusion process q( q( (2r) in Eq. 1. This allows the model to restart the denoising process from different region of the manifold, encouraging exploration of diverse modes. Compared with the fixed noise injection schedule used in SoP (Ma et al., 2025), DFS adopts score-adaptive exploration strategy that enables early backtracking for global restarts and prevents excessive compute from being allocated to easy problem instances. Together, these mechanisms enable more adaptive and computationally efficient global search.
key strength of DFS is its ability to allocate compute adaptively: difficult prompts and low-quality trajectories naturally trigger more backtracking and exploration, while easier instances are solved more directly. This dynamic behavior is driven purely by the verifier signal, without needing to know the difficulty in advance as in Snell al. (2024). Also, the threshold acts as a control knob for users to balance output quality and computation resources, where higher threshold automatically scales compute for better output. As shown in Sec. 5.2, this adaptive strategy leads substantial gains in efficiency and performance over prior methods, and even our improved BFS implementation. The pseudocode Alg. 5 and more details can found in Sec. D.2.
4.2 SCALING LOCAL SEARCH VIA LANGEVIN MCMC WITH VERIFIER GRADIENT
Global search can efficiently discover the high score modes within the base diffusion model, but struggles to generate higher quality samples with refined details and from low density regions. Thus, we aim to sample from the compositional distribution Po in Eq. 5 for higher quality samples. To optimize the compositional objective, we conduct local-search with hill-climbing methods, aiming to find the local maximum with high po. Specifically, we view the sampling problem as compositional optimization in measure space (Wibisono, 2018), and follow the gradient flow of KL-divergence, performing Langevin MCMC steps (details see Appendix. C.I).
Similar to annealed Langevin MCMC in Du al. (2023), we could construct series of annealed functions f(( with fo( = f(xo). Then we sample from the distributions qe (xx)) x (xx)) (x:) through Langevin MCMC in Eq. 2 (details see Appendix. C.2), Alternatively, training-free guidance in Eq. 3 utilizes the gradient of f (Zooje to optimize æt, which can be computed directly using the diffusion model output. However, naive gradient updates have been observed produce OOD and adversarial samples (Shen et al., 2024). In Ye et al. (2024), recurrence (Eq. 4) was found to help avoid adversarial samples in challenging guidance tasks, though its theoretical underpinnings remain poorly understood. We unify these two approaches by demonstrating that training-free guidance with recurrence, in the continuous limit, constitutes an instance of Langevin MCMC. For details see Appendix. C.3, and a rigorous convergence bound is in Theorem. 1.
Proposition 1. In the continuous limit where the number of diffission denoising steps T + oo, training free guidance with recurrence is equivalent to running Langevin MCMC on a series of annealed distributions {ūe (x:))))) with (o( = (o()))) x po(xa)[(2a))
Thus, the recurrence step (without guidance) can be interpreted as Langevin MCMC applied the original distribution of the diffusion model qi (..), and the guidance term A, in Eq. 3 then serves
'The original implementation of DAS uses a verifier- verifier-gradient saadient-guided guided proposal kernel and incorporates both the proposal probability and the base model prohability during resampling. For fair comparison, we disable the gradicnts of all methods in See. 5.1, and provide additional results in Sec. F.
6

---

Published as a conference paper at ICLR 2026
as defining practical annealing path fa( that bias the sampling path towards high reward areas beyond the modes of the base model. We are the first to propose this theoretical unification of the two lines of work, providing insights into efficient local search of diffusion models via gradients.
We implement local search by parameterizing the reverse transition kernel po (0 (x,) as a sequence of Langevin MCMC steps (Eq. 2), followed by denoising step using DDIM (Eq. 11) DDPM (Eq. 12); see Appendix C.5 for details. Unlike classifier-guidanee or naive training- faiiing-free guidance, which apply only gradient guidance in the denoising step, our approach incorporates explicit Langevin MCMC steps. In Sec. 5.3, we scale the number of local search steps for the first time and observe substantial improvements over pretrained models across multiple tasks.
EXPERIMENTS
In this section, we apply inference-time scaling with our search strategy across a range of domains. In Sec. 5.1, we present a strengthened BFS baseline that outperforms previous particle-based methods. In Sec. 5.2, we demonstrate the adaptivity and efficiency of our DFS method. In Sec. 5.3, scale up local search in challenging decision-making domains, highlighting the importance of jointly scaling local and global search. In Sec. 5.4, we propose a double-verifier strategy mitigate the reward hacking problem in inference-scaling.
5.1 ELUCIDATING THE DESIGN SPACE OF BFS
In this section, we explore the design choices of BFS, and present an improved BFS (Increase, Max. SSP) over prior particle-base or tree-search- Guo et al., 2025; Li et al., 2024; 2025b), where we disable the gradient guidance in DAS (Kim et al., 2025) as in other methods. We also compare with SoP (Ma et al., 2025) which adopts noise injection. To ensure a fair comparison, we directly use the official implementation of -t-steeiing (Singhal et al., 2025a), with the IeageReward (Xu et al., 2023a) verifier and prompts from the ImageReward paper". For ablation of design choices, we use the SD v1.5 model. For more details, see Appendix E.
methods (Singhal ct al., 2025b; Kim et al., 2025;
Table 1: Ablation of BFS design choices. For each section, we vary one design choice while keeping the others fixed. Left: Resampling methods (with Constant tempering, Max scoring). Center: Scoring functions (with SSP resampling, Constant tempering). Right: Tempering schedules (with SSP resampling, Max scoring).
Resampling
Scoeing
Tempering
Multiaomial
SSP
Current
Difference
Max
lnf
Constant
11.022 I1057
0.748 0.047
0.834
0.12 0.007
0,825 0.006
0.894
0,882 0.029
0.667 0,076
0.041
0.022
1,032
1.087
0.775 0.087
0.035
As mentioned in Sec.4.1.1, we experiment with different resampling, scoring, and tempering design choices. We begin with BES (Constant, Max, Multinomial) and evaluate various resampling strategies. As shown in Table I (left), SSP significantly outperforms naive multinomial resampling, and we adopt it in our design. We then assess scoring functions and tempering schedules under SSP resampling, as reported in Table (center) and (right). Here, we observe modest improvements with Max scoring and Increase tempering, leading to our final design: BFS (Increase, Max, SSP).
Table 2: Comparison our BES with prior methods
Maes
(Riggiii 3000
DAS pndroiencaa 3035
Io 2021
5V00 Cins, 2009
Decaccrti 2002
DFS (4E3)
5,212 1907
pues 5:851
0.322
2 020
6000
188 m8
L181 8153
115 0.603
1:002 e 14000
1,00
100 8.14
1
6014
1.144 0.056
1UXX]
3.113 3015
L345 1.45--8537 W537
1.104 0.60
1.578 0 5/615
1 2 8,005
1.104 2 4.002
1.360 24 0.222
1.205 1 1.0.03
To compare our improved BFS (Increase, Max, SSP) with prior baselines, we additionally exper- iment with the SDXL model (Podell et al., 2023) and FLUX.1 dev which differs from the model used in our ablations. As shown in Table 2, our improved BFS consistently outperforms previous methods across compute budgets and models. Among the baselines, DAS with out gradient guidance (Kim al., 2025) exhibits the smallest performance gap due resampling, while SoP (Ma et al., 2025) underperforms in global search efficiency because it allocates compute uniformly across all particles during the noising and denoising process when searching over paths. In the following experiments, we use the improved BFS as our strengthened baseline.
similar design space with SSP
Ww follow the settings of Table 1 in Singhal et al. (2025a)
wtw.:/acccccolaacafooccoeee wwweoooreeettaaaeeeaeeeeeee -abe//LU..a.aa.a/ee/eeaa dev
7

---

Published as a conference paper at ICLR 2026
5.2
ADAPTIVE AND EFFICIENT INTERECEE-SCALINE WITH DFS
DFS Pareto Curve
DFS Compute by Difficulty Level
Rase score
DTS score
Compute
0.76
0.72
Compute
0.20
Score
DFS-0.5
DFS-0.7
0.60
DF5-0.9
0.54
02
80
0.0
Compute
Difficulty
Figure 3: CompBench text-to-image results with DFS. Left: the Pareto curve of DFS, with DFS-8 denotes DFS with threshold 6; =. Right: average compute allocation by DFS for prompts of increasing difficulty.
In this section, we evaluate the adaptivity and efficiency of DFS on the CompBench dataset (Huang al., 2023), using the SSD-1B model (Gupta et al., 2024) and a VLM (Li et al., 2022) as our verifier. Through these experiments, we address the following questions:
Can DFS outperform Best-of-N and prior particle-based methods? As shown in Fig. E.2 (left), DFS consistently outperforms Best-of-N and also our improved BFS design across different compute budgets, with up 30% less computational cost.
Can DFS adjust compute allocation with different thresholds? We evaluate DFS across a wide range of practical threshold values (0.5, 0.7, and 0.9) and find that lower thresholds automatically allocate less compute, while higher thresholds scale up compute for better quality. DFS consistently outperforms baseline methods across all threshold choices, demonstrating the robustness of our method for different hyperparameters.
Can DFS dynamically adjust compute allocation for different instances? We measure the computa- tional cost DFS on prompts of varying difficulty under fixed thresholds, where the difficulty of prompt is defined as the average score of the base model over four random samples. As shown in Fig. (right), harder prompts with lower scores naturally consume more compute, without requiring prior knowledge of difficulty as in Snell al. (2024). This adaptive design reduces compute waste on easy instances and allocates additional compute to achieve greater improvement harder problems, evidenced by the score increase in Fig. (right).
The detailed setup is provided Appendix E.2, where we provide visualizations in Fig. 7 and comparisons with SoP (Ma et al., 2025) in Fig. 8. Unlike linear-searde methods that fixed exploration schedule, DFS offers higher efficiency and adaptivity, which may be of independent interest to the broader community.
5.3 JOINT SCALING LOCAL AND GLOBAL SEARCH
Although global search methods such as BFS and DFS can efficiently explore the generative space of the diffusion model, they are restricted to the modes of the base distribution and therefore cannot exceed the capabilities of the base model. To optimize the compositional objective in Eq, 5 and sample from high-reward regions beyond the base model, we propose scaling up local search steps via annealed Langevin MCMC, introducing a new scaling dimension for diffusion models. We validate the effectiveness of scaling local search in challenging decision-making domains, such as long-horizon planning and offline RI..
Baselines. To evaluate the effectiveness of scaling local search steps, we compare our method with particle-based DAS (Kim et al., 2025), which also leverages verifier gradients but only incorporates gradient guidance without recurrent local search. We also compare against the of-te-oftthe-at training-free free guidance method TFG (Ye et al., 2024), which performs multiple recurrence steps but lacks any global search. For faimess, compute is measured as the total NFEs of both local and global search. As shown in the following experiments, scaling local and global search separately yields suboptimal performance, whereas our joint scaling strategy establishes new Pareto frontier.
8

---

Published as a conference paper at ICLR 2026
5.3.1
LONG HORIZON PLANNING
Parero Curves with Different Local Search Steps
with
BoN-O
00
Bolk-1
BF5-6
Success rate (96
Buhl-
(99)
BoN-b
60
Dol-B
40
DAS
21
210
2
2
2
Compute (NFEs)
Compute (NFEs)
Figure 4: Pareto curves of local search. Left: Pareto curves of hest-of-N with different local search steps, where BoN-i denotes local search steps. Right: Pareto curves BFS and DFS with 6 local search steps.
Diffusion models have been widely adopted in planning for trajectory synthesis (Ubukata et al., 2024). We evaluate long-horizon planning in a challenging PointMaze environment, using the base model trained following Diffuser (Janner et al., 2022), with the verifier defined as total number of collisions between the trajectory and maze walls (see Appendix E.3 for details). (Luo al., 2024; Marcucci et al., 2023). As shown in Fig. (left), naive sampling without local search often resulted in failed trajectories that violates the maze layout in some of the segments. When scaling up local search steps in Fig. 5 (right), are able generate successful plans free of violation.
Figure 5: Illustration of Maze lay- out and task, with failed trajec- tory (left) and successfual sample (right).
To evaluate the compute efficiency of recurrent steps, we observe in Fig. 4 (left) that scaling up local search with BoN improves the overall Pareto frontier and significantly outperforms baseline methods. Scaling local search only as in TFG (Ye al., 2024) efficient with low compute budget but fails to scale with increased compute, as local search alone can become trapped in local optima. DAS (Kim et al., 2025) is more efficient than the corresponding BoN-O baseline without local search, but underperforms best-of-N when more local search steps are used. In Fig. 4 (right), we show that local search can be combined with global search techniques such as BFS and DFS to further improve scaling efficiency, demonstrating the flexibility of our framework.
5.3.2 OFFLINE REINFORCEMENT LEARNING
Table Performance D4RL locomotion tasks, highlighting results within 5% of the maximum.
Dataset
Environment
1QL
SIBC
DD
Diffuser
D-QL
QGPO
TFG
DAS
TISours)
Medium. Expert
HalfCheetah
79.8
Medium Expert
Hopper
91.5
108.6
111.8
107.2
110.7
108.0
100.2 1 3.5
105.4 L.1.
104.4 3.1
Medium- Expert
Walker2d
109.8
108.8
108.4
110.7
111.410.
Medium
HalfCheetah
47.4
45.9
49.1
44.2
50.6
54.1
53.1 03.1-:0..1
53.4 +: 0.1
54.8 0.1
Medium
Hopper
57.1
79.3
98.0
Medium
Walker2d
78.3
77.9
82,5
79.7
85.1
86.0
83.2
83.9 0.9
86.5 0.2
Modium--eplhe Replay
HalfChoctah
44.2
37.1
39.3
42.2
47.5
47.6
1..
42.2
47.8 0.4
Medium- Medium-Replan Replay
Hopper
94.7
86.2
100.0
96.8
100.7
96,9
03.1 0.1
96.7 3.0
97.4 4.0
Medium Replay
Walker2d
75.0
61.2
69.8
79.3 9.7
Average (Locomotion)
76.9
75.6
81.8
75.3
86.3
86.6
82.1
80.2
86.1
Recently, diffusion models have emerged as a powerful action prior in robotics due to their ability to model complex and multimodal distributions (Chi et al., 2023). However, these diffusion policies typically trained offline datasets and struggle adapt to reinforcement learning test-time requirements. Following Peters et al. (2010), we formulate the offline RL problem as sampling from Q Qereullarized distribution: x (a) s) x ((aas)e (s.a))) where Q is a learned Q-function representing preferences over actions, and is the behavior policy, which we model using diffusion prior. We approach this problem from the inference scaanin perspective, composing off-the-shelf pretrained diffusion policy with ground-truth Q-functions without training.

---

Published as a conference paper at ICLR 2026
Among the baselines, Diffuser (Janner et al., 2022). QGPO (Lu et al., 2023), and D-QL (Wang et al., 2022) require additional joint training of the diffusion model and Q-function. To demonstrate the effectiveness of our method test-time search (TTS), we allow TFG (Ye et al., 2024) and DAS (Kim et al., 2025) to use up to twice the compute of TTS. Since we use a small number of particles, we simply adopt BoN for global search of TTS and searched for the number of local search steps, with detailed hyper-parameter in Table 10. As shown in Table 3, TTS achieves performance comparable to training-baa -based baselines, while DAS and TFG struggles on the Medium and Medium- datasets where the model's capabilities are limited. Details see Sec. E.4.
Ad Advancing Online Learning with Policy Distillation. To enhance the capabilities of the base model, we fine-tune it on actions gener- ated by TTS and subsequently evaluate its one-shot sampling per- formance. We adapt the Medium datasets by replacing the original actions with those produced by TTS (see Sec. E.4.1 I for details). Per- haps surprisingly, this simple offline distillation outperforms SOTA diffusion-based online RL methods such as DPPO (Ren et al., 2024), as shown in Table 4. Compared with other offline RL methods such as PA-RL (Mark al., 2024),
Table 4: Results for policy distil- lation with TTS
Environment
TTS-distill
DPPO
Halfchoctah
6.7
47.8
Hopper
98.8 0.8
92.8
Walker2d
86.3 1 0.5
82.7
which also finetume the policy using optimized actions, our approach relies solely on pretrained verifiers and does not require iterative joint updates of the Q-function and policy. This enables more flexible composition of pretrained foundation models for verification and further highlights the effectiveness of inference scaling compared with naive action optimization. To the best of our knowledge, this is the first work to demonstrate self- self-improvement via inference scaling for diffusion models using pretrained verifiers, a direction that may be of independent interest.
5.4 MITIGATING REWARD HACKING WITH DOUBLE VERIFIER
In this section, we show that reward hacking and over-optimization can be mitigated by double-verifier strategy: employing separate verifiers for local and global search. As observed in Shen et al. (2024), training-frre guidance with verifier gradients is vulnerable to adversarial reward hacking: gradient guidance often over-optimize the verifier, causing it to classify them as belonging to the target class despite being out-of- distribuuut (OOD). Inspired by double-Q learning in reinforcement learning (Van Hasselt et al., 2016), we propose a double-veriffer approach, assigning distinct verifiers to local and global search to efficiently reduce overestimation.
Table 5: Results for ImageNet conditional generation with double-verifier using BoN and BFS.
Bingl
dou-le
BFS-single
dou-leuble
N
FID]
Acef
MSP
FIDI
Acct
MSP
FID.
Acc?
MSP
FID,
Acc
MSP
171.5
31.8%
0.161
151.2
37.5%
0.164
156.2
36.1
0.169
145.5
44.3%
0.181
8
155.7
35.8%
0.165
127.8
49.2%
0.184
133.3
46.5%
0.183
118.2
55.9%
0.214
We evaluate the proposed vorile-verifier on the challenging conditional ImageNet generation task, generating target-class samples from an unconditional model guided by a pretrained classifier defined only on clean samples. We report the FID and class accuracy of 256 samples. We also report the MSP score (Hendrycks & Gimpel, 2016), with higher MSP score indicating less OOD samples. For double-verifier, we use two distinct ImageNet classifiers for local' and global search respectively. We use another classifier for evaluating class accuracy avoid reward hacking. As shown in Table 5, using double-verifier significantly improves performance over single verifier with 2x less compute. Also, Gouble-verifier significantly reduces OOD samples indicated by the higher MSP score, improving robustness alongside efficiency. Visualizations and details see Appendix. E.5.
6 LIMITATIONS AND CONCLUSION
In this work, we present unified and principled framework for inference-time scaling of diffusion models, jointly scaling local and global search. A potential limitation of our approach is that inference- time scaling still requires additional hyperparameters. To address this, heuristics such evolutionary search can be adopted during hyperparameter tuning, which we leave for future work.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is con- ditional generative modeling all you need for decision Dacisinn-making? arXiv preprint arXiv:2211.156577 2022. 32
Michael Albergo and Eric Sanden-Eijnder Building normalizing flows with stochastic interpolants. arXiv preprint arXiv:2209./5571, 2022. 3
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. No. A iision-laguuage-cctii flow model for general robot control. arXiv preprini M.xiv:24100.4164. 2024. 1, 31
Huayu Chen, Cheng Lu, Chengyang Ying, Hang Su, and Jun Zhu, Offline reinforcement learning via high-fidelity generative behavior modeling. In The Eleventh International Conference on Learning Representations, 2023. 32
Xiang Cheng and Peter Bartlett. Convergence of langevin meme in kl-divergence. In Algorithmic learning theory, pp. 186 211. PMLR, 2018. 18
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal Robotics Research, pp. 02783649241273665 2023. 1, 9, 31
Hyungjin Chung and Jong Chul Ye. Score-based diffusion models for accelerated mri. Medical
image analysis, 80: 80:102479, 2022. 3, 17
Meihua Dang, Jiaqi Han, Minkai Xu, Kai Xu, Akash Srivastava, and Stefano Ermon. Inference- time scaling of diffusion language models with particle gibbs sampling. arXiv preprint arXiv:2507.08339. 2025. 3, 17
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis, Advances in neural information processing systems, 34:8780-8794, 2021. 1, 3, 4,
Yilun Du and Igor Mordatch. Implicit generation and modeling with energy based models. Advances in neural information processing systems, 32, 2019. 18
Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenhaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Composi- tional generation with energy-based diffusion models and memc. In International conference on machine learming, pp. 8489-8510. 10. PMLR, 2023. 2, 4, 6, 18
Yilun Du, Jiayuan Mao, and Joshua B. Tenenbaum. Learning Iterative Reasoning through Energy Diffusion, June 2024. URL http::tttt//it org. aa/aaaa///2// /20/.24066.00 11179. arXiv:2406.11177 [es]. 17
Alain Durmus and Eric Moulines. Non-asymptotic convergence analysis for the unadjusted langevin algorithm. arXiv preprint arXiv: 2015. 18
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari. Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024. 20
Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4rl: Datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219 2020. 32
Mathieu Gerber, Nicolas Chopin, and Nick Whiteley. Negative association. ordering and convergence of resampling methods, 2020. URL https:: :/apps:///arxvv.// /atxiv.vvvvv//////////a////avava org/ /t////77.018480 5, 25
Yingqing Guo, Yukang Yang, Hui Yuan, and Mengdi Wang. Training-fres free guidance beyond dif- ferentiability: Scalable path steering with tree search in diffusion and flow models, 2025. URL http://awxxvv.ooab/aas//as
https: fttts://arrivv wtg/ 11420. 2, 7, 16, 25, 27

---

Published as a conference paper at ICLR 2026
Yatharth Gupta, Vishnu V Jaddipal, Harish Prabhala, Sayak Paul, and Patrick Von Platen. Progressive knowledge distillation stable diffusion xl using layer level loss. arXiv preprint arXiv:2401.0267 2024. 8
Peter E Hart, Nils J Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics, 4(2):100-107, 1968.
Haoran He, Jiajun Liang, Xintao Wang, Pengfei Wan, Di Zhang. Kun Gai, and Ling Pan. Scaling image and video generation via test-time evolutionary search, 2025. URL httth:://aaxtii org/ abs/2505. 17618. 16
Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei- Hsiang Liao, Yuki Mitsufuji, Zico Kolter, Ruslan Salakhutdinov, et al. Manifold preserving guided diffusion. arXiv preprint arXiv:2311.16422 2023. 3, 17
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-o--distribution examples in neural networks. arXiv preprint 1r 2016. 10,
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12544 2022. 32
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020. 3, 3,20
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8646, 2022.
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723-7888 2023. 8,
Vincet Jain, Kusha Sarcen, Mohammad Pedramfar, and Siamak Ravanbakhsh. Diffusion tree sampling: Scalable inference-time alignment of diffusion models. arXiv preprint arXiv:2506.20701,2000 2025. 17
Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In International Conference on Machine Learning, pp. 9902-9915. PMLR, 2022. 9, 10, 30, 32
Daniel Kahneman. Thinking, fast and slow, macmillan, 2011. 16
Sunwoo Kim, Minkyu Kim, and Dongmin Park. Test-time alignment of diffusion models without
reward over-optimization. In The Thirteenth International Conference on Learning Representations, 2025. URL https: /ttp://oppp/eeeeeeeeee nwt/fooummiiiieeee http://rrmmmiiviiiiiii-ama 2, 3, 5, 6, 7, 8, 9, 10, 16, 24, 26, 27, 28, 32,
Scott Kirkpatrick, C Daniel Gelatt Jr. and Mario P Vecchi. Optimization by simulated annealing. science, 220(4598):671-666 1983. 18
Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXhv:2110.06769) 2021. 32
Gyubin Lee, Truong Nhat Nguyen Bao, Jaesik Yoon, Dongwoo Lee, Minsu Kim, Yoshua Bengio, and Sungjin Ahn. Adaptive inference-time scaling via cyclic diffusion search, 2025. URL https: /tttps://axivy www.aiiw..g//ab...... www.oogg//////ooooeee 14036. 3, 16
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vasion-language understanding and generation. In International conference on machine learning, pp. 12888-1299 PMLR, 2022.
Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Arsh Koneru, Yusuke Kato, Kazuki Kozuka, and Aditya Grover. Reflect-dit: Inference-time time scaling for text-to-image diffusion transformers via in-context reflection. arXiv preprint arXiv:2503.
00225533.22211 /2271, 2025a. 17
12

---

Published as a conference paper at ICLR 2026
Xiner Li, Yulai Zhao, Chenyu Wang, Gabriele Scalia, Gokeen Eraslan, Surag Nair, Tommaso Biancalami, Shuiwang Ji, Aviv Regev, Sergey Levine, et al. Derivative-fis guidance in continuous and discrete diffusion models with soft value-based decoding. arXiv preprint arX2v:2408.08825.. 2024. 2, 3, 5, 6, 7, 24, 25
Xiner Li, Masatoshi Uehara, Xingyu Su, Gabriele Scalia, Tommaso Biancalani, Aviv Regev, Sergey Levine, and Shuiwang Ji. Dynamie search for inference-time alignment in diffusion models. arXiv preprint arXiv:2503.02033, 2025b. 2, 3, 7, 16, 26, 27
Shanchuan Lin, Bingchen Liu. Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 5404-5411 2024. 26
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022. 3
Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.0788 2024. 1, 3,,1,,3
Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, and Jun Zhu. Contrastive energy prediction for exact energy-guided diffusion sampling offline reinforcement learning. In International Conference Machine Learning, pp.
PMLR, 2023. 1, 4, 10,
Yunhao Luo, Chen Sun, Joshua B Tenenbaum, and Yilun Du Potential based diffusion motion planning. arXiv preprint arXiv:2407.06169 2024. 9
Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijndee and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision, pp. 23-40. Springer, 2024. 3, 19, 20
Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, al. Inference-tine time scaling for diffusion models beyond scaling denoising steps. arXiv preprint arXiv:2501.09732 2025. 2, 3, 6, 7, 8, 16, 26, 27, 29, 30
Tobia Marcueci, Mark Petersen, David von Wrangel, and Russ Tedrake. Motion planning around obstacles with convex optimization. Science robotics, 8(84))ead)7843, 2023. 9, 9,30
Max Sobol Mark, Tian Gao, Georgia Gabriela Sampaio, Mohan Kumar Srirama, Archit Sharma, Chelsea Finn, and Aviral Kumar. Policy agnostic rl: Offline and online rl fine-tuning of any class and backbone. arXiv preprint arXiv:2412.06683 2024. 10
Edward F Moore. The shortest path through maze. In Proc. of the International Symposium on the Theory of Switching, pp. 285 282. Harvard University Press, 1959. 1
Mitsuhiko Nakamoto, Oier Mees, Aviral Kumar, and Sergey Levine. Steering your generalists:
Improving robotic foundation models via value guidance. 8th Annual Conference on Robot Learning, 2024. 31
Allen Newell, John C Shaw, and Herbert A Simon. Report on a general problem solving program. In IFIP congress, volume 256, pp. 64. Pittsburgh, PA, 1959. 16
Allen Newell, Herbert Alexander Simon, al. Human problem solving, volume 104. Prentice-hall Englewood Cliffs, NJ, 1972. 16
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusson probabilistic models.
In International conference on machine learning, pp. 8162-8171. PMLR, 2021. 3, 20
Scohong Park, Kevin Frans, Benjamin Eysenbach, and Sergey Levine. Ogbench: Benchmarking offline goal-conditioned rl. arXiv preprint arXiv:2410.20092, 2024. 30
Judea Pearl. Heuristics: intelligent search strategies for computer problem solving. Addison- -eellyy Longman Publishing Co., Inc., 1984. 1,5
13

---

Published as a conference paper at ICLR 2026
Jan Peters, Katharina Mulling, and Yasemin Altun. Relative entropy policy search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 24, pp. 1607-1612. 2010. 9
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952. 2023. 7
Allen Z Ren, Justin Lidard, Lars L Ankile, Anthony Simeonov, Pulkit Agrawal, Anirudha Majumdar, Benjamin Burchfiel, Hongkai Dai, and Max Simchowitz. Diffusion policy policy optimization. arXiv preprint arXiv:2409.00555 00588, 2024. 10, 33
Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall Press, USA, 3rd edition, 2009. ISBN 0136042597. 1
Yifei Shen, Xinyang Jiang, Yifan Yang, Yezhen Wang, Dongqi Han, and Dongsheng Li. Understand- ing and improving training- free loss-based diffusion guidance. Advances in Neural Information Processing Systems, 37:108974-109002, 2024. 10
Raghav Singhal, Zachary Horvitz, Ryan Teehan, Mengye Ren, Zhou Yu, Kathleen McKeown, and Rajesh Ranganath. A general framework for inference-time time scaling and steering of diffusion models. arXiv preprint 1xXiv:2501.06844 2025a. 2, 6, 7. 16, 17, 24, 26, 27
Raghav Singhal, Zachary Horvitz, Ryan Teehan, Mengye Ren, Zhou Yu, Kathleen McKeown, and Rajesh Ranganath. A general framework for inference-time scaling and steering of diffusion models, 2025b. URL https: //t///awvvvv////////////////aav/ www.aaiwwwaaaaaeaeaeaaaeeeooeeee hbrgg/abs//e tabs/2501. .00000006888.. 5,7
Steven A Sloman. The empirical case for two systems of reasoning. Psychological bulletin, 119(1):3, 1996. 16
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024. 6,
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020a. 3, 19
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference on Machine Learning, pp. 32483-32499 PMLR, 2023. 1, 17
Yang Song and Stefano Ermon. Generative Modeling by Estimating Gradients of the Data Distribution, October 2020. URL
.05600. 100.:1900.60000 18
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011. 13466, 2020b.
Robert Tarjan. Depth-fire feptt-first search and linear graph algorithms. SIAM journal computing, 1(2): 146-160, 1972.
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudcep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot. policy. arXiv preprint 2rXiv /22 2024. 1
Toshihide Ubukata, Jialong Li, and Kenji Tei. Diffusion model for planning: systematic literature review. arXiv preprint arXiv:2408. 10266, 2024.
Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double q- learning. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016. 10
dric Villani. Topics in optimal transportation. Graduate studies in mathematics, v. 58. American Mathematical Society, Providence, R.I, 2003. ISBN 082183312X. 22
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computa- fion, 23(7):1661-1674 201 2,3
14

---

Published as a conference paper at ICLR 2026
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference Computer Viston and Puttern Recognition, pp. 9228-8238, 2024."
Feng Wang and Zihao Yu. Coefficients presseern sampling for reinforcement learning with flow matching. arXiv preprint arXiv:2509.05952 2025. 27
Zhendong Wang. Jonathan Hunt, and Mingyuan Zhou, Diffusion policies as an expressive policy
class for offline reinforcement learning, arXiv preprint Sitiv::220.00699900 2022. 10, 31, 32
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-II), pp. 681-688. Citeseer, 2011. 4
Andre Wibisono. Sampling as optimization the space of measures: The langevin dynamics as composite optimization problem. In Conference on learning theory, pp. 2093-3027. PMLR, 2018. 6, 17, 22
Luhuan Wu, Brian Trippe, Christian Naesseth, David Blei, and John P Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems, 36:31372-31400 2023a. 2, 16
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.00 09341, 2023b. 28
Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Chengyue Wu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, et al. Sana 1.5: Efficient scaling training-time and inference-time compute in linear diffusion transformer. arXiv preprint arXiv:250J. 18427, 2025. 1
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong. Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: learning and evaluating human preferences for text-to-iman image generation. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pp.
15903-15935, 2023a. 7,7
Yilun Xu, Mingyang Deng. Xiang Cheng, Yonglong Tian, Ziming Liu, and Tommi Jaakkola. Restart sampling for improving generative processes. Advances in Neural Information Processing Systems, 36:76806-768335 2023b. 16
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36: 11011110911122 11822, 2023. 16
Haotian Ye, Haowei Lin, Jiaqi Han, Minkai Xu, Sheng Liu, Yitao Liang, Jianzhu Ma, James Y Zou, and Stefano Ermon. Tfg: Unified training-free guidance for diffusion models. Advances in Neural Information Processing Systems, 37:22370-2244 22417, 2024. 1, 3, 6, 8, 10, 17, 21, 23, 26. 32
Jaesik Yoon, Hyeonseo Cho, Doojin Baek, Yoshua Bengio, and Sungjin Ahn. Monte carlo tree diffusion for system 2 planning. arXiv preprint arXiv:2502.0721 2025, 17
Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training- free mmergy-guided conditional diffusion model. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 23174 23174-33184 23184, 2023. 3,4, 17
Stephen Zhao, Rob Brekelmans, Alireza Makhzani, and Roger Grosse. Probabilistic inference in language models via twisted sequential monte carlo. arXiv preprint 2rX5v.2404.17546 2024. 16
15

---

Published as a conference paper at ICLR 2026
A APPENDIX OVERVIEW
In Sec. B, we provide a in-depth review of literature related to inference-time scaling and diffusion models. In Sec. C, elaborate on local search with Langevin MCMC, and in Sec. D provide the pseudo code and design of global search algorithms BFS and DFS. In Sec. E, we provide the details of all the experiments.
B ADDITIONAL RELATED WORKS
B.1 DISCUSSIONS ON CONCURRENT WORKS
Adaptive Bi-directional Cyclic Diffusion (ABCD) (Lee et al., 2025) introduces search-based inference-scaling framework that combines elements of both DFS and BFS. Unlike DFS, which relies on a quality threshold to determine backtracking, ABCD maintains set of particles and backtracks by propagating them across all noise levels. The termination criterion is based on whether backtracking to higher noise levels improves sample quality. Compared with DFS, ABCD achieves smaller score estimation errors since particles are evaluated after full denoising, and it explores the generative space more thoroughly by combining BFS and DFS. However, ABCD cannot adaptively allocate compute across different instances due to its fixed termination condition, and it expends unnecessary compute on easier instances because it denoises full set of particles regardless of sample quality.
EvoSearch (He et al., 2025) proposes evolutionary search to scale inference compute for image and video generation. At selected timesteps, particles fully denoised; high-scoring particles retained, while low-scoring ones are perturbed via noise injection. This method improves over the FK-steering baseline (Singhal et al., 2025a).
In contrast, our work develops a uniform and principled framework grounded in classical search heuristics. BFS and DFS serve fundamental building blocks for more advanced search methods, and, crucially, we are the first to scale local search using annealed Langevin MCMC. This local search can be seamlessly combined with global search strategies, joint scaling approach that show is critical for success in challenging decision makinng domains.
B.2 INFERENCE- TIME SCALING WITH CLASSICAL SEARCH
We now review the use of classical search for inference- time scaling, including applications beyond diffusion models.
Inference-time compute scaling, often viewed as "slow thinking," " has long-standing roots in cognitive science, where it corresponds to "system 2" reasoning (Kahneman, 2011; 1; Sloman, 1996). In early work, Newell et al. (1959; 1972) formalized problem solving as tree search in combinatorial spaces. More recently, Yao et al. (2023) proposed tree-of-thoughts reasoning for LLMs, enabling exploration along multiple reasoning paths using BFS and DFS as strategic search primitives.
B.3 INFERENCE SCALING IN DIFFUSION MODELS WITHOUT DIFFERENTIABLE OPTIMIZATION
We next provide a more detailed overview of gradient-free inference-scaling approaches in diffusion models.
Inference-time compute for diffusion models largely determined by the number of denoising steps. Xu et al. (2023b) proposed recursive restart sampling to mitigate cumulative errors, effectively scaling the number of denoising steps. More recently, Singhal al. (2025a); Kim et al. (2025) introduced Sequential Monte Carlo (SMC) approaches (Wu et al., 2023a; Zhao et al., 2024), which can be inter- preted as BtS-style algorithms from our trme-iampling perspective, since they search over nodes at the same depth before denoising further. Notably, SMC style methods propose to use different proposal transition kernel such as the gradient guided sampler N (Ie (x;) + V.. log f (xojt) ot). which we disable during our global search experiments. Their implementations differ mainly in the choice of scoring functions and resampling strategies, with Kim et al. (2025) proposing an increasing tempering schedule. Li et al. (2025b); Guo et al. (2025) proposed beam- search style tree search methods, where Li et al. (2025b) proposed to adopt dynamic beam-size and tree-width during the sampling process. Ma et al. (2025) further explored inference-tine scaling for image generation
16

---

Published as a conference paper at ICLR 2026
via a combination of local zero-order search and global path search, experimenting with different verifiers (oracle, selff-uuprviseed, and lask-aligned) and analyzing verifier-tass alignment. However, they did not systematically evaluate and improve the compute efficiency of search methods, while demonstrate significantly improved efficiency with our novel design in search methods. Additionally, Yoon al. (2025) proposed Monte Carlo Tree Search-based diffusion framework tailored for trajectory synthesis, which denoises different trajectory segments at distinct steps. More recently, Jain al. (2025) proposed integrate the value backup in MCTS into the tree sampling and search of diffusion models, which can utilize information from low noise states and previous sampling paths. For diffusion language models, Dang et al. (2025) proposed particle Gibbs sampling methods that outperform naive SMC (Singhal et al., 2025a) approaches.
Beyond search- based methods, Li et al. (2025a) leverage the in-context learning abilities of foundation models to enable revision during sampling. Their approach exploits multimodal VIM capabilities to provide feedback on intermediate generations and trains the model condition on both prior samples and corresponding feedback.
B.4 GRADIENT GUIDANCE OF DIFFUSION MODELS
We review guidance methods for diffusion models that leverage gradients from verifiers or classifiers. To sample from conditional distribution
Po(xo) x (x p(ckzo).
where p denotes a classifier, one can define conditional diffusion process {āc( with initial distribution Po- The corresponding conditional score function
Væ, log To (xx)) = V log (xx)) + Va, Poglx())
where pe(c)ar.) [ Classifier guidance (Dhariwal & Nichol, 2021) trains noise- dependent classifier (c( noisy samples, using data from the diffusion model together with the base classifier. However, this training procedure is computationally expensive and impractical when the base diffuusion model cannot generate target-class samples.
Training-free guidance methods (Ye et al., 2024; Chung & Ye, 2022; Song et al., 2023; Yu et al., 2023; He et al., 2023) approximate the intractable (c(ca))) with p(c( D(c[Bezalee [aro]), which can be interpreted as a first-order approximation. These methods introduce bias, though recurrence has been shown mitigate outoo--distribution artifacts and improve performance in glassifier-guides generation (Ye et al., 2024). Nevertheless, the theoretical foundations and scaling behavior of recurrent guidance remain largely unexplored.
In this work, we instead adopt Langevin MCMC sampling in place of score-based ODE sampling. This approach guarantees asymptotic exactness using only the base verifier, without requiring additional training. Relatedly, Du et al. (2024) proposed annealed energy-based models (EBMs) combined with iterative MCMC for inference scaling and reasoning, but their method cannot be applied to pretrained diffusion models and verifiers. For the first time, establish connection between annealed Langevin MCMC and recurrent guidance, thereby enabling inference scaling with pretrained foundation models.
C DETAILS ABOUT LOCAL SEARCH WITH LANGEVIN MCMC
In this section we provide a comprehensive and detailed overview (annealed) Langevin MCMC based methods used in local search, well proving Proposition. 1.
C.1 LANGEVIN MCMC AS GRADIENT FLOW IN MEASURE SPACE
Following Wibisono (2018), the Langevin SDE in sample space corresponds to gradient flow of the KL-divergence in measure space. Here we provide brief overview.
Define our target distribution that we wish to sample from as v, and the distribution of our current sample p. We define the KL-divergence (relative entropy)
Hp(P) / p pog
(6)
17

---

Published as a conference paper at ICLR 2026
Thus, sampling from can be minimizing H, since the minimum of H is achieved at p with (p) O. Furthermore, v is the only stationary point H even for multimodal distributions. Thus sample from when optimizing H via gradient based methods.
We have the gradient flow of in Eq. 6 follows the following PDE:
3p_ V (-V(-log v)) + Ap;
Ōt
which is known as the Fokker-Planck equation. Here, / p( 1) is smooth positive density evolving through time, driven by the dynamics of the sample x. The dynamics in sample space corresponding to Eq. 7 is the Langevin SDE:
dx; = V log (z,) + v2dw,,
(8)
where (22 is a stochastic process with measure Pt. and (w,)r20 is standard Brownian motion. That is, Øy ~ Pt evolves according to the dynamics in Eq. 8, then the measure p(x; t) = Pt evolves according to the PDE in Eq. 7, conducting gradient optimization in measure space.
In practice, we implement Eq. 8 through discretization, which is known as the unadjusted Langevin algorithm (ULA):
x 5+1 = x' + V/V= xx' (og((x"))))) + 22
(9)
with e¹ ~ NO I). When 7) + 0. the ULA converges to Langevin SDE, providing exact sampling.
Previous works (Durmus& & Moulines, 2015; Cheng & Bartlett, 2018) show that ULA can efficiently converge to the target measure if v is log-concave and smooth. However, when facing complex and multimodal distributions, we can only guarantee convergence to the concave vicinity.
C.2 ANNEALED LANGEVIN MCMC SAMPLING
Langevin MCMC have been used to perform implicit sampling in energy-based models (Du & Mordatch, 2019) and acore-based models (Song & Ermon, 2020). However, these methods suffer from inaccurate score estimation and low density regions (Song & Ermon, 2020). In Song & Ermon (2020) they propose perturb the data with gaussian noise, eventually smoothing the data distribution:
g(are) =
no. Po(ar) To, of
20
and creating sequence of annealed distributions (g(x+) ) which converges to Po (arg). Since they are smoothed by gaussian noise, we can improve the mixing time of Langevin MCMC on multimodal distributions by sampling from these intermediate distributions, sharing similar spirits with simulated annealing (Kirkpatrick et al., 1983).
In Du et al. (2023), they extend this method to compositional generation of diffusion models. Specifically, we consider sampling from a product distribution ph p (ær) x pó (2:))) (z)). where (a())) and pă( (r)) are distributions of different diffusion models. Since we have access to the score functions Væ, log qi (2x) and Vz, lo (xx,) through the diffusion model, we can construct sequence of annealing distributions c. (xx) such that:
V log sprod (IL) (x1)) = Va, log (xx) + Va, log (xx).
{ (ame (c:)). }. we can arrive at % gmod (wo) which is equal to pb prxd (2t)).
By sampling from the sequence
key difference from sampling from ppood (x:)) and direct diffusion sampling is that the diffusion process with prod (xx) defined as
of prod (xx)
/ aco
(2ro)
is different from alood (xx). The score ge prod (xx) can be derived as:
V log of prod (xxe) = Va, log ( (o
18

---

Published as a conference paper at ICLR 2026
which is not equal to
V log gonod (x:)) Vu. log (/,, pr( ((11100)))))))) Va, log
(f pā ( 200
and thus intractable to compute directly.
key distinction between annealed Langevin MCMC sampling and reverse diffusion sampling is that we run multiple Langevin MCMC steps on the same noise level, while reverse diffusion goes from high noise level to low noise level via denoising. A minimal pseudo code shown in Alg. 1.
Algorithm Annealed Langevin MCMC sampling
Input: sequence of annealing distributions {ăe {x=))) number of MCMC steps N, step size {ne m (Optional) reverse transition kernel {5e (20 Init: 27 ~ (0,
for 1 = -T...... do
for 0,1, ,N - do
Perform Langevin MCMC steps:
x ++ x
+
n1 V log (x(x)) + +. /ni,, ct ~N(101
end for (Optional) transit to next time step: 2b_1 ). If no reverse kernel initialize x/_1 xx. end for Return Cy
C.3 ANNEALED LANGEVIN MCMC WITH RECURRENT TRAINING- GUIDANCE
In this section, we prove the connection between annealed Langevin MCMC (Alg. 1) and training-free guidance (Alg. 2) in Proposition. We divide the proof into two parts. Sec. C.3. we prove the equivalence between naive recurrence steps and Langevin MCMC. Then in Sec. C.3.2, we prove that adding the guidance term is defining an annealing path that biases towards high score regions. Finally, we provide rigorous convergence analysis in Sec. C.3.3.
C.3.1 EQUIVALENCE BETWEEN LANGEVIN MCMC AND NAIVE RECURRENCE
Consider the diffusion process with the following stochastic interpolant (Ma et al., 2024):
It CyTo + o.e.
We denote the score function of (xx)) as V æ, log (xx) s(x,, t). Recall the forward process in Eq. 1:
o²
oi of E.
2; a-11
x- +
af
(10)
a
In a recurrence step Line. 5, we first solve xi_ from x; using the learned score function s(x), f), then add noise to æi_ to obtain the recurrent sample a;+1, where the superscript denotes the recurrence step index:
0,1, .Ncur Depending on different solvers, we have different
formulations of aztt1
DDIM sampler. When using DDIM (Song et al., 2020a) sampler, we have the reverse step as:
Tz-1
0t-1
(1-1 0t-1
s(xe,)),
(11)
at
at
ot
where s(x,, t) is the score function Va, log ge (xx)) Thus, we have:
0 o of-16
at xt-1 + ail
x
a:-1
of
01-1
a + oz
1- ai 01-1
s s(xi 1) + +oo
af o2
0t-1
a:
0
o
19

---

Published as a conference paper at ICLR 2026
Denote A log or
then we have:
0,
++1 = xi + of (1 e s( s(z 1) + 0. .o.V e2lAr-At - 1)
æ + of (1 - s s(2// t) σIV - (1- (1 + e - Ae-1)E
where 1 + - + 2 when T - and denoising step size approaches 0, as A A--1 - 0. DDPM sampler. In DDPM (Ho al., 2020), we parametrize the posterior distribution as: po (x,) N (z pa t), (x,,,))
(12)
where the posterior mean is:
po( t) ar-1 xz +
- - ac-1 ( s(xr, t).
01:0011 -
at
Ho et al. (2020) parameterizes the posterior variance as (x) 1) 3,1 or (o( B,1:
oo at
of
Be = a
0
B. 0
3e,
oi
2
while Nichol Dhariwal (2021) propose to train the posterior variance as Ex( = exp lolog Br + (1 v) log (.)
Thus, a backward step can be written as:
Ott xt + o?
04-1 of
at
s(x1,t) t) + I//² (xx Epos
ae
04-1
where Epost denotes the noise added in the posterior sampling step. Then, we can write the recurrence step as:
a +1 = 0 0f -f- o2
ap-1 -2-
af
of of s(z), + 2x)/2(x0,, e) + ay
of
forward
x. + B,s(2), 1) + V Ex( 3,IC'
where Eo(x,, + 3.I when T co, and the denoising step size approaches 0.
Flow-Matching sampler. In flow-matching models (Ma et al., 2024; Esser et al., 2024), the noise schedule is a₂ = 1- o, t, with & [0. 1]. The denoising step can be written as:
3t 1t (2e, A)
xt tts( t)
Dt At aa + lAt
A
(1
s(x,, t)
1-t t
Then, the forward process in Eq. 10 can be written as:
1-t Z1-11
At (2t At 2f 2tAt)
It (t At)
C.
1 1-t At
Thus, the recurrence equation can be written as:
tAt
At (2t At 21 2tAt)
z
x
1- t
1- + At
2tAt (1 - t At) At2
x
s(a,,
t
1 1-t + At
Putting together. In general, we can write the recurrence step as:
2 ا+ = o + arr,s(a(, t) + 22
(13)
with d + 0 and rt 1 as the denoising step size approaches 0:
20

---

Published as a conference paper at ICLR 2026
For DDIM sampler, we have O₂ = }
and
r1 = 1te A1-1
2
For DDPM sampler, we have de f0²
o
and 1
For flow-matching sampler, have az
AAt
and 1 S rt $ 8(1-1)
Thus, it can be approximation of the ULA in Eq. 9, and also discretization of the Langevin SDE in Eq. 8.
C.3.2 ANNEALED LANGEVIN MCMC WITH GUIDANCE
When applying training free guidance (Ye al., 2024) during the recurrence, we have:
xt ++1 x; af), f) - /2a,E' + A (
where dt, by are the coefficients of the recurrence equation in Eq. 13 without guidance. In general, A = Pr V log f (aojt) + 14 V mote log f (Zojt). where Pt. controls the guidance strength. We then show that the guidance term can considered the score function of of annealed verifiers {f(zz)
When considering 'variance guidance" in Line. 7, we have A = Pt V.V. log ((toj)) Thus, we can define f(=" (auje), which satisfies for f200). Similarly, for 'mean guidance" in Line. 8, we have
mean - 11 Vaoi log f(
of Ze log ((
114
Σ0/z
where the second Equation follows from Lemma 3.3 in Ye at. (2024). Thus, there exists set of functions fimeam (xx) such that Va, log fr fream (xx) = Enfe
Vz. log f(xn and we can see that when t - V,V m, log (xx)) = V., log (mo)) If we additionally incorporate the 'implicit dynamics' in Line. our arguments still stands since the smoothed objective f(x) E8-N + 5),6) converges to f with + and o, +
Combining the two terms together, we have A, = ct Var, log fe( with fo = fra frmen Thus, recurrence with guidance can be written as:
a i+1 = at + u.ris(ar, 1) + 22 + a nn, log f(((x))
a + arr. log a: (Zx (
V20t6',
Thus, we have defined the annealing path as (x,) = q. (x,) (2x)) t = 1,2,
C.3.3 CONVERGENCE ANALYSIS
In this section, provide rigorous convergence analysis of recurrence to the target distribution Te (#x).
Theorem 1. Suppose de (2x)) has bounded support, is a-strongly log-concave and L-llo-ssmothh, and -V log q. is M -Lpisschitz. Denote x Nuw the sample after Nrew steps of recurrence, we can bound the Wasserstein distance between the distribution of xx New and qt as:
Nnw W2(p(x.Nun ), (.) - (O
c- -2 + 61-e-220 +e -2h--1 )mar)
where log Se is half of the log SNR. ae
Proof. Recall recurrence is equivalent to the following recursion equation, with ar Tt being sampler- dependent parameters in Eq. 13:
++ t
x + atr' Van æ, (x)) + 220c²
x as V a, log qt (x))** + c2a,c'
21

---

Published as a conference paper at ICLR 2026
Thus, recurrence is equivalent to running unadjusted Langevin algorithm (ULA) on the tempered distribution plempered x q.. Using Lemma and Lemma 2 from Wibisono (2018), given the regularity conditions 9:. bound the discretization error from ULA as:
Wpmprrde pp(,, (ax)N=") (a: (1 aa) ),a))
= 0 of 0
oi
0 Nucr
2
+ (1
+
i
1
ar-1
-0 (e
2d, 2he-1 (1
22 + 2Ae-1
To bound (ompered gt), we first bound the TV distance as TV (t).
Denote Z(ri)= f q (z) dx, and (b)) log (r)). We have:
KL lompered 4t)
E Eppeeens (r log q. (rr)) log Tcl
$ 0 ((r: poppan Dlo q: - (')) -1)
where the last step is using Taylor expansion. Since (r) Expreome Dlg del and (r) Var pempona (log gr). Using the Pinsker inequality, we have (p g.)) S 1 (L 4x) S O (Tre 1)v Var, (log q O(r, - 1), given the finiteness of Varge [log @: under bounded
support.
Following Proposition 7.10 in Villani (2003) for distributions with bounded support, we have:
W2 (p
0
TV (propered qb)
0
(Vre
oi-1 o?
040t-1
0
1 --m
a4-10,
Tt.
log ar-1 log σ1
log
+ max
---1
---1
0111
0t
log
+ loog
a4
0z-1
0 A -xx) t1)
where we recall the definition of r; in Eq. 13.
Putting together the bound on Wz (p tompered pr Nacu)) )) and Wytteenperod ,e)), we obtain our desired bound.
C.4 RELATIONSHIP BETWEEN LANGEVIN MCMC AND GRADIENT ASCENT
In training-free guidance, most prior works only apply gradient ascent without recurrence. Here we provide a theoretical analysis of both methods.
Recall the KL-divergence objective in Eq. 6, which can be further decomposed when are sampling from a compositional distribution of (o and verifier f(0o), with v x Po f:
Hp(p) Epl- fl H.. (p) + log Z.
where Z = f fof is a normalization constant. Thus, gradient ascent is optimizing the verifier objective E,I - log f). while Langevin MCMC in Eq. 13 is optimizing the divergence between current sample and base distribution (p). This explains why naive gradient updates leads to OOD samples, and recurrence effectively mitigates this issue, acting contraction force pulling the
22

---

Published as a conference paper at ICLR 2026
sample back to the original manifold. However, since start from Po the distribution of our initial sample, sometimes we can omit the recurrence the guidance strength is small. But we wish to traverse different modes with multiple gradient updates, introducing recurrence helps avoid OOD during optimization.
C.5 IMPLEMENTING LOCAL SEARCH WITH TFG HYPER-PARAMETE SPACE
Due to the equivalence between annealed Langevin MCMC and training free guidance with recur- rence, we can implement local search with Langevin MCMC using the TFG framework of Ye et al. (2024), efficiently searching the hyperparameters. Here we provide a overview of the algorithm and design space. Following Sec. C.3, every iteration of recurrence in Line. 5 is equivalent to an annealed Langevin MCMC step, thus Niccur is equal to the number of local search steps.
Algorithm 2 Training- Free Guidance
I: Input: Unconditional diffusion model 6ø, verifier f. guidance strength 44 7. number of steps T, Nuwar, Nieer
2: Ør (... I)
for to =T....... ,
4: Define function f(x) = 50,8)
for 1,... NNvcur do
#ojtt = (#:: 0,50 (ate f))/ar
A p vv, log (wopt) A mean menn + Mack V Vaoii log f(Woje + Amem) 2t-1 Sample(as, aofes 1)
Diterate New times starting from A
at-1 (Ane mema)
Sample follows DDIM or DDPM
at
10:
N
01 01-1; Oi Ge-1
c oi
07-1 )
Recurrent strategy
11:
end for
12:
end for
13:
Output: Conditional sample me
For time varying schedules Pt. Mrs follow Ye et al. (2024) and propose either the 'increase' schedule:
Qt 01/01-1
8: = T
(14)
Σ 01/01-11
where we increase the guidance strength as we denoise: < 8T-1 < ... < or the 'constant" schedule
(15)
which uses constant parameters throughout the denoising process. Thus, the dime-varying schedules can be computed P: 8:p and 8tf4, and we only need to determine the average and jr.
D GLOBAL SEARCH OF DENOISING DIFFUSION MODELS
In this section, we provide details about the global search algorithms: BFS and DFS.
D.1 BFS-BASED SEARCH
We present the pseudo code for BFS in Alg. In practice, we can evaluate and resample the particles at fixed subset of time steps, with detailed hyper-parameten in the details of each experiment.
D.1.1 OVERVIEW OF DESIGN SPACE
Tempering. When estimating the particle score via the denoised output f(= for larger t the estimation often has larger bias, due to the fact that [[f())) * ([[xxo/xx) f f( Thus, we propose to re-weight the score estimates with a tempering schedule t, before scoring the particles. For increasing schedule 7y 7T-1
ni < 7o, the estimated scores have more influence
23

---

Published as a conference paper at ICLR 2026
Algorithm 3 Diffusion BFS
Diffusion input:
diffusion model Cy with diffusion time steps 7 and proposal transition kernel
(pe (1 Verifier f.
BFS input: Tempering schedule тG. Budget of particles N. Scoring function and resampling method.
Init: Random sample N particles 24 ~ k; ....
N.
for L = T....... 1 do
for - 1,2,....A .N do Estimation. Estimate the conditional mean: 20 2 00,e0(0
Compute the verifier
score estimate f(z)))).
Scoring. Score the particles according to the scoring functions:
Current f(x,) = Ti. (
Difference: f(x,t) = Ti. Ta( Speet
Max M(x{) max (T (a Piec))
Resampling, Compute the weights
(w, wi:- -.. .., wx")
softmax (fl))) ... -F(x,())))
and resample the children
(nt, ni ...... n Resample(NV w! wp ,.,,,")),),
where Resample can be Multinomial resampling SSP resampling (Alg.4). end for for k = 1,... N do
Sample n children particles from z/: x/-1 -j= ... ,,, end for Update the score buffers for computing the Difference and Max score next timestep.
f (2. (=ppeeet(())
if scoring function is Max
Pr
parcentt))
ref
zofe
scoring function Difference
end for
Return Zy argmax, a..
".. a f(x())
smaller time steps with higher weights, where the estimation has less bias. For the increasing temper- ing schedule, we simply adopt the design in DAS (Kim et al., 2025) with T. ((1 + 7)) 1) T. with searched in (0.008, 0.024). We also consider the deterministic particle selection in SVDD (Li et al., 2024) where we only retain the highest scoring particle, which can be seen as т = cc. For the constant tempering schedule we simply set T. т, without any tempering different time steps.
Scoring. Given the score estimates f( one can construct scoring rules based on the scores of the entire path { T= In traditional sequential monte-carlo (SMC) based methods, the sequential-momtecaatee
score is computed the difference between two consecutive evaluations: f(x!)) (=)))) (= However, in FK-steering (Singhal et al., 2025a), they pointed out that the difference potential is unfair for sample path that saturates earlier, reaching a high reward at earlier time steps. Thus they propose the max scoring: f(xt) max Tif (x ..)... We also consider the simple baseline of current score: M(x)t Te/ (=)
24

---

Resampling. Given the scores f(x)), ). we can compute the softmax logits of particles via {we N softmax {f(xt) sK= Then, we resample the particles according the logits, and obtain the number n Published as a conference paper at ICLR 2026
J==
of children for each particle n². The sets { {nt k=1 should satisfy the following constraints:
Σ 2 =N
E[n,]
=Nw
Most previous approaches simply adopt the multinomial resampling, with {nt] k=1
N k=1). However, this sampling process introduces large variance since every Multinomial (N; (wt
component is sampled independently. To reduce variance, for example, residual resampling only resamples the residuals nnद from N R with weights प.f N==-(NW.) where R = I [N=I[Nuf]] Then, we obtain n [Nw*] n... This way we reduce the randomness in nē.
N-R
The SSP resampling views the sampling process as randomized rounding on the expectations Nuf, with pseudo code from Gerber et al. (2020) shown below in Alg. 4.
Algorithm 4 SSP resampling
Inputs: E [0, 1|ah and (EI E RN such that S.. Nw with wp" being the softmax
weight of the n-th particle, and we have Σ Sn = NNE N.
Output: (Yah (Y), Y (u)) E NV such that ΣN y (up(u)) (u) = Σ En.
Initialization: (Yd (u), ..... Y. (u)) (EI,...,EN) EN), (n.m, k) 21,
Iterate the following steps until (Y (u). (")) E N
(1) Let 5 be the smallest number in (0, 1) such that at least one of Y (u) + d or Y (u) - 8 is an integer, and let be the smallest number in (0, 1) such that at least one of ssp (u) or Y (u) + é is an integer.
(2) up I c/(ō + E) set (Y (u). Ym (u)) + (Y) + Yap (w) 8); otherwise set (Y (u), Yap (u)) (Ymp(u) E, Yap (u) + E).
(3) Update and
follows:
1. If (Yssp (u), Ysp(()) N2, (n, m) + (m + 1,m + 2);
Y (u) E N and Y.p (u) & N set (n, m) - (m, m + 1);
Ypp( (u) and 2p (u) set (n, m) < (n, m 1).
(4) 1 t k +1
There are more variance reduced sampling methods such systematic resampling and stratified resampling, and we adopt the Srinivasan sampling process (SSP) resampling as our design choice. For other variance-reduced methods, the performance is similar to SSP resampling, all outperforming naive multinomial resampling.
D.1.2 OVERVIEW OF PRIOR METHODS
Here, we provide an overview of prior methods.
SVDD (Li et al., 2024). In SVDD, the best sample is selected at each time step, from which M children are generated. This approach can be viewed as a variant of BFS with 7 = oo and M particles. Nodes evaluated using the current score estimate f(
TreeG (Guo et al., 2025). In TreeG, particles are ranked and the top M are either selected directly or resampled based their scores to obtain M samples. From each selected particle, K children
25

---

Published as a conference paper at ICLR 2026
sampled, resulting in effective tree width of K.M. Particles are evaluated using their current score f They also consider adding Gaussian noise to Tnj to approximate the posterior distribution (x)) However, the posterior distribution is intractable as not a Gaussian distribution, and exact Gaussian approximation requires computing the variance using the Jacobian of the score function (see Ye et al. (2024)).
DAS (Kim et al., 2025). In DAS, the authors propose an exponentially increasing tempering schedule 1, and also introduce an adaptive tempering schedule. They adopt advanced SSP resampling instead of multinomial resampling, and evaluate particles based on the difference in rewards from the previous evaluation, similar to SMC methods. In the original implementation DAS also modify the transition kernel with verifier gradients N (It-1; + 2V log (*]]:)) and include the log ratio in the resampling weights: wi = log po(2); [ log ( 1 | + + llg f(z)) A Alog f(x ). For a fair comparison, we disable the gradients in the BFS experiments in Table. 2. and provide additional results in Table. 7.
as the default, given by 7 = +1+
FK-steering (Singhal et al., 2025a). In FK, the authors propose several options for evaluating intermediate particles, including difference, max, and sum, with max adopted as the default. In the official implementation, multinomial resampling is used, which may lead to suboptimal performance.
SoP (Ma et al., 2025). In saarch-over-patee (SoP), we start with N samples and denoise them to low noise level After that, we sample M i.i.d noise for each of the noisy particles and run the forward process from a to a + Aʃ, and then denoise from o Aʃ to a + Aʃ Ds, After that we select the top N particles and repeat the above process until 0. Although the noise injection is similar to the backtracking in DFS, we point out that SoP adopts fixed schedule and all particles remain at the same noise level, which different from adaptive DFS.
DSearch (Li et al., 2025b). In DSearch, the authors improve upon beam-search based tree search by dynamically adjust the beam-size and tree width during sampling. At lower noise levels, adopt larger beam-size and lower tree width, facilitating more local exploration. This approach improves the diversity of the generated samples.
D.2 DFS-BASED SEARCII
In this section, we provide the details and pseudo code for DFS in Alg. 5. To better utilize previously explored sampling paths, we employ a buffer to store prior results. When the budget exhausted and no more backtracking is allowed, we retrieve the best sample from the buffer.
Similar to BFS, controlling the set of evaluation steps allows a trade-off between efficiency and accuracy. Evaluating at earlier time steps introduces higher uncertainty but enables backtracking. Additionally, adjusting the backtracking depth AT governs the search scope: a small ÂT reduces computation and favors local search, while larger Ar enables broader exploration at the cost of increased computation.
In practice, set the evaluation steps to S and T 3T 1.... 1} for PointMaze experiments. We the recurrence depth to T/2 for image tasks and T/4 for PointMaze, corresponding the denoised steps at which samples are first evaluated. The threshold schedule ó, is also set to 'increase' as in Eq. 14 in our PointMaze experiments, enforcing tighter constraints for samples with lower noise. We use simple constant ò, = ō schedule in our image experiments.
for image experiments to save compute,
In our experiments, we observed that when backtracking fner T' thus fully restarting- the nonzero terminal SNR at/ Or in many diffusion schedules (Lin et al., 2024) can lead to cumulative errors with repeated backtracking. Therefore, when backtracking to freat T, we initialize with fresh Gaussian noise.
26

---

Published as a conference paper at ICLR 2026
Algorithm 5 Diffusion DFS
Diffusion input: diffusion model Cy with diffusion time steps 7 and proposal transition kernel
( ( ) Verifier f.
DES imput: Budget for total number of backtracking = K, backtracking depth Ay and threshold
{
Init ær ~ Init buffer with empty sets: btffer(t) < 0 ,t 1.2.-... .T
while to 0 do
Compute the conditional mean #ojr ==-oreo(æx.)) and estimate the verifier score (()g))) (f
a:
Add the score-sample pair to the buffer: buffer(t).ade (f xz)
if' (Zooj) < 6g and budget B > 0 then
Backtrack: tex +- min(t + Ar. T), Dran ~ q( (2:) with q in Eq. 1
Decrease the budget: B t- B --1
else
if B then
Pop the best sample from buffer: 2, + buffer(t) max bo select the best sample from past explorations
end if
Sample posterior: fnnex 5- 1, t ~ po (00 end if'
f fnexts Dt f- Unext end while
Return Zu
E EXPERIMENT DETAILS
In this section provide the details of experimental setup and implementation for all our experiments. We run our experiments on clusters with Nvidia A A100 and RTX 4090 GPUs, with over 1000 GPU hours used.
E. ABLATION OF BFS DESIGN SPACE
We directly adopt the official code base of FK-steering (Singhal et al., 2025a) and use the samping methods provided in the code base of DAS (Kim et al., 2025). We use the ImageReward prompts as in Singhal et al. (2025a) and report the average and standard deviation over 4 independent trials. For the temperature and resampling interval, directly follow the implementation of FK-steering. For TreeG (Guo et al., 2025) we use a fixed branch out size of 2. For Dsearch (Li et al., 2025b) we set the ratio 6(7) = 4 as in the original paper, and for SoP (Ma et al., 2025) we directly adopt the noise levels of the original paper, and use Paths-2 for N = 4 and Paths-4 for N 8. For the FLUX model, we use the stochastic sampler from Wang & Yu (2025). The detailed arper-parameters are below in
b(O)
Table. 6:
Table 6: BFS implementation parameters
temperature
10
resampling interval
[20,40,80)
sampling steps
100
num seeds
4
E.I.1 ADDITIONAL RESULTS FOR THE BFS EXPERIMENTS
In this section, we provide additional results of the BFS experiments.
In the original implementation of DAS (Kim et al., 2025), the proposal transition kernel is biased with the verifier gradient N' (#,-15 15 Ma (æx) + XV x, log f()uj)) and include the log ratio
in the resampling weights: wf log po (or (a:)) ) log 7 (xc (xt) Alog f(x() A log f(
27

---

Published as a conference paper at ICLR 2026
In Table. 2, disable all the gradients in global search methods for fair comparison. Here provide the results using the original implementation with gradients (DAS-graul grad) in Table. We tune the KL coefficient that the relative magnitude of the reward divided by the KL coefficient remains consistent with the experiments in the original paper (Kim et al., 2025). We also evaluate our BFS method with the gradient-biased kernel (BFS-grad), where use the KI. coefficient from the DAS-grad method as the guidance strength, ensuring an identical transition kernel. From the results, we observe that both ImageReward and HPS scores benefit from the additional gradient guidance. with our BFS design enabling more efficient global exploration.
To validate the robustness of inference-scaling we further evaluate the generated samples with Human Preference Score (Wu et al., 2023b), and provide visualizations in Fig. 6.
BcN
DAS (W50 gradi
D4S-grd
BPS (olo gra))
BRS-gnd
ImsgeRzaoot
Insgsllowon
ImageRoward
ImegeRooon
SD v1.5
3,712 a 0,067
0,264 = 0.301
0,878 a 0,028
0,256 0.003
L084 0.022
0.209 0.002
0.522 0/02)
0,36 6.00
1191 0.041
0.271 =: 0.001
SD
0.001
0:707 00:00
1,522 4 0.005
0.306
0001
197 - 0.53
0.521
0101
1.07 4-000
0.38 0:000
1.002 0.043
0.273 0.002
0.033
0:77
1:181 8003
0.315
0.001
8047
8107
6:000
1/118
0,00L
0/000
1485 0/965
1.2% 0.005
Table Additional BFS experiment results
(a) Direct sampling
(b) BoN
(c) DAS
(d) DAS-grad
(c) BFS
(f) BFS-grad
Figure 6: Visualizations for SD v1.5 generated samples, where direct sampling denotes sampling without inference-scaling and the number of particles is N=4 for other methods. Prompt: beautiful painting of the jungle the morning with lots of smoke, fantasy art, matte painting.
28

---

Published as a conference paper at ICLR 2026
E.2 TEXT-TO-IMAGE COMPOSITIONAL GENERATION WITH DFS
We use the SSD-IB model' which is distilled from SDXL, and we use the default sampling config- uration with 50 steps of DDIM sampler. For DFS and BFS, we evaluate time steps {25, 35, 45) and set the backtrack depth Ar 25. For BFS we additionally sweep the temperature in range {0.5, 1, 4, 8 and report the best performance.
We provide example visualizations in Fig. 7.
(a) w/o inference scaling
(b) w/ inference scaling
(c) Example from color dataset with prompt: yellow book and red vasc
(d) w/o inference scaling
(e) w/ inference scaling
Example from texture datasct with prompt: a fabric hat and glass mirror
(g) w/o inference scaling
(h) w/ inference scaling
Example from shape dataset with prompt: round bagel a square toaster
Figure 7: Example of inference-time scaling with CompBench Above shows the example prompts from CompBench benchmark and images before and after inference scaling
Comparison with SoP. Compared with SoP (Ma et al., 2025), DFS adopts adaptive back- tracking schedule, which enables efficient quality-dependent exploration and full restarts to escape
29

---

Published as a conference paper at ICLR 2026
low quality modes. We provide the additional experiment results in Fig. 8. For SoP, follow the original backtracking schedule, and the uses Paths-2 for low compute and Paths-4 for scaling up more compute, following the insights of Ma et al. (2025). We use 6 = 0.7 for DFS.
0.74
BFS
SoP
0.70
Score
0.66
2
2
Compute
Figure 8: Comparison of DFS with SoP
E.3 LONG HORIZON MAZE PLANNING
Maze environment. For all our maze experiments we use the OGBench PointMaze environment (Park al., 2024). We created our maze layout using the same protocol of Figure 5 in Marcueci et al. (2023)*, but with a smaller size of 20x20 cells. Dataset collected following the protocal in OGBench (Park et al., 2024). We evaluate the model on the default task 1 of OGBench (Park et al., 2024), which is navigating from bottom left to top right. Empirically we discover that the diffusion model perform well on short-horizon tasks without extra inference compute, but struggles heavily in the long horizon tasks.
Model Training. We train the model following diffuser (Janner et al., 2022), where use temporal U-Net to denoise the trajectory
81
82
8H
a1
Q2
O.
Since our objective start and goal is more distant than trajectories in dataset, we sample at longer horizons than training, which is enabled by the temporal U-Net architecture. We train the model for 1.2M steps using the same configuration Janner et al. (2022). Model details are below in Table. 8.
Table 8: parammtrrseeters for maze model
hidden dimension multipliers
[1,4,8,16]
training horizon sampling horizon training steps batch size
2000
2800
1.2M
64
learning rate
le-4
Inference. We found that the model performance saturates with 16 denoising steps, which we use for all our experiments. For all the data points report the average success rate with over 40 samples.
For verifier design, we use the ground-trutt maze layout, and calculate the violation of each point in the trajectory using the position coordinates. Specifically, if point (z. y) is inside maze wall box with center (Ca, Gy) and half-width d, then the point loss can be calculated as the minimum distance from the point to box walls:
L(I,)) = min (z (c2 (cx + x, (c, d), (c, + d) y)
30

---

Published as a conference paper at ICLR 2026
5T-TI sohEssk--Aeaa
DES (howdelss
DES (100
80
DPS dirishal--oo0
51000a 10
Campute
Ablation of threshold 8
(b) Ablation of backtrack schedule
Figure Ablation DFS hyperparameters
and the total verifier score is computed as:
f(T) exp --
L(,, 91)
So if all the points are free of violation in the trajectory, then f(r) = 1. We point out that this does not indicate successful plan the connection between consecutive points (Iz yi) (I5+1: //++1) may violate the maze layout, and using only the verifier function can not generate a successful plan.
For local search, search the with BFS we evaluate at steps {12, 8, 4], and for DFS we evaluate at {12, 11,.1..... .... } with backtrack- ing depth Ar 4. We also observe that increasing backtracking depth to 12 and evaluate at smaller time steps {4, 3, 2, 1} helps to scale the performance with more compute. The hyperparameter
O. For global search f and ji in Sec. C.5 with
search results are below in T and Fig. 9
N
==
0.2
T == 0.005
== ==0.1 0.1
2
27.5
+ 4.3
32.5 1.1
31.2 4.2
4
42.5
5.2
1.1
45.5 2.3
8
67.6
2.2
70.1 111.11 1.1
Table 9: Hyperparameter search for temperature in PointMaze BFS
E.4 OFFLINE RL
Background. Diffusion policy (Chi al., 2023) is widely used for action generation in robot foundation models (Black et al., 2024; Liu et al., 2024). At inference time, policies can be guided by human trajectory constraints (Wang al., 2022) or LLM-based value functions (Nakamoto al., 2024). Exact diffusion sampling requires training a noise-dependent energy function (Lu et al., 2023), but this can degrade pretrained knowledge and demands additional data- often impractical in data-scarce robotic settings. In contrast, inference-scaling provides more flexible approach, allowing seamless composition of pretrained diffusion policies with Q-functions without retraining.
Setup. We follow the setup in Lu et al. (2023), and we directly use their pretrained diffusion model and Q-function, omitting the time-dependent energy function. The diffusion model was trained to generate action given state and we sample with 15 steps of DDIM.
For hyper-parameter search, we disable the implicit dynamics and set 5 O. and use the 'increase" schedule for p and μ.. For strength parameters p and j. we first search for the right magnitude. Then, we also follow Lu et al. (2023) and search with step size [1,2,3,5,8, 10] within the magnitude. Same use 5 different seeds with 10 samples per seed for each task. To avoid over
Lu et al. (2023), fitting, we use different seeds for parameter search and evaluation. We report the hyper-parameters and the performance within the parameter-searching dataset and evaluation dataset.
31

---

Published as a conference paper at ICLR 2026
For global search, use 4 particles for Medium Medium-Expert and Medium datasets, and 2 particles for Medium- Replay datasets. Since the number of particles are small, we do not carry out BFS or DFS methods and simply use Best-of-N. We point out that the number of particles use much smaller than the 50 particles in Wang et al. (2022) and the 32 particles in Chen et al. (2023), highlighting the effectiveness of local search.
Mlediu-RReplay
Baseline. We compare our method to a variety of baselines, including traditional state-of-the-ar methods IQL (Kostrikov et al., 2021) and diffusion-basee policies such diffuser (Janner al., 2022), decision-diffuser (DD) (Ajay et al., 2022), Diffusion-QL. (D-QL) (Wang et al., 2022), SIBC (Chen et al., 2023) and QGPO (Lu et al., 2023). We directly take the numbers from Lu al. (2023).
Among the baseline diffusion-basee methods, both Diffuser (Janner et al., 2022) and QGPO (Lu al., 2023) requires training a noise-dependent guidance function, and D-QL (Wang et al., 2022) requires updating the diffusion model during training using the Q-function iteratively, which needs to back propapro through the diffusion sampling chain, introducing high computation and memory overheads. DD (Ajay et al., 2022) uses classifier-free guidance (Ho & Salimans, 2022) to generate high-return trajectories that requires training a return-conditional model on labeled datasets, which can be expensive to obtain in robotics where only demonstration data is available (Liu et al., 2024).
For our reproduced baselines, TFG (Ye et al., 2024) is allowed up to recurrence steps and DAS (Kim et al., 2025) up 16 particles, resulting in hyperparameter space and computational cost approximately twice that of our method, We sweep across all configurations for the baseline methods and report the best performance. For fair comparison evaluate method on different seeds used for hyperparameter search, with the results shown in Table. 10.
Dataset
Environment
particles
Niear
Nier
p
set
Search set
Redium-Fxperd
HalfCheetah
1
1
0.008
0.02
93.9 + 0.3
94.3 0.5
Medium-Expe Expert
Hopper
1
4
0.001
0.00
104.4 +: 3.1
109.4 + 5.2
Medium Expert
Walker2d
1
0.005
0.10
111.4 0.1
111.4 +
Medium
HalfCheetah
4
1
4
0.003
0.05
54.8 + 0.1
54.8 + 0.2
Medium
Hopper
4
4
4
0.003
0.02
99.5 + 1.7
100.1 + 0.1
Medium
Walker2d
4
1
0.003 6
0.08
86.5 + 0.2
85.2 + 3.2
Medium- Medium-Replay
Replay
HalfCheetah
1
6
0.005
0.03
47.8 0.4
48 4 + 0.1
Medium- Rediumm-Repaay
Hopper
1
0.003
0.20
97.4
100.4 2.2
Medium- Replay
Walker2d
2
2
4
0.003
0.03
79.3 - 9.7
83. 2 + 2.8
Average
86.1
87.5
Table 10: Hyper-parameters D4RL locomotion tasks with test-time scaling. We report the performance on hyper-parameter search dataset and the evaluation dataset, highlighting the best number.
E.4.1 OFFLINE POLICY DISTILLATION
Setup. We adopt the Medium dataset from D4RL (Fu et al., 2020) and the corresponding pretrained models from Lu et al. (2023) as in the previous section. For the dataset, replace the action of each state with the sample generated with test-time search (TTS), using the syper-parameters in Table. 10. We then finetune the models on the modified dataset with early-stopping. For evaluation of finetuned models, we use 5 different seeds different from the seed used during training, with 10 samples per seed. During evaluation, we sample from the finetuned diffusion model directly, without the presence of any verifier. Detailed training hyper-parameters are below in Table. 11.
epochs learning rate batch size
5
le-4
16384
eval every epoch
1
sampling steps
15
Table Hyper-parameters policy distillation
32

---

Published as a conference paper at ICLR 2026
For the Baseline DPPO (Ren et al., 2024), directly adopt the numbers reported in their paper, which uses the same Medium dataset and models as ours.
The training curves for policy finetuning shown in Fig. 10. In both Hopper and Walker2d environments, performance converges rapidly, within a single epoch over the dataset. Notably, method is significantly faster than online finetuning approaches, it does not require online data collection. Training one epoch on the full Medium dataset takes only about 4 minutes on a single Nvidia RTX 4090 GPU, including the time for data generation via inference-scaling sampling, whereas DPPO requires several hours to reach convergence.
0.0
0.2
03
032
04
Eooth
Opert
town
Figure 10: Training curves for policy distillation
E.5 MITIGATING REWARD HACKING WITH DOUBLE VERIFIER
Experimental setup. We evaluate the proposed double-verifier on the challenging conditional ImageNet generation task, generating target-class samples from an unconditional model guided by pretrained classifier. Specifically, we use two independent classifiers verifiers for global and local search. We report the Fréchet Inception Distance (FID) computed on 256 generated samples against the corresponding ImageNet class, and measure class accuracy using a separate classifier", Since we only apply the global verifier sparsely, double-verfier introduces negligible computational costs. For local search we search the hyper-parameters with step size [1 1,2,4488110 10], and for global search with BFS, we search the temperature in [1.0,2.0,4.0]. For MSP score, we adopt different imagenet classifier 12.
For the MSP score (Hendrycks & Gimpel, 2016), it is defined as:
MSP( max p(cla:)
Thus, higher MSP score indicates higher confidence that the image belongs to one of the classes from the dataset, which is less OOD.
Visualizations. Here we provide some visualizations of adversarial reward hacking and the effective- ness of our double-verifier approach. As shown in Fig. 11, we can see the adversarial patches that exploits the weakness of a single verifier, making the verifier to predict the images as belonging to the wrong class.
F WALL-CLOCK TIME ANALYSIS
We conduct a wall-clock time analysis of our methods on the class-guided ImageNet generation task, measuring the average per-sample runtime when generating 10 samples under similar GPU utilization on a single RTX 4090. In DAS Kim et al. (2025) they only compute the verifier gradient while in our local search we additionally incorporates the recurrence the Langevin MCMC step. We observe that wall-clock time is primarily dominated by the number of NFEs. Local search and DAS (Kim al., 2025) slower due the additional gradient computations, while DFS is slightly slower because it generates particles sequentially, preventing full utilization of GPU batch parallelism.
33

---

Published as a conference paper at ICLR 2026
(a) Adversarial example with 0.96 class validity predicted by the lo- cal search verifier
(b) Adversarial example 0.95 class validity predicted by the lo-
(c) Adversarial example with 0.98 class validity predicted by the lo- cal search verifier
cal search verifier
Figure 11: Examples of adversarial samples generated by gradient over-optimization Here provide the adversarial samples generated by gradient guidance on class 222 (kuvasz). Although the samples have high probability of belonging to the target class as predicted by the verifier, they are caused by adversarial reward hacking.
Method
Settings
NFEs of diffusion model
Seconds
Direct sampling
N=1
100
14
DAS
N=1, gradient step
100
Local search
N=1, 1 additional local search step
200
40
Local search
N=1, additional local search steps
300
66
BoN
N=4
400
55
BFS
N=4
400
56
DAS
N=4, gradient step
400
106
DFS
budget=4
210
29
BoN
N=4, additional local search steps
1200
260
BFS
N=4, additional local search steps
1200
261
DFS
budget=4, additional local search steps
620
136
Table 12: Wall-clock time analysis
G LLM USAGE STATEMENT
this paper, LLMs are used exclusively grammar checks, with no changes made to any technical components.
34


Reviews:
[{"review_id": "otyixsB0Aj", "reviewer": "Reviewer_KEXB", "summary": "The paper proposes a unified inference-time scaling framework for diffusion models that performs global exploration with BFS/DFS over the denoising tree and local refinement via annealed Langevin MCMC guided by a verifier. It clarifies and improves BFS-style methods (tempering, scoring, SSP resampling) and introduces the first adaptive DFS with verifier-threshold backtracking to allocate compute on demand. It demonstrates superior Pareto efficiency across image synthesis, planning, and offline RL, including policy distillation and a double-verifier to curb reward hacking.", "questions": "- If the gradient-based guidance component were removed, could the proposed method generalize to discrete diffusion (e.g., MaskGIT, LLaDA)? Any results or insights on this would be valuable.\n- Could this approach be extended to Flow models as well?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Principled, general framework unifying global tree search and local gradient-based MCMC; theory linking recurrence to Langevin MCMC with convergence insights\n- Algorithmic contributions: improved BFS design and novel adaptive DFS that scales compute by instance difficulty, substantially boosting quality per NFE\n- Strong empirical results and ablations across tasks; new Pareto frontier, competitive offline RL without retraining, effective policy distillation, and robustness via double-verifier", "weaknesses": "- The proposed BFS and DFS methods were shown to be effective across various scenarios. However, establishing a unified methodology with default hyperparameter settings (beyond TTS in RL tasks) would make the approach more practical for broader use.\n- No comparisons with more recent T2I diffusion models beyond SD1.5 and SDXL and recent baselines (e.g., [1]).\n- How stable is the proposed method against reward hacking without the double verifier? Since reward hacking is a critical issue, it should be clarified whether the double verifier is essential for robustness.\n\n---\n[1] Inference-time scaling for diffusion models beyond scaling denoising steps, CVPR, 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dUNuPq8pWx", "reviewer": "Reviewer_TXqE", "summary": "This paper studies inference-time scaling for diffusion models via: (i) a global search layer with a BF-style design space (tempering, scoring, resampling) meant to unify existing particle/tree methods; (ii) a DFS variant that backtracks to higher noise when a per-step verifier threshold is violated; and (iii) a local search layer using annealed Langevin MCMC. The authors also relate recurrent training-free guidance to Langevin MCMC in the small-step limit. Experiments span text-to-image (SD 1.5/SDXL/SSD-1B; CompBench), planning (PointMaze), and offline RL, plus a double-verifier trick for reward-hacking mitigation.", "questions": "- In Figure 3, why is DFS-0.9 slightly lower than DFS-0.7 for the same amount of compute? Intuitively, it seems that thresholds should generally provide more high quality samples.\n- In Figure 4 why why is best-of-N with 8 local steps worse across compute than BoN with 6 local steps?\n- How do the BFS/DFS + local search compare with existing methods in terms of wall-clock times?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The various design choices for the BFS method are clearly presented and the studies help understand which components are helpful.\n- The threshold-based method is a useful modification for DFS that is practical, and experiments show it indeed scales up compute for harder problems as intended.\n- The theoretical result bridge that guided recurrence approximates annealed Langevin MCMC in the small-step limit.\n- The experiments span multiple domains including text-to-image generation, path generation in maze, and continuous control tasks for offline RL.", "weaknesses": "My main reservations concern the strength of the paper’s contribution and significance, the clarity of exposition, and a few statements that seem potentially misleading or inaccurate.\n\n### Contribution\n\nThe global search methods BFS/DFS proposed in the paper largely seem to be an alternate perspective or modifications to existing methods, rather than fundamentally new algorithms\n\n- The authors acknowledge that FK-steering/DAS are instantiations of BFS, but I do not see what new insights are provided by this new framing. The design choices for BFS in Section 4.1.1 are essentially ablation studies on existing techniques: FK-Steering [1] explored current/difference/max potential functions (and showed that max performs best), and DAS [2] showed that SSP sampling and tempering help improve performance. This is somewhat corroborated by Table 2, where BFS scores on all tasks are within one standard deviation of DAS scores.\n- DFS modifies the fixed sequential noising/denoising procedure of SoP [3] with a threshold-based adaptive procedure, which is a useful modification, but it is again not a fundamentally new algorithmic contribution.\n\n### Experiments\n\n- **Comparison with SoP.** The paper tries to set apart DFS by stating that SoP uses “small noise injection for local exploration” as opposed to $\\Delta_t \\geq T/4$ for DFS. This is incorrect as SoP in fact uses $\\Delta_t = 0.78\\,T$. Please change this statement to accurately reflect prior work. In addition, despite the conceptual proximity, I find it a bit odd that there is no empirical comparison of DFS with SoP to understand the relative improvement of the proposed thresholding-based modifications.\n- **DFS results.** The claim in Section 5.2 that DFS outperforms BFS/Best-of-N with “up to 2x less computational cost” is slight overclaiming based on the results in Figure 3 (left). The difficulty vs compute discussion in Figure 3 (right) does not show the resulting improvement from the additional compute. Please include baseline scores across the difficulty levels so we can compare the base model vs DFS results.\n- **Hyperparameter burden.** The local search introduces step size and number of steps, DFS introduces threshold $\\delta_t$ and backtracking depth. The paper has reasonable sweeps in its experiments, but introducing this many hyperparameters introduces a burden for practical adoption, as discussed briefly in the limitations. Specifically, regarding the threshold parameter of the DFS method which is introduced in this paper: (1) it is defined as a time-dependent threshold $\\delta_t$, but it seems experiments use a constant value for all $t$? How should one vary this threshold with time? (2) This parameter generally seems hard to tune - given a new reward function with an arbitrary range, how shall a practitioner decide the suitable threshold value?\n- **Insufficient experiment details in the main paper.** Many crucial experimental details that should be in the main paper are provided in the appendix, making the experiments section very hard to read.\n    - Section 5.1 does not specify the prompt dataset used for evaluation. Appendix E.1 says the experiments used ImageReward prompts following the setting in [1], which actually used GenEval and DrawBench prompts for text-to-image. Please provide the correct citation to avoid confusion, and the prompt dataset should be specified in the main paper.\n    - Section 5.3.2 does not specify the number of recurrent steps for local search that are used in the offline RL experiments. This information is not provided in the appendix as well.\n    - Section 5.4 does not explain what the two different verifiers are used for ImageNet conditional generation; this information is only provided in Appendix E.5.\n\n### Clarifications/corrections on statements\n\n- Section 2 states that tree-search methods like [3,4] are special cases of the BFS framework. [3] seems closer to DFS as discussed later in the paper, and [4] does not map trivially to the BFS framework. An explanation to support this statement would be helpful.\n- Section 3.2 discusses sampling from product distributions. However, the wording here is a bit misleading - it seems to suggest simply adding the score functions is enough to sample from product distributions. Also, authors define $\\hat{q_t}(x_0)$ as a distribution on $x_0$ but the equation involves $\\hat{q}_t(x_t)$ as a distribution over $x_t$. The appendix has a detailed explanation, but the wording here should be changed to avoid misunderstanding.\n- The scoring functions for BFS use posterior mean predictions $x_{0|t}$ to get intermediate rewards at noisy states; asserting this “accounts for reward variation during sampling by considering not only the current reward but also its trajectory” is misleading since this prediction is specifically designed to *not* consider the sampling trajectories. Please qualify/modify this statement.\n- The paper could benefit from discussing some relevant works: [5] uses an MCTS-style tree search, and [6] is a particle-based method that uses Gibbs sampling. Both methods outperform FK-steering and/or DAS by a significant margin.\n\n*[1] Singhal, Raghav, et al. \"A General Framework for Inference-time Scaling and Steering of Diffusion Models.\" Forty-second International Conference on Machine Learning, 2025.*\n\n*[2] Kim, Sunwoo, Minkyu Kim, and Dongmin Park. \"Test-time Alignment of Diffusion Models without Reward Over-optimization.\" The Thirteenth International Conference on Learning Representations, 2025.*\n\n*[3] Ma, Nanye, et al. \"Inference-time scaling for diffusion models beyond scaling denoising steps.\" arXiv preprint arXiv:2501.09732 (2025).*\n\n*[4] Li, Xiner, et al. \"Dynamic Search for Inference-Time Alignment in Diffusion Models.\" arXiv preprint arXiv:2503.02039 (2025).*\n\n*[7] Jain, Vineet, et al. \"Diffusion Tree Sampling: Scalable inference-time alignment of diffusion models.\" The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.*\n\n*[6] Dang, Meihua, et al. \"Inference-time scaling of diffusion language models with particle gibbs sampling.\" arXiv preprint arXiv:2507.08390 (2025).*", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wrk5cUBGpE", "reviewer": "Reviewer_JCZg", "summary": "This paper focuses on inference-time control of diffusion models to generate samples that maximize given objectives. It combines global search via breadth-first and depth-first search and local search via annealed Langevin MCMC. The method is applied across planning, offline reinforcement learning, and image generation to demonstrate improved performance and efficiency over baselines.", "questions": "1. The claim that local search via Langevin MCMC can 'go beyond base model', which repeatedly appears in the paper (Section 1 line 51-52, line 80-81, Section 4.2), may be misleading. Langevin MCMC still generates samples from the compositional distribution $p_0(x)f(x)^\\lambda$ where the support is a subset of the support of $p_0$. Thus, similar to global search, it's searching for high-reward samples in the support of the base model. Even if it can explore very low-probability (relative to the base model) regions, there seems to be no direct evidence in the paper.\n2. SVDD, DAS, and FK-Steering all fall under the umbrella of SMC. Categorizing them as BFS seems like simple reframing. Also, when resampling, does it consider log likelihood with respect to the base model, which is the case for DAS and FK-Steering?\n3. What is TTS in Section 5.3.2? Is it BFS + local search or DFS + local search?\n4. What's the purpose of double-verifier experiments? Solving reward hacking is an important problem, but the experiments don't demonstrate the additional benefit of the proposed BFS. It's rather just introducing an additional technique for a separate problem.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. It explores various design choices in inference-time scaling of diffusion models across global and local search and proposes a unified framework that jointly scales both global and local search.\n2. It proposes a method using depth-first search (DFS), which can adaptively allocate compute.\n3. Decomposed ablation on global search (Section 5.2) and local search (Section 5.3.1) shows the effectiveness and necessity of both methods.\n4. Experiment demonstrates that distilling TTS samples can be a powerful off-policy RL method. (I believe [1] has already done this, though they didn't use the term 'inference-time scaling', but referred to it as global and local optimization, so being the 'first' work seems inaccurate.)\n\n[1] MARK, Max Sobol, et al. Policy agnostic rl: Offline rl and online rl fine-tuning of any class and backbone. arXiv preprint arXiv:2412.06685, 2024.", "weaknesses": "1. Usage of Langevin MCMC prevents applying local search for non-differentiable rewards or naturally expanding the method to discrete diffusions. There are other MCMC alternatives, such as Metropolis-Hastings variants (e.g., predictor-corrector), but the choice of Langevin MCMC isn't justified sufficiently.\n2. By reporting only a single reward value in image experiments, it's hard to tell whether there is severe reward hacking and the method is generating broken images. For instance, excessive guidance can achieve high reward but generate a broken image that is off the data manifold. Please provide additional metrics (e.g., additional preference score or FID) or qualitative results to show the proposed method is searching for high-reward samples on the data manifold.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l3qBNWCYlU", "reviewer": "Reviewer_Vc1j", "summary": "This paper proposes a unified framework for inference-time scaling of diffusion models by leveraging principles from classical search. The authors formulate the inference process as a combination of global and local search. For global search, they employ tree-search algorithms such as Breadth-First Search (BFS) and Depth-First Search (DFS) to efficiently identify high-quality modes in the generative space. To further refine sample quality beyond the base model's distribution, they introduce a theoretically-grounded local search based on annealed Langevin MCMC. The proposed method demonstrates superior or comparable performance across a range of challenging domains, including long-horizon planning, offline reinforcement learning, and image generation. The paper is supported by comprehensive empirical analyses of its design choices and a theoretical convergence analysis in the appendix.", "questions": "- Could you please provide an empirical ablation study on the backtracking hyperparameter $\\Delta_T$​? Understanding its sensitivity would be valuable for practitioners seeking to apply your method.\n    \n- Regarding the scoring functions in BFS, I understand that 'Difference' and 'Max' scoring account for the trajectory's history. Could you elaborate on the necessity of this? What are the specific advantages of scoring based on the trajectory history compared to only using the score from the current timestep?\n    \n- As mentioned in the \"Limitations and Conclusion\" section, using evolutionary algorithms for hyperparameter tuning is an interesting direction. However, an alternative direction for future work could be to further unify the three search strategies (DFS, BFS, and local search) to be controllable by a smaller, more concise set of core hyperparameters. I would be interested to hear the authors' thoughts on the feasibility and potential of such a direction.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- **Clarity and Thoroughness of the Proposed Method:** The paper provides a detailed and well-structured discussion of the proposed methods, both in the main text and the appendix. This thoroughness contributes to a clear and concrete presentation of their framework.\n    \n- **Comprehensive Empirical Analysis:** The authors systematically present and analyze the empirical results for each component of their method. For example, they methodically ablate the design choices for BFS (resampling, scoring, and tempering) and subsequently demonstrate the efficiency gains achieved through the introduction of DFS and local search.\n    \n- **Unified Framework for Inference-Time Scaling:** The paper successfully presents a unified framework that integrates global and local search strategies. This framework cohesively combines key functionalities essential for controlled generation, such as branching, exploration, backtracking, and optimization beyond the pretrained distribution.", "weaknesses": "- **Large Number of Hyperparameters:** While powerful, the proposed integrated framework introduces a significant number of hyperparameters. These include the BFS tempering parameter () and number of particles, the DFS backtracking step size (​) and score threshold (), and the local search steps (​), among others. As indicated by the experimental results, performance is sensitive to the tuning of these parameters. The authors acknowledge this as a limitation in the \"Limitations and Conclusion\" section and suggest automatic tuning algorithms as future work. I agree with their assessment that this is a key area for improvement.\n    \n- **Fixed Backtracking Step Size $\\Delta_T$:** In the proposed DFS algorithm, when a sample's estimated score falls below a predefined threshold $\\delta$, the algorithm backtracks by adding a fixed amount of noise determined by the hyperparameter $\\Delta_T$​. However, the optimal amount of noise to inject may vary depending on the specific instance. In some cases, a small perturbation might be sufficient to escape a low-quality region, while in others, a near-complete restart from a higher noise level might be necessary. The current method lacks the flexibility to handle this adaptively. In contrast, other contemporary works have explored more dynamic approaches. For instance, Adaptive Bi-directional Cyclic Diffusion (ABCD) [1] addresses this by propagating particles across different noise levels to balance exploration and exploitation. Similarly, Monte Carlo Tree Diffusion (MCTD) [2] uses a selection policy inspired by MCTS to navigate the search space more dynamically.\n\n- **The Statement about the Use of Large Language Model:** The authors doesn’t state explicitly how they used LLM in their submission, which is guided in https://iclr.cc/Conferences/2026/AuthorGuide.\n    \n- **(Minor) Presentation Issues:**\n    - In line 200 on page 4, there appears to be a typo in the transition kernel notation: $xt-1$ should likely be written in LaTeX as $x_{t-1}$.\n\n[1] Lee, Gyubin, et al. \"Adaptive Cyclic Diffusion for Inference Scaling.\" _arXiv preprint arXiv:2505.14036_ (2025).\n\n[2] Yoon, Jaesik, et al. \"Monte carlo tree diffusion for system 2 planning.\" _arXiv preprint arXiv:2502.07202_ (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)