One step further with Monte-Carlo sampler to guide diffusion better
Authors: Minsi Ren
Wenhao Deng
Ruiqi Feng
Tailin Wu
Venue: ICLR 2026 Poster (2026)
Keywords: Conditional Generation; Diffusion Model; Training-free Guidance
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Stochastic differential equation (SDE)-based generative models have achieved
substantial progress in conditional generation via training-free differentiable
loss-guided approaches. However, existing methodologies utilizing posterior sam-
pling typically confront a substantial estimation error, which results in inaccurate
gradients for guidance and leading to inconsistent generation results. To mitigate
this issue, we propose that performing an additional backward denoising step and
Monte-Carlo sampling (ABMS) can achieve better guided diffusion, which is a
plug-and-play adjustment strategy. To verify the effectiveness of our method, we
provide theoretical analysis and propose the adoption of a dual-evaluation frame-
work, which further serves to highlight the critical problem of cross-condition
interference prevalent in existing approaches. We conduct experiments across var-
ious task settings and data types, mainly including conditional online handwritten
trajectory generation, image inverse problems (inpainting, super resolution and
gaussian deblurring), and molecular inverse design. Experimental results demon-
strate that our approach consistently improves the quality of generation samples
across all the different scenarios.

Published as a conference paper at ICLR 2026
ONE STEP FURTHER WITH MONTE-CARLO SAMPLER TO GUIDE DIFFUSION BETTER
Minsi Ren, Wenhao Deng, Ruiqi Feng, Tailin Wu Al for Scientific Simulation and Discovery Lab, Westlake University (renminsi, wutailin] @ ake.edu.co
ABSTRACT
Stochastic differential equation (SDE)-based generative models have achieved substantial progress in conditional generation via training-free differentiable loss-guided approaches. However, existing methodologies utilizing posterior sam- pling typically confront substantial estimation error, which results in inaccu- rate gradients for guidance and leading to inconsistent generation results. To mitigate this issue, propose that performing an additional backward denois- ing step and Monte-Carlo sampling (ABMS) can achieve better guided diffu- sion, which is plug and--ndd-pp play adjustment strategy. To verify the effectiveness of our method, we provide theoretical analysis and propose the adoption of a dual-focus evaluation framework, which further serves highlight the critical problem of cross-condition interference prevalent in existing approaches. We conduct experiments across various task settings and data types, mainly includ- ing conditional online handwritten trajectory generation, image inverse problems (inpainting, super resolution and gaussian deblurring) molecular inverse design and so on. Experimental results demonstrate that our approach can be effec- tively used with higher order samplers and consistently improves the quality of generation samples across all the different scenarios. Our code is available at htttp:://qithut wtt./I////AAi/cc/eeeeeeee Westlakeū)
INTRODUCTION
In recent years, with the rapid advancement of diffusion model techniques (Sohl-Dickstein al. 2015 Ho et al. 2020 Song et al. 2021b Song & Ermon 2019). conditional diffusion models have emerged a powerful paradigm generative for solving generative tasks such as high-resolution image synthesis, image restoration, tex--to-image generation, 3D shape modeling, molecular design and on (Dhariwal & Nichol, [021] Rombach et al. 2022 Zhang al. 2023 Mardaní et al., 2023]. By incorporating conditional information (e.g., text descriptions, class labels, degraded im- ages, specific molecular property) into the diffusion process, these models enable fine-graines con- trol over generated outputs. However, despite significant attention, conditional generation still faces a series of challenges in terms of cost and generalizability: typical conditional generation approaches such classifier guidance (Dhariwal & Nichol 2021) and classifier-free guidance (Ho & Salimans, 2022) cither require additional task-specific training of diffusion models or necessitate training extra noise-compatible conditional discriminators, thus limiting the range of applications. Moreover, for inverse problems that require highly precise conditions, such as the quantum numerical attributes of molecules, the classifier- free paradigm is not applicable.
In this context, a highly versatile approach is the training-free guidance methods (Wallace et al. 2023 Chung et al.) 2022 Lugmayr et al. 2022; Dou & Song 2024 Cardoso al. 2024 Feng
et al. 2023) for conditional generation, using off-the-shelf loss guidance during denoising to achieve satisfactory results. These approaches avoid task-specific training or extra conditional networks by directly leveraging pre-defined loss signals, enabling plug-and
conditional generation across diverse scenarios. Among these methods, DPS (Chung al., 2023) is a prominent approach lever- aging diffusion models to tackle inverse problems, with a series of subsequent improvements. Recent studies (He et al. 2024 Yang al., 2024) have made some improvements by enforcing manifold
"Corresponding author

---

Published as a conference paper at ICLR 2026
preservation during the guidance process, thereby enabling the use of larger guidance step sizes in denoising. This advancement not only accelerates sampling efficiency but also enhances the sta- bility of conditional generation. fostering stronger alignment between theoretical foundations and practical utility.
Nevertheless, despite recent progress, we observe that the majority of existing methods using the gradient direction delivered by the plain DPS formulation as guidance, suffer from systematically biased gradients. Empirically, guiding the sample toward one condition frequently perturbs other conditions that are intended to remain decoupled, revealing an inherent cross-talk implicit in the gradient estimate. Motivated by this observation, we argue that the evaluation of a guidance method must simultaneously account for two facets: (i) the degree of alignment between the generated samples and the target condition, and (ii) the preservation of global properties, egg. FID for images or molecular stability in drug design. Experiments exhibit the trade-off in practice: as the guidance weight increases, compliance with the specified condition improves at the systematic expense of sample quality, manifested by rising FID or diminished molecular stability. Consequently, reporting only alignment metrics paints an incomplete picture and risks selecting operating points that violate downstream requirements.
Furthermore, we analyze the root cause of the problems in existing methods: the large bias in es- timating the conditional expectation. To address this issue, propose ABMS, strategy that ef- fectively controls estimation bias through Monte Carlo sampling, yielding more accurate guidance gradients. We demonstrate its effectiveness across multiple tasks and data types. Our contributions are summarized as follows:
We highlight the limitations of previous methods: the significant estimation error typically results in imprecise guidance gradient, leading to inconsistent generation outcomes.
We advocate for the dual-focus evaluation framework to better assess guidance- based meth- ods and reveal the severe issue of cross-condition interference in existing methods.
We analyze the source of the estimation error and propose a simple, plug-and aing-and-paa im- provement strategy ABMS mitigate its impact and also provide theoretical support.
Experiments on various tasks and data types demonstrate the generality and effectiveness of the proposed strategy.
2 RELATED WORK
Training-free conditional diffusion sampling. recent years, the technology of training free
conditional sampling in diffusion models (Wang et al.) 2023 Kawar et al., 2022; Chung et al.) 2023
2022 Lugmayr et al. 2022) has witnessed rapid development (Dou & Song 2024 Cardoso al.,
2024 Feng et al. 2023 Janati al.) 2024). Essentially, these methods enable conditional generation under any differentiable constraint. For example, Red-Diff (Mardani et al., 2023) casts the condition
as differentiable loss and treats the diffusion prior regularizer, thereby formulating the prob- lem as an optimization task. Another popular and widely adopted paradigm is Diffusion Posterior Sampling (DPS))Chung al. 2023), which is the basic of method.
As mentioned earlier, the core idea of DPS is to leverage Tweedie's 's formula to approximate con- ditional scores, generalizing linear inverse problem solvers to arbitrary generation tasks. In ries of subsequent works on DPS, LGD-MC (Song et al., (023) reduces estimation bias by using Monte Carlo sampling from imperfect Gaussian distributions. However, the strong assumption that p( (z.)) follows a Gaussian distribution prevents any increase in sampling steps from adequately capturing the multi-modal distributions in real world scenarios. Other work such as UGD (Bansal et al. 2023) and DifiPIR (Zhu et al.) 2023) perform guidance on clean data 2o followed by pro- jection intermediate states Ft. Recently, MPGD (He et al.) (024) enforces guidance within the tangent space of the clean data manifold using autoencoders, improving sample quality at the cost of relying on pre-trained models and linearity assumptions. DSG (Yang et al. 20024), on the other hand, focuses on the high-density regions near the center of the high-dimensional Gaussian distribution, constraining the guidance step within the intermediate data manifold through optimization, and en- abling the use of larger guidance steps. However, it is worth noting that the existing improvements to DPS primarily focus on preventing the intermediate data x, from deviating from the manifold M, at time step f, without addressing the issue of the guidance gradient imprecision in the plain DPS.
2

---

Published as a conference paper at ICLR 2026
Inference time scale diffusion sampling. Several existing methods enhance sample quality with- out modifying model parameters by leveraging additional computational cost during sampling(Ma et al. 2025 Zhang al. 2025 Xu et al. 2023). These methods typically fully exploit the capability of pretrained generative models through noise search or reintroducing noise during the generation process restart sampling. Motivated by the same principle, our method aims to obtain more accurate guidance gradients for arbitrary differentiable conditions by increasing appropriate compu- tational budget.
PRELIMINARY
3.1 DIFFUSION MODELS FUNDAMENTALS
Diffusion models Sohl- Dickstein et al. 2015) construct forward process that gradually adds noise data samples through 7 time steps, then learn reverse this process for generation. The frame- work has evolved through two main perspectives: stochastic differential equations (SDEs) (Song
al. 202 and discrete-time Markov chains (Ho et al.) 2020).
Stochastic differential equation framework. The continuous-time perspective formulates diffu- sion through SDEs:
dæ hax, t)dt + g(t)w.
where h(.) is the drift coefficient, g(t) controls noise scaling, and w denotes Brownian motion (Anderson 1982). The reverse process becomes:
dæ = [b( 2 log (x))d( + g())dw.
where V log (x)) is estimated by a neural network. Various noise schedules yield distinct forward-noising and reverse trajectories but all are subsumed under Equation 2
3.2 TRAINING-FREE GUIDED DIFFUSION
Problem formulation. Let E X C Rd be the unknown clean signal we wish to generate or recover, and y E y arbitrary conditioning variable (e.g. a corrupted measurement, categories, or numerical properties). We assume access to
pre- trained diffusion priorp(xo) that models the marginal distribution of clean data;
a differentiable, blug-and-play loss L(xo: y) defined on the clean space X, which encodes the conditional likelihood p(y) (r) xx exp(-L(mo)) y)).
Our goal to sample from the posterior
p(zoly) p(æo) (xp(--C(x0:1))
the loss L at every diffusion time-step, yielding a fully training-free inference pipeline. No retraining or time-dependent classifier is required: guidance is performed by back-propagani
Diffusion posterior sampling. Diffusion posterior sampling (DPS) (Chung et al. 2023) casts de- terministic inverse problems as posterior sampling via the Bayesian identity:
V æe pog. (1) Ve. p(x:)) + Va. logp(y/æ:)
(3)
intractuble
To sidestep the intractable likelihood score, DPS performs two successive approximations. Approximation-l: Factories the likelihood and move the expectation inside the loss
p(yla)) =/ p(u (o) p(zo a.)) dæc ss 1Ero [x.))
(4)
Invoke Tweedie's 's formula and replace the posterior mean with the plug-in in esti- mate produced by the denoising network Zy. Combining the two steps yields the gradient surrogate:
Approximation-2:
V Va log p(g(a))))))
Vz, log ply ((oo(o)))))
(5)

---

Published as a conference paper at ICLR 2026
4 METHOD
In this part, we first analyze the limitations of current mainstream approaches Section and propose mitigation strategies. Then, in Section 4.2 we conduct analysis, providing the- oretical guarantee for the effectiveness of the proposed strategy. Finally, in Section 4.3 the complete pipeline of the method.
we present
4.1 ADDITIONAL BACKWARD STEP WITH MONTE-CARLO SAMPLER
The estimation error problem in DPS. As discussed in Section 3.2 more generally, for any differentiable conditional function far), with the conditioning variable y suppressed for notational simplicity: the core objective is to estimate Ezola, [xo)), i.e., the conditional expectation of f evaluated the clean signal Zo. given the noisy input H,. The common approach DPS uses the de- noising network output do (2x,) to predict 20 and uses the conditional gradient to guide the diffusion generation process as:
2t-1 f - a,
(6)
where _/-1 is the unconditional update term and wi is the guidance scale. However, this single-point approximation fails to account for the inherent uncertainty in p(ral=z)) and introduces significant bias, particularly when f is nonlinear (by Jensen's 's inequality) and Ia is highly noisy. Crucially, while the target quantity [ [xo)] depends the full posterior p(zo (za). this distribution is generally complex and analytically intractable, precluding direct sampling or exact computation.
Leveraging the trackable structure in diffusion backward process. Fortunately, notice that the reverse diffusion process defines a Markov chain in which the one-step transition kernel p( can be substituted by explicitly parameterized Gaussian in practice. This allows re-express the desired expectation via the law of total expectation:
Ezyla [] (zo)] B jx, (x0)
Rather than approximating Zo directly from 2t, our method propagates uncertainty through an inter- mediate step, effectively averaging over multiple plausible denoising paths. Based on this insight, define the ABMS strategy follows:
Sample M intermediate states: -.(m) ~ p(xx-- (2x) for m = I,...,M. M.
2. For each " obtain a denoised estimate (o(x (m) using the pretrained denoising network.
3. Evaluate the conditional function: compute [ (((((((())) for each sample.
4. Average the evaluations:
M
1 fARMS (M, x.) = M
(ORM(M,,EN
Σ f( m
Intuitively, by injecting stochastic intermediate step, ABMS lets the generative network explore multiple plausible denoising trajectories, naturally capturing the multi-modal shape of p( (x)) in- stead of being trapped in single point estimation. Furthermore, in the experiment section also demonstrate the method's 's suitability for higher order samplers involving stochasticity. not just for single step updates.
4.2 ESTIMATION ERROR ANALYSIS
We also present rigorous comparison of the estimation error between the DPS estimator and our ABMS estimator. establish our theoretical results, we make the following assumptions, which empirically well-supported and commonly adopted in diffusion modeling:
A1. The conditional function f X + Rd is K K-Lipschitz continuous and L-Lipschitz gradient.
A2. The accuracy of the denoiser improves monotonically along the reverse diffusion process. Specifically, for any state Zt, the expected reconstruction enror satisfies:
Er..le. [()) Ezulz-. [sa]]]] < [00();) E jx, [_o]]]
4

---

Published as a conference paper at ICLR 2026
This assumption reflects the intuitive and empirically observed fact that reconstructions from less noisy intermediate states (Iz-1) are closer to the true posterior mean than those from noisier states (æ²).
Then have the following proposition:
Proposition I: ABMS attains lower-bounded expected estimation error than plain DPS.
Proof. Error bound for DPS. The DPS estimator uses single reconstruction (n(x)) to approxi- mate Io [zo)), yielding:
fDrs f(fo(x))).
Its estimation error is bounded by decomposing:
||ErrordPs] = [[((((z))) Exole. (xx)]]
$ [((((()))) - f(Ezolz, + (E (mol) - E on [x
Jensen gap item::6 (
<K. L [zo]]]
5y(2)))
where (22) 2 0 known Jensen gap quantifies the deviation due the nonlinearity of f. and vanishes only if is affine.
Error bound for ABMS. Recalling the ABMS estimator defined in equation as M is large enough, converges to 12, (X, 1))] Using the Markov property.
Exgla [f(o)) = Ex--i/ar [
the asymptotic estimation error satisfies:
|ErtorBBMsI Br, |m. || Exol (zo))
< K Eeg, 1120 [000(++-1))) Ego ln [mo]]] + Te 1l2, (x)()=11))
Comparison of expected error bounds. Taking expectation over 24 p(a,) on both equation and equation yields:
E|JErrossss| K E (a(r))) Ex [2o]]] (
(10)
[||EororrAMMs S K E Ez, 1/m, (1:0 Ezalz, [xm]]]] + E [E,, 1lm (5
(11)
Step reconstruction term. By Assumption 2 we have:
E, Jar, | (1) Eng Io, [xo]]| S [1,) - Euglar [moll]
so the first term in equation which related to reconstruction error is larger than that in equation
Step 2: Jensen-gap term. According to the L-Liscchit gradient property in Assumption] I can get the upper bound of E6 (x:)) and E [ 16, (x.- respectively (the proof provided in Appendix A.I.
UB, = L E., Th Cov Cov (zol))] UB 1 Ezz, Bmy 1/21 Tr(Cov (CCovoolzz- 20l00 [mo)]]]
(12)
Law of total covariance gives the identity:
Covzolz, [mn]] Ex, .| [Cvv [mo]]] Cova, 1/m1 Beoler [mo]]
Taking trace and expectation over 2, and we find that:
UB, UB.., 1/ Ez, [TY (2o) 2 0.
Hence UB,1 < UB, showing that the Jensen gap in ABMS contributes to the expected error upper bound is no larger than that of DPS. Combining the two steps, we have proved the original proposition.

---

Published as a conference paper at ICLR 2026
4.3 THE COMPLETE FRAMEWORK OF ABMS GUIDANCE
In the preceding sections we introduced the ABMS estimator to obtain more accurate guidance direction, denoted as g. As regards the scale of the guidance, recall that the posterior p(1 xx follows Gaussian distribution: p(
(No (3r, (), Tf Inspired by DSG (Yang al, 2024), for any n-dimensional isotropic Gaussian æ (N,, when sufficiently large the distribution is concentrated on a hypersphere of radius vna centred at μ.. To prevent the guided sample from drifting away from the data manifold, we therefore rescale and constrain the magnitude of the guidance vector to lie in this hypersphere: g' = We vno, T where E (0, 1) is the guidance rate. We employ cosine schedule to ensure that wt increases smoothly throughout the denoising process and the complete procedure is provided in Algorithm [
Algorithm One Guided Diffusion Step of ABMS
Require: 11-1"
Wmaxx J. M, T.
I: radius VTL-r. Oz.
denotes dimensionality
0-0 0
3: for i=
====10 to M do
draw M Monte-Carlo samples
4:
Ei ~ 1)
5:
2t-1 t- 2i-1 Er
6:
$o + predxx((0
network predicts clean .oo
F FU((((
8: end for
9: f average F), - v
ABMS guidance direction
10: Wi "maa (1 + cos (T(1 ( t/T)))
11: g' +- Wi radius
T9l
9 + 12: new x- #t-1
mean
Ensure: peew
EXPERIMENTS
In this section, we evaluate our methods extensively in various tasks, including dual-conditional generation task (stylized handwritten trajectory generation). three prevalent image inverse problems (inpainting, super resolution and gaussian deblurring), molecular inverse design and text-style guid- ance. We primarily compare with the current state-of-the-art method, DSG (Yang et al., 2024), which has been proven to effectively prevent the manifold deviation phenomena.
A key aspect of our experimental design is the dual-focus evaluation criterion: simultaneously as- sessing (1) the consistency of the generated samples with the specified conditions, and (2) the re- sulting impact on other sample attributes. Below detail the metric pairs used in each experiment: content score vs style score S.11 distance vs FID in Section 5.22, and MAE vs mol sta- bility in Section [5.3] For the analysis of sampling time, please refer to Appendix A.2 For more experiment results, please refer to Appendix A..
5.1 STYLIZED HANDWRITTEN CHARACTER GENERATION
Implementation. The goal of this task to generate Chinese characters with specified categories
and writing styles. Following the setup in (Ren al. 2025), with dual-conditional generation capabilities. We use the CASIA-OLHWDB (1.0-1.2) dataset (Liu
the pretrained diffusion model
al.) 2011) as the training set and the ICDAR-2013 competition database (Yin al. 2013) as the Test set. We adopt the DDPM sampler with 1000 sampling steps.
For evaluation, we train two classifiers for character categories and handwriting styles the test set respectively, following the previous work (Ren et al.) 2025 Dai et al.) (023). The classification accuracy of the synthesized samples called the content score and the style score. Higher scores indicate that the synthesized samples are closer to the real target data.
Cruss-condition interference. In an ideal scenario (i.e., on a clean data manifold), the category and writing style are completely decoupled conditions. Therefore, applying a moderate gradient to

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative evaluation of conditional character generation under different guidance scales.
Sathod-Scale
No guidance DSG((.01)
DSG(0.1)
DSG(0.5)
Ours(0.01)
Ours(0.1)
Ours(0.5)
Content(1) 0.827
0.927
0.998
0.999
0.981
0.999
0.999
Style(t) 0.899
0.543
0.534
0.166
0.888
0.878
0.756
one condition should not have negative impact on the other. To this end, only the gradient from category classifier to guide the diffusion model, while observing any potential impact on the style score. We evaluate performance under varying guidance scales and the number of sampling time M is set to in our ABMS method.
OKE 22
GT:
OURS (o.)::
DSG (D.1):
Figure 1:
The visualization comparison results, where different colors represent different strokes. The guidance scale is set as 0.1. It can be observed that, even without manifold deviation, the fonts generated by DSG tend have connected strokes. regardless of the target writing style. On the other hand, our method is able to better preserve the style characteristics.
Evaluation results. As shown in Table T both the methods significantly boost the content score. However, even with a smaller guidance scale, DSG method based on plain DPS gradients still exerts sigmificant influence on style features. In contrast, our method shows substantial improve- ments in both quantitative and qualitative evaluations. Figure T demonstrates that at a guidance scale of 0.1, our proposed method basically maintains the writing style of the synthesized stroke trajectories, while DSG introduces pronounced stylistic distortions in calligraphic consistency.
5.2 IMAGE INVERSE PROBLEMS
Implementation. In this part, investigate three prevalent linear inverse problems of the form y Ax + c, which including image inpainting. super-resolution, and gaussian deblurring. We
evaluate our proposed method on two datasets: 1K images from FFHQ 256x256 (Karras al. 2019)
and ImageNet 256x256 validation dataset (Deng et al. 2009). using pre-trained diffusion models
sourced from (Chung al.) 2023 Dhariwal & Nichol 2021). For comparison, we select several
baseline methods: DPS (Chung et al. 2023), LGD (Song et al. (002)) and DSG (Yang et al. 2024).
We adopt the experimental setup from DSG generate noisy measurements. The loss function
guiding the reconstruction process is defined as follows:
C( y) [|Aio(r;)) vll
(13)
where y represents the noisy measurement, represents the deterioration model and mg denotes the image aim to reconstruct. For these tasks, employ DDIM [Song et al.) 202 sampler with 100 sampling steps.
For evaluation, as shown in Equation 13 this loss function essentially represents the pixel-wise difference between the generated image and the real ground-truth image after both undergo the same degradation process. We refer to this the 'Distance' metric, which is used to evaluate the degree to which the guidance method adheres to the conditions. Additionally, we evaluated image quality metrics including PSNR, SSIM, LPIPS, and FID.

---

Published as a conference paper at ICLR 2026
Evaluation results. The quantitative evaluation results for the tasks mentioned above presented in [ Notably, our method demonstrates consistent performance improvements over baselines across a range of evaluation metrics and task configurations. These results provide empirical support for the feasibility of our proposed modification.
More importantly, following the principle of dual-focus evaluation, measure the performance curves of the Distance vs FID metrics under different guidance scales, as illustrated in Figure 2 Our method achieves lower Distance while maintaining higher image quality. Due to space limitations, the results for the Gaussian Deblurring task are provided in the Appendix A.3 for reference.
For the case M therefore omitted for clarity. It is noteworthy that as the number of sampling steps (see Equation ? increases, the method exhibits relatively better performance. which aligns with the algorithm's intu- ition. Moreover, we observe that the performance gain becomes evident once M reaches 3, and the marginal benefit gradually saturates as M is further increased.
the resulting curve is visually relatively similar the initial DSG and is
Performance Comparison Inpalitinn Task
Performance Comparison on Super Resolution
21.3
DS0 Ours (M-5) Durs (M-3))
26.3
3.2-
(M+33
26.0
Digannc
Distance
4.5
25.0
24.5
24.2
4.0
M-111
23.5
3.5
*
20
22
24
26
28
30
FID (1)
FID
Figure 2: Performance curves of Distance Metric FID Metric. We select different guidance scales for each method to obtain performance trend curves. It can be clearly observed that our method achieves a better guidance effect and exhibits greater robustness to the selection of guidance scale.
Table 2: Quantitative image-quality evaluation Imagnet under the linear inverse problem.
Inpainting
Super resolution
Gaussian deblurring
Methods
PSNRT
SSIM
LPIPS
FID
PSNRT
SSIM
LPIPS
FID|
PSNRT
SSIMT
LPIPS.
FID
DPS
27.56
0.825
0.251
30.57
22.07
0.633
0.317
41.36
18.78
0.387
0.523
52.13
LGD
27.78
0.824
0.242
28.65
22.23
0.631
0.305
39.85
19.52
0.452
0.501
50.42
DSG
28.67
0.877
0.176
23.63
23.74
0.696
0.291
34.28
22.64
0.644
0.335
45.27
Ours
29.23
0.889
0.154
19.25
23.80
0.697
0.290
33.06
22.65
0.645
0.321
41.65
5.3
MOLECULAR INVERSE DESIGN
Implementation. Inverse molecular design aims to generate 3D molecular structures that ex- hibit desired quantum properties. Following the settings in EDM (Hoogeboom al. 2022) and EEGSDE (Bao et al. 2022), we adopt the QM9 (Ramakrishnan et al. 2014) dataset, which contains 130k organic molecules with up to nine heavy atoms (C, N, O, F) and their corresponding quan- tum properties. Six properties are selected: polarizability (a), dipole moment (n), HOMO/LUMO energies (ÉномO, fLUMO), HOMO-LUMO gap (Ae), and heat capacity at constant volume (C,). All experiments are conducted with the pre-trained EEGSDE model. More specifically, for each prop- crty, two separate predictors are needed: (i) a time-dependent, noise-robust predictor (Z,, t) used by EEGSDE to guide the diffusion process, and (ii) an evaluation predictor J (xx). which has been trained on clean data only used to assess the generated molecules. In Table 3 "L-bound" represents the MAE error of the evaluation predictor the test data.
The baseline conditional EDM (cEDM) receives the desired property as an additional input during training leam conditional generative model without guidance. Based EDM, BEGSDE incor-
8

---

Published as a conference paper at ICLR 2026
porates an equivariant energy function E(r,, () (ga () -)) into the reverse-time SDE, where s is tunable scale and c is the desired quantum property. For EEGSDE, since Jo must be robust noise, is conditioned on both the perturbed coordinates Iy and the time step t. In contrast, our training-free approach only requires a predictor defined on clean data (t = 0). However. for fair comparison, we reuse the EEGSDE-trained Iø. but freeze when providing guidance.
Table 3: Quantitative Evaluation of Molecular Property Prediction. MAE represents the numerical discrepancy between the specific properties of generated molecules and the target conditions, while MS denotes the molecular stability index.
MAE
MS
MAE (1)
MS
MAE
MS
Method
Method
Method
(1)
(1)
(T)
(1)
(1)
Ce
/
CEDM
1.0650
CEDM
1.1230
CEDM
2.7804
EEGSDE
0.9187
0.7836
EEGSDE
0.7518
0.8036
EEGSDE
2,4133
0.7954
DSG
0.8447
0.7654
DSG
0.7811
0.8001
DSG
2.1919
0.7863
Ours M 3)
0.8348
0.7718
Ours(M = 3)
0.7274
0.8059
Ours(M = 3)
2.1001
0.7745
L-bound
0.0400
L-bound
0.0430
L-bound
0.0900
Ae
CHOMO
6LUMO
CEDM
0.6710
CEDM
0.3714
CEDM
0.6015
EEGSDE
0.4854
0.7572
EEGSDE
0.2997
0.7890
EEGSDE
0.4450
0.8000
DSG
0.4558
0.7890
DSG
0.2673
0.7854
DSG
0.3969
0.8081
Ours(M
3)
0.4182
0.7909
Ours(M 3)
0.2449
0.7872
Ours(M 3)
0.3778
0.8181
L-bound
0.0650
L-bound
0.0390
L-bound
0,0360
For evaluation, we still adhere to the principle of dual-focus evaluation, assessing both the metric of Molecular Property Deviation from Conditions (MAE) and the Molecular Stability (MS) metric. More specifically, we observe that for each method, as the guidance scale increases, the MAE metric gradually decreases, but this may be accompanied by decline in the MS metric. Therefore, reproduced the parameters of EEGSDE, and then adjusted the guidance scales of the DSG and ABMS methods until their MS metries either outperformed EEGSDE's differed by no more than 2% before comparing their MAE metrics.
first
Evaluation Results. As shown in Table 3] under the condition of comparable molecular stability (MS), our method achieves superior MAE metrics across six distinct conditional molecular inverse design tasks. This further demonstrates that our method can provide more accurate conditional guid- ance compared to existing methods. It can be observed that tasks conditioned on precise numerical values exhibit significantly degraded performance when guidance is absent, corroborating the neces- sity of explicit guidance mechanisms. Moreover, we emphasize that our framework only requires property predictor defined on clean data; for fair comparison, we reused the noise-robust predictor trained by EEGSDE, which may partially limit the achievable performance.
5.4 SCALING TEXT STYLE GUIDANCE
To thoroughly validate the generalizability of our method, We conduct experiments on diffusion model of larger scale and distinct training strategy. Specifically, we perform text-style guidance task and adopt Stable Diffusion 3.5 (Rombach et al., [2022] Esser et al. 2024) as the generative diffusion prior; which a **flow matching*** mitmanng**_ipman al. [2023] et al. 2023 based model and
receives piece of text description input. We adopt the method proposed in (Liu et al. 2025 to enable SDE sampling. We apply the style loss for guidance, which is defined as:
C(ān(x:), ) = [((o(()))) - E(
(14)
where Zin is the style reference image, E represents the Gram matrix the third feature map extracted from the CLIP image encoder (Radford et al. 2021). and 11-11, denotes the Frobenius norm. The size of the generated images is 512x512. Figures 3 presents the visual results of our method with M 3 compared to the baseline method. As be seen, method also achieves satisfying performance on flow matching based models and produces clearer and much higher image quality while ensuring adherence to the conditional guidance. This is because more accurate guidance

---

Published as a conference paper at ICLR 2026
gradients enable the generation process satisfy the conditions much possible while not undermining the prior diffusion capabilities.
Style
DSG
Ours
Figure 3: Qualitative result of Text-style guidance, the text input "A corgi wearing a wizard hat". Our method generates much clearer and higher quality results than the baseline method across all the target style.
CONCLUSION & DISCUSSION
In this paper, we propose simple ald--ay strategy ABMS improve the existing DPS- based guidance methods and dual-focus evaluation paradigm to offer more discerning assessment of guidance methodologies. To mitigate the giant estimation error when calculating the guidance gradient, we demonstrate that the strategy of performing additional denoising step with Monte- Carlo sampling can reduce the variance interference, effectively improves the performance.
Limitation and future work. Although the proposed method has demonstrated its efficacy in integrating and enhancing existing approaches, constrained by computational resources, we have yet systematically investigate whether further increasing the number of reverse diffusion steps the sampling budget yields additional gains across diverse scenarios. Additionally, how the proposed methodology can be adapted to paradigms that enable very few-step generation is also open question. These will be exciting directions for future work.
7 ETHICS STATEMENT.
This paper focuses the research of general artificial intelligence technology, specifically the gen- erative diffusion models, and does not involve ethic issues.
REPRODUCIBILITY STATEMENT.
For fair comparison, all the experiments mentioned strictly follow the framework, data, and pre- trained diffusion models from existing articles, with corresponding references provided in the main text.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Ap- plications, 12(3):313-326, 1982.
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. CVPR, 2023.
Fan Bao, Min Zhao, Zhongkai Hao, Peiyao Li, Chongxuan Li, and Jun Zhu. Equivariant energy- guided sde for inverse molecular design. NeurIPS, 2022.
Gabriel Cardoso, Yazid Janati El Idrissi, Sylvain Le Corff, and Eric Moulines. Monte carlo guided diffusion for bayesian linear inverse problems. ICLR, 2024.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. NeurlPS, 2022.
Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L. Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. ICLR, 2023.
Gang Dai, Yifan Zhang, Qingfeng Wang, Qing Du, Zhuliang Yu, Zhuoman Liu, and Shuangping Huang. Disentangling writer and character styles for handwriting generation. CVPR, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, 2009.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. CVPR, 2019.
Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. NeurlPS, 2021.
Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. ICLR, 2024.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In /CML, 2024.
Berthy Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine L Bouman, and William T Freeman. Score-based diffusion models principled priors for inverse imaging. CVPR, 2023.
Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei- Hsiang Liao, Yuki Mitsufuji, Zico Kolter, Ruslan Salakhutdinov, et al. Manifold preserving guided diffusion. ICLR, 2024.
Jonathan Ho and Tim Salimans. arXiv:2207.1225 125.9987..28..... 2022.
Classifier-free diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffu- sion for molecule generation in 3d. In ICML, 2022.
Yazid Janati, Badr Moufad, Alain Durmus, Eric Moulines, and Jimmy Olsson. Divide-and-conquer posterior sampling for denoising diffusion priors. NeurlPS, 2024.
Tero Karras, Samuli Laine, and Timo Aila. A style- generator architecture for generative adversarial networks. CVPR, 2019.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. NeurlPS, 2022.
Yaron Lipman, Ricky TQ Chen, Heli Ben cen-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. ICLR, 2023.
11

---

Published as a conference paper at ICLR 2026
Cheng-Lin Liu, Fei Yin, Da-Han Wang, and Qiu-Feng Wang. Casia online and offline chinese handwriting databases. Icdar, 2011.
Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. NeurIPS, 2025.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. ICLR, 2023.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. NeurIPS, 2022.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. Machine Intelligence Research, pp- 1-22, 2025.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. CVPR, 2022.
Nanye Ma Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang. Yandong Li. Tommi Jaakkola, Xuhui Jia, et al. Scaling inference time compute for diffusion models. In CVPR, 2025.
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective on solving inverse problems with diffusion models. arXiv preprint vrXiv:2305.04391, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. JCML, 2021.
Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):177, 2014.
Min-Si Ren, Yan-Ming Zhang, and Yi Chen. Decoupling layout from glyph in online chinese hand- writing generation. ICLR, 2025.
Robin Rombach, Andreas Blattmann, Dominík Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. CVPR, 2022.
Jascha Sohl-Dickstein, Eric A Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. ICML, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. ICLR, 2021a.
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. ICML, 2023.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. NeurlPS, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. ICLR, 2021b.
Bram Wallace, Akash Gokul, Stefano Ermon, and Nikhil Naik. End-to-end diffusion latent opti- mization improves classifier guidance. ICCV, 2023.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. ICLR, 2023.
Yilun Xu. Mingyang Deng, Xiang Cheng, Yonglong Tian, Ziming Liu, and Tommi Jaakkola. Restart sampling for improving generative processes. NeurIPS, 36.76806-76838, 2023.
12

---

Published as a conference paper at ICLR 2026
Lingxiao Yang, Shutong Ding, Yifan Cai, Jingyi Yu, Jingya Wang, and Ye Shi. Guidance with spherical gaussian constraint for conditional diffusion. ICML, 2024.
Fei Yin, Qiu-Feng Wang, Xu-Yao Zhang, and Cheng-Lin Liu. Icdar 2013 chinese handwriting
recognition competition. Icdar, 2013.
Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training-free energy-guided conditional diffusion model. ICCV, 2023.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. ICCV, 2023.
Tao Zhang, Jia-Shu Pan, Ruiqi Feng, and Tailin Wu T-scend. Test-time scalable mets-enhanced diffusion model. arXiv preprint arXiv:2502.01999, 2025.
Yuanzhi Zhu, Kai Zhang. Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and and-play image restoration. CVPR, 2023.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1
ADDITIONAL PROOF
Lemma (Jensen Gap Upper Bound), Let f Rd - be continuously differentiable function with L-Liscchitz gradient, i.e.
IV Vf(e))l & -llu-vll YU.ERR".
(15)
Then for any random variable with finite second moment, the Jensen gap satisfies:
L
:= (B[] - B[f(x(=)
2 Tr(Cov[X|a)).
(16)
Proof. We proceed by Taylor expansion and Lipschitz gradient property.
Step 1: Second-order Taylor expansion. Expanding around IE. [x]]]] with remainder term:
f(X) = f(E[X + vf(E[X(=)) (x -[
+ / a1-- (1-xxxx (1-))(X--XXx)))>>>>>>XX s(X E(xx+)) - [[x(2]) ds.
(17)
Step 2: Taking conditional expectation. Taking B[||3c] on both sides of equation 17
[] = ([[[X(=) + (=(((X(=))) EEX [[X
E / (1 - s)(X (1-s)(X-
V -
d x (18)
The second term vanishes since B[X [[X|>|| Thus:
= E L - EXb) E[X[2]) ds
(x(x) =
(19)
Step 3: Bounding the Hessian norm. The T-LLipscitz gradient property implies that for any u, Rdd
(v) (f(ill s L||u (ell-=lll
(20)
This ensures that the Hessian (in the sense of weak derivatives) satisfies the spectral bound:
f() L,
(21)
where || denotes the operator norm (largest cigenvalue).
Step Upper bounding the quadratic form. For any vector Rd:
v f(u)u & ||V f(u)llll]
S T||oa|?.
(22)
Therefore:
(X
s -
(23)
Step 5: Taking expectation. Applying conditional expectation:
EE (X [[ f()(( -[(|=) LE ||X - ||
L. Tr(Cov(X(=))
(24)
14

---

Published as a conference paper at ICLR 2026
Step 6: Final bound. Substituting back into equation 19
16y(z) S E 1 (1- [1-s)[iIX-- - - [|X ds
L Tr(Cov(X(o)) /. (1 8) ds
2 Tr(Cov(X[a))
(25)
This completes the proof.
A.2 SAMPLING TIME OVERHEAD ANALYSIS
In practical applications, as shown in the experiment section we have found that small number of Monte-Carlo sampling times already yields satisfactory results. To save computational time, we notice that computing the gradients with respect to multiple samples æt-1 can be parallelized. We generally can choose the largest possible number of samples to ensure parallelization. In this case, Table reports the number of diffusion- model denoising iterations completed per second under different values M for the image inverse problem.
In addition, variety of acceleration techniques have been developed to speed up sampling in diffu- sion models (Song et al. 2021a Lu et al. 2022 2025 Lipman et al., 2023). Among them, DDIM and flow matching approaches fore verified experimentally that our method remains compatible with these strategies, rendering the resulting computational overhead entirely acceptable.
introduce stochasticity every sampling step, we have there-
Table 4: The inference time cost for different M.
DPS/DSG
ABMS(M =
ABMS(M = 2)
ABMS(M 3)
ABMS(M = 4)
ABMS(M = 5)
4.8 iter/s
2.5 iter/s
2.1 iter/s
iter/s
iter/s
1.3 iter/s
A.3 ADDITIONAL EXPERIMENT RESULTS
In this section, we present some additional experimental results. As shown in Figure the con- tent guidance accurately corrects the trajectory structure of the generated characters, verifying the effectiveness of guidance method.
Content
Baseline
Ours
Figure 4: Qualitative result of content guidance. Compared baseline model, the guidance we applied corrects the detailed strokes of the generated characters, making the character structures more accurate. The parts with structural emrors are circled black box.
15

---

Published as a conference paper at ICLR 2026
Figure displays the experimental results for the Gaussian deblurring task discussed in the main text. We also provide the visualization of the linear inverse problems as shown in Figure 6 can be observed that our improved strategy exhibits fewer artifacts and enhances the image quality.
Comparison
Deblurring
DSG
22.70
0.06
Ours (M=5) (M
22.85
22.60
Distance (L)
22.35
22.50
0.000
0-000
W 0
22.40
40.0
42.5
45.0
47.5
50.0
52.5
55.0
57.5
(4)
Figure 5: Performance curves of Distance Metric FID Metric on Gaussian Deblurring task.
We also evaluate our proposed methods on another nonlinear task: FacelD guidance. We utilize the pre-trained diffusion model from CelebA-HQ 256x256, provided by Freedom (Yu et al.) 2023). We pick 1000 samples from the FFHQ test set (Karras al. (019) as the reference images. To guide the process for input reference image I, we apply the FaceID loss, which is defined as:
((i CE(E((o(z;)) E(T)),
(26)
where F!()) denotes the face recognition network (Deng et al. 2019) and CE represents the Cross- Entropy Loss. We employ the DDIM sampler with 100 sampling steps.
Figures 7 presents the visual results of our method compared to other approaches. As can be seen, method produces clearer details and higher image quality while ensuring adherence to the con- ditional guidance.
16

---

Published as a conference paper at ICLR 2026
Gaussian Deblurring
Inpainting
Super Resulution
Measuromen
Ground Truth
DSG
Ours
Figure 6: Qualitative result of linear inverse problems. Compared existing methods, our improve- ments slightly enhance the image quality.
"Refornoce
Froedoor
Dvers
Figure 7: Qualitative result of FacelD guidance. Compared with existing methods, our improved approach generates more smooth results and less likely to produce incoherent regions.
17


Reviews:
[{"review_id": "4IHDCT1lEZ", "reviewer": "Reviewer_vUip", "summary": "This paper addresses a critical and widely recognized problem in training-free guided diffusion models: the degradation of sample quality (e.g., FID) under strong conditional guidance. The authors convincingly argue that this issue stems from a systematic estimation error and bias in the guidance gradient, which is typically derived from a single, noisy denoising step. To mitigate this, they propose ABMS, a plug-and-play Monte-Carlo sampling strategy. At each reverse diffusion step, instead of relying on a single estimation path, ABMS explores multiple (M) potential predecessor states, denoises each one to predict a clean output, and then averages the guidance function evaluations across these M predictions. This Monte-Carlo averaging yields a more stable and accurate estimate of the true guidance gradient. The paper provides a theoretical justification (Proposition 1) showing that ABMS achieves a lower estimation error bound compared to the standard DPS approach. The effectiveness of ABMS is demonstrated empirically across a diverse set of tasks, including stylized handwriting generation, standard image inverse problems (inpainting, super-resolution, deblurring) on ImageNet, and molecular property prediction. The results consistently show that ABMS allows for stronger guidance without the typical collapse in sample quality, outperforming existing methods.", "questions": "See the weakness.\nWhile the Weakness.3 is the most important, which affects the innovation of this paper, with an excellent explanation, I will increase the score, while a bad explanation, I will decrease the score.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper tackles a fundamental challenge in guided diffusion. The \"dual-focus evaluation\" paradigm, which explicitly calls for balancing task-specific performance (e.g., reconstruction error) with general sample quality (e.g., FID), is an excellent and necessary framing for this problem area. The authors clearly articulate why strong guidance often leads to poor results, providing strong motivation for their work.\n2. The proposed ABMS method is simple, well-motivated, and elegant. The core idea of using Monte-Carlo averaging to obtain a more robust estimate of an expectation is a classic statistical principle applied very effectively here. Its \"plug-and-play\" nature makes it broadly applicable to various diffusion frameworks and tasks without requiring model retraining, which is a significant practical advantage.\n2.1 While not a deep theoretical paper, the inclusion of the estimation error analysis and Proposition 1 provides a solid theoretical grounding for why ABMS should be expected to work better than single-path estimators. It connects the intuitive idea of averaging to the mathematical problem of reducing the bias introduced by Jensen's inequality.\n\n3. The experimental section is a major strength of this paper.\n  - Diversity of Tasks: Testing the method on stylized text, natural images, and molecular data convincingly demonstrates its generality.\n  - Rigorous Evaluation: The use of performance curves (Figure 3) that plot task-specific distance against FID is particularly effective. These plots provide a clear and powerful visualization of the core contribution, showing that ABMS dominates other methods by achieving a better trade-off frontier.", "weaknesses": "1. The primary drawback of ABMS is the increased computational cost, which scales with the number of Monte-Carlo samples, `M`. The paper demonstrates the effectiveness of `M=3` and `M=5` but never explicitly analyzes or reports the trade-off between performance and inference time/FLOPs. For a sampling method, this performance-cost analysis is crucial for researchers and practitioners to assess its viability. While Figure 3 implicitly shows the performance gain for different `M`, the associated cost is not quantified.\n2. While the use of ImageNet 256x256 is a standard and respectable benchmark, the field of generative models is rapidly moving towards much higher resolutions (512x512, 1024x1024) and significantly larger models (e.g., Stable Diffusion). The paper does not demonstrate whether the benefits of ABMS hold or are even more critical at this larger scale, where guidance is often essential. Demonstrating scalability to at least one high-resolution setting would substantially increase the paper's impact.\n3. The paper would benefit from a more detailed discussion of how ABMS relates to other methods that also aim to improve guided sampling by investing more computation. A key missing comparison is with \"Restart Sampling\"[1]. Both methods address quality degradation under strong guidance but seem to operate on different principles. A discussion clarifying these differences would better situate the paper's contribution within the current literature.\n\nRefrence:\n[1] Xu, Yilun, et al. \"Restart sampling for improving generative processes.\" Advances in Neural Information Processing Systems 36 (2023): 76806-76838.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UE3Ug4GuwV", "reviewer": "Reviewer_93q3", "summary": "The paper targets bias in training-free diffusion posterior sampling (DPS) for conditional generation and inverse problems. DPS approximates the posterior p(x0∣y)∝p(x0)exp⁡[−L(x0;y)]p(x_0|y) through two linearizations: (1) moving the expectation inside the loss, and (2) replacing the conditional mean with the denoiser output via Tweedie’s formula. The authors argue these lead to biased gradients and propose ABMS (Additional Backward step with Monte-Carlo Sampling): before estimating guidance at step ttt, it samples xt−1∼ p(xt−1∣xt) multiple times, evaluates the loss on each denoised x^0(xt−1), and averages the results to reduce bias. A dual-focus evaluation (alignment vs. global quality) is introduced. Experiments on classifier-guided digit synthesis, image inverse problems (SR, inpainting, deblurring), and molecular property conditioning show modest but consistent gains.", "questions": "Can the authors clarify whether the proposed MC sampling yields an unbiased estimator of gradients? If not, what assumptions make the bias negligible?\n\nDoes the ABMS correction still hold under deterministic DDIM/flow samplers?\n\nHow does the method compare to re-scoring or re-noise-based variance-reduction schemes such as RED-Diff or Score-DPO?\n\nAre there ablations showing the effect of the hypersphere projection alone versus MC averaging alone?\n\nFor molecular tasks, how sensitive is performance to the sampling count MMM?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The problem—bias in loss-guided diffusion—is relevant and well-motivated.\n\nThe proposed method (MC sampling one step earlier) is simple, plug-and-play, and compatible with existing samplers. Also parallelization of MC sampler justifies the added computations. \n\nEvaluation across multiple domains, with clear quantitative metrics.", "weaknesses": "The “unbiased” claim is overstated: ABMS still produces a lower-bias approximation but not a provably unbiased gradient of the tilted posterior. No formal unbiasedness proof or convergence result is provided.\n\nThe theoretical bound (Sec. 4.2) only compares error upper bounds under strong assumptions (Lipschitz fff, monotone denoiser accuracy). Variance of the stochastic gradient is unaddressed.\n\nScope limited to DPS. Extensions to other plug-and-play or variational samplers (e.g., RED-Diff, flow-matching, ODE solvers) are not discussed experimentally.\n\nThe empirical improvements, though consistent, are incremental; figures often lack statistical significance or ablations isolating the MC vs. scaling effects.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oovp7ezQty", "reviewer": "Reviewer_BtcN", "summary": "This paper addresses a training-free guidance method to improve the inverse problem in diffusion models. The authors argue that the existing method, DPS, suffers from a biased gradient. To mitigate this, ABMS leverages the gradient of the averaged multiple backward predictions of diffusion models. With theoretical justification, ABMS shows improved results compared to existing baselines.", "questions": "- **Clarification on Computational Cost**: Please refer to Weakness 1.\n\n- **Ablation Study Without DSG**: ABMS opts for the DSG for the stability of the algorithm. It could be helpful if the base performance of ABMS without DSG is analyzed compared to DPS.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- **Well-Motivated and Simple Solution**: The proposed ABMS method is intuitive, well-motivated by the law of total expectation, and directly targets the identified source of bias (single-point estimation). It seems to be easily adapted to the existing codebase due to its simplicity and can be widely utilized as it does not require any additional conditions.\n\n- **Comprehensive Experiments**: The method's effectiveness is demonstrated across three different domains. The consistent improvements across all tasks support the claims.\n\n- **Theoretical Analysis**: The estimation error analysis showing that ABMS's error bound under the given assumptions avoids the $\\delta_{f}(x_t)$, which plagues the DPS bound, provides a theoretical justification for the effectiveness of the ABMS.", "weaknesses": "- **Computational Overhead**: The most significant weakness is the increased computational cost. ABMS requires $M$ denoising network evaluations in addition to the original denoising steps. While it can be parallelized, the memory consumption can grow rapidly as it also requires additional gradient calculations. For a more comprehensive analysis, the additional computational time and memory consumption for ABMS should be reported.\n\n- **Novelty in Context**: The idea of using Monte Carlo sampling to get better estimates in diffusion guidance is not entirely new (e.g., LGD-MC). The paper's novelty lies in the one-step-back formulation. The paper could be stronger if it more directly compared against a simpler MC-DPS baseline (i.e., averaging $M$ estimates from $x_t$, not $x_{t-1}$) to isolate the benefit of the backward step from the benefit of MC sampling. Furthermore, while the authors argue that LGD-MC incurs high computational costs, the proposed ABMS is a more computationally intensive method as it requires multiple diffusion model calls with gradient calculation, while LGD-MC requires only one diffusion model call.\n\n- **Limited Scope of Samplers**: While the authors acknowledge this limitation, it is unclear how ABMS, which relies on the SDE-based one-step transition, would be adapted to faster ODE-based or higher-order solvers (like DPM-Solver++, etc.) that are now state-of-the-art for fast sampling. This potentially limits the method's practical application.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WSmP4DhRsZ", "reviewer": "Reviewer_Daim", "summary": "The paper proposes ABMS for solving inverse problems using diffusion models.\nIt identifies and addresses errors arising from the imprecise gradient guidance in a prominent baseline method, DPS.\nTo resolve this, ABMS computes the gradient guidance by drawing multiple samples at each denoising step and using the diffusion model's predictions for those samples.\nThe proposed method shows enhanced performance on a variety of tasks, such as character generation, image restoration, and monocular property prediction.", "questions": "Is the proposed methodology applicable to recent state-of-the-art diffusion models such as Flux-dev?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- It highlights that many existing training-free inverse problem methodologies including DPS rely on numerous assumptions and approximations, which often leads to suboptimal results.\n\n- It makes a significant contribution by demonstrating the trade-off inherent in using gradient guidance through a dual-focus evaluation.", "weaknesses": "- There are concerns regarding the practical applicability of the proposed method. ABMS is computationally heavy as it requires M diffusion model operations at each step, and 1000 sampling steps.\n\n- Most of the demonstrated tasks have limited practicality; for instance, the inpainting task only uses very small masks instead of large ones.\n\n- The results are shown on pixel-space diffusion models. However, state-of-the-art diffusion models like Stable Diffusion or Flux-dev operate in latent space. Including results on these models would broaden the paper's scope.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)