Adaptive Moments are Surprisingly Effective for Plug-and-Play Diffusion Sampling
Authors: Christian Belardi
Justin Lovelace
Kilian Q Weinberger
Carla P Gomes
Venue: ICLR 2026 Poster (2026)
Keywords: Guided Diffusion Sampling
Plug-and-Play Conditional Diffusion Sampling
Adaptive Moment Estimation
Primary area: generative models
TL;DR: Adaptive moments are surprisingly effective for plug-and-play diffusion sampling.

Abstract: Guided diffusion sampling relies on approximating often intractable likelihood scores, which introduces significant noise into the sampling dynamics. We propose using adaptive moment estimation to stabilize these noisy likelihood scores during sampling. Despite its simplicity, our approach achieves state-of-the-art results on image restoration and class-conditional generation tasks, outperforming more complicated methods, which are often computationally more expensive. We provide empirical analysis of our method on both synthetic and real data, demonstrating that mitigating gradient noise through adaptive moments offers an effective way to improve alignment.

Published as a conference paper at ICLR 2026
ADAPTIVE MOMENTS ARE SURPRISINGLY EFFECTIVE FOR PLUG-AND-PLAY DIFFUSION SAMPLING
Christian Belardi
Justin Lovelace
Kilian Q. Weinberger
Carla P. Gomes
Cornell University
@kb73@cornel1.cm
ABSTRACT
Guided diffusion sampling relies approximating often intractable likelihood scores, which introduces significant noise into the sampling dynamics. We pro- pose using adaptive moment estimation stabilize these noisy likelihood scores during sampling. Despite its simplicity, our approach achieves state-of-the- art results on image restoration and class-conditional generation tasks, outper- forming more complicated methods, which are often computationally more ex- pensive. We provide empirical analysis of our method on both synthetic and real data, demonstrating that mitigating gradient noise through adaptive mo- ments offers an effective way to improve alignment. Our code available https: //t:://iii //gtttb...... com/ chr iirriste st anbe wawaeeeerdda. ardi aaciiiiaamm-uuuu adam-
INTRODUCTION
2019 Ho al. 2020) have become one the most prevalent approaches to generative modeling, achieving state-of-the-a results in many domains, including text-to-image synthesis (Rombach et al. 2022 Ramesh et al., 2022]] Sa- haria et al. 2022a), image-to-image translation (Saharia et al. (0022b) audio generation (Kong et al.,
Diffusion models (Sohi-Dickstein et al. 2015; Song & Ermon
2020 Liu et al. (023), video synthesis (Ho et al. 2022 Brooks et al. 2024 and molecular design
Hoogeboom et al. 2022 Watson et al. 2023).
Plug-and-play conditional generation enables sampling from conditional distribution p(xly) using a diffusion model trained only on the marginal distribution p(r). These methods guide the sam- pling process toward desired conditions y without task-specific training. While some approaches like Classifier Guidance (CG) (Dhariwal & Nichol 202T) train time-aware models directly on the diffusion latents, many and-play methods leverage existing models that operate on clean data- whether analytical forward operators for inverse problems or pre-traine pre-trained classifiers making them highly flexible for diverse applications.
The plug-and-play guidance literature has evolved from early methods like Diffusion Posterior Sam- pling (DPS) (Chung al., (023) to increasingly sophisticated techniques. Recent work such as Universal Guidance for Diftusion Models (UGD) (Bansal et al. (023) and Training- Guidance (TFG) Ye et al., 2024) compose multiple algorithmic components, gradient computations both latent and data spaces, Monte Carlo approximations, and iterative refinement procedures.
At the heart of plug-and-play guidance lics the challenge incorporating the desired condition into the diffusion sampling process. Sampling in diffusion models can be understood annealed
Langevin dynamics using the score function Vx, log p(z,) (Song Ermon 2019: Karras et al, For soug-and-play diffusion sampling, Bayes' rule gives
2022
V 2, log p(x:ly) Va, log (og)(()))))))))))))())))))) Va, log p(yl()))))
Posterior Score
Prior Score
Likelibood Score
While the prior
is provided by the unconditional diffusion model, the likelihood score
Va, log p(y xx) is often intractable to compute directly, necessitating approximation strategies
Chung et al. 2023 He et al. 2023 Song et al. 2023b).
Prior work has predominantly studied improving the likelihood score approximation. Orthogonal to this, investigate whether information from earlier sampling steps can help mitigate approx-

---

Published as a conference paper at ICLR 2026
imation errors in later sampling steps. Instead of developing more sophisticated approximation methods, we use adaptive moment estimation- technique from stochastic optimization, Adam (Kingma 2014 to stabilize the noisy guidance gradients that arise in alug-and-py -lay methods.
Our approach maintains exponential moving averages of the first and second moments of the likeli- hood score estimates across sampling steps, dampening noise while preserving the guidance signal. Despite its simplicity, this modification yields substantial improvements for DPS and CG.
We also examine how task difficulty affects performance. Typically, plug-and -lay methods evaluated under mild degradations that provide strong guidance signal (e.g., 4x super-resolution) We show as difficulty increases (e.g., 16x super-resolution), many methods degrade rapidly. We find that across all difficulties, adaptive moment estimation consistently improves the DPS baseline while other guidance methods fall below DPS alone.
Our contributions are: (i) we demonstrate that adaptive moment estimation can substantially improve plug and ald- guidance methods, achieving state-of-the tft results with minimal added complexity; (ii) we provide a variety of empirical analyses that illustrate how our method stabilizes noisy gradi- ents and improves performance; and (iii) we reveal that task difficulty significantly impacts relative method performance, suggesting the need for more comprehensive evaluation standards.
2 BACKGROUND
Diffusion Models and Score-Based Sampling, Diffusion models (Sohl-Dickstein et al. 2015
Song & Ermon 2019, Ho et al. 2020) learn reverse a gradual process that adds noise to data. Starting from a clean data sample To. we repeatedly add small amounts of Gaussian noise until the result becomes almost pure noise. We describe this process using timestep E [0, 7], where each t corresponds to a noise level o,, The variable o, measures how much total noise has been added at that step, and 2y denotes the resulting noisy version of the data, As t increases, the amount noise also increases, choose of large enough that 2T is indistinguishable from Gaussian noise. The specific schedule of noise levels {o,) } can vary, but it always follows this pattern of gradually increasing noise.
that de < ... dr. In practice, set dy that zo is the original data, and
For two timesteps 8 and & with 8 < f, the noising process describes how a partially noised sam- ple T, becomes noisier sample ry. This transition follows Gaussian distribution p(ral (2,) (z,; "),, where the variance between the two steps is given by o² = of - a.. Because the sum of Gaussian variables is also Gaussian, we can treat many small noising steps as one com- bined step. Thus, the noising process also defines a direct marginal distribution from the original data zo to any timestep f,, p(zz/ (xo) = =N(xx;; Fo, oT Equivalently, we can express the noisy sam- ple as x2 = To + otC, where 6 AO, TI). This form shows that 24 is simply the clean data 2i plus Gaussian noise scaled by the noise level oz.
To reverse the noising process, we train a neural network eg to predict the noise that was added to the clean data zo in order to produce the noisy sample xı. The standard training objective minimizes the difference between the true noise e and the network's 's prediction (Ho et al. 2020
(0) Er,a [Ilea t) - <|12]
(2)
By learning to predict this noise, the network implicitly learns the gradient of the bog-rrbbabylity density of the data also known as the score function (Kingma & Gao 2023):
c0(x1.1) de
so so(
R Var log p(r.).
Intuitively, the added noise moves data away from regions of high likelihood. Thus, predicting the noise tells us how to move 24 to increase its probability under the data distribution, which is precisely the meaning of the gradient Vx, p(zz( Once the score function is known, we can generate
new data by starting from pure noise IT ~ NN(0, I) and gradually denoising it. This sampling procedure follows annealed Langevin dynamics (Song & Ermon 2019
Karras et al. 2022):
Vz) logp(x:) + Vo? o? ot
J 3t + (0
(4)
We adapt the notation of Kingma et al. 2021 to the variance exploding formulation for simplicity.
2

---

Published as a conference paper at ICLR 2026
This update incrementally removes noise until we obtain clean sample from the data distribution. At any timestep f, the denoising network provides an estimate of the underlying clean data:
Tole
[t] 2, of eo (2,,
(5)
which corresponds to the minimum mean squared error estimator under optimal training (Efron 2011). This estimate, denoted Zoje- plays a central role in many plug-and guidance methods.
The Plug-and-Play Guidance Challenge. The goal of plug-and-plas diffusion sampling is to draw samples from the posterior distribution p(roly). ly), which represents data consistent with the observa- tion y. This approach relies the decomposition of the posterior first term, the prior score, is provided by the diffusion model itself and well approximated using the trained network eg. The second term, the likelihood score, expresses how the observation y changes the probability of a noisy sample Ir. However, this term is much harder to compute in practice.
described in Equation 1| The
To evaluate the likelihood score, would need marginalize over all possible clean samples x0 that could have produced 2y:
/ p(y) ((zul drg-
p(y =
(6)
This integral is generally intractable, since it requires integrating over the entire data space. Even if we train a neural network to approximate p(y]ax)) directly, its gradients tend be noisy and unstable, leading to poor guidance during sampling.
3 GUIDANCE WITH ADAPTIVE MOMENT ESTIMATION
Plug-and-play diffusion sampling requires approximating the often intractable likelihood score V., log p(y]2;)) to guide the sampling process toward desired conditions. practice, this score is typically approximated by LL y/, where L is the negative log likelihood and the condition.
Existing Likelihood Score Approximations. There are two foundational approaches for this ap- proximation, distinguished by the domain in which the guidance function operates. DPS (Chung) ct al. 2023), a widely adopted plug-and-play method, approximates the likelihood score as,
Rx logl(y) & Vz, ((y an (( 3)
The DPS approximation replaces the noisy latent x, with the denoising model's estimate Zaje. providing point estimate of the likelihood. This allows the guidance model fo X - 3/ to be any differentiable function that operates clean both learned models (e.g.. pretrained neural networks) and analytical models (e.g., Gaussian blur kernels) though it requires backpropagation through the denoising network.
where Zoje is the predicted clean sample from Equation 5
Alternatively, Classifier Guidance (CG) (Dhariwal & Nichol 2021) uses the approximation,
Vz, log ()) C(fo t).
(8)
In contrast to DPS, CG trains specialized guidance model fø : X, x T y that operates directly on noisy latents 2t, conditioning on the diffusion timestep f. making the guidance model time-aimeeee aware. This eliminates the need for DPS's point estimate approximation and provides a more direct estimate of the likelihood score, though it requires training a classifier that depends on t.
examine the typical plug-and-play sampling update by replacing the prior score in Equation 4 with the posterior score and decomposing it as shown in [qquatio 1 Finally replacing the prior score with the denoising networks approximation 88, and the likelihood score with either of the approximations given in [qquatio 7 or [quation 8 we observe that the sodg-and-pla sampling update is performing gradient descent on the likelihood component at each timestep:
The Plug-and-Play Sampling Update. We
(ot -) + Vo 0Z -z- at
Is =
I, =
Stabilization via Adaptive Moments. Regardless the approximation strategy, guidance methods suffer from substantial noise in likelihood score estimates. In stochastic optimization, prominent
3

---

Published as a conference paper at ICLR 2026
Algorithm Adam Diffusion Posterior Sampling (AdamDPS)
Require: Diffusion Model 0, Guidance Model fe. Condition y, Guidance Strength p. Sampling Timesteps
tn S T, First Moment 0, Second Moment O, Adam Step & = 0, First Moment
Exponential Decay Rate B1, Second Moment Exponential Decay Rate f82
1: St, (0,1
2: for == bus ......,,,, do
3:
9t V ((a(()))
4: 9r.
= Aduppiveeomeensstiaacce m, v, 82)
5: 3y Sample(zaja 24, 8) p0:
end for
7: return Ito
Algorithm Adam Classifier Guidance (AdamCG)
Require: Diffusion Model Guidance Model Jø. Condition y, Guidance Strength p. Sampling Timesteps
Lo....,4g to _ T. First Moment m Second Moment O, Adam Step 0, First Moment
Exponential Decay Rate 31. Second Moment Exponential Decay Rate B2
I: St. ~ (0,
2: for t tn...., do
91 = af t), Se. m, v, = AdapiiveeoeelEEsiiaeee... m, u, kr (2)
5: 2, = Sample(zuje + pû.0? (,)
6: end for
7:
return Zio
methods like RMSProp (Tieleman & Hinton 2012) and Adam (Kingma 2014) use adaptive moment estimation to stabilize noisy gradients. Inspired by this insight, propose maintaining exponential moving averages of the likelihood score estimates and their squared values across sampling steps,
m4 81m=-1 + 1 B1)9:
-2 + (1 -32)9
(10)
where l: is a step counter, St = C.)),,, and of denotes element-wise squaring. The stabilized likelihood score is then computed as,
g: = mik
(11)
Vēx +
with bias-corrected moments vin = mm (1 - B4 and Ux = un/(1- 35), and ó a small constant for numerical stability (Kingma 2014).
Adaptive moment estimation serves two purposes: the first moment (momentum) smooths the op- timization trajectory by accumulating gradient information across steps, while the second moment adaptively scales the updates based on the historical variance of each gradient component. Since likelihood score approximations can vary dramatically across noise levels during diffusion sam- pling, this stabilization is essential for improving performance. We denote the resulting methods as AdamDPS (Algorithm [ when applied to DPS and AdamCG (Algorithm 2 when applied to CG, and demonstrate that this simple modification yields substantial improvements in sample quality.
4 SYNTHETIC STUDY
We begin by comparing DPS and AdamDPS in a synthetic setting, where both the true likelihood score and the ideal DPS likelihood score approximation can be computed exactly in closed form. This allows to study the effect noise has on guidance. To this end, the data distribution is defined by a dimensional Gaussian Mixture Model (GMM) with three components.
Guidance methods typically suffer from noisy likelihood score estimates due to limited conditioning information and taking gradients through large neural networks. Since our synthetic setting is free of these noise sources, we simulate them by adding to the guidance term Gaussian noise of magnitude < log ((y)) In Figure I (Left), we measure how close the empirical distributions are to function of the guidance noise coefficient Ç. We find that AdamDPS is much more robust to noise than DPS, achieving lower KL divergence with the target distribution as increases. Figuree 1 (Right) shows this qualitatively for specific Ç.
the target distribution
4

---

Published as a conference paper at ICLR 2026
20
Unconl
KL Divergence
DP5
15
AdamDPS
Unconditional
Oracle
10
5
DPS
AdamDPS
0
0.0
0.2
0.4
0.6
Empirical vs. Target Distributions
Guidance Noise Coefficient
Figure Left: The KL divergence between each method's empirical distribution and the target distribution function of the guidance noise coefficient Ç. Right: Visualization of the empirical
and target distributions at Ç 0.175.
Condition
Ground Truth
AdamDPS
DP5
TFG
Super Resolution
Gaussian Deblur
Figure 2: Qualitative comparison of AdamDPS, DPS, and TFG on Cats dataset for super resolution 12x downsampling and Gaussian deblurring blur intensity 9.
EXPERIMENTS
We evaluate adaptive moments for plug-and-play guidance across a range of tasks. All methods benchmarked were tuned with Bayesian Optimization (Jones et al.) 1998) on held-out valida- tion set of images. Reconstruction tasks were tuned to minimize LPIPS (Zhang et al., [0018), while class-conditional sampling was tuned for CMMD (Jayasumana al. 2024) since tuning for accuracy encouraged generating adversarial examples. We measure alignment with the desired con- dition using LPIPS for reconstruction tasks, and accuracy for conditiondi sampling. Accuracy computed as the harmonic mean across three held-out classifiers. We also report FID (Heusel measure of fidelity, computed
2048 samples following the evaluation proce- dure from Ye et al. (2024). We benchmark a variety of methods including Loss Guided Diffusion (LGD) (Song al. 2023b), Manifold Preserving Guided Diffusion (MPGD) (He et 2023), DPS, UGD, and TFG on ImageNet (Deng et al.] 2009)) CIFARIO (Krizhevsky et al. 2009) and the Cats subset of the Cats vs. Dogs dataset (Elson al. 2007). We set Nieur for TFG and sweep Nuer 1, 2, 4 for UGD and TFG, while tuning the remaining hyperparameters as recommended by Ye et al. (2024). We provide additional details in AAppendix A
al. 2017)
Reconstruction. Qualitatively, AdamDPS generates sharper, more realistic details in regions un- derdetermined by the conditioning information, where other methods tend to produce blurred or artifacted outputs. TFG reconstructions frequently exhibit visual artifacts, and DPS lacks the fine detail achieved by AdamDPS, see Figure 2) Quantitatively, for both ImageNet and the Cats dataset, AdamDPS outperforms all other methods on all reconstruction tasks: super resolution at 16x down- sampling. Gaussian deblurring at blur intensity 12, and inpainting with 90% random mask, Fie ure 3 Interestingly, the second best performing method across datasets in super resolution and Gaussian deblurring is DPS. We observe this happens challenging settings where the condition- ing information does not fully determine the target image, causing other methods to degrade signif-
5

---

Published as a conference paper at ICLR 2026
ImageNet
ImageNet
ImageNet
Super Resolution 16x
Gaussian Deblur 12
Inpainting 90%
MPGD
.38
LGD
MPGD
UGD
LPIPS
LGD
UGD
LGD
36
.30
DPS
UGD
TFG
.15
.34
DPS
TFG
AdDDS
AdanDPS
AdanDPS
40
40
50
30
Cats
Cats
Cats
Super Resolution
16x
Gaussian Deblur 12
Inpainting
90%
.25
-28
MPGD
LGD
UGD
RED RED-diff
.32
MPGG
.20
LPIPS
.26
30
DPS
DPS
TFG
.15
DPS
.24
TFG
AdanDPS
AdanDPS
AdanDPS
26
30
40
50
30
40
50
20
40
FID
Figure 3: Reconstruction performance measured in LPIPS and FID, where lower is better for both. Comparison on ImageNet and Cats dataset for super resolution at 16x downsampling, Gaussian deblurring at blur intensity 12, and inpainting with a 90% random mask.
CIFAR10
ImageNet
ImageNet
Standard Classifier
Standard Classifier
Time-Aware Classifier
50%
AdanDPS
Top 18 Accuracy
8%
AdarDPS
75%
AdanCG
Accuracy
DPS
TPG
6%
Accuracy
70%
30%
UGD
65%
20%
diff
2%
1160 ADPS
60%
Randon
RED di
10%
176
55%
100
200
100
200
30
31
FID
FID
FID
Figure 4: Class-conditional sampling performance measured classification accuracy and FID, where higher accuracy and lower FID is better. Accuracy is computed the harmonic mean across three held-out classifiers. Left & Center: Comparison plug-and-pla methods with a standard classifier CIFAR-10 and ImageNet, respectively. Right: Comparison of plug-and-play methods with a time-aware classifier on ImageNet.
icantly while DPS is more robust. Inpainting proves easier than 16x super resolution and intensity 12 Gaussian deblurring, evidenced by lower LPIPS for all methods. In this setting, TFG Niter can outperform DPS, yet still underperforms AdamDPS.
Cass-Conditional Sampling. Adaptive moments demonstrates strong performance on class- conditional tasks, see see Figure 4 On CIFAR-I0, AdamDPS outperforms the next best method, DPS, by 9.86% in classification accuracy. On ImageNet, all approaches except AdamDPS achieve top-10 classification accuracies at approximately 1,, equivalent to random chance. In contrast, AdamDPS achieves 10.49%, demonstrating substantial improvement for conditional generation in challenging settings. We also evaluate adaptive moments in the time-aware classifier guidance setting, where AdamCG improves upon CG by more than 19 percentage points in classification accuracy.

---

Published as a conference paper at ICLR 2026
AdamDP5
Relative Imp rovement
Naa
TFG New =2
20%
TFG New
0%
4x
8x
12x
16x
Super Resolution Factor
Gaussian Deblur Intensity
Figure 5: Relative improvement over DPS on ImageNet as task difficulty increases for super resolu- tion and Gaussian deblurring.
AdanDPS TFG Mae
Re Relative Imp rovementt
20%
0%
12
25
50
100
12
25
50
100
DDPM Sampling Steps
DDIM Sampling Steps
Figure 6: Relative improvement over DPS for super resolution 16x downsampling on ImageNet, across DDPM and DDIM sampling step budgets.
Task Difficulty Ablation. We examine how task difficulty affects the relative performance of AdamDPS and TFG compared DPS. Figure 5) shows the relative LPIPS improvement of AdamDPS and TFG over DPS on ImageNet as super resolution and deblurring tasks increase in difficulty. For super resolution, AdamDPS consistently outperforms TFG, maintaining a substan- tial improvement over DPS across all difficulty levels. TFG only improves upon DPS in the two easiest settings, and performs noticeably worse in the two hardest. We see similar trend for Gaus- sian deblurring. TFG only meaningfully improves upon DPS in the easiest setting, performing equivalently or worse than DPS as difficulty increases. AdamDPS demonstrates robust positive improvement that becomes more pronounced at higher blur intensities, TFG variants show de- clining performance and ultimately underperform DPS on the most challenging tasks. AdamDPS remains effective even conditioning information becomes increasingly limited.
Sampling Steps Ablation. We analyze the effect of reducing the sampling step budget on AdamDPS and TFG relative DPS for super resolution 16x downsampling ImageNet, see Figure For both DDPM and DDIM sampling, AdamDPS consistently improves over DPS across all step counts. TFG Nier fails outperform
high step counts, consistent with Figure 3 However, TFG becomes competitive at lower step counts, possibly because TFG performs I + Niser updates to the image per step, whereas DPS and AdamDPS only perform 1 update per step. Across both samplers, AdamDPS provides reliable gains over DPS regardless of step budget.
Wall Clock. Figure 7 shows wall clock time averaged over trials for ImageNet class-conditional sampling with 100 steps single H100 GPU, generating a batch of 8 256x256 images. AdamDPS adds negligible overhead compared DPS. Both AdamDPS and DPS outpace TFG, whose gradi- ent computations scale with N + Nier) through the guidance model and Nrecur through the denoising network.
Adam 31, B2 Ablation. Figure 7 shows an ablation of Adam hyperparameters B1 and B3 for AdamDPS across super resolution at 16x downsampling, Gaussian deblurring at blur intensity 12, and inpainting with 90% random mask. We compare the relative LPIPS improvement over DPS for three configurations: default AdamDPS, AdamDPS with 81 = O, and AdamDPS with 82 = 0. AdamDPS consistently outperforms both ablated variants across all tasks, demonstrating that both momentum and adaptive scaling are essential for optimal performance, with their relative impor- tance varying by task.

---

Published as a conference paper at ICLR 2026
Improvement
30%
AdanDPS
TFG Niter
Relative
AdanDPS
TFG Niter
20%
AdenDPS =
TFG Niter =
10%
AdamDPS
DPS
Inpainting
Super
Gaussian Deblur
0
10
20
30
48
Resolution
Wall Clock (s)
Figure Left: Ablation of Adam 31, 8g for super resolution at 16x downsampling, Gaussian deblurring at blur intensity 12, and inpainting at 90% random mask on the Cats dataset. Right: Wall clock comparison on a single HI00 GPU 100 step class-conditional sampling with standard classifier ImageNet for batch of 8 256x256 images.
Sampling Analysis. We visualize the sampling trajectories of DPS and AdamDPS for super resolu- tion 16x downsampling on ImageNet in Figure 8 We project the trajectories of interest: the first defined by the difference between the initial noise and the target image, and the second defined by the difference between the AdamDPS and DPS solutions. The contours indicate the Mean Squared Error (MSE) loss surface with respect to the target. Across samples, AdamDPS trajectories more directly approach the ground truth, while DPS trajectories may stray from the target, resulting in solutions less aligned with the condition.
dimensions
To better understand the performance gap between AdamDPS and DPS, we examine the cosine sim- ilarity between sequential guidance terms 9r and gt+1 throughout sampling for both methods. As shown in Figure 9] DPS guidance terms in adjacent steps frequently disagree in direction, with nega- tive cosine similarity for the majority of sampling. This suggests that DPS guidance is often fighting itself, pushing the sample in conflicting directions across consecutive steps. contrast, AdamDPS maintains consistently positive cosine similarity between adjacent guidance terms, indicating that its
Noise
AdarDPS
DPS
Initial
Target
Target
DPS Sample
AdamDPS Sample
DPS
AdamDPS Sample
DPS
AdanDPS Sample
Sample
Sample
Figure 8: Sampling trajectories for DPS and AdamDPS projected onto two dimensions for super resolution at 16x downsampling on ImageNet. The y-axis is defined by the difference between the initial noise and the target, and the x-axis by the difference between the AdamDPS and DPS solutions. Contours depict the MSE loss surface with respect to the target.
1
Cosine Similare lamitarity
AdamDPS
DPS
0
1000
800
600
400
200
Timestep
Figure 9: Cosine similarity between sequential guidance terms St and gt+1 throughout sampling for DPS and AdamDPS. Guidance terms collected from 16x super resolution task on ImageNet. Shading denotes the 25th to 75th percentile.
8

---

Published as a conference paper at ICLR 2026
Class-conditional (Time-Awar Classifier)
Super Resolution (4x)
10a
10a
AdanDPS
10
102
TFG
10-
10
16-5
AdanCG
Mass-Conditiona (Standard Classifier)
Super Resolution (16x)
10a
102
10
Loss
100
10
18
10-1
1800
880
600
400
200
0
1000
800
600
480
200
Timestep
Timestep
Figure 10: The guidance loss throughout sampling for class-conditional and super resolution tasks on ImageNet. Shading denotes the 25th to 75th percentile.
999
918
838
t= 757
676
0
Original Condition
TFG
DPS
AdamDPS
Figure 11: Qualitative example of sampling trajectory for super resolution at 16x downsampling on ImageNet. Displayed are the intermediate clean data predictions Znjz throughout sampling.
updates agree in direction for most of sampling. This coherence allows AdamDPS to make steady progress towards aligning with the condition rather than oscillating unproductively.
We track the loss L(fo(-), y) along the sampling trajectory to compare how each method mini- mizes the guidance objective, see Figure 10 For 4x super resolution on ImageNet, where condi- tioning information is relatively dense, TFG reduces loss more quickly than DPS and AdamDPS initially, though all methods achieve similar terminal loss values. Despite comparable final losses, AdamDPS still slightly outperforms TFG in reconstruction quality. For 16x super resolution on Ima- geNet, TFG struggles to reduce loss early sampling. only achieving rapid decrease near the end. Importantly. this lower terminal loss compared to DPS and AdamDPS does not translate to better reconstruction performance, as evident FFigure 3) When conditioning information severely lim- ited, over-opti the sparse signal can be detrimental rather than beneficial. In this case, TFG effectively overfits to the low-resolution image, producing the visual artifacts observed in Figure 2
For class-conditional sampling on ImageNet with standard classifier-a particularly challenging setting where baselines achieve near-random accuracy accuracy-neithee neither DPS nor TFG can meaningfully re- duce the guidance loss, shown in Figure 10 In contrast, AdamDPS achieves slow initial progress that accelerates toward the end of sampling, successfully reducing the guidance loss when all other methods could not, consistent with its accuracy improvement in Figure 4 For time-aware classifiers,

---

Published as a conference paper at ICLR 2026
both CG and AdamCG reduce loss throughout sampling, but AdamCG continues improving while CG plateaus, ultimately achieving substantially lower loss.
We visualize intermediate clean data predictions Zoje during the early stages of sampling for 16x super resolution on ImageNet in Figure II DPS and AdamDPS produce reasonable predictions of the target image from the earliest timesteps, with structure and color emerging quickly. In contrast, TFG's predictions vary wildly and bear little resemblance to either the target image or conditioning image. This is consistent with the guidance loss curves in Figure 10 where TFG fails to reduce loss early in sampling for 16x super resolution. These erratic early predictions further suggest that TFG's guidance is ineffective when conditioning information is sparse, leaving the model unable identify a coherent trajectory toward aligning with the condition. Moreover, TFG's final sample exhibits visual artifacts despite its aggressive loss reduction late in sampling, illustrating how overfitring to sparse conditioning information degrades reconstruction quality.
Additional quantitative and qualitative results are provided in Apppeddix C Cand Appendix
RELATED WORK
Beyond DPS and CG, numerous methods have been proposed to improve conditional gener- ation through more sophisticated likelihood score approximations and algorithmic refinements. MPGD (He et al. 2023) sidesteps backpropagation through the diffusion model by optimizing di- rectly in data space on the clean data estimate Tojz, while LGD (Song et al. 2023b) applies Monte Carlo smoothing to stabilize the DPS likelihood score approximation. Building upon these ap- proaches, compositional frameworks combine multiple guidance strategies. UGD (Bansal et al. 2023) unifies DPS and MPGD with recurrence, introducing separate hyperparameters to control the contribution of different components. Recurrence is a key technique employed by guidance methods such as FreeDoM Yuu et al.] (002)), which enables iterative refinement by revisiting timesteps (Mokady et al., 2023; Wang al. [2022] Lugmayr et al. 2022 Du et al. (023). TFG (Ye) et al. 2024) further extends this by combining DPS, MPGD, LGD, UGD and FrM into a uni- fied framework. TFG introduces hyperparameters for various algorithmic components, along with schedules that adjust some of these hyperparameters throughout sampling.
Additional methods exist which leverage specific structural assumptions about the guidance model to achieve improved performance. DDRM (Kawar et al., 2022) assumes linearity of the inverse problem for efficient posterior sampling through variational inference, while HIDM (Song et al.] 2023a) handles non-differentiable measurements by assuming the availability of pseudoinverse. TMPD (ooss et al., 2023) and FreeHunch (Rissanen et al., 2025) assume access to linear measure- ment operators estimate denoiser covariance, yielding improved likelihood score approximations.
In contrast to these approaches, which focus on improving the likelihood score approximation itself, we focus reducing the noise in guidance updates that arises from approximate likelihood scores and limited conditioning information. Our work is orthogonal to these plug-and-play methods; adaptive moment estimation can be applied to any of these methods stabilize guidance updates.
CONCLUSION
We demonstrate that adaptive moment estimation, which stabilizes noisy guidance during sampling, can substantially improve plug-and-play diffusion sampling. AdamDPS and AdamCG achieve state- of-the-art performance across diverse reconstruction and class-conditional generation tasks with minimal computational overhead. Adding adaptive moments is typically only a few-line modifi- cation an existing guidance implementation, and we show in(Appendix Cthat extends beyond DPS and CG, yielding improvements when applied to other plug-and-play guidance methods. Im- portantly, our evaluation across a range of task difficulties reveals that complex methods like TFG which outperform simpler baselines on easy tasks often degrade substantially as conditioning infor- mation becomes limited, ultimately underperforming DPS. More broadly, hlug-and-play guidance with adaptive moments opens promising direction for solving underdetermined inverse problems in scientific domains, where conditioning information is sparse and noisy. Noise mitigation offers a simple yet effective alternative to increasingly sophisticated likelihood score approximations.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We thank Chia-Hao Lee and David Muller for productive discussions which motivated this work. CB is supported by the National Science Foundation (NSF) through the NSF Research Traineeship (NRT) program under Grant No. 2345579. JL is supported by Google PhD Fellowship. This work is also supported by the NSF through Grant No. OAC-21 18310 and the Al Research Institutes program Award No. 2M3-3433348; the National Institute of Food and Agriculture (USDA/NIFA):; the Air Force Office of Scientific Research (AFOSR); New York- Pork-Preshyterian for the NYP-Come Cardiovascular AI Collaboration; and a Schmidt A12050 Senior Fellowship.
REFERENCES
Sebastian Ament, Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Un- expected improvements to expected improvement for bayesian optimization. Advances in Neural Information Processing Systems, 36:20577-20612, 2023.
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 843 443-852,000 2023.
Benjamin Boys, Mark Girolami, Jakiw Pidstrigach, Sebastian Reich, Alan Mosca, and Deniz Akyildiz. Tweedie moment projected diffusions for inverse problems. arXiv preprint arXiv:2310.06721, 2023.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, al. Video generation models world simulators. OpenAl Blog, 1(8):1, 2024.
Hyungjin Chung. Jeongsol Kim, Michael Thompson Mccann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. The Eleventh Interna- tional Conference on Learning Representations, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255 leee, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 11-8780-8794 2021.
Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and meme. International con- ference machine learning, pp. 8489-8510. PMLR, 2023.
Bradley Efron. Tweedie's 's formula and selection bias. Journal of the American Statistical Associa- tion, 106(4996::602-16614 2011.
Jeremy Elson, John (JD) Douceur, Jon Howell, and Jared Saul. Asirra: A captcha that exploits interest-aligned manual image categorization. In Proceedings of 14th ACM Conference on Com- puter and Communications Security (CCS). Association for Computing Machinery, Inc., October 2007.
Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei- Hsiang Liao, Yuki Mitsufuji. J Zico Kolter, Ruslan Salakhutdinov, and Stefano Ermon. Manifold preserving guided diffusion. arXiv preprint arXiv:2311.16422 2023.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pp. 6840-6851, 2020.
11

---

Published as a conference paper at ICLR 2026
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in neural information processing systems, 35:8633- 8646, 2022.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffu- sion for molecule generation in 3d. In International conference on machine learning, pp. 8867- 8887. PMLR. 2022.
Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Rethinking fid: Towards a better evaluation metric for image generation. In Pro- ceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition, pp. 9307- 9315, 2024.
Donald R Jones, Matthias Schonlau, and William J Welch. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13(4):455-49) 1998.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-265777 2022.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Advances neural information processing systems, 35:23593-233600 2022.
Diederik Kingma and Ruiqi Gao. Understanding diffusion objectives as elbo with simple data augmentation. Advances in Neural Information Processing Systems, 36:65484-65516, 2023.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Ad- vances in neural information processing systems, 34:21696-21707, 2021.
Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761 2020.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio generation with latent diffusion models. arXiv preprint arXiv:2301.12503, 2023.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11461-11471, 2022.
Ron Mokady, Amir Hertz, Kfrr Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6038-6047, 2023.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hicrarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.0611 06144, 2022.
Severi Rissanen, Markus Heinonen, and Arno Solin. Free hunch: Denoiser covariance estimation for diffusion models without extra costs. In The Thirteenth International Conference on Learning Representations, 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural informa- fion processing systems, 35:36479-36494, 2022a.
12

---

Published as a conference paper at ICLR 2026
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE transactions on pattrrn anal- ysis and machine intelligence, 45(4):4713-472 2022b.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2265, pmlr, 2015.
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guided diffusion models for inverse problems. 2023a.
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference Machine Learning, pp. 32483 22483332498. 32498. PMLR, 2023b.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, volume 32, pp. 11895-11907, 2019.
Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5 RMSProp. COURSERA: Neural Networks for Machine Learning, 2012.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint #xXiv:2212.00499. 2022.
Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E Eise- nach, Woody Ahern, Andrew J Borst, Robert J Ragotte, Lukas F Milles, et al. De novo design of protein structure and function with rfdiffusion. Nature, 620(7976):1089- -1000009900000011111111111100000 2023.
Haotian Ye, Haowei Lin, Jiaqi Han, Minkai Xu, Sheng Liu, Yitao Liang, Jianzhu Ma, James Zou, and Stefano Ermon. Tfg: Unified training-free free guidance for diffusion models. Advances in Neural Information Processing Systems, 37:22370-22411 2024.
Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training-fiee energy-guided conditional diffusion model. In Proceedings the IEEE/CVF International Con- ference on Computer Vision, pp. 231 74-23184, 2023.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018
13

---

Published as a conference paper at ICLR 2026
A ADDITIONAL EXPERIMENTAL DETAILS
We build on the TFG codebase using their implementations for CG, MPGD, UGD, and TFG. We slightly update the LGD and DPS implementations. For the TFG baseline itself we follow the authors' recommendation, tuning p. p. y, sweeping Niur 1, 2, 4. setting Nicour 1, and setting the schedules for p. μ4, y to be increasing, increasing, decreasing (Ye et al. 2024). All experiments use 100 step DDPM sampling unless stated otherwise. Gaussian deblurring blur intensity refers to the standard deviation of the Gaussian kernel used to blur the image. We split the datasets into validation and test splits and tune our hyperparameters on the validation split. Our validation and test splits contain 32 and 2048 images respectively. We use Bayesian Optimization to tune the hyperparameters of each method on our validation split. We perform 150 trials, the first 50 trials select candidates by Sobol sampling, the remaining 100 trials use Log Noisy Expected Improvement (Ament al.) 2023). Accuracy on class-conditional sampling tasks is computed with three held-out classifiers for each dataset downloaded from HuggingFace. For CIFARIO: ViT ViT Finetuned" and ConvNeX For ImageNet: ViT DINO v MobileViT v
B ADDITIONAL PSEUDOCODE
Algorithm Diffusion Posterior Sampling (DPS)
Require: Diffusion Model Guidance Model fa. Condition y. Guidance Strength Pla.... Ptg. Sampling Timesteps fn...., to
1: Tt, ~ (0,
2: for fn...... f do
3: It p C(fo ((fo(r))))))))
2, Sample(Foje: 21,
5: end for
6: return 1io
Algorithm Classifier Guidance (CG)
Require: Diffusion Model 0, Guidance Model fo. Condition y. Guidance Strength p, Sampling Timesteps t..... to T
I: St, (0,
2: for = bn.
do 3: St = V n. ((fo t), v)
4: 2t Sample(zaj + P9:07 Fz, 8)
5: end for
return Ito
Algorithm 5 Adaptive Moment Estimate
Require: Gradient g, First Moment Estimate m, Second Moment Estimate v, Adam Step k First Moment Exponential Decay Rate 31, Second Moment Exponential Decay Rate 82, = 10-5
I: k = h
2: u = $2u + (1 32)g
31
}g
4: A m/(1- 84)
5: 0 = v/(1 o5)
9 = m 6)
7: return g. m. U.
https: http:///ugniiiccccotttttttttaaer
14

---

Published as a conference paper at ICLR 2026
C ADDITIONAL QUANTITATIVE RESULTS
Table Reconstruction on ImageNet
Inpainting
Super
Resolution 4x
Super
Resolution
16x
Gaussian Deblur
Gaussian Deblur
12
Method
LPIPS
FID
IS
LPIPS
FID
IS
LPIPS
FID
IS
LPIPS
FID
I8
LPIPS
FID
18
P&GDM
0.30
66.83
36.85
0.48
63.73
26.34
0.48
49.41
30.11
0.48
65.04
24.69
0.50
51.97
27.76
RED-diff
0.31
75.86
36.60
0.20
30.42
102.09
0.57
100.30
9.66
0.27
40.13
76.44
0.52
209
90
5.85
DPS
132.32
0.19
128.03
0.23
27.66
109.29
40.25
45.07
LGD
0.18
37.37
89.52
0.24
31.97
105.63
0.32
38.10
49.85
0.31
37.02
74.39
0.39
46.11
36.46
142.47
UGD Mae
0.29
47.30
60.38
0.31
38.42
70.41
0.34
38.52
45.25
0.33
37.46
63.02
0.40
43.06
36,76
UGD Ma
0.20
33.18
95.94
0.18
28.31
123.13
0.32
40.05
49.68
0.26
32.76
98.00
0.38
44.48
40.1
Niw
29.51
106.57
TFG =1
0.20
41.49
81.77
0.14
25.37
132.58
0.32
42:27
46.69
0.20
27.21
113.49
0.36
47.34
36.11
TPG N
0.13
26.17
135.12
0.13
25.03
136.73
0.32
42.69
46.63
0.17
26.01
125.65
0.35
46.44
38.92
TRG
0.13
125.08
128.90
0.32
25.74
128.00
MdD
0.48
142.78
13.39
0.15
27.06
120.36
0.35
$6.51
32.74
0.20
29.96
114.47
0.37
51.43
35.29
AdamPIGDM
0.11
22.25
163.23
0.14
22.42
161.15
0.28
33.75
61.26
0.20
27.30
132.52
0.30
35.91
59.00
AdamDPS
0.12
23.42
160.63
0.12
180.91
0.27
65.69
23.45
144.23
38.35
49.11
Table 2: Reconstruction on ImageNet (Intermediate Difficulties)
Resalution 8x
Resolution
12x
Gaussian Dehlur
Gaussian Deblur 9
Method
LPIPS
FID
18
LPIPS
FID
IS
LPIPS
FID
IS
LPIPS
FID
18
DPS
0.23
29.02
88.64
0.26
32.63
71.02
0,26
31.39
71.41
0.30
36.65
54.54
TFG Neve m1
31.76
79.87
TPG Naurm
0.22
31.56
80.09
0.27
37.08
57.60
0.26
33.21
72.71
0.32
40.15
49.85
TFG
0.22
32.26
76.58
0.27
39.84
53.69
0.26
33.58
73.29
0.31
39.58
49.63
AdamDDP
24.36
85.87
90.08
Table 3: Reconstruction on Cats
Inpainting LPIPS
Super Resolution
Super Resolution
Deblur
Method
FID
LPIPS
FID
LPIPS
FID
L.PIS
FID
LPIPS
FID
PIGDM
0.32
80.77
0.40
60.26
0.44
$2.76
0.42
58,80
0.46
58.48
0.22
120.05
DPS
0.13
17.13
0.14
17.74
0.25
30.68
0.18
23.58
0.29
34.74
LGD
0.17
35.29
0.14
19.63
0.27
35.15
0.20
25.72
0.33
40.10
MPGD
0.42
76.60
0.09
15.70
0.28
44.16
0.14
20.88
0.32
49.50
LGD
0.32
64.22
0.25
38.53
0.29
33.60
0.26
35.70
0.35
43.10
Nav
26.97
LGD =4
0.21
37.68
0.11
18.56
0.27
38.07
0.17
23.91
0.32
47.14
TFG
0.15
18.24
0.09
14.81
0.27
39.75
0.15
20.78
0.32
50.49
Yuu
TFG
0.09
17.77
0.08
14.09
0.26
38.83
16.42
0.29
44.95
AdamMPGD
0.44
70.93
0.10
15.32
0.28
46.85
0.12
18.33
0.30
49.17
AdamPiGDM
17.79
AdamDPS
0.08
14.64
0.09
13.19
0.24
27.62
0.14
20.43
0.27
30.22
Table 4: Reconstruction on Cats (Intermediate Difficulties)
Super Resolution 8x
Super Resolution 12x
Gaussian Deblur 6
Gaussian Dehlur 9
Method
LPIPS
FID
LPIPS
LPIPS
FID
LPIPS
DPS
25.36
0.22
TFG Nu
0.17
23.70
0.22
31.46
0.24
31.36
0.27
38.37
TFG Ni
0.16
22.10
0.22
29.77
0.21
27.40
0.27
36.95
TEC
DPS
0.16
19.45
0.20
22.64
0.20
24.32
0.23
26.48
15

---

Published as a conference paper at ICLR 2026
Table 5: Sampling Step Ablation for Super Resolution at 16x Downsampling ImageNet
DDPM
DDIM
LPIPS
FID
LPIPS
FID
100
25
TIG T
0.32
0.34
0.39
0.50
42.92
44.55
52.35
118.59
0.35
0.35
0.35
0.41
58.38
59.20
53.84
60.34
AdamDPS
0.27
0.30
0.35
0.46
30.16
35.63
49.05
98.71
0.29
0.30
0.35
0.44
39.60
37.18
46.35
77.33
Table 6: Sampling Step Ablation for Super Resolution 16x Downsampling on Cats
DDPM
DDIM
LPIPS
FID
LPIPS
FTD
Method
100
25
12
TFG TEG
0.26
0.29
0.34
0.40
38.83
39.72
44.83
70.17
0.35
0.34
0.33
0.34
55.81
51.70
49.85
47,33
AdamDPS
0.24
0.26
0.29
0.38
27.62
30.32
34.15
72.19
0.27
0.27
0.29
0.37
27.53
27.20
32.51
61.72
Table 7: Class-Conditionss (Standard Classifier) CIFARIO
Guidance Classifier Accuracy (1%
Held out Classifier
Accuracy
Method
VIT
VIT Finetuned ConvNeXi
RED-diff
10.7
9.9
10.2
10.7
203.8
DPS
40.3
42.8
32.3
LGD
21.8
19.6
18.1
21.8
31.0
MPGD
26.7
24.9
24.3
26.7
34.1
UGD Nar
9.3
9.7
10.2
25.4
LGD
24.6
22.9
23.1
24.6
32.4
UGD 44
30.3
31.9
37.2
TFG Nu
TFG Nao
17.0
15.4
14.7
17.0
25.1
TFG
19.4
17.1
16.7
19.4
26.1
AdamMPGD
29.9
AdamDPSS
52.6
53.2
$1.2
52.6
58.0
Table 8: Class-Conditional (Standard Classifier) on ImageNet
Guidance Classifier Tsp-10 Accuracy
Held-our Classifier
Accurayy (%)
Top-10 Accuracy (%)
FID
Method
DINO
v2
RED-diff
12.7
23.0
3.2
L.4
272.7
DPS
0.7
0.8
1.2
1.0
43.0
MPGD
0.9
0.8
1.2
1.4
43.6
UGDN
0.7
42.3
UGD
1.4
UGD Ng
1.7
1,0
0.8
0.9
43.6
TFG
1.0
0.9
0.9
42.7
TFU =2
TPG Ng
1.4
0.8
0.9
1.0
43.1
AdamMPGD
1.4
1.6
1.8
45.8
AdamDPS
10.5
833
52.8
Table 9:
(Time-Aware Classifier) on ImageNet
Guidance Classitier Accuracy (8)
Held-out Classitier Accuracy (%)
FID
Mathod
VIT
DINO v2 MobileViT v2
61.0
63.4
58.3
$1.0
30.8
AdamCG
82.5
84.2
77.8
68.1
29.6
16

---

Published as a conference paper at ICLR 2026
D ADDITIONAL QUALITATIVE RESULTS
Super Resolution (16x) on ImageNet
Condition
Ground Truth
AdamDPS
DPS
TFG
Figure 12: Additional qualitative results for super resolution at 16x downsampling on ImageNet.
17

---

Published as a conference paper at ICLR 2026
Gaussian Deblur (12) on ImageNet
Condition
Ground Truth
AdamDPS
DPS
TFG
Figure 13: Additional qualitative results for Gaussian deblurring blur intensity 12 ImageNet.
18

---

Published as a conference paper at ICLR 2026
Super Resolution (12x) on Cats
Condition
Ground Truth
AdamDPS
DPS
TFG
awodd Archa
Figure 14; Additional qualitative results for super resolution 12x downsampling on Cats.
19

---

Published as a conference paper at ICLR 2026
Gaussian Deblur (9) on Cats
Condition
Ground Truth
AdamDPS
DPS
TFG
Figure 15: Additional qualitative results for Gaussian deblurring blur intensity 9 Cats.
20

---

Published as a conference paper at ICLR 2026
(Standard Classifier) on CIFAR10
Condition
AdamDPS
DPS
TFG
Dog
Ship
Automobile le
Deer
Truck
Ship
Figure 16: Additional qualitative results for class-conditional sampling with standard classifier on CIFAR10.
21

---

Published as a conference paper at ICLR 2026
Class - Conditional (Standard Classifier) on ImageNet
Condition
AdamDPS
DPS
TFG
Pirate
Hamster
9
Refrigerator
Tape Player
Space Heater
JT1
Gazelle
Figure 17: Additional qualitative results for class-conditional sampling with a standard classifier on ImageNet.
22

---

Published as a conference paper at ICLR 2026
Class - Conditional TTime-Award Classifier) on ImageNet
Condition
AdamCG
CG
Espresso
Black Stork
Soccer Ball
African Grey
King Penguin
Cowboy Boot
Figure 18: Additional qualitative results for fass-conditional sampling with a time-aware classifier on ImageNet.
23


Reviews:
[{"review_id": "PbRitf5JtJ", "reviewer": "Reviewer_br5n", "summary": "This paper introduces AdamDPS, a new plug-and-play method for diffusion sampling. By noting that the guided sampling update is gradient ascent using the score of the likelihood distribution, the authors introduce Adam-style moment exponential moving averages to stabilize the overall update. The resulting method is tested on a toy GMM example, demonstrating improvements in sampling quality over the baseline diffusion posterior sampling (DPS). Experiments are then extended to reconstruction tasks on image datasets, where AdamDPS demonstrates the best combination of both FID and LPIPs. Ablations are performed on task-difficulty, Adam hyperparams, and wall-clock time.", "questions": "**(Q1)**: Is there a missing increment to $k$ in algorithms 1 and 2?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "This paper is complete and well-presented. Overall, the method is effective and experiments are thorough. I would recommend it for acceptance.\n\n**(S1)**: Simple and effective. The core idea of introducing Adam-style moment stabilization is simple and effective for diffusion guided plug-and-play sampling. The paper clearly motivates the problem with previous approaches, proposes a sound improvement, and demonstrates the improvement via experimental results.\n\n**(S2)**: Clear improvement over prior work on reconstruction tasks. Results in Fig 3 clearly demonstrate superior reconstruction quality and sample quality of AdamDPS. \n\n**(S3)**: Comprehensive ablations and analysis. The ablations on task difficulty is valuable and demonstrates the core motivation for the method-- as the task gets noisier, the stabilization introduced by Adam-style moments results in better performance. Other ablations on number of sampling steps and moment coefficient values are useful for any future users of this method. \n\n**(S4)**: Computational efficiency. The ablation on wall-clock time confirms that this method induces little to no overhead over baselines, which is valuable.", "weaknesses": "**(W1)**: Slightly mixed results on class-conditional sampling. The FID on ImageNet and CIFAR-10 seems worse than baselines, even as the classification accuracy is much better. \n\n**(W2)**: Slight regressions over baselines on easy tasks. In Fig 6, for easier super-resolution or deblurring tasks, AdamDPS is slightly worse than TFG. This again slightly clouds the otherwise clear narrative of the paper. No explanation for this is given.\n\n**(W3)**: Details on the method are missing. A clearer explanation of the models used, experimental setup, metrics would be very valuable for clarity. For example, while an ablation on $\\beta_1$ and $\\beta_2$ were performed, it's not clear what the optimal values are for each task.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DaKJahzs9z", "reviewer": "Reviewer_sSbi", "summary": "This paper studies the likelihood approximation noise in training-free plug-and-play diffusion models such as DPS. These models estimate the posterior score using $\\nabla_xt log⁡p(xt∣y)=\\nabla_xt log⁡p(y∣xt) + \\nabla_xt log⁡p(xt)$, where the likelihood term $\\nabla_{x_t}\\log p(y|x_t)$ is approximated by a differentiable surrogate $\\nabla_{x_t}L(f_\\phi(\\hat x_{0|t}),y)$. At high noise levels (early and mid diffusion steps), this estimate becomes highly unstable, leading to biased and noisy sampling.\n\nThe paper proposes maintaining exponential moving averages (EMAs) of the first and second moments of the likelihood gradient, similar to Adam. By tracking running averages of the gradient and its squared values across timesteps and adaptively rescaling updates, the approach stabilizes the guidance direction and reduces variance. Two variants are presented: AdamDPS for diffusion posterior sampling and AdamCG for classifier guidance.\n\nExperiments demonstrate consistent improvements: on inpainting, deblurring, and $16\\times$ super-resolution tasks (ImageNet and Cat datasets), AdamDPS achieves 1–2 dB PSNR gains and lower LPIPS/FID than DPS, UGD, and LGD. On CIFAR-10 and ImageNet 64×64 class-conditional generation, AdamCG improves top-1 and top-10 accuracy without significant degradation in FID. Synthetic 2-D Gaussian mixture experiments confirm that adaptive moments smooth the gradient variance peak occurring at intermediate diffusion steps.", "questions": "How does AdamDPS fundamentally differ from optimization-based samplers such as RED-Diff that already employ Adam updates?\n\nDoes maintaining EMAs across timesteps bias the effective sampling trajectory?\n\nHow would AdamDPS perform against RED-Diff or TMPD under the same compute and step budgets?\n\nCould combining adaptive moments with Monte-Carlo smoothing (e.g., LGD) provide further gains?\n\nHow sensitive is the performance to the number of diffusion steps or when applied to ODE-based samplers such as DDIM?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Addresses a key limitation—noisy and biased likelihood gradients—with a simple, general, and low-cost modification. \n\nEmpirically robust across datasets, with clear ablations on noise scales and $(\\beta_1, \\beta_2)$.\n\nMinimal implementation effort with measurable gains in quality and stability.", "weaknesses": "Limited to gradient-based plug-and-play samplers; not applicable to variational or optimization-based methods such as RED-Diff.\n\nNo theoretical analysis showing that adaptive moments preserve unbiasedness or convergence to the correct posterior $p(x_0)\\exp[-r(x_0,y)]$.\n\nMissing comparisons with RED-Diff, MPGD, and TMPD, which already incorporate momentum or adaptive scaling.\n\nNo examination of how the moving averages interact with the diffusion dynamics or affect the stationary distribution.\n\n\nRED-Diff — M. Mardani, J. Song, J. Kautz, A. Vahdat. A Variational Perspective on Solving Inverse Problems with Diffusion Models, arXiv:2305.04391 (2023).\n\nMPGD — Y. He, N. Murata, C.-H. Lai, Y. Takida, T. Uesaka, D. Kim, W.-H. Liao, Y. Mitsufuji, J. Z. Kolter, R. Salakhutdinov, S. Ermon. Manifold Preserving Guided Diffusion, ICLR 2024.\n\nTMPD — B. Boys, et al. Tweedie Moment Projected Diffusions for Inverse Problems, arXiv:2310.06721 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Qi1v0y7DqN", "reviewer": "Reviewer_WUrY", "summary": "The paper investigates the classifier guidance and finds out that classifier guidance approximates the likelihood score which often includes a lot of noise. The paper proposes to regularize the guidance sampling process with ADAM, a popular method in neural network optimization.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well written\n2. the method is easy to understand", "weaknesses": "1. The novelty might be the concern. The Adam is not novel and the guidance is also not novel. The use of Adam during sampling process is also not new (this paper has similar idea https://www.ijcai.org/proceedings/2024/0157.pdf). The only difference is that Adam is applied to guidance term instead of denoising term as in the paper. \n2. The paper does not provide a new fundamental observations or scientific hypothesis.\n3. In terms of applications, the proposed method currently could not extend to classifier-free guidance which is a very popular guidance right now. \n4. The quantitative results are not provided, different metrics are not considered e.g FID, IS, Rec, Prec, CLIP, GenEval, T2I Bench.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xsqQ2Lr8MW", "reviewer": "Reviewer_FmMn", "summary": "This paper proposes a very simple plug-in modification to plug-and-play diffusion guidance: keep exponential moving averages of the likelihood-gradient during sampling (Adam-style first/second moments) and use the stabilized gradient to guide updates. The authors instantiate this on DPS (\"AdamDPS\") and on classifier guidance (\"AdamCG\"), provide a 2-D GMM toy study, and report empirical gains on various inverse problems.", "questions": "- Can you replicate gains on other plug-and-play frameworks (e.g., DDRM, PiGDM, MPGD-style data-space updates) and other backbones/samplers to support the \"surprisingly effective\" claim beyond DPS/CG?\n\n- Beyond demonstrating that AdamDPS smooths noisy guidance, is there substantive novelty beyond applying an optimizer wrapper to DPS/CG?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "- The change is minimal (Adam-style moments around the guidance gradient) and easy to graft onto existing DPS/CG code paths.\n\n- The toy GMM illustrates how DPS becomes unstable under noisy likelihood gradients whereas AdamDPS stabilizes trajectories; several restoration benchmarks also show improvements in LPIPS/FID vs. DPS and some training-free baselines.", "weaknesses": "- Technical novelty is minimal. The paper just adds Adam-style gradient update to DPS/CG at inference time. There is no new objective, solver, or estimator; the contribution is only an optimizer wrapper around an existing guidance gradient (Algorithms 1–2). This is a straightforward, well-known idea from stochastic optimization brought over without new theory specific to diffusion guidance beyond intuition. This falls short of ICLR's bar for conceptual advance. The empirical results are useful but do not compensate for the lack of technical depth or new insight into diffusion guidance.\n\n- Potential overfitting in Fig. 5. The combination of highest accuracy with worst FID suggests the guidance may be overfitting to the evaluation classifier rather than improving image quality. Classification accuracy alone does not imply good samples if the images are classifier-friendly but perceptually poor. Please use a different (held-out) classifier for evaluation than the one used during guidance/training, and report cross-model accuracy (and, ideally, human or ImageReward/VQA scores) to confirm the gains are not due to classifier overfitting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)