FAST‑DIPS: Adjoint‑Free Analytic Steps and Hard‑Constrained Likelihood Correction for Diffusion‑Prior Inverse Problems
Authors: Minwoo Kim
Seunghyeok Shin
Hongki Lim
Venue: ICLR 2026 Poster (2026)
Keywords: Inverse problem
Image reconstruction
Diffusion models
Primary area: generative models

Abstract: Training-free diffusion priors enable inverse-problem solvers without retraining, but for nonlinear forward operators data consistency often relies on repeated derivatives or inner optimization/MCMC loops with conservative step sizes, incurring many iterations and denoiser/score evaluations. We propose a training-free solver that replaces these inner loops with a hard measurement-space feasibility constraint (closed-form projection) and an analytic, model-optimal step size, enabling a small, fixed compute budget per noise level. Anchored at the denoiser prediction, the correction is approximated via an adjoint-free, ADMM-style splitting with projection and a few steepest-descent updates, using one VJP and either one JVP or a forward-difference probe, followed by backtracking and decoupled re-annealing. We prove local model optimality and descent under backtracking for the step-size rule, and derive an explicit KL bound for mode-substitution re-annealing under a local Gaussian conditional surrogate. We also develop a latent variant and a one-parameter pixel$\rightarrow$latent hybrid schedule. Experiments achieve competitive PSNR/SSIM/LPIPS with up to 19.5$\times$ speedup, without hand-coded adjoints or inner MCMC. Code and data: [here](https://github.com/ququlza/FAST-DIPS)

Published as a conference paper at ICLR 2026
FAST-DIPS: ADJOINT-FREE ANALYTIC STEPS AND HARD-CONSTRAINEE LIKELIHOOD CORRECTION FOR DIFFUSION- PRIOR INVERSE PROBLEMS
Minwoo Kim', Seunghyeok Shin', Hongki Lim
Department of Electrical and Computer Engineering, Inha University Incheon, 22212. South Korea
aaqqulza1520 ssh8 642 2 einea........mm...... edu,
hkl hklimeinki ac kr
Maasarenent
BAPS
SITCIM
OUISS (Praa))
Roforence
STTOOM
DURS(Foer)
SSate SL791
SSIN:8.851
Tine
3284
Tine:261s
10:355
(a) Gaussian deblurring
(h) Phase Retrieval
Mescorerenn
DAPs
BeSample
OURS (Latent)
Befievnoe
Mestarmment
Lakest DAPS
BeSempte
OUBS (Ladear)
Relemane
SSIM:9.822
SSis 0.322
LMP5:0512
LPIP5:0.239
0
Time9148
27850
T5:e:45344s
Time: 25e
Times 200.
(c) Inpaint (random)
(d) High Dynamic Range
Figure 1: FFHQ results four inverse problems: (a) Gaussian deblurring, (b) phase retrieval, (c)
random inpainting, (d) HDR. Each panel shows the measurement, basclines, our FAST-DIPS output, and the reference. SSIM/LPIPS and average per-image run-time (s) are overlaid; FAST-DIPS attains comparable or higher quality with markedly lower run-time.
ABSTRACT
Training-free diffusion priors enable inverse-problem solvers without retraining, but for nonlinear forward operators data consistency often relies on repeated derivatives or inner Opiiiization/MMMC loops with conservative step sizes, incur- ring many iterations and denoiser/score evaluations. We propose a training-fre free solver that replaces these inner loops with a hard measurement-spaaee feasibility constraint (closed-form projection) and an analytic, model-optimal step size, en- abling small, fixed compute budget per noise level. Anchored at the denoiser prediction, the correction is approximated via an adjoint-free, ADMM-style split- ting with projection and few steepest-descent updates, using one VJP and either one JVP or a forward-difference probe, followed by backtracking and decoupled re-annealing. We prove local model optimality and descent under backtracking for the step-size rule, and derive an explicit KL bound for mode-substitution re- annealing under local Gaussian conditional surrogate. We also develop a latent variant and a one one- parameter pixel->latent hybrid schedule. Experiments achieve competitive PSNR/SSIM/LPPN adjoints or inner MCMC. Code and data: ?
with up to 19.5x speedup, without hand-coded
'These authors contributed equally to this work "ooresponniig author

---

Published as a conference paper at ICLR 2026
1 INTRODUCTION
Inverse problems seek to recover an unknown signal x from partial and noisy measurements, y A(x) + n. Such problems are ubiquitous in science and engineering, yet they are often ill-posed: distinct x can produce similar y due to the structure of the operator A and measurement noise n. The Bayesian viewpoint constrains the solution via a prior and asks to sample from p(x y) ply
x) pxx).
Diffusion models have emerged as a powerful class of learned priors for modeling complex data dis- tributions, including natural images (Ho et al. (2020); Song & Ermon (2020); Song et al. (2021 Dhariwal & Nichol (2021); Karras et al. (2022); Song et al. (2023); Lu & Song (2025)). Through reverse-time dynamics, they progressively transform simple noise into samples from the target dis- tribution. This generative mechanism offers a natural framework for inverse problems, where the reverse-time SDE is guided by measurements to draw posterior samples.
Diffusion-based inverse problem solvers span both task-specific conditional models and training- free posterior samplers. Task-specific conditional diffusion/bridge models are trained for particular restoration or image-to-image tasks (Saharia et (2022); Liu et al. (2023)). In contrast, many training-free solvers start from an unconditional pretrained prior and impose data consistency at sampling time, including inference-time inpainting (Lugmayr et al. (2022)), linear-operator frame- works (Kawar et al. (2022); Wang et al. (2023)), and decoupped/posteriorr et al. (2023a;b); Dou & Song (2024); Zhang et al. (2025)). Other lines formulate plug-and-play op- timization with diffusion denoisers (Zhu et al. (2023); Rout et al. (2024); Wu et al. (2024); Xu & Chi (2024); Mardani et al. (2024); Wang et al. (2024); Zheng et al. (2025)), Monte-Care guidance (Car- doso et al. (2024)), aim for faster sampling via preconditioning, parallelization, fast/few-stey samplers (Garber& & Tirer (2024); Cao et al. (2024); Liu et al. (2024); Chung et al. (2024)).
updates (Chung
A central practical question is how data consistency is enforced. Many training-free designs inject measurements through an operator- aware data-consistency (likelihood) update (Kawar et al. (2022); Wang et al. (2023); Rout et al. (2023); Liu et al. (2024); Pandey et al. (2024); Cao al. (2024); Garber & Tirer (2024); Dou & Song (2024); Cardoso et al. (2024); Chung et al. (2024)). For some linear degradations, this update admits efficient closed-form structure by exploiting explicit adjoints and/or pseudo- pseudo-inverses (Kawar et al. (2022); Wang et al. (2023)). When such closed-form operator primitives are unavailable (e.g.. complex, ill-conditioned, or simulator- -ased forward models im- plemented in autodiff frameworks but lacking closed-form aejoints/pseudo-inveeess) the likelihood step often implemented via inner iterative solvers (e.g., multiple gradient steps on a data-fidelig objective) or MCMC/Langevin subloops with conservative step sizes, which increases wall-clock cost due to many inner iterations and repeated score/denoiser calls (Zhu et al. (2023); Wu et al. (2024); Xu & Chi (2024); Mardani et al. (2024); Wang et al. (2024); Zhang et al. (2025)).
complementary design axis is latent vs. pixel execution. Latent diffusion models reduce dimen- sionality and sampling cost, and many recent posterior samplers therefore operate in latent space (Song et al. (2024); Rout et al. (2024); Zhang et al. (2025)). with an encoder & and decoder D. When fidelity is defined in pixel space, latent-space likelihood updates require backpropagating through the decoder, e.g., (=)lA(D((z) yll which can be a throughput bottleneck. A pixel-space correction instead decodes z + x, applies data-fidelity correction in image space, then re-encodes x= this avoids decoder backprop during correction, but can be sensitive to how Jacobian-vector products (JVPs) are computed for highly nonlinear A.
Taken together, these considerations motivate vriniin-frree solvers that (i) impose data consistency without operator-specine primitives (e.g.. hand-coded adjoints. pseudo-inverses, or SVDs), (ii) keep the per-noise-level correction lightweight by using a small, fixed compute budget (few inner steps and few operator/gradient calls) while delivering competitive reconstruction quality, and (ii) bal- ance pixel- and latent-space computation to trade early-time throughput for late-time fidelity. We propose FAST-DIPS (Fast And STable Diffusion-prioa Inverse Problem Solver), which centers each noise-level update at the denoiser prediction and applies a measurement-spaac feasibility correction with an interpretable residual budget under additive white Gaussian noise (AWGN), followed by decoupled re-annealing. Our implementation is adjoint-free and uses only a small fixed number of inexpensive autodiff primitives per level, avoiding hand-coded adjoints and inner MCMC while remaining applicable to nonlinear forward operators. We also develop latent counterpart and
2

---

Published as a conference paper at ICLR 2026
(a) Annealing process Diffusion model
X
y
= X == A00
X
Xoje
Acquiza-Free Libelhuud Corraction via optimal stop ADMM
cofr
optical
Denoise
(s. Ott./,0000)
IBall ppll"+pUa.00 00gll
Initializ::
x Keps A(toz): u
x x-update via Analytic stop
Yt
F0O)
|x - xtrl $11000-b1
Zy:
Shore Bedkredk
A00
W.
[w yl
S
w y
(0) Adaptive analytical updatte in ADMM
(b) ADMM based MAP estimation
Figure 2: FAST-DIPS method sketch. At each noise level I, we (I) compute denoiser anchor Xoje (2) perform a hard-constrained measurement-space correction under an &a feasibility ball a few- step ADMM-style splitting (closed-form projection + a few descent steps with analytic ax using one VJP and either one JVP or a
probe), and (3) re-anneal to obtain Xt-1-
one-switch pixel-latent hybrid schedule that corrects in pixels early (cheap) and in latents late (manifold-faithful).
FAST-DIPS differs from PnP-ADMM (Chan et al. (2016); Venkatakrishnan et al. (2013)) in that the denoiser is not a (surrogate) proximal operator; it only provides an anchor, while an ADMM-style splitting enforces an explicit hard measurement-space constraint via projection. Unlike quadratic penalties Al)
y12 that require tuning and can noise-sensitive, we use an indicator confidence-set likelihood with an interpretable residual budget. Unlike coupled DPS-style guidance (Chung et al. (2023a)), we keep traversal decoupled (Zhang et al. (2025)) and apply diffusion- kernel transport after each correction via sudetsubbtitttion re-annealing. Our pixel->latent hybrid reduces decoder-Jacobian cost early while preserving manifold fidelity late. Finally, while we use autodiff for VJPs (and one JVP forward-difference surrogate) (Baydin et al. (2018)), we avoid hand- crafted adjoints and closed-fone Dncobian//sseudo-nneer inverses of A.
Our contributions can be summarized as follows:
Adjoint-free feasibility correction. At each diffusion step, we enforce measurement- space feasibility around the denoiser prediction with hard residual budget and schedule- aware trust region, using an adjoint-free splitting solver with a closed-form projection and small fixed inner compute budget, avoiding hand-crafted adjoints and inner MCMC.
Theory for step size and re-annealing. We prove the proposed step size is locally model- optimal and, with backtracking, yields descent for the current subproblem (Proposition I), and provide an explicit KL bound for the per-step Gaussian-injedt distribution in- duced by mode-substitution re-annealing under local Gaussian conditional surrogate (Proposition 6).
Latent and hybrid execution with speed gains. We derive latent counterpart via Ao D and a one-switch pixel->latent hybrid that balances early-time efficiency and late-time fidelity; across eight linear and nonlinear tasks, achieve competitive quality and up to 19.5x speedup on FFHQ pixel-space tasks with robust default hyperparameters.
Orthogonal to contributions, fast/few-step samplers and parallel sampling schemes reduce the diffusion sampling cost (Zhao et al. (2024); Cao et al. (2024); Liu et al. (2024); Chung et al. (2024)). FAST-DIPS complements such advances by minimizing inner correction cost and adjoint engineering while preserving an explicit measurement-spaae feasibility set via projection in the split variable, these techniques composable with approach.
3

---

Published as a conference paper at ICLR 2026
2 METHOD
2.1 HIGH-LEVEL OVERVIEW AND DESIGN CHOICES
We briefly summarize the goals and main design choices of FAST-DIPS before introducing the de- tailed derivations.
Setting and practical issues. We use pretrained diffusion model prior for inverse problems (((x))++)) + nn, where A may be nonlinear. Existing training-free solvers often enforce data consis- tency via many gradient steps inner Langevin/MCMMC chains each noise level, and control their influence through soft data-fidelity weight. This can be computationally heavy (many evaluations of A and its gradient) and sensitive step sizes and noise miscalibration.
= A(x))
Per-level update in FAST-DIPS. each diffusion level t, FAST-DIPS performs single correct- then-noise update with three ingredients:
Local hand-constrained MAP. The denoiser output Xoje is treated as the center of a local Gaussian prior, and data consistency is enforced via a simple mcasurement-space constraint [|A(x) yll s s. This defines constrained proximal problem whose solution x is the likelihood correction at level I.
Adjoint-free ADMM with analytic step size. We solve the constrained problem with a small, fixed number of deterministic ADMM-style iterations. The split measurement variable is up- dated by a closed-form projection onto the constraint set, while the image variable is updated by few steepest-descent steps whose step size is computed per step by minimizing local 1D quadratic model (and refined by backtracking). This keeps the number of evaluations of A and its derivatives small. No inner MCMC chain and hand-coded adjoint operator are required: VJPs/JVPs are obtained from autodiff, and when JVPs are unavailable we use a single finite- difference probe (Remark 4). This leads to substantially fewer total function calls than methods based on long gradient/MCMC inner loops.
3. Re-annealing, After the correction, re-anneal by adding Gaussian noise with the next diffu- sion variance, decoupling the measurement-await update from the diffusion noise.
Theoretical support. The inner x-update uses an analytic step size that minimizes the local quadratic model, and Armijo backtracking ensures accepted step decreases the current subproblem objective under mild regularity (Proposition 1 proof in Appendix A.3). For complete- ness, Appendix A.3 also includes (i) fixed-point KKT characterization for the exact scaled- ADMM mapping (Proposition 5), and (ii) a KL bound for the Gaussian-injectee distribution in- duced by mode-substitution re-annealing under local Gaussian conditional approximation (Propo- sition 6). With nonlinear A the per-level problem is generally nonconvex, and we do not claim global convergence.
Practical per-step recipe. Figure 2 and Algorithms 1-2 give the full procedure. Let Xden denote the pretrained denoiser. One reverse step of FAST-DIPS at level f proceeds as: (i) Anchor and ini- tialization: compute the denoiser anchor Xojt X (x,, στ) and set (0) Xooo, ((") ())
(()) <-0. O. (ii) Few-step ADMM-style correction: for A O...., K K-1, 1, update by applying few successive steepest-descent steps on F in Equation 17 using the gradient in Equation 18, with the step size computed a* in Equation 21 (via one VJP and one JVP. or the finite-difference probe in Equation 22/Remark 4) and then refined by backtracking. Then update (k++)) by the projection in Equation 14 Equation 16 and (k+1) by Equation 15. After K iterations, set x <- x (K), (iii) Re-anneal: sample x corr + 0:-16 with & ~ I)..
2.2 PROBLEM SETUP
Let X₂ EBCCHH denote the clean image stacked vector and
y A(xo) + n,
n~ N~,, 821)

---

Published as a conference paper at ICLR 2026
where A RCWW - R is (possibly nonlinear) forward operator. Throughout the paper
assume additive white Gaussian noise (AWGN) with variance 3 and use the standard Euclidean norm in measurement space.
2.3 PROBABILISTIC MOTIVATION AND THE PER-LEVEL OBJECTIVE
The reverse process of the diffusion model, conditioned on y, is described by the reverse-time SDE (Song et al. (2021b)):
dx: (t) (xgp(xx)y: dt + (t)o(t) dw,-
We do not use the SDE form directly; instead, we derive a fast per-level update from the conditional factorization in Equation At each diffusion level t we maintain a state x, and wish to trans- form the mime-marginal p(x; yy) into good approximation to p(x:-1 y) by performing local, measurement-awe -wwrre likelihood correction around the denoiser's prediction. The derivation proceeds from the conditional factorization
p(xo x) p(xo
xx) ply Xx));
(3)
and modeling choices: local Laplace surrogate for p(xo xx) and a set-valued likelihood in measurement space.
Local prior surrogate around the denoiser. Write
Xpjt := Xden (Xe: (t),
and approximate the intractable p(Xp x²) by a Gaussian centered at Xpiz=
p(xo x2)
(Xg
x2)
exp
|xx Xuje xujell?).
(5)
27
where "/ > plays the role of a local prior variance. Inspired by schedule-tied noise-annealing heuristics in decoupled inverse solvers (Zhang et al. (2025)), we set 7 = of so that the proximal trust region naturally tightens with annealing.
Conservative likelihood via a measurement-space feasibility ball. Under AWGN, the Gaussian likelihood
ply 8) & 3 2,52
1 ||A(xx)) (1bxk))-11 -112)
(6)
We replace p(y Xo. 3) by a set-valued surrogate likelihood ( xo) to improve robustness
to noise miscalibration. If B is known, then for any confidence level 6, (1 5) confidence region for the residual A(x,) - y under Equation 6 is the Euclidean ball {v ||v yll S c}
with E 3 3/ (Casella & Berger (1990)); using this region yields an indicator likelihood. In our implementation, we treat as user-chosen lata-consistency tolerance and do not calibrate via a specific choice of 5. If B is unknown, profiling it out gives p(y Xu, (xx)) xc m log (A(x))-y/ (Casella & Berger (1990)), which is monotone in the residual norm and motivates enforcing [|A(xx)) - yll < e for a chosen budget & > 0 (Engl & Hanke (1996)). Both viewpoints lead to
(y xo) x 1(x))(x)) - yl S $}
which replaces p(y Xo, B) in Equation and yields the hard constraint in Equation 9.
Per-level surrogate conditional and MAP. Combining Equation 5 and Equation 7 with Equa- tion 3 yields
pe (xp
xt,y) exp - 2%% ||x - xojel²) ([llA(x)) - y se).
We take the mode of Equation 8 as the likelihood correction at level 6, which solves
1
cOFF
E arg min XERCAW
||x
x
[A(()) y S
27+
Problem Equation is hard- constrannd proximal objective: the first term is schedule-awer trust region around the denoiser estimate, while the constraint enforces measurement feasibility within a user-specified residual budget in measurement space.
5

---

Published as a conference paper at ICLR 2026
2.4 DECOUPLED RE- ANNEALING AND CONNECTION TO TIME MARGINALS
Let Kt-st-1 (Xt-1 Xo) (xt-; Xo, of 1!) denote the diffusion kemnl that transports the clean image the next diffusion state, The exact time-marginal recursion (Zhang al. (2025)) is
p(x-1 1y) 1/
Ki 1 Xo) p(xo y) dxo p(x: yy) dx,.
(10)
Thus, transforming p(x; yy) to p(xx-1 yy) amounts to obtaining representative Xo ~ pxx X:: and injecting Gaussian noise of variance ai-1. We approximate p(xo x), by pt in Equation and substitute its mode, yielding
X
ooFr Oft
+ O
{~N(0,1).
(11)
Appendix A.3 provides an explicit KL bound for the mode-substitution re-annealing step (for fixed x.), comparing it to the per-step Gaussian-injected distribution obtained by sampling from local Gaussian surrogate for p(xo x) (Proposition 6).
2.5 PIXEL-SPACE ADMM SOLVER WITH ADIOINT ODJOINT-FREE UPDATES
We solve Equation 9 via variable splitting (Combettes & Pesquet (201 Boyd al. (2011)) in pixel
space. Introduce an auxiliary v > Ax) and the feasibility set C == {v : || - yll S E). Consider
min
x
xo +
<c(v)
s.t. A(x) v= 0, g(v)
(12)
f(x)
where ie is the indicator of C. Using a scaled ADMM-style augmented Lagrangian splitting with penalty >> and scaled dual n, we iterate
x k+1 arg min
1 _ ||x
Xuje 12 +
L (((A(x) v + +u
(13)
27
h+1 -Ilo A(x + u²
(14)
**1 u A(xkt1)
xt1
(15)
Let by* v u² brevity. In FAST-DIPS we run only a small fixed K and not solve Equation 13 exactly: approximate the x-update by steepest-descent step F with analytic step size and backtracking (below). Each correction iteration requires at least one forward evaluation of A (often reused across forming A(x) by the v-update, and the u-update by caching), one VJP to form (x)) (A(x) b²), and either one JVP or one additional forward probe for JA backtracking may add few extra forward calls. The v-update is closed-form.
Closed-form projection onto the measurement ball. For C = {v E RT V y/l - }}, the Euclidean projection in Equation is the standard radial shrink (Parikh Boyd (2014))
W.
||w yll S E,
[l((w)
W
(16)
Y+
otherwise.
yl
Efficient x-update.
Define the smooth objective for Equation 13
1 _ ||x X 11 + +1 - 111
F(x)
(17)
2
Let 8 = x Nojz and r A(x) - b Its gradient is
g = VF(x) = (x - x + p (=A( - b*),
X x ag.
(18)
where (x) is the Jacobian of A at x. Crucially, both the VJP (x) T and the JVP J (x) g can be obtained from autodiff (Baydin al. (2018)).

---

Published as a conference paper at ICLR 2026
We linearize A along the descent direction:
A(x og) R2 A(x) aJ))))
(19)
Substituting Equation 19 into F(x og. yields a one-dimensional quadratie model (Nocedal & Wright (2006))
F(a)
1
|s - agal + + - a
(20)
27
whose exact minimizer is
a (s, g) + prr, (x(x)
(21)
|gl 2 + (x)g)
The numerator can be rewritten as 1 (s. +pr, ()(x)g) = (,, p))()) 74 a' 2 whenever g * O (the clamp is numerical safeguard); if = 0 we are already stationary for the current x-subproblem and set a 0. Computing a" (when g * 0) requires one forward evaluation of A (to form r), one VJP to form and one JVP (or one additional forward probe) form J Ax(x); backtracking evaluates F few times but reuses the same descent direction. In practice we initialize a by a" and apply Armijo backtracking to ensure decrease of F.
g) 1g Hence
Proposition 1 (Local model-optimal step and descent). Under C' regularity of A near and local Lipschitzness of JA. if g + O then a" in Equation 21 minimizes the quadratic model F in Equa- tion 20. Moreover, using a" to initialize an Armijo backtracking line search guarantees that the accepted step yields monotone decrease of the current x-subproblem objective F in Equation 17 (with be fixed), even when Equation 19 is only locally accurate. Iʃ O, then x is stationary for F and no step laken.
Step size via finite-difference JVP (fallback). The analytic step size a* in Equation 21 requires JVP, JA(x)g When forward-mode JVPs are unavailable impractical, estimate the JVP by a single forward probe (Nocedal & Wright (2006)):
(x A(x + ng) A(x)
AA 1
(22)
"
where 17 0 is small scalar perturbation parameter. Appendix A.3 provides a stabilized closed form (Remark 4, Equation 38).
2.6 OPTIMALITY CONDITIONS AND FEASIBILITY
Remark 1 (Nonconvexity and scope). With nonlinear A. problem Equation 9 is generally non- convex; we do not claim global convergence. Our guarantees are local: the x-update descends F (Proposition I and Remark 4). Feasibility is enforced in the split measurement variable via the projection Equation 16 (so vk E C), while the ADMM coupling encourages A(x*) z v For completeness, Appendix A.3 provides (i) a fixed-point
KKT characterization for the exact scaled-ADMM mapping (Proposition 5), and (ii) a KL bound that formalizes mode-substitution in re-annealing under a local Gaussian approximation (Proposition 6).
2.7 LATENT FAST-DIPS
We extend FAST-DIPS to latent diffusion models (Rombach et al. (2022); Podell et al. (2024)) by applying the same per-level construction in latent space: replace A by the composite operator A o D (with D the pretrained decoder) and optimize over the latent variable. The ADMM split- ting. mpeasremenntsspace projection, and analytic step-size carry over via the chain rule and autodiff through D and A. To balance cost and fidelity, we use hybrid schedule: early steps (large oe) ap- ply cheaper pixel-space corrections, then we switch to latent corrections once a, S Tswitch to better conform to the learned manifold. Details appear in Appendix A.I.
3 EXPERIMENTS
3.1 EXPERIMENTAL SETUP
Our experimental setup, including the suite of inverse problems and noise levels, largely follows that of DAPS (Zhang et al. (2025)). We evaluate our method across eight tasks five linear and three nonlinear- to demonstrate its versatility.
7

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative evaluation on 100 FFHQ images and 100 ImageNet images for eight inverse
problems (five linear and three nonlinear). The best and second-best results within each task type (Pixel and Latent) are indicated in bold and underlined, respectively. Method names shown gray denote methods designed for noiseless settings
Task
Type
Method
PSNR (f)
(1)
FFHQ
LPIPS (1)
Run- Mun-time (s)
PSNR (?)
(1)
ImaggeNet
LPIPS (
Run-time
DAPS
28.774
0.774
0.257
40.229
25 35.685
0.651
0.364
97. 192
SITCOM
0.811
21,591
26.519
0.716
65.677
Plasf
CJIGOM
03007
HRDIS
30:45
0.67
26. 764
0.744
6.316
Super resolution 4x
Ouns
29.573
0.841
0.244
2,726
26.367
0.714
0.334
6,266
LatenDAPS
29.184
0.825
0.273
93.383
26.189
0.202
0.388
95.675
Latour
PSLD
ReSample
23.749
23.317
0.601
0.456
0.367
248, 865
92.799
21,262
0,423
0.405
0.501
149.29
999
28,634
0.792
0.283
26.298
0.704
DAPS
24.546
0.754
0.218
33.108
21.399
0.726
0.271
SITCOM
25.336
0.658
0.169
34.994
20.638
0.794
0.209
73 989
Paall
C-IGED
18.291
0.731
0.358
1,277
17. 17.51:
0.676
0.360
3,683
HRDIS
21.735
0.785
0.191
3.726
20.507
0.707
0,280
10.107
Inpuint (box)
19,630
Lanout
ReSample
PSLD
21.488
19.978
0873
0.796
0.247
253.162
91.189
21,084
18.087
0.713
0,803
0.309
281 X31
644
Onrs
24.048
0,829
0.247
45.276
19,349
0.716
0.389
45.989
DAPS
30,250
0.797
0.211
35.361
27.677
0.736
0,340
82.617
STECOM
32,550
0.911
0.148
35.499
29.736
0,853
0.168
106.182
Paael
1.281
HRDIS
28.722
0.823
0,202
4.518
21.614
0.676
0.321
9,703
Inpuint (nandon)
Ours
31,022
0.72
0.202
2,908
28.353
0.791
0,249
5.857
LateniDAPS
25,979
0.742
0.387
91.480
22. 29.695
0.567
0,519
95.442
Latenz
ReSample
PSLD
22,836
29.950
0.472
0.842
0.261
0.467
278.498
87,157
26.916
22.761
0.756
0.522
0.431
146 022
707
45.335
DAPS
28,895
0.755
0,253
50,400
25 25.966
0.662
0.352
94
STICOM
28.775
0320
0.261
32,841
26.201
0.200
0.351
338
Pinel
C-IGUU
24.432
0.678
0.368
1,305
23.201
0.595
0.352
4.581
HRDIS
27.674
0.791
0.259
2,569
24.575
0.633
0.119
5,960
Ganssian deblurring
29,406
0.836
2.612
0.705
LatenDAPs
25,742
Laiou
PSLD
ReSample
26.345
16,007
0.227
0.661
0.569
0.329
292.612
94,823
23,530
608
0.212
0.497
0.566
0.439
822
738
Ours
25,006
0.793
0,312
46,307
25.356
0.661
0.424
48:229
DAPS
$1,074
0.299
0.199
50.924
28.838
0.776
0.241
94 681
Piael
SITCOM
31,172
0.822
36,684
28.815
0.807
338
0.778
2.616
29.037
0.299
Motion deblurring
LatenDAPS
0.752
0.592
Latou
PSLD
ReSample
28.741
19.237
0.751
0.288
0.362
0.518
302.828
90.682
21.845
18.327
0.258
0.579
0.101
0.544
148. 151
316.985
Onrs
29,235
0,822
0.278
46,785
26.627
0.709
0,386
47.282
DAPS
30.25
0.801
0.202
22.354
0.519
926
SITCOM
28.512
18,704
0.393
Piud
23,670
0.537
12,020
14.019
0.195
20.90
Phuse retricval
Durs
29.253
0.851
0.218
10.354
19.238
0.430
0.479
16.629
LatenDDPP
23.450
0.695
0.418
193.005
13.067
0.446
0.621
202 426
Latew
ReSample
24.676
0.606
0.412
321.227
16.913
(1320
0.608
354 430
Ours
28.330
0.789
0.344
87.167
18.874
0.441
0.507
$5.520
DAPS
28,907
0.780
763.86 163866
0.734
1453.314
Pixel
HRDIS
SITCOM
21.929
29.770
0.844
0.658
0.357
3.091
11.400
28.138
23.553
0.791
0.501
0.448
6.653
Nonlineur deblur
Oun
27.818
0.803
0.280
57,903
25.607
0.695
0.373
62.350
LatenDDAP
25.151
0.727
0.384
220.700
22,516
0L568
0.530
249 6:39
Latou
ReSample
28.748
0.792
1276.326
26.047
0.697
0.301
1250.785 783
28.746
0.833
110:567
0.738
537
DAPS
0.344
26.568
0.819
286
Pixel
HIRDIS
SITCOM
26 346
27.625
0.836
0.808
0.178
0.214
38.150
2.428
26.849
24.623
0.825
0.796
0.199
0.207
109 9416
$,989
High dynamic rang
Ours
26.275
0.843
0.218
7.212
24.522
0.775
0,290
13.307
LatenDDPP
20.789
0.630
0,512
197.250
19,394
0,469
0.641
207 469
ReSample
25.058
0.822
261.558
24,950
0.783
495
25.869
0.822
0.247
83,790
24.415
0.773
Implementation details. For all experiments, we employ pretrained diffusion models in both pixel and latent domains. For the pixel-space setting, we use diffusion models trained on FFHQ (Chung et al. (2023a)) and ImageNet (Dhariwal & Nichol (2021)). For the latent-space setting, we use the unconditional LDM-VQ4 model (Rombach et al. (2022)) for both FFHQ and ImageNet. These models used consistently across all baselines and method ensure fair comparison. We adopt the time-step discretization and noise schedule from EDM (Karras et al. (2022)). Evaluation is performed on 100 images from FFHQ (256 256) and 100 images from ImageNet (256 x 256).
8

---

Published as a conference paper at ICLR 2026
DAPS
SUTDOM
C-IIGD (wriscled))
HRDDS (noieloo))
HRDES
Ours
(2) Goustian debluming
(b) Motion dellurring
350
LIPIS
6,5
27.5-
JPPP
23.-
255-
200-
Time (0)
(e) Inpeint (madoo))
Time
(d) Phase Retriiaal
Tisse (1)
0.8
0.5-
PSNR
8.4-
15.
0d-
20.
02
10
Time (4)
Timse (5)
Quantitative evaluations comparing image quality and computational time for baseline methods. Each point is derived from an experiment on 100 FFHQ images. The y-axis value (PSNR or LPIPS) is the mean of the scores from the 100 resulting images. The x-axis value is the average per-image run- run-time, calculated by dividing the total processing time for all 100 images by 100. The plots show results for three linear tasks (a-c) and one nonlinear task (d).
Figure 3:
Across all tasks, measurements are corrupted by additive Gaussian noise with a standard deviation of / 0.05, and performance is reported using PSNR, SSIM, and LPIPS. Inputs are in the range [-1, I] for PSNR/LPIPS and [o, 1] for SSIM. For LPIPS, we keep the default max-pooling layers in the VGG-based implementation. Additional experimental details are provided in Appendix A.6.
Baselines. We compare our method against range of state-of-the-art baselines in both pixel and latent spaces. In pixel space, we include recent fast-sampling methods such as SITCOM (Alkhouri et al. (2025)), C-IIGDM (Pandey et al. (2024)), and HRDIS (Dou et al. (2025)), alongside DAPS (Zhang et al. (2025)), which is recognized for its balance of performance and efficiency. For latent-space comparisons, we benchmark against prominent methods including PSLD (Rout et al. (2023)). ReSample (Song et al. (2024)), and Latent-DAPS (Zhang et al. (2025)). Details of the baseline methods provided in Appendix A.7.
3.2 MAIN RESULTS
Table presents the quantitative results on the FTHQ and ImageNet datasets, where all baselines are run with their official default settings. In pixel space, our method achieves comparable or su- perior performance to the baselines across nearly all tasks on both datasets, but with significantly lower run-tima unn-time. This acceleration is particularly evident in Gaussian and motion deblurring, where FAST-DIPS is about 19.4x faster than DAPS FFHQ and about 20.8x faster than SITCOM on ImageNet, while achieving comparable PSNR/SSIM/LPIPS phase retrieval, we follow the common practice of selecting the best of four independent runs. In this setting, our method is approximately 11.8x faster than DAPS on FFHQ and 19.3x faster on ImageNet, achieving higher SSIM on FFHQ and the second-best SSIM/LPIPS among pixel-space baselines on ImageNet. Furthermore, our approach addresses key inefficiencies commonly found in latent-space methods. While most guided techniques suffer from long run-times due the compu- tational cost of backpropagating through the decoder, our hybrid pixel-latent schedule avoids this bottleneck. By performing corrections pixel space during the early sampling stages and switch- ing latent-space correction later, our method effectively reduces sampling time while maintaining high-quality, faithul reconstructions across datasets.
For the challenging nonlinear task of
Table I alone does not fully capture how different methods compare under the same run-time bud- get. To offer a more comprehensive evaluation, Figure 3 reports PSNR and LPIPS while considering the computational run-time. For this benchmark, vary only the number of sampling steps/inner iterations per method, while all other hyperparameters were kept at their originally proposed opti- mal values to ensure a fair comparison. (Full details are provided in Appendix A.7). We evaluate

---

Published as a conference paper at ICLR 2026
three linear and one nonlinear task in total. Across all four tasks, method improves steadily as run-time increases while maintaining a clear gap over competing baselines. The advantage is particularly pronounced in motion deblurring and phase retrieval, where the superiority highlighted earlier is equally evident under identical run-time budgets. In Gaussian deblurring, even compared to noiseless baselines,
method quickly attains strong PSNR and LPIPS in the early stage and sustains or further improves them as sampling proceeds. The same trend is also observed in random inpainting. For this task, perceptual quality paramount, and our method generates natural-looking results while consistently maintaining low LPIPS.
We include additional experiments on FAST-DIPS in Appendix A.8, covering the effectiveness of the x-update step, hyperparameter robustness, the hybrid schedule trade-off, experiments with non- Gaussian noise and qualitative results in both pixel and latent spaces.
3.3 ABLATION STUDIES
We study two factors inside the per-level correction: whether enforce feasibility by projection and how we choose the step size for the x-update. The projection variant is our default FAST- DIPS (ADMM proj.); the no-projection control is an unsplit penalized solver we call QDP (no splitting, no proj.). which minimizes the same quadratic objective as the ADMM x-suhproblem. To compare fairly, we equalize compute by counting first-order autodiff' work: each x-gradient step uses one forward of A, one VJP, and one JVP (or a single forward probe for FD); projection and dual updates are negligible. With K ADMM iterations and gradient steps per iteration, FAST-DIPS spends K x S such triplets at each diffusion level, so we give QDP exactly K x S gradient steps per level. For step size we compare a tuned constant a, the analytic model-optimal a* (one VJP + one JVP), and a forward-only finite-difference surrogate orp- Full protocol and numbers are provided in Appendix A.4, Table
On representative linear pixel task (Gaussian blur), OFD reaches virtually the same quality a² at lower cost; on the nonlinear latent HDR task the optimization is sensitive to a fixed step and the JVP-based a is the robust choice, whereas app tends to underperform through the decoder-forward stack. Enforcing feasibility by projection consistently improves quality relative to the unsplit penalty path under the matched budget; the extra cost in latent space is dominated by backprop through the decoder rather than the projection. practical recipe is therefore to use orp in pixel space and ax in latent space within FAST-DIPS.
CONCLUSION
Our proposed method, FAST-DIPS, targets practical challenges in training-free diffusion-based in- verse problems. It is broadly applicable: by using VJP and JVP from automatic differentiation instead of hand-crafted adjoint, can handle wide range of linear and nonlinear forward models without requiring SVDs or pseudo-inverses.
For the guidance step, we replace generic optimizers (e.g., Adam with tuned learning rates) by single gradient update with an analytic step size from local quadratic model. This deterministic update removes step-size hyperparameters and improves efficiency and stability.
Empirically, FAST-DIPS shows predictable trade-off between computation and quality: more cor- rection steps consistently improve reconstructions. The framework also does not rely on carefully chosen initial samples. Limitations and future directions are discussed in Appendix A.9.
10

---

Published as a conference paper at ICLR 2026
5 REPRODUCIBILITY STATEMENT
Our experimental setup (datasets, pretrained models, forward operators, noise levels, metrics, and hardware) is specified in Section 3.1. In brief, we use publicly available pixel- and latent-space diffusion priors on the FFHQ-256 and the ImageNet-256, the EDM discretization, additive Gaus- sian measurement noise with 8 0.05, and evaluate PSNR/SSIM/LPIPS on 100 images. Most experiments are performed using single NVIDIA RTX 4090 GPU, whereas the experiments re- ported in Tables 5, 7, were conducted on single RTX 6000 Ada GPU. For phase retrieval, we follow the common "hest-of-4" protocol for all methods (ours and baselines). Baselines are run from the authors' official repositories with their recommended defaults; Appendix A.7 lists the packages used and task-specific settings. We set the random seed to 42 for all experiments, including all baselines. For evaluation, we compute PSNR with skimage. metrics (Van der Walt et al. (2014)), SSIM with TerchMetrics (Detlefsen et al. (2022)), and LPIPS with the lpips library (VGG backbone) (Zhang et al. (2018)).
ACKNOWLEDGEMEET
This work was supported in part by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2025-246831033 in part by Korea Basic Science Institute (National research Facilities and Equipment Center) grant funded by the Ministry of Science and ICT (No. RS-2024-00401 899), and part by Institute of Information & communications Technol- ogy Planning & Evaluation (ITTP) under the Leading Generative Al Human Resources Development (IITP-202--R) (000226RS-2024-00 2024-0030022)) grant funded by the Korea government (MSIT).
REFERENCES
Ismail Alkhouri, Shijun Liang, Cheng-Han Huang, Jimmy Dai, Qing Qu, Saiprasad Ravishankar, and Rongrong Wang. SITCOM: Step-wise triple-consistent diffusion sampling for inverse prob- lems. In International Conference on Machine Learning (ICML), 2025.
Atilim Gunes Baydin, Barak A. Pearlmutter, Alexey Radul, and Jeffrey Mark Siskind. Automatic differentiation in machine learning: A survey. Journal of Machine Learning Research, 18:153:1- 153:43, 2018.
Dimitri P. Bertsekas. Nonlinear Programming. Athena Scientific, edition, 1999.
Stephen P. Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed opti- mization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3-12 2011.
Jiezhang Cao, Yue Shi, Kai Zhang, Yulun Zhang. Radu Timofte, and Luc Van Gool. Deep equilib- rium diffusion restoration with parallel sampling. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Gabriel V. Cardoso, Yazid Janati El Idrissi, Sylvain Le Corff, and Eric Moulines. Monte carlo guided denoising diffusion models for bayesian linear inverse problems. In International Conference on Learning Representations (ICLR), 2024.
George Casella and Roger L.. Berger. Statistical Inference. Duxbury Press, 1990.
Stanley H. Chan, Xiran Wang, and Omar A. Elgendy. Plug-and-play ADMM for image restoration: Fixed-point convergence and applications. IEEE Transactions on Computational Imaging, 3(1): 84-98, 2016.
Hyungjin Chung, Jeongsol Kim, Michael T. McCann, Mare L. Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In International Conference on Learning Representations (ICLR), 2023a.
Hyungjin Chung, Jeongsol Kim, and Jong Chul Ye. Direct diffusion bridge using data consistency for inverse problems. In Advances in Neural Information Processing Systems (NeurIPS). 2023b.

---

Published as a conference paper at ICLR 2026
Hyungjin Chung, Suhyeon Lee, and Jong Chul Ye. Decomposed diffusion sampler for accelerating large-scale inverse problems. In International Conference on Learning Representations (ICLR), 2024.
Patrick L. Combettes and Jean- Chan-Chrittophe Pesquet. Proximal splitting methods in signal process- ing. In Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 185-212. Springer, 2011.
Nicki Skafte Detlefsen, Jiri Borovec, Justus Schock, Ananya Harsh Jha, Teddy Koker, Luca
Di Liello, Daniel Stancl, Changsheng Quan, Maxim Grechkin, and William Falcon. TorchMetrics measuring reproducibility in PyTorch. Journal of Open Source Software, 7(70):4101, 2022.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs image synthesis. In
Advances in Neural Information Processing Systems (NeurIPS), 2021.
Hongkun Dou, Zeyu Li, Jinyang Du, Lijun Yang, Wen Yao. and Yue Deng. Hybrid regulariza- tion improves diffusion-basee inverse problem solving. In International Conference on Learning Representations (ICIR), 2025.
Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In International Conference Learning Represenlations (ICLR), 2024.
Heinz Werner Engl and Martin Hanke. Regularization of inverse problems, volume 375. Springer
Science & Business Media, 1996.
Tomer Garber and Tom Tirer. Image restoration by denoising diffusion models with iteratively preconditioned guidance. In IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2024.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusson probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Gongye Liu, Haoze Sun, Jiayi Li, Fei Yin, and Yujiu Yang. Accelerating diffusion models for inverse problems through shorteut sampling. In International Joint Conference Artificial Intelligence (IJCAI), 2024.
Guan- Guan-Horg Liu, Arash Vahdat, De-An Huang, Evangelos A. Theodorou. Weili Nie, and Anima Anandkumar. PSB: Image-to-image schrödinger bridge. In International Conference on Machine Learning (ICML), 2023.
Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. In International Conference on Learning Representations (ICLR), 2025.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. RePaint: Inpainting using denoising diffusion probabilistic models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective solving inverse problems with diffusion models. In International Conference on Learning Representa- tions (ICLR), 2024.
Jorge Nocedal and Stephen J. Wright. Numerical Optimization. Springer, 2 edition, 2006.
Kushagra Pandey, Ruihan Yang, and Stephan Mandt. Fast samplers for inverse problems in iterative refinement models. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
Neal Parikh and Stephen P. Boyd. Proximal algorithms. Foundations and Trends in Optimization, (3):133- 2014.
12

---

Published as a conference paper at ICLR 2026
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for nigh-resolution image synthesis. In International Conference on Learning Representations (ICLR), 2024.
Christoph Reich, Biplob Debnath, Deep Patel, and Srimat Chakradhar. Differentiable JPEG: The Devil is in the Details. In IEEE/CVF Winter Conference Applications of Computer Vision (WACV), 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkot- tai. Solving linear inverse problems provably via posterior sampling with latent diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Litu Rout, Yujia Chen, Abhishek Kumar, Constantine Caramanis. Sanjay Shakkottai, and Wen- Sheng Chu. Beyond first-order Tweedie: Solving inverse problems using latent diffusion. In IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2024.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, ((44)4473-4722) 2022.
Bowen Song, Soo Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen. Solving inverse problems with latent diffusion models via hard data consistency. International Conference on Learning Representations (ICLR), 2024.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations (ICLR), 2021a.
Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations (ICLR), 2021b.
Yang Song, Prafulla Dhariwal, Mark Chen, and llya Sutskever. Consistency models. In International Conference on Machine Learning (ICML), 2023.
Stefan Van der Walt, Johannes L Schönberger, Juan Nunez-Iglesias, François Boulogne, Joshua D Warner, Neil Yager, Emmanuelle Gouillart, and Tony Yu. scikit-image: image processing in python. PeerJ, 2:c453, 2014.
Singanallur V. Venkatakrishnan, Charles A. Bouman, and Brendt Wohlberg. slug-and-play priors for model based reconstruction. In IEEE Global Conference on Signal and Information Processing (GlobalSIP). 2013.
Hengkang Wang, Xu Zhang, Taihui Li, Yuxiang Wan, Tiancong Chen, and Ju Sun. DMPlug: A plug- in method for solving inverse problems with diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. In International Conference on Learning Representations (ICLR), 2023.
Zihui Wu, Yu Sun, Yifan Chen, Bingliang Zhang, Yisong Yue, and Katherine Bouman. Principled probabilistic imaging using diffusion models plug-and-play priors. In Advances in Neural Information Processing Systems (NeurIPS). 2024.
Xingyu Xu and Yuejie Chi. Provably robust score-based diffusion posterior sampling for plug-and- play image reconstruction. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
13

---

Published as a conference paper at ICLR 2026
Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing. In /EEE//VV Con- ference Computer Vision and Pattern Recognition (CVPR), 2025.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Jiankun Zhao, Bowen Song, and Liyue Shen. CoSiGN: Few-step guidance of consistency model to solve general inverse problems. In European Conference Computer Vision (ECCV), 2024.
Yang Zheng, Wen Li, and Zhaoqiang Liu. Integrating intermediate layer optimization and projected gradient descent for solving inverse problems with diffusion models. In International Conference Machine Learning (ICML), 2025.
Yuanzhi Zhu, Kai Zhang. Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and-play image restoration. In IEEE/CVF Con- ference on Computer Vision and Puttern Recognition Workshops (CVPRW), 2023.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.I LATENT-SPACE FAST-DIPS AND A HYBRID PIXHI.ILATENT SCHEDULE
The pixel-space method in Section 2.5 corrects the denoiser's proposal directly in image space. In many diffusion systems, however, the prior is trained a lower-dimensional latent space. Let E RCHW + R and ! R -> RCHW denote pretrained encoder-decoder with Zo E(xo) and Xo Dz Measurements are still acquired in pixel space via Equation 1. A latent denoiser Zden Le, σe) is available from the diffusion prior. We now derive latent analogue of the per-level objective and show that the pixel-space construction transfers under the substitution A + AoD and ++
Per-level surrogate in latent space. At level f, the denoiser proposes Zoje := Zden (Ze, o²). As in Section 2.3, we approximate p(zo z:) by local Gaussian centered at Zoje with variance parameter 7= > 0 (we use 7/2 = of for schedule-awareness)) and we employ the same set-valued / hard- constraint likelihood surrogate in the measurement space, now expressed through the decoder:
(z0 y) exp .イ:
I|z0 - =uj: l|²) ((lA((((2)) -yll S $t).
(23)
Since feasibility is enforced in the same measurement space, in our experiments we simply set Ex (unless noted otherwise). Taking the mode yields the latent per-level MAP:
Zoit arg min zER* 2%
|z
zo/e1 s.t. [|A(((z)) yyll Es
(24)
Re-annealing then follows the same transport rule as Equation 11:
-.-1E. & I),,
X₂-1 (((
(25)
ADMM in latent space and adjoint-free updates. Introduce v A(D(z)) and the same feasi- bility set C = (v: || - yyyl S } The scaled ADMM iterations mirror Equation 13- -3-Eutati 15:
k+1 arg min 1
||z Zo1z 112 + Ps [IA(((z)) v + uu
(26)
2
k+ TT A(D(2 (z1) + u²)
(27)
u k+1 = n² A(D(z*+1))
v++1
(28)
The projection Hc is identical to Equation 16 because feasibility is enforced in measurement space. As in pixel space, in practice we run only a small fixed number of iterations and do not solve the z z-subproblem Equation 26 exactly; instead we use one (or few) adjoint-free gradient steps on F; with analytic step initialization and backtracking. For the z-update, define
(z) = 27s ||z z + Ps |LA(((z)) - b$1 bt = v u²
(29)
2
and r = A(D(z)) bR Its gradient is
1 (z 201e) + P= JAOD (A(((2)) - b*),
g: (F,(z) =
z + 2 g:
(30)
72
As in pixel space, both the VJP [x and the JVP (Z)D(Z) are obtained directly from autodiff (backprop through D and A: forward-mode single finite-difference for the JVP needed), so the update remains adjoint-free.
Analytic step size in latent space. Let S2 := Zofz Linearizing A. D along -g. gives A(D(z A(D(Z) JoDD g.. The scalar quadratic model
Fa(a)=
||s - ag.
Pz |r AJAo((Z) 2
(31)
27
is minimized at
(82, 8=) + Pz (r. (2cD
(32)
+ Ps | (2)
followed by backtracking to ensure descent of F;; if gf = O we are already stationary for the current z-subproblem and
15

---

Published as a conference paper at ICLR 2026
Proposition 2 (Latent model-optimal step and descent). Under C' regularity of AcD near and lo- cal Lipschitzness of its Jacobian, if g. + then the step o in Equation 32 minimizes the quadratic model Fz(a). Moreover, using 0g (or its positive clamp) to initialize an Armijo backtracking line search guarantees that the accepted step yields monotone decrease of the current latent z- subproblem objective Fg in Equation 29 (with ba fixed). gf =0, 0, then z is stationary for Fz and no step laken.
Proposition 3 (KKT latent ADMM fixed points). If (z" ,v, is fixed point of the exact scaled-ADMM updates Equation 26- E--Eqaation 28 (i.e., z solves the z-subproblem Equation 26 exactly), then z² satisfies the KKT conditions of the latent proximal problem Equation 24.
Remark 2 (Transfer of pixel-space results). All pixel-space results in $2.4-$2.6 transfer to the latent case by replacing A with AoD and x with z. The projection statement is unchanged because feasibility is enforced in measurement space. Propositions 2 and 3 follow from the corresponding pixel-space arguments by the chain rule. Proposition 6 also applies to the latent Gaussian injected distributions in Equation 25 by replacing with z; after deterministic decoding, Kl. cannot increase (data processing).
Why (and when) prefer latent updates. Late in the schedule, de is small, the denoiser's latent prediction Zojz lies near the generative manifold, and optimizing in z respects that geometry by construction. Early in the schedule, however, correcting in pixel space is often cheaper (no back- prop through D) and sufficiently robust because injected noise dominates the time time-marginal. This observation motivates a hybrid schedule.
Hybrid pixel-latent schedule. We adopt single switching parameter Tswitch: for a >> Sswitch correct in pixel space using Equation 9-Fquation 15, then re-encode z +- Ex) before annealing in latent space; once or S switch we correct directly latent space using Equation 24-Equation 28. This keeps early iterations light and late iterations manifold-faithful
Complexity and switching. latent z-gradient step costs one pass through D and A to form r, one VJP through A to form JAOD", and one JVP to form JANDg:; we found this JVP-based step is effective for nonlinear-deblur in latent space. In pixel space, for strongly nonlinear A we recommend the FD variant Equation 22 and 38, which swaps the JVP for single extra forward call and was both faster and more stable in our nonlinear-deblur experiments. The switch w trades early-time efficiency for late-time fidelity; stable default is place it where the SNR of the denoiser's 's prediction visibly improves (e.g., where "/2 becomes comparable to the scale of ||x-Xxja in Equation 17).
Remark (Consistency of pixel encode with latent correction). If E and D are approximately inverses near the data manifold (i.e., (x) =x and (D(z)) z) and are locally Lipschitz. then pixel correction followed by z t {xx) produces latent iterate whose discrepancy is controlled by the local reconstruction error of the autoencoder (i.e., how close DoE is to ld near the manifold), Thus the hybrid scheme is coherent approximation of the pure latent method early in the schedule.
16

---

Published as a conference paper at ICLR 2026
A.2 ALGORIHTMS
Algorithm FAST-DIPS in Pixel Space
Require: measurement y; schedule }or}; denoiser Xten ot); forward A: parameters ps (7t) 7)
Ensure: reconstructed image Xo
1: Sample XT (0,00.11)
2: for f down to 1 do
3: predict X Xden (Xt, o4)
4: Initialize x Xujti
4 A(x); u 0
5: for
K do 6: b 4-+17 u:
F(x) - 1 27c xx xxooe1111?+ -11lA(x)x b|l2
7:
for i to S do
x-update: gradient step + backtracking
8:
r f A(x)
b; S + x x X0le
9:
Edata
Vx ( A(x) - b113)
D via automatic differentiation
10:
g
1 s + P i AA f- A(x 71)) A(x)
7
(5. + ap (r.
x1² 7t Iill' + P ||
12:
Backtrack on a until F(x - ag) < F(x) + x -ag
13:
end for
14:
W <- A(x) + V -llcsw(
15: u + u u A(x) -v
16: end for
17: Sample & ~ NO00 I) and set X,-1 t xx x 0
18:
end for
19:
return X0
17

---

Published as a conference paper at ICLR 2026
Algorithm 2 FAST-DIPS in Latent Space
Require: measurement y; schedule {or); ;; latent denoiser Zden (+, oı); encoder E; decoder D; for- ward A: parameters Px: Ta. Ka, S2,5z; Szy Pz, Yz. K2, S2, S2, Owwich
Ensure: reconstructed image Xo
1: Sample ZT ~ (00, T)
2: for = down to 1 do
3: predict (latent) Zoje + Zden (Ze, Fz)
4: if oo > swich then
b early: pixel correction
5:
Xojt - (Z)o)) X Xotti V V A(x); ut 0
6:
for to K₂ do
7:
b +- u:
F3 (x) < - xx xolell2 2 + 2 [|(A(xxx bl2
8:
for = 1 to Sp do
x-update with analytic step
9:
g +- 1 (x Xoje) + AA) (A(x)) - b)
10:
Form (x)x (JVP) and set by Equation 21;
Backtrack on a until Fz (x ag) < (x): set x t- X -0
12:
end for w f A(x) + u: v -y 1l,,,,
13:
ut u+-+A((x)
14:
end for
15:
re-encode 2 t {(x) else
16:
> late: latent correction
17:
2 201z V A(D(z)); u for h: to K= do
18:
19:
b < u;
F, (z) + 1
2 e 112 + + ||A(((z)) - -bl
27
20:
for s = 1 to S² do
& z-update with analytic step
g: - 7: (z ZO + LoDo
AoD (z) (A(()z)) - -b)
22:
Form (Z) (JVP) and set a by Equation 32;
23:
Backtrack until F₂ og.) F (z); set 0g: end for
24:
25:
+ A(D(z)) +u; V -1 (w); ut u A(D(z))
26: end for
end if
28:
re-anneal 2:-1 2 o N(o,1
29:
end for
30:
return Xg - -D(()))
A.3 THEORY AND PROOFS
This appendix first summarizes the proposed FAST-DIPS procedure and its modeling assumptions (App. 1.3. We then restate and prove the pixel-space results used by the method (App. A.3.2). For the latent variant, the corresponding statements follow from the same arguments by the substitution A + AoD and the chain rule; we provide brief transfer explanation and omit redundant proof's (App. A.3.3). Finally, we give step-by-step derivations of the analytic step sizes used in the pixel and latent updates and explain how they can be computed with autodiff VJP/JVP or single forward- difference probe (App. A.3.4).
A.3.1 OVERVIEW AND ASSUMPTIONS
Method in one paragraph. At diffusion level f, the pretrained denoiser returns an anchor Xoje = Xden (X,, ot). We then solve a hard-constrained proximal problem around X
1 2%
min X
||x
xo
s.t. [IA(x) -yll S
(33)
18

---

Published as a conference paper at ICLR 2026
in the standard (Euclidean) measurement space. We solve Equation 33 by scaled ADMM-style augmented Lagrangian splitting with variables v,
x ++1 = arg min
||x xoxell 21? + |A(xx) - v + 1
(34)
27
v x+1 = IITel ++ + +
C= (v 1Iv yyl Et)
(35)
u ++1 = ut A(x v
(36)
The v-update is a closed forred-form projection onto ball; the x-update is one (or a few) adjoint- gradient steps with an analytic, model-optimal step size, where the needed directional Jacobian term JA(x g is obtained either by autodiff JVP or by a single forward-difference probe. After correction,
re-anneal by sampling
x
coTr Off
+ --
& 1~,(
(37)
which implements the decoupled time-marginal transport.
Standing assumptions.
A1 (Noise model and metric) We assume additive white Gaussian noise (AWGN) with covari- ance 3²1 / and work in the standard Euclidean metric in measurement space; the feasibility set is the ball {v ||v y < $).
A2 (Regularity) A is C in neighborhood of the iterates, and AA is locally Lipschitz.
A3 (Tolerance) The radius is treated user-chosen data-consistency tolerance; do require [|A(xx)) - yll S S.
A.3.2 PIXEL-SPACE PROPOSITIONS AND PROOFS
We restate the pixel-space results referenced in the main text and provide detailed proofs.
Proposition 4 (Closed-form projection onto the measurement ball). Let C = {v E RT -|v--yl yll S } in the measurement space. Then the Euclidean projection (l((w) in Equation 14 is exactly the radial shrink (Parikh Boyd (2014))
|| Se
Ile(w)
W
y W yll
Y
otherwise.
We solve miny fll convex and the feasible set is closed and convex; hence there a unique minimizer.
Proof of Proposition 4.
w| s.t. 1|v yll S e. The objective is 1-strongly
KKT derivation.
The Lagrangian is
L(v, A) = 1 w/ A(IIv - -(Iv-~llll -)),
A O.
Stationarity gives
(v 0 = - -v v-y yll if
There are two cases.
(i) Interior case. If the constraint is inactive the optimum, then À b complementary slackness and stationarity gives v = w. Feasibility requires [[ yll < e, i.c., w E C.
(ii Boundary case. Otherwise || yll and À 0. Stationarity implies v is colinear with V y; hence the optimizer lies on the ray from to w. Write v = y + T(W y) with + 2 0. Enforcing ||v y|| yields e/ e/llw w| yll. Substituting gives
w yll
This is exactly the radial projection formula in Equation 16. Uniqueness follows from strong con- vexity.
19

---

Published as a conference paper at ICLR 2026
Proposition 1 (Local model-optimal step and descent). Under C' regularity of A near and local Lipschitzness of J. if g # 0 then a* in Equation 21 minimizes the quadratic model F in Equa- tion 20. Moreover, using a" to initialize an Armijo backtracking line search guarantees that the accepted step yields monotone decrease the current x-subproblem objective F in Equation 17 (with ba fixed), even when Equation 19 is only locally accurate. If = O, then x is stationary for F and no step laken.
with Xojz and A(x) b. The Proof of Proposition 1. Write F(x) 27, ||s 21|1|1 gradient is
g g= VF(() 1s p
Consider the steepest-descent trial x(a) og. A first-order Taylor expansion along -g gives
A(x(a)) A(x) ox)g(x)gg c(o),
|e((a)] 1
for some local Lipschitz constant L of JA (from A2). Plugging this into P(x(a)) yields
F(x(a))= 270 I|s agl - + p aJa(x)g e(a)) + & [e(())]
F())))
The model a convex quadratic a with derivative
F'(a) (s.g) g) (r, (A()))) a
( 1gl +
and curvature (a) = + | + أ| ( N 0, with equality only at stationary points where and JA(x)g = 0. Setting Fv (a) = 0 yields the model minimizer a* in Equation 21.
Descent of the true F. Using the expansion above and Schwarz with the bound |c((a]],, we obtain
F(x ag) s F(a) + p|| aJa(x)gl by a + ? La a² 11s)1²)
a = a F(a)) = min, F(a) and the improvement over ((0) = F(x) is
F(() F(a²) = (1. prr (x))))
2 1g p (x)g](
The remainder terms are 00 and ([g|" shrinking a by a constant factor (standard Armijo backtracking) ensures these are dominated by the quadratic-mode decrease, yielding strict descent of F.
Remark 4 (Step size from difite-iffeeeene JVP), Replacing JA(x)g in Equation 21 by AAm from Equation 22 yields the numerically stable
step
17 L
np AA) (s
CFD
Lg + P 1 2
where AA= A(x + +g) A(x)).
(38)
which is algebraically equivalent to substituting (x)x se AA/T in Equation 21 (the scaling by n? avoids division by small 1)). Since JA locally Lipschitz. AA/T Ja(x)g + Ofrllgl?), so aFD - a" as 17 + 0: backtracking preserves monotone decrease of F.
Remark 5 (Linear A yields exact optimal line search). If A is linear, then Equation 19 is exact and Equation 21 gives the true optimal line-search step for F along -g (Nocedal Wright (2006)), delivering the fastest progress among steepest-descent steps.
20

---

Published as a conference paper at ICLR 2026
Justification. If A(x) Hx, then Jx(x) H and the linearization is exact: A(x ag) A(x) aHg. Hence F coincides with F(x og) along the line, and the model minimizer in Equation 21 is the exact optimal line-search step.
Proposition 5 (Fixed points satisty KKT for Equation 9). Let (x*,, v", ,') be a fixed point of the exact scaled-ADMM updates Equation 13 13-Equation 15 (i.e., x solves the x-subproblem Equa-
tion 13 exactly). Then A(x") = -v,, v" E EC, and there exists X 20 such that
1 (x' xxo+) + X JA(X)* =0, x*I1A((') - = = (39)
where
{ A(x*) -y
11A(x*) -yll E,
v³ E
[A(x)) - (1)).
IIA(x*) -yll <.
Hence x" satisfies the KKT conditions of Equation (Bertsekas (1999)).
Proof of Proposition S. At fixed point (x", v", u²), the u-update satisfies u* uu + +A(x") -v", hence primal feasibility A(x*) - v" = 0. The v-update is the metric projection onto C:
IIc(A(x)) + u²),
so v and the optimality condition of the projection reads
0 0ic(v") + p(v" (A(x') + u")) -ic(v") - pu',
i.c. pu dic(v") NN(vv), the normal cone of C at v*. For the x-subproblem, first-order optimality gives
0 (x" xole) + (JA((")T((((X) - - ur) Tt 1x -x0 p
using primal feasibility. The normal cone for the ball {vv ||v - yll S $} is
Ne(v") = ( [ A 0}, ||v y/| E, with v³ v= y ||v* yll <
10))
1|v yil
Thus pu' X0D" for some X 2 when the constraint is active and otherwise. Substitut- ing into the x-optimality condition yields
1 [x' Te
x xx" 0.
Complementarity X A([IA((x)) yll s) 0 follows by construction of the normal cone. Hence (x*, ( satisfies the KKT conditions of Equation 33.
Proposition 6 (Mode-substitution re-annealing under local Gaussian surrogate). Fix x, and as- sume the conditional admits a local Gaussian (Laplace) surrogate p(xo x1,y) se N(m,, Ex)
with D, ž O, where my is the local mode/mean of the surrogate. Let ecorr denote the determin- istic corrected estimate used by FAST-DIPS as the center of the re-annealing step in Equation 11 (for analysis, take X to be the solution of Equation 9). For at-1 >> O, consider the two per- step boussian-iijecterr distributions: (i) sample X9 ~ Nmm,, Ex) and inject N Ii giving xcon centered at Xot giving N N(x _ T). Then
N(m,, t T); versus (ii) inject N (0,0%
|
|
KI. (T,(N E,
E,+ + oi (x (0-00)0of-11) oi-1) <
(40)
207_1
4o
Consequences. The bound quantifies the approximation incurred by FAST-DIPS replacing sampling from the local Gaussian surrogate by mode-centered re- re-annealnng It is small early when oi_ is large, and it is small late when the surrogate conditional concentrated (small [[E][[] and the corrected estimate xcort is close to the surrogate mode/mean m4. The first term also makes explicit that an arbitrary substitution can be poor; the approximation is meaningful only insofar xcor acts as a good mode proxy under the local surrogate.
21

---

Published as a conference paper at ICLR 2026
Proof of Proposition 6. Let P (mm, Eg + I2 and Q NN(x o in Rd. The Gaussian
KL. formula gives
KL(PIIQ) = = tr(E( Ep) + (uQ up) " SuQ up) d + log det EQ det 25p/
With Eq Ep 021 + Ee. pQ
up corr
my, we get
KL(P]IQ)= = | m& ||
+ (tr(I + Ex) - d log det([ + 0² Ex))
2x²
Diagonalize E2 TAUT with eigenvalues X; 2 Then
KL(PIQ) |xopr
mm/1
1 d
log (1+ (1+
Σ
20²
2 i=1
Use r - log(1 + < x2
5) f2r for 2 20 termwise to obtain
||x
m,
1|
1
corr
mm 1|2
|| E2 1
KL(PIIQ)
20
202
4
Tightness regimes. The second term vanishes 0² + (early in the schedule) and as ||>||| (late in the schedule); the first term quantifies bias between the mode Xuje and the posterior mean m,
A.3.3 LATENT-SPACE COUNTERPARTS
Why the substitution A AbD is valid. If A and the decoder D are C' in a neighborhood of the iterates and have locally Lipschitz Jacobians, then the composite AoD C with locally Lipschitz Jacobian on the same neighborhood. Consequently, all arguments that rely on local linearization (and VJP/JVP computations) transfer verbatim to A via the chain rule, while the projection step is unchanged because fcasibility is enforced in measurement space.
Latent proofs. The latent descent guarantee and the latent fxed-poiin = KKT statement follow from the pixel-space proofs by replacing A with AoD and with z, and applying the chain rule to the VJP/JVP terms. Since these arguments are structurally identical, we omit the redundant proofs for brevity.
Remark 6 (Mode-substitution transport in latent space). Proposition 6 applies verbatim the latent re-annealing step Z;-1 = Zojt comr + ot-1& by replacing x with z, since the transport kernel in latent space is Gaussian. If one compares the decoded distributions in pixel space, deterministic decoding D cannot increase KL (data processing). so the latent KL bound also upper-bounds the discrepancy after decoding.
A.3.4 DERIVATION OF ANALYTIC STEP SIZES AND AUTODIFF COMPUTATION
Pixel space:
Recall
1 ||x Xo a112 2 [11A(x)) -bl?, s=x Xplts T= A(x) b.
F(x) 2%v-
Then g Ls + +xJA(x) T.. For the trial x(a) ag.
7t
A(x(a)) = A(x) - (x)g
gives the scalar quadratic model
1 Is ag] 2 P o.
F(a) 2%
2
22

---

Published as a conference paper at ICLR 2026
whose derivative is
((a) = (s (8.8) g) (A())) + +a + +l|JA(x)=1²)
Setting (a = 0 yields a² in Equation 21. The curvature (a)a)) lgl + (l)))(x))]) 20
shows uniqueness unless g = 0. Autodiff computation recipe (pixel):
1. Evaluate A(x) to get A(x) - b.
2. Compute the VJP (x(x) Tr (reverse-mode autodiff) and form g.
3. Obtain the directional Jacobian (x(x) either
by forward-mode autodiff (preferred when available), or
by single forward-difference probe
AA
JA(x)g
AA: A(x + ng) A(x),
in which case it is numerically convenient to assemble the FD-stabilized closed form Equation 38 (equivalent to substituting AA/n into Equation but avoiding division by small 17).
4. Assemble the numerator/denooiinaoo and perform Armijo backtracking.
Latent space:
With
Fz(z) ||z Zofel 1
1
Pr (l))((
1 (z (z-zo))) ( (A((())) -b), g= 7s
27:-
linearize AoD obtain
1 ZoiE ag: P= (L(((z))) Ao
(((a)
27
The VJP/VPP are computed end-to-end through D and A by autodiff; a single finiteddifference through the composition is valid JVP fallback:
whose minimizer Equation 32.
(z)D(z)g A(D(z + ōg.)) A(D(z))
Complex-valued measurements. When measurements are complex, work with real- imaginary stacking (dimension 2m) and the Euclidean norm; all expressions remain valid verbatim, with JA denoting the real Jacobian.
use a monotone backtracking line search: starting from an initial candidate step (e.g., a* or OFD), repeatedly shrink a t- Ta (e.g., = 2)) until acceptance,
Backtracking and safeguards.
ie.,
F(x - ag) < F(x) (up to a small numerical tolerance).
A.3.5 ADDITIONAL REMARKS
Remark (Trust-region scaling along the schedule). Setting = o ties the proximal weight to the diffusion noise: early (large σ:) the correction can move farther from the denoiser anchor, while late (small σe) the local trust region tightens.
Remark 8 (Constraint and closed-form projection in implementation), Under the AWGN setting adopted throughout, the measurement-spen space constraint is an Ez (Euclidean) ball and the v-update is the closed-form radial shrink in Equation 16. The remaining steps follow standard scaled ADMM- style updates, with the x- x-updete handled approximately by adjoint-free gradient steps with analytic step sizing and backtracking.
23

---

Published as a conference paper at ICLR 2026
Table 2: Ablation of step-size selection inside two per-level solvers. Left: Gaussian blur (pixel). Right: HDR (latent). We compare constant analytic/JVP a", and forward-only ap within QDP (no splitting, no proj.) and FAST-DIPS (ADMM + proj.). For fairness, compute matched by allocating K x S gradient steps per level to QDP when FAST-DIPS uses K ADMM iterations with gradient steps each; projection/dual updates are negligible. All results are evaluated on FFHQ256 10 samples.
Gareun iSt (Pro6)
High
lorge (Laker)
10"
72.854
11:655
0.074
1.891
10
32.113
[1671
0.59
31 837
10
25.028
11.700
0.314
1.867
10
23.486
(.760)
0.356
22.07%
glining no 2
2,047
0.162
0.779
1,055
quiningg peg)
16.356
INP
20:480
(0830)
0.211
2,355
JVP
24,356
0,337
MUUS
FO3
79:377
01822
2018
25.955
(177)
0.394
11.198
24.829
00.14
0.991
1981
10"
31,532
0.09
25,01
limd
28:647
01811
0.29
2029
10-8
25.221
0.788
0.399
25.110
RAST-DIPS (ARMM poo)
d 10"
3.851
00151
0.772
1,93
PAST DAPS (ADMM ppn))
m 10-1
23.328
0.390
0.294
25.199
FFI (Olin)
79433
035
0.787
7053
F13
31.011
0.766
0.35
16.99
Remark (Empirical choice: FD in pixel, JVP latent). Our analytic step size depends on a di- rectional Jacobian term along the current descent direction: Jx(x)g in pixel space and A (z)g, in latent space. In pixel space, a single forward-different estimate of this term lower overhead than autodiff JVP in our implementation, yielding similar performance with reduced run-time. In latent space, however, the composed map A A D rypically much more curved, the FD estimate becomes sensitive to the perturbation scale: too large yields a biased directional derivative (poor local linearization), while too small suffers from numerical cancellation (especially with mixed precision), leading unstable step-size estimates and degraded reconstructions. We therefore use autodiff JVPs through A a DD for the latent variant, In both cases, backtracking guarantees descent of the current subproblem objective (with ADMM auxiliary vari- ables fixed).
probe provides a sufficiently accurate
A.4 ABLATION STUDIES
Goal and tasks. We assess the impact of measurement-space feasibility via projection and the choice of step size inside the x-update. Experiments use 10 FFHQ images on two representa- tives: Gaussian blur in pixel space and HDR in latent space, with PSNR/SSIM//PIPS and average per-image run-time.
Baseline and objective. To isolate projection, we evaluate an unsplit penalized baseline that opti- mizes the same quadratic objective as the x-subproblem inside ADMM, but without variable split- ting or projection:
min XERCHW
||x
Xojell
+
2,32 |A(x) y1
27
which we refer to QDP (no splitting, proj.). In all runs we match the ADMM instantiation by setting "' = of identically to FAST-DIPS and choosing the data-penalty weight so that & 28
Compute-matched fairness. Each x-gradient step entails one forward pass of A, one VJP, and one JVP (or single forward probe for FD); projection and dual updates negligible. With K ADMM iterations and S gradient steps per iteration, FAST-DIPS (ADMM + proj.) spends x S such triplets per level, QDP is allotted K S gradient steps per level to match compute. Step-size mechanisms are kept identical between solvers: constant o, analytic/IVP a", and finite-difference QpD-
Findings. As shown in Table 2. in pixel space, aFD is competitive with a* at lower cost; in la- tent space, a* provides the stability needed for the nonlinear foroaer-forward composition, while arp lags. Under the matched budget, enforcing fcasibility via projection improves quality over the unsplit penalty path; latent run-times primarily reflect decoder backprop.
24

---

Published as a conference paper at ICLR 2026
Table 3: The hyperparameters of experiments in paper for all tasks.
Alpettn
Super Nocletio
Iepuint (Bod)
Laxar Bak lepuant (Bandeoy)
Geamin dethenin
Meiio detheriu
Placc ectrical
Nee Lancer tack Nestinar defeumsg
dyeami rarge
75
75
150
150
150
3
300
333
CJ5
1.6
048
1L.6
011
1665
6.5
ute
J K.)
(5,5)
(5.3)
(5,5)
(3.3)
(5.3)
(10.10)
(10,10)
186,01
(S,, Si
(X.3)
(3.1
(1.3)
(1.3)
(1.5)
(3.3)
170000000
5
CUS
0.66
0U6
0,05
0.88
CAS
0.66
A.5 HYPERPARAMETERS OVERVIEW
Throughout our experiments, hyperparameter settings are summarized in Table 3. We fix 10-3 all experiments. In the annealing process, we set max = 100 pixel space and 10 in latent space, with min 0.1 in both, to enhance robustness measurement noise.
A.6 EXPERIMENTAL DETAILS
Validation set information. For reproducibility, explicitly specify the indices of the samples used for validation. FFHQ (256 x 256). We use 100 images corresponding to dataset indices 00000-00099. ImageNet (256 256). We use 100 images from the ImageNet validation set corresponding to indices 4 49000-49099.
run-time measurement. The run-times reported in Table and Figure 3 correspond to the pure sampling time per generated sample. Since sample with batch size we measure the sam- pling time for each individual sample, sum over samples, and report the average run-time as (E)=(²) /N. We exclude data loading, saving, and metric evaluation from the timing, and mea- sure only the time spent inside the sampling loop. The same timing protocol is applied FAST-DIPS and all baselines. For phase retrieval (best-of-4), we report the mean per-run run- ountiime over four
independent runs: EEL Tt /4.
A.7 BASELINE IMPLEMENTATION DETAILS
All baselines were run using the authors' public repositories:
DAPS/LatentDAPS:
gittab.com//annggggggnngg//99999
SITCOM: gittub.ccm/jjmsse//SSTcCC
HRDIS: github..ommdenn---llabbHHRDDD
C-IGDM: github.commmand--lll---pipd
PSLD: github.comLLituRou/PSLD.
ReSample: gittbb.om/ssmmikvoo/rssnn..
We followed each method's original paper and default repository settings. Additionally, for phase retrieval applied best-of-four protocol uniformly across all compared baselines.
Measurement noise setting. Because the SVD-based operator becomes unstable when noise is in- jected in super-resolution and Gaussian deblurring, we evaluate HRDIS with noise on the remaining tasks, whereas C-IIGDM is evaluated only in the noiseless setting across all tasks.
Details of Figure 3.
For the uun-time-qualle trade-off in Figure 3, we varied only the number of solver steps/iterations per method, keeping all other hyperparameters at their recommended defaults:
DAPS: The number of ODE steps was fixed at 4, while the number of annealing steps was swept over {2, 5, 10, 15, 20, 25).
25

---

Published as a conference paper at ICLR 2026
Gaussian deblurring
Inpaint (random)
Motion deblurring
Measurement
Table 4: Quantitative results under Pois- measurement noise (Apiisson 1).
FAST-DIPS remains accurate and perceptu- ally faithful across tasks.
Reconstruction
Task
PSNR
SSIM
LPIPS
Gaussian dehlurring
28,730
0.814
0.273
Random Inpainting
0.192
Nonlincar deblurring
27.016
0.781
0,266
Reference
Figure 4: Qualitative reconstructions under Poisson measurement noise (Apuisson 1): FAST-DIPS pre- serves edges and textures across tasks.
SITCOM: swept pairs of diffusion steps and inner iterations over (N, {(3,2), 2), (5, (5, 5), (5, 10), (5, 15), (5, 20)}.
HRDIS: We varied the number of diffusion steps over {10, 15, 50, 80, 100, 130).
C-IGGD: We varied the number of diffusion steps over {20, 50, 75, 100, 150, 200}.
Automatic Differentiation Primitives. To implement the adjoint-free analytic updates without manually deriving gradients for the forward operator A, we leverage the automatic differentiation capabilities PyTorch. Specifically, the Vector-Jacobiee Product (VJP) term Jx()) r, which is necessary for computing the gradient of the data-consis term, is obtained via standard reverse- mode differentiation using torch. autograd grad. For the Jacobian- Vector Product (JVP) term JA (x)g, we utilize the functional transformation API, specifically torch. func. jvp. This allows us to efficiently compute the directional derivative required for the local quadratic model in fully differentiable manner.
A.8 ADDITIONAL. EXPERIMENTS
Table 5: The off between quality and cost in the x-update step. For complex nonlinear tasks such as nonlinear deblurring, increasing the number of gradient steps improves reconstruction qual- ity but also increases computational cost. All experiments conducted 10 samples using RTX 6000 Ada GPU.
Super Resolution 4x
Nonlincar Blur
K
S
SSIM
LPIPS
SSIM
29.627
0.837
0.260
1.444
25. 101
0.711
0.367
8.711
2
29.675
0.838
0.259
1,778
27.275
0.780
0.302
17.284
29.678
0.838
0.259
2.113
27.914
0.800
0,285
26.766
740
0.839
0.256
1.569
25.491
0.725
0.347
10.937
3
29.734
0.839
0.256
2.092
27.079
0.785
0.287
26.888
29.734
0.839
0.256
2.608
27.546
0.801
0.274
38.314
0.838
0.252
1.794
0.734
15.707
5
29.737
0.838
0.252
2.641
26.706
0.777
0.267
39.492
0.838
0.252
0.786
63.044
Effectiveness of K ADMM iterations and S gradient steps. We performed ablation study to evaluate the influence both the number ADMM iterations (K) and the number gradient steps (5) in the x-update stage. We tested K (2, and S (1,3,5) across both linear and
26

---

Published as a conference paper at ICLR 2026
Table 6: Sensitivity analysis of the main hyperparameters for Super resolution 4x, evaluated on 10 FFHQ images. The table shows the performance while sweeping the ADMM penalty and the constraint radius e. The results demonstrate that our method is robust, with performance remaining remarkably stable across wide range of values, which reduces the need for extensive hyperparam- eter tuning.
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
10
27.546
0.783
0.339
0
29.739
0.839
0.255
100
29.614
0,836
0.267
0.01
29.739
0.839
0.255
200
29.740
0.839
0.256
0.05
29.740
0.839
0.256
500
29.565
0.828
0,260
0.1
29.740
0.839
0.256
1000
29.363
0.816
0.276
1
29.726
0.839
0.258
Table 7: Performance of the hybrid pixel-latent schedule with varying Owiich values for 4x super- resolution on 10 FFHQ images. The schedule performs correction in pixel space when > switch and in latent space otherwise. The results, measured on an RTX 6000 Ada GPU, show that a bal- anced approach soswith 1.0 for the linear task) is more effective than purely pixel-space (< 0.0) or purely latent-space (> 10.0) correction strategy.
PSNR
SSIM
LPIPS
Run-time (s)
0.0
24.283
0.553
0.469
3.082
0.2
27.185
0.681
0.374
11.727
1
28.809
0.793
0.302
38.014
5
28.828
0.791
0.306
73.138
>10.0
28.819
0.791
0.307
90.646
nonlinear tasks. As shown in Table 5, increasing K consistently improves performance for both linear tasks (c.g., super-resolutionn and nonlinear tasks (c.g., nonlinear deblurring), though naturally at the cost of higher computation. This suggests that the refinement offered by additional ADMM iterations is broadly beneficial regardless of task difficulty.
In contrast, the effect of increasing the number of gradient steps S is task-dependent. For linear tasks, the performance gain is marginal relative to the additional run-time. However, for nonlinear tasks, the reconstruction metrics steadily improve S increases, indicating that additional gradient refinement helps the solver locate more accurate correction points in challenging settings. Overall, both K and S present a clear quality-cost trade-off, with K providing general improvements and S offering additional benefits especially for complex nonlinear problems.
Hyperparameter Robustness. We investigate the robustness of our method its main hyperpa- rameters. Table 6 shows the results for the super-resolution task when sweeping the ADMM penalty p. and the constraint radius The performance remains stable across wide range of values for each parameter. This highlights a key advantage of FAST-DIPS: it not sensitive to fine-tuning and delivers strong results with default settings, enhancing its practicality and ease of use.
Hybrid Schedule Trade-off. our hybrid pixel-latent framework, the Tswitch parameter deter- mines the point at which the correction process transitions from pixel space to latent space. Table illustrates the resulting trade-off between performance and run-time. Performing the initial cor- rection steps in pixel space (o switch > 0) provides a fast and effective rough update, significantly reducing the overall computation time. The subsequent switch to latent-space updates allows for more stable, ene-ggained corrections that respect the generative manifold. This hybrid strategy proves highly effective, and an intermediate awwitch value offers an optimal balance between speed and reconstruction fidelity.
Experiments with non-Gaussian noise. Figure 4 and Table evaluate FAST-DIPS under Pois- son measurement noise with rate Apoisson = 1, showing that our method remains accurate and perceptually faithful beyond the additive white Gaussian noise (AWGN) setting. The robustness arises from replacing a parametric likelihood with set-valued surrogate: at each diffusion level, curd-constrained proximal problem that enforces feasibility within a
we solve a denoiser-anchool measurement-space feasibility ball. This set-valued constraint robust noise miscalibration and
27

---

Published as a conference paper at ICLR 2026
JPEG Quality
Measurement
Run 0
Reference
Measurement
Reference
Q-25
Q-10
Qe5
Figure 5: Qualitative results for JPEG restoration FFHQ using FAST-DIPS with differentiable surrogate operator. We display the mcasurement, reconstruction, and the ground-trune reference across three compression levels: JPEG Quality 5, 10, and 25.
Table 8: Quantitative evaluation of JPEG restoration FFHQ across JPEG quality factors 5, 10, and 25.
Table 9: Quantitative evaluation of high-resolution (512x512) Gaussian deblurring on FFHQ using 10 samples, conducted on RTX 6000 Ada GPU.
JPEG Quality
PSNR
SSIM
LPIPS
Method
(s)
Latent DAPS
28. 28.308
0.809
0.428
580. 664
10
29.343
0.834
0.267
Ours([aatent
31.438
0.852
0.356
247.399
remains effective empirically beyond the AWGN setting. Our analytic step-size rules yield stable optimization across tasks, supporting practical insensitivity corruption type.
Extension to Non-Differentiable Operators. To address the applicability of FAST-DIPS to non- differentiable degradations, we evaluated our framework on JPEG restoration. While the standard JPEG compression pipeline involves non-differentiable quantization step, it can be effectively handled using a differentiable surrogate (Reich et al. (2024)).
We applied FAST-DIPS using this differentiable surrogate to guide the restoration process under measurement noise 3 = 0.05. Qualitative results are presented in Figure 5, demonstrating that our method effectively suppresses blocking artifacts and restores high-frequency details. Quantitative metrics in Table further confirm competitive reconstruction performance. These results suggest that FAST-DIPS remains highly effective for formally non-differentiable problems, provided differ- entiable proxy of the forward operator is available.
High-resolution image data. We further conducted high-resolution experiments on the FFHQ dataset 512x512 resolution in the latent setting, going beyond the standard 256x256 regime. As shown in Table 9, our method improves PSNR, SSIM, and LPIPS compared to Latent DAPS- the most recent state-of-the-art latent diffusion-based inverse problem solver-while also achieving approximately 2.3x faster run-time. These results suggest that our hybrid approach can handle high-resolution inputs effectively and benefit from efficient computations in the latent space.
Qualitative Results. Figures 6-22 provide additional qualitative samples for comprehensive set of eight problems on FFHQ and ImageNet datasets. These results visually demonstrate the high- quality and consistent reconstructions achieved by both the pixel-space (FAST-DIPS) and latent-space (Latent FAST-DIPS) versions of our method.
A.9 FUTURE WORK AND LIMITATIONS
Our proposed method, FAST-DIPS, provides a robust framework for solving inverse problems, and its hyperparameter stability opens up several promising directions for future work. The framework
28

---

Published as a conference paper at ICLR 2026
defined by few key hyperparameters (p, switch)), and as shown in the additional experiments (Tables and 7), exhibits robustness across wide range of their values, enhancing its practical usability. Among these, the ADMM penalty parameter While our experiments show stable performance with fixed value, integrating adaptive penalty selection strategies could further improve convergence and robustness. Similarly, exploring op- timal or adaptive schedule for the hybrid switching point Jswitc remains another interesting avenue for research.
be considered the most influential.
Despite these strengths and opportunities, we also acknowledge a primary limitation of the current framework: its dependency on differentiable forward operators. FAST-DIPS is "adjoint-free" free""""""""" in the sense that it does not require a hand-coded adjoint operator. However, its efficiency heavily relies on automatic differentiation to compute VJP and JVP needed for the analytic step size a*. This implic- itly assumes that the forward operator A is (at least piecewise) differentiable. For problems involv- ing non-differentiable operators or black-box simulators where gradients are unavailable, our current approach cannot be directly applied. Future work could explore extensions using zeroth-order opti- mization techniques or proximal gradient methods that can handle non-differentiable terms.
29

---

Published as a conference paper at ICLR 2026
COFT Xajc
Xoj
X
D(zo))
D(z,)
Figure 6: Phase Retrieval trajectory under FAST-DIPS and Latent FAST-DIPS with intermediate iter- ates along the diffusion schedule.
30

---

Published as a conference paper at ICLR 2026
Figure 7: Additional qualitative results for Phase Retrieval We show Measurement, Reconstruc- tion, and Reference for both FAST-DIPS and Latent FAST-DIPS across four runs (Run 0-3).
BAST DIPS
Latent BAST-DIPS
Reference
Run
Run
Rum 2
Run 3
Reference
Ran
Run
Run
Run
Figure 8: Qualitative results for Phase Retrieval on ImageNet dataset. We show Measurement, Reconstruction, and Reference for both FAST-DIPS and Latent FAST-DIPS across four runs (Run 0-3).
31

---

Published as a conference paper at ICLR 2026
Measurement
Reconstruccion
Refernce
Figure 9: Additional qualitative results for and Reference are shown for FAST-DIPS and Latent FAST-DIPS.
x4, Measurement, Reconstruction,
Messumment
Reconstruction
Reference
Figure 10: Qualitative results for Super- Resolution x4 ImageNet dataset. Measurement, Re- construction, and Reference are shown for FAST-DIPS and Latent FAST-DIPS.
Mracarement
Recanstruction
Figure 11: Additional qualitative results for Inpaint(box). We display Measurement, Reconstruc- tion, and Reference for FAST-DIPS and Latent FAST-DIPS.
RAST-DIPS
Latent FAST-DIPS
Messuremett
Reference
aafondeamun
Figure 12: Qualitative results for Inpaint(box) on ImageNet dataset. We display Measurement, Reconstruction, and Reference for FAST-DIPS and Latent FAST-DIPS.
32

---

Published as a conference paper at ICLR 2026
Measurement
Reconstraction
Reference
Figure 13: Additional qualitative results for Inpaint(random). Measurement, Reconstruction, and Reference with FAST-DIPS and Latent FAST-DIPS.
PAST-DIPS
Latent FAST-RIP
Reconstruttion
Figure 14: Qualitative results for Inpaint(random) on ImageNet dataset. Measurement, Recon- struction, and Reference with FAST-DIPS and Latent FAST-DIPS.
FAST-DIPS
Latest FAST DIPS
Reconstruction
Reference
Figure 15: Additional qualitative results for Gaussian deblurring. We show Measurement, Recon- struction, and Reference for FAST-DIPS and Latent FAST-DIPS.
FAST-DIPS
Latent BAST-DUPS
Measurement
Reconstruction
Reference
Figure 16: Qualitative results for Gaussian deblurring on ImageNet dataset. We show Measure- ment, Reconstruction, and Reference for FAST-DIPS and Latent FAST-DIPS.
33

---

Published as a conference paper at ICLR 2026
FAST-DIPS
Latest FAST DIPS
Mossurement
Rnconstruction
Reference
Figure 17: Additional qualitative results for Motion deblurring. Measurement, Reconstruction, and Reference provided for FAST-DIPS and Latent FAST-DIPS.
Messarmment
Reconstruction
Reference
Figure 18: Qualitative results for Motion deblurring on ImageNet dataset. Measurement, Recon- struction, and Reference are provided for FAST-DIPS and Latent FAST-DIPS.
FAST-BIPS
Labett FAST-DIPS
Measurement
Reconstruction
Reference
Figure 19: Additional qualitative results for Nonlinear deblurring. We present Measurement, Re- construction, and Reference for FAST-DIPS and Latent FAST-DIPS.
Measurement
Reconstruction
Referince
Figure 20: Qualitative results for Nonlinear deblurring on ImageNet dataset. We present Measure- ment, Reconstruction, and Reference for FAST-DIPS and Latent FAST-DIPS.
34

---

Published as a conference paper at ICLR 2026
FAST-DIPS
Labent EAST DIPS
Measurement
Reference
Figure 21: Additional qualitative results for High Dynamic Range. Measurement, Reconstruction, and Reference for FAST-DIPS and Latent FAST-DIPS.
FAST-DIPS
Latent FAST-DIPS
Measurement
Reconstruction
Figure 22: Qualitative results for High Dynamic Range ImageNet dataset. Measurement, Re- construction, and Reference for FAST-DIPS and Latent FAST-DIPS.
35


Reviews:
[{"review_id": "kIIt5pi3aU", "reviewer": "Reviewer_mDLA", "summary": "This paper introduces FAST-DIPS, a training-free solver for diffusion-prior inverse problems, including those with nonlinear forward operators. The method's core is a hard-constrained proximal correction via an adjoint-free ADMM. This approach replaces costly inner MCMC loops or iterative optimization with an analytic step size, computable from one VJP and one JVP. Experiments across eight linear and nonlinear tasks demonstrate comparable or superior quality to state-of-the-art baselines, while achieving faster runtimes.", "questions": "Please see the Weaknesses for the details.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1.  The method’s adjoint-free design is both interesting and practical, eliminating the need for hand-coded adjoints. By relying on standard automatic differentiation (VJP/JVP), the framework is directly applicable to a broad class of nonlinear inverse problems—a setting that remains challenging for many competing methods.\n\n2.  The framework is evaluated extensively and demonstrates a substantial speedup over baselines such as DAPS by replacing costly inner MCMC loops with an analytic step size, while maintaining—often even improving—reconstruction quality.", "weaknesses": "1.  One concern is the method's reliance on differentiable forward operators. The entire framework is built upon the availability of VJP and JVP, making it inapplicable to common non-differentiable degradations such as JPEG restoration or quantization. This may limit its utility for many real-world degradation types.\n\n2.  Another concern is the need for task-specific hyperparameter tuning. The authors show that key parameters were set to different values for each of the eight tasks. This implies that users must perform a new, potentially expensive hyperparameter search for any novel problem, undermining its practicality as a plug-and-play solver. Moreover, I wonder if this hyperparameter selection is essential for other baselines as well. If so, we need to include the cost of hyperparameter selection to show practical speed-up in the usage of the proposed method.\n\n3. Moreover, I am skeptical about the method's scalability to higher resolutions. The computational cost of the VJP/JVP, especially for the latent variant, which requires backpropagation through the decoder, was manageable for the $256 \\times 256$ experiments. However, this computation and memory overhead is likely to become a significant bottleneck as resolution increases, potentially limiting the framework's applicability to large-scale problems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gfBpKNw0yE", "reviewer": "Reviewer_6kKx", "summary": "This paper introduces FAST-DIPS, a training-free diffusion-prior inverse problem solver. The core contributions are: (1) an adjoint-free correction step that (2) enforces a hard-constrained likelihood ($||\\mathcal{A}(x)-y||\\le\\epsilon$) using (3) an ADMM formulation, where the primal update is solved efficiently with an analytic, non-iterative step size derived from a local quadratic model. The method is further extended to latent and hybrid (pixel/latent) settings.", "questions": "1. How was the hyperparameter $\\epsilon$ (the radius of the credible set) determined for each of the eight experiments?\n\n2. The paper claims to use no inner MCMC, but the ADMM iterations ($K=3$–$5$) with $S=1$ descent appear to form mini-loops. Please clarify if I have misunderstood this point.\n\n3. For the latent variant, have you analyzed the computational cost of the JVP $J_{\\mathcal{A}\\circ\\mathcal{D}}(z)g$?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The adjoint-free ADMM formulation with analytic step sizes (via VJP/JVP or finite-difference approximations) is a clever way to minimize engineering overhead while ensuring efficient and deterministic updates.\n\n2. The paper provides strong local guarantees, including exact minimization of quadratic models (Proposition 3), KKT satisfaction at fixed points (Proposition 4), and descent properties with backtracking.", "weaknesses": "1. My main concern is that, while adjoint-free, the method still requires autodiff through $\\mathcal{A}$, and the latent mode incurs repeated decoder calls. In my view, the engineering benefit of avoiding adjoints is somewhat offset by the computational cost of automatic differentiation through $\\mathcal{A}$.\n\n2. The method’s hard constraint ($||\\mathcal{A}(x)-y||\\le\\epsilon$) shifts the tuning burden from the likelihood weight to the credible set’s radius $\\epsilon$. Although this is briefly acknowledged in the limitations section, the paper should more thoroughly discuss how this critical hyperparameter is selected.\n\n3. Experiments are limited to FFHQ (faces only), which lacks diversity; no tests are conducted on broader datasets such as CelebA-HQ, LSUN, or natural images (e.g., ImageNet subsets). This raises concerns about generalization to non-face domains or higher resolutions.\n\n4. The complete algorithm, as presented in the appendix, is somewhat complex, making the method less elegant. Moreover, the writing could be improved for better clarity and flow.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PMa0CKOXOc", "reviewer": "Reviewer_pSwc", "summary": "This paper proposes FAST-DIPS, a fast and training-free solver for diffusion-prior inverse problems. The key idea is to perform a hard-constrained likelihood correction at each diffusion step through an adjoint-free ADMM scheme with analytic step size computation, thereby avoiding the need for hand-coded adjoints or inner MCMC loops. \nExperimental results across multiple linear and nonlinear inverse problems demonstrate that FAST-DIPS achieves comparable or superior reconstruction quality while reducing runtime compared to state-of-the-art training-free baselines.", "questions": "1.The performance of the baseline algorithms in Table 1 differs from that reported in their original papers, even under the same experimental settings. For instance, the SITCOM paper reports a PSNR of 30.68 for the SR task, whereas Table 1 reports 29.555. A similar discrepancy is also observed for the DAPS algorithm. While Figure 3 effectively demonstrates the superiority of the proposed method under the same runtime, I would appreciate clarification regarding these inconsistencies.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Clear and principled framework**:\nThe paper presents FAST-DIPS, a training-free and adjoint-free framework for diffusion-prior inverse problems, offering a clean and principled alternative to existing plug-and-play or posterior-sampling approaches.\n\n2. **Broad applicability and ease of use**:\nThe method supports both pixel-space and latent-space diffusion models through an adjoint-free design, making it versatile and easy to apply to a variety of inverse problems without hand-crafted adjoints or retraining.\n\n2. **Strong empirical performance**:\nThe proposed analytic-step ADMM correction eliminates the need for hand-crafted adjoints or inner MCMC loops, making the method broadly applicable to both linear and nonlinear operators while achieving 5×–25× faster inference.", "weaknesses": "1. **Presentation issue**:\nTable 1 exceeds the page width and is difficult to read in its current format. The authors should consider reformatting or splitting the table across pages to improve readability and compliance with ICLR formatting guidelines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0EawWhJ5pg", "reviewer": "Reviewer_9UpX", "summary": "This paper introduces FAST-DIPS which aims to improve both the speed and accuracy of existing diffusion-prior solvers. The core contribution is a two-stage update process applied at each step of the reverse diffusion chain: \"analytic step\" and \"hard-constrained likelihood correction\" step. The experimental results show that FAST-DIPS consistently outperforms state-of-the-art methods in terms of reconstruction quality and/or computational time.", "questions": "1. Could the author please clarify the practical limitations of the adjoint-free formulation? For which classes of forward operators $A$ does the pre-computation of $(A A^T)^{-1} A$ become a bottleneck that outweighs the per-iteration speed-up?\n\n2. Though the experiment section has included various inverse problems, the selection of the dataset is very limited. Can the author please add at least one more dataset other than human face to benchmark the performance?\n\n3, The hybrid schedule introduces a switching threshold $\\sigma_\\text{switch}$. How is this parameter selected in practice, and how does performance vary with its value?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed combination of an analytic data consistency step with the hard-constrained likelihood correction is novel and appears highly effective. \n2. The paper provides compelling empirical evidence of its superiority, often delivering higher quality in a fraction of the time required by competing methods.\n3. The paper is theoretically sound and clearly structured.", "weaknesses": "1. The Method section is dense and overly technical, relying heavily on acronyms and mathematical derivations with minimal intuition.\n\n2. The \"adjoint-free\" claim could be misleading. The method avoids computing the adjoint operator $A^T$ at each iteration by pre-computing a term involving $(A A^T)^{-1}$. This is only advantageous for a specific class of operators where this matrix is easy to compute and invert. \n\n3. The choice of $\\epsilon$ in the hard constraint is not well explained\n\n4. The extension of the method to non-linear problems like phase retrieval is handled by linearizing the forward operator at each step. This is a reasonable approach, but the paper provides very little detail or justification for it. \n\n5. The exclusive use of FFHQ restricts the generality of the conclusions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)