Solving Inverse Problems using Diffusion with Iterative Colored Renoising
Authors: Matthew C Bendel
Saurav K Shastri
Rizwan Ahmad
Philip Schniter
Venue: Accepted by TMLR ()

Abstract: Imaging inverse problems can be solved in an unsupervised manner using pre-trained diffusion models, but doing so requires approximating the gradient of the measurement-conditional score function in the diffusion reverse process.
We show that the approximations produced by existing methods are relatively poor, especially early in the reverse process, and so 
we propose a new approach that iteratively reestimates and ``renoises'' the estimate several times per diffusion step.
This iterative approach, which we call Fast Iterative REnoising (FIRE), injects colored noise that is shaped to ensure that the pre-trained diffusion model always sees white noise, in accordance with how it was trained.
We then embed FIRE into the DDIM reverse process and show that the resulting ``DDfire'' offers state-of-the-art accuracy and runtime on several linear inverse problems, as well as phase retrieval.

Published in Transactions on Machine Learning Research (08/2025)
Solving Inverse Problems using Diffusion with Iterative Col- ored Renoising
Matthew C. Bendel
bendel. Saiikkeeiiick
Dept. Electrical and Computer Engineering, The Ohio State University, Columbus, OH, USA
Saurav K. Shastri
shastri. shastri. 19w.wnckeyemaiil.......... osw. Depl. Electrical and Computer Engineering, The Ohio State University, Columbus, OH, USA
Rizwan Ahmad
rizwan, www.aannahmmd@osumccccom
Dept. Biomedical Engineering, The Ohio State University, Columbus, OH, USA
Philip Schniter
schniter. @osu. edu
Dept. Electrical and Computer Engineering, The Ohio State University, Columbus, OH, USA
Reviewed OpenReview: https:// o/pprrveeppereveeeu. net/ formm id= R2v8Fc(DPM
Abstract
Imaging inverse problems can be solved in an unsupervised manner using pre-trained dif- fusion models, but doing so requires approximating the gradient of the measurement- conditional score function in the diffusion reverse process. We show that the approximations produced by existing methods are relatively poor, especially early in the reverse process, and propose
approach that iteratively reestimates and "renoises" the estimate sev- eral times per diffusion step. This iterative approach, which we call Fast Tterative REnoising (FIRE), injects colored noise that is shaped to ensure that the pre-trained diffussin model always sees white noise, in accordance with how was trained. We then embed FIRE into the DDIM reverse process and show that the resulting "DDfire" offers state-of-the-an accuracy and runtime on several lincar inverse problems, as well as phase retrieval. Our implementation available al https: https:///gthubb
1 Introduction
Diffusion has emerged as powerful approach to generate samples from complex distribution (Sohl- Dickstein et al.) 2015 Song & Ermon 2019 Ho et al. 2020 Song et al.) 2021b a). Recently, diffusion has also been used solve inverse problems (Daras al. 2024], where the goal to recover my ~ Po from incomplete, distorted, and/or noisy measurements y an unsupervised manner. There, diffusion model is trained generate samples from Po and, test time, the reverse process modified to incorporate knowledge of the measurements y, with the goal of sampling from the posterior distribution p(zoly).
When implementing the reverse process, the main challenge is approximating the conditional score function V. In (x (y) at each step 4, where æ; an Additive-wiile-Gaassiinnn-- version of wo E Rd, and E Rm treated as a draw from a likelihood function p(y æo). (See Sec. 2 for more details.) Many existing approaches fall into one of two categorics. The first. uses Bayes rule write Va In pr (zx ly) V. pn (mp)(x)) + V. In (1 22), where an approximation of Va In (x(2)) is readily available from the po-trained diffusion model, and then approximates Va In (r (x)) (e.g., Chung et al.
(AWGN) corrupted and scaled
2023a
Song

---

Published in Transactions on Machine Learning Research (08/2025)
al. 2023
The second approach uses Tweedic's formula
Efron 2011 to write
V Inpo( Ex (c,, y)
(1)
of
and then approximates the conditional denoiser E( Zt. ye (c.g., Kawar al. (2022a) Wang et al. (2023
Zhu al. 20233; Chung et al. (2024a)).
A key shortcoming of the aforementioned approaches is that their conditional-score approximations not very accurate, especially carly in the reverse process. For the methods that approximate Ef y). we can assess the approximation quality both visually and via mean-square error (MSE) or PSNR, since the exact E æt, y) minimizes MSE given 24 and y. For the methods that approximate V. p we can compute their equivalent conditional-denoiser approximations using
2 y) = xt + of V in (x,
which follows from 1- Figure shows Fa .t 01, -approximations from the DDRM
(Kaaar et al. 2022a
DifIPIR (Zhu et al. 2023). DPS (Chung et al. 2023a and DAPS Zhang et al. (2025 solvers at times 25%
50%, and 75% through their reverse processes for noisy box inpainting with By 0.05. The approximations
show unwanted artifacts, especially early in the reverse process.
DDRM
DifIPIR
DPS
DAPS
DDfire
PSNR: 16.17
PSNR 21.16
PSNR: 11.64
PSNR: 20.35
PSNR: 24.73
25% NFEs
PSNR: 20.14
PSNR 22.13
PSNR: 22.39
PSNR: 22.84
PSNR 25.81
50% NFEs
PSNRY
PSNR:
PSNR 22.90
PSNR: 23.60
PSNR
PSNR:
75% NFEs
FIRE
Figure 1: Left column: True æu, noisy bax inpainting y, and 50-iteration FIRE approximation of E( }
Other columns: Approximations of E( æ,, y) at different t (as measured by % NFEs). Note the over- smoothing with DDRM and DPS. Additionally, note the cut-and-paste artifacts of DifIPIR and DAPS.
In this paper, we aim to improve the approximation of (F(ool=,,)) v) at each step L. Since have observed that methods using a single neural function evaluation (NFE) to approximate E( y or V In (æ, perform poorly, we consider using several NFEs. In particular, we propose an iterative approach to ap- proximating E( wr リ} that we call Fast Iterative REnoising (FIRE). FIRE is like a plug-and-play (PnP) algorithm (see the PnP survey Ahmad et al. (020)) in that iterates unconditional denoising with lincar estimation from 2, and y. But unlike traditional PnP algorithms, which aim to minimize an implicit loss function, FIRE is based on colored renoising, where carefully designed colored Gaussian noise is added to the linear-sstimation output that the denoiser's input error approximates AWGN. Since the denoiser is trained to remove AWGN, renoising aims to minimize the distribution shift experienced during inference. Figure shows the 50-iteration FIRE approximation to E( Ør. y = E{ for noisy box inpainting, which is relatively free artifacts. We propose two versions of FIRE: one that handles AWGN-corrupeed lincar measurement models like (6), and one that handles generalized-
inverse problems such as phase retrieval (Shechtman et al. 2015). Poisson regression (Figueiredo Bioucas- Bioucas-Dias 2010), and dequantization (Zymnis et al. 2010). The latter is based expectation propagation (EP) (Minka] 2001 Bishop 2007
2

---

Published in Transactions on Machine Learning Research (08/2025)
We then embed FIRE into the DDIM diffusion reverse process (Song et al. 2021a), yielding the "DDfire" posterior sampler. Figure 1 shows examples of DDfire's o/ y) approximation when run noisy box inpainting. DDfire's B Ft. y) approximations have fewer structural artifacts and higher PSNR than its competitors.
The contributions of this work are as follows:
We propose linear FIRE, an iterative approach to solving AWGN-corruplee linear inverse problems that injects carefully designed colored Gaussian noise in order to whiten the denoiser's input error.
2. We theoretically analyze the convergence of linear FIRE.
We use expectation propagation (EP) to extend linear FIRE to generalized- inverse problems.
Combining FIRE and DDIM, we propose the DDfire diffusion posterior sampler.
5. We demonstrate the excellent accuracy of DDfire on several imaging inverse problems: box inpaint- ing, Gaussian and motion blur, super-resolttin, and phase retricval.
2 Background
Given training data drawn from distribution po, diffusion models corrupt the data with ever-increasing amounts of noise and then learn to reverse that process in a way that can generate new samples from Po- In the main text, we assume the variance-exploding (VE) diffusion formulation (Song et al. 2021b), whereas Appendix A provides details on the variance- preservvngg (VP) formulation, including DDPM (Ho et al. 2020) and DDIM (Song al. 2021a
The VE diffusion forward process can be written as a stochastic differential equation (SDE) dæ /dlo² (t))/(t))/ dt dw over t from to T, where (t) is a variance schedule and dw is the standard Wiener process (SWP) (Song et al., 2021b). The corresponding reverse process runs the SDE dæ = - (t) a In (x( di + /dlo² (t)l dt dwr backwards over t from T to O, where p-()) the marginal distribution æ at and dw is the SWP run backwards. The "score function" V., In p4 (x) can be approximated using a deep neural network
(DNN) sa(x,, t) trained via denoising score matching (Hyvärinen 2005).
In practice, time is discrctized to t 10.1..... T), yielding the SMLD from Song & Ermon 2019 whose forward process, T++1 2, + Volu ofw, wi, with iid {w,} NNO, and of = 0, implics that
x v + dec,, a~ I)
(3)
for all 1 E 1,,1, 1o,1,..,,TT, The SMLD reverse process then uses i.idd {n+} ~ N(0, I) in
= =011 + (02+1 - of) In (npe)) 0f(0771 - -o))
n4+1
(4)
2241
To exploit side information about To. such as the measurements y in an inverse problem, one can simply replace Pc(-) with pe(:1y) in the above equations (Song al., 2021b). However, most works aim to avoid training y-dependent approximation of the conditional score function Va In pe Rather, they take "unsupervised" approach, where s0(2;; () R Va In (xx)) is learned during training but y is presented only at inference (Daras et al. 2024). In this case, approximating Va In (npp(pl))) is the key technical challenge.
There are two major approaches to approximate Va In (e The first uses the Bayes rule to write Va la (r (y) = Va pn + Va. pn and then replaces V. In (x with the score approximation so s0 t). But the remaining term, V In Pr (y)zz), is intractable because pr(y)<x;) = f p(y))=)))(()) (o)/=)((()(z)) dæy with unknown p(z, æ2), and so several approximations have been proposed. For example, DPS (Chung) et al. 2023a) uses p(zglz>) (z;) 5(zg- Zojp), where Zop. is the approximation of E(xo|æ,) computed from 80 s0 1) using Tweedie's formula:
znjt Ez + of s0 se t).
(5)
3

---

Published in Transactions on Machine Learning Research (08/2025)
Similarly, IIGDM (Song et al. 2023 uses p(Zu) (Z Zoje: (TI) with some Se. However, drawback to both approaches is that they require backpropagation through (o(:;), which increases the cost of generating single sample. In Fig. 4
we show that DDfire offers a 1.5x speedup over DPS at an equal number of NFES.
The second major approach to approximating V In y) uses ! with Ef x y) approximated by a quantity that we'll refer to as Fojt.y For example, with AWGN-corrppeed linear measurements
= Axo + Jy E R", w ~ N0,11),
(6)
DDNM (Wang ct al. 2023) approximates E/ n y) by first computing Tn. from 5 and then performing
the hard data-consistencyy step Fole.y= A y + (I A+ A oojp where (.)+ is the pseudo-inverse. DDS
(Chung et al. 2024ab and DiffPIR (Zhu al. 2023 instead use the soft data-consistency step Bott. =
arg min ly AæI 7 Tojell with some Tt O. DDRM (Kawar et al., 2022a) is a related technique
that requires singular value decomposition (SVD), which is prohibitive in many applications.
There are other ways to design posterior samplers. For example, Laumont et al. 2022 and Jalal et al. (0011) use Langevin dynamics. RED-diff (Mardani et al. 2024) and SNORE (Renaud et al. 2024) inject white
noise into the RED algorithm (Romano et al. 2017 whose regularizer's gradient equals the score function
RReehorst & Schniter 2019). Bouman & Buzzard (2023); Coeurdoux et al. (2024); Wu et al. (2024); Xu &
Chi (0024) ;
Zhang et al. (2025) use Markov-chain Monte Carlo (MCMC) in the diffusion reverse process.
3
Approach
We aim to accurately approximale the conditional denoiser E[ ar; y) at each step of the diffusion reverse process. We treat the standard linear model (SLM) 6 first and the generalized linear model (GLM) later.
3.1 Fast iterative renoising (FIRE) for the SLM
In this section, we describe the FIRE algorithm, which approximates E (o Minit y} assuming y from (6 and Tink 26 + CinieC with c ~ No I) and some iinit > FIRE performs half half-quadratic splitting (HQS) PnP with a scheduled denoising variance o², similar to DPIR from Zhang et al. (2021). : but injects colored noise to ensure that the error in the denoiser input remains white. It is beneficial for the denoiser to see white input error during inference, because it is trained to remove white input error. The basic FIRE algorithm iterates the following steps N 2 times, after initializing r + Tinit and a t- Dinit:
S1) Denoise r assuming AWGN of variance o², giving Æ.
82) MMSE estimate zo given from 6 and the prior xo ~ N N(, with some v > 0, giving a.
S3)
Update the denoising variance r² via 2² +- 02/p with some > 1,
S4) Update + 2 using colored Gaussian noise created to ensure Cov(r - =o} = I2
$1 $3 are essentially DPIR with regularization strength controlled by and a geometric denoising schedule with rate controlled by p, while S4) injects colored noise. In contrast, other renoising PnP approaches like
SNORE (Renaud et al. 2024) inject white noise. Section 4.1 examines the effect removing c or replacing it with while noise. Next we provide details and enhancements of the basic FIRE algorithm.
In the sequel, we use "d)(x,o)" denote a neural-net approximation of the conditional-mean denoiser E(xo|æ>) of = Zn DE with N/O.,) I). Given score function approximation sø t) z V (n
discussed in Sec. 2 the denoiser can be constructed via (5) as
do x + o² ooso(z,t) t) with t such that 0,
(7)
When FIRE estimates Do from the measurements y and the denoiser output E, it employs a Gaussian approximation of the form 2n W(E,vI) similar to DDS, DiffPIR, and prox-based PnP algorithms. But it differs in that v is explicitly estimated. The Gaussian approximation æn ~ NN(x, vI) is equivalent to
Eo v Vie, e~ 1).
(8)
4

---

Published in Transactions on Machine Learning Research (08/2025)
Suppose To Σ + vibe with I), where My denotes the true error variance. Then 6 and (8)
imply
Bllly Aæ | EX| Amo + oyw- Awo + vii = B[llaw + An may + A
(9)
assuminn independence between e and w. Consequently, unbiased estimate of vo can be constructed
(ly ATI -0)---mm))))
moo)//1A11 A..
(10)
Figure
12
shows that, in practice, the estimate 110
accurately tracks the true error variance 2
x/13/d.
Under and
8 the MMSE estimate of ve from and I can be written as Poor 1994
2 - arg min
200 ly Azll² +
2v 211 =|
1
A (1)
A
of
(11)
Equation
can be computed using conjugate gradients (CG) or, if practical, the SVD = USVT
via
o (1)
"
ST (SSTTyy
S v S S s
(12)
In any case, from { and
the error in @ can be written
2o =
A A
2 v
o 7( A [Axo + ryw) +
v xo -vel -z0 = (AAA A+ 1'
+1))
(0,AA
w
o3
-e)
vv
(13)
and so the covariance of the error & can be written as
Cov mo0 A
AT A
=1) (0PATAA+ + (TT+ 21
1 (
AT A A C. (14) d9
From [1] we see that the crror in E can be strongly colored. For example, in the case of inpainting, where A is formed from rows of the identity matrix, the error variance in the masked pixels equals v, while the error variance in the unmasked pixels equals (1/o² + 1/v) I og. These two values may differ by many orders of magnitude. Since most denoisers are trained to remove white noise with specified variance of o² direct denoising of eor performs poorly, as we show in Sec. 4.1
To circumvent the issues that arise from colored denoiser-input error, we propose to add "complementary" colored Gaussian noise c a~d NO, Σ) to ø so that the resulting r e has an error covariance of I²I, i.e., white error. This requires that.
1 ( + )
Σ -= -
V Diag(A)V for x = =0²
(15)
22/0 +1/0
for S) By setting o² 2 v, we ensure that x 2 Vi, needed for Σ to a valid covariance matrix. In the case that the SVD practical implement, we can generate e using
V Diag(A) E I(,
(16)
In the absence of SVD, we propose to approximate Σ by
m -
(17)
with some Ş 620 0. Note that E agrees with E in the nullspace of A (i.c., when sn = 0) for any &. By choosing
2 -(( vー
1
< =
(18)
22 m /03 + 1/v
5

---

Published in Transactions on Machine Learning Research (08/2025)
Algorithm FIRE for the SLM: E FIRESLM (v, A, By. Pinit: Onitt N.
Require: do(::)) 3), y, A, 8max, By, N, 1, Finit Finit- Also U (Digg()) V if using SVD.
r Pinit and
Oinit
Inútialize
2: for n = 1..... ......N. NN do
3:
I - dø (r. 0) + (o(0). I0,(0
Stochastic denoising > Error variance of a
4:
V + (ly - Azl 2 o2m)/||A|??
T arg min ly Awll²/o5 + ||00 z//2/v
D Estimate wn NN vI) from y N ot I)
o² < max(o²/pm /p, m}
Decrease largel voriance
if have SVD then
8:
A; + o² - (s? o + 1/v) i= 1.......
ct V Diag(A (A) 1/2,, g~ I)
b Colored Gaussian noise
10:
else
11:
{ t (v - (s /0 + 1111))
12:
c c Vo=-v1 V | e, E g~ (0,
b Colored Gaussian noise
13:
r x +
Renoise so that Cov zu} 0²)
14:
return
Σ will also agree with Σ in the strongest measured subspace (i.e., when 8n = Smax). Without an SVD, Smax can be computed using the power iteration (Parlett 1998). Finally, e ~ (N, Σ) can be generated via
[vo2 VEAT ~~ IO, E IR
(19)
Figure 0 shows close agreement between the ideal and approximate renoised error spectra in practice. Next, we provide the main theoretical result on FIRE.
Theorem 1 Suppose that, for any imput r
+ ae with c~ N/00, I), the denoiser output do(r, σ) has while Guussian error with known variance v < o² and independend of the noise w in 0 Then if initialized using Tinit = zg + E with arbitrarily large but finite Onnit and ~ (N,00 I), there exists a > under which converges to the true Zo-
the FIRE iteration $1 54)
Appendix provides a proof. Note that key assumption of Theorem is that the denoiser output error is white and Gaussian. Because this may not hold in practice, we propose to replace [1]] with a "stochastic denoising" step (21), in which AWGN is explicitly added to the denoiser output. As the AWGN variance increasos, the denoiser output becomes closer to white and Gaussian but its signal-to-noise ratio (SNR) degrades. To balance these competing objectives, we propose to add AWGN with variance approximately equal to that of the raw-denoiser output error. We estimate the latter quantity from the denoiser input variance o² by training a predictor of the form
(Ø(a) xe E[Ildo(>o + oe,0) xool2/d),
(00)
where the expectation is over 6 ~~ (O I) and validation images æp Po- Recall that d is the dimension of ao- In our experiments, (.)-) implemented using a lookup table. The stochastic denoising step is then
da σ) + Vo(()), U N(O,).
(21)
Algorithu summarizes the FIRE algorithu for the SLM 6 In App. D we describe minor enhancement
to Alg. that speeds up the MMSE istimation step when CG is used.
3.2 DDfire for the GLM
We now propose to extend the SLM-FIRE from Sec. 3.1 to the generalized linear model (GLM)
y ~ (>)p())))))))
= II
My (, (0.s) with 2o A Axo
(22)
1=1

---

Published in Transactions on Machine Learning Research (08/2025)
MMSE inference
y. By
SLM-FIRE
y
of 20.j ~ (22, from Mj Py12(-120.)
with y = Axo + w and w ~ (O,
2,
Figure High-level overview of GLM-FIRE, which uses EP-style iterations between SLM-FIRE and an MMSE inference stage that involves the scalar measurement channel Py/z-
where Dylz is some scalar "measurement channel." Examples include Pylz(Ul=) = Nu: (<::1=l,03) for phase [[T; oT) dr for dequantization.
retrieval,
>/y! for Poisson regression, and Pylz(vl=) = v+
Our extension is inspired by expectation propagation (EP) Minka 2001 Bishop 2007 and its application to GLMs [Schniter et al. 2016 Meng et al.) 2018). The idea is to iterate between i) constructing "pseudo- measurements" = Ano + w with w ~ N (0.05 using Pylz and an STM-FIRE-constrrue
belief that =o ~ (Z v,I), and then ii) running SLM-FIRE with those pseudlo-messuremeit and updating its belief
on 2n- Figure 2 shows a high-level summary. Details are given below.
To construct the belief 2o, we the SLM-FIRE denoiser output model 2g NVx vI) from ( Because that 2a N(ZVVAA") where z A AE. For simplicity, however, use the white-noise approximation 2p M(z, PzT), where
Azo,
/l Using the scalar belief =o.j M( 22) and the
likelihood model yj Psjz( 2o,)); EP suggests to first compute the posterior mcan E. 3z. 12) - 2, and
variance 1 Eラ= m
3,, F2) A D2 and then pass the "extrinsic" versions those quantities:
o [1/2 1/u,)
-
(23)
back to SLM-FIRE, where they are used to construct the pseudo- measurement model
y = Amo + oyw. w ~ NO,
(24)
In Fig. 13
we show that GLM-FIRE's & accurately tracks the true noise power in D. i.e., |
A
The GLM-FIRE algorithm is summarized as Alg. 3 When Pyz((u2z) = ( is it straightforward to show that y = y and oy = og for any and V2, in which case GLM-FIRE reduces to SIM-FIRE.
3.3 Putting FIRE into diffusion
detailed the FIRE algorithms for the SLM 6 and the GLM 22 respectively, In both cases, the FIRE algorithm approximates E( given the measurements y and the side-information = ID + oe, where ~ (O, I). Thus, recalling the discussion in Sec. 2 FIRE can be used in the SMLD reverse process as an approximation of Ef2ol2, y} by setting r = x, and σ = Gt-
Scctions 3.1 and 3.2
Instead of using SMLD for the diffusion reverse process, however, we use DDIM from Song et al. (2021))
which can be considered as generalization of SMLD, In the sequel, we distinguish the DDIM quantities by writing them with subscript k. As detailed in App. C DDIM is based the model
3t z. + TKE EK N/ I).
(25)
for ==1 1,.... K, where {o a specified sequence of variances. The DDIM reverse process iterates
Øk-1 h4 + 9k E{zul Bk fA + SRT
(26)
0h ( oz
02
2
Sk Mddim
of
hI
9h
bx
(27)
over k K,,,,,2,1, starting from øK NNO. x I), using i.dd (nz [ A I) and some 7]dimm
When Madim and K T, DDIM reduces to SMLDD But when 7ddim 0, the DDTM reverse process
26

---

Published in Transactions on Machine Learning Research (08/2025)
is deterministic and can be considered discretization of the probability-flow ODE Song al. 2021a which can outperform SMLD when the number of discretization steps K is small (Chen et al. 2023
For a specified number K of DDIM steps (which we treat as tuning parameter), we set the DDIM variances [ k== as the geometric sequence
2 o
max
1= K
(28)
min
min
for some min and max that are typically chosen to match the minimum and maximum variances used to train the denoiser d-(-.) or score approximation sa(:.-). So for example, if sa(:-)) was trained over the DDPM
steps E {l,.... ,I....,T) [ for T 1000, then we would set o (1 ai)/a and max (1 /1000)/01000 with 04; see (37 for additional details.
Next we discuss how we set the FIRE iteration schedule (Na [ and variance-deccress--attto >> 1. In doing so, we have two main goals:
G1) Ensure that, at every DDIM step k, the denoiser's output-crror variance at most V/thresh at the final FIRE iteration, where Uthresh is some value to be determined.
G2) Meet fixed budget of Noot & N& total NFEs.
Note that, because the denoiser's output-error variance increases monotonically with its input-crror variance, can rephrase
G1)
as
G1+) Ensure that, at every DDIM step k the denoiser's 's input-error variance is at most oiresh at the final FIRE iteration, where othresh is some value to be determined.
Although ohress could be luned directly, it's not the most convenient option because a good search range can be difficult to construct. Instead, we tune the fraction 8 [0, 1) of DDIM steps k that use a single FIRE iteration (i.c., that use N₂ 1) and set o thresh at the DDIM variance of of the first reverso-process step k that uses single FIRE iteration, i.e., 1 1 +(K - 1)8] kehresh (Note that kithresh = 1 when d 0 and kthresh = K - for 8 z 1.) All subsequent DDIM steps h < kehresh will then automatically satisfy G1*) because of decreases with k.
To ensure that the earlier DDIM steps kihreshh also satisfy G1*) we need that oi/pNe-1 S o since of the denoiser input-error variance at the first FTRE iteration and pe is the denoiser input-error variance at the last FIRE iteration. For fixed p > 1, we can rewrite this inequality as
No In on In thresh +12M N
(29)
Inp
Because N₂ is positive integer, it suffices to choose
N₁ = [max 11,x N1 Vk.
(30)
Finally, p chosen as the smallest value that meets the NFF budget G2) under (30 We find this value using bisection search. For a given kihresh, lower bound on the total NFEs kitrrssh I (K kithres)) 22. The
definition of kihresh then implics that Noot 2 K(2 5) 5 - 1 and thus K S (Not + 1 6)/(2 8) & Kmin-
In summary, for a budget of Nost total NFEs, we treat the number of DDIM steps K E (1,.... Kmin} and the fraction of single-FIREEiteration steps 8 E [0,1) as tuning parameters and, from them, compute {o [N& and p. Figure shows an example. The pair (K, 5) can be tuned using cross-validation. Algorithm 2 details DDIM with the FIRE approximation of E(xo @ y}, which we refer to as "DDfire."
3.4 Relation to other methods
To solve inverse problems, a number of algorithms have been proposed that iterate denoising (possibly scoro-basod), data-consistency (hard or soft), and renoising. Such approaches are referred as either plug- and-play, Langevin, or diffusion methods. (Recall the discussion in Sections 2 While existing approaches use white renoising, the proposed DDfire uses colored renoising that whitens the denoiser input error.
Recall that the reverse process counts backwards, i.e., & K, K- 1,..., 2.
8

---

Published in Transactions on Machine Learning Research (08/202))
Ohees
104
FIRE d²
10a
DDIM of
10a
10
variance
10a
10-1
10
10-1
10
10
DDIM step
Figure 3: For an FFHQ denoiser: the geometric DDIM variances {oAj versus DDIM step k for K = 10, the Thresh corresponding to a 0.4 fraction of single-FIRE-iterattio DDIM steps, and the denoiser input variance o² at each FIRE iteration each DDIM step, for Noo = 25 total NFEs.
Algorithm 2 DDfire
Require: y, A. By or Pylz P. (o {N& æk N(00 o I)
Mddim
for k K. K 1,
do
3:
Tojk = FIRE(y y, A, k, 0k, σh, P),
FIRE via Alg.
Alg.
o (o
4:
Sk = Addim
of
5:
+
1
Tojk + Skns
7& ~ (0,
> DDIM update
o
ok
2
return Zojk
When estimating 2a from the measurements y of (6) methods such DDS, DiAPIR, DAPS, and SNORE use Gaussian prior approximation of the form zo ([E But while they control with tuning parameters, DDfire explicitly estimates v via 10 See App. G for detailed comparison of DDfire to DDS, DifPPIR, and SNORE.
Although we are unaware prior work combining EP with diffusion, there a paper by Meng & Kabashima (2024) that applies EP the annealed Langevin dynamics approach from [Song & Ermon (2020). although for the specific task of dequantization. Like with DPS and IIGM,, Bayes rule used to write Va In Pc(aryl)) Va In pr(2;) Va In poylzx) and the first term is approximated by so (0 t) (recall Sec. 2 The sccond term is approxinated with multiple EP ilerations, each of which solves a linear system using SVD. DDfire differs in using colored renoising, an SVD-free option, and DDIM diffusion, which allows far fewer NFEs.
4 Numerical experiments
We use 256 x 256 FFHQ (Krrras et al. 2019 and ImageNet (Deng et. al. 2009) datasets with pretrained diffusion models from Chung et al. (2023a) and Dhariwal & Nichol 20021). respectively. As linear inverse problems, we consider box inpainting with 128 x 128 mask, Gaussian deblurring using x 61 blur kernel with 3-pixel standard deviation, motion deblurring using 61 61 blur kernel generated using Borodenko 20200) with intensity 0.5, and 4x bicubic super-resolution. We compare to DDRM (Kaaar et. al.] 2022a

---

Published in Transactions on Machine Learning Research (08/2025)
Table 1: DDfire ablation results for noisy FFHQ box impainting with By 0.05 at 1000 NFEs.
Method
PSNR1
LPIPS)
Runtime
DDfir
24.31
0.1127
34.37s
DDfire w/o renoising
18.48
0.2349
34.37s
DDfire w/o colored remoising
23.61
0.1553
34.7s
DDfire w/ stochastic denoising
24.30
0.1143
34.7s
DDfire w/o estimating
23.02
0.1755
34.37s
DDfire w/o CG early stopping
24.31
0.1127
52.12s
DDfire w/ SVD
21.31
0.1124
30.97s
DifIPIR Zhu ct al.
2023
IGDM (Song et al. 2023): ; DDS Chung al. 20244.)
DPS
Chung et al. 2023a
RED-diff (Mardani el al. 2024), and DAPS (Zhang el al. 2025
We also consider phase retrieval with the shot-noise corruption mechanism from Metzler et al. (2018) for both oversampled Fourier (OSF) and coded diffraction pattern (CDP) (Candès ct al. 2015 A at 4x oversampling
with O'shat = and 45, respectively. Here, O'shat is the shot-noise strength, as detailed App. H We compare to prDeep (Metzler et al., 2018), DOLPH (Shoushtari et al., 2023 DPS, R.D-diff, DAPS, and the classical
for all algorithms that hybrid input-output (HIO) algorithm (Fienup) 1982), using (y)z( N(vs og)) accept likelihood function.
Unless specified otherwise, DDfire was configured as follows. For the linear inverse problems, CG is used (no SVD), 1000 NFEs are used without stochastic denoising, and the (K, 6) hyperparameters are tuned to minimize LPIPS (Zhang et al.) 2018) on a 100-sample validation set (see Table 5p. For phase retrieval, we set NFEs to 800 for OSF to fairly compare to prDeep, while only 100 NFEs were used for CDP as further steps offered no improvement. Stochastic damping was appliod in both cases, and (K, 5) were the hand-tuned values in Table 0 Neither CG nor SVD are needed since (11] can be solved in analytically. Appendix H contains additional details the implementation of DDfire and the competing methods.
4.1 Ablation study
We first perform an ablation study on the SLM-DDfire design choices in Sec. 3 using noisy FFHQ box inpainting and 100-image validation set. The results are summarized in Table
We first see that both
PSNR and LPIPS suffer significantly when FIRE is run without renoising (i.c., c = 0 in line 13 in Alg. 1. Similarly, renoising using white noise (i.e., c e~ N00, o2I) in line 13 of Alg, [ gives noticeably worse PSNR and LPIPS than the proposed colored noise. Using stochastic denoising gives nearly identical performance plain demoising (i.e., (o)
in line of Alg. D and so we use plain denoising by default with linear inverse problems. A more significant degradation results when the denoiser output-crror variance is not adapted to w in line of Alg. 1 but set at the data-average value (a). On the other hand, when CG doesn't 't use early stopping (as described in App. D the runtime increases without improving PSNR or LPIPS. Thus, we use carly stopping by default. Finally, using an SVD instead of CG, which also avoids the noise approximation in 17h. gives essentially identical PSNR and LPIPS but with a slightly faster runtime. Figure shows another comparison of the SVD and CG versions of DDfire.
4.2 PSNR, LPIPS, and FID results
For noisy linear inverse problems, Tables 2 show PSNR, LPIPS, and FID (Heusel et al. 2017) on 1000- sample test set for FFHQ and ImageNet data, respectively. DDRM was not applied to motion deblurring due to the lack of an SVD. Tables 2 show that DDfire wins in most cases and otherwise performs well.
Fig. 5 shows image examples for inpainting, motion deblurring, Gaussian deblurring, and 4x super-resolution on ImageNet. The zoomed regions show that DDfire did a better job recovering fine details. Additional examples can be found in Fig. 15
For OSF and CDP phase retrieval, Table 4 shows PSNR, LPIPS, and FID on a 1000-sample test set for FFHQ and TeageNet data. The table shows that DDfire outperformed the competitors in all cases. With
10

---

Published in Transactions on Machine Learning Research (08/2025)
Table 2:
Noisy FFHQ results with measurement noise standard deviation By 0.05.
Inpaint (box)
Deblur (Gaussian)
Deblur (Motion)
4x Super-resolution
Model
PSNRT
LPIPS1
FID
PSNR1
LPIPS
FID4
PSNR1
LPIPSI
FID4
PSNR1
LPIPSL
FIDJ
DDRM
21.71
0.1551
40.61
25.35
0.2223
51.70
27.32
0.1864
45.82
DiPPI
22.43
0.1883
31.98
24.56
0.2394
34.82
26.91
0.1952
26.07
24.89
0.2486
32.33
TIGDM
21.41
0.2009
44.41
0.2525
45.34
25.14
0.2082
41.95
24.40
0.2520
51.41
DDS
20.28
0.1481
30.23
26.74
0.1648
25.47
27.52
0.1503
27.59
26.71
0.1852
27.09
DPS
22.54
0.1368
35.69
25.70
0.1774
25.18
26.74
0.1655
27.17
26.30
0.1850
27.38
RED-dif
23.58
0.1883
48.86
26.99
0.2081
38.82
16.47
0.5074
128.68
25.61
0.3569
70.86
DAPS
23.61
0.1415
31.51
26.97
0.1827
31.10
27.13
0.1718
30.74
26.91
0.1885
30.83
DDfir
24.75
0.1101
25.26
27.10
0.1533
24.97
28.14
0.1374
26.12
27.13
0.1650
25.73
Table 3: Noisy ImageNet results with mcasurement noise standard deviation By = 0..55.
Inpaint (box)
Deblur (Gaussian)
Deblur (Motion)
4x Super-resolution
Model
PSNR
LPIPS!
FID
PSNR
LPIPS
FID
PSNR
LPIPSU
FID
PSNR
LPIPS
FID.
DDRM
18.24
0.2423
67.47
22.56
0.3454
68.78
24.49
0.2777
64.68
DiIPR
18.03
0,2860
65.55
21.31
0.3683
56.35
24.36
0.2888
0.3383
63.48
TIGDM
17.69
0.3303
86.36
20.87
0.4191
75.43
22.15
0.3591
70.91
21.25
0.4149
78.57
DDS
16.68
0.2222
63.07
23.14
0.2684
50.84
23.34
0.2674
50.08
23.03
0.3011
52.13
DPS
18.23
0.2314
59.10
21.30
0.3393
50.46
21.77
0.3307
80.27
23.38
0.2904
49.86
RED-diff
18.95
0.2909
108.88
23.45
0.3190
65.65
15.21
0.5647
198.74
22.99
0.3858
83.06
DAPS
19.99
0.2199
61.53
23.91
0.2863
56.87
24.58
0,2722
24.04
0.2729
55.54
DDfire
20.39
0.1915
55.54
23.71
0.2353
50.05
24.59
0.2314
49.25
23.58
0.2629
49.67
Table 4: Noisy phase retricval results
FFHQ OSF
FFHQ CDP
ImageNet OSF
ImageNet CDP
Model
PSNR1
LPIPS1
FIDJ
PSNR
LPIPSŞ
FID
PSNR1
LPIPS!
FID
PSNR1
LPIPS1
FID
HIO
23.66
0.4706
130.6
17.59
0.5430
84.87
21.53
0.4584
111.5
17.65
0.4527
69.01
DOLPH
14.73
0.7220
389.9
25.76
0.1686
32.93
14.33
0.6844
258.6
26.15
0.2256
48.26
DPS
23.63
0.2908
53.91
29.19
0.1394
27.87
16.69
0,5314
140.2
27.21
0.1799
45.98
RED-diff
25.47
0.2828
65.74
28.75
0.1734
28.87
18.41
0,4385
108.0
26.87
0.2173
48.12
DAPS
24.10
0.2891
57.73
28.26
0.1927
34.97
17.62
0.4934
121.8
21.76
0.3332
54.17
pr prDeep
30.90
0.1132
31.51
19.24
0.4183
59.44
26.16
0.1977
57.86
19.51
0.3931
39.34
DDfire
33.56
0.0691
28.94
30.16
0.1186
23.30
29.93
0.1640
56.99
28.91
0.1377
44.35
0.45
0.40
DiffPIR
0.35
DDRM
DPS
DDfire-20
RED-diff
DDfire-50
0.30
DDfi
DAPS
DDf
0.25
DDfire-1000
0.20
0
20
40
60
80
100
120
Single image sampling time on one A100 GPU (s)
Figure 4: LPIPS vs. single image sampling time for noisy Gaussian deblurring on an A100 GPU. The evalustion used 1000 ImageNet images. Solid line: DDfire with CG for various numbers of NFEs. Dashed line: DDfire with SVD.
11

---

Published in Transactions on Machine Learning Research (08/2025)
37g
DDRM
DifIPIR
IIGDM
DPS
RED-diff
DAPS
DDfire
impainting
y
370
DDRM
DifPIRR
IIGDM
DPS
RED-diff
DAPS
DDfire
Gaussian deblurring
DiffPIR
DPS
RED-diff
DAPS
DDfire
motion deblurring
y
2Cg
DDRM
DifRPIR
IIGDM
DPS
RED-diff
DAPS
DDfire
super-resolution
Figure 5: Example recoverics from noisy linear inverse problems with ImageNet images.
FFHQ OSF, for example, DDfire outperformed the best competitor (prDeep) by 2.6 dB and the second best (RED-diff) by 8.1 dB in PSNR. Example reconstructions can be found in Fig.
4.3 Runtime results
Figure shows LPIPS vs. average runtime (in seconds on an A100 GPU) to generate single image for noisy Gaussian deblurring on the 1000-sample ImageNet test set. The figure shows that DDfire gives a better performance/compleee /complexity tradeoff than the competitors. also shows that DDfire is approximately 1.5 times faster than DPS when both are run at 1000 NFEs, due to DPS's use of backpropagation.
Figure shows LPIPS vs. runtime for noisy OSF phase retrieval on the 1000-sample FFHQ test set, showing that DDfire gives significantly better perfornanceecomplexiity tradeoff than all diffusion-based competitors.
12

---

Published in Transactions on Machine Learning Research (08/2025)
5 Conclusion
To solve linear inverse problems, we proposed the Fast Iterative Renoising (FIRE) algorithm, which can be interpreted as the HQS plug-and-play algorithm with a colored renoising step that aims to whiten the denoiser input error. We then extended the linear FIRE algorithm to the generalized-linear case using expec- tation propagation (EP). Since these FIRE algorithms approximate the meeaureeeen--oondiiee Ef y). or equivalently the cnassurement-condition
denoiser score v In (e (y), they can be readily combined with DDIM for diffusion posterior sampling, giving the "DDfire" algorithm. Experiments on box inpainting, Gaussian and motion deblurring, and 4x super-resolution with FFHQ and ImageNet images show DDfire outperforming DDRM, IIGDM, DDS, DiffPIR, DPS, RED-diff, and DAPS in PSNR, LPIPS, and FID met- rics in nearly all cases. Experiments on noisy FFHQ and ImageNet phase retrieval (both OSF and CDP versions) show DDfire outperforming HIO, prDeep, DOLPH, DPS, RED-diff, and DAPS in all cases. Finally, DDfire offers fast inference, with better SrIPS-versus--unbiie curves than the competitors.
Acknowledgements
This work was funded in part by the National Institutes Health under grant RO1-EB029957.
References
Rizwan Ahmad, Charles A. Bouman, Gregery T. Buzzard, Stanley Chan, Sizhou Liu, Edward T. Reehorst, and Philip Schniter. Plug and play methods for magnetic resonance imaging. IEFE Signal Process. Mag.. 37(1):105 116, March 2020.
Tamir Bendory, Robert Beinert, and Yonina C Eldar. Fourier phase retrieval: Uniqueness and algorithms. In MATHEON Conf. Compressed Sensing and its Applications, pp. 55 91, 2015.
Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, New York, 2007.
Levi Borodenko. motionblur. Downloaded from
2020.
Charles A Bouman and Gregery T Buzzard. Generative plug and play: Posterior sampling for inverse problems. In Proc. Allerton Conf. Commun. Control Comput., pp. 1-7, 2023.
Emmanuel Candès, Xiaodong Li, and Mahdi Soltanolkotabi. Phase retrieval from coded diffraction patterns.
Appl. Comput. Harmonic Anal., 39(2):277 299, 2015.
Sitan Chen, Sinho Chewi, Holden Lee, Yuanzhi Li, Jianfeng Lu, and Adil Salim. The probability flow ODE is provably fast. In Proc. Neural Info. Process. Syst. Conf., volume 36, pp. 08552-68574 2023.
Hyungjin Chung, Jeongsol Kim, Michael T McCann, Mare L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In Proc. Intl. Conf. Learn. Rep.. 2023a.
Hyungjin Chung, Jeongsol Kim, Michael T McCanm, Mare L Klasky, and Jong Chul Ye. diflusion-posterior sampling.
March 2023b.
Hyungjn Chung, Suhyeon Lee, and Jong Chul Ye. Decomposed diffusion sampler for accelerating large-scale inverse problems. In Proc. Tntl. Conf. Learn. Rep., 2024a.
Hyungji Chung, Suhyeon Lee, and Jong Chul Ye. DDS. 2024b.
chung/DD
Florentin Coeurdoux, Nicolas Dobigeon, and Pierre Chainais. Plug-and-play split Gibbs sampler: Embedding deep generative priors in Bayesian inference. IEEE Trans. Image Process., 33:3496 3507, 2024.
Giannis Daras, Hyungjin Chung, Chich-Hsin Lai, Yuki Mitsufuji, Jong Chul Ye, Poyman Milanfar, Alexandros Dimakis, and Mauricio Delbracio. A survey on diffusion models for inverse problems. arXiv:2410.00083,, 2024.
13

---

Published in Transactions on Machine Learning Research (08/2025)
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proc. TEEE Conf. Comp. Vision Pattern Recog,, pp. 248 118-255, 2009.
Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. In Proc. Neural Info. Process. Syst. Conf., volume 34, pp. $780-8794, 2021.
Bradley Efron. Tweedie's formula and selection bias. J. Am. Statist. Assoc., 106(496):1602 1611100001111,,,--1-1111111-1144 2011.
James R. Fiemup. Phase retrieval algorithms: A comparison. Appl. Optics, 21(15):2758-22 2769, August 1982.
Mario A. T. Figueiredo and Jose M Bioucas-Dias. Restoration of Poissonian images using alternating
direction optimization. IEEE Trans. Image Process., 19(12):3133-314 3144,,,,,,, 2010.
Docke Hekstra, Ian Hunt-Isaak, Jack Greisman, and John Russell. phase-retrieval. https://github..cm com/
Lakkstra-Lbb/haaeeee retrieaal 2018.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge to local Nash equilibrium. In Proc. Neural Info. Process. Syst. Conf., volume 30, 2017.
Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising diffusion probabilistic models. In Proc. Neural Info. Process. Syst. Conf., volume 33, pp. 6840-6851, 2020.
Aapo Hyvärinen. Estimation of non-normalized statistical models by score matching. J. Mach. Learn. Res., 6:695-709, 2005.
Ajil Jalal, Marius Arvinte, Giannis Daras, Eric Price, Alex Dimakis, and Jonathan Tamir. Robust compressed sensing MRI with deep generative priors. In Proc. Neural Info. Process. Syst. Conf., 2021.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proc. IEEE Conf. Comp. Vision Pattern Recog,, pp. 4396 4405, 2019.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Proc. Neural Info. Process. Syst. Conf., 2022a.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Downloaded from http:///gittuuu..oooo/aaattttttt
May 2022b.
Rémi Laumont, Valentin De Bortoli, Andrés Almansa, Julie Delon, Alain Durmus, and Marcelo Pereyra. Bayesian imaging using plug & play priors: When Langevin meets Tweedie. SIAM J. Imag. Sci., 15(2): 701-737, 2022.
David G Luenberger and Yinyu Ye. Linear and Nonlinear Programming, Springer, 2016.
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective on solving inverse problems with diffusion models. In Proc. Intl. Conf. Learn. Rep., 2024.
Xiangming Meng and Yoshiyuki Kabashima. QCS-GGM+: Improved quantized compressed sensing with score-based generative models. In Proc. AAAI Conf. Artificial Intell., volume 38, pp. 14341-143499 2024.
Xiangming Meng, Sheng Wu, and Jiang Zhu. A unified Bayesian inference framework for generalized linear models. IEEE Signal Process. Lett., 23(333398 402, 2018
C. A. Metzler. prDeep. htttgp///ttttuu..cmm//////////e/
2018.
Christopher A. Metzler, Philip Schniter, Ashok Veeraraghavan, and Richard G. Baraniuk. prDeep: Robust phase retrieval with flexible deep neural networks. In Proc. Intl. Conf. Mach. Learn., pp. 3501 3510, 2018.
Thomas Minka. Family of Approximate Algorithms for Bayesian Inference. PhD thesis, Dept. Comp. Sci. Eng., MIT, Cambridge, MA, USA, 2001.
14

---

Published in Transactions on Machine Learning Research (08/2025)
NVlabs. RED-diff. Downloaded from httpps//ggttabb..c/////aabb//b/a diff 2023.
Beresford N. Parlett. The symmelric eigenvalue problem. SIAM, 1998.
H. Vincent Poor. An Introduction to Signal Detection and Estimation. Springer, New York, NY, USA, 2nd edition, 1994.
Edward T. Reehorst and Philip Schniter. Regularization by denoising: Clarifications and new interpretations. IEEE Trans. Comput. Imag.. 5(1))52-67, 67, March 2019.
Marien Renaud, Jean Prost, Arthur Leclaire, and Nicolas Papadakis. Plug-ana and--lay image restoration with stochastic denoising regularization. In Proc. Intl. Conf. Mach. Learn., 2024.
Yaniv Romano, Michael Elad, and Peyman Milanfar. The little engine that could: Regularization by de- noising (RED). SIAM J. Imag, Sci., 10(4):1804 1844, 2017.
Philip Schniter, Sundeep Rangan, and Alyson K Fletcher. Vector approximate message passing for the generalized linear model. In Proc. Asilomar Conf. Signals Syst. Comput., pp. 1525 1629, 2016.
Yoav Shechtman, Yonina C Eldar, Oren Cohen, Henry Nicholas Chapman, Jianwei Miao, and Mordechai Segev. Phase retrieval with application to optical imaging: A contemporary overview. IEEE Signal Process. Mag., (113)88--109) 2015.
Shirin Shoushtari, Jiaming Lin, and Ulugbek S Kamilov. Diffusion models for phase retrieval in computational imaging. In Proc. Asilomar Conf. Signals Syst. Comput., pp. 779 783, 2023.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proc. Intl. Conf. Mach. Learn. pp. 2256-22655 2265, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Proc. Intl. Conf. Learn. Rep., 2021a.
Jiaming Song, Arash Vahdat, Morteza. Mardani, and Jan Kautz. Pseudoinverso-guide.. diffusion models for inverse problems. In Proc. Intl. Conf. Learn. Rep., 2023.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Proc. Neural Info. Process. Syst. Conf., 2019.
Yang Song and Stefano Ermon. Improved techniques for training score-based generative modols. In Proc. Neural Info. Process. Syst. Conf., 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Proc. Intl. Conf. Learn. Rep., 2021b.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. In Proc. Intl. Conf. Learn. Rep., 2023.
Zihui Wu, Yu Sun, Yifan Chen, Bingliang Zhang, Yisong Yue, and Katherine Bouman. Principled proba- bilistic imaging using diffusion models as plug-and-plas priors. In Proc. Neural Info. Process. Syst. Conf., 2024.
Xingyu Xu and Yuejie Chi. Provably robust score-based diffusion posterior sampling for plug-and-play image reconstruction. In Proc. Neural Info. Process. Syst. Conf., 2024.
Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing. Downloaded from https://githut: com/ chanbbinggllagg2219///5S]] 2024.
15

---

Published in Transactions on Machine Learning Research (08/2025)
Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing. In Proc. IFFE Conf. Comp. Vision Pattern Recog., 2025.
Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Lue Van Gool, and Radu Timofte. Plug-and-play image restoration with deep denoiser prior. IEEE Trans. Pattern Anal. Mach. Intell. 44(10):6360-633 6376, 2021.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. IEEE Conf. Comp. Vision Pattern Recog,, pp. 586-595, 2018.
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and-play image restoration. In Proc. IEEE Conf. Comp. Vision Pattern Recog,, pp. 1219-1229, 2023.
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Diffpir. Downloaded from https://gitee
http:///gggtuuu..oo.ooaaaaaaaaia conm//maanzzhi-uu//ffffif July 2024.
A. Zymnis, S. Boyd, and E Candès. Compressed sensing with quantized mcasurements. IEEE Signal Process. Lett, 17(2):149 152, 2010.
16

---

Published in Transactions on Machine Learning Research (08/2025)
A VP formulation
In the main text, we describe DDfire for the VE SDE formulation from Song et al. (2021)) and the corre- sponding SMLD discrotization from Song & Ermon (2019). Here, we describe it for the VP SDE from Song
et al.
2021b) and the corresponding DDPM discretization from Ho et al. (2020)
From Song al. (2021b), the general SDE forward process can be written as dæ ((x, dt + g(t) du
(31)
for some choices of f(::) -) and g(-), where dw is the standard Wiener process (i.e., Brownian motion). The reverse process can then be described by
dæ = (f(x,1) (1)V (npe(()))))))))))))))))))))))))) di + g(1) dre
32)
where pr(-) is the distribution of x at time and dw is the reverse Wiener process. In the VE-SDE, f(z t) = 0 and s()) ([oo(())/ di for some variance schedule d²((), but in the VP-SDE, f(x 1) - 3())) and g(t) Btt) for some variance schedule B(t). When discretized to t E 10, ,,,, the VP forward process becomes
3t = VI Brūr-1 + BBū..1
(33)
with Lid. {wt) NO, I), so that
3 = Vājao + vI d7E
(34)
with ar - 11 02 -=1 as and é ~ (O. Throughout, we write the VP quantities with tildes distinguish them from the VE quantities. The DDPM reverse process then takes the form
a1 3r
Et + Be Vn, m, Inp(#r)) + E with Σ
(35)
Vor
C2e
and is typically initialized at कr NO I). By rewriting 34 as
1
at E
F. = 3n
(36)
Vā
Ot
and comparing it 3 we recognize the VP/VE relationships
1- =o²
Va. E. we
and
a4 1 o
(37)
a1
Furthermore, assuming that or 1, the VP initialization initialization ØT ~ o0
क7 N) is well approximated by the VE
B DDIM details for VP
The DDIM reverse process from Song et al. (2021a) provides an alternative to the DDPM reverse process that offers a flexible level of stochasticity. When describing VP DDIM, we will write the quantities as
Cy. Ek. Tig ag to distinguish them from the corresponding VP DDPM quantities o,, Ec. ny a., and we will write the total number of steps as K. Like 334. DDIM is built around the model
Z = 2 + VI TgEk:
ES ~ N(O,I.
(38)
Adapting the first two equations from Song et al.
2021a A.p.D.3) to our notation, we have
Or E E Et. v)
- v ak-1
+ V ax-1 2 Ble Tt. y) + 7.
(39)
v a
ak-1
0
5k Tjddim
(40)
1 TeA
7
17

---

Published in Transactions on Machine Learning Research (08/2025)
with ng NO I) and tunable A 2 0. When dddim 1 and K T, DDIM reduces to DDPM. But when Vddimm 0, the reverse process is deterministic. In fact, it can be considered discretization of the probability flow ODE (Song et al.) (2021a), which often works much better than the SDE when the number of discretization steps K is small. We now write (39 in a simpler form. Applying E[-[0x, y) to both sides of 38 gives
En
OE
E Dn Th. y
K V 0ik
ao Tk. 4 + v1 ae
EfEK y
BE y)
(41)
O
and plugging
41 into 39 gives
Vo Ef20 k y
2g 1 V ak Exxp k y + T2 v ax-1 %
(42)
v1 -0k
V Vas-1 E(zpp T y + 7 + hx (āt - Va Ex [k 8))
(43)
Ty
+ Du E 20 k y + 7k
(44)
for
a - 22
and Th 2 v 0k-1 Ti v Ckk
(45)
Thus the VP DDIM reverse process can be described by [0 and initialization ög I).
45
with ne ~N I) Vk and
C DDIM details for VE
We now provide the details for the VE version of DDIM. Starting with the VP DDIM reverse process we can divide both sides by vax-1 to get
44
kk-1 hvva ak T
5k
E(aro
F2.
+
(46)
0--1
Voh-1 Vāk
-
and leveraging the VP-Lo-VE relationship
37
to wrile
Tex Vok
9k
5x
Φk-1 = h4Ex + 9h E( f) + Stnt with
hy =
Sk
(47)
Vāg
with ny IN I) wk and initialization ØR N(0 of I). Plugging Is from
45 into 47
find
Von- T Vok h4-
gh
(48)
vāg-1
Then plugging 40 into
we find
!]ddim
Cik
Ogg
T]ddim
1 1 Ch-1
1
CF
(49)
Th-1
ag-1
1 Cz.
Oha-11
1]ddim
1 α 1
C
1
1
1/ddim
2 0k-1
(11
of -
[1 + a o111)
(50)
@k-1
1
0kk
Oik-1
o
02.1
k-
Maaim
(51)
Finally, noting from
37
47
and
51
that
Nddim
of
(o o2
aadim (1
o
(52)
Sh
1
T
o
o
18

---

Published in Transactions on Machine Learning Research (08/2025)
plug h4 from
45
into
47
to find
hk
- 00
Sk
0
1 - 0k-1
1
Sk
(53)
V 1 ak.
k-
akk
0k
- Ük--1
%
0 K-1 1
(1
o
Okk
1
1
a
o
Iddim
o
(54)
Sk
of k-1
22
o k-
Sk
(55)
R
o
o
The VE DDIM reverse process is summarized in
26
27
D Speeding up CG
In this soction, we describe a small modification to FIRE that can help to speed up the CG step. When CG is used lo solve 11 its convergence speed is determined by the condition number of A A + (d)/o)T (Luenberger Ye 2016 Thus CG can converge slowly when og/v is small, which can happen in early DDiire iterations. To speed up CG, we propose solve 111 using By in place of σy, for some ōy > ay. Since guarantee conditional number of
the condition number of A + [v)] is at most vs² og + 1, at most 001 by setting
2 vs max max . / (ns
(56)
Although using ōy By in 11 will degrade the MSE of ã. the degradation is partially offset by the fact that less noise will be added when renoising r. In any case, the modified [1] can be written
π = AATA/ + 1/-)- y/ x/vv)
(57)
A/T + I/v)~ Azo + wywl/o [mo Vuel/u)
(58)
= x6 + (ATA/o) +1/1) (AT))-(AATwo,//0 - e/ e/v
(59)
in which case T ~ N(ro; C) with covariance
+ +/21 (A/A/G/+I/>> C =
- +1/v)
(60)
= VVSTSV 16 + +1/v) (VS SVTo I/v) SV 18/ I/v)
0//
(61)
L.0//0 +1/V
V Diag(y)) T for 7 |0 1/1
(62)
The desired renoising variance then becomes
I C V for d; = 0² 2,02/05 + 1/v
(63)
1st + 1/v|2
and we can generate the colored noise via 16 if the SVD is practical. If not, we approximate Σ by
max o f + 1/v
+ with {=
(64)
$2
max 03 + 1/v|2
and generate the colored noise via 19 Tt is straightforward to show that 8 2 O whenever o, By, in
which case x² V guarantees that valid covariance matrix. Figure 6 shows the close agreement between the ideal and approximate E-renoised error spectra both when o, = By and when o, > dy-
19

---

Published in Transactions on Machine Learning Research (08/2025)
eigenspectrum cov(x Xo)
eigenspectrum of cov(r- xo)
10
695.000
10
694.975
694.950
10
target o²
694.925
ideal
10
error with of
694.900
approx with of
10-5
with d7
694.875
approx with oz
694.850
20000 40000 sorted index
60000
20000
40000 sorted index
60000
Figure 6: For FFHQ Gaussian deblurring, the left plot shows the eigenspectrum of the error covariance Cov - oo} with either 83 from (56) (if CG speedup) or 02 = og (if no CG speedup), as well as the eigenspectrum of the target error covariance o²I to aim for when renoising. The right plot shows the eigenvalues of the renoised error covariance Cov(r-200 for the ideal case when is used (possible with SVD) and the case when Σ from (17) is used (if no SVD), with either of or oy. Here we used ož = 10 V = 0.16 (corresponding to the first FIRE iteration of the first DDIM step), and 35.7 (corrosponding to the example in Fig. 3
E Proof of Theorem 1
To prove Theorem
we begin by writing the key FIRE steps with explicit iteration index n 2
T do (r(nn] (m))
(65)
og =
( ( A y
of
a[n] A A +
v/nl +
x|n|
(66)
u(n]]
2 [ 1] max (ml//////
(67)
[n[n] - (5//0² 1/v/m)) i ==
(68)
n(n]] V Diaag(A(n))
e|n]- ~0,
(69)
r r|++1] 1] [mn] n/n2
(70)
Our proof uses induction. By the assumptions of the theorem, we know that there exists an iteration n (in particular n = 1) for which r|n] = xn + a[n]e[n]] with E[n] ~ No I) and finite [m). Then due to the
denoiser assumption, we know that [[n] = xo [ with e(n] = =W I) and known w/n] < [n]). We
assume that this value of v|n] is used in lines 666 68
Using these results and (6 we can rewrite 666 as
oy
o3 v/n]
a[n]
A A
1
AT A (Axo oyw)
(zo Ve(nle(ni))
(71)
v/n]]
01
07
1)
-1
og Vvln
+
AT A +
Joy ( ww
e(m]
~N(xo, C(r))
(72)
w/n]
for
of 2 1) v[m]]
of " (A
of (ATA+
1
Cln) A ATA +
1
A
7
(73)
vinl
by leveraging the independent-Gaussee Ganssian assumption on e(n]. From this and 668
69
we can then deduce
nnn] (0 E(n]) with E[n]]
V Diag(A(n)))))llo = = C[n]
(74)
20

---

Published in Transactions on Machine Learning Research (08/2025)
Algorithm 3 FIRE for the GLM: H FiREGLM (y. A, Pylz: Pinie: Fniit p).
Require: y, A, 8max, Pylz: N, p > 1, Pinit: Tinit- Also Diag(s)V) V if using SVD. r Pinit and Onni
Initialize
for = 1..... .N do
I do(r.o) Voolo). w~ N(0.1
Stochastic denoising
4:
V + (0)
Error variance of w
5:
I f AT
Denoised version of Anoi
D2 + v ||Allf/m
Error variance of z
2 E(20,51//
l..... m
Estimate 20.j ~ (2), from Yj ~ -Pyi(--/zug))
8:
D2 + 1 -== var Z.
> Averaged posterior variance of {zo,)
o - 11/2-1/DD.
2
Eatrinsic variance Extrinsic meany Error variance of x D Estimate Bn ~ NT,, vt) from y ~ (A o T) Decrease target variance
10:
y (2/022 2/
11:
D V Az - 2
12:
④ < arg min 1
Axi 1 + | 2//2/1/
13:
o² t 12x,0"/p,11
14:
if have SVD then
15:
A 0² (ot 2 1/v)-1 i= i=1,...,;
16:
ct V Diag 1/28, 8 ~~ I)
b Colored Gaussian noise
17:
else
< (v max 0 + ++1100 1/v) / max
18:
19:
c x Vo -I V E, E E~ (0,1)
D. Colored Gaussian noise
20:
r + x + e
Renoise so that Cov - 20) = 0
21:
return
that, from 70
r[n+1] ~ (m Cin] + [m]] = N( ( + r[n+1] = an + a +[n+1]e[n++ [][[++]], 1], <[m+1] ~ NO
(75)
(76)
Thus, by induction, [i]] 3g + [fn]c[n]] with c(n] ~ -0 I) holds n then it holds at all n >
Recall that the theorem also assumed that v/n] [2] for all n. Thus, there exists p >> for which
o² on vin] for all n, for which rewrite 67
Vn.
(77)
Consequently, for any iteration n 2 we can write
o² 0² [111//00 = pm=
(78)
Finally, because the error covariance on [|n] obeys
(
1 A
1
of 02(n)
Clrl
A A
u(m]
I pm-1
(79)
we that the error variance in @[n]] decreases exponentially with and thus @ [n]] converges to the true mp.
F GLM-FIRE algorithm
The full algorithm for the GLM version of FIRE given in Alg. 3 There, the dashed blue box indicates the lines used for the EP update; the remaining lines mirror those in FIR, summarized Alg.
21

---

Published in Transactions on Machine Learning Research (08/2025)
Algorithm 4 DDS (Chung et al. 2024a
Require: (: :). y. A, Ydds, Meg. f K k=1 Addim 0
1: ~ *0, 0K
2: for k = K, 1e-K-1,.... 1,..,, do
3: Fy = do (d.( 0k)
D Denoising
4: Dojk = CG(ATA + Ydds I, A,A, A y + Tdds 2k. Mcg) xe arg ming {1ly Ax 12 + Ydds | -x xx
k ( x
S: 1dddim|
o
0k Sk kk
Kk
k-
%
T k
Sh Tkk
ny NO I)
DDIM update
k
0
return Bojk
Algorithm 5 DiffPIR [Zuu et al. 2023)
Require: do(::)). y. A, Ty, Adiffir for k Adifiir 1: Øk ~ ~N
for h K, 1,..,, do
Is ( T
D Denoising
Adiffpir ||0 Ex 2
Bojk arg min
y Ax
Approzimate MMSE estimation
2 Ok-1
Ok lddifiirr
0
6: -k-1 2* + (1 - 2k Dojk + Maiffpir k-1 17k; Th ~0,
>> DDIM-like update
7:
return Boo
G
Comparison to other renoising proximal-data-fidelits PnP schemes
In this section, compare SLM-FIRE/DDirr to other renoising PnP schemes for the SLM that involve a proximal data-fidelity step equivalently, MMSE estimation of z under the Gaussian prior assumption 8 DDS, DiffPIR, and SNORE, which are detailed in Alg. Alg, 5 and Alg. respectively.
Comparing FLM-FIRE/DDiis (see Alg. and Alg. 2 to DDS, we see that both approaches use CG for approximate MMSE estimation under a Gaussian prior approximation and both use a DDIM diffusion update. But DDS uses the hyperparameters "dds and Meg (the number CG iterations, which is usually small, such four) to adjust the noise variance in the Gaussian prior approximation (8), while SLM-FIRE/DDEE istimates at each iteration. Also, STM-FIRF/DDDrr injects colored Gaussian noise to whiten the denoiser input error while DDS injects no noise outside of DDIM.
Comparing FLM-IRE/DDfrre to DifiPIR, we see that both perform MMSE estimation under a Gaussian prior approximation and both use a DDIM-like diffusion update. But DimPIR uses the hyperparameter Adiffpir to adjust the noise variance V in the Gaussian prior approximation ( while SLM-FIRE/DDére estimates at each iteration. Also, SLM-FIRE/DDGrr injects colored Gaussian noise whiten the denoiser input error while DiffPIR injects noise outside of its DDIM-like step.
Comparing SIM-FIRE to SNORE, we see that both perform MMSE estimation under a Gaussian prior approximation and both use renoising. But SNORE uses white renoising while SLM-FIRE uses colored renoising to whiten the denoiser input error. Also, SNORE uses the hyperparameter &snore to adjust the noise variance in the Gaussian prior approximation (8), while SLM-FIRE estimates at cach iteration. Furthermore, since SNORE based on the RED algorithm (Romano al. (017), its denoiser output is scaled and shifted. Finally, SNORE has many more tuning parameters than ST.M-FIRF.
22

---

Published in Transactions on Machine Learning Research (08/2025)
Algorithm 6 Annealed Proximal SNORE (Renaud et al. 2024]
Require: do(:--).8 A, By, Snore. Manore: for (= {ai) {K;) Maon Dinit
KMrer Dinitt
for = Mnorey Manore
do
3:
for k Ki, Ki- 1, 1,..., do & ~ I((,1)
4:
= Tolk - ii
D Renoising
Dojk S
Th
of
do do 0i)
RED updale
To k-1 arg min
1y Axl + 28 1 Fxll?
Approcimate MMSE estimation
20
return Book
H Implementation details
H.1 Inverse problems
For the linear inverse problems, the measurements were generated as
(80) with appropriate A. For box inpainting, Gaussian deblurring, and super-resolution we used the A and A implementations from Kawar et al. (2022b). For motion deblurring, implemented our A and A with reflect padding. All methods used these operators implementations except DiffPIR, which used the authors' implementations. Motion-blur kernels were generated using Borodenko 2020).
Ano + 0, w ~ I),
For phase retrieval, the measurements were generated using the method from Metzler et al. 2018):
リ = 1= + w/j w ~ N(0, aghor |=o.) P). j j= i....,,
(81)
where ashot controls the noise level and A with the values of æp scaled lic in the range [0, 255). This is approximation of the Poisson shot-noise corruption model in that the intensity vÿ / is approximately Poisson (() (1=0..)//shhol
distributed for sufficiently small values of ashot- We implemented the oversampled- Fourier A by zero-par sero-padding the image by 2x in each direction and then passing the result through unitary FFT. For CDP phase retrieval, we set. A = [A]...., [
A]] for A, = I,-1/2 F Diag(cı), where F is d x d FFT and e contain i.i.d. random entries uniformly distributed on the unit circle in the complex plane, and where 4. In both cases, A
H.2 Evaluation protocol
For the linear inverse problems, run each method once for each measurement y in the 1000-sample test set and compute average PSNR, average LPIPS, and FID from the resulting recoveries.
For OSF phase retrieval, following Chung et al. (202aa), we run each algorithm four times and keep the reconstruction & that minimizes the measurement residual y A Performance metrics are then evalu- ated after resolving the inherent spatial shift and conjugate flip ambiguities associated with phase retrieval (see, e.g., [endory et al. (015)). Note global phase ambiguity is not an issue due to the non-negativity of our images. For the CDP experiments, run each algorithm only once and don't perform ambiguily resolution, because it is unnecessary.
H.3 Unconditional diffusion models
For the FFHQ experiments, all mothods used the pretrained model from Chung et al. (203a For the ImageNet experiments, all methods used the pretrained model from Dhariwal & Nichol 20021 In both cases, T = 1000.
23

---

Published in Transactions on Machine Learning Research (08/2025)
Table 5: Hyperparameter values used for DDfire.
Inpaint (box)
Deblur (Gaussian)
Deblur (Motion)
4x Super-resolution
Dataset
dy
K
K
K
K
8
FFHQ
0.05
100
0.50
650
0.60
500
0.20
650
0.60
ImageNet
0.05
100
0.50
500
0.20
500
0.20
650
0.60
Table 6: Hyperparameter values used for DDfire phase retrieval.
Operator
Oshott
K
OSF
8
20
0.00
CDP
45
10
0.00
H.4 Recovery methods
DDfire. Our Python/Pytorch codebase modification of the DPS codebase from Chung al. (202bb) and is available at https://github. http///tttbeedeeeDDttte For all but one row of the ablation study in Table and the dashed line in Fig. ran DDfire without SVD and thus with the approximate renoising in 19
For the linear inverse problems, unless noted otherwise, we ran DDfire for 1000 NFEs using Rddim 1.5, and we did not use stochastic denoising (i.e., (a) = Yo in Alg. D as suggested by our ablation study. We tuned the (K,5) hyperparameters to minimize L.IPS on a 100-sample validation set, yielding the parameters in Table 5 For the runtime results in Fig. we used Addim 1.0 for Noo E (50, 100, 200, 500} and Mddim for Neot 20, and we used K NFE/2 and d 0.2 for all cases. For the u-estimation step in Alg. we
used A |l
~ E || Aw; with i..dd wi ~ 10,, and L = 25.
For phase retrieval, we ran DDfire using a default of 800 NFEs for OSF and 100 NFEs for CDP. We set Mddim 0.85 and the hand-tuned (K values listed in Table 6 Also, did use stochastic denoising, since it helped in all metries. Since the likelihood Py)z(y|2) (1r lz]. makes the conditional mean and variance in lines of Alg. 3 intractable, we used the Laplace approximation (Bishop) 2007]. For the special case of Image et OSF, we guided DDfire using the HTO estimate; all other cases, DDfire received no guidance. Appendix H.5 details the guidance.
DDRM.
We ran DDRM for 20 NFEs using the authors' implementation from Kawar et al. (202bb) with minor changes work with codebase.
DifiPIR. We ran DifPPI for 20 NFFs using the authors' implementation from Zhu et al.
2024 without
modification. Hyperparameters were set according to the reported values in Zhu et al. 2023
IIDMM. We ran HIGM for 100 NFEs. Since the authors do not provide a IIDM implementation for noisy inverse problems NVlabs (2023), we coded IIDDM ourselves Python Torch. With problems for which an SVD available, we computed (AAT + (x using the efficient SVD implementation of A from the DDRM codebase Kawar et al. 2022b), and otherwise used CG.
We ran DDS for 100 NFEs. We leveraged the authors' implementation in Chung et al. (024b)) to reimplement DDS in our codebase. We tuned the DDS regularization parameter idds via grid search and used Addim = 0.8 and 50 CG iterations.
DDS.
DPS. For the linear inverse problems, we ran DPS for 1000 NFEs using the authors' implementation from Chung al. (2023b) without modification, using the suggested tuning from Chung et al. (2023a Sec. D.1).
24

---

Published in Transactions on Machine Learning Research (08/2025)
For phase retrieval, we also used 1000 NFEs, but made minor adjustments to the DPS authors' implemen- tation to accommodate the likelihood Pyız(y]2) = N ( (v:l=l,o4), which was used by all methods for fairness. We used grid-search tuning to minimize LPIPS on a 100-image validation set.
RED-diff We ran RED-diff for 1000 NFEs using the authors' implementation from NVlabs 2023)) with minor changes to work with our codebase. We tuned the RED-diff learning ratc, X, and data fidelity weight is to minimize LPIPS with 100-image validation set. For phase retrieval, we made similar code modifications handle our likelihood as with DPS.
We ran DAPS for 1000 NFEs using the authors' implementation from Zhang al. (2024), with minor changes to work in codebase. The tuning parameters were set in Zhang et al. (2025 For phase retrieval, made similar code modifications to handle our likelihood with DPS.
DAPS.
We ran DOLPH for 1000 NFEs. Since the DOLPH authors did not release implementation, implemented it ourselves in Porch and used grid-search to find the step-size that minimized LPIPS on 100-image validation set. For phase retrieval, we made similar code modifications to handle our likelihood as with DPS.
DOLPH.
We translated the MATLAB implementation of HIO from Metzler (2018) to Python and set the step-size parameter to 0.9, We then followed the runtime procedure described Metzler et al. (2018): For the OSF experiments, HIO first rum times, for 50 itcrations each, from a random initialization. The estimate & with the lowest measurement residual |y A || then used to reinitialize HIO, after which it is run for 1000 more iterations. Finally, the second and third color channels in the result are shifted and flipped as needed to best match the first color channel. For the CDP experiments, HIO is run once for 200 iterations from random initialization.
HIO.
prDecp. We used the Python implementation from Hekstra et al. 2018). As recommended in Metzler et al. (2018), we initialized prDeep with the HTO estimate for OSF experiments and with an all-ones initialization for CDP experiments (The HIO estimate was also used by DDfire for ImageNet OSF), We tuned on a grid minimize LPIPS on a 100-image validation set.
H.5 DDfire with guidance
Algorithm 7 details an extension of DDfire that takes guidance from an externally provided estimate Eguide. The approach can be explained as follows. Recall from 33 and App. that the iteration-k VE DDIM estimate 3y assumed to obey the model
p(æx|20) = N (axx; Co, I)
(82)
If we set guuide Fguide thide with us ~ (0, and sufficiently large guide, then it is reasonable to model
p( pa@gid (wo) = N NN Do, o 2 guide D)
(83)
Furthermore, if Un is generated independently of an each iteration k, then
p(a Tguide (a) = Nazr; Fo, of D. (T (Fguide; Fe, o2 guidd I)
(84)
guide I) = Na Dz. T) guide o²
(85)
σ² guide
o
rk
Lk +
Eguide
x Na I) for
o +
o
guide
o +
guide
(86)
1
-1
Ux. 2
k
guuide
=N(rgz0 vel),
(87)
25

---

Published in Transactions on Machine Learning Research (08/2025)
Algorithm 7 DDfire with guidance
Require: y, A. By or Pylz, [ookkmn JN 1K 1: иK ~ (0, o
Tlddim 2 O, Eguide guide
for h K ,,.,, do
T guide x guide
guide Uk
Uk N(0,1)
b Randomized guidance
T guide
o I guide
4:
Pk
Lk + Ok o
Guided estimate
guide
guide
Pk k
D Approzimate error-uariance of Tk
guide
Bojk
= FIRE(y: y. A, *, rk: Vuk: P)
FIRE via Alg. OT Alg.
o (o) o x
Ck 7ddim
o
0 k-1 - c
Kk
1
To k: Shaee nn (N,
DDIM updale
k
o
return Dojk
which means that i) rn sufficient statistic for the estimation of zo from an and g'uide (Poor 1994) and ii) that тą can be modeled an AWGN-corrupted version of By with AWGN variance Vi- Thus, whereas standard DDfire passes (z) to FIRE in iteration k, the guided version of DDfire instead passes (rk, ve) to FIRE at iteration k. Note that this necessitates that of be replaced by P& when computing N, in (29) Thus concludes the explanation of Alg. 7 As for the choice of o guuide it should be large enough relative to the error variance in Eguide for 83 to hold. Thus suggest choosing of guide to be at least 10x the error variance in guidde which can be estimated using (10). But as dguide grows larger, less guidance is provided. For example, if quide is large relative to o then essentially no guidance will be provided (since ты x æ and V& of for all & E {1...., K} when oquide >> oxx). For ImageNet OSF phase retrieval, set o guide 50x the error variance of @guide = THIO.
H.6 Compute
All experiments were run on a single NVIDIA A100 GPU with S0GB memory. The runtime for each method on the GPU varies, as shown Figure 4
DDfire hyperparameter tuning curves
Figures 10 show PSNR and LPIPS over the parameter grids K E {10, 20, 50, 100, 200, 500, 1000} and 8 E (0.05.0.1.0.2. 0.5, 0.75) for box inpainting, Gaussian deblurring, motion deblurring, and 4x super reso- lution, respectively, using 50 ImageNet. validation images. While there are some noticeable trends, DDfire is relatively sensitive to hyperparemeter selection, particularly in cases where the degradation not global (e.g., in inpainting).
J Additional experimental results
Figure 11 showws LPIPS average runtime (in seconds A100 GPU) to generate single image for noisy OSF phase retrieval on the 1000-sample FFHQ test set. The figure shows that DDfire gives significantly better performance/copplee coopllextty tradeoff than all diffusion-based competitors.
Figure 12 shows SLM-FIRE's o² versus iteration i, for comparison to the true denoiser imput variance | Fo l//d and STM-FIRE's v, for comparison the true denoiser output variance || xo l for
25 FIRE iterations with p = 1.5 for noisy 4x super-resolution t[k] 1000. We that the SLM-FIRE estimates o² and track the true error variances quite closely.
26

---

Published in Transactions on Machine Learning Research (08/2025)
Inpainting
Inpainting
0.1
0.26
0.2
0.5
6 0.75
0.25
20
0.24
19
18
0.21
6 0.1
0.20
0.5
10
6 0.75
10a
10a
101
10a
Figure 7: PSNR and LPIPS tuning results for box inpainting with 50 ImageNct validation images.
Gaussian Deblurring
Gaussian Deblurring
0.05
0.38
0.05
0.1
25.0
0.36
6 0.5
0.5
24.5
LPIS
23.5
0.30
23.0
0.28
22.5
0.26
10
10'
10a
10
10a
10
Figure 8: PSNR and LPIPS tuning results for gaussian deblurring with 50 ImageNet validation images.
Motion Deblurring
Motion Deblurring
0.42
25
0.1
0.1
0.2
0.5
0.40
0.5
6 0.75
6 0.75
24
0.38
23
0.34
22
0.32
21
10P
10a
Figure 9: PSNR and LPIPS tuning results for motion deblurring with 50 ImageNet validation images.
27

---

Published in Transactions on Machine Learning Research (08/2025)
4x Super Resolution
4x Super Resolution
0.05
0.05
- 0.1
0 0.1
25.0
0.5
0.38
5 0.5
0.36
24.0
LPIPS
0.34
23.5
23.0
10a
10
10a
Figure 10: PSNR and LPIPS tuning results for 4x super resolution with 50 ImageNet validation images.
DOLPH
0.7
0.5
0.5
HIO
LPIPS
0.4
DDfire-50
0.3
RED-diff DPS
DAPS
DDfire-100
DDfire-200
prDeep
DDfire: 500
DDfire-1000
10
20
30
40
50
60
70
80
90
Single image sampling time on one A100 GPU (s)
Figure 11: LPIPS vs. single image sampling time for noisy OSF phase retrieval on an A100 GPU. The evaluation used 1000 FFHQ images.
Figure 13 shows a similar figure for GLM-FIRE. In particular, shows GLM-FIRE's σ² versus itcration i, for comparison to the true denoiser input variance ||r - 2o x0/l3/d d, GLM-FIRE's v, for comparison to the true denoiser output variance || - 2n /2//d, d, and GLM-FIRF's og for comparison the true useudo-measurement variance j Aw //m for noisy FFHQ phase retrieval t[k] 1000. We see that the GLM-FIRE estimates o² V, and σ track the true variances quite closely.
Example recoveries for noisy phase retrieval with FFHQ images are given in Figure 14
Additional example recoveries for the noisy linear inverse problems with FFHQ images are shown in Figure 15
28

---

Published in Transactions on Machine Learning Research (08/2025)
|| xol13/d
10
FIRE a²
10a
oo - xoll//d
FIRE
Variance
10a
10
10
15
25
SLM-FIRE iteration
Figure 12: SLM-FIRE o², true denoiser input variance variance 200 xo 20112/d d vs. SLM-FIRE iteration for noisy 4x super-resolution at [[k] 1000 for a single validation sample Zg-
x. 1o--oll/// SLM-FIRE v, and true denoiser output
||r ||r-xolf xol13/d
GLM-FIRE O²
1
10a
GLM-FIRE
Variance
102
llýo Axol13/m
GLM-FIRE oz
101
10-1
5
10
GLM-FIRE iteration
Figure 13: GLM-FIRE a2, true denoiser input variance | xx 1//3//d GLM-FIRE, v, and true denoiser output
variance - - xn / GLM-FIRE o², and noise variance || Awolle/m vs. GLM-FIRE iteration for noisy CDP phase retrieval at [k] = 1000 for a single validation sample vo.
29

---

Published in Transactions on Machine Learning Research (08/2025)
DOLPH
RED-diff
prDeep
DDfire
050
DOLPH
RED-diff
DAPS
prDp
DDfire
OSF
20g
DOLPH
RED-diff
DAPS
prDeep
DDfire
CDP
y
F8g HIO
DOLPH
DPS
RED-diff
DAPS
prDeep
DDfire
CDP
Figure 14: Example recoveries from noisy phase retrieval with FFHQ images.
30

---

Published in Transactions on Machine Learning Research (08/2025)
y
a:0
DDRM
DifFIR
IGDM
DPS
RED-diff
DAPS
DDfire
box impainting
y
4
DDRM
DiffPIR
TTGDM
DPS
RED diff
DAPS
DDfire
Gaussian deblurring
y
40g
DRAFIIR
ITGDM
DPS
RED-diff
DAP5
DDfire
deblurring
y
2tg
DDRM
DifPPIR
HIGDM
DPS
RED-diff
DAPS
DDfire
super-resslu rescluttont
Figure 15: Example recoveries from noisy lincar inverse problems with FFHQ images.
31


Reviews:
[{"review_id": "oCmk0h6rHr", "reviewer": "Reviewer_31od", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength**\n- The algorithms are thoughtfully designed and technically sound.\n- The proposed method demonstrates competitive performance and relatively fast inference compared to several benchmark methods.\n\n**Weakness**\n\n- Clarity of Writing: The paper’s exposition is often difficult to follow, even for readers with a strong background in the field. For instance, the phrase “possibly scaled version of $x_0$\" in the second paragraph of the introduction is ambiguous and could benefit from clarification. The presentation becomes increasingly unclear in the method section. In Section 3.1, Steps S1–S4 introduce key variables such as $\\nu, \\rho, \\hat{x}$ and $c$ without immediately defining them. These definitions are only gradually revealed across the section, which impedes readability. A more effective structure would be to begin with an intuitive overview of the method, followed by a systematic introduction of notations and their roles.\n\n- Comparison to DDS: The proposed approach appears closely related to DDS. While the paper briefly notes that FIRE differs from DDS and DiffPIR in that $\\nu$ is explicitly estimated, this distinction is not elaborated on sufficiently. A more thorough explanation of how FIRE diverges from DDS—both in terms of algorithmic structure and inference dynamics—would greatly improve clarity. Additionally, a direct empirical comparison with DDS (e.g., on one or two benchmark tasks) is warranted to contextualize the performance of FIRE.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a novel approach aimed at improving the approximation of the posterior expectation $E[X_0|X_t, y]$ in inverse problems. The proposed method, termed FIRE, incorporates a technique referred to as colored renoising, where carefully structured colored Gaussian noise is added to the output of a linear estimator. This is done to ensure that the input to the denoiser mimics additive white Gaussian noise (AWGN), thereby reducing the distribution shift between training and inference. Two variants of FIRE are proposed: one tailored for linear inverse problems with AWGN-corrupted observations, and another for generalized models, which builds on expectation propagation (EP). Finally, the authors integrate FIRE into the DDIM framework, resulting in a new posterior sampler dubbed DDfire. Experimental results demonstrate that the proposed method achieves both high performance and computational efficiency across several benchmarks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would like to request the following:\n\n- Improve the clarity and structure of Section 3.1 by: (1) Providing intuitive explanations before delving into mathematical details, (2) Clearly defining the variables at their first appearance.\n\n- Include a more explicit comparison between DDS and FIRE: (1) Clearly articulate algorithmic differences; (2) Provide at least one quantitative benchmark comparing the two methods.", "broader_impact_concerns": "There are no additional concerns related to broader societal impact that require further elaboration beyond the existing statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0rTIjjFbmh", "reviewer": "Reviewer_RZJr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Strengths\n\nThe authors provide a rigorous theoretical analysis of flaws in existing methods, particularly focusing on how noise variance estimation can mitigate distribution shift during both training and inference.\n\nEmpirically, their method outperforms all existing approaches across varying numbers of function evaluations (NFEs), as demonstrated in Figure 4. This is particularly impressive, as most competing methods are only effective within specific NFE regimes.\n\n- Weaknesses\n\nThe paper’s presentation does not match the quality of its results. For instance, in the theoretical section, certain concepts are introduced without clear definitions, requiring readers to infer their meaning. This ambiguity necessitates repeated reading to fully grasp the content. Additionally, there are theoretical flaws and missing points, which are detailed in the following section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work addresses the inaccurate estimation of the gradient of the measurement-conditional score function. The authors propose a novel approach that iteratively re-estimates and \"renoises\" the estimate multiple times per diffusion step. Their method, termed Fast Iterative REnoising (FIRE), injects colored noise to ensure pre-trained diffusion models consistently encounter white noise, thereby minimizing distribution shift during inference. Theoretically, the authors prove the convergence of the FIRE algorithm. By integrating this estimation method with the DDIM reverse process, they develop DDFire, which achieves state-of-the-art performance in both accuracy and runtime across several linear inverse problems.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Method\n\nThe stochasticity in reverse diffusion models is known to play a crucial role in correcting inaccurate trajectories. However, the relative contributions of (a) correct variance computation and (b) noise injection should be more thoroughly analyzed to better understand their individual impacts.\n\n- Theoretical Validity\n\nThe Gaussian assumption in Eq. (8) appears flawed, which calls into question the validity of subsequent derivations (Eqs. 9-14). Could the authors empirically verify any of these formulas? For instance, does the MMSE estimate in Eq. (11) hold in practice, or are there other formulas that could be more directly linked to the algorithm's performance?\n\n- Algorithm\nRegarding Eqs. (15)-(16): How should $\\sigma^2 \\ge \\nu$ be chosen in practice? Since $\\bar{x}$ is the denoising result given variance $\\sigma^2$, and $\\nu$ is computed using $\\bar{x}$, there appears to be a circular dependency. Could the authors clarify whether my understanding is correct and explain how this is resolved?\n\n- Approximation Error\n\nWhat is the quantitative impact of the colored Gaussian noise approximation error (Eq. 17) on the final reconstruction error? A more detailed analysis of this relationship would strengthen the theoretical foundation.\n\n- Experiments\n\nWhile Figure 4 shows impressive results, how does the method perform on more challenging inverse problems? For example:\n1. Different tasks beyond those shown\n2. More diverse datasets\n3. FIRE combined with other state-of-the-art methods\n\n- Minor Comments\n\n1. The abstract claims that existing approximations perform poorly early in the reverse process, but this point is not fully developed in the main text.\n\n2. In the introduction:\n\nHow does FIRE relate to Plug-and-Play (PnP) algorithms? What implicit losses are traditionally used in such frameworks?\nMany existing algorithms use pre-trained diffusion models for inverse problems - do they operate in a PnP-like manner? Clarifying these connections would better motivate FIRE's approach.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "IK4xjfUgwD", "reviewer": "Reviewer_tbY8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: \n\n- The paper provides a theoretical convergence guarantee for the FIRE algorithm and empirically demonstrates that DDfire achieves significantly improved reconstruction performance across a variety of tasks.\n\n- Despite performing multiple inner iterations in FIRE, DDfire maintains runtime efficiency.\n\n- The paper offers comprehensive studies for FIRE and DDfire frameworks.\n\nWeakness and questions:\n- colored re-noising step is the key contribution of the paper, it would be beneficial to include a review of related literature and empirical comparisons with other plug-and-play methods that also incorporate re-noising strategies. For example, \n\n   M Renaud, Plug-and-play image restoration with stochastic denoising regularization\n\n   Yuyang Hu, Stochastic Deep Restoration Priors for Imaging Inverse Problems\n\n- how does the level of ill-posedness of A affect the reconstruction performance of both the proposed method and existing baseline approaches?\n\n- how sensitive is DDfire to the choice of hyperparameters", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work aims to more accurately estimate the gradient of the measurement-conditional score function for solving inverse problems within the plug-and-play (PnP) framework. The authors first introduce FIRE, an iterative method that combines neural denoising with MMSE estimation and a novel colored renoising step to ensure that the denoiser input matches the white noise distribution it was trained on. FIRE is then embedded into the DDIM reverse process, resulting in the DDfire sampler. The paper provides a convergence analysis of FIRE and demonstrates, through extensive empirical results, significant improvements in reconstruction quality.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- see weakness above\n\n- in algorithm 1, should d_theta be as part of the input", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)