SITCOM: Step-wise Triple-Consistent Diffusion Sampling For Inverse Problems
Authors: Ismail Alkhouri
Shijun Liang
Cheng-Han Huang
Jimmy Dai
Qing Qu
Saiprasad Ravishankar
Rongrong Wang
Venue: ICML 2025 poster (2025)
Keywords: Diffusion Model; Inverse Problems; Image Restoration; MRI
Primary area: applications->computer_vision

Abstract: Diffusion models (DMs) are a class of generative models that allow sampling from a distribution learned over a training set. When applied to solving inverse problems, the reverse sampling steps are modified to approximately sample from a measurement-conditioned distribution. However, these modifications may be unsuitable for certain settings (e.g., presence of measurement noise) and non-linear tasks, as they often struggle to correct errors from earlier steps and generally require a large number of optimization and/or sampling steps. To address these challenges, we state three conditions for achieving measurement-consistent diffusion trajectories. Building on these conditions, we propose a new optimization-based sampling method that not only enforces standard data manifold measurement consistency and forward diffusion consistency, as seen in previous studies, but also incorporates our proposed step-wise and network-regularized backward diffusion consistency that maintains a diffusion trajectory by optimizing over the input of the pre-trained model at every sampling step. By enforcing these conditions (implicitly or explicitly), our sampler requires significantly fewer reverse steps. Therefore, we refer to our method as **S**tep-w**i**se **T**riple-**Co**nsistent Sa**m**pling (**SITCOM**). Compared to SOTA baselines, our experiments across several linear and non-linear tasks (with natural and medical images) demonstrate that SITCOM achieves competitive or superior results in terms of standard similarity metrics and run-time.

SITCOM: Step-wise Triple-Consistent Diffusion Sampling For Inverse Problems
Ismail R. Alkhouri 1 Shijun Liang'
Cheng-Han Huang? Jimmy Dai
Qing Qu'
Saiprasad Ravishankar 23
Rongrong Wang" 24
Abstract
1. Introduction
Diffusion models (DMs) are a class generative models that allow sampling from a distribution learned over a training set. When applied to solv- ing inverse problems, the reverse sampling steps are modified to approximately sample from measureeeent-condiioone distribution. However, these modifications may be unsuitable for certain settings (e.g.. presence of measurement noise) and non- non-linear tasks, as they often struggle to correct errors from earlier steps and generally require large number of optimization and/or sampling steps. To address these challenges, we state three conditions for achieving measurement-consiseen diffusion trajectories. Building on these condi- tions, we propose a new optimization-bel pling method that not only enforces standard data manifold measurement consistency and forward diffusion consistency, as seen in previous stud- ies, but also incorporates our proposed step-wise and network- backward diffusion con- sistency that maintains diffusion trajectory by optimizing over the input of the pre-trained model at every sampling step. By enforcing these con- ditions (implicitly or explicitly), our sampler re- quires significantly fewer reverse steps. There- fore, refer
Inverse problems (IPs) arise in wide range of science and engineering applications, including computer vision (Li et al., 2024), signal processing (Byme, 2003), medical imaging (Alkhouri et al., 2024b), remote sensing (Levis et al., 2022), and geophysics (Bnil am & Al-Khoury, 2020). In these applications, the primary goal is to recover an un- known image or signal x E R" from its measurements or degraded image y R", which are often corrupted by noise. Mathematically, the unknown signal and the mea- surements are related as
A(x) + n,
where A(-) R" R (with S. n)) represents the linear or non-linear forward operator that models the measurement process, and R denotes the noise in the measurement domain, e.g., assumed sampled from Gaussian distribution (O0, I)), where σy > 0 denotes the noise level. Exactly solving these inverse problems is challenging due to their ill- posedness in many settings, requiring advanced techniques to achieve accurate solutions.
sam-
Deep learning techniques have recently been utilized form priors aid in solving these problems (Ravishankar et al., 2019; Lempitsky et al., 2018). One framework that has shown significant potential is the use of generative models, particularly diffusion models (DMs) (Ho et al., 2020). Given training dataset, DMs are trained to learn the underlying distribution p(x).
method Step-wise Triple- Consistent Sampling (SITCOM). Compared to SOTA baselines, our experiments across several linear and non-linear tasks (with natural and med- ical images) demonstrate that SITCOM achieves competitive or superior results terms of stan- dard similarity metrics and run-time.
During inference, DMs enable sampling from this learned distribution through iterative procedure (Song et al., 2021b). When employed to solving inverse problems, DM- based IP solvers often modify the reverse sampling steps to distri-
allow sampling from the cesiuurmments-coodiiiiin bution p(xly) (Chung et al., 2023b; 2022; Cardoso et al., 2024). These modifications typically rely on approxima- tions that may not be suitable for all tasks and settings, and in addition to generally requiring many sampling iterations, often suffer from emoor accumulated during early diffusion sampling steps (Zhang et al., 2025).
Eqqual contribution 'Electrical Engincering Computer Sci- ence at University of Michigan-Ann Arbor "Computational Math- ematics, Science, & Engineering at Michigan State University (MSU) 'Biomedical Engineering at MSU "Mathematical Sci- ences MSU. Correspondence mailal @ umich.edu; alkhour3 @ mdu..
Ismail Alkhouri <is-
Proceedings of the 4dnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
In most DM-ba IP solvers, these approximations are designed enforce standard measurement consistency on

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
the estimated image (or posterior mean) at every reverse sampling iteration, as in (Chung et al., 2023b), and may also include resampling using the forward diffusion process (which we refer to as forward diffusion consistency), such as in (I (Lugmayr et al., 2022; Song et al., 2023a))
forward process and the reverse process
3r
dx: = 2 Xidl + v BBdw.
(la)
dx, - x + Vxx, (ogp((x)) dt V Bidw, (1b)
key bottleneck in DMs is their computational cost, as they are slower than other generative models due to the large number of sampling steps. Although various methods have been proposed to reduce sampling frequency (e.g., (Song et al., 2023c)), these improvements have yet to be fully realized for DMs applied to IPs. Most existing methods still require dense sampling, which continues to pose speed challenges. Based on the aforementioned discussion, in this paper, we make the following contributions.
where B :......, T + (0. 1) is pre-defined function that controls the amount of additive perturbations at time t, w (resp. w) is the forward (resp. reverse) Weiner process (An- derson, 1982), Pt Pt((x) is the distribution of x, at time 4, and Xx, log pt (xxx the score function that replaced by a neu- ral network (typically time-encoded U-Net (Ronneberger et al., 2015)) 0. In practice, given the score function so. the SDEs can be discretized as in (2) where n), 1 (0,1)).
RR x {O...., T} + R", parameterized by
Formulating three conditions for DM-based IP solvers: By identifying key issues in accelerating DMs for IPs, we present a systematic formulation of three conditions where the goal is to (i) improve the accuracy of each intermediate reconstruction and (ii) maintain a diffusion trajectory.
X = V1 3,x,- + VB
(2a)
X
1
B X + 3rsu (X:: 3iou
0) + Ben
(2b)
VI
When employed to solve inverse problems, the score func- tion in (1) is replaced by a conditional score function which, by Bayes' rule, is
Introducing the step-wise Backward consistency: Based on the pre-trained network regularization, we pro- pose the step-wise backward diffusion consistency by leveraging the implicit bias of the pre-trained network at each iteration.
Vx,
Vx, log (xxs) + Vx, log (y)
Solving the SDEs in (1) with the conditional score is re- ferred to as posterior sampling (Chung et al., 2023b). As there doesn't exist a closed-form expression for the term Vx, log Pe (y) (x.) (which is referred to the measurements matching term in (Daras et al., 2024)), previous works have explored different approaches, which we will briefly discuss below. We refer the reader to the recent survey in (Daras et al., 2024) for an overview on DM-based methods for solving IPs.
Presenting a new optimization- sampler: We present an optimization- baaaed sampler that allows for arbitrary sampling step sizes. We refer to our method as Step-wise Triple-Consistent Consistent Sampling (SITCOM). The formulation, initialization, and regularization of the opti- mization problem along with the preceding steps ensure that all consistencies/conndiiioss are enforced with mini- mal deviation at every sampling iteration.
A well-known method is Diffusion Posterior Sampling (DPS) (Chung et al., 2023b), which uses the approxima- tion p(y|xx) R p(y (8)) where 8x(x)) (or simply 8u) is the estimated image at time / as a function of the pre-trained model and x, (Tweedic's 's formula (Vincent, 201 I)), given as
Extensive Evaluation: We evaluate SITCOM one medical image reconstruction task (MRI) and 8 image restoration tasks (5 linear and 3 non-linear). Compared to lcading baselines, our approach consistently achieves either state-of -hee--rr-t---tttt-e or highly competitive quantitative results, while also reducing the number of sampling steps and, consequently, the computational time.
1
{x())) =
X - VI- ((xxx))(x Vār
(3)
=: =: =(xittt,)))
where
2. Background: DMs & Their Usage for Solving IPs
c0 --11==-1- = (180 !),
In this section, we discuss preliminaries on Diffusion Mod- els (DMs) and their usage in solving IPs.
output the noise in X2 (Luo, 2022), and ã = IT' dj with ar = 1- 3. We call the function f. defined in (3), as "TTeeddie-nttoork denoiser" (also termed as 'posterior mean predictor' in (Chen et al., 2024)).
Pre-trained DMs generate images by applying a pre-defined iterative denoising process (Ho et al., 2020). In the Variance- Preserving Stochastic Differentiable Equations (SDEs) set- ting (Song et al., 2021 2021b;a), bba), DMs are formulated using the
Tweedie's 's formula, like in our method, is also adopted in other DM-based solvers such as (Rout et al., 2023; Chung
2

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
al., 2023c; Wang al., 2022). The drawback of most of these methods is that they typically require large number of sampling steps.
the measurements y into these steps, common approach adopted in previous works that demonstrate superior perfor- mance (e.g., (Song et al., 2023a; Zhang et al., 2025; Li et al., 2024)) is to encode Xa. computed via (3), in the following
The work in ReSample (Song et al., 2023a), solves an op- timization problem on the estimated posterior mean in the latent space to enforce a step-wise measurement consistency, requiring many sampling and optimization steps.
optimization problem:
(x(x,) = arg min || - y11 + A||x - x0xxx) (4)
The work in (Mardani et al., 2024) introduced RED-Diff, variational Bayesian method that fits a Gaussian distribution to the posterior distribution of the clean image conditional on the measurements. RED-Diff solves also an optimization problem using stochastic gradient descent (SGD) to mini- mize a data-fitting term while maximizing the likelihood of the reconstructed image under the denoising diffusion prior (as a regularizer). However, the SGD process requires mul- tiple iterations, each involving evaluations of the pre-trained DM on different noisy image at some randomly selected time. While RED-diff reduces the run-time, their qualitative results are not highly competitive on some tasks in addition to requiring an external pre-trained denoiser.
where À E R+ is regularization parameter. The (x,) obtained from (4) is close to X (x,) while also remaining consistent with the measurements. When using X (xx) to sample x,-1, the second formula in (2) can be rewritten as in (5), where the derivation is provided in Appendix A.
Vor(1 - x4-1))
V 10t-19t
X1-1
X2 +
Xx
(x1)
(5)
1- at
1- at
+ BBm
By substituting X (xx) in the second term of (5) with the measurement-consissee 8/ (xx), the modified sampling for- mula becomes:
Recently, Decoupling Consistency with Diffusion Purifica- tion (DCDP) (Li al., 2024) proposed separating diffusion sampling steps from measurement consistency by using DMs as diffusion purifiers (Nie et al., 2022; Alkhouri et al., 2024b), with the goal of reducing the run-time. However, for every task, DCDP requires tuning the number of forward dif- fusion steps for purification for each sampling step. Shortly after, Decoupled Annealing Posterior Sampling (DAPS) (Zhang al., 2025) introduced another decoupled approach, incorporating gradient descent noise annealing via Langevin dynamics. DAPS, similar to DPS, also requires large num- ber of sampling and optimization steps.
v (1 - 02-1)
V 13t & (x:)
X
x1 +
(6)
1- āc
1- ā
+ BBm.
While this approach effectively ensures data consistency at each step, it inevitably causes X to deviate from the diffusion trajectory, leading to two major issues:
(II): The image (xx). initially constructed through Tweedie's formula, usually appears quite natural (e.g.. columns 3 to 5 of Figure 1); however, the modified version, xxxx). likely to exhibit severe artifacts (e.g., columns 6 to 8 of Figure 1).
3. Step-wise Triple-Consisten Sampling
(I2): Since the DM network, Eg, is trained via minimizing the objective function (denoising score matching (Vincent, 201
3.1. Motivation: Addressing the Challenges in Applying DMs to IPs
E (x ||e (V(oo + v1 -āge, 0/12
Most inverse problems are ill-conditioned and undersam- pled. DMs, when trained on dataset that closely resembles the target image, can provide critical information to allevi- ate ill-conditioning and improve recovery. Despite various previous efforts, a key challenge remains: How to efficiently integrate DMs into the framework of inverse problems? We will now elaborate on this challenge in detail.
finite dataset, it performs best noisy images ly- ing in the high-density regions of the training distribu- tion N(x: Vā,xo. (1 - a,) We define an algorithm as forward-consistent if it likely applies eg only to in- distribution inputs (i.e., those from the same distribution used for training). For example, if the forward diffusion used to train Eg adds Gaussian noise, the in-distribution in- put to Eg should ideally be sampled from a Gaussian with specific parameters. If Poisson noise is used in the forward process, inputs drawn from suitable Poisson distributions are more likely to fall within the well-trained region of the network. In summary, forward consistency requires that inputs to Eg during sampling align with the forward process. While the x,-1 generated from (5) is forward-consisterr by
The standard reverse sampling procedure in DMs con- sists of applying the backward discrete steps in (2) for t E (1,T-- 1}. forming the standard diffusion trajec- tory for which xo is the generated image". To incorporate
'Diffusion trajectory refers to the path that leads to an in- distribution image. where the distribution is the one learned by the DM from the training set.
3

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
design, the one generated from the modified formula (6) is not. Therefore, in the latter case, the DM network, 60, may be applied to many out-of-distribution inputs, leading to degraded performance.
network bias, as "network regularization".
By employing network regularization, we can address (II) by ensuring that the data-consistent X is also network- consistent. We refer the latter condition as Backward Con- sistency and define it formally as follows.
We pause to verify our claimed Issue (TI) through a box- inpainting experiment. Columns to 8 of Figure show (x,) at various t'. The results clearly demonstrate suc- cessful enforcement of data consistency, as the region out- side the box aligns with the original image. However, this enforcement compromises the natural appearance of the image, introducing significant artifacts in the reconstructed area inside the box. Details about the setting of the results in Figure are given in Appendix C.1.2.
Definition 3.1 (Backward Consistency), We say a recon- struction X is backward-consistent with posterior mean predictor J( : Eg) at time l if it can be expressed as X1 = f(v;; t, Ea) with some Vr. In other words, backward consistency requires & to be an output of time t.
The subset of images that are in the range of function (i.e., backward-consistent)) is denoted by C, and is defined as
Issue (12) was previously observed in (Lugmayr et al., 2022), which proposed a remedy known as 'resampling'. In this approach, the sampling formula in (6) is replaced by
C. == (f(vitttt ce) v. E R").
(8)
Enforcing X, to be both measurement- and backward- consistent involves solving the following optimization prob- lem:
x V0 + v 71-0
Provided Xn is close to the ground truth Xg. x generated this way will stay in-distribution with high probability. For more detailed explanation of the rationale behind this remedy, we refer the reader to (Lugmayr et al., 2022). This method has since been adopted by subsequent works, such as (Song et al., 2023a; Zhang et al., 2025), and we will also employ it to address (12).
Ky v. argmin (11A(xx)) -y1
(9)
Vi.xo
subject to X²= f(v'; Eo)]
However, (9) may violate forward consistency, as v. could possibly be far from x,. Therefore, we propose adding a regularization term, for which (9) becomes
% V. == argmin (ILA(x()) y112 A1|x, - vill2 (10)
3.2. Network-Regularinn Backward Diffusion Consistency
subject
x = ([V(ii)
Previous studies, such as (Song et al., 2023a; Zhang et al., 2025), mitigate issue (II) by using a large number of sam- pling steps, which inevitably increases the computational burden. In contrast, this paper proposes employing a net- work regularization to resolve issue only accelerates convergence but also enhances reconstruc- tion quality. Let's first clarify the underlying intuition.
During the reverse sampling process, at each time f, with the given Xt, we seek vy in the nearby region (i.e., ||x, - -vill is small), such that v can be denoised by ʃ to produce a clean image x/ (i.c., x²= = f(vi;t, t, eæ)). which is also consistent with the measurements y (i.c., (|A(x)) - yll is small). We need to identify such a vz because x; itself cannot be directly denoised by to yield an image consistent with the measurements. By substituting the constraint into the objective function, the optimization problem in (10) is reduced to
(I1). This approach not
It is widely observed that the U-Net architecture or trained transformers exhibit an effective image bias (Ulyanov et al., 2018; Liang al., 2025; Hatamizadeh et al., 2025). From columns 3 to 5 of Figure 1, we observe that without enfore- ing data consistency, the reconstructed Xo. derived directly from Tweedie-network denoiser f(x;; t, Eq) for each time t, exhibits natural textures. This indicates that the reconstruc- tion using the combination of Tweedie's ' formula and the DM network has a natural regularizing effect on the image.
v. == argmin { [IA(f(() Ea)) - -yl + (11)
A||x:- - v/
with X - ( E0). The benefit of the considered back- ward consistency constraint is shown in columns 9 to 11 of Figure 1. After obtaining X/y the resampling formula in (7) is used to obtain X₁-1-
By definition, the output of f(x;; t, Ea) in (3) represents the denoised version of x, at time t using the Tweedie's 's formula and the DM denoiser eg. Due to the implicit bias of Ea, this denoised image tends to align with the clean image manifold, even if x, does not correspond to a training image, as shown in columns 3 to 5 of Figure 1. We refer to this regularization effect of [xx; t, 60), which arises from
3.2. 1. RELATION TO GENERATIVE PRIORS
The use of network regularization to define step-wise back- ward consistency is inspired by Compressed Sensing with
4

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
Ground Truth
Masked Image
Tweedie's Formula without Measurement Consistency
Measurement Consistency by Equation (4)
Measurement & Backward Consistency by Equation (12)
t' 200
t'= 400
600
r' 200
t' 400
t' 600
t' 200
t' 400
'= 600
Figure Effects enforcing backsard-consistenccy in box-inpainting: Results using Tweedie's 's formula without measurement via (4) (columms to 8), and enforcing both maassurement-consisten and backward-comsistencyy via (11) (columns 9 to 1 11) at different time steps t'. Experimental details are given in Appendix C.1.2.
consistency (columms 3 to 5), enforcing measurement--cnsistte
generative models (CSGM) (Bora et al., 2017). In CSGM, given a pre-trained generative model do with Φ as its weights, can regularize the reconstruction of inverse prob- lems by solving the following optimization problem:
tencies along the diffusion trajectory.
3.4. The Proposed Sampler
Given Xr, Eg, and towards satisfying the above conditions, our method, at sampling time t, consists of the following three steps:
à argmin (z) - -112
(12)
and (²) is the reconstructed image. Here, is the input of the pre-trained network and the optimization vari-
v. == argmin v
||A{
VV - v1 - co(v),
-
able. In this setup, the reconstruction x is constrained to be
Vas
the output of the network 9ø- Similarly, in Definition 3. % required to be the output of the posterior mean estimator f, which is defined by the network Eg.
+ Al|x: - vi
($1)
3.3. Triple Consistency Conditions
X f(viit,e)) = -oa Iiv v1 ā co(i
(S2)
We now summarize the three key conditions that apply at each sampling step.
X var-ix + VI a
(S3)
C1 Measurement Consistency: The reconstruction X consistent with the measurements. This means that
In (S1), the argument of the forward operator is [v'ri 60) Ja. [V] vI ai (V( which is dif- ferent from (S2).
A(%) R y.
C2 Backward Consistency: The reconstruction X/ is denoised image produced by the Tweedie-network denoiser f. More generally, we define the backward consistency to include any form of DM network regularization (e.g.. using the DM probability-foo (PF) ODE (Karras et al., 2022) as in Appendix F) applied to X/-
The minimization in the first step optimizes over the input v of the pre-trained diffusion model at time t, where the first term of the objective enforces measurement consistency for the posterior mean estimated image, satisfying condition C1. The second term serves as a regularization term, implicitly enforcing closeness between v. and x; (i.e., condition C3). with À O acting as the regularization parameter.
The pre-trained DM network Ep is provided with in-distribution inputs with high probability. To ensure this, we apply the resampling formula in (7) and enforce that v, remains close to X2.
C3 Forward Consistency:
The argument of the forward operator in (S1) and the second step in (S2) enforce that v, and X/, respectively, maintain the diffusion trajectory through obeying Tweedie's formula, thereby satisfying the backward consistency condition, C2.
We note that the three considered consistencies are step- wise, meaning they are enforced at every sampling step. This approach contrasts with enforcing these consistencies solely on the final reconstruction at = O, which represents a significantly weaker requirement.
After obtaining the crtasurrmeet-conssitte estimate, X/- given in (S2), it must be mapped back to time / - to generate X;-1- This is achieved through the forward diffusion step in (Sa) as outlined in the forward consistency condition, C3. A diagram of the SITCOM procedure is provided in Figure (left).
C1-C3 aim ensure that all intermediate reconstructions x.(+) f( :t, <g) (with & > 0) accurate possible, allowing us to effectively reduce the number of sampling steps. Previous works, such as (Song et al., 2023a; Zhang et al., 2025), enforce measurement consistency by applying A(Es) = y exactly, whereas DPS (Chung et al., 2023b) does not ensure forward and a step-wise backward consis-
Remark 1. Obtaining the estimated image at time 0 given some x, using the standard DM PF-ODE (Karras et al., 2022) is more accurate compared to the one-step Tweedie's formula. However, since PF-ODE an iterative procedure,
5

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
Algorithm 1 Step-wise Triple-Consistent Sampling (SITCOM).
Input: Measurements y, forward operator A(.), pre-trained DM :o("."). number of diffusion steps N, DM noise schedule ã; for i E {1,...,N] N,, number of gradient updates K, stopping criterion 8, learning rate 7, and regularization parameter X.
Output: Restored image 8.
Initialization: XN (0, At [
1: For each (N, N ,....; 1}. (Reducing diffusion sampling steps)
2: Initialize v; (0) k- X4. (Initialization ensure Closeness: C3) )
3: For each & E {1,..., K3. (Gradient used in Adam achieve measurement & backward consistency: C1, C2)
4: v(M))
- v. || V vI a, Co.V.,
Att)) i >112 + A||x, v.ll M
5:
[||A (v! ( VT a Eo (v #)+t)) - y < 8² (Stopping criterion)
6:
Break the For loop step 3. (Preventing noise overfitting)
7: Assign V. + vv!" (Backward diffusion consistency of v,: C2)
8: Obtain & f(vs,, Voi [+. - vT- 5, EV(V,, i.t)]. (Backward consistency s/;: C2)
9: Obtain X = 186 + /1 - 04-177, n ~ NO 1). (Forward diffusion consistency: C3)
10: Restored image: X Xo.
it requires more computational time. In SITCOM, PF-ODE could replace Tweedie's 's formula in (S2). In the main body of the paper, we chose not to use it, as this would increase the run time, and our empirical results are already highly competitive using Tweedie's 's formula. To show this trade off, refer the reader to the study in Appendix F.
Instead, we use the stopping criterion
|A VV' Vāl
v1 0e co(v:. - -1112 6²,
where 8 E R4 is a aaper--arameter that indicates tolerance for noise and helps prevent overfitting. This is equivalent to enforcing an l2 constraint. In our experiments, similar to DAPS (Zhang et al., 2025) and PGDM (Song al., 2023b), use 5 slightly larger than the actual level of noise in the measurements, i.e., 5 By /m.
conceptual illustration of SITCOM is shown in Figure 2 (right). The DM generative manifold, Mr, is defined as the set of all x2 sampled from
Remark 2. Setting the stopping criterion based on the noise level in the measurements may not be practical, as noise estimators may be inaccurate. However, in Appendix J.2.1, empirically show that SITCOM is not sensitive to the choice of this threshold by demonstrating that even when the stopping criterion is set higher or lower than the actual measurement noise level, the performance remains largely unaffected.
(xx] Nxxx; Vā,xo, (1 II)
and Xy (x).. This set coincides with the entire space Rn equipped with the probability measure induced by the distribution of Xt, which we denote as Pr. In Figure 2 (right), the variation of color around each M₁ indicates the concentration of the measure Pt, with darker colors representing higher concentration.
3.5. SITCOM with Arbitrary Stepsizes
SITCOM's Step (1) and Step (2) enforce measurement consistency and backward consistency, thus map x, to X = ff,;; co) which lies within the intersection of (i) measurement-consisterr set {x) : A(*))
Here, we explain how to apply SITCOM with a large step- size and present the final algorithm. The DM trained with T diffusion steps. Given that our method is designed to sat- isfy the three consistencies, SITCOM requires N & T sam- pling iterations, using step size of At == [ Thus, introduce the index i instead of with a relation t; iAt.
)+{+ A y (the shaded set C, (the yel-
black line) and (ii) the backward-consisee low ellipsoid). Subsequently, x, is generated by inserting X into the resampling formula, which enforces the forward consistency.
The procedure of SITCOM is outlined in Algorithm 1. As inputs, SITCOM takes y, A(.), cg. the number of sampling steps N, a; for all E {l...., , N). the number of optimiza- tion steps K per sampling step. stopping criteria 8, and the learning rate 7,
Handling Measurement Noise: To avoid the case where the first term of the objective in (S)) reaches small values yielding noise overfitting (i.e., when additive Gaussian mea- surement noise is considered, σy > 0), we propose refrain- ing from enforcing strict measurement fitting A(x) = y.
Starting with initializing v; as x, (satisfying condition C3), lines through 6 correspond to the gradient updates
6

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
DM Gencrative Manifolds
Measurement & Backward Consistency
Measurement- Consistent Postertor Mean
Estimation by Tweedie's
Mossurement- Consistent Image at
M,
&
Mc-1
Optimization Over
Forward
Consistency
Network Inpur
X
M
% (())),EE))
Xt-1
X
C, (f(viit,c)) : v, E R*)
Step (1) SITCOM
Step (2)
Step (3) SITCOM
SITCOM
{&6 Al(%)) = y3
Ground Truth
C2
Figure 2: Illustrative diagram of the proposed procedure in SITCOM (left). Conceptual illustration of SITCOM, where Me is the DM generative manifold at time and C. is the subset of images that are backward-sonsistent (righr).
for solving the optimization problem in the first step of SIT- COM, ie., ( _ In lines 5 and 6, the stopping criterion is applied to prevent strict data fidelity (avoiding noise overfit- ting). Following the gradient updates in the inner loop, v, is obtained in line 7, which is then used in line 8 to obtain x/ as specified (52), satisfying condition C2. Note that line requires no additional computation, as the & calculated here was already obtained while checking the stopping con-
network, requiring an additional backward pass. This back- ward pass has the same computational cost as the forward pass. Consequently, this procedure is significantly more efficient than network training. where the network weights are updated instead of the input.
In Appendix D, we offer high-level insights to enhance the understanding of SITCOM, particularly about where its ac- celeration comes from. In essence, interpret SITCOM with K inner iterations as accelerated variant of DPS, under certain approximations, via ADAM or other precon- ditioned GD (Proposition D.1).
dition in line 6. After obtaining the double-consistent Xy, the resampling is applied to map the image back to time /-1 while ensuring X,-1 to be in-distribution, as indicated in line 9 of the algorithm. In the next iteration, the requirement that v is close to x;-1 ensures that the input v to the DM network, co. also in-distribution, thus satisfying the forward-consistency (condition C3).
3.6. Relation to Existing Studies
Both SITCOM and the works in DAPS (Zhang et al., 2025), DCDP (Li et al., 2024), and ReSample (Song et al., 2023a) are optimization- the sampling steps to enforce measurement consistency. DAPS and ReSample involve mapping back to time (as in step of SITCOM) and DCDP uses this step prior to purification. However, there is major difference between them: The optimization variable in these works is the es- timated image at time & (the output of the DM network), whereas in SITCOM, it is the noisy image at time / (the input the network). This means that these studies enforce C1 and C3 (only DAPS and ReSample), but not C2.
The computational requirements of SITCOM are determined by (i) the number of sampling steps N and (ii) the number of gradient steps K required for each sampling iteration. Given the proposed stopping criterion, this results in at most NK Number Function Evaluations (NFEs) of the pre- trained model (forward pass), N K backward passes through the pre-trained model, and NK applications each for the forward operator and its adjoint solve the optimization problem in (S1). With early stopping, the computational cost is lower. For example, for linear operator AA with dimensions m x n, the cost of applying it (or its adjoint) to a vector (mn)). For a network with width M and depth L, the cost for making a forward pass (LM²).
methods that modify (or decouple)
Previous works, RED-diff (Mardani et al., 2024) and DM- Plug (Wang et al., 2024), also utilize the implicit bias of the pre-trained network. However, they adopt the full diffu- sion process as regularizer, applied only once. In contrast, SITCOM uses the neural network as the regularizer at each iteration and focuses specifically on reducing the number of sampling steps for given level of accuracy.
The gradients are computed w.r.t. the input of the DM
Althoughh lines to of Algorithm describe using fixed-step- size gradient descent for (S1), we note that, in our experiments, we use the ADAM optimizer (Kingma & Ba, 2015).

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
FFHQ
ImageNet
Task
Method
PSNR (1)
SSIM (t)
LPIPS (
Run-time ())
PSNR (1)
SSIM (t)
LPIPS (L)
Run-time ())
DPS
24,44+0.56
0.801
0.26+a.02
15.6
23.86+034
0.76-004
0.357+00m
142.80 202220
DAPS
29. 29.24
0.851 1unus
0.135
14.40
25 2777
0.802
0.256
129 10
DDNM
28.02 28.02+a075
0.842+0002
0.197
64. 25
23.96:4039
a.767
0.475
76.20
DMPlug
30.18+007
0.862+0.056
0.149
68.2-2112
SR
RED-diff
24.89
0.789
0.324:+0042
18.02::400
PGDM
25. 22:1099
0.798:20177
0,289
26.24:7.20
DCDP
0,825 +000
0,211
19.10=4.00
24. 24.12+1.24
0.772
0.351
15.00=11.22
SITCOM (ours)
L006
+.042
27.00=480
26.35
0.812
0.232
67.20
DPS
23.20
0.754
0.19620003
24.20
19.782077
1
0.312:0002
136.80
DAPS
24.17210n
0.78720012
0.135
11.00
21 21.4330040
0.7362amu
0.218
116.40=15300
BIP
DDNM
24.37
0.792
0.
0.232
61. 11.20.1..z
21 14
0.732am02
0.319
17.00.116.22
DCDP
13.6621167
0.762=aam
0. 2005
16.60
20.45
0.712
0.298
51.62:115.00
SITCOM (ours)
24.68
0 20042
0.121 go0s
21.00 21.00
21 2:98
0.742gmma
0.214
67. 67.20
DPS
28.39
=002 0.844=0042
91. 18.20
24. 24.26+0042
0.772+tam
0.326=0054
136, 136.20=15.00
DAPS
31.02++0.45
0, 0.902
0.098
93. 23.60
28 44:4
0.872++um24
0.135
128. 40::27.00
DDNM
29.93+0.67
0.889+a.02
0.122+0.066
87.0 14.00
29. 29.22+0.55
0.912+0004
0.19140044
12.40
RIP
DMPlug
31.01+046
0.899
0.099
12.02+15a2
DCDP
28.59
0.852
0.202:004
20.14::k00
26.22
0.791 2006
0.289
49.50
SITCOM (ours)
32.05
0,909
0,095=00025
27.00+12.00
29.60=0.75
0.915
0.127
68.40+222.00
DPS
25.52
0.826100s2
0.211 10007
90.00
21.86.045
0.772
0.362:10044
153.00
DAPS
29.22
0.88440005
0.1 14.
84,0 +31.00
26. 12:00.5
0.832
0.245+0002
133.80-31.20
DDNM
28. 22....
0.8672200
0,21622002
93.60
28.0611052
0.879.umm
0.278:200M
105.00 105,00=21.00
GDB
DCDP
26.67 26.67..075
0.835 aane
0. 1m19
21.07:18.00
23.24
0,781
0.343
48.25
DMPlug
29.79 19.
0.883+0045
+0.00
65 :22.12
SITCOM (ours)
20.25
0.89220m2
0.135 g0.00
27.6 20.60
27.40
0.8542004
0.236
66.00
DPS
23.40=142
0.737
0.270
144.00
21.86:2205
0.724
0.357
153.60
223.600224.00
RED-diff
27. 27.35
0.84220082
20044
19.02:2834
MDB
PGDM
27. 27.48+0075
0,848
0.225
24.1+7.45
DAPS
29.66
0.872+0027
0.157
91.60
27 1.2
+.862+amm2
0.196
118.00
SITCOM (ours)
30.34
0.902=007
0.148:004
30.00=7.10
28.65
0.876gam1
0.189
88.80
DPS
17.34
0.67
0.41
90.00
16.82.112
0.64
0.447
130..20
DAPS
30,97
0 10041
0.112
80 80.40
25.761233
0.797
0.255
134,40 1100000
DCDP
28. 22:522
0.892
0. 0.167
108.00 00:27.00
24.25+225
0.778 20.44
0.287=00099
102, 40-33.20
SITCOM (ours)
30.67+3.10
0,915 +0.044
0.122+0.000
28.50=
25.45++278
0.808 sanss
0.246++0058
84.00+22..0
DPS
23. 23.42::2.15
0.757=0042
0.279
93.00+22640
22.57=00t
0.310 0.778=a057
141.00022..0
DAPS
28. 25555
0.833ga052
0.155g0041
85. 25.20
27. 27.65
0.82220056
0.1 0.169
128. 128.40_258.0
NDB
DCDP
28. 18.78
0.162
12..0amm.m
26.5612100
0.803
0.182a0m
89.00
DMPlug
30.31
0.901
0.142
182.4430.00
RED-diff
29.51
0.828 20:8
0.21
31.15
SITCOM (ours)
30.12
2.903
0. 10088
33.45.9..
28.78
0.83220096
0.16gams
75.00
DPS
12288
0.722
0.264
20.00
19.33
0.688
0.503
145.20=227.0
RED-diff DAPS
23.45
0.746
0.257
24.4::3.00
HDR
27.12+0.30
0.8 a.05s
0. 0.166
75.00+21.00
26.30
0.792+0046
0.177
130.80=33.00
SITCOM (ours)
2
0.158 0.832anu22
31.20
26.97
0.167:20052
92,40 a2..00
Table 1: Average PSNR, SSIM, LPIPS, and run-time (seconds) with σy 0.05. The best (resp. second-best) results are bolded (resp. underlined). Values after + represent the standard deviation. Dashes indicate cases where a method did not consider a dataset.
4. Experimental Results
(Appendix F) is empirically more stable than SITCOM and all other baselines on PR. For baselines, we use DPS (Chung et al., 2023b), DDNM (Wang et al., 2022), RED-diff (Mar- dani et al., 2024), PGDM (Song et al., 2023b), DCDP (Li et al., 2024), DMPlug (Wang et al., 2024), and DAPS (Zhang et al., 2025). For latent space DMs, in Appendix E, we in- troduce Latent-SITCOM and compare the performance to ReSample (Song et al., 2023a) and Latent-DAPS (Zhang et al., 2025). We use 100 test images from the validation set of FFH (Karras et al., 2019) and 100 test images from the validation set of ImageNet (Deng et al., 2009) for which the FFHQ-trained and ImageNet-trained DMs are given in (Chung et al., 2023b) and (Dhariwal & Nichol, 2021), respectively, following the previous convention.
4.1. Settings
Our setup for the image restoration problems and noise lev- els largely follows DPS (Chung et al., 2023b). For linear IPs, we evaluate super resolution (SR), Gaussian deblurring (GDB), motion deblurring (MDB), box inpainting (BIP), and random inpainting (RIP). For non-linear IPs, eval- uate phase retrieval (PR), non-uniform deblurring (NDB), and high dynamic range (HDR). For phase retrieval, the run- time is reported for the best result out of four independent runs (applied to SITCOM and baselines), consistent with (Chung et al., 2023b; Zhang et al., 2025). See Appendix G for a discussion on phase retrieval, and how SITCOM-ODE
8

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
For evaluation metrics, we use PSNR, SSIM (Wang al., 2004), and LPIPS (Zhang et al., 2018). All experiments were conducted using single RTX5000 GPU machine. In Appendix C, we show the impact of each individual con- sistency on the performance of SITCOM. Ablation studies are given in Appendix J. For SITCOM, Table 17 in Ap- pendix K. lists all the hyper-parameters used for every task. Our code is available online."
tings prioritize achieving the highest possible PSNR, which comes at the cost of increased computational time. For in- stance, SITCOM demonstrates significant improvements of over 3dB in SR, RIP, and GDB for FFHQ, as well as in RIP, GDB, and HDR for ImageNet, when compared to DCDP, REDdiff, and PGDM. To ensure a fairer comparison, we provide additional results in Appendix 1.2, where all meth- ods are constrained achieve the same PSNR. Under these conditions, SITCOM once again proves to be faster than DCDP, REDdiff, and PGDM.
4.2. Main Results
In Table 1, we present quantitative results and run-time (seconds). Columns
SITCOM-MRI results are given in Appendix H. and visual examples are provided in Appendix L.
(resp. 7 to 10) correspond to the
FFHQ (resp. ImageNet) dataset. The table covers 8 tasks, 3 restoration quality metrics, and 2 datasets, totaling 51 results. Among these, SITCOM reports the best performance in 42 out of 51 cases. In the remaining 8, SITCOM achieved the second- best performance.
5. Conclusion
We introduced step-wise backward consistency with net- work regularization and formulated three conditions to achieve measurement- and diffusion-consistern trajectories for solving inverse imaging problems using pre-trained dif- fusion models. These conditions formed the base of our optimization-basee sampling method, optimizing the dif- fusion model input at every sampling step for improved efficiency and measurement consistency. Experiments across eight image restoration tasks and one medical imag- ing reconstruction task show that our sampler consistently matches or outperforms of-te-af- baselines, under different measurement noise levels.
Generally, SITCOM demonstrates strong reconstruction ca- pabilities across most tasks. SITCOM reports PSNR im- provement of over 1 dB when compared to the second best for the tasks of RIP with FFHQ (with nearly 1/3 of the run- time of DAPS), and in the task of NDB with ImageNet (with nearly one minute less run-time than DAPS and excluding RED-diff).
There are tasks where in terms of PSNR, we present a slight improvement (less than I dB) when compared to the sec- ond best scheme. However, we achieve great speed-up up. Examples include (i) SR where SITCOM requires nearly half of the run-time when compared to DMPlug for FFHQ and PGDM for ImageNet, and (ii) HDR where, when com- pared to DAPS, SITCOM requires less than half the time for FFHQ, and approximately 40 seconds less time for Ima- geNet. This observation is noted in nearly all other tasks.
Impact Statement
The Step-wise Priple-Cossistent Sampling (SITCOM) method improves diffusion models for inverse problems by introducing three consistency conditions. SITCOM ac- celerates sampling by reducing reverse steps while ensuring measurement, forward, and network-regularizze backward diffusion consistency. This enables efficient, robust im- age restoration, particularly in noisy settings, and advances optimization-basee approaches in generative modeling.
There are 9 cases where we report the second best results. In these cases, we slightly under-perforn in one or two of the three restoration quality metrics (i.e., PNSR, SSIM, and LPIPS). An example of this case is GDB on ImageNet where we report the best LPIPS but the second best PSNR (under- performing by 0.66 dB) and SSIM (under-performi by 0.025).
A Acknowledgments
This work was supported by the National Science Foun- dation (NSF) under grants CCF-2212065. CCF-2212066. BCS-2215155, and CAREER Award CCF-2442240. The authors would like to thank Xiang Li (University of Michi- gan) for discussions on compressed sensing using generative models. The authors would also like to thank Michael T. McCann (Los Alamos National Laboratory) for valuable feedback about the setting of our optimization problem.
For PR, DAPS's PSNR is higher than SITCOM's by a small margin. However, in Appendix F, we show that SITCOM with ODE solver achieves better PSNRs than DAPS for the task of PR at a cost of increased run time (see Table 6).
In terms of run-time, SITCOM outperforms DPS, DAPS, DDNM. and DMPlug in almost all cases, as shown in Ta- ble 1. However, DCDP, REDdiff, and PGDM often hibit faster run times compared to SITCOM. This discrep- ancy arises because SITCOM's default hyperparameter set-
httpe: // /tttt/gttttaaee/////o//. wtt.sstteet/tttts
9

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
References
Chung, H., Kim, J., Kim, S., and Ye, J. C. Parallel dif- fusion models of operator and image for blind inverse problems. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023a.
Alkhouri, I., Liang, S., Bell, E., Qu, Q., Wang, R., and Ravishankar, S. Image reconstruction via autoencoding sequential deep image prior. In The eight Annual Conference on Neural Information Processing Systems, 2024a.
Chung, H., Kim, J., Mecann, M. T., Klasky, M. L.,, and Ye, J. C. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Confer- ence on Learning Representations, 2023b.
Alkhouri, I., Liang, S., Wang, R.. Qu, Q. and Ravishankar, S. Diffusion- basid adversarial purification for robust deep mri reconstruction. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024b.
Chung, H., Kim, J., and Ye, J. C. Direct diffusion bridge using data consistency for inverse problems. 2023c.
Chung, H., Lee, S., and Ye, J. C. Decomposed diffusion sampler for accelerating large-scale inverse problems. In The Twelfth International Conference on Learning Represemtations, 2024.
Anderson, B. D. Reverse-time diffusion equation models. Stochastic Processes and their Applications. 12(3):313- 326, 1982.
BniLam, N. and Al-Khoury, R. Parameter identification algorithm for ground source heat pump systems. Applied energy, 264:11441 14712, 2020.
Daras, G., Chung, H., Lai, C.-H., Mitsufuji, Y., Ye, J. C., Milanfar, P., Dimakis, A. G., and Delbracio, M. A survey diffusion models for inverse problems. arXiv preprint arXiv:2410.000 2024.
Bora, A., Jalal, A., Price, E., and Dimakis, A. G. Com- pressed sensing using generative models. In Proceedings of the 34th International Conference on Machine Learn- ing (ICML), 2017.
Deng, J., Dong, W., Socher, R., L.-,,,, Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. leee, 2009.
Byrne, C. A unified treatment some iterative algorithms in signal processing and image reconstruction. Inverse problems, 20(1):103, 2003.
Dhariwal, P. and Nichol, Diffusion models beat gans image synthesis. Advances in neural information processing systems, 2021.
Cardoso, G., Le Corff, S., Moulines, E., et Monte carlo guided denoising diffusion models for bayesian linear in- verse problems. In The Twelfth International Conference on Learning Representations, 2024.
et al, J. Z. fastMRI: An Open Dataset and Benchmarks for Accelerated MRI. https: /ts/trasttriiii med nyu. , arXiv preprint arXiv: 1811.088811..88
Hatamizadeh, A., Song. J., Liu, G., Kautz, J., and Vahdat, A. Diffit: Diffusion vision transformers for image generation. In European Conference on Computer Vision, pp. 37-55. Springer, 2025.
Chen, G., Zhu, F., and Ann Heng, An efficient statistical method for image noise level estimation. In Proceed- ings of the IEEE International Conference Computer Vision, pp. 477-485, 2015.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 2020.
Chen, S. Zhang, H., Guo, M., Lu, Y., Wang, P., and Qu, Q Exploring diw-eimensional subspace in diffusion mod- els for controllable image editing. In The Thirth-eighth Annual Conference on Neural Information Processing Systems, 2024.
Karras, T., Laine, S., and Aila, T. A style-based generator architecture for generative adversarial networks. In Pro- ceedings of the IEEE/CVF conference on computer vision and paltern recognition, 2019.
Chung, H. and Ye, J. C. Score-based diffusion models for accelerated mri. Medical Image Analysis, 80:102479, 2022. ISSN 1361-8415.
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 2022.
Chung. H., Sim, B., and Ye, C. Come-closerddiffuse faster: Accelerating conditional diffusion models for in- verse problems through stochastic contraction. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, 2022.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
10

---

SITCOM: Step-wise Triple-Consistett Diffusion Sampling for Inverse Problems
Lempitsky, V., Vedaldi, A., and Ulyanov, D. Deep image prior. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
Song, B., Kwon, S. M., Zhang, Z.., Hu, X., Qu, Q., and Shen, L. Solving inverse problems with latent diffusion models via hard data consistency. In The Twelfth International Conference on Learning Representations, 2023a.
Levis, A., Srinivasan, P., Chael, A. Ag, R., and Bouman, K. L. Gravitationally lensed black hole emission tomography. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022.
Song, J., Meng, C., and Ermon, Denoising diffusion implicit models. In International Conference Learning Representations, 202
Li, X., Kwon, S. M., Alkhouri, I. R., Ravishankar, S., and Qu, Q. Decoupled data consistency with diffu- sion purification for image restoration. arXiv preprint arXiv:2403. 0405044 2024.
Song, J., Vahdat, A., Mardani, M., and Kautz, J. Pseudoinverse seeudoinverseguuided diffusion models for inverse prob- lems. In International Conference on Learning Represen- tations, 2023b.
Liang, S., Bell, E.,, Qu, Q. Wang, R., and Ravishankar, S. Analysis of deep image prior and exploiting self- guidance for image reconstruction. IEEE Transactions Computationnll Imaging, 2025.
Song, Y., Durkan, C., Murray, I., and Ermon, Maximum likelihood training of score-based diffusion models. Ad- vances in neural information processing systems, 2021b.
Song, Y., Dhariwal, P., Chen, M., and Sutskever, I. Consis- tency models. In International Conference on Machine Learning, pp. 3221 1-32252. PMLR, 2023c.
Liu, C., Freeman, W., Szeliski, R., and Kang, S. B. Noise estimation from a single image. 2006 IEEE Com- puter Society Conference on Computer Vision and Pattern Recognition (CVPR'06) 2006.
Sriram, A., Zbontar, J., Murrell, T., Defazio, A., Zitnick, C. L.,, Yakubova, N., Knoll, F,, and Johnson, P. End- to-end variational networks for accelerated mri recon- struction. In Medical Image Computing and Computer Assisted Intervention-MICCA. 2020. Springer, 2020.
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., and Van Gool, L. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022.
Tamir, J. I., Ong, F., Cheng, J. Y., Uecker, M., and Lustig, M. Generalized magnetic resonance image reconstruction using the berkeley advanced reconstruction toolbox. In ISMRM Workshop on Data Sampling & Image Recon- struction, 2016
Luo, C. Understanding diffusion models: unified perspec- tive. arXiv preprint arXiv:2208. 1970, 2022.
Mardani, M., Song, J., Kautz, J., and Vahdat, A. A varia- tional perspective on solving inverse problems with diffu- sion models. In The Twelfth International Conference on Learning Representations, 2024.
Tran, P., Tran, A. T., Phung, Q., and Hoai, M. Explore image deblurring via encoded blur kernel space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021.
Nie, W., Guo, B., Huang. Y., Xiao, C., Vahdat, A., and Anandkumar, A. Diffusion models for adversarial purifi- cation. In International Conference on Machine Learning, pp. 16805-168221 PMLR, 2022.
Uecker, M., Lai, P., Murphy, M. J., Virtue, P., Elad, M., Pauly, M., Vasanawala, S. S., and Lustig, M. Es- pirit- an eigenvalue approach to autocalibrating parallel mri: where sense meets grappa. Magnetic resonance in medicine, 71(3):990-1001, 2014.
Ravishankar, S., Ye, J. C., and Fessler, J. A. Image recon- struction: From sparsity to data-adaptive methods and machine learning. Proceedings of the IEEE, 108(1):86 109, 2019.
Ulyanov, D., Vedaldi, A., and Lempitsky, V. Deep image prior. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolu- tional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisterd Inter- vention - MICCAI 2015, pp. 234-241, 2015.
Vincent, P. A connection between score matching and de- noising autoencoders. Neural computation, 23(7):1661- 1674, 2011.
Rout, I.., Raoof, N., Daras, G., Caramanis, C., Dimakis, A., and Shakkottai, S. Solving linear inverse problems provably via posterior sampling with latent diffusion mod- els. Advances in Neural Information Processing Systems, 2023.
Wang. H., Zhang, X., Li, T., Wan, Y., Chen, T., and Sun, J. Dmplug: A plug-in method for solving inverse prob- lems with diffusion models. In The Thirty- Annual Conference on Neural Information Processing Systems, 2024.

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
Wang, Y., Yu, J., and Zhang, J. Zero-shot image restora- tion using denoising diffusion null-space model. In The Eleventh International Conference on Learning Repre- sentations, 2022.
Wang. Z., Bovik, A. C., Sheikh, H. and Simoncelli, E. Image quality assessment: From error visibility to struc- tural similarity. IEEE transactions image processing, 13(4):600-6) 660000662,000 2004.
Zhang, B., Chu, W., Berner, Meng, C., Anandkumar, A., and Song. Y. Improving diffusion inverse problem solving with decoupled noise annealing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2025.
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
12

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
Appendix
In the Appendix, we start by showing the equivalence between the second formula in (2) and (5) (Appendix A). Then, present experiments to
discuss the known limitations and future extensions of SITCOM (Appendix B). Subsequently, highlight the impact of the proposed backward consistency and other consistencies in SITCOM (Appendix C). Appendix D provides further insights to aid in understanding SITCOM. In Appendix E and Appendix F, we present the latent and ODE versions of SITCOM, respectively. This is followed by a discussion on phase retrieval (Appendix G). Appendix shows the results of applying SITCOM on multicoil Magnetic Resonance Imaging (MRI). In Appendix I, we provide further comparison results, and in Appendix J, we perform ablation studies examine the effects of the stopping criterion and other componnent/hyppe--praaeeee parameters in SITCOM. Appendix K covers the implementation details of tasks, baselines, and SITCOM's 's hyper-parameters. followed by visual examples of restored/recoonsrruieed images (Appendix L).
A. Derivation of Equation (5)
From (Luo, 2022), we have
So sx(x1,
Eo (x(x))))
(13)
VI āt
Rearranging the Tweedie's formula in (3) to solve for eØ 1) yields
X; V (xx))(X))
(xi,))
(14)
VI āi
Now, we substitute into the recursive equation for x,-1
1
X1-1
VI
B
[x,
Biso(xi,tt] + BBm
1
---=_c((x.1)) + Ben
+ 3e
VI
VI- ae
1
31
Ea (x(x,t))
+ VBn
VI-
B
V
a
1
3
Xx ((rx
) + Ben
X2
VI
B
0
v1 O2
1
3
X
(x,
Ban
V1 Br
ā
1
1
B
van xx((x)
+ -Bm
X1
VI D
0t
at
(1 a.
-8)
VarB,
xx((x)) /im
X1
VI 87 (1 - ②1)
VI Br (1 (1-6) (1-a+)
(a: a1)
Vargi
*xo(x)) + BBn
Xt.
Vau (1 -ar)
Vot (1 - -
(Vot (ar((t11)
1at-13r (x())) VBn.
Xt +
1-ai
00 (1 - -1-1)
10
xx(x)) + BBTt
X2
Cr.
which is equivalent to the second formula in (2).
13

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
B. Limitations & Future Work
The stated conditions and proposed sampler are limited to the non-blind setting, as SITCOM assumes full access to the forward model, unlike works such as (Chung et al., 2023a). which perform both image restoration and forward model estimation. Additionally, SITCOM uses 2D diffusion models for 2D image reconstrnctionrrettortioo only. Extending SITCOM to the 3D setting may require specialized regularization and modifications.
For future work, we aim to address the above two settings (3I) reconstruction and/or blind settings) and explore its applicability in medical 3D image reconstruction.
C. Impact of Each Individual Consistency on SITCOM's Performance
Here, we show the effect of removing each of the individual consistencies demonstrate their necessity.
C.1. Impact of the Backward Consistency
In SITCOM, a key contribution the imposition of step-wise backward consistency, which allows us to fully exploit the implicit regularization provided by the network. Removing stepwise backward consistency is equivalent to removing the network regularization constraint & [vz; 0, from SITCOM. More specifically, in our algorithm, the optimization variable becomes the output of the pre-trained network for which, given some x,, the steps at sampling time / are:
X = = (| y112.
(15a)
X v v 0t-177 7 ~ (O,,I)
(15b)
where the initialization of x' in (15a) is set f(x;; 1, 60). We call the resulting algorithm as "SITCOM without backward consistency". In (15a), we already obtain an estimated image. Therefore, there is no need for second step. Furthermore, the pre-trained model here is only used for initializing the optimization in (15a), SITCOM without backward consistency
does not require to back-propagate through the pre-trained DM network which will subsequently result in reduced run-tin unntimmes. However, the steps in (15) require more sampling iterations to converge to decent PSNRs. This statement will be supported in the next subsubsections.
We remark that this reduced algorithm is essentially equivalent to the Resample Algorithm (Song et al., 2023a) in the pixel space. In other words, Resample in the pixel space resembles SITCOM without backward consistency.
In this subsection, we will show the impact of the step-wise DM network regularization of the backward consistency by examining the results of (i) all sampling steps, and (ii) intermediate steps.
C. FULL SAMPLING STEPS STUDY
Here, we run SITCOM vs. SITCOM without backward consistency (i.e., Algorithm 1 vs. the steps in (15)) for the task of super resolution. We report the average PSNR and run-time (seconds) of 20 FFHQ test images with K 1000 and N 100J. We select large K to allow the optimizer to converge. Results are given in Table 2. As observed, SITCOM without backward consistency indeed requires less run-time but the PSNRs significantly lower than SITCOM (more than 5 dB for both cases) even if run until convergence (i.e., with very large K).
N = 20
N = 100
Method
PSNR
PSNR
Run-time
SITCOM
31.34
608.2
31.39
2418.2
SITCOM without Backward Consistency
25.97
372.4
26.1
1405.7
Table 2: Impact of the backward consistency in SITCOM: Average PSNR and run-time results (in seconds) of 20 FFHQ test images at using different values of N with K = 1000, reported by running Algorithm I with the optimization in (S₁) (first row) vs. the optimization in (15a) (second row).
C. .... I.. INTERMEDIATE SAMPLING STEPS VISUALIZATIONS
First, for the box- eox-panntin task, we compare SITCOM with optimizing over the output of the DM network (i.e., the steps in (15)) at time steps 1' E (200, 400, 600). More specifically, for each case (selection of i'), we start from t = T and
14

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
400
t'= 600
800
Task: Box Inpainting
Degr Image
Optimizing over output
Optimizing over the input the DM network (t K iterations.
200
400
600
800
Deblurring
Degracidd
Optimizing over the output of the DM network at ( for 20 iterations.
Optimizing over the input of the DM network (ours) att = for iterations,
Figure 3: Results of applying optimization baasadd measurement consistency, for which the optimization variable is the DM output (resp. input), shown in the first (resp. second) row for each task: Box Inpainting (top) and Gaussian Deblurring (bottom).
run SITCOM with step size of T At = given Xy, we perform two separate optimizations with intializing the optimization variable as Ny: one iteratively over the DM network input (i.e.. (S1)) and another iteratively over the DM network output (i.e., (15a)), both running until convergence (i.e., when the loss stops decreasing). Figure shows the results at different time steps. The consistency between the ground truth and the unmasked regions of the estimated images suggest the convergence of the measurement consistency. As observed, SITCOM produces significantly less artifacts in the masked region when compared to optimizing over the output. This is evident both at earlier time steps (t' = 600) and later steps (t' = 400 and /' = 200).
For the second experiment, the goal show that SITCOM requires much smaller number of optimization steps to remove the noise as compared SITCOM without backward consistency. The results are given in Figure 3, where we repeat the above experiment with two tasks: Box-in Box-inpainting (top) and Gaussian Deblurring (bottom)), this time using fixed number of optimization steps for both SITCOM, and SITCOM without backward consistency. Specifically, we run SITCOM from t = 7' to = t' + 1. Then, we apply K- = 20 iterations (the setting in SITCOM) in (S1), and K 20 for (15a) where measurement noise is σy = 0.05. As shown, compared to SITCOM without backward consistency, SITCOM significantly reduces noise across all considered t', underscoring the effect of the proposed ttep-wise network regularization for backward diffusion consistency.
C.2. Impact of the iterative step-wise measurement/dataa consistency
While all DM-IP methods aim to impose data consistency in the final solution, not all enforce step-wise data consistency. This step-wise consistency requires that the estimate &o computed using Tweedie's formula at each intermediate step remains consistent with the data. SITCOM, step-wise measurement consistency enforced by minimizing the data misfit term [[:())(v:)) y1 each step. However, if the minimizer is not found and only |[A(xx - y 1 with respect V, is performed (as in DPS), step-wise data consistency is no longer enforced, although global data consistency at i ( is still maintained. Therefore, we refer to SITCOM with K = 1 (i.e., performing only one gradient update of the data misfit) as SITCOM without data consistency and compare it to SITCOM itself. Note that the former is also equivalent to DPS plus resampling.
gradient update of
More specifically, we compare SITCOM with = vs. K 20 (converged) for various sampling steps (N). The study
15

---

SITCOM: Step-wise Triple-Consisten Diffusion Sampling for Inverse Problems
20 FFHQ images with øy 0.05. Average PSNR and run-time (in seconds) for the tasks SR and NDB are given in
Table 3.
Super Resolution
Non-linear Deblurring
N
K
K
20
K
K
20
PSNR
Run-time
PSNR
Run-time
PSNR
Run-time
PSNR
Run-time
10
11.28
0.99
27.75
19.12
12.12
0.99
26.92
19.02
20
11.41
2.11
30.40
35.20
12.30
2.30
30.27
36.05
11.44
30.88
55.04
12.32
3.40
30.72
57.05
17.04
16.46
9,12
1000
25.44
60.43
24.89
60.45
Table 3: Impact of the step-wise iterative measurement consistency SITCOM: Average PSNR/run-tinn results 20 FFHQ test images at using different values of N and to show the impact of using multiple gradient updates in applying the step-wise measurement consistency in SITCOM.
We observe that K = 20 (i.e., imposing the step-wise data consistency) yields better result than K = (i.e., not imposing the step-wise data consistency). In addition, for K (i.e., DPS+resampling) achieving decent results requires N 1000, leading to longer run-times.
C.3. Impact of the step-wise forward consistency
This experiment examines the impact of forward consistency imposed through the resampling in step 3 of SITCOM. Specifically, we compare SITCOM versus SITCOM with replacing the resampling formula with the traditional sampling formula in (5) at each sampling step.
We use 20 FFHQ test images with By 0.05, setting N = 20. Table shows average PSNR/runtime (in seconds) for linear and non-linear task. As observed, the use of resampling formula is more effective than the standard sampling,
Super Resolution PSNR Run-time
Non-linear Deblurring
Method
Mapping
PSNR
Run-time
SITCOM
Equation (Sa)
30.40
35.20
30.27 22.42
36.05
SITCOM without Forward Consistency (resampling)
Equation (6)
20.78
35.12
36.22
Table 4: Impact of the forward consistency SITCOM: Average PSNR and run-time (seconds) results of 20 FFHQ test images using two re-mappings: First is (Sx) (first row), and second is (6).
demonstrating the effect of the forward consistency SITCOM.
D. Further Insights & Relation to Existing Studies
Here, we discuss similarities and differences between DPS and SITCOM which leads to providing further insights into why our proposed sampler can (f) achieve competitive quantitative results, and (ii) allow for arbitrary sampling iterations.
DPS (Algorithm 2) is known to approximately converge to the posterior distribution p(xly) at 0.
16

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
Algorithm 2: DPS- Gaussian (Chung et al., 2023b)
Algorithm 3: -PSS with resampling
Require: y. / {or)
Require: N.y, ((i) (6:)
I: XN ~ (0, I)
1: XN ~ (0,
2: for i = N 1 to do
2: for N to do
3:
$ + so(x,, i)
3:
å - so i)
4:
Eo ya,
(x: (1 k,)
4:
Xo s
(x; + (1 k:)8)
5:
z ~~a (0, 1)
5:
2~ 10,1
6: X--1 yarll - 8 Xi V v
80 + 5,2
6:
v;
4: V x: ly A(xxn)1
1-a.
7: X;-1 - (i Vx: l1y -A(x))1
7:
x/-1 --i-vai--xp 1xi (ii + vI 1
8: end for
8: end for
9: return
9: return
Proposition D.I. SITCOM with K = 1 is DPS with the resampling formula in (7).
Proof. First, we replace the x; in line 6 of DPS (Algorithm 2) with its approximation x; ~ Vā,2 + -I- ā,z, where z ~ NV I). Under this substitution, line becomes x = vā + /I i which corresponds to the resampling formula. The resulting algorithm is referred to as DPS with resampling (DPStresampling)
To better illustrate its connection to SITCOM, we rename the variable x/_1 v, and swap the resampling formula with line 7 of DPS. These changes do not alter the algorithm's 's outcome. The resulting is equivalent form of DPS+resampling and is presented in Algorithm 3.
We observe that line 6 in the Algorithm corresponds to the gradient descent step for the objective function ||y Bly--Ale (0xo(((xxxx)))))) with respect to x,, and line 7 is the resampling. Compared to Algorithm I, we see that DPS+resampling is equivalent to SITCOM with K- =1
Remark 3. From Proposition 1, we observe that each iteration of DPS performs a single gradient update the objective function ||y - Axxx(xx))1 12. By grouping K consecutive sampling steps together, the process effectively performs K gradient updates. These updates correspond to very similar objective functions due to the continuity of the diffusion model, where (x, f) Rs (x, (') for Rs t'.
Specifically, within each group of K consecutive sampling steps, the associated time steps are closely spaced, say they are all within an interval (42, 12) with f22 f1 < 1. Due to the continuity of the diffusion model, the so(x, t) term in Xy can be approximated by its value at the fixed time 12, i.e., 88 t) RJ s0 (2) for all & E 42). With this approximation, the (x)) 1| depends solely on x., rather than both x; and t.
objective function ||y - -((xo(x:))
This simplification implies that the K gradient updates now act on the same objective function. As a result, optimization can be accelerated using Adam. The use of Adam is a key reason why SITCOM achieves faster convergence compared to DPS, as the latter relies solely on gradient descent. Finally, as shown ReSample (Song et al., 2023a), resampling can be performed rather sparsely. In SITCOM, resampling performed only once every K iterations, further enhancing its efficiency.
It is important to note that this explanation provides high-level intuition to aid in understanding SITCOM, rather than a rigorous proof of its performance.
E. Latent SITCOM
In this section, we extend SITCOM to latent diffusion models. Define & R - (parameterized by (5) the pre-trained encoder and decoder, respectively, with < n. Let RR x (0...., T) + R (parameterized by 0') be pre-trained DM in the latent space. We note that È is only needed during training, At inference, given some 2, E R", the steps in latent SITCOM are formulated as
R" (parameterized by Φ) and D R + R
1 [w] Vav
vI - (ve, - >11 + 2 >/|Ww 21 20
We = argmin w!
(SL,)
17

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
1
2/
Wr
vI aarer (ve (wr
(SL2)
0tx
24-1 vāl-1 12 + /1 ac- Se I,(,),,)
(SLg)
FFHQ LPIPS (1)
ImageNet LPIPS ())
Task
Method
PSNR (t)
run-time (L)
PSNR (t)
run-time ())
Latent-DAPS (Zhang et al., 2025)
27.48
0.192
84.24
25.00
0.343
125.40
Super Resolution 4x
ReSample (Song al., 2023a)
23.33
0.392
194.30
22.19
0.370
265.30
Latent-SITCOM (ours)
27.98
0.142
65.00
26,35
0.232
123.20
Latent-DAPS (Zhang et al., 2025)
23.99
0.194
83.00
17.19
0.340
115.20
Box In-Painting
ReSample (Song ol., 2023a)
20.06
0.184
189. 45
18.29
0.262
226.24
I
(ours)
24.22
0.177
58.20
20.88
0,314
114,00
Random Painting
Latent DAPS (Zhang el al., 2025)
30.71
0.146
93.65
27,89
0,202
128
SITen- (ours)
31,05
0.135
63.24
28.44
118.30
DAPS (Zhang et al., 2025)
28.03
0.232
28.420
0.341
135.24
Gaussian Debluming
ReSample (Song et al., 2023a)
26.42
0.255
186.40
25.97
0.254
234.10
Latent SITCOM (ours)
28.21
0.223
78.30
25.72
0.316
132.24
Latent-DAPS (Zhang et al., 2025)
27.32
0.278
118.2
0.349
139.2
Motion Deblurring
ReSample (Song al., 2023a)
27.41
0.198
236.2
26.94
0.227
264.2
Latent-STTCOM (ours)
27.74
0.182
64.25
27.25
0.202
121.4
Latent DAPS (Zhang et al., 2025)
29.16
0.199
80,92
20.54
0.361
105.65
Phase Retrieval
ReSample (Song 2023a)
21.60
0.406
204.5
19.24
0.403
234.25
Latent SITCOM (ours)
28.14
0.241
1 114,52
20.45
0.326
156.2
Laleat-DAFP (Zhang et al., 2025)
28.
0.211
89.24
25.34
0,314
130.2
Non-Uniform Deblurring
ReSample (Song al., 2023a)
28.24
172.4
215.2
Latent-SITCOM (ours)
28.41
0.171
112.4
26.48
0.201
134.5
Latent-DAPS (Zhang et al., 2025)
26.14
0,221
78.2
23.78
0.261
132.2
High Dynamic Range
ReSample (Song et al., 2023a)
25.55
0.182
178.2
25.11
0.196
215.2
Latent-SITCOM (ours)
25.85
0.174
102.45
25.67
0.182
145.4
Table Average PSNR, LPIPS, and run-time (scconds) results of our method and other latent-space baselines over 100 FFHQ and 100 ImageNet test images. The measurement noise setting is σy = 0.05.
Table 5 presents the results of Latent-SITCCM for the 8 image restoration tasks as compared to ReSample (Song et al., 2023a) and Latent-DAPS (Zhang et al., 2025). All methods used the pre-trained latent model from ReSample*.
For SITCOM, we used N = 20 and and set K = 30 (resp. K = 50) for linear (resp. non- non-linear) problems. Default hyper-parameter settings were used for ReSample and Latent-DAPP for each task as recommended in the authors' papers and code base.
As observed, Latent-SITCOM achieves better results than ReSample on all tasks and on both metrics (PSNR and LPIPS). When compared to Latent-DAPS, SITCOM achieves improved results on all tasks other than phase retrieval where we under-perform by 1BB on FFHQ and nearly dB on ImageNet.
In terms of run- time, MMieent-SITCOMMM generally requires less time when compared to ReSample. As compared to Latent- DAPS, our run-time is generally less. However, there are cases where Latet-DAPS faster than our method. An example of this is MDB on FFHQ
In summary, in this section, show that SITCOM can be extended to the latent space, and our results are either better very competitive when compared to two recent state-of-the-art latent-space baselines.
F. SITCOM with ODEs for Step (2) in (S²)
The Tweedie's 's network denoiser [ar;; I, <w) (S2) can be replaced by other denoisers without affecting the main idea of imposing the three consistencies SITCOM. In this section, we conduct a study for which we replace it with an ODE solver at each sampling step t. Specifically, given v, from (S1), obtain an estimated image & by solving following ODE
"ht pie /f /ggit thab. con/ soom nl nkwwnr/rewwsg reserple
18

---

SITCOM: Step-wise Triple-Consistett Diffusion Sampling for Inverse Problems
with initial values V and
dx: - Vx logp(x;;0))
(16)
dt
In (16), Vx, logp(x;; 0r) se se(Vr, b) where we implement the Euler solver (Karras et al., 2022) and select or = This
reduces (16) solving the ODE dx;-- (Ve t)dt using NODE discrete steps.
FFHQ LPIPS (L)
ImageNet LPIPS (4)
Task
Method
PSNR (t)
run-time (L)
PSNR (t)
run-time (1)
SITCOM
30.68
0.142
27.00
26.35
0.232
67.20
Super Resolution 4x
SITCOM-ODE
30.86
0.137
76.10
26.51
0.228
132.00
SITCOM
24.68
0.121
21.00
21.88
0.214
67.20
Box In-Painting
SITCOM-ODE
24.79
0.118
81,30
22.44
0.208
152.00
SITCOM
32.05
0.095
27.00
29.60
0.127
68.40
Random In- Pa-Paannin
SITCOM-ODE
32.18
0.091
93.50
30.11
0.114
128.30
SITCOM
30.25
0.135
27.60
27.40
0.236
66.00
Gaussian Deblurring
SITCOM-ODE
30.42
0.132
85.10
27.87
0.232
134.10
SITCOM
30.34
0.148
30,00
28.65
0.189
88.80
Motion Deblurring
SITCOM-ODE
30,54
0.145
112.34
28.81
0.184
139.00
SITCOM
30.67
0.122
28.50
25.45
0.246
84.00
Phase Retrieval
SITCOM-ODE
31.10
0.109
80.40
25.96
0.242
138.50
SITCOM
30.12
0.145
33.45
28.78
0.16
75.00
Mon-Uniform Dehlurring
SITCOM-ODE
30,31
0.141
85,25
28.86
0.152
129.20
SITCOM
27.98
0.158
31.20
26.97
0.177
92.40
High Dynamic Range
SITCOM-ODE
28.12
0.155
75,20
27.14
0.164
132.00
Table 6: Average PSNR, LPIPS, and run-time results of SITCOM with Tweedie's formula (i.e., Algorithm vs. SITCOM with the ODE solver for (16) instead of Tweedie's in (S2). Results are averaged over FFHQ and ImageNet test images with measurement noise level of σy = 0.05. For SITCOM ODE, similar to DAPS (Zhang et al., 2025), we use NODE = 5.
Table 6 presents a comparison study between SITCOM with Tweedic's formula vs. SITCOM with ODE in terms of PSNR, LPIPS, and run-time.
In general, we observe that the run-time for SITCOM-ODE is significantly longer than the required run-time for SITCOM. Specifically, the run-time is either doubled (or, in some cases, tripled) for PSNR gain of less than I dB. These results show the tra between PSNR and run-time when ODEs are used in SITCOM, as discussed in Remark 1.
is important to note that SITCOM-ODE demonstrates greater stability for the task of PR. In particular, we observe fewer failures when selecting the best result from runs for each image. See the next section for further details.
G. Discussion on Phase Retrieval
As discussed in our experimental results section, for the phase retrieval task, we report the best results from 4 independent runs, following the convention in (Chung et al., 2023b; Zhang et al., 2025). For the phase retrieval results of Table 1, we use this approach across all baselines where the run-time is reported for one run.
The forward model for phase retrieval adopted from DPS where the inverse problem is generally more challenging compared other image restoration tasks. This increased difficulty arises from the presence of multiple modes that can yield the same measurements (Zhang et al., 2025).
In Figure 4, present two examples. For each ground truth image, show four results from which the best was selected. In the first column, SITCOM avoids significant artifacts, while DAPS and SITCOM-ODE produce one image rotated by 180 degrees. In the second column, both SITCOM and DAPS exhibit one run with severe artifacts whereas SITCOM-ODE shows no image with extreme artifacts. The last image from SITCOM does exhibit more artifacts compared to the second warst-case result from DAPS. Additionally, the DPS results show severe perceptual differences in both cases, with artifacts being particularly noticeable in the second column.
19

---

SITCOM: Step-wise Triple-Consistend Diffusion Sampling for Inverse Problems
Ground Truth
Ground Truth
SITCOM (Ours)
SIDCOMODE (Oun)
Figure 4: Results of Phase Retrieval on two images (top row) from the FFHQ dataset. Rows 2, 3, 4, and 5 correspond to the results of DPS, DAPS, and SITCOM (ours), and SITCOM-ODE (ours), respectively.
H. SITCOM MRI Setup and Results
In this section, we extend SITCOM 2D medical image reconstruction. In particular, we consider the linear task of multi-coil Magnetic Resonance Imaging (MRI) where the forward operator is A = MFS. Here, M denotes the coil-wise undersampling, F is the coil-by-coil Fourier transform, and S represents the sensitivity encoding with multiple coils, which are incorporated into the operator A for all scenarios, are obtained using the BART toolbox (Tamir et al., 2016). We consider two mask patterns and two acceleration factors (Ax).
For baselines, we compare against DM-based solvers (DDS (Chung et al., 2024) and Score-MRI (Chung & Ye, 2022)), supervised learning solvers (Supervised U-Net (Ronneberger et al., 2015) and E2E Varnet (Sriram al., 2020)), and dataless approaches (Self-Guided DIP (Liang al., 2025), Auto-encoded Sequential DIP (aSeqDIP) (Alkhouri al., 2024a), and TV-ADMM (Uecker et al., 2014)), We use the fastMRI dataset (et al, 2019). We note that for training the Supervised U-Net (Ronneberger et al., 2015) and E2E Varnet (Sriram et al., 2020), we selected 8,000 training images from the 973 available volumes, omitting the first and last five slices from each volume. For testing, we used 50 images taken from the validation dataset.
SITCOM and the other two DM-based solvers (DDS and Score-MRI) use the pre-trained model DDS5. We used the recommended parameters for baselines given in their respective papers. For SITCOM, we used N 50, K 10, and
==0 O.
PSNR and SSIM results are presented in Table 7, while run-time results are provided in Table 8. As observed, SITCOM achieves the highest PSNR and SSIM values compared to the considered baselines. In terms of run-time, SITCOM requires significantly less time compared to DPS and Score-MRI. When compared to DDS (the best DM-based baseline), SITCOM achieves more than a 1 dB improvement in 3 out of the 4 considered mask patterns/aaccelcaaion factors, though it requires nearly twice the run-time. DDS faster than SITCOM because, in DDS, broo-propng through the pre-trained network required, as it is in SITCOM, Visualizations are given Figure 5.
ht pa //ggi it thab con' I4 hhhr
20

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
Mask Pattern
Ax.
TV-ADMM
Supervised
Var-VarNeet
Self-G DIP
aSeqDIP
Score-MRI
DPS
DDS
SITCOM
43
31.67
33.06s0.09
32.59
32.75
33.12+1.18
30.56x0.00
34.95=0.74
36.33+0.57
0.667+0.17
0.847=0.0
0.854+o.12
0.851
0.852=0.08
0.849+0.0e
0.841
0,954
0.962+0.08
Uniform ID
25. 13.12+2.11
29.24+0.a7
32.03+o.ss
32. 19+099....
32.33
32.10+2.90
30.33+0.28
31.72=1.88
32.78+0.64
x
0.535
0.784
g.829
0.827 16
20.11
0.821
0.81229.18
0.876 ce
0.892
32. a2.78=0.00
34.1 10.11
13
=0.28
=4.21
32.37
35.24=1.oz
36.42
0.785+0.06
0,861
0.913+0.12
0,904 uo
=,
0.891
0.832+0.15
0,963 16
0.969
Gaussian ID
28. 28.08+1122
31.52+1.00
33.21
32.81=1.42
32.95=0.as
32.34+0.58
30.52+2.92
33.32-1.6
33.99+1.29
0.747
0.841+0.12
0.868+0.1s
0,873
0.877
0.853+0.11
0.833+a.18
0,933=0.c7
0,943++0.00
Table 7: Average PSNR and SSIM results for SITCOM and various methods for MRI reconstruction using Uniform ID and Gaussian ID masks with 4x and acceleration factors. The measurement noise level is dy 0.01. Values past + represent the standard deviation. Best results are bolded whereas the second-bes best results are underlined.
Method
Score-MRI
DPS
DDS
SITCOM
Avg. Run-time (seconds)
342.20=41
145.52#27
25.40
52.30
Table 8: Average run- time seconds for SITCOM and MM-based baselines for the case Uniform ID x4.
Ground Truth
Input
Score MRI
DDS
SITCOM
Uniform 1D 4x
Uniform 1D 8x
PSNR
PSNR 17.45
PSNR 30.75
PSNR 32.02
PSNR
Gaussian 1D 4x
PSNR = dB
PSNR = 20.85 dB
PSNR = 32.67 dB
PSNK = 34.52 dB
PSNR = 35.36 dB
Gaussian 1D 8x
PSNR
PSNR 18.95
PSNR 30.45
PSNR 31.89 dB
PSNR 32.72 JB
Figure 5: Reconstructed images using proposed approach, SITCOM, and DM-based baselines (DDS and Score-MRI) Each row corresponds different mask pattern and acceleration factor. The ground truth and degraded images shown in the first and second columns, respectively, followed by the reconstructed imaged from the baselines. last column presents our method. PSNR results are given at the bottom of each reconstructed image. For all tasks, SITCOM reconstructions contain sharper and clearer image features than other methods.
21

---

SITCOM: Step-wise Triple-Consisent Diffusion Sampling for Inverse Problems
I. Additional Comparison Results
I.I. Near Noiseless Setting
In Table 9, we present the average PSNR, SSIM, LPIPS, and run-time (seconds) of DPS, DAPS, DDNM, and SITCOM using the FFHQ and ImageNet datasets for which the measurement noise level is set to dy 0.01 (different from Table 1).
The goal of these results is to evaluate our method and baselines under less noisy settings.
FFHQ
ImageNet
Task
Method
PSNR (f)
SSIM (1)
LPIPS (U)
Run-time (U)
PSNR (1)
SSIM (t)
LPIPS ())
Run-time ())
DPS
25
0.806.20044
0.242.00m
78.60
24,45
0.79220002
2.33 2o
139.80 809.80
DAPS
29.60_007
0.871 1:00
0.1321000s
74.40
25,981004
0.794.1000
0.234:ome
106.00131.00
Super Resolution 4x
DDNM
28. 28.82-002
0.851+0043
0,188++c1
64.20
24.67+028
0.771+006
0.432
82.80 82.80-31.00
Ours
30.95-000 20
0.872
0.137
30.00
26.89
0.80220007
0.224
10.40
DPS
23.56_007
0.762.10044
0.191
91.20
20.22,.007
0.69gna0
0.297..mm
93.00
DAPS
24.41 =0:2
0.791 1:0014
0.129 go0es?
69.80
21:7
0.734:0.44
0.214
96.4 96:40:2040
Box In-Painting
DDNM
24.67 +0006
0.788++0024
0.229 +0.05
+1,20+15.20
21. 21.99+a.44
0.737
0.737
0.315=aum
85 20.0000
Ours
24.97
0.804
1.
22.20
22
0.745
0.
73.80
DPS
28.77
0.847+0002
+
93.00+220.0
0.775 24.57+
0.318+022
127.20+
Random In- Painting
DAPS
31.56-0a4
0,905
0:09420002
29.20
28.8 27
0.131 0.877
120.602:300
DDNM
30.56_.0.
0.902.100
0.11610.22
75.00 1::20
30.12 30.12:104
0.917
0.124
113.40
Ours
33.02_004 a4
0.91910011
0.0912gam3
28.20
30.67
0.918
0.118go07
84.00.20.40
DPS
25.78-0.8
0.831 #:0024
0.202+a.14
79.80++26.40
22. 22.45+a42
0.778=
0.34 0.34
127.20
Gaussian Deblurring
DAPS
20.67=00s
1.089
0.
79000
26. 16
0.836
0
133,20)
DDNM
28.56_004
0.872
0.211 1 10204
74.40 ::040
28.44 comm
0.882 2o00e
0.26720044 2004
105. (05.60)....
Ours
-4.1
0,913+0022
0.139 +0,04
27.00+10.00
28. i2:45
0,891=00044
0.216=0m2
80.40=15..00
DPS
23.78
0.742+0042
0.265
99.00
22.33
0.726
0.352+000
132.60 222.00+22400
Motion Deblurring
DAPS
30.78_0.
0.89211001
0.146
66.40)7704
28. 28.24
0.867
0.191
127,20)1.6
Ouss
32,34 0..4
0.908 too2s
0.135 e0002
:
29.12
0.882 20.025
0,182 o.
87.00:11.00
DPS
17.56-215
0.681 +0056
+.392
91.20+25.20
16.77+178
0.65 =
0.442
130.80
Phase Retrieval
DAPS
31
4.909
4.109
81.00
26.12
0.80200000
0.247 20
139,20 20:2:000
Ours
31,88_2
0.921
1oosT
0,102100m
32.40 1:1.40
15.76
0.813
0.238
78.60.1..00
DPS
23.78-223
0.761
10001
0.269
23.60
22.97=1157
0.781
0.30220.09
140, 20:
Non-Uniford Deblurring
DAPS
28.89
01.455
0.150
84.60 1-2.20
18.02.1155
0.831 20.00
0.162 1004
133, 50
Ours
31.09-0.0
0,911+0006
+000s
0.132+t045
33.60=8.0
29.56
0.844=0045
0.147 00.42
80,40=16440
DPS DAPS
23.33-134
0.734+0049
0,251 +0.0ms
80.40++25.20
-9.67=0005
0.693
0.498
140.40+22.00
High Dynamic Range
27.58
0.828200
0.161
75.60
26.71=0008
0.802
0.172200
127.20
Ouss
28.52__0.9
0.844 100045
0.148 10.oss
30.60-2..0
27.561008
0.825 20.077
0.162.0006
87.00
Table 9: Average PSNR, SSIM, LPIPS, and run-time (seconds) of SITCOM and baselines using 100 test images from FFHQ and 100 test images from ImageNet with a mcasurement noise level ay = 0.01. The first five tasks are linear, while the last three tasks are non-linear (underlined). For each task and dataset combination, the best results are in bold, and the second-best results are underlined. Values after 1 represent the standard deviation. All results were obtained using single RTX5000 GPU machine. For phase retrieval, the run nin-tmmein is reported for the best result out of four independent runs. This is applied for SITCOM and baselines.
Overall, we observe similar trends to those discussed in Section 4 for Table 1. On the FFHQ dataset, SITCOM achieves higher average PSNR values compared to the baselines across all tasks, with improvements exceeding 1 dB in 5 out of 8 tasks. For the ImageNet dataset, we observe more than 1 dB improvement on the non-linear deblurring task, while for the remaining tasks, the improvement is less than dB, except for Gaussian deblurring (where SITCOM underperforms by 0.22 dB) and phase retrieval (underperforming by 0.36 dB).
In terms of run-time, generally, SITCOM outperforms DDNM, DPS, and DAPS, with all methods evaluated on a single RTX5000 GPU. For the FFHQ dataset, SITCOM is at least twice fast when compared to baselines. On ImageNet, SITCOM consistently requires less run-time compared to DPS and DAPS. When compared to DDNM, SITCOM's run-time is similar or slightly lower. For example, on the super-resolution task on ImageNet, both SITCOM and DDNM average around 81 seconds, but SITCOM achieves over a 2 dB improvement.
1.2. Performance Comparison Under Time Constraints
In this subsection, we compare the performance of SITCOM with baselines where their default settings require less run-time than SITCOM. Namely, DCDP on linear IP's RED-diff, and PGDM. In Section 4, we demonstrated that the proposed step- wise backward consistency significantly reduces the number of sampling steps. However, enforcing backward consistency also increases the runtime of each iteration, as the gradient computation SITCOM requires backpropagation through the
22

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
neural network. Here, we assess the combined impact of these two factors-fewer sampling steps and increased runtime per step-by conducting time-hudgeted comparisons with fast baselines. We will compare SITCOM with DCDP, which currently achieves SOTA runtime performance, as well as with RED-diff and PGDM, which enforce (non-stepwise) network consistency.
For fair comparison, we evaluate the runtime of each method in relation to achieving the same PSNR.
Table 10 includes a comparison SITCOM and DCDP. DCDP has two versions, and we only compare with its version I as it delivers notably better results than version 2. The parameters N and K in SITCOM also appear in DCDP, so we report results of the methods for various values of N and K. DCDP has extra inner for-loop (total of 3 nested-for-lom in DCDP) which applies diffusion purification. We fixed the time schedule for the diffusion purification to be the default one. In addition, we set By = O because SITCOM and DCDP handle measurement noise differently.
Method
K
PSNR
Run-time (scconds)
SITCOM
10
31.6
8.72
DCDP (Default)
10
10
29.52
10,50
SITCOM (Default)
20
20
32.28
28. 18
DCDP
20
20
31.67
16.24
SITCOM
30
31.66
22.11
DCDP
10
30
31.11
12.42
SITCOM
31.34
22.08
DCDP
30
10
30.81
12,04
SITCOM
32.32
32.22
DCDP
50
10
31.69
26.20
SITCOM
100
10
32.34
51.23
DCDP
100
10
31.71
42.45
SITCOM
200
10
32.41
124.20
DCDP
200
10
31.76
112.10
SITCOM
500
10
32.46
320.20
DCDP
500
10
31.82
309.10
Table 10: Comparison SITCOM vs. DCDP using the linear inverse problem, Gaussian Deblurring (we consider linear problems here since SITCOM already demonstrates significant advantage in run-time for nonlinear settings, as evidenced in Table 1 of the main text). Results are reported using the FFHQ dataset with σy O. Different values of N and K are used to report the average PSNR and run-time (seconds).
As confirmed by the table, SITCOM is slower than DCDP (which does enforce the backward consistency) when the number of iterations is fixed due to the need back muok-propagate. However, SITCOM allows more significant reduction of N and K while achieving competitive PSNRs, so the total run time of SITCOM is smaller. For instance, the PSNR of SITCOM with N = 10 matches DCDP with N 20.
In Table 11 and Table 12, we compare SITCOM with RED-diff and PGDM, respectively. Both of these baselines utilize ideas related to network regularization. However, they utilize the entire diffusson process (from time to 0) as a single regularizer of the reconstructed image, instead of applying the network regularize stepwise. Therefore, they' 're not imposing the step backward consistency.
Method
Super Resolution Motion Deblurring
High Dynamic Range
RED-diff
24.89/18.02
27.35/19.02
23.45/24.40
SITCOM (Default) SITCOM (N) = (8
26.35/62.02 24.99/17.80
28.65/62.04 27.38/17.92
26.97/62.50 25.24/20.12
Table 11: Average PSNR/nune-time of SITCOM as compared to RED-diff using two linear tasks and one non-linear task. Here, ImageNet is used with noise level of oyy 0.05.
As observed in Table 11, for Super Resolution and Motion Deblurring, the default SITCOM (second line) requires more run- time and achieves better PSNR than RED-diff. However, the third line shows that if we just want to match the performance
23

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
of RED-diff, can use smaller N and K for SITCOM that greatly reduces the run-time below RED-diff.
Method
Super Resolution
Motion Deblurring
PGDM
25.22/26.20
27.48/24.10
SITCOM (Default)
26.35/62.02 25.15/19.72
28.65/62.04
SITCOM K) (10, 10)
27.55/19.65
Table 12:
Average PSNR/rune-time of SITCOM as compared to PGDM for super resolution and motion deblurring, Here, ImageNet used with noise level of σy = 0.05.
As observed in Table 12, given the same PSNR, SITCOM requires less run-time than PGDM. Additionally, the default N and K allow SITCOM to achieve more than IdB improvement.
J. Ablation Studies
J.1. Impact of K& & N and Convergence Plots
In this subsection, we perform an ablation study on the number of optimization steps, K, and the number of sampling steps, N. Specifically, for the task of Gaussian Deblurring,
SITCOM using combinations from N E {4, 12, ... 40) and K (5, 10, 15, ................................ 40). The average PSNR results over 20 test images from the FFHQ dataset are presented in Table 13.
We observe that with N = 12 and K = 15, we obtain 28.3dB (with only 18.3 seconds) whereas at N = K = 40, we obtain 29 dB which required nearly 110 seconds. This indicates that there is indeed a trade-off between computational cost and PSNR values. Furthermore, we observe that 61 entries (out of 80) achieve a PSNR values of more than or equal to 28 dB, This indicates that SITCOM is not very sensitive to the choice of N and K. We note that we observe similar patterns for all tasks. See the tables of all the tasks here".
The selected (N, K) values for main results are listed in Table 17 of Appendix K.1.
(N,
K = 5
K = 10
K = 15
K = 20
K = 25
K = 30
K = 35
= 40
N = 4
19.3, 5.22
23.6, 7.14
25.9, 9.00
26.7, 10.56
27.3, 11.34
27.7, 12.48
27.9, 13.74
28.1, 12.96
=
23.7.7.32
27.2, 11.04
27.9, 13.98
28.3, 17.16
28.4, 19.80
28.5, 22.98
28.5, 25.32
28.6, 23.76
N 12
2.42.
27.9, 14.28
28.3. 18.30
28.5,
28.6, 26.76
30.7,
35.94
28.7, 34.50
N = 16
26.2,
28.2, 17.58
23.70
28.6, 29.76
28.7,
28.8. 42.18
28.8, 47.82
28.8. 44.82
= 20
13.02
21.4,
28.6, 28.86
36.5,
28.8, 43.98
21.1,
28.9, 57.96
28.9, 54.72
N 24
24.1,
24.36
23.7.43.20
18.90
28.8, 60.48
28.9, 98.04
28.8, 65.10
N = 28
27.3,
28.5,39.72
56.58
28.8, 72.18
28.9, 88.38
28.9, 103.44
29.0, 117.90
28.9, 75.54
N 32
25.62
28.5, 45.00
28.7. 64.14
28.8, 82.86
28.9, 98.34
28.9, 114.84
28.9, 123.96
29.0, 86.64
N 36
27.8, 24.66
28.7, 44.46
28.8. 61.44
28.9, 74.16
28.9, 84.78
29.0, 89.04
29.0, 99.78
29.0, 96.12
N = 40
27.8, 21.72
28.6, 38.04
28.9, 53.82
28.9, 69.06
29.0, 84.06
29.0, 99.48
29.0, 121.62
29.0, 109.80
Table 13: Performance comparison for different (N, K) using the task Gaussian Deblurring. Each cell shows values the format PSNR, Run-Time (seconds).
Figure shows PSNR curves of SITCOM using different values of N and K for Gaussian Deblurring and High Dynamic Range. As observed, with N = 400, (N, = (10, 40) yields the best results. Notably, after completing the optimization steps at each sampling step, we observe drop in PSNR, attributed to the application of the resampling formula in (S3) where additive noise is added the estimated image at each sampling discrete time step.
J.2. Ablation Studies on the Stopping Criterion For Noisy Measurements
In SITCOM, we proposed to use a stopping criterion to prevent the noise overfitting, determined by ay. In this subsection, (i) show how sensitive SITCOM's performance w.r.t. the choice of 6, and (ii) we visually show the impact of using
ht pi /gi thab. con/ i 34 /e mwwoooo/ss hst ol/mm Extt endeed Ablat con.nd
24

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
PSNR Progress Over Iterations
Progress
PS 20+20
PSWR
PS'V 40x10
FSNR 40x30
PSTRR 30:-00
PSNR. 10v40
PSMR
FSNR
1.93
50
100
150
203
250
200
350
400
Itoratio
Iteration
(a) Gaussian Deblurring.
(b) High Dynamic Range.
Figure 6: SITCOM PSNR curves for Gaussian Deblurring (left) and High Dynamic Range (right) tasks using different values of N and K (labeled as K in legends).
stopping criterion vs. not using it; (ii) We evaluate SITCOM using measurement noise sourced from a non-Gaussian distribution.
J.2.1. SENSITIVITY TO THE CHOICE OF THE STOPPING CRITERION 8
In SITCOM, we proposed to use stopping criterion to prevent the noise overfitting, determined by ay. Here, we try to answer the question: How sensitive is SITCOM's 's performance to the choice of the stopping criterion parameter 5? To answer this question, we run SITCOM using different values of 5 and report the average PSNR values and run-time (in seconds).
We use N = K 20 and oy 0.05 and set values above and below Ty. Results for Super-Resolutionn(inees IP) and Non-uniform deblurring (non-linear IP) given in Table 14, where 20 FFHQ test images with 256 x 256 3. The first shows the values of for which 8 a.m.
Task
0.01
0.017
0.025
0.04
0.05
0.051
(1,055
0.07
0.085
0.1
0.5
SR
27. 27.80/40.20 20
28 28.45/38.82
82
29.02/38.41 41
30.22/36.26 26
20.39/28.52
30.40/28.22 26
30.37/24. 10
30. 1/23,5
29.87/22.89
28.87/22 12
23.10/15.87
NDB
28.40/40.40
28. 87/38.58
29. 32/.3033.32
29. 82.35/25112
30.12/32.12
30.21/29.46
30.20/27.44
30.01/26.56
29.67/24.52
29. 12/22//2
25.79/15.10
Table 14: Performance comparison for Super Resolution (SR) and Non-lincar Deblurring (NDB) of different values of a (first row) for which 8 a/m and σy 0.05. Each cell shows values in the format PSNR/run-times (scconds).
As observed, the PSNR values vary between 29.02 30.37 (resp. 29.80 to 30.20) only for BIP (resp. NDB) with stopping criterion between 0.025 to 0.055 which indicates that even if values lower (or slightly higher) than the measurement noise level are selected, SITCOM can still perform very well. This means that if we classical methods (such as (Liu et al., 2006; Chen et al., 2015)) to approximate/estimate the noise, we can achieve competitive results.
We note that other works, such as DAPS (Zhang et al., 2025) and PGDM (Song et al., 2023b), also use σy but not to estimate the stopping criteria. these papers, By encoded in the updates of their algorithm (see Eq. (7) in PGDM and Eq. (9) in DAPS).
J.2.2. IMPACT OF USING THE STOPPING CRITERION VS. NOT USING IT VISUALLY
In this subsection, demonstrate the impact of applying the stopping criterion in SITCOM when handling measurement noise. For the tasks of super resolution and motion deblurring, we run SITCOM with and without the stopping criterion for the case of σy = 0.05. The results are presented in Figure 7. As shown, for both tasks, using the stopping criterion (i.e., 5 > 0) not only improves PSNR values compared to the case of = O, but also visually reduces additive noise the restored images. This is because, without the stopping criterion, the measurement consistency enforced by the optimization in (S1) tends to fit the noise in the measurements.
25

---

SITCOM: Step-wise Triple-Consistenl Diffusion Sampling for Inverse Problems
Super Resolution 8=0
8>0
DegradedImage
PSNR 27.267
PSNR 30.62
Motion Deblurring 6=0
Degraded Image
PSNR 22.819
PSNR 31.911
Figure Impact of stopping criterion preventing noise overfitting. the most right column, 5 set as Table 17.
J.2.3. HANDLING NON-GAUSSIAN MEASUREMENT NOISE
Here, we evaluate SITCOM with additive measurement noise vector sampled from the Poisson distribution. We use N = 20 and A = 5 (which determines the noise level). We set å (the stopping criterion) to different values (top row). Results for Super Resolution (linear IP) and Non-uniform deblurring (non-linear IP) are given in Table 15 where we use 20 FFHQ test images with 256 256 3. The first row shows the values of for which å aym.
Task
0.02
0.035
0.05
0.06
0.061
0.065
0.08
0.1
0.5
SR
24.52/47.30
27. 19
29.19/38.1
30,42/36.23
30.49/28.22
30.43/28.21
30.37/24.16
29.67/24.32
26.77/22.08
25. 12/15888
NDB
25.04/43.42 28.60440.34 30.17/37.52
30,22/35.44
30.24/32.02
30.23/29.31
30.15/27.51
29.71/24.02
29.03/22.25
26.44/15.15
Table 15: Average PSNR/run-time (seconds) of SITCOM for the tasks of Super-Resolution Resolution (SR) and Non-uniform Deblurring (NDB) across various stopping criterion thresholds (top row is the value of a for which ò avm)) using additive noise from the Poisson distribution (code from DPS ?)
As observed, the results of setting 5 to values between 0.035 to 0.08 return PSNR values within approximately 1 dB. This indicates that SITCOM can perform reasonably well with different values of the stopping criterion even if its settings designed for the additive Gaussian measurement noise.
J.3. Effect of the Regularization Parameter À
In this subsection, we perform ablation study assess the impact of the regularization parameter, A, in SITCOM. Table 16 shows the results across four tasks using various À values. Aside from phase retrieval, the effect of X is minimal. We hypothesize that initializing the optimization variable in ($1) with is sufficient to enforce forward diffusion consistency in C3. Therefore, we set X = for phase retrieval and X = 0 for the other tasks. The impact of À for all tasks are online".
Additionally, for all tasks other than phase retrieval, we observed that when À = 0, the restored images exhibit enhanced high- high-frequency details. For visual examples, see the results of À = 0 versus
in Figure 8.
Task
X = ()
X = 0.05
0.5
X =
X 1.5
Super Resolution 4x
29.952
29.968
29.464
29,550
29.288
Motion Deblurring
31.380
31.393
31.429
31.382
31.150
Random Inpainting
34.559
34.537
34.523
34.500
34.301
Phase Retrieval
31.678
31.892
32.221
32,342
32.124
Table 16: Impact of the regularization parameter À terms of PSNR four tasks.
bt pa 2/ aoeor mmooss Anpeen wnwee/////// r 17CMM 5.58// /Fist enderl All ws Fonund
26

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
Ground Truth
Degroded Image
A=1
PSNR 28.99
PSNR 28.97
PSNR 27.47
Figure Results of running SITCOM using different regularization parameters (S1) for the task of Motion deblurring,
K. Detailed Implementation of Tasks, Baselines, and Hyper-parameters
Our experimental setup for image restoration problems and noise levels largely follows DPS (Chung et al., 2023b). For linear IPs, evaluate five tasks: super resolution, Gaussian deblurring, motion deblurring, box inpainting, and random inpainting, For Gaussian deblurring and motion deblurring, we use 61 661 kernels with standard deviations of 3 and 0.5, respectively. In the super-resolution task, a bicubic resizer downscales images by factor of 4. For box inpainting, random 128x 128 box is applied to mask image pixels, and for random inpainting, the mask is generated with each pixel masked with probability of 0.7, as described in (Song et al., 2023a).
For nonlinear IP tasks. we consider three tasks: phase retrieval, high dynamic range (HDR) reconstruction, and nonlinear (non-uniform) deblurring. In HDR reconstruction, the goal is restore higher dynamic range image from lower dynamic range image (with factor of 2). Nonlinear deblurring follows the setup in (Tran et al., 2021).
For MRI, we follow the setup in Decomposed Diffusion Sampling (DDS) (Chung et al., 2024) for which we consider different mask patterns and acceleration factors (Ax).
Baselines & Datasets: For baselines, we use DPS (Chung et al., 2023b), DDNM (Wang et al., 2022), RED-diff (Mardani et al. 2024), PGDM (Song et al., 2023b), DCDP (Li et al., 2024), DMPlug (Wang et al., 2024), and DAPS (Zhang et al., 2025). The selection criteria is based on these baselines' competitive performance on several linear and non-linear inverse problems under measurement noise. For MRI, compare against diffusion-based solvers (DDS (Chung et al., 2024) and Score-MRI (Chung & Ye, 2022)), supervised learning solvers (Supervised U-Net(Ronneberger et al., 2015) and E2E Varnet(Sriram et al., 2020)), and dataless approaches (Self-Guided DIP(Liang et al., 2025), Auto-encoded Sequential DIP et al., 2024a), and TV-ADMM (Uecker et al., 2014)). We evaluate SITCOM and baselines using 100 test images from the validation set of FFHQ (Karras et al., 2019) and 100 test images from the validation set of ImageNet (Deng et al., 2009) for which the FFHQ-trained and ImageNet-trained DMs are given in (Chung et al., 2023b) and (Dhariwal Nichol, 2021), respectively, following the previous convention.
For baselines, used the codes provided by the authors of each paper: DPS DDNM DAPS"!, and DCDP!2, Default configurations used for each task.
K.1. Complete List of hyper-parameters in SITCOM
Table 17 summarizes the hyper-parameters used for each task in our experiments, determined by the ablation studies in the previous subsections. Notably, the same set of hyper-parameters is applied to both the FFHQ and ImageNet datasets.
L. Additional Qualitative results
Figure 9 (resp. 10) presents results with SITCOM, DPS, and DAPS using FFHQ (resp. ImageNet). See also Figure 11, Figure 12, Figure 13, and Figure 14 for more images.
10 st tpa //itthas. cwn/w///////eo. http///aaa//D0
he tee /gg whaithab..ee. ss shangsii rngl Sang2019//an
12 tp a / /gght thab con//orwfffi www.wrrwwweeeee www.///eg///////e ed Date: Cone ency wi DJ Fuuii on Puri Phrrifiiin Fiiessoon For: Trage Bextorat lan
27

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
Task
Sampling Steps N
Optimization Steps K
Regularization X
Stopping criterion ó for dy E [0.05,0.01]
Super Resolution
20
(0.051 v mssk ,0.011
MASK
Box In-Painting
20
{0.051,/m /m .0.011
m
Random tn-Painting
30
{0.05) v / .0.01
Gaussian Deblurring
30
{0.05) y .0.01
m
Motion Deblurring
30
{0.05) y (0.01.001111
Phase Retrieval
20
30
(0.051 Mex. 00.011
Non-Uniform Deblurring
30
(0.05l//m.0.011/// {0.051 .0.011
High Dynamic Range
40
Table 17: Hyper-parameten SITCOM for every image restoration tasks considered this paper. The same of hyper-parametery used for FFHQ and ImageNet. The learning rate in Algorithm 1 set to y = 0.01 for all tasks, datasets, and measurement noise levels.
For the stopping criterion column, mess = 64 x 64 x 3, 256 x 256 x 3, and vipe = 384 x 384 x 3.
Super Resalution
Motion Deblurring
Gomund Massummmm
DAPS
PSUS = 28,6
POUR a 30.23
0132
PSWR 82.39
Paskk - 22:96
PANR
Non-linear Deblurring.
SITCOM jours
SITCOH (ours)
27.55
PSNA 17:1
PSNE $2.67 LMPS 0118
PSNR 1099
LPPB - 0.277
LPIPE -n 167
LHIMS 0.6
LIIPS CA01
LOPPS 0.112
linear tasks (top) and two non-linear tasks (bottom) under measurement noise By 0.05. PSNR LPIPS values are given below each restored image. Zoomed-in in regions show how SITCOM captures greater image details when compared to two general (non linear DM-based methods (DPS (Chung et al., 2023b) and DAPS (Zhang et al., 2025)).
Figure 9: Qualitative results the FFHQ dataset
28

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
Super Resolution
Inpeinting
Nesasneeents
(ours)
ann
PS3I 22.10
POWR 26.9 LPIPS 185
PaNR 24.15
Deblurring,
Gaussian Deblurring
Mestaummenss
Massurements
FSBR # 72,9
PSll 27.9
PSNR 20.22
PANK 21.72
PSNB 27.36
28570
LIREES 9345
19995 11 191
Random Inpsinting
High Dynamic Renge
Mestrunmenr
DFS
DAPB
SITCOM (ouns)
DPS
DAPS
SITCON
PAN
POW 2015 LinS 0.002
26-25
PONE 20.07
LMIN 6:115
Lits
LISINS 0.400
LEINS 0445
LEMP1 0.56
Non lineor Debluming
Phose Retrieval
SITCOM
PONE 23.50
PONIL 20U1
PSSEE 2012
POVE TARS
PONIL 30.12
PSHI
Figure 10: Qualitative results on the ImageNet dataset for five linear tasks and three non-linear tasks under measurement noise of Ty 0.05. The PSNR and LPIPS values given below each restored image.
Measurements
Figure 11: Super resolution (left) and box inpainting (right) results. First (resp. last) three rows are for the FFHQ (resp. ImageNet) dataset.
29

---

SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
Ground Iruth
Mosturamems airco
DAPS
DPS
Ground Toush
Messurement
SITCOM (curs)
DAPS
DPS
L
L0J
Figure 12: Motion deblurring (left) and Gaussian deblurring (right) results. First (resp. last) three for the FFIQ (resp. ImageNet) datasct.
Ground Trath
Moasurtmoms
SITCOM
Ground Touch
Mansurements
SITCOM jours
DAPS
DPS
2084
Figure 13: Random inpainting (left) and non-linear (non-uniform) deblurring (right) results. First (resp. last) three rows for the
FFHQ (resp. ImageNet) dataset.
30

---

SITCOM: Step-wise Triple-Consisten Diffusion Sampling for Inverse Problems
Ground Truth
Mesconments
SITCOM lours!
DAPS
OPS
Graund Inuhh
Heasurements
SITCOH tound)
DAFS
DPS
Figure 14: Phase retrieval (left) and high dynamic range (right) results. First (resp. last) three for the FFHQ (resp. ImageNet) dataset.
31


Reviews:
[{"review_id": "ls3uRJZgrq", "reviewer": "Reviewer_uV3o", "summary": "This paper systematically analyzes three necessary conditions—forward consistency, measurement consistency, and backward consistency—that enable accurate inverse problem solving using diffusion models. A sampler is proposed to enforce these conditions while reducing the required number of reverse diffusion steps. The method is extensively evaluated on five linear tasks, three nonlinear tasks, and one medical imaging task (in the appendix).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments. We refer the reviewer to (https://anonymous.4open.science/r/SITCOM-B65F/rebuttal_table.md) for Tables A & B. \n\nWe're glad the reviewer finds our paper well-motivated and well-written and acknowledges SITCOM's consistent improvements over most baselines with fewer inference steps. Our point-by-point response follows.\n\n### (W1) **Ablation study on the implicit regularizatoin of Tweedie-network denoiser.**\n\nWe appreciate the reviewer’s feedback. While forward and measurement consistencies have been explored before, our key contributions are: 1. **Backward consistency** is explicitly formulated and enforced through optimization. To our knowledge, this is the first work to introduce such a formulation and demonstrate its regularizing effect during sampling. 2. We integrate all three consistencies—forward, measurement, and backward—into a unified sampling framework, improving both reconstruction quality and efficiency. Notably, our approach reduces sampling steps without sacrificing accuracy, leading to computational savings.\n\nFor the reviewer’s concern about ablation on the denoiser’s implicit regularization, see Appendix C.1, where we isolate the effect of backward consistency by comparing results with and without it. The findings show that enforcing backward consistency improves reconstruction accuracy and stability. As suggested by the reviewer, in the revised paper, we will include this part in the main body of the paper. \n\nAppendices C.2 and C.3 further provide ablations on measurement and forward consistency, respectively, supporting our claim that each component contributes meaningfully and coherently to the overall performance of SITCOM.\n \n### (W2) **The inference time using number of function evaluations (NFEs).**\n\nWe appreciate the reviewer’s question. All run-time results were obtained on the same RTX5000 GPU for fair comparison, with baselines run on our machine using their default settings, including NFEs.\n\nWe agree that reporting NFEs is valuable and include this in Table A for super-resolution. Due to time constraints, we couldn't provide a full performance vs. NFE plot but offer partial results for insight.\n\nDPS and DDRM are competitive only at $N=1000$ (Figure 11, DPS paper). For DCDP, we found fewer NFEs suffice for linear tasks, while non-linear tasks still require many. Thus, we evaluated DCDP across a wide NFE range in Table 10 (Appendix I.2). SITCOM was similarly evaluated in Tables 10 (Appendix I.2) and 13 (Appendix J.1).\n\nOur results suggest reducing NFEs degrades baseline performance, though some methods are less sensitive. Additional experiments (Appendix I.2) show that under comparable runtime constraints, SITCOM achieves the best reconstruction quality (Tables 10, 11, and 12).\n\n### (W3) **Choice of baselines and RED-Diff results on nonlinear deblurring.**\n\nWe thank the reviewer for pointing this out. Table B shows comparison between RED-diff and SITCOM using ImageNet for non-linear deblurring (NDB) with measuement noise of 0.05. \n\nIn our experiments, this method was not very good on LPIPS which we consider one of the most widely used perceptual metrics. As a result, we chose to compare against DAPS and DCDP for this task, as both achieved better LPIPS scores. The high PSNR reported in RED-diff's Table 1 (the ~45dB for NDB) is for the measurement noisless case. \n\nIn general, the selection criteria for baselines is based on these baselines’ competitive performance on several linear and non-linear inverse problems under measurement noise (Appendix K). \n\nWe would also like to point out that in Appendix I.2 (Table 11), we provide additoinal comparison results between SITCOM and RED-diff under time constraints using two linear tasks and one non-linear task. \n\n### (W4) **Theoretical justification on the three consistency conditions.**\n\nWe thank the reviewer for their comment. Enforcing forward consistency can pull intermediate solutions off the reversed diffusion trajectory (Figure 1), making establishing theoretical guarantees highly challenging. While we can show partial results, such as enforcing data consistency (up to a small error) using standard SGD convergence results, establishing overall error bounds for the final reconstruction remains difficult.\n\nSimilar challenges exist in prior methods like ReSample, DCDP, DAPS, and DMPlug, which enforce data consistency at each step but lack overall reconstruction guarantees. The nonlinear nature of network regularization in our backward consistency further complicates the analysis.\n\nNonetheless, Appendix D provides an intuitive explanation of why SITCOM approximates posterior samples similarly to DPS and why it may offer faster performance. We hope this analysis enhances understanding and confidence in our method.", "claims_and_evidence": "The paper makes two primary claims:\n\n1. Forward consistency, measurement consistency, and backward consistency are three necessary conditions for accurately solving inverse problems with diffusion models.\n2. By implicitly enforcing backward consistency through network regularization, the required number of reverse diffusion model steps is significantly reduced.\n\nThese claims are partially supported by empirical experiments.", "methods_and_evaluation_criteria": "The method is evaluated on five widely used linear inverse problems, three nonlinear tasks, and one medical imaging task. Standard evaluation metrics, including PSNR, SSIM, FID, and LPIPS, are used to assess performance.", "theoretical_claims": "None.", "experimental_designs_or_analyses": "The experiment design is sound and valid.", "relation_to_broader_scientific_literature": "This work contributes to the field by systematically analyzing the role of three consistency conditions in determining reconstruction accuracy for inverse problems solved using diffusion models.", "essential_references_not_discussed": "I did not identify any critical missing references.", "other_strengths_and_weaknesses": "Strength:\n\n1. The paper is well-motivated and well-written, providing a systematic analysis of how different consistency conditions impact inverse problem solving. It also offers intuitive explanations for most design choices.\n2. The proposed method achieves consistent improvements over most selected baselines across both linear and nonlinear tasks while requiring fewer inference steps.\n\nWeakness:\n\n1. My main concern is that the technical novelty of the paper appears limited. While the paper is the first to formally propose and explicitly formulate the three consistency conditions, each has already been individually explored in prior work. More specifically, forward consistency and measurement consistency have been widely used, while the main enhancement in this paper is the introduction of backward consistency through optimization in the input space of the Tweedie-network denoiser rather than the output space. Since the Tweedie-network denoiser serves as an implicit regularizer, an ablation study on this component should be included in the main paper.\n2. The reported inference time for the proposed method is significantly lower than the baselines, but this comparison may be misleading because each baseline uses a different number of function evaluations (NFEs). A more informative way to present the results would be to plot performance (y-axis) against NFEs (x-axis) for each method as a curve.\n3. The choice of baselines is confusing. For example, the paper does not report RED-Diff results on the nonlinear deblurring task. Based on my experience, RED-Diff should perform particularly well on this task. Could the authors provide insights into why this comparison was not included?\n4. The paper doesn't provide theoriratical justification on three consistency condition in the proposed sampler.", "other_comments_or_suggestions": "None.", "questions_for_authors": "Please see above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bqjxyKoeIl", "reviewer": "Reviewer_VNrg", "summary": "This paper introduces a diffusion posterior sampling method, SITCOM, for solving inverse problems. SITCOM is designed to hold three consistency conditions during the diffusion sampling procedure to guide the generation. The backward consistency aims to ensure that the solution after data consistency optimization is a valid output of diffusion models. Experiments on several image inverse problems validate the effectiveness of SITCOM.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's recognition that our paper establishes three reasonable consistency conditions to guide the algorithm's design and acknowledges SITCOM's superior performance in image restoration and MRI.\n\nWe refer the reviewer to (https://anonymous.4open.science/r/SITCOM-B65F/rebuttal_table.md) for Tables C, D, and E.\n\nWe thank the reviewer for their willingness to revisit their decision. \n\n### (W1) **Backward consistency**\n\nThank you for raising this question. The reviewer is essentially asking whether the estimate $\\hat{x}_0$ produced by other methods (which do not explicitly enforce backward consistency) might satisfy backward consistency automatically, given that the space of possible $v_t$'s.\n\nOur response is that this scenario is unlikely. The key point is that the output space of $f(.;t,\\epsilon_\\theta)$ (Tweedie's network denoiser) at a fixed noise level $t$ is not the entire ambient space $\\mathbb{R}^d$, but rather a small subset of it. Therefore, the probability that an arbitrary vector $\\hat{x}_0$ produced by another method lies within this subset is low—unless the method is explicitly designed to ensure this consistency. \n\nThis is not an intuitive claim but is supported by the following reasoning:\n\n1. \"Diffusion models encode the intrinsic dimension of data manifolds\" ICML24. This paper demonstrates that for small noise levels, DMs approximate the normal bundle of the data manifold. Consequently, our Tweedie denoiser $f(.;t,\\epsilon_\\theta)$ at small $t$ approximates the tangent bundle. When the training data (such as imagenet) lies on a low-dimensional manifold, the output of $f(.;t,\\epsilon_\\theta)$ is therefore concentrated near this manifold—within a region whose thickness is proportional to the noise level $t$. This indicates that the denoiser’s output is constrained to a small subset of the full space.\n\n2. \"The intrinsic dimension of images and its impact on learning\" ArXiv21. Table 1 of this paper estimates the intrinsic dimension of common datasets and confirms that it is significantly lower than the ambient dimension. For example, ImageNet's intrinsic dimension is reported to be approximately 43, which is much smaller than the raw pixel space dimension.\n\nTaken together, these results suggest that the trained DMs has an inherent regularizing effect: its outputs, especially at low noise levels, lie close to a low-dimensional manifold of the ambient space. Therefore, our backward consistency condition—requiring $\\hat{x}0 = f(v_t;t,\\epsilon_\\theta)$ for some $v_t$ —acts as a meaningful constraint rather than a vacuous one.\n\nWe refer the reviewer to the experiment in (https://anonymous.4open.science/r/SITCOM-B65F/experiment.png) to support our theoretical argument. \n\n### (W2 & Q2) **Other baselines**\n\nWe appreciate the reviewer for pointing out these references. In our evaluation, we compare SITCOM with 7 pixel-space and 3 latent-space baselines (all from 2023+), as well as 7 baselines for MRI. Baselines were chosen based on their strong performance on linear and non-linear inverse problems under measurement noise (Appendix K).\n\nSome references, such as MGPS (ICLR25), were very recent and not initially considered. To ensure SITCOM's position, we include comparisons in Table C for MGPS, FPS, and DCPS, noting that dashes indicate tasks not covered by these methods. These baselines do not report PSNR or SSIM, but SITCOM remains competitive.\n\nTable D compares SITCOM to PnP-DM, showing that SITCOM outperforms it in all tasks. PnP-DM did not report results on ImageNet.\n\nFor MCG-Diff, we found no common settings (pre-trained model, noise, task, dataset), so due to time constraints, we could not compare it but will include results/discussion in the revised paper.\n\nTDS is a general conditional sampler applied to image inpainting on MNIST (not high-dimensional) and motif-scaffolding (not an imaging inverse problem). We will include this discussion in the revision.\n\n### (Q1) **The determination of $\\delta$, $\\gamma$, $\\lambda$, and $K$**\n\nWe thank the reviewer for their comment. For $\\delta$, it is computed based on the noise in the measurements. See the discussion in Appendix J.2.1. For the case of $K$ and $\\lambda$, see Appendix J.1 and Appendix J.3, respectively.  \n\nFor the case of the learning rate $\\gamma$, we use 0.01 (see Table 17 of Appendix K.1), which is the default for ADAM.\n\nTo address the reviewer's concern, we conduct the ablation study in Table E. As observed, the results are very close around the default value. **We will include these results in the Appendix.**\n\n### (Q3) **SITCOM running faster than baseline methods.**\n\nWe agree that SITCOM requires iterative gradient updates to solve Eq.(S1), including backpropagation through the pre-trained DM. However, enforcing the three consistencies enables SITCOM to perform well with fewer sampling steps, leading to a shorter overall runtime than other baselines. For an intuitive explanation, see Remark D.2 in Appendix D.", "claims_and_evidence": "The authors claim three conditions to hold for diffusion-based inverse problem solvers and a new optimization-based sampler to enforce the three conditions, which are corroborated by the experimental results.", "methods_and_evaluation_criteria": "SITCOM optimizes the noisy sample $v_i$ at time $i\\Delta t$ to satisfy the measurement constraints and perform a resampling step in each diffusion step. The proposed method is evaluated on inverse problems of natural images and MRI data, measured by common reconstruction metrics, PSNR, SSIM, and LPIPS.", "theoretical_claims": "The authors state three desirable consistency conditions, measurement, backward, and forward consistency, to solve inverse problems. The backward consistency is claimed to be new compared to previous literature. The backward consistency means a reconstruction $\\hat x_0^\\prime$ is the Tweedie output of the diffusion model for some input $v_t$. The proposed method indeed enforces three consistency conditions.", "experimental_designs_or_analyses": "The authors choose eight image inverse problems commonly studied in previous works. The experimental design is reasonable and demonstrates the effectiveness of the proposed algorithm.", "relation_to_broader_scientific_literature": "This paper may be of interest to a broader audience in other scientific domains that consider inverse problems with rich prior data.", "essential_references_not_discussed": "Some recent works on diffusion-based inverse problem solving are not discussed in this paper. \n\n- Moufad et al. \"Variational Diffusion Posterior Sampling with Midpoint Guidance.\" arXiv:2410.09945\n- Janati et al. \"Divide-and-Conquer Posterior Sampling for Denoising Diffusion priors.\" NeurIPS 2024.\n- Wu et al. \"Principled Probabilistic Imaging using Diffusion Models as Plug-and-Play Priors.\" NeurIPS 2024.\n- Dou et al. \"Diffusion Posterior Sampling for Linear Inverse Problem Solving: A Filtering Perspective.\" ICLR 2024.\n- Cardoso et al. \"Monte Carlo guided Denoising Diffusion models for Bayesian linear inverse problems.\" ICLR 2024.\n- Wu et al. \"Practical and Asymptotically Exact Conditional Sampling in Diffusion Models.\" NeurIPS 2023.", "other_strengths_and_weaknesses": "#### Strengths\n\n- This paper establishes three reasonable consistency conditions to guide the design of the algorithm.\n- SITCOM demonstrates better performance in image restoration and MRI compared to existing methods.\n\n#### Weaknesses\n\n- The definition of backward consistency is confusing. It requires $\\hat x_0^\\prime$ to be the output of the diffusion model for some input $v_t$. It seems very hard to check whether this requirement is satisfied, given $v_t$ is an arbitrary input to the diffusion model. While SITCOM automatically satisfies this condition, it is not convincing that previous methods do not satisfy this condition just by looking at the definition.\n- This paper is missing several recent references. It is possible that the comparison to these works will influence the position of this paper.", "other_comments_or_suggestions": "- Page 6 looks too compact in its current form.", "questions_for_authors": "- How are the hyperparameters $\\delta, \\gamma, \\lambda, K$ determined? Does the performance of SITCOM depend on specific choices of these hyperparameters?\n- Could the authors comment on the missing references, and why are these methods not included in the comparison?\n\n- What are the reasons for SITCOM running faster than baseline methods given its additional overhead of resampling and multiple gradient updates?\n\nJustification: I have some concerns about this paper as listed above, but I am open to revisiting my decision if the authors can address these concerns.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3CySPgD7Z6", "reviewer": "Reviewer_Q3J7", "summary": "Authors propose Step-wise Triple-Consistent Sampling (SITCOM) an optimization-based sampling method that enforces measurement consistency, forward diffusion consistency, and incorporates step-wise and network-regularized backward diffusion consistency that maintains a diffusion trajectory by optimizing over the input of the pre-trained model at every sampling step. Authors show that these conditions require significantly less number of reverse diffusion steps. Authors demonstrate the efficacy of their method for different levels of measurement noise, various operators (five linear and three nonlinear image restoration tasks and one medical image reconstruction) and show that SITCOM achieves competitive or superior results in terms of standard image similarity metrics and required run-time.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments.  \n\nWe appreciate the reviewer’s recognition of our clear and well-supported claims, as well as their positive feedback on our experimental analysis and writing.\n\nOur point-by-point response is provided below.\n\n### (W2 and S1) **Memory usage for SITCOM.**\n\nThank you for the comment! Yes, in the paper we only discussed computation load instead of memory usage. Indeed, due to the inclusion of back-propagation, we require the output of each intermediate layer to be stored, increasing the memory usage from $\\mathcal{O}(C\\times H\\times W)$ to $\\mathcal{O}(L\\times C\\times H\\times W)$, where $C$, $H$, $W$ are the maximum number of channels, maximum image height and width across outputs of all layers, and $L$ is the total number of layers. \n\nAs a result, our memory usage is approximately $L$ times higher than that of methods requiring only a forward pass, assuming that the output sizes are roughly the same across all layers. However, **the specific Unet architecture** currently used in the diffusion model has an unbalanced layer structure**, where each layer output in the encoder is roughly half the size of the previous one, and each layer output in the decoder is approximately twice the size of the previous one. Consequently, the peak memory consumption due to storing all intermediate outputs is only moderately increased, **roughly four times that of forward-only propagation**.\n\nIn addition, **it is possible to greatly reduce the memory by incorporate gradient checkpointing** such as in [A] and [B], so overall, we do not deem memory usage as a major bottleneck of our method.\n\nWe will include this discussion into the revised paper as a limitation, as suggested.\n\n[A] Memory-Efficient Back-propagation Through Time (https://arxiv.org/abs/1606.03401)\n\n[B] Aligning Text-to-Image Diffusion Models with Reward Backpropagation (https://arxiv.org/abs/2310.03739v2)\n\n### (W1) **Combination of existing ideas from ReSample, DPS and DCDP.**\n\nOur method is indeed motivated by the limitations of DPS and ReSample in terms of requiring a large number of sampling steps. However, we found that incorporating the proposed **network-regularized iterative backward consistency** not only improves quality but also significantly reduces the number of sampling steps and, consequently, the run-time. \n\nFurthermore, explicitly formulating the three consistency conditions may help clarify the individual effect of each condition and inspire future methods of a similar nature. We note that DCDP also does not utilize the network-regularized backward consistency. \n\nWe refer the reviewer to Appendix C where we show the impact of each individual consistency on SITCOM’s performance. Further insights and relation to DPS and ReSample are given in Appendix D and Appendix E, respectively. \n\n### (S3) **Arrows in the visualization of Figure 2 (left).**\n\nThank you for pointing this out. We will fix it in the revised paper. \n\n### (Q1) **Range of images before passing it to the measurement operator?**\n\nFor natural images and MRI, the real and imaginary part, the range is [-1,1]. Thank you for pointing this out. We will include it in the revised paper.", "claims_and_evidence": "Claims made in the submission are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "Selected baselines, tasks, and the datasets are all relevant and appropriate for the application at hand.", "theoretical_claims": "The paper does not have theoretical claims. The derivations of equation 5 (Appendix A) appears correct to me.Experimental design follow what is standard in multitute of works (DPS, ReSample, etc.). Analysis of experiments are sound.", "experimental_designs_or_analyses": "Experimental design follow what is standard in multitude of works (DPS, ReSample, etc.). Analysis of experiments are sound.", "relation_to_broader_scientific_literature": "The core ideas follow from the papers ReSample, DPS, and DCDP. This work generalizes ReSample method which is extensively discussed in Appendix D.1 by the authors.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "***Strengths:***\n* Experimental results in terms of baselines and tasks are diverse, extensive and convincing.\n* Paper is written very well. I enjoyed reading it.\n* Extensive ablation studies are conducted. These include noise robustness, choice of consistency terms, choice of $N$ and $K$, stopping criteria.\n\n***\n***Weaknesses:***\n* The paper combines existing ideas from ReSample, DPS and DCDP. In that sense, technical novelty is limited.\n* The method is inherently more computation heavy due to the need for back-propagating through the denoising model. While the runtime can be offset with smaller choice of $N$ and $K$, it will be more memory hungry compared to baseline models that do not back-propagate through the model.", "other_comments_or_suggestions": "***Suggestions:***\n* I would expect increased peak GPU memory usage when back-propagating through the denoising model. This should be disclosed as a limitation.\n* I would recommend revising the SITCOM algorithm visualization in Figure 2 (left). The arrows at the bottom guide the reader to look from left to right whereas it is the opposite. Moreover, left and right figures are too close to each other which makes it look visually crowded.", "questions_for_authors": "1) Could you clarify the range of images before passing it to the measurement operator? [0,1], [-1,1], etc.?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper handles three necessary conditions such as forward consistency, measurement consistency, and backward consistency for introducing better posterior sampling on solving inverse problems.

All three reviewers are convinced of papers' quality, especially on analysis of three conditions (uV3o, VNrg) and on clarity of supportive experiments (Q3J7). Even there are negative comments on the technical novelties (Q3J7,uV3o), computational burden (Q3J7), and missing references(VNrg), the authors resolved such concerns in rebuttal phase. Finally, three reviewers recommend accepts to this paper. I also agreed on reviewers' comments.

I recommend publication of this paper.
Decision: Accept (poster)