Fast samplers for Inverse Problems in Iterative Refinement models
Authors: Kushagra Pandey
Ruihan Yang
Stephan Mandt
Venue: NeurIPS 2024 poster (2024)
Keywords: Inverse Problems
Diffusion models
Fast sampling
Primary area: diffusion_based_models
TL;DR: We present fast samplers for solving inverse problems in diffusion and flow models

Abstract: Constructing fast samplers for unconditional diffusion and flow-matching models has received much attention recently; however, existing methods for solving *inverse problems*, such as super-resolution, inpainting, or deblurring, still require hundreds to thousands of iterative steps to obtain high-quality results. We propose a plug-and-play framework for constructing efficient samplers for inverse problems, requiring only *pre-trained* diffusion or flow-matching models. We present *Conditional Conjugate Integrators*, which leverage the specific form of the inverse problem to project the respective conditional diffusion/flow dynamics into a more amenable space for sampling. Our method complements popular posterior approximation methods for solving inverse problems using diffusion/flow models. We evaluate the proposed method's performance on various linear image restoration tasks across multiple datasets, employing diffusion and flow-matching models. Notably, on challenging inverse problems like 4x super-resolution on the ImageNet dataset, our method can generate high-quality samples in as few as *5* conditional sampling steps and outperforms competing baselines requiring 20-1000 steps. Our code will be publicly available at https://github.com/mandt-lab/c-pigdm.

Fast Samplers for Inverse Problems in Iterative Refinement Models
Kushagra Pandey*
Ruihan Yang"
Department of Computer Science University of California Irvine pandeyk1 fucii.uuiiioo @uci
Department of Computer Science
University of California Irvine ruihan. ruihan.yang@un yang@uci. edu
Stephan Mandt
Department of Computer Science University of California Irvine mandt@uci. edu
Abstract
Constructing fast samplers for unconditional diffusion and flow-matching models has received much attention recently; however, existing methods for solving inverse problems, such as super-resolution, inpainting, or deblurring, still require hundreds to thousands of iterative steps to obtain high-quality results. We propose plug-and- play framework for constructing efficient samplers for inverse problems, requiring only pre-trained diffusion or flow-matching models. We present Conditional Conjugate Integrators, which leverage the specific form of the inverse problem project the respective conditional diffusion/flow dynamics into a more amenable space for sampling. Our method complements popular posterior approximation methods for solving inverse problems using diffusion/flow models. We evaluate the proposed method's performance various linear image restoration tasks across multiple datasets, employing diffusion and motthhnhhnng models. Notably, on challenging inverse problems like 4x super-resolution on the ImageNet dataset, our method can generate high-quality samples in as few as conditional sampling steps and outperforms competing baselines requiring 20- 1000 steps. Our code will be publicly available at (itttt::: / com/mande 1ab/ c- c-pigdm
Introduction
Iterative refinement models, such diffusion generative models and flow matching methods [Sohl
Dickstein et al. 2015 Ho et al. 2020 (ong et al. 2020 Lipman et al. 2023 Alhergo and Vanden-
Fijnden [023)
have increasing popularity in recent months, and much effort has been invested
in accelerating unconditional sampling these models Pandey et al. 2024 Shaul et al. 2024 Sauer
al. 2024
Karras al. 2022 Salimans and Ho 2022 Zhang and Chen 2023 Lu
2022 Song
et al.
2021 However, while most efficient samplers have been designed in the unconditional setup,
current methods for solving inverse problems, such deblurring, inpainting,
super-resolution,
still require hundreds to thousands of neural network evaluations to achieve the highest perceptual quality. Moreover, in addition to score function evaluation, class of existing methods for solving inverse problems using pre-trained unconditional iterative refinement models often involves expensive Jacobian- vector products [Song et al. 2022 Chung al. 2022a making single sampling step quite expensive and therefore, intolerably slow for most practical applications.
Equal contribution
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

0
+ 1 H, w. 0)
Belter-Cnnditoned dynamics
Ill-Conditioned dynamics
Project back when done
①b (It,, H, W. t,)
C t,
Project to another space
@ to
Figure 1: Illustration of Conditional Conjugate Integrators for Fast Sampling in Inverse Problems. Given an initial sampling latent xt, at time 4, our sampler projects the diffusion/flow dynamics more amenable space for sampling using a projector operator which is conditioned on the degradation operator H and the sampling guidance scale w. The diffusion/flow sampling is then performed in the projected space. Post completion, the generated sample in the projected space is transformed back into the original space using the inverse of the projection operator, yielding the final generated sample. We define the form of the operator in Section 2.2 Conditional Conjugate Integrators can significantly speed up sampling in challenging inverse problems and can generate high-qualit samples in as few as NFEs as compared existing baselines, which require from 20-1000 NFEs (see Section 3).
This paper presents a principled framework for designing efficient samplers for guided sampling in iterative refinement models, accelerating existing samplers like ITGDM by order of magnitude. We present our framework for inverse problems where the degradation operator is known and might be corrupted with additional noise. Crucially, transformations do not require any re-training and merely rely on some algebraic manipulations of the equations to be simulated.
Intuitively, expand on the concept of Conjugate Integrators [Pandey et al. 2024 by projecting the conditional generation process inverse problems to another space that might be better conditioned for faster sampling (See Figure []. To this end, we separate the linear and non-linear components in the generation process and parameterize the transformation by analytically solving the linear coefficients. By the end of the sampling procedure, map back to the original sampling space, leading to the concept of Conditional Conjugate Integrators that apply to various iterative refinement models such diffusion models, flows, and interpolants.
In more detail, our main contributions are as follows.
Conditional Conjugate Integrators: We repurpose the recently proposed Conjugate In- tegrator framework (Pandey et al.) 2024] for fast guided sampling in iterative refinement models (diffusions and flows) for linear inverse problems and refer it Conditional Con- jugate Integrators. Next, we design a specific parameterization of the proposed framework, which encodes the structure of the linear inverse problem in the sampler design itself.
Theoretical Analysis: Our parameterization exhibits theoretical properties that help us identify key parameters for sampler design. More specifically, we show that our parameteri- zation (by design) enables recovering high-frequency details early on during sampling. This further enables fast- fast-guided sampling while maintaining good sample quality in the context of inverse problems.
Empirical Results. Empirically, we show that our proposed sampler significantly improves over baselines in terms sampling efficiency on challenging benchmarks across inverse problems like super-resolution, inpainting, and Gaussian deblurring. For instance, on challenging 4x superresolution task on the ImageNet dataset, our proposed sampler achieves better sample quality at 5 steps, compared to 20-1000 steps required by competing baselines.

---

Additionally, we extend the proposed framework for noisy and non -oneainear inverse problems with qualitative demonstrations.
2 Fast Samplers for Inverse Problems using DiffusionssFlows.
2.1 Background and Problem Statement
Diffusion models define a continuous-time forward process (usually with an affine drift) to convert data X E IR into noise. A learnable reverse process is trained to generate data from noise. In this work, we only consider deterministic reverse processes specified as an ODE [Song et al. 2020].
G.G. Xx, (x)) dt. Fxx:
dx:-
2
The score is usually intractable and is approximated using a parametric estimator Sy Xe. f), trained using denoising score matching (Vincent, [211T (onn and Ermon! 2019 /Song et al. 2020] Analo- gously, oiedsided stochastic interpolants (Alerggo and Vanden- Vanden-Eijnden [023] define an interpolan" " X2 0yX1 + NZ where X1 Pdata .and
NN0 I) to define transport map between the generative prior (typically an isotropic Gaussian) and the data distribution. Interestingly, the one-sided interpolant induces vector field b(x,, t) R[ + fxx/xx]] where di, te represent the time derivatives of 04 and "t, respectively. The vector field b(.) is typically learned using a neural network approximation bø(x,, t). The deterministic interpolant process can then be specified as dx; = (x(x, t) dt. Numerically solving these deterministic generative processes with sufficient sampling budget can generate plausible samples from noise.
Problem Statement. Given a noisy linear degradation process (we will consider non- oon-linear processes later) with degradation operator H specified over unobserved data point Xo.
== Hxo + oz z ~~ (O, Xg ~ Pdata:
(2)
the goal is to recover the original signal Xy- Additionally, given an unconditional pre-trained diffusion or flow matching model, one approach for solving inverse problems is to infer the posterior distribution over the data given the degraded observation, i.e., p(xoly) x p(y (x))((x)) (xn) by simulating the conditional reverse process dynamics i.e.
dx: = Fix: 1 G.G. xx, logp(x:lu) dl,
Diffusion:
(3)
2
Flows:
dx; = b(x,, y, t)dt,
where Xx, logl(x))g) and y are the conditional score and velocity estimates, respec- tively. One approach could be to directly model the conditional score velocity estimates using a conditional iterative refinement model (Saharia et al. 2022a bj. However, such approaches are groblem-dependene requiring expensive training pipelines to account for the lack of generalization across inverse problems. Additionally, such methods rely on the availability of paired (x,, y) mea- surements, which can be expensive to acquire. Alternatively, problen-agnossti agnostic methods leverage pre-trained unconditional iterative refinement models to estimate the conditional score or velocity fields and generalize different inverse problems without extra training. In this work, we restrict our discussion to the latter and discuss estimating conditional score/velocity fields next.
Estimating Conditional Score/Velocity from Pretrained Models:
For diffusion models, ap- proximating the conditional score follows directly from Bayes Rule, i.e. Vx, log logp(x;ly) a sa (x(x,, 1) + w. Xx, log ply xx) where wy is the guidance weight (or temperature) of the distribu- tion p(y xx). Analogously for interpolants (or flows), Pokle et al. [2044] propose the conditional flow dynamics,
2t a₁
b 6xx,, y.t) (x t) + w.
node haad Vx. logp(y)):).
(4)
We include formal proof for the result in Eqn. from an interpolant perspective in Appendix A. Since the conditional score and velocity estimates require approximating the term Vx, x, log p(y xx). we discuss estimation next.
In this work, use the terms interpolant and flows interchangeably.
3

---

Estimation of the Noise Conditional Score Xx, plyp(y) The noise conditional distribution p(y| can be represented as p(y = / p(y (xx (x)) For problem -agnostic models, is common to approximate the posterior p(xo xx) using unimodal Gaussian distribution [Chung
al. 2022a Song al. 2022]. In this work, we restrict our discussion to the posterior approximation
in IIGDM [Song et al. 2022 and its flow variant (Pokle et al. [024] (named as IIGFM in our work),
p(xo|xt) R (Xo, ri [a), which yields the following estimate the conditional score:
xx logp(y/xx) 0&o T
H H
+ + H
(5)
Õx
where &o is the first-order Tweedie's moment estimate [Stein 1981 Our choice of using the ITGDM approximation is motivated by its expressive posterior approximation p(ro) (xx) compared to other methods such as DPS MCG. This makes it an excellent starting point for low-budget sampling.
2.2 Conditional Conjugate Integrators
Conjugate Integrators The main idea in conjugate integrators [Pandey et al. 2024] is to project the diffusion dynamies in Eqn. iinto another space where sampling might be more efficient. The projected diffusion dynamics can then be solved using any numerical ODE solver. On completion, the dynamics can be projected back to the original space to generate samples from the data distribution. To this end. Pandey et al. (2024) introduce an invertible time-dependent affine transformation &, = A, Interestingly, conjugate samplers have theoretical connections to prior work in fast sampling for unconditional diffusion models [Song et al.) 2022 I
Zhang and Chen 2023
Lu et al. 2022 We refer
the readers Pandey et al. 2024 for exact details.
2.2.1 Conjugate Integrators for Inverse Problems
Next, we design conjugate integrators for linear inverse problems. For simplicity, we discuss noiseless inverse problems, By 0, and defer the discussion of noisy inverse problems to Section Furthermore, due to space constraints, we present our analysis for diffusion models and defer the discussion of flows to Appendix [ Lastly, without loss of generality,
2.4
assume the standard score network parameterization, &n X,, t) ( (t)eg(x,,)) t) where Coun (t) is the notation from the score
precondition defined in Karras al. 2022
straightforward way to define conditional conjugate integrators is to treat the score estimate Xx, log p(y) xx) black-box i.e., ignore the structure of the inverse problem. For this case, we formally specify the conjugate integrator formulation as,
Proposition 1. (Extended LIGDM) For the conditional diffusion dynamics defined in Eqn. introducing a diffeomorphism, X₁ Aixi, where,
A exp (
(B.- B, FF
$ -1-- (-¹ A.G, Coul
(6)
induces the following projected diffusion dynamics,
Wert A,G.G,
O&o
d&: = A, *tdtt + dæcco (x,,t)
(H dx
2
Ox,
where Ht H (H and P HH (HHT)-1 represent the pseudoinverse and the orthogonal projector operators for the degradation operator H. (Proof in Appendix A.2
Similar to Pandey et al. 2024), the matrix B, is design choice. We refer to the formulation in Eqn. as Extended IIGDM since for B, = 0, the ODE in Eqn. 7 becomes equivalent to the IIGDM formulation proposed in Song et al. 2022 This is because, for B, = O, Conjugate Integrators cquivalent to DDIM Song et al. 2021 (See Pandey et al. (2044 for proof). Therefore, the projected diffusion dynamics in Eqn. 7 already present a more generic framework for designing samplers for inverse problems over HIDMM. In this work, we only explore the parameterization in Eqn. 7 for
B, and hence refer to it simply IIGM (analogously IIGFM for flows; see Appendix B).
One characteristic of the formulation in Eqn. 7 is the black-box nature of the conditional score Vx, log p(y)),). However, the inherent linearity in the conditional score can be used to design better conditioned (more on this in Section 2.3 conjugate integrators, which we illustrate formally the form of the following result.

---

Analogously, for Proposition 2. (Conjugate IIGDM) Given a noiseless linear inverse problem with oy 0, design matrix B [0. 1] + gaxd, and the conditional score Xx logply'', pxy(x;)))))))))))))))))))) approximated using Eqn. 5
introducing the transformation X A, where
A
- exp [/, B. (B.+
W.T.
G.G. P)ds) do
(8)
2p
induces the following projected diffusion dynamics:
dx; = A, &,dt + d yy + d t) d Ox co (HH' PŁo)],
(9)
where exp(.) denotes the matrix exponential, H and P are the pseudoinverse and projector
operators (as defined previously). Proof in Appendix A.3
In this case, the coefficients yy 3j. and D, depend on time t and the degradation operator H (See Appendix A.3 for full definitions). Intuitively, by including information about the degradation operator H and the guidance scale in the transformation A, in Eqn. [] we incorporate the specific structure of the inverse problem in the sampler design, which can have several advantages (more this in Section 2.3 Moreover, the matrix B, is a design choice of our parameterization (we will discuss exact choices in Section 2.2.2 We refer this parameterization as C-IIGDM (analogously C-1IGFM for flows; see Appendix B). In this work, we restrict our discussion to this parameterization and discuss some practical and thcoretical aspects next.
2.2.2 Practical Design Choices
Choice of Diffusions and Flows:
While our proposed integrators are applicable to generic diffusion
processes [Dockhorn al.
2022
Pandey and Mandt 2023] and flows MM et al., 2024 restrict
follow-up discussion to VP-SI SDE
Song et al.
2020 diffusion for which F, = 3 a. G, V
and OT-flows [Liu al. 20223
Lipman al.
For score 1/r,, corresponding to the standard e-prediction (Ho
2023] for which an = I, 7t I
network parameterization, we set (c.
al. 2020 Song et al. 2020 parameterization in diffusion models.
Choice of BN: Similar to Pandey et al. [0024], we set B, = Ala, where À is a time-invariant scalar hyperparameter tuned during inference for optimal sample quality.
Choice of wy: Similar to prior work [Song et 2022 Poklee al. 2024), we use an adaptive guidance weight schedule. For diffusion models,
wy wpir? where ri
of
flows, we set wafr? where r²
af+h
Having an extra multiplicative factor of μf (for VP-SDE) or af (for flows) stabilizes the numerical computation of coefficients in Eqn. before sampling. We tune the static guidance weight w during inference for optimal sample quality.
Choice of Start Time: Given degradation output it is common to start diffusion or flow sampling at 7 < T T O, respectively [Chung al., 20226 Song al. 2022 Pokle et al. 2024). Consequently, we initialize the diffusion sampling process as X, H, H y + Analogously for flows, we initialize sampling x, a, + 7+z.
Choice of the ODE Solver: Unless specified otherwise, we use the Euler discretization scheme for C-IIG(DFFM samplers.
illustrate a generic C-IIGDM sampling routine in Algorithm and include additional implemen- tation details in Appendix D Next, we present some theoretical aspects of our proposed method.
2.3 Theoretical Aspects
With the simplifications in Section 2.2.2] the transformation A, in Eqn. simpliiees to:
A = exp /// (A+} dsI (f Bads) p].
1
(10)
where P H. (HH)) T
y -1 H is orthogonal projection operator.
5

---

Algorithm Conjugate IIGDM sampling
1: Input: Corrupted observation y, Corruption operator H. Denoiser ca(., .). Choice B,, NFE budget N, Timestep discretization {t:} fi=0. Diffusion kemel p(xx) (x) NNuxx of Ta), Start time T.
Output: Clean sample Xo
Pre- Pre-Compute (Ac, 1N (Eqn.
Pre-compute coefficients
4: Pre-Compute { $ $ 1
(see App. A.3
5: P(xT)
Draw initial samples from the generative prior
6: X μ. H + 0,2
> Initialize using the pseudoinverse (See Chung et al. [2022b] )
7: X A-X
Initial Projection Step
8:
for 110 N - do
9:
h (tn+1 - tn)
Time step differential
10:
X A -x x
11:
& Peo Ix o1, co (x, [n
Tweedie's 's Estimate
12:
= hAc, BI, A, x x ( 100+1-$0 -b,1
> Linear drift
13:
Mni ( E0) (n) + + $3) (x, Pxo)
Non-Linear drift
14:
x == x + vi Vna
D Euler Update
15:
end for
return A
Project back to original space when done
Computing A.: While computing the matrix exponential in Eqn. has several interesting properties that make tractable compute. More specifically, the matrix exponential Eqn. T0 can be simplified as (Proof in Appendix A.4
10 mighht seem non-trivial,
(1 - -/,' (A+53.))s (2(1)
f Bads,
A. exp(xi La+
( [=+(ex((ss())))))))
(11)
where exp(-) in Eqn. represents the scalar exponential. Furthermore, the integrals in Eqn. 11 trivial compute analytically numerically, making A, easier to compute. Moreover, A, also be compactly represented as,
A,- exp(-xx [a (exp(-<2(()) - 1)
(12)
and is also tractable to compute. Due to the tractability of A, and A the projected diffusion dynamics in C-IGGDM straightforward simulate numerically.
Intuition behind A₁: Next, we analyze several theoretical properties of the transformation matrix A, in Eqn. 11 More specifically,
8; A,x, exp(ki (f)) x, (1 exp (xx)
(13)
Since P P H - is an orthogonal projector, the matrix Ia P is also an orthogonal projector which projects any vector the nullspace of P. Therefore, we can decompose the state into two orthogonal components Pxt + (Ia P)x, Plugging this form in Eqn. [3
x, = cxp((x ( P)x, (xp(x2(t)) Px;
(14)
Intuitively, near t T (i.c., at the start of reverse diffusion sampling). for a large static guidance weight w, ex)(x2(t)) + 0. In this limit, from eqn. 14. X; R3 (I. P)x,. This implies that for large guidance weight w, the diffusion dynamics are projected into the nullspace of the projection operator P. Intuitively, for an inverse problem like superresolution, this implies that near the start of the diffusion process, the projected diffusion dynamics correspond to the denoising of the high- frequency details missing in Px, This is because the projector operation, Px, H Hx, can be interpreted as the pseudoinverse of the noisy degraded state x,, and, therefore, (I, - P)x, represents the figh-freuuency details missing from the signal component in Px.-
Moreover, near i (i.e., near the end of reverse diffusion sampling), assuming the guidance weight is not too large, both coefficients exp(1i (t)) and exp(x2(t)) - 1, which implies x,. This

---

implies that near = O, diffusion happens in the original space, which can prevent over-sharpening artifacts towards the end of sampling. Therefore, we hypothesize that a large w can also lead to over-sharpened results near the end of sampling, resulting in artifacts in the generated samples. Therefore, introducing the projection A, as defined in Eqn. [T0 introduces tradeoff in the choice of w to control for sample quality. Lastly, since the parameter À controls the magnitude of x,, it exhibits a similar tradeoff. Indeed, we will empirically demonstrate these tradeoffs in Section 3.3 While our discussion has been limited to diffusion models, similar theoretical intuition also holds for flows (See Appendix for proof).
2.4 Extension Noisy and Non-Linear Inverse Problems
While our discussion has been primarily in the context of noiseless linear inverse problems, the conditional Conjugate Integrator framework can also be extended to develop samplers for noisy linear and non-linear inverse problems. We provide a more detailed explanation for the same in App. C
3 Experiments
Next, empirically demonstrate that our proposed samplers C-IIGDM/GFM outperform recent baselines on linear image restoration tasks regarding sampling speed vs. quality tradcoff. We then present ablation experiments highlighting the key parameters of our samplers. Lastly, we present design choices for solving noisy and non-linear inverse problems using our proposed framework.
Models and Dataset: For diffusion models, we utilize an unconditional pre- pre-trained ImageNet [Dng et al. 2009) checkpoint at 256x 256 resolution from OpenAl (Dhariwal and Nichol 2021 For
evaluations on the FFHQ dataset Karras et al.] [2019] we use pre-trained checkpoint from Choi
ct al. [202T] also at 256 256 resolution. For flow model comparisons, we utilize three publicly
available model checkpoints from Liu al. [20223 trained on the AFHQ-Cat (Choi et al. 2020).
LSUN- SSUN-Bedroom/ Yu et al. 20155, and CelebA-HQ Karras et al., 2018) datasets. Each flow model was trained at pixel resolution of 256 x 256. For diffusion models, we conduct evaluations lk
subset of the evaluation set. For flows, we conduct evaluations on the entire validation set.
We evaluate our samplers qualitatively (see Figure 2 and quantitatively on three challenging linear inverse problems under the noiseless setting. Firstly, we test Image Super- Super-Resolutin Resolution, enhancing images from dicubie-downsampled 64 x 64 pixels to 256 x 256 pixels. Secondly, we assess Image Inpainting performance on images with fixed free-form center mask. Lastly, we evaluate our samplers on Gaussian Deblurring, applying a Gaussian kernel with a = 3.0 across a 61 x 61 window. We evaluate the performance of each task based on three perceptual metrics:
Tasks and Metrics:
FID Heusel et al. 2017]. KID [Binkowski et al. 2018) and LPIPS (Zhang al. 2018
Methods and Baselines: We assess the sample quality of our proposed C-IIGDM and C-IIGFM samplers using 5, 10, and 20 sampling steps (denoted as Number of Function Evaluations (NFE)). We conduct an extensive search to optimize the parameters w, À and J to identify the best-perf configuration based on sample quality. For diffusion baselines, we include DDRM (Kawar et al.
2022 DPS [Chung et 2022a and IGDMM (Song et al. 2022]. As recommended for DPS [Chung et al. 2022a]. J. we use 1PF==10000 for all tasks. For DDRM, we adhere to the original implementation and run with пe = 1.0 and 7 0.85 at NFE=20. We test our implementation of IIGDM (see Section 2.2 with NFE values of 5, 10, and 20 and use the recommended guidance schedule of Wt = r across all tasks. For flow models, we consider the recently proposed method inspired by IIGDM running on OT-ODE path by Pokle et al. (2024) (which we refer to as IIGFM; see Appendix
B). and similarly run with NFE values of 5, 10, and 20. We optimize all baselines by conducting an
extensive grid search over w and 7 for the best performance (in terms of sample quality).
3.1 Quantitative Results
We present the results of our method applied to inverse problems in Table specifically using the CelebA-HQ dataset for flow-based models and the ImageNet dataset for diffusion-based models. For
7

---

Results
NFF
LPIPS
KIDx10
FID.
C-IIGEM
IIGM
C-IGGEM
IIGMM
C-IIGEM
IIGM
0.125
0.240
17.6
167.0
26.95
161.99
Inpainting
10
0,074
0.188
8.0
86.6
14.64
94,91
20
0.063
0. 144
4.6
54.4
10.93
65.39
5
0.063
0.091
5.5
17.5
13,08
21.84
Super-Resolution
0.058
0.076
3.6
12.2
10.65
16.73
20
0.064
0.069
3.9
3.5
11.07
10.23
12.86
Deblunring
10
0.077
0.088
5.0
7.0
14.41
15.09
20
0.080
0.073
7.9
3.1
17.10
11.35
Diffusion Results
C-IIGDM IGGDM
DPS
DDRM C-IIGDM IIGDM DPS DDRM C-IIGDM IIGDM
DPS
DDRM
0.220
0.306
2.7
6.3
37.31
49.06
Super. Resolution
10
0.206
0.252
0.252
0.318
1.6
4.8
5.8
14.1
34,22
44.30
38.18
51.64
20
0.207
0.222
1.7
34.28
37.36
0.272
0.349
44.42
63.94
Deblurring
10
0.272
0.294
0.619
0.336
3.6
5,3
59.5
12.3
43.37
47.80
139.58
62.53
20
0.268
0.259
4.2
43,70
44.20
Table Comparison between Conjugate TTG(D/F)M and other baselines for noiseless linear inverse problems. Top: Flow models (CelebA-HQ) and Bottom: Diffusion Models (ImageNet). Entries in bold show the best performance for given sampling budget.
a comprehensive review of additional results across different datasets, please refer to Appendix Our method consistently surpasses other approaches across all sampling budgets (indicated by NFE) for the inpainting task. Similarly, our flow-based sampler (C-ITGFM) exhibits superior perceptual quality for image super-resolution at NFEs of 5 and 10. The ITGFM model only reaches comparable performance at higher NFEs. Remarkably, our diffusion- sampler C-IIGDM outperforms all baselines across the entire range of NFEs. Notably, C-IIGDM outperforms competing baselines requiring 20- 10-1000 NFEs in just sampling steps on the challenging ImageNet dataset, demonstrating significant speedup in sampling speed while preserving sample quality. A similar pattern is observed in the image deblurring task, where the performance of IIGDM/IGGM approaches that of our method only when the NFE is increased 20 steps.
Interestingly, we observe plateau performance improvements at NFE=20 for both super-resolution and deblurring tasks using our method. This suggests that while our method efficiently utilizes the iterative model under deterministic path with an Euler solver, further enhancements in performance, particularly at higher NFEs, might require integrating stochastic sampling techniques or more advanced solvers. This potential next step could unlock further gains from our approach complex image processing tasks.
3.2 Qualitative Results
Figure presentsss qualitative comparison between our proposed method and the ITG(D/F)M baseline. The inpainting results in the first column reveal that ITGFM tends to introduce gray artifacts within the inpainted areas. This issue may stem from the initialization of the parameter т; optimal performance is achieved when 7 2 0.2, as established during our parameter tuning phase and corroborated by Pokle et al. 2024). Consequently, insufficient NFE means IIGFM cannot effectively eliminate artifacts associated with the inpainting mask in our experiments. For image super-resolution. our method excels restoring fine details, particularly evident in high-frequency image components such human hair and wheat cars. Similarly, for the deblurring task, method qualitatively outperforms the baseline, especially in mitigating the over smoothin artifacts (Figure last column). Additional examples are provided in Appendix E.4
3.3 Ablation Studies
In this section, we further explore the impact of the hyperparameters w, X, and т. which were identified during our tuning phase and link to the theoretical insights discussed in Section 2.3 We recognize that particularly task-specific and relatively straightforward to adjust. For instance, tasks such as inpainting require smaller to prevent masking artifacts, whereas tasks like super-

---

Reference
Distorted
C-LIG(D/FM
IIG
(a) AFHQ
(b) LSUN
(c) CelebA-HQ
(d) ImageNet
(c) FFHQ
Figure 2: Qualitative comparison between C-TIG(D/FMM and ITG(//FMM baselines five different Inpainting, De-blurring, and 4x Super-resolution with C-IGFFM, respectively. (dle)
datasets. aal b
4x Image Super-resolution and De-blurring with C-LIGDM, respectively. (a, 0, NFE=5)
resolution or deblurring benefit from larger 7 to ensure effective initialization. Consequently, our discussion will primarily focus on the effects of w and X. Figure iillustrates the impact of varying w and À on sample quality for image super-resolution on the CelebA-HQ and ImageNet datasets.
From Figure 3 we make the following observations. Firstly, for both C-HGGM and C-IIGFM samplers, observe that the optimal value of À can differ from À O. This illustrates the usefulness of parameterizing B, in our sampler design. On the contrary, IIGDM or IIGFM samplers do not have this flexibility and, therefore, yield sub-optimal sample quality at different sampling budgets. Secondly, we observe that deviating from the optimal À can lead to degradation in sample quality. More specifically, we observed that deviating from our tuned value of leads either over-sharpening artifacts or blurry samples (See Figs. TT). This is intuitive since X controls the scale of the transformation X; A.x; (see Eqn. (4) and thus plays a significant role in conditioning the projected diffusion dynamics. We observe similar tradeoff on varying the static guidance weight w where large magnitude wr can lead to over-sharpened artifacts while a very small guidance weight can lead to blurry samples (See Figs. 6 10 These empirical observations are consistent with our theoretical analysis in Section 2.3 confirming our theoretical intuition on the role the sampler parameters and X.
Related Works
Fast Unconditional Sampling: Recent research has significantly advanced the efficiency of the
sampling process unconditional diffusion/flow models [Song al., 2020 Lipman et al., 2023] Manduchi et al. 2024 One line of research involves designing efficient diffusion models to improve
sampling by design Karras al. 2022 Dockhorn al. 2022 Pandey Mandi 2023] Song et al.)
2023 Since treatment of conditional Conjugate Integrators is quite generic, our method is readily

---

0085
420
336
130
123
424
332
030
-30
00 9.5
-20
-65
65
10
(a) Flow À
(b) Flow
(c) Diffusion
(d) Diffusion w
Figure 3: Impact of X and sampling quality, Red curves and labels represent the LPIPS scores,
while blue curves and labels indicate the FID scores.
compatible with most advancements in diffusion model design. Another line of work focuses on distilling student model from a teacher model, enabling sampling in even a single step [Salimans)
and Ho 2022 Meng al. 2023 Sauer et al. 2024]. However, since these methods require expensive
re-training, there has been a significant interest in the development of fast samplers applicable to
pretrained diffusion/flow models [iu al. 2022
Pandey al.) 2024 Shaul al., 2024 Zhang
and Chen 2023 Lu et al., [022] Song et al. 2021
Gonzalez et al. 2023 Our work falls under the
latter line of research, where we develop fast conditional samplers that can be applied to pretrained diffusion models.
Conditional Iterative Refinement Models
have become prevalent for tasks requiring controlled generation. These models often involve training specialized conditional diffusion models (Saharia
iet al. 2022b Yang and Mandt [0033 Kong et al.] [202]] Panddyy et al. [2022] Preechakul et al. 2022
Rombach et al. [0222] Podell et al. Ramesh et al. 2022 Peebles and Xie [023] Ma et al. 2024
Esser et al. 2024 Chen et al. 2024 and may incorporate classifier-free guidance [Ho and Salimans
2021] or classifier guidance DDhriwal and Nichol 2021 Song al.) 2020 for conditional sampling.
These approaches have also spurred research into sol solving inverse problems related to various image
degradation transformations. such inpainting and super-resolution (Kawar et al.. 2022 Chung
ct al. 2022a Song et al. 2022 Mardani et al. 2023 Pokle et al. 2024]. Although these methods demonstrate promising outcomes, they typically bottlenecked by costly sampling process. emphasizing the need for fast sampler to address inverse problems efficiently. Recent work Xu al. [2024] employs a consistency model [Song al. [2023] enhance posterior approximation. but incorporating an additional model may deviate from our proposal of using a single pre-trained model.
[Lu al. 2023) also tackles the problem of accelerating guided sampling in diffusion models. However, unlike [Lu et al. 2023 we incorporate the structure of the inverse problem in the sampler design.
Discussion
We present a generic framework for designing samplers for accelerating guided sampling in iterative refinement models. In this work, we explore a specific parameterization of this framework, which incorporates the structure of the inverse problem sampler design. We provide a theoretical intuition behind our design choices and empirically justify its effectiveness in solving linear inverse problems as few as sampling steps compared to 20- 1000 NFEs required by competing baselines. While our method can serve as an important step toward designing fast-guided samplers, there are several important future directions. Firstly, our parameterization of the transform A, can be more expressive by learning it directly during the sampling stage. Secondly, in this work, we consider inverse problems with a known degradation operator. Extending our framework for solving blind inverse problems could be important research direction. Lastly, it would be interesting adapt our solvers to techniques for solving inverse problems in latent diffusion models [Rout et al. 2024 to enhance sampling efficiency further.
While our work has the potential to make synthetic data generation accessible, the techniques presented in this work should be used responsibly. Moreover, despite good sample quality in a limited sampling budget, restoration can sometimes lead to artifacts in the generated sample which can be undesirable in some domains like medical image analysis.
Broader Impact:
10

---

Acknowledgments and Disclosure of Funding
The authors thank Justus Will for providing valuable feedback on our manuscript. KP acknowledges support from the HPI Research Center in Machine Learning and Data Science at UC Irvine. SM acknowledges support from the National Science Foundation (NSF) under an NSF CAREER Award IIS-2047418 and IIS- NSS-0907,,, the NSF LEAP Center, by the Department of Energy under grant DE-SC0022331, the IARPA WRIVA program, the Hasso Plattner Research Center UCI, and by gifts from Qualcomm and Disney.
References
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference Machine Leurning, pages 2256-2265. PMLR, 2015.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
Yaron Lipman, Ricky T. Q. Chen, Heli Ben- Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In International Conference Learning Representations, 2023. URL htttps::/ppenreviieet wwt.:nee//ourmme wtt.::/eeeoororndddPPPPt......
Michael Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. In ICLR 2023 Conference, 2023.
Kushagra Pandey, Maja Rudolph, and Stephan Mandt. Efficient integrators for diffusion generative models. The Twelfth International Conference on Learning Representations, 2024. URL
https://openreviex wwwpeevwwwwwwweeeeeeeee met/ffrunmt http//orrrn/d/a///f0000000
Neta Shaul, Juan Perez, Ricky TQ Chen, Ali Thabet, Albert Pumarola, and Yaron Lipman. Be- spoke solvers for generative flow models. In The Twelfth International Conference on Learning Representations, 2024.
Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann. Patrick Esser, and Robin Rombach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. arXiv preprint wrXiv:2403.12015. 2024.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in Neural Information Processing Systems, 35:26565-26577, 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffuusion models. In International Conference on Learning Representations, 2022. URL https::/opeerevieee net/ Eorun?id-T1dIIIpznn
Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In The Eleventh International Conference on Learning Representations, 2023. URL https
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In- ternational Conference on Learning Representations, 2021. URL https:://pperreiierr htttteeerrreieeeeeeeeeeeee net/ #frumm?dd-StlgiarCHI
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guided diffusion models for inverse problems. In International Conference Learning Representations, 2022.
11

---

Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2022a.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Compula- lion, 23(7):1661-1674 2011. doi: 10.1162/ __10042..
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022a.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pages 1-10, 2022b.
Ashwini Pokle, Matthew J. Muckley, Ricky Q. Chen, and Brian Karrer. Training- free linear image inverses via flows. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreviex. wtw.:eew..ee/fooooeeoeeeee Patttaia
Charles M Stein. Estimation of the mean a multivariate normal distribution. The annals of Statistics, pages 133-1151, 1981.
Tim Dockhorn, Arash Vahdat, and Karsten Kreis. Score-based generative modeling with critically- damped langevin diffusion. In International Conference on Learning Representations, 2022. URL
htttps://openreviee fertforum?id--- -2ceum222ddC222
Kushagra Pandey and Stephan Mandt. A Complete Recipe for Diffusion Generative Models. In 2023 Los Alamitos, IEEE/CVF International Conference on Computer Vision (ICCV), pages 200.0.0.22.00099.. URL CA, USA, October 2023, IEEE Computer Society. doi: 10. 10.11000/ https:://oi.ieet http://a....eeemmmtttteerrooaeoa org/10 1119/ICCV11100...... 2023. 003933
Nanye Ma, Mark Goldstein, Michael S. Albergo, Nicholas M. Boffi, Eric Vanden- Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-basee generative models with scalable interpolant transformers, 2024.
Xingchao Liu, Chengyue Gong, et al. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 20223.
Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. Come-cllser-diffus--fnne conditional diffusion models for inverse problems through stochastic contraction. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 12413-12422, 2022b.
Accelerating
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pages 248-255. Iece, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401-4410, 2019.
Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. Ilvr: Conditioning method for denoising diffusion probabilistic models. 202/ JEEE/CVF International Conference on Computer Vision (ICCV), pages 14347-14356. IEEE, 2021.
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and lung-Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEFF/CVE conference on computer vision and pattern recognition, pages 8
2020.
12

---

Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large- scale image dataset using deep learning with humans in the loop. arXiv preprint 1tth::550003365. 2015.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In International Conference on Learning Representations, 2018.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Mikotaj Binkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. In International Conference on Learning Representattonss 2018.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unrcasonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 586-595, 2018.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 35:23593-23606, 2022.
Laura Manduchi, Kushagra Pandey, Robert Bamler, Ryan Cotterell, Sina Däubener, Sophie Fellenz, Asja Fischer, Thomas Gärtner, Matthias Kirchler, Marius Kloft, et al. On the challenges and opportunities in generative ai. arXiv preprint 24030:2403.00025, 2024.
Yang Song, Prafulla Dhariwal, Mark Chen, and llya Sutskever. Consistency models. In Proceedings of the 40th International Conference on Machine Learning, pages 32211-32252, 2023.
Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2023.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models manifolds. In International Conference on Learning Representations, 2022.
Martin Gonzalez, Nelson Fernande Pinto, Thuy Tran, Hatem Hajri, Nader Masmoudi, et al. Seeds: Exponential sde solvers for fast high-quality sampling from diffusion models. Advances in Neural Information Processing Systems, 36, 2023.
Ruihan Yang and Stephan Mandt. Lossy image compression with conditional diffusion models. Advances in Neural Information Processing Systems, 36, 2023.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2021.
Kushagra Pandey, Avideep Mukherjee, Piyush Rai, and Abhishek Kumar. Diffusevae: Efficient, controllable and high-fidelig generation from now-dimensional latents. Transactions on Machine Learning Research, 2022.
Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasorn Suwajanakorn. Dif- fusion autoencoders: Toward a meaningful and decodable representation. In Proceedings of the IEFF/CVF Conference Computer Vision and Patlern Recognition, pages 10619-10629, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF confer- ence on computer vision and pattern recognition, pages 10684-10695, 2022.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference Learning Representations.
13

---

Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents, 2022. URL https::/arxiv..or wttpv//.rrgabb//22.... 06125
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the /EEE/CVF International Conference on Computer Vision (ICCV), pages 4195-4205 October 2023.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion En- glish, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis, 2024.
Junsong Chen, Jincheng YU, Chongjian GE, Lewei Yao, Enze Xie, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- -talphaS: Fast training of diffusion transformer for photorealistic text-to exx-too-mmae synthesis. In The Twelfth International Conference on Learning Representations, 2024. URL
Jonathan Ho and Tim Salimans. Classifier-free diffussion guidance. In NeurlPS 202/ Workshop on Deep Generative Models and Downstream Applications, 2021.
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective solving inverse problems with diffusion models. In The Twelfth International Conference on Learning Representations, 2023.
Tongda Xu, Ziran Zhu, Dailan He, Yuanyuan Wang, Ming Sun, Ning Li, Hongwei Qin, Yan Wang, Jingjing Liu, and Ya-Qin Zhang, Consistency models improve diffusion inverse solvers. arXiv preprint arXiv:2403. 12063, 2024.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver+++ Fast solver for guided sampling of diffusion probabilistic models, 2023.
Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkottai. Solving linear inverse problems provably via posterior sampling with latent diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Michael Albergo, Nicholas M Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint
2023.
Ricky T. Q Chen. torchdiffeq, 2018. URL htl.lttss/ggttto..ccoo////geeeer
J.R. Dormand and P.J. Prince. A family of embedded runge-kutta formulae. Journal of Computational and Applied Mathematics, 6(1):19 26, 1980. ISSN 0377-0427. doi: https://doi.or//10.. 1016/
0771-00500889)90013333, URI http::tpp::// http::::s/wwww..iinneceeee www.le/ccarreet..ooo/ciieeeeeeie 0771050X80900183
Alexia Jolicceur-Marttnaauu Rémi Piché-Taillefer, Ioannis Mitliagkas, and Remi Tachet des Combes. Adversarial score matching and improved sampling for image generation. In International Conference on Learning Representations, 2021. URL htttp:://pperreviee www.:nee//eeeeee feet/forummid
eLfqM13231q
Anton Obukhov, Maximilian Seitzer, Po-Wei Wu, Semen Zhydenko, Jonathan Kyl, and Elvis Yu-Jing
Lin. High-fidelity performance metrics for generative models in pytorch. 2020. URL https Version: 0.3.0. DOI: 2e..588/2eenddo.99577777
David Minnen, Johannes Ballé, and George Toderici. Joint autoregressive and hierarchical priors for learned image compression. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL http:: httpt//ep/////.oeeenin....ss iwwp:/oeeeddiin................. cc/paper pdf
14

---

Contents
Introduction
Fast Samplers for Inverse Problems using Diffusions/Flows,
Background and Problem Statement
2.2
Conditional Conjugate Integrators
2.2.1
Conjugate Integrators for Inverse Problems
2.2.2
Practical Design Choices
2.3
Theoretical Aspects
2.4
Extension Noisy and Non-Linear Inverse Problems
3
Experiments
3.1
Quantitative Results
3.2
Qualitative Results
3.3
Ablation Studies
8
Related Works
Discussion
10
Proofs
16
Proof of Conditional flow dynamies
16
A.2
Proof of Proposition
17
A.3 Proof of Proposition
2
18
Simplification in Eqn.
20
A.5
Proof of Proposition for Solving Noisy Inverse Problems
Conditional Conjugate Integrators: Flows
22
B.
Background
22
B.2
Conditional Conjugate Integrators for Flows
22
B.2.1 Conjugate-IIGFM (C-IIGFM)
22
Extension to Noisy and Non-linear Inverse Problems
23
D Implementation Details
24
D.I C-IIGDM Practical Aspects
24
D.I.I VP-SDE
24
D.1.2 C-TIGDM Simplified Expressions
24
D.2
C-IIGPM: Practical Aspects
25
D.2.1 OT-Flows
25
D.2.2 C-IGFF:: Simplified Expressions
25
D.3 Coefficient Computation
26
15

---

D.4
Choice of Numerical Solver
27
D.5
Timestep Selection during Sampling
27
D.6
Last-Step Denoising
27
7
Evaluation Metries
27
D.8 Baseline Hyperparameters
27
Additional Results
27
E.1
Additional Baseline Comparisons
27
Comparison of Perceptual Recovery Metrics
28
Traversing the Recovery vs Perceptual trade-off
28
E,4
Qualitative Results
29
A
Proofs
A.1
Proof of Conditional flow dynamics
Proof. Given a one-sided interpolant, X, = 0, + 7 X1 Pdnta: z Wo F) satisfying
regularity conditions as stated in [Albergo al. 2023]. and degraded signal y generated using Equ. 2. the conditional velocity field 6, v,t) can be approximately estimated from the unconditional velocity field by (X,, t) as [Pokle et al. 2024
Yt b(x:, Rd (x,, () + wi
"rrce - tear Vx, logp(y(x:)
(15)
Ot
where Wi represents a time-dependent scalar guidance schedule and o,, å represent the first-order time derivatives of a, and (ı, respectively. Our proof consists of two parts: Firstly, establish the connection between the unconditional velocity field b(x;, t) for the one-sided interpolant and the score function (x,, t) associated with the marginal distribution p(x;). Secondly, use this connection to estimate the conditional velocity b(x,, y, t) in terms of b(x,, t) to establish the required result.
Connection between b(x,, and s(x,, t): For the one-sided interpolant, by definition,
04 + 7
Pdata; 2 NO. I)
(16)
Taking the expectation w.r.t p(x1, z) on both sides conditioned on the noisy state x,, we have,
or ax xx] + 7xZ/z|x,]
(17)
Furthermore, we have the following result from Albergo et al. 2023
E E
(x,,
(18)
where s(x,,t) t) represents the score function. From Eqns. 17 18 it follows,
E
Ex [x]
+ (x,, 0)
(19)
Intuitively, the above result represents Tweedie's 's estimate [Stein] 1981 for estimating x; Exx [x] in the context of one-sided stochastic interpolants. Next, the one-sided interpolant also induces an unconditional velocity field specified as:
b(x,, 1) c, E x xx] + E[2|xx
= = X.] - (xx)
(20)
where it. & represent the first-order time derivatives of 7. and cu, respectively. Substituting the result from Eqn. 19 into Eqn. 20 we have the following result,
dr "x + [yáe - Shros s(x,,t)
b(x,,t) t)
(21)
at
ar
16

---

This concludes the first part of the proof.
Estimating the conditional velocity b(x,, 9, () in terms of b(x,, (). For transport condi- tioned on y, the conditional velocity can be expressed as (following the result in Eqn. 21)
ár
bx,, v.
n: - a
s sxx,, y.t)
(22)
X
2t
8xx,, 1) wi Vx, log logp(y|x:))
(23)
02
di
7t
mede - fr 8(xc, () + w 2t (rdr - Seas Xx, lygp(y|x()
X1
(24)
b(x,,)) 1) + wy
(rcir - Star Xx, log p(y|())
(25)
Cx
Approximating the unconditional velocity b(xe: () using a parametric estimator (x (), we get the required result.
b(xt, y. 1) ss (x(, 1) we 7t (rár Seare Xx, logp(y(xx)
(26)
A.2 Proof of Proposition
We restate Proposition for convenience,
Proposition. For the conditional diffusion dynamics defined in Eqn. x, A,x, induces the following projected diffusion dynamics.
introducing the transformation
0& G.G.
d&, A.B.A, 2 + ddp.6o (xt,1)
(H'y Px1)dll
(27)
2
0x.
A, = exp ( B,
[ A.G.G. Cou(s))s
- F,ds)
D =
(28)
where Ht = H (HH and P = H (HH - represent the pseudoinvesse and the orthogonal projector operators for the degradation operator H.
Proof. We have the following form of the conditional diffusion dynamics
dx,
F,X: G.G. Vx, log ((x
(29)
dl
Fix: G.G. so (x,, t) G.G.G. Vx, logp(y(xx)
(30)
Given an affine transformation which projects the state X₁ to X,,
x, A,X,
(31)
Therefore, by the Chain Rule of calculus,
dx:
d.A.
A. dxi
X1
(32)
dl
dl
dll
Substituting the ODE in Eqn.
30
Eqn.
32
dx:
d.A.
X, + A, F,x: G.G. Vx, (x1,
2 w. G.G.G Vx, logp(y|x:)
(33)
dt
dt
= dA,
1
+ A,F, x: -
1 A.G.G. so (x
A.G.G Vx, logp(y/xx)
(34)
dt
Since, we have 80(X,, t) = Comi(t) f), the above equation can be simplified as,
dx. dAA
A,F, Xt A.G.G. Com (x.. -,, A.A.G.G. Vx. logogg(())
(35)
dt
17

---

Further parameterizing,
dA, dt
+ A,F, A.B.
(36)
d A,G,G, Cour
(37)
dt 2
which yields the required diffusion ODE in the projected space:
dx = A,B,A, & + :&e (xx,t) 2 ur W, Vx, log ply p(ylxid
(38)
We have the following approximation for the conditional score Vx, log (xgp(x (for the noiseless case By
Ixu
Vx, logp(y(xx) r1
H H'HH
" H
(39)
Ox
0xo
T
.t 0x
(H (HHH H H THH H
(40)
2 0%0
T
(Hly PXn)
(41)
Uxi
where H -H and P H ( H represent the pseudoinverse and the or- thogonal projector operators for the degradation operator H. Substituting this form of the conditional score in projected diffusion dynamics, we have.
Were A,G,G, Ox. (H'y P
O%0
d& A,B,A,' &,dt + de (x1,t)
(42)
2
This concludes the proof.
A.3 Proof of Proposition
We restate the theorem here for convenience.
Proposition. For noiseless linear inverse problem with By = O and the conditional score approxi-
mated using Eqn. [ introducing the transformation X, A,x, also induces the following projected diffusion dynamics.
dx, A,B,A 8x ddb, dd + dD, 0x () (
(43)
A, = exp 1// B. - -F,F+ wor=²
G.G. P)ds]
wor G H
d
(44)
2/12
214
worr Coue
wer, 200
dΦ = 2 A.G.G. Ia-
A
dΦ;
A,G,G, Cow (45)
2/4
where exp(.) denotes the matrix exponential, H and P are the pseudoinverse and projector operators (as defined previously).
Proof. The proof consists two parts. Firstly, we simplify the conditional score Vx, log pxy().)
Secondly. we plug the simplified form of the conditional score into the conditional diffusion dynamics and develop conjugate integrators.
Exploiting the linearity in Vx, logp(y|>::) From the definition the score Vx, log (ogg(yy(.))
8%,
Xx, logg(p//xx))
' (2 H
(46)
Ox.
where Xy is the Tweedie's estimate of E(x1|xx) given by:
1
Xo
- (xe + of (xi,
(47)
Mt
18

---

where pe, on are the mean coefficient and standard deviation of the perturbation kernel p(x)(x1) = N of a), respectively, and Σ = (HHH is the variance of the ITGDM approximation of ply xx) (for the noiseless case i.c. dg = 0). Therefore,
ORA 0x.
Xx, logp(y(>x)
H E (y- H
(48)
1
(Ia oi 2
Vx, () Ex 2y (y-Hix) H
(49)
Pt.
(xx(x).))
H -; 1 (y - H8o) + So 1)H " 'y
(y- - HKoo]
(50)
Pr.
1 Pt.
1
H Σ (y-
H(x (x t))) 50 H)H m, (y H
(51)
1 k
И Σ y
H Σ Hx
(52)
Ht
Lincar Terms
Σ HHHHo ()) + o se
H 1
(x,,
1) E (v- H
(53)
Non-Lincar Torms
where So denotes the second-order derivative of the score function sp (x,, t). Therefore, the condi- tional score Vx, p(y) xx). can be decomposed into a combination of linear and non-linear terms. Next, we use this decomposition to design conjugate integrators for noiseless linear inverse problems.
Conjugate Integrator Design: From Eqn. specified as:
30
the conditional reverse diffusion dynamics can be
dx: F,x,
1 G.G. so (x(,,))
1
2 w. G. Vx, x: logp(y(>))
(54)
dt
2
Plugging in the form of the conditional score in Eqn. 53 in the above equation, we have,
dx. dt
Fix:
G.G. (x
2 w, G.G.G. Vx, (og(())x;)
(55)
2
1
1 H Σ, H E y E.'y
= Fix
G.G.
t)
G -/
Hxi
(56)
2
M4
of
H E H800(4 1)) + of
So 1) H (y Hxn)
(57)
#t
Fr + Wt
G.G. H Σ H X 2/1t
Wt
G.G.G T H Σ y
(58)
241
: H nn
w G.G.T So(Xii)
G.G.
la
H
so(x
T2, (gy
"
H
2prr
(59)
Given an affine transformation which projects the state x, to X..
x, Aix,
(60)
the projected diffusion dynamics can be specified as:
dx,
dt dA, + A.
+ Wt 2n²
G.G.
H Σ, H) X1
Wi A.G.G.GHHH
(61)
dt
214
A.G.G. Ia
wio 1
A. Σ H 80(x),
wio A.G.G.T (o((), Σ, (g- H&i)
2/41
(62)
Furthermore, the score network is parameterized as s)(xi,t) Cou(t))e(((, t). Consequently, So (xx, t) (om(()) t). Lastly, we reparameterize E. r where E = -HH Plugging
19

---

these parameterizations in the projected diffusion dynamics, we have,
dx.
dAA
+
(F
H H X W 20
G.G.
were
A.G,G, H'
(63)
dt
dt
2/xx
of
A.G.G.
La
AH H (x)
(64)
1
Wor, 2 o0, A.G.G. Coultt)
H
(65)
2/11
We then parameterize.
dA.
A Fi +
W
G,G, H E H) A,B.
(66)
dt
2p²
This implies,
A, = exp [// B. B w G.G.
_ P ds
(67)
2nd
where exp(.) denotes the matrix exponential. Furthermore, we parameterize,
Dy = Jo !/" "or'"" A.G.G, H'
(68)
2ps
/ (,' A, G.G.G. Ia-
wor
4,
A. Coul
(69)
2
L.
W Wwr 20?
i --=-
A,G, (s
(70)
2p
With this paramcterization, the projected diffusion dynamics can be compactly specified as follows:
dx, A, A,B,A, + ddb, y dd, co 1) d [ (H PXu)
(71)
This concludes the proof.
A.4 Simplification in Eqn. 11
We restate the result for convenience. The matrix exponential Eqn. 10
A. exp [ (* (.)
2 2 SL d, P
3.))o. ds Ia
(72)
can be simplified as,
K2 = (1 (o
A = exp(k:) La (exp(x²)
1)P
K2 2 I
3,ds
(73)
where P is the orthogonal projector corresponding to the degradation operator H.
Proof. We have,
A exp( La + xxP) = exp(x; Ia) exp(
(74)
The above result follows since I,P PIa (commutative under multiplication). Moreover, we can further simplify the matrix exponential P as follows.
(nt))P' p
exp(n = Σ
(75)
i!
1=0
20
(N 1
p
8 (n2)
Ia
Ia
P
(76)
i!
The above result follows from the property of orthogonal projectors p2 P. Therefore, 20
(k?)
00
(x
exp(n<P) = Ia+
P
I +
P
(77)
i!
i!
exp(x = Ia +
eex(xx?) - 1 P
(78)
which concludes the proof.
20

---

A.5 Proof of Proposition for Solving Noisy Inverse Problems
We restate the result here for convenience. Proposition. For the noisy inverse problem,
y Hxu + Oyz, z Z~~(0, Ia).
(79)
given the transformation A. for the noiseless case as defined in Equ. 17 the corresponding noisy transformation can be approximated as,
A, A. + (k((t))((HH)) 0(og))
(80)
(t wo ( p.
(2 exp ( -
(81)
Consequently, the inverse of the transformation A can be approximated as,
(A ~ A,- KS (t HHHH) A, 0(()))
(82)
Proof. We have,
o A exp 1 (A+23.)0 ( (HH
7 1a) -a)
(83)
From perturbation analysis, introduce the following first-order approximation,
H (H la H H
(HH))-
HH - H +
(84)
(HH
(85)
~ P y H (T))
(86)
Substituting this approximation in the expression for A." (and ignoring terms in O(or)),
A," exp [ (*+
2 33.) dsIa - w
1
_ [ff Bg
o² 2
H
HiHH)
ds))
(87)
exp //' ()
2 3r
1
w ds Ia
2 (ʃ Bods)
P+
wo
2
y
3r
ds
H
H )T]
(88)
2
r
(t)
=Rg(t)
woy (/ H
= exp kn (1) (1)
(89)
r?
From the definition of the matrix exponential, it can be shown that the A in Eqn. 89 approximated as:
can be
A = exp [ + wo
Ba ds exp (kI(t)= + ne(t)) -1 H H (H²) T + O(o9))
2
r?
(ka(1)
(90)
Ignoring the higher-order terms, we have,
A A. + (HI))
(91)
Consequently, we can also approximate the inverse of A as follows,
(A.")
[A, + kg H -1-1
(92)
~ A (
A +06,))
(93)
which concludes the proof.
21

---

B Conditional Conjugate Integrators: Flows
B.1
Background
This section discusses conditional conjugate integrators in the context of flows. For brevity, we skip deriving our results for flows since the derivations can be similar the analysis of diffusion models with minor parameterization changes, Recall that the conditional dynamics for flows are specified as follows [Pokle et al. 2024]
b(x,, se (x(, + we nrire Sear) Vx. log p(y(p)))
(94)
Ott.
where (x,,; represents the pre-trained velocity field for flow. Moreover, we restate the form of the conditional score Vx, log ((y(x( for convenience.
0% T
Xx, (ogp(y/
HT + Ta 'g H&1)
(95)
0xc
where & represents the Tweedie's estimate of the first moment of E(xe|xi).
E x
x
x + 72 12
(96)
02
where s(x,, t) represents the score function associated with the marginal distribution p(x.). It can be shown that x can also be expressed in terms the pre-trained velocity field by (Xr, () as follows,
7ext + n.o (xe: t)
1 ?pd t404
(97)
B.2 Conditional Conjugate Integrators for Flows
Analogous to diffusion models, design conditional conjugate samplers for flows that treat the conditional score xx, log p(y x,) as black box. Similar to Proposition [ by introducing the transformation A,X,, we have the projected flow dynamics,
0x1
T (H PX2)dl
d&, A,B,A, &idl + ddb (xe:1) 1) + were
(98)
Ox,
A = exp Bads)
p 1-¹ A,ds,
(99)
where H H and P= H (HH represent the pseudoinverse and the orthogonal projector operators for the degradation operator H. For B, 0, the formulation in Eqn. 98 becomes equivalent to the IIGDM formulation proposed for OT-flows in Pokle et al. [2024]. For simplicity, since in this work, we only explore the parameterization Eqn. 98 for B, 0, we refer to this parameterization as IIGFM.
B.2.1 Conjugate-TIGFM (C-IIGFM)
Analogous to the discussion of C-IGGM samplers Section 2.2
More specifically, given
noiseless linear inverse problem with σy = 0, and the conditional score Vx, logp(y|Xx, xx), introducing the transformation x; A,x,, where
A, exp { B.
Wor 2
Pds
(100)
2ar Jrdre - -teck)
induces the following projected flow dynamics.
dx, A,B.A,'x,l d y + dD + d Oxx bu,
PXI)
(101)
where,
WeT 270 A. ds
T
(102)
22

---

Algorithm 2 Conjugate IIGFM sampling
1: Input: Corrupted observation Corruption operator H, Pretrained Flow ba(..)). Choice of B,, NFE
budget N, Timestep discretization {ts fx=o, Flow kernel X, 0x.1 + Tye, Start time Output: Clean sample X
Pre- Pre-Compte (At, J (Eqn. 100
Pre-compute coefficients
4: Pre-Compute ( $.. + Ji=o (see Eqns. 102
104)
5: NOO, Id)
Draw initial samples from the generative prior
6: X a, HH' + 7
> Initialize using the pseudoinverse (See Chung et al. [2022b] )
7: X A-X
Initial Projection Step
8:
for 110 N 1 do
9:
h (ta+1 tn)
Time step differential
10:
X A x
11:
X1 Ti - 1
1xx (xbo(x,
Tweedie's Estimate
12:
hAn, Bi, A x ($2 $b
Linear drift
13:
vnt (
D6 bo (.) + ( 1 0xbx - P&))
> Non-Linear drift
14:
x= x + Ui + Vnl
Di Buler Update
15:
end for
return X A
Project back to original space when done
Ia Wor, 27290
$
Ag
P
ds
(103)
as(%
kssss)
$ Wst, 2n2
A,ds
(104)
ag
where exp(.) denotes the matrix exponential, H and P are the pseudoinverse and projector operators (as defined previously). Lastly, the matrix B, is a design choice of our method. We specify recipe for C-IIGFM sampling in Algorithm
C Extension to Noisy and Non-linear Inverse Problems
Here, we discuss extension of Conditional Conjugate Integrators to noisy and non-linear inverse problems. While our discussion is primarily in the context of diffusion models, similar theoretical arguments also apply to Flows.
Noisy Linear Inverse Problems: For noisy linear inverse problems of the form,
y Hxg + dyZ,
(105)
for VPSDE diffusion, the noisy transformation A can be approximated from the transformation A, for the noiseless case (i.e., By 0) as illustrated in the following result (Proof in Appendix A.5):
A." A, Kz (t)
+ +O(O)) (H*)
A. K3a(t) (H
(106)
fo
(f ds exp (t)(t) k²)()) i]
B.
(107)
2
r?
Consequently, the inverse projection analysis.
(A -1 can be approximated from A from perturbation
(A²-)-' R A n2 (1) ( +
(108)
Therefore, the transformation matrix A and its inverse (see Appendix A.5) can also be computed tractably for the noisy case. Since, for most practical purposes, By is pretty small, higher order terms
of can safely ignored, making our approximation accurate. We include qualitative examples for 4x super-resolution with oy 0.05 for the ImageNet dataset in Figure
23

---

Non-Linear Inverse Problems: For non-linear inverse problems of the form,
y (xn)) uy 2~ (0, Ia),
(109)
similar Song et al. 2022 we heuristically re-define linear operations like H'x, Hx, and Px, by their non-linear equivalents h (x+), h(xx) and h (h(x+)) respectively. Consequently, analogous to Eqn. [T the projection operator for a noiseless non-linear inverse problem, in this case, can be defined as,
A, = exp((x1 Ia+ (exp((2())) 1) (1(t) L (A+ (A+ (22()) = 2 / Byds,
(110)
where p= h (h()) non-linear "projector" operator. For instance, in non-linear inverse problems like compression artifact removal, h(x,) and hf (xx) can realized by encoders and decoders. We illustrate some qualitative examples in Figure [2] It is worth noting that this a purely heuristic approximation, and developing our framework remains an interesting direction for further work.
principled framework for non-linear inverse problems within
D Implementation Details
In this section, we include additional practical implementation details for both C-IIGDM and C- IIGMM formulations.
D.1 C-IGGDM: Practical Aspects
D.1.1 VP-SDE
We work with the VP-SDE diffusion [Sngg et al. 2020] with the forward process specified as:
dx, 3,x, dt + BB, dw,,
E [0,T].
(111)
This implies, F. -2 and G. VB.. For the VP-SDE the perturbation kernel is given by,
p(x+|xo)= N(uxx of Ia)
(112)
He exp (- 2
1 L B,ds)
of of= exp (-
f( (ads)
(113)
The corresponding deterministic reverse process parameterized as:
Bt x + (x t)] dt.
dx,
(114)
Moreover, adopt the standard c-prediction parameterization which implies Coul (t) - 1/0r. Lastly, the Tweedies estimate & can be specified as:
1 P4
Xo
Xx + 00
(115)
D.1.2 C-IIGDM Simplified Expressions
We choose the parameterization B, Ala and set the adaptive guidance weight w, wpirr?
?
where r = d?tp? The projected diffusion dynamics are then specified as:
dx; dx,dt + d
d y + d (xx, t) + +d co (H' Pxt)
(116)
where
1 3. dsl. 12 ( Bods) Bods) pp A, - exp 1// (A+
w
(117)
2
which further simplifies to.
x1(t) = / (*+ B.) ds,
w f
A, (xp(k)(()) (cxpp(()))))))
N2(t) = 2 3.ds
(118)

---

Moreover, we have,
WeTs
Φ y
A, G.G.G. Hids
(119)
2/1s
WBrks
A. H
ds
(120)
Worls exp(xi(s)) La (exp(xa(s)) - l)p| Hiids
(121)
WBI exp(Xi(()) H (exp((x(())) - 1) ds
(122)
WilHl exp(xs (8)) H (exp(<)g())) - H ds
(123)
2
-1//
WBrMe (x)( + ks)))ds H
(124)
2
1
Ware20' (om(s)ds
2,
A G.G.G,
Ia
(125)
2
m3
J0
B1
A.
A.(To
[o
Wo ds
(126)
20.
Bg
A,ds - [1/² "w..00
1'12200 exp((x(()) + k2(s)) P
(127)
0
20,
Bx
exp(15 (8)) la (exp(x2(s)) - 1) ds -1/² wB
exp(ni(s) + k2(2s))
20.
2
(128)
Bx
(xp(x((1))dd + +I[
Bx (xp((())))) -1) w3,05, 20.
exp(1i + x2(s))
20,
2
(129)
Wors -02
A,G.G, (dd(s) ("' "BB.p100s A.ds
4
(130)
2ps
2
exp(ni (s)) la + (exp(~2(6))- 11 P)
(131)
w.SHH.o. exp(m (s)) I wBssllos
exp(x (s)sxp((ass)) - 1)ds p (132)
2
2
D.2
C-TGGF:: Practical Aspects
D.2.1
OT-Flows
We work with OT-Flows Albergo et al. 2023 Lipman et al. 2023 Liu et al. 20223 due to its wide adoption. More specifically, the corresponding interpolant can be specified as,
X t)y + (x1, (O Ia) Xi
Pdota
(133)
For this case a4 and 7/ 1 t. Therefore, the Tweedie's estimate of E(x.(x)) can be specified as (from Eqn. 97p
* x, (1 (x,,(
(134)
D.2.2 C-IGFM: Simplified Expressions
We choose the parameterization B, Ala and set the adaptive guidance weight as wr = wafr?
where r The projected diffusion dynamics then specified as follows:
dx, Xx,di d y + (o0,bo((x 1) + d [Ox, bo (xx ()H PX1
(135)
25

---

where,
A. - exp [1/ Ala + wt(1 t)
2 Pds
(136)
which further simplifies to,
A, = exp(11 Ia + (exp(1<2(t)) - 1)p]. (1(t) = S." Ads, k2(t)= = w //- s(1 - s)ds
J
(137)
Moreover, we have.
Woi 7o9s
By
A, H'ds
(138)
0xs
was A, H ds = /-" ws((1 - ,s,,H
(139)
/ ws(1 5) exp(<i(s)) La (exp(r2(s)) 1)P H'ds
(140)
[/ ws(1 8) exp(xi(() k2()))ds H
(141)
A, La + W.r, 2737
Dbb
ds
(142)
0
as(7'd, a,)
Ag Ia + w0.767 P ds
(143)
(Tods k)0,s)
As La w8(1 - 8)² 2 ds
(144)
A,ds - S' ws((1 - =))² A,P ds
(145)
A,ds 11/" (us(1 5)² exp(ni(s) + rs(s))
(146)
exp(xi(s))ds 1//² exp(s(()) 1) ws((1 5)2 eep((x((s) + (2(s))
P
(147)
W. Wor 2n
A,ds = [' wook2224 A2A,ds
(148)
ws(1 - 5)² exp((x (s)) La + (exp(R2(s)) - 1)p ds
(149)
ws(1 8)8 (s)) [f""
ws( 5)² exp(nı (ex)(<)()))))) 1)ds P (150)
D.3 Coefficient Computation
From the above analysis, most integrals are one-dimensional and can be computed in closed form or numerically with high precision. To clarify, with a predetermined timestep schedule {t:), the coefficients O can be calculated offline just once and then reused across various samples. Therefore, this computation must only be done once offline for each sampling run. For numerical approximation of these integrals, we use the odeint method from torchdiffeg package [Chen] 2018 with parameters atol=le-5, rtol=le-5 and the RK45 solver [Dormand and Prince 1980 We set the initial value 4imi for all coefficients as an initial condition for both C-IIGDM and C-TIGFM samplers.
26

---

D.4 Choice of Numerical Solver
We use the Euler method to simulate projected diffusion/flow dynamics for simplicity. However, using higher-order numerical solvers within framework is also possible. We leave this exploration to future work.
D.5
Timestep Selection during Sampling
We use uniform spacing for timestep discretization during sampling. We hypothesize our sampler can also benefit from more advanced timestep discretization rechniques/aaran [0222] commonly
used for sampling in unconditional diffusion models in the low NFE regime.
D.6 Last-Step Denoising
It is common to add Euler-based denoising step from cutoff to optimize for sample quality [Song et al. 2020 Dockhorn et al. 2022 Jolicoeur Marttnnea et al. [021] at the expense of another sampling step. In this work, do not last-step denoising for our samplers.
D.7 Evaluation Metrics
We use the network function evaluations (NFE) to assess sampling efficiency and perceptual metries KID [Binkowski et al. 2018). LPIPS [Zhang et al. 2018 and FID [Heusel et al. [017]] to assess sam- ple quality. In practice we use the torch-fidel Siteelttyy Obukhov et al. 2020 package for computing all FID and KID scores reported in this work. For LPIPS, we use the torchmetrics package with Alexnet embedding.
D.8 Baseline Hyperparameters
Diffusion Baselines: For DPS (Chung et al. 2022a we set NFE=1000 and set the step size for
each task to the value recommended in Appendix D in Chung al. [2022a] For DDRM (Kaaar
et al. 2022], we set the number of sampling steps to NFE=20 with parameters recommended in Kawar al. [2022]. For both DPS and DDRM we start diffusion sampling from / = T. For our implementation of II-GDM, we set the start time parameter to 0.6 for super-resolution and deblurring. We set the guidance weight w; wr? where is tuned using grid search between 1.0 and 10.0 for best sample quality for super-resolution and deblurring. For implementation all diffusion-baser baselines, we use the official code for RED-Diff (Mardani et al. 2023 at https: //tp//ggittt//tt www.ccomm////eeeeeee /at////Vaaaabas/ RREDddiff..
1.0 and 0.85
Flow Baselines: In developing our flow-based baseline, we adhere to the approach outlined in IIGMM (Pokle et al., 2024), which advocates for a consistent guidance schedule characterized
by w, and r. 77t For each task, perform comprehensive grid search the parameters a, = 0.. different datasets identify the optimal configuration that minimizes the LPIPS score. For the
0.7} and w {1, , 5} (35 combinations in total) across
,,
implementation of Flows, we use the official implementation of Rectified Flows L et al. 20223 at com/ coo//gobbtaab.oooooooooooo Retp/iffeiif...oo....
https://ght
thub. //tt:/b..ttttt...//
E
Additional Results
E.1
Additional Baseline Comparisons
We include additional comparisons between our proposed samplers and competing baselines on the AFHQ-Cat (see Table 2) LSUN Bedroom (see Table 3), and the FFHQ (see Table 4 datasets.
note on Inpainting evaluations for ImageNet. We find that for diffusion model evaluations, continuous sampler for ITGDM suffers from noisy artifacts for the inpainting task. Consequently, Conjugate IIGDM suffers from similar artifacts. Therefore, we do not report results on this task for the ImageNet dataset.
27

---

LPIPSI
XIDx10 $1
FID.
Task
NFE
C-ITGFM
IGGEM
C-IIGFM
TIGMM
C-IGGFM
TGGFM
5
0.151
0.177
6.5
15.6
21.76
30.82
Inpainting
10
0.122
0.136
8.5
9.4
22.50
24.87
20
0.115
0.117
6.4
10.4
20.39
24.42
0.129
0.133
4.1
18.43
19.55
Super-Resolutin
10
0.132
0.121
4.0
4.6
18.32
17.65
20
0.134
0.119
4.5
4.0
18.75
16.97
5
0.176
0.177
6.6
6.9
23.28
23.66
Deblurring
10
0.182
0.164
9.4
7.1
28.12
23.62
20
0.191
0.170
12.4
7.2
31.76
23.65
Table Quantitative evaluation on superresolution, inpainting, and Gaussian deblurring on the AFHQ-Cat dataset.
LPIPS)
10
FID
Task
NFE
C-IIGFM
IIGFM
C-IIGFM
IIGEM
C-NIGFM
IIGFM
5
0.208
7.0
45.66
Inpainting
10
0.176
4.4
40.69
20
0.167
4.2
40.35
5
0.174
0.219
3.1
7.7
37.54
46.03
Super-Resolution
0.150
0.193
1.1
32.41
37.34
20
0.148
0.175
0.9
2.5
32,26
32.15
5
0.209
0.220
5.0
44.78
49.27
Deblurring
10
0,204
0.193
10.7
4.7
53.53
44.21
20
0.224
0.175
18.0
3.5
62.87
39.95
Table 3: Quantitative evaluation 4x superresolution, inpainting, and Gaussian deblurring the LSUN-Bar BeNN-Bdooom dataset. We note that IIGFM fails to generate reasonable texture in the masked region with the maximum NFE=20, choose not to report the results here. (See qualitative examples in Figure
E.2 Comparison of Perceptual vs Recovery Metries
Here, we highlight the robustness of C-IIGDM in both perceptual and recovery metrics the context of inverse problems. For completeness, we provide comparison between DPS, IIGDM, and C- IIGDM in terms of PSNR, SSIM, FID, and LPIPS in Tables and [ on the ImageNet-256 and FFHQ-256 datasets on the 4x super-resolution task. It is worth noting that the PSNR and SSIM scores for all methods correspond with the best FID/L PIPS scores presented in the main text for these methods. Our method achieves competitive PSNR and SSIM scores for better perceptual quality than competing baselines like DPS/II-GDM, even for very small sampling budgets. For instance, on the FFHQ dataset, method achieves PSNR of 28.97 compared to 28.49 for DPS while achicving better perceptual sample quality (LPIPS: 0.095 for ours vs 0.107 for DPS) and requiring around 200 times less sampling budget (NFE=5 for method vs 1000 for DPS). Therefore, argue that perceptual quality to recovery trade-off is better than competing baselines.
E.3 Traversing the Recovery vs Perceptual trade-off
In addition the guidance weight w, our method also allows tuning additional hyperparameter which controls the dynamics of the projection operator (See Sections 2.3 and 3.2 for more intuition). Therefore, tuning w and À can help traverse the trade-off curve between perceptual quality and distortion for a fixed NFE budget. We illustrate this aspect in Table 7 (fixed À with varying w) and Table (fixed w with varying A) for the SR(x4) task on the ImageNet-22 LPIPS, and FID metrics. Therefore, our method offers greater flexibility to tune the sampling process towards either good perceptual quality or good recovery for a given application while maintaining the same number of sampling steps. In contrast, other methods like DPS II-GDM do not offer such
dataset using the PSNR,
28

---

Task
NFF
LPIPS
KIDXI0
FID
C-IIGDM
IIDDM
DDRM
C-IIGDM
HIGDM DPS DDRM C-IGGDM HIGDM DPS DDRM
0.095
0.133
10,9
17.4
32.01
41.39
Super Resolution
10
0.086
0.106
0.106
0.106
8.8
10.2
7.8
22.8
29.07
32.79
30.86
36.95
0.083
0.087
5.8
26.37
26.17
0.127
0.147
7.3
14,6
31.18
39.63
Deblurring
0.111
0.123
0.348
0.132
1094
11.5
29.08
31.49
142.26
33,94
0.112
0.103
4,4
3.1
27.68
26.30
Table 4: Quantitative evaluation on 4x superresolution and Gaussian Deblurring tasks for the FFHQ dataset. DPS was evaluated with NFE=1000 but failed to perform well on the deblurring task. DDRM was evaluated with NFE=20.
PSNR
SSIM f
FID +
LPIPS +
DPS NNFE=1000)
23.81
0.708
38.18
0.252
IIGDM (NFE=20)
21.92
0.646
37.36
0.222
C-LIGDM (NFE=5)
22.32
0.641
37.31
0.220
C-IGGDM (NFF=10)
23.00
0.651
34.22
0,206
C-TIGDM (NFF=20)
23.16
0.654
34.28
0.207
Table 5: Comparison between C-IIGDM and other baselines in terms of the Recovery (a.k.a distortion) Perception tradeoff for ImageNet-256 dataset for the SR(x4) task.
flexibility. Moreover, tuning the guidance weight in methods like DPS could be very expensive due to its high sampling budget requirement (around 1000 NFE).
E.4 Qualitative Results
Diffusion Models:
1. We include additional qualitative comparisons between II-GDM and our proposed C-ITGDM sampler for the ImageNet dataset in Fig. 4
2. We include a qualitative comparison between sample quality at different sampling budgets for the C-1IGDM sampler 5
3. We qualitatively study the impact of varying w on sample quality in Fig. 6 and the impact of varying À on sample quality in Fig. 7
4. We qualitatively present the performance the C-ITGDM sampler for noisy inverse prob- lems in Fig. 8 In just 5 steps, our method can also generate good-quality samples for noisy inverse problems.
Flow Models:
1. We include additional qualitative comparisons between II-GFM and our proposed C-IIGFM sampler with different sampling budget for the all three datasets in Fig. 9
2. We qualitatively study the impact of varying sample quality in Fig. 10 and the impact of varying À on sample quality in Fig. []
29

---

PSNR t
SSIM 1
FID 4
LPIPS 4
DPS (NFE=1000)
28.49
0.834
30.86
0.107
TIGDM (NFE=20)
28.26
0.818
26.17
0.087
C-LIGDM (NFE=5)
28.97
0.832
32.01
0.095
C-IIGDM (NFE=10)
29.03
0.821
29.07
0.086
C-IIGDM (NFE=20)
28.79
0.809
26.37
0.083
Table 6: Comparison between C-ITGDM and other baselines in terms of the Recovery (a.k.a distortion)
Perception tradeoff for FFHQ-250 dataset for the SR(x4) task.
PSNR 1
LPIPS
FID
22.91
0.339
48.48
23.37
0.306
45.03
23.49
0.274
42.68
23.44
0.266
40.96
10
23.28
0.254
40.27
12
22.89
0.246
40.13
14
22.74
0.239
40.16
Table 7: Illustration of the impact of w for fixed À = 0.0 on the sample recovery (PSNR) vs sample perceptual quality (LPIPS, FID) NFE=5 for our method. The task is SR(x4) on the ImageNet-256 dataset.
PSNR t
LPIPS +
FID 4
110
20.96
0.291
42.56
008
21.33
0.265
40.97
-0.6
21.69
0.240
39.38
-0.4
22.04
0.223
37.83
-0.2
22.32
0.220
37.31
0.2
22.73
0.257
45.27
0.4
22.90
0.275
48.98
0.6
23,03
0,283
47.47
0.8
23.11
0.285
46.2
1.0
23.15
0.285
46.41
Table 8: Illustration of the impact of for a fixed w: = 15.0 on the sample recovery (PSNR) sample
perceptual quality (LPIPS, FID) NFE=5 for our method. The task SR(x4) on the ImageNet-255 dataset.
30

---

Original
Pseudoinverse
IIGDM
C HIDMM
2222
Figure 4: Qualitative comparison between IIGDM and C-IGGDM NFE=5 for the ImageNet dataset on the 4x Superresolution task. C-IGGMM can generate high-frequency details even for a low compute budget compared to the baseline [[-GDM (Best Viewed when zoomed in)
31

---

Original
Pseudoinverse
NFE=5
NFE=10
NFE=20
Figure 5: Qualitative comparison for different sampling budgets for the ImageNet dataset on the 4x Superresolution task. C-IIGDM can generate high-quality samples in just 5 steps (Best Viewed when zoomed in)
32

---

Original
Pseudoinverse
W
w=10
w=15
Figure 6: Impact of varying C-TIGDM guidance weight on sample quality for the ImageNet dataset on the 4x Superresolution task. High guidance weight crucial to generate good quality samples from C-IIGDM (NFE=5 steps) (Best Viewed when zoomed in)
Original
Pseudoinverse
A == -1.0
x= -.2
==
Figure 7: Impact of varying C-IIGDM À on sample quality for the ImageNet dataset on the 4x Superresolution task. High can lead to blurry samples while a very low À can lead over- sharpened artifacts (NFE=5 steps) (Best Viewed when zoomed in)
33

---

Original
Pseudoinverse
C HGGM
Figure 8: C-IIGDM can also generate good quality samples for noisy inverse problems (4x superres
with NFE=5, dy = 0.05). For this case naively computing the pseudoinverse fails to get rid of the noise.
34

---

IIGFM
C-IIGFM
IIGFM
C-IIGFM
IIGFM
C-IIGFM
Reference
Distorted
NFE-5
NFE=10
NFE=20
Figure 9: Qualitative comparison between 1IGFM and C-IIGFM at NFE=(5, 10) for the datasets on 3 tasks. C-IGGF can generate high-frequency details even for a low compute budget as compared the baseline II-GFM (Best Viewed when zoomed in). We did not report IIGFM inpainting results in Table as it failed to generate "reasonable" textures even after extensive on and т.
search
AR
AR
Reference
Distorted
1 20
1
28
40
Figure 10: Impact of varying C-IIGFM guidance weight sample quality for the ImageNet dataset on the 4x Superresolution task. High guidance weight crucial generate good quality samples from C-IIGMM (NFE=5 steps) (Best Viewed when zoomed in)
35

---

Reference
Distorted
== 0.2
==1
Figure 11: Impact of varying C-IIGFM À on sample quality for the ImageNet dataset the 4x Superresolution task. High À can lead to blurry samples while a very high À can lead to over- sharpened artifacts (NFE=5 steps) (Best Viewed when zoomed in)
Reference
Neual Compression
NFE=5
NFE=10
NFE=20
Reference
JPEG 9=5
NFE=5
NFE=10
NFE=20
Figure 12: C-IGGM for solving compression inverse problem. Top: decoding compressed latents from pretrained mean-scale hyperprior neural codec [Minnen al. 2018]; Bottom: JPEG image restoration.
36

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Abstract, Section Section 2 and Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Section 5
37

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't ' acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: See Section
2 and Appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
See the beginning of Section 3
Guidelines:
The answer NA means that the paper does not include experiments.
38

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code is not available at the time of submission, but will be published later.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://niis.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ /ttppp.cc/pbllc/guudeeee http://oeesuiiiiinoooooooooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
39

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See the beginning of Section 3
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
We don't have any error bar data our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
show in tables
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification: Our approach utilizes established models for range of downstream tasks, ensuring that hardware variations do not affect the outcomes.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
40

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: the research conducted in the paper conform with the NeurIPS Code of Ethies
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See Section
S
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: Our approach utilizes published models for a range of downstream tasks, so do not need add any safeguard.
Guidelines:
41

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cited all the related works and packegs we used
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. www.coo/caaa hom/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pane se-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our approach utilizes published models for range of downstream tasks. Details are available in Section
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details the dataset/codeemooell as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
42

---

Justification:
We don't 't have experiment with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We don't 't have experiment with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
43


Reviews:
[{"review_id": "p1WT2jvjSE", "reviewer": "Reviewer_hLds", "summary": "This work proposes a plug-and-play based method that leverages pretrained diffusion and flow models to solve inverse problems. The proposed method called Conditional Conjugate Integrators adapts previously proposed Conjugate Integrators framework for fast sampling of diffusion and flow models to solve linear inverse problems. The key idea is to project the diffusion (flow) dynamics into another latent space where sampling can be more efficient. Upon completion, the dynamics is projected back to the original pixel space. The paper provides the mathematical forms of the projection operator for conditional diffusion (flow) dynamics, then adapts it to get the projection operator for linear inverse problems. The paper also provides tractable forms of the projection matrix and its inverse. This derivation can be seen as a more general form of previously proposed method $\\Pi$GDM ($Pi$GFM). The method can be extended to nonlinear and noisy settings. The paper provides promising results on datasets like LSUN Bedrooms, AFHQ Cats, FFHQ, ImageNet etc. on tasks like inpainting, superresolution, gaussian deblurring etc.", "questions": "Questions:\n1. Why does Eq. 117 hold true for VPSDE? I’m missing the simplification step here. \n2. Why does Table 1 not include results of inpainting for diffusion models on ImageNet? Similarly, Table 4 in the appendix skips results for inpainting on FFHQ.\n\nSuggestions:\n1. All the tables and figures must explicitly state all the relevant settings of the inverse problems. It is not immediately apparent that some of these quantitative results are only for noiseless linear inverse problems. \n2. Consider including an algorithmic box that summarizes C-$\\Pi$GFM and  C-$\\Pi$GDM. This would provide a concise overview of the method to the readers.", "limitations": "The paper discusses relevant limitations.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Methodology: The proposed method seems efficient — it can solve linear inverse problems in 5 steps. In comparison, previous methods need 20-100 or even 1000 steps.  \n2. Experiments: The paper includes ablation studies on the choice of hyper-parameters and provides comparisons against former state-of-the-art methods like $\\Pi$GDM, DPS, DDRM,$\\Pi$GFM etc.  The proposed method out-performs previous methods by a significant margin while using 5-10 steps and is on par with or better than previous method for >=20 steps.  \n3. Writing: Paper is well written. The core ideas and the methodology have been presented well, and derivations are easy to follow.", "weaknesses": "The benefits of the proposed method for the settings of noisy linear inverse problem setting as well as non-linear inverse problems remain unclear. The paper does not include any quantitative results for these two problem settings.  It only provides some limited qualitative results for super-resolution for the noisy setting with $\\sigma_y$=0.05 and compression inverse problem and JPEG restoration problem for non-linear settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "$\\newcommand\\dag\\dagger$\nWe thank the reviewer for their insightful comments and feedback. We provide a detailed response to several concerns below:\n\n>The benefits of the proposed method for the settings of noisy linear inverse problem setting as well as non-linear inverse problems remain unclear. The paper does not include any quantitative results for these two problem settings. It only provides some limited qualitative results for super-resolution for the noisy setting with \n=0.05 and compression inverse problem and JPEG restoration problem for non-linear settings.\n\nAs illustrated in Figures 7 and 11 in the main text, we would like to highlight that our proposed method can be applied to challenging noisy and non-linear inverse problems in as few as 5-10 sampling steps. Therefore, the proposed method can also be beneficial for these settings. However, we acknowledge that our qualitative results for noisy and nonlinear problems serve as a proof of concept and that further evaluation and refinements are necessary. We intend to pursue more detailed investigations and improvements in future work and will highlight the same in the Conclusion section of a subsequent revision. We would also like to point the reviewer to our common response (Point 1) for a more comprehensive justification of the utility of our method for noisy and non-linear inverse problems.\n\n>Why does Eq. 117 hold true for VPSDE? I’m missing the simplification step here.\n\nFor a given degradation operator $H$, the core idea in this step is the property of the projection matrix $P=H^\\top(HH^\\top)^{-1}H$, which shows:\n$$PH^\\dag = \\big[H^\\top(HH^\\top)^{-1}H\\big]\\big[H^\\top(HH^\\top)^{-1}\\big] = H^\\top(HH^\\top)^{-1} = H^\\dag$$\nNow, Eqn. 116 reads as:\n$$\\Phi_y = -\\int_0^t \\frac{w\\beta_t\\mu_s}{2}\\exp(\\kappa_1(s))\\Big[H^\\dag + (\\exp(\\kappa_2(s)) - 1)PH^\\dag\\Big] ds$$\nReplacing $PH^\\dag = H^\\dag$ in the above equation, we get,\n\n\\begin{align}\n\\Phi_y &= -\\int_0^t \\frac{w\\beta_t\\mu_s}{2}\\exp(\\kappa_1(s))\\Big[H^\\dag + (\\exp(\\kappa_2(s)) - 1)H^\\dag\\Big] ds\\\\\\\\\n&= -\\int_0^t \\frac{w\\beta_t\\mu_s}{2}\\exp(\\kappa_1(s))\\exp(\\kappa_2(s))H^\\dag\\Big] ds \\\\\\\\\n&= -\\big[\\int_0^t \\frac{w\\beta_t\\mu_s}{2}\\exp(\\kappa_1(s) + \\kappa_2(s))ds\\big]H^\\dag \n\\end{align}\nTherefore, we arrive at Eqn. 117. We hope this resolves any confusion. We will also add these simplifying steps in our subsequent revision.\n\n> Why does Table 1 not include results of inpainting for diffusion models on ImageNet? Similarly, Table 4 in the appendix skips results for inpainting on FFHQ.\n\nWe thank the reviewer for pointing this out. We couldn't include these comparisons due to time constraints but will include them in a subsequent revision.\n\n> All the tables and figures must explicitly state all the relevant settings of the inverse problems. It is not immediately apparent that some of these quantitative results are only for noiseless linear inverse problems.\n\nWe thank the reviewer for pointing this out. We are planning to update the paper with more detailed captions and agree that we should mention the settings of the problems more explicitly in the figures and tables\n\n> Consider including an algorithmic box that summarizes C-$\\Pi$GFM and C-$\\Pi$GDM. This would provide a concise overview of the method to the readers.\n\nWe thank the reviewer for this suggestion. We plan to add a pseudocode/algorithm box summarizing the proposed samplers in the main text. We would also like to point the reviewer to our common response (Point 2) for more details regarding improved illustrations of our method.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uFlbBBRSMb", "reviewer": "Reviewer_PpUR", "summary": "Inverse problems like super-resolution and deblurring remain computationally inefficient with current diffusion/flow models. The paper introduces a plug-and-play framework with Conditional Conjugate Integrators that use pre-trained iterative refinement models to simplify sampling. The method generates high-quality samples in as few as 5 steps for tasks like 4× super-resolution on ImageNet, outperforming existing methods.", "questions": "1. Can the method be applied to nonlinear inverse problems? If not, please clarify the reason.\n2. Though the Conditional Conjugate Integrators work well and can speed up the sampling process to 5 steps, a more detailed computational comparison is needed as each iteration becomes more complicated. Can you provide more detailed computational results?\n3. Can you clarify the reason why you restrict yourself to the posterior approximation in $\\Pi$GDM? Though the experimental results are promising, a direct comparison with DPS is preferred.", "limitations": "Yes, the paper mentioned its broader impact at the end, which looks good to me.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written, and the literature review is thorough.\n2. The method is well-supported by theoretical foundations.\n3. The experiments on both diffusion and flow-matching methods across various tasks demonstrate the robustness of the proposed speed-up algorithm.", "weaknesses": "1. The reason why the paper is restricted to the $\\Pi$GDM paradigm is unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "$\\newcommand\\dag\\dagger$\nWe thank the reviewer for their feedback. We provide a detailed response to several concerns below:\n\n> The reason why the paper is restricted to the $\\Pi$GDM paradigm is unclear.\n\nWe chose to work with the $\\Pi$GDM baseline due to the following reasons:\n\n1. Firstly, $\\Pi$GDM is a very strong and widely-used baseline for tackling inverse problems, providing a more expressive posterior approximation $p(x_0|x_t)$ compared to other methods such as DPS or MCG. This makes it an excellent starting point for low NFE budget sampling. As demonstrated in our experiments (See Table 1), DPS typically requires 1000 NFEs to achieve optimal results. In contrast, $\\Pi$GDM serves as a competitive baseline and a strong foundation for our method. Additionally, $\\Pi$GDM offers greater flexibility since it does not require a differentiable image degradation transform, unlike DPS. Thus, PGDM is a natural choice for us to demonstrate the effectiveness of our method.\n\n2. Secondly, in the context of popular diffusions and flows, using the posterior approximation in $\\Pi$GDM results in a closed-form derivation of the conjugate operator and its inverse (See Eqns. 11 and 12 in the main text). Note that this is not a limitation of our method since the projection operator can also be derived for DPS, as highlighted in the following theoretical result (for brevity, we omit the full proof and would include it in our revision).\n\n**Proposition.** For a time-dependent design matrix $B: [0,1] \\rightarrow \\mathbb{R}^{d\\times d}$ and the posterior approximation proposed in DPS [Chung et al.] with guidance step size $\\rho$ such that the conditional score $\\nabla_{x_t} \\log p(y|x_t) = \\rho \\frac{\\partial \\hat{x}_0}{\\rho x_t}^\\top H^\\top (y - H\\hat{x}_0)$, introducing the transformation $\\hat{x}_t=A_tx_t$, where\n$$A_t = \\exp{\\Big[\\int_0^t B_s - \\Big(F_s + \\frac{\\rho}{2\\mu_s^2}G_s G_s^\\top H^\\top H\\Big)ds \\Big]}$$ induces the following projected diffusion dynamics\n$$d\\hat{x}_t = A_t B_t A_t^{-1}\\hat{x}_t dt + d \\Phi_y y + d \\Phi_s \\epsilon(x_t, t) + d \\Phi_j \\Big[\\partial\\epsilon(x_t, t)H^\\top (y - H\\hat{x}_0)\\Big] $$\nwhere $\\exp(.)$ denotes the matrix exponential, $\\hat{x}_0$ represents the Tweedie's moment estimate and $H$ denotes the degradation operator. The proof roughly follows similar ideas from Appendix A.3 in our paper and we will include a complete proof in our revision.\n\nDespite our method applying to DPS, it is worth noting that in contrast to $\\Pi$GDM, the projection operator $A_t$  and its inverse for DPS do not exhibit a closed-form solution and need to be approximated using perturbation analysis (if the step size $\\rho$ is small) or computed using standard routines in packages like PyTorch. Therefore, though our method is also applicable to DPS, we choose to stick with the $\\Pi$GDM framework in this work.\n\n> Can the method be applied to nonlinear inverse problems? If not, please clarify the reason.\n\nWhile our presentation of the proposed method is primarily in the context of linear-noiseless inverse problems, we also present an extension of our method to noisy-linear and non-linear inverse problems in Section 3.4. Furthermore, in Figure 11 of our paper, we illustrate the applicability of our method to non-linear inverse problems in the context of challenging problems like JPEG and lossy neural compression-based restoration. We would also like to point the reviewer to our common response (Point 1) for a more comprehensive justification of the utility of our method for noisy and non-linear inverse problems.\n\n> Though the Conditional Conjugate Integrators work well and can speed up the sampling process to 5 steps, a more detailed computational comparison is needed as each iteration becomes more complicated. Can you provide more detailed computational results?\n\nWe provide a computational comparison in Table 1 using NFE as the metric, which is standard in the existing literature on inverse problems like DPS [Chung et al.] and PiGDM [Song et al.]. It is important to note that the compared models utilize the same pre-trained backbone diffusion/flow models, ensuring no differences in model inference speed. The only additional computation incurred involves a simple linear transform $A_t$ and some **scalar** integral calculations, which are computationally inexpensive. Moreover, these integrals can be precomputed offline (i.e., before sampling starts, as only the noise schedule influences the integral results). Therefore, the computational cost can be amortized between samples, reducing potential overhead. We will make this point more explicit in our revised manuscript.\n\n> Can you clarify the reason why you restrict yourself to the posterior approximation in $\\Pi$GDM\n\nPlease see our response to your first comment.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mrjjO52NbE", "reviewer": "Reviewer_X2Z6", "summary": "This paper introduces a inverse problem approach to reconstruct low-resolution blurred or obfuscated images, from pre-trained generative models. The method leverages conjugate integrators to project the diffusion dynamics in an easier space, solve the inverse problem and then map back.", "questions": "Minor details:\n- Line 608, link the approximation to Eq. (5)", "limitations": "I don't have any concern.", "rating": 7, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The integration of Conjugate integrators for inverse problems in the paper is rigorously defined and proved.\n- The proposed method drastically reduces the number of sampling steps compared to other inverse problem approaches.\n- The presented approach achieve better reconstructed images compared with state-of-the-art inverse problem methods.", "weaknesses": "The paper is extremely dense with equations and mathematical formulations. I would suggest moving sections 2.3 and 3.4, as well as the numerical details of the experiments, to the appendix. This would allow space in the main paper for additional qualitative results and a broader discussion on the intuition behind the conjugate integrators for inverse problems. Including a figure illustrating the method would also be appreciated. While the mathematical formulation is the main essence of the paper, a visual representation or descriptive insight would be appreciated for the understanding.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback. We provide a detailed response to several concerns below:\n\n>  I would suggest moving sections 2.3 and 3.4, as well as the numerical details of the experiments, to the appendix.....\n\nWe thank the reviewer for their suggestion. We stress that the final version allows for an extra page, which we will use to make the paper more easily accessible. Re. Section 2.3, we would like to highlight that this section provides an intuitive understanding of our method's tractability and working. Therefore, we think this section serves as an intuitive premise for the observations presented in our experimental section.  Re. Section 3.4, we will consider moving this section to the appendix since our presented results for non-linear and noisy inverse problems serve as a proof of concept. Additionally, we plan to add an overview figure in the revised manuscript. We also provide a sample overview figure as Figure 1 in the rebuttal pdf (Also see the second point of our shared response).\n\n>  Line 608, link the approximation to Eq. (5)\n\nThanks for pointing it out. We will revise our manuscript accordingly", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their insightful comments and are glad that the reviewers found our paper well-written (Reviewers hLds, PpUR), well-supported by theoretical arguments (Reviewer PpUR), our proposed method efficient (Reviewer hLds), and rigorously defined (Reviewer X2Z6). Below we highlight some shared concerns that the reviewers have and address reviewer-specific concerns as individual responses. We also provide a rebuttal pdf with our response which illustrates the main figure we plan on including in our paper revision for better understanding of the proposed method.

**Shared comments**:

**Re. applicability to Noisy and Nonlinear Inverse Problems**:

Reviewers hLds and PpUR wished to see more details and results of our approach applied to noisy and nonlinear inverse problems. While we appreciate the suggestions and are willing to expand on them (see below), we stress that our primary focus in this work is on the *linear and noiseless case*, which captures a set of diverse and important inverse problems encountered in image restoration applications (e.g., super-resolution, deblurring, and inpainting). Similar to other works (e.g., on $\Pi$GDM), we presented the selected results on nonlinear and noisy setups as proof of concept.

In our paper, we discuss our proposed approach for addressing noisy and nonlinear inverse problems in Section 3.4, with some qualitative results presented in Figures 7 and 11. More specifically, for noisy linear inverse problems, our approximations for the noisy projection operator $A_t^{\sigma_y}$ are accurate to an order of $O(\sigma_y^4)$ [See Eq. 15 in the main text], where $\sigma_y$ is the noise level added to the output of the degradation operator. Therefore, unless the noise levels are very high, this implies that our method is applicable in most practical scenarios. Empirically, Figure 7 illustrates the validity of our theoretical arguments for the task of noisy super-resolution. Therefore, while we do not include quantitative comparisons on noisy inverse problems, our qualitative results serve as a good proof-of-concept of the generality of our proposed conditional sampling framework. We will add further results for noisy-linear inverse problems in our revised manuscript to highlight this aspect further. 

Next, for nonlinear problems, our approximations follow a heuristic approach that is similar to $\Pi$GDM (Song et al.) but can be empirically effective as demonstrated in the context of challenging non-linear inverse problems like JPEG and lossy neural compression-based restoration (see Figure 11) under a limited compute budget. Thus, our method offers promising avenues for speeding-up nonlinear inverse problems. However, we acknowledge that our qualitative results for noisy and nonlinear problems serve as a proof-of-concept, and that further evaluation and refinements are necessary. We intend to pursue more detailed investigations and improvements in future work.

**Re. Illustration of the proposed method**

Reviewers X2Z6 and hLds made nice suggestions about improving the illustration of our proposed method. As a follow-up, we present an intuitive visualization of our proposed sampling framework in Figure 1 in our rebuttal pdf. To summarize, as illustrated in the figure, given a starting time $t_s$, our method tries to map the conditional diffusion dynamics into a more well-conditioned space where sampling is more efficient. The hyperparameter \lambda controls the amenability of the projected space for faster sampling. Moreover, the projection operator itself is a function of the degradation operator and the guidance scale resulting in more robust sampling even at high guidance scales. We revert back to the original space after sampling concludes in the projected space. While we will continue to improve upon these visualizations, we hope that the accompanying figure can help resolve any confusion that the readers or the reviewers may have regarding our proposed approach. In subsequent revision, we will also include the algorithmic pseudocode of our method as suggested by Reviewer hLds for more clarity.

Meta-review: This paper presents a numerical sampling method for solving inverse problems using diffusion and flow models. The proposed technique is based on conjugate integrators which provide high-quality solutions to linear inverse problems in a few sampling steps. Overall the reviewers have acknowledged the efficacy of the method and its rigorous treatment of the problem and they have recommended accepting the paper.
Decision: Accept (poster)