FIG: Flow with Interpolant Guidance for Linear Inverse Problems
Authors: Yici Yan
Yichi Zhang
Xiangming Meng
Zhizhen Zhao
Venue: ICLR 2025 Poster (2025)
Keywords: Flow Matching
Diffusion Model
Linear Inverse Problems
Primary area: generative models
TL;DR: We propose Flow with Interpolant Guidance (FIG), where the reverse-time sampling is efficiently guided with measurement interpolants to solve linear inverse problems.

Abstract: Diffusion and flow matching models have recently been used to solve various linear inverse problems in image restoration, such as super-resolution and inpainting. Using a pre-trained diffusion or flow-matching model as a prior, most existing methods modify the reverse-time sampling process by incorporating the likelihood information from the measurement. However, they struggle in challenging scenarios, such as high measurement noise or severe ill-posedness. In this paper, we propose Flow with Interpolant Guidance (FIG), an algorithm where reverse-time sampling is efficiently guided with measurement interpolants through theoretically justified schemes. Experimentally, we demonstrate that FIG efficiently produces highly competitive results on a variety of linear image reconstruction tasks on natural image datasets, especially for challenging tasks. Our code is available at: https://riccizz.github.io/FIG/.

Published as a conference paper at ICLR 2025
FIG: FLOW WITH INTERPOLANT GUIDANCE FOR LIN- EAR INVERSE PROBLEMS
Yici Yan"", Yichi Zhang", Xiangming Meng', Zhizhen Zhao' 'Department of Electrical and Computer Engineering, University of Illinois at Urbana- Chaama-Champ DDepartment of Statistics, University of Illinois at Urbana-Chane -haaapaaa *ZJU-UIUC Institute, Zhejiang University
ABSTRACT
Diffusion and flow matching models have recently been used to solve various linear inverse problems in image restoration, such as super-resolution and inpainting. Using a pre-trained diffusion or flow-matching model as a prior, most existing methods modify the reverse-time sampling process by incorporating the likelihood information from the measurement. However, they struggle in challenging sce- narios, such as high measurement noise or severe ill-posedness. In this paper, we propose Flow with Interpolant Guidance (FIG), an algorithm where reverse-time sampling efficiently guided with measurement interpolants through theoretically justified schemes. Experimentally, we demonstrate that FIG efficiently produces highly competitive results on a variety of linear image reconstruction tasks on natural image datasets, especially for challenging tasks. Our code is available at: http://:e/.cccii..ittbubb...b.. www.hg..thhhhcoo
hattps://ricon
INTRODUCTION
Linear inverse problems have long been an active field of research in applied mathematics, statistics, and signal processing. In particular, linear inverse problems in image restoration have many real-world applications. A typical mathematical model for linear inverse problems reads as follows:
= Ax + n2
where y E IR" is the observed measurement, x= E. IR denotes the underlying true signal, A E Rad linear forward measurement operator, and n E IR" is the measurement noise independent of signal x". The goal of solving a linear inverse problem is to retrieve the underlying signal x" given the observed measurement. In real-world scenarios, the problem is often ill-posed, i.e., the underlying signal is high dimensional compared the number of observations. Formally, when n d, the solution to the inverse problem is not unique. From a Bayesian perspective, the prior is given by a data representation of the underlying true signal, and the inverse problem can be solved by calculating the posterior using the measurement likelihood (Idier 2013).
Recent developments in continuous normalizing flows have shown tremendous success in sampling from underlying complex high-dimensional distributions (Chen et al. 2018). Particularly, flow
matching models (Lipman et al. 2023 Liu et al. 20236 Albergo & Vanden-Eijnden 2023 Albergo
et al. 22023) Ma et al. (0224) together with diffusion models <Song et al.] [021b Ho et al. 2020
Song et al. [2221a] Rombach et al. 2022). emerge as powerful tools for image synthesis (Dhariwal
| The idea behind both models is to simulate process that gradually perturbs data until becomes random noise. Moving from data to noise, usually referred to as the forward process, can be represented by either a stochastic differential equation (SDE) or an ordinary differential equation (ODE) (Song et al. 2021b Lipman et al. (023). A score or velocity network trained using the forward process to "memorize" the dynamics. Then, one can generate new data from noise by rinerreversing the learned dynamics, which is usually referred to as the reverse process. In practice, velocity and score networks can be accurately learned so that the generation quality of flow matching and diffusion models is high, especially for image data.
Equal Contribution in Alphabetical Order.

---

Published as a conference paper at ICLR 2025
21-1
1ry-1
aF2
2F1
1
ST
9r-1
Mo+1
3/+
V11
yo
Figure Overview of our FIG algorithm during the conditional sampling process. Black arrows (++ denote the unconditional update. Orange arrows (->) represent K times conditional updates with unconditional sample x and measurement interpolant y/ at each timestep t. Blue arrows (-) indicate the measurement interpolation.
Due to the aforementioned advantages, flow matching and diffusion models have frequently been adopted to solve inverse problems for imaging data (Jalal et al. 2021). Specifically, for linear inverse problems, recent works use pre-trained flow matching or diffusion models as prior and guide towards the posterior by incorporating the measurement likelihood. Though existing methods achieve highly competitive results, most of them are either slow inference or struggle in challenging tasks.
In this work, we propose a simple yet effective algorithm for linear inverse problems named "Flow with Interpolant Guidance" (FIG). We summarize its mechanism Fig. [ FIG, the measurement y is interpolated parallel to the forward process to obtain measurement interpolants (3/, [oE(o Then, the likelihood y/: used to perturb the reverse process. Our key contributions are summarized as follows:
We propose novel algorithm FIG that applies to all linear forward measurement operators and pre-trained models irrespective of complexity.
We show that the updating scheme of FIG is theoretically justified.
We implement FIG using both flow matching and diffusion models to solve various linear image reconstruction tasks, including challenging scenarios such as high measurement noise and severe ill- posedness. Results demonstrate that FIG achieves state-of-the-aa of-the-art performance with competitive runtime and memory consumption.
2 BACKGROUND AND RELATED WORKS
2.1 FLOW MATCHING AND DIFFUSION MODELS
Though various formulations were proposed, flow matching generally defines a continuous time stochastic process (æ, (E for some T E (0, oo]. where 26 x² refers to the data distribution that one aims to draw samples from, and 2T = Ea is noise variable that is easy to obtain. The marginal distribution of an for / (0, is defined as interpolant between the two boundaries, i...
xt agæo + J,TT,
where {0t)<t(o.T] and (at)efo,7) are smooth interpolation coefficients such that og oT and at = oo = 0 (Albergo & Vanden- the dependence of v, on & when there no ambiguity. The interpolation Eq. (2) is the forward process of flow matching, whose dynamics can be represented by the probability flow ODE (Ma al. 2024
20233 Ma et al. 2024). For readability, we remove
a == (x)
Here, the velocity field satisfies
(z) E(d, (z, = x) = 1, x x) 0, x).
(4)
Typically, the velocity is parameterized by neural net ny and trained by minimizing the loss
() ["
E (100 - cg.ao ōgær 2dt.
(5)
2

---

Published as a conference paper at ICLR 2025
Given sufficiently well trained ve. sample the target x' by numerically solving Eq. in reverse time, i.e.,
a (x),
(6)
with initial condition Øт This corresponds to the reverse process of flow matching.
In contrast to flow matching, diffusion models directly define the dynamics of the forward process through an SDD. Given certain choices of the drift and diffusion coefficients, the forward process converges a noise distribution. More specifically, the forward diffusion model corresponds the Ito process
dæ f t) + g(tddW,
(7)
where f and g are predetermined, and W denotes standard Brownian motion. Specific choices of ()(())
and g lead different diffusion models. For example, taking f - and g(t) ((())
for some (tt) > 0, we obtain the sariance-preservve (VP) SDE (Song et al. 2021b b Ho et al. 2020
dx (t) + 3(t)dW.
(8)
When T + oo, the noise Dy ~ N I). The reverse process of Eq.
shown (Anderson 1982) correspond to dæ f(x, t) g())² 8 (x) dt + 9())dW,
where W a Brownian motion independent of W, and 8, (x(x) denotes the score function ær- Using the forward process defined in Eq. ( 0 ) = Va log p(x, t) via a loss similar to Eq. (5). where p(z, t) is the PDF of æt. One then runs the reverse process detailed in Eq. (9) with the trained score obtain data sample (Song et al.) 2021b).
trains neural net learn the score function
2.2 RELATION BETWEEN FLOW MATCHING AND DIFFUSION MODELS
There are both common grounds and differences between flow matching and diffusion models. For the flow matching setting, Ma et al. (0224) shows that
Uy (xx) = a,æ: + (xs;(x))
(10)
where a, by are determined by a and oe in Eq. (2). demonstrating the fact that score and velocity are interchangeable. However, it can also be shown by Tweedie's formula (Efron 2011 that
st (xx;)) = a, ((xo(xx)
(11)
which reveals singularity in the score function at time T, since or As v does explode so long as the interpolation coefficients are smooth, training velocity via flow matching is numerically more stable than training the score.
Though score and velocity are closely related, diffusion and flow matching models still differ significantly in terms of dynamics. In flow matching, there only one noise variable e.. Interpolation between data and noise leaves the path-wise trajectory of æ/ooT] smooth in f. In contrast, diffusion models are driven by Brownian motion, such that the path-wise trajectory is non-differentiable. Therefore, the law of diffusion process and a flow matching process can never agree. Nevertheless, with certain choices of coefficients, flow matching model and a diffusion model can share the same marginal distributions. For example, consider the VP SDE given in Eq. Let T O0,
0g = V - Se B(s)ds and a; e a f (())ds Then the distribution of æ; generated by Eq. is exactly the same as the one of the VP SDE given in Eq. (8). This suggests that number of algorithms such as DPS (Chung et al., 2023b) can replace their diffusion based priors with flow matching priors, since often the marginal distribution instead of the law of the whole process that matters.
2.3 LINEAR INVERSE PROBLEMS WITH FLOW MATCHING AND DIFFUSION PRIOR
Recent attempts addressing linear inverse problems using flow matching diffusion models can be classified into two categories: 1) task-specific methods that train a conditional diffusion model
(Saharia al. 2023 2022 Whang al. 2022); and 2) task-agnostic methods that only rely
3

---

Published as a conference paper at ICLR 2025
pre-trained unconditional diffusion models. For task specific methods, Liu et al. (2023a) Chung et al. (2023c) formulate the conditional diffusion as a Schrödinger bridge problem. which can be further related to the Doob's h- d--rrasformm (Särkkä & Solin 2019 Zhang et al. 2021 and stochastic optimal control (Uehara et al. 2024).
Task-agnostic methods tackle Bayesian inverse problems, where pre-trained flow matching or dif- fusion models are treated as priors. In contrast to the usual Bayesian settings, the prior can only be sampled by solving the reverse process. The key question is how incorporated measurement likelihood into the reverse process such that it leads to solution of the problem. Ideally, one method to "hijack" the reverse process by adding a guidance such that the process is conditioned on the measurement y. For instance, for diffusion priors, one aims to sample from the following SDE:
dæ [ (z. + v(t)²Va log Pt (alyy] dt + g(t)dW.
(12)
Since Væ, log Pt (2rly) = V log Pt (xx) + Væ log (y/xz) (Bayes rule), v log g (y/a)) is the guidance leading the posterior. However, the conditional density q (y (x,)) is intractable, because the measurement y only depends on 2o x due to the linear inverse problem specified in Eq. ①
Though the likelihood is intractable, various algorithms have been proposed tackle linear inverse problems with diffusion or flow matching prior. Among the existing methods, Chung & Ye (002);;
Chung et al. (2022); Zhu al. (2023) adopt a projection onto the measurement subspace: Kawar
et al. (2022): : Meng & Kabashima (2022) approximate q(y (r:)) via SVD; Mardani et al. (024);
Feng et al. (2023) use variational inference; Fei et al.] (2023): Chung et al. (202bb): [Song al.
(2023); y; Pokle et al. (2024); Pandey al. (2024) estimate the likelihood score V aa log 4 (
Dou & Song (2024); Trippe et al. (2023); Cardoso et al. (2024) carry out exact Bayesian posterior estimation with sequential Monte Carlo (SMC): Choi et al. (2021); (Song et al. (2022) enforce data consistency via optimization at each time step: Gen- et al. (2024) optimize the starting noise so that the generation process yields the intended results; Xu & Chi (2024; Li et al.) (2024) use the variable splitting techniques (Geman & Yang 1995] Afonso et al. 2010, Boyd et al. 2011 Venkatakrishnan al., 2013) decouple the optimization with respect prior and data consistency; Zhang et al. (2024) introduce decoupled noise annealing process to handle highly nonlinear problems. Despite promising experimental results, we find prior works struggle to efficiently handle challenging linear inverse problems. More specifically, we find methods of high reconstruction quality are often time consuming, and efficient methods fail to consistently obtain high quality reconstruction throughout different tasks. Furthermore, current methods struggle to effectively handle high levels of measurement noise.
METHOD
In this section, we propose 'Flow with Interpolant Guidance' (FIG), a simple yet effective algorithm
that can leverage all pre-trained flow matching and diffusion models to solve linear inverse problems.
During initialization, we perturb the measurement Le te parallel to the forward process of æ to get measurement interpolants y similar to (Song et al., 2022). The intuition of measurement interpolants comes from filtering (Dou & Song 2024). With the interpolants, the likelihood (1 (z)) is Gaussian for all t. Then, at each time E [0, we incorporate qu (y: xx)) as guidance for sampling follows. In Section [3.l. we state how to obtain measurement
ar, This section is organized interpolants. Section presents the FIG algorithm. We then show in Section 3.3 that the update scheme is theoretically justified. Section (.4] discusses details regarding the practical implementation of our algorithm. For readability, we choose to adopt a flow matching narrative in most of this section, although FIG can also use a diffusion prior. Empirical results using both flow matching and diffusion priors are shown in Section
3.1 GENERATING MEASUREMENT INTERPOLANTS
Given a pre-trained flow model, T, is interpolated following Eq. 22) with wo following the data distribution and Ty = Ex ~ (00 I). We consider parallel interpolation of i.e.,
Y areoo d,yr, where yo y. and yr Ey = AE.. The following sequence of equalities reveals the distribution of qo (y ax.): yi - a + n) + As = A(a + agn = Axx, + agn.
(13)
+
(14)
4

---

Published as a conference paper at ICLR 2025
Algorithm 1 Flow with Interpolant Guidance (FIG)
Require: T. c, K, w. Yo
1: Initialize FT Ey ~ 1)
b Initialize a;
2: A/ 1/T
3: for = T to do
4:
i/TTt'
1)/T
5: yi-1 = or + way AE,
D measurement interpolation with rescaled variance Unconditional update
6:
T;-1 T; vo At
7:
for k to K do
8:
0;-1 #i-1 cdeg. Al 20
V Nz
9:-1 A
K K-steps conditional update
9: end for
10:
end for
Hence, we have ge(ye = N (yr: Am,, T). We note that the generation the process y relies on the fact that the measurement Yo has a linear relationship with the signal Zo. therefore the interpolant measurement only works for linear models.
It is worth mentioning that æ; in Eq. [4) is defined by the interpolation process given in Eq. 2 instead probability flow ODE detailed Eq. (3). In this case, is fair to ask: is the conditional distribution of y 2 still Gaussian when 2; is sampled from Eq. 3 The answer is yes because æ; defined in Eq. 2 and Eq. (3) have the same distribution (Liu et al. 2023b).
3.2
ALGORITHM OVERVIEW
Before diving into the detailed analysis, we first provide an overview the FIG algorithm (see Algorithm D). The algorithm numerically solves reverse-time ODE to generate samples from the correct posterior.
dæ = oz + A V log (x) with B7 ~ NO0
(15)
Here, denotes the (unconditional) velocity field from the pre-trained model, X = Ō àe ot, and
Vr log de (y)ar)
V Azll2 The derivation and justification of Eq. 15 are detailed
20
in Section 3.3
Numerically, time is discretized on uniform grid with A/ referring the step size. In the following, we use and instead of æt, and yı, for readability. The updating scheme at each time t, corresponding to Euler's method with splitting (Leimkuhler & Matthews 2015). as follows:
a = V7. (x: At
(16)
03-1 =00_1
y:-1
A
20
Each step the two-step updating scheme has a different aim. The first step in Eq. (6) corresponds to the unconditional update, which is exactly numerical discretization of the reverse process specified in Eq. (6). practice, it performed via Euler's method (see line line Algorithm The second step is a conditional update that incorporates measurement information. In practice K gradient descent steps with learning rate carry out the conditional update (see line t line Algorithm []] We next introduce the theoretical reasoning for this simple but effective updating scheme.
3.3 THEORETICAL JUSTIFICATION
We now provide the theoretical reasoning underlying FIG. Importantly, note that for all t E (O,T), the marginal distribution of y: is uniquely determined by yo and E, as detailed Eq. (13) We make the following technical assumption:
The conditional distribution of x, given and e is equivalent to the conditional distribution of x, given yr.
Assumption
Let De(-)))) be the density of n, given y: The following theorem reviews the dynamics of p.(:)))).
5

---

Published as a conference paper at ICLR 2025
Theorem Let (z(l)) be the conditional velocity field that generates Pt. i.e., po(xly)) solves the reverse-time continuity equation with initial condition pr being a standard Gaussian:
Ocpe( ye) Vx [ (p) (x)))) O.
(17)
Let (((a))))) V. (alp)( be the score function. Then under Assumption
v = de x (xl)))
(18)
We defer the proof to Appendix A. Theorem T describes the conditional dynamics, which demon- strates consistency, i.c., posterior sampling will be achieved if the conditional continuity equation is solved in reverse time. The corresponding (reverse-time) probability flow ODE gives rise to the numerical updating scheme Eq. (16):
dæ (x)ye))
(19)
with initial condition æy ~ (O, I). In view of Eq. 18 rewrite the conditional probability flow ODE detailed in Eq. (19) as
7 V. log p. dt U (a: y.) - (01x- -
dx
- x N v log (x:) di + X V. log (y
(20)
0f
dt + Aeo v log (Ye dt)
Here, v log (eyla)) 2n10 V Ar. 2 Further, e, denotes the (unconditional) velocity field from the pre-trained trained model. Note that this equation identical to Eq. (15). An Euler discretization and o, i.e., when
of Eq. (15) yields exactly Eq. (T6). Interestingly, when we let at we follow the rectified flow (Liu et al., 2023b), the coefficient Xe = I-? = 1/SNR, where
SNR, 1-t ft. The following corollary summarizes the direction of the conditional update.
a,
Corollary The conditional update direction time 14 is equivalent that maximizes an Co-regularized posterior log bggliikel at the next time-step log
gradient flow direction
(6;-1 1|
|0:-1 112
loggp- Pi- 1 11-1)
(21)
0;
SNR,
2
Corollary 1 demonstrates that FIG updating direction is equivalent to maximizing a regularized logllike,hoodd, providing further intuition into the algorithm. The proof is provided in Appendix A.2
While all previous derivations use flow matching base model, FIG can also be applied to diffusion models. When adopting diffusion prior, the following corollary states the SDE version of FIG.
Corollary Let the pre-trained unconditional generation be diffusion model, as detailed in Eq. 7 then FIG admits the form:
dæ ff + g(t)) (zly1) dt g())dW
(22)
Numerically solving Eq. gives the FIG algorithm with diffusion prior. It is interesting to observe that Eq. (22) coincides with the limiting process of FPS (Dou Song 2024), yet the sampling procedure of FIG remains different. The proof is given in Appendix A.3
3.4 PRACTICAL IMPLEMENTATION
In practice, we observe that a conditional update can be viewed as minimizing the distance between the data 2t projected onto the measurement space and the measurement interpolants y:- The likelihood score represents the gradient the conditional update. Therefore, a single conditional update corresponds to step of gradient descent. If we perform only one conditional update per time-step and adjust the magnitude the update solely by tuning the weight e of the likelihood score, the effectiveness of the conditional update within single time-step will be significantly

---

Published as a conference paper at ICLR 2025
SR (4x)
Gaussian Deblur
Motion Deblur
Inpainting
Method
PSNR 1
S
LPIPS
PSNR f SSM
LPIPS
PSNR
SSIM
LPIPS 4 PSNR + SSM LPIPS
FIG-Flow
29.41
0.811
0.201
26.83
0.738
0.261
30.84
0.871
0,209
27.08
0.814
0.274
DPS-Flow
28.73
0.777
0.221
22.95
0.568
0.337
21.95
0.571
0.382
27.05
0.801
0.244
DMPS
29.00
0.807
0.222
26.74
0.725
0.255
29.81
0.828
0.226
12.66
0.284
0.658
OT-ODE
28.40
0.769
0.225
26.34
0.708
0.252
29.07
0.818
0.226
18.86
0.475
0.500
FIG-Diffusion
29.89
0.846
0.163
29.83
0.845
0.147
31.81
0.882
0.137
27.36
0.848
0.163
DPS-Diffusion
28.85
0.801
0.172
28.32
0.797
0,136
30.69
0.843
0.127
28.23
0.793
0.164
DDNM+
28.82
0.750
0.340
7.84
0.021
0.840
28.70
0.843
0.182
DAPS
29.59
0.809
0.158
29.74
0.808
0.146
31.67
0.859
0.119
27.85
0.791
0.183
Table 1: Quantitative comparison (PSNR, SSIM, LPIPS) of different algorithms for different tasks on the CelebA-HQ 256 256 test dataset. All input images have a measurement Gaussian noise of Tn 0.05. Bold for the best.
reduced. However, this distance is fully optimized, the log likelihood of measurement interpolant will be overwhelming compared to the prior information, resulting in severe overfitting. Thus, maintaining a balance between the measurement information and the prior information is crucial for reconstruction problem. To remedy the issue, introduce new parameter K. After an unconditional update, we perform K conditional updates with theoretically justified weights and directions as previously shown. We also note that due the nature of ODEs, once the velocity field and initialization are determined, the final generated data is uniquely determined. However, the random initialization does not correspond the desired reconstruction result. Therefore, we opt to average multiple random initializations, thereby reducing the impact of incorrect initial values during measurement interpolation. Given that our measurement model linear, this approach is equivalent to introducing another parameter w to generate y:. i.e., the new ye is generated by ox + worey.
By combining the practical design considerations, we obtain Algorithm T. K are constants, with c and K being task-specific, governing the balance between unconditional and conditional updates. See Appendix [ for more details on the choice of hyper-parameters. The parameter helps us obtain an empirically better log- logliikeldhood of the measurement interpolants. The effectiveness of both strategies is examined in later ablation studies.
The parameters c, w,
Although FIG is effective in solving most linear imaging inverse problems, it struggles with inpainting tasks where a large region is masked. The reason is that the masked pixels carry zero information for the conditional update, they are totally "masked". Therefore, we propose simple remedy termed FIG+. It improves the performance for inpainting tasks. Concretely, for each æ; update, we first estimate to using Tweedie's formula, and perturb it to time i get क;- Then mix the masked parts of é with the current &; to update the masked region. A hyper-parameter mixing weight m E [0, 1] determines how much we trust &t. We provide the pseudocode in Appendix C When m get the original FIG algorithm. The additional three steps of FIG+ only involve simple addition and matrix (mask) multiplication. Hence they don't 't affect the runtime and memory consumption much In our experiments below, we only use FIG+ for the inpainting task.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets. We conduct experiments on 3 natural image datasets: CelebA-HQ (Karras et al. 2018). LSUNN-edroom YYu et al. 2015). and AFHQ-Cat (Choi al. 2020). All images are taken from the official test data splits and are preprocessed to the size of 256 x 256 x 3.
Tasks. Our evaluation focuses on linear inverse problems including super-resolution, deblurring. and inpainting. For super-resolution, we apply 4x bicubic downsampling across all datasets. For deblurring tasks, we use Gaussian blurring with a kernel size of 61 x 61 and a standard deviation of 3.0, and motion blurring with the same kernel size but standard deviation of 0.5. For inpainting, we perform random inpainting by masking out 90% of the total pixels. We follow the SVD-based measurement operations defined in DDRM (Kawar al. 2022) DDNM, DMPS, and OT-ODE

---

Published as a conference paper at ICLR 2025
(a) Reference
(b) Mcasurement
(c) DPS
(d) DMPS
(c) OT-ODE
(f) FIG+ (ours)
Figure 2: Results for 90% random inpainting with noise on = 0.05 on the CelebA-HO dataset using flow matching base model.
(a) Reference
(b) Mcasurement
(c) DPS
(d) DDNM+
(c) DAPS
(f) FIG+ (ours)
Figure 3: Results for 90% random inpainting with noise o, 0.05 on the CelebA-HQ dataset using a diffusion base model.
require an SVD for their conditional updates. For all tasks above, we add measurement Gaussian noise ~ NO0 o I) with on 0.05. To further demonstrate the capabilities of our algorithm, conducted two additional, more complex experiments: 4x super-resolution with high noise (a,, = 1.0), 16x super-resolution with noise (a,, 0.2).
Metrics. For the quantitative comparison, Similarity (LPIPS) distance (Zhang et al.) 2018), along with two standard metrics: peak signal-to- noise-ratio (PSNR), and structural similarity index (SSIM).
the perceptual Learned Perceptual Image Patch
Baselines. We compare the performance of our algorithm to several state-of-the-art training- free algorithms for solving inverse problems, including DPS (Chung al. 2023b), OT-ODE (Pokle et al. 2024). DMPS (Meng & Kabashimal 2022), DDNM/DDNM+ (Wang et al. 2023), and DAPS (Zhang al. 2024). OT-ODE an improved version of LIGDM (Song al.) 2023) applied flow matching models, thus it is used instead of LIGDM as the baseline. Likewise, we transferred DPS and DMPS to flow matching using Eq. (88). Leveraging the advantages of flow matching models, we fine-tune the baseline methods to ensure they all achieve their best performance at 50 NFEs except for OT-ODE. It requires less than 50 NFEs due its start time strategy.
Base Models. We implemented our FIG algorithm for both flow matching and diffusion models. We refer both incarnations via FIG-Flow and FIG-Diffusion. For each base model, we then compared with three tfate-of-the-an algorithms respectively. For FIG-Flow, we use the pre-trained Rectified Flow model from Liu et al. (2023b) as our base model and implement all baselines on it to ensure a fair comparison. In the diffusion category, utilize EDM (Karras et al. 2022) the base model for our algorithm, DPS, and DAPS. Since DDNM/DDNM+ employs a unique time travel structure that may not transfer well, retain their original DDIM base model and conduct experiments using their original code. For unconditional generation, we choose the Euler ODE solver for simplicity as makes it easier add the measurement information at each sampling step.
8

---

Published as a conference paper at ICLR 2025
SR (4x) On 1.0
SR(16x) On 0.2
Method
PSNR
SSI
LPIPS
PSNR
SSM
LPIPS
FIG (ours)
24.47
0.722
0.315
19.92
0.553
0.405
DPS
22.50
0.593
0.378
19.20
0.531
0.424
DMPS
24.06
0.678
0.342
19.91
0.536
0.441
OT-ODE
15.14
0.191
0.695
17.79
0.408
0.570
Table 2: Quantitative comparison (PSNR, SSIM, LPIPS) different algorithms for high noise bicubic super-resolution the CelebA-HQ 256 x 256 test dataset. Bold for the best.
(a) Reference
(b) Mcasurement
(c) DPS
(d) DMPS
(c) OT-ODE
(f) FIG (ours)
Figure 4: Results for 4x bicubic super-resolution with high noise on 1.0 on the CelebA HQ dataset.
4.2 EXPERIMENTAL RESULTS
The quantitative comparisons are listed in Tables and 12 with samples of reconstructed images shown in Figs. to 6 Additional experimental results are presented in Appendices and G Our algorithm FIG demonstrates highly competitive performance across all tasks, metrics, and datasets. For all these tasks, DPS and OT-ODE tend to produce sharper edges, but this sometimes results in unrealistic details and textures, leading to worse metrics. In contrast, our algorithm generates faithfully smooth images with good balance of detail and realism.
Our algorithm exhibits superior per- formance compared to other baselines with flow matching prior, particularly for the challenging super-resolution tasks with high levels noise. We do not include the results from diffusion- based algorithms for those tasks be- cause they all fail yield reason- able reconstruction. Note that [Wang (2023) reported similar task with average pooling super-resolution but it fails on bicubic super-resolution. See Appendix E for more details of DDNM/DDNM+, Additionally, to fur- ther demonstrate the robustness of our algorithm. we test on 4x super- resolution with non-uniform and high noise. As shown in Fig. 6, our algorithm performs well even under these challenging conditions.
Algorithm
Total Memory
NFEs
Avg. Inf. Time (s)
Flo-llow
7735MB
50
2.80=0.12
DPS-Flow
8109MB
50
4.41=0.
DMPS
7847MB
50
2.83+0.15
OT-ODE
8257MB
40
3.69=0.1
FIG-Diffusion
2809MB
100
1,06=1.01
OPS-DDifusioo
7761MB
100
8. 8.19:1.04
DDNM+
28707MB
1210
28.36#0.61
DAPS
2809MB
1000
59.56=1.70
Table 3: The runtime and memory consumption required for different algorithms on the CelebA-HQ 256 x 256 test dataset for the super-resolution task. All experiments conducted on a single NVIDIA RTX A6000 GPU for recon- structing one image. Bold for the best.
In addition delivering outstanding performance, algorithm maintains low computational cost in terms of both runtime and memory usage as shown Table
To validate theoretical results and the effectiveness of practical design components, perform ablation studies (detailed in Appendix [B) on 1) K K-steps of conditional update with results in Fig. and Tables to Appendix [B. and 2) measurement interpolant variance rescaling with results in

---

Published as a conference paper at ICLR 2025
(a) Reference
(b) Mcasurement
(b) DPS
(b) DMPS
(b) OT-ODE
(b) FIG (ours)
Figure 5: Results for 16x bicubic super-resolution with high noise o = 0.2 on the CelebA-HQ dataset.
RT AIR
(a) Reference
(b) Mcasurement
(c) FIG (ours)
(d) Reference
(c) Measurement
(f) FIG (ours)
Figure 6: Results for 16x bicubic super-resolution with high non-uniform Gaussian noise (first row)
and non-uniform Laplacian noise (second row) CelebA-HQ dataset.
Fig. and Tables 7 in Appendix B.2 Both the visual outputs and quantitative results support the principles underlying the development of FIG.
CONCLUSION
In this paper, presented FIG, task-agnostic algorithm for solving linear inverse problems. Our key observation: adding noise to the measurement is easier compared to denoising the signal. We thus measurement interpolants to guide the unconditional generation. The FIG algorithm simple yet effective, as is shown by experiments on multiple tasks over several high-dimensions datasets.
Limitations: First, since the measurement interpolants rely on the linear relationship between the measurement and the underlying signal, FIG can not directly be applied to general non-linear inverse problems. For the same reason, the algorithm is not compatible with pre-trained models with latent encodings. Second, the FIG result is still an approximated posterior. To obtain true posterior samples, one should consider methods based on stochastic optimal control (Guo et al. 2024 Uehara that it generates the intended image (Ben-Hamu
2024). Another way is to sequentially update 2024).
et al.
Future Work: Although FIG can not be extended to non-linear problems in general, its potential application bilinear problems such as blind motion deblurring (Chung et al. 2023aa Fei et al. 2023) could be interesting direction. When fixing either the blurring kernel the image, the noise model linear, thus measurement interpolants can be constructed. Using FIG's updating scheme could potentially lead better reconstruction. However, initialization and regularization the kernel highly non-tring non-trivial, and require future explorations. Another interesting direction is to extend FIG to latent models (Rout et al. 2024 Song et al. 2024
10

---

Published as a conference paper at ICLR 2025
Acknowledgements: YY, YZ and ZZ are supported in part by the NSF grant 1934757 and the Alfred Sloan Foundation. XM is supported by NSFC No. 62306277, and the Fundamental Research Funds for the Zhejiang Provincial Universities Grant No. K20240090.
REFERENCES
Manya V Afonso, José M Bioucas-Dias, and Mário AT Figueiredo. Fast image recovery using variable splitting and constrained optimization. IEEE transactions on image processing, 19(9): 2345-2356, 2010.
Michael S Albergo, Nicholas M Boffi, and Eric Vanden-Eijnden Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint arXiv:2303.08797 2023.
Michael Samuel Albergo and Eric Vanden-Fijnden. Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations, 2023.
Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-326, 1982.
Heli Ben-Hamu, Omri Puny, Itai Gat, Brian Karrer, Uriel Singer, and Yaron Lipman. D-Flow: Differentiating through flows for controlled generation. In Proceedings of the 41st International Conference on Machine Learning, 2024.
Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends@ in Machine learning, 3(1):1-122, 2011.
Gabriel Cardoso, Yazid Janati idrissi, Eric Moulines, and Sylvain Le Corff. Monte carlo guided denoising diffusion models for bayesian linear inverse problems. In The Twelfth International Conference on Learning Representations, 2024.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in neural information processing systems, volume 31, 2018.
Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. ILVR: Conditioning method for denoising diffusion probabilistic models. In 202/ IEEF/CVF International Conference on Computer Vision (ICCV), pp. 14347-14356, 2021.
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-We Woo Ha. StarGAN v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pp. 8188-8197, 2020.
Hyungjin Chung and Jong Chul Ye. Score-based diffusion models for accelerated MRL. Medical image analysis, 80:102479, 2022.
Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. Come--loser-diffusse--ttt. Accelerating condi- tional diffusion models for inverse problems through stochastic contraction. In 2022 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12403-12412, 2022.
Hyungjin Chung, Jeongsol Kim, Sehui Kim, and Jong Chul Ye. Parallel diffusion models of operator and image for blind inverse problems. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 6059-6069, 2023a.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mecann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023b.
Hyungjin Chung, Jeongsol Kim, and Jong Chul Ye. Direct diffusion bridge using data consistency for inverse problems. In seirth- Conference on Neural Information Processing Systems, 2023c.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In Advances in Neural Information Processing Systems, 2021.
11

---

Published as a conference paper at ICLR 2025
Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In The Twelfth International Conference on Learning Representations, 2024.
Bradley Efron. Tweedie's s formula and selection bias. Journal of the American Statistical Association, 1061496):1002-1614, 2011.
Ben Fei, Zhaoyang Lyu, Liang Pan, Junzhe Zhang, Weidong Yang, Tianyue Luo, Bo Zhang, and Bo Dai. Generative diffusion prior for unified image restoration and enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 0935-9946, 2023.
Berthy T. Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine L. Bouman, and William T. Freeman. Score-based diffusion models as principled priors for inverse imaging. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10486-10497, 2023.
Donald Geman and Chengda Yang. Nonlinear image recovery with half-quadratic regularization. IEEE transactions on Image Processing, 4(7):932-946, 1995.
Yingqing Guo, Hui Yuan, Yukang Yang, Minshuo Chen, and Mengdi Wang. Gradient guidance for diffusion models: An optimization perspective. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pp. 6840-6851, 2020.
Jérôme Idier. Bayesian approach to inverse problems. John Wiley & Sons, 2013.
Ajil Jalal, Marius Arvinte, Giannis Daras, Eric Price, Alex Dimakis, and Jonathan Tamir. Robust com- pressed sensing MRI with deep generative priors. In Advances in Neural Information Processing Systems, 2021.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. In International Conference on Learning Representations, 2018.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Advances in Neural Information Processing Systems, 2022.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In ICLR Workshop on Deep Generative Models for Highly Structured Data, 2022.
Benedict Leimkuhler and Charles Matthews. Molecular Dynamics: With Deterministic and Stochastic Numerical Methods. Interdisciplinary Applied Mathematics. Springer, 2015.
Xiang Li, Soo Min Kwon, Ismail R Alkhouri, Saiprasad Ravishanka, and Qing Qu. Decoupled data consistency with diffusion purification for image restoration. arXiv preprint arXiv:2403.06053. 2024.
Yaron Lipman, Ricky TQ Chen, Heli Ben Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023.
Guan-Home Horng Liu, Arash Vahdat, De-An Huang, Evangelos Theodorou, Weili Nie, and Anima Anandkumar. SSB: Image-to-image schrödinger bridge. Proceedings of the 40th International Conference Machine Learning, volume 202, pp. 22042-22062, 2023a.
Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023b.
Nanye Ma, Mark Goldstein, Michael S Albergo. Nicholas M Boffi, Eric Vanden- and Saining Xie. SiT: Exploring flow and diffusion based generative models with scalable interpolant transformers. In European Conference on Computer Vision, pp. 23-40, 2024.
12

---

Published as a conference paper at ICLR 2025
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective solving inverse problems with diffusion models. The Twelfth International Conference on Learning Representations, 2024.
Xiangming Meng and Yoshiyuki Kabashima. Diffusion model based posterior sampling for noisy linear inverse problems. arXiv preprint arXiv:2211. 2022.
Kushagra Pandey, Ruihan Yang, and Stephan Mandt. Fast samplers for inverse problems in iterative refinement models. In The Thirtt-eighth Annual Conference on Neural Information Processing Systems, 2024.
Ashwini Pokle, Matthew J. Muckley, Ricky T. Q Chen, and Brian Karrer. Training-free linear image inverses via flows. Transactions on Machine Learning Research, 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF confer- ence on computer vision and pattern recognition, pp.
2022.
Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkottai. Solving linear inverse problems provably via posterior sampling with latent diffusion models. In Advances in Neural Information Processing Systems, volume 36, 2024.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, SIGGRAPH '22, 2022.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713 2023.
Simo Särkkä and Arno Solin. Applied stochastic differential equations, volume 10. Cambridge University Press, 2019.
Bowen Song, Soo Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen. Solving inverse problems with latent diffusion models via hard data consistency. In The Twelfth International Conference on Learning Representations, 2024.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations, 2021a.
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guiide diffusion models for inverse problems. International Conference on Learning Representations, 2023.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference Learning Representations, 2021b.
Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. Solving inverse problems in medical imaging with score-based generative models. In International Conference Learning Representations, 2022.
Brian L. Trippe, Luhuan Wu, Christian A. Naesseth, David Blei, and John Patrick Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. In ICML 2023 Workshop Structured Probabilistic Inference & Generative Modeling, 2023.
Masatoshi Uchara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M Tseng, Tommaso Biancalani, and Sergey Levine. Fine-tuning of continuous-tima diffusion models as entropy control. arXiv preprint arXiv:2402. 15194, 2024.
Singanallur V Venkatakrishnan, Charles A Bouman, and Brendt Wohlberg. Plug-and-pl priors for model based reconstruction. In 2013 IEEE global conference on signal and information processing. pp. 945-948. IEFE, 2013.
13

---

Published as a conference paper at ICLR 2025
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. The Eleventh International Conference on Learning Representations. 2023.
Jay Whang, Mauricio Delbracio, Hossein Talebi, Chitwan Saharia, Alexandros G. Dimakis, and Pey- man Milanfar. Deblurring via stochastic refinement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16293-16303, June 2022.
Xingyu Xu and Yuejie Chi. Provably robust boore-based diffusion posterior sampling for plug- and and-play image reconstruction. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. LSUN: Construction of a large- scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365. 2015.
Benjamin Zhang, Tuhin Sahai, and Youssef Marzouk. Sampling via controlled stochastic dynamical systems. In I (Still) Can't it Believe It's Not Better! NeurIPS 2021 Workshop, 2021.
Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing. arXiv preprint arXiv:2407.01521, 2024.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and paltern recognition, pp. 586-595, 2018.
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and-plas and-play image restoration. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1219-1229, 2023.
14

---

Published as a conference paper at ICLR 2025
APPENDIX: FIG: FLOW WITH INTERPOLANT GUIDANCE FOR LINEAR INVERSE PROBLEMS
A PROOFS
A.I PROOF OF THEOREM 1
The proof follows from Appendix C of Ma et al. (2024). Pe (a: yo: Ey) and later use Assumption to get pa a y:). Let
We start with the conditional density
(c(k)
eth (r uo: Eyddr
(23)
be the characteristic function, which is also equal to
(k) = i.
(24)
Here E denote the expectation on an yo, By and e. yo Ey. Similar Ma al. 2024 we have
Oep:(k) = 0 E, ikm
(
ikE,, e (2x))
(25)
ik-B,, [i0 dhe (cs ) e
ikEp, Üū (z: (
where we define (z(z;) = B( + o (z:) = Eáá,zo + o yo. Eg).
Now using the two formulations of the characteristic function given in Eq. 23 and Eq.
24 we have
/
ch 7, E,)dæ ik f (xx))) 2. 2 (7, E dd)
0,
(26)
Thus we have
/
Mk euk- (20, Ex)dd
(x( V ( ck p Bo. Ey
(27)
Vx ( (x, (7, (x: e,)]e ik dx,
where the last equality uses integration by parts. Using the inverse Fourier transform we have
(2 Ey) + V (xx Dr, (x (alyo= Ey)] 0.
(28)
Since start from T, minus sign should be added ö. Using Assumption gives Eq. [7).
Assumption [ can also be equivalently stated as follows: let the density function of o; given yo and E, be D. (ac yo, Ey), then there exists pt such that
(= ey) pt (x/ge))
(29)
The equality holds when E, is independent of Ez. The proof follows the one of Equation 9 by Ma et al. (2024).
vy (x(g))))
á; E
x. a.. y:
Og ( ax, 9t)
0,0
àgE
2,, y/
J.E [e, x. y1
a₁
x
+
0:
E [ lx, yr)
(30)
0n
a
du
àig
r.
+
0;
(xot)((x)))))
Ct1
a
dtt
15

---

Published as a conference paper at ICLR 2025
A.2 PROOF OF COROLLARY
From Eq. ( the first update for æ uses information of Φ; If we consider the second step first and then it's next update step, the update scheme reads as follows:
1:-1 =
V "ai_, /,c,,/ii1 Vi-1 2070R
Ax 1
A
(31)
1/-2 21-1 Ut, (2)-1)A
which is splitting of
0tti O V log ( (x)Bt..))
(32)
This completes the proof.
A.3
PROOF COROLLARY
The corollary direct result of Theorem The velocity field of diffusion models is defined in the probability flow ODE (Song et al. 2021b) which is exactly Eq. (22) when conditioning on y:-
B ABLATION STUDY
B.1 K-STEPS CONDITIONAL UPDATE
In this section, we perform the ablation study the K-steps conditional update strategy.
(a) Reference
(b) Measurement
(c) K== =
(d) Overfit
(e) optimal K
Figure 7: Our FIG algorithm for Gaussian deblurring and random inpainting with noise o, 0.05 on the CelebA-HQ dataset.
SR (4x)
Gaussian Deblur
Colorization
Inpainting
Method
PSNR
SSIM f LPIPS L
PSNR
SSIM 1 LPIPS
PSNR
SSIM
LPIPS
PSNR
SSI
IPIPS
K ==
28.51
0.812
0.222
25.43
0.691
0.271
24.21
0.902
0.241
30.02
0.895
0.189
optimal K
28.51
0.812
0.222
26.45
0.738
0.257
24.21
0.902
0.241
33.33
0.920
0.131
Table Quantitative comparison (PSNR, SSIM, LPIPS) of our algorithm for different K's on the CelebA-HQ 256 x 256 validation set. All input images have measurement Gaussian noise of Tn = 0.05. Note that for SR (4x) and colorization, the optimal K = 1. so they have the same metrics.
16

---

Published as a conference paper at ICLR 2025
SR (4x)
Gaussian Deblur
Colorization
Inpainting
Method
PSNR
SSM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
K
24.48
0.646
0.370
22.75
0.533
0.445
24.27
0.862
0.284
27.25
0.815
0.246
optimal K
24.48
0.646
0.370
23.36
0.554
0.433
24.27
0.862
0.284
29.17
0.872
0.156
Table 5: Quantitative comparison (PSNR, SSIM, LPIPS) of our algorithm for different K's on the LSUN-Bedroom 256 x 256 validation set. All input images have mcasurement Gaussian noise of on 0.05. Note that for SR (4x) and colorization, the optimal K 1, so they have the same metrics.
SR (4x)
Gaussian Deblur
Colorization
Inpainting
Method
PSNR
SSIM + LPIPS 1 PSNR
SSIM f LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
K = 1
28.45
0.730
0.260
25.33
0.590
0.330
25.52
0.906
0.246
31.04
0.867
0.178
optimal K
28.45
0.730
0.260
26.25
0.632
0.310
25.52
0.906
0.246
31.04
0.867
0.178
Table 6: Quantitative comparison (PSNR, SSIM, LPIPS) of our algorithm for different K's on the AFHQ-Cat 256 x 256 validation set. All input images have a measurement Gaussian noise of 0.05. Note that for SR (4x), colorization, and inpainting, the optimal K 1, so they have the same metrics.
on
We conduct experiments on separate validation set (100 images from the official test data split) using different values of K and report results for K = and the optimal K. In each experiment, we finetune other hyperparameters so that our algorithm achieves the best performance for the given K. Fig. 7 contains examples of Gaussian deblurring and random inpainting
the CelebA-HQ dataset. When we set = 1. the results are blurry and exhibit unnatural details compared to the result for the optimal K. If we fully minimize the distance 3/e AAxzll
1/2 at each timestep. the output shows severe overfitting problems as illustrated in Fig. 7d To mitigate the risk, we tune K and e for better
performance. The quantitative results shown in Table 4 Table 5 and Table 6
B.2 MEASUREMENT INTERPOLANT VARIANCE RESCALING
In this section, we ablate the measurement interpolant variance rescaling technique.
(a) Reference
(b) Mcasurement
(c) ==1=
(d) optimal ur
Figure 8: Our FIG algorithm for super-resolution and random inpainting with noise on = 0.05 on the and AFHQ-Cat datasets.
17

---

Published as a conference paper at ICLR 2025
SR (4x)
Gaussian Deblur
Colorization
Inpainting
Method
PSNR
SSIM
LPIPS
PSNR
SSIM
LPPPS
PSNR
SSIM
LPPPS
PSNR
SSIM
LPIPS
25.66
0.585
0.376
24.08
0.533
0.383
23.24
0.673
0.400
19.36
0.407
0.495
optimal w
28.51
0.812
0.222
26.45
0.738
0.257
24.21
0.902
0.241
33.33
0.920
0.131
Table 7: Quantitative comparison (PSNR, SSIM, LPIPS) of our algorithm for different w's on the CelebA-HQ 256 256 validation set. All input images have a mcasurement Gaussian noise of n 0.05.
SR (1x)
Gaussian Deblur
Colorization
Inpainting
Method
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSI
LPIPS
PSNR f SSIM
TPIPS
=
21.02
0.327
0.566
20.06
0.273
0.575
19.80
0.486
0.557
19.48
0.346
0.571
optimal w
24.48
0.646
0.370
23.36
0.554
0.433
24.27
0.862
0.284
29.17
0.872
0.156
Table 8: Quantitative comparison (PSNR, SSIM, LPIPS) of our algorithm for different w's on the LSUN- 256 256 validation set. All input images have measurement Gaussian noise of
Tn 0.05.
SR (4x)
Gaussian Deblur
Colorization
Inpainting
Method
PSNR f
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR f SSIM
LPIPS
PSNR f SSIM
LPIPS
=
26.02
0.538
0.395
24.74
0.466
0.419
23.95
0.685
0.441
21.76
0.388
0.523
optimal w
28.45
0.730
0.260
26.25
0.632
0.310
25.52
0.906
0.246
31.04
0.867
0.178
Table Quantitative comparison (PSNR, SSIM, LPIPS) of algorithm for different w's on the
AFHQ-Cat 256 x 256 validation set. All input images have a measurement Gaussian noise of on 0.05.
our algorithm, the measurement interpolant defined as y: = Oz:Jo + watey. Without rescaling, ie., for w 1, as shown in the first row in Fig. 8 we observe that the reconstructed result appears as if two images have been combined: one representing our measurement and the other determined by the initialization and the ODE. Increasing the weight of the measurement guidance this issue, but it leads to overfitting, resulting in a high level of noise in the final output as shown in the second row in Fig. 8 With smaller optimal we can achieve balance between these two situations, resulting in much better outcomes. The quantitative results are presented Tables to 9 All experiments are conducted on the validation set by carefully fine-tuning the hyper-parameters. Fig. 9 gives a visual illustration of effects on different w for super resolution on CelebA with flow matching prior. It's clear that w affects the performance of our algorithm and there exists an optimal w. We only show the PSNR and LPIPS because the SSIM scores are close.
address
C FIG+ ALGORITHM
We provide the FIG+ algorithm in Algorithm 2
18

---

Published as a conference paper at ICLR 2025
0.2458
-
PSNR
- LPP5
+0.2425
28.
0.2400
PSNR
28.60
-0.2375
0,2350
28.66
-0.2325
-0.2275
Variance Reszaling Factor w
Figure 9: Metrics for 4x super resolution CelebA-HQ 256 x 256 validation set with different variance rescaling factor w.
Algorithm 2 FIG+
Require: C, K, w. m, Yo
I: Initialize FT En ~ NN I)
> Initialize
2: A/ = 1/T
3: for = T to do
4: = i/T, = (i T
5:
Yi-1 ag 0 w0 AE
measurement interpolation with rescaled variance
6:
I = I; Un x;, A)
Unconditional update
7:
for
to K do
8:
21-1 = .i-1
cdva.At V -:-1 9:-1 A 20fa
D. K K-steps conditional update
9:
end for
10:
2/1 (1 1)20 (xi 1) + 03
use Tweedie's formula to estimate æn perturb estimated mo time 4--1
11: a/ 0;-12% 03-1Ex
12: T. Ax;-1 + m) A) + m(I A)
D mix a;_ with x;
end for
D RECOVERY VS PERCEPTUAL TRADE-OFF
Method
PSNR t
SSIM
LPIPS $
FID
KID J
FIG-Diffusion
29.89
0.846
0.163
25.46
0.013
DPS-Diffusion
28.85
0.801
0.172
19.31
0.005
DDNM+
28.82
0.750
0.340
64.85
0.052
DAPS
29.59
0.809
0.158
23.69
0.013
C-IIGDM
28.14
0.787
0.106
30.51
0.019
Table 10: Quantitative comparison (PSNR, SSIM, LPIPS, FID, KID) of different algorithms for 4x super-resolution with noise o, 0.05 on the CelebA-HQ 256 256 test dataset. Bold for the best.
In this section, we report FID and KID along with the metrics used in the main paper, i.c., PSNR, SSIM, and LPIPS for 4x super-resolution with noise o, 0.05 the CelebA-HQ 256 256 (see Table 10 and discuss the recovery and perceptual trade-off in those metrics.
PSNR and SSIM considered recovery metrics that measure pixel-level fidelity while LPIPS, FID, and KID are referred to as perceptual metrics that assess perceptual similarity or high-level semantic fidelity.
19

---

Published as a conference paper at ICLR 2025
(a) Reference
Average pooling
(c) Bicubic
Figure 10: DDNM+ results for 4 bicubic super-resolution with noise or 0.05 on the CelebA-HQ dataset.
In this paper, focus image reconstruction tasks involving noisy measurements. In this case, metrics like FID and KID, while effective for generative models emphasizing perceptual realism, evaluate distribution-level similarity and may fail capture details in structural recovery, particularly in high-noise scenarios where subtle details matter, Additionally, FID and KID can be misleading if the reconstructed images perceptually plausible but deviate significantly from the ground truth. PSNR and SSIM, however, provide clear and objective measure of how well the noise has been suppressed and the original content preserved, which remains critical in our experiments. In all, PSNR and SSIM, LPIPS are baseline metrics widely used in image restoration domain.
As shown in Table TO our algorithm achieves balanced trade-off between recovery and perceptual metrics. In the task of noisy image reconstruction, it not only delivers the best recovery metrics but also achieves impressive perceptual scores.
E BASELINE IMPLEMENTATIONS
For the pretrained model, flow matching base model, we use the rectified flow For measurement operators. since our baseline methods OT-ODE and DMPS require an SVD, we inherit codes from the DDRM Github repository.
both flow matching and diffusion models. For all algorithms with
DPS-Flow. In the codes for the Rectified flow model, a₂ and og = 1- 1 f. Following Eq. (18).
we have ž. (1 t) (xx, f) + 24. Given eo, we implement DPS strictly following Algorithm Chung et al. (023b) and fine-tune the hyperparameter Çi to achieve the best performance.
DMPS. With the SVD operations from DDRM codes, we implement DMPS strictly following Algorithm in Meng & Kabashimal (2022) and fine-tune the hyperparameter À to get the best results. Again, the score function obtained by the velocity following Eq. (18).
OT-ODE. Using the same do as DPS and the SVD operations from the DDRM implementation, we implement the OT-ODE strictly following Algorithm of et al. (2024) and test the start time for each task to get the best performance.
DAPS. We use the official repository P and strictly follow the Algorithm of Zhang al. (0224) to implement DAPS.
DPS-Diffusion. We implement DPS on the EDM implementation from the DAPS repository.
DDNM/DDNM+. We implement DDNM and DDNM+ strictly following Algorithm and 2 of Wang et al. )2023) We reproduce the metrics of the tasks reported their paper find that the performance of DDNM/DDNM+ varies significantly across different tasks. Our experiment shows that DDNM/DDNM+ fails on Gaussian deblurring with noise, which matches the result of Meng & Kabashimal (2022). We also find that DDNM/DDNM+ can't handle super-resolution with bicubic down-sampling well, resulting in noisy reconstructions as shown in Fig. TO However for super-resolution with average pooling, achieved good results for both SR(xx) with noise standard
http:://gttthbb..om////aaaaaaa/
20

---

Published as a conference paper at ICLR 2025
SR (4x)
Gaussian Deblur
Colorization
Inpainting
Method
PSNR
SSI
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSM
LPIPS
FIG (ours)
24.54
0.682
0.327
22.94
0.588
0.402
21.87
0.865
0.320
30.24
0.895
0.139
DPS
24.23
0.649
0.346
19.82
0.433
0.471
14.30
0.610
0.596
27.21
0.771
0.283
DMPS
24.08
0.653
0.368
22.76
0.575
0.413
21.26
0.865
0.326
27.34
0.841
0.212
OT-ODE
23.82
0.639
0.349
22.16
0.566
0.389
22.06
0.781
0.385
26.56
0.768
0.278
Table 11: Quantitative comparison (PSNR, SSIM, LPIPS) of different algorithms for different tasks on the --UN-Berroom 256 x 256 test dataset. All input images have a measurement Gaussian noise of n 0.05. Bold for the best.
SR (4x)
Gaussian Deblur
Colorization
Inpainting
Method
PSNR
SSIM + LPIPS 4
PSNR
SSIM f LPIPS
PSNR
SSIM
LPIPS
PSNR
SSMM
LPIPS
FIG (ours)
27.50
0.707
0.272
25.14
0.591
0.334
26.36
0.915
0.227
28.99
0.835
0.202
25.70
0.607
0.332
21.40
0.407
0.429
21.82
0.763
0.407
26.20
0.709
0.300
DMPS
27.49
0.709
0.286
24.97
0.577
0.343
26.12
0.913
0.221
27.02
0.781
0.262
OT-ODE
25.95
0.617
0.326
24.42
0.536
0.357
25.39
0.849
0.291
28.22
0.789
0.235
Table 12: Quantitative comparison (PSNR, SSIM, LPIPS) of different algorithms for different tasks on the AFHQ-Cat 256 x 256 test dataset. All input images have measurement Gaussian noise of on 0.05. Bold for the best.
deviation 0.05 and SR(16x) with noise standard deviation 0.2. Motion deblur is not implemented since DDNM requires SVD operations.
F ADDITIONAL EXPERIMENTAL RESULTS FOR FLOW MATCHING MODEL
We show additional results with flow matching priors. Reconstructed images for motion deblurring with a, 0.05 on CelebA-HQ are shown in Fig. [1 Additional quantitative results on other datasets (LSUN-Bedroom, AFHQ-Cat) are shown in Tables
and 12
(a) Reference (b) Measurement
(c) DPS
(d) DMPS
(e) OT-ODE
(f) FIG (ours)
Figure 11: Results for motion deblurring with noise o, 0.05 the CelebA-HQ dataset.
21

---

Published as a conference paper at ICLR 2025
hilf
Reference
(b) Measurment
(c) DPS
(d) DMPS
(e) OT-DE
(I) FIG (ours)
Figure 12: Results for super-resolution (4x) with noise a, = 0.05 on AFHQ-Cat, CelebA-HO, and datasets.
22

---

Published as a conference paper at ICLR 2025
Reference
(b) Messurement
(c) DPS
(d) DMPS
(e) OT-DE
(I) FIG (ours)
Figure 13: Results for Gaussian deblurring with noise a, = 0.05 on AFHQ-Cat, CelebA-HQ, and SSUN-Bedroom datasets.
23

---

Published as a conference paper at ICLR 2025
ent
ent
ent
Reference
Measurement
(c) DPS
(d) DMPS
(e) OTODE
(I) FIG (ours)
Figure 14: Results for colorization with noise on = 0.05 on AFHQ-Cat, CelebA-HQ and LSUN- Bedroom datasets.
24

---

Published as a conference paper at ICLR 2025
G ADDITIONAL EXPERIMENT RESULTS FOR DIFFUSION MODEL
In this section, we show that our algorithm works for diffusion models. Qualitative results are shown below.
CE
LE
(a) Reference (b) Mcasurement
(c) DPS
(d) DDNM+
(c) DAPS
(I) FIG (ours)
Figure 15: Results for 4x bicubic super-resolution with noise oN = 0.05 on the CelebA-HQ dataset.
(a) Reference (b) Measurement
(c) DPS
(d) DDNM+
(e) DAPS
(f) FIG (ours)
Figure 16: Results for Gaussian deblurring with noise on. 0.05 the CelebA-HQ dataset.
(a) Reference (b) Measurement
(c) DPS
(d) DAPS
(e) FIG (ours)
Figure 17: Results for motion deblurring with noise a, 0.05 on CelebA-HO dataset.
25


Reviews:
[{"review_id": "2YtWUbu62h", "reviewer": "Reviewer_BWWP", "summary": "The authors propose a method, FIG, to pre-train a flow matching or diffusion model to be used as a prior to solve linear inverse problems. FIG considers the conditioning on the measurements in the diffusion process. However, instead of using the direct measurement y, it conditions on the “measurement interpolants” y_t, as it generates x_t. The authors show that for specific choices of the interplant, the relationship between the signal x_t and the measurement y_t hold with the same linear model of the inverse problem (and adjusted additive noise magnitude).\nTheoretical guarantees are provided for the correctness of the proposed optimiser and sampler. FIG is also tested on several inverse problems (SR x4/x16, Gaussian/Motion deblurring, inpainting) and yields state of the art results.", "questions": "1) I would like to know the motivation for the measurement interpolants. I have an intuition of why they are used, but I could not find that in the paper and I would like the authors to explain; More precisely, I am asking what was wrong with the conditioning of other methods and how this new approach makes that right.\n2) I would like to have an explanation for what each of the theorems contributes. Possibly this is connected to the previous point. I suggest to 1) add a brief paragraph after each theorem explaining its practical implications; 2) include a discussion subsection that ties together how the theorems collectively justify or enable the proposed method; 3) provide intuitive explanations or examples to complement the formal mathematical statements.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The idea of measurements interpolants seems intuitively good and is clearly supported by the experiments (with state of the art performance especially in very challenging cases).\nThe authors also introduce the background in both flow matching and diffusion models, and provide some theoretical proof that the proposed method is valid given that Assumption 1 holds.\nThe proposed method in essence is quite simple and easy to implement (as a change to existing uses of pre-trained flow matching or diffusion models).", "weaknesses": "The presentation could be improved to explain much more clearly the motivation behind FIG.\nWhat are the issues that other methods could not address and that FIG does address? \nMy current understanding is that one _can_ condition on the measurement interpolants and then in the conclusions there is a comment that it is easier to create the interpolants than to denoise the signal.\nI would suggest to include a specific subsection that explains:\n1) What limitations exist in how other methods condition on measurements\n2) How measurement interpolants address those limitations\n3) Why conditioning on interpolants is theoretically or empirically advantageous\n\nIt would then be necessary to demonstrate in the experiments how these limitations are not addressed\nby other methods but are addressed by FIG.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "35D3pAEH3k", "reviewer": "Reviewer_V1uU", "summary": "This paper proposed a solution for solving linear inverse problem with flow matching model. The author provides both theoritecal justification and experiment results to support the claim that the proposed method can efficiently handle challenging linear inverse problems.", "questions": "See weakness above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper is well-written with both theoretical claim and experiment results.\nThe model can be deployed in both diffusion and flow matching scenarios.", "weaknesses": "Technically, the paper lacks significant contributions that clearly distinguish it from prior work. My concerns are as follows:\n\n1. **Inverse Problem Approach**: Solving an inverse problem with pre-trained diffusion/flow matching models typically involves classifier guidance by designing a robust $q(y|x_t)$ term, as Bayes’ rule suggests: $\\nabla p(x|y) = \\nabla p(x) + \\nabla p(y|x)$. However, the authors do not propose a novel method for posterior estimation. Equation (16) indicates that this work adheres to the scheme outlined in the [DPS](https://arxiv.org/pdf/2209.14687) paper, using reconstruction guidance.\n\n2. **Application of Existing Methods**: This paper seems primarily focused on adapting DPS to flow matching models, with only minor adjustments to weighting terms.\n\n3. **Theorem 1**: This theorem has been discussed in previous work. For example, the [stochastic interpolant](https://arxiv.org/pdf/2209.15571) paper (Appendix F) provides a proof connecting unconditional flow and score-based models. Similarly, [OT-ODE](https://arxiv.org/pdf/2310.04432) and [C-PG(D/F)M](https://arxiv.org/pdf/2405.17673) offer similar proofs for inverse problem solutions. Thus, Theorem 1 could be moved to the appendix, rather than highlighted as a primary contribution.\n\n4. **Omission of Related Work**: The paper omits mention of highly related work, such as [C-PG(D/F)M](https://arxiv.org/pdf/2405.17673), which specifically addresses efficiency in solving inverse problems using both flow matching and diffusion models. As this work was posted on arXiv in May, it should be acknowledged, and I recommend that the authors review it thoroughly.\n\n5. **Efficiency Claims**: The authors assert, \"Despite promising experimental results, we find prior works struggle to efficiently handle challenging linear inverse problems,\" which is questionable. It seems the key contribution here is an additional iterative update at each ODE solving step (Algorithm 1, lines 7-8), introducing a hyperparameter $K$. For $K > 1$, this approach could increase inference time due to additional gradient calculations. Although this gradient term may not be computationally prohibitive for linear transformations $A$, it could become costly for non-linear transformations. While the authors focus on the linear case, the method could potentially be extended to non-linear cases using heuristics, as seen in [$\\Pi$GDM](https://openreview.net/pdf?id=9_gsMA8MRKQ).\n\n    - **Efficient Inverse Problem Solutions**: Achieving efficiency typically involves (1) reducing NFE (Number of Function Evaluations) without sacrificing quality (e.g., [C-PG(D/F)M](https://arxiv.org/pdf/2405.17673) achieves NFE=5 for inverse problem solutions), and (2) avoiding costly gradient calculations (e.g., [Red-Diff](https://arxiv.org/pdf/2305.04391)).\n\n6. **Evaluation Metrics**: The evaluation could be improved. PSNR and SSIM are no longer reliable metrics for generative model assessment, especially when leveraging pre-trained unconditional models. Instead, metrics like FID and KID should be considered.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a6Ho7BaZlO", "reviewer": "Reviewer_9aCd", "summary": "The paper introduces FIG, a novel algorithm that leverages pre-trained flow matching and diffusion models to solve linear inverse problems. The main contributions include:\nFIG introduces a simple but effective method of guiding the reverse-time sampling process with measurement interpolants.\nThe authors provide a detailed theoretical explanation of FIG, deriving the updating scheme and showing its relation to the posterior log-likelihood.\nFIG is evaluated on several image reconstruction tasks using flow matching and diffusion models, demonstrating its effectiveness and efficiency compared to state-of-the-art baselines.", "questions": "To further strengthen the evaluation and demonstrate the broad applicability of FIG, could the authors please consider comparing their approach against additional methods within the same domain, if available.\nCould the authors discuss potential strategies for extending FIG to handle non-linear inverse problems?\nWhat modifications would be needed to apply FIG to models with latent encodings?\nThe paper mentions the possibility of overfitting when applying FIG. Could the authors elaborate on this aspect, providing insights into how to mitigate this risk?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "FIG is a novel approach to solving linear inverse problems that combines pre-trained generative models with a theoretically justified interpolation-based guidance scheme. It distinguishes itself from existing methods by directly incorporating measurement information during the reverse sampling process.\nThe paper presents a well-defined algorithm with strong theoretical justification. The empirical evaluation is comprehensive, covering various tasks and datasets, with thorough comparisons to relevant baselines.\nThe paper is clearly written and well-structured. The authors effectively explain the core concepts of flow matching and diffusion models, and provide a detailed explanation of FIG. The algorithm is presented in a clear pseudocode, making it easy to follow.\nFIG offers a promising approach for tackling linear inverse problems with pre-trained generative models. Its efficiency, effectiveness, and theoretical grounding suggest a significant contribution to the field.", "weaknesses": "The paper acknowledges that FIG, as currently formulated, is not directly applicable to non-linear inverse problems or models with latent encodings. While this is a valid limitation, the authors could further discuss potential extensions or modifications to address these limitations.\nThe ablation study on the measurement interpolant variance rescaling technique could be further expanded. For example, visualizing the effects of different weight values (w) on the reconstructions could provide a more intuitive understanding of the impact of this parameter.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YIEDTo7v4W", "reviewer": "Reviewer_iHSi", "summary": "This paper introduces an algorithm that efficiently guides reverse-time sampling using measurement interpolants, supported by theoretically grounded frameworks, to tackle image restoration tasks. The performance in the experiment is strong and surpasses previous methods like DDNM+. However, the novelty of the guidance design appears somewhat limited, and the scope of experimental comparisons could be expanded to provide a more convincing evaluation.", "questions": "Performance with more base models. While the current experimental setup effectively demonstrates the validity of your approach, it would be interesting to assess how it performs with more recent open-source diffusion and flow models, like Flux 1.0, SD3, SDXL and other DiTs.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed method is sound and the theoretical framework is well-written with mathematical formulations.\n2. The experimental results seem nice.\n3. The writing is clear and concise.", "weaknesses": "1. The method of using the gradient of reconstruction function to guide the generation direction has been widely explored in recent works. The technique in FIG lacks sufficient novelty in comparison to prior works. \n2. The experiments can be more detailed. The author can involve more related works like GDP[a] in comparisons. \n3. The effectiveness of the measurement interpolation is not verified in the experiments. The author should analyze it with some ablation studies.\n4. The method has not been evaluated in blind scenarios. All degradation settings in this paper are fixed and known. However, real-world images suffer from complex and blind degradations. The author should show the performance on real-world samples.\n\n[a] Generative Diffusion Prior for Unified Image Restoration and Enhancement, CVPR 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)