Title: Improving Diffusion-based Inverse Algorithms under Few-Step Constraint via Linear Extrapolation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Inverse problem

diffusion model

few-step

high order solver
Abstract: Diffusion-based inverse algorithms have shown remarkable performance across various inverse problems, yet their reliance on numerous denoising steps incurs high computational costs.  While recent developments of fast diffusion ODE solvers offer effective acceleration for diffusion sampling without observations, their application in inverse problems remains limited due to the heterogeneous formulations of inverse algorithms and their prevalent use of approximations and heuristics, which often introduce significant errors that undermine the reliability of analytical solvers. 
In this work, we begin with an analysis of ODE solvers for inverse problems that reveals a linear combination structure of approximations for the inverse trajectory. Building on this insight, we propose a canonical form that unifies a broad class of diffusion-based inverse algorithms and facilitates the design of more generalizable solvers.
Inspired by the linear subspace search strategy, we propose Learnable Linear Extrapolation (LLE), a lightweight approach that universally enhances the performance of any diffusion-based inverse algorithm conforming to our canonical form. LLE optimizes the combination coefficients to refine current predictions using previous estimates, alleviating the sensitivity of analytical solvers for inverse algorithms.
Extensive experiments demonstrate consistent improvements of the proposed LLE method across multiple algorithms and tasks, indicating its potential for more efficient solutions and boosted performance of diffusion-based inverse algorithms with limited steps. Codes for reproducing our experiments are available at https://github.com/weigerzan/LLE_inverse_problem.

Improving Diffusion-based Inverse Algorithms under Few-Step Constraint via Linear Extrapolation

Abstract

Diffusion-based inverse algorithms have shown remarkable performance across various inverse problems, yet their reliance on numerous denoising steps incurs high computational costs. While recent developments of fast diffusion ODE solvers offer effective acceleration for diffusion sampling without observations, their application in inverse problems remains limited due to the heterogeneous formulations of inverse algorithms and their prevalent use of approximations and heuristics, which often introduce significant errors that undermine the reliability of analytical solvers. In this work, we begin with an analysis of ODE solvers for inverse problems that reveals a linear combination structure of approximations for the inverse trajectory. Building on this insight, we propose a canonical form that unifies a broad class of diffusion-based inverse algorithms and facilitates the design of more generalizable solvers. Inspired by the linear subspace search strategy, we propose Learnable Linear Extrapolation (LLE), a lightweight approach that universally enhances the performance of any diffusion-based inverse algorithm conforming to our canonical form. LLE optimizes the combination coefficients to refine current predictions using previous estimates, alleviating the sensitivity of analytical solvers for inverse algorithms. Extensive experiments demonstrate consistent improvements of the proposed LLE method across multiple algorithms and tasks, indicating its potential for more efficient solutions and boosted performance of diffusion-based inverse algorithms with limited steps. Codes for reproducing our experiments are available at https://github.com/weigerzan/LLE_inverse_problem.

1 Introduction

Diffusion models have demonstrated remarkable capability in modeling complex data priors [1, 2, 3, 4], which has led to their widespread application in solving inverse problems. Extensive efforts have been devoted to the development of diffusion-based inverse algorithms [5, 6, 7, 8, 9, 10, 11, 12, 13, 14], achieving impressive performance in numerous tasks including inpainting [15], super-resolution [16], deblurring [17], and compressed sensing [18].

One major drawback of diffusion-based inverse algorithms is the need for multiple neural network inferences, resulting in high computational complexity. Directly reducing the number of inference steps often degrades the performance of these inverse algorithms. Therefore, enhancing the performance of diffusion-based inverse algorithms under limited steps has emerged as a critical research

Xto

Sampler Φti

Xo,ti

Corrector hti

Xti-1

xti-y

y

Xti

→

Degradation

xo,ts,., xo,ti+1

LLE

xo,ti

Original dataflow

Xts

Xo

L(·，·)

LLEdataflow

direction, leading to several recent attempts such as shortcut sampling [19, 20], learned correction processes [21], and the introduction of conjugate integrators [22]. Recently, high-order diffusion ODE solvers [23, 24, 25, 26, 27] have shown strong performance in the few-step sampling of diffusion models without observations. However, although conceptually these solvers can also accelerate inverse algorithms, two challenges remain in practice. First, not all inverse algorithms admit an explicit ODE or SDE formulation, necessitating a more general framework to unify their analysis and the design of generalizable solvers. Second, the prevalence of approximations and heuristics in existing methods often leads to substantial estimation errors, undermining the reliability of analytical high-order solvers.

To tackle these challenges, we begin with an analysis of ODE solvers in inverse problems from the perspective of posterior sampling, revealing an equivalent interpretation in the form of linear combinations. Based on the insights, we propose a canonical form for diffusion-based inverse algorithms, which serves as a unifying framework for a broad class of existing methods, facilitating a more systematic understanding and more generalizable development of inverse problem solvers. Building on this canonical form, we design a universal enhancement strategy for diffusion-based inverse algorithms inspired by optimal linear subspace search [27]. Our proposed method, termed Learnable Linear Extrapolation (LLE), learns a set of linear combination coefficients to boost current corrected estimate with previous results, thereby mitigating the unreliability of analytical high-order solvers under large estimation errors or heuristics. For linear inverse problems, we further leverage the structure of the observation function by decoupling the coefficients associated with the range space and null space, leading to enhanced performance for LLE.

LLE is highly lightweight and can be trained with 50 diffusion-generated samples within minutes on a single NVIDIA RTX 3090 GPU, which makes it practical and easy to integrate into existing inverse algorithms. We reformulate nine mainstream diffusion-based inverse algorithms into the proposed canonical form and conduct extensive experiments to validate the effectiveness of LLE in both linear and nonlinear tasks. The results demonstrate that LLE effectively improves the performance of these algorithms under few steps, providing a novel and more general perspective to enhance diffusion-based inverse algorithms in the future.

2 Backgrounds

2.1 Diffusion models

Diffusion models [1, 2, 28] consist of a forward diffusion process that transforms a clean data distribution into a pure Gaussian distribution, and a reverse denoising process that generates clean data from the noise distribution. In this paper, we consider the setting of Variance Preserving (VP) diffusion. Given a target data distribution q ( x 0 ) , the forward diffusion process is modeled by the

following stochastic differential equation (SDE):

where x 0 ∼ q ( x 0 ) and w t is a standard Wiener process. VP diffusion specifics the drift term f ( t ) and the diffusion term g ( t ) as

respectively, where α t is a predefined monotonic parameter controlling the noise level at each timestep, with α 0 = 1 and α T = 0 .

The forward SDE (1) can be exactly reversed as [29]

where w t is the reverse-time standard Wiener process, x T ∼ N ( 0 , I ) , and ∇ x t log q t ( x t ) is known as the (Stein) score function [30, 31]. Practical implementation aims to estimate the unknown score function ∇ x t log q t ( x t ) via a parameterized neural network ϵ θ ( x t , t ) as

which can be learned via score matching [32, 33].

Song et al. [34] demonstrate that the above reverse SDE shares the same marginal distributions as the following reverse-time ordinary differential equation (ODE):

which facilitates the introduction of efficient ODE solvers to enhance sampling efficiency. Using a first-order solver to solve the diffusion ODE (3) leads to the iterative formula of the well-known DDIM sampler [3], which is

2.2 High-order solvers for diffusion ODE

Numerous works have developed a variety of fast ODE solvers for diffusion models [35, 23, 24, 25, 26, 36, 37, 38, 39, 40]. Recent studies have adopted exponential integrators [41] to solve the diffusion ODE as follows

where λ ( t ) = log √ α t / (1 -α t ) . High-order diffusion ODE-solvers [23, 24, 26, 25] leverage the Taylor expansions of ϵ θ ( · , · ) estimated from previous score functions to solve the integral more accurately, which reduces the dependency on small step sizes and enables efficient sampling with fewer steps. Despite differences in derivation and specific formulation, these solvers share a common structure in that x t can be expressed as a linear combination of the initial point and the scores from all previous steps [27], i.e.,

Thus [27] proposes the optimal subspace search method to optimize the coefficients of x t in the subspace shown in (4), aiming to align the trajectory of a few-step solver with a target solver using more steps (e.g., a 999-step DDIM). The learnable solver achieves remarkable performance with few steps while remaining lightweight as it only requires learning a set of linear combination coefficients per timestep. In this paper, we design a lightweight learnable method tailored for inverse algorithms to generally enhance their performance under few steps leveraging similar insights from the optimal subspace search approach.

2.3 Diffusion-based inverse algorithms

Consider the degradation equation

where A is the degradation function, y is the observation, and σ y is the standard deviation of the observation noise. The goal of inverse problems is to recover the original data x 0 from the degraded observation y . The ability of diffusion models to capture complex data priors has inspired a wide range of studies exploring their applications in inverse problems from diverse perspectives.

DDRM [5] defines a new set of variational distributions to approximate the posterior q ( x 0 | y ) and derives an inverse algorithm for linear problems based on projection operations. DDNM [6] directly employs the null-range decomposition and heuristically projects the intermediate estimation of x 0 during sampling onto the hyperplane corresponding to the linear observation. Posterior sampling methods [8, 7, 10] attempt to estimate the conditional score ∇ x t log q t ( x t | y ) to sample from the posterior distribution q ( x 0 | y ) . DiffPIR [9] proposes a plug-and-play approach by guiding the intermediate estimation of x 0 with a proximal point problem and further refining the reverse sampling trajectory. ReSample [12], originally designed for latent diffusion, can be interpreted as optimizing the intermediate estimation via gradient descent to enforce hard data consistency and guiding the reverse sampling trajectory with a resample algorithm. More recently, maximum a posteriori (MAP)based methods such as RED-diff [11] have gained attention, which explicitly express the data prior q ( x 0 ) using diffusion models and solve inverse problems by optimizing max x 0 q ( x 0 | y ) . DAPS [14] adopts Langevin dynamics to enforce consistency with the observations and progressively anneals the timestep to achieve noise reduction.

Despite the effectiveness of diffusion-based inverse algorithms, they typically require 100 to 1000 steps during inference. In this paper, we first establish a canonical form for these algorithms and then propose a lightweight and general approach to enhance their performance under few steps, achieving a better trade-off between performance and computational efficiency.

3 Analysis of ODE solvers in inverse problems

It can be shown that the associated probability flow ODE for sampling from the posterior distribution p ( x 0 | y ) is given by

In the context of inverse problems, the primary focus is to restore the clean signal. Thus we define

To assist our analysis, we assume the temporal discretization for the numerical solver as { t S , t S -1 , . . . , t 0 } ⊆ [0 , T ] with t S > t S -1 > · · · > t 0 = 0 , and S denoting the total number of steps. Then the evolution of x 0 ( t, y ) from t i +1 to t i can be described as the following proposition. Proposition 1. The evolution of x 0 ( t, y ) follows

The detailed derivations are deferred to Appendix B. Note that term ( ∗ ) contains an integral involving x 0 ( τ, y ) , which can be approximated using a standard k -th order Taylor expansion, i.e.,

The high-order derivatives d l x 0 ( t i +1 , y ) d t l i +1 in (9) can be estimated using multi-step methods. In particular, under uniform time discretization t j +1 -t j = h, ∀ j , the forward differences approximation [42, 43] of the l -th derivative yields

which implies that the Taylor expansion approximation for ( ∗ ) is a linear combination of previous steps x 0 ( t j , y ) , for j ≥ i +1 .

Term ( ∗∗ ) involves the score at both time steps t i +1 and t i , which can be reformulated as

where

Note that ˆ x t i serves as the sample calculated by a deterministic DDIM sampler. Consequently, the reformulation (11) can be interpreted as a linear combination of a coarse estimation at the current time step t i and the previous result at time t i +1 . Based on the above analysis on terms ( ∗ ) and ( ∗∗ ) , we arrive at the following conclusion.

Corollary 2. Approximating the evolution of x 0 ( t, y ) from t i +1 to t i can be interpreted as computing a linear combination of an estimate of the current step t i and all previous results at t j for ∀ j ≥ i +1 .

We note that many inverse algorithms introduce random noise, which makes them more naturally aligned with the SDE perspective. We provide a detailed analysis of the SDE case in Appendix B.3 and obtain conclusions similar to those in Corollary 2.

This insight provides a foundation for generalizing diffusion ODE and SDE solvers to a broad class of inverse algorithms. Conceptually, for most algorithms that are difficult to characterize via posterior sampling, we reinterpret x 0 ( t, y ) as an estimate of the clean sample at time t corrected with the observation y . Meanwhile, the linear combination structure enables learning the combination coefficients in a data-driven manner, thus avoiding the failure of analytical coefficients when the posterior score estimation is highly inaccurate or heuristic.

4 Designing unified solvers for inverse algorithms

In this section, we first establish a canonical form that unifies a wide range of diffusion-based inverse algorithms and clarify its connection to the analysis in Section 3. We then propose the LLE method, which is compatible with any algorithm adhering to the canonical form and effectively improves their performance in the few-step regime by learning a set of linear combination coefficients.

4.1 A canonical form for diffusion-based inverse algorithms

As illustrated in Figure 1, our proposed canonical form decomposes each iteration of an inverse algorithm into three modules. Below we detail these modules at step t i for any 1 ≤ i ≤ S .

(1) Sampler generates an estimate of the clean data x 0 ,t i from the noisy input x t i using a pre-trained diffusion model:

(2) Corrector incorporates the observation y to refine the sampled result, ensuring consistency with the measurement:

Algorithm 1 Training of LLE.

Require: Pretrained diffusion model ϵ θ , discrete timesteps t S , t S -1 , . . . , t 0 , inverse algorithm in the canonical form Φ t i , h t i , Ψ t i , reference samples x (1) 0 , . . . , x ( N ) 0 , observation function A , observation noise deviation σ y .

1: for n = 1 to n = N do 2: Get observation y ( n ) as (5); 3: end for 4: Initialize { x ( n ) t S } N n =1 as x ( n ) t S ∼ N ( 0 , I ) ; 5: for i = S to i = 1 do 6: for n = 1 to n = N do 7: Calculate ˆ x ( n ) 0 ,t i = h t i ( Φ t i ( x ( n ) t i ) , A , y ( n ) ) ; 8: end for 9: Optimizing γ t i , 0 , . . . , γ t i ,S -i as (19); 10: for n = 1 to n = N do 11: Calculate ˜ x ( n ) 0 ,t i as (18); 12: Update x ( n ) t i -1 = Ψ t i (˜ x ( n ) 0 ,t i ) ; 13: end for 14: end for 15: return Coefficients γ t i , 0 , . . . , γ t i ,S -i for i = S, . . . , 1

Algorithm 2 Inference with LLE.

Require: Pretrained diffusion model ϵ θ , discrete timesteps t S , t S -1 , . . . , t 0 , observation y , observation function A , observation noise deviation σ y , inverse algorithm in the canonical form Φ t i ,

h

t

i

,

Ψ

t

i

, optimized coefficients

1: Initialize x t S ∼ N ( 0 , I ) ;

2: for i = S to i = 1 do

3: Calculate ˆ x 0 ,t i = h t i ( Φ t i ( x t i ) , A , y ) ;

4: Calculate ˜ x 0 ,t i as (18);

5: Update x t i -1 = Ψ t i (˜ x 0 ,t i ) ;

6: end for

7: return x t 0

Noiser maps the corrected clean sample back to the next noise level for the subsequent iteration:

One immediate advantage of this decomposition is that (14) broadens the definition of x 0 ( t i , y ) in (7), which is applicable to a wider range of inverse algorithms beyond posterior sampling, including but not limited to projection-based and plug-and-play approaches. Another benefit lies in that, by decoupling an inverse algorithm into discrete modules, we can reorganize these modules and directly obtain an estimate at t i from the results at t i +1 as

which provides an effective alternative to the approximation term in (11). Consequently, we can construct unified solvers for any inverse algorithms conforming to the proposed canonical form, requiring only the specification of a set of linear combination coefficients, which we propose to optimize in a data-driven manner.

This modular decomposition is applicable to a broad class of inverse algorithms. In this work, we reformulate nine mainstream inverse algorithms into the proposed canonical form, with detailed decompositions presented in Appendix E.

4.2 Iterative learnable linear extrapolation

Intuitively, the proposed LLE method attempts to enhance the performance of inverse algorithms under few steps by iteratively learning a linear combination of the corrected sample (14) at t i and

γ

t

i

,

0

, . . . , γ

t

i

,S

-

i

for

i

=

S, . . . ,

1

.

all previous results at t j , j > i , to obtain a better estimate ˜ x 0 ,t i . Below, we introduce the training objective of LLE at timestep t i inductively.

Consider a reference training set generated by the diffusion model X = { x (1) 0 , x (2) 0 , . . . , x ( N ) 0 } , where N is the number of reference samples. Suppose we have obtained the estimates from the previous S -i steps, i.e., ˜ x ( n ) 0 ,t k , k = i +1 , . . . , S , n = 1 , . . . , N . Then we replace each ˆ x ( n ) 0 ,t i +1 with ˜ x ( n ) 0 ,t i +1 in (16) and compute an estimate at step t i as

where y ( n ) represents the observation of x ( n ) 0 . Now we optimize a set of linear combination coefficients, denoted as γ t i , 0 , . . . , γ t i ,S -i , and update the estimates at step t i as

Unlike unconditional generation tasks that often require sampling long trajectories as the training targets for learning fast solvers, inverse problems provide explicit supervision in the form of known ground-truth reconstructions. This eliminates the need to sample or store complete trajectories, significantly reducing training cost. As a result, the optimization objective of LLE is formulated as:

where the loss function is chosen as

L MSE is the Mean-Square Error loss that measures the pixel-level discrepancy between samples, while L LPIPS represents the Learned Perceptual Image Patch Similarity [44] loss that quantifies the perceptual discrepancy between two samples. ω is introduced in our loss function (20) as a weight to balance the MSE loss and LPIPS loss. Smaller ω encourages the algorithm to minimize MSE, resulting in higher PSNR, whereas a larger ω biases the algorithm toward minimizing LPIPS, leading to higher perceptual similarity.

Once all coefficients are optimized, we fix them and perform inference in exactly the same manner as the data flow in the training process. The pseudo-codes for the training and inference of LLE are presented in Algorithm 1 and Algorithm 2, respectively.

4.3 Decoupled coefficients for linear problems

For linear inverse problems, we propose using decoupled linear combination coefficients to estimate the null-space and range-space components of a sample with respect to the observation matrix separately. The rationale is that observations impose strong constraints on the range-space component, while the null-space component remains more flexible. Consequently, decoupled sets of linear combination coefficients would perform better to capture the differences between these components.

Consider the observation equation y = Ax 0 + σ y n , where A is the observation matrix. The rangespace and null-space components of a sample x with respect to A are denoted as x ∥ = A † Ax and x ⊥ = ( I -A † A ) x , respectively, where A † represents the Moore-Penrose pseudoinverse of matrix A . At timestep t i , we optimize a set of coefficients γ ∥ t i ,j , γ ⊥ t i ,j , j = 0 , . . . , S -i to estimate ˜ x ( n ) 0 ,t i as

where

The optimization method and objective remain consistent with those described in Section 4.2.

5 Experiments

In this section, we validate the effectiveness of LLE across various inverse algorithms and tasks.

5.1 Experimental setup

Inverse algorithms and hyperparameters. We consider nine mainstream diffusion-based inverse algorithms with distinct formulations and design principles, including DDRM [5], DDNM [6], Π GDM[7], DMPS [10], DPS [8], DiffPIR [9], ReSample [12], RED-diff [11], and DAPS [14]. All these algorithms are decomposed following the canonical form described in Section 4.1, with the details provided in Appendix E. We evaluate the performance of all algorithms under total steps of S = 3 , 4 , 5 , 7 , 10 , 15 . Due to space constraints, we report part of the results in the main text and defer the complete results to Appendix H. For the training of LLE, we fix ω = 0 . 1 and utilize the schedule-free AdamW [45] to optimize the coefficients on 50 diffusion-generated samples. More training details are supplemented in Appendix G.

Datasets and diffusion checkpoints. Two datasets are used: CelebA-HQ [46] and FFHQ [47]. The checkpoint for CelebA-HQ is obtained from [48] and we follow [6] to use the 1k test set. The checkpoint for FFHQ is obtained from [8], and the evaluation follows [12, 14] with 100 randomly selected images from the validation set.

Inverse problems and metrics. We consider five inverse problems in this work: (1) gaussian deblurring (anisotropic), (2) inpainting ( 50 %random), (3) 4 × super-resolution (average pooling), (4) compressed sensing (CS) with the Walsh-Hadamard transform ( 50 % ratio), and (5) nonlinear deblurring. Except for nonlinear deblurring, all other four tasks are linear and we apply the decoupled coefficients method proposed in Section 4.3. The primary metrics include peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM) [49], and Learned Perceptual Image Patch Similarity (LPIPS) [44]. Results of Fréchet Inception Distance (FID) [50] are also included in the Appendix. All of the experiments are performed on a single NVIDIA RTX 3090 GPU.

5.2 Main results

Table 1 presents the performance of five algorithms on four noisy linear tasks from the CelebA-HQ dataset with 3 and 5 steps. Table 2 shows the results of nonlinear deblurring task on CelebA-HQ and FFHQ under both noiseless and noisy settings with 3 and 10 steps. The LLE method consistently improves the performance across different tasks and algorithms.

Figure 2 illustrates the PSNR and LPIPS metrics for four algorithms on linear tasks from FFHQ with steps varying from 3 to 15 . LLE consistently achieves superior objective and perceptual metrics

4

6

8

10

12

14

Steps

25.5

26.0

26.5

27.0

27.5

PSNR ( )

Deblur (aniso)

(

σ

y

=0)

RED-diff

4

6

8

10

12

14

Steps

20.0

22.5

25.0

27.5

30.0

32.5

Inpainting

(

σ

y

=0

.

05)

ReSample

4

6

8

10

12

14

Steps

26

27

28

29

4

×

SR

(

σ

y

=0

.

05)

DAPS

4

6

8

10

12

14

Steps

17

18

19

20

21

22

CS

50% (

σ

y

=0)

DDRM

0.27

0.28

0.29

0.30

0.31

0.32

0.33

0.2

0.3

0.4

0.5

0.6

0.275

0.300

0.325

0.350

0.375

0.400

0.35

0.40

0.45

0.50

0.55

LPIPS (

)

Original PSNR

LLE PSNR

Original LPIPS

LLE LPIPS

under different steps. Some qualitative results are provided in Figure 3, and more visualization of the results is postponed to Appendix I.

A notable observation is that LLE tends to show more significant improvements when the original algorithm performs suboptimally, while achieving comparable or slightly better results when the original algorithm already performs satisfactorily. This aligns with our expectations since LLE searches for better solutions in a linear subspace spanned by previous steps. When an inverse algorithm's trajectory is close to optimal, LLE is expected to approximate this original trajectory.

5.3 Ablation studies

In this section, we present the ablation studies of the decoupled coefficients for linear problems and the effect of different ω in LLE. We defer more ablation studies and analysis of the results to Appendix D, including the visualizations of the learned coefficients, the training and inference time of LLE, as well as the cross-dataset and cross-task generalization results.

Decoupled coefficients for linear observations.

Table 3 compares the performance of LLE on linear tasks using decoupled coefficients versus a single set of coefficients. All the algorithm uses LLE and 3 steps. Decoupled coefficients consistently achieve better results, validating the advantage of handling the null-space and rangespace components separately.

Π

GDM - Orig.

Π

GDM - LLE

DPS - Orig.

3 Steps

DPS - LLE

4 Steps

5 Steps

7 Steps

10 Steps

15 Steps

y

x

0

GU

GU

Effect of ω . Figure 4 presents the PSNR and LPIPS metrics of the LLE method when varying the hyperparameter ω in the range of 0 to 0.5. Both tasks are noiseless and the algorithms use 3 steps. Existing works [11, 13] have observed that objective metrics (PSNR) and perceptual quality (LPIPS) often trade off against each other in certain tasks. Figure 4 indicates that LLE provides a straightforward mechanism to control this balance by adjusting ω .

6 Conclusion

In this paper, we design a unified canonical form for existing diffusion-based inverse algorithms and propose the LLE method to enhance the performance of these algorithms under few steps. Extensive experiments validate the effectiveness of LLE across nine different inverse algorithms and five tasks on two datasets. LLE learns improved solvers for diffusion-based inverse algorithms in a lightweight approach, providing novel insights for the design and acceleration of diffusion-based inverse algorithms in the future.

Limitations and Future Work. One limitation of the LLE method is that its search space is confined to the linear subspace spanned by all previous steps. Future work could explore the introduction of nonlinear methods to further improve the algorithm's performance while maintaining lightweight and universal as LLE. Additionally, the canonical form proposed in this paper offers a flexible framework for combining and designing inverse algorithms. Research could focus on leveraging this framework to design more effective algorithms in the future. Moreover, we note that the CelebA-HQ and FFHQ datasets used in this paper have demographic imbalances, which may cause our results to fail to generalize to underrepresented groups. A more diverse and balanced dataset is needed in the field to ensure fairness across factors such as ethnicity, gender, and age. A more representative diffusion model trained on such data is also preferred to better support real-world applications.

References

Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning , pages 2256-2265. PMLR, 2015.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840-6851, 2020.

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations , 2021.

Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems , 34:8780-8794, 2021.

Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Advances in Neural Information Processing Systems , 35:23593-23606, 2022.

Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. In The Eleventh International Conference on Learning Representations , 2023.

Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations , 2023.

Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations , 2023.

Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and-play image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , pages 1219-1229, June 2023.

Xiangming Meng and Yoshiyuki Kabashima. Diffusion model based posterior sampling for noisy linear inverse problems. In Vu Nguyen and Hsuan-Tien Lin, editors, Proceedings of the 16th Asian Conference on Machine Learning , volume 260 of Proceedings of Machine Learning Research , pages 623-638. PMLR, 05-08 Dec 2025.

Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective on solving inverse problems with diffusion models. In The Twelfth International Conference on Learning Representations , 2024.

Bowen Song, Soo Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen. Solving inverse problems with latent diffusion models via hard data consistency. In The Twelfth International Conference on Learning Representations , 2024.

Jiawei Zhang, Jiaxin Zhuang, Cheng Jin, Gen Li, and Yuantao Gu. Unleashing the denoising capability of diffusion prior for solving inverse problems. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems , volume 37, pages 47636-47677. Curran Associates, Inc., 2024.

Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing. arXiv preprint arXiv:2407.01521 , 2024.

Marcelo Bertalmio, Guillermo Sapiro, Vincent Caselles, and Coloma Ballester. Image inpainting. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques , pages 417-424, 2000.

Sung Cheol Park, Min Kyu Park, and Moon Gi Kang. Super-resolution image reconstruction: a technical overview. IEEE signal processing magazine , 20(3):21-36, 2003.

Phong Tran, Anh Tuan Tran, Quynh Phung, and Minh Hoai. Explore image deblurring via encoded blur kernel space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11956-11965, 2021.

Le Wang and Shengmei Zhao. Fast reconstructed and high-quality ghost imaging with fast walsh-hadamard transform. Photonics Research , 4(6):240-244, 2016.

Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12413-12422, 2022.

Gongye Liu, Haoze Sun, Jiayi Li, Fei Yin, and Yujiu Yang. Accelerating diffusion models for inverse problems through shortcut sampling. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence , IJCAI '24, 2024.

Hanyu Chen, Zhixiu Hao, and Liying Xiao. Deep data consistency: a fast and robust diffusion model-based solver for inverse problems. arXiv preprint arXiv:2405.10748 , 2024.

Kushagra Pandey, Ruihan Yang, and Stephan Mandt. Fast samplers for inverse problems in iterative refinement models. Advances in Neural Information Processing Systems , 37:26872-26914, 2024.

Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems , 35:5775-5787, 2022.

Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-solver++: Fast solver for guided sampling of diffusion probabilistic models, 2023.

Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipc: A unified predictor-corrector framework for fast sampling of diffusion models. Advances in Neural Information Processing Systems , 36:49842-49869, 2023.

Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics. Advances in Neural Information Processing Systems , 36:55502-55542, 2023.

Zhongjie Duan, Chengyu Wang, Cen Chen, Jun Huang, and Weining Qian. Optimal linear subspace search: Learning to construct fast and high-quality schedulers for diffusion models. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management , pages 463-472, 2023.

Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in neural information processing systems , 34:21696-21707, 2021.

Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications , 12(3):313-326, 1982.

Kacper Chwialkowski, Heiko Strathmann, and Arthur Gretton. A kernel test of goodness of fit. In International conference on machine learning , pages 2606-2615. PMLR, 2016.

Qiang Liu, Jason Lee, and Michael Jordan. A kernelized stein discrepancy for goodness-of-fit tests. In International conference on machine learning , pages 276-284. PMLR, 2016.

Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research , 6(4), 2005.

Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation , 23(7):1661-1674, 2011.

Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in neural information processing systems , 34:1415-1428, 2021.

Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In The Eleventh International Conference on Learning Representations , 2023.

Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around 5 steps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7777-7786, 2024.

Shuchen Xue, Mingyang Yi, Weijian Luo, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhi-Ming Ma. Sa-solver: Stochastic adams solver for fast sampling of diffusion models. Advances in Neural Information Processing Systems , 36:77632-77674, 2023.

Gen Li, Yu Huang, Timofey Efimov, Yuting Wei, Yuejie Chi, and Yuxin Chen. Accelerating convergence of score-based diffusion models, provably. In International Conference on Machine Learning , pages 27942-27954. PMLR, 2024.

Gen Li and Changxiao Cai. Provable acceleration for diffusion models under minimal assumptions. arXiv preprint arXiv:2410.23285 , 2024.

Gen Li, Yuchen Zhou, Yuting Wei, and Yuxin Chen. Faster diffusion models via higher-order approximation. arXiv preprint arXiv:2506.24042 , 2025.

Marlis Hochbruck and Alexander Ostermann. Exponential integrators. Acta Numerica , 19:209-286, 2010.

Bengt Fornberg. Generation of finite difference formulas on arbitrarily spaced grids. Mathematics of computation , 51(184):699-706, 1988.

Endre Süli and David F Mayers. An introduction to numerical analysis . Cambridge university press, 2003.

Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 586-595, 2018.

Aaron Defazio, Xingyu Yang, Ahmed Khaled, Konstantin Mishchenko, Harsh Mehta, and Ashok Cutkosky. The road less scheduled. Advances in Neural Information Processing Systems , 37:9974-10007, 2025.

Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In International Conference on Learning Representations , 2018.

Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4401-4410, 2019.

Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11461-11471, 2022.

Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing , 13(4):600-612, 2004.

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems , 30, 2017.

Neta Shaul, Uriel Singer, Ricky T. Q. Chen, Matthew Le, Ali Thabet, Albert Pumarola, and Yaron Lipman. Bespoke non-stationary solvers for fast sampling of diffusion and flow models. In Forty-first International Conference on Machine Learning , 2024.

Daniel Watson, William Chan, Jonathan Ho, and Mohammad Norouzi. Learning fast samplers for diffusion models by differentiating through sample quality. In International Conference on Learning Representations , 2021.

Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your steps: Optimizing sampling schedules in diffusion models. In International Conference on Machine Learning , pages 42947-42975. PMLR, 2024.

Defang Chen, Zhenyu Zhou, Can Wang, Chunhua Shen, and Siwei Lyu. On the trajectory regularity of ODE-based diffusion sampling. In Forty-first International Conference on Machine Learning , 2024.

Vinh Tong, Dung Trung Hoang, Anji Liu, Guy Van den Broeck, and Mathias Niepert. Learning to discretize denoising diffusion ODEs. In The Thirteenth International Conference on Learning Representations , 2025.

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 , 2020.

Kendall Atkinson, Weimin Han, and David E Stewart. Numerical solution of ordinary differential equations . John Wiley & Sons, 2009.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming-Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference on Machine Learning , pages 32483-32498. PMLR, 2023.

Xiang Li, Soo Min Kwon, Shijun Liang, Ismail R Alkhouri, Saiprasad Ravishankar, and Qing Qu. Decoupled data consistency with diffusion purification for image restoration. arXiv preprint arXiv:2403.06054 , 2024.

Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei-Hsiang Liao, Yuki Mitsufuji, J Zico Kolter, Ruslan Salakhutdinov, and Stefano Ermon. Manifold preserving guided diffusion. In The Twelfth International Conference on Learning Representations , 2024.

Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In The Twelfth International Conference on Learning Representations , 2024.

Zihui Wu, Yu Sun, Yifan Chen, Bingliang Zhang, Yisong Yue, and Katherine Bouman. Principled probabilistic imaging using diffusion models as plug-and-play priors. Advances in Neural Information Processing Systems , 37:118389-118427, 2024.

Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkottai. Solving linear inverse problems provably via posterior sampling with latent diffusion models. Advances in Neural Information Processing Systems , 36:49960-49990, 2023.

Litu Rout, Yujia Chen, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Beyond first-order tweedie: Solving inverse problems using latent diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9472-9481, 2024.

Tongda Xu, Ziran Zhu, Jian Li, Dailan He, Yuanyuan Wang, Ming Sun, Ling Li, Hongwei Qin, Yan Wang, Jingjing Liu, et al. Consistency model is an effective posterior sample approximation for diffusion inverse solvers. arXiv preprint arXiv:2403.12063 , 2024.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684-10695, 2022.

Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference on Machine Learning , pages 32211-32252. PMLR, 2023.

Hengkang Wang, Xu Zhang, Taihui Li, Yuxiang Wan, Tiancong Chen, and Ju Sun. Dmplug: A plug-in method for solving inverse problems with diffusion models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems , volume 37, pages 117881-117916. Curran Associates, Inc., 2024.

Phong Tran, Anh Tuan Tran, Quynh Phung, and Minh Hoai. Explore image deblurring via encoded blur kernel space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11956-11965, 2021.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The claims made in the abstract and introduction are consistent with this paper's main contributions and scope, and have been verified through extensive experiments and ablation studies.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: See Section 6.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The code is included in the supplemental matrials and the experiment details are presented in Appendix G.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code and instructions are included in the supplemental materials.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: All of the experiment details and implemention details are presented. See Appendix G.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Following the convention of inverse problem research, we report the performance comparison of all algorithms with a fixed random seed.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The computer resources needed to reproduce the experiments are provided in Section 5 and Appendix D.4. All of the experiments are conducted on a single NVIDIA 3090 GPU.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have read the NeurIPS Code of Ethics in detail and conformed with it in this paper.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Facial datasets such as CelebA-HQ and FFHQ are used in our experiments. While our method is general-purpose, we acknowledge that related technologies may potentially be misused, for example in DeepFake scenarios. Improved techniques such as digital watermarking and DeepFake detection may need to be further developed to mitigate the potential societal risks posed by such misuse.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No datasets or models are proposed and there is no such risk of misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All databases, models, and tookits used in this paper are cited from the original papers.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The code provided in the supplementary materials is anonymized and well documented with the running details.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: We did not use any large language model in the development of our core methods.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Appendix

In the appendix, we provide additional discussions and details that are omitted in the main paper due to space limitations. Appendix A reviews existing methods for solving inverse problems with diffusion models under few steps, highlighting how our work is complementary to these approaches and has the potential to be combined with them. We also discuss the relation of LLE to non-stationary solvers and its possible synergy with learnable timesteps. Appendix B presents the derivations of the probability flow ode for posterior sampling, the main results shown in the main text, and also the analysis of solvers for inverse algorithms in the SDE form. Appendix C provides additional experiments on the ImageNet dataset as well as additional nine baselines, further demonstrating the generality of LLE on more challenging tasks as well as its applicability to latent-space and consistency model-based methods. Appendix D supplements more analysis and ablation studies omitted in the main text due to space constraints. Appendix E elaborates on the detailed decomposition under the proposed canonical form of all inverse algorithms adopted in our experiments. Appendix F introduces our tailored design of LLE for DDRM and DDNM on noisy linear problems, which is specifically adapted to better align with both algorithms. Appendix G describes additional experimental details, including inverse problem settings, the hyperparameter configurations of the inverse algorithms, and the optimization methods for LLE training. Appendix H provides comprehensive results across CelebA-HQ and FFHQ datasets using 3, 4, 5, 7, 10, and 15 steps, covering four or five tasks (depending on the algorithm's ability to handle nonlinear problems) in both noiseless and noisy scenarios. Appendix I presents additional qualitative results.

A More related works

A.1 Other methods for few-step diffusion-based inverse algorithms

Recently, several works [19, 20, 21, 22] have attempted to design algorithms capable of solving inverse problems with diffusion models under few steps. We provide a detailed explanation of these methods and describe their differences from this work.

Chung et al. [19] point out that it is unnecessary to sample from pure Gaussian noise for solving inverse problems. Instead, starting from a lower noise level is more efficient by initializing with a noised version of the degraded image. Liu et al. [20] adopt a similar idea and refines it by using DDIM inversion to obtain the noisy image from the degraded observation as initialization. Both methods can essentially be seen as utilizing only a short segment of the diffusion sampling chain close to the clean image rather than the full sampling chain. In our work, we default to the use of the complete sampling chain for inverse algorithms. The proposed LLE method can also be applied to these methods, which only requires adjusting the sampling interval of timesteps t S , . . . , t 0 and using the corresponding initialization methods. Moreover, it can be verified that the algorithms in [19] and [20] can both be decomposed into our proposed canonical form.

Chen et al. [21] introduce deep data consistency (DDC), which employs an additional UNet to learn the data consistency process. We note that this is equivalent to using a UNet as the Corrector in our canonical form, making LLE directly compatible with DDC. Chen et al. [21] also observe that existing methods essentially are equivalent to imposing constraints on x 0 ,t . While [21] primarily focuses on the data consistency step, our proposed canonical form systematically modularizes the diffusion model-based inverse algorithms into three components, providing a more comprehensive framework that fits well with our LLE method.

Pandey et al. [22] propose enhancing the efficiency of inverse problem solving by integrating the Conjugate Integrators method from diffusion sampling. They design a specialized Conjugate Integrator for Π GDM, named CΠ GDM, which transforms the solving process into a space that is more flexible and more friendly to inverse problems, thereby reducing the solver's dependency on small step sizes. The solver in CΠ GDM remains first-order, and the concept of LLE can still be applied to introduce higher-order information to further improve its performance under few steps. Additionally, the observation-based correction step in CΠ GDMcan also been transformed to process in the original space by using projections between the original and transformed spaces, making it well-suited for our canonical form and LLE method.

In summary, existing few-step diffusion-based inverse algorithms are complementary to the proposed LLE method, and all fit within our canonical form. Combining LLE with these approaches has the

potential to further enhance the performance of inverse algorithms under few steps, and we leave the exploration of these combinations for future work.

A.2 Relation to non-stationary solvers

Recently, [51] formulates non-stationary solvers as a general family of iterative schemes where the update at each step is expressed as a linear combination of all previous estimates and their corresponding velocity fields (or equivalently, score functions). Let x t i denote the current estimate at timestep t i and with the predicted velocity v t i , then a non-stationary solver can be written as

where the coefficients { a t i ,j , b t i ,j } are adaptively determined and can vary across timesteps. From this perspective, our proposed LLE can be regarded as a data-driven realization of non-stationary solvers tailored for diffusion-based inverse problems. By learning the combination weights directly from data, LLE adapts the solver dynamics to the empirical characteristics of the ideal trajectory.

A.3 Combination with learnable timesteps

Recent works [52, 53, 54, 55] have proposed to improve the performance of diffusion solvers by learning the timestep discretization. Specifically, the timesteps can be parameterized and optimized under the supervision of a teacher solver's trajectory to obtain an optimal scheduler. We note that this approach and our proposed LLE represent two complementary perspectives for enhancing few-step solvers. Incorporating a learnable scheduler into the LLE framework could potentially lead to further performance gains for diffusion-based inverse algorithms in the few-step regime. We leave this as a promising direction for future work.

B Derivations of the main results

B.1 Probability flow ODE for posterior sampling

The derivation is similar to the probability flow ODE for diffusion sampling without observation in [56]. Recall the reverse time SDE for posterior sampling

where d w t denotes the reverse-time Wiener process. Denote τ = T -t , then we can rewrite (24) as

Its corresponding Fokker-Planck equation follows

with the fact that the marginal distribution of x at time τ is p ( x | y , τ ) . Note that

Thus (26) becomes

which is exactly the Liouville equation for reverse-time probability flow ode

B.2 Derivations of Proposition 1

By definition, we have

Thus (6) can be rewritten as

Given x t i +1 , the solution to the semilinear ODE (31) is known to be [57]

where

Note that

Thus (32) is

Rearrange (35) yields

B.3 The case with SDE

Empirical evidence has shown that the injection of stochastic noise plays a critical role in the effectiveness of diffusion-based inverse algorithms. Accordingly, we extend our discussion to cover the SDE-based formulation in this section.

Considering the SDE for posterior sampling (24), we may follow [24] to solve the SDE as

where

We have

Then the Itô integral is computed as

where

Thus with (30) one have

where η ( τ ) = α τ (1 -α τ ) 2 g 2 ( τ ) .

Similarly, the integral of x 0 ( τ, y ) may be approximated by the (informal) Taylor expansion, with high-order derivatives estimated from previous-step results. An estimate ˆ x t i can be sampled using a standard DDPM update, i.e.,

If the sampled noise is exactly the same ϵ in (43), then 1 ⃝ + 2 ⃝ in (43) is

which leads to a similar conclusion as Corollary 2.

C Additional results and comparisons

In this section, we present additional experimental results on the ImageNet dataset, as well as the performance of LLE over nine additional baseline algorithms.

C.1 Results on ImageNet

Here we supplement the results of Π GDM, RED-diff, DPS, and ReSample on the ImageNet dataset [58] with 4 steps and 10 steps. Following [12, 14], we sample 100 images from the ImageNet test set for evaluation. For noisy tasks, we add Gaussian noise with σ = 0 . 1 with respect to data range [-1, 1]. The ADM pre-trained checkpoint in [4] is adopted. The results are shown in Table 4, which demonstrates that LLE consistently improves performance across all methods in the few-step setting on ImageNet. This confirms that LLE is effective and broadly applicable on more complex datasets.

C.2 Additional baselines

To further validate the general applicability of LLE, we include nine additional inverse algorithms as baselines, including five pixel-space methods (LGD [59], DCDP [60], MPGD [61], FPS [62], and SGS-EDM [63]), three latent-space methods (Latent-DPS [12], PSLD [64], and STSL [65]), and one consistency model-based method (CMInversion [66]). For pixel-space methods, we follow the same validation settings on FFHQ as used in the main content. For latent-space methods, we perform evaluation on the FFHQ256 dataset using the pretrained LDM checkpoint provided by [67]. For CMInversion, we adopt the pretrained consistency model from [68], which was trained for class-conditional generation on ImageNet-64 with ℓ 2 loss. We randomly sample 100 images from class 0 ('tench') in the ImageNet validation set and resize them to 64 × 64 for evaluation. The corresponding results are presented in Table 5, Table 6, and Table 7. The results indicate that LLE consistently improves the performance of all nine baselines in the few-step regime, further confirming its generality and broad applicability.

We now elaborate on how latent diffusion-based and consistency model-based inverse algorithms can be incorporated into the LLE framework. There are no conceptual difficulties in extending the canonical form and LLE from pixel space to latent space. The observation model with LDMs can be formulated as

where D ( · ) denotes the decoder of the VAE, A ( · ) is the observation operator, and z 0 is the latent. This corresponds to a nonlinear inverse problem with respect to z 0 , since the composed operator A◦D is highly nonlinear. Thus the theoretical foundation of linear extrapolation in LLE still holds, as it does not rely on specific assumptions about the observation function. The only modification required for training LLE in the latent space is a minor adjustment to the loss function (20), which becomes

since LPIPS is computed in the pixel space.

Regarding consistency model-based methods, though consistency models do not provide an explicit sampling trajectory, they can always be viewed as one-step efficient samplers. Inverse algorithms built upon consistency models (e.g., CMInversion [68]) also follow a sampling-correction-noising paradigm, which naturally fits into our canonical form and can therefore benefit from the LLE framework as well.

The detailed decomposition of all nine algorithms above into our canonical form is supplemented in Appendix E.

D More ablation studies and additional analysis

In this section, we supplement the ablation studies and discussion that are omitted in the main text due to space constraints.

D.1 Visualization of the LLE coefficients

Here, we present and analyze the visualization of the learned coefficients from several algorithms trained on the CelebA-HQ dataset. Figure 5, Figure 6, Figure 7, and Figure 8 correspond to DDNM on noisy inpainting, Π GDMon noisy compressive sensing, DPS on noiseless super-resolution, and DiffPIR on noiseless anisotropic deblurring, respectively. All visualizations are based on 7-step inference. The vertical axis indicates the current step in the inverse process, and each row represents the linear combination weights applied to all previous steps. The left subfigure shows the coefficients for range space, while the right subfigure shows the coefficients for null space.

Different algorithms yield distinct patterns of the learned linear combination coefficients, which is attributed to their underlying different architectures and designs. At the same time, several shared characteristics emerge across methods, aligning well with the design philosophy of our framework.

First, we observe a clear difference between the range space and null space coefficients. This supports our hypothesis that learning separate coefficients allows the algorithm to better capture the varying strengths of the observation constraints in the range and null spaces. Notably in DDNM, the range space coefficients are almost entirely concentrated along the diagonal with values close to one, while off-diagonal elements remain near zero. This is consistent with the design philosophy of DDNM: DDNM's correction step explicitly projects the sample onto the hyperplane defined by the observation, thereby enforcing a hard constraint in the range space while preserving greater flexibility in the null space. Some non-zero off-diagonal coefficients in the range space are attributed to the observation noise. The other three methods do not employ explicit projection operations. Consequently, these methods impose relatively less stringent constraints on the range space compared to DDNM, which may result in estimation discrepancies that LLE could compensate for by learning a more effective linear combination.

Range Coefficients (Ill)

Null Coefficients (L)

66x

1.0

1.02

0.78

Z58x

1.0

-0.00

1.00

0.8

-0.17

0.98

O1LX

0.8

Current Step

-0.00-0.001.00

Current Step

-0.13

0.09

1.17

0.6

X568

-0.00

-0.00-0.001.00

89Sx

0.01

-0.21

0.07

1.14

0.6

X284X426

X284X426

-0.00

0.00

-0.00-0.001.00

0.4

0.01

-0.00-0.160.05

1.12

0.4

-0.00

0.00

-0.00-0.000.00

1.00

0.01

0.01

-0.04-0.03 0.11

0.96

0.2

0.2

X

0.05

0.05

0.05

0.05

0.05

0.05

0.73

0.00

0.00

-0.02

0.05

0.13

0.12

0.74

0.0

0.0

0.03

0.03

0.03

0.03

0.03

0.03

-0.11

0.91

0.00

-0.01

0.02

-0.06-0.02-0.03-0.09

1.17

-0.2

X994

X852

X710

X568

X426

X284

X142

Xo

X994

X852

X710

X568

X426X284

X142

Xo

Referenced Step

Referenced Step

Second, we observe that the learned coefficients are not always dominated by diagonal elements. For instance, in Π GDMat step 770 and in DiffPIR at steps 142 and 0 , the model assigns larger weights to earlier steps rather than the current one. This suggests that LLE may, in certain cases, favor earlier estimates over the most recent one. Such behavior reveals an implicit form of regularization, akin to 'early stopping', within the dynamics of LLE, echoing a similar strategy reported in prior studies [69]. This demonstrates LLE's ability to dynamically assess the reliability of predictions and turn to past estimates when appropriate, thereby enhancing robustness.

Moreover, a consistent pattern across different inverse algorithms is that the LLE coefficients tend to concentrate around the diagonal and its immediate neighborhood. This suggests that more recent steps exert a stronger influence, while earlier steps contribute less as their estimates become increasingly outdated. According to Equation (10), elements farther from the diagonal correspond to higher-order derivatives of the sampling trajectory. Since most sampling trajectories are expected to be smooth, the high-order derivatives gradually decay to zero as the order grows, which results in the weight matrix being dominated by the diagonal and first off-diagonal elements. Such behavior is also intuitive, since earlier predictions are typically less accurate and thus less informative for refining later outputs. Building on this insight, one possible improvement is to introduce a limited look-back window for LLE, which could reduce both computational and memory overhead during training and inference. This modification may further improve the scalability of LLE, particularly in inverse problems involving longer sampling trajectories. We leave a deeper investigation to future work.

D.2 Cross-task and cross-dataset generalization

Although conceptually LLE requires training with the same prior distribution and inverse task as inference, we also evaluate its generalizability across datasets and tasks. Table 8 reports LLE's performance with cross-dataset training between FFHQ and CelebA-HQ, Table 9 shows the crossdataset results between FFHQ and ImageNet, and Table 10 presents performance with corss-task training.

We observe that LLE trained on one dataset generalizes well to another, even in the more challenging ImageNet-to-FFHQ cross-domain setting. Cross-dataset training achieves comparable performance to training on the matching dataset, and occasionally even surpasses it. This indicates that LLE possesses a certain degree of cross-dataset generalization capability.

Range Coefficients (ll)

1.2

1.2

0.77

0.77

X852

1.0

758x

1.0

-0.04

0.79

0.01

0.78

0.8

0.8

Current Step

0.01

0.77

0.16

Current Step

-0.00

0.77

0.09

89Sx

895X

0.6

0.01

0.05

-0.09

0.81

0.6

0.01

-0.06-0.07

0.80

X284X426

X284X426

0.4

-0.00-0.04

0.07

0.13

0.81

0.4

0.01

-0.03

0.03

-0.02

0.92

0.00

-0.01

0.01-0.05-0.051.10

0.2

0.01

-0.05

0.04-0.14-0.07

1.21

0.2

X142

-0.00-0.000.00-0.02-0.10-0.081.20

0.00

-0.040

0.03

-0.07

-0.17

0.07

1.17

0.0

-0.0

-0.00-0.000.00

-0.01

0.02

-0.21

0.10

1.10

0.00

-0.040

0.05

-0.01

0.01

-0.29

0.14

1.14

-0.2

-0.2

X994

X852

X710

X568

X426X284

X142

Xo

X994

X852

X710

X568

X426X284

X142

Xo

ReferencedStep

ReferencedStep

Range Coefficients (Il)

X994

0.77

0.77

758x

1.0

1.0

-0.03

0.78

-0.00

0.77

Current Step

OTLX

0.02

0.02

0.78

0.8

Current Step

O1LX

-0.00

0.00

0.77

0.8

895x

0.00

0.06

-0.02

0.80

0.6

895x

0.00

0.01

-0.01

0.78

0.6

X426

9

0.00

0.00

0.05

0.06

0.81

0.00

-0.01

0.03

0.10

0.78

0.4

0.4

0.00

-0.00

0.00-0.01-0.021.02

-0.00

0.00

-0.01-0.000.02

0.87

0.2

-0.2

0.00

-0.00-0.00-0.00-0.04-0.151.19

-0.00

0.00

-0.01

-0.02-0.07-0.201.17

-0.0

-0.0

0.00

-0.00-0.00-0.00

0.00

0.01

-0.14

1.13

-0.00-0.00

0.00

0.01

0.03

-0.02

0.06

1.08

--0.2

X994

X852

Referenced Step

X710

X568

X426

X284

X142

Xo

X994

X852

Referenced Step

X710

X568

X426X284

X142

Xo

When the training and testing tasks differ, LLE performance generally degrades. However, in some cases, the cross-task trained LLE coefficients still outperform the original baseline without LLE. For instance, LLE trained on super-resolution or compressive sensing can improve performance on inpainting. This suggests that certain inverse problems may share similar underlying structures, potentially leading to similar optimal coefficient patterns. Future work could further explore these shared structures to enhance LLE's generalization across tasks.

D.3 Sensitivity to the hyperparameters of original algorithms

It is well known that some inverse algorithms require careful tuning of hyperparameters for each task and dataset, for example, the step size ζ in DPS and the regularization weight λ in RED-diff. Small perturbations in these hyperparameters can often lead to significant performance degradation.

We note that the linear combination mechanism in LLE can be interpreted as equivalently involving a learnable scaling factor, which may reduce the sensitivity of the original algorithm to its hyperparameters. We conduct an experiment analyzing the performance of DPS and RED-diff with and without LLE under varying hyperparameter settings on the noiseless compressed sensing task from FFHQ, with hyperparameter ζ or λ ranging from 0 . 2 to 0 . 8 . The performance is shown in Figure 9. The results demonstrate that LLE mitigates the influence of hyperparameter variations and consistently achieves better performance, indicating that LLE can potentially reduce the reliance on precise hyperparameter tuning.

Range Coefficients (Il)

X994

0.87

0.77

1.0

1.0

X852

Z58x

0.05

0.95

-0.08

0.79

OTLX

0.8

01LX

0.8

Current Step

0.00

0.14

0.86

Current Step

0.00

0.08

0.80

895x

-0.00

0.05

0.14

0.81

0.6

895x

0.00

0.05

0.11

0.79

0.6

X284X426

X426

-0.00

0.01

0.09

0.11

0.79

- 0.4

0.00

0.03

0.09

0.08

0.78

0.4

0.00

-0.01

-0.01

0.07

0.75

0.20

0.00

-0.00-0.010

0.07

0.76

0.17

0.2

0.2

X142

0.00

0.00

0.01

0.04

-0.17

0.90

0.21

0.00

0.01

0.03

0.06

-0.19

0.86

0.19

-0.0

-0.0

--0.00

0.00

0.01

-0.05

5-0.18-0.09

1.15

0.17

0.00

0.00

0.01

-0.04-0.18-0.06

1.16

0.10

X994

X852

X710

X568

X426X284X142

Xo

X994

X852

X710

X568

X426X284X142

Xo

ReferencedStep

ReferencedStep

D.4 Training cost, inference time, and momery footprint

Table 11 presents the training time of LLE for DDNM, DPS, DiffPIR, and DAPS on noiseless inpainting tasks from CelebA-HQ. LLE requires only 2 to 20 minutes for training, depending on the task, the complexity of the original algorithm, and the number of steps. This demonstrates the lightweight advantage of our LLE method.

We also compare the inference time of each algorithm with and without the LLE module. Table 12 presents the inference time on the CelebA-HQ 1k test set for the inpainting task. The results indicates that incorporating LLE introduces negligible additional computational overhead during inference. The efficiency of LLE stems from the simplicity of its operation, which only involves computing a linear combination of previous samples without introducing any complex transformations.

Regarding the memory footprint, here we further report the GPU memory usage during inference (in MB) of DDNM and DPS with and without LLE in Table 13. We measure the CUDA peak allocated memory, and we account only for the memory used by model parameters, data, and additional storage required by the algorithm. The results show that LLE introduces only a few to several dozen MB of additional memory overhead, even under the 15-step setting. This is consistent with our expectations: during inference, LLE only needs to store a few 256 × 256 RGBimages from previous steps, typically around 0.7 MB per image, which is negligible compared to model weights or other algorithm-specific overhead (e.g., back-propagation in DPS).

D.5 Analysis with default steps

Though our primary focus is on performance under limited steps, we also present the 100-step results of DDNM on CelebA-HQ in Table 14 as a reference. These results further indicate that LLE

is particularly effective when there is a large performance gap between the few-step setting and the optimal setting. For example, in the inpainting and compressive sensing tasks, the difference between the few-step performance and the default-step (100 steps) performance is substantial, thus LLE provides significant improvements, with PSNR gains ranging from 1 dB to 8 dB. In contrast, for deblurring and super-resolution tasks, the performance gap between few steps and 100 steps is relatively limited, and correspondingly, LLE yields smaller improvements. This is expected as LLE is constrained to searching within the linear combinations of previous predictions; thus, when the few-step trajectory already approximates the optimal performance, LLE tends to learn a trajectory similar to the original one.

D.6 Performance of LLE with more steps

Here we supplement an additional experiment comparing DDNM with and without LLE under a larger number of sampling steps (25, 50, and 100) on the FFHQ dataset, as presented in Table 15. To avoid the need to store an excessive number of past results when the number of steps is large, we limit the lookback window to 10 steps. The results show that as the number of steps increases, the performance gap between methods with and without LLE becomes smaller. This is consistent with our expectation as LLE searches for an improved sampling trajectory within the linear subspace spanned by previous samples. When the original algorithm takes more steps, the discretization error is reduced, and its trajectory already closely approximates the optimal one. In such cases, LLE tends to follow the original trajectory more closely, resulting in smaller gaps. Therefore, LLE brings more significant improvements in few-step regimes, which aligns with our primary objective of enhancing the performance of inverse algorithms under a limited number of steps.

D.7 Effect of Optimization Iterations on LLE Performance

To assess how sensitive LLE is to the optimization of Equations (19), we further investigate its convergence behavior. We optimize LLE for { 25 , 50 , 100 , 150 , 200 } iterations and report the corresponding performance on the FFHQ noiseless task using the 5-step DDNM. The results are summarized in Table 16. We find that LLE achieves consistently strong and comparable results across a wide range of iterations, indicating that it converges quickly and shows no significant overfitting even with extended

0.2

0.3

0.4

0.5

0.6

0.7

0.8

ζ

15.0

15.5

16.0

16.5

17.0

17.5

18.0

18.5

PSNR ( )

DPS

0.2

0.3

0.4

0.5

0.6

0.7

0.8

λ

15.0

15.5

16.0

16.5

17.0

17.5

RED-diff

0.50

0.55

0.60

0.65

0.70

0.525

0.550

0.575

0.600

0.625

0.650

0.675

0.700

LPIPS (

)

Original PSNR

LLE PSNR

Original LPIPS

LLE LPIPS

training. Theoretically, this behavior is expected because LLE learns interpolation coefficients by minimizing a weighted sum of the MSE loss and the LPIPS loss. When the LPIPS weight is set to zero, the problem reduces to a standard least squares formulation, which is convex and therefore free from local minima. Although the LPIPS loss introduces non-convexity, its weight is chosen to be sufficiently small such that the overall loss landscape remains smooth and easy to optimize. These findings confirm that LLE's optimization process is stable and computationally efficient in practice.

D.8 Diversity of the solution

We conduct an additional visualization experiment comparing the diversity of Π GDM and DPS with and without the proposed LLE. As shown in Figure 10, for typical image restoration tasks, the posterior distribution is generally narrow, meaning that the plausible solutions tend to concentrate around the ground truth. Consequently, the ideal output should remain close to the reference image rather than exhibiting large variations. LLE effectively stabilizes the output and maintains high reconstruction quality.

E Detailed decomposition of the inverse algorithms

Here, we present the specific forms of DDRM [5], DDNM [6], DPS [8], Π GDM[7], RED-diff [11], DiffPIR [9], DMPS [10], ReSample [12], DAPS [14], DCDP [60], LGD [59], MPGD [61], FPS [62], SGS-EDM [63], Latent-DPS [12], PSLD [64], STSL [65], and CMInversion [68] within the canonical form proposed in this paper. For DDNM, DDRM, DMPS, Π GDM, FPS, and PSLD, we consider linear observations,

For the other algorithms, we consider general observations

E.1 DDRM

The Sampler of DDRM is a single-step DDIM, i.e., tweedie's formula, which is

Consider A as a diagonal matrix and s k as the k -th singular value. A general matrix A can be equivalently transformed into a diagonal matrix using Singular Value Decomposition. The correction process is defined element-wise as

The Noiser combines noise addition which is similar to DDIM sampling:

where ϵ ∼ N ( 0 , I ) , η and η b are hyperparameters. x k denotes the k -th element of the vector x , and the same notation applies to other vectors.

E.2 DDNM

DDNMis similar to DDRM, where the Sampler is the single-step DDIM as (50). Considering A as a diagonal matrix, the Corrector is a modified projection operator as

where Σ t i = diag { λ 1 , . . . , λ n } is a diagonal matrix with elements defined as

The Noiser is defined element-wise as

where ϵ ∼ N ( 0 , I ) and σ t i = √ 1 -α t i .

E.3 DPS

We rewrite the update formula of DPS as

where x 0 ,t i is the estimation using Tweedie's formula as (50). Thus the Sampler of DPS is a single-step DDIM. The Corrector is

where ζ t i is the learning rate hyperparameter. The Noiser is exactly the same as DDIM sampling

Π

GDM - Orig.

Π

GDM - LLE

DPS - Orig.

Sample 1

DPS - LLE

Sample 2

Sample 3

Sample 4

Sample 5

Sample 6

y

x

0

where

with η as a hyperparameter and ϵ ∼ N ( 0 , I ) . Similar transformation is applicable to all posterior sampling approaches, including Π GDMand DMPS. The derivation is omitted hereafter.

E.4 Π GDM

The Sampler of Π GDMis a single-step DDIM as (50). The Corrector is

where r t i = √ 1 -α t i . The Noiser is the same as DDIM sampling as (58).

E.5 RED-diff

The Sampler of RED-diff is a single-step DDIM as (50). RED-diff differs from other algorithms in that it uses an optimization process to update ˆ x 0 ,t i . Here we consider only the gradient descent update, where the Corrector of RED-diff is equivalently given by

where ξ is the learning rate, and λ controls the trade-off between the prior and likelihood terms. However, this formulation fails to reconstruct reasonable results under few steps in our experiments. We find the issue lies in the likelihood term, so we adjust it as follows to improve its performance unders few steps

where we use the x 0 ,t i to calculate the loss for the likelihood term rather than ˆ x 0 ,t i +1 . The Noiser of RED-diff is directly noise addition as

E.6 DiffPIR

The Sampler of DiffPIR is a single-step DDIM as (50). The Corrector solves a proximal point problem as

where ρ t = λσ 2 y α t / (1 -α t ) and λ is a hyperparameter. The Noiser of DiffPIR is a modified version of (58) as

where they calculate an effective ˆ ϵ θ ( x t i , t i ) using the corrected sample ˆ x 0 ,t i , and ϵ ∼ N ( 0 , I ) .

E.7 DMPS

The Sampler of DMPS is the single-step DDIM (50). Consider the SVD of the observation matrix A = UΣV T , then the Corrector is

where α t i = α t i /α t i -1 and

The Noiser is as (58).

E.8 ReSample

ReSample [12] is originally designed for latent diffusion, thus we set the Encoder and Decoder in ReSample to identity mappings for pixel diffusion models. The Sampler of ReSample is the single-step DDIM as (50). The Corrector is an optimization process which is termed hard data consistency as

with the initial point of the optimization algorithm set as x init = x 0 ,t i . The Noiser of ReSample is the ReSample method, which is

where and

E.9 DAPS

The Sampler of DAPS is a k -step DDIM sampler, i.e.,

The Corrector follows a Langevin dynamics process, with the iteration given by

where ϵ j ∼ N ( 0 , I ) , and r t i is a heuristics hyperparameter, which can be set as r t i = √ 1 -α t i [7]. The Noiser is direct noise addition as (63).

E.10 DCDP

The Sampler of DCDP is either a single-step DDIM as (50) or a k-step DDIM as (72). The Corrector starts from ˆ x 0 ,t i and performs fixed number of gradient descent steps to minimize ∥A ( x 0 ) -y ∥ 2 . The Noiser is direct noise addition as (63).

E.11 LGD

The Sampler of LGD is a single-step DDIM as (50). The Corrector is

where ζ t i is the step-size, x ( j ) ∼ q ( x 0 | x t i ) , and ℓ y ( x ) = ∥ y -A ( x ) ∥ 2 . q ( x 0 | x t i ) can be approximated as N ( x 0 ,t i , r 2 t i I ) where r t i = σ t / √ 1 + σ 2 t with

The Noiser is as (58).

E.12 MPGD

We only consider MPGD without manifold projection. The Sampler is a single-step DDIM as (50). The Corrector is

The Noiser is as (58).

E.13 FPS

For simplicity, we only consider FPS with particle size 1 . The Sampler is a single-step DDIM as (50). The Corrector is

where

and c is the DDIM hyperparameter. The Noiser is

where ϵ ∼ N ( 0 , I ) , and ϵ y ,t i is defined recursively as

with y t S = A ϵ t S .

E.14 SGS-EDM

The Sampler is a k-step DDIM as (72). The Corrector is

where

E.15 Latent-DPS

For latent-space methods, we denote E ( · ) , D ( · ) , and z t i as the Encoder, Decoder, and latent variable, respectively. The decomposition of Latent-DPS is similar to that of DPS, with the Corrector adjusted as

E.16 PSLD

The Sampler is a single-step DDIM as (50). The Corrector is

The Noiser is as (58).

E.17 STSL

The Sampler is a single-step DDIM as (50). The Corrector is

where d is the dimension of the latent variable, and ϵ j ∼ N ( 0 , I ) . The Noiser is as (58).

E.18 CMInversion

We denote the consistency model as g θ ( · , · ) . The Sampler of CMInversion is a single consistency model inference, i.e.,

The correction process can be recursively defined as

where x 0 ,t i , 0 = x 0 ,t i , and n t i , 1 ∼ N ( 0 , I ) . Then the Corrector is

The Noiser is direct noise addition as

F Specific design for DDNM and DDRM on noisy linear tasks

DDRMand DDNM are specifically designed to take aware of the observation noise in noisy scenarios. In particular, the variance of the additional noise is reduced such that the variance of the observation noise adds the additional noise equal to the desired variance of the next step in the diffusion model. This implies that in noisy linear tasks, the expected ground truth of the output of the Correctors in DDRM and DDNM is not a noiseless image, but rather a noisy image perturbed by the observation noise. Therefore, we adjust the ground truth for the LLE learning in noisy scenarios for DDRM and DDNMto better align with the characteristics of these algorithms.

The implementation is straightforward by simply passing the noiseless image through the Correctors of DDRM and DDNM, i.e.,

The objective for the LLE training is then

where the ground truth in (19) is replaced by (94) and (95). The remaining training details remain unchanged.

G Experimental details

G.1 Inverse problems settings

All the linear inverse problems follow [5], while the nonlinear deblurring follows [8]. We describe the details of all these problems to maintain completeness. For 4 × Super-Resolution, we use a 4 × 4 average pooling operation for downsampling. For Inpainting, we apply a random 50% mask to all pixels, where the mask is generated using the same random seed to ensure fairness. For anisotropic deblurring, we use Gaussian blur kernels with standard deviations of 20 and 1 in two directions, respectively. For compressed sensing, we use Walsh-Hadamard transform with a downsampling rate of 50% . For nonlinear deblurring, we adopt the neural network from [70] as the observation function. Unlike [8], we follow [14] by using a deterministic observation equation generated with a fixed random seed to ensure fairness.

For the noisy tasks, the noise standard deviation is multiplied by 2 to account for the data range of [ -1 , 1] , i.e., in the implementation, we actually add Gaussian noise with a standard deviation of 0 . 1 to the observations.

G.2 Hyperparameters for base inverse algorithms

We carefully tune the hyperparameters of the inverse algorithms to ensure they achieve satisfactory performance under few steps. The detailed hyperparameter settings are as follows.

DDRM. We use the recommended values from [5], i.e., η = 0 . 85 and η b = 1 . 0 .

DDNM. We use the recommended value from [6], i.e., η = 0 . 85 .

DPS. We find that the learning rate form recommended by [8] is too small for few steps setting. Therefore, we follow the learning rate from [12], i.e., ζ t i = ζ √ α t i , which demonstrated superior performance with fewer steps. The specific value of ζ is adjusted for different tasks, as shown in Table 17. The DDIM hyperparameter η is set to 1 . 0 .

Π GDM. We set the DDIM hyperparameter η = 1 . 0 .

RED-diff. We set the learning rate ξ = 1 . 0 . The weight λ is adjusted for different tasks, as shown in Table 18.

DiffPIR. We set the DDIM hyperparameter η = 1 . 0 and λ = 7 . 0 , which performs well across all tasks. For the optimizer of the proximal point problem (64), we use the schedule-free AdamW [45], which offers better stability than SGDM in some tasks under few steps. The learning rate is set to 0.1 and the number of optimization steps is 50.

DMPS. We specify the DDIM parameters as c 1 = η √ 1 -α t i -1 , c 2 = √ 1 -η 2 √ 1 -α t i -1 , and set η = 0 . 85 . The weight λ is carefully tuned task-by-task and step-by-step.

ReSample. We set the DDIM hyperparameter η = 1 . 0 and γ = 100 . The optimizer for hard consistency (68) is the SGDM algorithm, with a learning rate of 0.01, momentum of 0.9, and 50 optimization steps.

DAPS. We use k = 5 steps for the DDIM sampler and perform the Langevin dynamics for 100 iterations per timestep with the step size as η t i = η 0 ( δ + t i /T (1 -δ )) , where η 0 = 0 . 0001 , δ = 0 . 01 , and T = 1000 . For noiseless linear inverse problems, we find that replacing the gradient term in (73) with

leads to better results under few steps. For nonlinear problems and noisy scenarios, we follow the original setting and set σ y in (73) to 0 . 02 instead of 0 . 05 for better results, which is consistent with the observations in [14].

DCDP. We use Tweedie's formula approximation in the purification stage. The optimizer is SGDM, with the learning rate of 0.01 and momentum of 0.9. The number of optimization steps is set to 50 for Super-Resolution and 25 for all other tasks.

LGD. The particle number n is fixed as 50 in all experiments. The step-size ζ is set to 0 . 5

for deblurring, 1 . 0 for inpainting, 5 . 0 for super-resolution, and 0 . 1 for compressive sensing.

MPGD. The step-size ζ is set to 0 . 5 for deblurring, 1 . 0 for inpainting, 5 . 0 for super-resolution, and 0 . 5 for compressive sensing.

FPS. The DDIM hyperparameter c is fixed as √ c = 0 . 85 for all tasks.

SGS-EDM. We use 1 -step DDIM (i.e., Tweedie's formula) in the prior step.

Latent-DPS. The step-size is set as ζ t i = α t i · ζ , with ζ = 0 . 01 for deblurring, ζ = 0 . 5 for inpainting and super-resolution, and ζ = 0 . 005 for compressive sensing.

PSLD. We fix γ t i = 0 . 1 η t i , and set η t i as 8 . 0 for deblurring, 10 . 0 for inpainting, 25 . 0 for super-resolution, and 9 . 0 for compressive sensing.

STSL. We fix γ = 0 . 02 and set the number of Monte Carlo samples to n = 1 . η t i is chosen as 0 . 01 for deblurring, 0 . 02 for inpainting, 0 . 1 for super-resolution, and 0 . 015 for compressive sensing.

CMInversion. We fix the correction steps K = 5 . The step-size ζ is set to 0 . 3 for deblurring, 1 . 0 for inpainting, 3 . 0 for super-resolution, and 0 . 095 for compressive sensing.

In this paper, we default to sampling evenly spaced timesteps in [0 , T ] . For all algorithms, we initialize from standard Gaussian noise. Specifically for RED-diff, this is equivalent to using the random initialization method proposed in [13], which provides better stability. Starting from intermediate steps like the original setting of [7] does not affect the applicability of our method, as discussed in Appendix A.

G.3 Details of LLE training

For the training of LLE, we generate N = 50 reference samples using 999-step DDIM sampler with the corresponding diffusion model. We fix the weight of PSNR and LPIPS as ω = 0 . 1 . The Schedule-free AdamW [45] is employed with the epochs set to 100 at each timestep t i and warmup steps set to 50. Gradients are calculated directly using the full batch. In most cases, the learning rate is set to 0 . 04 /S , where S denotes the total number of steps used by the inverse algorithm. For ReSample and DAPS on the nonlinear deblurring task on the FFHQ dataset, we adopt a dynamic learning rate that at timestep t i , the learning rate is set to 0 . 2 α t i +1 /S . It is worth noting that we have conducted minimal learning rate tuning for different algorithms, tasks, and steps. Such general learning rates have already yielded satisfactory performance. In practical applications, further tuning of learning rates and other optimization hyperparameters could potentially further enhance the performance of LLE.

We design an adaptive initialization method for the learnable coefficients at each timestep. Before the learning stage at timestep t i , we compute the loss for ˆ x ( n ) 0 ,t i and ˜ x ( n ) 0 ,t i +1 , n = 1 , . . . , N , and determine the initialization strategy accordingly. For linear inverse problems, if

then we initialize as

Otherwise, we initialize as:

All other parameters are randomly sampled from N (0 , 10 -6 ) . Intuitively, this initialization allows the coefficients to initial with a smaller loss, thereby improving optimization efficiency.

For nonlinear inverse problems, we further introduce a soft initialization to prevent the algorithm from prematurely converging to local optima. Similarly, if

we initialize as

Otherwise, we initialize as:

All other coefficients are randomly sampled from N (0 , 10 -6 ) . This approach allows LLE to follow the original trajectory in earlier timesteps, while searching for better linear combination coefficients more aggressively in later timesteps.

H Complete quantitive results

Here, we present the complete results of all algorithms on the CelebA-HQ and FFHQ datasets using a total of 3, 4, 5, 7, 10, and 15 steps. PSNR, SSIM, LPIPS, and FID are reported on both noiseless and noisy tasks. The table index is illustrated in Table 19 for convenience.

Condition

Steps

Strategy

PSNR

↑

/SSIM

↑

/LPIPS

↓

/FID

↓

Deblur (aniso)

Inpainting

4

×

SR

CS

50%

σ

y

= 0

.

05

3

-

27.80 / 0.7577 / 0.3192 / 53.22

16.64 / 0.4424 / 0.4923 / 73.29

27.09 /

0.7732

/

0.2961

/ 60.81

16.55 / 0.4415 / 0.5394 /

104.7

LLE

28.08

/

0.7842

/

0.2909

/

43.53

24.38

/

0.5523

/

0.4329

/

64.67

27.84

/ 0.7702 / 0.2989 /

46.29

17.29

/

0.4731

/

0.5197

/ 155.4

4

-

28.98 / 0.7948 / 0.2845 / 47.36

20.27 / 0.5102 / 0.4568 / 75.48

28.35 /

0.8034

/ 0.2758 / 59.34

17.39 / 0.4464 / 0.5179 /

87.57

LLE

29.27

/

0.8172

/

0.2558

/

42.83

25.29

/

0.5915

/

0.4073

/

61.19

28.65

/ 0.7921 /

0.2700

/

39.87

18.43

/

0.4994

/

0.4742

/ 96.63

5

-

29.63 / 0.8192 / 0.2590 / 45.43

22.76 / 0.5501 / 0.4310 / 73.25

28.97 /

0.8182

/ 0.2619 / 58.20

18.20 / 0.4742 / 0.4914 / 77.15

LLE

29.82

/

0.8307

/

0.2394

/

45.10

26.35

/

0.6586

/

0.3661

/

54.84

29.02

/ 0.8056 /

0.2522

/

37.16

19.41

/

0.5358

/

0.4410

/

74.41

7

-

30.19 / 0.8377 / 0.2377 / 50.72

25.93 / 0.6269 / 0.3880 / 68.29

29.52

/

0.8325

/ 0.2418 / 54.07

20.28 / 0.5536 / 0.4344 / 65.31

LLE

30.29

/

0.8409

/

0.2247

/

44.66

27.69

/

0.7188

/

0.3279

/

50.63

29.33 / 0.8177 /

0.2333

/

35.57

21.32

/

0.6084

/

0.3852

/

54.22

10

-

30.45 / 0.8446 / 0.2274 / 51.12

28.19 / 0.7163 / 0.3325 / 60.74

29.07 /

0.8210

/ 0.2410 / 51.61

21.30 / 0.6356 / 0.3792 / 55.50

LLE

30.50

/

0.8454

/

0.2137

/

42.71

29.13

/

0.7886

/

0.2789

/

45.50

29.08

/ 0.8075 /

0.2353

/

33.47

22.42

/

0.6738

/

0.3408

/

44.97

15

-

30.50 / 0.8461 / 0.2165 / 47.03

30.57 / 0.8206 / 0.2589 / 46.30

29.39

/

0.8289

/ 0.2274 / 47.58

23.39 / 0.7475 / 0.2961 / 38.77

LLE

30.52

/

0.8463

/

0.2058

/

40.68

30.74

/

0.8540

/

0.2281

/

38.23

29.17 / 0.8120 /

0.2266

/

33.42

24.04

/

0.7687

/

0.2706

/

31.03

σ

y

= 0

3

-

39.44 / 0.9577 / 0.1004 /

11.60

19.39 / 0.4237 / 0.5946 / 137.9

30.20 / 0.8603 / 0.2413 / 60.50

17.27 /

0.5188

/ 0.5521 / 252.3

LLE

39.56

/

0.9588

/

0.0986

/ 11.72

26.31

/

0.7180

/

0.3790

/

78.34

30.29

/

0.8628

/

0.2309

/

58.37

17.69

/ 0.5185 /

0.5385

/

229.1

4

-

39.80 / 0.9608 / 0.0931 /

9.790

24.11 / 0.6077 / 0.4427 / 88.72

30.76

/

0.8728

/ 0.2043 / 51.84

18.16 / 0.5751 / 0.4969 / 187.6

LLE

39.97

/

0.9623

/

0.0914

/ 10.01

29.03

/

0.8419

/

0.2840

/

67.03

30.71 / 0.8710 /

0.1979

/

48.91

18.96

/

0.5840

/

0.4695

/

154.7

5

-

39.99 / 0.9623 / 0.0897 / 8.810

27.83 / 0.7580 / 0.3322 / 66.63

30.97

/

0.8766

/ 0.1876 / 46.19

18.98 / 0.6191 / 0.4496 / 149.1

LLE

40.20

/

0.9640

/

0.0865

/

8.770

30.93

/

0.8868

/

0.2206

/

52.78

30.90 / 0.8743 /

0.1823

/

43.52

20.15

/

0.6395

/

0.4147

/

118.1

7

-

40.24 / 0.9643 / 0.0846 / 7.480

26.20 / 0.6481 / 0.3722 / 55.35

31.17

/

0.8802

/ 0.1700 / 39.58

21.64 / 0.7143 / 0.3512 / 91.41

LLE

40.47

/

0.9662

/

0.0815

/

7.390

32.56

/

0.9159

/

0.1688

/

38.79

31.04 / 0.8772 /

0.1663

/

38.15

22.43

/

0.7270

/

0.3275

/

77.24

10

-

40.42 / 0.9656 / 0.0805 / 6.650

33.54 / 0.9282 / 0.1440 / 30.86

31.26

/

0.8818

/ 0.1583 / 34.82

22.68 / 0.7591 / 0.2992 / 68.94

LLE

40.70

/

0.9677

/

0.0773

/

6.500

33.63

/

0.9294

/

0.1386

/

29.31

31.16 / 0.8792 /

0.1564

/

33.90

23.81

/

0.7744

/

0.2773

/

57.84

15

-

40.55 / 0.9667 / 0.0776 / 5.880

34.57

/

0.9402

/

0.1122

/ 22.14

31.32

/

0.8828

/

0.1506

/ 30.93

25.25 / 0.8213 / 0.2347 / 43.85

LLE

40.78

/

0.9684

/

0.0748

/

5.830

34.55 / 0.9398 / 0.1125 /

21.95

31.28 / 0.8812 / 0.1507 /

30.79

26.27

/

0.8316

/

0.2232

/

38.99

Condition

Steps

Strategy

PSNR

↑

/SSIM

↑

/LPIPS

↓

/FID

↓

Deblur (aniso)

Inpainting

4

×

SR

CS

50%

Deblur (nonlinear)

σ

y

= 0

.

05

3

-

23.82 / 0.4509 / 0.4872 /

130.6

18.11 / 0.3216 / 0.6280 / 193.8

21.35 / 0.4240 / 0.5346 / 135.2

16.43 / 0.3556 / 0.6324 /

368.5

18.97 / 0.2847 / 0.6416 /

168.9

LLE

24.18

/

0.4741

/

0.4858

/ 132.1

19.98

/

0.3377

/

0.5652

/

160.2

24.42

/

0.5647

/

0.4596

/

128.3

16.54

/

0.3637

/

0.6309

/ 377.4

19.10

/

0.2913

/

0.6370

/ 173.0

4

-

24.67 / 0.4989 / 0.4454 /

115.9

21.80 / 0.4299 / 0.5197 / 142.0

23.37 / 0.4856 / 0.4714 /

123.8

16.99 /

0.3922

/ 0.5894 / 282.5

20.85

/ 0.3521 /

0.5584

/

141.8

LLE

25.05

/

0.5243

/

0.4449

/ 117.0

23.49

/

0.4817

/

0.4676

/

127.3

24.74

/

0.6058

/

0.4287

/ 124.5

17.20

/ 0.3870 / 0.5894 /

272.0

20.78 /

0.3525

/ 0.5629 / 143.7

5

-

25.22 / 0.5352 / 0.4220 /

110.9

23.95 / 0.5043 / 0.4546 / 120.1

24.24 / 0.5262 / 0.4421 /

115.7

17.59 /

0.4209

/ 0.5552 / 228.0

21.85

/

0.3871

/

0.5195

/

129.5

LLE

25.59

/

0.5626

/

0.4181

/ 112.6

24.48

/

0.5311

/

0.4346

/

118.1

24.94

/

0.6293

/

0.4113

/ 119.0

17.92

/ 0.4171 /

0.5516

/

212.8

21.81 / 0.3862 / 0.5236 / 131.0

7

-

25.97 / 0.5866 / 0.3904 / 103.5

25.71 / 0.5741 / 0.3979 /

102.1

25.03 / 0.5806 / 0.4060 /

111.3

19.05 /

0.4769

/ 0.4918 / 172.3

22.80

/

0.4227

/

0.4749

/

120.5

LLE

26.25

/

0.6165

/

0.3825

/

103.4

25.75

/

0.5803

/

0.3961

/ 105.1

25.44

/

0.6648

/

0.3814

/ 114.6

19.33

/ 0.4740 /

0.4900

/

161.7

22.76 / 0.4218 / 0.4807 / 121.8

10

-

26.59 / 0.6330 / 0.3612 / 98.73

26.71 / 0.6146 /

0.3649

/

91.47

25.59 / 0.6249 / 0.3776 / 106.0

20.12 / 0.5216 / 0.4417 / 137.8

23.35 / 0.4527 / 0.4472 /

116.2

LLE

26.79

/

0.6617

/

0.3537

/

97.59

26.72

/

0.6194

/ 0.3664 / 92.47

25.92

/

0.6897

/

0.3565

/

105.7

20.47

/

0.5224

/

0.4385

/

133.6

23.37

/

0.4532

/

0.4461

/ 117.8

15

-

27.22

/ 0.6802 / 0.3348 /

90.30

27.67 / 0.6557 / 0.3315 /

83.41

26.16 / 0.6677 / 0.3513 / 101.2

21.77 / 0.5804 / 0.3883 /

108.6

23.93

/ 0.4884 / 0.4172 / 112.2

LLE

27.11 /

0.7017

/

0.3278

/ 95.27

27.83

/

0.6684

/

0.3296

/ 85.67

26.43

/

0.7120

/

0.3357

/

97.74

22.00

/

0.5853

/

0.3854

/ 110.5

23.89 /

0.4914

/

0.4119

/

111.0

σ

y

= 0

3

-

28.66 / 0.6918 / 0.3179 / 80.09

18.48 / 0.3716 / 0.6071 / 200.4

25.54 / 0.5555 / 0.4055 /

109.2

16.70 / 0.4218 / 0.6093 /

379.6

20.10 / 0.4128 / 0.5904 / 166.6

LLE

30.70

/

0.7863

/

0.2683

/

74.22

20.55

/

0.3887

/

0.5328

/

166.7

27.11

/

0.6894

/

0.3851

/ 123.6

16.83

/

0.4423

/

0.6054

/ 392.9

20.21

/

0.4244

/

0.5810

/

164.7

4

-

29.56 / 0.7336 / 0.2900 / 73.34

22.70 / 0.5194 / 0.4807 / 147.8

26.34 / 0.6033 / 0.3720 /

96.68

17.29 /

0.4727

/

0.5619

/ 297.8

22.37

/ 0.5376 / 0.4839 / 131.5

LLE

31.61

/

0.8245

/

0.2394

/

66.55

24.98

/

0.6101

/

0.4120

/

127.2

27.78

/

0.7226

/

0.3480

/ 111.8

17.52

/ 0.4680 / 0.5622 /

292.1

22.36 /

0.5502

/

0.4787

/

129.0

5

-

30.22 / 0.7626 / 0.2700 / 68.05

25.48 / 0.6333 / 0.3982 / 119.0

26.90 / 0.6380 / 0.3491 /

90.89

17.95 /

0.5137

/ 0.5232 / 246.7

23.72 / 0.6058 / 0.4300 /

113.3

LLE

32.03

/

0.8424

/

0.2261

/

63.79

26.37

/

0.6934

/

0.3662

/

116.5

28.21

/

0.7453

/

0.3227

/ 103.3

18.32

/ 0.5100 /

0.5194

/

231.8

23.72 /

0.6241

/

0.4253

/ 115.6

7

-

31.21 / 0.8025 / 0.2417 / 62.32

28.18 / 0.7534 / 0.3146 / 98.21

27.71 / 0.6864 / 0.3184 /

83.45

19.56 /

0.5952

/ 0.4473 / 181.1

24.98 / 0.6651 / 0.3759 /

102.5

LLE

32.89

/

0.8753

/

0.1991

/

57.87

28.31

/

0.7732

/

0.3085

/

97.76

28.61

/

0.7709

/

0.3027

/ 96.55

19.89

/ 0.5931 /

0.4445

/

173.5

25.25

/

0.6991

/

0.3700

/ 105.2

10

-

32.14 / 0.8362 / 0.2150 / 54.74

29.86 / 0.8130 / 0.2653 /

80.17

28.39 / 0.7276 / 0.2891 /

78.51

20.75 / 0.6570 / 0.3857 / 144.2

25.56 / 0.6986 / 0.3454 /

95.53

LLE

33.35

/

0.8907

/

0.1847

/

54.45

29.91

/

0.8292

/

0.2629

/ 86.08

29.01

/

0.7923

/

0.2823

/ 87.14

21.19

/

0.6603

/

0.3816

/

139.3

25.80

/

0.7309

/

0.3426

/ 100.6

15

-

33.15 / 0.8676 / 0.1876 /

47.85

31.36 / 0.8576 / 0.2203 /

66.01

29.08 / 0.7677 /

0.2603

/

70.31

22.66 / 0.7312 / 0.3172 /

107.8

26.04 / 0.7251 /

0.3161

/

92.32

LLE

33.75

/

0.9046

/

0.1715

/ 51.07

31.51

/

0.8727

/

0.2151

/ 67.73

29.41

/

0.8090

/ 0.2626 / 77.98

22.91

/

0.7338

/

0.3163

/ 111.4

26.27

/

0.7556

/ 0.3213 / 96.30

I More qualitative results

Here we present more visualization results. Anisotropic Deblurring results are shown in Figure 11. Inpainting results are shown in Figure 12. Super-Resolution results are shown in Figure 13. Compressed Sensing results are shown in Figure 14. Nonlinear Deblurring results are shown in Figure 15.

DMPS - Orig.

CelebA-HQ

DMPS - LLE

CelebA-HQ

DMPS - Orig.

FFHQ

DMPS - LLE

FFHQ

Π

GDM - Orig.

CelebA-HQ

Π

GDM - LLE

CelebA-HQ

Π

GDM - Orig.

FFHQ

3 Steps

Π

GDM - LLE

FFHQ

4 Steps

5 Steps

7 Steps

10 Steps

15 Steps

y

x

0

DAPS - Orig.

CelebA-HQ

DAPS - LLE

CelebA-HQ

DAPS - Orig.

FFHQ

DAPS - LLE

FFHQ

DiffPIR - Orig.

CelebA-HQ

DiffPIR - LLE

CelebA-HQ

DiffPIR - Orig.

FFHQ

3 Steps

DiffPIR - LLE

FFHQ

4 Steps

5 Steps

7 Steps

10 Steps

15 Steps

y

x

0

8771

7

3 Steps

4 Steps

5 Steps

7 Steps

10 Steps

15 Steps

y

x

0

Π

GDM - Orig.

CelebA-HQ

Π

GDM - LLE

CelebA-HQ

Π

GDM - Orig.

FFHQ

Π

GDM - LLE

FFHQ

DDRM - Orig.

CelebA-HQ

DDRM - LLE

CelebA-HQ

DDRM - Orig.

FFHQ

3 Steps

DDRM - LLE

FFHQ

4 Steps

5 Steps

7 Steps

10 Steps

15 Steps

y

x

0

DPS - Orig.

CelebA-HQ

DPS - LLE

CelebA-HQ

DPS - Orig.

FFHQ

DPS - LLE

FFHQ

RED-diff - Orig.

CelebA-HQ

RED-diff - LLE

CelebA-HQ

RED-diff - Orig.

FFHQ

3 Steps

RED-diff - LLE

FFHQ

4 Steps

5 Steps

7 Steps

10 Steps

15 Steps

y

x

0

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes learnable linear extrapolation (LLE), designed to improve the performance of any diffusion-based inverse algorithm that follows the steps: (i) predict the clean signal, (ii) enforce measurement consistency, and (iii) map back to the noisy manifold to continue the reverse sampling process. The main insight of this paper, as far as I can tell, comes from Corollary 2, where they show that the estimate of the clean sample conditioned on the measurements, denoted $\\mathbf{x}_{0}(t, \\mathbf{y})$, is a linear combination of the estimate at the current timestep along with all previous timesteps. Motivated by this, they propose solving an optimization problem to learn such linear coefficients in order to obtain an accurate estimate of the clean signal, provided that a reference dataset exists.\n\nSTRENGTHS AND WEAKNESSES:\nI would say that the main strength of this paper is that they can improve upon existing diffusion-based inverse problem solvers in a lightweight fashion. Table 7 shows that LLE does not induce a large overhead, but can improve upon the existing methods up to some extent.\n\nFor weaknesses, I am not sure how effective this method would actually be considering that the improvements are not great. I would suggest the author to put more qualitative results such that we can look at the performance differences a bit better. On top of that, this method assumes that there exists a reference dataset to perform the optimization, which I am not sure holds in practice. How were these reference points chosen for experiments? I perhaps missed this in the main text.\n\nI also think that the presentation of this paper could be improved; I found the presentation of the methodology a bit convoluted. Perhaps there is a way to condense the main text to bring the algorithm box up to the main part.\n\nQUESTIONS:\n- Why are the results for DMPS in Table 1 so poor? I suggest the authors to look into this to provide a more fair comparison.\n- How precisely do Equations (19–20) need to be optimized? Did the authors find that the loss associated with this problem is easy to optimize? In cases where the optimization falls into a local minimum, did the authors find that LLE still outperforms the original method? I believe these questions are fundamental to understanding the effectiveness of LLE, if they have not already been addressed.\n\nLIMITATIONS:\nI have stated the limitations under weaknesses above. I would be willing to raise my score if my questions can be effectively addressed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the high computational cost of diffusion-based inverse problem solvers, particularly in few-step scenarios. First, it proposes a \"canonical form\" that deconstructs diffusion-based inverse algorithms into three distinct modules: a Sampler, a Corrector, and a Noiser. This framework is designed to unify a wide range of existing methods. Second, based on this canonical form and an analysis of ODE solvers, the paper introduces Learnable Linear Extrapolation (LLE), a lightweight method that improves few-step performance by learning to linearly combine the current estimate with previous ones. The authors demonstrate that LLE consistently improves results across nine reformulated diffusion algorithms on several inverse problem tasks, such as deblurring and inpainting on facial datasets.\n\nSTRENGTHS AND WEAKNESSES:\n#### **Strengths**\n\n1.  LLE method is a lightweight and intuitive approach, well-motivated by an analysis of high-order ODE solvers. \n2.  The proposed canonical form is an insightful contribution that provides a structured, unified view of diverse diffusion-based inverse algorithms.\n\n#### **Weaknesses**\n\n1. The claim of enhancing performance lacks validation against the true state-of-the-art. The baseline set, while broad, omits recent, competitive methods like  LGD (Song et al., 2023), DCDP (Li et al., 2024), MPGD (He et al., 2024), FPS-SMC (Dou and Song, 2024), and  SGS-EDM (Wu et al., 2024).\n2. The over-reliance on facial datasets (CelebA-HQ, FFHQ) is not representative of general-purpose restoration; validation on a diverse benchmark like ImageNet is needed to prove general applicability. \n3. The paper's complete focus on pixel-space models is a critical weakness. As LDMs are the de facto standard for high-resolution restoration, a method that does not demonstrate a clear path for applicability to them has severely limited practical significance.\n4. The paper fails to discuss or compare its approach with consistency models (e.g., Consistency Posterior Sampling). This is a major concurrent research direction for accelerating inverse problem solvers, and its omission makes the literature review and experimental comparison incomplete.\n\nQUESTIONS:\n1. How is LLE expected to perform against the baselines mentioned in the weakness section?\n\n2. What are the specific technical challenges in applying the canonical form and LLE to Latent Diffusion Models? Does linear extrapolation hold in the latent space, and how does the VAE's compression affect the Corrector module?\n\n3. How can the generalization of results from facial datasets to more complex domains like ImageNet be justified? The distinct priors of faces versus general scenes could affect the validity of the learned extrapolations.\n\n4. Please elaborate on the conceptual differences and key trade-offs (e.g., performance vs. training cost) between the LLE method and consistency-based approaches like Consistency Posterior Sampling.\n\nLIMITATIONS:\nA more thorough discussion of the challenges involved with LDMs and a justification for not addressing them is warranted.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces \"Learnable Linear Extrapolation\", a method to improve the performance of diffusion-based inverse algorithms with few sampling steps by leveraging previous estimation.\nThe authors propose a canonical form that unifies several diffusion-based inverse algorithms by decomposing them into three modules: Sampler, Corrector, and Noiser.\nThe method  operates by learning optimal linear combination coefficients to refine current predictions using previous estimates predictions.\nThe coefficient are pre-learned my minimizing a ``reconstruction + w * perceptual`` loss with respect to a reference dataset.\nThe authors validate the method through extensive experiments across multiple algorithms and tasks.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:\n\n- A rich framework that encompasses various diffusion posterior samplers\n- A novel method that improves performance with reduced required number of sampling steps\n\n\n## Weaknesses:\n\n**Theoretical Justification:**\n- Ambiguity in the justification for $\\hat{x}_0$ being a linear combination of previous estimate, particularly in Equation (10), it is unclear why it holds and a reference would great.\nIn fact, empirical observations of the extrapolation coefficient in Figures 6-8 suggests minimal dependence of the current iterate on previous estimates, in fact the matrix of weights is dominated by the diagonal or first diagonal elements, while the rest are almost zero.\n- The choice and size of the ground-truth datasets are not adequately addressed. Also the robustness of the method to out-of-distrbution samples. \nWhile the authors assess the the robustness on datasets FFHQ and CelebA-HQ, it is of little relevance as both dataset represent human faces. A more relevant discussion would be on Imagenet dataset, for instance.\n\n**Practical Considerations:**\n- **Computational Overhead:** Essential computational aspects, including the training time required to obtain the extrapolation coefficients and the memory requirements of the method during inference (as it requires storing previous estimates), are not discussed.\n\n**Inconsistencies:**\n- **Error in equation (2):** There is a missing logarithm in Equation (2) \n- **Figure 4 Clarity:** Figure 4 is unclear and lacks essential information. Specifically, the values of $ w $ are missing, and while the axes are labeled (LPIPS and PSNR), the context or meaning of $ w$ in relation to these metrics is not provided.\n\nQUESTIONS:\nNone\n\nLIMITATIONS:\n- Memory Footprint: The method necessitates storing previous estimate, which could lead to considerable memory overhead\n- Out-of-Distribution Performance: the method might suffer from out-of-distribution samples as the extrapolation coefficient are computed beforehand on a selected dataset.\n- Algorithm Dependence: the method is dependent on  the specific diffusion algorithm employed. changing the diffusion sampler requires re-training to get the extrapolation coeffcients.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Learnable Linear Extrapolation (LLE) which learns coefficients to combine previous estimates and the current prediction \nfor improved inference in solving inverse problems using diffusion models with few steps. The authors rewrite several diffusion-based algorithms for inverse problems \nin a canoncial form which can be combined with LLE. They show consistent improvements for the different algorithms on several standard reconstruction problems.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- LLE shows consistent improvements for a wide range of different algorithms \n- The canoncial form combines several existing algorithms in a unified framework, which is useful for practitioners\n- The paper is well written and easy to follow\n\nWeaknesses:\n\n- LLE requires additional finetuning and cannot be used out of the box\n- Corallary 2 that motivates LLE is based on ODEs, while the canonical form is non-deterministic due to the additional noiser step. I saw in the appendix that there is an extension to the more general SDE case; this should be highlighted more in the main paper\n- Improvements are only shown in the few-step regime (up to 14 steps); how large are the gaps to methods taking significantly more steps?\n- I would appreciate if the experiment section included problems where the diversity of solutions/full posterior is evaluated.\n\nQUESTIONS:\n- Using the history of estimates combined with learned coefficients has also been used in Bespoke Non-Stationary Solvers (https://arxiv.org/pdf/2403.01329).\n  Can the LLE method be seen as learning a type of non-stationary solver as defined in the paper?\n- How are the time steps chosen? Are they just uniformly spaced? Are there methods to choose them in a more optimal way?\n- What happens if we choose a large value for the number of steps $S$ and LLE? Do we still see improvements?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}