Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problems
Authors: Hongkai Zheng
Wenda Chu
Austin Wang
Nikola Borislavov Kovachki
Ricardo Baptista
Yisong Yue
Venue: Accepted by TMLR ()

Abstract: When solving inverse problems, one increasingly popular approach is to use pre-trained diffusion models as plug-and-play priors. This framework can accommodate different forward models without re-training while preserving the generative capability of diffusion models.  Despite their success in many imaging inverse problems, most existing methods rely on privileged information such as derivative, pseudo-inverse, or full knowledge about the forward model. This reliance poses a substantial limitation that restricts their use in a wide range of problems where such information is unavailable, such as in many scientific applications. We propose Ensemble Kalman Diffusion Guidance (EnKG), a derivative-free approach that can solve inverse problems by only accessing forward model evaluations and a pre-trained diffusion model prior. We study the empirical effectiveness of EnKG across various inverse problems, including scientific settings such as inferring fluid flows and astronomical objects, which are highly non-linear inverse problems that often only permit black-box access to the forward model. We open-source our code at https://github.com/devzhk/enkg-pytorch.

Published in Transactions on Machine Learning Research (05/2025)
Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problems
Hongkai Zheng Caltech
hazheng@caltech... edu
Wenda Chu Caltech
wchu@caltech.com edu
Austin Wang
abwang@caltech.coo edu
Caltech
Nikola B. Kovachki
nbovachbi@nnviddi.com
NVIDIA
Ricardo Baptista
rsh@cclteccc.c edu
Caltech
Yisong Yue Caltech
wyuescaltech.. edu
Reviewed on OpenReview:
https: // openreview. net/ forum? XPEEsKNeKa
//
Abstract
When solving inverse problems, one increasingly popular approach is to use pre-trained dif- fusion models as plug-and-play priors. This framework can accommodate different forward models without re-training while preserving the generative capability of diffuusin models. Despite their success in many imaging inverse problems, most existing methods rely on priv- ileged information such as derivative, pseudo-inverse, or full knowledge about the forward model. This reliance poses a substantial limitation that restricts their use in a wide range of problems where such information is unavailable, such as in many scientific applications. We propose Ensemble Kalman Diffusion Guidance (EnKG), a derivative-free approach that solve inverse problems by only accessing forward model evaluations and pre-trained diffusion model prior. We study the empirical effectiveness of EnKG across various inverse problems, including scientific settings such as inferring fluid flows and astronomical objects, which are highly non-linear inverse problems that often only permit black- box access to the forward model. We open-source our code at https://github.coom deezan//eee///eee Benkhhk//nkggnae wewgeenkggyttee pytorchh
1 Introduction
The idea of using pre-trained diffusion models (Song al. 2020 Ho et al. 2020 plug-and-play priors for solving inverse problems has been increasingly popular and successful across various applications includ- ing medical imaging (Song et al. 2021 Sun et al. 2023). image restoration (Chung et al. 2022b Wang ct al., 2022), and image and music generation (Rout et al. 2024b Huang et al.) [024]. A key advantage of this approach is its flexibility to accommodate different problems without re-training while maintaining the expressive power diffusion models to capture complex and high-dimensional prior data distributions. However, most existing algorithms rely on privileged information of the forward model, such as deriva- tive (Chung et al. 2022a Song al. 2023b), pseudo-inverse (Song al. (023a), or knowledge of its parameterization (Chung al.] 2023a This reliance poses substantial limitation that prevents their

---

Published in Transactions on Machine Learning Research (05/2025)
use in problems where such information is generally unvailable. For instance, in many scientific applica- Tglesias 2015), the forward model consists of a system of partial differential equations whose derivative or pseudo-inverse is generally unavailable or even undefined.
tions (Oliver et al.] 2008: Evensen & Van Leeuwen 1996;
The goal of this work is to develop an effective method that only requires black-box access to the forward model and pre-trained diffusion model for solving general inverse problems. Such an approach could signifi- cantly extend the range of diffusion- based inverse problems studied in the current literature, unlocking new class of applications especially many scientific applications. The major challenge here arises from the dif- ficulty of approximating the gradient of general forward model with only black-box access. The standard zero-order gradient estimation methods are known to scale poorly with the problem dimension (Berahas et al. 2022).
To develop our approach, we first propose a generic prodiction-correction (PC) framcwork using an opti- mization perspective that includes existing diffusion guidance-based methods (Chung al.) 2022a Song et al. 2023b a Peng et al., 2024 Tang et al. 2024 special cases. The key idea of this PC framework is to decompose diffusion guidance into two explicitly separate steps, unconditional generation (i.e., sampling from the diffusion model prior), and guidance imposed by the observations and forward model. This modular viewpoint allows us to both develop new insights of the existing methods, as well as to introduce new tools to develop a fully derivative-fi guidance method. Our approach, called Ensemble Kalman Diffusion Guidance (EnKG), uses an ensemble of particles to estimate the guidance term while only using black-box queries to the forward model (i.e., no derivatives are needed), using a technique known as statistical linearization EEeesse 2003 Schillings & Stuart 2017) that introduce to diffusion guidance via PC framework.
Contributions
We present a generic prodiction-correctio (PC) framework that provides an alternative interpreta- tion of guided diffusion, as well as additional insights of the existing methods.
Building upon the PC framework, we propose Ensemble Kalman Diffusion Guidance (EnKG), a fully derivative-free frree approach that leverages pre-trained model in plug-and-play manner for solving general inverse problems. EnKG only requires black-box access to the forward model and can accommodate different forward models without any re-training.
We evaluate on various inverse problems including the standard imaging tasks and scientific problems like the Navier-Stokes equation and black-hole imaging. On more challenging tasks, such as nonlinear phase retrieval in standard imaging and the scientific inverse tasks, our proposed EnKG outperforms bascline methods by a large margin. For problems with very expensive forward models (e.g., Navier- Stokes equation), EnKG also stands out being much more computationally efficient than other ferivative--ree methods.
2 Background & Problem Setting
Problem setting Let G R" + RM denote the forward model that maps the true unobserved source x to observations y. We consider the following setting:
y = G(x) + &ER" E R
(1)
where we only have black-box access to G (generally assumed to be non-linear). & represents the observation noise which is often modeled as Gaussian, ie., I',, and y represents the observation. Solving the inverse problem amounts to inverting Eq. [ i.c., finding the most likely æ (MAP inference) or its postcrior distribution P(æly) (posterior inference) given y. This inverse task is often expressed via Bayes's rule as p( yy) x p(y|(z) p(æ). Here p(æ) is the prior distribution over source signals (which we instantiate using a pro-trained diffusion model), and p(y]a:) is defined as ① Because we only have black-box access to G, we can only sample from p(y|z)) (z), and do not know its functional form. For simplicity, we focus on finding the MAP estimate: arg max (z)) p(x).
2

---

Published in Transactions on Machine Learning Research (05/2025)
Diffusion models Diffusion models (Song et al. 2020 Karras et al. 2022 capture the prior p(æ) implicitly using diffusion process, which includes forward process and backward process. The forward process transforms a data distribution 2o ~ Pdata into a Gaussian distribution 2T ~ (T))) defined by a pre-determined stochastic process. The Gaussian distribution often referred to as noise, and so the forward process (1 going from 0 to T) is typically used lo create training data (iteratively noisier versions of #o Paata) for the diffusion model. The backward process (t going from T to 0), which typically learned in diffusion model, the standard generative model and operates by sequentially denoising the noisy data into clean data, which can be done by either a probability flow ODE or reverse-time stochastic process. Without loss of generality, consider the following probability flow ODE since every other probability flow ODE is equivalent to it up to a simple reparameterization as shown by Karras et al. [2022]:
dæ. - (1)) log
(2)
Training a diffusion model amounts to training the so-called score function Va, log which we assume is already completed (and not the focus of this paper). Given trained diffusion model, we can sample p(x) by integrating (2) starting from random noise.
Diffusion guidance for inverse problems As surveyed in Daras et al.
2024), arguably the most popular
approach to solving inverse problems with pre-trained diffusion model is sellance-based (Chung et al.
2022a: Wang et al.) 2022 Kawar et al. 2022 Song et al. 2023a Zhu et al. 2023 Rout et al., 2023 Chung
et al. 2023b Tang et al. 2024]. Guidance-based methods are originally interpreted as the conditional reverse
diffusion process targeting the posterior distribution. For ease of notation and clear presentation, we use the
probability flow ODE to represent the reverse process and rewrite it with Bayes Theorem.
dæ; à (t) lz)l
=ttot)((t log (tt) log (d)
where Va, log)((#)) is the unconditional score and the Vz, (og is the guidance from likelihood. In practice, the unconditional score is approximated by pre-trained diffusion model so(2,,t). The likelihood term, (yyl), = Sa,, p(æ; (0)))()/2)))0. is computationally intractable as it requires integration over all possible Ty- Various tractable approximations have been proposed in the literature, which we denote as p.(y)a,). The corresponding approximated reverse process is:
dæ, (f)) w, Vn, log P. dtt
(4)
where Wy is the adaptive time-dependent weight. The design of w, in Eq. 4 varies across different methods but it is typically not related to (t)) that Eq. (3) suggests, which makes it hard to interpret from posterior sampling perspective. In this paper, we will take optimization perspective develop useful interpretation for designing our proposed algorithm.
One key issuc with Eq. ( is that many algorithms for sampling along Eq. 4 assume access to the gradient Va, log (y/æx)) When this gradient unavailable (e.g., only black-box access to fx())), then one must develop a derivative-free approach, which is our core technical contribution.
Two existing derivative-free diffusion guidance methods are stochastic control guidance (SCG) Huang al., 2024), and diffusion policy gradient (DPG) (Tang et al. 2024). Both SCG and DPG are developed from the stochastic control viewpoint, and guides the diffusion process via estimating a value function, which can be challenging to estimate well (as seen in our experiments).
3 Related work
Ensemble Kalman methods Ensemble Kalman methodology was first introduced by Evensen (1994) in the context of data assimilation and later revisited by Iglesias et al. (2013) for inverse problems from an optimization perspective, resulting in the derivativo-free algorithm known as Ensemble Kalman Inversion
(EKI). Subsequent advancements inchude momentum-basee EKI for training neural networks (Kovachki &
Stuart 2019) and various regularization techniques to improve stability and efficiency (Iglesias 2016 Chada
3

---

Published in Transactions on Machine Learning Research (05/2025)
While the original derivations of these methods (Evensen, 1994 Iglesias et al. 2013 may not explicitly highlight it, the overarching idea behind the ensemble Kalman met hodologyg can be interpreted as linearization of the forward model G as elucidated in Schillings & Stuart (2017); Law et al. (2016). In essence we approximate the potentially complex forward model G(z) with the best-fitting linear surrogate model y = Ar b. In other words, we seek the minimizers of the following optimization problem:
al. 2020
min E. |[(()) - Ax - -bb. A.b
which has closed-form solutions given by
= B, b=,[ [x[r),
where C is the pseudoinverse of the covariance matrix. Our approach builds upon this linear surrogate model to approximate the forward model in our likelihood step defined in Eq. (4p. enabling fully derivative- free algorithm. Furthermore, prior works (Bergou et al. 2019 Chada & Tong 2022) establish the convergence results for certain EKI variants in the non-linean setting. However, these proofs do not directly apply to our algorithm. Instead, we develop tailored convergence results for our analysis.
Derivative-free optimization Traditional derivative-free optimization (DFO) algorithms include direct search, which includes the coordinate search, stochastic finiteddifference approximations of the gradient. Nelder-Mead simplex methods, and model- based methods; see Berahas et al (0022) for an overview. Among modern DFO techniques, Gaussian smoothing methods (Nesterov & Spokoing 2017) have demonstrated robust empirical performance (Salimans et al. 2017). These gradient estimates can be plugged into gradient- based algorithms directly, which we use to establish strong baselines in this paper.
Predictor-Correctt method for diffusion models The term "predictor-correttee has been used several prior works on diffusion model sampling, but their objectives and mechanisms differ from our PC framework. Song al. (2020) introduces Predietor-Corrector sampler for the sampling of diffusion models,
where both the predictor and corrector aim to sample from the same target distribution by simulating different stochastic processes (reverse-time SDE and annealed Langevin dynamics, respectively). Lezama et al. 2022) Uhen extends this framework to discrete space. More recently, Bradley & Nakkiran (2024) applics a similar perspective to classifier-free guidance, using PF-ODE solver (DDIM) as the predictor and Langevin dynamics as corrector, in order to sample from the comuna-poweres data distribution. Other works, such Zhao al. (2023) and Zhao et al. (2024), draw inspiration from predictor-correcto methods in the literature of classical numerical ODE solvers, focusing on solving the probability flow ODE (PF-ODF) with higher-order accuracy and adaptive step sizes. These cxisting PC methods aim to sample from the diffusion model, which is related to the prediction step (sampling from diffusion prior) in our framework but irrelevant to our correction step that interacts with the forward model G.
4 Method
To develop our Ensemble Kalman Diffusion Guidance (EnKG) method, we first provide an interpretation of diffusion guidance through the lens of the prediction-corretioo framework. EnKG can be viewed as an instantiation which enables derivative-free approximation of the guidance term.
4.1 Prediction-correcti interpretation of guidance-base methods
Inspired by the idea of the Predickor-Correccoor continuation method in numerical analysis (Allgoer & Georg 2012), we show how to express the guidance- based methods within the following prediction-correctin franework. Suppose the time discrctization scheme T to > t, > in = 0. Let w; = wy, for light guidance-basee methods for inverse problems can be summarized
notation. As illustrated in Algorithm | into the following steps.
4

---

Published in Transactions on Machine Learning Research (05/2025)
Algorithm Generic Guidance- based Method (ODE version)
Require: G, so, (4))=1 {wi)/
sample (O, (to)
2: for E (0, 1] do
x (()) (0t, 4:) (le+1 44)
Prior prediction step
4: log ( (z))
xe log p(y (2s) w:+1 arg m
b. Log-likelihood estimation
|| 11 =1
log ply) (0:41)
Guidance correction step
2wa
6: end for
return EN
Figure 1: Illustration of the predittion-correction interpretation for guidance-based methods a 1D Gaussian mixture example. From left to right, the probability flow ODE transforms pr(x;) from a Gaussian into a mixture of two Gaussians. The grey lines represent the trajectorics of the probabil- ity flow. The prediction step corresponds to nu- merical integration step along this flow. The correc- tion step adjusts the particle towards the MAP es- timator while keeping them close to the imitial pre- diction point. In contrast to prior methods with in- dependent particles, the proposed EnKG introduces interactions between particles to eliminate the need for gradient access.
Particle Ground truth
Prediction step
Correction step
Prior prediction step This step is simply a numerical integration step of the unconditional probability flow ODE, ie., by moving one step along the unconditional ODE trajectory:
x =; (tt))otttt (z), (t++1 tz).
(5)
Log-likelihood estimation step
This step estimates the log-likelihood log p(y]a;)):
log (gg((,) log
Guidance correction step This step solves the following optimization problem that formulates promise between maximizing the log- likelihood and being near ø/;
| - xf112
0;+1 = arg min
log (1)+))
(6)
20;
where the larger guidance scale w; gives the solution point near the MAP estimator and smaller value leads to small movement towards the MAP estimator. Eq. (6) is essentially proximal operator (Parikh et al.
2014) if w; is lower bounded by positive number. This optimization problem is often non-convex in most practical scenarios. As a result, the optimization algorithm may converge to a local maximum rather than a global one.
To solve Eq. 6 efficiently, one can use first-order Taylor approximation of log p(y) 2++1) 2f/
1og (1+1) log (gyl())) V log (gyla))))) (#141 xx)) O (1æe41 2;,13).
(7)
5

---

Published in Transactions on Machine Learning Research (05/2025)
Algorithm 2 Our method: Ensemble Kalman Diffusion Guidance (EnKG).
Require: G, y. so, solver Φ, (4))=11 (=1)
sample T0 ~N(00 (to) (to) J) 1, l.....J
b Initialize particles
2: for E (O,...,N--............. 1 do
3:
2 ( (l)) ( (4) (li+1 - ui)
b Prior prediction step
4: EN t Φ ti .j = l.....J
())
5: 9 + E G( (k)) -G( (2/())
(11l + x w.gh") ,j 1, l....
Guidance correction step
7: end for
return [# 1
Substituting the approximation Eq.
7 into the correction step
6 gives:
|
Li+1 = arg min
log p(ylaz)) - V log((1)/z)) (#141 - x())
(8)
2w,
= o + w/ V. log ((gf(y)))))
(9)
which can recover the gradient step structure of most existing guidance-based methods CChung el al.
2022a
Song et al. 2023b Mardani et al. 2023
Putting it together. Figure depicts the Prediction- Correction interpretation in 1D Gaussian mixture example, where guidance-based hoth iteratively step towards the MAP estimator while staying close to the initial unconditional generation trajectory defined by the prediction step. Importantly, the PC framework allows more degrees of freodom in method design.
4.2
Our approach: Ensemble Kalman Diffusion Guidance
We now demonstrate how the correction step can be performed in a derivative-free manner using the idea of statistical linearization. Our overall approach is described Algorithm 2
Likelihood estimation. The likelihood term can be factorized as follows:
ply)æ.) = / ( /d = Ex le
(10)
which is intractable in general. We use the following Monte Carlo approximation:
ply x.) = Emy mm P(y((e) = p(y 2N),
(11)
where vy is obtained by rumning the PF ODE solver starting at x,. One attractive property of this estimate compared to popular ones based on E|zy æ. and isotropic Gaussian approximations in previous works Chung et al. (2022a): Song et al. (2023a]b is that our approximation stays on data manifold but the Gaussian approximations include additive noise that do not live on data manifold. This aspect is particularly important for scientific inverse problems based on partial differential equations (PDEs), where staying on the data manifold is important for reliably solving the forward model p(y]æ:). For instance, we observe that Gaussian approximations frequently violate the stability conditions of numerical PDE solvers, leading meaningless estimates.

---

Published in Transactions on Machine Learning Research (05/2025)
Derivative-free correction step. Consider an ensemble of particles ( 1==1- empirical mean and Of denote their empirical covariance matrix, at the i-th iteration:
)/ Let. D; denote their
3. = マ
Σ o
C
(1) (a Bi
)))
J j=1
Instead of the commonly used scalar weight wss use a weighting matrix C, in Eq. (8:
x 1it1
(
arg min
[2:+1 - - 22(1)) (u:C./) -1 (0it1 -2, 22(0))
(12)
mi+1
V log f (y)l2:1))
(3/411 m. (21)))
(13)
= ( +w, C.i v log ())
(14)
Note that in practice, Cf can be singular when the number of particles is smaller than the particle dimension. In such cases, the matrix inverse in Eq. (12) is generalized to the sense of the Moore-Penronn inverse as C
is always positive semi-definite. Eq. (14 effectively becomes gradient projected onto the subspace spanned by the particles. At its current form, Eq. (14) still requires the gradient information. Next, we show how to approximate this gradient step without explicit derivative by Leveraging the idea of statistical linearization in the ensemble Kalman methods (Bergemann & Reich 2010 Schillings & Stuart 2017).
Assumption 1. G & has bounded first and second order derivatives. Let vb denote Go There exist constants M1, M2 such that for all u, u', u, v' E 12d
l1v(u) (u'')ll S Millu u'll, Hy(o')u M2||o|12 where Hy denotes the Hessian matric of sb Assumption 2. The dislance belween ensemble particles is bounded. There erists a constand M3. such that ||= - - < M3.j j = I....,J.
Assumption The observation empirical covariance matriz does not degenerate to zero unless the covari- ance matrir collapses to zero. In other words, tr (((1) 0 if and only if ((() 0, and
(+) 2x
#0 + tr (c((t > M4 >0, 0,
where
J
J .b Σ (z))))
O J Σ
((((0)) (5)) - - (W(a!()) ;.)
())
(15)
Remark To verily that Assumption holds in most cases of interest, plot the traces ir (Czz) and tr (Cyy) across three experiments of different inverse problems. As shown in Figure 5 while larger tr ( (Caz) does not always indicate larger tr (Cvy) due to the ill-posedness, tr(Cny) approaches zero only when tr (C()a also approaches zero, aligning with our assumption.
Proposition 1. Under Assumption and suppose the correction step is implemented as follows with w, 1/ (tr (18
1+1 = xi + wxC(k) my
(15)
y - & (z, (2)
(16)
= x
(6)
+ wi
1
( (2((1)
G. +6 (2/(1)) (,(0))), (z(()) x..
(17)
k=1
where
= 1 1=1
Σ
((1)
Tz)) (w: (zx V51
x
7

---

Published in Transactions on Machine Learning Research (05/2025)
After sufficient iterations, have the following approzimation:
C my y - vi (a. - Σ
-
Σ (w (2, (1) G.y v (2. )
2 x;
(18)
C+ v log& (y)
(19)
where
J G
G -
2b
Σ
(x (5))
The detailed derivation can be found in Appendix A.2 Proposition I shows that the ensemble update step defined in Eq. 177 effectively approximates the preconditioned gradient step defined in Eq. (12) without explicit derivative:
2 x 16) + w;C(i (y (z!
~ x. (1) +w, w, V log&
(20)
Algorithm puts it all together and provides a complete description of the proposed method. Implementation details are provided in Appendix A.4
5 Experiments
We empirically study our EnKG method on the classic image restoration problems and two scientific inverse problems. We view the scient inverse problems as the more interesting domains for evaluating our method, particularly the Navier-Stokes equation where it is impractical to accurately compute the gradient of the forward model.
Baselines We focus comparing against. methods that only use black-box access to the forward model. The first two baselines, Forward-GSG and Central-GSG (Algorithm 3), use numerical estimation methods instead of automatic differentiation to approximate the gradient of the log-likelihodd, and plug it into a standard gradient-based method, Diffusion Posterior Sampling (DPS) (Chung et al. 2023b) Specifically, Forward-CSG uses a forward Gaussian smoothed gradient (Eq. 37) and Central-CSG uses a central Gaussian smoothed gradient (Eq. 38 More details are in Appendix A.3 The last two baselines are Stochastic Control Guidance (SCG) (Huang et al. (2044) and Diffusion Policy Gradient (DPG) (Tang al. (024)), discussed in Sec. For Navicr-Stokes, also add the conventional Ensemble Kalman Inversion (EKI) [Iglesiss et al.
2013
5.1 Image inverse problems
Tackling image inverse problems (e.g., deblurring) is common in the literature and serves reasonable reference domain for evaluation. We note that we consider harder version of the problem where we do not use the gradient of the forward model. Moreover, most imaging problems use a linear forward model (except for phase retricval), which is significantly simpler than the non-lincar forward models more often used in scientific domains.
Problem setting We evaluate our algorithm on the standard image inpainting, superresolution, deblurring (Gaussian), and phase retrieval problems. For inpainting, the forward model is box mask with random- ized location. For superresolution, deblurring, blurring kernel of size 61 x 61 with standard deviation 3.0. Finally, phase retrieval takes the magnitude of the Fourier transform of the image as the observation. We use measurement. noise = 0.05 in all experiments except for superresolution on 64 x 64 images, where we set o 0.01. The pre-trained diffusion model for FFHQ 64 x 64 is taken unmodified from Karras al. (2022). The model for FFHQ 256 256 is taken from Chung et al. (2022a) and converted the EDM framework (Karras et al. 2022
employ bicubic downsampling (either x2 or x4) and for Gaussian
using their Variance- Pariance-Preservinn Preserving (VP) preconditioning.
8

---

Published in Transactions on Machine Learning Research (05/2025)
Table 1: Quantitative evaluation on FFHQ 256x256 dataset. We report average metrics for image quality and consistency on four tasks. Measurement noise o 0.05 unless otherwise stated.
Inpaint (box)
SR (x4)
Deblur (Gauss)
Phase retrieval
PSNR
SSIM
LPIPSL
PSNR1
SSIMI
LPIPSL
PSNRT
SSIM
LPIPS,
PSNRT
SSIM
LPIPS,
For
17.82
0.562
0.302
18.08
0.469
0.384
24.43
0.704
0.206
7.88
0.070
0.838
Cen
18.76
0.720
0.229
26.55
0.740
0.169
25.39
0.775
0.180
10.10
0.353
0.691
DPG
20.89
0.752
0.184
28.12
0.831
0.126
26.42
0.798
0.143
15.47
0.186
0.495
SCG
4.71
0.302
0.763
4.71
0.302
0.760
4.69
0.300
0.759
4.623
0.294
0.764
BnkG((Ours)
21.70
0.727
0.286
27.17
0.773
0.237
26.13
0.723
0.224
20.06
0.584
0.393
Evaluation metrics
We evaluate the sample quality of all methods using peak signal signal-to-noise
ratio (PSNR), structural similarity (SSIM) index (Wang et. al. 2004), and learned perceptual image patch similarity (LPIPS) score (Zhang et al. 2018).
Results We show the quantitative results in Table 1 Appendix), and qualitative results in Figure (Ap- pendix). On the easier linear inverse problems (inpainting, superresolution, and deblur), EnKG comes in second to DPG. On the harder non-linear phase retrieval problem, EnKG is comfortably the best approach. This trend is consistent with our results the scientific inverse problems, which are all non-linear.
5.2 Navier-Stokes equation
The Navier-Stokes problem is representative of the key class of scientific inverse problems (Iglesias et al,, 2013) that our approach aims to tackle. The gradient of the forward model is impractical to reliably compute via autoo-differetiation, as it requires differentiating through a PDE solver. Having effective derivative-free methods would be highly desirable here.
Problem setting We consider the 2-d Navier-Stokes equation for a viscous, incompressible fluid in vorticity form on a torus, where u E C' ([0, Ti; (4per((0,))) m²)) for any r >> 0 is the velocity field, w = V x u is the vorticity, wo E Lper ((0, R)) is the initial vorticity, E R+ is the viscosity coefficient, and E ee ((0, 2;; is the forcing function. The solution operator g is defined the operator mapping the vorticity from the initial vorticity to the vorticity at time T. G wp wr. Our experiments implement it as a pseudo-spectral solver (He & Sun 2007).
O,w uxx Vw t) vAw( t) f(z),
(0, 20, (0, x E (0, 2T) i [0,
V u(x t) O, w 0) = wo(x)
(21)
x E (0, 2T))
The goal is to recover the initial vorticity field from a noisy sparse observation of the vorticity field at time T 1. Eq. (21 does not admit a closed form solution and thus there is no closed form derivative available for the solution operator. Moreover, obtaining an accurate numerical derivative via automatic differentiation through the numerical solver is challenging due to the extensive computation graph that can span thousands of discrete time steps. We first solve the equation up to time T = using initial conditions from Gaussian random field, which is highly complicated due to the non-linearity of Navier-Stokes equation. We sample 20,000 vorticity fields to train our diffusion model. Then, we independently sample 10 random vorticity fields as the test set.
Evaluation metrics We report the relative 2 error to evaluate the accuracy of the algorithm, which is given by -10p-w;l1.z where uo the predicted vorticity and w is the ground truth. To comprehensively analyze the computational requirements of inverse problem solvers, we use the following metrics: the total number of forward model ovaluations (Total # Fwd); the number of sequential forward model evaluations (Seq. # Fwd), where each evaluation depends on the previous one.; the total number of diffusion model evaluations (Total # DM); the number of sequential diffusion model evaluations (Seq. # DM), which is
9

---

Published in Transactions on Machine Learning Research (05/2025)
Table 2: Comparison on the Navier-Stokes inverse problem. Numbers in parentheses represent the sample standard deviation. Metrics to evaluate computation costs are defined in Sec. 5.2 are excluded from the results due numerical instability. Runtime is reported on single A100 GPU.
*: one or two test cases
Onteee
Onoten 1.0
Computation cost
Relative L2
Relative L2
Relative L.
Total # Fwd
Total # DM
Seg # Fwd
Sey # DM
Runtime
EKI
0.577 (0.138)
0.609 (0.119)
0.673 (0.107)
1024k
0.50k
121 mins
GSG
1.687 (0.156)
1.612 (0.173)
1.454 (0.154)
2049k
105 mins
Central-GSG
2.203* (0.314)
2.117 (0.295)
1.746 (0.191)
2048k
105 mins
DPG
0.325 (0.188)
0,408 (0.173)
0,466 (0.171)
1000k
1k
228 mins
SCG
0.908 (0.600)
0.928 (0.537)
0.966 (0.546)
384k
384k
0.75k
1
119 mins
EnKG((Our))
0.120 (0.085)
0.191 (0.057)
0.294 (0.061)
295k
3342k
0.14k
1.3k
124 mins
Observation
Ground truth
EKI (baseline)
Central-GSG
SCG
DPG
EnKG
Figure 2: Visualization of results Navier-Stokes inverse problem with different levels of observation noise. Each observation is subsampled by factor of 2, representing sparse measurement. Note that the results of GSttral-GSG are here for demonstration purpose because neither Contral-GSC nor Forward-GSB is able to produce reasonable results.
analogous Seq. # Fwd but focuses on diffusion model evaluation; the total number of diffussio model gradient evaluations (Total # DM grad); the number of sequential diffusion model gradient evaluations (Seq. # DM grad). These metrics designed to reflect the primary computational demands: forward model queries and diffusion model queries. Sequential metrics particularly important because they determine the minimum runtime achievable, independent of the available computational resources. By isolating sequential evaluations, we can better assess the parallelization potential of the algorithm, akin to the "critical path" concept in algorithm analysis from the computer science literature [Kohler 1975).
Results In Table 2 we show the average relative 1,2 error of the recovered ground truth at different noise levels of the obscrvations. Our EnKG approach dramatically outperforms all other mothods. Qualitatively, we see in Figure that EnKG give solutions which qualitatively preserve important features of the flow, while some methods completely fail (i..,, overly noisy outputs).
On the computational aspect, the Navier-Stokes forward model (which requires PDE solve) is extremely expensive, as shown in Figure a). As such, the nunber of calls to the forward model dominates the computational cost. We see in Table our EnKG approach actually makes the fewest calls to the forward model (since it uses statistical lincarization rather than trying to numerically approximate the gradient or value function), and thus EnKG is also the most computationally efficient approach, as seen in Figure b). The traditional Ensemble Kalman Inversion (FKT) approach also employs statistical linearization, and we
10

---

Published in Transactions on Machine Learning Research (05/2025)
S40g PM
PerwerdCanttaa-055
127
DPG
Relative error
Runtime 53
EKI
EnKG
30"1
Fwd evall
DM evaal
DM grad
300 150
200
250
300
Total OM Ored
Time cost (min)
(a)
(b)
(c)
Figure 3: (a): runtime of single evaluation of the forward model, diffusion model, and diffusion model gradient (tested on single A100). (b): comparison of computational characteristics of different algorithms on Navier-Stokes problem. Metrics are defined in the 'Evaluation metrics' paragraph of Sec. 4.2. Each axis is normalized by dividing by the maximum over the algorithms. (c): compare EnKG with EKI on compute cost versus error.
do a detailed comparison in Figure 3(c) c),, where we see that EnKG dominates EKI in the computational cost versus error trade-off curve.
Ablation study For practical insights into selecting hyperparameters for EnKG, perform ablation studies on the ensemble size, guidance scale, and the number of diffusion model queries. The results are shown in Figure and Figure [ We observe clear trend: increasing the ensemble size consistently leads to improved performance, as measured by the relative L error. However, the gains become marginal beyond 2048 particles. Our experiments also reveal that a guidance scale 2.0 yields the best average performance across our experiments. Increasing the guidance scale beyond this point introduces instability, resulting higher relative L errors. This instability likely due to the fact that large guidance scale might lead to divergence as Lemma [ only guarantees the convergence with small guidance scale. Furthermore, we vary the number of diffusion model queries using different mumber of ODE steps in the likelihood estimation (a))). We also examine the effect of varying the number of diffusion model queries by changing the number of ODE steps used in estimating 2 Performance improves with more diffusion model queries but exhibits diminishing returns beyond 500 steps.
5.3 Black-hole imaging inverse problem
The black-hole imaging problom is interesting due to its highly non-lincar and ill-posed forward model (i.c., the sparse observations capt ured by telescopes on Earth). For evaluation purposes, we assume only black-box access to the forward model.
Problem setting The black hole interferometric imaging system aims to reconstruct image of black holes using a set of telescopes distributed on the Earth. Each pair of telescopes produces a measurement V called visibility, where (a, b) is a pair of telescopes and is the measuring time. To mitigate the effect of measurement noise caused by atmosphere turbulence and thermal noise, multiple visibilities can be grouped together to cancel out noise (Chacl et al., 2018), producing noisc-invariant measurements, termed closure camp
We specify the likelihood of these measurements similar phases (a.be) and log closure amplitudes Vt.(a.b.e.d)'
to Sun & Bouman (2021):
(x) y
eply
A camp ( y
camp
Σ x;
fux
(yylx) =
(22)
2
2ph
2,92
camp
2
11

---

Published in Transactions on Machine Learning Research (05/2025)
Ground Truth
FaKG Sample
EnKG Sample 2
BakG Sample 3. EiKK Meon Sample
DPS-cCSG
SCG
DPG
Ground Iruth
EnKG Sample
EnKG Sample 2
BnKG Sample
lanK Mean Sample
DPS cCS
S0G
DPG
Visualization of generated samples on the black-hole imaging inverse problem. The first row shows the results the original resolution, while the second row shows the blureed images in the intrinsic resolution of the imaging system.
Figure 4:
Table 3: Quantitative evaluation of the reconstructed black-hole images.
PSNR f
Blurred PSNR
Xaph +
Xeamp
Central-GSG
24.700
30.011
1.616
79.669
SCG
20.201
20.976
1.103
1.134
DPG
13.222
14.281
5.116
15.679
EnKG (Ours)
29.093
32.803
1.426
1.270
where A,ph and Acam measures the closure phase and log closure amplitude of black hole images æ. Seph and Bemmp known parameters from the telescope system. The first two terms the sum of chi-square values for closure phases and log closure amplitudes, and the last term constrains the total flux of the black-hole image. We trained diffusion model on the GRMHD dataset (Wong et al. 2022 with resolution 64x 64 to generate black hole images from telescope measurements.
Evaluation metrics We report the chi-square errors closure phases Xeph and closure amplitudes Xaamp to measure how the generated samples fit the given measurement. We calculate the peak signal-to- noise ratio (PSNR) between reconstructed images and the ground truth. Moreover, since the black-hole imaging system provides only information for low spatial frequencies, following conventional evaluation methodology EHTC 2019), we blur images with circular Gaussian filter and compute their PSNR on the intrinsic resolution of the imaging system.
Results Figure shows the reconstructed images of the black-hole using our EuKG method and the baseline methods with black box access. EnKG is able to generate black hole images with visual features consistent with the ground truth. Table 3 shows the quantitative comparison. EnKG achieves relatively low measurement error (i.c., consistency with observations) and the best (blurred) PSNR compared with bascline methods (i.e., realistic images). SCG achieves slightly better data fitting metrics, but produces much noisier images than those by EnKG (Figure
6 Conclusion and discussion
In this work, we propose EnKG, a fully derivative-free approach solve general inverse problems that only permit black-box access. EnKG can accommodate different forward models without any re-training while maintaining the expressive ability of diffusion models to capture complex distribution. The experiments on various inverse problems arising from imaging and partial differential equations demonstrate the robustness and effect iveness of our methodology.
Despite its strengths, EnKG has certain limitations. First, as an optimization-basee approach, does not aim recover the full posterior distribution and therefore cannot provide reliable uncertainty quantifica- tion- important feature in some applications. Second, while EnKG reduces per-sample computational
12

---

Published in Transactions on Machine Learning Research (05/2025)
cost compared standard gradient-based methods, its total runtime is higher due to maintaining and up- dating ensemble of interacting particles. Future work could explore adaptive strategies to dynamically adjust the number of particles based on the optimization landscape to improve efficiency. Additionally, in-
tegrating fast diffusion model sampling techniques (Zheng et al. 2023 Song et al. 2023c Yin ct al. 2024] may further reduce computational overhead.
References
Eugene L Allgower and Kurt Georg, Numerical continuation methods: an introduction, volume 13. Springer Science & Business Media, 2012.
Albert S Berahas, Liyuan Cao, Krzysztof Choromanski, and Katya Scheinberg. A thcorctical and empirical comparison of gradient approximations in derivalive-Iree optimization. Foundations of Compulational Mathematics, 22(2):507-5600 2022.
Kay Bergemann and Sebastian Reich. mollified ensemble kalman filter. Quarterly Journal of the Royal Meteorological Society, 136(651):1636-1 1643, 2010.
El Houcine Bergou, Serge Gratton, and Jan Mandel. On the convergence of non-linear ensemble kalman smoother. Applied Numerical Mathematics, 137:151-168, 2019.
Arwen Bradley and Preetum Nakkiran. Classifier-free guidance is a predictor-corrector, arXie preprint arXiv:2408.000 090000 2024.
Neil Chada and Xin Tong. Convergence acceerattin of ensemble kalman inversion in nonlinear settings. Mathematics of Computation, 91 (335):11477-1188 2022.
Neil K Chada, Andrew M Stuart, and Xin T Tong. Tikhonov regularization within ensemble kalman inversion. SIAM Journal on Numerical Analysis, 38(2):1263-12M)) 2020.
Andrew A. Chael, Michael D. Johnson, Katherine L. Bouman, Lindy L. Blackburn, Kazunori Akiyama, and Ramesh Narayan. Interferometric imaging directly with closure phases and closure amplitudes. The Astrophysical Journol, (53(1): apr 2018. doi: 11.384//138844355 4357/aab6as, aab6a8. URL htttps://dx: doi. org/
10. 10.3847/1538-4377aabbb0
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Marc Louis Klasky, and Jong Chul Ye. Diffu- sion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2022a.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 35:25683 25696, 2022b.
Hyungjin Chung, Jcongsol Kim, Schui Kim, and Jong Chul Ye. Parallel diffusion models of operator and image for blind inverse problems. In Proceedings of the IEEE/CVI Conference on Compuler Vision and Pattern Recognition, pp. 6059 6069, 2023a.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Marc Louis Klasky, and Jong Chul Ye. Diffu- sion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Represenlations, 2023b. URL
Hyungjin Chung, Jong Chul Ye, Peyman Milanfar, and Mauricio Delbracio. Prompt-tuning latent diffusion models for inverse problems. In International Conference on Machine Learning, pp. 8941-8967 PMT.R, 2024.
Giannis Daras, Hyungjin Chung, Chich-Hsin Lai, Yuki Mitsufuji, Jong Chul Ye, Peyman Milanfar, Alexan- dros G Dimakis, and Mauricio Delbracio. A survey on diffusion models for inverse problems. arXiv preprint arXiv:241000000 2024.
13

---

Published in Transactions on Machine Learning Research (05/2025)
The Event Horizon Telescope Collaboration EHTC. First m87 event horizon telescope results. iv. imaging the central supermassive black hole. The Astrophysical Journal Lellers, 875(1):L44 apr 2019. doi: 10.3847/ 2041-8213/ 0041-8213/abbe85. ab0e85. URL https://dx.doi: doi. /tt.:///r..//..//////00001111//0
Geir Evensen. Sequential data assimilation with nonlinear quasi-geostrophic model using monte carlo methods to forecast error statistics. Journal of Geophysical Research: Oceans, 19(CG):10133-10162, 1994.
Geir Evensen. The ensemble kalman filter: Theoretical formulation and practical implementation. Ocean dynamics, 53:343-361 367, 2003.
Geir Evensen and Peter Jan Van Leeuwen, Assimilation of geosat altimeter data for the agulhas current using the ensemble kalman filter with a quasigeostrophie model. Monthly Weather Review, 124(1):85 96, 1996.
Yinnian He and Weiwei Sun. Stability and convergence of the crank dicolson/adamm bashforth scheme for the time-dependent navier-stoker stokes equations. SIAM Journal on Numerical Analysis, 45(2):837-886 869, 2007.
Jonathan Ho, Ajay Jain, and Pieter Abbecl. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840- 0-000,661111 2020.
Yujia Huang, Adishree Ghatare, Yuanzhe Liu, Ziniu Hu, Qinsheng Zhang, Chandramouli S Sastry, Siddharth Gururani, Sageev Oore, and Yisong Yue. Symbolic music generation with non-differentiable rule guided diffusion. arXiv preprint ar arv::22402 14285, 14285, 2024.
Marco A Iglesias. Iterative regularization for ensemble data assimilation in reservoir models. Computational Geosciences, 19:177-212, 2015.
Marco A Tglesias. A regularizing iterative ensemble kalman method for pde-constrained inverse problems. Inverse Problems, 32(2):025002, 2016.
Marco A Tglesias, Kody JH Law, and Andrew M Stuart. Ensemble kalman methods for inverse problems. Inverse Problems, 29( (9)40005,,,, 2013.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in Neural Information Processing Systems, 35:26565-26577 2022.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 2022.
Walter H. Kohler. A preliminary evaluation of the critical path method for scheduling tasks on multiprocessor systems. JEEE Transactions on Compulers, (00(12):1235 1238, 1975.
Nikola B Kovachki and Andrew M Stuart. Ensemble kalman inversion: a derivative-free technique for machine learning tasks. Inverse Problems, 35(9):095005, 2019.
Kody JH Law, Hamidou Tembine, and Raul Tempone. Deterministic mean-field ensemble kalman filtering,
SIAM Journal on Scientific Computing, 38(3):A1251-A122) 2016.
Jose Lezama, Tim Salimans, Lu Jiang, Huiwen Chang, Jonathan Ho, and Irfan Essa. Discrete predictor- corrector diffusion models for image synthesis. Tn The Fleventh International Conference on Learning Representations, 2022.
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective on solving inverse problems with diffusion models. In The Twelfth International Conference Learning Representations, 2023.
Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics, 17(2):527-566 2017.
Dean S Oliver, Albert C Reynolds, and Ning Liu. Inverse theory for petroleum reservoir characterization and history matching. 2008.
14

---

Published in Transactions on Machine Learning Research (05/2025)
Neal Parikh, Stephen Boyd, et al. Proximal algorithms. Foundations and trends8 in Optimization, 1(3): 127-239, 2014.
Xinyu Peng, Ziyang Zheng, Wenrui Dai, Nuoqian Xiao, Chenglin Li, Junni Zou, and Hongkai Xiong, Improving diffusion models for inverse problems using optimal posterior covariance. ar preprint arXiv:2402.0214 021444 2024.
Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkottai. Solving linear inverse problems provably via postcrior sampling with latent diffusion models. In Thirty- seventh Conference on Neural Information Processing Systems, 2023. URL httpp://opeereviers net/
forumPid=xMBPaYTtto
Litu Rout, Yujia Chen, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Beyond first-order tweedie: Solving inverse problems using latent diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024a.
Litu Rout, Yujia Chen, Nataniel Ruiz, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Rb-modulation: Training-fie free personalizatton of diffusion models using stochastic optimal control. arxiv preprint arXiv:2405. 17011, 2024b.
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Πlya Sutskever. Evolution strategies as scalable alternative to reinforcement learning. arXiv preprint arxiv: 703. 700.03864, 2017.
Claudia Schillings and Andrew M Stuart. Analysis of the ensemble kalman filter for inverse problems. SIAM Journal on Numerical Analysis, 55(3):1264 1290, 2017.
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoiinvess--uidded diffusion models for inverse problems. In International Conference on Learning Representations, 2023a. URL https:
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morleza Mardani, Ming-Yi Yu Liu, Jan Kaulz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference on Machine Learning, pp. 32483 32498. PMLR, 2023b.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. Solving inverse problems in medical imaging with score-based generative models. arXiv preprint arXim:211108000 2021.
Yang Song, Prafulla Dhariwal, Mark Chen, and Tlya Sutskever. Consistency models. In International Conference on Machine Learning, pp. 32211- 122252. PMLR, 2023c.
He Sun and Katherine L Bouman. Deep probabilistic imaging: Uncertainty quantification and multi-modal solution characterization for computational imaging. In Proceedings of the AAAI Conference on Arlificial Intelligence, volume 35, pp. 2628 2637, 2021.
Yu Sun, Zihui Wu, Yifan Chen, Berthy T Feng, and Katherine Bouman. Provable probabilistic imaging using score-based generative priors. arXiv preprint ar 1xXi:233001188/0 2023.
Haoyue Tang, Tian Xie, Aosong Feng, Hanyu Wang, Chenyang Zhang, and Yang Bai. Solving general noisy inverse problem via posterior sampling: policy gradient viewpoint. In International Conference on Artificial Intelligence and Statistics, pp. 2116 2124. PMLR, 2024.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. In The Fleventh International Conference Learning Representations, 2022.
15

---

Published in Transactions on Machine Learning Research (05/2025)
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. TEEE Transactions on Image Processing, 13(4):600-61) 612, 2004. doi: (0.1109/TIP. 2003.819861.
George N. Wong, Ben S. Prather, Vedant Dhruv, Benjamin R. Ryan, Monika Mošcibrodzka, Chi kwan Chan, Abhishek V. Joshi, Ricardo Yarza, Angelo Ricarte, Hotaka Shiokawa, Joshua C. Dolence, Scott C. Noble, Jonathan C. McKinney, and Charles F. Gammie. Patoka: Simulating electromagnetic observables of black hole accretion. The Astrophysical Journal Supplement Series, 259, apr 2022. doi: 10.3847/ URL lhttps://dx: http://xxxdoii.ooo www.dd......o.goooooo org/o0....000......o... 3847 / 401088-4365/cccc5222 ac522.. 4388//5882..
1538-4365/
Zihui Wu, Yu Sun, Yifan Chen, Bingliang Zhang, Yisong Yue, and Katherine I. Bouman. Principled proba- bilistic imaging using diffusion models as plug-and-play priors. arXiv preprint ar (xXx:22403118555 2024.
Tianwei Yin, Michaöl Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William Freeman, and Taesung Park. One-step diff'usion with distribution matching distillation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6613 0623,622,-4 2024.
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features a perceptual metric. In 2018 TEFE/CVP Conference on Computer Vision and Pattern Recognation (CVPR), pp. 586 595, Los Alamitos, CA, USA, jun 2018. IEEE Computer Society. doi: 10.1109/CVPR. 2018.00068. URL http::://ol..eeeccoppttttoooooo
Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipc: unified predictor-correcti framework for fast, sampling of diffusion models. Advances Newral Information Processing Systems, 36: 49842 49869, 2023.
Wenliang Zhao, Haolin Wang, Jie Zhou, and Jiwen Lu. De-solver: Improving predictor-coreectr diffusion sampler via dynamic compensation. In European Conference on Computer Vision, pp. 450-466. Springer, 2024.
Hongkai Zheng, Weili Nic, Arash Vahdat, Kamyar Azizzadenesheli, and Anima Anandkumar. Fast sampling of diffusion models via operator learning. In International conference on machine learning, pp. 42390- 42402. PMLR, 2023.
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Lue Van Gool. Denoising diffusion models for plug-and-play image restoration. In IEEE Conference Computer Vision and Pattern Recognition Workshops (NTIRE), 2023.
16

---

Published in Transactions on Machine Learning Research (05/2025)
Table 4: Table of notations.
Notation
Description
G
the forward model of the inverse problem Probability ODE solver Composition of G and &
Df
Jacobian matrix of function f
per
Lebesgue space of periodic r-integrable functions
H
Sobolev space of r-times weakly differentiable periodic functions Covariance matrix of the Gaussian noise model
per
F
Weighted Euclidean inner product, (r:)r = (-,
A
approximate gradient of Gaussian smoothing factor number of gradient estimation queries
1
Q
log-likelihood gradient scale at step
N
number of sampling steps
Exo(f(())
gradient estimator of f(x) using smoothing factor μ and Q queries
A Appendix /
supplemental material
A.1 Notation
A.2 Proofs
Lemma 1. Under Assumption 1/ tr (c()) as follows,
1 and
suppose the correction step is implemented with wi
x. + wi C(( y v (1! )).
x 1
(23)
where E {1,.... JJ and
C
m. (v: (v(aa()) -
"
x
Then tr (C#)
monotonically decreases to zero in the limit as i goes to infinity.
We first start from the ensemble update of the correction step given in Eq. 23 at iteration as follows
Proof.
# (1 + w;C (y- (a. ().
(24)
where j e JJ. The covariance matrix at the next iteration given by
J
cli+1) = ~
( (~ 2411
(1)
(25)
17

---

Published in Transactions on Machine Learning Research (05/2025)
Plugging the update rule in Eq. 23 into Eq. 25 we have
C(++1)
Σ (ao))) x;) + w. C, ( -ba (aa) 2,) u. w. (b: - -xx(()))
ー ト ト Σ (ax (x) - =z)² + C (0) )((o (3r v(oo")) c
+
1
w1O (0h ( x() (z. - xi) + ux(x! =z)(0; xxxx(²)) T
w
(26)
1=1
We notice that
J
Σ
w.C
( (we ((x)))) (2. 199-20 xz)² = -w,CChy C())
j
zy
j=1
1
Σ
(z..)) - =ii) (Ør v(aot))
C ay
= -w: C(i ( C(i) 22
W4
/=1
Therefore, we rewrite Eq.
26 as follows:
cli+1)
C 2w:C C( + wf C 29 my
my 29
Further, by linearity of trace, we have
tr (C1t4))) - tr (C!1) 22 2wser
(CH)CITT) + wntr (Cli) 2g
By cyclic and submultiplicative properties, we have
wrtr (C.i)
((()) C(1)) 49 sy w tr (c())
C13) S wftr (c(3) ((C() c(1)
ay
Since wA = 1/ tr (clb) ) we have
c(i+1)
(1(1) 2tar
2
tr (Clt) ((1)T
Ir (c()) tr (Cl)
C()) Cay
tr
S tr
+
str
(1)
Ir
tr
(1) 1a
(K) tr ( ((())())
(27)
tr
By Assumption and we know that both tr (cl+) and tr (cl+) are upper bounded. By Assumption 3 tr (cl) C(STT is lower bounded unless all the ensemble members collapse to a single point. Thus, there exists a > 0 such that tr (1)) C(S 2 a--i tr (c10) tr (CW) Therefore,
tr (Clit1) < tr
(((1) 23
tr (c1b)) tr (cLi) C(+TT) s (1 a) tr (C(i ax
Note that from Eq. 227 we have a S. 1. Therefore, ir (C0 monotonically decreases zero. Additionally, empirically check how quickly the average distance decays we iterate in our experiments as shown in Figure 5
18

---

Proposition 1. Under Assumption 1 and 3 suppose the correction step is implemented as follows with my = 1/ (tr (c())) Published in Transactions on Machine Learning Research (05/2025)
FFHQ256 (SR)
Navier Stokes
Black Hole
396.3
38
bnrabons
3.08
FFHQ256 (SR)
34
Navier Stokes
Black Hole
3.05
2.4
130
042
200
Enrshons
Figure 5: Empirical verification of Lemma and Assumption 3 quickly decays over update steps. Bottom: while larger tr(Cza) does not always indicate larger tr(Cy)); tr( tr(Cg)) close zero only when tr(Czz) is close zero.
Top: the distance of ensemble members
++ F.
(1)
Yy 2 (2((61))
(28)
(()
+ w/s
4 (2(()))
G. y
4 (1,10) )
2
10) xc
(29)
where
1 Σ
x2/(01 (1) x.) (v (2))00) =i)²
After sufficient iterations, we have the following approzimation:
(( yy 2 (2((11) = = = 1 Σ (v (x(())) G.y v (00/0)) (z(0) xz
(30)
C Va 1 (y)
(31)
Proof. Note that we can always normalize the problem so that T identity. Therefore, without loss of generality and for the ease of notation, we assume throughout the whole proof. Given the inverse problem setting in Eq. where the observation noise is Gaussian, we rewrite the preconditioned gradient
19

---

Published in Transactions on Machine Learning Research (05/2025)
10) w.r.t 23 as
Cl1 V log& ( (1)
(32)
1
F3z
(k))
x
(k)
Fi
T 1
4/ ( (1) 3/|
(33)
J
2
1
Σ
((k) Tz: ( x
((K)
Tt
D
sb (a!((1) ( (2:200 - 3)
(34)
Σ
((M)
xi D (2)( x; (R) Do (z)(() x.) (0 (z.(3) 2)
(35)
(k))
x.)
(Du
( ) (a,((a) - (1)) (w (m. ( - 2)
(36)
J
1=1
By definition, we have
Ir
((1) 22
= 1r
1
Σ
j
(1)
きz (a.5)) Ti
J j=1
((z!) - xz ) T (xx!0) xi)
tr
1=1
1 J
| (1) - x
which represents the average distance between ensemble members. By Lemma know that tr monotonically decreases to zero in the limit. Therefore, the ensemble members will get sufficiently close as we iterate. Therefore, we can apply first-order Taylor approximation to to at a²(6) under Assumption and obtain
(C
vb ( (Kk)) = +b
(1j)
(M) + Díb (2) (a (e(())=((0) 2((") + O (II, (00) - x 16) 13)13).
-z, 22,60)
a + L
((:)
where & E JJ. Therefore for any k, l E (l,.... J}, by applying the approximation above at both [(() and 20(0) we have
(k))
1/ (m,:0)
Dub (x:()) (ar)"
We then plug it into Eq.
36
log
yy(21(1)) (33)
T (w (1x150) 3) Fr - -m:) (vl (1x (())) Φ (0c.") ))
I(N)
J 1= 1=1
- Σ
(1)
oh:) (vi (2o - -wi) T (vb (an,00) - y
-
Σ
vb (24(1t) - - e (x((50) - 8) (a.1 (((( xx
(G(E)))) - G. y -c(((())) >) (2((1) - :Z.)
20

---

Published in Transactions on Machine Learning Research (05/2025)
Algorithm 3 Contral/Forward-Ge bascline with (tt) and s(t)
Require: G,y, Do, [(i] [we ke P
1: sample 2g (0, t)
for EE :O....:N: - 13 do
to Dg (0i li)
x/ f 0; do (li+1 14)
Prior prediction step
5: Va, logp(y/<e;) t Va, ( (log(ly ko)))
Gradient estimation
2i+1 f at + wi Vz, logp(y)<;;
b Guidance correction step
7: end for
8: return NN
concluding the proof.
A.3 Zero-order gradient estimation baseline
We use the forward Gaussian smoothing and central Gaussian smoothing gradient estimation methods to establish a baseline to compare against. These methods approximate the gradient of a function using only function evaluations and can be expressed the following (Forward-GSG) form
f(x + pmu) - f(x) u.
v((x)
Σ
(37)
11
And Central-GSG:
Q f(x + pu.) f(x pr))) ū,
f(()) = Σ
(38)
2/
For Gaussian smoothing, follows the standard normal distribution and ú; Q u, The smoothing factor and number of queries Q are both tunable hyperparameters.
Posterior sampling requires computation of the scores Vz, logp(æ:) and Va, logp(y Øt); the former is learned by the pre-trained diffusion model, and the latter can be estimated by various approximation meth- ods. In our baseline derivative-free inverse problem solver, we substitute the explicit automatic differentiation used in algorithms such DPS with (37 and (38) We estimate this gradient by leveraging the fact that probability flow ODE deterministically maps every Do; Va, log (t)) is approximated with Gaussian smoothing, and vector-Jacobian product (VJP) is used to then calculate V., p(y xx). Our gradient estimate is defined as follows:
V., log x.) ) = V., log 20) = DTT Zo Vao logp(y 20)
(39)
DT, is the transpose of the Jacobian matrix; 39 can be efficiently computed using automatic diffcrentiation. Note that although automatic differentiation used, differentiation through the forward model does not occur. Thus, this method is still applicable to non-differentiable inverse problems. Furthermore, we choose to perturb å and use a VJP rather than directly perturb æ, so that we can avoid repeated forward passes through the pre-trained network, which very expensive. Pseudocode for these algorithms provided in A Algorithm g
21

---

Published in Transactions on Machine Learning Research (05/2025)
0.5
9.4
0.4
Bel 12 Eroor
0.4
0.3
0.3
0.3
0.2
0.2
0.0 800 1000
3000
4000
2.0
1400 1600 1800
perticles
queries (per particle)
Figure 6: Ablation studies the impact of ensemble size, guidance scale, and the number of diffusion model queries, conducted on the Navier-Stokes problem. Left: Relative L error versus ensemble size; the shaded region indicates the range between the best and worst particle. Middle: Relative L error versus the guidance scale. Right: Relative I,2 error versus the number of diffusion model queries per particle; error bars indicate one standard deviation.
Table 5: Qualitative evaluation on FFHQ 64x64 dataset. We report average metrics for image quality and samples consistency on four tasks. Measurement noise level 0.05 used if not otherwise stated.
Inpaint (box)
SR (x2, 0.01)
Deblur (Gauss)
Phase retrieval
PSNR1
SSIM
LPIPS]
PSNR1
SSIM
LPIPS]
PSNR
SSIM
LPIPSU
PSNR
SSIM
LPIPS,
Forward- GSG
19.62
0.612
0.189
25.25
0.836
0.093
20.27
0.606
0.170
10.307
0.170
0.493
Central-GSG
21.37
0.764
0.095
27.11
0.916
0.030
20.88
0.729
0.123
11.36
0,283
0.619
DPG
0.799
SCG
20.27
0.734
0.098
27.02
0.910
0.036
20.73
0.754
0.100
0.233
0.617
EnKC(Ours)
23.53
0.822
0.067
29.52
0.930
0.036
22.02
0.698
0.136
26.14
0.840
0.122
A.4 EnKG Implementation Details
There are mainly two design choices in our algorithm to be made. The first is the step size wy which controls the extent to which the correction step moves towards the MAP estimator. In the ensemble Kalman literature [Kvaachi & Stuart 2019), the following adaptive step size is widely used, and adopt it for our experiments as well.
((a (()) G|' a(
w
(40)
Secondly, find it useful to perform two correction steps in Eq. (6) when solving highly nonlinear and high- dimensional problems such as Navier Stokes. Therefore, we perform two correction steps at each itcration when rumning experiments on Navier Stokes.
A.5
Baseline Details
A.6
Additional results
We include more qualitative results for inverse problems FFHQ 256x256 dataset in Figure
22

---

Published in Transactions on Machine Learning Research (05/2025)
Observation
Ground truth
SCG
Forward-CSG
Central-GSG
DPG
EnKG
Gaussian
deblur
Super resolution
Inpainting
Phase retrieval
Figure 7: Qualitative results FFHQ 256.
Table 6: Hyperparameter choices for Forward-GSG and Central-GSG (64 64).
Inpaint (box)
SR (x2, = 0.01)
Deblur (Gauss)
Phase retrieval
Forward GSG
0.001
0.001
0.001
0.001
Q
N 10000
10000
10000
10000
1.0
1.0
1.0
1.0
1000
1000
1000
1000
Central GSG
p Q
0.001
0.001
0.001
0.001
10000
10000
10000
10000
Wi
1.0
1.0
1.0
1.0
N
1000
1000
1000
1000
A.7
Details of black hole imaging
The measurement of black hole imaging is defined as Sun & Bouman, 2021
oph
L(Vi, a,6 Vot pet = cph
b(a,b.c) (x)
(41)
(a,(a,
[i Vo aal Vi a.c VVV
y,
camp (a,bb,
log
camp
((a,bee,d) (z)
(42)
where Vas the visibility defined by
(x) = g expepii(c) - +()) I (2) Ja.b
(43)
Ia: 9b are telescope-based gain errors, Φi oj are phase errors, and na.b bascline-based Gaussian noise. The measurements consist of () (oln22), 2)/2 closure phases rh and M(M 3)/2 log closure amplitudes camp for array of M telescopes. Our experiments use M = telescopes from Event Horizon Telescope.
23

---

Published in Transactions on Machine Learning Research (05/2025)
Table 7: Hyperparameter choices for baselines Forward- Gorward-GSG and Central-GSG (256 x 256).
Inpaint (box)
SR (x4, 0.05)
Deblur (Gauss)
Phase retrieval
Forward-GSG
P
0.01
0.01
0.01
0.01
Q
10000
10000
10000
10000
Wi
1.0
1.0
3.0
0.7
N
1000
1000
1000
1000
Central-GSG
0.01
0.01
0.01
0.01
10000
10000
10000
10000
1.0
1.0
3.0
0.7
N
1000
1000
1000
1000
Ground truth
J
256
= 512
== 1024
= 2048
J= 4096
Figure 8: Vorticity field predicted by EnKG with different number of particles. From left to right, the result gets better as we increase the number particles.
A.8 Additional comparison
To provide a more comprehensive evaluation, we provide comparisons against several gradient- based methods across different tasks.
Image restoration FFHQ256 Table S presents comparisons with DPS (Chung et al. b023b and
DilIIIR (Zuu et al. 2023 on four image restoration tasks: inpainting, super-resolution (x4), deblurring, and phase retrieval. We observe that EnKG achieves performance comparable to gradient- based methods, with no single approach emerging as a clear winner across all tasks. This demonstrates that EnKG offers competitive performance while maintaining its derivative-Iree property.
Navier-Stokes equation Table reports comparison with DPS and PnP-DM (w et al. 2024g the Navier-Stokes equation problem. We observe that EnKG clearly outperforms the gradient- based methods, while PnP-DM encounters numerical instability, resulting in either crash or timeout. Since DPS and PnP- DM do not have such experiments in their paper, we perform a grid search for its guidance scale over range 1 to find the best choice. For PnP-DM, we explore all hyperparameter combinations mentioned in their paper; however, all result in a numerical crash within the PDE solver. Although reducing the Langevin Monte Carlo learning rate improved stability, it led to infeasible runtimes (e.g., exceeding 100 hours). Consequently, mark PnP-DM autograd encounters out-of-memory issues when the pscudospectral solver unrolls beyond approximately 6k steps an A100-40GB GPU. This limitation suggests that, gradient-based methods may not be applicable to more complex problems that require a large number of PDE solver iterations.
"aashed/timeout"" in Table g Additionally, in this problem,
Black hole imaging As shown Table shows additional comparisons for the black hole imaging problem, including DPS and PnP-DM. Once again, EnKG delivers performance comparable to gradient-basse methods. For DPS, we performed grid search to optimize hyperparameters, while for PnP-DM, we used
24

---

Published in Transactions on Machine Learning Research (05/2025)
the settings provided in their paper. These results further demonstrate the robustness and competitiveness of EnKG across diverse scientific inverse problems.
Discussion As shown in the experiments above, gradient-based methods do not always perform better than derivative-free methods. We believe this due to two reasons. First, gradient-based methods are more prone to getting stuck in local optima, especially in noisy, non-smooth settings. In contrast, many derivative- free methods, including EnKG, incorporate implicit smoothing, which can improve robustness such cases. Second, gradient- braseent-bbsed methods often rely on Gaussian approximations via Tweedie's formula, as taking gradients through the full unrolled reverse process is computationally infeasible. In contrast, EnKG gets better likelihood approximation by simulating the reverse process without the need of backpropagation. More recent. gradient-based works, such as Rout et al. (2024a): Chung et. al. (2024), focus on solving linear inverse problems with text-to-image latent diffusion models. In contrast, EnKG does not rely on text-to-image latent diffusion model, which is generally unavailable in many inverse problem applications.
Table 8: Additional comparison with a few gradient-based methods on FFHQ 256x256 dataset. We report average metries for image quality and consistency on four tasks. Measurement noise is a = 0.05 unless otherwise stated.
Impaint (box)
SR (x4)
Deblur (Gauss)
Phase retrieval
PSNR1 SSIM
LPIPSI
PSNR1
SSIM
LPIPS)
PSNR
SSIM
LPIPS]
PSNR1
SSIM
LPIPSI
Gradient-based
DPS
21.77
0.787
0.213
21.90
0.710
0.265
25.46
0,708
0,212
14.14
0.401
0.486
DiRPIR
22.87
0.653
0.268
26.48
0.744
0.220
24.87
0.690
0.251
22.20
0.733
0.270
Black-box access
(Ours)
21.70
0.727
0.286
27.17
0.773
0.237
26.13
0.723
0.224
20.06
0.584
0.393
Table 9: Additional comparison of relative L2 error on the Navier-Stokes inverse problem. parentheses represent the sample standard deviation.
Numbers in
Ohosse
Gotsse 1.0
Onoise 2.0
Gradient-based
DPS
0.308 (0.214)
0.349 (0.246)
0.382 (0.228)
PuP-DM
Crashed timoout
Crashed timcout
Crashed or timeout
Black-box access
Enurs(Our))
0.120 (0.085)
0.191 (0.057)
0.294 (0.061)
Table 10: Additional comparison with a few gradient- based methods on the black hole imaging problem.
PSNR f
Blurred PSNR
Xepa
Xeamp
DPS
23.984
26.220
1.212
1.079
PnP-DM
28.211
32.499
1.120
1.224
EnKG (Ours)
29.093
32.803
1.426
1.270
A.9 Robustness to the pretrained prior quality
In this section, conduct controlled experiment Navier-Stokes equation problem to investigate the performance dependence on the quality of pre-trained diffusion models. Specifically, we trained a diffusion model prior using only 1/10 of the original training set and limited the training to 15k steps to simulate a lower-quality model. We evaluate the top two algorithms, EnKG and DPG, with the same byperparamoters used the main experiments.
Robust performance As shown in Table 11 we observe that while both algorithms experienced per- formance drop due to the reduced quality of the diffusion model, the decline was relatively small compared to the significant reduction in training data. Notably, our EnKG demonstrated greater robustness, with
25

---

Published in Transactions on Machine Learning Research (05/2025)
smaller performance drop than the best baseline method, DPG. These results indicate that while EnKG benefits from high-quality diffusion models, it is not overly sensitive to their quality. It maintains strong performance even with reduced model capabilities.
Table 11: Relative L2 error of DPG and EnKG (ours) with different diffusion model quality.
Original model trained with full data New model trained with 1/10 data
DPG
0.325 (0.188)
0.394 (0.178)
EnKG (Ours)
0,120 (0.085)
0.169 (0.117)
Table 12: Per-particle computational resource usage of EnKG across three inverse problems. We use 1024 particles for FFHQ and black hole imaging, and 2048 particles for Navier-Stokes. NFE-DM: number of diffusion model evaluations. NFF-Fwd: number of forward model evaluations. Runtime averaged per particle.
NFE-DM
NFE-Forward
Runtime (s)
Peak GPU memory (GB)
FFHQ 256x256
1632
144
11.6
23.9
Navier-Stokes equation
1632
144
3.6
7.1
Black hole imaging
771
60
1.1
1.4
26


Reviews:
[{"review_id": "vzrP8U9t9s", "reviewer": "Reviewer_STRX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n- This paper introduces ensemble Kalman methods for diffusion-based inverse problems, a novel concept in this area.\n- This paper proposes a unique guidance term formulation that employs statistical linearization, effectively replacing the derivative of the measurement operator with the covariance of evaluations produced by the measurement operator. This derivative-free approach is broadly applicable across various scenarios.\n- The effectiveness of the proposed method is demonstrated in three key scenarios: (1) when the measurement model is known and differentiable, (2) when it is known but impractical to differentiate, such as in PDE-based models, and (3) when the measurement model is a black box, with only observational data available.\n\n### Weaknesses\n- The proposed method proves ineffective in situations where derivative computation is efficient while point evaluation is computationally costly.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Ensemble Kalman Diffusion Guidance (EnKG), a novel derivative-free optimization method for solving inverse problems using pre-trained diffusion models. Unlike traditional methods, EnKG requires only black-box evaluations of the forward model, eliminating the need for explicit gradient computations or pseudo-inverses. It estimates guidance terms through statistical linearization from ensemble Kalman methods, approximating gradients using the empirical covariance matrix of ensemble particles along diffusion ODE trajectories. The effectiveness of EnKG is demonstrated across three types of inverse problems—computational imaging, fluid dynamics (Navier-Stokes equation), and black-hole imaging—highlighting its flexibility and robustness.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I appreciate the paper's contributions. However, I believe it could be further improved by addressing the following.\n\n -  What distinguishes the predictor-corrector scheme presented in this paper from those described in existing works? Specifically, can you provide a comparative analysis and elucidate any connections to the following papers:\n1)  Song, Yang, et al. \"Score-Based Generative Modeling through Stochastic Differential Equations.\" ICLR 2021.\n2) Lezama, Jose, et al. \"Discrete predictor-corrector diffusion models for image synthesis.\" ECCV 2022.\n3) Zhao, Wenliang, et al. \"Unipc: A unified predictor-corrector framework for fast sampling of diffusion models.\" NeurIPS 2023.\n4) Zhao, Wenliang, et al. \"DC-Solver: Improving Predictor-Corrector Diffusion Sampler via Dynamic Compensation.\" ECCV 2024.\n5) Bradley, Arwen, and Preetum Nakkiran. \"Classifier-Free Guidance is a Predictor-Corrector.\" NeurIPSW 2024.\n\n-  I question the plausibility of Assumption 3, particularly in low-dimensional and ill-posed operator scenarios. While it may hold true in the high-dimensional problems explored in this paper, I request plots illustrating the trace or singular values of the covariance matrices for x and y.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "naPJhFJ8Fk", "reviewer": "Reviewer_uaeR", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1.The work creatively combines ensemble Kalman filtering techniques with diffusion model guidance, addressing the limitation of requiring gradient or derivative information, especially in scenarios (e.g., PDE) where gradients are impractical to obtain.\n2.The paper provides a clear derivation of the prediction–correction framework along with supporting theoretical results, which adds depth to the contribution. \n3.The method is evaluated across a diverse set of inverse problems—from standard imaging tasks to challenging scientific problems—demonstrating its efficiency.\n\nWeaknesses:\n1.Writing needs improvement. (1). most readers are unfamiliar with the Ensemble Kalman and there should be a brief introduction to the core idea EnK (2) some symbols appear for the first time but are not explained, e.g., \\hat{p}_t in Eq. 4\n2.Although per-sample cost is reduced, maintaining an ensemble of particles can be computationally heavy and may not scale well in very high-dimensional settings. \n3.The sensitivity of the method to key hyperparameters (such as ensemble size and guidance scaling) is not thoroughly explored, leaving questions about robustness and optimal settings.\n4.Since the Prediction-correction framework is not new [1], you should clarify the significant differences between yours and [1].\n\n[1] Score-based generative modeling through stochastic differential equations. ICLR 2021", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces Ensemble Kalman Diffusion Guidance (EnKG), a novel derivative-free approach for solving inverse problems using diffusion models. Unlike previous methods that rely on privileged information (e.g., derivatives or pseudo-inverses of the forward model), EnKG only requires black-box access to the forward model and a pre-trained diffusion model prior.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The reviewers suggest the following improvements:\n1.Investigate whether the number of particles in EnKG could be dynamically adjusted to improve efficiency.\n2.Improve the writing and address the issues mentioned in Weakness 1.\n3.Provide a detailed sensitivity analysis and ablation study that investigates how hyperparameters—particularly the ensemble size and guidance scaling factors—affect performance and computational cost.", "broader_impact_concerns": "none", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "rK1N40ZBJ6", "reviewer": "Reviewer_huKD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The proposed method is novel and provide insights on applying agradient-free aproaches for the inverse problems.\n2. The performace (both qualitative and quantitative) is good compared to other alternatives.\n3. Experiment settings are comprehensive.\n\nWeakness:\n1. Needs more clarification for how you applied the motivations from the Ensemble Kalman methods since it's the core part of the paper, it will help the readers build the insight and relation on where the motivation come from to explain for \"why this\"?\n2. Though the main focus is on gradient-free method, it good that the authors also comapre with more popular gradient-based method which helps building the bridge for the explanation, but more clarification is needed on the results, and more recent/compatitive baselines would be better for the comparison. For example STSL[1], P2L[2]. I would expect gradient-based models performs better but it doesn't seem to be the case, why?\n\n \n\n[1] Rout, Litu, Yujia Chen, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. \"Beyond first-order tweedie: Solving inverse problems using latent diffusion.\" In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9472-9481. 2024.\n[2] Chung, Hyungjin, Jong Chul Ye, Peyman Milanfar, and Mauricio Delbracio. \"Prompt-tuning latent diffusion models for inverse problems.\" ICML 2025.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposed a new gradient-free method to solve the inverse problems. Specifically, it leveraged the advancement of both pre-trained diffusion model and the Ensemble Kalman methods. The key controbutions of this method is using black-box model without relying on gradient when they are inaccessable, as is in many real-world applications. The two-stage PC framework makes is feasible to avoid the gradient by using the forward model evaluations with the proposed ODE. The overall performance is good compared with other gradient-free alternatives.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. More discussions of the relations to Ensemble Kalman methods.\n2. More disussions with the other stronger gradient-based baselines as noted in the weakeness. Since gradient-based is not the focus, I'm more looking for discussions on the relations and explanations instead of additional experiments, now that it's mentioned in the paper alreday.", "broader_impact_concerns": "no concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: See above.
Decision: Accept (poster)