InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciences
Authors: Hongkai Zheng
Wenda Chu
Bingliang Zhang
Zihui Wu
Austin Wang
Berthy Feng
Caifeng Zou
Yu Sun
Nikola Borislavov Kovachki
Zachary E Ross
Katherine Bouman
Yisong Yue
Venue: ICLR 2025 Spotlight (2025)
Keywords: inverse problem
benchmark
diffusion model
Primary area: datasets and benchmarks

Abstract: Plug-and-play diffusion priors (PnPDP) have emerged as a promising research direction for solving inverse problems. 
 However, current studies primarily focus on natural image restoration, leaving the performance of these algorithms in scientific inverse problems largely unexplored. To address this gap, we introduce \textsc{InverseBench}, a framework that evaluates diffusion models across five distinct scientific inverse problems. These problems present unique structural challenges that differ from existing benchmarks, arising from critical scientific applications such as optical tomography, medical imaging, black hole imaging, seismology, and fluid dynamics. With \textsc{InverseBench}, we benchmark 14 inverse problem algorithms that use plug-and-play diffusion priors against strong, domain-specific baselines, offering valuable new insights into the strengths and weaknesses of existing algorithms. To facilitate further research and development, we open-source the codebase, along with datasets and pre-trained models, at  [https://devzhk.github.io/InverseBench/](https://devzhk.github.io/InverseBench/).

Published as a conference paper at ICLR 2025
INVERSEBENCH: BENCHMARKING PLUG-AND-PLAY DIFFUSION PRIORS FOR INVERSE PROBLEMS IN PHYS- ICAL SCIENCES
Hongkai Zheng'.', Wenda Chu Bingliang Zhang" Zihui Wu".', Austin Wang', Berthy T. Feng', Caifeng Zou', Yu Nikola Katherine L. Bouman', Yisong Yue! 'California Institute Technology, "Johns Hopkins University, "NVIDIA
Nikola Kovachki², Zachary E. Ross',
ABSTRACT
Plug-and-play diffusion priors (PnPDP) have emerged as a promising research direction for solving inverse problems. However, current studies primarily fo- cus on natural image restoration, leaving the performance of these algorithms in scientific inverse problems largely unexplored. To address this gap, we intro- duce INVERSEBENCIL a framework that evaluates diffusion models across five distinct scientific inverse problems. These problems present unique structural challenges that differ from existing benchmarks. arising from critical scientific applications such as optical tomography, medical imaging, black hole imaging, seismology, and fluid dynamics. With INVERSEBENCH, we benchmark 14 in- verse problem algorithms that use plug-and-play diffusion priors against strong, domain-specific baselines, offering valuable weaknesses of existing algorithms. To facilitate further research and develop- ment, we open-source the codebase, along with datasets and pre-trained models, at http:://ddwwwkkkgiiiub.oo///eees
insights into the strengths and
INTRODUCTION
Inverse problems are fundamental in many domains of science and engineering, where the goal is to infer the unknown source from indirect and noisy observations. Example domains include astronomy (Chael et al. 2019), geophysics (Virieux & Operto 2009), optical microscopy (Choi al. 2007), medical imaging (Lustig al.) 2007), fluid dynamics (Iglesias et al., 2013), among others. These inverse problems are often challenging due to their ill-posedness, complexity in the underlying physics, and unknown measurement noise.
The use of diffusion models (DMs) (Sohl-Dickstein et al. 2015 Dhariwal & Nichol 2021 for solv- ing inverse problems has become increasingly popular. One attractive approach is PnPDP methods that use the DM as plug-and-play prior (Wang et al. [022] Dou & Song (0024). where the infer- ence objective is decomposed into the prior (using pre-trained diffusion model) and the likelihood of fitting the observations (using suitable forward model). The advantage of this idea is twofold: (1) As powerful class of generative models, DMs can efficiently encode the complex and high- dimensional prior distribution, which is essential to overcome ill-posedness. (2) As plug-and-pla priors, DMs can accommodate different problems without any re-training by decoupling the prior and likelihood. However, current algorithms are primarily evaluated and compared row set of image restoration tasks such as inpainting, super-resolution, and deblurring (Kadkhodaie) & Simoncelli 2021] Song al. 2023a Mardani et al. 2024). These problems differ greatly from those from science and engineering applications such as geophysics (Virieux & Operto) 2009). as- tronomy (Porth al. 2019). oceanography (Carton & Giese 2008), and many other fields, which have very different structural challenges arising from the underlying physics. It is unclear how much insight can be carried over from image restoration to scientific inverse problems.
fairly nar-
In this paper, we introduce INVERSEBENCH, a comprchensive benchmarking framework designed to evaluate PnP diffusion prior approaches in a systematic and easily extensible manner. We curate
"These authors contributed equally this work.

---

Published as a conference paper at ICLR 2025
Linear inverse scattering
Compressed sensing MRI
Black hole imaging
Full waveform inversion
Nevier-Stokes equation
loli
kapace
Mölinage
Blaccholc Image
Obserned
Compressional
Qssereed
bed
DAPS
Accurtte
Mare Accurote
More Accurato
Accurtte
RED-SY
150 cfs
DOKAA
INTTOM
Qusiin
More Efficient
More Effcient
More Effecient
More Efficient
More Efficient
Figure I: Illustration of five benchmark problems in the INVERSEBENCH. G represents the forward model that produces observations from the source. G represents the inverse map. In the linear inverse scattering problem (left two), the observation is the recorded data from the receivers and the unknown source we aim infer is the permittivity map of the object. The bottom panel displays the efficiency and accuracy plots for our benchmarked algorithms. Certain characteristics of the problem cause the efficiency and accuracy trade-offs of each algorithm to vary across tasks. In these plots, the larger radius of the points indicates greater interaction with the forward function G, as measured by the number forward model evaluations.
a diverse set of five inverse problems from distinct scientific domains: optical tomography, black hole imaging, medical imaging, seismology, and fluid dynamics. These problems present structural challenges that differ significantly from natural image restoration tasks (cf. Figure and Table 2 and encompass broad spectrum of complexities across multiple scientific fields. Most notably, the forward model (which maps the source to observations) is defined using various types of physics- based models which can be highly nonlinear and difficull to evaluate.
We select 14 representative plug-and-play diffusion prior algorithms proposed for solving inverse problems, providing thorough comparison of their performance across different scientific inverse problems and further insights into their efficacy and limitations. Additionally, we establish strong, domain-specifit baselines for each inverse problem, providing meaningful reference point for sessing the effectiveness of diffusion model-based approaches against traditional methods.
Through extensive experiments, we find that PnP diffusion prior methods generally exhibit strong performance given suitable dataset for training a diffusion prior. This performance consistent even we vary the forward model (which is strength of PnP approach), given appropriate tuning. However, for forward models that require certain constraints on the input (e.g., uses a PDE solver), performance can be very sensitive hyperparameter tuning. Moreover, the strength of using a diffusion prior can also be limitation, as PnP diffusion prior methods have difficulty when the source image is out of the prior distribution (i.e., the use of diffusion models makes it difficult to recover "surprising" results). Additionally, we find that PnP methods that use multiple queries of the forward model tend to outperform simpler methods like DPS, the cost of requiring additional tuning and computation, which points to an interesting direction for future method development.
INVERSE implemented as a highly modular framework that can interface with new inverse problems and algorithms to run experiments at scale. We open-source the codebase, along with datasets and pre-trained models, at
PRELIMINARIES
2.1 INVERSE PROBLEMS
Following the typical setup, we have observations y E Im from an unknown source 2 E C via forward model G C - mm The inverse problem is to design mapping Gf to infer z from y:
- Gt(y): where y G(z,g)
2

---

Published as a conference paper at ICLR 2025
Here, & represents noise in the forward model. In scientific applications, G represents the measure- ment or sensing device (telescopes, infrared cameras, seismometers, electron microscopes, etc.). Inverse problems typically present four major challenges: (1) Many inverse problems ill-posed, meaning that a solution may not exist, may not be unique, or may not be stable (Hadamard 2014). For example, in black hole imaging, there could be multiple solutions that match the same sparse measurements. (2) The measurement noise is generally not separately observed (it is part of the ob- servations y), and accounting for it in the inverse problem can be challenging, especially for poorly characterized noise profiles (e.g., non- Gaussaansi.. (3) The forward model might be highly nonlinear and lack closed-form expression, leading to computational and numerical challenges in method design. (4) Designing an appropriate prior for the unknown source is also a critical challenge. For some problems, is necessary for the designed prior to capture the complex structure of the solution space while remaining computationally tractable.
All these challenges necessitate some kind of regularization. While classic optimization approaches often employ simple regularizers (e.g., local isotropic smoothness), these fail to capture global or anisotropic properties. The use of diffusion models as a prior attractive as a way to capture these more complex properties.
2.2 DIFFUSION MODELS
Diffusion models are a powerful class of deep generative models that can capture complicated high- dimensional distributions such as natural images (Rombach et al.] 2022), proteins (Fu et al. 2024). small molecules (Luo et al., 2024), robotic trajectories (Chi et al./ [2023. amongst other domains. Given their strong performance and compatibility with Bayesian inference, using diffusion models to model the solution space prior is a promising idea (Chung et al. 2023 Song al. 2022).
We consider the continuous formulation of diffusion models proposed by Song et al. (2020), which expresses the forward diffusion and backward denoising process stochastic differential equations (SDEs). The forward process transforms a data distribution 2o Pdata into an approximately Gaussian one ær (0, o (T) by gradually adding Gaussian noise according
dæ; f t)dt + g(t)du:,
where f is predefined vector- drift, 9 is the diffusion coefficient, w is the standard Wiener process with time / flowing from 0 to T. The backward process sequentially denoises the Gaussian noise into clean data, which is given by the reverse-time SDE:
dx; (f(or.t) 299 (t)V w. log pi:)) dt +g()dd
(3)
where Pe (xx,) is the probability density of æ. at time and w; is the reverse-time Wiener process. The diffusion model is trained to learn the score function V log Pr (x;). Once trained, the diffusion model can generate new samples from the learned data distribution by solving Eq. (3).
2.3 PLUG-AND-PLAY DIFFUSION PRIORS FOR INVERSE PROBLEMS
We use the term Plug-and- PPay Diffussio Prior (PnPDP) to refer to the class of recent methods that diffusion models (or the denoising network within) plug-and-play priors (Venkatakrishnan ctt al., (2013) for solving inverse problems. The basic idea is to either modify or use Eq. (3) to gen- erate samples from p(z: (y) rather than the prior p(z), which under Bayes rule can be expressed as p(aly) & p())p(U a.). The first term p(z) can modeled using a diffusion prior, and the sec- ond term p(y (z) can be computed using the forward model. Broadly speaking, existing PnPDP approaches can be grouped into four categories described below. Table Iists the 14 representative algorithms we selected, and notes their different requirements the forward model. To avoid con- fusion, we use Courier. font when referring to a specific algorithm in the main text throughout the paper (c.g., PnP- for Wu et (2024)).
Arguably the most popular approach to solving inverse problems with a pretrained diffusion model is cridance-based methods (Song et al.) 2023a Wang al. 2022 Kawar
Guidance- based methods
et al. 2022 Rout et al. 2023) Chung et al.) 2023), which modify Eq. (3) by adding a likelihood score term, m, DggPp [yyn (..). along the diffusion trajectory. This term is related to the forward model G
3

---

Published as a conference paper at ICLR 2025
Table I: Requirements on the forward model of the algorithms evaluated in our experiments.
Category
Method
SVD
Pseudo inverse
Lineur
Gradient
Linear guidance
DDRM
Kawar al
2022
DDNM
Wang
2122
HGGM
Song et al
2023a
x
General guidancce
DPS
Chung
2023
X
X
Song
2023b
DPG
Tang ct
X
SCG Huang et al.
2024
X
X
X
EnKG Zhong al 2024
X
X
X
Viriahle-splittnng
DiAPIR Zhu et
PnP-DM Wa ct
2024
X
X
DAPS Zhang al.
2024
X
X
Variational Bayes
Mardani
2023
Sequential Monte Carlo
FPS Dou & Song 2024
x
MCGDiff (Cardoso
2024
Table 2: Characteristics of different inverse problems in INVERSEBENCH, from left to right: whether the forward model is linear, whether can compute the SVD from the forward model, whether the inverse problem operates in the complex domain, whether the forward model can be solved in closed form, whether one can access gradients from the forward model, and the noise type.
Problem
Linar
Complex domain
Closed-form forward
Gradient
type
Linear scattering
Guussian
Compressed sensing MKI
/
Real world
Black hole imaging
x
Nan-additive
Navier-Stukes equation
Glussian
if the final clean æo is a candidate source z, in which case p(y) can be estimated by querying G. However, log Pr (y (y/ xx, is generally intractable various approximations have been proposed (Song
et al. 2022 Chung et al., 2023: Song et al. 2023a Boys et al. 2023).
Variable splitting is a widely used strategy for solving regularized optimization problems and conducting Bayesian inference (Vono et al. 2019 Chen et al., 2022, Lee al. 2021
Variable splitting
2024 Zhu et al. 2023 The core idea is to split the inference into two alternating steps (Wu et al.
Li et al.) 2024at (oogg et al., 2024| Zhang et al. 2024 Xu & Chil 2024). The first step uses the forward model to update or sample in the neighborhood of the most recent 2t. The second step runs unconditional inference p(æz), which amounts running Eq. (3) for small amount of time.
Variational Bayes Variational Bayes methods approximate intractable distributions such as p(zly) using some simpler parameterized distribution qe (Zhang al.) 2018). The key idea is to find go. that, in a KL-divergence sense, both fits the observations and agrees with the prior p(x). Instead of directly sampling according to Eq. (3). uses the diffusion model as prior within a variational inference framework (Mardani et al. 2023 Feng et al. [223 Feng & Bouman 2024).
Sequential Monte Carlo Sequential Monte Carlo (SMC) methods draw samples iteratively from sequence of probability distributions. These methods represent probability distributions by a set of particles with associated weights, which asymptotically converge target distribution following sequence of proposal and reweighting steps. Recent works have extended SMC methods to the sequential diffusion sampling process (Wu et al., 2023 Trippe et al.) 2023); Cardoso et al.) [024] Dou & Song 2024), enabling zero-shot posterior sampling with diffusion priors. However, these methods are typically applicable only to inverse problems with linear forward models.
3 INVERSEBENCH
In this section, we introduce the formulation and specific challenges of the five scientific inverse problems considered in INVERSEBENCH: linear inverse scattering, compressed sensing MRI, black
4

---

Published as a conference paper at ICLR 2025
hole imaging, full waveform inversion, and the Navier-Stokes equation. The characteristics of these inverse problems are summarized in Table Their computational characteristics are summarized in Figure Detailed descriptions and formal definitions can be found in Appendix B
Linear inverse scattering Inverse scattering is an inverse problem that arises from optical mi- croscopy. where the goal is to recover the unknown permittivity contrast IR" from the measured scattered lightfield Use E C We consider the following formulation of inverse scattering
Ysc H H 2) E C.
where
14x
G( @ z).
Here G E CR and H E Cm are the discretized Green's 's functions that model the responses of the optical system, uin and Usx are the input and total lightfields, O is the elementwise product, and n is the measurement noise. Since this problem is linearized version of the general nonlinear inverse scattering problem based on the first Born approximation, we refer to it as linear inverse scattering. This problem allows to test algorithms designed specifically for linear problems.
Compressed sensing MRI Compressed sensing MRI is a technique that accelerates the scan time of MRI via subsampling. We consider the parallel imaging (PI) setup of CS-MRI, which is widely adopted in research and practice. Mathematically, PI CS-MRI can be formulated as an inverse problem that aims recover image C from
yj = PFS,> + ny E C jol..,,,..,
where P E {0, 10 is subsampling operator and the Fourier transform; yy. Sj. and n; the measurements, sensitivity map, and the noise of the j-th coil, respectively. Compressed sensing MRI is a linear problem, but it poses significant challenges due to its high-dimensional nature, involvement of priors in the complex domain, and attention fine-grained details.
Black hole imaging The measurements for black hole imaging (BHI) are obtained through Very Long Baseline Interferometry (VLBI). In this technique, each pair of telescopes (a, b) provides visibility (van Cittert 1934 Zernike I938): ; a measurement that samples a particular spatial Fourier frequency of the source image related to the projected baseline between telescopes time
V = gi.bee -'d' (%) (z) + n'.
(5)
The ideal visibilities (x), representing the Fourier component of the image x, are corrupted by Gaussian thermal noise nh well as dedescope-dependeee amplitude errors ga. g and phase errors &i of (EHTC 2019a). To mitigate the impact of these amplitude and phase errors, derived data products called closure quantities, namely closure phases and log closure amplitudes, can be used to constrain inference (Blackburn al. 2020
V Vi [ Vi
Vi.la.b.c) L(Vo, 0,b Vit v()c) E R,
logen (r,(a,b.c,d) log
E IR.
(6)
Here, L and |.| denote the complex angle and amplitude. Given total of M telescopes, the number of closure phase measurements (i,(,, at time (M-IXM-2 2) and the number of log logea closure amplitude measurements Vt.(a,b,c.d) is M( after accounting for redundancy. Closure 2 quantities are nonlinear transformations of the visibilities, making a forward model that uses them for black hole imaging non-convex. The inverse problem is further complicated by the need for super-resolution imaging beyond the intrinsic resolution of the Event Horizon Telescope (EHT) observations (i.e., maximum probed spatial frequency). ). as well as phase ambiguities, which can lead to multiple modes in the posterior distribution (Sun & Bouman, 2021; Sun et al., (024).. Another challenge of BHI is that measurement noise is non-additive due to the usage of the closure quantities.
op
2
Full waveform inversion Full waveform inversion (FWI) aims infer subsurface physical prop- erties (e.g. compressional and shear wave velocities) using the full information of recorded wave- forms. In this work, consider the problem of recovering the compressional wave veloeity := v(x) (discretized as z E IR") from the observed wavefield u, (discretized as y E RT")::
y Pu.
5

---

Published as a conference paper at ICLR 2025
where P is the sampling operator for receivers where observational data is available, and is the discretization of the pressure wavefield u. := u(2; t), which is a function of location æ and time t. Here, is the solution to the acoustic (scalar) wave equation that models seismic wave propagation in heterogeneous acoustic media with constant density:
1 8'u 2
vuu=
(8)
2 01
where 9 == q t) is the source function (discretized as q). Eq. 8
can be discretized as:
Au = q,
8 where A represents the discretized operator x गर the free surface, the inverse problem has non-unique solutions. One of the major challenges for FWI is the prohibitive computational expense, especially for large problems, it usually requires numerous calls to the forward modeling process. Moreover, the conventional method for FWI, called the adjoint-state method, casts it local optimization problem (Virieux al.) 2017 Virieux & Operto 2009). This means that a sufficiently accurate initial model is required, as the solution is only sought in its vicinity. FWI conventionally needs start with smoothed model derived from simpler ray-based methods (Liu et al., 2017 Maguire et al. 2022). which imposes a significantly strong prior. A general method with less reliance on initialization highly desired.
V2. Typically we only have observations at
Navier-Stokes equation Navier-Stokes equation is classic benchmarking problem from fluid dynamics (Tglesias et al.) (013). Its applications range from ocean dynamics climate modeling where observations of the atmosphere are used to calibrate the initial condition for the downstream numerical forecasting. We consider the forward model that is given by the following 2D Navier- Stokes equation for viscous, incompressible fluid in vorticity form torus.
0,w(2, t) + uxxxt) (w( t) vAw(x t) f(z),
(0, 2T) t (0,
V u(a () 0, w< 0) = wa))
(0, (0,2m)), [0,
x E (0, 2T)²
where u E C (10, 7); Haer(( (00, 2+);; for any r >> O is the velocity field, w V x u is the 14, is the viscosity coefficient, and vorticity, wo L ((0, 2;; is the initial vorticity,
the forcing function. The solution operator G defined as the operator mapping the vorticity from the initial vorticity to the vorticity at time T, i.e. G wh - wr- We consider the problem of recovering the initial vorticity field z == wy from the noisy partial observation y of the vorticity field wr at time T given by
f E T.ger (0,2z))2
PL(z) +
where P is the sampling operator. n is the measurement noise, and L(-) is the discretized solution operator of Eq. (9) The Navier-Stokes equation does not admit closed-form solution and thus there is no closed-form gradient available for the solution operator. Moreover, obtaining an accurate numerical gradient via automatic differentiation through the numerical solver is challenging due to the extensive computation graph expanded after thousands of discrete time steps.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Here we provide a brief summary of our experimental setup. More details about the inverse problems and their corresponding datasets can be found in Appendix B Technical details of DM pretraining can be found in Appendix B.6
Black hole imaging We leverage a dataset of General Relativistic MagnetoHHydnoDynammi (GRMHD) (Mizuno 2022) simulated black hole images as our training data. The training set consists of 50,000 resized 64x64 images. Since this dataset is not publicly available, gener- ate synthetic images from a pre-trained diffusion model for both the validation and test datasets. Specifically, we use sampled images for the validation set and 100 sampled images for the test set.

---

Published as a conference paper at ICLR 2025
Full waveform inversion We adapt the CurveFaultB dataset (Deng et al. 2022). which presents the velocity maps that contain faults caused by shifted rock layers. We resize the original data to resolution x28x 128 with bilinear interpolation and unti-aliasing. The training set consists of 50,000 velocity maps. The test and validation sets contain 10 and velocity maps, respectively.
Linear inverse scattering We create dataset of fluorescence microscopy images using the online simulator (Wiesner et al., 2019). The training set consists of 10,000 HL60 nucleus permittivity images. The test and validation sets contain 100 and 10 permittivity images, respectively. We curate the test and validation samples so that all test samples have less than 0.6 cosine similarities to those in the training
Compressed sensing MRI We use the multi-coil raw k-space data from the fastMRI knee dataset (Zbontar al. 2018). We exclude the first and last 5 slices of each volume for training and valida- tion as they do not contain much anatomical information and resize all images down to 320 x 320 following the preprocessing procedure of (Jalal et al. 202T). In total, we use 25,012 images for training. 6 images for hyperparameter search, and 94 images for testing.
Navier-Stokes We create dataset of non-trivial initial vorticity fields by first sampling from Gaussian random field and then evolving Eq for five time units. The equation setup follows Igle- sias al. (2013); Li al.) (2024b). We set the Reynolds number 200 and spatial resolution to 128 x 128. The training set consists of 10,000 samples. The test and validation sets contain 10 and samples, respectively.
Pretraining of diffusion model priors For each problem, we train a diffusion model on the train- ing set using the pipeline from (Karras et al. (202). and use the same checkpoint for all diffusion hou-and--layy methods on each problem for fair comparison. See more details in Appendix B.6
4.2 EVALUATION METRICS
Accuracy metries We use the Peak Signal-to-Noise Ratio (PSNR), Structure Similarity Index Measure (SSIM), as the generic ways quantify recovery of the true source. For all the problems except for black hole imaging, we use the l2 error [|G(E) 1---111 yll2 to measure the consistency with the observation For black hole imaging, the closure quantities are invariant under translation, and so we measure the best fit under any shift alignment. We also assess the Blur PSNR, where images are blurred to match the target resolution of the telescope. We evaluate data misfit via the x² statistic two closure quantities: the closure phase (Xip) and the log closure amplitude (Xiogaa) ). A x" value close to I indicates better data fitting. To facilitate comparison between underfitting (x* > 1) and overfitting (x" < 1), we report a unified metric defined as
x² = x2 11x 13+ 1
1(x < 1}.
(10)
For FWI and Javier-Stoker experiments, we also use the relative & error 1 2||2/ || as it is commonly used primary accuracy metric in PDE problems (Iglesias et al. 2013).
Efficiency metrics We define a set of efficiency metries in Table evaluate the computational complexity of inverse algorithms more thoroughly. These metrics fall into two categories: (1) to- tal metrics that measure the overall computational cost; (2) sequential metries that help identify bottlenecks where forward model or diffusion model queries cannot be parallelized.
Ranking score To assess the relative ranking of different PnP diffusion models across various problems, we define the following ranking score for each problem. Given set of accuracy or efficiency metrics ( we rank the algorithms according to each individual metric. Suppose algorithm has the rank R2 (1) out of I, algorithms under the metric k. Its ranking score on this 100 (L - (!(() + +1) 1)/L. For each problem, we calculate the
metric is given by score;(1) average ranking score to assess overall performance:
K
1
mrohem (1)
score
K

---

Published as a conference paper at ICLR 2025
Ground truh
FIS1A
D2190
DUNM
TK-1M
DP'S
Linear inwerse scatbering
Pseudo
PHP
DAPS
RED-drf
FF5
MCG di
Compressed sensing Mil
CSGM
DPS
DHMPIR
DAPS
Blackk hale imaging
Ground muth
EKTilmoging
SMILI
DPS
LGD
DHFR
PMP-OM
DAPS
RED dtf
Ground truth
Adam
Adam
LaiGs*
DPS
Ground buth
EdI
Fall waveformm inversion
Naviler Stohee Seuaaaio
SCG
DP5
Figure 2: Qualitative comparison showing representative examples of PnP-DP methods and domain- specific baselines across five inverse problems. Note that for full waveform inversion, Adam" and LBFGS' initialized with coussian-blurred ground truth, serving references.
4.3
MAIN FINDINGS
The full experimental results for each problem provided in Appendix A.1 as tables. Below, highlight some key insights distilled from these results.
How do PnPDP methods work compared to conventional baselines? Our primary finding is that, given a suitable dataset for training a DM prior, PnPDP methods generally outperform conven- tional baselines. This evident in Figure [ where the PnPDP methods generally lie higher along the vertical axis. This finding is as expected given that the baselines do not incorporate such strong prior information.
8

---

Published as a conference paper at ICLR 2025
However, if the classic optimization baselines initialized well, then they sometimes outperform PnPDP methods, most of which cannot naturally incorporate an initialization beyond white noise. For example, in FWI, PnPDP methods clearly outperform the classic baseline methods if the base- lines are initialized randomly or from a constant. But initialized with a good guess (e.g.. heavily blurred ground truth image), they consistently outperform the current PnPDP methods. That being said, the fact that PnPDP methods rely much less on initialization than the traditional optimization methods is already an intriguing property. See qualitative comparison Figure [ and quantitative comparison in Table 7
How do PnPDP methods compare with each other? In the problems where the forward model has a closed-for foremd-fomm expression, methods that require more gradient queries, such as DAPS and P FnP-DM, tend to more accurate. However, since they have more queries the forward model, they are also Additionally, these methods require more careful tuning as they usually have larger hyperparameter spaces, as shown Table 12
more expensive, as shown in Figure [
In the problems where the forward model has no closed-form expression, particularly forward model defined by a PDE system and implemented as a numerical PDE solver, this trend does not hold. In fact, DAPS and PnP-DM perform poorly, as shown in Figure and Table 7 These meth- ods also demonstrate an increased level of numerical instability and sensitivity to hyperparameters, shown in Figure minor adjustments in step size can lead to either unconditional generation results that ignore measurements (with slightly smaller steps) or complete failure (with slightly larger steps). This performance degradation stems from critical limitation in many current PnPDP algorithms: they do not account for stability conditions required to query a forward model. For example, in the FWI and Navier-Stokes equation, the input of the forward model must satisfy the Couran-Frieddichs-Leew
(CFL) condition (Courant et al. 1967) to produce stable solutions. This issue is particularly pronounced for methods like DAPS and PnP-DM, which incorporate Langevin Monte Carlo (LMC) as an inner loop as LMC introduces additional Gaussian noise at each step, further exacerbating instability compared other PnPDP methods.
Ground truth
Stable region but poor results
Better results but unstable
Failure region
step size: 1e-4
step size: 3e-4
step size: 5e-4
Increase step size
X Figure 3: Illustration of the failures of PnPDP methods (DAPS as an example) on full waveform inversion. With small learning rate, DAPS numerically stable but does solve the inverse problem effectively. With slightly larger learning rate, DAPS produces noisy velocity map that breaks the stability condition of the PDE solver, resulting a complete failure.
Stability condition
How does the performance vary with different levels of measurement sparsity? As measure- ment sparsity increases, making the inverse problem more ill-posed, we observe an increasingly wide performance gap between PnPDP methods and baselines. Figure illustrates this trend across three problems, showing that the average performance gain of top PnPDP methods over baselines grows with increasing measurement sparsity.
How well do PnPDP methods deal with different forward models? For linear inverse prob- lems, our results demonstrate that PnPDP methods can effectively deal with varying forward models without the need for parameter tuning. To validate this, we conduct controlled experiment in CS- MRI, where we maintain consistent measurement sparsity while altering the subsampling pattern
Note that tuning the hyperparameters of PnPDP approaches is still much more efficient than retraining neural network that typically required for end-to-end approaches.

---

Published as a conference paper at ICLR 2025
Scatarig
Figure 4: Relative performance of plug-and-play diffusion prior methods compared with traditional baselines under different levels of measurement sparsity on different tasks. Metries are averaged over multiple PnPDP methods. The performance difference increases in general the measurement becomes sparser.
I-P-DM
DAPS
Ground truth
DDRM
DONM
TIGOM
DPS
LGP
S
5
2,
PSNB 0253
FSrt 1234
F94h 22:00
2012
DIfFFR
PMP.MM
DAPS
RED
FPS
Xi. = 7.71
Ry 1.858
X
1.30
X
PSV8-7904
PSWB-122
PSW
PIA--78-1
(b)
Figure 5: PnPDP methods on out-of-distribution test samples. (a) Black-hole imaging problem on digits inputs; and (b) inverse scattering sources that contain 9 cells, while the prior model is trained on images with to cells.
(from vertical to horizontal lines). We assess the average performance variation across three method categories: traditional baselines, end-to-end approaches, and PnPDP methods. The average abso- lute performance change for PnPDP methods is 0.48dB (PSNR) and 0.016 (SSIM), comparable to the traditional baseline methods 1.62dB (PSNR) and 0.027 (SSIM), but significantly smaller than the end-to-end methods, which exhibit changes of 9.58dB (PSNR) and 0.21 (SSIM). These find- ings indicate that PnPDP methods are more robust than both baseline and end-to-end methods when handling different forward models
How well do PnPDP methods handle out-of-distribution sources? In general, if the unknown source falls outside the diffusion prior distribution, PnPDP methods tend to generate solutions that are biased toward the prior. As illustrated in Figure5a, most solutions produced by PnPDP methods exhibit a black hole ring feature characteristic of the diffusion prior. This suggests that while PnPDP approaches are flexible in capturing high-dimensional priors, they are limited in their ability to reliably recover "surprising" sources that outside the support of the diffusion prior distribution. However, when the unknown source is close the diffusion prior distribution, PnPDP methods can recover it effectively, as demonstrated in Figure 5b
DISCUSSION
We conclude by highlighting key research opportunities for advancing PnPDP methods in solving inverse problems. One research challenge we identify is that the current PnPDP methods do not account for stability conditions required to query a forward model, which leads to degraded per- formance and numerical instability. However, many scientific inverse problems are based on PDE systems that requires certain conditions on the inputs to simulate numerically and violating these constraints result in meaningless solutions. Another direction for improvement is inference speed. As shown Figure [ almost all the PnPDP methods are less computationally efficient than the conventional baselines. There remains substantial room for optimization. Beyond these challenges, additional promising research directions such as robustness to model error and prior mismatch are further discussed Appendix C
For end-to-end approaches, this is considered as out-of-distrihuttin test.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
This research is funded in part by NSF CPS Grant #1918655, NSF Award 2048237, NSF Award 2034306 and Amazon AI4Science Discovery Award. H.Z. is supported by the PIMCO and Amazon AlAscience fellowship. Z.W. is supported by the Amazon A14Science fellowship. B.Z. and W.C. are supported by the Kortschak Scholars Fellowship. B.F. is supported by the Pritzker Award and NSF Graduate Research Fellowship. Z.E.R. and C.Z. are supported by a Packard Fellowship from the David and Lucile Packard Foundation.
We thank Ben Prather, Abhishek Joshi, Vedant Dhruv, C.K. Chan, and Charles Gammie for the synthetic blackhole images GRMHD dataset used here, generated under NSF grant AST 20-34306.
REFERENCES
Hemant Aggarwal, Merry Mani, and Mathews Jacob. Modl: Model based deep learning architecture for inverse problems. IEEE Transactions on Medical Imaging, PP, 12 2017. doi: 10. 1009TML. 2018.2865355
Kazunori Akiyama, Shiro Ikeda, Mollie Pleau, Vincent L Fish, Fumie Tazaki, Kazuki Kuramochi, Avery E Broderick, Jason Dexter, Monika Mošcibrodzka, Michael Gowanlock, et al. Superreso- lution full- coll-poaaimetricc imaging for radio interferometry with sparse modeling. The Astronomical Journal, 153(4):159, 2017a.
Kazumori Akiyama, Kazuki Kuramochi, Shiro Ikeda, Vincent Fish, Fumic Tazaki, Mareki Honma, Sheperd S Doeleman, Avery E Broderick, Jason Dexter, Monika Mošcibrodzka, al. Imaging the schwarzschildrraaduusssal structure of m87 with the event horizon telescope using sparse modeling. The Astrophysical Journal, 838(1):1, 2017b.
Kazunori Akiyama, Fumie Tazaki, Kotaro Moriyama, Ilje Cho, Shiro Ikeda, Mahito Sasada, Hiroki Okino, and Mareki Honma. Smili: Sparse modeling imaging library for interferometry. Astro- physics Source Code Library, ascl-1904, 2019.
Albert Berahas, Liyuan Cao, Krzysztof Choromanski, and Katya Scheinberg. A theoretical and empirical comparison of gradient approximations in derivative-free optimization. Foundations of Computational Mathematics, 22(2):507-560, 2022.
Lindy Blackburn, Dominic W. Pesce, Michael D. Johnson, Maciek Wielgus, Andrew A. Chael, Pierre Christian, and Sheperd Doeleman. Closure statistics in interferometric data. The As- doi: 11..8477/538-43357//88888 1357/ab8469
trophysical Journal, 894(1):31, may 2020.
URL https:
//dx. f/dx.doi.ore ot1e2i..rr11..0 wtt.oogg///..../1....
C. Bouman and K. Sauer. A generalized gaussian image model for edge-preserving map estimation. IEEE Transactions on Image Processing, 2(3):296-310, 1993. doi: 011100/88.236566
Benjamin Boys, Mark Girolami, Jakiw Pidstrigach, Sebastian Reich, Alan Mosca, and O. Deniz Akyildiz. Tweedie moment projected diffusions for inverse problems, 2023.
D. J. Brady, K. Choi, D. L. Marks, R. Horisaki, and S. Lim. Compressive holography. Opt. Express, 17005::3340-13040 2009.
M. M. Bronstein, A. M. Bronstein, M. Zibulevsky, and H. Azhari. Reconstruction in diffraction ultrasound tomography using nonuniform FFT. IEEE Trans. Med. Imag., 21(11):1395-1401 November 2002.
Ruiming Cao, Nikita S Divekar, James Nuñez, Srigokul Upadhyayula, and Laura Waller. Neural space-time model for dynamic multi-shot imaging. Nature Methods, pp. 1-6, 2024.
Gabriel Cardoso, Yazid Janati el idrissi, Sylvain Le Corff, and Eric Moulines. Monte carlo guided denoising diffusion models for bayesian linear inverse problems. In The Twelfth International Conference on Learning Representations, 2024, URUjttts://oppeeeeetee net forum? H.=nHESSXXxwWk

---

Published as a conference paper at ICLR 2025
James A Carton and Benjamin S Giese. A reanalysis of ocean climate using simple ocean data assimilation (soda). Monthly weather review, 136(8):2999-301 2008.
Andrew Chacl, Katie Bouman, Michael Johnson, Maciek Wielgus, Lindy Blackburn, Chi-Kwan Chan, Joseph Rachid Farah, Daniel Palumbo, and Dominic Pesce. cht-imaging: vl. 1.0: Imaging interferometric data with regularized maximum likelihood. Zenodo, 2019.
Andrew A Chael, Michael D Johnson, Ramesh Narayan, Sheperd S Doeleman, John FC Wardle, and Katherine I. Bouman. High-resolution linear polarimetric imaging for the event horizon telescope. The Astrophysical Journal, 829 2016.
Andrew A Chael, Michael D Johnson, Katherine L. Bouman, Lindy L Blackburn, Kazunori Akiyama, and Ramesh Narayan. Interferometric imaging directly with closure phases and closure amplitudes. The Astrophysical Journal, 857(1):23, 2018.
Michael Chen, David Ren, Hsiou-Yusee Liu, Shwetadwip Chowdhury, and Laura Waller. Multi-layer born multiple-scaterrigg model for 3d phase microscopy. Optica, 7(5):394 403, 2020.
Yongxin Chen, Sinho Chewi, Adil Salim, and Andre Wibisono. Improved analysis for a proximal algorithm for sampling, 2022.
06386
Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shu- ran Song. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.04137, 2023.
W. Choi, C. Fang- Yen, K. Badizadegan, S. Oh, N. Lue, R. R. Dasari, and M. S. Feld. Tomographic phase microscopy. Nat. Methods, 4(9):717-719, September 2007.
Hyungjin Chung and Jong Chul Ye. Score-based diffusion models for accelerated mri. Medical Image Analysis, 80: 102479, 05 2022. doi: 10. 10.10166..eeia.22222.. 2tt.ia..022.102799999
Hyungjin Chung. Jeongsol Kim, Michael Thompson Mccann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh Interna- tional Conference on Learning Representations, 2023. URL http::::////eeevie......
BG Clark. An efficient implementation of the algorithm "clean". Astronomy and Astrophysics, vol. 89, no. Sept. 1980, p. 377, 378., 89:377, 1980.
Robert Clayton and Björn Engquist. Absorbing boundary conditions for acoustic and elastic wave equations. Bulletin of the seismological society of America, 67(6):152--1540, 1977.
Richard Courant, Kurt Friedrichs, and Hans Lewy. On the partial difference equations of mathemat- ical physics. IBM journal of Research and Development, 11(2):215- 234, 1967.
Chengyuan Deng, Shihang Feng, Hanchen Wang, Xitong Zhang, Peng Jin, Yinan Feng, Qili Zeng, Yinpeng Chen, and Youzuo Lin. Openfwi: sarge-scale multi-structural benchmark datasets for full waveform inversion. Advances in Neural Information P Processing Systems, 35:6007-6020, 2022.
J. J. Devaney. Inverse-scattering theory within the Rytov approximation. Opt. Lett. 6(8):374-376. August 1981.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-877 87948794.222 2021.
Noe Dia, MJ Yantovski-Barbh Alexandre Adam, Micah Bowles, Pablo Lemos, Anna MM Scaife, Yashar Hezaveh, and Laurence Leveasser, Bayesian imaging for radio interferometry with score-based priors. arXiv preprint arXiv:2311.18011 2023.
Zehao Dou and Yang Song. Diffusson posterior sampling for linear inverse problem solving: A filtering perspective. In The Twelfth International Conference Learning Representations, 2024.
12

---

Published as a conference paper at ICLR 2025
Event Horizon Telescope Collaboration EHTC. First sagittarius a event horizon telescope results. iii. imaging of the galactic center supermassive black hole. The Astrophysical Journal Letters, 930(2):L14, may 2022. doi: 10.384472041-82113aaaa 0000/224118213/aar
URthttps:/ /tt///xxx// /xp doi. 10.
The Event Horizon Telescope Collaboration EHTC. First m87 event horizon telescope results. iii.
data processing and calibration. The Astrophysical Journal Letters, 875 2019a. doi: 10. 384772044-88113/bbbe5. URL /7dx
doi wtt.:ooo..ogggggg..... /ttp://c.............///202211//
The Event Horizon Telescope Collaboration EHTC. First m87 event horizon telescope results.
iv. imaging the central supermassive black hole. The Astropysical Journal Letters, 875(1):
/ttpdxxccccc...ooooooooooodddoit www.i.orrro..... L4, apr 2019b. doi: 1-384772044--211//aa60 URL httttt://ddd.eti....
The Event Horizon Telescope Collaboration EHTC. The persistent shadow of the supermassive black hole of 87-i. observations, calibration, imaging, and analysis. Astronomy & Astrophysics, 681:A79, 2024.
Berthy Feng and Katherine Bouman. Variational bayesian imaging with efficient surrogate score- based prior. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https:
//openeeviww.co wwwp:/eeeeetttoneeeeeeeeeeeeeeee /ettt//oorumo1-ddd22/2e/ii
Berthy T Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine L Bouman, and William T Freeman. Score-based diffusion models as principled priors for inverse imaging. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10520-10531, 2023.
Berthy T Feng, Katherine L Bouman, and William T Freeman. Event-horizon-scal imag- ing of m87* under different assumptions via deep generative image priors. arXiv preprint arXiv:2406.027855 2024.
Cong Fu, Keqiang Yan, Limei Wang. Wing Yee Au, Michael Curtis McThrow, Tao Komikado, Koji Maruhashi, Kanji Uchino, Xiaoning Qian, and Shuiwang Ji. latent diffusion model for protein structure generation. In Learning on Graphs Conference, pp. 29-1. PMLR, 2024.
Jacques Hadamard. Lectures on Cauchy's 's problem linear partial differential equations. Courier Corporation, 2014.
Kerstin Hammernik, Teresa Klatzer, Erich Kobler, Michael P. Recht, Daniel K. Sodickson, Thomas Pock, and Florian Knoll. Learning a variational network for reconstruction accelcrated MRI data. CoRR, abs/1704.00447, 2017.
Yinnian He and Weiwei Sun. Stability and convergence of the crank-niollsn//ddas--astii scheme for the time-dependent navier-stokes equations. SIAM Journal on Numerical Analysis, 45(2):837-86,, 2007.
JA Högbom. Aperture synthesis with a neg-reg distribution of interferometer baselines. Astron- omy and Astrophysics Supplement, Vol. 15, p. 417, 15:417, 1974.
Yue Huang, John W. Paisley, Qin Lin, Xinghao Ding, Xueyang Fu, and Xiao-Ping (Steven) Zhang- Bayesian nonparametric dictionary learning for compressed sensing MRI. IEEE Trans. Image Process., 23(12):5007-5019 2014.
Yujia Huang, Adishree Ghatare, Yuanzhe Liu, Ziniu Hu, Qinsheng Zhang, Chandramouli Shama Sastry, Siddharth Gururani, Sageev Oore, and Yisong Yue. Symbolic music generation with con-differentiable rule guided diffusion. In Ruslan Salakhutdinov, Zico Kolter, Kather- ine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Pro- ceedings of the 41st International Conference on Machine Learning, volume 235 Proceed- ings of Machine Learning Research, pp. 19772-19797. PMLR, 21-27 Jul 2024. URL. https: /proceedings. mlr. wwwpp/eeeeaaa Trress/v235//nang2e2
Chang Min Hyun, Hwa Pyung Kim, Sung Min Lee, Sungchul Lee, and Jin Keun Sco. Deep learning for undersampled MRI reconstruction. CoRR, abs/1 709.02576, 2017.
13

---

Published as a conference paper at ICLR 2025
Marco A Iglesias, Kody JH Law, and Andrew M Stuart. Ensemble kalman methods for inverse problems. Inverse Problems, 2944)04400 2013.
Ajil Jalal, Marius Arvinte, Giannis Daras, Eric Price, Alex Dimakis, and Jonathan Tamir. Robust compressed sensing MRI with deep generative priors. In A. Beygelzimer. Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances Neural Information Processing Systems, 2021. URL https://cpenree/
net www.moiirrooorr..e. ?id= jrT 6MMb
Zahra Kadkhodaie and Eero P Simoncelli. Stochastic solutions for linear inverse problems us- ing the prior implicit in a denoiser. In A. Beygelzimer, Y. Dauphin, Liang, and J. Wort- man Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https: //openzeview net/ forttffoum?? wtw.mmmmmddmhxteeee d=xShh6N9UUUh
Avinash C Kak and Malcolm Slaney. Principles of computerized tomographic imaging. SIAM, 2001.
Ulugbek S Kamilov, Ioannis N Papadopoulos, Morteza H Shoreh, Alexandre Goy, Cedric Vonesch, Michael Unser, and Demetri Psaltis. Learning approach to optical tomography. Optica, 2(6): 517-522, 2015.
Tero Karras, Mila Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in Neural Information Processing Systems, 35:26565 2022.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 35:23593-23666, 2022.
Rahima Khanam and Muhammad Hussain. Yolov11: An overview of the key architectural enhance- ments, 2024. URL htt.::tttt::aaat http://aaaa....or//////////00
Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint 11X.691 2014.
Nikola B Kovachki and Andrew M Stuart. Ensemble kalman inversion: derivative-free technique for machine learning tasks. Inverse Problems, 35(9)0095005, 2019.
Dongwook Lee, Jae Jun Yoo, and Jong Chul Ye. Deep residual learning for compressed sensing MRI. In 14th IEEE International Symposium on Biomedical Imaging, ISBI 2017, Melbourne, Australia, April 18-21, 2017, pp. 15-18. IEEE, 2017.
Yin Tat Lee, Ruoqi Shen, and Kevin Tian, Structured logconcave sampling with a restricted gaussian oracle, 2021. URL wttpt::t://// http:/ttt::/xxiiitit http:://axxvvvve www.bvs.rrgg...o.. wwwp://p.aa...oooooo/o// 2tt.//0000000006666
Lianlin Li, Long Gang Wang, Fernando L Teixeira, Che Liu, Arye Nehorai, and Tie Jun Cui. Deep- nis: Deep neural network for nonlinear electromagnetic inverse scattering. IEEE Transactions on Antennas and Propagation, (((3)::819-11)
2018a.
Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. Hyperband: A novel bandit-based approach to hyperparameter optimization. Journal of Machine Learning Research, 18( (1115):1-52, 2018b.
Xiang Li, Soo Min K Kwon, Ismail R. Alkhouri, Saiprasad Ravishankar, and Qing Qu. Decoupled data consistency with diffusion purification for image restoration, 2024a. URL httpp://arxiv. orq/abs 2403. 06054
Yunzhe Li, Yujia Xue, and Lei Tian. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica, 1810)1 2018c.
Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadeneshelli and Anima Anandkumar. Physics-informed neural operator for learning partial differential equations. ACM/JMS Journal of Data Science, 133):1227, 2024b.
H.-Y. Liu, D. Liu, H. Mansour, P. T. Boufounos, L. Waller, and U. S. Kamilov. SEAGLE: Sparsity- driven image reconstruction under multiple scattering. IEEE Trans. Comput. Imaging, 4(1):73-86, March 2018.
14

---

Published as a conference paper at ICLR 2025
Jiaming Liu, Yu Sun, Cihat Eldeniz, Weijie Gan, Hongyu An, and Ulugbek S. Kamilov. RARE: image reconstruction using deep priors learned without groundtruth. IEEE J. Sel. Top. Signal Process., 14(6): (466):0888-199, 2020.
Jiaming Liu, Yu Sun, Weijie Gan, Xiaojian Xu, Brendt Wohlberg, and Ulugbek S. Kamilov. Sed-net: Efficient model-based deep learning with theoretical guarantees. CoRR, abs/2101.09379, 2021.
Renhao Liu, Yu Sun, Jiabei Zhu, Lei Tian, and Ulugbek S Kamilov. Recovery of continuous 3d refractive index maps from discrete intensity-only measurements using neural fields. Nature Ma- chine Intelligence, 4(9):781-75) 2022.
Yaning Liu, Fenglin Niu, Min Chen, and Wencai Yang. 3-d crustal and uppermost mantle structure beneath ne china revealed by ambient noise adjoint tomography. Earth and Planetary Science Letters, 461:20-29, 2017.
M. Louboutin, M. Lange, F. Luporini, N. Kukreja, P. A. Witte, F. J. Herrmann, P. Velesko, and G. J. Gorman. Devito (v3.1.0): embedded domain-specifts language for finite differences and geo- physical exploration. Geoscientific Model Development, 12(3): 1653):116--1188 1187 2019. doi: 10.5194/ gmd- 12-1 1651122-1
URLpttts::/ wwww www.geeossiiiooo dev wtw.ooeee-e....ee/////1/1///////
Guanxiong Luo, Na Zhao, Wenhao Jiang, Edward S. Hui, and Peng Cao. Mri reconstruction us- ing deep bayesian estimation. Magnetic Resonance in Medicine, 84:2246 2261, 2019. URL https://appl www.nteettttt.e scw.nttccooonoco.rrrc....coc..a.
II: htt.:/oo.....or/oooooss:::::::::
Guanxiong Luo, Moritz Blumenthal, Martin Heide, and Martin Uecker. Bayesian mri construction with joint uncertainty estimation using diffusion models. Magnetic Reso- nance in Medicine, 90:295 311, 2022. URL http:: /tpp. www.ssttcccccola......
CorpusID:25669700
Yanchen Luo, Junfeng Fang, Sihang Li, Zhiyuan Liu, Jiancan Wu, An Zhang, Wenjie Du, and Xiang Wang. Text-guided small molecule generation via diffusion model. iScience, 2024.
M. Lustig, D. L. Donoho, J. M. Santos, and J. M. Pauly. Compressed sensing MRI. IEEE Signal Processing Magazine, 25(2):72 82, 2008. doi: 10. 109/MSP2200.9914728..
Michael Lustig, David Donoho, and John Pauly. Sparse mri:
The application of compressed sensing for rapid mr imaging. Magnetic resonance in medicine: : official journal of the Society of Magnetic Resonance in Medicine / Society of Magnetic Resonance in Medicine, 58:1 182 95, 12 2007. doi: 10.1002/mmm.21391
Shiqian Ma, Wotao Yin, Yin Zhang, and A. Chakraborty. An efficient algorithm for compressed mr imaging using total variation and wavelets. In 2008 IEEE Conference on Computer Vision and Patlern Recognition, pp. 1-8, 2008. doi: 10.1 10.1 4587391.
Y. Ma, H. Mansour, D. Liu, P. T. Boufounos, and U. S. Kamilov. Accelerated image reconstruction for nonlinear diffractive imaging. In Proc. IEEE Int. Conf. Acoustics, Speech and Signal Process. (ICASSP 2018), Calgary, Canada, April 15-20, 2018. arXiv: 1xiv:11088 708.01663 [cs.CV].
Ross Maguire, Brandon Schmandt, Jiaqi Li, Chengxin Jiang, Guoliang Li, Justin Wilgus, and Min Chen. Magma accumulation at depths of prior rhyolite storage beneath yellowstone caldera. Science, 10006663):0001-00060 2022.
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective on solving inverse problems with diffusion models. arXiv preprint arXiv:2305.0433 04391, 2023.
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective on solving inverse problems with diffusion models. In The Twelfth International Conference on Learning Representatiinss 2024. www.https:.. / revi ew. net. lew.
Yosuke Mizuno. Grmhd simulations and modeling for jet formation and acceleration region in agns. Universe, 8(2)85, January 2022. ISSN 1298-1997, doi: 10.3390/mmiveree8020555. URL http://dx..o doi wwg/10.g/o0... 1
15

---

Published as a conference paper at ICLR 2025
R-E Plessix. A review of the adjoint-state method for computing the gradient of a functional with geophysical applications. Geophysical Journal International, 167(2):495-55) 503, 2006.
Oliver Porth, Koushik Chatterjee, Ramesh Narayan, Charles F Gammie, Yosuke Mizuno, Peter Anninos, John G Baker, Matteo Bugli, Chi-kwan Chan, Jordy Davelaar, al. The event horizon general relativistic magnetohydrodynamiic code comparison project. The Astrophysical Journal Supplement Series, 243(2):26, 2019.
Tran Minh Quan, Thanh Nguyen-Duc. and Won-Ki Ki Jeong. Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans. Medical Imaging, 37(6): 1488-1497, 2018.
Saiprasad Ravishankar and Yoram Bresler. MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans. Medical Imaging, 1025):102--1 2011.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkot- tai. Solving linear inverse problems provably via posterior sampling with latent diffusion mod- els. In Thirly-seventh Conference on Neural Information Processing Systems, 2023. URL
net. forum? wtt.:ooooooooo-0---------a..aa.. 1ddXXBPPYERRRR
Jo Schlemper, Jose Caballero, Joseph V. Hajnal, Anthony N. Price, and Daniel Rueckert. A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Medical Imaging. 37(2):491-503 2018.
Martin Shepherd. Difmap: Synthesis imaging of visibility data. Astrophysics Source Code Library, pp. asc]-1103, 2011.
MC Shepherd. Difmmap an interactive program for synthesis imaging. In Astrononical Data Anal- ysis Software and Systems VI, volume 125, pp. 77, 1997.
Jascha Sohl-Dickstein, Bric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-22666 2265. PMLR, 2015.
Bowen Song, Soo Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen. Solving inverse problems with latent diffusion models via hard data consistency, 2024. URL https:
08123
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guidee diffusion models for inverse problems. In International Conference on Learning Representations, 2023a.
URL httptttss///ppeeeeeee net. feee//formmid=: 2t__gmii__________S33
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 32483-32498, PMLR, 23-29 Jul 2023b. URL
www.ppte........mm.....e v20 song23k....k html
Yang Song. Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. baore-based generative modeling through stochastic differential equations. arXiv preprint 2rXiv:2011.13456. 2020.
Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. Solving inverse problems in medical imaging with based generative models. In International Conference on Learning Representations, 2022. URL httpttppss///ppeeeieeeee net. wwtett/foormmmmdaaaaaa... (OuGT
Anuroop Sriram, Jure Zbontar, Tullie Murrell, Aaron Defazio, C. Zitnic,, Nafissa Yakubova, Florian Knoll, and Patricia Johnson. End-to-End Variational Networks for Accelerated MRI Reconstruc- fion, pp. 64-73. 09 2020. ISBN 978-3-030-59712-22 doi: 10. 10.10077978833-00-5577133 7.
16

---

Published as a conference paper at ICLR 2025
He Sun and Katherine L Bouman. Deep probabilistic imaging: Uncertainty quantification and multi- modal solution characterization for computational imaging. In AAAI, pp. 2628-2637, 2021.
Yu Sun, Zhihao Xia, and Ulugbek Kamilov. Efficient and accurate inversion of multiple scattering with deep learning. Optics express, (6(11):44678-444
2018.
Yu Sun, Brendt Wohlberg, and Ulugbek S. Kamilov. An online plug-and-play algorithm for regular- ized image reconstruction. IEEE Transactions on Computational Imaging. 5(3):395-408, 2019. doi: 10.1009/C1.20192288556.
Yu Sun, Zihui Wu, Yifan Chen, Berthy T. Feng, and Katherine L. Bouman. Provable probabilisstc imaging using score- score-based generative priors. IEEE Transactions Computational Imaging, 10:
1290-1305, 2024.
doi: 10.1 10.11099TC112224.444499
Yongjin Sung, Wonshik Choi, Christopher Fang- Beng Kamran Badizadegan, Ramachandra Dasari, and Michael S Feld. Optical diffraction tomography for high resolution live cell imaging. Optics express, 17(1):266-2777 2009.
Haoyue Tang, Tian Xie, Aosong Feng, Hanyu Wang, Chenyang Zhang, and Yang Bai. Solving noisy inverse problems via posterior sampling: A policy gradient view-point. In The Symbiosis of Deep Learning and Differential Equations III, 2023. httlhttts:///peeeie
net forum?
id 175
Kerem Tezcan, Christian Baumgartner, Roger Luechinger, Klaas Pruessmann, and Ender Konukoglu. Mr image reconstruction using deep density priors. IEEE Transactions on Medi- cal Imaging, PP:1-1, 12 2018. doi: 10. 10.1109/710 0099/MM..018.28877777
A Richard Thompson, James M Moran, and George W Swenson. Interferometry and synthesis in radio astronomy. Springer Nature, 2017.
L. Tian and L. Waller. 3D intensity and phase imaging from light field measurements in LED array microscope. Optica, 2: 2:104-111, 2015.
Brian L. Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi S. Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif- scaffolding problem. The Eleventh International Conference on Learning Representations, 2023. BR..tttps:
/tctps:/ooperreet ew, net. feet/forummad==6Tee0 6=xT:21XM6xXXXXXX111
Martin Uecker, Peng Lai, M. J. Murphy, Patrick Virtue, Michael Elad, John M. Pauly, Shreyas S. Vasanawala, and Michael Lustig. Espirit an eigenvalue approach to autocalibrating parallel MRI: Where sense meets grappa. Magnetic Resonance in Medicine, 71, 2014.
Pieter Hendrik van Cittert. Die wahrscheinliche schwingungsvertellung in einer von einer lichtquelle direkt oder mittels einer linse beleuchteten ebene. Physica, 1(1-6):001-210, 1934.
Singanallur V. Venkatakrishnan, Charles A. Bouman, and Brendt Wohlberg. Plug-and-play priors for model based reconstruction. In 2013 IEEE Global Conference on Signal and Information Processing, pp. 945-948, 2013. doi: 10. (0.1109/GlobbSSSP221.3.63333
Jean Virieux and Stéphane Operto. An overview of full-waveform inversion in exploration geo- physics. Geophysics, 746)) 2009.
Jean Virieux, Amir Asnaashari, Romain Brossier, Ludovic Métivier, Alessandra Ribodetti, and Wei Zhou. An introduction to full waveform inversion. In Encyclopedia of exploration geophysics, pp. RI-1. Society of Exploration Geophysicists, 2017.
Maxime Vono, Nicolas Dobigeon, and Pierre Chainais. Split-and-augmented gibbs sam- pler- appliaatiia to large-scale inference problems. IEEE Transactions on Signal Process- doi: 10. 1099tp..019.2299482 URL
ing, 6766):1648-16111 March 2019. ISSN 1941-0476.
doi. / 11w. 2019.011222 2894825
Shanshan Wang, Zhenghang Su, Leslie Ying, Xi Peng, Shun Zhu, Feng Liang, Dagan Feng, and Dong Liang. Accelerating magnetic resonance imaging via deep learning. In 13th IEEE Inter- national Symposium on Biomedical Imaging, ISBI 2016, Prague, Czech Republic, April 13-16, 2016, pp. 514-517. IEFF, 2016.
17

---

Published as a conference paper at ICLR 2025
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. In The Eleventh International Conference on Learning Representations, 2022.
David Wiesner, David Svoboda, Martin Maška, and Michal Kozubek. Cytopacq: a web-interface for simulating multi-dimensional cell imaging. Bioinformaties, 35(21):4531-45)5 -43333,,, 2019.
Emil Wolf. dineesdimenio structure determination of -ransitrasp objects from holographic data. Optics communications, 1(4):153-156, 1969.
Luhuan Wu, Brian Trippe. Christian Naesseth, David Blei, and John P Cunningham. Practi- cal and asymptotically exact conditional sampling in diffusion models. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp- 31372-31403. Curran Associates, Inc., cc 2023. URL http:s///prrceeednns....eee cttp:ippppccppppppeeeeeeeeeeeeee httpp//peeeeiiillepppppee/// File/ http:////ooooooeeeeeaaaaaaaaaaea Papeesaaeeeeepeeeerreeeeeeee - Conference.
Zihui Wu, Yu Sun, Alex Matlock, Jiaming Liu, Lei Tian, and Ulugbek S. Kamilov. Simba: Scalable inversion optical tomography using deep denoising priors. IEEE Journal of Selected Topics in Signal Processing, 14(6): 1eee11166117..1 2020. doi: 10. (100//STSP.22222299988888
Zihui Wu, Yu Sun, Yifan Chen, Bingliang Zhang, Yisong Yue, and Katherine L Bouman. Prin- cipled probabilistic imaging using diffusion models as alag-and-play priors. arXiv preprint arXiv:2405. 18782, 2024.
Xingyu Xu and Yuejie Chi. Provably robust score- diffusion posterior sampling for plug-am and- play image reconstruction, 2024. URL htttp:::/aa......cr/aaab/////
Guang Yang, Simiao Yu, Hao Dong, Gregory G. Slabaugh, Pier Luigi Dragotti, Xujiong Ye, Fangde Liu, Simon R. Arridge, Jennifer Keegan, Yike Guo, and David N. Firmin. DAGAN: deep de- aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans. Medical Imaging, 37(6):110--1321 2018
Yan Yang, Jian Sun, Huibin Li, and Zongben Xu. Deep admm-net for compressive sensing MRI. In Advances in Neural Information Processing Systems 29; Annual Conference on Neural Informa- tion Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 10-18, 2016.
Jure Zbontar, Florian Knoll, Anuroop Sriram, Matthew J. Muckley, Mary Bruno, Aaron Defazio, Mare Parente, Krzysztof J. Geras, Joe Katsnelson, Hersh Chandarana, Zizhao Zhang, Michal Drozdzal, Adriana Romero, Michael Rabbat, Pascal Vincent, James Pinkerton, Duo Wang, Nafissa Yakubova, Erich Owens, C. Lawrence Zitnick, Michael P. Recht, Daniel K. Sodickson, and Yvonne W. Lui. fastMRI: An open dataset and benchmarks for accelerated MRI. 2018
Frederik Zernike. The concept of degree of coherence and its application optical problems. Phys- ica, 5(8):785-795, 1938.
Zhifang Zhan, Jian-Feng Cai, Di Guo, Yunsong Liu, Zhong Chen, and Xiaobo Qu. Fast multi-class dictionaries learning with geometrical directions in MRT reconstruction. CoRR, 2015.
Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing, 2024. URL
https::/arxxv..org//
Cheng Zhang, Judith Bütepage, Hedvig Kjellström, and Stephan Mandt. Advances in variational inference. IEEE transactions on pattern analysis and machine intelligence, 41(8):2008 -026, 2018.
Ruiyang Zhao, Burhaneddin Yaman, Yuxin Zhang, Russell Stewart, Austin Dixon, Florian Knoll, Zhengnan Huang, Yvonne W Lui, Michael Hansen, and Matthew P Lungren. fastmrit: Clin- ical pathology annotations for knee and brain fully sampled multi-coil mri data. arXiv preprint arXiv:2109.0388 009.03 2021.
18

---

Published as a conference paper at ICLR 2025
Hongkai Zheng, Wenda Chu, Austin Wang, Nikola Kovachki, Ricardo Baptista, and Yisong Yue.
Ensemble kalman diffusion guidance: A derivative- free method for inverse problems, 2024. URL
https::/arxxi: www.oogg//..... abs / 409.20172
Kevin C. Zhou and Roarke Horstmeyer. Diffraction tomography with deep image prior. Opt. Express, 280):12872-12886, Apr 2020. doi: 10.1364/0E.3792000 URL https://opg.
pttica.orc/oe/abttraaaa
cfm.URI=oe-28- 9-1872
Bo Zhu, Jeremiah Z. Liu, Stephen F. Cauley, Bruce R. Rosen, and Matthew S. Rosen. Image reconstruction by domain-transform manifold learning, Nat., 555(7697)4487-492, 2018.
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc
Van Gool. Denoising diffusion models for plug-and-play image restoration. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1229, 2023.
19

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 TABLES OF MAIN RESULTS
Table 3: Results on Linear inverse scattering. inverse scattering. Noise level By = 10
PSNR and SSIM of different algorithms linear
Methods
PSNK
Meas (2)
Mess (16
Meeas (95)
Traditional
PISTA-TV
$2,126 (2.1M)
0.9% (3.030)
(0.5))
26.521 (2.677)
01914 (1004)))
2.65 (1 909
20.91 70.916 (3.513)
0.709 (1. 6,55 00.65)
PaP diffusion prisr
(1.823)
(1.56)
(0.019)
(3.39)
(0.037)
(0.92)
27.925
(6441.(3.410)
(2.163
0.845
31,798 163)
(1
27.572(3.415)
(0.133)
4..3(1.31)
LGD
27,001 (2.346)
0.313 (0.03))
1.17 (0.20)
23,833 2.337)
0.803 (0.034)
1.06 (1 16)
20.491 (3.088)
0.532 (0.077)
1.45
DINPER
34.241 (2.310)
0.968 (0.030)
1.11 (0.24)
34.110 269)
0.957 (0.005)
1.04 (0.23)
26.321 (3.272)
0.915 (0.028)
1.37 (0.23)
PSP-M
31,914 (2.04)
0,988 (11030)
1.21 (0.25)
31,817 (2.073)
11951 (01011))
1.42 00 25-
26.715 (2,84)
0.009 00.0465
2.20 m. 34)
DAPS
34,641 (1.69)
0.987 (0.036)
1.03 (0.25)
33.060(1.704)
0.944 (0.005)
1.11 (0 25)
(3.100)
0.885 (0.050)
1.51 (0.25)
RED deT
36.556 2.292)
0.981 (0.025)
0.89 (0.25)
35.411 (2.16) 166
0.954 (0.004)
0.57 (0.21)
22,072 23.072(3.330)
0.935 (0,05))
1.18 (0.23)
AMD5
(1.01)
651)
0.710
01)
31
10.0/90
0.88 (0.02)
MCG
30.95 (1.564)
28,05 (1.072)
(0.042)
(0.01)
21.004 (1-570)
0.21 (0.00)
Table Results on compressed sensing MRI. Mean and standard deviation are reported over test cases. Underline: the best across all methods. Bold: the best across PnP DM methods.
Kar
2945 (177g
3647 (LSE))
(L122)
3LADE 2061
0.51 ELIS
22 7331
334611201 50)
(622
BUESD
71309-19 414)
5413119KI
5-27 BUTI
TUNSOTS
Feel mem
126 Natter
3148 (1997)
64h 717799
SEST CL340
0756 S1180
ESCH
2019 (L.17)
3770
22.17 (6240
3.50 (L167)
31642 15 9021
21317-03314
3-43 (L152)
22A86 (1274))
2018 (2774)
33333 13.313
63096077)
2671 (6.15)
3.55 231130
31.41 15 368
26 To4 230
DURT (8-14)
0485 70660
27.39 (244)
3.50 FL177)
105
3619 (-247)
653 13 1055
0800 120
25.3 (2.17)
558 23.100
15365 15 90))
28 82 771
033415-110
6727 1100
01.00 23235
3.40 01100
56842/1-300
31.8 01411
1.78 (1/
4331 (3.77)
2742 (1425
3555 (L177)
32241 (3 (60)
25 25.35 2411
5521 01550
25.55 03122
036 (L177)
31279-12113
Table Generalization results on compressed sensing MRI with x4 acceleration and raw measure- ments. Mean and standard deviation are reported over 94 test cases.
Coneratioaion
Vertical - Hortnoatal
Knee - Bestt
x4 ++ X
Methats
PSNRR
SSIM
Dara nistr
PSNR 1
SSEM
Data mistr:
PSNR
SSIM 1
DaL milti
Tradiion
Waoeletets
27 75 (1,68)
MAZT (1133)
31.244 (15)
25 (1 253)
0.44 II (11250
7.966 (1 MUS)
24.08 (L))))
U.513 (00705)
28,900 CTUL TRT)
TV
28 (1.777)
0.533 (1 1380
32 (15)
25 96 307)
BA86 (1 (49)
9.396 1D MODn
23,70 357)
B.477 (0:177)
23 OOI (10885)
Endi-t-end
Recdual UNi
20 06--1 (1-77)
61605 (1
10 C1 164)
03881 (I (1))) 0.857 (0.035)
23.95 (7. (76)
B.5 (D006
ELE NalNet
22 (2:925)
0,545 (0.105)
31.97 (1.A52)
24.59 (2.012)
0L537 (DL009)
Pall diffedoo prior
CSCM
26.56 (3.64)).447)
0,629 (0.129)
31,866 (15:479)
27.19 (7.1917.551)
0.779 (0. 189)
2,779 (1
21.17 (8.334)
0.425 (0.192)
22,088 (10.
(1.661))))
0.475 (0.01))
19.001 (14.234)
75
0.674
34.34 48.1)
(04000)
34.075
28,95
31,540
30.61
0.311
9,750
(0.117)
DNMIPER
(759)
27.75
0:325
(0.11.3)
(3.255)
(2.911
0.552
8.255
PoP-IM
27
0.689 (2.121)
961 (15.235)
29
08882
9.769
(0.117)
20

---

Published as a conference paper at ICLR 2025
Table 6: Results on black hole imaging. PSNR and Chi-squared of different algorithms on black hole imaging. Gain and phase noise and thermal noise are added based on EHT library.
Mitaiie
2556
F5NR
FSNR
Bkr 1500
Tradiional
1S0T linging
1.97 (3 525
(156
33.67 (3
35.001149)
1.16 (3 150)
1.30(13325)
31.56.0 (7.47)
75.55 (1
1.51 (9:50)
1359 313
TuP 4ifteies
33.9955
6664 (36760)
2536-2-000 900
33,84
5406
23.51 3.50
13860 (1360)
(3155
(5.00
50100330
(3:53)
4586 (81.211
25.75 m0
29-13
3435 0.40
1:08:
500
(5.5)
1:38 (0477)
511
1,899 (1221)
1667713 7075
3.39 950
23
11:0111100
385)
133.99 576
5,06
3440//1
Table 7: Results on FWI. Mean and standard deviation are reported over 10 test cases. t: initialized from data blurred by Gaussian filters with o 20. *: one test case is excluded from the results due to numerical instability.
Methods
Relative L.2
PSNR1
SSIMt
Data misfit.
Traditional
Adam
0.333 (0.086)
9.968 (2.083)
0,305 (0. 120)
115.14 (52.10) 10)
Adum?
0.089 (0.021)
21,273 (2.045)
0.679 (0.073)
15.89 (10.16)
LBBGST
0.070 (0.023)
23.398 (2.749)
0.704 (0.077)
(6.47)
PnP diffusion prior
DPS
0.250 (0.154)
14.111 (6.820)
01.991 (0.161)
155.08 (92. 17)
LGD
0.244 (0.024)
12.288 (0.889)
0.341 (0.047)
258.47 (26.40)
DiTPIR
0.204 (0.129)
16.113 (6.962)
0.554 (0.191)
88.53 (56.91)
DAPS!
0.201 (0.103)
14.914 (4. 184)
0.321 (0.067)
111.13 (71.33)
PmP-DM
0.259 (0.075)
11.983 (2.269)
0.431 (0.073)
308.84 (26.34)
REDDiff
0.319 (0.102)
10.372 (2.650)
0.280 (0.108)
94.67 (41.33)
Table 8: Results on Havier-Stokes equation. Relative l error of different algorithms on 2D Navier- Stokes inverse problem, reported over 10 test cases, *; one or two test cases are excluded from the results due to numerical instability.
Subsampling maio
x2
Traditional
FKI
0,577 (38)
0,609 (0.119)
0.633 (0 107)
0.579 (0.145)
0.689 1315
0.805 (0.117)
0.852 (67)
0.9400.111
1,1100.000
Pal' ditudon priee
DPS-9186
1,887 (0. 186)
1,612 (0. 1731
1.454 (1 154)
1,701* (0 132)
1,209* (0
1,00P (1) 100)
1.2469 (6 1080
1.221 (0.82)
1..70)
DPS-CS8
2,209 (0:31)
2.117 (0.305)
1.246 (0 101)
1.176 (0.079)
1.155* (0095)
1.114 (1) 144)
1885 (0
1.304* (1.111)
1.218 113)
DPG
0.325 (0.185)
0.408* (0.173)
0u856 (0L (0.171)
0.322 (0.200)
0.361 (0.187)
0.454 (0.207)
0.596 (0.301)
0.591 (0.262)
0.546 (0.251)
SCG
0.905 (0:000)
0.925 (0.557)
(1966 (0 546)
0.365 (0.513)
0.926 (0.546)
0.929 (0.505)
1,200 (0.133)
1.284 (0:117)
1,547 (0.14)
EaKG
0.120 (0:085)
(0.057)
0.294 (0.061)
0.115 (0.064)
0.271 (0.055)
0.523 (0.136)
0.287 (0:273)
0.546 (0.213)
6:773
Table 9: Table of metrics we use to capture the computation complexity of each algorithm.
Metric
Description
# Fwdiotll
total forward model evaluations
# DM
total diffusion model evaluations
# Fwd Gradinal
total forward model gradient evaluations
# DM Gradwoul
total diffusion model gradient evaluations
Costuull
total runtime
# Fwd.eg
sequential forward model evaluations
# DM
sequential diffusion model evaluations
# Fwd Gradke
sequential forward model gradient evaluations
# DM Gradieg
sequential diffusion model gradient evaluations sequential runtime
Cost,e
21

---

Published as a conference paper at ICLR 2025
Unear Inverse scattering
Magnetic resonance Imaging
Bback hole imaging
Futl Waveform Inversion
Ntvien equation
13-0
Fwd
DM
Run Timeee
64581
DM Gred
Fwd Grad
10-
104
30cet
1M
Fed tour
Fat ooor
GM oko
Figure 6: Computational characteristics of each forward model. Fwd: runtime of a single forward model evaluation tested on single A100 GPU. DM: runtime of single diffusion model evaluation. Fwd Grad: runtime of a single forward model gradient evaluation. DM Grad: runtime of a single diffusion model gradient evaluation. Note that the inverse problem of the Navier-Stokes equation only permits black-box access to the forward model so its Fwd Grad has no value.
A.2 EXTENDED EVALUATION OF CS-MRI
Table 10: Diagnostic performance of compressed sensing MRI reconstructions.
Method
Precision
Recall
mAP50
mAP50 ranking
PSNR
SSIM
Data mislit
PSNR ranking
Traditional
Wavelete
0,532
0.332
0.385
28.16 (1.724)
0.685 (0.064)
23.501 (10.475)
TV
0.447
0,251
0,263
11
28.31 (1.834)
0,662 (0.079)
24.152 (10.613)
End-to-End
Residual UNet
0,482
0.462
0.439
8
31.62 (1.635)
0,803 (0.050)
VarNee
0.610
0.514
0.500
32,25 (1,901)
0.805 (0.056)
PnP diffusion prior
CSGM
0.501
0.528
0.454
6
27.34 (2,770)
0.673 (0.082)
(10.65))
ScorMMRI
0,412
26.86 (2.583)
(0.092)
25.677 (10.491)
RED-diff
0.478
0.468
0.448
7
31.56 (2.357)
0.764 (0.080)
23,406 (10.571)
3
DiaPIR
0.536
0.484
0.496
28.41 (1.403)
0.632 (0.061)
26.376 (10.555)
DPS
0,346
0.380
0.362
10
26.49 (1.550)
0.540 (0.067)
27.603 (11.127)
DAPS
0.514
0.556
0.480
30.15 (1.429)
0,725 (0.053)
23,978 (10.630)
PaP-DM
0.527
(2.934)
(10.413)
Fully sampled
0.573
0.581
0.535
23.721 (10.824)
For compressed sensing MRI, achieving good performance on general purpose metrics such as PSNR and SSIM is not always sufficient signal for high-quality reconstruction, as hallucinations might lead to wrong diagnoses. We quantify the degree of hallucination by employing a pathology detector on the reconstructed images of different methods. Specifically,
finetune medium-sin YOLOv11 model (Khanam & Hussain 2024) on a training set of fully sampled images with the fastMRI+ pathology annotations (Zhao al. 2021) (22 classes in total). We calculate the mAP50 metric over the reconstructed results on 14 selected volumes with severe knee pathologies, which includes 171 test images in total. For each method, we report the Precision, Recall, and mAP50 metrics for detection, and PSNR, SSIM, and Data Misfit for reconstruction, as shown in Table I0 We also provide the rankings based on mAP50 and PSNR. Overall, the two rankings are correlated, which means that better pixel-wise accuracy indeed leads to more accurate diagnosis. However, there are a few algorithms for which the two rankings disagree: Residual UNet, Score MRI, and RED-diff. The best methods for pathology detection are 22E-VarNed and PnP-DM.
B INVERSE PROBLEM DETAILS
B.1 LINEAR INVERSE SCATTERING
Problem details Consider a 2D object with permittivity distribution e(r) in a bounded sample plane Ω E R², which is immersed in the background medium with permittivity eg. The permittivity contrast is given by Ac(r) = e(r) Cb. At each time, the object is illuminated by an incident light field un (r) emitted by one of N > 0 transmitters, and the scattered light field u (r) is measured by M. > 0 receivers. We adopt the experimental setup in (Sun et al.] 2018) where the transmitters and receivers are arranged along circle I' E R22 that surrounds the object. Here, (2r, y) denotes
22

---

Published as a conference paper at ICLR 2025
the spatial coordinates. Under the first Born approximation (Wolf 1969,, the interaction between the light and object is governed by the following equation
Und (r)( [ g(r (((--r)) f(r') unn(r') dr', r E
(11)
where uoo (r) is the total light field, and f(r) Ae2))(rr) is the scattering potential. Here, k 2TT X is the wavenumber in free space, and A is the wavelength of the illumination. In the 2D space, the Green's function is given by
15 The
g(r - r') = 4 H (kollr -arr-rrll r'112)
(12)
where Kú = /Egk is the wavenumber of the background medium, and H²!) is the zero-order Hankel function of the first kind. Given the total field unt inside the sample domain Ω2 the scattered field at the sensor plane I is given by
Use (r) = - ggrr-r)) f(r') uhod (r)) reT.
(13)
Note that r denotes the sensor location in r, and the integral is computed over Ω.
By discretizing Eq. [II) and Eq. (13), we obtain a vectorized system that describes the linear inverse scattering problem. We denote the 2D vectorized permittivity distribution of the object by == f(r), and the corresponding measurement by Yse = (r) for notation consistency.
The forward model can thus be written
Ysc H Huuo 0 - A
(14)
where Utx = G(uin O 2). matrices G and H are discretizations of the Green's function I and Q respectively, and A : H diag Una)) We split and concatenate the real and imaginary part of A, and pre-compute the singular value decomposition of A to facilitate the plug-and-pla, diffusion methods that exploit SVD of linear inverse problems.
We set the physical size of test images to 18cm x 18cm, and the wavelength of the illumination to À = 0.84cm specified in (Sun et al. 2019). The forward model consists of N = 20 transmitters, placed uniformly on a circle of radius be air with permittivity c 1. We specify the number of receivers to be M 360, 180, 60 in our experiments.
1.6m. We further assume the background medium to
Related work Linear inverse scattering aims reconstruct the spatial distribution of an object's dielectric permittivity from the measurements of the light it scatters (Wolf 1969 Kak & Slaney 2001). This problem arises in various applications, such ultrasound imaging (Bronstein et al,, 2002 optical microscopy (Choi et al.] [007]] [Sung et al. (009). and digital holography (Brady ett al. 2009). Due to the physical constraints on the number and placement sensors, the problem often ill-posed, as the scattered light field is undersampled. Linear inverse scattering is com- monly formulated as linear inverse problem using scattering models based on the first Born (Wolf, 1969) Rytov (Devaney] 1981) approximations, These models enable efficient computation and facilitate the use of convex optimization algorithms. On the other hand, nonlinear approaches have been developed image strongly scattering objects (Kamilov al., 2015 Tian & Waller 2015 Ma et al. 2018: Liu et al. 2018 Chen et al. 2020), although these methods generally have a higher computational complexity. Deep learning- based methods have also been explored for linear inverse scattering. A common approach is to train convolutional neural networks (CNNs) to directly invert the scattering process by learning an inverse mapping from the measurements to permittivity dis- tribution (Sun et al.) 2018 Li et al. 20188a с Wu et al.) (020).. Recent research has extended these efforts to more advanced deep learning techniques, such as neural fields (Liu et al. 2022 Cao al., 2024) and deep image priors (Zhou & Horstmeyer 2020).
B.2
COMPRESSED SENSING MULTI-COIL MRI
Problem details We use the raw multi-coil k-space data from the fastMRI knee dataset (Zbontar et al. 2018). We then estimate the coil sensitivity maps of each slice using the ISPTRiT T (Lecker
23

---

Published as a conference paper at ICLR 2025
et al. 2014) method implemented in SigPy Since different volumes in the dataset have different shapes, we adopt the preprocessing procedure in (Jalal et al., 202T). leading to images with x shape. The ground truth image is given by calculating the magnitude image of the Minimum Vari- ance Unbiased Estimator (MVUR), which is used for all numbers reported in Table I and Table The MVUE images are also used as ground truths for training the end-to-end deep learning methods Residual UNet and E2E -arreet (Sriram et al. 2020).
Related work Compressed sensing magnetic resonance imaging (CS-MRI) is a medical imaging technology that enables high-resolution visualization of human tissues with faster acquisition time than traditional MRI (Lustig et al.] 2007). Instead of fully sampling the measurement space k.. k-space), CS-MRI only takes sparse measurements and then solves an inverse problem that recov- ers the underlying image (uuttg et al. 2008). The traditional approach is to solve a regularized optimization problem that involves data-fit fit term and regularization term, such as the total vari- 1993). and the f1-ormm after a sparsifying transformation, such as 2008) and dictionary decomposition (Ravishankar & Bresler 2011 Huang et al. 2014 /Zhan et al.
ation (TV) (Bouman & Sauer
Wavelet transform (Ma et al.
2015)). End-to-end deep learning methods have also demonstrated
strong performance in MRI reconstruction. Prior works have proposed unrolled networks (Yang
ct al. 2016 Hammernik et al. 2017 Aggarwal al. 2017 [Schlemeer et al. 2018 Liu al. 202
Uased networks (Lee et al. (2yu et al.] 2017)) GAN-based networks (Yang et al. 2018
Quan et al.
2018), among others (Wang al.) 2016 Zhuu et al. 2018; Tezcan al. 2018 Luo
et al. 2019
Liu et al. 2020). These learning methods have achieved state-of-the-ard performance
on the fastMRI dataset (Zbontar al. 2018). Another line of work
employ image denoisers as
plug-and-play prior (Liu et al. 2020 Jalal et al. 2021 Sun et al. 2024) Recently, diffusion model-
based methods have been designed for CS CS-MRI reconstruction (Chung Ye 2022
Luo al. 2022
Chung et al. (023)).
B.3 BLACK HOLE IMAGING
Problem details Measurements for black hole imaging are obtained using Very Long Baseline Interferometry (VLBI). The cross-correlation of the recorded scalar electric fields at two telescopes, referred to as the (ideal) visibility, is related to the ideal source image through Fourier transform, as given by the van Cittert-Zemike theorem:
Hass (z) (xxx==/ J.1o 12 ( (p, 8)dpdö.
(15)
Here, (p, 8) denotes the angular coordinates of the source image, and (u(a,t) ((a,)) is the dimen- sionless baseline vector between two telescopes (a, b), orthogonal to the source direction. prac- tice, these measurements can be time-averaged over short intervals.
Due to atmospheric turbulence and instrumental calibration errors, the observed visibilities are cor- rupted by amplitude and phase errors, along with additive Gaussian thermal noise (EHTC 2019a Sun et al. 2024):
V(a.b) = g' =(d--o6)
(z) -60
+ m
(16)
Note that there are three main sources of noise at time 1: gain errors g gh. phase errors &i of and baseline-based additive white Gaussian noise n/a.b) While the phase of the visibility cannot be
directly used due to phase errors, the product of three visibilities among any set of three telescopes, known the bispectrum, can be computed to retain useful information. Specifically, the phase of the bispectrum, termed the closure phase, effectively cancels out phase errors in the observed visibilities. Similarly, strategy can be employed cancel out amplitude gain errors and extract information from the visibility amplitude (Blackburn et al. 2020 Formally, these quantities are defined as:
cp Va,a.d = <(V (a.b) (b.c) Vi V(ac));
| ((,,)) (V(,,))lVV VVV (o,oe) (K(ao)llV (V.aa))
(17)
logca =log
y (i,(bbbe)
24

---

Published as a conference paper at ICLR 2025
Here, 7 denotes the complex angle, and | computes the complex amplitude. For total of M telescopes, the number of closure phase measurements y (a.(a.).e) at time is (M-1)(M-2) and the Jogen number of log closure amplitude measurements (t,(a,b,c) is MMM-33) after accounting for redun- 2
dancy. To avoid having to solve for the calibration terms, black hole imaging methods can constrain closure quantities during inference. Since closure quantities are nonlinear transformations of the visibilities, the forward model in black hole imaging then becomes non-convex.
The total flux of the image source, representing the DC component of the Fourier transform, given by:
y nux = CClu 1 =(, 20p,8)dpds
(18)
To aggregate data over time intervals and telescope combinations, the forward model of black hole imaging can be expressed as:
y G(z,( = (G (loos0
(z)) (2)))
(v y logea yy )
(19)
The data consistency is typically assessed using the x² statistic:
Cly 2)
2 Xep
+
Xlogen
2
ffux
1 GT
8P
1
logca (z)
logea 1
G(lux()))
y fux
(20)
+
Nep
Cep
Mogga
Ologaa
Oflux
Here, acp. Ologaa and Tlux the estimated standard deviations of the measured closure phase, log closure amplitude, and flux, respectively. Additionally, Nap and Nogec represent the total number of time intervals and telescope combinations for the closure phase and log closure amplitude measure- ments.
Related work The Event Horizon Telescope (EHT) Collaboration aims to image black holes using global network of radio telescopes operating around 1mm wavelength. Through very-long- baseline interferometry (VLBI) (Thompson et al. 2017). data from these telescopes are synchro- nized to obtain measurements in 2D Fourier space of the sky's image, known as visibilities (van Cittert 1934 Zemiike 1938). These measurements only sparsely cover the
-pequiallfreuuencyy
space and are corrupted by instrument noise and atmospheric turbulence, making the inverse prob- lem of image recovery ill-posed. Using traditional imaging techniques, the EHT Collaboration has successfully imaged the supermassive black holes M87* (EHTC, 2019bb 2024) and SgrA (EHTC) 2022). The classical imaging algorithm is CLEAN HHggoom 1974; Clark 1980), implemented in the DIFMAP (Shepherd 1997 2011) software. DIFMAP is an inverse modeling approach that starts with the "dirty" image (given by the inverse Fourier transform of the visibilities) and iteratively de- convolves the image with an estimate point-spread function to "clean" " the image. Since DI FMAP somman--n-thh-loog
often requires developed and used regularized maximum-likelilen
chose not to present results from DIFMAP. The EHT has also
approaches, namely eht sont-iime ..mmiinn (Chael al., 2016 2018 2019) and SMILI (Akiyama et 2017b 2019). Although they regularize and optimize the image differently (EHTC (2019b). 2019)) eht - immgiiiiii ing and SMILI both iteratively update an estimated image to agree with the measured data and regularization assumptions. Because of the simple regularization they choose to use, these baseline methods are limited in the amount of visual detail they can recover and do not recover detail far beyond the intrinsic resolution of the measurements. Some deep-learning beasen regularization approaches have been proposed for VLBI (Feng et al. 2224 Feng & Bouman, 2024 Dia et al. (202)), but most plug-and-plaa inverse diffusion solvers have not been validated on black hole imaging.
Multi-modal observation As previously discussed, the non-convex and sparse measurement char- acteristics of black hole imaging cause the inverse problem to exhibit multi-modal behavior. Con- sequently, the resulting samples may follow systematic modes that, while potentially quite different from the true source images, fit the observational data well and exhibit high prior likelihood. Fig- ure 7 iillustrates two such modes discovered by DAPS and PnP-DM This motti-modal behavior has not been extensively formulated or discussed in previous literature, and we believe it represents a phenomenon worthy of further investigation.
25

---

Published as a conference paper at ICLR 2025
PnP-DM
DAPS
Ground truth
Xcy 1.149
Xpp 1.110
Xep
1.295
Xep
1.095
Figure 7: Multi-modal example on black hole imaging. discovered by DAPS and PnP DD.
The image shows two systematic modes
B.4 FULL WAVEFORM INVERSION
Problem details In Eq. S solving for the pressure wavefield u given the velocity v and source term g defines the forward modeling process. We use open-source software, Devito (Louboutin et al. 2019), for both forward modeling and adjoint FWI. We use a 128 128 mesh to discretize physical domain of 2.54km 1.27km, with horizontal spacing of 20m and vertical spacing of 10m. The time step is set to 0.001s which satisfies the Couran-Frreddich--LLeyy (CFL) con- dition (Courant et al. 1967 We use Ricker wavelet with central frequency of 5Hz excite the wavefield and model it for 1s. The natural boundary condition is set for the top boundary (free surface) which will generate reflected waves, while the absorbing boundary condition (Clayton & Engguist 1977) is set for the rest boundaries to avoid artificial reflections. The absorbing boundary width set to 80 grid points.
Inferring the subsurface velocity from observed data at receivers defines the inverse problem. We put 129 receivers evenly near the free surface (at a depth of 10m) to model realistic scenario. We excite 16 sources evenly at depth of 1270m. This source-receiver geometry is designed for the entire physical medium to be sampled by seismic waves, making it thcoretically feasible to invert for Devito uses the adjoint-state method to estimate the gradient by cross-correlating the forward and adjoint wavefields at zero time lag (Plessix 2006):
m T
2 ultoou [t] where
(21)
z
where & is the objective function, N₁ the number of time steps, is the forward wavefield, and du is the adjoint wavefield. du is generated by treating the receivers sources and back pock-propagating the residual öd between the modeled and observed data into the model:
A ou = P P'
(22)
=d=y- d.
This process is repeated for each source, and the gradients are summed to update the parameters of interest.
B.5 HAVIER-STOKES
Forward modeling The following 2D Navier-Stokes equation for viscous, incompressible fluid in vorticity form on a torus defines the forward model.
u, t) + u(x t) Vw( t) vAw(a, t) + f(x),
E (0, E (0, 7
V u(xx, t) 0,
(0, (0,27)² t 2 [0,7]
(23)
w(x, 0) wo(
(0,2m) 2m)²
26

---

Published as a conference paper at ICLR 2025
where E C (10, T7; Her (0, 2x)) for any O is the velocity field, V X is the per (00, 2n; is the initial vorticity, v E RR+ the viscosity coefficient, and E per ((0, IRR) is the forcing function. The solution operator G ww - wr is defined as the operator mapping the vorticity from the initial vorticity to the vorticity time T. We implement the forward model using a pseudo-spectral solver with adaptive time stepping (He & Sun 2007))
vorticity, wg I
Dataset To generate the training and test samples, we first draw independent identically distributed samples from the Gaussian random field N (0, (-. 97) ()))-))))) )) where A denotes the negative Laplacian. Then, we evolve them according to Equation
| for 5 time units to get the final vorticity filed, which generates an empirical distribution of the vorticity field with rich flow features. We
the forcing function f(x) = -4 cos( (4x2).
B.6 PRETRAINED DIFFUSION MODEL DETAILS
We train diffusion models following the pipeline from Karras et al. 2022), using UNet architectures from (Dhariwal & Nichol 202T) and (Song et 2020 Detailed network configurations can be found in Table!
Table 11: Model Card for pre-trained diffusion models.
Inverse scattering
Black holc
MRI
FWI
2D Navier-Stokes
Input resolution
128 x 128
64 64
x 320 x 320
128 x 128
128 x 128
Attention blocks encoder/decoder Residual blocks per resolution Attention resolutions
5
1
]
(16)
{16)
(16)
(16)
{16)
# Parameters
26.8M
20.0M
26.0M
26.8M
26.8M
Training steps
50,000
50,000
100,000
50,000
50,000
B.7 ALGORITHMS AND PARAMETER CHOICES
B.7.1 PROBLEM-SPECING BASELINES
Black hole imaging We use SMILI AAiyymaa et al. 2017b a: 2019) and eht eht- teat-imaging (Chael et al. 2016, 2018 2019) as our baseline methods. To ensure compatibility with the default hyper- parameters of these methods, preprocess the test dataset accordingly.
Full waveform inversion A classic baseline for full waveform inversion is LBFGS. We set the maximum iteration to 5 and perform 100 global update steps with a Wolfe line search. The second baseline consider is the Adam optimization algorithm (Kingma 2014). We implement the Adam optimizer with a learning rate of 0.02 with the learning rate decay to minimize the data misfit term. For the traditional method, the initialization is smoothed version of the ground truth, which is blurred using a Gaussian filter with = 20. We perform the inversion for 300 iterations.
Linear inverse scattering We include FIS IISTA-VV (Sun et al. 2019) as a traditional optimization- based method. We set batch size B 20 and 5 x 10 for all experiments.
Compressed sensing multi-coil MRI We utilize both traditional methods, such as Wa (Lustig et al.] 2007; Lustig et al. 2008) and TV, as well as end-to-end models like Residual UNet and B2E-VarNes (Sriram et al. 2020). For the traditional methods, apply the same hyperparameter search strategy for fine-tuning, while the end-to-end models are trained using the Adam optimizer with learning rate of 1 x 10 until convergence.
Navier-Stokes equation The traditional baseline we implement is the Ensemble Kalman Inversion (EKI) first proposed in!lglesias et al. (2013). is implemented with 2048 particles, 500 update steps, and adaptive step size used in (Kovachki & Stuart 2019) to ensure similar computation budget. Additional baselines include DPS-EGSS and DPS CGSC, which are natural D2S extensions that replace gradient by zeroth-order gradient estimation first introduced in Zheng et al. (2024). More
27

---

Published as a conference paper at ICLR 2025
DAPS
PnP-DM
DPS
Ground truth
Recsiver 180
PSNR: 30.40
PSNR: 27.60
PSNR: 25.25
Receiver 60
22.56
PSNR:
Figure 8: Robustness to a human face prior in linear inverse scattering shows that methods requiring more data gradient steps tend to generalize better than those that prioritize the prior more.
specifically, use forward and central Gaussian Smoothed Gradient estimation technique (Berahas et al. 2022).
B.7.2 HYPERPARAMETER SELECTION
To ensure sufficient tuning of the hyperparameters for each algorithm, we employ hybrid strategy combining grid search with Bayesian optimization and early termination technique, using small validation dataset. Specifically, we first perform a coarse grid search to narrow down the search space and then apply Bayesian optimization. For problems where the forward model fast such as linear inverse scattering, MRI, and black hole imaging, we conduct 50-100 iterations of Bayesian optimization select the best hyperparameters. For computationally intensive problems such as full waveform inversion and Navier-Stokes equation, we use 10-30 iterations of Bayesian optimization combined with an early termination technique (Li et al.) 2018b), based on data misfit. The details of the search spaces for Bayesian optimization and the optimized hyperparameter choices are listed in Table 12
C FUTURE DIRECTION
In this section, we outline few additional future directions for benchmarking PnPDP methods for solving inverse problems.
Robustness to forward model mismatch In our paper, only consider the setting where the forward model exact and explicit. However, in real-world scenarios, the forward model is often approximation. Studying the robustness of algorithms when there is mismatch between the assumed and actual forward model would be an essential step toward their practical deployment. This includes exploring how algorithms handle noisy or imperfect forward model representations.
Robustness to prior mismatch The robustness of a diffusion prior often depends on two key fac- tors: the degree of ill-posedness and the balance between the prior and the observation in the sam- pling algorithm. Highly ill-posed tasks, such as black hole imaging, require more in-distribution prior to achieve reasonable results, whereas less ill-posed tasks, such as linear inverse scattering, are less sensitive to this requirement. Regarding the balance between prior and observation, methods like DAPS and PnP-DM, which incorporate more data gradient steps, tend to be more robust to out- of-distribution prior than methods like DP We include preliminary discussion presents comparison of the robustness of different algorithms in the linear inverse scattering using a prior trained on FFHQ x56 human face images. We will pursue further exploration this direction in the future.
this in Figure
Multi-modal posterior In this work, we focus exclusively on scenarios where the ground truth is unimodal. However, some real-world inverse problems might exhibit posteriors that inherently
28

---

Published as a conference paper at ICLR 2025
Table 12: Hyperparameter search space and final choices of the -aae-- algorithms on all five inverse problems. Columns marked with task names present the chosen values for the reported main results Appendix A. These values are selected by a hybrid hyperparameter search strategy described in Appendix B.7.2
Parmees/
Search space
Linear inverse scattering (260/ (80)
Black hole
MRI (Sim. Raw]
pw1
2D Navier Stokes
DPS
Guiduaee wnde
[1o72 _0
280/3800,620 (625
0.003
0.580 128
10
LGD
Grodance scale
[10 7. 10 [1, [1,
31000, 1330000
010082
11.33
MC simples
200
REDDIT
Leorning rnee
[1011 1.0]
0.04 0,0005
0.05
x 10-2 22:06 x 1o-2
Regulariztionn Xoses
[0 1, 1.0)
0.25
2.51 10 /2.72 x 10 "
0.1 linear
Regulantzation schodule
constant, lnner, sqrt [10 " _.0"
constans
communt
sqgt
Graliest weight
1500
0.0004
6.68 10' /1.7 x 10
DiMPER
sampling
(200, 600..... 1000) [1,10F]
200
1000
1000 1633 1:31 /1.47
1000
Regulanzation
x 10- 1075/10-1
133.6
50.6
Soccastkktty
[10-1.1
0.34
[10-2,
10-1 10-1
PaPDM
Arecaling step
200] [10,
100
100
100
150
10
Asscaling doeay
[0.60,
0.9
0.95
Langein step size
[o 1,
1/d */10
Langvin step
900
Nouse
10t1
/1.15 10-22
Asscaling step
2001
200
200
Diffussion step
[1,
Langevia step size
[10"), 10
11005//2 10
10 /1.52 10
Langevia step number
[B0,500]
100
Neese
/4.77
Step vize daeay
[01,]] []
1/1.0.5
DDRM
Stochasiiity
1]
0.55
DUNM
Stochastictty
0.95
time movel soep
[0.5]
FIGOM
Sucdhasicity q
Suechaticity
punticles.
MCG-dir
patticles
[1.
16
DPS-IGSG
Guiduse
[1072 10.
DPS-etiSG
Guilurez
10-",30"
DPG
samplex
(1000, 2000.......00000
4000
scale
SCG
4
G12
Guidunez scale
(1.0. 2.0.2.0.4 4.0) (512, 1021,
2,0
panticle
2045
multimodal, reflecting multiple plausible solutions given the observations. Developing systematic benchmarks to evaluate how accurately different PnPDP methods can capture such multimodal pos- teriors is an interesting and challenging research question.
29


Reviews:
[{"review_id": "NhSs6txAZf", "reviewer": "Reviewer_69mk", "summary": "A paper entitled “INVERSEBENCH: BENCHMARKING PLUG-AND-PLAY DIFFUSION MODELS FOR SCIENTIFIC INVERSE PROBLEMS” proposes a systematic benchmark for a number of scientific inverse problems that go beyond natural mages. The authors suggest optical tomography, black hole imaging, medical imaging, seismology, and fluid dynamics as a set of tasks for the benchmark. Next, the authors evaluate a handful of SOTA models with respect to their efficiency and accuracy on respective inverse tasks.", "questions": "Perhaps the authors could propose an approach to quantify the contribution of hallucinations? This is especially important for medical images, as hallucinations can lead to a wrong diagnosis.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- This is an important and timely contribution. It has the potential to improve generalisation and standardisation across solutions addressing inverse problems in science.", "weaknesses": "- Using SSIM and PSNR is not always sufficient in medical imaging. Perhaps the authors could propose an approach to quantify the contribution of hallucinations.\n- Although the title claims “Scientific Inverse Problems”, the collection of problems is extremely physics-heavy. The only non-physics problem the authors are looking at is MRI, I suggest toning down the claim in the title.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r3Cd21JO55", "reviewer": "Reviewer_TH8o", "summary": "This submission made technical contributions by creating and demonstrating the INVERSEBEHCN framework, which is designed for evaluating plug-and-play diffusion models on scientific inverse problems such as black hole imaging, fluid dynamics, medical imaging, and so on. The authors compared 14 representative methods in different problem settings (i.e., forward model), considering the computation efficiency and solution accuracy. What's more, the adaptability of each method to different types of forward models was investigated.", "questions": "1. How to make sure that each method is optimally tuned for different types of inverse problems, such as implementation details, hyperparameter choice? Even we understand that this would be a huge amount of work to tune every method. \n\n2. As shown in Table 11, methods like Reddiff, DiffPIR, PnPDM, and DAPS have more hyperparameters for tuning than other methods. Meanwhile, these methods tend to have better positions in the benchmark shown in Figure 1. Could this suggest that the other methods are genuinely worse or that they may be undertuned?\n\n3. For the domain-specific evaluation like compressed sensing MRI, the papers that apply generative models to MRI reconstruction in a plug-and-play way have been published in some domain-specific journals. To reach a broader impact, it is reasonable to briefly introduce the following works.\n\n     1. Tezcan, Kerem C., et al. \"MR image reconstruction using deep density priors.\" IEEE transactions on medical imaging 38.7 (2018): 1633-1642.\n     2. Luo, Guanxiong, et al. \"MRI reconstruction using deep Bayesian estimation.\" Magnetic resonance in medicine 84.4 (2020): 2246-2261.\n     3. Luo, Guanxiong, et al. \"Bayesian MRI reconstruction with joint uncertainty estimation using diffusion models.\" Magnetic Resonance in Medicine 90.1 (2023): 295-311.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed INVERSEBECH fills a gap in testing diffusion models beyond natural image tasks by evaluating them to scientifically meaningful, diverse, and challenging problems. The selected 14 methods are evaluated in multiple perspectives, such as efficiency, accuracy and adaptability.", "weaknesses": "1. The discussion on sensitivity to hyperparameters and initialization for each method is limited.\n\n2. The investigation into out-of-distribution could be more elaborated, as this could usually happen in the real-world inverse problem. Would a better sampling method help us to overcome the problem caused by out-of-distribution, or is this more related to trained prior?\n\n3. Referring to the selected 14 methods as \"plug-and-play\" methods is potentially misleading, especially since one of them is specifically a plug-and-play diffusion method (Wu et al., 2024). This raises the question of whether \"plug-and-play\" should be considered a more narrowly defined concept.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8IvhjfKslW", "reviewer": "Reviewer_SNSr", "summary": "Authors propose InverseBench, a benchmark/framework for evaluating diffusion prior based inverse problem solvers across five distinct scientific inverse problems that arise from black hole imaging, optical tomography, etc. Each of these problems present unique challenges for reconstruction that is different than the existing benchmarks. Authors benchmark 14 diffusion based algorithms that can be grouped into 4 categories (Guidance based, Variable splitting, Variational Bayes, Sequential Monte Carlo) and compare against problem specific baselines.", "questions": "* Line 316: \"Since this dataset is not publicly available, we generate synthetic images from a pre-trained diffusion modes...\" Do you use the model that is trained using 50k images to synthetically generate 105 more images for validation and test purposes? In general this part is not clear to me.\n* As a future direction, robustness of diffusion prior methods can be systematically investigated. That is suppose measurement is obtained via forward model $G$ but reconstructed believing that it came from \n$\\tilde{G}$. The experiments conducted on CS-MRI already suggests that PnP diffusion prior methods are more robust than baseline and end-to-end methods but it is also interesting to see comparison between different PnPDP algorithm.\n* Have the authors considered benchmarking diffusion model checkpoints that are pre-trained on natural images (such as ImageNet) on these tasks? It would be interesting to see if natural image priors are useful for the tasks in the InverseBench.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* The paper is written very well. I truly enjoyed reading it.\n* Having a standardized benchmark to evaluate diffusion prior based algorithms is an important contribution to the inverse problems community. I believe the forward models that are included in the InverseBench are diverse, practically relevant with diverse characteristics (linear vs non-linear, with or without SVD decomposition, etc.).\n* The diffusion algorithms that are considered in this paper are comprehensive.", "weaknesses": "* While I appreciate the comprehensiveness of experiments. I would expect more detailed discussion on experimental results. For instance, which methods (or category of methods) work better for each task based on the experimental results? If a method is competitive when used in natural image inpainting, super-resolution but not in InverseBench, is there some intuition/explanation why that might be the case?\n* See the questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EGUvj14qAn", "reviewer": "Reviewer_MUia", "summary": "The paper provides a comprehensive summary of the performance of 14 different diffusion-based inverse problem solvers on 5 scientific applications, each with unique forward operators. Extensive experiments on each application reveal general trends such as a bias towards the prior with out-of-distribution data and strong performance compared with conventional baselines. They contribute an open-source codebase of datasets and pretrained models to enable expedited exploration in this space of inverse problems.", "questions": "- Are the reported results (say in Table 4) for a single posterior sample? Or do you draw many samples and compute a posterior mean?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- With such a wide and rapidly expanding collection of diffusion-based methods, this paper provides a valuable assessment of existing models across diverse problems. For those working in the space of inverse imaging problems, this gives a strong starting point to assess which model to use for a given application or as a baseline.\n- The experiments are extensive and thorough. As such, they are able to discover high-level insights and trends across problems. This breadth of experimentation would be larger unavailable to most researchers; thus, the results present a significant contribution to the community in my view.\n- Open-sourced datasets and pretrained models contribute a well-curated foundation for other researchers to build upon\n- The paper is very well-written and concise with clear introductions to the complex inverse problems and well-thought-out analyses of the results.", "weaknesses": "- The value of some of the images in Figure 2 is limited since its difficult to actually see differences between the reconstructions. Particularly, the MRI reconstructions for the PnP methods look the exact same. It may be more useful to provide a zoomed-in region where the differences are more apparent.\n\nMinor Points:\n- L161: Typo with \"... generally intractable as so various approximations...\"\n- L508: Typo with \"inverse scattering on sources that contain more than cells...\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)