Guided Diffusion Sampling on Function Spaces with Applications to PDEs
Authors: Jiachen Yao
Abbas Mammadov
Julius Berner
Gavin Kerrigan
Jong Chul Ye
Kamyar Azizzadenesheli
Anima Anandkumar
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion Models
Function Spaces
Partial Differential Equations
Inverse Problems
Primary area: machine_learning_for_sciences
TL;DR: We present a general framework to solve inverse PDE problems through function space diffusion models in a plug-and-play way.

Abstract: We propose a general framework for conditional sampling in PDE-based inverse problems, targeting the recovery of whole solutions from extremely sparse or noisy measurements.
This is accomplished by a function-space diffusion model and plug-and-play guidance for conditioning.
Our method first trains an unconditional discretization-agnostic denoising model using neural operator architectures. 
At inference, we refine the samples to satisfy sparse observation data via a gradient-based guidance mechanism.
Through rigorous mathematical analysis, we extend Tweedie's formula to infinite-dimensional Banach spaces, providing the theoretical foundation for our posterior sampling approach.
Our method (FunDPS) accurately captures posterior distribution in function spaces under minimal supervision and severe data scarcity. Across five PDE tasks with only 3\% observation, our method achieves an average 32\% accuracy improvement over state-of-the-art fixed-resolution diffusion baselines while reducing sampling steps by 4x. Furthermore, multi-resolution fine-tuning ensures strong cross-resolution generalizability and speedup. To the best of our knowledge, this is the first diffusion-based framework to operate independently of discretization, offering a practical and flexible solution for forward and inverse problems in the context of PDEs. Code is available at https://github.com/neuraloperator/FunDPS.

Guided Diffusion Sampling on Function Spaces with Applications to PDEs
Jiachen Yao; Abbas Mammadov* California Institute of Technology
Julius Berner NVIDIA
Gavin Kerrigan University of Oxford
Jong Chul Ye KAIST
Kamyar Azizzadenesheli
Anima Anandkumar
NVIDIA
California Institute of Technology
Abstract
We propose general framework for conditional sampling in PDE-based inverse problems, targeting the recovery of whole solutions from extremely sparse or noisy measurements. This accomplished by function-space diffusion model and plug-and-pla guidance for conditioning, Our method first trains an unconditional discretization-agnen aagnosiic denoising model using neural operator architectures. At inference, we refine the samples to satisfy sparse observation data via a gradient- based guidance mechanism. Through rigorous mathematical analysis, we extend Tweedie's is formula to infiniteddimensionall Banach spaces, providing the theoretical foundation for our posterior sampling approach. Our method (FunDPS) accu- rately captures posterior distribution in function spaces under minimal supervision and severe data scarcity. Across five PDE tasks with only 3% observation, our method achieves average 32% accuracy improvement over state-of- the-art fixed- resolution diffusion baselines while reducing sampling steps by 4x. Furthermore, multi-resolution fine-tuning ensures strong cross-resolution generalizability and speedup. To the best our knowledge, this is the first diffusion- framework operate independently of discretization, offering practical and flexible solution for forward and inverse problems in the context of PDEs. Code available at httpss//ttttb..ccommmmuuaappp//t
Introduction
Conditional sampling is ubiquitous task in machine learning and scientific computing that involves generating samples from a distribution conditioned on certain constraints or observations. This task appears naturally in many applications where we need reconstruct nigh-dimensional data given partial, corrupted, or indirect measurements. It has been extensively studied in the image domain for tasks like inpainting, deblurring, and super-resolution [15).
In scientific domains, an example of conditional sampling is climate modeling, where scientists predict future atmospheric states based on limited sensor measurements. Traditional forecasting methods try to produce a single. best-case outcome based on the available data. However, the chaotic nature of weather systems means small uncertainties in initial measurements different outcomes [6]. Conditional sampling methods can generate multiple plausible weather states consistent with the available data (7]. This probabilistic approach is especially valuable for predicting extreme weather events and making informed policy decisions. Beyond weather prediction,
lead to drastically
"Equal contribution.
Now at the University of Oxford.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Observation
FunDPS Error (Qurs)
DittusionPDE Erooe
Observation
FunDPS Firor (Ours)
DiflisionDDE Frror
Heknholle
Dovey Forward
Forwarl
Helmbollt Inverse
Darey
Figure 1: Comparison between our method (FunDPS) with the state-of-thh-aan diffusion baseline, DiffusionPDE, the Darcy Flow and Helmholtz problems. The left column shows the sparse observation measurements (3% of total points), while the other two columns display the absolute reconstruction error of our method and DiffusionPDE, respectively. FunDPS achieves superior accuracy with an order of magnitude fewer sampling steps.
conditional sampling naturally arises in scientific tasks such as solving inverse problems in physical systems. These examples include recovering permeability fields in subsurface flows (8) inferring
material properties in elasticity [9], and identifying initial conditions for fluid simulations [10]
Traditional numerical methods often struggle with ill- ill-posed problems such as when observations are sparse or noisy, they cannot effectively leverage prior knowledge about the solutions. While Markov Chain Monte Carlo (MCMC) methods can theoretically sample from the posterior distribution, they are computationally intensive and often require exponentially many iterations to converge, making them impractical for high-dimensional problems [I]] Moreover, constructing effective proposal distributions for MCMC is particularly challenging, leading to poor mixing times [12].
Bayesian approach to conditional sampling A principled approach to conditional sampling from Bayesian perspective [13). We aim sample from the posterior distribution p(a|ut u)) p(a) p( a). where u. represents our observations. In inverse problems, this posterior defined through a forward operator A and the likelihood function p(ula) p(u/ (a)). The forward operator A can take many forms depending on the application. For instance, when is masking operator that selects specific coordinates, forms the reconstruction problem from sparse observations. When A represents a PDE solution operator mapping from parameters to solutions, we obtain parameter inference problems in physics. The likelihood function can also incorporate different noise assumptions to handle varying measurement uncertainties. This general framework effectively consolidates many conditional sampling problems into one unified mathematical formulation.
Diffussin Models Function Spaces Physical systems are inherently described by continuous functions rather than discrete grids. Consequently, methods developed for mnite-dimensional settings tend to perform poorly when applied to different discretizations
infinite-din diniitedddmeniionall problems. While diffusion models have been extended to function space settings, existing approaches have limitations. Denoising Diffusion Operator [14] uplifts the generative process to infinite dimensions by umntion-vallus' annealed Langevin dynamics, but is limited
uncontrollable generation pipeline. Baldassari et al. [15] suggests a conditional denoising estimator in infinite dimensions, yet its adaptability is limited by the requirement of pre-trained conditional
model. In real-world applications, the density and configuration of sensors can vary significantly, and training a separate model for each measurement setup is costly. This highlights the need for method capable of performing diffusion posterior sampling in function spaces using an unconditional score model. The ability use one such model for various downstream tasks offers substantial flexibility, effectively decoupling the core model development from specific application requirements.
Contributions We introduce Function-space Diffusion Posterior Sampling (FunDPS), novel framework that leverages diffusion operators to address inverse problems by sssolution-independeee conditional sampling. Our contributions can be regarded tri-fold:

---

Reverse Diffusion Sampling with FunDPS Guidance
Training
a
0
Joint data with different
Reconstruction
High-res
FunDPS
Gaussian Random Field
a
Denoising Diffusion Neural Operator
Denoising Dillision
Neural Operator
Likelihood ((lA)
denstied
Optimior EDM Loss
Figure 2: The sampling and training pipelines of FunDPS. During inference, we utilize a standard reverse diffusion approach with additional FunDPS guidance to drag the samples the posterior. During training, the model is based on a U-shaped neural operator, and Gaussian random fields are used the noise sampler ensure consistency within function spaces. Notations are detailed in blue box in the bottom-left corner, where function a jointly represents the PDE parameters and solution.
aefinite-dimensionn Tweedie's formula.
We develop a novel theoretical framework for posterior sampling in infinite-din sienitt-dimeesionall spaces by extending Tweedie's formula the Banach space setting. Tweedie's formula gives a closed-form expression for the posterior mean of the reverse diffusion process and serves as the basis for many diffusion solvers. However, prior to this work, it has only been shown hold in finite dimensions. We rigorously establish generalization of Tweedie's 's formula to function spaces, thereby providing plug-and-play approach for guided sampling we study the scenario where given data measure is perturbed via an additive Gaussian measure, and show an equivalence between the score of the resulting noisy distribution and the conditional expectation of the noise-free sample given noisy one. This generalization, combined with measure- theoretic decomposition of the conditional score, enables us inject measurement consistency (or PDE constraints) into an unconditional function-space diffusion model at inference time.
inverse problems. Concretely,
2) Multi-resolution conditional generation pipeline in function space (FunDPS).
We propose the function-space diffusion posterior sampling method that builds on the above theo- retical framework. the pipeline which is shown in Figure An unconditional diffusion model is first pretrained using the score matching objective as defined in [14]. The model works on joint functional representation of PDE parameters and solutions, which allows to solve forward prob- lems, inverse problems, and combination thereof using partial observations. The model employs a multi-resolution training strategy, which begins training on a coarse grid, then fine-tunes on finer one. It reduces GPU training hours by 25% which is enabled by the Gaussian random field (GRF)- based noise model and neural operator architecture. After training the unconditional model, we generate conditional samples follows: The sample is initialized with GRF noise, followed by iterative denoising via a second-order deterministic sampler [16]. At each step, the sample evolves according to the conditional score operator, which decomposes into data prior (from the pretrained model) and observation- based likelihoods. Our generalized Tweedie's formula enables us to effi- ciently approximate the measurement lig-liiel FunDPS can also employ multi-resolution inference by performing most sampling steps at a lower resolution and only upsampling towards the end, which alone yields 2x speedup while maintaining accuracy.
3) Extensive experiments with SotA performance in both speed and accuracy. We test our approach on five challenging PDEs, which vary widely in their difficulty due to complex input distributions (GRF's multi-scale features), different boundary conditions (Dirichlet and periodic), and nonlinear patterns (Navier-Stokes). We simulate extreme obfuscation in measurements by masking. Specifically, for forward tasks (e.g., recovering the final state), we only observe 3% of the initial condition and reconstruct the entire solution, while for inverse tasks, observe

---

only 3% of the final state and reconstruct unknown initial or coefficient fields. Despite these difficulties, our method consistently achieves top performance (see Figure I]. Across all five tasks, we reduce the average error by roughly 32% compared to the state-of-the-art diffusion- -ased solver (DiffusionPDE:: [7], which operates at fixed trained resolution) and surpass deterministic neural PDE solvers by even larger margins. Moreover, our reverse diffusion process requires only 200-500 steps to converge- to 10x fewer than DiffusionPDE- while maintaining superior accuracy (see Figure Bp). We attribute these improvements primarily to FunDPS's ' function-space formulation, which aligns better with physics functions and applies smoother guidance, cutting inference wall-clock time by 25x compared to DiffusionPDE with minimal impact accuracy.
2 Related Works
Neural PDE solvers Neural PDE solvers aim approximate PDE solutions or operators using deep networks. Physics-informed neural networks (PINNs) [8] incorporate PDE residuals and
boundary conditions into the loss function, which enables both forward and inverse problem solving
but often at the cost of challenging optimization or limited scalability [19 [2]] Operator-lcarning
approaches such as Fourier Neural Operator (FNO) 21 and DeepONet [2 23 learn resolution-
invariant mappings between function spaces. Meanwhile, graph-based neural PDE solvers 24 25
treat spatial discretization as a graph rather than functions and apply message passing to approximate PDE dynamics on potentially irregular domains. Despite these breakthroughs, most existing works yield single (deterministic) outputs rather than sampling from a posterior distribution over PDE equations in function spaces, an aspect that we address through our diffusion-based approach.
Inverse PDE problems Inverse problems in PDEs, like recovering material properties or flow states from sparse or noisy measurements, have been mainly tackled by PDE-constrained optimization or Bayesian inference settings. Recently, data-driven approaches like PINN, FNO, and DeepONet [18] 21 have emerged as efficient approaches tackle PDE problems. However, their main target is not recovering the full fields from partial observations, but learning the function More recently. Energy Transformer [26] can reconstruct full fields from incomplete or irregular data, but they are limited by the patch size. Others have explored cGANO for broadband ground- motion synthesis (27). unified latent representations for forward-inverse subsurface imaging (28), measurement- -uided diffusion in geophysical tasks (29]. and physically consistent score-based methods with PDE constraints [30]. Although these works demonstrate promising directions and partial overlap with our goals, they typically rely on discretized image domains or separately trained physical models, lacking issolution-invaiian features and flexibility.
mappings.
Diffusion models have demonstrated high quality and stability in generation tasks 33 16] By learning a score function-a a vector field pointing towards high- density data regions at different noise levels-we can reverse the noising process to sample from the prior distribution v(a) (4).6)) Diffusion models have been widely used to solve inverse problems in a plug-and-play manner, utilizing a pre-trained unconditional diffusion model as a prior and its sampling process to integrate constraints [37 33). One key ad advantage of these methods is their flexibility- the same unconditional model can be used for various downstream tasks without retraining. Various approaches have been proposed to leverage diffusion priors, ranging from guidance terms or resampling strategies within the generative process (39 40 to integrations within variational frameworks [4]] 42). While widely researched for fixed-resolution inverse problems like images, their application to partial differential equations (PDEs) in function spaces is less explored. DiffusionPDE [17 learns joint distributions of PDE parameters and solutions, but its reliance on fixed discretizations limits its practical applicability in scientific computing.
Diffusion- based posterior sampling
Generative neural operators For physics-informed problems, developing true function-space generative methods is crucial as data are inherently functions. Lim et al. [4] proposes Denoising Diffusion Operators (DDOs), which extend diffusion processes to Gaussian random fields and prove ssolution-indeepeddenc under discretization, while Pidstrigach et al. mathematical framework for infinite-dimensional diffusion models that preserve key properties as discretization is refined. Another work uses adversarial training on function spaces (44) which introduces Generative Adversarial Neural Operators (GANO) to learn push-forward maps between probability measures in
4 provides a rigorous
Hilbert spaces. Flow matching techniques [45] offer an

---

alternative by directly learning velocity field whose induced transport matches the target distribution. This framework has very recently been extended anfinite-dimensionll settings 48]. Kerrigan et al. [9 further applies functional flow matching in a conditional inference context, providing rigorous training scheme for downstream tasks. contrast, we leverage a pre-trained functional prior and simply compose it with arbitrary measurement operators at test time in plug-and-play manner, without requiring task-specific re-training.
Method
We now present our function-space diffusion posterior sampling (FunDPS) framework. We begin by formulating the inverse problem in function spaces and establishing the Bayesian framework. We then derive the conditional score through the decomposition of likelihood and prior terms and develop the necessary approximations for practical implementation. Finally, we describe the complete FunDPS algorithm, including our multi-resolution training strategy.
3.1 Problem Settings
PDE-based inverse problems We focus on general inverse problems in function spaces, with the goal of retrieving an unknown function E A from measurement function u E li given by
A(a) + with A:A- + U,
(1)
where E is an U/valued random variable representing the measurement noise and A and U are separable Hilbert spaces. Such inverse problems frequently appear in the context of PDEs
Lef = 0 (on D) and flad= = g.
(2)
where Le is differential operator depending on coefficient function c, the function g encodes boundary or initial values on the domain D, and f is the solution function. For classical inverse problems, A can be the operator mapping the parameter functions of PDE, i.e. coefficient functions and boundary values g. to the corresponding solution f. Additionally, A can combine both a PDE solution operator and masking operator for solving inverse problems with sparse measurements.
Bayesian perspective For many practical solutions and masking operators, the inverse problem ill-posed, i.e., A is not injective nor stable, meaning that small changes in u can cause large variations in a. To address these issues, we consider a Bayesian perspective [3], where we assume a prior distribution v(a) on with the aim of sampling from the corresponding posterior distribution y" v(a u). All distributions, including the prior // and posterior 2 are considered as probability measures, and the existence of densities in fitnite-dimensional spaces often requires careful treatment.
We assume that the measurement noise is a Gaussian random field (GRF) with covariance op- erator Cy,, i.e., E is drawn from the Gaussian measure r = (O, C,,) on U4 which is the nat- ural extension of Gaussian noise to function spaces [50]. Observe that, given we have that u a a~ ا = (a)) C.2) is drawn from a Gaussian measure A with mean A(a). When the noise-free observation A(a) is an element of the Cameron-Martin space c (44) ((r) 4 associated with C,,, Cameron-Mann Martin theorem allows us to compute the Radon-Nikoday derivative
Ala) A(a) (u) exp ((u, (a)) (()) - (a)))))))
dry
In other words, Eq. gives the density of 17th with respect to the noise measure 17. We will use : 21 + R to represent the function (a u) = (u, A(a)),, Al) Loosely speaking, (aa, u) is the log-like likelihood of u given a. the special case that u E RR" is finite dimensional (for instance, when A is the composition of a PDE solution operator and a finite observation mask), the measure 77 corresponds
mean-zero zero Gaussian random variable with covariance matrix C, gax", In this case, when C, has full rank, the Cameron-Mann Marrrnn space is (R²)(R") = R" so that the log- likelihood is defined for any value of A(a).
We assume that the considered PDEs allow for unique, strong solutions in suitable space of functions.
We recall that the inner product and norm on the Cameron-Martin space Uo are readily computed via
(o. 41 (G) (C, 1/2 uo, C,, 1/2 uihu: Uo. U1 H()-
5

---

By Bayes' rule [3], the posterior y/ is absolutely continuous with respect to the prior measure v, and moreover, the corresponding Kadon-Nikodym derivative is proportional to the likelihood, i.e. du " do (a) x exp (+(a, u)) with the constant of proportionality depending on u but not a.
Diffusion prior We propose to learn the prior distribution v/(a) from data using the recent extension of diffusion models to function spaces [14] [1]. These methods define a sequence of distributions (a,) that progressively add noise to the data until approximately reaching tractable latent distri- bution for R r. Learning& a score approximator Da 1) Re E[ [, from the data via a variant of score matching, one can approximately reverse the noising process and sample from the data distribution. Moreover, by replacing the score with a conditional expectation, one can solve the reverse SDE given initial conditions 43 Notably, the denoiser Dy is parametrized as neural operator [1 (52), and the noise is chosen to be Gaussian random fields, as opposed to neural networks and multivariate Gaussian random variables for the finite-dimensional setting (see Appendix A for more details). The diffusion prior will be updated by (approximated) conditional likelihood during inference in order solve inverse problems.
3.2 Conditional Score via Likelihood and Prior Score
In our setting, the unconditional forward process of the diffusion model may be simulated via
a ag + 7,Et
Ee % do
for some specified time-dependent constants or > 0 14 Here, 7 C.2) is now a Gaussian measure on the space A. We will also use 7/2 (0, C,C,) ) to represent the corresponding Gaussian measure with scaled covariance. The spaces H (7), (7x) represent the Cameron-Martin spaces associated with these Gaussian measures. We will write Po (ao)) v(a) for the prior distribution over and vi (a.) for the marginal distribution of a, obtained during the forward process. Similarly, the conditional forward process is obtained in the same manner as Eq. 4 except with initial conditions ag ~ n drawn from the posterior corresponding to a given, fixed u. We will use v = va and v? for the corresponding measures. Informally, u/ can be thought of as VaPt (a. uu).
make the assumption that the prior Vn is supported on H()), i.e., (7)) = 1. Note that as the posterior v/ < vo is absolutely continuous with respect to the prior, we also have that v (7)) = Under this assumption, both v4 and 1/," are equivalent Ys the sense of mutual absolute continuity (see Appendix C and [14, Lemma 13]).
As in prior work 14 43
In this case, by the Madon-Nikodyyn theorem there exists density dut" dd/. We assume that the logarithm of this density is Fréchet differenniable along H (?a). The object of interest when seeking to sample from the posterior 1/" using a diffusion model the conditional score of 1/, defined
du dy
D DH log
A H(x)".
The unconditional score of v4 defined analogously. Previous work [4 uses this notion of score to build function-space diffusion models. To simplify the notation, we write Va, DH())) as shorthand for this Fréchet derivative.
However, in our setup, we do not assume we have access to the score of ut,, but rather only to the unconditional score of U2 and the log-likelihood function o u.. To overcome this, note that
dut (a.) Vay log Va, log d%
dut d=²((a)) (a:) + Va, log duc (a:) Va (a w) + Vay log dut du.
(at),
(6)
dy
dry
where (ar, u) = log(( / /dz)"/d (ax)) is the log-likelihood of u az Thus, we have managed to decompose the conditional score into a sum of a likelihood term and the prior score as desired.
3.3 Approximating the Likelihood in Function Spaces
While the calculations in the previous section in principle allow for plug-and-play posterior sampling, major difficulty is that the time-dependent likelihood क. is intractable. In particular, from Eq, 33) we know the log-likelihoon of u given a nrise-free sample ag, but Eq. (6 requires us have access
$ Following common conventions, we will also refer the noising process forward process.

---

to the ligeliied of u given a noisy sample In the next two sections, we develop a method for approximating this likelihood through a conditional expectation.
Let us write pi for the conditional distribution of u a, This measure can be sampled from by first predicting the clean dg from the noisy at, followed by sampling from the noise measure 17 A according to Eq. (T. Hence, for any measurable - l, we have
t (U) - A(oo) (U) dr
where 0/j: is the conditional measure of the reverse diffusion process given noisy ae Denote by (a,) = Ea ar] the expected value of this measure. (We will sometimes write åg simplify the notation when the dependency a, is clear from context.) Approximating Vojz ~ ö|ăg] with its mean, under the assumption that A(an) E H. (n) is an element of the CM space of 17, we have
11 1 (U)) = 1 Alaa)(()) (U) (V) = -A/e dn
dry A
(u) dej
(u) dn(u).
(8)
dn
dn
Note further that Eq. ④7) shows that if the prior un is such that A(ao) H(n) almost surely, then 14 ae < 17. In all, we have shown
dus (u) =
dn A
dn (u).
(9)
dn
Carrying this approximation over into the log-likelihood, we obtain &, u) RS o(ān, u) as defined by Eq. (3 Thus, when we are able to calculate V.. ((ás(a,), u), we may substitute this expression into Eq. (6) in order to guide the diffusion process to approximately sample from the posterior na.
In the special case that 2/ is finite dimensional (e-g., when A represents observing our PDE finitely many locations) and C, is full rank, observe that via straightforward calculation we have
( u) (1|u Aão Pa (
(10)
and so in this case we are justified in using an approximation to our log- ligeliieldoood of the form
Va, (ar, u) Va. (- (u A
(11)
as the two differ by a constant depending only on u. However, when I is infinite dimensional, the CaameronMarttn space (1) has measure zero under and so u A(ág) almost surely not be an element of (1) in which case Eq. (10) is ill-defined. It is also worth noting that this approximation is less accurate when noise level is high (33 which opens future work in this topic.
3.4 Approximation of Conditional Expectation via Tweedie's Formula
Tweedic's ' formula [54] plays fundamental role in diffusion solvers by providing a link between score functions and conditional expectations. It is cornerstone in many guided sampling methods, including MCG (38], DPS [40] and PSLD (55). In our framework, the approximated likelihood also relies on this efficient estimation of conditional expectations E[ay az), but in function spaces.
Generalization of Tweedie's 's formula function spaces requires careful treatment of several concepts. In definiteddimensionll spaces, we can no longer rely probability density functions or standard gradients as in the finite-dimensional case. The notion conditional expectation must be handled through weasure-theoretie tools, and the score function needs to be redefined using the Fréchet derivative and the Riesz representation theorem. Here we present our extension below.
Theorem 3.1 (Tweedie's formula in infinite-dine dimensioal Banach spaces). Let be separable Banach space. Assume that pi H())) = 1, the score of u is Fréchet differentiable along H(y), and that the Fréchet derivatives of dry" dr are p-almost surely bounded by a p-integrabbe function. Then, for v-almost every y.
E Y =R (Paco) log dy
dv
(v)
(12)
Proof. See Appendix

---

3.5 FunDPS: Function-Space Diffusion Posterior Sampling
With the analytically tractable likelihood approximation, sampling step. We will introduce the guidance term for the reverse diffusion process by defining a differentiable likelihood term given a measurement function. The FunDPS algorithm follows, with special attention to some design details of our implementation.
now proceed to the posterior
Guidance With the extended Tweedic's formula, we now have a closed form representation of áo(a,) = Eiao ag] given by Theorem 3.1 becomes differentiable when we use a neural operator to simulate the score function, enabling us to compute the likelihood function given in Eq. (IT
In practice, our observations u will be finite dimensional. We also assume that the covariance C₁ of the observation noise is approximately uncorrelated, such that C₂ can be well-approximated by a scaled version CT se cl of the identity. Plugging this and Eq. (11 into Eq. (6) with the trained score operator Do, we can now formalize the conditional score operator, acting as the guidance for posterior sampling
dut dry
du
Va. log
(a=) = Vol log
(a:) + Va, Dr((s, u) =o Do(a,, t) [V, -Vaollu-A(aa dry.
(13)
When there are multiple types of observation, we further define the total log likelihood as weighted sum of individual observation log likelihoods. This formulation introduces a vector of hyperpa- rameters Ç, where each component scales its corresponding likelihood term. These weights can be interpreted as confidence measures for observations. For simplicity, we also absorb & into Ç.
Algorithm depicts this reverse diffusion process, where we iteratively update the samples a; using the consistency between the given measurement u and the one obtained from the denoiser, i.e., A (DDo(a, a,, f.). Specifically, we propose the update rule
a.+1 < a, Ç Va, l|u A(Da(a,, (.)...
(14)
where Ç is predefined guidance weight above.
Joint embedding In practice, the solution and the parameter functions typically measured by sensors and can only be partially observed [56]. Hence, we focus on the general case where the operator A is an arbitrary masking operator, specifying the coordinates of the sensors, and the function a jointly represents the PDE parameters and solution. Our goal is learning the prior distribution of both initial and solution states via diffusion models. We then recover classical inverse problems by fully masking the coefficients. On the other hand, forward problems correspond to reconstructing the masked full solution from partial observations of the parameters. By partially masking either channel, sparse reconstruction problems are resolved.
we introduce a new training technique to learn the prior distribution with reduced computational costs. We first train the diffusion model on low-resolution data for a majority of the epochs and only train on higher resolution for the final epochs. This curriculum learning approach guides the model to efficiently learn coarser information at the earlier stages of training and finer high-frequency details in the later stages. Due to the discretization invariance of neural operators, the resulting model exhibits similar performance as training only on high resolutions, almost at the cost of oowressouution training.
Multi-resolution training Inspired by other works on operator learning 57 58
Muuti-resolution inference Beyond reducing steps, inference time per step be accelerated by initially processing at a lower resolution and progressively increasing it. This aligns with the concept of diffusion models generating autoregressively in resolution or frequency space (59]. We propose ReNoise, a bi-level multi-resolution inference method. Initially, samples are processed at a lower resolution. Subsequently, they upscaled, and inspired by ReSample(60) additional noise is introduced to mitigate artifacts and noise level mismatch arising from upscaling, followed by a few denoising steps the target resolution. This enables us achieve equivalent accuracy in half the computing time. We illustrate the pipeline in Figure 3b with implementation details in Appendix H.4
4 Experiments
We will now showease the efficiency, robustness, and multi-resolution generalizability of our proposed FunDPS through various tasks. Additional and ablation experiments can be found in Appendix H
8

---

(b)
10
llig-maektionn sampling Milni resolution sampling
FandiPS-Mica
Prant 112
10"
102M
Timee
100
125
150
175
Eap
Figure 3: (a) Comparison of FunDPS and DiffusionPDE terms accuracy and inference time with varying step sizes; (b) Demonstration of multi-resolution inference pipeline. is in lcals.
4.1 Experimental Setup
Datasets & Tasks We validate our approach by solving both forward and inverse problems on five different PDE problems. The problems span range of difficulties, characterized by complex input distributions arising from the multi-scale features GRFs, varying boundary conditions (Dirichlet and periodic), and nonlinear patterns inherent in the Navier-Stokes equations. We provide detailed description each PDE in Appendix E The objective
solve forward and inverse problems in sparse sensor settings. For forward tasks, we only observe 3% of the initial condition before reconstructing the entire solution. Conversely, for inverse tasks, we only observe 3% the final state before reconstructing unknown initial or coefficient fields.
Training & Inference We base our code on EDM-FS [61]. In particular, we adopt a U-shaped neural operator architecture [52] the denoiser Dg and modify the noising process according the discussion above. Our denoiser has 54M parameters, which is similar to DiffusionPDE's network size. The implementation is detailed in Appendix F The guidance weight Ç in Eq. (14) is tuned on small validation set, provided in Appendix G
Baselines Baseline comparisons include well-known deterministic PDE solvers such as FNO 211, PINO (62] DeepONet [22) and PINN [18]. well the state-of of-the-art ceftusiio-bbaee approach, DiffusionPDE [17]. We had correspondence with DiffusionPDE'ss authors about its reproducibility issues. We use reproduced results for comparison. Please refer to the Appendix for details.
4.2
Results
Main results We evaluate our method on both forward and inverse problems across five PDE tasks with sparse observations. Results shown in Table Even with severe occlusions, where only 3% of function points are visible, we are able to reconstruct both states effectively. Our approach always achieves the best results among all the baselines with the lowest error rates, surpassing all baselines with 32% higher accuracy with one-fourth steps. We provide qualitative results in Figure [ and Appendix K We also compared all methods on classical fully-observed forward and inverse problems, where FunDPS also showed superior performance as in Table 6 Further, we test our framework with diverse guidance methods to show its adaptability (Figure 6 6land Table We also Additionally, we provide results by using FDM as forward operator and compare with joint learning in Appendix [
conducted an ablation study on the effectiveness of our design choices in T2]
Inference speed We compare the inference time of method with the diffusion- based solver, DiffusionPDE,, as shown in Figure 3 With 200-step discretization of the reverse-time SDE, our method achieved superior accuracy with only one-tenth of the integration steps compared to DifIPDE. When we increase the number of steps, FunDPS further reduces the error, which suggests that the scaling law of inference time may hold for solving PDE problems with guided diffusion. This highlights the effectiveness of our function space formulation and guidance mechanism.
Regarding the actual run time, our model averages 55sssample for 500 steps (without multi-resolution inference technique) on single NVIDIA RTX 4090 GPU, while DiffusionPDE: takes 190s/sample for 2000 steps the same hardware and the same 128 x 128 discretization. This superior performance

---

Table I: Comparison different models on five PDE problems (in L relative error)
Steps (N)
Darcy Flow
Poisson
Helmholtz
Navier-Stokes
Navier-Stokes (BCs)
Forward
Inverse
Forward
Inverse
Forward
Inverse
Forward
Inverse
Forward
Inverse
FunDPS (ours)
200
2.88%
6.78%
2,04%
24.04%
2.20%
20.07%
3.99%
9.87%
5.91%
4.31%
FunDPS (ours)
500
2.49%
5.18%
1.99%
20.47% %
2.13%
17.16%
3.32%
8.48%
4.90%
4.08 %
DiffusionPDE
2000
6.07%
7.87%
4.88%
21.10%
12.64%
19.07%
3.78%
9.63%
9.69%
4.18%
FNO
28.2%
49.3%
100.9%
232.7%
98.2%
218.2%
101.4%
96.0%
82.8%
69.6%
PINO
35.2%
49.2%
107.1%
231.9%
106.5%
216.9%
101.4%
96.0%
81.1%
69.5%
DeepONet
38.3%
41.1%
155.5%
105.8%
123.1%
132.8%
103.2%
97.2%
97.7%
91.9%
PINN
48.8%
59.7%
128.1%
130.0%
142.3%
160.0%
142.7%
146.8%
100.1%
105.5%
can be attributed to two factors: our efficient implementation increases our steps per second, and we require significantly fewer steps than pixel-space models.
Multi-resolution training For our main results, we employed a two-phase training strategy: training primarily on oowresolutionn data before switching high resolution in the final epochs. As shown in Table this approach achieves comparable accuracy to models trained solely on high-resolution grids, while requiring only 25% of the GPU hours.
Multi-resolution inference Thanks to its multi-resolution nature, neural operators can be trained on multiple resolutions and applied to data of different resolutions. We can reduce significant inference time by performing most sampling steps at low resolution and upsampling only near the end to finalize figh-rreueecy de- tails. We found that upsampling in the middle of the diffusion process worked poorly, which we attribute the difficulty of preserving GRF properties during upscaling. Hence, we propose bi-level sampling process, ReNoise, that mit- igates the upsampling artifacts by adding noise for improved correction potential. The imple- mentation details are given in Appendix (H.4
12
Daucy Farvasl
Foewad
NS Invene
Bas Rute (60)
Time quiomppe)
(Klampl))
6b
Figure 4: Comparison of the accuracy of FunDPS with ReNoise under different ratios of low- resolution inference steps. steps.
We provide the multi-resolution inference results in Figure With 80% of steps performed at low resolution, ReNoise sustains similar accuracy, yielding a further 2x speed improvement to 7. 7.5s/samlle-3 25 times faster than DiffusionPDE.
5 Conclusion
We introduce novel peteretizzaiionagnnsii generative framework for solving inverse problems in function spaces. Our framework supports sampling from posterior distributions with generalizability across different resolutions. We provide theoretical foundations for our framework with the extension of Tweedie's formulation to function spaces. We verified our approach with various settings and PDEs, achieving 32% higher accuracy than baselines while reducing time by an order of magnitude.
Limitations Incorporating PDE loss in FunDPS yields only marginal improvements over pure observation-bant based guidance. We attribute this partially to numerical errors introduced by finite difference approximations of PDE operators. Second, our approach still requires manual tuning of guidance weights for different problems. Future work could explore improved techniques to better preserve the continuous nature of the PDEs and include an adaptive guidance scheme.
Outlook While we focused on sparse spatial observations, our framework could naturally extend to temporal observations in dume-dependent PDEs, enabling spatiotemporal evolution from limited measurements. Second, function-space diffusion models could serve as a unifying methodology for diverse physical systems-enabling foundation models that can trained once on multiple PDE and domain types and then adapted to various downstream tasks. Lastly, while FunDPS demonstrates promise for general PDE-based inverse problems, its performance on specialized inverse tasks such as MRI reconstruction
663 and full waveform inversion [64] remains to be seen.
10

---

Acknowledgments
The authors want thank Christopher Beckham for his efficient implementation of diffusion operators. Our thanks also extend to Jiahe Huang for her discussion on DiffusionPDE, and Yizhou Zhang for his assistance with baselines. Anima Anandkumar is supported in part by Bren endowed chair, ONR (MURI grant N00014-23-1-2654), and the A12050 senior fellow program at Schmidt Sciences.
References
[1] Giannis Daras, Hyungjin Chung, Chich-Hsin Lai, Yuki Mitsufuji, Jong Chul Ye, Peyman Milanfar, Alexandros G Dimakis, and Mauricio Delbracio. A survey on diffusion models for inverse problems. arXiv preprint arXiv:2410.00083, 2024.
[2] Mauricio Delbracio and Peyman Milanfar. Inversion by direct iteration: An alternative to denoising diffusion for image restoration. arXiv preprint arXiv:2303. /435, 2023.
[3] Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. Come-closer-diffuse-faattt Accelerating conditional diffusion models for inverse problems through stochastic contraction. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition, pages 12413-12422, 2022.
[4] Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverss-Guudee Diffusion Models for Inverse Problems. In International Conference Learning Representations, 2023.
URL
[5] Yuanzhi Zhu, Kai Zhang. Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-am and-play image restoration. In Proceedings of the IEEESCVF Conference on Computer Vision and Pattern Recognition, pages 1219-1229,
2023.
[6] Edward Lorenz. Deterministic nonperiodic flow. Journal of atmospheric sciences, 20(2): 130-141, 1963.
[7] Ilan Price, Alvaro Sanchez-Gonzalez Ferran Alet, Tom R Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, et al. Gencast: Diffusion-based ensemble forecasting for medium-range weather. arXiv preprint arXiv:2312.15796 2023.
[8] Yinhao Zhu and Nicholas Zabaras. Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification. Journal of Computational Physics, 366: 415-447, 2018
[9] Ali Kamali, Mohammad Sarabian, and Kaveh Laksari. Elasticity imaging using physics- informed neural networks: Spatial discovery of clastic modulus and poisson's 's ratio. Acta biomaterialia, 155:400-409, 2023.
[10] Nicholas Geneva and Nicholas Zabaras. Modeling the dynamics of pde systems with physics- constrained deep auto-regressive networks. Journal of Compulational Physics, 403: 109056, 2020.
[11] Omer Ronen, Theo Saarinen, Yan Shuo Tan, James Duncan, and Bin Yu. A mixing time lower bound for simplified version of bart, 2022. https://t//// org/ 22100 2210.0935222.2
[12] Alexandros Beskos, Mark Girolami, Shiwei Lan, Patrick E Farrell, and Andrew M Stuart. Geometric meme for infinite-dimensionaal inverse problems. Journal of Computational Physics, 335:327-351, 2017.
[13] Masoumeh Dashti and Andrew M Stuart. The Bayesian approach inverse problems. Handbook of Uncertainty Quantificatio, pages 311 311-428, 2017.
[14] Jae Hyun Lim, Nikola B Kovachki, Ricardo Baptista, Christopher Beckham, Kamyar Az- izzadenesheli, Jean Kossaifi, Vikram Voleti, Jiaming Song, Karsten Kreis, Jan Kautz, al. Score-based diffusion models in function space. arXiv preprint arXiv:2302.074000 2023.

---

[15] Lorenzo Baldassari, Ali Siahkoohi, Josselin Garnier, Knut Solna, and Maarten V. de Hoop. Conditional score-based diffusion models for bayesian inference in infinite dimensions, 2023.
http::///xxvvvv......o/ 2305. 19147
[16] Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35: 26565-26577, 2022.
[17] Jiahe Huang, Guandao Yang, Zichen Wang, and Jeong Joon Park. Diffussinnde: Generative pde-solving under partial observation. arXiv preprint arXiv:2406.17763, 2024.
[18] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. innfree- neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physies, 378:686- 2019.
[19] Zhongkai Hao, Jiachen Yao, Chang Su, Hang Su, Ziao Wang, Fanzhi Lu, Zeyu Xia, Yichi Zhang, Songming Liu, Lu Lu, and Jun Zhu. Pinnacle: A comprehensive benchmark of physics-ifformed neural networks for solving pdes, 2023. URL httthsts//aaiiii wwwp//iiioror/eeee aws/aaa///aa 08827
informed
[20] Songming Liu, Chang Su, Jiachen Yao, Zhongkai Hao, Hang Su, Youjia Wu, and Jun Zhu. Preconditioning for physics-informen neural networks, 2024. URL https://arxii org/
abs/ 0bs/2400.om 2402.00531
[21] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differen- tial equations, arXiv preprint arXiv:2010.08895, 2020.
[22] Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218 2-111822222220 2021.
[23] Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar, Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1-97, 2023,
[24] Johannes Brandstetter, Daniel Worrall, and Max Welling. Message passing neural pde solvers. arXiv preprint 22Xx:22002.03376 2022.
[25] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W Battaglia. Learning mesh-based simulation with graph networks. arXiv preprint arXiv:2010.03409, 2020.
[26] Qian Zhang, Dmitry Krotov, and George Em Karniadakis. Operator learning for reconstruct- ing flow fields from sparse measurements: an energy transformer approach. arXiv preprint (rXiv:2501.08333 2025.
[27] Yaozhong Shi, Grigorios Lavrentiadis, Domniki Asimaki, Zachary E Ross, and Kamyar Aziz- zadenesheli. Broadband ground-motion synthesis via generative adversarial neural operators: Development and validation. Bulletin of the Seismological Society of America, 1 114(4):1151- 2171, 2024.
[28] Naveen Gupta, Medha Sawhney, Arka Daw, Youzuo Lin, and Anuj Karpatne. A unified frame- work for forward and inverse problems in subsurface imaging using latent space translations. arXiv preprint 4rXiv:2410.112477 2024.
[29] Matteo Ravasi. Geophysical inverse problems with geeasreement-uuider diffusion models. arXiv preprint MxXi::2501.0488), 2025.
[30] Christian Jacobsen, Yilin Zhuang, and Karthik Duraisamy. Cocogen: Physically-consittr and conditioned score-based generative models for forward and inverse problems. arXiv preprint ar2iv:2312. 10527, 2023.
[31] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
12

---

[32] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances Neural Information Processing Systems, 34, 2021.
[33] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[34] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
[35] Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661-16) 2011.
[36] Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications. 12(3):313-326 1982.
[37] Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. arXiv preprint arXiv:2201.11793, 2022.
[38] Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 35:25683-25666, 2022.
[39] Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing. arXiv preprint arXiv:2407.0152), 2024.
[40] Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc I. Klasky, and Jong Chul Ye. Diffu- sion posterior sampling for general noisy inverse problems. arXiv preprint 2XXi::2209.14687 2022.
[41] Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective on solving inverse problems with diffusion models. arXiv preprint arXiv:2305.04351, 2023.
[42] Abbas Mammadov, Hyungjin Chung, and Jong Chul Ye, Amortized posterior sampling with diffusion prior distillation. arXiv preprint 1rXiv:2407.17907, 2024.
[43] Jakiw Pidstrigach, Youssef Marzouk, Sebastian Reich, and Sven Wang. Infinite-dimensional diffusion models. arXiv preprint 2rXiv:2302.101.00 2023.
[44] Md Ashiqur Rahman, Manuel A Florez, Anima Anandkumar, Zachary E Ross, and Kamyar Azizzadenesheli. Generative adversarial neural operators. arXiv preprint 22X 2022.
[45] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv.2210.02747, 2022.
[46] Gavin Kerrigan, Giosue Migliorini, and Padhraic Smyth. Functional flow matching. arXiv preprint 22X10 2023.
[47] Yaozhong Shi, Angela F Gao, Zachary E Ross, and Kamyar Azizzadenesheli. Universal functional regression with neural operator flows. arXiv preprint arXiv:2404.02986 2024.
[48] Yaozhong Shi, Zachary E Ross, Domniki Asimaki, and Kamyar Azizzadenesshlli Stochastic process learning via operator flow matching. arXiv preprint arXiv:2501.04126,000 2025.
[49] Gavin Kerrigan, Giosué Migliorini, and Padhraic Smyth. Dynamic conditional optimal transport through simulation-free flows. Advances in Neural Information Processing Systems, 37, 2024.
[50] Vladimir Igorevich Bogachev. Gaussian measures. American Mathematical Society, 1998.
[51] Gavin Kerrigan, Justin Ley, and Padhraic Smyth. Diffusion generative models in infinite dimensions. arXiv preprint arXiv:2212.00888 2022.
[52] Md Ashiqur Rahman, Zachary E Ross, and Kamyar Avizzadenesheli, U-no: U-shaped neural operators. arXiv preprint arXhv:2204.11127, 2022,
13

---

[53] Gabriel Cardoso, Yazid Janati EI Idrissi, Sylvain Le Corff, and Eric Moulines. Monte carlo guided diffusion for bayesian linear inverse problems, 2023. /ttps:////a
2308 07983.
[54] Herbert E Robbins. An empirical bayes approach to statistics. In Breakthroughs in Statistics: Foundations and basic theory, pages 388-994. Springer, 1992.
[55] Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alexandros G. Dimakis, and Sanjay Shakkottai. Solving linear inverse problems provably via posterior sampling with latent diffusion models, 2023. URL https:////aa http://awww....aaa//a/aaaaaaaaaa
[56] Yan Yang, Angela F Gao, Jorge C Castellanos, Zachary E Ross, Kamyar Azizzadenesheli, and Robert W Clayton. Seismic wave propagation and inversion with neural operators. The Seismic Record, (3)::26-113) 2021.
[57] Jiawei Zhao, Robert Joseph George, Yifei Zhang, Zongyi Li, and Anima Anandkumar. Incre- mental Fourier neural operator. arXiv preprint arXiv:221 1./5188, 2022.
[58] Samuel Lanthaler, Andrew M Stuart, and Margaret Trautner. Discretization error of Fourier neural operators. arXiv preprint arXiv:2405.02221, 2024.
[59] Sander Dieleman. Diffusion is spectral autoregression, 2024. URL htttps://sandee aii http:////2//speccraaattttttr////
html
[60] Bowen Song, Soo Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen. Solving inverse problems with latent diffusion models via hard data consistency. In The Twelfth International Conference on Learning Representations, 2024. URL https://openrevieg
[61]
ChristppherBBechhan., Beckhan.. hagpth/.....eaaaaaapeeoooo/rroop
2024. URL https
[62] Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. Physics-informed neural operator for learning partial differential equations. ACM/JMS Journal of Data Science, 1(3):1-27, 2024.
[63] Jure Zhontar, Florian Knoll, Anuroop Sriram, Tullie Murrell, Zhengnan Huang, Matthew J. Muckley, Aaron Defazio, Ruben Stern, Patricia Johnson, Mary Bruno, Marc Parente, Krzysztof J, Geras, Joe Katsnelson, Hersh Chandarana, Zizhao Zhang, Michal Drozdzal, Adriana Romero, Michael Rabbat, Pascal Vincent, Nafissa Yakubova, James Pinkerton, Duo Wang, Erich Owens, C. Lawrence Zitnick, Michael P. Recht, Daniel K. Sodickson, and Yvonne Lui. fastMRI: An open dataset and benchmarks for accelerated MRI, 2018.
[64] Chengyuan Deng, Shihang Feng, Hanchen Wang, Xitong Zhang, Peng Jin, Yinan Feng, Qili Zeng. Yinpeng Chen, and Youzuo Lin. Openfwi: Large-scale multi-structural benchmark datasets for seismic full waveform inversion, 2023. URL https:t/arxiii..oo org www.oorgg////iiooooooeee ab1/2111 02926
[65] Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint arXiv:22/2 00490, 2022.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The Abstract and $1 Introduction explicitly list our main contributions in terms of theory (the extension of Tweedie's formula to anfrite-dimmensioaal Banach spaces), algorithm (the FunDPS framework for function-spaet -ppetinoonncceeeeeeeeeeeee diffusion posterior sampling), and results (demonstrated through many experiments and ablation studies). These contributions are well-aligned with the paper's scope solving inverse problems in anfnite-dimensiona spaces.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: $6 Conclusion (Limitations) describes the need to tune guidance weights and open extensions to temporal data.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specifiiattin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
15

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: All assumptions are stated with each theorem. and full proofs appear in Appendix B; the main text gives proof sketches for intuition.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: $4 Experiments and Appendix E,F.G detail data splits, hyper- syper-parammters, training and inference details; code and scripts are also provided in the supplementary, with exact commands to reproduce the results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
16

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: Codes are provided in the supplementary zip file. Datasets have been taken from DiffusionPDE's work, which is openly available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https SonddeSubmisssooPPoon
/ wttppssccpppbllc/////coc
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the necessary setup details have been provided throughout $4 Experiments and Appendix F.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Error bars were not included primarily due to the extensive scope of experi- ments (30+ setups across five large PDE datasets) and limited computational resources. The main claims are supported by substantial and consistent performance improvements (e.g..
17

---

32% average accuracy gain, 4x fewer sampling steps) demonstrated on five test sets with 1000 samples each.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Appendix details the complete set of hardware, total training time and per-sample inference time.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: /ttper///////i////e www.rrissccppubpl/eee hctppublic/thhicccuuueeeell.eee.
Answer: [Yes]
Justification:
The work adheres to all points of the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
18

---

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss possible positive and negative impacts in Appendix K. This paper presents work whose goal is to advance the field of machine learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The released models and code are for specialized scientific applications involving PDE-based simulations with non-sensitive, synthetic data. They present a very low risk for misuse.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
19

---

Answer: [Yes]
Justification: All used assets, including data, code, and models, are properly cited, credited, and explicitly mentioned throughout the paper. Although the explicit license terms for each of these assets are not reiterated in the manuscript, they originate from publicly accessible sources that are governed by standard open-source licenses (MIT for NeuralOperator and EDM-FS; CC-BY-SA 4.0 for DiffusionPDE).
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pwwpssppppeeseeeeeeeeeseepp thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new datasets are introduced; only code and trained weights are released.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codd/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The work does not involve any research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
20

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The work does not involve any research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Large language models were not part of the core methodology.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy htttss//iorriiss.cc/Cfffooorrrre for what should or should be described.
21

---

A Diffusion Models in Function Spaces
A.1 Forward Diffusion with Gaussian Random Fields
dimenie-dimessonnall data (such as functions defined on continuous domain) require careful definition of the diffusion process on them. We adopt a meesure-theoretee approach following recent score-based generative models in function spaces [14]. Let H be a separable Hilbert space of functions (the function space of interest), and let μ be the data distribution on H (a probability measure over H for our training data). In infinite dimensions, there no Lebesgue density, so we work relative to a reference Gaussian measure. We introduce a centered Gaussian prior measure (O, C) on H, with covariance operator C chosen be self-adjoint, non-negative, and trace-class (so that N C) is well-defined GRF measure). We refer to (0, C) as the GRF prior measure.
Noising process
Given a sample μ., we perturb it by adding independent Gaussian random function (drawn from the GRF prior) with appropriate variance. In other words, for a noise scale 0, we define the noisy function level a
Gg a + 7
(15)
where 7) (0, o² C²) is a Gaussian random element in H with covariance o²C. As increases, more Gaussian noise is added to the function. For a = O, we have 0g = a (no noise), and at large a, ag is dominated by noise (In fact as a + 00, d, approaches a draw from the zero-mean GRF prior). This construction defines the forward diffusion process in function space in a distributional sense: it transforms the data distribution μ into a family of perturbed distributions Maa where Pa is the law of 0, = + 17. Equivalently, Ho is the convolution of μu with the Gaussian measure NO, o CCC). By varying a from to some large value, we obtain a continuum (or a discrete set) of distributions bridging No = 1a and an almost pure noise distribution (when ø is high). This is analogous to the forward noising process in standard diffusion models, but defined on an infinte-diimensioall function space via GRFs.
A.2 Score Function and Denoising Objective
With the forward process defined, we now consider the score function on the function space. At given noise level a, the score is defined as the gradient of the log-density of the perturbed distribution Ma. In our infinie-dimeensioaal setting, this gradient is understood with respect to the Gaussian reference measure (the GRF prior). Formally, for each Ø, we define the score function s(a, σ) as the H-valued gradient of log Pa (a). where Po is the density of Pa relative to (O oC Intuitively, s(a, o) points in the direction H that increases the likelihood of a under Pa the most. In finite dimensions, this recovers V. log Po (a); here we assume s(a, r) exists as an element of H [14)
Denoising score matching In practice, s(a, o) is unknown because the true data distribution μ (and hence (a)) is unknown. Instead of trying to directly estimate the score, we train a denoising model Do(a,, or) to recover the underlying clean function a from a noisy sample 0g. This is called denoising score matching, which is particularly convenient in function spaces. Following the EDM [6] training strategy, sample pairs of clean and noisy functions and train Da to predict the clean input. In particular, given a sample a ~ j and its noisy version y = a + with ~ - NN, OO)), we train Da to output (the ground truth) when given (y, a) as input. The training objective, averaged over the data distribution and noise, is formulated as a weighted mean squared error:
L(O)= E Bawa, (0, X(a Da + 7/, σ) a 11
(16)
where X(o) is a positive weighting function that balances the loss contributions across different noise levels. Lim et al. [14] shows that minimizing this denoising objective for all a is equivalent to learning the true score function on H. In fact, there is an explicit relationship between the optimal denoiser and the score operator in function space, analogous to Tweedie's formula. Given a noisy observation y = a + 17 at scale o, the score can be written as:
((y = Do( -
(17)
Thus, by training the model to minimize T.(0) across many noise levels, we are effectively teaching to approximate the score operator in H. We implement the above training with discrete noise level
22

---

schedule and random sampling of noise intensities, similar to the EDM methodology (Algorithm D We randomly sample a noise levels log- ungguonfl over [Tmin. maxx as in Karras et al. [[6]. As result, Dg becomes function-space denoiser that can gradually refine noisy input at any noise scale within the range.
Algorithm 1 FunDPS Training (Training unconditional diffusion model in function spaces)
Require: Data distribution μu, GRF prior covariance C, noise-level distribution p(o)
1: Initialize model parameters 0
2: repeat
3: a ~ μ, ~ p(o)
[Draw clean function and noise level)
4: 7~ 7~~ii N(0,
[Sample GRF noise [Construct noisy sample) { [Compute denoised prediction] (Compute training loss)
5:
y + a+7
6:
at Doly.o)
7:
L M(a)ll - a
8:
Update parameters 0 by minimizing L
9:
until converged
10:
return Dg
A.3 Reverse Diffusion and Sampling
Once the denoising model (score model) is trained, we can generate new function samples from the learned distribution by running the diffusion process in reverse starting from noise and iteratively removing noise. The key idea is to start with an initial random field drawn from the prior and then repeatedly apply the denoiser D3 while decreasing in stages. Here we adopt a discrete reverse diffusion approach aligned with EDM's deterministic solver. First, we choose high noise level maax (e.g. the upper bound used in training) and sample an initial function av ~ (O(, o C), i.c. a pure noise sample in H drawn from the GRF prior. Then we define decreasing sequence of noise levels max ON > ON-1 adjacent levels have small differences in terms of signal-to-noise ratio as in EDM [66.. The noise scheduler spans from the highest noise to zero (5 mmax corresponds to the prior and o = 0 corresponds to a clean sample). At each step N, N 1, 1,..., 1, given the current noisy sample a, at noise level og we apply the denoiser get à Da o;), the model's s estimate of the clean underlying function. followed by adding the a;-1 noise level to reach the next step's sample a;-1 along with higher-order updates. After iterating down to the final level dp - 0. we obtain dg. which is an approximate sample from the original data distribution μ (since no noise remains). We can then use ao newly generated function sample drawn from the learned generative model. We implement our framework using a deterministic sampler based on Euler's 2nd order method. Algorithm 2 without the FunDPS guidance corresponds to this unconditional sampling procedure.
>
do R O, where the levels are spaced polynomially so that
B Pseudocode for FunDPS
Algorithm 2 FunDPS Sampler
Require: Observation u, forward operator A, denoising diffusion operator D,, variance schedule (o)) with a (lo) 0, guidance weights Ç.
1: aN NO0 C) (Initialize a from GRF)
2: for = N to 1 do
3: âg + Do(a,,a(tt))
(Estimate ag by Tweedie's formula) {Evaluate da dt at t; (Take Euler step from a (1;)) (1;-1))
4: d, + (a, ao)/o(t+)
5:
a.1 t a; + (a((())))) (l:))d
6:
if (4:44-)) t
7:
a + o
8:
d t (a:-1 (t-)
9:
a-- a; + (a(4-1) - (l.)) (d))
(Apply 2nd-order correction)
10: end if
11: 0.-1 -; S. Va, Iu A(ab))
(Invoke the FunDPS guidance in Eq. (14
12:
end for
13: return ag
23

---

C Measure-Theoretic Tweedie's Formula
In this section, prove generalization of Tweedie's formula which holds for separable Banach spaces. This will furnish us with link between score functions [14] and conditional expectations, which is key step in enabling function-space inverse problem solvers. We begin with some preliminaries on Gaussian measures before proceeding with our proof.
Notation Throughout this section, B will represent a separable Banach space. For two measures J,, v on B, we write μ < if y absolutely continuous with respect to v, i.e., if E BB) Borel are equívalent if v < and & For any h E B, we will write 7, + B for the translation map I + I + h. For an arbitrary Borel measure μ B we will write μ (Th) p(- h) for the translated measure.
measurable and v E)) = 0, then ()) = 0. We write
C.1 Banach Space Gaussian Measures
We briefly review the key definitions necessary for our constructions. We refer to (50) for an in-depth treatment of this material. A centered Gaussian measure y on B is a Borel probability measure such that the pushforward of along any bounded linear functional f E B* is mean-zero Gaussian distribution on R. Note that, as separable, Fernique's theorem guarantees that we have B C (7). The reproducing kernel Hilbert space (RKHS) B,, associated with y is the closure of B" with respect to the (1) norm, i.e.,
B = [ (
(18)
and for f.g E B their inner product is (f.9) B: = f(z)g(z)
The measure is uniquely determined by its covariance operator Cy B + B defined by
C2( / ((x))(2)d((()))
v B B E B"
(19)
The Cameron-Martin space H(+) C is defined as ((>) = (B,) (C, BT).
(20)
Although (n)) C 8 in general, because B is separable Banach space we may take H(?) C B.
That is, we write (f) for some B and f E B if f(g) = (f)(()) for all g B". Since
every h, k E H1) are of the form h C,,,),, k (k(k) for some h k E By the space H()) has an induced inner product
(h, (h, = (h), C. (,)( = (h,
(21)
The space (() is Hilbert space under this inner product. We will write [hl (6)) for the associated norm. Note that we may equivalently define
[M|{() = sup (J(h) / B", C,())) 1)
(22)
where the
space H()) may be identified as [50] Theorem 3.2.3]
H(>) = (h E X |||HG) < x.).
(23)
Note that the map C, B H()) is an isometric isomorphism. Since H is Hilbert space, the Riesz representation theorem furnishes with canonical isometry R H(x)* H(y). Thus H(y)" ~ B,, Somewhat more explicitly, we obtain an isometric isomorphism
J B - H
(C,(0)
(24)
Note further that (R J)() Cf( for e B gives us an isometry Ro B H()).
The following shows that Cameron-Marte space H()) is precisely those directions under which we may translate the measure y while remaining absolutely continuous (50 Corollary 2.4.3, Theorem
3.2.3]. Moreover, the celebrated Cameron-Martin formula allows us explicitly calculate the associated density.
24

---

Theorem C.1 "Cameron-Marting On separable Banach space B, an element h E H()) is an element of the Cameron-Marinn space if and only 7f 7 are equivalent in the sense of being mutually absolutely continuous. In this case,
dn (x) =exp -(16(r))))()())))))) (x) dr
C.2
Tweedie's Formula
We now proceed to give proof of a generalized Tweedie's formula for centered Gaussian measures on Banach spaces. Let X ~ be a random variable on B with distribution μ and let ~ be distributed according an independent centered Gaussian measure with covariance Cy. Define new random variable
Y is obtained by the convolution of these two measures, i.e.,
X + Z
(25)
whose distribution -= j
v(E) / 7(E - 2) du(x)
VE E B(B).
(26)
Observe that conditioned fixed value of X z, have Y X according to a Gaussian measure 7 with mean x and covariance Cy-
7tth is distributed
We begin by showing that v ~ ry are equivalent when je is supported on the CM space (7)). This is generalization of [14] Theorem I]. who show an analogous claim for separable Hilbert spaces. The proof is essentially the same in both cases, but we include it here for the sake of completeness.
Proposition C.2. Suppose (TH( = 1. Then, v/ ~ 7 are equivalent.
Proof. Let E E B(B) be an arbitrary Borel set. Suppose that >(E) = 0. By Theorem C.I for y-almost every æ. and hence 7= (E) 0 for yralmost every It follows that
7
(EE) = / (F) 0.
Conversely, suppose v(E)= = 0. follows that 7f for y-almost every x. Since ((H(?)) Theorem C.1 shows the measures 7" and 7 are almost surely equivalent and thus ((F) 0.
Hence, if p(U(())) 1, the Radon-Nikodym theorem provides us with Borel measurable d IR with
dv (y) exp(o())) dr
7-a.c. E B.
We henceforth assume this is the case. Assume further that & is Fréchet differentiable along H (>). The score of y is defined as
Duc,0: B - H())*
That is, O = DHT()) log dr
de the logarithmic derivative of the density along H The value (z) is the derivative of & at in the direction h E H()). We refer to 14 for further discussion of this notion of score and the differentiability assumption.
In the following lemma, we prove that the density of v with respect to the noise measure y can also be understood in terms of the corresponding conditional distributions y This lemma will be used to aid our later calculations.
Lemma C.3. Assume that ((H(?)) For 7-almost every y E we have
dv
/ d (v) dp(x) Ennw
dr
(v)
(v)
(27)
d
dy
d7
Moreover, for y-almost every B,
dy (g)
dv du
(y)
(28)
dy
d7
25

---

Proof. Fix a measurable E E (B). Since (7)) - 1, then p-almost surely we have that y" < Hence, by the Radon-Nikodyn theorem. the density (dr" d7) exists p-almost everywhere.
Now, recall that μl * y a convolution of measures, so that
J <(E 2) du(x)
v())
(29)
B
| x2 (F) du(x)
(30)
d (y) (y dp(x)
(31)
J dy
dy E dr
(v) du(z))d(())
(32)
where the last equality follows by Tonelli's 's theorem and the fact that the densities are nonnegative. By Proposition C.2 v < yy and so by the Radon -addon- theorem the density dv/ dr is uniquely defined up to a set of m-masur zero. Thus,
du (v) dr (v)= - -/
d7 dr((0)
(v) du(x)
a-a.c. y E B
(33)
dr
as claimed. The second claim follows because y under the assumption ((H())) 1.
We now proceed to calculate E|X|Y= = w]- First, we directly calculate this using the definition of conditional expectation. Note that this proof follows closely calculation shown in 43 Appendix F.I.
Proposition C.4. Suppose that ((H( = 1. Then, for v-almost every E B, the conditional expectation is given by
dy dy (1)
dr x
( dy
(2)
E X
Y
y|
(34)
Proof. Write fly) for the right-hand side of (34) and let E oYY) be a --measurable event. We show Ey ( = Ez-y [LAz], from which the claim follows. Indeed,
J [(y)du())
/ 1B
(())d" (v)du(z)
(35)
dy
d.e
x
(V) dr (w) du(3))
(36)
du
dy
B
dxt
dr (v) dr du du(x)
(37)
dy
dy
(v)
(v) du(a) dr(v)
(38)
dy
de
(v)
dr (v) dr(x) du(x)
(39)
dy
B
d7
x
(1) dr(w) dp(x)
(40)
dy
B AAA].
(41)
which completes the proof.
We now proceed to calculate the Cameron-Martes space gradient of the score. In particular, we show that it is equal to the same expression obtained in Proposition C.4 This requires assumption on the measure j to ensure that the derivatives of dy / dr are bounded by an integrable function in order justify a derivative-integral exchange. Using the continuity of C.,, the condition in Eq. 443 can be relaxed to finding integrable v/ L (*)) such that for all y E B and p-almost every 2,
((| exp (IC, |Bg |ar (ll lvle - (lm(r()) S (x)x).
(42)
26

---

While this condition depends on the specific choice of μu, it will be satisfied when e.g. is compactly supported in H (7) or when μ has tails which decay sufficiently fast.
Proposition C.5. Assume (()( = and that log du is Fréchet differentiable along H(?). Consider the score Di(()) log de dy B + H(y)", Let R H(y)" + H()) be the Riesz isometry.
dy
Assume further that there exists a non-negative function v/ E L*(4) such that for all E B and y-almost every B,
dh Duti dy [(())*
(v)
S (xx).
(43)
Then, the Cameron-Martin score
R Dnt log dr ) B + H())
de
(44)
given for v-almost every y B as
dv (v)
d7 dy du
[ dr dy
R DDC log
(v)lBem
x
(1)
(45)
dy
Proof. Since we assume the score of v is Fréchet differentiable, we may apply the chain rule to see that for E B,
DH()) log de (v) du dr
dy (v)
de dy (v).
(46)
Morcover, by the Cameron-Martin formula, if x E H()), then
DEC log dy (v) J(C)
(47)
d-
where J : By + H" is the isomorphism defined in Eq. 24). Note this isomorphism is required as C.yy (x)) B"y is an element of the RKHS By, whereas the Fréchet derivative is an element of H())*. Using Lemma C.3 and the assumption that there exists dominating function & E L (14), we may use the Leibniz integral rule to calculate that
du DHwm dr (v) = [ ((),
dy (v) dr
(48)
in the sense that
dv dy
(w) J J (c, (h)
d (y) dx)))
4h E H1))
(49)
d7
Using the fact that R is bounded and Ro J = Oy, we obtain
du (n(()) R Da
/
dh
x d7
(w) dr())
(50)
B
1 dr" d7
Bonyy
(v)
(51)
Combined with Eq. (46). this yields the claim.
Combining Proposition C.4 and Proposition C.5 yields the Banach space generalization of Tweedie's formula, which concludes the proof for Theorem 3.1
C.3 Special Case: Euclidean Setting
Here, we will informally replicate our proof of Tweedie's 's formula in the special case of = R²" to provide some intuition and to sanity check this result. In this case suppose everything admits densities, so that v = py (y), Px (z), and 7 = P2(2) = A (2 0, C). Moreover Moreoverrpy p 2) = Ny xr, C). While many of these steps in this section can be done a more straightforward fashion when B= = IR", purposefully follow the structure of our previous calculations.
27

---

In this setting, we seek to compute
dv dy
py (3)
log (1)
= V. log
(52)
pz (y)
the score is now taken with respect the noise measure pz (y). Now, using the densities, we may calculate in analogous fashion
Py (1)
Pz(U) V.
V. log
(y (x) dx
(53)
(v)
(z
JRn
PY (V) /
pz(y)
(y xx)
py (y xt)
V
Volog log
p(x) dæ
(54)
RR
P2
Pz (v)
pz(u)
py (y x))
(c- - y) + C (x))
pz (v) p(x) dz
(55)
(V) RD
pz(y)
(z) py x)) pz(v) p(x) dr.
(56)
PY (y)
This expression is the finite-dimensional analogue of the one we obtain in Eq.
48) This yields
(v) pz(() / pr(ul I x) p(x) dz. pz()) py( pz(1)
CVv log
(57)
On the other hand, we may explicitly calculate the conditional expectation by
E Y = vl
xxxiv 1)) dz R
(58)
SE XPYIX ())x(x)
(59)
PY PV)
pz(U)
/ Pr
pz(u) px (x) da
(60)
py
R
pv(v)
CV, y log Pz
(61)
Let us take a step further towards the standard expression of Tweedie's formula. The gradient (y) V, y log pxly) can be expanded as V, log PY (v) V, y log pz (y). For (2) = N( 0, C), we have V, loggz(y) -C-ly. Substituting, this becomes the more familiar expression
E[X Y vl y CV, y log log (y).
(62)
28

---

D Inverse PDE Solver with an FDM Forward Operator
In order to show the diverse adaptability of our methodology, we further apply our framework to solve inverse PDE problems by using Finite Difference Method (FDM)-based forward operator. In this problem, we only model the prior distribution on the coefficient space using the function-space diffusion model. We then reconstruct the initial states from full, noisy, and sparse observations of the solution space. Specifically, the forward operator here is
A(a) FDM_Solve( VVVaa 1),
(63)
and we formulate the inverse problem as estimating initial state a from corrupted observations of solution state Our goal is sample from p(a|ut by applying the function-space reverse diffusion steps with guidance term V AallA(a) u/l. Since A is standard FDM solver, we can compute gradients via automatic differentiation. We present qualitative examples Figure and show in Table 2 that our method can achieve small relative L'-errors in these challenging cases with both FDM torward operator and joint learning. In practice, considering similar or better performance. we use joint learning approach for all our experiments as the inference speed is considerably faster, since the FDM-based method requires backpropagating through the linear solve FDM_Solve.
Ground
Observed Solution
FDM Method
Joint Learning
Truth
Masked
Noisy
Pure Solution
Figure 5: Reconstruction of coefficient functions from partially-observed Darcy Flow problems.
Table 2: The reconstruction on the inverse Darcy Flow problem with diverse settings.
Corruption
Steps (N) Time
None
Noisy Masked
FDM
50
57s
4.28% 4.67%
4.47% 3.64% 6.32% 3.95%
Joint Learning
500
15s
E Detailed Dataset Description
Darcy Flow Darcy Flow is fundamental model that describes the flow of viscous incompressible fluid through porous medium. The governing equations are given by:
(u<)) = f(x),
I E (0, 1)3
(64)
with constant forcing ((x) and zero boundary conditions. We follow the strategies in Li et al. 21 to generate coefficient functions a ~ A0, (-A + 91) where h: -> is set be 12
for positive numbers and 3 otherwise.
29

---

Poisson Equation
The Poisson equation describes steady state diffusion processes:
Viu(z) = a(x), x E (0,
(65)
with homogeneous Dirichlet boundary conditions u lan = O. We generate coefficient fields a(x) from Gaussian random fields (0, (-À -))- The PDE guidance function is V a.
Helmholtz Equation
The Helmholtz equation models wave propagation heterogeneous media: Vu((x) + k2 u(x)) = a(x), * E (0,1)²
(66)
with i and Dirichlet boundary conditions ul aa = Coefficient fields a(x) are GRFs generated as in 17. The PDE guidance function is f = 2 + kZ - a.
Ntvier-Stokes Equations We further evaluate the performance on the Navier-Stoker equations by generating its initial and terminal states as in [21. In particular, we consider the evolution of velocity field u(x t) over time given by
(1, () u(x () (w(x, 1) = vAn (z, f(x), x (0, L (0,7),
(67)
V u(x, = 0, x (0, tE [0,7]. (2x 0) = a()), x E (0,
(68)
(69)
where w X u. is the vorticity; V T000 viscosity; and J, fixed forcing term. The initial
condition (z)) is sampled from ((0, ( A + 491) -6/2). The forcing term is defined as f(z) =
TO (sin(2i (x1 + x2)) c2s(2 + x)))) We simulate the PDE for T using a pseudo-spectral
method. It should be noted that the PDE guidance formulated in Huang et al. 17 is invalid:
ww= (V x u) =
(70)
Furthermore, due to the lack of information, calculating PDE loss is non-trivial here. While the experiments in this work use the original incorrect formulation for consistency with prior benchmarks, we anticipate minimal impact on the final results given the relatively small loss weight APDE-
Navier-Stokes Equations with Boundary Conditions (BCs) cylindrical obstacles, governed by:
We study bounded flow around
0,0(I,) t) + v(x,t). Vo(a, = -VVp+-V+un V v(z 00 BENtt (0,7),
x R
(71)
(72)
with // = 0.001, 1.0, and no-slip boundaries on Wie The domain contains randomly placed cylinders. We learn the joint distribution of un and Uy T 4. its original PDE guidance has the same етror as the non-bounded case.
F Detailed Experiment Setup
Datasets We validate our approach by solving both forward and inverse problems on five different PDE problems. These PDEs include Darcy Flow, Poisson, Helmholtz, and Navier-Stokes with and without boundary conditions. We follow the same strategy as in DiffusionPDE [17 to generate datasets, where we prepare 50, 000 training samples and 1, 000 test samples for each PDE. The Navier-Stokes equation with boundary conditions specifically consists of 14, 000 train and 000 test samples. The resolution is 128 x 128, and in some settings we downsample the data by 2x. For quantitative comparisons, error rates are calculated using the T,2 relative error between the predicted and true solutions, except for the inverse Darcy Flow problem, where we use the binary error rate.
Implementation We adopt levell U-shaped neural operator architecture [52] as the denoiser Do. which has 54M parameters, similar to DiffusionPDE's network size. The network is trained using 50, 000 training samples for 200 epochs. For the multi-resolution training, we begin training on coarser grid (64 64) for 200 epochs, then switch to higher resolution (128 x 128) for 100 epochs. The hyperparameters we used for training and inference are listed in Table We source the quantitative results of deterministic baselines from DiffusionPDE [17] 's table.
30

---

Table 3: Hyperparameters of Choice.
Hyperparameter
Value
learning rate
0.0001
learning_ pattingg warmup _pr___u
5 million samples
ema_hall life
0.5 million samples
dropout
0.13
ibf_scale
0.05
sigma_mat max
80
sigma min rho
0.002
Speed comparison All the experiments are conducted using a single NVIDIA RTX 4090 GPU, To determine per-sample inference time, we averaged batch inference time 10 runs and divided by the batch size. Batch sizes were optimized to fully utilize GPU memory; specifically, for 128 x 128 data, these were 13 for FunDPS and for DiffusionPDE.
G Implementation Details of the Guidance Mechanism
For inference, we tuned the guidance strength Ç on a small validation set, resulting in the values shown in Ta We noted that PDE loss calculations are unreliable in early sampling stages due to high noise levels. Hence empirically, we only apply PDE loss when o, Furthermore, to ensure smooth convergence the posterior, we found it beneficial to dial down the guidance weights as the noise level decreases. Therefore, we implement a simple but effective scheduling scheme for guidance weights of both observation and PDE loss:
غ { o
if <
a 2
We the Huber loss for PDE guidance instead of mean squared error because it provides robustness against potential outliers caused by finite difference approximation errors, which improves the stability of gradient updates. We investigated the method's sensitivity the guidance strength in Appendix H.6
Table 4: Guidance strength used for each PDE problem.
Darcy Flow
Poisson
Helmholiz
Navier Stokes
Navier-Stokes with BC's
Forward
Inverse
Forward
Forward
Inverse
Forward
Inverse
Forward
Inverse
Observation Loss Type
MSE 10000
MSE
MSE
MSE
MSE
L2
MSE
L2
MSE
L2
Observation Loss Weight
50000
10000
20000
10000
5000
5000
7500
2000
PDE Loss Type
Huber
Huber
Huber
Huber
Huber
Huber
Huber
Huber 10
Huber 100
Huber
PDE Loss Weight
0
0
15
31

---

H Additional Experiments
In this section, we will present additional experiments to demonstrate key aspects of our model. Due to time constraints, primarily focus two representative PDEs: Darcy Flow and Navier-Stokes equation. These systems are significant interest and range from smooth elliptic problems to highly nonlinear dynamics.
H.1 Plug-and-play inverse solvers
In order to show the adaptability of the framework, we further test our method with various guidance methods. Namely, Table | and Figure demonstrate the reconstruction results on Poisson PDE equation with various inverse solvers (40 39 65 on different priors. FunDPS (Function Space + DPS) consistently outperforms other methods within a smaller number of steps.
The sigmificant underperformance DDNM and DAPS in our setting primarily due to the extreme sparsity of observation data. DDNM relies projecting the sample at each step. With only 3% of observation points, the measurement subspace extremely low- dimensional compared to the overall function space. As result, the projection provides very weak guidance and leads to poor reconstruction. The core issue DAPS comes from the localness of guided updates during Langevin dynamics. Only the points with observation updated and others are just added with noise. This makes the intermediate state after Langevin dynamics discontinuous and out-of-distribution, which hinders performance. we increase the number of sampling steps to 20,000, the accuracy can match DPS, but with 100x more time.
measurement subspace
Table 5:
The relative errors for Poisson equation under varying priors and plug-and and-play inverse solvers. FunDPS corresponds to the intersection of Function Space prior with DPS solver. FunDPS results are based on 500 steps, whereas other methods are performed with at least 2000 steps.
Function Space
Euclidean Space
Inverse Solvers
Forward
Inverse
Forward
Inverse
40 DPS
1.99 %
20.47%
4.88%
21.10%
DDNM 65
10.17%
41.67%
20.66%
43.84%
DAPS 39
40.35%
77.72%
492.6%
274.8%
DDNM
DAPS
Space
Sprce
Spoce
Spece
Spece
Problem
Fraur: 2. 10%
Ener: 1.96%
Froor: 8.50%
Froe: 30%
Fror 24 58%
Froor: 369 1%
Problem
23.88%
32.45%
Figure 6: The qualitative results of Table wiith with Poisson equation. First and second rows correspond to the forward problem reconstructions and maps, respectively. The third and fourth rows correspond to the inverse problem reconstructions and error maps, respectively. Relative errors are also reported for this specific data under each error map, where FunDPS achieves the minimum among all the tasks.
32

---

H.2 Fully-observed problems
Deep learning approaches have been extensively researched for solving classical forward and inverse problems, where either the initial state or the final state is fully known. While the deterministic baselines powerful in these tasks, our method still demonstrates superior performance in out of 4 cases, highlighting the strong modeling capabilities of our framework. Ours also outperforms DiffusionPDE in all cases, shown Table
Table 6: Comparison of different methods on forward and inverse problems with full observation. Error rates are calculated using the I relative error between the predicted and true solutions, except for the Darey Flow inverse problem, where binary outperforms the fixed-resolution diffusion baseline and achieves top performance even when compared to deterministic baselines in 3 out of 4 cases. The best results are highlighted in bold.
rate is used. FunDPS significantly
Darcy Flow
Navier-Stokee
Steps (N)
Forward
Inverse
Forward
Inverse
FunDPS (ours)
200
1.1%
4.2%
4.9%
7.8%
FunDPS (ours)
500
1.4%
3.0%
3.0%
7.0%
FunDPS (ours)
2000
0.9%
2.1 %
1.6%
6.6%
DiffusionPDI
2000
2.9%
13.0%
2.4%
8.4%
FNO
5.3%
5.6%
2.3%
6.8%
PINO
4.0%
2.1%
1.1%
6.8%
DeepONet
12.3%
8.4%
25.6%
19.6%
PINN
15.4%
10.1%
27.3%
27.8%
H.3
Multi-resolution training
Our multi-resolution training combines efficient low-resolution learning with high high-resolution finetun- ing. We first train for 200 epochs on 64x64 resolution data to learn coarse features, then train for 100 epochs 128x128 resolution to capture fine details. This approach leverages the resolution independence of neural operators while reducing computational costs.
Table Table compares training configurations across resolutions. Direct high-resolution training performs well but requires substantially more parameters and computing resources. Our multi-resolution approach achieves comparable or better performance while maintaining the smaller model size, reducing total GPU hours by about 25%.
Table 7: Comparison of different training resolution strategies. "Mixed" refers to our two-phase approach, which achieves comparable or better performance to training directly at high resolution while using significantly fewer parameters. We use 500 steps for evaluation.
Darcy Flow
Ntvier-Stokes
Training Res.
Inference Res.
# Params
Forward
Inverse
Forward
Inverse
64
64
54M
3.03%
6.75%
3.20%
8.85%
64
128
54M
3.64%
5.24%
3.81%
8.48%
128
128
184M
2.74%
5.03%
3.35%
8.20%
Mixed
128
54M
2.49%
5.18%
3.32%
8.16%
H.4 Multi-resolution inference
Our multi-resolution sampling pipeline shown in Figure 3p. It includes two stages with noise increase in between. The first stage a complete diffusion sampling at low resolution, from omax (1) min over fup steps. Samples then upscaled (e.g., using bicubic interpolation, though
We again found reproducibility issues with DiffusionPDE. Our reproduced results are here. Please refer to Appendix for details.
33

---

method is not sensitive to this choice). To counter upscaling artifacts and initiate subsequent high- resolution diffusion, noise added at Fmax. Subsequently, a second diffusion sampling process is performed. Empirically, we found that o min the initial low-resolution stage need not be very low, as details refined in the second stage. Furthermore, o for initiating the high-resolution process is typically set to a modest value (e.g-- in the 1-10 range), considerably lower than the initial (1) 80, want to keep the existing information in the low-resolution sample. We provide the nuuttiresolluion inference results in Figure 4 Table g further shows that FunDPS generalizes resolutions, while DiffusionPDE fails transfer and must be retrained for each fixed grid.
(2)
Table Results on Darcy Flow under different mrining-inference resolution pairs. DifiPDE- and DifIPDE-500 represent 2000 and 500 steps for Diffusion PDE, respectively. FunDPS, using 500 steps, achieves superior generalization and maintains accuracy under multi-resolution settings, whereas DiffusionPDE fails to generalize between grids.
Training
Inference
DiftPDE-2000
DinIPDE-500
FunDPS
64
64
6.38%
7.96%
3.03% %
64
128
33.72%
35.57%
3.64%
128
128
6.07%
4.60%
2.74%
Mixed
128
3.83%
4.53%
2.49
H.S Number of observations
We investigate how number of observations affects model performance in both forward and inverse Darey Flow problems. We use 500 steps for evaluation. Figure S shows that performance improves consistently as we increase the number of observed points from TOO (0.6%) to 2000 (12%) of the spatial domain. It is worth noting that method can achieve reasonable accuracy even with extremely sparse observations, demonstrating its effectiveness.
We further analyze the performance FunDPS and DiffusionPDE: under different observation sparsities on both forward and inverse Darcy Flow problems. As shown in Table FunDPS maintains high accuracy even with extremely sparse observations (as low as 0.5% of spatial points), while DiffusionPDE performance degrades significantly under sparse settings.
Table 9: Comparison of FunDPS and DiffusionPDE across varying numbers of observations on Darcy Flow. DifPPE--200 and DifPPD-500 represent 2000 and 500 steps for Diffusion PDE, respectively. FunDPS, using 500 steps, maintains < <10% relative error even with as few as 0.5% observed points.
Forward
Inverse
Obs.
DifIPDE-2000 DiAIDE--50 FunDPS DifiPDE-2000 DifiPDE-500 FunDPS
100
7.2%
13.8%
7.2%
10.7%
16.4%
8.0%
200
6.5%
7.8%
4.8 %
9.1%
10.2%
6.3%
500
6.1%
4.6%
2.9%
7.9%
8.1%
5.2%
2000
2.2%
2.3%
1.7%
3.9%
4.7%
4.1%
H.6
Sensitivity to the guidance strength
The guidance weight is key hyperparameter in framework, manually chosen for each task by tuning on a small validation set. This is common practice in guided diffusion models, as the optimal weight often depends on the forward operator and prior data distribution. We provide list of tuned Ç values in Table 4 In our experience, the model's performance is stable across wide ranges of Ç. However, values that values that are too low result in weak guidance and less accurate reconstructions.
high can cause sampling become unstable and diverge, while
We conducted an ablation study to evaluate performance on Darcy Flow and Poisson equation forward/inverse problems across a range of Ç values, using 500 sampling steps. The results, shown in Table 10 demonstrate expected behavior.
34

---

0.08
Forward Invetse
Relative Error
0.06
0.05
0.04
0.03
0.02
100
200
500
1000
2000
Number of Observatiooss
Figure 7: Comparison of the accuracy of method with respect the number of observations for he forward and inverse Darcy Flow problems.
Table 10: Ablation study on guidance weight for Darcy Flow and Poisson problems. Errors are I relative errors (%). NA indicates divergence due to instability.
Darcy Forward
Darcy Inverse Poisson Forward
Poisson Inverse
1000
6.92%
24.02%
3.31 %
34.38%
5000
2.92%
11.01 1
2.56%
28.05%
10000
2.49%
9.16%
1.99%
19.47%
20000
NA
9.86%
18.86%
20.47%
50000
NA
5.18%
NA
NA
H.7
Additional experiments on Navier-Stokes with boundary conditions
In Table [ present experiments for the Navier-Stokes equations with boundary conditions by using both boundary observations and 1% random interior points. We also compare our methodology against a standard setup in which only 3% of the data revealed for state reconstruction. Numerical results for this comparison can be found in Table
Table 11:
Quantitative results of the Navier-Stokes equations with boundary conditions, where sparse observation consists of 3% of data, the same as in other PDE experiments.
Navier-Stokes with BCs
Steps (N)
Forward
Inverse
DiffusionPDE
2000
9.78%
4.71%
FunDPS (ours)
200
4.62%
3.14%
FunDPS (ours)
500
3.48%
3.07%
H.8 Design Choices of DiffusionPDE. and FunDPS
In order show the effectiveness of our design choice, we include an ablation study on Darcy Flow PDE by adding different components to achieve FunDPS starting from DiffusionPDE. Details are shown in Table 12 and FunDPS's superiority can be observed upon adding the components.
Reproducibility of DiffusionPDE
We replicated DiffusionPDE''s results using their provided code and weights. Due to reproducibility issues, we reran all experiments in communication with DiffusionPDE'ss authors and present the comparison in Table B
35

---

Table 12: Ablation study the design choices of DiffusionPDE and FunDPS, tested forward and inverse Darcy Flow problems. Functional diffusion noise significantly boosts capability. FunDPS achieves superior performance with fewer steps by using functional noise and neural operators.
Functional Noise?
Neural Operator?
Darcy Flow
Steps (N)
Forward
Inverse
DiffusionPDE FunDPS w/o NO FunDPS
X
x
2000
6.07%
14.50%
/
X
500
3.59%
7.77%
/
500
2.49%
5.18%
Table 13: Comparison between reported results in the DiffusionPDE paper and our reproduced results using the official code and weights.
Darey Flow
Poisson
Helmholtz
Navier-Stnkes
Navier- (BCs)
Forward
Inverse
Forward
Inverse
Forward
Inverse
Forward
Inverse
Forward
Inverse
DiffusionPPD (reported)
2.5%
3.2%
4.5%
20.0%
8.8%
22.6%
6.9%
10.4%
3.9%
2.7%
DiffusionPDE (
6.07%
7,87%
4.88%
21.109
12.64%
19.07%
3.78%
9.63%
9.69%
4.18%
J Impact Statement
This research introduces FunDPS, novel framework that significantly advances the solution of PDE-based inverse problems in function spaces. The main positive societal impact is the acceleration of scientific and engineering research by enabling more accurate and efficient modeling from sparse, noisy data in many areas.
Our released models and code are specialized for these scientific applications and are trained on simulated, non-sensitive data. Thus, they carry low risk societal misuse and do not produce general-purpose generative content for public consumption.
36

---

K Qualitative Comparison
Here conduct qualitative comparisons between the prediction of FunDPS and DiffusionPII: Darcy Flow (Figure 8). Poisson (Figure 9), Helmholtz (Figure T0) and Navier-Stokes (Figures 11 and l2) problems.
Obazrcation
Ground Truth
FunDPS (Ouns)
DiflasionPDE
FunDPS Ernor (Ouns)
DiliusienPD Faror
Foroeni
Enor: 3.69%
Errer: 10.41%
Inoure
Figure 8: We compare the results of our method with the diffusion- baseline, DiffusionPDE. on the Darcy Flow problem. The first column shows the 3% observed measurements, while the second column shows the corresponding ground truth (note that these two states are in different spaces). The middle two columns show the reconstruction results of our method and DiffusionPDE, respectively. The last two columns present the absolute error between the predictions and the ground truth. We provide relative errors for this specific test sample as well.
(Ours)
(Ours)
Porward
Erer: 1.51%
Ener: 16.99%
Invone
Froer: 22.68%
79%
Figure 9: We compare the results of our method with the diffusion-based baseline, DiffusionPDE, on the Poisson problem. The first column shows the 3% observed measurements, while the second column shows the corresponding ground truth (note that these two states are in different spaces). The middle columns show the reconstruction results of our method and DiffusionPDE, respectively. The last two columns present the absolute error between the predictions and the ground truth. We provide relative errors for this specific sample as well.
37

---

Trufh
PanDPSS (Orin)
DifinsionPDD
FonDDS
Foreonl
Ianer: 1.94%
Emor: 12.81%
Inverse
Figure 10: We compare the results of our method with the diffusion basoed baseline, DiffusionPDE. on the Helmholtz problem. The first column shows the 3% observed measurements, while the second column shows the corresponding ground truth (note that these two states are in different spaces). The middle columns show the reconstruction results of our method and DiffusionPDE, respectively. The last two columns present the absolute error between the predictions and the ground truth. We provide relative errors for this specific test sample as well.
Observation
Ground Trihh
FunDPS (Our)
DiflssioPDE
FunDPS Froo (Ouns)
DiftisionPD Eror
Frreend Briblen
Imoom
Enoe: 8.90%
Eroor 9.99%
Figure 11: We compare the results our method with the diffusion-based baseline, DiffusionPDE. on the Navier-Stokes problem. The first column shows the 3% observed measurements, while the second column shows the corresponding ground truth (note that these two states are in different spaces). The middle two columns show the reconstruction results of our method and DiffusionPDE. respectively. The last columns present the absolute error between the predictions and the ground truth. We provide relative errors for this specific test sample as well.
(Curs)
(Ouns)
Foraad
Exror: 3.74%
Emor: 7.68%5
Lmur: 2.5%%
Emne: 3.62%
Figure 12: We compare the results of method with the diffusion-based baseline, DiffusionPDE. on the Navier-Stokes with boundary conditions problem. The first column shows the 3% observed measurements, while the second column shows the corresponding ground truth (note that these two states are in different spaces). The middle two columns show the reconstruction results of our method and DiffusionPDE, respectively. The last two columns present the absolute error between the predictions and the ground truth. We provide relative errors for this specific test sample as well.
38


Reviews:
[{"review_id": "Sid65ARnuE", "reviewer": "Reviewer_wgk8", "summary": "The paper introduces FunDPS, a novel framework for solving inverse problems involving partial differential equations (PDEs) using diffusion models in function spaces. It combines a discretization-agnostic denoising diffusion model with a plug-and-play guidance mechanism that uses sparse observations for conditional sampling. FunDPS achieves state-of-the-art accuracy and speed, demonstrating significant improvements over prior methods in recovering solutions from as little as 3% of observed data.", "questions": "(1) What are the main intuition of generalizing the Tweedie's formula to the function space?  \n(2) When you compare your method with DiffusionPDE, are you using the same neural network backbone architecture?  \n(3) What are the challenges to generalize this algorithm to more practical diffusion usecase like image generation?  \n(4) Do we need to apply any assumption (theoretical or practical) to this inverse problem formulation if we want to extend Tweedie's formulation to infinite-dimensional Banach spaces?", "limitations": "NA", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: The paper adopts a function space formulation of diffusion and can achieve faster and better sampling from experiment. The paper has a clear explanation of the theoretical innovations and is well written. The paper proposes a novel discretization-agnostic framework which can generalize across different resolution\n\nWeakness: \n(1) My main concern is incremental innovation compared to DiffusionPDE, although theoretically fancy, Alorithm 1 has exactly the same procedure as DiffusionPDE, so could you point out what is the main difference with diffusionPDE from the implementation perspective? also why the PDE loss is not included in the loss term? Is it a negative contribution to both DiffusionPDE and FunDPS? If it is, it will be more fair to compare both method without this term together.  \n(2) It will be better to highlight more the intuition and motivation to introducing the functional space especially for this PDE problem, the current version reads more like motivation for diffusion inverse model + functional space diffusion..  \n\n(3) Since the backbone is relatively small scale and influence of hyperparameters will be more obvious, it will be good to add more ablations on the influence of hyperparameters. \n\n(4) Also will be more fair if you can perform careful hyperparameter on the baseline method for different problems as well, especially the error you report here as baseline is much higher than the original numbers reported in the DiffusionPDE paper.  \n\n(5) It will be good to include more diffusion-based PDE solver besides DiffusionPDE, also the model 54M is relatively small, not sure if it can generalize well on more general or ood test datasets.  \n\n(6) Current experiment applies 3\\% observation and will be interesting to see how the method along with diffusionPDE can handle fewer observation and what are the lower limit of observation of diffusion-based approach.  \n\n(7) I did not fully understand the multi-resolution training, but want to know will this training method influence the sampling steps, final accuracy, and does using coarser grid break the 3\\% observation assumption? If this multi-resolution training will influence final accuracy, are you applying it to the diffusion-based baselines as well?\n\n(8) As in DiffusionPDE paper which compares DiffusionPDE with Diffusion with CFG, does it make sense to compare FunDPS with the functional-space diffusion with CFG as well?", "was_revised": true, "final_justification": "I hold my opinion of borderline reject as I was not convinced of the necessity of going beyond euclidean space (also skeptical of the euclidean space experiment results). But I am open to any final paper decision.", "rebuttal": "We thank the reviewer for their time and their thoughtful, constructive feedback.\n\n> [W1] Concern about incremental innovation compared to DiffPDE. Plus, is PDE loss a negative contribution to DiffPDE and FunDPS?\n\nA: Thanks for the question. Inside the sampling loop, our work offers three key improvements over DiffPDE:\n- **Architecture**: Our core contrib. is a discretization-agnostic generation framework,  by using U-NO and GRF noise model operating on continuous representations. However, DiffPDE, which is tied to a fixed grid resolution, cannot generalize to different resolutions well. **This generalization is important because in scientific domains, data are often collected in different resolutions.** Our ablation study (Tab 9) further confirms the benefits of function-space approach.\n- **Theory**: We introduce and prove the first generalization of Tweedie’s formula to Banach spaces, a critical theoretical guarantee for guided sampling in function spaces, to provide principled guided sampling when using GRF noise.\n- **Multi-resolution Capabilities and SotA Performance**: These contributions unlock new multi-resolution strategies, achieving a 25x inference speedup and a 32% accuracy improvement.\n\nAs for the second question, PDE loss **is** included. It is expressed in a uniform way with the obs. loss. Contrary to the concern, both our work (App F) and DiffPDE (Sec 4.7) demonstrate that PDE loss improves accuracy.\n\n> [W2] Highlight the intuition and motivation to introducing the functional space, especially for this PDE problem.\n\nA: Thank you for suggestion. Our main intuition is that physical systems described by PDEs are inherently continuous, and numerical solutions are discretizations of these dynamics. One major challenge in scientific computing is that a model trained on one specific grid often fails to generalize to another. Our function-space formulation addresses this by learning the resolution-agnostic mapping between the parameter and solution functions. This approach is more natural and well-suited as the real-world data inherently comes at varying resolutions. It is also becoming standard in AI4Sci community. Furthermore, this unlocks significant efficiency gains from our multi-resolution training and inference strategies.\n\n> [W3] It will be good to add more ablations on the influence of hyperparameters.\n\nA: We agree that selecting hyperparameters is important. In our submission, we analyze various factors: # of observations (App G.5); different inverse solvers (App G.1); different training schemes (App G.3); ratio of low resolution sampling steps (App G.4); and varying design choices (App G.7). Notably, our model has a wide stable range for guidance weights, shown in the rebuttal to mb5h.\n\n> [W4] It will be more fair to perform careful hyperparameter on the baseline, especially the error you report here is much higher than the original numbers.\n\nA: Thanks for the important question. To ensure a fair comparison, we used the official implementation and weights from the DiffPDE authors. We were in direct communication with them, and they confirmed that the results are the same as they could reproduce. In fact, for Helmholtz and NS1 cases, the reproduced baseline is even stronger than the original. We're confident this provides the most rigorous foundation for comparison.\n\n> [W5] It will be good to include more diffusion-based PDE solver. Also the model 54M is relatively small, not sure if it can generalize well on more general datasets.\n\nA: Thank you for your suggestion. We have included an exp. in App G.1 that evaluates three plug-and-play inverse solvers.\nRegarding model scale, we chose a 54M parameter model to make a fair comparison with DiffPDE. In ablation, the largest model has 180M parameters, comparable to the typical diffusion model size of 300M for ImageNet64.\n\nTo further confirm generalizability, we provide additional exp. on real-world geophysical Darcy dataset, where we outperform all tuned baselines. The dataset is generated following [1].\n||Forw.|Inv.|\n|-|-|-|\n|FunDPS|2.6|2.6|\n|DiffPDE-2000|3.0|3.9|\n|FNO|5.85|7.29|\n|U-NO|5.53|6.92|\n|U-FNO|3.12|3.80|\n\nFinally, we agree that scaling to larger models and more complex datasets is an exciting future direction.\n\n[1] Wen, Gege, et al. \"U-FNO—An enhanced Fourier neural operator-based deep-learning model for multiphase flow.\"\n\n> [W6] How the method along with DiffPDE can handle fewer observation and what are the lower limit?\n\nA: Good question! We have investigated this in App G.5. We further conducted exp. with DiffPDE, shown below. Our method consistently outperforms across varying # of observations. As for the lower limit, it really depends on the use case; it maintains under 10% error with as little as 0.5% obs.\n|||Forw.|||Inv.|||\n|-|-|-|-|-|-|-|-|\n|# of Obs|DiffPDE-2000|DiffPDE-500|FunDPS|DiffPDE-2000|DiffPDE-500|FunDPS|\n|100|7.2|13.8|7.2|10.7|16.4|8.0|\n|200|6.5|7.8|4.8|9.1|10.2|6.3|\n|500|6.1|4.6|2.9|7.9|8.1|5.2|\n|2000|2.2|2.3|1.7|3.9|4.7|4.1|\n\n> [W7] Will multi-resolution training influence sampling steps and accuracy? Does using coarser grid break the 3% observation assumption? Are you applying it to the diffusion-based baselines as well?\n\nA: i) The number of inference steps and guidance weight are not affected by the strategy because our model can generalize.\n\nii) The observation is properly handled. 3% observation means 125 and 500 observed pts for 64 and 128 resolution.\n\niii) The multi-resolution inference cannot be applied to UNet as it cannot process different resolutions without training. The training strategy can be applied to DiffPDE. As shown below, FunDPS still outperforms baseline by a large margin with any training strategy. Notably, the UNet trained on 64x64 performs very badly on 128x128.\n|||Darcy Forw.|||\n|-|-|-|-|-|\n|Training|Inference|DiffPDE-2000|DiffPDE-500|FunDPS|\n|64|64|6.38|7.96|3.03|\n|64|128|33.72|35.57|3.64|\n|128|128|6.07|4.60|2.74|\n|Mixed|128|3.83|4.53|2.49|\n\n> [W8] Compare FunDPS with the functional-space diffusion with CFG?\n\nA: Thanks for this insightful comment. However, we believe such comparison is out of scope for the current work because\n- Adapting CFG to function space is a non-trivial task and could be a new paper in itself. Our focus is on a principled posterior sampling framework with a plug-and-play solver.\n- We compare primarily against DiffPDE, as it was already shown to outperform CFG models in its original paper. This provides a more informative evaluation against a stronger baseline.\n\n> [Q1] The main intuition of generalizing the Tweedie's formula to the function space?\n\nA: At a high level, Tweedie’s formula states that if $Y = X + Z$ is a noisy observation of a random variable $X$ corrupted by Gaussian noise $Z \\sim N(0, C)$, then the best guess (i.e. cond. expectation) for the value of $X$ given a noisy obs $Y=y$ is given by $E[X|Y=y]=y+C\\nabla_y\\log p(y)$. In other words, we take the observation $y$ and correct it by adding the vector $\\nabla_y\\log p(y)$, moving $y$ in the direction which most quickly increases its likelihood. The additional $C$ term accounts for correlations in the Gaussian noise.\n\nWhen generalizing this to function spaces, the intuition remains the same, with the key difference being $X,Y$ are now random functions, and $Z$ corresponds to a Gaussian random field. This means that the gradient must be replaced with Frechet derivative, which is no longer a vector but rather a function itself. Moreover, $C$ is no longer a covariance matrix, but rather a covariance operator which acts on functions. The shape of this corrective function is determined by the functional derivative of the log-probability density, smoothed by the covariance operator of the noise process.\n\nHowever, a key theoretical difference in function spaces is that probability densities only exist under certain strict conditions. To ensure a well-defined density exists in function space, the Cameron-Martin theorem (B.1) gives us the precise conditions, effectively requiring our data to live in a specific subspace (the Cameron-Martin space) where the signal and noise measures “agree” with each other.\n\nWe will update Sec 3.2 for the sake of more clarity.\n\n> [Q2] Compared with DiffPDE, are you using the same neural network arch?\n\nA: No, FunDPS uses a U-shaped Neural Operator, which is designed to be resolution-agnostic. This is the one aspect of our contribution.\n\n> [Q3] What are the challenges to generalize this algorithm to more practical diffusion usecase like image generation?\n\nA: Thanks for great question. PDE problems and image generation are important usecases. Generalizing function-space methods to image domain is an active research area. A key challenge is that NOs can oversmooth the sharp features in natural images. In principle, our framework could be adapted, but it is beyond the scope of this paper.\n\n> [Q4] Do we need to apply any assumption to this inverse problem formulation if we want to extend Tweedie's formulation to infinite-dimensional Banach spaces?\n\nA: Yes, the core assumption is that the data is supported on the Cameron-Martin space of the noise process. We refer to our Thm B.6.\nThis assumption is the infinite-dimensional equivalent of ensuring a well-defined probability density exists, which is a prerequisite for defining the score function at the heart of Tweedie's formula. As we noted, it is a standard requirement in function-space diffusion literature.\nWhile verifying this assumption theoretically can be difficult, a common workaround is to pre-smooth the data, for instance by applying the operator $C^{1/2}$ (the square root of the noise covariance). This effectively projects the data onto the Cameron-Martin space, meeting the assumption. For more details on this technique, we point to [1, Sec 4.2].\n\n[1] Lim, Jae Hyun, et al. \"Score-based diffusion models in function space.\" (2023)\n\nWe hope our detailed answers are helpful, and we kindly ask for a re-evaluation after this discussion."}, {"review_id": "qwStjF3svf", "reviewer": "Reviewer_8pVy", "summary": "The paper focusses on solving PDE-based inverse problems formulated in infinite dimensions using conditional diffusion processes. They formulate an infinite-dimensional conditional score via a density with respect to a Gaussian measure. They show that this can be decomposed into a likelihood term and prior or unconditional score. The likelihood is approximated in closed form by approximating the conditional reverse diffusion process with its conditional expected value. This expected value is proved to satisfy an infinite-dimensional Tweedie’s formula. Using the resulting closed form approximation of the guidance alongside a trained (unconditioned) score operator allows one to obtain samples from the posterior. The method is compared experimentally to other methods on forward and inverse problems for different PDE tasks.", "questions": "For Tweedie’s formula, is there anything that can be said about the assumption that the score of $\\nu$ is Fréchet differentiable? When will this hold?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. In general, I find the submission to be well-written.\n\n2. The method is compared experimentally to well-known PDE solvers across five different PDE experiments, which are the same as those considered in DiffusionPDE. The method performs well on these baselines, outperforming the other methods whilst using a quarter of the time discretisation steps used in DiffusionPDE.\n\n3. The submission proves an infinite-dimensional Tweedie’s formula. Tweedie’s formula is an important result used in finite-dimensional diffusion models. I find its extension to infinite-dimensions to be a valuable result.\n\n4. The paper also details a new training technique for learning the prior distribution at a lower cost using multi-resolution training. This is coupled with what they term multi-resolution inference also accelerating the inference time per step. I think this is a valuable contribution which can also be used in the infinite-dimensional unconditional problem.\n\n\nWeaknesses:\n\n\n1. In order to obtain a closed form expression for the likelihood, the conditional measure of the reverse diffusion process is approximated with its expectation. Can you comment on the error induced by this approximation?\n\n2. The paper introduces the conditional score of $\\nu^u_t$ as the object of interest, however it’s not clear to me exactly how it is used in the given setup for sampling. I think it would be beneficial to make this explicit, for example, by giving the time-reversal of (4) in terms of the conditional score.\n\n3. The submission introduces a lot of different notations for various measures and conditional distributions. I think this is necessary, however I would find a paragraph on the notation useful to refer back to, containing the main measures e.g. $\\mu, \\eta, \\gamma, \\nu$.\n\n\n4. The following weaknesses are minor and do not affect my score.\n\n    - In line 239 “firt” should be “first”.\n\n    - In Equation (13) should the scalar be $\\frac{c^2}{2}$ or $\\frac{c}{2}$? Equation (11) uses $C_{\\eta}^{-1/2}$, which according to line 277 should be given by $c^{1/2}I.$\n\n\nIn summary, I find the paper gives an important and novel contribution in extending diffusion models for Bayesian inverse problems into function space. Experimentally, the method is shown to work well in comparison to existing methods and I recommend accepting this paper. However, I think there are some presentational issues which if fixed would improve the paper. Most importantly, making the infinite dimensional sampling scheme clearer.", "was_revised": true, "final_justification": "I think the paper has strong theoretical results (formulating the conditional diffusion problem to function spaces, and an infinite-dimensional Tweedie's formula) coupled with good experimental results.", "rebuttal": "We thank the reviewer for their time and their thoughtful, constructive feedback.\n\n## W1\n**Problem**: In order to obtain a closed form expression for the likelihood, the conditional measure of the reverse diffusion process is approximated with its expectation. Can you comment on the error induced by this approximation?\n\n**Answer**: This is an important question and we thank the reviewer for pointing it out. In the Euclidean setting, the error of this kind of approximation is described precisely by the Jensen gap of the conditional density, i.e., $E_x [p(y|x)] - p(y|E[x])$. Depending on the form of the noise, an upper bound on this gap can be given [1].  We believe a similar result will hold in our setting, where the notion of a density is now replaced by an appropriate notion of a Radon-Nikodym derivative. However, further research is needed for a formal proof, and we plan to spend more effort in this direction.\n\n[1] Chung, Hyungjin, et al. \"Diffusion posterior sampling for general noisy inverse problems.\" arXiv preprint arXiv:2209.14687 (2022).\n\n## W2\n**Problem**: The paper introduces the conditional score of \\nu_t^u as the object of interest, however it’s not clear to me exactly how it is used in the given setup for sampling. I think it would be beneficial to make this explicit, for example, by giving the time-reversal of (4) in terms of the conditional score.\n\n**Answer**: The conditional score of $\\nu_t^u$ can be informally thought of as $\\nabla_{a} p_t(a_t \\mid u)$, i.e., the usual notion of a score function conditioned on an additional variable $u$. One of the main contributions of our work is an approximation of this conditional score in function spaces using an unconditional score and an additional guidance term (Eqn 13). To sample using this score, one starts by initializing $a_T$ as a Gaussian random field, and iteratively moving in the direction of this conditional score (Eqn 14 and L3-11 in Alg 1). \n\nWe agree with you that this could be explained more precisely in our submission. Your suggestion of writing the time reversal of (4) in terms of this conditional score will make this more clear to readers, and we will update our paper to include this. \n\n\n## W3\n**Problem**: The submission introduces a lot of different notations for various measures and conditional distributions. I think this is necessary, however I would find a paragraph on the notation useful to refer back to, containing the main measures e.g. \\mu, \\eta, \\gamma, \\nu.\n\n**Answer**: We thank the reviewer for their constructive feedback. In the final manuscript, we will include a more comprehensive notation paragraph to enhance the paper’s readability.\n\n\n## W4\n**Problem**: \n- In line 239 “firt” should be “first”.\n- In Equation (13) should the scalar be c^2/2 or c/2? Equation (11) uses C_\\eta^{-1/2}, which according to line 277 should be given by c^{1/2}I.\n\n**Answer**: We thank the reviewer for reading the paper thoroughly and finding these typos. Indeed, both points are valid and fixed in the paper. For the second point, we will fix Equation (13) to be $\\frac{c}{2}$. As we note in lines 286-288, the constant term is $\\frac{c}{2}$ (not $\\frac{c^2}{2}$) and absorbed into the final guidance weight hyperparameter $\\zeta$. This hyperparameter has been tuned for each specific problem based on a small validation set (Appendix F).\n\n\n## W*\n**Problem**: In summary, I find the paper gives an important and novel contribution in extending diffusion models for Bayesian inverse problems into function space. Experimentally, the method is shown to work well in comparison to existing methods and I recommend accepting this paper. However, I think there are some presentational issues which if fixed would improve the paper. Most importantly, making the infinite dimensional sampling scheme clearer.\n\n**Answer**: Thank you very much for the positive feedback and constructive suggestions. We will revise the final manuscript based on all valuable reviews we get to make the main idea clearer. The infinite-dimensional sampling scheme is one of our core contributions, and we will enhance the content by rethinking definitions, better organization, and additional discussions.\n\n---\n\n## Q1\n**Problem**:  For Tweedie’s formula, is there anything that can be said about the assumption that the score of \\nu is Fréchet differentiable? When will this hold?\n\n**Answer**: This is a good question that touches on a subtle theoretical point. The answer depends on the regularity of the data distribution $\\mu$. \nThe Fréchet differentiability of the score of $\\nu$ is fundamentally tied to the smoothness of the Radon-Nikodym derivative $\\frac{d\\nu}{d\\gamma}$, where $\\gamma$ is the base Gaussian measure.\n\nFor a Gaussian measure $\\gamma$, the natural space of directions for differentiation is its Cameron-Martin space. Since our perturbed measure $\\nu$ is equivalent to $\\gamma$ (i.e., they are mutually absolutely continuous, per Prop B.2), $\\nu$ inherits much of the structure of $\\gamma$. Therefore, assuming the score of $\\nu$ is Fréchet differentiable with respect to perturbations from the Cameron-Martin space is a reasonable starting point, provided the data distribution $\\mu$ is sufficiently regular. Without further assumptions on $\\mu$, however, a more general statement is difficult. For a complete technical discussion, we refer to [2, Chapter 5].\n\nOne special case, though, where it is possible to give a definitive answer is when the data distribution $\\mu$ is itself Gaussian. In this scenario, the perturbed measure $\\nu$, being the convolution of two Gaussians, is also a Gaussian measure, and thus its Frechet differentiability is fully characterized by its Cameron-Martin space. See [3, Example 1] for a full proof. \n\n[2] Bogachev, Vladimir Igorevich. Gaussian measures. No. 62. American Mathematical Soc., 1998.\n\n[3] Lim, Jae Hyun, et al. \"Score-based diffusion models in function space.\" arXiv preprint arXiv:2302.07400 (2023)."}, {"review_id": "r3T5z1LmnF", "reviewer": "Reviewer_uwzC", "summary": "The papers propose a general framework for conditional sampling in PDE-based inverse problems, to recover the whole solutions from extremely sparse or noisy measurements, by a function-space diffusion model and plug-and-play guidance for conditioning.", "questions": "Please see the weaknesses for specific questions.", "limitations": "Yes, the author discusses the limitations of the proposed method in the conclusion.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n+ This paper is well written and easy to follow.\n\n+ The investigated problem is well-motivated to design the diffusion model in the function space for solving PDE problems. \n\n+ The proposed method looks reasonable to achieve the goal, with promising results achieved compared with previous baseline such as DiffusionPDE.\n\n\nWeaknesses:\n\n- The key contribution lies in the directly extending DiffusionPDE to the function space through function representation. The contribution on the multi-resolution training and inference for acceleration, and conditional sampling algorithm for solving PDE problems may be limited, since they are mostly integrated from previous works such as DiffusionPDE. \n\n- The results with different inverse solvers in Table 5 are interesting. The results show that DAPS and DDNM is far worse than the DPS, which may diverge from the results in previous works. It would be helpful to elaborate what could be the underlying reason, like due to the function space formulation or PDE task?\n\n- A key baseline compared in the paper is the DiffusionPDE, with the main difference in finite v.s. infinite representation space for training diffusion models, with claimed key contribution in extended Tweedie’s Formula. But another possible simpler approach is to represent measurement in the function space and inject to each diffusion sampling step to avoid estimating the Tweedie’s Formula. This was introduced in the diffusion inverse solvers before the introduction of DPS. Would this be possible to solve the target problem?\n\n- The paper claims the sampling from posterior distribution with DPS guidance, while DPS is shown not to provably sample from posterior, such as in [1]. Can the author elaborate more on how to guarantee the posterior sampling as claimed?\n\n[1] Cardoso, G., Janati El Idrissi, Y., Le Corff, S., & Moulines, É. (2024). Monte Carlo guided Denoising Diffusion Models for Bayesian Linear Inverse Problems. In Proceedings of ICLR 2024", "was_revised": true, "final_justification": "The author provides extensive rebuttal to reply my questions, which are helpful to address most concerns. See comments for more details. Considering this, I am pleased to raise my score.", "rebuttal": "We thank the reviewer for their time and their thoughtful, constructive feedback.\n\n> [W1] The key contribution lies in the directly extending DiffusionPDE to the function space through function representation. The contribution on the multi-resolution training and inference for acceleration, and conditional sampling algorithm for solving PDE problems may be limited, since they are mostly integrated from previous works such as DiffusionPDE.\n\n**Answer**: Thank you for this critical question. We would like to clarify the distinction and novelty of our work compared to DiffusionPDE. While we use their datasets for a fair and direct comparison, our methodology is fundamentally different. Shortly speaking, the only design choice we borrowed from them is the joint-embedding mechanism. In other aspects, FunDPS has important contributions as follows:\n- **A New Discretization-Agnostic Framework**: Our core contribution is a framework that is inherently discretization-agnostic. FunDPS achieves this by using a U-shaped Neural Operator (U-NO) and Gaussian Random Field (GRF) noise model, which operates on continuous function representations. This differs sharply from DiffusionPDE, which is based on convolutional networks and is fundamentally tied to a fixed grid resolution (Please check Table 2 in the response to reviewer wgk8). This is not only a change in representation, but a critical shift in the underlying model that leads to inherent resolution-invariance, which was the key limitation of prior works. **Our ablation study in Table 9 further confirms the benefits of function-space approach** with significant performance boost.\n- **Novel Theoretical Guarantees in Function Spaces**: To provide principled guided sampling on the infinite-dimensional settings, we propose and prove the generalization of Tweedie’s formula to Banach spaces. This theoretical result is a critical contribution that provides a mathematical relation between the score function and the denoising operator in function spaces, which is crucial for many plug-and-play guidance mechanisms, including ours.\n- **Multi-resolution Capabilities and State-of-the-Art Performance**: Our function space formulation inherently leads to powerful multi-resolution training and inference strategies that were impossible with prior works as they are tied to fixed resolutions. For instance, **our ReNoise inference pipeline achieves 25x wall-clock speedup** over DiffusionPDE. Furthermore, our resulting framework obtains 32% average accuracy improvement with 4x fewer sampling steps. \n\nWe will revise the introduction to more explicitly highlight these fundamental distinctions.\n\n\n> [W2] The results with different inverse solvers in Table 5 are interesting. The results show that DAPS and DDNM is far worse than the DPS, which may diverge from the results in previous works. It would be helpful to elaborate what could be the underlying reason, like due to the function space formulation or PDE task?\n\n**Answer**: We thank the reviewer for highlighting this interesting result. The significant underperformance of DDNM and DAPS in our setting is primarily due to the extreme sparsity of observation data, not the function space formulation, which we found after extensive investigation.\n- DDNM relies on projecting the sample onto measurement subspace at each step. With only 3% of observation points, the measurement subspace is extremely low-dimensional compared to the overall function space. As a result, the projection provides very weak guidance and leads to poor reconstruction.\n- DAPS uses a decoupled noise annealing schedule. The core issue comes from the localness of guided updates during Langevin dynamics. Only the points with observation are updated and others are just added with noise. This makes the intermediate state after Langevin dynamics discontinuous and out-of-distribution, which hinders performance. If we increase the # of sampling steps to 20,000, the accuracy can match DPS, but with 100x more time. The same trend is also observed in image-domain frameworks as we provide the quantitative results. \n\nWe have figures available to back our findings. We will include them in the camera-ready version and hope that it can provide insights into the advantages and disadvantages of different inverse solvers.\n\n\n> [W3] A key baseline compared in the paper is the DiffusionPDE, with the main difference in finite v.s. infinite representation space for training diffusion models, with claimed key contribution in extended Tweedie’s Formula. But another possible simpler approach is to represent measurement in the function space and inject to each diffusion sampling step to avoid estimating the Tweedie’s Formula. This was introduced in the diffusion inverse solvers before the introduction of DPS. Would this be possible to solve the target problem?\n\n**Answer**: Thank you for this insightful question about alternative approaches. To ensure we understand accurately, we would be grateful if the reviewer could clarify which specific prior works they have in mind. This may refer to resampling-based or projection-based methods. Proceeding with that assumption, we can explain why such an approach, while powerful in some domains, is ill-suited for the PDE inverse problems here.\n- **Instability of forward operators on noisy states**: Representing measurement would often require applying the forward operator $A$ (i.e. masking or PDE solver) to a noisy intermediate state $a_t$ to enforce consistency. PDE solvers are often ill-posed or numerically unstable when their input is a noisy, non-physical function, which is the case in huge noise spaces. FunDPS prevents this by first using denoiser to predict a clean state $\\hat{a_0} = D_\\theta(a_i, t_i)$, and then applying the forward operator to this physically plausible estimate $A(\\hat{a_0})$. This leads to more stable gradient guidance. \n- From a practical standpoint, **our generalized Tweedie's formula is computationally inexpensive**. Plus, both methods involve approximations, so there is no clear advantage of one over another.\n\nWith that said, our generalization of Tweedie’s formula bridges the denoising target and the score function required for guidance. We believe this contribution is not just crucial for our method but can also empower a wide range of future guided diffusion solvers that operate in function spaces.\n\n\n> [W4] The paper claims the sampling from posterior distribution with DPS guidance, while DPS is shown not to provably sample from posterior, such as in [1]. Can the author elaborate more on how to guarantee the posterior sampling as claimed?\n\n**Answer**: Thank you for this important point and for providing the reference. The reviewer is correct that our method, like other guided diffusion techniques, provides an approximation of true posterior distribution. The approximation arises because the time-dependent log-likelihood term in the conditional score (Equation 6) is intractable, and we approximate using the conditional expectation of the clean sample (Equation 9 & 10).\nWe want to highlight that **even in standard finite dimensions, there is no inference-time inverse solver that has guarantees for true posterior (w/o training)**. Methods, such as Sequential Monte Carlo (SMC, TDS, etc.), have stronger *asymptotic* guarantees, but they are computationally more intense. It would be interesting to extend such methods to function spaces in future work.\nWe will revise our manuscript to claim “approximate posterior sampling”. We will further add a sentence in Section 3.3 to explicitly state the source of this approximation by referring to the suggested paper [1].\n\n[1] Cardoso, G., Janati El Idrissi, Y., Le Corff, S., & Moulines, É. (2024). Monte Carlo guided Denoising Diffusion Models for Bayesian Linear Inverse Problems. In Proceedings of ICLR 2024\n\nWe hope our detailed answers are helpful, and we kindly ask for a re-evaluation of our submission in light of this discussion."}, {"review_id": "8VpnI2ye0t", "reviewer": "Reviewer_mb5h", "summary": "FunDPS proposes a general framework for solving PDE-based inverse problems through conditioning sampling. The core idea is to first learn the prior distribution of the underlying functions by training an unconditional diffusion model in function space. Then, conditioning on observations is achieved at inference time using a plug-and-play guidance mechanism derived from an extension of Tweedie's formula.", "questions": "1. How sensitive is the model's performance to the choice of $\\zeta$? Please provide an ablation study showing performance across a range of $\\zeta$. Moreover, is $\\zeta$ manually chosen or is automatically determined under certain theoretical background?", "limitations": "1. Refer to weakness.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. The primary contribution is the rigorous extension of diffusion models to infinite-dimensional function spaces for inverse problems.\n2. The method demonstrates superior empirical performance on a variety of challenging PDE-based inverse problems, often achieving better results in fewer sampling steps than competing methods. The inclusion of discussion regarding wall-clock time addresses potential concerns about the computational overhead of calculating the guidance term's gradient.\n3. The proposed multi-resolution training and inference strategy is a key practical advantage, enabling efficient handling of functions at different levels of detail and likely contributing to the model's overall performance and speed.\n\nWeakness\n1. While the paper's contributions are significant, the core components are not entirely novel in isolation. The concept of function-space diffusion models builds upon prior work (e.g., Score-based Diffusion Models in Function Space), and plug-and-play guidance is a well-established technique in finite-dimensional diffusion models.", "was_revised": true, "final_justification": "All of my concerns have been addressed. \nHowever, I will keep my score as the concerns were minor.", "rebuttal": "We thank the reviewer for their time and their thoughtful, constructive feedback.\n\n> [W1] While the paper's contributions are significant, the core components are not entirely novel in isolation. The concept of function-space diffusion models builds upon prior work (e.g., Score-based Diffusion Models in Function Space), and plug-and-play guidance is a well-established technique in finite-dimensional diffusion models.\n\n**Answer**: Thank you for your comment. We agree that our work builds upon important prior research. However, our primary contribution is based on a novel integration of these ideas. Most importantly, FunDPS provides essential theoretical and practical contributions to make this combination work and be effective for solving ill-posed inverse problems arising in PDEs.\nWhile plug-and-play guidance was established in finite dimensions, a rigorous theoretical framework in the infinite-dimensional setting of function spaces was absent. Our work introduces the first generalization of Tweedie’s formula to Banach spaces. This is a key novelty that enables us to propose principled gradient-based guidance for diffusion models operating directly on a function. It goes beyond a simple combination of existing works, enabling design choices that are crucial for superior accuracy, speed, and multi-resolution strategies.\n\n\n> [Q1] How sensitive is the model's performance to the choice of \\zeta? Please provide an ablation study showing performance across a range of \\zeta. Moreover, is \\zeta manually chosen or is automatically determined under certain theoretical background?\n\n**Answer**: This is an excellent question regarding the key hyperparameter in our framework. Currently, the guidance weight $\\zeta$ is manually chosen for each task by tuning on a small validation set. This is a common practice of guided diffusion models, as optimal weight often depends on the forward operator and prior data distribution. We provide a list of tuned $\\zeta$ values used for all five PDE tasks in Appendix F, Table 4. In our experience, the model’s performance is stable in wide ranges. However, values that are too high can cause the sampling to become unstable and diverge, while the values that are too low result in weak guidance and less accurate reconstructions.\n\nNote that after several trials, we follow ideas from the DPS paper, and we use adaptive guidance weighting based on noise level as highlighted in Appendix F. We agree that developing a method to automatically determine $\\zeta$ is an important direction for future research, which could help overcome inherent challenges in this field.\n\nFinally, we followed the suggestion and conducted an ablation study to show the performance of Darcy and Poisson equation forward/inverse problems across a range of $\\zeta$ values. We can observe that FunDPS has expected behaviour across different ranges.\n|zeta|Darcy Forw.|Darcy Inv.|Poisson Forw.|Poisson Inv.|\n|-|-|-|-|-|\n|1000|6.92|24.02|3.31|34.38|\n|5000|2.92|11.01|2.56|28.05|\n|10000|**2.49**|9.16|**1.99**|**19.47**|\n|20000|NA|9.86|18.86|20.47|\n|50000|NA|**5.18**|NA|NA|\n\nWe hope our detailed answers are helpful, and we kindly ask for a re-evaluation of our submission in light of this discussion."}, {"review_id": "f95Gy37ady", "reviewer": "Reviewer_dkME", "summary": "The authors solve the problem of finding samples for a posterior function where the likelihood is given by a data fitting term and the regularization is given by a diffusion model. The setup for this problem is infinite dimensions where the parameter to be recovered belong to some functional space (rather R^n).\n\nThis types of papers are important from a theoretical point of view as they justify multiresolution work and can aid in any multiscale process.", "questions": "My questions are more on the practical side of the work.\n\nEquation 13 is the approximation you use for the velocity when solving the ODE. This approximation is highly inefficient in the context of inverse problems and gradient descent. This is because if c is large (you want to actually fit the data) the ODE becomes highly stiff and you will need an implicit type method. This is why almost every PDE constrained optimization problem is solved using a Gauss-Newton type method rather than SGD. Your comparisons with other highly sub-optimal methods does not strike confidence.", "limitations": "No", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The paper may be strong theoretically although I am having a hard time following the technical content (this is more my issue that the authors). I think that researchers that are more versed in the branch of functional analysis and probability could give a more precise evaluation. \nI do think that the overall content is difficult to read and work is needed to make it easier if you want a CS type publication.\n\nI have only one question/comments (see below)", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and their thoughtful, constructive feedback.\n\n> [Strengths And Weaknesses]\n\n**Answer**: We thank the reviewer for their valuable feedback and for recognizing the theoretical importance of our work. Due to the special care needed in function spaces, our formulation requires introducing notations from functional analysis.\nTo enhance clarity, we will add a high-level introductory paragraph at the beginning of the Method section. Our goal is to ensure that foundational concepts are presented more simply to make the technical details accessible to a broader audience.\n\n> [Q1] Equation 13 is the approximation you use for the velocity when solving the ODE. This approximation is highly inefficient in the context of inverse problems and gradient descent. This is because if c is large (you want to actually fit the data) the ODE becomes highly stiff and you will need an implicit type method. This is why almost every PDE constrained optimization problem is solved using a Gauss-Newton type method rather than SGD. Your comparisons with other highly sub-optimal methods does not strike confidence.\n\n**Answer**: We agree that for traditional PDE-constrained optimizations, where the goal is to find a single optimal solution, the gradient descent methods are often inefficient due to stiffness. However, our approach operates in a different framework (generative posterior sampling), solving the probability flow ODE directly with an explicit and well-posed definition. Specifically, our goal is not to find a single point estimate but to draw samples from an approximate posterior distribution. This process is governed by a reverse-time ODE that integrates a powerful data-driven prior (diffusion model) with a likelihood guidance term. We would like to further clarify several points about our implementation:\n- **FunDPS is sampler agnostic**: Equation 13 defines a conditional score that provides the velocity for the reverse ODE. Since it does not prescribe any specific numerical solver nor do we use any specific solver throughout the proofs, our framework is general and can be integrated with any suitable ODE solver. The choice of sample is separate from the formulation of the guidance term (i.e. likelihood term).\n- **We use a higher-order solver, not SGD**: We do not use first-order Euler steps (analogous to not SGD). As shown in Algorithm 1, we combine the method with a second-order deterministic sampler inspired by the EDM framework [1]. This higher-order solver mitigates the instability and stiffness issues related to simpler first-order schemes. \n- **Extensive Comparisons**: During this work, we compared deep learning and GenAI baselines, including the SotA works (DiffusionPDE, FNO, PINN), and demonstrated our method's superior speed and accuracy.\n\n[1] Karras, Tero, et al. \"Elucidating the design space of diffusion-based generative models.\" Advances in neural information processing systems 35 (2022): 26565-26577.\n\nWe hope our detailed answers are helpful, and we kindly ask for a re-evaluation of our submission in light of this discussion."}]

Author rebuttal:

Meta-review: The paper introduces a new framework for solving inverse problems for partial differential equations. The main contribution is an extension of a generative diffusion framework—originally developed for jointly modeling the distribution of PDE solutions and their parameter space in finite dimensions—to infinite-dimensional function spaces. The method first trains an unconditional diffusion model in function space and then conditions at inference time on observations. This conditioning leverages a plug-and-play mechanism based on an extension of Tweedie’s formula to infinite-dimensional spaces. The framework implements a neural operator, is resolution-agnostic, and exploits this property to accelerate training. As in the original finite-dimensional diffusion work, the focus is on solving PDE inverse problems under partial observation. Experiments and comparisons are conducted on simulations from a variety of PDEs, addressing both inverse and forward problems.

The reviewers find that the paper makes several contributions, including: a novel function-space formulation of joint distribution modeling, an extension of Tweedie’s formula from finite-dimensional diffusion models to infinite-dimensional spaces, and promising experimental results compared with several baselines across diverse PDE simulations. Some reviewers note, however, that while novel, the originality of the contribution is somewhat limited.

During the rebuttal, the authors provided detailed answers and clarifications, along with new experimental results and ablations. All reviewers agree that most of their concerns were addressed: one increased their score, while the others maintained their initial evaluation.
Considering the discussion, I recommend acceptance. However, in its present form, the paper will be accessible to only a narrow audience. I encourage the authors to provide a clearer and more accessible description of their contribution.
Decision: Accept (poster)