DiffusionPDE: Generative PDE-Solving under Partial Observation
Authors: Jiahe Huang
Guandao Yang
Zichen Wang
Jeong Joon Park
Venue: NeurIPS 2024 poster (2024)
Keywords: Guided Diffusion Model
Partial Differential Equation
Sparse Observation
Inverse Problem
Primary area: machine_learning_for_physical_sciences
TL;DR: DiffusionPDE solves forward and inverse PDEs from partial observations using diffusion models.

Abstract: We introduce a general framework for solving partial differential equations (PDEs) using generative diffusion models. In particular, we focus on the scenarios where we do not have the full knowledge of the scene necessary to apply classical solvers. Most existing forward or inverse PDE approaches perform poorly when the observations on the data or the underlying coefficients are incomplete, which is a common assumption for real-world measurements. In this work, we propose DiffusionPDE that can simultaneously fill in the missing information and solve a PDE by modeling the joint distribution of the solution and coefficient spaces. We show that the learned generative priors lead to a versatile framework for accurately solving a wide range of PDEs under partial observation, significantly outperforming the state-of-the-art methods for both forward and inverse directions.

DiffusionPDE: Generative PDE-Solving Under Partial Observation
Jiahe Huang'
Guandao Yang² Zichen Wang' Jeong Joon Park'
University of Michigan
Stanford University
{chloehjh, zzzichen, jjparkev) (earkcc)@@michh @umf edu
guandao@stanf ord edu
Abstract
We introduce general framework for solving partial differential equations (PDEs) using generative diffusion models. In particular, we focus on the scenarios where we do not have the full knowledge of the scene necessary to apply classical solvers. Most existing forward or inverse PDE approaches perform poorly when the ob- servations on the data or the underlying coefficients are incomplete, which is a common assumption for real-world measurements. In this work, we propose Diffu- sionPDE that can simultaneously fill in the missing information and solve PDE by modeling the joint distribution of the solution and coefficient spaces. We show that the learned generative priors lead to a versatile framework for accurately solving wide range of PDEs under partial observation, significantly outperforming the state-of- -ice-o--hhe-rrt the-art methods for both forward and inverse directions. See our project page for results: jhhuangchloe github. www.ggiiio/i.iifssiiiieeeee Dtt.iiiiiiioo-DDDDDD.
Introduction
Partial differential equations (PDEs) are a cornerstone of modern science, underpinning many contemporary physical theories that explain natural phenomena. The ability to solve PDEs grants us the power to predict future states a system (forward process) and estimate underlying physical properties from state measurements (inverse process).
To date, numerous methods [1, 2] have been proposed to numerically solve PDEs for both the forward and inverse directions. However, the classical methods can be prohibitively slow, prompting the development of data-driven, learning- based solvers that are significantly faster and capable of handling family of PDEs. These learning- ceasning-based based approaches [3-6] typically learn deterministic mapping between input coefficients and their solutions using deep neural networks.
Despite the progress, existing learning- based approaches, much like classical solvers, rely complete observations of the coefficients map solutions. However, complete information on the underlying physical properties or the state of a system is rarely accessible; in reality, most measurements are sparse in space and time. Both classical solvers and the state-of-the-arr data-driven models often overlook these scenarios and consequently fail when confronted with partial observations. This limitation confines their use primarily to synthetic simulations, where full scene configurations are available by design, making their application to real-world cases challenging.
We present a comprehensive framework, DiffusionPDE,, for solving PDEs in both forward and inverse directions under conditions of highly partial Abservations--tyicee information. This task is particularly challenging due to the numerous possible ways to complete missing data and find subsequent solutions. Our approach uses a generative model to formulate the joint distribution of the coefficient and solution spaces, effectively managing the uncertainty and simultaneously reconstructing both spaces. During inference, sample random noise and iteratively
just 1-3% of the total
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Training
Joint Distribution
Diffusion Model
Predictions
Coethciens/mitaal State: Solution/Final State:
(on both a and it Generated a
Observations
Inference
(on either side or both)
Generated a
Sparse Truth Sparse Ground
Diffusion Model
Noise
PDE Guidance
Observation Guidance
Figure 1: We propose DiffusionPDER, generative PDE solver under partial observations. Given family PDE with coefficient (initial state) and solution (final state) u, we train the diffusion model on the joint distribution of and u. During inference, we gradually denoise a Gaussian noise, guided by sparse observation and known PDE function, to recover the full prediction of both and that align well with the sparse observations and the given equation.
denoise it following standard diffusion models [7]. However, uniquely guide this denoising process with sparse observations and relevant PDE constraints, generating plausible outputs that adhere the imposed constraints. Notably, DiffusionPDE handle observations with arbitrary density and patterns with a single pre-trained generative network.
We conduct extensive experiments to show the versatility of DiffusionPDE: general PDE-solving framework. evaluate on diverse set of static and temporal PDEs, including Darcy Flow, Poisson, Helmholtz, Burger's, s, and Navier-Stokes equations. DiffusionPDE: significantly outperforms existing state-of-the-art learning- methods for solving PDEs [3-6, 8] in both forward and inverse directions with sparse measurements, while achieving comparable results with full observations. Highlighting the effectiveness our model, DiffusionPDE accurately reconstructs the complete state of Burgers' equation using time-series data from just five sensors (Fig. 4), suggesting the potential of generative models to revolutionize physical modeling in real-world applications.
2 Related Works
Our work builds on the extensive literature of three areas: forward PDE solvers, inverse PDE solvers, and diffusion models. Please see relevant surveys for more information [9-13].
Forward PDE Solvers. PDE solvers take the specification of physics system and predict its state in unseen space and time by solving an equation involving partial derivatives. Since Most PDEs solve analytically, people resolve to numerical techniques, such Finite Element Method [14, 2] and Boundary Element Method [1, 15). While these techniques show strong performance and versatility in some problems, they can be computationally expensive or difficult to set for complex physics systems. Recently, ad adancements in deep-learning methods have inspired new
very challenging
PDE solvers. Raissi et al. [16. 6] introduce Physics- Informed Neural Networks (PINNs), which optimize a neural network using PDE constraints as self-supevvised losses to output the PDE solutions. PINNs have been extended solving specific fluid [17, 18], Reynolds- Navier-Stokes equations [19], heat equations [20], and dynamic power systems [21]. While PINNs can tackle wide range of complex PDE problems, they are difficult to scale due to the need for network optimization. An alternative approach, neural operators [3, 5]. directly learn the mapping from PDE parameters (e.g. initial and boundary condition) to the solution function. Once trained, this method avoids expensive network optimization and instantly output the solution result. This idea

---

has been extended to solve PDE in 3D [22, 23]. multiphase flow [24], seismic wave [25, 26). 3D turbulence [27, 28], and spherical dynamics [29]. People have also explored using neural networks as part of the PDE solver, such as compressing the physics state [30-33]. These solvers usually assume known PDE parameters, and applying them to solve the inverse problem can be challenging.
PDE inverse problem. The inverse problem refers to finding the coefficients of PDE that can induce certain observations, mapping from the solution of a PDE solver its input parameters. People have tried to extend traditional numerical methods to this inverse problem [34-38], but these extensions are non- non-trivial to implement efficiently. There are similar attempts to inverse deep-lcarning PDE solvers. For example, one can inverse PINNs by optimizing the network parameters such that their outputs satisfy both the observed data and the governing equations. iFNO [39] and NIO [40] tries to extend FNO [3]. Other methods [41, 42] directly learn the operator functionn for the inverse problem. PINO [4] further combines neural operators with physics constraints to improve the performance of both forward and inverse problems. These methods assume full observations are available. To address the inverse problem with partial observations, people have tried to leverage generative priors with Graph neural networks [43, 8). These works have not demonstrated the ability to solve high-resolution PDEs, possibly limited by the power of generative prior. We want to leverage the state-of-the-ee generative model, diffusion models, to develop a better inverse PDE solver.
Diffusion models. Diffusion models have shown great promise in learning the prior with higher resolutions by progressively estimating and removing noise. Models like DDIM [44], DDPM [7], and EDM [45] offer expressive generative capabilities but face challenges when sampling with specific constraints. Guided diffusion models [46-49] enhance generation processes with constraints such as image inpainting, providing more stable and accurate solutions. Prior works on diffusion models for PDEs highlight the potential of diffusion approaches by generating PDE datasets such as 3D turbulence [50, 51] and Navier-Stokes equations [52] with diffusion models. Diffusion models can also be used to model frequency spectrum and denoise the solution space [53], and conditional diffusion models are applied to solve 2D flows with sparse observation [54]. However, the application of diffusion models to solve inverse problems under partial observation remains underexplored. In this work, we aim to take the initial steps towards addressing this gap.
3 Methods
3.1 Overview
To solve physics-informe forward and inverse problems under uncertainty, we start by pre-training a diffusion generative model on a family of partial differential equations (PDEs). This model is designed to learn the joint distribution of the PDE coefficients (or the initial state) and its corresponding solutions (or the final state). Our approach involves recovering full data in both spaces using sparse observations from either or both sides. We achieve this through the iterative denoising of random Gaussian noise as in regular diffusion models but with additional guidance from the sparse observations and the PDE function enforced during denoising. The schematic description of our approach is shown in Fig. 1.
3.2 Prelimary: Diffusion Models and Guided Diffusion
Diffusion models involve predefined forward process that gradually adds Gaussian noise to the data and a learned reverse process that denoises the data to reconstruct the original distribution. Specifically, Song et al. [55] propose deterministic diffusion model that learns an N-step denoising process that eventually outputs a denoised data N and satisfies the following ordinary differential equations (ODE) each timestep 4; where i (0, 1, ..., N -11 1}
dx = (1) pog a dt)
Here V. log p az; a (f) is the score function [56] that helps to transform samples from normal distribution .VO,, or (I) to a target probability distribution p(æ;; (()). To estimate the score function, Karras et al. [45] propose to learn a denoiser function D(æ; o)) such that
Va logp(x;; (t)) = (D(x; a(t) - xt)
(2)
To enable control over the generated data, guided diffusion methods [48] add guidance gradients to the score function during the denoising process. Recently, diffusion posterior sampling (DPS) [46]
3

---

Algorithm Sparse Observation and PDE Guided Diffusion Sampling Algorithm.
1: input DeterninisticSampler (x;x;rr) a (l4e TotalPointCount m, ObservedPointConnt
n, Observation y. PDEFunction f. Weights Sobs: Spde.
2: sample X (0, a IT)
Generate initial sampling noise
3: for E {0, ....,,,,..........
13 do
4: ''y f- Do (2;;o(tt))
b Estimate the denoised data at step t, Evaluate dæ ad/da(()) at step t; Take an Euler step from a (1,)) to a (14+11)
5:
d, x (2; -2,--i -xx) (a(tt)
6:
#141 x
7:
if # then
8:
Eyy + Do(= (((
Apply 2nd order correlation unless = 0 Evaluate dæ/ /do(() at step 4++1
9:
d; + (ai+) axx) /o((++)
10:
end Di+1 Ej + (a(tit1) (t:)) (2d: ]d))
Apply the trapezoidal rule at step 4
11:
12: Lobs f- 1 l -x'11
Evaluate the observation loss of s'y >> Evaluate the PDE loss of & Guide the sampling with Cobs and ppde.
12
13:
ppde f- 1 10 - f(
14: I1+1 Ii+1 Sobe Va, Lobe Spde Vae, Spde
15:
end for
16: return EN
Return the denoised data
made notable progress in guided diffusion for tackling various inverse problems. DPS uses corrupted measurements y derived from to guide the diffusion model in outputting the posterior distribution p(aly). A prime application of DPS the inpainting problem, which involves recovering a complete image from sparsely observed pixels, which suits well with our task. This approach modifies Eq. to
dæ = - o (t) (t) V a log p(2; (t)) + V logp(y x; a()) dt.
(3)
DPS [46] showed that under Gaussian noise assumption of the sparse measurement operator M(-), ie., u/ ~ ((M(x), 6'I) with some S.D. 5, the log- likelihoodd function can be approximated with:
1 8² Va.ly- M( (n, (1:))
Va log logopu (t+)) = Va, log Rix: (t:)) =
(4)
where a'v == (zx;; (t+)) denotes the estimation of the final denoised data each denoising step i. Applying the Baye's rule, the gradient direction of the guided diffusion is therefore:
Va, log p( (g)) ~ s(x;) - lly M(xx))1
(5)
where s(x) v log is the original score function, and Ç 1/82.
3.3 Solving PDEs with Guided Diffusion
Our work focuses on two classes PDEs: static PDEs and dynamic time-dependent PDEs. Static systems (e.g., Darcy Flow or Poisson equations) are defined by a time-independent function f: 02,
f(c; a, u) where Ω is bounded domain, c E Q is spatial coordinate, a E AA is the PDE coefficient field, and u E 2/ is the solution field. an is the boundary of the domain Q and u lan = g the boundary constraint. We aim to recover both a and u from sparse observations on either a or or both.
in Ω C IRd
u(c) = g
Similarly, we consider the dynamic systems (e.g., Navier-Stokes):
f(c,T:a, u) = 0, u(c,T) T) = -c, u(c, =0=00 a(c,)),
in f2 x (0, 00)
in 82 (0)
in Ω {0)
where 7 is temporal coordinate, un A is the initial condition, u is the solution field, and ul (0, = g is the boundary constraint. We aim to simultancously recover both and the solution ur == : T) at a specific time from sparse observations on either a, ur, or both.
Finally, we explore the recovery of the states across all timesteps Ug.7 in 1D dynamic systems governed by Burger's equation. Our network Da models the distribution all ID states, including the initial condition un and solutions ui.r stacked in the temporal dimension, forming a 2D dataset.

---

Obscrvations Coefficeons
Buth Confiisiens and Secotions
Soluttons
Truth
Obsrrnation
Roovertl Coeffriian
Roooerrs
Figure Different from forward and inverse PDE solvers, DiffusionPDE can take sparse observations either the coefficient the solution recover both of them, using trained network. Here, we show the recovered a and u the Darcy's 's eqaution given sparse observations on a, u, or both. that method successfully recovers the PDE in all cases.
Compared with the ground truth,
Guided Diffusion Algorithm In the data-driven PDE literature, the above tasks can be achieved by learning directional mappings between and (or uy for dynamic systems). Thus, existing methods typically train separate neural networks for the forward solution operator F A 2 and the inverse solution operator I U A.
Our method unifies the forward and inverse operators with single network and an algorithm using the guided diffusion framework. DiffusionPDE can handle arbitrary sparsity patterns with one pre-trained diffusion model D9 that learns the joint distribution A and Ut concatenated on the channel dimension, denoted X. Thus, our data E X, where X' == A x U. We follow the typical diffusion model procedures [45] to train our model on a family PDEs.
Once we train the diffusion model Da. we employ our physics physics-informed informed DPS [46] formulation during X' that satisfies the sparse observations and the given PDE, as
inference to guide the sampling detailed in Algorithm 1. We follow Eq. 5 to modify the score function using the two guidance terms:
V Va, log ((g, f) Vx, logp(z;)) Vob Cobs Spde Ve, Cpde:
(8)
where a; is the noisy data denoising step Yots the observed values, and f(.) 0 is the underlying PDE condition. Lots and fpde respectively represent the MSE loss of the sparse observations and the PDE equation residuals:
Labss Dobsi Do)
-lyobt 2'y
Yots (0z) (x;))))
11
n
(9)
Cpde f) 110 f( m
= m ΣΣ flc,,
where Φyy (o(x;) the clean image estimate denoising timestep i, which can be split into coefficient ú; and solution à; Here, m is the total number of grid points (i.c., pixels), n is the number of sparse observation points. o, represents the temporl coordinate jth observation. Note that, without loss of generality, Lpde can be accumulated for all applicable PDE function o in the system, and the time component тę ignored for static systems.
4 Experiments
4.1 PDE Problem Settings
We show the usefulness of DiffusionPDE across various PDEs for inverse and forward problems and compare it against recent bearning-based techniques. We test on the following families PDEs.
5

---

Grabed wib DeGemurte
Geleet 22 206 and Oiburraaee
Crouad Truls
Obamaatee Loss
and Cilurratio Lows
Curtccrr Abont
SALCE
Figure 3: Usefulness of PDE loss. We visualize the absolute errors of the recovered coefficient and solution of the Helmholtz equation with and w/o PDE loss. We compare having only the observation loss with applying the additional PDE loss. The errors drop significantly when using PDE loss.
Darcy Flow. Darcy flow describes the movement of fluid through a porous medium. In our experiment, consider the static Darcy Flow with no-slip boundary in
-V (a(c)Vuu)() = q(c), C
(10)
u(c) 0,
0Ω
Here the coefficient a has binary values. We set g(c) = for constant force. The PDE guidance function is thus f V (a(c)Vu(c)) q(c). Inhomogeneous Helmholtz Equation. We consider the static inhomogeneous Helmholtz Equation with no-slip boundary on Ön, which describes wave propagation:
vu(() + u²)( a(c), u(c) = 0,
E Ω
(11)
c E 0Ω
The coefficient is piecewise constant function and k is constant. Note 11 is the Poisson equation when k = 0. Setting = for Helmholtz equations, the PDE guidance function is = Vuu(c) + k²u(() a(c).
Non-bounded Navier-Stokes Equation. We study the non-bounded incompressive Navier-Stokes equation regarding the vorticity.
+ + T)= vAw(c, = +(C), C S2 E (0,
(12)
V v(((c,))= 0,
C E [0,7]
Here w = V x v the vorticity, v(c, r) is the velocity at c at time т, and g(c) is a force field. We 1000.
set the viscosity coefficient
10 and correspondingly the Reynolds number Re
DiffusionPDE learns the joint distribution of wi and wr and we take T = which simulates 1 second. Since T > 0, we cannot accurately compute the PDE loss from our model outputs. Therefore, given that V V-w(e, Bounded Navier-Stokes Equation. We study the bounded 2D incompressive Navier Stokes regarding the velocity and pressure p.
V (V x v) = O, we use simplified = V w((,
a,v(c,T) + Vu(c,T)
- = p V V = 0,
Ayp=vVPo(e.)
C Q2 (0,
(13)
CEQ,TE (0,7).
We set the viscosity coefficient 0.001 and the fluid density p= 1.0. We generate 2D cylinders of random radius at random positions inside the grid. Random turbulence flows in from the top of the grid, with the velocity field satisfying no-slip boundary conditions at the left and right edges, well as around the cylinder ft
DiffusionPDE. learns the joint distribution up and ur at T = 4, which simulates 0.4 seconds. Therefore, similarly use f V o(c, r) before.
Burgers' Equation. We study the Burgers' equation with periodic boundary conditions on a ID spatial domain of unit length !2 = (0, 1). We set the viscosity to v = 0.01. In our experiment, the initial condition uo has shape of 128
and we take 127 more time steps after the initial state to
form a 2D Uorr of size 128 x 128.
dcu(c, 7) (e(u2( (c, T)/2) v)c
C Q, (0,T)
(14)
u(c, 0) no(c).
CEΩ
We can reliably compute f dju(c, 7) + (u (c. r)/2) u(e 7) with finite difference since we model densely on the time dimension.

---

Table 1: Relative errors of solutions (or final states) and coefficients (or initial states) when solving forward and inverse problems respectively with sparse observations. Error rates are used for the inverse problem of Darcy Flow.
DiffusionPDE
PINO
DeepONet
PINNs
FNO
Forward Inverse
2.5%
35.2%
38.3%
48.8%
28.2%
Darcy Flow
3.2 %
49.2%
41.1%
59.7%
49.3%
Forward Inverse
4.5 %
107.1%
155.5%
128.1%
100.9%
Poisson
20.0% %
231.9%
105.8%
130.0%
232.7%
Forward Inverse
8.8%
106.5%
123.1%
142.3%
98.2%
Helmholtz
22.6% %
216.9%
132.8%
160.0%
218.2%
Non-bounded
Forward Inverse
6.9%
101.4%
103.2%
142.7%
101.4%
Navier-Stoker
10.4%
96.0%
97.2%
146.8%
96.0%
Bounded
Forward Inverse
3.9%
81.1%
97.7%
100.1 %
82.8%
Havier-Stokes
2.7%
69.5%
91.9%
105.5%
69.6%
4.2
Dataset Preparation and Training
We first test DiffusionPDE. jointly learning the forward mapping A U and the inverse given sparse observations. In our experiments, we define our PDE over the unit square ? (0, 1) which represent 128 x 128 grid. We utilize Finite Element Methods (FEM) to generate our training data. Specifically, we run FNO's [3] released scripts to generate Darcy Flows and the vorticities of the Navier-Stokes equation. Similarly, we generate the dataset of Poisson and Helmholtz using second-order finite difference schemes. To add more complex boundary conditions, use Difftaichi [57] to generate the velocities of the bounded Navier-Stoke equation. We train the joint diffusion model for each PDE on three A40 GPUs for approximately 4 hours, using 50,000 data pairs. For Burgers' equation, we train the diffusion model on dataset of 50,000 samples produced as outlined in FNO [3]. We randomly select 5 out of 128 spatial points on Q to simulate sensors that provide measurements across time.
mapping I: U
4.3 Baseline Methods
We compare DiffusionPDE with state-of-the-an caaming-based methods, including PINO [4], Deep- ONet [5], PINNs [6], and FNO [3]. However, note that none these methods show operation on partial observations. These methods can learn mappings between a and un and urr with full observations, allowing them to also solve the mapping between Ug and ur. PINNs map input a to output by optimizing combined loss function that incorporates both the solution and the PDE residuals. DeepONet employs a branch network to encode input function values sampled at discrete points and trunk network to handle the coordinates of the evaluated outputs. FNO maps from the parametric space to the solution space using Fourier transforms. PINO enhances FNO by integrating PDE loss during training and refining the model with PDE loss finetuning. We train all four baseline methods on both forward and inverse mappings using full observation of a or u for both static and dynamic PDEs. We tried training the baseline models partial observations, but noticed degenerate training outcomes (see supplementary for details). Overall, they are intended for full observations and may not be suitable for sparse measurements.
More closely related to our method, GraphPDE [8] demonstrates the ability to recover the initial state using sparse observations on the final state, task that other baselines struggle with. Therefore, bounded Navier-Stoker (NS) equation,
we compare against GraphPDE for the inverse problem which is the setup used in their report. GraphPDE uses a trained latent space model and bounded forward GNN model to solve the inverse problem with sparse sensors and thus is incompatible with unbounded Navier-Stokes We create bounded meshes using bounded grids to train the GNN model and train the latent prior with Vp.T for GraphPDE.
While we employ guided sampling to reconstruct the solutions, Classifier-Free Guidance (CFG) [58] offers an alternative approach where the diffusion model is conditioned on sparse input data. Shu et al. [54] extend this method by developing optimized CFG approach that conditions on the PDE loss,
7

---

Olservations
DillsiinPDE
DespOlil
PINO
Noz bounded Novier Stotes Equation
taiua Stoee
Forward
500 Rarslon
Ruaaive Relaive 103.5% Rebiive
Riccaiie 101.4% 142.%
biid Sute
Relutive Eine: 10:45 Relative Emon: $7.28 Relative Eine: 96.06 Relative Binu; 96.0% Relative Eine: 1468%
Bargees Lquatten
Bandem Samsor
Xelaiive Timor: 2.60% Rebative Emorr 92.0% Relative Emor 63.7% Relative Timer 90.02% Relative Emor 40.4%
Figure 4: We compare DiffusionPDE with fatte-of-the-art neural PDE solvers [3-6]. In the forward Navier-Stokes problem, we give 500 sparse observations of the initial state to solve for the final state. In the inverse set-up, take observations of the final state and solve for the initial. For the Burgers' equation, we use sensors throughout all time steps and want to recover the solution at all time steps. Note that train on neighboring snapshot pairs for the baselines in order add continuous observations of the Burgers' equation. Results show that existing methods do not support PDE solving under sparse observations, and believe they are not easily extendable do We refer readers to the supplementary for complete set of visual results.
using the observation low-resolution input. Additionally, OFormer [59] is another model designed to reconstruct the full solution using transformers, offering shorter inference runtime. Consequently, we compare our approach against these methods for solving the unbounded Navier-Stoke equation.
4.4 Main Evaluation Results
We respectively address the forward problem and the inverse problem with sparse observations of For the forward problem, randomly select coefficients (initial states) sparse observations and then compare the predicted solutions (final states) with the ground truth. Specifically, we select 500 out of 128 x 128 points, approximately 3%, the coefficients of Darcy Flow, Poisson equation, Helmholtz equation, and the initial state of the non-bounded Navier-Stokes equation. For the bounded Navier-Stokes equation,
1% observed points beside the boundary of the cylinder in 2D. Similarly, for the inverse problem, we randomly sample points on solutions (final states) as sparse observations, using the same number of observed points in the forward model for each PDE.
We show the relative errors of all methods regarding both forward and inverse problems in Table 1. Since the coefficients of Darcy Flow are binary, evaluate the error rates of prediction. Non-binary data is evaluated using mean pixel-wise relative error. We report error numbers aver- aged across 1,000 random scenes and observations for each PDE. DiffusionPDE outperforms other methods including PINO [4], DeepONet [5], PINNs [6], and FNO [3] for both directions with sparse observations, demonstrating the novelty and uniqueness of our approach. For the inverse problems of the Poisson and Helmholtz equations, DiffusionPDE exhibits higher error rates due to the insufficient constraints within the coefficient space, produced from random fields. In Fig. 4, we visualize the results for solving both the forward and inverse problem of the non-bounded Navier-Stokes. We refer the supplementary for additional visual results. While other methods may produce partially correct results, DiffusionPDE outperforms them and can recover results very close to the ground truth.
For the inverse problem of the bounded Tavier-Stokes equation, we further compare DiffusionPDE with GraphPDE, as illustrated Fig. 5. Our findings reveal that DiffusionPDE surpasses GraphPDE [8] in accuracy, reducing the relative error from 12.0% to 2.7% with only 1% observed points.
further show whether DiffusionPDE can jointly recover both a and u by analyzing the retrieved and with sparse observations on different sides as well as both sides. In Fig. 2, we recover the coefficients and solutions Darcy Flow by randomly observing 500 points on only coefficient space, only space solution space, and both. Both coefficients and solutions can be recovered with low emrors

---

Gruza Truth
Uherrved Final State
Calinder
GopaPDB
DiftsiioPD0
Croand Touhh
Frina State
19 Rasdom
Relative
Reluive Broor 27%
Figure 5: We compare GraphPDE [8] and our method for solving the inverse bounded Navier-Stokes equation. Given the boundary conditions and 1% observations of the final vorticity field, solve the initial vorticity field. We set the fliuds to flow from the top, with boundary conditions the edges and middle cylinder. While GraphPDE can recover the overall pattern of the initial state, it suffers from noise when the fluid passes the cylinder and misses the high vorticities at the bottom.
for each situation. We therefore conclude that DiffusionPDE can solve the forward problem and the inverse problem simultaneously with sparse observations any side without retraining our network.
Advantage of Guided Sampling
To demonstrate the clear advantage of our guided sampling method, evaluate both the forward and inverse processes of the unbounded Navier-Stokes equation, comparing our DiffusionPDE approach with Diffusion using CFG when considering only the initial and final states given 500 observation points, as illustrated in Fig. 6. Our DiffusionPDE method consistently achieves lower relative errors across both evaluations.
Furthermore, in Fig. 7, we compare our results with those of Shu et al. [54], where the full time intervals solved autoregressively using optimized CFG method. In their approach, the error in the final state increases to approximately 13%, which notably higher than that of our two-state model. Additionally, the relative errors of the transformer-basee approach, OFormer [59], are around 17% and 23%, which are significantly larger than those observed with DiffusionPDE.
4.6 Recovering Solutions Throughout Time Interval
We demonstrate that DiffusionPDE capable retrieving all time steps throughout the time interval [0, 7]] from continuous observations sparse sensors. To evaluate its ability to recover up: with sparse sensors, we study the ID dynamic Burgers' equation, where DiffusionPDE learns the distribution of uo:r using 2D diffusion model. To apply continuous observation DeepONet, FNO, and PINNs, we train them on neighboring snapshot pairs. Our experiment results in test relative error of 2.68%. depicted in Fig. 4, which is significantly lower than other methods.
PINO,
4.7 Additional Analysis
We examine the effects of different components of our algorithm such as PDE loss and observation samplings. We strongly encourage readers to view the supplementary for more details of these analyses well additional experiments.
PDE Loss. To verify the role of the PDE guidance loss of Eq. during the denoising process, we visualize the ervors of recovered a and of Helmholtz equation with or without PDE loss. Here, we run our DPS algorithm with 500 sparse observed points on both the coefficient and solution u and study the effect of the additional PDE loss guidance. The relative error of reduces from 9.3% to 0.6%, and the relative error of a reduces from 13.2% to 9.4%. Therefore, we conclude that PDE guidance helps smooth the prediction and improve the accuracy.
Number of Observations. We examine the results of DiffusionPDE in solving forward and inverse problems when there are 100, 300, 500, and 1000 random observations on a, u. or both and u. The error of DiffusionPDE decreases as the number sparse observations increases. DiffusionPDE is capable of recovering both and with errors 1% 10% with approximately 6% observation points any side for most PDE families. DiffusionPDE becomes insensitive the number of observations and can solve the problems well once more than 3% of the points are observed.

---

DiflusioPDD
Diffusion
Initia State
Forward
Piann State
Fooward
Random: Points
Relative Emor: 6.9%
Relative Emor: 9.6%
Fitzal State
Enveess
State
Relutive Eroor: 10.4% Relative Emor: 17.4%
Figure 6: We compare the performance of DiffusionPDE and Diffusion with CFG for the unbounded Navier-Stokee equation, and visualize the error. With 500 observation points, DiffusionPDE demon- strates superior accuracy, achieving lower emrors in both forward and inverse problem-solvingg
Observations
Ground Trrsh
DiffesicPPD
Sha et al.
OFoomer
Iniial State
Finil State
Forward
500 Randum Puints
Relative Errn: 6.9%
Rebiivv Erner: 12.8%
Relative Emron: 16.2%
Finui State
Inverse
Initil Stite
Relative Eroor 10.4% Relative Emor: 18.6% Relazive Error: 23.5%
Figure 7: We compare our DiffusionPDE method with the approaches of Shu et al. [54] and OFormer [59] for the unbounded Navier-Stoke equation. Using 500 observation points, DiffusionPDE effectively solves both the forward and inverse problems, achieving significantly lower errors.
Observation Sampling Pattern. While CFG struggles with robustness, we show that DiffusionPDE robust different sampling patterns of the sparse observations, including grid and non-uniformly concentrated patterns. Note that even when conditioned on the full observations, our approach performs on par with the current best methods, likely due the inherent resilience of our guided diffusion algorithm. Additionally, DiffusionPDE can leverage continuous coordinates with bilinear interpolation in the prediction space obtain predicted values for points that do lie directly on the grid, without compromising accuracy.
5 Conclusion and Future Work
In this work, we develop DiffusionPDE: a diffusion-basee PDE solver that addresses the challenge of solving PDEs from partial observations by filling in missing information using generative priors. We formulate a diffusion model that learns the joint distribution the coefficient (or initial state) space and the solution (or final state) space. During the sampling process, DiffusionPDE. generate plausible data by guiding its denoising with sparse measurements and PDE constraints. Our new approach leads to significant improvements toward a general PDE-solving framework that leverages the power of generative models.
flexibly
existing state-of- feft--hann art methods, advancing
Several promising directions for future research have emerged from this work. Currently, Diffusion- PDE limited to solving slices of 2D dynamic PDEs; extending its capabilities to cover full time intervals of these equations presents significant opportunity. Moreover, the model's 's struggle with accuracy in spaces that lack constraints another critical area for exploration. DiffusionPDE also suffers from slow sampling procedure, and faster solution might be desired.
10

---

References
[1] Ferri MH Aliabadi. Boundary element methods. In Encyclopedia of continuum mechanics, pages 182-193. Springer, 2020.
[2] Pavel Solín. Partial differential equations and the 2005.
the finite element method. John Wiley & Sons,
[3] Zongyi Li, Nikola Kovachki, Kamyar Arizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differen- tial equations. arXiv preprint arXiv:2010.0889) 2020.
[4] Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. thyyics-informee neural operator for learning partial differential equations. ACM/JMS Journal of Data Science, 2021.
[5] Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, (00:218-2299 2021.
[6] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: dcep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
[7] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[8] Qingqing Zhao, David B Lindell, and Gordon Wetzstein. Learning to solve pde-constrained inverse problems with graph networks. arXiv preprint arXiv:2206.00 00711, 2022.
[9] Lawrence C Evans. Partial differential equations, volume 19. American Mathematical Society, 2022.
[10] Kenji Omori and Jun Kotera. Overview of pdes and their regulation. Circulation research, 100(3):30--327, 2007.
[11] Ryan Po, Wang Yifan, Vladislav Golyanik, Kfir Aberman, Jonathan T Barron, Amit H Bermano, Eric Ryan Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, et al. State of the art on diffusion models for visual computing. arXiv preprint arXiv:2310.07200 2023.
[12] Walter A Strauss. Partial differential equations: An introduction. John Wiley & Sons, 2007.
[13] Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys, 56(4):1-39) 2023.
[14] Alfio Quarteroni and Alberto Valli. Numerical approximation of partial differential equations, volume 23. Springer Science & Business Media, 2008
[15] Sergio R Idelsohn, Eugenio Onate, Nestor Calvo, and Facundo Del Pin. The meshless finite element method. International Journal for Numerical Methods in Engineering, 58(6):893-912, 2003.
[16] Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics informed deep learn- ing (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint 1711v:1711.10661 2017.
[17] Shengze Cai, Zhiping Mao, Zhicheng Wang, Minglang Yin, and George Em Karniadakis. Physics-informed neural networks (pinns) for fluid mechanics: A review. Acta Mechanica Sinica, 37722):177-11738 2021.
[18] Zhiping Mao, Ameya D Jagtap, and George Em Kariiadakis. Physics-informed neural networks for high-speed flows. Computer Methods in Applied Mechanics and Engineering, 360: 12789, 2020.
11

---

[19] Hamidreza Eivazi, Mojtaba Tahani, Philipp Schlatter, and Ricardo Vinuesa. Physics-informed neural networks for solving reynolds-averee navier-stokes equations. Physies of Fluids, 34(7), 2022.
[20] Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, and George Em Karniadakis. Physics-informed neural networks for heat transfer problems. Journal of Heat Transfer, 143(6))06080,, 2021.
[21] George S Misyris, Andreas Venzke, and Spyros Chatzivasileiadis. Physics-informed neural networks for power systems. In 2020 IEEE power & energy society general meeting (PESGM), pages 1-5, IEEE, 2020.
[22] Zongyi Li, Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Moham- mad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, et al. Geometry-informed neural operator for large-scale 3d pdes. Advances in Neural Information Processing Systems, 36, 2024.
[23] Louis Serrano, Lise Le Boudec, Armand Kassai Koupai, Thomas X Wang, Yuan Yin, Jean-Noël Vittaut, and Patrick Gallinari. Operator learning with neural fields: Tackling pdes general geometries. Advances in Neural Information Processing Systems, 36, 2024.
[24] Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M Benson. U-fnno-an an enhanced fourier neural operator- baseed deep-learning model for multiphase flow. Advances in Water Resources, (63:104180 2022.
[25] Fanny Lehmann, Filippo Gatti, Michaël Bertin, and Didier Clouteau. Fourier neural operator surrogate model to predict 3d seismic waves propagation. arXiv preprint 2rX1v:2304.10242, 2023.
[26] Bian Li, Hanchen Wang, Shihang Feng, Xiu Yang, and Youzuo Lin. Solving seismic wave equations variable velocity models with fourier neural operator. IEEE Transactions Geoscience and Remote Sensing, 61:1-18, 2023.
[27] Zhijie Li, Wenhui Peng. Zelong Yuan, and Jianchun Wang. Fourier neural operator approach large eddy simulation of three-dimensional turbulence. Theoretical and Applied Mechanics Letters, 1006), 2022.
[28) Wenhui Peng, Zelong Yuan, Zhijie Li, and Jianchun Wang. Linear attention coupled fourier neural operator for simulation of three-dimensional turbulence. Physics of Fluids, 35(1), 2023.
[29] Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical fourier neural operators: Learning stable dynamics on the sphere. In International conference on machine learning, pages 2806-2823. PMLR, 2023.
[30] Peter Yichen Chen, Jinxu Xiang, Dong Heon Cho, Yue Chang, GA Pershing, Henrique Teles Maia, Maurizio M Chiaramonte, Kevin Carlberg, and Eitan Grinspun. Crom: Continu- ous reduced-ordes modeling of pdes using implicit neural representations. arXiv preprint arXiv:2206.0260 2022.
[31] Zilu Li, Guandao Yang, Xi Deng, Christopher De Sa, Bharath Hariharan, and Steve Marschner. Neural caches for monte carlo partial differential equation solvers. In SIGGRAPH Asia 2023 Conference Pupers, pages 1-10, 2023.
[32] Thomas Müller, Fabrice Rousselle, Jan Novák, and Alexander Keller. Real-time neural radiance caching for path tracing. arXiv preprint arXiv:2 /2372, 2021.
[33] Hong Chul Nam, Julius Berner, and Anima Anandkumar. Solving poisson equations using neural walk-on-spheres. In ICLR 2024 Workshop on A44iifferentaalaauttoon Science, 2024.
[34] M Cho, Jadamba, Kahler, AA Khan, and M Sama. First-order and second-order adjoint methods for the inverse problem of identifying non-linear parameters in pdes. Industrial Mathematics and Complex Systems: Emerging Mathematical Models, Methods and Algorithms, pages 147-163, 2017.
12

---

[35] Colin Fox and Geoff Nicholls. Statistical estimation of the parameters of a pde. Can. appl. Math. Quater, 10:277-810, 2001.
[36] Bastian Harrach. An introduction finite element methods for inverse coefficient problems in elliptic pdes. Jahresbericht der Deutschen Mathematiker- Vereinigung, (23(3):183-210 2021.
[37] Krishna Kumar and Yonjin Choi. Accelerating particle and fluid simulations with differentiable graph networks for solving forward and inverse problems. In Proceedings of the SC'23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, pages 60 65, 2023.
[38] Tristan van Leeuwen and Felix Herraann. A penalty method for pde-constrained optimization in inverse problems. Inverse Problems, 32(1) (0(())015007,,.0 2015.
[39] Da Long and Shandian Zhe. Invertible fourier neural operators for tackling both forward and inverse problems. arXiv preprint Xix::24422.11222 2024.
[40] Roberto Molinaro, Yunan Yang, Björn Engquist, and Siddhartha Mishra. Neural inverse operators for solving pde inverse problems. arXiv preprint arXiv:2301. 2023.
[41] Maarten V de Hoop, Matti Lassas, and Christopher A Wong. Deep learning architectures for nonlinear operator functions and nonlinear inverse problems. Mathematical Statistics and Learning, 4(1): 86, 2022.
[42] Samira Pakravan, Pouria A Mistani, Miguel A Caagon-Calvo, and Frederic Gibou. Solving inverse-pde problems with physics-aware neural networks. Journal of Computational Physics, 440:1 10414, 2021.
[43] Valerii lakovlev, Markus Heinonen, and Harri Lähdesmäki. Learning continuous time pdes from sparse data with graph neural networks. arXiv preprint arXiv:2006.00 00w:20000.88566, 2020.
[44] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
[45] Tero Karras, Mika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In Proc. NeurIPS, 2022.
[46] Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Mare L Klasky, and Jong Chul Ye. Diffu- sion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:22009 14687, 2022.
[47] Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 35:25683-25696, 2022.
[48] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794 2021.
[49] Sihan Xu, Yidong Huang, Jiayi Pan, Ziqiao Ma, and Joyce Chai. Inversion-free image editing with natural language. arXiv preprint arXiv:2312.04965. 2023.
[50] Christian Jacobsen, Yilin Zhuang, and Karthik Duraisamy. Cocogen: conditioned score-based generative models for forward and inverse problems. arXiv preprint arXiv:2312. 10527, 2023.
consistent and
[51] Marten Lienen, David Lüdke, Jan Hansen- Palmuu, and Stephan Günnemann. From zero to turbulence: Generative modeling for 3d flow simulation. In The Twelfth International Conference on Learning Representations, 2023.
[52] Gefan Yang and Stefan Sommer. A denoising diffusion model for fluid field prediction. arXiv preprint arXiv:2301. 11.00.11661,,. 2023.
[53] Phillip Lippe, Bas Veeling, Paris Perdikaris, Richard Turner, and Johannes Brandstetter. Pde- refiner: Achieving accurate long rollouts with neural pde solvers. Advances in Neural Informa- tion Processing Systems, 36, 2024.
13

---

[54] Dule Shu, Zijie Li, and Amir Barati Farimani. A physics-informed diffusion model for high- fidelity flow field reconstruction. Journal of Computational Physics, 478:111972, 2023.
[55] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[56] Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal Machine Learning Research, 6(4), 2005.
[57] Yuamming Hu, Luke Anderson, Tzu-Mao Li. Qi Sun, Nathan Carr, Jonathan Ragan- Kelley, and Frédo Durand. Difftaichi: Differentiable programming for physical simulation. arXiv preprint arXiv-1910.00955, 2019.
[58] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. 22N.:2200.//5588. 2022.
arXiv preprint
[59] Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations' operator learning. arXiv preprint arXiv:2205.13671, 2022.
[60] Evan F Bollig, Natasha Flyer, and Gordon Erlebacher. Solution to pdes using radial ba- sis function finite-difference (rbf-fd) multiple gpus. Journal of Computational Physics, 231(21):733-7155, 2012.
14

---

Appendix
A Overview
In this supplementary material, we provide additional details to complement the main paper. Section B elaborates on the data generation process. Section C outlines the sampling implementation, and Section D highlights error reductions achieved by integrating PDE loss. Section E presents comprehensive visual results for both forward and inverse computations using sparse observations, which are not included in the main text. In Section F, we discuss results from full observation scenarios across all methods. Section G justifies observation data, while Section H shows results from optimized baseline methods. Section I and J provide standard deviation and runtime analyses, and Section K examines the model's robustness against random noise and varying observation locations, as well as the stochasticity of the model. Section L. and M explores how different observation numbers and resolutions affect result accuracy. offering further insight into the model's performance under varying conditions. Lastly, Section N compares DiffusionPDi with additional baseline methods, including RBF kernel and U-Net.
decision to train the baselines complete
B Data Generation Details
We generate 50,000 samples for each PDE and all diffusion models are trained on Nvidia A40 GPUs.
B.1 Static PDEs
We derived the methods of data generation for static PDEs from [3]. We first generate Gaussian
random fields on (0, 1)² so that (O, (-A + 9I) ()) For Darcy Flow, we let a f(u) so that:
[a(x) 12, if p(x))) 2
[a(x) = 3, if p()) <
For the Poisson equation and Helmholtz equation, we let - as the coefficients. We then use second-order finite difference schemes solve the solution u and enforce the no-slip boundary condition for solutions by multiplying mollifier sin (in() for point x = (x1, 22) E (0. Both and have resolutions of 128 x 128.
B.2 Non-bounded Navier-Stokes Equation
We derived the method to generate non-bounded Navier-Stokes equation from [3]. The initial condition wp is generated by Gaussian random field field (/00, (09 The forcing function follows the fixed pattern for point (2);
1
(x)) = 1/ (sin(2 + x2)) + cos(2 x2)))
10
We then use the eseudo-spectral method to solve the Navier-Stoker equations in the stream- stream-function formulation. We transform the equations into the spectral domain using Fourier transforms, solving the vorticity equation in the spectral domain, and then using inverse Fourier transforms to compute nonlinear terms in physical space. We simulate for I second with 10 timesteps, and wy has resolution of 128 x 128.
B.3 Bounded Navier-Stokes Equation
We use Difftaichi [57] generate data for the bounded Navier-Stokes equation. Specifically, we apply the Marker-and-Cell (MAC) method by solving a pressure-Poisson equation to enforce incompressibility and iterating through predictor and corrector steps update the velocity and pressure fields. The grid is of the resolution 128 x 128 and the center of the cylinder is at a random location in [30, 60] x [30, 90] with a random radius [5, 20). The fluid flows into the grid from the upper boundary with a random initial vertical velocity in [0.5, 3). We simulate for I second with 10 timesteps and study steps
when the turbulence is passing the cylinder.
B.4 Burgers' Equation
We derived the method to generate Burgers' equation from [3]. The initial condition is generated by Gaussian random field (O, 625(-A + 25I)- We solve the PDE with a spectral method and simulate 1 second with 127 additional timesteps. The final up.T space has a resolution of 128 x 128.
15

---

C Guided Sampling Details
For experiments with sparse observations sensors, find that DiffusionPDE performs the best when weights Ç are selected as shown Table During the initial 80% of iterations in the sampling process, guidance is exclusively provided the observation loss Lohs. Subsequently, after 80% of the iterations have been completed, we introduce the PDE loss Lpde; and reduce the weighting factor Sobs for the observation loss, by factor of 10. This adjustment shifts the primary guiding influence to the PDE loss, thereby aligning the diffusion model more closely with the dynamics governed by the partial differential equations.
Table The weights assigned the PDE loss and the observation loss vary depending on whether the observations pertain to the coefficients (or initial states) to the solutions (or final states)
Darcy Flow
Poisson
Helmholtz
Non- bounded Bounded
Burgers' equation
Navier-Stokes
Navier-Stokes
Sobe
a
2.5 x 103
x 102
x 10
5 x 102
2.5 x 10
3.2
10a
106
x 10a
x 10a
5 x 102
2.5 x 102
Spde
103
10a
10a
10a
102
10a
D Improvement in Prediction through PDE Loss Term
DiffusionPDE performs better when we apply the PDE loss term Lyde in addition to the observation loss term Lobs as guidance, as shown in Table The errors in both the coefficients (initial states) and the solutions (final states) significantly decrease. We also visualize the recovered a and u and corresponding absolute errors Darcy Flow, Poisson equation, and Helmholtz equation in Fig. 8. is demonstrated that the prediction becomes more accurate with the combined guidance of PDE loss and observation loss than with only observation loss.
Table DiffusionPDE' prediction errors coefficients (initial states) a and solutions (final states) with sparse observation on both and u, guided by different loss functions.
Loss Function
Side
Darcy Flow
Poisson
Helmholtz
Non-bounded Navier-Stokes
Bounded Navier-Stoke
Lobs
a
4.6%
12.1%
13.2%
8.2%
6.4%
4.8%
6.5%
9.3%
7.6%
3.3%
3,4%
10.3%
9.4%
4.9%
1.7%
Lobo + fapde
u 1.7%
0.3%
0.6%
0.6%
1.4%
with
Guilded with Olservation
Guided with PDE Loss and Oiverssiion
Observation
Cocfficient Absulute Eloo
Recovered Coetiiiiis
Error
Emer Rate: 2.6%
Recovered Sotutors
Sobaiinn Absoluic Elon
Relutive
(a) Recovered cocfficients, solutions, and corresponding absolute of Darey Flow.
16

---

Guiddel with Observosion Lees
Guided Observatio Loss
Guided and Observation Less
Ubseevatin
Ground
Rocovered Cooffictons
Coeffiiiis: Absolute
Rebetive Ecrur: 12.15
Relative Eneer 10.3%
Recovered Solzzons
Relative Eroor 6.5%
Relative Erse: 0.3%
(b) Recovered coefficients, solutions, and corresponding absolute emrors of Poisson equation.
Guided wits Observation Loos
Grided with PDE Loss and Observaton Loss
Grided with Observziio Loss
Guided with Loss and Observation Loss
Ground Truth
Recovered Coetficinss
Coofieteon Absohen
Relative Emor: 13.2%
Releive Emoor 9,4%
Recovered Solisoon
Solution Abolter
Relative Emue 9.3%
Relative Emor: 6.6%
(c) Recovered coefficients, solutions, and corresponding absolute errors of Helmholtz equation.
Figure 8: Recovered coefficients, solutions, and their corresponding visualized absolute errors for
various PDE families.
E
Additional Results on All PDEs with Sparse Observation
We present the recovered results of another Burgers' equation in Fig. 9. DiffusionPDE outperforms all other methods with 5 sensors for continuous observation. We also present the recovered results for both the forward and inverse problems of all other PDEs with sparse observations, as shown in Fig. 10. Specifically, we solve the forward and inverse problems for the Darey Flow, Poisson equation, Helmholtz equation, and bon-bounded Navier-Stokes equation using 500 random points observed in either the solution space or the coefficient space. Additionally, for the bounded Navier-Stoker equation, we observe 1% of the points in the velocity field. Our findings indicate that DiffusionPDE outperforms all other methods, providing the most accurate solutions.
To further demonstrate the generalization capability of our model, we conducted additional tests on different data settings for Darcy Flow. In Fig. 11, we solve the forward and inverse problems of Darcy Flow with 500 observation points, adjusting the binary values of a to 20 and 16 instead the original 12 and in Section B, i.e..
Additional Data Setting for Darcy Flow
[a(x) 20, ((xx) = 16,
if (()))
((x)) <
Our results indicate that DiffusionPDE performs equally well under these varied data settings, showcasing its robustness and adaptability.
17

---

Obscratton
Truls
DifteiinDEE
DeepUNer
Bunden Scroees
1ine
Emer:
Figure 9: Results of another Burgers' equation recovered by 5 sensors throughout the time interval.
Utservations
Ground Truth
DitfisiooPDE
DoepONar
ENO
PINNs
Forward
Solitios
Peims
Firor: Reletive Froor
Solntonn
laverse
Coettioiaas
Erre Rote: 3.2%
Ero Rue 41.19
Eror Ruze 49.2%
Erix Rate: 49.3%
Ernx Rate: 29.7%
(a) Forward inverse results of Darey Flow recovered by 500 observation points.
Observaions
Ground Trubh
DefrsioaPDE
DeepCiNet
PINO
PNO
PINN
Controen
Solation
500 Random Points
Relaiis Emcer 4.5% Reislive Emun 155.5% Relative Emer: 107.1% Relative Ervor: 100 Rebrive Emor 128,
Cheffiieet
Relotive Brnr: 2005 Relative Emor 105 8S Relstvve 9% Relative Binor 283 Rebrive 1300%
(b) Forward and inverse results of Poisson equation recovered by 500 observation points.
Orserovione
Ground Trub
DefraionDD
DoppONa
PINO
PNO
PINNS
Uoetooens
Ferward
Soluainna
(eRelaie
Chefitaiants
Inveze
Relative Rehutiv Emur:
(c) Forward and inverse results Helmholtz equation recovered by observation points.
Obscrvations
DiffusioNOOE
DeepONai
PINN:
Innall Staee
State
200 Randon Poirs
Relative Bror. 5.7% Relative Broo. 100.1% Relative Broc: 99.5% Relative Enor 99.1% Relutive Enor. 141.1%
Inverner
Iniiaa State
Relutive Eror: 0.5% Rebsive Froo 96.0% Relaive Fioe 08.2%, Reletive Frree: 98.1% Rehuivve Eror 7%6
(d) Forward and inverse results of another non-bounded Navier-Stokes equation recovered by 500 observation points.
18

---

Obscrvations
Cround Trath
DiffusionPDE
DeepONei
Inital State
Ferword
Fitol State
1S Randeai Poins
Rolative Enor. 3.9%
Relottvve Iuron 97216
Relative Broor $1.1% Relative Enor. $2.549 Relative Eror. 100.15
Pina State
Inverne
lobis State
Reisivve Fioon 2.76
Reluiive Frror: 01.9%, Relatve Fimor 60.5% Relslive Frmm: 60.6% Rebrive Enoor 105.
(c) Forward and inverse results of bounded Navicr-Stokes equation recovered by 1% observation points.
Ground Trush
Observaa Final State
Cylinder Boundey
Goyprtte:
DXffesoowPE::
Ground Tnrth
Fital Store
Invaru
Initall Stze
Ruxdin Points
Relative Enor. 10.5%
Relative Emor. 5.6%
(f) Inverse results of DiffusionPDE: and GraphPDE of another hounded Navicr-Stokes equation recovered by 1% observation points and the known boundary of the cylinder.
Figure 10: Results forward and inverse problems for different PDE families with sparse observation.
Observad Coefficient
Recovered Solutions
Ground Truth
Ohserved Solutions
Recoverod Solutions
Groond Truth
20
Forwand
Tevene
500 Rundom Points
Relative Eroor: 2.0%
500 Random Points
Eruur Rate: 4,9%
Figure 11: Forward and inverse results of Darcy Flow recovered by 500 observation points under different data setting.
DiTusionPDE
Ground Truth
DiftisiouPDE
Ground Troth
DiflesionPDE
Ground Truth
Naviee- Navie--Sood Stokes (Forward)
Navicr Navicr-Stokee Stokke (Invens))
Burgers Equation
Relative Enro: 7.7%
Relative Eroe: 6.8%
Relative Eror 8.1%
Figure 12: Results of Navier-Stokk Stokes equation and Burgers' equation with 10 times smaller viscosity.
Additional Data Setting for Non-bounded Navier-Stokes Equation and Burgers' Equation We also test DiffusionPDE the Burgers' equation with viscosity of x 10-3 and on the bon-bounded Navier-Stokes equation with viscosity of 1 x 10 which are 10 times smaller than the ones in the main paper, shown in Fig. 12. For the Burgers' equation, we are able to recover the full time interval with 5 fixed sensors at a relative error of approximately 6%, which is close to the error of approximately 5% in the main paper. For the Navier-Stokes equation, and inverse problems with relative eиrors of approximately 7% and 9%, respectively, using 500 observation points. The errors are also close the ones the main paper, where the forward and inverse errors of Navier-Stokes equation are approximately 7% and 10%.
can solve the forward
Solving Forward and Inverse Problems with Full Observation
We have also included the errors of methods when solving both the forward and inverse problems with full observation, as displayed in Table 4.
19

---

Table 4: Relative errors of solutions (or final states) and coefficients (or initial states) when solving forward and inverse problems with full observations. Error rates are used for the inverse problem of Darcy Flow.
DiffusionPDE
PINO
DeepONet
PINNs
FNO
Forward Inverse
2.2%
4.0%
12.3%
15.4%
5.3%
Darey Flow
2.0 %
2.1%
8.4%
10.1%
5.6%
Forward 2.7%
3.7%
14.3%
16.1%
8.2%
Poisson
Inverse
9.8%
10.2%
29.0%
28.5%
13.6%
Forward Inverse
2.3 4.0% %
4.9%
17.8%
18. 1%
11.1%
Helmholtz
4.9%
28.1%
29.2%
5.0%
Non-bounded Navier-Stokes
Forward 6.1%
1.1%
25.6%
27.3%
2.3%
Inverse
8.6%
6.8%
19.6%
27.8%
6.8%
Bounded
Forward 1.7%
1.9%
13.3%
18.6%
2.0%
Navier-Stokes
Inverse
1.4%
2.9%
6.1%
7.6%
3.0%
In general, DiffusionPDE and PINO outperform all other methods, and DiffusionPDE performs the best for all static PDEs. DiffusionPDE is capable of solving both forward and inverse problems with errors of less than 10% for all classes of discussed PDEs and is comparable to the state-of of-the-art. Results of all methods regarding Darcy Flow and non- non-bounded Navier-Stoker equation included in Fig. 13.
Obscrvations
Grunnd Trsh
DifleisPDD
DespONis
PINO
IND
PINN
Coetiicenss
Foeward
Sotusass
Relaive
Relutive
3.09 Erun:
Sotztious
Inverse
Coetiooeess
Eroy Rony: 0.71
Eroo Rave: 9.5%
Limor Ratn: 2.1%
Eroe Rate: 5.5%
Enerr Rate: 10
(a) Forward inverse results Darcy Flow recovered by full observation.
Obscrvations:
Geound Tush
DiflaisoPDE
DeepONa
ENO
PINNS
Initill sizze
Ferward
Floal State
Pull Observation
Relitive Bror. 3.25
Relitive Enor. 23.4%
Relative Broc: LIS
Relative Brooe 1.2%
Relative Erior. 26.39
Pinal State
Inverne
lobtal State
Rebeive Froe:: 3-4%, Rebeiiv Firor: 24.1% Robrive Froor 3.6% Relative From: 3.7% Rebeive Froor 20.05
(b) Forward and inverse results of non-bounded Navier-Stokes equation recovered by full observation.
Figure 13: Results of forward and inverse problems for different PDE families with full observation.
G Training Baselines Methods on Partial Inputs
For our main experiments, we opt to train the baseline models (PINO, DeepONet, PINNs, FNO) on full observations for several compelling reasons: First, physics-informed models such as PINNs and PINO are unable effectively compute the PDE when only sparse observations are available. Second, other models like DeepONet and FNO perform poorly with sparse observations. For instance,
20

---

Table Relative errors of solutions (or final states) and coefficients (or initial states) when solving forward and inverse problems respectively with sparse observations after optimizing the baselines. Error rates are used for the inverse problem Darcy Flow.
DiffusionPDE
DeepONet
PINO
FNO
PINNs
Forward Inverse
2.5%
31.3%
32.6%
27.8%
6.9%
Darcy Flow
3.2%
41.1%
49.2%
49.3%
59.7%
Forward Inverse
4.5%
73.6%
79.1%
70.5%
77.8%
Poisson
20.0%
75.0%
115.0%
118.5%
73.9%
Forward Inverse
8.8
77.6%
67.7%
84.8%
79.2%
Helmholtz
22.6%
100.7%
125.3%
131.6%
103.7%
Non-bounded
Forward Inverse
6.9%
96.5%
93.3%
91.6%
106.1%
Navier-Stokes
10.4%
71.9%
87.8%
89.3%
108.6%
Bounded
Forward 3.9%
89.1%
80.8%
81.2%
84.4%
Navier-Stokes
Inverse
2.7%
88.6%
47.3%
48.7%
82.1%
training the DeepONet model on 500 uniformly random points for each training sample in the context of the forward problem of Darcy Flow leads to testing outcomes that are consistently similar, as illustrated in Fig. 14, regardless of the testing input. This pattern suggests that the model tends generate a generalized solution that minimizes the average error across all potential solutions rather than converging based specific samples. Furthermore, the purtilliinput-rranee model exhibits poor generalization when faced with different distribution of observations from training, indicating that it lacks flexibility-4a critical attribute of our DiffusionPDE.
500 Observed Points
1000 Observed Points
Full Observation
Figure 14:
Predicted solutions obtained using the DeepONet model trained with 500 observation points across different numbers of observation points.
H Baseline Optimization
We further refine the noisy outputs generated by baseline methods such DeepONet, PINO, FNO, and PINNs. Specifically, given partially observed parameter a for the PDE ((e a, u) = and pre-trained forward operator F', we address the problem by solving the optimization equation:
min Cpde FaaF
(15)
and the results are shown in Table 5 and Fig. 15. Optimization reduces errors and smooths the solutions. However, the resulting values are smaller due to the smoothing effect from minimizing PDE loss, and the overall error compared to the ground truth remains much higher than DiffusionPDE. This may be due to the difficulty in optimizing the derivatives of noisy a and
I Standard Deviation of DiffusionPDE Experiment Results
We further assess the statistical significance of our DiffusionPDE by analyzing the standard devia- tions for forward and inverse problems under conditions of 500 sparse observation points and full observation, respectively, as detailed Table We evaluate our model using test sets comprising 1,000 samples for each PDE. Our findings confirm that full observation enhances the stability of
21

---

DilluiusPDE
DespONee
PINO
Forward
300 Randoo Poidss
Relative Erno: 4.5%
Relative Enor. 13,65 Relolvv Eiro: 79
Relative Eioor 70.5%
Relutvve Brror. 77.5%
Invore
Cocffriett
Rebaive Fioor 20.5 Rebetie Fioor: 75.0% Robeive Fioor 115.0% Relative From: 118,9% Reiatvve Froor 71.9%
Figure 15: Results of Poisson equation after optimizing baseline methods.
the results, predictable outcome as variability diminishes with an increase in observation points. The standard deviations are notably higher for more complex PDEs, such as the inverse problems of the Poisson and Helmholtz equations, reflecting the inherent challenges associated with these computations. Overall, DiffusionPDE demonstrates considcrable stability, evidenced by relatively low standard deviations across various tests.
Table 6: Standard deviation of DiffusionPDE when solving forward and inverse problems with sparse or full observations.
Sparse Observations
Full Observations
Forward Inverse
2.5 + 0.7%
2.2 0.1%
Darcy Flow
3.2 0.9%
2.0 0.1%
Forward
4.5 + 0.9%
2.7 : 0.1% 9.8 0.7%
Poisson
Inverse
20.0 + 1.8%
Forward
8.8 +: 1.0%
2.3 + 0.1%
Helmholtz
Inverse
22.6 + 1.7%
4.0 0.6%
Non-bounded
Forward
6.9 + 0.9%
6.1 0.2%
Navier-Stokes
Inverse
10.4 1.0%
8.6 0.3%
Bounded
Forward
3.9 + 0.2%
1.7 + 0.1%
Navier-Stokes
Inverse
2.7 + 0.2%
1.4 + 0.1%
Runtime Analysis
We evaluate the computing cost during the inference stage by testing a single data point on a single A40 GPU for the Navier-Stokes equation, shown in Table 7. DiffusionPDE has lower computing cost compared to Shu et al. [54], which autoregressively solves the full time interval. This advantage becomes more significant when increase the number of time steps.
Table 7: Inference computing cost of sparse-observation-benn based methods.
Method
DiffusionPDE (Ours)
GraphPDE
Shu et al. (2023)
OFormer
#Parameter (M)
54
1.3
63
1.6
Inference time
| 140
84
180
3.2
GPU memory (GB)
6.8
3.6
7.2
0.1
Further, we evaluate the inference runtimes one single A40 GPU of vanilla full-observation-basee methods and also the optimization time of them during the inference as introduced in Appendix H. The optimization runtimes significantly slower, especially when using Fourier transforms.
22

---

Table Average inference runtimes (in seconds) full-observation based methods with and without
optimization.
Method
PINO
FNO
DeepONet
PINNS
Vanilla
1.0c0
9.8e-1
7.4e-1
1.5e0
With Optimization
6.7e2
6.7e2
3.5e1
3.7el
K
Robustness of DiffusionPDE
We find that DiffusionPDE is robust against sparse noisy observation. In Fig, 16, we add Gaussian noise to the 500 observed points Darcy Flow coefficients. Our DiffusionPDE can maintain relative eπror of around 10% with a 15% noise level concerning the forward problem, and the recovered solutions are shown in Fig. Baseline methods such as PINO also exhibit robustness against random noise under sparse observation conditions; this is attributed to their limited applicability to sparse observation problems, leading them address the problem more randomized manner.
0.4
0.3529
0.35
Relative Error
0.25
0.2
0.15
0.1
0.078
0.074
0.076
0.05
0.025
01
15
3%
5%
10%
15%
Goussinn Noise Level
DiffisionDDE
PINO
Figure 16: Relative errors of recovered Darcy Flow solutions with sparse noisy observation.
18
Nois:
Noise
(5%
Recoveeed Solaiass
Costooons
Ferword
Add Noine
7.85:
time:
Figure 17: Recovered solutions for Darcy Flow with noisy observations.
Robustness on Sampling Patterns Moreover, as mentioned in the main document, we investigate the robustness of DiffusionPDE on different sampling patterns of the observation points. Here, we address the forward problem of Darcy Flow using 500 observed coefficient points, which are non-uniformly concentrated on the left and right sides or are regularly distributed across the grid, as depicted in Fig. 18. Our results demonstrate that DiffusionPDE flexibly solves problems with arbitrary sparse observation locations within the spatial domain, without re-training the neural network model. However, the CFG method faces challenges when solving with varying sampling patterns, as demonstrated in Fig. 19. this figure, we compare reconstruction results of DiffusionPDE and Diffusion with CFG for the unbounded Navier-Stokes equation, where all observation points are located on the left side of the grid. The CFG approach struggles with this asymmetric sampling pattern, while DiffusionPDE maintains more accurate reconstructions.
23

---

Observel Confficints
Resoveral Solatons
Observel Coefficiens
Solution:
Observe
Recovered Solutions
Forward
Forward
Forward
Releiive Error 6.2%
Relaive Frooet 8.89
Relstive Eaon 2.3%
Figure 18: Recovered solutions Darcy Flow with observations sampled using non-uniform distributions.
Observations
Truth
DiftusionPDE
Diflasion
Initall State
Fizal
500 Random Points
Relative Eraor: 24.7% Relative Eroo:: 39.6%
Figure 19: Comparison between DiffusionPDE and Diffusion CFG under different sampling patterns for non-bounded Navier-Stokes equation.
Stochasticity Evaluation Since we employ a deterministic diffusion model, with partial observa- tions as input, the only source of stochasticity or uncertainty in our approach arises from the initial random noise. To examine this, we conducted experiments to assess the impact of different noise seeds on both the initial and final states of the Navier-Stoke equations, as demonstrated in Fig. 20. Our findings indicate that the diffusion model exhibits some degree of uncertainty in its predictions, despite the deterministic nature of the underlying framework.
Observations
Noise Seed
Noise Seed
Noise Seed
Enitaa State
Initial State
Recover
Random
Final State
Figure 20: Different predictions of DiffusionPDE generated by different initial noise for non-bounded Navier-Stokes equation.
L Solving Forward and Inverse Problems with Different Numbers of Observations
We also investigate how our DiffusionPDE handles varying degrees of sparse observation. Experi- ments are conducted on the Darcy Flow, Poisson equation, Helmholtz equation, and non-bounded Navier-Stokes equation. We examine the results of DiffusionPDE in solving forward and inverse problems when there are 100, 300, 500, and 1000 random observations as shown in Fig. 21. We have observed that the error of DiffusionPDE decreases as the number of sparse observations increases. Overall, we recover better than a. DiffusionPDE can recover with approximately 2% observation points at any side pretty well. DiffusionPDE is also capable of recovering both and u with errors 1% 10% with approximately 6% observation points at any side for most PDE families. We also conclude that our DiffusionPDE becomes insensitive to the number of observations once more than 3% of the points are observed.
u, or both and

---

0.35
Error Ratee (Darcy Flow) or Relative Error (Others)
0.25
0.2
0.05
Observed
Observed
Observed
Observed
Observed
Observed a&u
Observed
Observed
Observed
Observed
Observed
Observed adu
Non-bounded Navier-Stokes
Darey Flow
Poisson
Holmholtz
Error rates for Darcy Flow and relative errors for other PDEs recovered coefficients or initial states
Relative Error
Observed
Observed
Observed aduu
Observed
Obsurved
Obwerved on adu
Observed
Observed
Observed adu
Observed
Observed
Observed a&u
Non-bounded Navier-Stokes
Darey
Peisson
Helmholtz
Observations
300 Observations
Observations
1000 Observations
(b) Relative errors of recovered solutions final states
Figure 21: Error rate or relative error of both coefficients (or initial states) a and solutions (or final states) with different numbers of observations.
M Solving Forward and Inverse Problems across Varied Resolutions
To evaluate the generalizability of DiffusionPDE, we implemented the model on various resolutions, including 64 x 64 and 256 256, while maintaining the same percentage of observed points. For resolutions of 64 x 64, 128 x 128, and 256 x 256, we observe 125, 500, and 2000 points on or respectively, which are approximately 3% for each resolution. Overall, DiffusionPDE is capable of handling different resolutions effectively. For instance, Table 9 presents the forward relative errors of the solution u and inverse error rates of the coefficient a for the Darcy Flow, demonstrating that DiffusionPDE performs consistently well with similar error rates across various resolutions.
25

---

Table Forward relative errors and inverse error rates of Darcy Flow across different resolutions.
Resolution
Forward Relative Error
Inverse Error Rate
64 64
2.9%
4.3%
128 x 128
2.5%
3.2%
256 x 256
3.1%
4.1%
N Comparison with Other Baselines
We have compared the results using the RBF kernel [60], as shown in Fig. 22. For the forward process of solving the Poisson, Helmholtz, and Darey Flow equations, the RBF kernel achieved solution errors of approximately 14.3%, 23. 23.1%, and 18.4%, respectively, with 500 random observation points. However, when addressing the inverse problem, errors increased significantly to 44.2%, 143.1% and 34.0%, respectively. This increase in error is likely due to the inherent challenges of solving inverse problems with such a straightforward method.
RBF
Ground Truth
RBF
Ground Truth
Poisson (Forward)
Poisson (lavers)
Relative Enoor 15.3%
Relative Error: 140.7%
Figure 22: Forward and Inverse Results of Poisson equation recovered by 500 observation points using RBF Kernel.
Additionally, we compare our DiffusionPDE method with single U-Net model. The U-Net is trained based on our EDM diffusion model, where we initially train it to map between 500 fixed input points and the full output space, as illustrated in Fig. 23. For the Navier-Stokes equation, the prediction of the final state results in average test error of approximately 39%, which is significantly higher than the error produced by our diffusion model. Furthermore, when making predictions using 500 different sampling points, the relative eπror increases approximately 49%. We also train another U-Net model map between 500 random input points and the full output space, but this model results in test error of 101 %, indicating that the U-Net struggles to adapt to varying sampling patterns and fails to flexibly solve different configurations.
UNet Trained Fixed UNet Trained Random Observation Points Observation Points
Ground Truth
DiffusionPDE
Final State
Relative Erroe: 6.5%
Relative Error: 38.9%
Relative Error: 100.9%
Figure 23: Comparison between DiffusionPDE and U-Net regarding non-bounded Navier-Stokes equation.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly state our contribution to solving forward and inverse problems of PDEs simultaneously with partial observation in the abstract and introduction (Section I).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We include the limitation of work and future direction in Section 5,
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
27

---

Justification:
We include theoretical results with full assumptions and complete proofs in
Section 3.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We include all implementation details for reproducibility supplemental materials (Sections B and C).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
28

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: In the Abstract, we include the URL project page, which provides access to the data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
(ogllc/guides/ooeeeee
www.ssssssoolillooo onPolicy) icy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. wctpublic//aeccccccccccccccc ggwii/gg///////eeee// /Ceoddeodmubiiic cttssuuumisiioon. ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We include implementation details of data generation and sampling in the supplemental materials (Sections B and C).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the standard deviation of our DiffusionPDE in the supplemental materials (Section I).
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
29

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
We talk about the compute resources in Sections 4.2 and J.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https / /ttp///eerrpp.eeeee htcp/p/bccc/thhhcssuieeleeeeeele
Answer: [Yes]
Justification:
We follow the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work, which concentrates solving both forward and inverse problems partial differential equations (PDEs) with sparse observations, promises significantly advance scientific and engineering disciplines that rely on these computations. The potential negative impacts are minimal, as this research primarily aims to provide more accurate and efficient computational methods without adverse effects existing systems practices.
30

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out, For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
31

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
32

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
33


Reviews:
[{"review_id": "nZRQ0jN4rd", "reviewer": "Reviewer_nBsW", "summary": "The paper uses score based generative diffusion models to find the forward and backwards solution of a set of PDEs given partial observations of the solution and/or incomplete knowledge of the coefficients. The method performs well, and outperforms other ML  methods such as FNO, as well as 'standard' FE type methods, for a range of standard test problems. The method reconstructs This is a novel approach, which delivers good performance, with low errors  at a competitive speed. Extensive tests are given, with careful analysis of the results.", "questions": "1. How does this method compare with a data assimilation approach\n2. How easy would it be to extend the method to full time intervals \n3. How easy is it to extend the method to higher dimensions\n4. Have the authors tried out the method on more challenging PDE examples.\n5. Also consider tests on NS and Bergers' eqn with much smaller viscosity.", "limitations": "The model as described only looks at slices of solutions of 2D problems. This has been clearly identified by the authors. In this sense it is vrather limited when compared to other ML based approaches, and of course traditional FE based methods. I am pleased that this is recognised and that the authors plan to address this. The DiffusionPDE method will only be truly competitive when this is done, but this paper is a good step in this direction.", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The use of score based generative methods in this context, where both the solution and the parameter estimates are updated, is novel. The method is clearly effective for the problems considered and should have good applications to real world examples. Extensive tests on a series of standard test problems show that the errors of the method are much lower than other ML based methods such as FNO.", "weaknesses": "This paper suffers as do many similar papers from a limited range of examples. It concentrates on the usual examples of PDEs such as NS and Bergers, and in both cases of these it looks at problems with quite moderate viscosity, which are realtively east to solve. This is more or less inevitable for such a short paper as this, especially as comparisons are needed with other method. But I would have liked to have seen more novel examples than the usual ones. This is not really a criticism of this paper, but is something to consider for future work. It would be imporoved by a fairer comparison with other methods which work with incomplete data and measurements. A clear exanple of this being the data assimilation widely used in physical modelling for just this range of problems. These should be descibed somewhere in the introduction and in Section 2. (Although of course these latter methods are slow in comparison.) The method is also limited (see later) to looking at certain slices of the solution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comments on our work! We feel much encouraged that you recognize the novelty of our work. \n\n> **a fairer comparison with other methods that work with incomplete data and measurements** \n\nIn addition to GraphPDE, we further compare our method with OFormer [1], Shu et al. [2], and UNet baselines. Please see Figures 7-10 in the PDF.\n\n> **Q1: data assimilation**\n\nWe have compared the results using the RBF kernel (Figure 1 in the PDF). For the forward process of solving the Poisson, Helmholtz, and Darcy Flow equations, the RBF kernel achieved solution errors of approximately 14.3%, 23.1%, and 18.4%, respectively, with 500 random observation points. However, when addressing the inverse problem, the errors increased significantly to 141.2%, 143.1%, and 34.0%, respectively. This increase in error is likely due to the inherent challenges of solving inverse problems with such a straightforward method. We are happy to include such discussions in the revision of the paper.\n\n> **Q2: full time intervals**\n\nPlease see our reply in the common response above.\n\n> **Q3: higher dimensions**\n\nYes, we can extend the method to higher dimensions using state-of-the-art diffusion models designed for such cases. For instance, 3D diffusion models (e.g., [3]), especially latent diffusion models, could be appropriate for handling 3D geometric PDEs.\n\n> **Q4: more challenging PDE examples**\n\nTo further demonstrate the generalization capability of our model, we conducted additional tests on different data settings for Darcy Flow. In Figure 2 in the PDF, we solve the forward and inverse problems of Darcy Flow with 500 observation points, adjusting the binary values of $a$ to 20 and 16 instead of the original 12 and 3. Our results indicate that DiffusionPDE performs equally well under these varied data settings, showcasing its robustness and adaptability.\n\n> **Q5: NS and Burgers' equation with much smaller viscosity**\n\nYes, we also test DiffusionPDE on the Burgers’ equation with a viscosity of $1 \\times 10^{-3}$ and on the Navier-Stokes equation with a viscosity of $1 \\times 10^{-4}$, which are 10 times smaller than the ones in the main paper. For the Burgers’ equation, we are able to recover the full time interval with 5 fixed sensors at a relative error of approximately 6%, which is close to the error of approximately 2~5% in the main paper. For the Navier-Stokes equation, we can solve the forward and inverse problems with relative errors of approximately 7% and 9%, respectively, using 500 observation points. The errors are also close to the ones in the main paper, where the forward and inverse errors of NS are approximately 7% and 10%. Please see Figure 3 in the PDF.\n\n[1] Transformer for Partial Differential Equations' Operator Learning. Li et. al. TMLR 2023.\n\n[2] A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction. Shu et. al. JCP 2023.\n\n[3] 3D Neural Field Generation using Triplane Diffusion. Shue et. al. CVPR 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C9OuJt8BIo", "reviewer": "Reviewer_4PC9", "summary": "The work uses a guided diffusion process to solve the PDE forward and inverse problems with partial observations. Instead of learning the parameter-to-solution map ($a\\rightarrow u$) as in Neural Operators, the method learns the diffusion process on the joint distribution $(a,u)$, and use guided diffusion for inference under sparse observations. Compared with several baseline method, the proposed method shows improved performance for solving forward and inverse problem with sparse observations.", "questions": "(1) The results from the baseline methods (PINO, DeepONet, PINNs, and FNO) are so bad. The claim that these methods are \"not easily extendable\" invites further scrutiny:\n\n(a) All the baseline models are supposed to represent smooth functions. However, in Figure 4, 7, 8, they look discontinuous at the training points.\nAn explanation of how these models were trained and how inference was conducted could clarify why these discrepancies appear.\n\n(b) Taking PINNs as an example in the Darcy flow problem.\nLet $\\hat{a}(x)$ and $\\hat{u}(x)$ be the (potentially noisy) observation at $x$.\nWe can represent $a(x)$ by a neural network, $a_V(x)$ (use neural net for convenience, could be other representations), and the PDE solution $u(x)$ by a neural net $u_W(x)$, where $V$ and $W$ are the weights of the neural network.\nWe can solve the following optimization problem:\n\n$$\\min_{V,W} \\sum_{x\\in T_d} (u_W(x) - \\hat{u}(x))^2 + (a_V(x) - \\hat{a}(x))^2 + \\sum_{x \\in T_r} (\\nabla \\cdot (a_V(x) \\nabla u_W(x)) - q(x))^2$$\n\nwhere $T_d$ is the observational point, and $T_r$ is the residual points, which does not need to be the same as $T_d$.\n\n(c) Similar, for a trained neural operator parametrized by W, $G_W[a] = u$. We can solve the following optimization problem:\n\n$$\\min_V \\sum_{x\\in T_d} (a_V(x) - \\hat{a}(x))^2 + (G\\[a_V\\](x)-\\hat{u}(x))^2$$\n\nwhere $G[a_V]$ is the solution of the PDE with parameter $a_V$.\n\nIt seems that all the baseline methods can be used for forward and inverse problems with sparse observation. \nIt is unclear why the proposed method would offer superior performance compared with the baselines. \n\nIn contexts where full observation are available, as shown in Table 4, one might intuitively expect methods like PINNs—which utilize residual losses to ensure adherence to PDE constraints—or Neural Operators—which establish a direct parameter-to-solution mapping—to deliver more accurate results compared to a method that relies on a diffusion process. This leads to a critical inquiry on why diffusion process gives better accuracy for PDE problems.\n\n(2) How is the PDE loss computed? Is it by finite difference on a regular grid? Detailing this in the main text could help readers assess the accuracy and applicability of the PDE loss in different scenarios.", "limitations": "The author mention several limitation in the conclusion.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The work uses a guided diffusion process to solve the PDE forward and inverse problems with partial observations.The authors compare with several baseline methods. The idea is clearly presented, and might be useful for the community.", "weaknesses": "The paper presents an interesting approach to solving PDE forward or inverse problems with sparse observations, which is an appealing concept given the minimal data requirement. However, this approach raises some concerns about the well-posedness of the problem. For example, in forward problems where sparse observations of the parameter $a(x_i)$ are available, there are infinitely many ways to interpolate $a$ and solve the PDE to obtain $u$. They are all valid solutions that satisfy both the PDE and the observations. This suggests that the method's ability to achieve good recovery might heavily rely on the strong regularization imposed by the training dataset, potentially limiting its practical utility as it may only favor solutions resembling those in the training set.\n\nAdditionally, in Appendix C, Table 2, the weightings for observation and PDE loss are significantly higher (by two to six orders of magnitude) than those for $\\nabla_x \\log(p(x))$ as described in Equation 8, which might indicate a predominance of data fitting over the diffusion process. It would be beneficial if the authors could provide more guidance on how these weights were chosen and discuss the implications of using smaller weights. Understanding the rationale behind these choices could help clarify the model's dependency on these parameters and their impact on the solution's behavior.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed and constructive review!\n\n> **“The method’s success may heavily depend on the strong regularization from the training dataset.”**\n\nWe use the same data generation methods as other studies [1, 2, 3], ensuring fair comparisons. The process does not favor any specific subset of PDEs. It’s reasonable to assume that a dataset can be obtained containing patterns or regularization about field distributions. For example, factories manufacture parts with material properties following specific distributions, and our method can solve PDEs for these parts. While our method may struggle with out-of-distribution cases, we know of no existing method that performs well under sparse observations without assumptions about PDE coefficients to the best of our knowledge. This is an open and exciting field with great potential.\n\n> **“The weightings for observation and PDE loss are significantly higher.”**\n\nThe weights depend on the scale of the coefficients or solutions. In our static PDE dataset, the absolute values of $u$ are much smaller than those of $a$. To ensure their distributions fall between -1 and 1, $a$ and $u$ are scaled differently during dataset preparation. During inference, they are scaled back to their original values to calculate PDE and observation losses. Thus, $\\zeta_u$ should be larger than $\\zeta_a$ to ensure the accurate recovery of both $a$ and $u$, as $L_u$ is much smaller than $L_a$. For example, in the Darcy Flow equation, $a$ is 3 or 12, while the absolute value of $u$ can be less than 0.01, making $L_u$ approximately $10^{-4}$ times $L_a$, leading to $\\zeta_u$ being $10^4$ times $\\zeta_a$. Similarly, the choice of $\\zeta_{pde}$ is also influenced by the data scale.\n\n> **Q1(a): “The baseline models… look discontinuous”**\n\nAll models are trained with full observations. During inference, we apply a mask to set non-observed input values to zero, which can cause output discontinuities, especially with FNO as the model does not guarantee smooth output. Results are smoother after applying the optimization from Question 1(b) (Figure 5 in the PDF). We also considered training on partial observations (Appendix G), where the model maps sparse inputs to full outputs, but this method doesn’t generalize to different sampling patterns. Additionally, using a different PINN codebase with modified hyperparameters [4] results in smoother outputs for the Burgers’ equation (Figure 4 in the PDF).\n\n> **Q1(b)(c): Suggest optimizing baselines methods**\n\nWe acknowledge that optimizing the sampling space is another way to run the baselines. We applied this approach, and the results are shown in Table C below and Figure 5 in the PDF. Optimization reduces errors and smooths the solutions. However, the resulting values are smaller due to the smoothing effect from minimizing PDE loss, and the overall error compared to the ground truth remains much higher than DiffusionPDE. This may be due to the difficulty in optimizing the derivatives of noisy $a$ and $u$. Additionally, optimizing FNO and PINO is slow. While other models were optimized within 30 seconds, FNO and PINO took over 10 minutes to converge. Table B above shows these optimized baselines are slower than DiffusionPDE, likely due to the computational complexity of the Fourier transform.\n\n**Table C: Relative errors of solutions (or final states) and coefficients (or initial states) when solving forward and inverse problems respectively with sparse observations after optimizing the baselines. Error rates are used for the inverse problem of Darcy Flow.**\n| | | DiffusionPDE | DeepONet | PINO | FNO |  PINNs  |\n|-|-|:-|:-:|:-:|:-:|:-:|\n| **Darcy Flow** | Forward | **2.5%** | 31.3% | 32.6% | 27.8% | 6.9% |\n| | Inverse |**3.2%**|41.1%|49.2%| 49.3% | 59.7%|\n| **Poisson**| Forward | **4.5%** |73.6%| 79.1%  |  70.5% |  77.8%  |\n| | Inverse | **20.0%** | 75.0% | 115.0% | 118.5%  |  73.9%  |\n| **Helmholtz** | Forward | **8.8%** | 77.6% | 67.7%  |  84.8%  |  79.2%  |\n| | Inverse |  **22.6%**   | 100.7%   | 125.3% | 131.6%  | 103.7%  |\n| **Non-bounded NS** | Forward | **6.9%** | 96.5% | 93.3%  |  91.6%  | 106.1%  |\n| |Inverse |  **10.4%**   |  71.9% | 87.8%  |  89.3%  | 108.6%  |\n| **Bounded NS** | Forward | **3.9%** |  89.1% | 80.8%  |  81.2%  |  84.4%  |\n| | Inverse |   **2.7%**   |  88.6%   | 47.3% | 48.7% |  82.1% |\n\n> **Q1(c): “Why diffusion process gives better accuracy for PDE problems than other methods like PINNs”**\n\nFor partial observations, PINNs [1] does not learn any knowledge about the PDE, making it difficult to inpaint the missing parts. Neural operators [2] learn to map the entire coefficient space to the entire solution space and hence have difficulty taking partial observations as input. For full observations, PINNs require iterations of optimization to converge to the solution, which can be vulnerable to local minimal or failure to convergence, leading to higher error. It is also less robust compared to our data-driven method. Our method, on the other hand, enjoys the advantage of combining PDE knowledge and observation guidance through an iterative generative model. Iterative generative models, such as diffusion models, tend to beat feedforward models like GANs.\n\n> **Q2: “How is the PDE loss computed?”**\n\nSobel filters are applied to the grid with asymmetric padding to compute derivatives. The PDE loss at the boundary is manually set to zero to handle edge effects. This procedure is applied to the pixel space rather than the continuous coordinate space. We promise to supplement these details in the final version of the paper.\n\n[1] Physics-informed neural networks. Raissi et. al. JCP 2019.\n\n[2] Fourier Neural Operator for Parametric Partial Differential Equations. Li et. al. ICLR 2021.\n\n[3] Learning to solve pde-constrained inverse problems with graph networks. Zhao et. al. ICML 2022.\n\n[4] Physics-informed deep-learning applications to experimental fluid mechanics. Eivazi et. al. Measurement Science and Technology 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AXEQgaiI3m", "reviewer": "Reviewer_AaET", "summary": "The paper proposes to solve PDEs given only sparse measurements by jointly modeling the solution and coefficient space (e.g. the initial conditions) using a diffusion model. By applying diffusion posterior sampling (DPS) the authors obtain samples that are consistent with the sparse measurements and the underlying PDE equations. Several experiments show superior performance of the method compared to standard baselines such as PINNs and FNOs.", "questions": "- Algorithm 1 shows an adaption of DPS to EDM (Karras et al. 2022). Is this adaptation novel? Can the authors give some intuition why they apply the DPS losses in line 12 and 13 to the 2nd order correction (line 8) and not apply any trapezoidal rules in this case?  \n- Are sparse measurements located on a grid that matches the resolution of the diffusion model or do they have continuous coordinates? In the second case, how are they interpolated to match the data resolution of the diffusion model? Does that make classifier-free guidance difficult to apply?\n- As noted in the weaknesses: why not use classifier-free guidance? I would like to see a discussion of different methods for inverse problems and diffusion models that can be used here instead of DPS and what are the advantages of using DPS. Reconstructing the solution/coefficient space from sparse measurements alone is a linear inverse problem with a number of different methods that can be used (e.g. Denoising Diffusion Restoration Models; Kawar et al. 2022, among many others) which oftentimes have much nicer theoretical guarantees/higher quality reconstructions and faster sampling speed. When considering these methods, is adding the PDE loss $\\mathcal{L}_{pde}$ and thus making the problem a non-linear inverse problem really beneficial?\n- Likewise, as mentioned above: what are the parameter counts and runtimes of the method and the baselines?", "limitations": "The authors have mentioned slow sampling speed as a limitation in the conclusion, but I think an extended discussion of this would be important to include.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Solving PDEs under partial observation is an important problem in real-world applications\n- The proposed method is technically sound and improves upon existing baseline methods (PINN, FNO) that do not work well for sparse measurements\n- Leveraging a pretrained diffusion model as a generative prior to model the joint distribution of solution and coefficient space is a good idea\n- The presentation of the method is clear and supported by concise algorithms and equations. The paper is well written overall\n- Experiments consider standard baseline methods for PDEs and cover a sufficient range of different dynamics\n\n----\n\nPost-rebuttal: the authors have addressed quite a few of the initial concerns, and while some concerns (e.g. about the magnitude of the contributions remain), I'd be happy to support an accept. I've raised my score accordingly.", "weaknesses": "- The main weakness of the method is the limited novelty. Both sparse measurements and physics-based losses have been considered together with diffusion models, see e.g. Shu et al. (2023). So it seems to me that the main technical novelty is to apply diffusion models to model the joint distribution of two simulation states at different points in time and apply DPS during inference for consistency with the sparse measurements and PDE constraints.     \n- The experiments do not take into account any stochasticity or uncertainty. In principle, DPS will give a distribution of solutions, which is not the case for the other baseline methods, but this is not explored further in the paper. \n- Since the joint distribution models two states at time 0 and time T (for all experiments except Burgers' equation) and $0 \\ll T$, the authors need to simplify the PDE loss $\\mathcal{L}_{pde}$ to drop any time derivatives. This is a serious limitation.   \n- It is not clear if DPS works better than classifier-free guidance, as used e.g. in Shu et al. (2023), or other methods for solving inverse problems with diffusion models. \n- DPS requires a lot of compute during inference for calculating $\\mathcal{L}_{pde}$. For a fair comparison, it would be important to show the number of parameters, training time and inference time for all methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for agreeing with us that we consider an important problem and propose a technically sound method.\n\n> **“The main weakness of the method is the limited novelty”**\n\nPlease see our common response above.\n\n> **“The experiments do not take into account any stochasticity or uncertainty”**\n\nOn one hand, we use a deterministic diffusion model. Thus, given partial observations as input, the only stochasticity or uncertainty in our method is the initial random noise. On the other hand, we agree it is important to take this into consideration and hence supplement an experiment where we test the effect of different noise seeds on the initial and final state of Navier-Stokes equations. Please see Figure 6 in the attached PDF. We promise to add this to the final version of the paper.\n\n> **“The authors need to… drop any time derivatives”**\n\nPlease see the common response.\n\n> **Q1: “Can the authors give some intuition why they apply the DPS losses in lines 12 and 13 to the 2nd order correction (line 8) and not apply any trapezoidal rules in this case?”**\n\nDPS guides the sampling procedure by the end of each iteration, and we are not specifically modifying it. \n\n> **Q2: “Are sparse measurements located on a grid?”**\n\nYes, we use a 128*128 grid to represent the different spaces/states of the PDEs and the sparse measurements lie on the grid points. This is a common practice for most works in area [1][2].  However, we can also easily extend our method to be not restricted to grid points by interpolating on the grid and supervising the inerpolated value.\n\n> **Q3: “It is not clear if DPS works better than classifier-free guidance”**\n\nIn Figure 8 in the PDF, we evaluate the forward and inverse processes of the non-bounded Navier-Stokes equation, comparing DiffusionPDE with Diffusion with CFG when only initial and final states are considered. Our DiffusionPDE method achieves lower relative errors in both evaluations. We also compare our results with those of Shu et al. [3], where full time intervals are solved autoregressively. In this approach, the error of the final state increases to approximately 13%, which is higher than that of the two-state model (see Figure 10 in the PDF file).\n\n> **Q4: “Show the number of parameters, training time and inference time for all methods.”**\n\nPlease see the common response.\n\n[1] Physics-informed neural networks. Raissi et. al. JCP 2019.\n\n[2] Fourier Neural Operator for Parametric Partial Differential Equations. Li et. al. ICLR 2021.\n\n[3] A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction. Shu et al. Journal of Computational Physics 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2v5ZG9boCi", "reviewer": "Reviewer_y14J", "summary": "This paper introduces diffusion methods to tackle the partially observed PDEs, named DiffusionPDE. By learning the joint distribution of solution and coefficient space, the proposed model can handle both forward and inverse problems. The authors experiment with diverse PDEs and settings to demonstrate the model's effectiveness.", "questions": "1.\tFigure 4, do both forward and inverse tasks use the same diffusion model? Or do we need to train two models for these two different tasks?\n\n2.\tI think the base model is U-Net. So how does DiffusionPDE handle the spatially scattered partial observations? Is the input still in the regular grid, but only the sampled locations have ground-truth values?", "limitations": "I appreciate that they have discussed the limitations. But I think the mentioned issues about efficiency and limitations on temporal modeling are not trivial. More discussions are expected.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "-\tThis paper successfully utilizes the diffusion methods in solving PDEs, covering both forward and inverse problems.\n\n-\tThe main text and supplementary materials provide diverse experiment settings, which can well support the model’s effectiveness on partial observations.\n\n-\tThis paper is overall clear and well-written.", "weaknesses": "1.\tThe technical contribution is limited.\n\nFrom a technical view, this paper is an application of the diffusion model in PDE solving. There are also some previous methods that also use diffusion methods and leverage the PDE loss [1]. Thus, I think the technical novelty is limited.\n\n[1] A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction, JCP 2023\n\n2. Some powerful baselines are missing.\n\n- According to Figure 1, I think the base model of DiffusionPDE is U-Net. How about comparing it with a single U-Net? I think U-Net could be a powerful baseline.\n\n- There are also some latest models that are good at processing partially observed or irregularly placed PDEs, such as OFormer [1] and Transolver [2]. They should include them as baselines.\n\n[1] Transformer for Partial Differential Equations' Operator Learning, TMLR 2023\n\n[2] Transolver: A Fast Transformer Solver for PDEs on General Geometries, ICML 2024\n\n3. Model efficiency comparisons are needed, including GPU memory and running time.\n\n4. I think the proposed model cannot predict the future evolution of a time-dependent PDE. Current tasks are all about “reconstruction” or “imputation”.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful comments! We are happy that you find our paper provides diverse experiments and is well-written.\n\n> **“The technical contribution is limited.”**\n\nPlease refer to the common response above.\n\n> **“Some powerful baselines are missing.”**\n\n1. U-Net: We trained a U-Net model based on our EDM diffusion model. Initially, we trained the model to map between 500 fixed points in the input space and the full output space (Figure 9  in the PDF). For the Navier-Stokes equation, the prediction of the final state resulted in an average test error of approximately 39.1%, which is significantly higher than the error of our diffusion model. Additionally, we made predictions for the final state using 500 different sampling points, which increased the relative error to approximately 49.2%. Furthermore, we trained another U-Net model to map between 500 random points in the input space and the full output space. This U-Net model resulted in a 101% error during testing, indicating that the model fails to flexibly solve different patterns. \t\n\n2. OFormer [1]: We examined the forward and inverse process of Navier-Stokes equation for OFormer taking 500 observation points, and the relative errors are approximately 17% and 23% respectively, which are much larger than DiffusionPDE, as shown in Figure 10 in the PDF file.\n\n3. Transolver [2]: We believe that Transolver is applied to 3D geometric PDEs and is not suitable for comparison with our DiffusionPDE without significant modifications to their method.\n\n> **“Model efficiency comparisons”**\n\nPlease see the common response.\n\n> **“time-dependent PDE”**\n\nPlease see the common response.\n\n> **Q1: “Figure 4, do both forward and inverse tasks use the same diffusion model?”**\n\nYes. We can do forward and inverse tasks with a single model.\n\n> **Q2: “So how does DiffusionPDE handle the spatially scattered partial observations?”**\n\nWe use a 128*128 grid to represent the different spaces/states of the PDEs, so yes the input is still in the regular grid. This is a common practice for most works in area [3][4]. In addition, we leverage DPS [5] to handle the spatially scattered partial observations, which is a technique designed for noisy inverse problems (e.g. inpainting). \n\n[1] Transformer for Partial Differential Equations' Operator Learning. Li et. al. TMLR 2023.\n\n[2] Transolver: A Fast Transformer Solver for PDEs on General Geometries. Wu et. al. ICML 2024.\n\n[3] Physics-informed neural networks. Raissi et. al. JCP 2019.\n\n[4] Fourier Neural Operator for Parametric Partial Differential Equations. Li et. al. ICLR 2021.\n\n[5] Diffusion Posterior Sampling for General Noisy Inverse Problems. Chung et. al. ICLR 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank all reviewers for their constructive feedback! We will first clarify common concerns from the reviewers.

> **The method drops time derivatives and cannot solve for full time intervals (Reviewer nBsW, AaET)**

Our method can in theory support time derivatives and solve for full time intervals, as we have demonstrated in the 1D Burger’s equation (Figure 4 in the paper). Due to our choice of the diffusion model,  which enables channel concatenation, we decided to model the joint distribution on the initial state and an arbitrary time state for other time-dependent PDEs such as the Naiver-Stokes equations. However, we think this is not a limitation of our method because:

1. The choice of diffusion model is orthogonal to our proposed method. For example, we can extend our solutions of the Naiver-Stokes equations to full time intervals by replacing the current 2D diffusion model with ones that support higher dimensions, such as 3D or video diffusion models [3,4,5], without changing the algorithm we described in the paper (Lines 12-14, Algorithm 1).

2. Even at its current state,  where the method is applied to solve time-dependent PDE by modeling the joint distribution between two time states for time-dependent PDEs, our method can achieve more accurate results in solving both forward and inverse PDEs. Plus, our method is faster than methods that autoregressively solve time-dependent PDEs [1].

> **“There is limited novelty of this paper compared with [1] (Shu et. al. 2023)” (Reviewer AaET, y14J)**

We would like to reiterate the novelty of this paper in that we propose to use diffusions to model the joint distribution of different spaces/states of PDEs, achieving state-of-the-art performance in solving forward/inverse PDEs under sparse observation. We would like to emphasize our main novelty/contribution as proposing to use joint distribution as a better way to model PDE problems (as stated in L32-34 in the paper and recognized by reviewer AaET). Compared with [1], we point out a few advantages brought by our method:

1. In terms of PDE types, Shu et al. (2023) is designed for fluid flow fields; as a result, they only show demonstration on limited PDE types such as Naiver-Stokes. Our method can work for a broader category of PDEs, such as Darcy Flow and Poisson equation.

2. In terms of diffusion models, we propose to use DPS [2] as a better choice for the problem. DPS is designed for solving inverse problems and thus suits well the task of solving PDEs given partial observation. We demonstrated that our proposed method can achieve lower errors and better robustness against different sparse sampling patterns (see Figure 7,8 in the PDF). 

3. In terms of the problem setup, Shu et al. (2023) requires consistent observations across time. Our method, on the other hand, requires observations at only the initial or final state or both for time-dependent PDEs, enjoying more flexibility on the partial observation.

4. In terms of inference time, Shu et al. (2023) is slower because it is an autoregressive method. In comparison, our method using the joint distribution adopts a unified model that can handle both forward and inverse and shows more accurate results (see Figure 10 in the PDF file).

> **“Show the number of parameters, training time, inference time, and GPU memory” (reviewer AaET, y14J)**

As mentioned in Section 4.2 of our main paper, training the DiffusionPDE model takes approximately 4 hours on 3 A40 GPUs. In comparison, training the PINO or FNO model on a single A40 GPU takes approximately 8 hours, while training DeepONet takes around 40 minutes.

We evaluate the computing cost during the inference stage by testing a single data point on a single A40 GPU for the Navier-Stokes equation, as shown in Table A. DiffusionPDE has a lower computing cost compared to Shu et al. (2023), which autoregressively solves the full time interval. This advantage becomes more significant when we increase the number of time steps.

**Table A: Inference computing cost of sparse-observation-based methods.**
| Method        | DiffusionPDE (Ours) | GraphPDE | Shu et al. (2023) | OFormer |
|---------------|:------------:|:--------:|:-------------:|:-------:|
| **#Parameter (M)**          |   54  | 1.3  | 63 | 1.6  |  
| **Inference time (s)**           |   140     |  84  |     180     |   3.2   |
| **GPU memory (GB)**	| 6.8 |  3.6   | 7.2 | 0.1  | 

Further, we evaluate the inference runtimes on one single A40 GPU of vanilla full-observation-based methods and also the optimization time of them during the inference as suggested by Reviewer 4PC9. The optimization runtimes are significantly slower, especially when using Fourier transforms.
   
**Table B: Average inference runtimes (in seconds) of full-observation-based methods with and without optimization.**
| Method          |  PINO  |  FNO   | DeepONet | PINNs  |
|-----------------|:------:|:------:|:--------:|:------:|
| Vanilla |  1.0e0  | 9.8e-1 |  7.4e-1  |  1.5e0 |
| With Optimization | 6.7e2  | 6.7e2  |  3.5e1   |  3.7e1 |

[1] A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction. Shu et. al. JCP 2023.

[2] Diffusion Posterior Sampling for General Noisy Inverse Problems. Chung et. al. ICLR 2023.

[3] Diffusion Probabilistic Models for 3D Point Cloud Generation. Luo et al. CVPR 2021.

[4] Flexible Diffusion Modeling of Long Videos. Harvey et al. NeurIPS 2022.

[5] Open-Sora: Democratizing Efficient Video Production for All. Zheng et al. Github.

Meta-review: This diffusion models proposed here solve PDEs with unknown coefficients by modeling the joint distribution of the coefficients and solution space.  An advantage of this generative approach over some other learning-based PDE solvers is that it can flexibly handle partial observations with irregular patterns.  While one borderline-negative review is skeptical of this probabilistic approach, overall we found the authors' rebuttal to these concerns to be convincing.  There were some concerns about the range of experiments and baseline methods compared to in the original manuscript, but a thorough author rebuttal addressed most of these points.  Please be sure that your final manuscript incorporates these additional experiments and comparisons.
Decision: Accept (poster)