Title: Guided Diffusion Sampling on Function Spaces with Applications to PDEs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Diffusion Models

Function Spaces

Partial Differential Equations

Inverse Problems
Abstract: We propose a general framework for conditional sampling in PDE-based inverse problems, targeting the recovery of whole solutions from extremely sparse or noisy measurements.
This is accomplished by a function-space diffusion model and plug-and-play guidance for conditioning.
Our method first trains an unconditional discretization-agnostic denoising model using neural operator architectures. 
At inference, we refine the samples to satisfy sparse observation data via a gradient-based guidance mechanism.
Through rigorous mathematical analysis, we extend Tweedie's formula to infinite-dimensional Banach spaces, providing the theoretical foundation for our posterior sampling approach.
Our method (FunDPS) accurately captures posterior distribution in function spaces under minimal supervision and severe data scarcity. Across five PDE tasks with only 3\% observation, our method achieves an average 32\% accuracy improvement over state-of-the-art fixed-resolution diffusion baselines while reducing sampling steps by 4x. Furthermore, multi-resolution fine-tuning ensures strong cross-resolution generalizability and speedup. To the best of our knowledge, this is the first diffusion-based framework to operate independently of discretization, offering a practical and flexible solution for forward and inverse problems in the context of PDEs. Code is available at https://github.com/neuraloperator/FunDPS.


Peer reviews:
comments: SUMMARY:
The authors solve the problem of finding samples for a posterior function where the likelihood is given by a data fitting term and the regularization is given by a diffusion model. The setup for this problem is infinite dimensions where the parameter to be recovered belong to some functional space (rather R^n).

This types of papers are important from a theoretical point of view as they justify multiresolution work and can aid in any multiscale process.

STRENGTHS AND WEAKNESSES:
The paper may be strong theoretically although I am having a hard time following the technical content (this is more my issue that the authors). I think that researchers that are more versed in the branch of functional analysis and probability could give a more precise evaluation. 
I do think that the overall content is difficult to read and work is needed to make it easier if you want a CS type publication.

I have only one question/comments (see below)

QUESTIONS:
My questions are more on the practical side of the work.

Equation 13 is the approximation you use for the velocity when solving the ODE. This approximation is highly inefficient in the context of inverse problems and gradient descent. This is because if c is large (you want to actually fit the data) the ODE becomes highly stiff and you will need an implicit type method. This is why almost every PDE constrained optimization problem is solved using a Gauss-Newton type method rather than SGD. Your comparisons with other highly sub-optimal methods does not strike confidence.

LIMITATIONS:
No
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-24

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nFunDPS proposes a general framework for solving PDE-based inverse problems through conditioning sampling. The core idea is to first learn the prior distribution of the underlying functions by training an unconditional diffusion model in function space. Then, conditioning on observations is achieved at inference time using a plug-and-play guidance mechanism derived from an extension of Tweedie's formula.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The primary contribution is the rigorous extension of diffusion models to infinite-dimensional function spaces for inverse problems.\n2. The method demonstrates superior empirical performance on a variety of challenging PDE-based inverse problems, often achieving better results in fewer sampling steps than competing methods. The inclusion of discussion regarding wall-clock time addresses potential concerns about the computational overhead of calculating the guidance term's gradient.\n3. The proposed multi-resolution training and inference strategy is a key practical advantage, enabling efficient handling of functions at different levels of detail and likely contributing to the model's overall performance and speed.\n\nWeakness\n1. While the paper's contributions are significant, the core components are not entirely novel in isolation. The concept of function-space diffusion models builds upon prior work (e.g., Score-based Diffusion Models in Function Space), and plug-and-play guidance is a well-established technique in finite-dimensional diffusion models.\n\nQUESTIONS:\n1. How sensitive is the model's performance to the choice of $\\zeta$? Please provide an ablation study showing performance across a range of $\\zeta$. Moreover, is $\\zeta$ manually chosen or is automatically determined under certain theoretical background?\n\nLIMITATIONS:\n1. Refer to weakness.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe papers propose a general framework for conditional sampling in PDE-based inverse problems, to recover the whole solutions from extremely sparse or noisy measurements, by a function-space diffusion model and plug-and-play guidance for conditioning.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n+ This paper is well written and easy to follow.\n\n+ The investigated problem is well-motivated to design the diffusion model in the function space for solving PDE problems. \n\n+ The proposed method looks reasonable to achieve the goal, with promising results achieved compared with previous baseline such as DiffusionPDE.\n\n\nWeaknesses:\n\n- The key contribution lies in the directly extending DiffusionPDE to the function space through function representation. The contribution on the multi-resolution training and inference for acceleration, and conditional sampling algorithm for solving PDE problems may be limited, since they are mostly integrated from previous works such as DiffusionPDE. \n\n- The results with different inverse solvers in Table 5 are interesting. The results show that DAPS and DDNM is far worse than the DPS, which may diverge from the results in previous works. It would be helpful to elaborate what could be the underlying reason, like due to the function space formulation or PDE task?\n\n- A key baseline compared in the paper is the DiffusionPDE, with the main difference in finite v.s. infinite representation space for training diffusion models, with claimed key contribution in extended Tweedie’s Formula. But another possible simpler approach is to represent measurement in the function space and inject to each diffusion sampling step to avoid estimating the Tweedie’s Formula. This was introduced in the diffusion inverse solvers before the introduction of DPS. Would this be possible to solve the target problem?\n\n- The paper claims the sampling from posterior distribution with DPS guidance, while DPS is shown not to provably sample from posterior, such as in [1]. Can the author elaborate more on how to guarantee the posterior sampling as claimed?\n\n[1] Cardoso, G., Janati El Idrissi, Y., Le Corff, S., & Moulines, É. (2024). Monte Carlo guided Denoising Diffusion Models for Bayesian Linear Inverse Problems. In Proceedings of ICLR 2024\n\nQUESTIONS:\nPlease see the weaknesses for specific questions.\n\nLIMITATIONS:\nYes, the author discusses the limitations of the proposed method in the conclusion.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focusses on solving PDE-based inverse problems formulated in infinite dimensions using conditional diffusion processes. They formulate an infinite-dimensional conditional score via a density with respect to a Gaussian measure. They show that this can be decomposed into a likelihood term and prior or unconditional score. The likelihood is approximated in closed form by approximating the conditional reverse diffusion process with its conditional expected value. This expected value is proved to satisfy an infinite-dimensional Tweedie’s formula. Using the resulting closed form approximation of the guidance alongside a trained (unconditioned) score operator allows one to obtain samples from the posterior. The method is compared experimentally to other methods on forward and inverse problems for different PDE tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. In general, I find the submission to be well-written.\n\n2. The method is compared experimentally to well-known PDE solvers across five different PDE experiments, which are the same as those considered in DiffusionPDE. The method performs well on these baselines, outperforming the other methods whilst using a quarter of the time discretisation steps used in DiffusionPDE.\n\n3. The submission proves an infinite-dimensional Tweedie’s formula. Tweedie’s formula is an important result used in finite-dimensional diffusion models. I find its extension to infinite-dimensions to be a valuable result.\n\n4. The paper also details a new training technique for learning the prior distribution at a lower cost using multi-resolution training. This is coupled with what they term multi-resolution inference also accelerating the inference time per step. I think this is a valuable contribution which can also be used in the infinite-dimensional unconditional problem.\n\n\nWeaknesses:\n\n\n1. In order to obtain a closed form expression for the likelihood, the conditional measure of the reverse diffusion process is approximated with its expectation. Can you comment on the error induced by this approximation?\n\n2. The paper introduces the conditional score of $\\nu^u_t$ as the object of interest, however it’s not clear to me exactly how it is used in the given setup for sampling. I think it would be beneficial to make this explicit, for example, by giving the time-reversal of (4) in terms of the conditional score.\n\n3. The submission introduces a lot of different notations for various measures and conditional distributions. I think this is necessary, however I would find a paragraph on the notation useful to refer back to, containing the main measures e.g. $\\mu, \\eta, \\gamma, \\nu$.\n\n\n4. The following weaknesses are minor and do not affect my score.\n\n    - In line 239 “firt” should be “first”.\n\n    - In Equation (13) should the scalar be $\\frac{c^2}{2}$ or $\\frac{c}{2}$? Equation (11) uses $C_{\\eta}^{-1/2}$, which according to line 277 should be given by $c^{1/2}I.$\n\n\nIn summary, I find the paper gives an important and novel contribution in extending diffusion models for Bayesian inverse problems into function space. Experimentally, the method is shown to work well in comparison to existing methods and I recommend accepting this paper. However, I think there are some presentational issues which if fixed would improve the paper. Most importantly, making the infinite dimensional sampling scheme clearer.\n\nQUESTIONS:\nFor Tweedie’s formula, is there anything that can be said about the assumption that the score of $\\nu$ is Fréchet differentiable? When will this hold?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces FunDPS, a novel framework for solving inverse problems involving partial differential equations (PDEs) using diffusion models in function spaces. It combines a discretization-agnostic denoising diffusion model with a plug-and-play guidance mechanism that uses sparse observations for conditional sampling. FunDPS achieves state-of-the-art accuracy and speed, demonstrating significant improvements over prior methods in recovering solutions from as little as 3% of observed data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: The paper adopts a function space formulation of diffusion and can achieve faster and better sampling from experiment. The paper has a clear explanation of the theoretical innovations and is well written. The paper proposes a novel discretization-agnostic framework which can generalize across different resolution\n\nWeakness: \n(1) My main concern is incremental innovation compared to DiffusionPDE, although theoretically fancy, Alorithm 1 has exactly the same procedure as DiffusionPDE, so could you point out what is the main difference with diffusionPDE from the implementation perspective? also why the PDE loss is not included in the loss term? Is it a negative contribution to both DiffusionPDE and FunDPS? If it is, it will be more fair to compare both method without this term together.  \n(2) It will be better to highlight more the intuition and motivation to introducing the functional space especially for this PDE problem, the current version reads more like motivation for diffusion inverse model + functional space diffusion..  \n\n(3) Since the backbone is relatively small scale and influence of hyperparameters will be more obvious, it will be good to add more ablations on the influence of hyperparameters. \n\n(4) Also will be more fair if you can perform careful hyperparameter on the baseline method for different problems as well, especially the error you report here as baseline is much higher than the original numbers reported in the DiffusionPDE paper.  \n\n(5) It will be good to include more diffusion-based PDE solver besides DiffusionPDE, also the model 54M is relatively small, not sure if it can generalize well on more general or ood test datasets.  \n\n(6) Current experiment applies 3\\% observation and will be interesting to see how the method along with diffusionPDE can handle fewer observation and what are the lower limit of observation of diffusion-based approach.  \n\n(7) I did not fully understand the multi-resolution training, but want to know will this training method influence the sampling steps, final accuracy, and does using coarser grid break the 3\\% observation assumption? If this multi-resolution training will influence final accuracy, are you applying it to the diffusion-based baselines as well?\n\n(8) As in DiffusionPDE paper which compares DiffusionPDE with Diffusion with CFG, does it make sense to compare FunDPS with the functional-space diffusion with CFG as well?\n\nQUESTIONS:\n(1) What are the main intuition of generalizing the Tweedie's formula to the function space?  \n(2) When you compare your method with DiffusionPDE, are you using the same neural network backbone architecture?  \n(3) What are the challenges to generalize this algorithm to more practical diffusion usecase like image generation?  \n(4) Do we need to apply any assumption (theoretical or practical) to this inverse problem formulation if we want to extend Tweedie's formulation to infinite-dimensional Banach spaces?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-04"}