Title: FNOPE: Simulation-based inference on function spaces with Fourier Neural Operators
Venue: NeurIPS 2025
Accepted: yes
Keywords: simulation-based inference

Fourier Neural Operators

flow matching

applications
Abstract: Simulation-based inference (SBI) is an established approach for performing Bayesian inference on scientific simulators. SBI so far works best on low-dimensional parametric models. However, it is difficult to infer function-valued parameters, which frequently occur in disciplines that model spatiotemporal processes such as the climate and earth sciences. Here, we introduce an approach for efficient posterior estimation, using a Fourier Neural Operator (FNO) architecture with a flow matching objective. We show that our approach, FNOPE, can perform inference of function-valued parameters at a fraction of the simulation budget of state of the art methods. In addition, FNOPE supports posterior evaluation at arbitrary discretizations of the domain, as well as simultaneous estimation of vector-valued parameters. We demonstrate the effectiveness of our approach on several benchmark tasks and a challenging spatial inference task from glaciology. FNOPE extends the applicability of SBI methods to new scientific domains by enabling the inference of function-valued parameters.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes FNOPE, a simulation-based inference method that leverages Fourier Neural Operators with a flow-matching objective to infer function-valued parameters, including over irregular discretizations. The authors demonstrate the method across several benchmark and real-world scientific simulation problems. While the paper studies a problem of impacts, I find it difficult to identify a clear novelty in its core contribution.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: Nothing stood out as commendable.\n\nWeaknesses:\n- My main concern lies in the novelty and positioning of the contribution. Using neural operators, e.g., FNOs, to approximate mappings between function spaces is well-established. Applying them to inverse problems, including probabilistic ones, has been widely explored in recent works such as Invertible FNO (iFNO), Latent Neural Operators (LNO), and Neural Inverse Operators (NIO). If the primary novelty is the integration of NUDFT to support irregular meshes, that seems incremental, as this mechanism is neither new nor unique to this work. Several transformer-based neural operators also support irregular discretizations and are known to perform well, especially under sparse observations, as acknowledged by the authors themselves in the SIRD task (Fig. 4, pg. 6).\n- Another problem is that I fail to see a fair baseline comparisons against other neural-operator-based inverse methods. It is unclear why the authors exclude these from the baseline set. Without such comparisons, it is difficult to assess whether FNOPE contributes meaningfully beyond existing neural operator architectures tailored for inverse tasks.\n- While the paper includes multiple benchmarks, it lacks deeper ablations that would clarify which components are key to the method's performance. For instance, the impact of NUDFT versus FFT, the number of retained Fourier modes, or the role of GP noise shaping is not systematically studied. This makes it hard to disentangle whether FNOPE’s gains arise from architectural choices, training strategy, or task-specific tuning.\n\n1. Long, D., Xu, Z., Yuan, Q., Yang, Y. and Zhe, S., Invertible Fourier Neural Operators for Tackling Both Forward and Inverse Problems. In The 28th International Conference on Artificial Intelligence and Statistics.\n2. Molinaro, R., Yang, Y., Engquist, B. and Mishra, S., 2023, July. Neural inverse operators for solving PDE inverse problems. In Proceedings of the 40th International Conference on Machine Learning (pp. 25105-25139).\n3. Kaltenbach, S., Perdikaris, P. and Koutsourelakis, P.S., 2023. Semi-supervised invertible neural operators for Bayesian inverse problems. Computational Mechanics, 72(3), pp.451-470.\n4. Wang, T. and Wang, C., Latent Neural Operator for Solving Forward and Inverse PDE Problems. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.\n5. Behroozi, A., Shen, C. and Kifer, D., Sensitivity-Constrained Fourier Neural Operators for Forward and Inverse Problems in Parametric Differential Equations. In The Thirteenth International Conference on Learning Representations.\n6. Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. GNOT: a general neural operator Transformer for operator learning. In Proceedings of the International Conference on Machine Learning (ICML), 2023.\n7. Wu, H., Luo, H., Wang, H., Wang, J. and Long, M., 2024, July. Transolver: A Fast Transformer Solver for PDEs on General Geometries. In International Conference on Machine Learning (pp. 53681-53705). PMLR.\n\nQUESTIONS:\nFNOPE appears to be a technically sound implementation of familiar ideas, but as presented, it lacks sufficient novelty and comparative context to justify its contributions. Please consider \n- Clarify the novelty of the method in the context of existing operator-based inverse problem literature. \n- Include direct comparisons with other neural operator methods. \n- Conduct deeper ablation studies to isolate the impact of NUDFT, GP priors, and other design decisions. \n- Strengthen the motivation for using FNOs specifically over more flexible architectures that already support irregular discretizations.\n\nLIMITATIONS:\nI suggest the authors to more clearly articulate the novel contributions of the paper, especially in light of extensive prior work on using neural operators for inverse problems in functional spaces. Please strengthen the positioning and clarify the methodological innovations.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors introduce a novel approach to Simulation-based inference (SBI) by integrating Fourier Neural Operators (FNO) with a flow matching objective. The method enables high-dimensional inference while requiring several orders of magnitude fewer simulations than traditional SBI techniques. The superior performance is demonstrated across different simulated case studies and one real-world case study.\n\nAdditionally, they are able to produce posterior samples on arbitrary discretizations and avoid reliance on any additional basis functions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. _Innovative combination of FNO with a flow-matching training scheme:_ The FNO component effectively learns a compact representation of the data's global structure. On the other hand, the flow-matching training scheme helps perform likelihood-free inference of conditional distributions.\n\n2. _Efficient posterior estimation in high-dimensional parameter space:_ The authors of the paper demonstrate efficient performance of FNOPE on high dimensional data under limited simulation budgets compared to FMPE, NPE and other benchmarks. Crucially, it achieves this performance without relying on fixed discretizations.\n\n3. The authors demonstrate FNOPE's excellent performance across different synthetic benchmarks and a real-world application. \n\nWeaknesses. Despite the strengths, the paper has certain weaknesses that need to be addressed:\n1. The absence of ablation studies on key hyperparameters makes it difficult to assess the robustness of the method. Understanding this would clarify different performance bottlenecks. (Check limitations and questions for more details)\n\nQUESTIONS:\n1. How sensitive is the method to the number of Fourier modes retained in the FNO layers? An ablation study to assess the trade-off between accuracy and computational cost would help.\n2. How dependent is the model's performance to the choice of prior distribution used? Would deviations from the assumed prior significantly degrade performance?\n\nLIMITATIONS:\n1. Several important evaluation metrics, like SWD, are not formally defined in the appendix. For example, the loss is referenced briefly on line 869 without an accompanying equation or detailed explanation.\n2. Key theoretical concepts referenced in the main text (Bochner's theorem - line 120) are not formally stated or explained in the appendix.\n3. The paper lacks ablation studies on several key hyperparameters. For instance, an ablation on the number of discretizations, number of fourier modes retain etc could help better understand performance bottlenecks of FNOPE.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a method for performing simulation-based inference (SBI) of function-valued latent variables. The method is a combination of flow-matching posterior estimation (FMPE) and Fourier neural operators (FNOs): FMPE provides the objective function, while FNOs provide a means of parameterizing function-valued outputs given $x$ (plus its discretization) as input. A key innovation of the authors is their explicit conditioning on the (variable) discretization positions $l^\\theta, l^x$ in both the generative (simulation) process and the inference procedure. This formulation allows for flexible choice of discretization grid, in particular the use of *any* grid desired at inference time once the model is fitted.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The exposition is clear and overall the paper is well written and easily understandable.\n- The use case for practitioners is an important one; NPE and other simulation-based methods are increasingly popular, and performing inference on increasingly complex quantities like function-valued latents  is a useful advancement (although I’m less familiar with other related work in the function-valued SBI space that may tackle similar problems).\n- The sample efficiency point is compelling; some simulators are expensive to sample, so doing well with fewer samples is necessary.\n\nWeaknesses:\n- The paper makes claims about the difficulties of high-dimensional inference problems for SBI without talking about the marginalization properties of NPE – this is an important point that I think needs to be addressed (more on this in next section).\n- The sample efficiency results are empirical in nature, rather than a theoretical guarantee. In experimental settings, it seems that NPE (spectral) can perform as well as the proposed method with enough simulations (10^4 or 10^5). \n- The experimental results do not show the degree of improvement over NPE or related methods that one might expect relative to the claims. Also, some of the metrics for evaluation of the experiments raise questions about marginalization as well. \n- The method is somewhat combinatorial, as both FMPE and FNOs are established approaches in the literature.\n\nQUESTIONS:\nLet me start by providing more detail on the marginalization point discussed above. Imagine a high-dimensional latent variable $\\theta \\in \\mathbb{R}^{100}$, for example. In NPE, a common approach to avoid high-dimensional difficulties is to use a mean-field factorization, i.e. parameterize $q(\\theta \\mid x)$  as $q(\\theta_1 \\mid x) q(\\theta_2 \\mid x) \\cdots q(\\theta_{100} \\mid x)$. This yields 100 univariate inference problems to solve. Certainly, fitting a hundred different networks is less than ideal, but it can be done. What is lost in this approach is posterior correlations between the latent variables, but in some settings this may be acceptable.\n\nFor the type of data in this work, losing correlations is unacceptable: if $\\theta$ represents evaluations of a smooth function  on a grid, $\\theta_{76}$ and $\\theta_{77}$ are highly correlated, for example; they should not deviate from each other much if the grid being evaluated on is fine enough. A discussion similar to the above I think might be a useful addition to the paper, as function-valued latent variables make a compelling case to *not* use the mean-field approach with its marginalization properties. You may want to read or cite Forward Amortized Variational Inference (Ambrogioni et al.) or works that cite this one for more on marginalization.\n\n\nQuestions:\n- The proposed method uses the FMPE objective. I wonder if you tried (or if it possible to formulate) a variant based on the NPE objective. Does this exist in other related work? Presumably it would look something like \n$$\n-\\mathbb{E}_{l^\\theta, \\theta, l^x, x} \\log q^\\phi(\\theta \\mid x, l^x, l^\\theta).\n$$\n- The section on “inferring additional parameters” seems less useful for the same marginalization argument presented above. If a user has to perform inference on both a parameter $\\eta$ and a function valued latent $\\theta$, I suspect in practice one might fit the mean-field variational posterior $q(\\theta, \\eta \\mid x) = q(\\theta \\mid x)q(\\eta \\mid x)$ with the first term fit by FNOPE and the second by a standard NPE approach. Again, unless posterior correlations between $\\eta, \\theta$ are desired, fitting a joint posterior is unnecessary.\n- The lack of discussion on marginalization clashes with the fact that SBC on the marginals is used as an evaluation metric (line 183). If the quality of the marginals is what’s being evaluated, modeling posterior correlations seems unnecessary. Adding some type of NPE competitor based on a mean-field parameterization would help resolve this point. \n- Can you explain the MSE metric in more detail? What two quantities is the MSE computed between? For a fixed evaluation point $i$, is it between the ground truth draw $\\theta_i$ and a single posterior sample at $i$? Or something else?\n- In the first experiment, can you explain intuitively why FNOPE (fix) outperforms FNOPE? \n- It’s difficult to gauge a take-away message from the second experiment given that FNOPE does not outperform the Simformer. \n- In the third experiment, NPE (spectral) seems to achieve the same MSE as the proposed methods. Yet, the images in panel (a) for NPE look much poorer relative to the ground truth. Why is this?\n- In the third experiment, is NPE (spectral) present in panel d), or not? If not, why?\n- In the third experiment, it’s stated that FNOPE uses 32 Fourier modes, while NPE/FMPE only 16 (line 248). Should these be equal for a fair comparison?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes to combine flow matching and Fourier neural operators to extend simulation-based inference to model posteriors over function-valued parameters, e.g., parameters that vary in time and/or space. To do this, the authors use Fourier neural operators to model the flow field and further adapt them to handle non-uniform input data using a non-uniform fast Fourier transform. The full model is able to handle both function-valued and vector-valued parameters.\n\nThe authors test the proposed model on several synthetic benchmark tasks, as well as a real-world example involving glacial flow. The method is generally on par with or outperforms baselines, requiring orders of magnitude fewer simulations to achieve the same accuracy, and having more realistic posterior samples.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper presents an interesting solution to an important problem in simulation-based inference by combining two recent and promising techniques, flow matching and Fourier neural operators. \n2. The model appears to work well in practice and requires many fewer simulations than the baselines, although there still are some computational concerns (see weaknesses).\n3. The paper is well-written and should be of broad interest to the NeurIPS community.\n\n**Weaknesses**\n1. The main weakness of the paper is the computational complexity of the proposed model, which scales exponentially with the parameter dimensionality, both in the number of parameter discretisation points and the number of Fourier modes considered. So while the model seems like an interesting approach, it is unlikely to be useful in many scenarios in its current form.\n2. The model appears to be powerful in the sense that it generally seems to require only a fraction of the number of simulations that the baselines do, but it is unclear how big the advantage is in actual compute since training and evaluation times are not reported.\n3. The hyperparameters of the model seem to have been chosen heuristically and not compared to other choices.\n\nQUESTIONS:\n1. Did you measure the training and evaluation (wall-clock) times for each of the methods you consider? I'm curious to see how FNOPE compares to baselines. Also, how stable is the training?\n2. The hyperparameters of the model seem to have been chosen somewhat heuristically. Did you investigate other choices? For instance, did you try other kernels for the Gaussian process prior, e.g., Matérn?\n3. Further to the question above, you choose to fix the length scale of the Gaussian process kernel to a particular value, but would it not be possible to optimise it? Is there a reason not to do so?\n4. Simformer seems like a strong baseline. What is the reason for not comparing against it in all of your experiments?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-09"}