Title: Stochastic Process Learning via Operator Flow Matching
Venue: NeurIPS 2025
Accepted: yes
Keywords: Operator Learning

Fourier Neural Operator

Bayesian Methods

Gaussian Process

AI for Science
Abstract: Expanding on neural operators, we propose a novel framework for stochastic process learning across arbitrary domains. In particular, we develop operator flow matching (OFM) for learning stochastic process priors on function spaces. OFM provides the probability density of the values of any collection of points and enables mathematically tractable functional regression at new points with mean and density estimation. Our method outperforms state-of-the-art models in stochastic process learning, functional regression, and prior learning.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a new framework, called Operator Flow Matching (OFM), for learning stochastic processes on function spaces. In brief, the authors extend flow-based generative modelling via Fourier neural operators (FNO) to map a reference Gaussian process (GP) to an unknown target process. Formally, they define a continuous-time ODE on a Hilbert space that “flows” samples from a simple GP prior ($\\nu_0$) to the data distribution ($\\nu_1$). By parameterizing the time-dependent vector field $G_t$ with a neural operator $G_\\theta(t,\\cdot)$, they can (in principle) learn a bijection between function spaces. This yields an explicit transport map so that any finite collection of points from the learned process has a tractable probability density under OFM. Once trained, the model can perform Bayesian regression: given partial noisy observations of a function, it can compute the posterior over the function values at new points (via Bayes’ rule and stochastic gradient Langevin dynamics (SGLD) in the latent GP space). Several claimed contributions are highlighted: (1) Novelty: extending flow-matching or stochastic interpolant methods to infinite-dimensional stochastic processes via operator learning; (2) Unifying prior and posterior inference: providing a unified framework for prior sampling and exact (tractable) likelihood estimation, enabling “universal functional regression” (UFR) with learned priors; (3) Exact densities: learning an exact (invertible) mapping so that the model supplies the true prior and posterior densities over any finite evaluation of the process. Empirically, OFM is tested on several synthetic datasets (1D/2D Gaussian processes, truncated GPs, and highly non-Gaussian processes from Navier–Stokes equations, simulated black-hole data, MNIST-SDF fields).\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n### __1) Integration of flows and neural operators:__\n\nThe paper introduces an original approach by combining flow-matching models with neural operators. By treating functions as first-class objects, OFM learns an infinite-dimensional transport map, which is innovative. This method naturally extends normalizing flows and stochastic interpolant models from finite-dimensional data to functional data. The authors provide a clear formulation of the continuous-time flow on a Hilbert space, showing how a diffeomorphism $\\Phi_t$ pushes a reference GP measure to the data measure. The use of a Fourier neural operator (FNO) to parametrize the vector field $G_\\theta(t,h)$ is appropriate to ensure resolution agnosticism (the model can handle discretized functions at arbitrary grids).\n\n### __2) Mathematically grounded:__ \n\nThe framework is built on solid theoretical ideas from optimal transport and infinite-dimensional analysis. The authors derive a continuity equation for the time-varying measure path and explain how the log-density can be obtained by integrating the divergence of $G_t$. They further leverage Kantorovich’s formulation of dynamic optimal transport together with a linear interpolation strategy for Gaussian measures to obtain a simple vector field that couples a reference GP sample $h_0$ with a target sample $h_1$. While I am not expert in these topics, I find that these constructions are internally consistent and grounded in prior work (e.g. Chizat et al. 2018, Tong et al. 2023). In short, the method is not ad-hoc: it extends finite-dimensional flow matching to function spaces in a principled way.\n\n### __3) Tractable likelihoods and UFR:__ \n\nA key strength is that OFM can compute exact log-likelihoods for any finite set of function values. Once $G_\\theta$ is trained, one can evaluate $\\log p(u_1)$ for a new function evaluation $u_1$ by integrating as in Eq. (15). The paper carefully notes the $\\mathcal{O}(m^2)$ cost of divergence computation and employs the Hutchinson estimator to make it $\\mathcal{O}(m)$. This allows the use of Bayes’ rule for regression. In practice, the authors draw posterior samples via SGLD in the GP input space, then map them to function space. This gives a “universal functional regression” (UFR) procedure that generalizes GP regression to arbitrary learned priors. I find that the fact that OFM retains exact densities (unlike prior implicit models) is a strong advantage.\n\n### __4) Empirical evidence:__ \n\nThe experimental results are compelling. On synthetic GP tasks (1D and 2D), OFM matches or surpasses classical GP performance, and on more challenging non-Gaussian tasks it significantly outperforms baselines. The paper' supplementary material includes ablation and scaling studies to justify design choices, and also provides detailed architecture and training parameters. Moreover, the authors emphasize reproducibility: the code and logs are provided.\n\n\n\n## Weaknesses\n\n### 1) Dense presentation and clarity: \n\nWhile I acknowledge that writing operator learning papers for general ML audience is particularly challenging due to high-level of abstraction and mathematical interaction of diverse fields, I feel that the paper is mathematically dense, which may hinder its accessibility. Many readers may find the notation and measure-theoretic language heavy. For example, Section 3 immediately introduces pushforward measures on a Hilbert space and continuity equations, which can be difficult to parse. While the technical content is correct, the exposition is terse: key concepts like Gaussian measures on function spaces or the Cameron–Martin condition are assumed known. The authors do provide Appendix B–D as “foundational overview”, but the main narrative would benefit from more intuition. The introduction of the conditional flow matching in Sec. 3.2 is similarly rapid, with many equations in succession. Since NeurIPS audience is broad; more high-level guidance or illustrative text would improve clarity (e.g. a step-by-step summary of the two-phase algorithm in plain language). I am aware that this is not an easy task due to the limited length of the main body, but I would still strongly encourage authors to improve on this aspect. \n\n### __2) Novelty relative to recent work:__\n\nThe paper claims to be _the first to extend flow matching to stochastic processes__. However, it cites a recent paper (Ref [29]: “Functional Flow Matching” by Kerrigan et al., 2023) that appears to do something very similar. Indeed, the related work explicitly notes that [29] generalizes flow matching to function spaces. The authors argue that prior works (including [29]) “do not support UFR” or density evaluation, but this distinction should be made clearer. If [29] already develops the theory of infinite-dimensional flows, the novelty here is partly in combining it with operator networks and Bayesian regression. It would strengthen the paper to more explicitly state __“_unlike [29], we can compute densities and do conditional sampling_,”__ perhaps with a succinct comparison. As it stands, __the originality claim needs nuance__: OFM is novel in its application and integration (and outperforms [29] presumably), but flow-in-function-space per se is not entirely new.\n\n### __3) Theoretical assumptions and scope:__ \n\nThe method relies on strong assumptions that deserve discussion. For instance, the reference measure $\\nu_0$ is a GP with some covariance operator, and $\\nu_1$ is assumed to be absolutely continuous with respect to $\\nu_0$ (so that an invertible transport exists). In practice this means the true process must lie in the Cameron–Martin space of the GP prior. The paper does not discuss what happens if this is violated. Also, computing a flow on infinite dimensions typically requires technical conditions (e.g. continuity of $G_t$, compactness of support, etc.). While the appendices address these issues, it would be helpful to acknowledge in the main text any limitations (e.g. “we assume the target measure has full support on the same space as the GP”) and what that implies. Relatedly, the choice of a linear interpolation for Gaussian measures is convenient, but may not be optimal for complex target distributions. To me it is unclear how the approach handles non-Gaussian $\\nu_1$ during training, since the conditional flow matching still assumes $\\mu_t(\\cdot|z)$ is Gaussian. If $\\nu_1$ is far from Gaussian, the learned map may suffer. The empirical results suggest it works, but a discussion of this modeling assumption would strengthen the theory section.\n\n### __4) Empirical evaluation details:__\n\nWhile the results are promising, some experimental questions remain: \n- (a) Statistical robustness: The reported results (e.g. Table 2) do not include confidence intervals or multiple-trial averages. It is unclear how sensitive OFM is to random initialization or data sampling. \n\n- (b) Baselines and fairness: The baselines chosen (GP, deep GP, NP family, “OPFlow”[1]) are reasonable, but other recent methods could be considered. For example, Neural Processes with latent flows or diffusion-based process models (e.g. [34] Neural Diffusion Processes) might be relevant, although those are mostly unconditional. \n\n- (c) Computational cost: The paper notes in passing that “OFM demands more computational resources”, but does not quantify it in the main text. From Appendix tables we see training can take many GPU-hours and memory (dozens of GB). This could limit applicability, especially on high-resolution 2D/3D domains. Including some runtime/memory comparison or at least discussing scalability would be useful.\n\n### __5) Lack of ablation in main text:__\n\nThe contributions bullet promises _“extensive ablation and scaling studies”_, but no ablation figures appear in the main body. It would help to include at least one ablation table or figure (for example, the effect of using Hutchinson estimation vs exact divergence, or the impact of neural operator depth) in the paper, rather than hiding all such results in appendices. This would boost confidence that each component (e.g. neural operator architecture, trace estimator, SGLD sampling) is necessary and correctly implemented.\n\n### __6) Evaluation on realistic tasks:__ \n\nAll experiments are on synthetic or simulation-based data. While this is acceptable for a methods paper, testing on at least one real-world dataset (e.g. geospatial data, time series, etc.) would demonstrate broader applicability. Also, the noise model is fixed to i.i.d. Gaussian in all cases; exploring non-i.i.d. or non-Gaussian noise could be informative, since many applications (e.g. cosmology, climate) have complex noise.\n\nQUESTIONS:\nNA\n\nLIMITATIONS:\nNeither limitations of the proposed framework and its theoretical analysis, nor negative impact have been discussed in the main body. The high computational cost is hinted in the conclusion, but I feel that this deserves more serious report and discussion in the main body rather than in Appendix N.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel method, Operator Flow Matching (OFM), for learning stochastic processes. The approach consists of two key phases. First, it learns a neural operator that transforms samples from a Gaussian Process (GP) into samples from the target data distribution. Then, this learned operator is used to generate posterior samples given noisy observations. To address scalability and computational efficiency, the authors incorporate Stochastic Gradient Langevin Dynamics (SGLD) into the training procedure.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is the first to extend flow matching to stochastic process learning via operator learning.\n2. By leveraging a learned operator, the method can directly model data functions, enabling access to the exact posterior distribution over functions.\n3. The approach is validated on several synthetic GP datasets and demonstrates improved performance compared to existing methods.\n\nWeaknesses:\n1. The paper lacks mathematical rigor and clarity in presentation.\na) It is not immediately clear that Eq. 14 is a solution to Eq. 11, nor is this connection explicitly stated. The authors are encouraged to include propositions or theorem-style statements to improve readability. Each phase should clearly state the optimization problem and how it connects to the method.\nb) In line 230, the authors write: \"For evaluating ... on the right-hand side of Eq. 18, we can efficiently calculate it with the likelihood estimation tool described above.\" It is unclear which likelihood estimation tool is being referenced. The statement should be made more precise.\n\n2. The paper lacks a high-level explanation of flow matching, making it difficult to understand the intuition behind why this method might outperform alternatives like Neural Processes (NP). A conceptual overview of how flow matching contributes to the method would improve accessibility.\n\n3. While an ablation study is included, it only focuses on the mini-batch size for optimal transport. However, the method involves other important hyperparameters. A broader sensitivity analysis would improve the robustness of the empirical evaluation.\n\n4. The paper only evaluates on synthetic GP datasets. Real-world datasets commonly used in Neural Process literature—such as MNIST and CelebA for image completion—are missing. Including such benchmarks would strengthen the empirical impact (significance) of the work.\n\nI would be inclined to raise my score if the clarity issues are addressed and additional experiments on real-world datasets and hyperparameter sensitivity are included.\n\nQUESTIONS:\nBesides the questions raised in the weaknesses, I also wonder how Eq. 16 is computed in practice and why its complexity is claimed to be $O(m)$. A more detailed explanation would help clarify this point.\n\nAdditionally, the proposed method appears to align well with the denoising literature. It would be interesting to see how the flow matching framework performs on denoising tasks, which could further demonstrate its versatility and practical value.\n\nLIMITATIONS:\nThe authors have discussed the limitations in the Appendix, and the identified issues are common in both the Neural Process (NP)and the operator learning literature.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The authors introduce a method to perform flow matching in function spaces relying on a generalization of modelling functions to functionals. They demonstrate the applicability of the proposed approach to academic examples.

STRENGTHS AND WEAKNESSES:
## Strengths
**S1: Interesting Generalization** The studied problem of generalizing flow matching to functionals is interesting and novel. It addresses a common limitation of CNP that can be of relevance in practice. The derivations seem correct to me.

**S2: Insights into the model** The experiments are described in detail, and the appendix provides comprehensive material on the analysis and comparison to other methods.

## Weaknesses

**W1:  Important insights are pushed to the appendix.** I appreciate the detailed appendix, but limitations or a short introduction on the main neural network baseline, the ConvCNP, should not be pushed to the appendix.

**W2: Excessive Data Requirements** The main strength of using GP and Bayesian methods in general is their performance in low-data regimes. The data requirements seem excessive for the low-dimensional examples provided. I read the limitation section in the appendix, but IMO it should be part of the main paper, as neither the data requirement nor the limitation to low-dimensional settings are mentioned in the main text.

QUESTIONS:
**Q1: Scalability** The authors mention, their method is limited to low-dimensional settings and, so far, to problems where a simulator is available for unlimited data generation. Can you comment on which theoretical or practical advancements would be needed to overcome these challenges and apply the OFM method to real-world problems?

LIMITATIONS:
Limitations are discussed, but should be included in the main text.
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Operator Flow Matching (OFM), an extension of (optimal transport) conditional flow matching (CFM) to stochastic processes using neural operators. In a nutshell, the authors show that the vector fields in the CFM objective can be replaced with neural operators, thus generalising CFM to (infinite-dimensional) function spaces. This allows for evaluating the posterior probability of observations under any stochastic process as well as efficiently sampling from the process.\n\nThe authors evaluate their method on a range of experiments from simple synthetic 1D Gaussian and non-Gaussian data, where the true posterior is available, to more advanced data from simulations and MNIST. It shows promising performance, although at a high computational budget.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper combines two recent and promising techniques, flow matching and neural operators. It appears original and should be of broad interest to the NeurIPS community.\n2. The model itself is rigorously developed.\n3. The model appears to work well in practice, despite some computational concerns (see weaknesses).\n4. I could see this work forming the basis for many follow-up papers, so it could become quite significant.\n\n\n**Weaknesses**\n1. The main weakness of the paper, to me, is its clarity. It is very advanced material, which the authors are aware of and have tried to address by adding an FAQ and additional details to the appendix. While these are helpful, the main paper is still very detailed, and the sentences are long and dense, which makes the work difficult to understand.\n2. As the authors themselves point out, the model is computationally heavy and is currently limited to low-dimensional data (the authors experiment on just 1D and 2D data). \n3. The experimental section is a little weak, with many of the results deferred to the appendix. Some of the results are purely qualitative. The only quantitative experiments are some simple 1D and 2D synthetic datasets, which should at least have been extended to more complex examples (e.g., more complex GP kernels would be easy to test).\n\nQUESTIONS:\n**Suggestions**\n1. My main suggestion would be to work on the clarity of the paper. For instance, it would have helped me to have a basic introduction to neural operators and (conditional) flow matching in the appendix. Also, I would suggest moving the technical details of the main paper to the appendix, and focusing the main paper on building intuition for your model and how to use it, perhaps with some proof sketches if necessary. \n2. The figures and tables in the experiments section are placed rather chaotically, and table 1 is never mentioned, which is all somewhat distracting. It would help the reader if figures and tables were placed on the same page as where they are mentioned.\n\n**Questions**\n1. You mention that you extend neural operators to map between collections of points. Can you clarify how this is different from a standard flow?\n2. In Eq. (11), what is the argument for replacing the suprema in Eq. (8) with expectations? When is this valid?\n3. The OFM posterior samples in figures 8, 10, and 11 look quite noisy. Do you have ideas as to why this could be and how one might mitigate it?\n4. For the GP experiments, the training set seems to be constructed from samples from the same GP (i.e., with fixed hyperparameters) that was also used for the test sample. This seems like an unrealistic setting in general. Is OFM able to learn from samples from GPs with different hyperparameters? If so, did you try that?\n5. A minor question regarding the 1D GP experiment to make sure I understand the setup correctly: you write in appendix K that you sample the training set at 256 input locations, but table 3 says that you use a batch size of 1024 during training? Is this a mistake?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}