Title: Flow Matching Neural Processes
Venue: NeurIPS 2025
Accepted: yes
Keywords: flow matching

neural processes
Abstract: Neural processes (NPs) are a class of models that learn stochastic processes directly from data and can be used for inference, sampling and conditional sampling.  We introduce a new NP model based on flow matching, a generative modeling paradigm that has demonstrated strong performance on various data modalities. Following the NP training framework, the model provides amortized predictions of conditional distributions over any arbitrary points in the data. Compared to previous NP models, our model is simple to implement and can be used to sample from conditional distributions using an ODE solver, without requiring auxiliary conditioning methods.In addition, the model provides a controllable tradeoff between accuracy and running time via the number of steps in the ODE solver. We show that our model outperforms previous state-of-the-art neural process methods on various benchmarks including synthetic 1D Gaussian processes data, 2D images, and real-world weather data.


Peer reviews:
comments: SUMMARY:
This paper proposes a flow matching neural process (FlowNP), which combines the neural process with flow matching by using flow matching to generate target value given target position and context. The proposed FlowNP is simple to implement and can sample from conditional distributions without auxiliary methods. Moreover, the model provides a controllable tradeoff between accuracy and running time via ODE solver steps. Empirically, they show that FlowNP outperforms SOTA in 1D GP, 2D image and weather data.

STRENGTHS AND WEAKNESSES:
**Strength**: the idea is simple and show improvement from previous NP methods. The writing is clear and very easy to follow.

**Weakness**: The proposed method combines FM and NP in a straightforward way, which may raise questions regarding novelty. However, if it works fine, I’m happy to solve problems in a simple way. But I’m not sure if overlaying FM with NP is necessary: what’s the benefit of FlowNP, comparing to using FM directly (FM is flexible enough, and can be used for inference, sampling and conditional sampling)? If the improvement is not very significant but can potentially give insights/ inspirations to new methods, I’m also OK with it.

QUESTIONS:
See my question in weakness: what’s the gain for overlaying FM with NP? Can it significantly beat FM, or give us some insight/ inspiration for new methods?

LIMITATIONS:
The paper already discussed their limitations: iterative sampling and likelihood computation.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose to train a model in the neural process framework with the flow matching algorithm. In their experiments, they outperform other neural process models on in-painting tasks for images and weather data and in terms of test-set likelihood.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is easy to follow and the authors evaluate their proposed model on a variety of data types.\n\nWeaknesses:\n1. The abstract claims to outperform state-of-the-art methods on various benchmarks but only includes neural process models.\n2. Line 192: The claim that \"it is independent of the number of target points $N$\" is slightly misleading. While the number of function evaluations is independent of it, the run time of each individual function evaluation does scale with $N$.\n3. Line 273: The authors claim that FlowNP is the fasted method in Figure 2. However, ANP is actually the fastest by a factor of 100.\n4. Table 2 and Line 281: You report continuous log-likelihoods, which is why they can grow without bounds. However, the standard for images would be to report negative log-likelihoods of the discretized data in bits per dimension, see for example Appendix B in [1] for an explanation. Then, your numbers could also be compared to other models directly, e.g. diffusion models or neural ODEs.\n5. Figure 4 does not include true data for reference.\n6. Both data and samples of CelebA in Figure 7 are much blurrier than they should be. Did you downsample the data? If so, did you make sure to plot the images at the same resolution that they actually are? If I zoom in, it looks like a blurry high-resolution image.\n\nRemarks:\n1. In Section 2.1, all $y_{i:n}$ should be $y_{1:n}$, I believe.\n2. The caption of Figure 2 has a typo in \"condinting\".\n\n[1] Lienen, Kollovieh and Günnemann. \"Generative Modeling with Bayesian Sample Inference.\", https://arxiv.org/abs/2502.07580\n\nQUESTIONS:\n1. Line 190: Why does TNP require $2N + M$?\n2. Figure 2: Which kernels are in which column? What do true samples look like for reference?\n3. Do I understand Appendix A correctly that FlowNP is conditionally consistent but not marginally consistent?\n4. What is $\\alpha_t$ in Algorithm 2?\n5. How well does your model perform on unconditional image generation? Would it reduce to flow matching?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-11"}

comments: SUMMARY:
The paper introduces FlowNP, a novel Neural Process (NP) model that uses flow matching for learning stochastic processes from data. FlowNP is designed to address limitations of existing NP models, such as underfitting, sequential sampling inefficiencies in autoregressive models, and lack of global uncertainty representation. The model uses a transformer architecture to predict velocity vectors for target points, allowing parallel sampling via an ODE solver. Experiments are provided to demonstrate that FlowNP outperforms existing NP models in terms of log-likelihood and sampling efficiency.

STRENGTHS AND WEAKNESSES:
Strengths:
- The integration of flow matching with NPs is interesting, providing a new way to model conditional distributions without relying on autoregressive or auxiliary conditioning methods 
- The proposed parallel sampling method improves efficiency during generation, as it avoids sequential generation of autoregressive approaches 

Weaknesses: 
- Lack of consistency guarantees, since consistency (marginal and conditional) is not formally guaranteed
- Lack of scalability evaluation, as it is not clear how well the method would scale to settings with very large context/target sets (e.g., data with higher resolutions)
- Lack of ablation study on the design of the flow matching framework (sensitivity to choice of noise schedule/probability path, choice of ODE sampler, etc.)

QUESTIONS:
- Can FlowNP be used to provide quantification of epistemic uncertainty? 
- Is FlowNP robust to noise perturbation in the training datasets?
- Does FlowNP suffer from spectral bias (over-smoothing high frequencies)?
- What are the limitations of the proposed method? For what data regimes the method excel and fail?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-26

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- The authors use Flow Matching to improve sample generation from Neural Processes.\n- The authors use the known ability of transformers to aggregate context data points very effectively. \n- The authors use Flow matching as a method to generate coherent samples with more expressive densities than Transformer Neural Processes and better guidance than Neural Diffusion Processes. \n- The authors benchmark their method against existing approaches on both synthetic and real world data sets,  where  their method outperforms alternatives.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- Well written paper with a good background for the field.\n- Good benchmarking, building from simpler to more complex examples.\n- NPs are an exciting field which show great promise in meta learning priors and performing inference and the proposed approach significantly improves the ability of the proposed method to make coherent predictions at large numbers of points as well as providing accurate predictions.\n- While proposed method did not seem to require significant theoretical advances on previous methods, the two selected techniques compliment one another very well and solve a significant issue with Neural Processes.\n\nWeaknesses \n- The authors refer to their method as using an ODE solver for sampling, however they add noise in their sampling method, so their method more closely resembles sampling using Langevin dynamics. This addition to flow matching raises two concerns for me: \n\t- Empirically, I would have liked to see an ablation study comparing performance without this additional element.\n\t- Theoretically, this significantly changes how I would think about the proposed method, to something closer to an energy based model sampled with Langevin dynamics. I believe that this detail merited significantly more discussion in the main body of the paper.\n\nQUESTIONS:\n- How did your results look without the added noise in the sample generation? \n- Why are ERA5 results not reported with standard deviation?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

comments: SUMMARY:
This paper proposes a probabilistic model called flow matching neural process that trains a neural process with the flow matching objective.
The main idea is to train a conditional flow matching model that maps from a Gaussian to the desired distribution at the target location.
They use a transformer architecture to parameterize the vector field so that the flow model is able to condition on arbitrary number of context tokens.

STRENGTHS AND WEAKNESSES:
1. The idea seems to be very clean---combining transformers and flow matching objective for neural processes.
1. The empirical performance seems quite strong.
The proposed flow matching neural process outperforms all baselines consistently.

QUESTIONS:
1. As mentioned by the author, diffusion neural processes have already been published before.
Given the connection between diffusion models and flow matching models (in fact some would argue they are equivalent), I wonder if the method proposed in this paper apply to diffusion neural processes.
Also, the authors perhaps should elaborate a bit more on the key difference between this paper and diffusion neural processes.

LIMITATIONS:
NA
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-07