Title: Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions
Venue: NeurIPS 2025
Accepted: yes
Keywords: Neural stochastic flows

stochastic differential equations

normalising flows

continuous-time models

latent dynamics

time series modelling
Abstract: Stochastic differential equations (SDEs) are well suited to modelling noisy and/or irregularly-sampled time series, which are omnipresent in finance, physics, and machine learning applications. Traditional approaches require costly simulation of numerical solvers when sampling between arbitrary time points. We introduce Neural Stochastic Flows (NSFs) and their latent dynamic versions, which learns (latent) SDE transition laws directly using conditional normalising flows, with architectural constraints that preserve properties inherited from stochastic flow. This enables sampling between arbitrary states in a single step, providing up to two orders of magnitude speedup for distant time points. Experiments on synthetic SDE simulations and real-world tracking and video data demonstrate that NSF maintains distributional accuracy comparable to numerical approaches while dramatically reducing computation for arbitrary time-point sampling, enabling applications where numerical solvers remain prohibitively expensive.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents neural stochastic flows, a novel framework for modelling trajectories of stochastic differential equations for time series data. The method utilizes normalising flows to get a direct access to the probability distribution of SDE transitions, which allows it to obtain a sample from an arbitrary time step without requiring a numerical solver. Experimental results show that the proposed approach significantly speeds up the sampling process while maintaining high sample quality.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**   \nThe paper structure is good, the proposed method is well-motivated and its theoretical justification is strong.\n\n**Weaknesses**   \nThe experimental evaluation lacks clarity and depth in analysis (see Questions section). Some of the important implementation details are not presented in the main text of the paper. For example, the architecture of the proposed normalizing flow is a crucial part of the approach and it should not be moved to the appendix.\n\nQUESTIONS:\nAll my questions are related to the Experiments section.\n\n**Stochastic Lorenz Attractor**   \n1. In Table 1, the meaning of the variable $t$ is unclear. What does it represent, and why does the KL divergence vary non-monotonically with changes in $t$?\n2. Why does NSF with $t_{\\text{max}} = 0.5$ perform worse than NSF with $t_{\\text{max}} = 1$ and $t_{\\text{max}} = 0.25$?\n3. How does Figure 3 relate to Table 1? Were the depicted trajectories generated in a one-step fashion, or sequentially by conditioning each point on the previous one?\n\n**CMU Motion Capture**   \n1. In the Latent SDE paper [1] I see that the reported MSE for Setup 1 is 4.03, which is notably better than the result of your reproduction. Please explain this discrepancy.\n2. Figure 4 (a) clearly indicates that MSE grows with increasing runtime for Latent NSF. How can you explain this behavior?\n3. You attribute the better performance of Latent NSF in Setup 2 to the SDE's inductive bias. Could you please clarify the idea of inductive bias here and elaborate on why Setup 2 constitutes extrapolation while Setup 1 does not for this particular dataset?\n\n**Stochastic Moving MNIST**   \n1. The variance of the Fréchet Distance in Table 3 and Figure 5 appears high relative to the mean differences between methods. This raises concerns about the statistical significance of the results. How many samples were used to compute these metrics, and what is the level of significance?\n2. There is no analysis of the results. How can you explain that recursive NSF is much better then SDE in terms of Static FD and much worse in terms of Dynamics FD? How does the speed of recursive NSF compare with one-step NSF? Are the results aligned with expectations, and how do they compare with findings on the other datasets? Why does recursive NSF perform better for Moving MNIST, while appearing to underperform on the Motion Capture dataset?\n\n**General questions**   \n1. I noticed that you use different dimensionality of latent space for different methods (Appendix E.3 and E.4), which might have an impact on quality. Could you please explain how you chose these values and all other hyperparameters, and how different methods compare in terms of the number of parameters?\n2. All experiments are conducted on relatively small-scale problems (latent space dimensionality ≤ 10). Is the proposed method applicable to larger-scale settings, such as in the video domain?\n\n[1] X. Li, T. Wong, R. T. Q. Chen, and D. Duvenaud. Scalable gradients for stochastic differential\nequations. In Proceedings of the 23rd International Conference on Artificial Intelligence and\nStatistics (AISTATS), 2020.\n\nLIMITATIONS:\nThe limitations are discussed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose to use conditional normalizing flows to approximate transition densities in SDE. This yields a solver-free method to sample between arbitrary timepoints in a single step. \"Neural stochastic flows (NSF)\" is a conditional coupling flow aiming to estimate transition densities $p(x_t|x_s, s, t-s)$ of SDEs (for all s,t). The authors enforce that some known properties of them are architecturally embedded, and develop a regularization term to enforce the \"flow\" property. It's shown in experiments on synthetic SDEs and real-world tracking and video data that NSF stays distributionally accurate while strongly reducing computational cost.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Well written, structured, and motivated, i.e., first pointing out the target's properties, then incorporating them in the model.\n- Overall, the empirical results show that the proposed method is accurate and comparable to other approaches, but much more computationally efficient.\n\nWeaknesses:\n- One of the main methodological novelties is the regularization term for the Flow property. However, it only occupies ~0.5 page in the method section and some design decisions about it remain unclear (i.e., \"A natural design for such a regularization loss is to combine both forward and reverse KL ...\" Why is this the natural choice? There are several other possibilities to achieve the same. \n- Missing ablations e.g. given that this regularization term adds quite a lot of complexity to the approach, I miss an ablation that this term does empirically improve upon the standard maximum likelihood loss (which should enforce the Flow Property implicitly in a data-driven manner). How much slower is it to train with regularization? I expect quite a bit.\n- Table 2 raises questions: SDE Matching performs best in Setup 1 (by quite a big margin compared to the proposed method), but results for this approach on Setup 2 are not reported (why?). Given that the authors claim \"state-of-the-art long-horizon extrapolation performance\" this result should be included.\n\nQUESTIONS:\nSee weaknesses i.e.:\n- How much does the regularization term help (compared to no regularization) ?\n- Why forward and backward KL for regularization? This also introduces the \"weighting\" hyperparameters. As reverse and forward KL are often on quite different \"scales,\" this might be important to \"tune\". Did the authors investigate just using the forward term (or just the backward term)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes neural stochastic flows, which learn transition laws via conditional normalizing flows. The normalizing flows are constructed in such a way that the identity and Markov properties are respected and trained with an additional regularization loss for the flow property/Chapman Kolmogorov equation. The model is tested on three experiments, improving over baselines in terms of speed and performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is well written and the method seems mathematically sound\n- Neural stochastic flows show strong empirical performance while being much faster than non solver-free methods. \n\nWeaknesses:\n- A lot of the desired properties of the flows are incentivized through additional loss terms leading to multiple hyperparameters (K, $\\lambda$, $\\lambda_{\\text{1to2}}$, $\\lambda_{\\text{2to1}}$, $\\beta$, $\\beta_{\\text{skip}}$). It is not clear how to best select all these parameters making extensive hyperparameter tuning necessary. How sensitive is the method to the choice of hyperparameters in practice? \n- The dependence on affine coupling layers makes the method not very scalable. Is there a way to lift this dependence?\n\nMinor: \n- l26: before consistency models, general distillation approaches for diffusion models also tried to achieve this using the probability flow ODE to define the mapping [1]\n- l130: \"regularisaion\" -> regularisation\n \nI am not familiar enough with the recent related work to properly assess the novelty.\n\n[1] Progressive Distillation for Fast Sampling of Diffusion Models https://openreview.net/pdf?id=TIdIXIpzhoI\n\nQUESTIONS:\n- The way $x_{t_i}$ is sampled in the expectations in eq. (8) and eq. (10) is a bit unclear. Is it just obtained as a random sample from the dataset at time $t_i$?\n- l238: \"this is due to the inductive bias of SDEs for better generalization\" What is meant by that?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on using neural stochastic flows to model stochastic dynamic data without solving SDEs. To do this, it is proposed to use conditional normalising flows to learn the SDE transition distribution. It also allows efficient one-step sampling to facilitate the computational efficiency. Experiments were conducted with Stochastic Lorenz Attractor, CMU Motion Capture, and Stochastic Moving MNIST.\n\nSTRENGTHS AND WEAKNESSES:\n* The work focused on important problem. To avoid learning the transition law via simulations with a large amount of steps, the paper follows a solver-free approach. The paper proposed a specific design (architecture) for parameterising the conditional normalising flows, and the loss to train this. To capture the noisy scenarios, the framework is also extended to Latent-NFS.\n\n* The paper is written in self-contained way. Figure 1 can well-illustrate the concept behind the proposed approach (neural stochastic flows-NSF). Section 2 briefly introduce to the SDEs and Stochastic Flows to follow the next sections of the paper. \n\n* The approach (NSF and Latent NSF) has strong motivations and technically sound. \n   - To avoid costly computation of characterising strong solutions of SDEs, the approach derives the alternative weak solutions. A parametric model is then proposed to approximate it (Eq (2) and (3), with design in section 3.1). The conditions for the converted weak solutions of SDEs are described provided in section 3.\n   - To enforce the flow property (especially the property 2 in Section 3) in NSFs, the approach utilised variational technique to construct upper-bounds on KL divergence for $\\mathcal{L}_{\\text{flow}}$ beside the normal supervised loss. \n   - The connection made for the choice of the structure of Latent NSF is described in lines 157-164 with support in the Figure 2. \n\n* The experiments is showed that the computational cost is significantly reduced, which is an important contribution in term of the efficiency of this approach. Compared to other Latent- and Neural- SDE existing approaches, NSF obtained low KL divergence (high performance) with much lower computational cost (Table 1). Analyses about the trade-off between computation and performance is also conducted in Figure 4 (with CMU Motion Capture dataset) and Table 3 (for Stochastic Moving MNIST).\n\nQUESTIONS:\nAlgorithm in Appendix C can be written in the main text, since it helps to better follow the process. Minor: (1) line 130: typo in \"regularisation\"; (2) Line 170: in the description of the total objective, \"the flow loss\" might refer to the Eq (25) in the appendix instead of Eq (24) as in the current version.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}