Title: On Flow-based Generative Models for Probabilistic Forecasting
Venue: NeurIPS 2025
Accepted: no
Keywords: Flow based generative models

stochastic interpolation

time series

probabilistic forecasting

mean-field variational inference

conditional random fields
Abstract: Flow-based generative models (FBGM) have emerged as a dominant approach to generative modeling in many domains for their scalability and controllability, but have notably not made the same impact on autoregressive probabilistic forecasting.  Although the methodology behind these models can be applied directly to the time series setting, and in theory offers the potential to apply the advances in generative modeling to time series, this direct approach is difficult to use in practice.  In this work, we investigate this methodological gap by generalizing the key elements of flow-based generative modeling to the time series setting to devise a more practical related algorithm.  We show that FBGMs based on linear stochastic differential equations are instances of a more general mean-field variational inference algorithm for conditional exponential family distributions that constructs Bayes estimators of natural parameters.  This insight yields a family of mean-squared error based latent probabilistic forecasters that contains a discrete time counterpart of FBGMs for time series.  We demonstrate that the models we develop inherit the convenient theoretical properties of FBGMs while being easy to work with in practice.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to use autoregressive flow based generative models for probabilistic forecasting (i.e., time series generation step by step). First, they relaxe the framework of stochastic interpolants to generalized stochastic interpolants, which do not exactly arrive at a point but at a gaussian potential. Then the authors derive several models, based on the SDEs, which have the correct path measure and also propose autoregressive solutions to solve the variational inference problem, which do not need to be trained with maximum likelihood. Experiments are conducted on several toy data sets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1) The maths is written pretty clearly and is well done. The authors have done a great job at developing rigorous mathematics for their algorithms. \n\n2) The idea is very timely. Whereas generative AI has made great progress on images, the generation of time series (or probabiistic forecasting) lags behind. Further, there is great application potential in finance. \n\nWeaknesses: \n\n1) The framework of generalized interpolants is a bit confusing to me/the motivation is not super clear. So adding a bit of Gaussian noise at the time series end points introduces an error which blurs the distribution a bit and therefore could hinder precision. What is the intuition behind WHY one should do it. \n\n2) The model in the experiments is an LSTM type model. This is in conflict with the continuous perspective (SDE) since LSTMs have 'fixed' hidden states/discretization. If one would employ the time as a continuous input, this would be more clear. (Correct me if i am wrong here) \n\n3) The paper does not really compare to other state of the art algos. While I mainly see this as a nice theoretical paper, I would argue that getting a feeling for how well other algos perform should be there. \n\nOne interesting comparison could be: Trajectory Flow Matching with Applications to Clinical Time Series Modeling, Zhang et al, Neurips 2024\n\n4) The paper sometimes states that the extension to the irregular case is simple. I see this with the autoregressive models but with the FBGM i dont really understand how this should work.\n\nQUESTIONS:\n1) Where does the $q^{Neural_SDE}$ trained as an bayes estimator appear in the experimental tables? \n\n2) Are the gaussian potential fixed in the experiments and how is $\\sigma$ chosen then? Basically you also wrote down an ELBO/NLL so it should be okay'ish to learn EM style?\n\nLIMITATIONS:\nmainly theory paper so fine", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates flow-based generative models for forecasting and generalizes stochastic interpolation via Gaussian potential, leading to a Gaussian random field with relaxed endpoints. The resulting model is a latent SDE via variational inference and closely relates to previous MSE-based autoregressive models. The authors demonstrate their methodology on multiple synthetic datasets using the NRMSE and NLL.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- The CMFVI framework generalizes stochastic interpolation, resulting in a latent SDE via variational inference. All assumptions and modelling choices are theoretically motivated and justified.\n- Gaussian potential functions allow endpoint relaxations and incorporate fixed ones as special cases.\n- The authors connect to stochastic interpolants, describe how their framework relates to previous models, and show that their model is similar to autoregressive MSE-based models.\n- The code is attached, and the appendix contains additional information and details on the implementation.\n\n### Weaknesses\n\n- The paper reads like an incomplete draft. There are various typos, incorrect grammar, missing citations, missing references, and missing proofs. (see Minors for examples)\n- The experimental evaluation is insufficient. Only synthetic datasets are used, and no standard baselines are included. It is hard to see a trend in these. Furthermore, there is no discussion of the results, and the section ends in the middle of a sentence (L352).\n- There is no qualitative evaluation. An example forecast with additional confidence intervals would provide valuable insights.\n- The specifics of different models in the experimental evaluation are not clear. A detailed description would help interpret the results.\n\n\n### Minors\n\n- Equation 2: There should be no $x_0$ in $s_t$\n- L97-99: Sentence seems cut off\n- L113: \"understand\" should be removed\n- L130-131: \"the the\" -> \"is the\"\n- L137: \"(CITE)\" instead of citations\n- L177: in linear SDEs ***in*** ...\n- L198: wide range ***of*** ...\n- L324: though -> thought\n- L328: depend -> depends\n- Multiple references missing, e.g., L646\n- Proof missing (L925)\n\nQUESTIONS:\n- L71: Shouldn't it be conditional distribution instead of marginal?\n- How is the likelihood evaluated (Table 1)?\n- \"[...] the uncertainty in the models only depend on the time in between observations and not the observations themselves.\" This seems like a limitation to me. Can you elaborate on that?\n\n\nTo increase the score, the authors would need to polish the paper, i.e., fix missing references, and include a broader experimental evaluation including real-world datasets and different baselines.\n\nLIMITATIONS:\nWhile the authors discuss the connection to previous time series models, I encourage them to include an additional limitations paragraph.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work proposed a solution to apply flow-based generative modelling to the time series setting. They show connections of flow-based generative modelling with a mean field variational inference algorithm for conditional exponential family distributions and Markovian projection to propose a discrete time version of flow-based generative models that use Stochastic differential equations. The theoretical gains are supported by empirical evidence from experiments on a bunch of synthetic datasets.\n\nSTRENGTHS AND WEAKNESSES:\nI think the paper is overall strong with a good theoretical contribution. I will add some of the minor points I have here.\n\n1. The paper is overall well written, but there are some lines that get overly long and require reading them multiple times to understand them, i.e., the readability can be improved at times. A good example is in the abstract, i.e \"We show that FBGMs based on linear stochastic differential equations are instances of a more general mean-field variational inference algorithm for conditional exponential family distributions that constructs Bayes estimators of natural parameters.\" \n\n2. Line 19: This is not inherently true for flow matching, which works with the velocity field, and sample generation is done via ODE. Maybe it’s better to write ’stochastic process’ instead of SDE.\n\n3. Line 37-38: The Authors wrote \"several years ago\", but the citations are rather recent; consider rephrasing or updating to relevant citations.\n\n4. I think generalising to a large class of models (flow-based generative models) was a good idea, but the entire analysis seems to be heavily based on Stochastic Interpolants; it would be better to make it clear early in the paper.\n\n5. Line 98-99: The statement seems incomplete.\n\n6. Line 5 102-104, The statement looks repeated as the one in the introduction.\n\n7. Probably my biggest critic if any would be the limited evaluation, it would be good to have more experiments not only synthetic datasets.\n\nQUESTIONS:\nPlease see the strengths and weaknesses above.\n\nLIMITATIONS:\nI have not read any limitations addressed directly in the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies how to adapt flow-based generative models to forecasting for time series data. The authors show why directly using these models for time series is hard in practice and propose a new way to adapt them into a simpler, discrete-time version. They connect this idea to variational inference and show that their method links naturally to common time series models like mean-squared-error predictors and Gaussian autoregressive models. They test the method on synthetic data and show it keeps useful properties of flow models while being easier to train for forecasting.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\nThe paper tackles a clear gap in flow-based generative models by making them more practical for time series forecasting. It gives a clear and solid theoretical way to move from complex continuous-time models to simpler discrete-time ones that are easier to train. The main ideas are well motivated, connect nicely to known models, and could help bring modern generative tools into everyday forecasting tasks. The experiments, while simple, show that the method works as intended.\n## Weaknesses\nThe main weakness is that the experiments are only on simple synthetic data, so it’s unclear how well the method works on real-world time series. Also, from the results in the selected benchmarks, it is unclear whether the method perform better than the baselines. Furthermore, no comparison to strong baselines is provided, making it hard to position this method in the broader literature of models for forecasting.\nSome parts of the paper are quite dense, and I think there is a lot of room for improving the exposition, as it took several passes before getting a grasp of the proposed method. Some parts of the text should be polished:\n\n- line 99 is it missing something?\n- line 113 should use understand or find, not both\n- line 137 missing citation?\n- line 352 missing part of sentence\n\nQUESTIONS:\n1. Your method uses Gaussian potential functions for the stochastic interpolation and CRF parts. Could this choice limit how well the model can handle time series with strongly non-Gaussian or multimodal behavior? Do you have ideas for extending the framework to use more flexible (non-Gaussian) potentials?\n2. Can you clarify how the choice of discretization level (e.g., number of time steps or grid resolution) affects model accuracy and computational cost? Is there a practical guideline for choosing how fine or coarse the discrete time steps should be for different types of time series?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 1, "CLARITY": 1, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}