Title: True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics
Venue: NeurIPS 2025
Accepted: yes
Keywords: Dynamical Systems

Dynamical Systems Reconstruction

Chaos

Recurrent Neural Networks

Foundation Model
Abstract: Complex, temporally evolving phenomena, from climate to brain activity, are governed by dynamical systems (DS). DS reconstruction (DSR) seeks to infer generative surrogate models of these from observed data, reproducing their long-term behavior. Existing DSR approaches require purpose-training for any new system observed, lacking the zero-shot and in-context inference capabilities known from LLMs. Here we introduce *DynaMix*, a novel multivariate ALRNN-based mixture-of-experts architecture pre-trained for DSR, the first DSR model able to generalize zero-shot to out-of-domain DS. Just from a provided context signal, without any re-training, DynaMix faithfully forecasts the long-term evolution of novel DS where existing time series (TS) foundation models, like Chronos, fail -- at a fraction of the number of parameters (0.1%) and orders of magnitude faster inference times. DynaMix outperforms TS foundation models in terms of long-term statistics, and often also short-term forecasts, even on real-world time series, like traffic or weather data, typically used for training and evaluating TS models, *but not at all part of DynaMix' training corpus*. We illustrate some of the failure modes of TS models for DSR problems, and conclude that models built on DS principles may bear a huge potential also for advancing the TS prediction field.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents DynaMix, a foundation model for dynamical systems based on a mixture of experts with ALRNN models.\n\nThe model is pretrained on a large amount of dynamical systems (synthetic 3d data from chaotic systems), and can be used for zero-shot reconstruction of unseen dynamical systems.\n\nDynaMix is tested on both simulated and real-world datasets, and greatly outperforms existing time-series foundation models while also being very computationally efficient.\n\nSTRENGTHS AND WEAKNESSES:\n__STRENGTHS__\n\n1. Novelty: The paper introduces a new class of foundation model for time series, specifically designed for dynamical systems reconstruction, which is a direction not previously explored to this extent as far as I know.\n\n2. Performance: DynaMix demonstrates superior long-term forecasting performance compared to state-of-the-art time series foundation models, with orders of magnitude faster inference.\n\n3. Motivation: The model is grounded on the hypothesis that many real-world time series exhibit local dynamics reflective of underlying dynamical systems.\n\n4. Clarity on Limitations: the paper clearly acknowledges that the model is not suited for more complex time series, setting realistic expectations.\n\n5. Thorough Literature Review: The paper provides a comprehensive overview of existing approaches in both time series modeling and dynamical systems.\n\n\n__WEAKNESSESS__\n\n\nThe paper lacks few analyses/clarifications to better understand the role of different components of the model\n\n1. Why did you choose the ALRNN? How does the model perform with other RNN architectures like LSTM/GRU?\n2. How does the model perform without the temporal delay embeddings? Can SOTA models be improved with similar ideas?\n\nFinally, the model is not suited for more complex time series that stationary ones, but this is a limitation that is acknowledged by the authors and I still find the paper valuable as is.\n\nQUESTIONS:\nSee questions in the weaknesses section above.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes DynaMix, a foundation model for dynamical systems. The model is a mixture-of-experts based on an RNN and trained via sparse teacher forcing. The authors perform an extensive empirical evaluation of their method across a range of dynamical systems and time series forecasting tasks. DynaMix exhibits strong predictive performance while simultaneously requiring less inference time than existing well-known time series foundation models.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n- The proposed architecture is clearly motivated and seems well-suited for the task at hand\n- The empirical evaluation in the paper is thorough and convincing. The proposed model architecture exhibits clear gains compared against well-known time series foundation models in terms of zero-shot DSR (Fig 4) and time series forecasting (Tab 1) while being significantly faster (Fig 5).\n- I especially appreciate that the authors clearly demonstrate various short-comings of existing methods (Section 4.2) rather than only reporting metrics. This provides significant insight. \n- Section 4.3, on MoE similarity, is an interesting qualitative evaluation of the model.\n- The paper is generally very clearly written and well-presented throughout\n\n# Weaknesses\n\n- The paper could potentially be strengthened by additional ablations. It wasn't clear to me which components of the model (the AL-RNN, the MoE, the specifics of the training setup, etc.) were most important \n- It also would have been interesting to see how the proposed method compares to DSR methods which are not based on foundation models -- while these certainly have drawbacks (e.g., requiring task-specific training or fine-tuning), it is unclear if this would result in better performance.\n\n# Minor Comments\n- I found Section 3.2 a bit dense; while things are explained in the appendix, it would be ideal to provide more details in the main paper given that this seems to be an integral part of the method\n- The curves in Figure 8 are difficult to see/distinguish from one another\n\nQUESTIONS:\n- From my understanding, the context dimension $N$ is fixed and shared across tasks during model training. L153-165 describe techniques for applying the model to signals of smaller dimension. Is it possible to apply the model to signals of higher dimension, or is this a limitation of the model?\n- Is the model applicable in settings where your signal is not observed uniformly in time?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes DynaMix, a lightweight AL-RNN–based mixture-of-experts architecture. The proposed DynaMix architecture leverages a mixture-of-experts (MoE) framework employing almost-linear RNNs (AL-RNNs) as experts. In this framework, the next hidden state is predicted as a weighted sum of the outputs from multiple experts, with the weights determined by a gating network which computes these weights using the context and the current hidden state as inputs. By using lightweight experts and gating network, DynaMix is computationally efficient compared to recent transformer-based time series (TS) foundation models. Experimental results demonstrate that the proposed method performs well on dynamical system reconstruction (DSR) tasks and even on short-term forecasting without any fine-tuning.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nS1. DynaMix can reconstruct and forecast previously unseen dynamical systems from only a short context without need for retraining or fine-tuning\n\nS2. The model achieves high performance with a lightweight architecture—requiring orders of magnitude fewer parameters and delivering much faster inference compared to existing foundation models, which makes it suitable for practical deployment at scale\n\nS3. The effectiveness of the proposed method is validated through various experiments on both DSR datasets and general time series datasets.\n\nWeaknesses:\n\nW1. The paper lacks quantitative comparisons with the state-of-the-art DSR models in term of both DSR tasks and time series forecasting, which limits the fairness and practical context of its benchmarking. \n\nW2..The Mixture-of-Experts framework is not itself a novel concept, and given that strong performance is reported in a broad sense, it remains unclear to what extent the authors’ specific architecture choices contribute to the observed gains, making the source of novelty somewhat ambiguous\n\nW3. The scalability and generalization of the model remain uncertain, as increasing the number of experts to cover more diverse or fundamentally different dynamical systems could lead to substantial growth in model size and potential degradation in out-of-distribution performance\n\nQUESTIONS:\nQ1. Could the authors provide quantitative comparisons with the state-of-the-art DSR models in term of both DSR task and time series forecasting?\n\nQ2. Could the authors conduct ablation studies to clarify which specific architectural choices, beyond the general MoE framework, are responsible for the observed performance gains?\n\nQ3. How does the model scale and generalize as the number of experts increases, and are there strategies in place to control model size or improve robustness to highly novel dynamical systems?\n\nQ4. The rationale for selecting AL-RNNs as experts within the MoE framework is not sufficiently justified. Could you provide a more detailed explanation to explain why this choice is effective or necessary? \n\nQ5. Can you compare the test performances of the proposed scheme, benchmark SOTA DSR models and time series foundation models, for data set with more complicated dynamics (e.g., 6D) in terms of term of both DSR task and time series forecasting?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces DynaMix a new type of zero-shot forecasting / dynamical system reconstruction approach.\nDynaMix uses a mixture of expert approach, where each expert is a multi-dimensional almost linear RNN model. \nGiven a contex window of a time series signal, an attention mechanism is used to gate the experts. The model is trained across a wide set of dynamical systems to reproduce dynamical system behavior.\n\nThe paper demonstrate in simulations that the model can forecast and re-construct the statistical properties of many dynamical systems and in particular for chaotic dynamics -- a setting where other foundational time series forecasting models fail. The paper also shows that DynaMix can give reasonable forecast accuracy on real world time series forecasting benchmarks, despite never being trained on such datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Very novel and innovative approach\n- Strong numerical results for dynamical system and real world time series\n- Well written and easy to follow.\n- Comprehensive details about model setup, training procedure, parameter selection etc.\n\nWeaknesses:\n- On the forecasting task, MAE is not the best metric when evaluating accuracy across time series with different magnitude, because the metric can be dominated by some few time series in the dataset with larger magnitude. It would be better to use MASE or normalize MAE per metric and report geometric averages.\n- Somewhat complex training setup relying on not so standard methods (STF). But everything is well documented and the code is available, which mitigates this to some extend.\n\nQUESTIONS:\n- Is there any kind of pre-processing of the time series done? For example scaling, normalization, or time scaling?\n- The model seems to be able to match the overall period of a signal well, but it essentially only does a local re-construction of the phase space dynamics. Do you have any insights on how the model is able to match the overall period? It would be interesting to see how the model predicts a simple limit cycle as you change the period.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-08"}