Title: Learning Stochastic Multiscale Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multiscale Models

Stochastic Differential Equations

Spatio-Temporal Data

Variational Inference

Partial Differential Equations
Abstract: The physical sciences are replete with dynamical systems that require the resolution of a wide range of length and time scales. This presents significant computational challenges since direct numerical simulation requires discretization at the finest relevant scales, leading to a high-dimensional state space. In this work, we propose an approach to learn stochastic multiscale models in the form of stochastic differential equations directly from observational data. Drawing inspiration from physics-based multiscale modeling approaches, we resolve the macroscale state on a coarse mesh while introducing a microscale latent state to explicitly model unresolved dynamics. We learn the parameters of the multiscale model using a simulator-free amortized variational inference method with a Product of Experts likelihood that enforces scale separation.  We present detailed numerical studies to demonstrate that our learned multiscale models achieve superior predictive accuracy compared to under-resolved direct numerical simulation and closure-type models at equivalent resolution, as well as reduced-order modeling approaches.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce a data-driven framework for learning stochastic multiscale models of complex physical systems, comprising three core components: an encoder, a latent stochastic differential equation model, and a decoder. Large-scale features are explicitly separated into macroscale and microscale latent states, with their dynamics modeled via coupled SDEs and reconstructed into the high-dimensional space. By leveraging a Product of Experts likelihood and variational inference, the framework achieves efficient training. The authors demonstrate the accuracy across multiple partial differential equation systems.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper tackles the critical challenge of modeling multiscale physical systems \n2. The paper explicitly separates the microscale and macroscale latent states, enhancing physical interpretability and model accuracy\n3. visual explanation: fig 4, 6, 8 clearly visualize the macroscale state and microscale state \n\n**Weaknesses**\n1. The idea of a multiscale framework comprising of encoder, latent dynamics, and decoder is generally not novel. [1] [2] [3]\n2. Clarity issues: \n      -  l72-76: the definition of fully-resolved state $y$ is a bit confusing. This part can be improved for better comprehension. \n      -  Brief introductions to Large-Eddy Simulation (LES) and Product of Experts (PoE) would enhance understanding\n      -  The notation $S_{\\theta}, D_{\\theta}, T_p, T_r, \\mathcal{E}_{\\theta}$ is complex. Are $\\theta, p, r$ all trainable? Is there any special reason to use $p, r$ instead of $\\theta$ here?  Adopting symmetric notation for microscale and macroscale operators would enhance clarity\n3. See question 1 \n\n--------\n[1] Chen, Peter Yichen, et al. \"CROM: Continuous reduced-order modeling of PDEs using implicit neural representations.\" arXiv preprint arXiv:2206.02607 (2022).\n\n[2] Boral, Anudhyan, et al. \"Neural ideal large eddy simulation: Modeling turbulence with neural stochastic differential equations.\" Advances in Neural Information Processing Systems 36 (2023): 69270-69283.\n\n[3] Stachenfeld, Kim, et al. \"Learned simulators for turbulence.\" International conference on learning representations. 2021.\n\nQUESTIONS:\n1. The authors discuss reduced-order modeling (ROM) and closure modeling in lines 51–66. I agree the framework differs from closure models, as they focus on coarsened state predictions, while this approach aim to accelerates full-state simulation via latent dynamics. However, doesn’t this framework fall within the ROM category? It seems the proposed framework is a form of ROM, and you can simply called the baseline \"implicit-scale model\" as ROM. How does the proposed method distinguish itself from traditional ROM approaches?? \n2. Can the framework be applied to PDEs with nonuniform spatial discretization?\n3. The latent SDE is described in eq 2. what is the aim of sec 4.2, where a linear SDE in eq 7 is used to approximate the latent SDE in eq 2? \n4. l.101 can you explain why $\\mathbb{E}[\\bar{y}|y] = \\mathcal{S}_{\\theta}(y)$ and how do you guarantee this? \n5. For the experiment parts, will the stochastic multiscale framework give totally the same result as the implicit-scale model when $n_{\\eta}=0$? Is the framework and training procedure of the multiscale framework and baseline totally the same, where the only difference is $n_{\\eta}$? \n6. is it difficult to train with the multiscale model with ELBO in eq 8? \n7. I’m surprised by the poor performance of the baseline in Figure 7 for the cylinder flow experiment. I assume this is not a very tough task when the dimension of latent macroscale state is 512. Can you show the result of the following baselines on the cylinder flow system as in fig 7: naive coarse-scale simulation of the PDE system on 32 * 8 grids; dynamic mode ecomposition with 512 modes?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, authors propose to learn stochastic multiscale models in the form of stochastic differential equations (SDEs) from observations to better adapt to those various resolutions in real dynamical systems. By optimizing with a modern forward-solver-free amortized variational inference method, the proposed method achieves superior predictive accuracy compared to previous ones.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The idea of explicitly modeling multiscale dynamics within systems using coupled SDEs is novel and valuable for dealing with real applications which have uncertain resolutions.\n2. Authors use a forward-solver-free amortized variational inference method to improve efficiency. Instead of using an SDE numerical solver, they adopt a variational approximation and reparameterize the drift and diffusion terms of the SDE to avoid the integration process. \n3. Detailed mathematical proofs and sufficient ablation studies make the claims more convincing.\n\nWeaknesses:\n1. As authors claimed, the latent state is divided into a macroscale representation and a microscale one. However, how to decide the real resolution corresponding to them still needs to be considered. E.g., in Sec.1, authors use the example of atmosphere modeling, if the macro scale corresponds to planetary waves (5000km) and the micro scale corresponds to cloud microphysics (< 1m), what about modeling at 10m and 100m? Is there a need for more scales? \n2. In the experimental part, it seems that all the test sets are synthetic ones, it could be better if the method can be evaluated on real data. Besides, the baseline is only implicit-scale models. According to the remarks given on closure models and implicit-scale models in Sec.3, if the closure models predict state matching true state y instead of $\\bar{y}$, it is equivalent to the implicit-scale model, correct? If not, authors should also include closure models into baselines.\n\nQUESTIONS:\nPlease refer to the Weaknesses part.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe proposed method is concerned with learning stochastic dynamics from observed data. The central idea is to de-compose the observed evolution into a coarse macroscopic one $(\\bar y)$, and a fine micro-scale ($\\tilde y = y - \\bar y$). Once learned, a coupled SDE can be used to evolve both macro- and microstate, including their learned interactions. The method is purely data-driven, which sets it apart from numerous existing closure-type works that model sub-grid effects using neural architecture while relying on classical (linear) model-order reduction methods on the coarse grid (e.g. [1] and the references therein). The performance of the method is compared to a modified version where the subgrid evolution is not explicitly handled.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well written and I found it easy to follow. The selfcontained nature of the paper (together with the appendix) is appreciated. All plots, tables and math are clear. The code used to generate the experiments is provided in its entirety for the review process already. The general idea to leverage multi-scale modeling is compelling, albeit not new.\n\nThe topic of neural emulators and data-driven methods to accelerate the simulation of fluid dynamics problems is clearly of big interest, as evidenced by the numerous works on this topic in recent years (although one should also mention more skeptical voices as in [2]).\n\nThe central weakness of the paper is that I am not convinced that the claims made are truly backed up by the numerical results. The examples shown (KdV equation, Burger's equation, flow past a cylinder) are classical and can be solved to high precision by most non-linear projection methods (e.g. registration methods as in [3]) or, in the case of the flow past a cylinder, classical POD methods [4]. Especially for the latter, I am not convinced that the proposed method outperforms a linear projection method.\n\nA comparison to other multi-scale inspired projects such as GraphCast [5] and [6], which operate on much larger problems, is not given.\n\nReferences:\n\n[1] A. Ivagnes, G. Stabile, G. Rozza: Data-driven Closure Strategies for Parametrized Reduced Order Models via Deep Operator Networks. https://arxiv.org/abs/2505.17305v1 (2025)\n\n[2] N. McGreivy and A. Hakim: Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations. Nature Machine Intelligence (2024)\n\n[3] T. Taddei: Compositional Maps for Registration in Complex Geometries. SIAM Journal on Scientific Computing (2024)\n\n[4] D. Deep, A. Sahasranaman, S. Senthilkumar: POD analysis of the wake behind a circular cylinder with splitter plate. European Journal of Mechanics - B/Fluids (2022)\n\n[5] R. Lam et al: Learning skillful medium-range global weather forecasting. Science (2023)\n\n[6] Oskarsson et al.: Probabilistic Weather Forecasting with Hierarchical Graph Neural Networks. Neurips (2024)\n\nQUESTIONS:\nThe abstract claim that the learned multi-scale model achieves \"superior predictive accuracy compared to direct numerical simulation (...) at equivalent resolution\". When making claims about comparison to the full order model, I would ask for a comparison between run-time of the FOM (at high/moderate resolution) as well as the online/offline cost of the proposed method.\n\nI believe the paper needs to compare to at least one competing method to make a compelling argument. A classical POD method might already be a good candidate.\n\nLIMITATIONS:\nThe statement that the paper \"fundamentally reimagines how to model multi-scale phenomena\" is, in my opinion, too strong.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an interesting approach to modeling stochastic multiscale problems by separating scales through coupled SDEs. The work introduces a thoughtful application of Product of Experts (PoE) likelihood to determine the separation between macro and micro scales. The experimental evaluation demonstrates promising results across several PDE problems including KdV, Burgers, and Kármán vortex, showing improvements over implicit scale methods.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\nThe paper tackles an important and challenging problem in computational physics and machine learning. The use of variational inference for scale separation is mathematically sound and well-motivated. The theoretical framework connecting stochastic multiscale modeling with neural operators is noteworthy and could inspire future work in this direction.\n\n## Weaknesses and Questions:\n- Why not different dynamcis: The latent dynamics equation (2) treats macro and micro scale states similarly. The work could benefit from exploring differentiated dynamics—for instance, allowing macro scale components to be more global (perhaps through convolution) while keeping micro scale components more local. This architectural choice could better reflect the inherent nature of different scales.\n- Why not more scales: can this framework be generalized to more than two scales?\n- Baseline Comparisons: While the variational inference approach is interesting, the current baseline could be too weak. When using implicit-scale, one can use high latent dimension. Generally speaking the machine learning community has seen success with larger number of model parameters in recent machine learning-based surrogate models. To fully demonstrate the method's capabilities, comparisons with established machine learning baselines such as FNO and PDE-refiner, which are multiscale in nature, would strengthen the evaluation and provide better context for the results.\n- Problem Complexity: Although the paper is motivated by large-scale turbulence problems, the current PDE test cases may not fully showcase the method's advantages. These problems could potentially be resolved using simpler approaches like finite difference solvers or standard ML-based surrogate models. The work would benefit from evaluation on more challenging turbulence problems, such as Navier-Stokes equations with higher Reynolds numbers, where scale separation treatment such as LES becomes truly necessary and beneficial.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nThe paper does not seem to fully discuss its limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-04"}