Title: INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers
Venue: NeurIPS 2025
Accepted: yes
Keywords: machine learning for science

hybrid PDE solvers

autoregressive error analysis

differentiable physics
Abstract: When simulating partial differential equations, hybrid solvers combine coarse numerical solvers with learned correctors. They promise accelerated simulations while adhering to physical constraints. However, as shown in our theoretical framework, directly applying learned corrections to solver outputs leads to significant autoregressive errors, which originate from amplified perturbations that accumulate during long-term rollouts, especially in chaotic regimes. To overcome this, we propose the Indirect Neural Corrector ($\mathrm{INC}$), which integrates learned corrections into the governing equations rather than applying direct state updates. Our key insight is that $\mathrm{INC}$ reduces the error amplification on the order of $\Delta t^{-1} + L$, where $\Delta t$ is the timestep and $L$ the Lipschitz constant. At the same time, our framework poses no architectural requirements and integrates seamlessly with arbitrary neural networks and solvers. We test $\mathrm{INC}$ in extensive benchmarks, covering numerous differentiable solvers, neural backbones, and test cases ranging from a 1D chaotic system to 3D turbulence. INC improves the long-term trajectory performance ($R^2$) by up to 158.7\%, stabilizes blowups under aggressive  coarsening, and for complex 3D turbulence cases yields speed-ups of several orders of magnitude. INC thus enables stable, efficient PDE emulation with formal error reduction, paving the way for faster scientific and engineering simulations with reliable physics guarantees.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper Introduces Indirect Neural Corrector (INC), a neural corrector implemented as a source term in the partial differential equation. Authors provide a perturbation analysis justifying the superiority of the indirect method compared to direct correction term which directly adjusts the field. Experiments are conducted over a wide range of partial differential equations (PDEs) to provide empirical support of the proposed framework. These experiments support the theoretical analysis, and demonstrates up to ~158% improvement on the long trajectory performance.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The necessity of hybrid neural solvers are well justified and argued in the paper\n2. INC directly addresses the error accumulation issues of the hybrid neural solvers\n3. The perturbation analysis is very well written and correctly reaches the point. This seems to be a novel analysis which is very well structured to demonstrate why adding the correction terms as a source term yields significant improvements. \n4. A wide range of experiments are conducted across 6 different PDEs to support the claims. This provides a solid ground to observe generalization of the framework as well as the limitations.\n\n**Weaknesses**\n1. The major concern regarding this paper is the novelty of the proposed framework itself. The Indirect Neural Corrector performs the corrections inside the equation as a source term. This similar approach is used in other literature as well. Some of these approaches are not acknowledged by the authors. One example of this is \"Deep neural network enabled corrective source term approach to hybrid analysis and modeling\" [1].\n\n2. The theoretical analysis is only conducted for the backward Euler case, meanwhile in the experimental configuration authors discuss the difference in implementation of INC for different time stepping schemes. Can authors discuss this further and explain what was the difference between implementation INC for backward Euler, and other schemes like Runge-Kutta? \n\n3. The learning dynamics is not described well. The authors need to explain how the model is trained in more detail. For instance, they mention the training is done in a supervised manner, does this mean the model was trained separately and then used online with the coarse solver? \n\n4. The effect of multi-step unrolling strategy is something that is completely neglected. Does the framework backpropagate through all the temporal steps or just a few steps? This can lead to significant inefficiencies that authors must acknowledge or briefly discuss.\n\n5. For clarity purposes authors must clarify the notations better at the beginning of the theoretical justification. \n\n\n(W1 Remains the primary weakness for the reviewer) \n\n**Ref**\n\n[1] Blakseth, S.S., Rasheed, A., Kvamsdal, T. and San, O., 2022. Deep neural network enabled corrective source term approach to hybrid analysis and modeling. Neural Networks, 146, pp.181-199.\n\nQUESTIONS:\n1) How is this paper different than other neural correctors implemented as source term? (Read W1) Addressing this can significantly help the readers to better understand the novelties.\n\n2) Can authors discuss further if the function calls would be any different between the direct and indirect correction methods? \n\nPlease refer to the weakness for other questions\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduce An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers,which integrates learned corrections into the governing equations rather than applying direct state updates.They claim that INC reduces the error amplification on the order of  $\\Delta t^{-1} + L$. And poses no architectural requirements and integrates seamlessly with arbitrary neural networks and solvers.They evaluate INC on 1D to 3D cases, and achieve the improvement compare with baselines.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- The proposed method demonstrates improved performance over established baselines by leveraging the idea of an Indirect Neural Corrector (INC).\n- The model is conceptually simple and demonstrates potential in correcting low-resolution numerical solutions, which is practically relevant given the high computational cost of traditional solvers.\n\n### Weaknesses\n- The idea of an indirect neural corrector may not be entirely novel. For instance, P2C2Net employs a similar strategy—computing derivatives on coarse grids and correcting errors over time using neural networks, with strong generalization results on Kolmogorov flow.\n- Some relevant baselines are missing from the comparisons, such as PDE-Refiner [2] and Modern-UNet [3], which are important references for learning-based PDE solvers.\n- Although the authors claim model-agnosticism, including one or two concrete architectural examples would greatly enhance clarity and reader comprehension.\n- Details regarding the numerical data generation process are lacking, including time discretization schemes, step sizes, and grid parameters, which are crucial for reproducibility and fair evaluation.\n\n***Ref***\n\n[1] Wang et al. P2C2Net: PDE-Preserved Coarse Correction Network for Efficient Prediction of Spatiotemporal Dynamics. NeurIPS, 2024.\n\n[2] Lippe et al. PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers. NeurIPS, 2023.\n\n[3] Gupta et al. Towards Multi-spatiotemporal-scale Generalized PDE Modeling. TMLR, 2023.\n\nQUESTIONS:\n1. Please clarify the conceptual and methodological differences between the proposed Indirect Neural Corrector and prior approaches such as P2C2Net , which also utilize neural networks to correct coarse-grid errors over time.\n2. Would applying the proposed method to more advanced backbone models—such as Learned Correction (LC) [1]  or PDE-Refiner—lead to further performance improvements, particularly on challenging benchmarks like Kolmogorov flow?\n3. The current evaluation is limited to standard statistical metrics. It would be beneficial to include physically meaningful metrics, such as the energy spectrum curve in turbulent flows, to provide a more comprehensive assessment.\n4. While hybrid solvers offer interpretability and some physical grounding, they often ignore historical information and tend to be slower than neural operator-based methods. Incorporating operator-based techniques could potentially lead to significant speed-ups and broader applicability. Such integration would represent a promising and impactful research direction.\n\n***Ref***\n\n[1] Kochkov et al. Machine learning–accelerated computational fluid dynamics. PNAS, 2021.\n\nLIMITATIONS:\n- The proposed method is heavily dependent on the underlying numerical solver, which may limit its generalizability across different simulation settings or solver configurations.\n- The approach appears tailored specifically for hybrid solver frameworks and may not be easily applicable to purely data-driven or operator-based methods.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to improve the long rollout accuracy of the hybrid solvers by adjusting the correction term from the original (after the numerical simulation) to the right-hand-side (RHS) part. This paper also provides the analytical analysis as well as numerical analysis about how accurate the proposed approach is compared to the conventional direct correction, where they defined the error dominance ratio. The proposed approach has been validated on different cases, from a 1D chaotic system to 3D turbulence.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths:\n    - Although the change of the proposed method to the direct correction approach is incremental, they provide a clear theoritical proof about why it works and how accurate it will be.\n    - The paper provides sufficient examples to demonstrate the generality of the proposed method to different numerical solvers, neural networks, as well as different PDEs.\n    - Comprehensive and convincing experiments support the claim and demonstrate the improvement of the proposed approach in terms of improving the long rollout accuracy.\n    - The written and presentation are good.\n- Weaknesses:\n    - The paper mentioned the multi-step unrolled optimization to ensure the stability, which will cause a huge training cost, especially for 3D problems and long unrolling steps. However, it is not clear how to choose the best multi-step strategy. Another follow-up question is how the proposed method performs without this strategy?\n    - The theoretical analysis (Section 4) is based on the linearized error evolution and the \"small perturbation\" assumption. In strongly chaotic or strongly nonlinear systems, the learned correction terms (i.e., perturbations) may not always be sufficiently small, and the system may deviate from the neighborhood where the linearized analysis is valid within a few steps. A further explanation on this would strengthen the paper.\n    - The hybrid approach reply on the differential solvers. However, for many industry applications, the legacy solvers might not be differentiable, which will limit the implementation of the proposed approach.\n\nQUESTIONS:\n- In the benchmark comparison of Section 6.1, you compared SITL and SITL*. I understand that the previous one is the direct correction approach. But is not clear for readers about the latter one. Also, what are the specific differences between these two direct correction methods? Why do you need to include both of them as benchmarks?\n- How do you choose the unrolling steps in the training and how it influence the final performance?\n- How the INC method perform when applied to PDEs with stiff source terms? If you embed a neural network as part of the source term in a PDE, and the output of this network is potentially unbounded (e.g., not constrained by activation functions like sigmoid or tanh), could such an “uncontrolled” neural source introduce additional challenges to the stability of time integrators (such as Runge-Kutta or implicit solvers)?\n\nLIMITATIONS:\n- The author mentioned that for problems that are inherently strongly damped or use unconditionally stable formats, the advantages of INC may be weakened. This point is worth emphasizing, and users need to judge the applicability of INC according to the dynamic characteristics of their own problems and the numerical formats they use.\n- The deployment of long-term sequence training requires a lot of computational resources, which may become a bottleneck in applying this method to ultra-large-scale problems.\n- A large number of existing, mature scientific computing code libraries cannot be directly applied to the training paradigm of INC.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work introduces Indirect Neural Correctors (INC), a new method of embedding neural networks within traditional numerical solvers (hybrid solvers). INC learns a correction of the form (with abuse of notation): $u_{t+1} = \\mathcal{T}[u_{t} + G_{\\theta}(u_{t})]$ where $\\mathcal{T}$ is the temporal integration and $G_{\\theta}$ is a neural network parameterized by $\\theta$. The authors also include a theoretical analysis of INC to show that INC reduces errors proportional to $\\Delta t^{-1} + L$ where $L$ is the Lipschitz constant and improves trajectory stability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is well written and properly motivated. The problem being tackled is significant to the ML for dynamical systems research community.\n- The proposed method shows strong improvements over the evaluated baselines and tests a variety of different dynamical systems.\n- I believe the theoretical analysis is strong.\n\nWeaknesses: There are two primary concerns I have with the paper.\n1. To me, this seems very similar to [1, 2], whom the authors cite. The learned correction (LC) variant from both papers learns a correction of the form  $u^{n+1}=u^{n^*} + LC({u^{n^{\\star}}})$. This seems to be similar to INC when rolled out: $u^{n+2} = \\mathcal{T}[u^{(n+1)^\\star}] + LC(u^{(n+1)^\\star})= \\mathcal{T}[u^{n^\\star} + LC(u^{n^\\star})] + LC(u^{(n+1)^\\star})$\nHere, the LC implicitly goes through the temporal integration step, albeit one timestep later. While I don’t think these are entirely the same, I would appreciate some further discussion on what makes INC uniquely different from LC [1, 2].\n2. In a related vein, the baselines the paper evaluates against are weak compared to the hybrid solvers cited in the related works. Architecturally, I think the baselines (FNO, DeepONet, ResNet, U-Net) are sufficient. However, the STIL baselines do not, as far as I can tell, correspond to [1, 2] which would act as a more direct point of reference. \n\n[1] Dmitrii Kochkov, Jamie A Smith, Ayya Alieva, Qing Wang, Michael P Brenner, and Stephan Hoyer. Machine learning–accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21):e2101784118, 2021\n[2] Gideon Dresdner, Dmitrii Kochkov, Peter Norgaard, Leonardo Zepeda-Nunez, Jamie A Smith, Michael P Brenner, and Stephan Hoyer. Learning to correct spectral methods for simulating turbulent flows. arXiv preprint arXiv:2207.00556, 2022.\n\nQUESTIONS:\n1. See weaknesses 1. How is the work different from LC in [1, 2]?\n2. See weakness 2. Can the authors include comparison to recent learned correction methods? I point to [1, 2] since I know they have code available. \n3. What does SITL* mean? It is not clear to me what “a pre-correction variant” means. Similarly, can the authors provide more details on what the RNN baseline is? \n4. How are $R^2$ scores being computed?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}