Title: Integration Matters for Learning PDEs with Backward SDEs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Backward Stochastic Differential Equations

Partial Differential Equations

Deep Learning

Scientific Machine Learning
Abstract: Backward stochastic differential equation (BSDE)-based deep learning methods provide an alternative to Physics-Informed Neural Networks (PINNs) for solving high-dimensional partial differential equations (PDEs), offering potential algorithmic advantages in settings such as stochastic optimal control, where the PDEs of interest are tied to an underlying dynamical system. However, standard BSDE-based solvers have empirically been shown to underperform relative to PINNs in the literature. In this paper, we identify the root cause of this performance gap as a discretization bias introduced by the standard Euler-Maruyama (EM) integration scheme applied to one-step self-consistency BSDE losses, which shifts the optimization landscape off target. We find that this bias cannot be satisfactorily addressed through finer step-sizes or multi-step self-consistency losses. To properly handle this issue, we propose a Stratonovich-based BSDE formulation, which we implement with stochastic Heun integration.  We show that our proposed approach completely eliminates the bias issues faced by EM integration.  Furthermore, our empirical results show that our Heun-based BSDE method consistently outperforms EM-based variants and achieves competitive results with PINNs across multiple high-dimensional benchmarks. Our findings highlight the critical role of integration schemes in BSDE-based PDE solvers, an algorithmic detail that has received little attention thus far in the literature.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe article is concerned with deep learning based methods for PDEs and more specifically, the deep BSDE method based on a stochastic reformulation of the PDE. Authors study specific loss function, so-called self-consistent loss, with Euler-Maruyama scheme for discretization of underlying stochastic processes. Paper analyses mathematically the limiting behaviour of loss function as the discretization step-size tends to zero and proves that a bias arises. Then, authors introduce an alternative discretization based on Stratonovich formulation of underlying stochastic differential equations and prove that in this case no bias arises.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n-\tThe paper raises an important point: PINNs and deepBSDE type methods have not been systematically compared in the literature\n- The paper is mostly well-written and presented\n-\tThe observation that the Euler scheme might create a bias for the deep BSDE method is interesting and new\n-\tThe numerical experiments appear to be sound and indicate that the new method performs well, at the expense of higher computational cost\n- The method appears to be new and innovative\n\n\n### Weaknesses\n\n**Imprecise mathematical statements:** My main concern with the paper is its lack of mathematical rigour. Broadly speaking, the paper is written in a way that indicates that probably there are conditions under which the conclusion holds true. However, in the present form of the paper it is impossible to see clearly what these conditions are. To provide a few examples:\n\n-\tTo derive (4.3), which is crucial for the main contribution of the paper, the authors do not spell out formally any assumptions, but just mention *basic regularity assumptions on $R$, $H$, and $u_\\theta$* and *strong convergence of the EM scheme*. What precisely are these conditions? What are the conditions for strong convergence of the EM scheme? Can you provide a reference where this statement can be found?\n-\tAlso in the proofs, there are several statements that are too informal to be checked properly. For example, in line 903: *assuming enough regularity on R*. Again, can you specify what are the precise assumptions here? Similarly, line 905: *where the penultimate equality holds from the standard approximation properties of a left Riemann sum to the Riemann integral*. Can you be more precise here? How do you get the desired decay with $\\tau$? \nMoreover, some assumptions are just swept under the carpet, see line 909: *We assume that sufficient regularity for the order 1/2 strong convergence of EM jointly on …* This seems to be a strong assumption, that is not mentioned in the main text. Here I feel like the presentation would need to be significantly improved to clarify what are the assumptions under which the claimed statements hold.  \n\n**Background on PDEs limited:** The paper does not provide context on the considered class of PDEs. For example, deep BSDE methods apply to specific types of parabolic PDEs (so-called Kolmogorov PDEs). Why are the considered PDEs relevant? In what applications do they arise? Which important PDEs are of this type?\n\n**Benchmarks not reflecting state-of-the-art** The overall motivation is based on numerical experiments from reference [6], which is from 2021 and does not reflect the state-of-the-art. In [6] it appears that PINNs outperform BSDE methods, but in the meantime, several other papers improving upon the deep BSDE method have been employed. To justify the strong conclusions made in the paper, some of these methods would need to be considered as well. Examples:\n\n- Beck, Christian and Becker, Sebastian and Cheridito, Patrick and Jentzen, Arnulf and Neufeld, Ariel. Deep Splitting Method for Parabolic PDEs. SIAM Journal on Scientific Computing.\n- Riu Naito, Toshihiro Yamada. An acceleration scheme for deep learning-based BSDE solver using weak expansions.  International Journal of Financial Engineering 2020 07:02\n- Akihiko Takahashi a, Yoshifumi Tsuchida, Toshihiro Yamada . A new efficient approximation scheme for solving high-dimensional semilinear PDEs: Control variate method for Deep BSDE solver. Journal of Computational Physics, 2022. \n- Jean-François Chassagneux, Junchao Chen, Noufel Frikha. Deep Runge-Kutta schemes for BSDEs, arXiv Preprint 2212.14372. \n- Huyên Pham, Xavier Warin & Maximilien Germain. Neural networks-based backward scheme for fully nonlinear PDEs, SN Partial Differential Equations and Applications.\n\nQUESTIONS:\n-\tCan you spell out formally the assumptions that guarantee that (4.3) holds? Moreover, what are the conditions for strong convergence of the EM scheme? Can you provide a reference where this statement can be found?\n-\tCan you provide a summary of all the assumptions needed to make the overall conclusion of the paper valid?\n-\tCan you specify what precise assumptions are needed to make lines 903, 905, 909 valid?\n-\tCan you provide more background on the considered PDEs? Why are these relevant? In which applications does each of them arise? \n-\tCan you discuss how your method compares to state-of-the art methods based on stochastic representations?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce a new integrator based on Stratonovich viewpoint of SDEs to reduce biases induced with Euler based solvers. The authors consider this in the case of solving backwards SDEs for use in semi-linear parabolic PDEs with applications in control. They illustrate theoretically and numerically the advantages of the Stratonovich based BSDE integrator.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe proposed integrator has theoretical and empirical advantages over existing Euler Maruyama and PINNs based approaches for solving high dimensional semi linear parabolic PDEs.\n\nThe problem is studied from a numerical analysis perspective, which provides useful results for other fields beyond machine learning. \n\nWeaknesses:\n\nThe main weakness is the induced computation time from the proposed method. \n\nThe empirical results could probably use a bit more analysis. For example, in one case the vanilla EM method performs better which is quite surprising given that there’s only four examples of empirical studies conducted on the methods. \n\nThis applies to a limited set of PDEs and perhaps some more examples should be motivated within the text.\n\nQUESTIONS:\nIt’s surprising to see how well FS-PINNs works in the experiments, do the authors have an idea as to why this is the case?\n\nDo the authors have any suggestions for how they can improve upon the computation speed of the proposed method?\n\nUnder a similar computational budget, how do the different methods compare? E.g. for a time constrained problem, which would be the correct method to use?\n\nI’m a bit curious as to why PINNs are much faster, my initial impression is that computing derivatives manually especially in high dimensions would be slower? Is this mainly because these are vector hessian products?\n\nI think a study on dimensionality vs performance could also be quite a bit helpful. \n\nFrom table 3, is there any intuition why the 32 bit performance makes the Euler Maruyama baselines perform better than 64 bit?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nBackward SDE-based deep learning methods offer a powerful framework for solving high-dimensional PDEs. Most existing approaches rely on the Euler–Maruyama (EM) scheme for time integration. This paper revisits that choice carefully and points out a discretization bias introduced by the EM scheme. To address this, the authors propose a Stratonovich-based BSDE formulation paired with a stochastic Heun integrator, which eliminates the bias. The paper presents detailed theoretical analysis of both schemes and supports the findings with three high-dimensional PDE examples, demonstrating notable improvements with the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper challenges a commonly used but under-examined component—time integration—in BSDE-based PDE solvers and highlights the significant role that the choice of integration scheme plays in solution quality. It is of great originality.\n\n2. The authors conduct a careful and thorough numerical analysis of discretization errors associated with both schemes. The implications are consistent with prior observations in the literature and are well supported by numerical experiments.\n\nWeaknesses:\n\n1. As the authors themselves note, the method proposed in Ref [6] is closely related but is not included as a baseline. This seems like a notable omission.\n\n1. The additional computational cost introduced by the second-order derivatives required in the Stratonovich formulation appears to be a key limitation of the method. Further comparisons that account for this overhead would provide a clearer assessment of the method’s practical value.\n\nQUESTIONS:\nBeyond the points raised above, here are several additional comments:\n\n1. Literature discussion contains several inaccuracies:\n\n    - The claim in the abstract that \"existing BSDE-based solvers have empirically been shown to underperform relative to PINNs in the literature\" is quite misleading. In the main text, this is attributed to Ref [6], but the diffusion loss proposed in that work itself is based on a BSDE formulation and shows notably better accuracy than both PINNs and the original Deep BSDE method. So this statement undercuts the actual strengths shown by BSDE-based solvers in the high-dimensional PDE setting relative to PINNs; see also the recent development in arXiv:2409.08526.\n\n    - In the introduction, the authors cite Refs [1,2]—the original PINN papers—as “scalable alternatives” meant to address the curse of dimensionality. This characterization is inaccurate. Those early PINN works (and many follow-ups) primarily focused on low-dimensional problems and were not designed with high-dimensional PDEs in mind. In contrast, BSDE-based methods were motivated from the start by high-dimensional settings, as discussed in Refs [4,22]. To the best of reviewer's knowledge, interest in applying PINNs to high-dimensional PDEs only developed more recently; see, e.g., arXiv:2307.12306 and references therein.\n\n    - Ref [3], the Deep Ritz method, is not a PINN-type approach; it is a distinct variational method and should not be grouped together with PINNs.\n\n1. Table 2 presents computational time differences across methods. It would be helpful to also compare performance under equal computation time rather than fixed iteration counts, as that would give a more meaningful picture of practical efficiency. Again, including diffusion loss from Ref [6] as a baseline is helpful.\n\n1. There is a substantial body of theoretical work on Itô-based BSDE formulations that supports the analysis of deep learning–based methods (e.g., arXiv:1811.01165, arXiv:2307.15496). It would be helpful if the authors could comment on whether a similar theoretical framework could be developed for the Stratonovich-based formulation. Such discussion could strengthen the theoretical appeal of the proposed method.\n\nLIMITATIONS:\nAs acknowledged, the overhead due to second-order derivatives is a main limitation.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies backward stochastic differential equation (BSDE) based deep learning methods for solving high-dimensional partial differential equations (PDEs). These methods provide an alternative to the popular PINNs and have an advantage that the underlying equation do not need to be known, but their performance has been lagging behind PINNs. The authors identify the reason of this performance gap as the Euler-Maruyama (EM) integrator typically used for BSDE methods, which is based on ito SDE formulation and is shown by the authors to induce significant discretization bias. The authors propose to employ Stratonovich SDE interpretation and use Heun integration scheme, which eliminates the raised issue. The authors empirically demonstrate that the fixed BSDE algorithm is competitive to PINNs in solving three PDE benchmark problems.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nS1. The paper addresses an important problem of engineering alternative approaches to PINNs for PDE solving which have qualitatively different characteristics (e.g., in the required information for training).\n\nS2. The authors theoretically show that BSDE methods with EM integrator suffers from a bias term which can be problematic even in the limit of infinite training data, and demonstrate it through an informative example (Theorem 4.1; Figure 1). The algorithmic fix based on Stratonovich BSDEs and Heun integration are well-motivated and analyzed to avoid the discretization error (Theorem 4.2; Figure 1). A potential fix based on multi-step losses is further analyzed in Section 5.\n\nS3. The experimental results in Section 6 support the theoretical claims made in Sections 4 and 5. The baselines are properly chosen, the results are presented clearly, and limitations are transparently discussed.\n\nWeaknesses\n\nMy concerns are mostly about the the limitations already discussed in the paper by the authors.\n\nW1. As the authors discussed in Section 7, the fact that the proposed method requires computing the Hessian leads to a large computational overhead, including the requirement to use float64 precision. While this is properly acknowledged and discussed (including Appendix E.2), it would be informative to, e.g., have a controlled EM-BSDE baseline which is allowed to use a similar amount of computation for integration, if it is possible.\n\nMinor comment\n- Line 35: involving -> involves?\n\nQUESTIONS:\nI have no particular questions, but would like to hear the authors' opinion on W1.\n\nLIMITATIONS:\nThe authors discussed limitations in Sections 6, 7 and Appendix A.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}