Numerical methods for ordinary differential equations

Numerical methods for ordinary differential equations are computational schemes to obtain approximate solutions of ordinary differential equations (ODEs).
Background
Since ODEs appeared in science, many mathematicians have studied how to solve them.Arnolʹd, V. I., Ordinary differential equations. Springer.Wolfgang Walter, Ordinary differential equations. Springer.Logemann, H., & Ryan, E. P. (2014). Ordinary differential equations: Analysis, qualitative theory and control. Springer.Chicone, C. (2006). Ordinary differential equations with applications. Springer Science & Business Media. However, only few of them can be mathematically solved. This is why numerical methods are needed. One of the most famous methods are the Runge-Kutta methods,Butcher, J. C. (1996). A history of Runge-Kutta methods. Applied Numerical Mathematics, 20(3), 247-260. but it doesn't work for some ODEs (especially nonlinear ODEs). This is why new ODE solvers are developed. The following list includes frequently used methods:

 Bulirsch-Stoer algorithmMonroe, J. L. (2002). Extrapolation and the Bulirsch-Stoer algorithm. Physical Review E, 65(6), 066116.
 Euler's method (named after Leonhard Euler) and their variants
 Backward Euler methodPeskin, C. S., & Schlick, T. (1989). Molecular dynamics by the Backward‐Euler method. Communications on pure and applied mathematics, 42(7), 1001-1031.
 Semi-implicit Euler method
 Euler-Maruyama methodEmma Gau (2020). Euler–Maruyama Method (https://www.mathworks.com/matlabcentral/fileexchange/69430-euler-maruyama-method), MATLAB Central File Exchange. Retrieved May 24, 2020.
 Exponential integratorHochbruck, M., & Ostermann, A. (2010). Exponential integrators. Acta Numerica, 19, 209-286.Al-Mohy, A. H., & Higham, N. J. (2011). Computing the action of the matrix exponential, with an application to exponential integrators. SIAM Journal on Scientific Computing, 33(2), 488-511.
 Leapfrog method
 Linear multistep methods
 Shooting method
 Symplectic integratorHairer, E., Lubich, C., & Wanner, G. (2006). Geometric numerical integration: structure-preserving algorithms for ordinary differential equations. Springer Science & Business Media.Symplectic integrators: An introduction, American Journal of Physics 73, 938 (2005); https://doi.org/10.1119/1.2034523
Denis Donnelly.Y. B. Suris, Hamiltonian Runge-Kutta type methods and their variational formulation (1990) Matematicheskoe modelirovanie, 2(4), 78-87.Iserles, A., & Quispel, G. R. W. (2016). Why geometric integration?. arXiv preprint arXiv:1602.07755.
 Taylor series methodHirayama, H. (2002). Solution of ordinary differential equations by Taylor series method. JSIAM, 12, 1-8.Hirayama, H. (2015). Performance of a Higher-Order Numerical Method for Solving Ordinary Differential Equations by Taylor Series. In Integral Methods in Science and Engineering (pp. 321-328). Birkhäuser, Cham.

Validated Numerics for ODEs

Not only approximate solvers, but the study to "verify the existence of solution by computers" is also active. This study is needed because numerically obtained solutions could be phantom solutions (fake solutions). This kind of incident is already reported.Breuer, B., Plum, M., & McKenna, P. J. (2001). "Inclusions and existence proofs for solutions of a nonlinear boundary value problem by spectral numerical methods." In Topics in Numerical Analysis (pp. 61–77). Springer, Vienna.Gidas, B., Ni, W. M., & Nirenberg, L. (1979). "Symmetry and related properties via the maximum principle." Communications in Mathematical Physics, 68(3), 209–243. The popular methods are based on the shooting method or spectral methods.Lloyd N. Trefethen (2000) Spectral Methods in MATLAB. SIAM, Philadelphia, PA.D. Gottlieb and S. Orzag (1977) "Numerical Analysis of Spectral Methods : Theory and Applications", SIAM, Philadelphia, PA. Today, European research teamsLohner，R．J.，Enclosing the Solution of Ordinary lnitial and Boundary Value Problems, Computer arithmetic：Scientific Computation and Programming Languages，Kaucher，E.，Kulisch，U., Ullrich，Ch．(eds.)， B．G．Teubner，Stuttgart (1987), 255−286.Rihm, R. (1994). Interval methods for initial value problems in ODEs. Topics in Validated Computations, 173-207.Hungria, A., Lessard, J. P., & Mireles-James, J. D. (2014). Radii polynomial approach for analytic solutions of differential equations: Theory, examples, and comparisons. Math. Comp.Nedialkov, N. S., Jackson, K. R., & Pryce, J. D. (2001). An effective high-order interval method for validating existence and uniqueness of the solution of an IVP for an ODE. Reliable Computing, 7(6), 449-465.Corliss, G. F. (1989). Survey of interval algorithms for ordinary differential equations. Applied Mathematics and Computation, 31, 112-120.Nedialkov, N. S. (2000). Computing rigorous bounds on the solution of an initial value problem for an ordinary differential equation (Ph.D. thesis). University of Toronto.Eijgenraam, P. (1981). The solution of initial value problems using interval arithmetic: formulation and analysis of an algorithm. MC Tracts.Nedialkov, N. S., & Jackson, K. R. (1999). An interval Hermite-Obreschkoff method for computing rigorous bounds on the solution of an initial value problem for an ordinary differential equation. Reliable Computing, 5(3), 289-310.Nedialkov, N. S., Jackson, K. R., & Corliss, G. F. (1999). Validated solutions of initial value problems for ordinary differential equations. Applied Mathematics and Computation, 105(1), 21-68. and Japanese expertsBerz, M., & Makino, K. (1998). Verified integration of ODEs and flows using differential algebraic methods on high-order Taylor models. Reliable computing, 4(4), 361-369.Kashiwagi, M. (1995). Numerical Validation for Ordinary Differential Equations using Power Series Arithmetic. In Numerical Analysis Of Ordinary Differential Equations And Its Applications (pp. 213-218). are working on this topic.
ODEs and Related Topics Studied in the Context of Validated Numerics

 Blow-up solutionsTakayasu, A., Matsue, K., Sasaki, T., Tanaka, K., Mizuguchi, M., & Oishi, S. I. (2017). Numerical validation of blow-up solutions of ordinary differential equations. Journal of Computational and Applied Mathematics, 314, 10-29.Matsue, K., & Takayasu, A. (2019). Rigorous numerics of blow-up solutions for ODEs with exponential nonlinearity. arXiv preprint arXiv:1902.01842.
 Lorentz equationHassard, B., Zhang, J., Hastings, S. P., & Troy, W. C. (1994). A computer proof that the Lorenz equations have “chaotic” solutions. Applied Mathematics Letters, 7(1), 79-83.Mischaikow, K., & Mrozek, M. (1995). Chaos in the Lorenz equations: a computer-assisted proof. :en:Bulletin of the American Mathematical Society, 32(1), 66-72.Mischaikow, K., & Mrozek, M. (1998). Chaos in the Lorenz equations: A computer assisted proof. Part II: Details. :en:Mathematics of Computation, 67(223), 1023-1046.Mischaikow, K., Mrozek, M., & Szymczak, A. (2001). Chaos in the lorenz equations: A computer assisted proof part iii: Classical parameter values. Journal of Differential Equations, 169(1), 17-56.Galias, Z., & Zgliczyński, P. (1998). Computer assisted proof of chaos in the Lorenz equations. Physica D: Nonlinear Phenomena, 115(3-4), 165-188.Tucker, W. (1999). The Lorenz attractor exists. Comptes Rendus de l'Académie des Sciences-Series I-Mathematics, 328(12), 1197-1202.
 Rossler equationZgliczynski, P. (1997). Computer assisted proof of chaos in the Rössler equations and in the Hénon map. Nonlinearity, 10(1), 243.

Related Software

ChebfunDriscoll, T. A., Hale, N., & Trefethen, L. N. (2014). Chebfun guide.Platte, R. B., & Trefethen, L. N. (2010). Chebfun: a new kind of numerical computing. In Progress in industrial mathematics at ECMI 2008 (pp. 69-87). Springer, Berlin, Heidelberg.Hashemi, B., & Trefethen, L. N. (2017). Chebfun in three dimensions. SIAM Journal on Scientific Computing, 39(5), C341-C363.Wright, G. B., Javed, M., Montanelli, H., & Trefethen, L. N. (2015). Extension of Chebfun to periodic functions. SIAM Journal on Scientific Computing, 37(5), C554-C573.
COSY INFINITY Makino, K., & Berz, M. (2006). Cosy infinity version 9. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 558(1), 346-350.Berz, M., Makino, K., Shamseddine, K., Hoffstätter, G. H., & Wan, W. (1996). 32. COSY INFINITY and Its Applications in Nonlinear Dynamics.
 INTLABS.M. Rump: INTLAB - INTerval LABoratory. In Tibor Csendes, editor, Developments in Reliable Computing, pages 77-104. Kluwer Academic Publishers, Dordrecht, 1999. and kvOverview of kv – a C++ library for verified numerical computation, Masahide Kashiwagi, SCAN 2018. are interval arithmetic libraries which include ODE solvers
 MATLABWouwer, A. V., Saucez, P., & Vilas, C. (2014). Simulation of Ode/Pde Models with MATLAB®, OCTAVE and SCILAB: Scientific and Engineering Applications. Springer.Houcque, D. (2008). Applications of MATLAB: Ordinary differential equations (ODE). Robert R. McCormick School of Engineering and Applied Science-Northwestern University, Evanston.Shampine, L. F., & Reichelt, M. W. (1997). The matlab ode suite. SIAM Journal on Scientific Computing, 18(1), 1-22.Ashino, R., Nagase, M., & Vaillancourt, R. (2000). Behind and beyond the MATLAB ODE suite. Computers & Mathematics with Applications, 40(4-5), 491-512. - made by MathWorks
 NAG libraryGladwell, I. (1979). Initial value routines in the NAG library. ACM Transactions on Mathematical Software (TOMS), 5(4), 386-400.Gladwell, I. (1979). The development of the boundary-value codes in the ordinary differential equations chapter of the NAG library. In Codes for Boundary-Value Problems in Ordinary Differential Equations (pp. 122-143). Springer, Berlin, Heidelberg.Berzins, M., Brankin, R. W., & Gladwell, I. (1988). The stiff integrators in the NAG library. ACM SIGNUM Newsletter, 23(2), 16-23.
 Wolfram MathematicaBaumann, G. (2013). Symmetry analysis of differential equations with Mathematica®. Springer Science & Business Media.Abell, M. L., & Braselton, J. P. (2016). Differential equations with Mathematica. Academic Press.Gray, A., Mezzino, M., & Pinsky, M. A. (1997). Introduction to ordinary differential equations with Mathematica: an integrated multimedia approach. Springer.Ross, C. C. (2013). Differential equations: an introduction with Mathematica®. Springer Science & Business Media. - made by Wolfram Research

References

Further reading

 Mitsui, T., & Shinohara, Y. (1995). Numerical analysis of ordinary differential equations and its applications. World Scientific.
 Iserles, A. (2009). A first course in the numerical analysis of differential equations. Cambridge University Press.
 Hairer, Ernst; Nørsett, Syvert Paul; Wanner, Gerhard (1993), Solving ordinary differential equations I: Nonstiff problems, Berlin, New York: Springer-Verlag.
 Wanner, G. & Hairer, E. (1996), Solving ordinary differential equations II: Stiff and differential-algebraic problems (2nd ed.). Springer Berlin Heidelberg.
 Butcher, John C. (2008), Numerical Methods for Ordinary Differential Equations, New York: John Wiley & Sons.
 John D. Lambert, Numerical Methods for Ordinary Differential Systems, John Wiley & Sons, Chichester, 1991.
 Deuflhard, P., & Bornemann, F. (2012). Scientific computing with ordinary differential equations. Springer Science & Business Media.
 Shampine, L. F. (2018). Numerical solution of ordinary differential equations. Routledge.
 Dormand, John R. (1996), Numerical Methods for Differential Equations: A Computational Approach, Boca Raton: CRC Press.

Other websites
 Joseph W. Rudmin, Application of the Parker–Sochacki Method to Celestial Mechanics , 1998.
 Dominique Tournès, L'intégration approchée des équations différentielles ordinaires (1671-1914), thèse de doctorat de l'université Paris 7 - Denis Diderot, juin 1996. Réimp. Villeneuve d'Ascq : Presses universitaires du Septentrion, 1997, 468 p. (Extensive online material on ODE numerical analysis history, for English-language material on the history of ODE numerical analysis, see e.g. the paper books by Chabert and Goldstine quoted by him.)
INTLAB 
Verified ODE (IVP) Solver

Category:Differential equations
Category:Numerical analysis