Meta-Learning Runge-Kutta

1 INTRODUCTION . Differential equations in their general form cover an extremely wide variety of disciplines : While many applications are rather intuitive , as for instance simple Newtonian physics and engineering , other more exotic use cases include the governing of price evolution in economics ( Black & Scholes , 1973 ) , the study of rates in chemical reactions ( Scholz & Scholz , 2014 ) , and the modeling of population growths in biology ( Lotka , 1925 ; Volterra , 1926 ) . In medicine , differential equations may be used to model cancer growth ( Ilea et al. , 2013 ) , diabetes and the glucose metabolism ( Esna-Ashari et al. , 2017 ) as well as for pharmaceutical drug design ( Deuflhard , 2000 ) . Recently , differential equations have also been used as a way to design neural networks ( Chen et al. , 2018 ) . Unfortunately , finding an analytical solution in closed form is in many cases very difficult , if not impossible . Therefore , a variety of numerical integration methods have been developed to obtain accurate , but approximate solutions . Arguably , the most prominent ones are Runge-Kutta methods , a family of integration methods for initial value problems . However , setting up Runge-Kutta involves several design choices , one of which is the step size controller . Using an adaptive step size strategy instead of a constant step size can often increase efficiency by several orders of magnitude , c.f . ( Söderlind , 2006 ) . Their performance is hampered by the fact that they only make use of hand-designed features . Contribution . We show how to cast the design of a step size controller for Runge-Kutta as a learning problem , c.f . Fig . 1 . The key ingredients of the resulting Meta-Learning Runge-Kutta ( MLRK ) are the identification of a good performance measure and appropriate inputs . Related Work . Various approaches to control the step sizes in Runge-Kutta methods have been proposed . While some rely on signal processing principles where the goal is to produce a smooth step size sequence in conjunction with acceptable local errors , others are based on the assumption that step sizes should be adapted to a prescribed function of the solution , e.g . to preserve structure in geometric integration ( Söderlind , 2006 ) . In this work we will focus on control theoretic approaches which aim to keep the local error associated with a single step close to a tolerance parameter . Specifically , the design of the step size control algorithm is learned within MLRK . Indeed , casting algorithm design as a learning problem is not new . Andrychowicz et al . ( 2016 ) learned a gradientbased optimizer for nonlinear unconstrained optimization problems . Wichrowska et al . ( 2017 ) extended this work by introducing a hierarchical RNN architecture which improves the generalization ability of the optimizer . Schramowski et al . ( 2018 ) demonstrated the benefit of a learned projectionfree convex optimization algorithm , which relies on conditional gradients . Finally , Chen et al . ( 2017 ) cast the design of gradient-free black-bock optimization as a learning problem . Furthermore , other approaches to solve differential equations using neural networks have been proposed , too . Lagaris et al . ( 1998 ) suggested to use a feed-forward neural network to approximate the solution of an ordinary or partial differential equation with initial or boundary conditions . Inspired by the Galerkin method , Sirignano & Spiliopoulos ( 2018 ) used a deep neural network to directly approximate the solution of a high-dimensional partial differential equation . E & Yu ( 2018 ) proposed the Deep Ritz method as a means to solve variational problems that arise from partial differential equations . Han et al . ( 2018 ) cast the problem of solving semilinear parabolic partial differential equations as a learning problem by using the reformulation of these differential equations as backward stochastic differential equations . In all of the above approaches , a neural network is used to help approximate the solution of the differential equation . The learning process corresponds to the numerical computation of a solution of the differential equation . In contrast , MLRK learns to improve the numerical integration process itself . We proceed as follows . As the first step , i.e . identifying a good performance measure , we consider the general objective of step size control as well as the simplified and more practical objective that many controllers are based on . Then , we identify useful inputs by analyzing existing step size control algorithms . Furthermore , we show how local information about the ODE can be used as additional favorable inputs . Before concluding , we demonstrate empirically how our proposed controller can be used to improve step size control and investigate the benefit of different inputs . 2 INITIAL VALUE PROBLEMS AND RUNGE-KUTTA METHODS . We give a brief introduction to initial value problems and Runge-Kutta methods , a numerical method for solving these problems . Furthermore , we will recap standard step size controllers and point out their underlying assumptions . Basics of Initial Value Problems and Runge-Kutta Methods . An ordinary differential equation describes a system that depends on one variable , often referred to as time . An initial value problem additionally provides initial values : y′ = g ( t , y ) , y ( t0 ) = y0 , with a function g : [ a , b ] × Rm → Rm , m ∈ N. As closed-form solutions can be very hard to find , numerical methods such as RungeKutta methods have been developed . Their mathematical underpinning is quite evolved ( Butcher , 2008 ; Hairer et al. , 2000 ; Hairer & Wanner , 2002 ) . A brief recap can be found in the appendix . For iterative solvers , the local truncation error—the error committed in a single step using step size h—constitutes an important tool to control the step size as it allows to access the fitness of a step size . Definition 2.1 . The local truncation error in step n+ 1 is defined as errn+1 ( h ) = ‖y ( tn + h ) − yn+1‖ = C ( tn ) hp+1 +O ( hp+2 ) , ( 1 ) where yn+1 = Φ ( tn , yn , h ) is a single Runge-Kutta step of a p-th order method using step size h. We call C ( tn ) the principal error term . We will also denote the local truncation error as local error and refer to the appendix for simple ways to estimate it . We denote the error estimates as errn . The Objective of Step Size Control . As numerical integration is used to approximate a solution , high accuracy is a desired property . However , in practice , computational resources might be limited and efficient use of these resources can be crucial in some applications . Hence , an efficient step size controller should maximize the accuracy of the approximate solution while minimizing the computational cost . These are competing goals and require a trade-off , which is achieved by considering the Lagrangian of the error E ( H ) and the work cost W ( H ) produced by a sequence of step sizes H = ( hn ) N−1n=0 , E ( H ) + λ ·W ( H ) . Butcher ( 2008 ) uses two integrals describing the error E ( H ) and the work cost W ( H ) . Under some assumptions one can show that optimal step sizes w.r.t . this objective causes local errors equal to the Lagrange multiplier λ , which is often referred to as the tolerance parameter . Namely , we need to assume that the global error is equal to the sum of local errors and all step sizes are small enough so that the loss can be approximated by the integrals suggested by Butcher . However , these are quite strong assumptions . The optimal step size depends on the tolerance parameter , and hence , might be too large . Furthermore , the assumption that the global error is equal to the sum of the local errors is not true in general . Classical Step Size Control . Most common step size control mechanisms aim to keep the local errors equal to a tolerance parameter so that a standard controller is based on the following idea : For a RungeKutta method of order p , the local error for step size h at time t is approximately err ≈ C ( t ) hp+1 , and , accordingly , the desired optimal step size hopt is given by hopt ≈ h ( tol/err ) 1 p+1 . This formula is then used to adapt the step size as hn+1 = rnhn , rn = max ( α , min ( β , γ ( tol errn+1 ) 1 p+1 ) ) , ( 2 ) where a safety factor 0 < γ < 1 and a minimal and maximal factor α , β ∈ R≥0 , α ≤ β are included to avoid excessive step rejection and ensure a smooth step size control . Söderlind ( 2002 ) pointed out that the underlying assumptions of Eq . ( 2 ) are rather strong . It assumes slow variations in C and requires h to be sufficiently small so that it exhibits its theoretical asymptotic behavior . Both are not true in general . Control Theory on Step Size Control in Runge-Kutta Methods . Gustafsson et al . ( 1988 ) discussed step size control in the context of a proportional-integral-derivative ( PID ) controller . The classical step size control mechanism in Eq . ( 2 ) can be regarded as an I-controller . They then demonstrated the oscillatory behavior of this controller when applied to certain problems . They report the poor stabilizing capability of an I-controller as the origin of these oscillations , which are further accentuated by a large integration gain . To overcome this , they Gustafsson et al . ( 1988 ) suggested a PI-controller , which in addition to the integral term used in a standard step size control , also includes a proportional term and can be expressed as hn+1 = rnhn , rn = max ( α , min ( β , γ ( tol errn+1 ) n1 ( tol errn ) n2 ) ) . ( 3 ) A Predictive Controller . Both the classical step size controller , see Eq . ( 2 ) , and the PI-controller in Eq . ( 3 ) rely on the assumption that the local truncation error can be described as a function of h that remains independent of t. This is certainly not true for all cases . Therefore , Gustafsson ( 1994 ) , e.g. , discussed a controller based on a prediction of the principal error term . A simple model that assumes a constant linear trend in logC ( t ) was suggested . Implementing the predicted principal error leads to hn+1 = hn hn−1 ( tol errn+1 ) n1 ( errn errn+1 ) n2 hn . ( 4 ) 3 META-LEARNING RUNGE-KUTTA : NEURAL STEP-SIZE CONTROLLER . Step size controllers for Runge-Kutta are typically still designed by hand . Formulas such as Eq . ( 2 ) , ( 3 ) and ( 4 ) require parameter fine-tuning . In this work , we take a different tack and instead suggest to replace these hand-designed update rules with a learned controller , hereinafter referred to as optimizer . The key steps to cast the design of a controller as a learning problem is to determine a good performance measure and appropriate inputs . Performance measure . As the first step we need to identify an appropriate performance measure . As discussed , a trade-off between numerical accuracy and computational cost of the solution must be made , e.g . by considering the Lagrangian of error and work cost . The work cost is generally correlated with the number of integration steps . It is crucial to take the length of the integration interval tn − t0 into account as well , which leads to the following performance measure : lLagrange ( tn , yn , errn ) = ‖y ( tn ) − yn‖+ tol · n tn − t0 . ( 5 ) In contrast , if we follow the concept of most step size controllers , an optimal step size is achieved if the local error is equal to the tolerance , which suggests the performance measure llocal ( tn , yn , errn ) = ‖errn − tol‖ . ( 6 ) Both types of loss functions offer various strengths and weaknesses over one another . The loss Eq . ( 5 ) incorporates the essential quantities that an efficient algorithm should minimize . However , the true global error of numerical integration is generally unknown and can only be approximated . In general this can be computationally demanding and as a result make the training process very expensive . Furthermore , when the global error is employed , optimal step sizes may depend on the integration interval as well , e.g . the choice of step size in the first few steps may have different effects on the global error at different points in the integration interval . Classic step size controllers are typically based on local errors alone without regards to global strategies . The objective of these controllers is reflected by the loss Eq . ( 6 ) . Although this objective minimizes the Lagrangian of the global error and the number of steps only under strong assumptions it does qualify as an appropriate practical objective for step size controllers.The advantage here is that estimates of the local error associated with a step size are available and no additional computation to determine the loss of a step size is required . This allows efficient training even when the analytical solution of an initial value problem is unknown . Input Features . Existing step size controllers rely on error estimates to update the step size . Specifically , tolerrn is important , as evident in Eq . ( 2 ) , ( 3 ) and ( 4 ) . We intuitively expect a data driven approach to be able to utilize these features . This gives rise to the first set of input features , ψ ( errn+1 , · ) = log ( tol errn+1 ) . ( 7 ) Here , ψ ( errn+1 , · ) denotes the input for the optimizer with optional variables . Next , we want to investigate additional input features that may be beneficial . The principal error term as well as higher order terms of the local errors can be expressed with elementary differentials of the ODE , c.f . ( Hairer et al. , 2000 , Section II.3 ) . These elementary differentials are formed of function values and partial derivatives of g and allow a Taylor approximation of the local error function errn+1 ( h ) = φ ( tn , h ) h k+1 +O ( hk+2 ) , where φ ( tn , h ) is a polynomial in h with coefficients formed from partial derivatives of g up to order k > p. The roots of the polynomial φ ( tn , h ) hk+1 − tol approximate the optimal step size . Moreover , it is reasonable to assume that the local error takes values greater tol for some h > 0 , implying the existence of real roots . Since the complex roots of polynomials are known to depend continuously on the coefficients of the polynomial , c.f . ( Rahman & Schmeisser , 2002 , Theorem 1.3.1 ) , it is clear that the optimal step size , a real root of p ( h ) , depends continuously on the partial derivatives and function values of g as well as the tolerance tol . From the Universal Approximation Theorem ( Hornik , 1991 ) , it then follows that the optimal step size function can be approximated by a neural network with input ( ( ∂αg ( tn , yn ) ) |α|=k , . . . , ∂g ( tn , yn ) , g ( tn , yn ) , tol ) . This suggests that the partial derivatives of g can also be an appropriate input for our controller , ψ ( errn+1 , · ) = ( log ( tol errn+1 ) , ( ∂αg ( tn , yn ) ) |α|=p , . . . , ∂g ( tn , yn ) , g ( tn , yn ) ) . ( 8 ) However , providing partial derivatives requires additional computation whereas the p function values that were computed during the Runge-Kutta step are conveniently available . As a trade-off between “ perfect ” information in the form of higher order partial derivatives and the computational effort to compute them , we also propose to use the following input : ψ ( errn+1 , · ) = ( log ( tol errn+1 ) , g ( 1 ) n , . . . , g ( p ) n ) , ( 9 ) where g ( 1 ) n , . . . , g ( p ) n denotes p function values of the initial value problem . In contrast to the traditional input ( 7 ) , our novel inputs ( 8 ) and ( 9 ) both provide local information about the structure of the problem at hand , which allows one to keep the local error close to the tolerance parameter . Moreover , hand-designed controller and existing Runge-Kutta methods do not make use of these information , and it is difficult—if not impossible—to do so . Training an optimizer with input ( 8 ) or ( 9 ) , as shown next , does so automatically and designs novel Runge-Kutta methods . Meta-Learner . With a good performance measure and appropriate inputs at hand , we can now solve the step-size control problem as a meta-learning problem as sketched already in Fig . 1 . Let the input for the optimizer be xn = ψ ( errn , · ) . We parameterize the optimizer c using φ and update the step size as follows , log hn+1 = log hn + cn ( ψ ( errn+1 , · ) , φ ) . An approach in log space has the advantage that that we do not need to constrain the output of c to be positive . Since cn corresponds to log rn , we will adjust the notation in a similar fashion . Due to their natural ability to handle sequential tasks , an LSTM was chosen in our experiments as optimizer c , and its prediction determines the step size which will be used in the next Runge-Kutta iteration . We denote the model of the optimizer , represented by the blue block in Fig . 1 , by m , its parameters by φ and the hidden state by ĥn . It can be expressed by ( log rn−1 ĥn+1 ) = m ( xn , ĥn , φ ) . ( 10 ) The subsequent Runge-Kutta update in the yellow block of Fig . 1 first updates the step size log hn = log hn−1 + log rn−1 , ( 11 ) and then uses it to execute the next Runge-Kutta step , tn+1 = tn + hn , ( yn+1 errn+1 ) = Φ ( tn , yn , hn ) . ( 12 ) Here , Φ denotes the Runge-Kutta step from yn to yn+1 as well as the error estimate errn+1 that is computed during that step . Afterwards , the next input for the model will be computed . Learning Objective . Different initial value problems can require very different step sizes . Using an optimizer that is specialized for a certain class of problems allows to exploit the structure of these problems . The behavior of an initial value problem is described by a function g , therefore we can represent a class of initial value problems with a distribution over the functions g. Thus , an optimizer can be considered optimal for a class of problems , if it minimizes the expected loss : L ( φ ) = Eg ( ∑N n=1 l ( tn , yn , errn ) ) , ( 13 ) where tn , yn , errn were computed according to Eqs . ( 10 ) , ( 11 ) , ( 12 ) . Here , l denotes the loss function defined for an approximation yn at time point tn and the local error estimate errn . Note that our training loss directly corresponds to the performance measure we are interested in . The model m can then learn the behavior of the given class of problems and use this knowledge to generalize to new examples of the same class and new , unseen classes . Specifically , since the provided performance measure l is differentiable a.e. , we can optimize the learning objective Eq . ( 13 ) using gradient descent on the parameters φ . An estimate of the gradient ∂L ( φ ) ∂φ can be computed by sampling a function g from the distribution of the class of initial value problems and applying backpropagation , c.f . ( Rumelhart et al. , 1986 ) .

Summary: The paper proposes to learn the step size for a Runge-Kutta numerical integrator for solving ordinary differential equations initial value problems. The authors frame the stepsize control problem as a learning problem, based on different performance measures, on ODE dependent inputs and on a LSTM for predicting the next step coefficient. Experiments are performed on 3 ODEs by training and testing in different contexts.