# Solution of Higher-Order ODEs Using Backward Difference Method

Authors: Mohamed Bin Suleiman; Zarina Bibi Binti Ibrahim; Ahmad Fadly Nurullah Bin Rasedee
Journal: Mathematical Problems in Engineering
 (2011)

---

## Abstract

The current numerical technique for solving a system of higher-order ordinary differential equations (ODEs) is to reduce it to a system of first-order equations then solving it using first-order ODE methods. Here, we propose a method to solve higher-order ODEs directly. The formulae will be derived in terms of backward difference in a constant stepsize formulation. The method developed will be validated by solving some higher-order ODEs directly with constant stepsize. To simplify the evaluations of the integration coefficients, we find the relationship between various orders. The result presented confirmed our hypothesis.


---

## Body

## 1. Introduction

Differential equations constantly arise in various branches of science and engineering. Many of these problems are in the form of higher-order ordinary differential equations (ODEs). A few examples where these problems can be found are, in the motion of projectiles, the bending of a thin clamped beam and population growth.The popular practice for solving a system of higher-order ODEs is by reducing it to a system of first-order equations and then solving with first-order methods. These methods worked, so that methods for solving higher-order ODEs have been disregarded as robust codes. However, the work by Krogh [1], Suleiman [2], Majid and Suleiman [3], and Omar and Suleiman [4] has revived the interest in solving higher-order ODEs directly and the theoretical development of the methods.Related works for solving higher-order ODEs can be found in Collatz [5], Gear [6], Krogh [1, 7], and Suleiman [2]. Krogh [7] proposed the direct integration (DI) method for nonstiff problems using modified divided difference while Suleiman [2] proposed the DI method using the standard divided difference. In this paper, we will derive the constant stepsize backward difference formulae of solving higher-order ODEs up to third order. The main reason for developing the constant stepsize formulae is that, in developing the theory on convergence and stability, the approach is through constant stepsize formulation. Another reason is that it is possible to use this formula in conjunction with other similar formulae as in Majid and Suleiman [3] to develop a code for variable stepsize and order.The advantage of such a code is that the integration or differentiation constants are calculated only once at the start of the first step of integration, whereas other formulations calculate the constants at every step.In this paper, we will focus only on nonstiff ODEs of the form(1.1)y(d)=f(x,Ỹ),(1.2)Ỹ(x)=f(x,y,y′,y′′,…,y(d-1)),η̃=(η,η′,η′′,…,η(d-1)),
where Ỹ(a)=η̃ in the interval a≤x≤b and d is the order of the ODE.Without loss of generality, we will be considering the scalar equation in (1.1).This paper will be organized as follows. First, the integration coefficients of the explicit constant stepsize backward difference formulation of the DI method will be derived. Then, the coefficients of the implicit method are formulated and their relationship with the explicit coefficients is shown. We start the derivation with the coefficients of the first-order system, which is given in Henrici [8]. Next, the second-order coefficients are derived and their relationship with the corresponding first-order coefficients is given, likewise the relationship of the coefficients for the second- and third-order systems. Finally, the method developed using backward difference will be validated numerically.


## 2. The Formulation of the Predict-Evaluate-Correct-Evaluate (PECE) Multistep Method in Its Backward Difference Form (MSBD) for Nonstiff Higher-Order ODEs

The code developed will be using the PECE mode. The predictor and corrector will have the following form:predictor:(2.1)yprn+1(d-t)=∑i=0t-1hii!yn(d-t+1)+hd∑i=0k-1γ(d-t),i∇ifn,t=1,2,…,d,
corrector:(2.2)yn+1(d-t)=∑i=0t-1hii!yn(d-t+1)+hd∑i=0k-1γ(d-t),i*∇ifn+1,t=1,2,…,d.
The corrector will be reformulated, so that it will be in terms of the predictor. The reformulated corrector can be written as(2.3)yn+1(d-t)=yprn+1(d-t)+γ(d-t),k∇kpr(fn+1),t=1,2,…,d,
where fn+1pr indicates fn+1 evaluated using predicted values. The integration coefficient γ(d-t),i and γ(d-t),i* will be derived using the method of generating function. Finally, the constant stepsize algorithm will be constructed and validated with some test problems and examples from physical situations.


## 3. Derivation up to Third-Order Explicit Integration Coefficients

Integrating (1.1) once yields (3.1)y(xn+1)=y(xn)+∫xnxn+1f(x,y,y′,y′′)dx.
Let Pn(x) be the interpolating polynomial which interpolates the k values (xn,fn),(xn-1,fn-1),…,(xn-k+1,fn-k+1), then(3.2)Pn(x)=∑i=0k-1(-1)i(-si)∇ifn.
Next, approximating f in (3.1) with Pn(x) and letting(3.3)x=xn+shors=x-xnh
gives us(3.4)y(xn+1)=y(xn)+∫01∑i=0k-1(-1)i(-si)∇ifnds,
or(3.5)y(xn+1)=y(xn)+h∑i=0k-1γ1,i∇ifn,
where(3.6)γ1,i=(-1)i∫01(-si)ds.
The generating function G1(t) for the coefficients γ1,i is defined as follows:(3.7)G1(t)=∑i=0∞γ1,iti.
Substituting γ1,i in (3.6) in G1(t) gives(3.8)G1(t)=∑i=0∞(-t)i∫01(-si)ds,G1(t)=∫01(1-t)(-s)ds,G1(t)=∫01e-slog⁡⁡(1-t)ds
which leads to(3.9)G1(t)=-[(1-t)-1log⁡⁡(1-t)-1log⁡⁡(1-t)].
Equation (3.9) can be written as(3.10)-(∑i=0∞γ1,iti)log⁡⁡(1-t)=[t(1-t)]or(3.11)(γ1,0+γ1,1t+γ1,2t2+γ1,3t3+⋯)(t+t22+t33+t44+⋯)=t(1+t+t2+t3+⋯).
Hence, the coefficients of γ1,k are given by(3.12)∑i=0k(γ1,ik-i+1)=1,γ1,k=1-∑i=0k-1(γ1,ik-i+1)k=1,2,…,γ1,0=1.


## 4. Second-Order ODE Formulae

Integrate (1.1) twice for second-order ODEs where d=2. Integrating once leads to the same coefficients as given in (3.6). Integrating twice yields(4.1)y(xn+1)=y(xn)+hy′(xn)+h2∑i=0kγ2,i∇ifn.
Substituting x with s gives(4.2)γ2,i=(-1)i∫01(1-s)1!(-si)ds.
The generating function G2(t) of the coefficients γ2,i is defined as follows(4.3)G2(t)=∑i=0∞γ2,iti.
Substituting (4.2) into G2(t) above gives(4.4)G2(t)=∫01(1-s)1!e-slog⁡⁡(1-t)ds.
Substituting G1(t) into (4.4) yields(4.5)G2(t)=11![1log⁡⁡(1-t)-1!G1(t)log⁡⁡(1-t)].
Equation (4.5) can be written as(4.6)(∑i=0∞γ2,iti)log⁡⁡(1-t)=11![1-1!G1(t)]or(4.7)(γ2,0+γ2,1t+γ2,2t2+⋯)(t+t22+t33+⋯)=11![-1+1!(γ1,0+γ1,1t+γ1,2t2+⋯)].
Hence the coefficients of γ2,k in relation to coefficients of the previous order γ1,k are given by(4.8)∑i=0kγ2,ik-i+1=γ1,k+1,γ2,0=γ1,1,γ2,k=γ1,k+1-∑i=0kγ2,ik-i+1,k=1,2,….


## 5. Third-Order Formulae

Next, the case of the third-order ODE whered=3 will be considered. In the case of y′′,y′, the corresponding coefficients are γ1,i,γ2,i as in (3.6) and (4.2). For the solution y(x), integrating three times yields(5.1)y(xn+1)=y(xn)+hy′(xn)+h22!y′′(xn)+∫xnxn+1(xn+1-x)(2)(2)!f(x,y,y′,y′′)dx
or in the backward difference formulation, given by(5.2)y(xn+1)=y(xn)+hy′(xn)+h22!y′′(xn)+h3∑i=0kγ3,i∇ifn,
where(5.3)γ3,i=(-1)i∫01(1-s)2!2(-si)ds.
The generating function G3(t) of the coefficients γ3,i is defined as follows:(5.4)G3(t)=∑i=0∞γ3,iti.Substituting (5.3) into G3(t) above yields(5.5)G3(t)=∫01(1-s)2!2e-slog⁡⁡(1-t)ds.
As in (4.4), we now substitute G2(t) in (5.5) which gives(5.6)G3(t)=12![1log⁡⁡(1-t)-2!G2(t)log⁡⁡(1-t)].
Equation (5.6) can be written as(5.7)(∑i=0∞γ3,iti)log⁡⁡(1-t)=12![1-2!G2(t)]or(5.8)(γ3,0+γ3,1t+γ3,2t2+⋯)(t+t22+t33+⋯)=12![-1+2!(γ2,0+γ2,1t+γ2,2t2+⋯)].
Hence, the coefficients of γ3,k in relation to coefficients of the previous order γ2,k are given by(5.9)∑i=0kγ3,ik-i+1=γ2,k+1,γ3,0=γ2,1,γ3,k=γ2,k+1-∑i=0kγ3,ik-i+1,k=1,2,….


## 6. Derivation up to the Third-Order Implicit Integration Coefficients

Integrating (1.1) once yields (6.1)y(xn+1)=y(xn)+∫xnxn+1f(x,y,y′,y′′)dx.
Let Pn(x) be the interpolating polynomial which interpolates the k values (xn,fn),(xn-1,fn-1),…,(xn-k+1,fn-k+1):(6.2)Pn(x)=∑i=0k(-1)i(-si)∇ifn+1.
As in the previous derivation, we choose(6.3)x=xn+1+sh,ors=x-xn+1h.
Replacing xby s yields (6.4)y(xn+1)=y(xn)+∫-10∑i=0k(-1)i(-si)∇ifn+1ds.
Simplify (6.5)y(xn+1)=y(xn)+h∑i=0kγ1,i*∇ifn,
where(6.6)γ1,i*=(-1)i∫-10(-si)ds.
The generating function G1*(t) of the coefficientsγ1,i* is defined as follows:(6.7)G1*(t)=∑i=0∞γ1,i*tior(6.8)G1*(t)=∑i=0∞(-t)i∫-10(-si)ds,G1*(t)=∫-10(1-t)(-s)ds,G1*(t)=∫-10e-slog⁡⁡(1-t)ds
which leads to(6.9)G1*(t)=-[1log⁡⁡(1-t)-(1-t)log⁡⁡(1-t)].
For the case d=2, the approximate solution of y has the form(6.10)y(xn+1)=y(xn)+hy′(xn)+∫xnxn+1(xn+1-x)(1)!(1)f(x,y,y′,y′′)dx.
The coefficients are given by(6.11)γ2,i*=(-1)i∫-10(-s)1!(-si)ds,
where γ2,i* are the coefficients of the backward difference formulation of (6.11) which can be represented by(6.12)y(xn+1)=y(xn)+hy′(xn)+h2∑i=0kγ2,i*∇ifn.
The generating function G2*(t) for the coefficients γ2,i* is defined as follows:(6.13)G2*(t)=∑i=0∞γ2,i*ti.
Substituting (6.11) into G2*(t) above,(6.14)G2*(t)=∫-10(-s)1!e-slog⁡⁡(1-t)ds.
Solving (6.14) with the substitution of (3.8) produces the relationship(6.15)G2*(t)=11![(1-t)log⁡⁡(1-t)-1!G1*(t)log⁡⁡(1-t)].
Integrating (1.1) thrice yields(6.16)y(xn+1)=y(xn)+hy′(xn)+h22!y′′(xn)+∫xnxn+1(xn+1-x)(2)(2)!f(x,y,y′,y′′)dx.
The coefficients are given by(6.17)γ3,i*=(-1)i∫-10(-s)2!2(-si)ds,
where γ3,i* are the coefficients of the backward difference formulation of (6.17) which can be represented by(6.18)y(xn+1)=y(xn)+hy′(xn)+h22!y′′(xn)+h3∑i=0kγ3,i*∇ifn.The generating functionG3*(t) of the coefficients γ3,i* is defined as follows:(6.19)G3*(t)=∑i=0∞γ3,i*ti.Substituting (6.17) into G3*(t) above yields(6.20)G3*(t)=∫-10(-s)2!2e-slog⁡⁡(1-t)ds.
Solving (6.11) with the substitution of (6.20) produces the relationship(6.21)G3*(t)=12![(1-t)log⁡⁡(1-t)-2!G2*(t)log⁡⁡(1-t)].


## 7. The Relationship between the Explicit and Implicit Integration Coefficients

Calculating the integration coefficients directly is time consuming when large numbers of integration are involved. A more efficient way of computing the coefficients is by obtaining a recursive relationship between the coefficients. With this recursive relationship, we are able to obtain the implicit integration coefficient with minimal time consumption. The relationship between the explicit and implicit coefficients is expressed below.For first-order coefficients,(7.1)G1*(t)=-[1log⁡⁡(1-t)-1-tlog⁡⁡(1-t)].It can be written as(7.2)G1*(t)=-(1-t)[1(1-t)log⁡⁡(1-t)-1log⁡⁡(1-t)].
By substituting (7.3)G1(t)=1(1-t)log⁡⁡(1-t)-1log⁡⁡(1-t)
into (7.2), we have (7.4)G1*(t)=(1-t)G1(t),(∑i=0∞γ1,i*ti)=(1-t)(∑i=0∞γ1,iti).
Expanding the equation yields(7.5)(γ1,0*+γ1,1*t+γ1,2*t2+⋯)=1(1+t+t2+⋯)(γ1,0+γ1,1t+γ1,2t2+⋯),(γ1,0*+γ1,1*t+γ1,2*t2+⋯)(1+t+t2+⋯)=(γ1,0+γ1,1t+γ1,2t2+⋯).
This gives the recursive relationship(7.6)∑i=0kγ1,i*=γ1,k.
For second-order coefficient, (7.7)G2*(t)=-11![1log⁡⁡(1-t)-1!G1*(t)log⁡⁡(1-t)].
It can be written as(7.8)G2*(t)=(1-t)1![1log⁡⁡(1-t)-1!G1*(t)(1-t)log⁡⁡(1-t)].
Substituting (7.4) into the equation above gives(7.9)G2*(t)=(1-t)1![1log⁡⁡(1-t)-1!(1-t)G1(t)(1-t)log⁡⁡(1-t)]or(7.10)G2*(t)=(1-t)1![1log⁡⁡(1-t)-1!G1(t)log⁡⁡(1-t)].
Substituting (4.5) into (7.10) gives(7.11)G2*(t)=(1-t)G2(t),(∑i=0∞γ2,i*ti)=(1-t)(∑i=0∞γ2,iti).
Expanding the equation, we have(7.12)(γ2,0*+γ2,1*t+γ2,2*t2+⋯)=1(1+t+t2+⋯)(γ2,0+γ2,1t+γ2,2t2+⋯),(γ2,0*+γ2,1*t+γ2,2*t2+⋯)(1+t+t2+⋯)=(γ2,0+γ2,1t+γ2,2t2+⋯).
The above gives the relationship(7.13)∑i=0kγ2,i*=γ2,k.
For third-order coefficient, we have(7.14)G3*(t)=-12![1log⁡⁡(1-t)-2!G2*(t)log⁡⁡(1-t)].
It can be written as(7.15)G3*(t)=(1-t)2![1log⁡⁡(1-t)-2!G2*(t)(1-t)log⁡⁡(1-t)].
Substituting (7.10) into (7.15) gives(7.16)G3*(t)=(1-t)2![1log⁡⁡(1-t)-2!(1-t)G2(t)(1-t)log⁡⁡(1-t)]or(7.17)G3*(t)=(1-t)2![1log⁡⁡(1-t)-2!G2(t)log⁡⁡(1-t)].
Substituting  (7.18)G3(t)=12![1log⁡⁡(1-t)-2!G2(t)log⁡⁡(1-t)]into (6.15) leads to(7.19)G3*(t)=(1-t)G3(t),(∑i=0∞kγ3,i*ti)=(1-t)(∑i=0∞γ3,iti).
Expanding the equation into,(7.20)(γ3,0*+γ3,1*t+γ3,2*t2+⋯)=1(1+t+t2+…)(γ3,0+γ3,1t+γ3,2t2+⋯),(γ3,0*+γ3,1*t+γ3,2*t2+⋯)(1+t+t2+…)=(γ3,0+γ3,1t+γ3,2t2+⋯)which leads to a recursive relationship(7.21)∑i=0kγ3,i*=γ3,k.Tables1 and 2 are a few examples of the explicit and implicit integration coefficients.Table 1
The explicit integration coefficients fork from 0 to 6.
k0123456γ1,k11/25/123/8251/72095/28819087/60480γ2,k1/21/61/819/1803/32863/10080275/3456γ3,k1/61/247/24017/72041/2016731/403208563/518400Table 2
The implicit integration coefficients fork from 0 to 6.
k0123456γ1,k*1−1/2−1/12−1/24−19/720−3/160−813/60480γ2,k*1/2−1/3−1/24−7/360−17/1440−41/5040−731/120960γ3,k*1/6−1/8−1/80−1/180−11/3360−89/40320−5849/3628800


## 8. Numerical Results

For error calculations, we will be using the three error tests, namely, absolute error, relative error, and mixed error tests. The error formula is given by,(8.1)Error=Max⁡xn⁡‖y(xn)-ynA+By(xn)‖,
where A=1, B=0 gives the absolute error test, A=0, B=1 gives the relative error test, and A=1, B=1 gives the mixed error test.In (8.1), y(xn) is the exact solution for the problem considered and yn the computed solution. In a general code when the exact solution is not available for the relative error, y(xn) is replaced by yn the computed value.When∥y(xn)-yn∥ is small, the error in (8.1) will approximate the absolute error. However, when it is large, the mixed error test will approximate the relative error. The numerical results give the three errors.The following notations hold MAX ABS: maximum error when using absolute error test, MAX MIX: maximum error when using mixed error test, MAX REL: maximum error when using relative error test,h: step size selected.For the choice of problems to be tested, we choose four linear problems consisting of a second- and a third-order problem. The third problem is a mix system of second- and first-order equations and the fourth problem is a system of three second-order equations. Our reason for choosing the linear problems is that if the formulae are correct, then they should solve linear problems. The choice of system of equations is to raise the degree of difficulty of solving the problems. The rest of the problems are nonlinear, which occur in physical situations. The choices of the physical problems are those with exact solutions known.  We give our comments on the numerical results right after the numerical Tables3, 4, 5, 6, 7, 8, and 9.Table 3hMAX ABSMAX MIXMAX REL10-14.73734 + 0261.18857 − 0031.18857 – 00310-24.70433 + 0231.16697 − 0061.16697 – 00610-34.42360 + 0201.18335 − 0091.18335 − 00910-44.05706 + 0201.01668 − 0091.01668 − 00910-53.69529 + 0219.26023 − 0099.26024 − 009Table 4hMAX ABSMAX MIXMAX REL10-11.60072 + 0231.40364 − 0031.40364 − 00310-21.52595 + 0201.33620 − 0061.33620 − 00610-31.55425 + 0171.36098 − 0091.36098 − 00910-41.27710 + 0161.11807 − 0101.11807 − 01010-52.20767 + 0171.93311 − 0091.93311 − 009Table 5hMAX ABSMAX MIXMAX REL10-11.74071 − 0021.68285 − 0036.12179 − 00310-22.32581 − 0054.04838 − 0065.34483 − 00510-32.38274 − 0084.39132 − 0095.74568 − 00710-45.33390 − 0095.47762 − 0109.42528 − 00910-54.54132 − 0084.54132 − 0085.25218 − 007Table 6hMAX ABSMAX MIXMAX REL10-11.40347 + 0011.55464 + 0006.27279 + 00210-21.78122 − 0021.75004 − 0021.60254 + 00110-32.32649 − 0052.32643 − 0059.97054 − 00110-42.44607 − 0082.44600 − 0089.82644 − 00110-59.90539 − 0109.90531 − 0109.99724 − 001Table 7hMAX ABSMAX MIXMAX REL10-12.34537 − 0014.96059 − 0041.00000 + 00010-22.85286 − 0042.86491 − 0071.00000 + 00010-32.91084 − 0076.19617 − 0101.00083 + 00010-43.00856 − 0087.74497 − 0131.80152 + 00010-52.43305 − 0071.09024 − 0117.08729 + 000Table 8hMAX ABSMAX MIXMAX REL10-11.01336 − 0048.29238 − 0055.15346 − 00210-28.33352 − 0088.33254 − 0089.08294 − 00410-39.71863 − 0119.71317 − 0114.58972 − 00610-45.45085 − 0105.45075 − 0103.45420 − 00410-54.75544 − 0094.75543 − 0091.91598 − 002Table 9hMAX ABSMAX MIXMAX REL10-16.95439 − 0025.63803 − 0022.97860 − 00110-28.31669 − 0057.12977 − 0054.99583 − 00410-38.60027 − 0087.37166 − 0085.16016 − 00710-48.66052 − 0117.42330 − 0115.19631 − 01010-51.86645 − 0121.55538 − 0129.33222 − 012Problem 1.
(8.2)y′′=2y′-y,y(0)=0,y′(0)=1,0≤x≤64.Solution 1.
(8.3)y=xex.Source: Krogh [7].This is a linear equation used by Krogh [7] to test his code. The solution increases exponentially to a maximum value of y(64)=64e64 which is considered large and therefore not suitable for absolute error test and hence the large values of the error.Problem 2.
(8.4)y′′′=2y′′-4,0≤x≤30,y(0)=1,y′(0)=2,y′′(0)=6.Solution 2.
(8.5)y(x)=x2+e2x.Source: Omar and Suleiman [4].This is a third-order problem with an exponential solution. The difference between Problems1 and 2 is that one is third order and the other is second order. Again, absolute error test does not work for the same reason given above.Problem 3.
(8.6)y1′′=-2y1′-5y2+3,y2′=y1′+2y2,0≤x≤16π,y1(0)=0,y1′(0)=0,y2(0)=1.Solution 3.
(8.7)y1(x)=2cos⁡⁡x+6sin⁡x-2-6x,y2(x)=-2cos⁡⁡x+2sin⁡x+3.Source: Bronson [9].For this problem, all error tests worked well.Problem 4.
(8.8)y1′′=1-y2-y3,y2′′=y3-y1,y3′′=y1′+y2′,0≤x≤4π,y1(0)=0,y1′(0)=1,y2(0)=0,y2′(0)=0,y3(0)=-1,y3′(0)=1.Solution 4.
(8.9)y1(x)=sin⁡x,y2(x)=cos⁡⁡x-1,y2(x)=sin⁡x-cos⁡⁡x.Source: Bronson [9].This problem does not work for relative error test because of the small value of the solution for certain values ofx.Problem 5.
(8.10)y′′′=-1xy′′+1x2y′+1x,1≤x≤50,y(1)=2621ln⁡2⁡(2)+99104,y′(1)=-4021ln⁡⁡(2)-513,y′′(1)=326+47ln⁡⁡(2).Solution 5.
(8.11)y(x)=x28(2ln⁡⁡(x2)-3313-23ln⁡⁡(2))+(13-2621ln⁡⁡(x2))ln⁡⁡(2)+3326.Source: Russel and Shampine [10].This problem is the symmetrical bending of a laterally loaded circular plate.The numerical results of this problem show the failure to control the error using relative error test. This is because the solution is zero whenx=2.Problem 6.
(8.12)y1′′=-y1r3,y1(0)=1,y1′(0)=0,y2′′=-y2r3,y2(0)=0,y2′(0)=1,r=(y12+y22)1/2,0≤x≤16π.Solution 6.
(8.13)y1(x)=cos⁡⁡x,y2(x)=sin⁡x.Source: Shampine and Gordon [11].This problem is Newton’s equations of motion for the two-body problem.Again, relative error test does not work too well for this problem becauseyn is very small at certain points xn.Problem 7.
(8.14)y′′=2y3,0≤x≤5,y(0)=1,y′(0)=-1.Solution 7.
(8.15)y(x)=1x+1.Source: Robert Jr. [12].For this problem, all error tests worked well.All the numerical results show that the errors in the mixed error mode give a reliable error estimate for all the problems given. The absolute error mode failed to give meaningful error results for Problems1 and 2. This is because the value of ∥yn∥ increases as x increases and this becomes large. Similarly, for Problems 4, 5, and 6, the relative error failed to give an acceptable result because ∥yn∥ is small.The research work done shows that the method developed for solving higher-order ODEs directly using the backward difference is successful. We recommend that, for multistep method, the error control procedure should use the mixed error test. This research suggests the potential of this work developing a robust code for solving higher-order ODEs directly.



---

*Source: 810324-2011-07-12.xml*