# Local Polynomial Regression Solution for Differential Equations with Initial and Boundary Values

Authors: Liyun Su; Tianshun Yan; Yanyong Zhao; Fenglan Li
Journal: Mathematical Problems in Engineering
 (2013)

---

## Abstract

Numerical solutions of the linear differential boundary issues are obtained by using a local polynomial estimator method with kernel smoothing. To achieve this, a combination of a local polynomial-based method and its differential form has been used. The computed results with the use of this technique have been compared with the exact solution and other existing methods to show the required accuracy of it. The effectiveness of this method is verified by three illustrative examples. The presented method is seen to be a very reliable alternative method to some existing techniques for such realistic problems. Numerical results show that the solution of this method is more accurate than that of other methods.


---

## Body

## 1. Introduction

Linear ordinary differential equation boundary value problems are very common theoretical and practical issues. Some solutions are put forward in [1–6]. Most solutions usually include finite difference method and shooting method. The shooting method is the most widely used. Both single and multishooting methods: (1) select the initial value of needing to be solved; (2) use the differential equation initial value algorithm to compute step by step and then get to the end of boundary value; (3) compute the error function between boundary value to compute and real boundary value then readjust the initial value of needing to be solved until the error is up to a certain level of accuracy. In general, this computing process is more tedious. This text makes use of local polynomial estimation to solve the boundary value issues of homogeneous and nonhomogeneous linear differential equations and compare it with the method in [4].The linear differential equation below has a unique solution:(1)y′′(x)+p(x)y′+q(x)y=f(x),y(a)=y0,y(b)=y1.Among them,y0,y1 are constant, and the function p(x), q(x), f(x) is continuous in [a,b].The presented method is useful for obtaining numerical approximations of differential equations, and it is also quite straightforward to write corresponding codes in any programming languages. Also, roundoff errors and necessity for large computer memory are not faced in this method. The computed results obtained by this way have been compared with the exact solution to show the required accuracy of it and the existing methods. Furthermore, the current method is of a general nature and can therefore be used for solving some partial differential equations arising in various areas.


## 2. Local Polynomial Estimator for Differential Equations

In this paper, we take advantage of the numerical estimation with the use of local polynomial regression to solve the boundary value problem. The basic idea of this method is first proposed in [5]. First, we introduce the mathematical thoughts of local polynomial regression. This idea was mentioned in [7–10]. Since the form of regression function is not specified, so the data points with long distance from x0 provide little information to y(x0). Therefore, we can only use the local data points around x0. We suppose that y(x) has p+1 derivative at x0, by the Taylor expansion, for point x, located in the neighborhood of this point x0; we can use the p-order multivariate polynomials to locally approximate y(x) and the surrounding local point of x0; we model y(x) as
(2)y(x)≈∑j=0p‍βj(x-x0)j,
where parameter βj depends on x0, so-called local parameter. Obviously, the local parameter βj=y(j)(x0)/j! fit the local model with local data, and it can be minimized:
(3)∑i=1n[Yi-∑j=0pβj(Xi-x0)j]2Kh(Xi-x),whereh controls the size of the bandwidth of local area. Using matrix notation to represent the local polynomial regression is more convenient. Below is the design matrix corresponding to (4) with X and Y:
(4)X=(1X1-x0⋯(X1-x0)p⋮⋮⋱⋮1Xn-x0⋯(Xn-x0)p),Y=(Y1Y2⋮Yn).The weighted least squares problem (4) can be written as
(5)min(Y-Xβ)TW(Y-Xβ),
here,
(6)W=diag(Kh(X1-x0),…,Kh(Xn-x0)),
so the solution vector is
(7)β=(XTWX)-1XTWY.For the initial value problem (1) and (3), we choose the best quadratic kernel function (Fan and Yao have proved in [10]): (8)Kh(u)={34(1-u2)+,if|u|≤1;0,otherwise.


## 3. Local Polynomial Estimator

Consider the following:(9)y′′(x)+p(x)y′(x)+q(x)=f(x),a≤x≤b,y(a)=ya,y(b)=yb.Here, we suppose thatX1=a<X2<⋯<Xn=b, then we obtain the following relation:
(10)(∑j=0pβj(x-x0)j)′′+∑j=1pp(x)jβj(x-x0)j-1+∑j=0pq(x)βj(x-x0)j≈f(x),a≤x≤b.Then, the corresponding minimization function is(11){[ya-∑j=0pβj(X1-x0)j]2v+∑i=2n-1[f(Xi)-∑j=2pj(j-1)βj(Xi-x0)j-2v-∑j=1pp(x)jβj(Xi-x0)j-1v-∑j=0pq(x)βj(Xi-x0)j]2vv+[yb-∑j=0pβj(Xn-x0)j]2}Kh(Xi-x).We can write the solution vector:(12)β~=(X~TWX~)-1X~TWY~,
where
(13)Y~=(yaf(X2)f(X3)⋮f(Xn-1)yb).In order to find the expression ofX~, we need to find the elements of X~ at first.Consider the following:(14)i=1,a1j=(X1-x0)j,j=0,…,pi=2,…,n-1,bij=j(j-1)(Xi-x0)j-2,111111111111j=2,…,p;cij=p(Xi)j(Xi-x0)j-1,11111111111j=0,…,p;tij=q(Xi)(Xi-x0)j,11111111j=0,…,p;i=n,an,j=(Xn-x0)j,j=1,…,p.Consequently, we can get theX~(15)X~=[a10a11⋯⋯a1p-c20-t20⋯b22-c22-t22⋯b2p-c2p-t2p-c30-t30⋯b32-c32-t32⋯b3p-c3p-t3p⋮⋮⋮⋮⋮-cn-1,0-tn-1,0⋯bn-1,2-cn-1,2-tn-1,2⋯bn-1,p-cn-1,p-tn-1,pan,0an,1⋯⋯an,p]n,p+1.SubstitutingX~ and Y~ into
(16)β~=(X~TWX~)-1X~TWY~.Now, we can obtain the value ofβ~.


## 4. Example Simulation

In this part, we verified this method by solving three examples, solved the solution of specific differential problems and integration with local polynomial estimator, calculated the residual sum of squares, and compared with the existing algorithms. Calculations were performed by using MATLAB 7.0 programming solution.


### 4.1. Initial Value Problem of Homogeneous Linear Differential Equation

Example 1.
Boundary value problem of linear differential equation with constant coefficients:(17)y′′(x)+y(x)=0,0≤x≤π2,y(0)=0,y(π2)=1,
The eigenvalue method can be used to obtain the real solution of initial value problem. Different parametersn, h,   and  p correspond to different residual sum of square, given in Figure 1 and Table 1. LPR represents local polynomial regression. The exact solution is y=sin(x). In Figure 1, LPR is carried out for n=21 and p=5 giving h=π/30, and we get the en2 and percentage error (Figure 1: Solution of local polynomial estimator and exact solution of Example 1).
In Table1, comparison of the two methods, when h′ is π/2/100, we can get the percentage error of optimization algorithm. By comparing the two methods, we can know the difference of percentage error. Certainly, local polynomial regression has more advantages. Then, we can get better results for n=45 and p=6 given h=π/60, n=60, p=8, and h=π/80.
In Table1, LPR represents local polynomial regression, OPA stands for optimization algorithm, Pe stands for Percentage error and h represents step size in [3]. Residuals en2=∑i=1n‍[y(Xi)-y^(Xi)]2.Table 1
Comparison of LPR and OPA.
Parameter(n,p,h)
Residualsen2
Pe of LPR
Pe of OPA
h
′
n
=
21, p=5, h=π/30
1.0435
×
1
0
-
6
0.032
0.342
π
/
2
/
100
n
=
45, p=6, h=π/60
4.0214
×
1
0
-
8
0.0004
0.035
π
/
2
/
1000
n
=
60, p=8, h=π/80
3.6503
×
1
0
-
9
0.00001
0.003
π
/
2
/
10000Figure 1
LPR of Example1 with n=21,p=5andh=π/30.Example 2.
Boundary value problem of homogeneous linear differential equations:(18)y′′-(2sinx)y′-(cosx-sinx2)y=0,0≤x≤1,y(0)=0,y(1)=e-cos1.
Different parametersn, h,  and p correspond to different residual sum of square, given in Figure 2 and Table 2. LPR represents local polynomial regression. The exact solution is y=xe-cosx.  In Figure 2, local polynomial estimator is carried out for n=50 and p=7 given h=1/20, and we can get the en2. Here, en2=∑i=1n‍[y(Xi)-y^(Xi)]2. We conclude that local polynomial regression is more effective. Certainly, we can get better results for n=60 and p=9 given h=1/135 (Figure 2: Solution of local polynomial estimator and exact solution of Example 2).Table 2
Local polynomial regression.
Parameter(n,p,h)
Residual sum of squareen2
n
=
24, p=5, h=1/12
6.2523
×
1
0
-
6
n
=
50, p=7, h=1/20
1.5162
×
1
0
-
7
n
=
70, p=9, h=1/32
4.4037
×
1
0
-
8Figure 2
LPR of Example2 with n=50,p=7andh=1/20.Example 3.
Boundary value problem of nonhomogeneous linear differential equations boundary value problem:(19)xy′′+2(1-x)y′+(x-2)y=2ex,y(12)=12e1/2,y(1)=e.
The exact solution isy=xex. Different parameters n, h,andp correspond to different residual sum of square, given in Figure 3 and Table 3. LPR represents local polynomial regression. In Figure 3, local polynomial estimator is carried out for n=20 and p=4 giving h=1/11, and we can get the residual sum of square en2 which is very small. Here, en2=∑i=1n‍[y(Xi)-y^(Xi)]2. In Table 3, local polynomial regression, it can be concluded that the effect of local polynomial regression is good and the residual sum of square en2 is very small. It is sure that we can get better results for n=70 and p=10 giving h=0.0045 (Figure 3: Solution of local polynomial estimator and exact solution of Example 3).Table 3
Local polynomial regression.
Parameter(n,p,h)
Residual sum of squareen2
n
=
20, p=5, h=1/11
5.4816
×
1
0
-
6
n
=
40, p=5, h=1/19
3.6711
×
1
0
-
7
n
=
70, p=5, h=2/67
8.5220
×
1
0
-
8Figure 3
Absolute errors figure of Example3 with n=21,p=5andh=π/30.Figures4, 5, and 6 present the absolute errors of this example under three groups of parameters. We can see that the absolute errors obtain ×10-4 magnitude given different parameters n,h with the same p=5. Moreover, with the increase in n, the absolute errors decrease gradually. From Examples 1–3, we can conclude that the residual sum of square and absolute error are smaller by using local polynomial method which shows good accuracy.Figure 4
Absolute errors figure of Example3 with n=20, p=4 and  h=1/11.Figure 5
Absolute errors figure of Example3 with n=40, p=6  and  h=1/19.Figure 6
Absolute errors figure of Example3 with n=70, p=10 and  h=2/67.


## 4.1. Initial Value Problem of Homogeneous Linear Differential Equation

Example 1.
Boundary value problem of linear differential equation with constant coefficients:(17)y′′(x)+y(x)=0,0≤x≤π2,y(0)=0,y(π2)=1,
The eigenvalue method can be used to obtain the real solution of initial value problem. Different parametersn, h,   and  p correspond to different residual sum of square, given in Figure 1 and Table 1. LPR represents local polynomial regression. The exact solution is y=sin(x). In Figure 1, LPR is carried out for n=21 and p=5 giving h=π/30, and we get the en2 and percentage error (Figure 1: Solution of local polynomial estimator and exact solution of Example 1).
In Table1, comparison of the two methods, when h′ is π/2/100, we can get the percentage error of optimization algorithm. By comparing the two methods, we can know the difference of percentage error. Certainly, local polynomial regression has more advantages. Then, we can get better results for n=45 and p=6 given h=π/60, n=60, p=8, and h=π/80.
In Table1, LPR represents local polynomial regression, OPA stands for optimization algorithm, Pe stands for Percentage error and h represents step size in [3]. Residuals en2=∑i=1n‍[y(Xi)-y^(Xi)]2.Table 1
Comparison of LPR and OPA.
Parameter(n,p,h)
Residualsen2
Pe of LPR
Pe of OPA
h
′
n
=
21, p=5, h=π/30
1.0435
×
1
0
-
6
0.032
0.342
π
/
2
/
100
n
=
45, p=6, h=π/60
4.0214
×
1
0
-
8
0.0004
0.035
π
/
2
/
1000
n
=
60, p=8, h=π/80
3.6503
×
1
0
-
9
0.00001
0.003
π
/
2
/
10000Figure 1
LPR of Example1 with n=21,p=5andh=π/30.Example 2.
Boundary value problem of homogeneous linear differential equations:(18)y′′-(2sinx)y′-(cosx-sinx2)y=0,0≤x≤1,y(0)=0,y(1)=e-cos1.
Different parametersn, h,  and p correspond to different residual sum of square, given in Figure 2 and Table 2. LPR represents local polynomial regression. The exact solution is y=xe-cosx.  In Figure 2, local polynomial estimator is carried out for n=50 and p=7 given h=1/20, and we can get the en2. Here, en2=∑i=1n‍[y(Xi)-y^(Xi)]2. We conclude that local polynomial regression is more effective. Certainly, we can get better results for n=60 and p=9 given h=1/135 (Figure 2: Solution of local polynomial estimator and exact solution of Example 2).Table 2
Local polynomial regression.
Parameter(n,p,h)
Residual sum of squareen2
n
=
24, p=5, h=1/12
6.2523
×
1
0
-
6
n
=
50, p=7, h=1/20
1.5162
×
1
0
-
7
n
=
70, p=9, h=1/32
4.4037
×
1
0
-
8Figure 2
LPR of Example2 with n=50,p=7andh=1/20.Example 3.
Boundary value problem of nonhomogeneous linear differential equations boundary value problem:(19)xy′′+2(1-x)y′+(x-2)y=2ex,y(12)=12e1/2,y(1)=e.
The exact solution isy=xex. Different parameters n, h,andp correspond to different residual sum of square, given in Figure 3 and Table 3. LPR represents local polynomial regression. In Figure 3, local polynomial estimator is carried out for n=20 and p=4 giving h=1/11, and we can get the residual sum of square en2 which is very small. Here, en2=∑i=1n‍[y(Xi)-y^(Xi)]2. In Table 3, local polynomial regression, it can be concluded that the effect of local polynomial regression is good and the residual sum of square en2 is very small. It is sure that we can get better results for n=70 and p=10 giving h=0.0045 (Figure 3: Solution of local polynomial estimator and exact solution of Example 3).Table 3
Local polynomial regression.
Parameter(n,p,h)
Residual sum of squareen2
n
=
20, p=5, h=1/11
5.4816
×
1
0
-
6
n
=
40, p=5, h=1/19
3.6711
×
1
0
-
7
n
=
70, p=5, h=2/67
8.5220
×
1
0
-
8Figure 3
Absolute errors figure of Example3 with n=21,p=5andh=π/30.Figures4, 5, and 6 present the absolute errors of this example under three groups of parameters. We can see that the absolute errors obtain ×10-4 magnitude given different parameters n,h with the same p=5. Moreover, with the increase in n, the absolute errors decrease gradually. From Examples 1–3, we can conclude that the residual sum of square and absolute error are smaller by using local polynomial method which shows good accuracy.Figure 4
Absolute errors figure of Example3 with n=20, p=4 and  h=1/11.Figure 5
Absolute errors figure of Example3 with n=40, p=6  and  h=1/19.Figure 6
Absolute errors figure of Example3 with n=70, p=10 and  h=2/67.


## 5. Conclusions

In this paper, a local polynomial-based fitting method has been proposed for numerical solutions of differential equations with boundary values. Comparisons of the computed results with exact solutions and existing methods showed that the method has the capability of solving the differential equations with boundary values. It is also capable of obtaining highly accurate results with minimal computational effort for both time and space. It was seen that the local polynomial-based technique approximates the exact solution very well. Numerical results demonstrated that the proposed method is more effective, and local polynomial estimator is more accurate compared with other algorithms, for instance, optimization algorithm. Furthermore, by using these methods, we can also study the numerical solutions of boundary value problems for differential equations on whole line or fractional differential equations which have been studied by the authors in [11–13]. In order to illustrate our method further, we will focus on these kinds of fractional differential, integral differential equations.



---

*Source: 530932-2013-05-20.xml*