# Solving Fractional Differential Equations by Using Triangle Neural Network

Authors: Feng Gao; Yumin Dong; Chunmei Chi
Journal: Journal of Function Spaces
 (2021)

---

## Abstract

In this paper, numerical methods for solving fractional differential equations by using a triangle neural network are proposed. The fractional derivative is considered Caputo type. The fractional derivative of the triangle neural network is analyzed first. Then, based on the technique of minimizing the loss function of the neural network, the proposed numerical methods reduce the fractional differential equation into a gradient descent problem or the quadratic optimization problem. By using the gradient descent process or the quadratic optimization process, the numerical solution to the FDEs can be obtained. The efficiency and accuracy of the presented methods are shown by some numerical examples. Numerical tests show that this approach is easy to implement and accurate when applied to many types of FDEs.


---

## Body

## 1. Introduction

Fractional differential equations (FDEs) have been a hot topic in many scientific fields, such as dynamical system control theory, fluid flow, modelling in rheology, dynamic process of self-similar porous structure, diffusion transport similar to diffusion, electric network, and probability statistics [1–9]. These problems in science and engineering sometimes require us to get the solutions of various fractional differential equations. But as we know, it is difficult to find the exact solutions in most cases. So, we have to use numerical methods to solve fractional differential equations.In the literature, some numerical methods for solving FDEs have been proposed, such as nonlinear functional analysis methods, including monotone iterative technique [10], topological degree theory [11], and fixed point theorem [12]. In addition, someone proposed the following numerical methods: random walk [13], Adomian decomposition method and variational iteration method [14], homotopy perturbation method [15–17], etc.In recent years, some scholars try to use the neural network to solve differential equations [18–20]. Lagaris et al. [21] proposed an artificial neural network method for solving initial and boundary value problems. In their work, a trial solution is adopted and written as the sum of two parts. The first part satisfies the initial or boundary conditions and does not contain adjustable parameters while the construction of the second part does not affect the initial and boundary conditions. Then, the neural network is trained to satisfy the differential equation at many selected points. The question for this method is that it is difficult to construct the first part of the trial solution and this method cannot be applied to fractional partial differential equations.Piscopo et al. [22] also introduced a method to find the numerical solutions of many types of differential equations. The proposed method does not depend on the trial solution and therefore has more flexibility in many cases. It can be used for solving many types of ODE and PDE. The two mentioned neural network techniques motivate us to develop more neural network methods to solve FDEs, but how to get the fractional order of the neural network is a difficult problem.To overcome this difficulty, in this work, we use a triangle base neural network as basis function to propose an alternative method called triangle neural network methods. This paper is organized as follows. In Section2, we study the fractional derivative of the triangle base neural network and present the numerical method for solving many types of FDEs. In Section 3, we show the efficiency of the proposed method by some numerical examples. Section 4 is the conclusion.


## 2. Fractional Derivative of Triangle Neural Network and Numerical Algorithm

### 2.1. Ordinary Fractional Differential Equation

To solve the following fractional initial value problem (1) and boundary value problem (2),
(1)Dαy=fx,y,yk0=yk,k=0,1,⋯,m−1,where m=α:
(2)axDαy+bxDβy=fx,y0=y0,yl=y1,where ax and bx are real functions, 1<α≤2, 0<β≤1. Dα and Dβ are Caputo fractional derivative operators.We consider the following triangle base neural network 1 (see Figure1) to approximate the solution of problems (1) and (2), where wj are weights for the neural networks and Cjx are triangle base functions as the following:
(3)Cjx=cosjx,j=0,1,2,⋯,N,sinj−Nx,j=N+1,N+2,⋯,2N,where Cjx are activation function of neurons in the hidden layer of the above neural network and N is an integer and x∈0,π.Figure 1
Triangle neural network 1.Let the weight matrix bew=w0,w1,⋯,w2NT and the activation matrix be Cx=c0x,c1x,⋯,c2NxT. The triangle base neural network can be written as
(4)y=∑j=02NwjCjx.When this neural network is used to be the numerical solution of problem (1), the loss function is
(5)J=12∑k=1me2t=12∑k=1mDαyxt−fxt,yxt2.For problem (2), The loss function is
(6)J=12∑k=1me2t=12∑k=1maxtDαyxt+bxtDβyxt−fxt2,where xt,t=1,2,⋯,m are training points. We have two methods to minimize the loss function to get the corresponding numerical solution. One is the gradient descent algorithm, and another one is the optimization process. For both methods, we need to compute the α derivative of the triangle neural network. For this purpose, we have the following theorems.Theorem 1.
For givenα∈R+, fx∈C1R, then
(7)Dαfλx=λαfuu=λx.Proof.
SinceDαfx=1/Γ1−α∫0xf′τ/x−ταdτ.We have(8)Dαfλx=1Γ1−α∫0xfλτ′x−ταdτ.Letλτ=u, we have
(9)Dαfλx=1Γ1−α∫0λxλf′ux−u/λαdu=1Γ1−α∫0λxf′uλx−uαλ−αdu=1Γ1−α∫0λxλαf′uλx−uαdu=λαΓ1−α∫0λxf′uλx−uαdu=λαDαfλx.We also have the following.Theorem 2.
Givenα∈R+, λ,b∈R, then
(10)Dαsinλx+b=λαsinλx+b+απ2,Dαcosλx+b=λαcosλx+b+απ2.Proof.
(11)Dαsinλx+b=Dαsinλxcosb+cosλxsinb.
From Theorem1, we have
(12)cosbDαsinλx+sinbDαcosλx=cosbλαsinλx+απ2+sinbλαsinλx+απ2=λαsinλx+b+απ2.
The second part of this theorem can be verified in the same way. Based on Theorems1 and 2, we can get the α derivative of the triangle base neural network.In fact, let the solution to problems (1) and (2) be
(13)y=∑j=0Nwjcosjx+∑j=0NwN+jsinjx=w0+∑j=1Nwjcosjx+∑j=1NwN+jsinjx.We can get(14)Dαy=∑j=1Njαwjcosjx+α2π+∑j=1NjαwN+jsinjx+α2π.Thus, we get the loss function for problem (1):
(15)J=12∑k=1me2t=12∑k=1mDαyxt−fxt,yxt2=12∑t=1m∑j=1Njαwjcosjxt+α2π+jαwN+jsinjxt+α2π−fxt,yxt2.To carry out the gradient descent process, we have(16)∂J∂wk=∑t=1m∑j=0Njαwjcosjxt+α2π+wN+jsinjx+α2π−fx,y⋅kαcoskxt+α2π−fyxt,ycoskxtfor j=0,1,2,⋯,N, and we also have
(17)∂J∂wk=∑t=1m∑j=0Njαwjcosjxt+α2π+wN+jsinjxt+α2π−fxt,y⋅kαsinkxt+α2π−fyxt,ysink−Nxtfor j=N+1,N+2,⋯,2N.So, we can see that getting the numerical solution of (1) is equivalent to finding wjs by minimizing the loss function J. Usually, we have two methods to do this work. One is the gradient descent method, and another one is adopting the optimization process.The gradient descent method is as below:(18)wkn+1=wkn−η∂J∂wk,k=1,2,⋯,2N,where η is the step size for the gradient descent. If the function fx,y is a linear function of y, the initial value problem can also be reduced to
(19)minJ=12∑k=1mDαyk−fxk,yk2s.t.y0=y0.That is,(20)minJ=12∑k=1mDαyk−fxk,yk2s.t.∑i=0Nwi=y0,which is a quadratic optimization problem. For fractional boundary value problem (2), the numerical solution can be reduced to the following optimization process:
(21)minJ=12∑k=1maxkDαyk+bxkDβyk−fxk2s.t.∑i=1Nwi=y0,∑i=0Ncosilwi+sinN+ilwN+i=y1.So, there are two methods to solve this problem. One method is to get the solution through the gradient descent method. Another method is using the optimization technique.


### 2.2. Fractional Partial Differential Equation

For fractional partial differential equation problem(22)∂αux,t∂tα=a∂2ux,t∂x2+ft,x,u0,t=u0t,u1,t=u1t,ux,0=vx,where 0<α≤1, and problem
(23)∂ux,t∂t=a∂βux,t∂xβ+ft,x,u0,t=u0t,u1,t=u1t,ux,0=vx,where 1<β≤2. We use the triangle base neural network (see Figure 2) to approximate the solution of problems (22) and (23). The triangle base neural network can be written as
(24)ux,y=∑i=1Nwisinai1x+ai2y−bi,where ai1,ai2 are weights for the import layer in the neural network, bi are bias parameters for the hidden layer in the neural network, wi are weights for the export layer in the neural network, and sinx is the activation function of neurons in the hidden layer.Figure 2
Neural network 2.Based on Theorems1 and 2, we can get the fractional derivative of the neural network as below:
(25)∂αux,y∂xα=∑i=1Nwiai1αsinai1x+ai2y+bi+α2π,∂βux,y∂yβ=∑i=1Nwiai2βsinai1x+ai2y+bi+α2π.The loss function for problem (22) is
(26)minJ=12∑k=1m∂αu∂tkα−∂2u∂xk2−ftk,xk2+12∑tk,xk∈Bmutk,xk−Btk,xk2,where tk,xk are training points and B is the boundary of problem (22) and
(27)B0,t=u0t,Bl,t=u1t,Bx,0=vx.The loss function for problem (23) can be given in the same way.We use the gradient descent algorithm to train the neural network. In fact, we can train the neural network by layers. First, we train the export layer to getwis, then the bias parameters to get bis in the hidden layer, and finally, we train the import layer to get ai1s and ai2s.


## 2.1. Ordinary Fractional Differential Equation

To solve the following fractional initial value problem (1) and boundary value problem (2),
(1)Dαy=fx,y,yk0=yk,k=0,1,⋯,m−1,where m=α:
(2)axDαy+bxDβy=fx,y0=y0,yl=y1,where ax and bx are real functions, 1<α≤2, 0<β≤1. Dα and Dβ are Caputo fractional derivative operators.We consider the following triangle base neural network 1 (see Figure1) to approximate the solution of problems (1) and (2), where wj are weights for the neural networks and Cjx are triangle base functions as the following:
(3)Cjx=cosjx,j=0,1,2,⋯,N,sinj−Nx,j=N+1,N+2,⋯,2N,where Cjx are activation function of neurons in the hidden layer of the above neural network and N is an integer and x∈0,π.Figure 1
Triangle neural network 1.Let the weight matrix bew=w0,w1,⋯,w2NT and the activation matrix be Cx=c0x,c1x,⋯,c2NxT. The triangle base neural network can be written as
(4)y=∑j=02NwjCjx.When this neural network is used to be the numerical solution of problem (1), the loss function is
(5)J=12∑k=1me2t=12∑k=1mDαyxt−fxt,yxt2.For problem (2), The loss function is
(6)J=12∑k=1me2t=12∑k=1maxtDαyxt+bxtDβyxt−fxt2,where xt,t=1,2,⋯,m are training points. We have two methods to minimize the loss function to get the corresponding numerical solution. One is the gradient descent algorithm, and another one is the optimization process. For both methods, we need to compute the α derivative of the triangle neural network. For this purpose, we have the following theorems.Theorem 1.
For givenα∈R+, fx∈C1R, then
(7)Dαfλx=λαfuu=λx.Proof.
SinceDαfx=1/Γ1−α∫0xf′τ/x−ταdτ.We have(8)Dαfλx=1Γ1−α∫0xfλτ′x−ταdτ.Letλτ=u, we have
(9)Dαfλx=1Γ1−α∫0λxλf′ux−u/λαdu=1Γ1−α∫0λxf′uλx−uαλ−αdu=1Γ1−α∫0λxλαf′uλx−uαdu=λαΓ1−α∫0λxf′uλx−uαdu=λαDαfλx.We also have the following.Theorem 2.
Givenα∈R+, λ,b∈R, then
(10)Dαsinλx+b=λαsinλx+b+απ2,Dαcosλx+b=λαcosλx+b+απ2.Proof.
(11)Dαsinλx+b=Dαsinλxcosb+cosλxsinb.
From Theorem1, we have
(12)cosbDαsinλx+sinbDαcosλx=cosbλαsinλx+απ2+sinbλαsinλx+απ2=λαsinλx+b+απ2.
The second part of this theorem can be verified in the same way. Based on Theorems1 and 2, we can get the α derivative of the triangle base neural network.In fact, let the solution to problems (1) and (2) be
(13)y=∑j=0Nwjcosjx+∑j=0NwN+jsinjx=w0+∑j=1Nwjcosjx+∑j=1NwN+jsinjx.We can get(14)Dαy=∑j=1Njαwjcosjx+α2π+∑j=1NjαwN+jsinjx+α2π.Thus, we get the loss function for problem (1):
(15)J=12∑k=1me2t=12∑k=1mDαyxt−fxt,yxt2=12∑t=1m∑j=1Njαwjcosjxt+α2π+jαwN+jsinjxt+α2π−fxt,yxt2.To carry out the gradient descent process, we have(16)∂J∂wk=∑t=1m∑j=0Njαwjcosjxt+α2π+wN+jsinjx+α2π−fx,y⋅kαcoskxt+α2π−fyxt,ycoskxtfor j=0,1,2,⋯,N, and we also have
(17)∂J∂wk=∑t=1m∑j=0Njαwjcosjxt+α2π+wN+jsinjxt+α2π−fxt,y⋅kαsinkxt+α2π−fyxt,ysink−Nxtfor j=N+1,N+2,⋯,2N.So, we can see that getting the numerical solution of (1) is equivalent to finding wjs by minimizing the loss function J. Usually, we have two methods to do this work. One is the gradient descent method, and another one is adopting the optimization process.The gradient descent method is as below:(18)wkn+1=wkn−η∂J∂wk,k=1,2,⋯,2N,where η is the step size for the gradient descent. If the function fx,y is a linear function of y, the initial value problem can also be reduced to
(19)minJ=12∑k=1mDαyk−fxk,yk2s.t.y0=y0.That is,(20)minJ=12∑k=1mDαyk−fxk,yk2s.t.∑i=0Nwi=y0,which is a quadratic optimization problem. For fractional boundary value problem (2), the numerical solution can be reduced to the following optimization process:
(21)minJ=12∑k=1maxkDαyk+bxkDβyk−fxk2s.t.∑i=1Nwi=y0,∑i=0Ncosilwi+sinN+ilwN+i=y1.So, there are two methods to solve this problem. One method is to get the solution through the gradient descent method. Another method is using the optimization technique.


## 2.2. Fractional Partial Differential Equation

For fractional partial differential equation problem(22)∂αux,t∂tα=a∂2ux,t∂x2+ft,x,u0,t=u0t,u1,t=u1t,ux,0=vx,where 0<α≤1, and problem
(23)∂ux,t∂t=a∂βux,t∂xβ+ft,x,u0,t=u0t,u1,t=u1t,ux,0=vx,where 1<β≤2. We use the triangle base neural network (see Figure 2) to approximate the solution of problems (22) and (23). The triangle base neural network can be written as
(24)ux,y=∑i=1Nwisinai1x+ai2y−bi,where ai1,ai2 are weights for the import layer in the neural network, bi are bias parameters for the hidden layer in the neural network, wi are weights for the export layer in the neural network, and sinx is the activation function of neurons in the hidden layer.Figure 2
Neural network 2.Based on Theorems1 and 2, we can get the fractional derivative of the neural network as below:
(25)∂αux,y∂xα=∑i=1Nwiai1αsinai1x+ai2y+bi+α2π,∂βux,y∂yβ=∑i=1Nwiai2βsinai1x+ai2y+bi+α2π.The loss function for problem (22) is
(26)minJ=12∑k=1m∂αu∂tkα−∂2u∂xk2−ftk,xk2+12∑tk,xk∈Bmutk,xk−Btk,xk2,where tk,xk are training points and B is the boundary of problem (22) and
(27)B0,t=u0t,Bl,t=u1t,Bx,0=vx.The loss function for problem (23) can be given in the same way.We use the gradient descent algorithm to train the neural network. In fact, we can train the neural network by layers. First, we train the export layer to getwis, then the bias parameters to get bis in the hidden layer, and finally, we train the import layer to get ai1s and ai2s.


## 3. Numerical Experiment

### 3.1. Numerical Test 1

Consider the following example 1:(28)Dαy=1x2+1y+Γ3Γ3−αx2−α,y0=1,where 0<α≤1. The exact solution to this problem is y=x2+1. We let α be 0.5,0.9 and use the optimization method when N=5,N=15 and gradient descent method when N=5,N=15, respectively. The computational results are listed in Tables 1 and 2.Table 1
Computational errors for example 1 whereα=0.5,h=0.1.
x1 h2 h3 h4 h5 h6 h7 h8 h9 h10 hoptimi (N=5)0.0100.0130.0150.0160.0180.0190.0210.0220.0240.026optimi (N=15)0.0030.0030.0080.0090.0120.0140.0130.0140.0150.016grad,N=50.0110.0130.0170.0190.0250.0210.0200.0190.0270.030grad,N=150.0080.0080.0070.0040.0050.0090.0120.0180.0150.013Table 2
Computational errors for example 1 whereα=0.9,h=0.1.
x1 h2 h3 h4 h5 h6 h7 h8 h9 h10 hoptimi (N=5)0.0000.0100.0100.0110.0120.0140.0180.0170.0180.020optimi (N=15)0.0000.0020.0060.0080.0100.0100.0110.0100.0120.012grad,N=50.0090.0130.0120.0140.0210.0200.0200.0160.0210.021grad,N=150.0050.0060.0030.0020.0070.0120.0130.0100.0100.010


### 3.2. Numerical Test 2

Consider the following example 2 for boundary value problem:(29)D1+αy+Dαy=2sinx+π4cosα2+2cosx+π4sinα2,y0=0,yπ2=1,where 0<α≤1. The exact solution to this problem is y=sinx.We letα be 0.5 and N=30,N=45, respectively; the computational error is listed in Figure 3.Figure 3
Solution of Test 2 whenα=0.5.We letα be 0.8 and N=20,N=30. The computational error is listed in Figure 4.Figure 4
Solution of Test 2 whenα=0.8.As we see in example 1, the solution becomes more accurate whenN is increased. And for the boundary value problem, we use two constraints when we use the optimization process.


### 3.3. Numerical Test 3

Consider the following example 3:(30)∂αux,t∂tα=−Γ4Γ4−αt−α∂2ux,t∂x2,u0,t=uπ,t=0,ux,0=0,where 0<α≤1. The exact solution to this problem is u=t3sinx.We use the gradient descent method to solve this problem, and the computational error is listed in Figure5. In the training process of the neural network, we set a stopping criteria J<10−5 for the computing process to stop. If this stopping criteria cannot be achieved, the computing will be stopped when 105 times of training is completed.Figure 5
Solution of Test 3 whenα=0.8.


### 3.4. Numerical Test 4

Consider the following example 4:(31)∂ux,t∂t=∂βux,t∂xβ−t3sinx+β2π+3t2sinx,u0,t=uπ,t=0,ux,0=0,where 1<β≤2. The exact solution to this problem is u=t3sinx.We use the gradient descent method to solve this problem, and the computational error is listed in Figure6.Figure 6
Solution of Test 4 whenα=0.8.


## 3.1. Numerical Test 1

Consider the following example 1:(28)Dαy=1x2+1y+Γ3Γ3−αx2−α,y0=1,where 0<α≤1. The exact solution to this problem is y=x2+1. We let α be 0.5,0.9 and use the optimization method when N=5,N=15 and gradient descent method when N=5,N=15, respectively. The computational results are listed in Tables 1 and 2.Table 1
Computational errors for example 1 whereα=0.5,h=0.1.
x1 h2 h3 h4 h5 h6 h7 h8 h9 h10 hoptimi (N=5)0.0100.0130.0150.0160.0180.0190.0210.0220.0240.026optimi (N=15)0.0030.0030.0080.0090.0120.0140.0130.0140.0150.016grad,N=50.0110.0130.0170.0190.0250.0210.0200.0190.0270.030grad,N=150.0080.0080.0070.0040.0050.0090.0120.0180.0150.013Table 2
Computational errors for example 1 whereα=0.9,h=0.1.
x1 h2 h3 h4 h5 h6 h7 h8 h9 h10 hoptimi (N=5)0.0000.0100.0100.0110.0120.0140.0180.0170.0180.020optimi (N=15)0.0000.0020.0060.0080.0100.0100.0110.0100.0120.012grad,N=50.0090.0130.0120.0140.0210.0200.0200.0160.0210.021grad,N=150.0050.0060.0030.0020.0070.0120.0130.0100.0100.010


## 3.2. Numerical Test 2

Consider the following example 2 for boundary value problem:(29)D1+αy+Dαy=2sinx+π4cosα2+2cosx+π4sinα2,y0=0,yπ2=1,where 0<α≤1. The exact solution to this problem is y=sinx.We letα be 0.5 and N=30,N=45, respectively; the computational error is listed in Figure 3.Figure 3
Solution of Test 2 whenα=0.5.We letα be 0.8 and N=20,N=30. The computational error is listed in Figure 4.Figure 4
Solution of Test 2 whenα=0.8.As we see in example 1, the solution becomes more accurate whenN is increased. And for the boundary value problem, we use two constraints when we use the optimization process.


## 3.3. Numerical Test 3

Consider the following example 3:(30)∂αux,t∂tα=−Γ4Γ4−αt−α∂2ux,t∂x2,u0,t=uπ,t=0,ux,0=0,where 0<α≤1. The exact solution to this problem is u=t3sinx.We use the gradient descent method to solve this problem, and the computational error is listed in Figure5. In the training process of the neural network, we set a stopping criteria J<10−5 for the computing process to stop. If this stopping criteria cannot be achieved, the computing will be stopped when 105 times of training is completed.Figure 5
Solution of Test 3 whenα=0.8.


## 3.4. Numerical Test 4

Consider the following example 4:(31)∂ux,t∂t=∂βux,t∂xβ−t3sinx+β2π+3t2sinx,u0,t=uπ,t=0,ux,0=0,where 1<β≤2. The exact solution to this problem is u=t3sinx.We use the gradient descent method to solve this problem, and the computational error is listed in Figure6.Figure 6
Solution of Test 4 whenα=0.8.


## 4. Conclusion

The neural network method is a promising approach for solving fractional differential equations. The difficulty for this method is how to calculate the fractional derivatives of the involved neural network. In this paper, we propose numerical methods for solving fractional differential equations including the initial problem, boundary value problem, and partial FDEs by using the triangle base neural network and gradient descending method. All the involved fractional derivatives in this work are considered as Caputo type. We first analyze the fractional derivative of the triangle base neural network. Then, based on the loss function, the proposed numerical methods reduce the fractional differential equation into the gradient descent process or the quadratic optimization problem. By carrying out the gradient descent process or the quadratic optimization process, we can get the numerical solutions. Numerical tests show that this approach is easy to implement and the solution is accurate when applied to many types of FDEs.



---

*Source: 5589905-2021-04-14.xml*