# A Numerical Method for Solving Fractional Differential Equations by Using Neural Network

Authors: Haidong Qu; Xuan Liu
Journal: Advances in Mathematical Physics
 (2015)

---

## Abstract

We present a new method for solving the fractional differential equations
of initial value problems by using neural networks which are constructed from
cosine basis functions with adjustable parameters. By training the neural networks
repeatedly the numerical solutions for the fractional differential equations were
obtained. Moreover, the technique is still applicable for the coupled differential equations
of fractional order. The computer graphics and numerical solutions show
that the proposed method is very effective.


---

## Body

## 1. Introduction

Recently, fractional differential equations have gained considerable importance due to their frequent appearance applications in fluid flow, rheology, dynamical processes in self-similar and porous structures, diffusive transport akin to diffusion, electrical networks, probability and statistics, control theory of dynamical systems, viscoelasticity, electrochemistry of corrosion, chemical physics, optics and signal processing [1–7], and so on. These applications in interdisciplinary sciences motivate us to try to find out the analytic or numerical solutions for the fractional differential equations. But for most ones it is difficult to find out or even have exact solutions. Thus, necessarily, the numerical techniques are applied to the fractional differential equations.Now, many effective methods for solving fractional differential equations have been presented, such as nonlinear functional analysis method including monotone iterative technique [8, 9], topological degree theory [10], and fixed point theorems [11–13]. Also, numerical solutions are obtained by the following methods: random walk [2], matrix approach [14], the Adomian decomposition method and variational iteration method [15], HAM [16–19], homotopy perturbation method (HPM) [20], and so forth. Not long ago, in [21], Raja et al. by applying Particle Swarm Optimization (PSO) algorithm along with feedforward ANN obtained the numerical solutions for fractional differential equations. But the convergence of the algorithm has not been proven, and this method is only applied to the single fractional differential equations. In this paper, we construct two different neural networks based on cosine functions and obtain the conditions of algorithm convergence.The first neural network (NU) is applied to linear and nonlinear fractional differential equations of the form(1)D0+αyx=fx,yx,0<x≤1,0<α≤1with initial condition as follows:(2)y0=C,where D0+α is the Caputo fractional derivatives of order α.The second neural network (NU) is applied to the fractional coupled differential equations of the form(3)D0+αyx=fx,yx,zx,0<x≤1,0<α≤1,D0+αzx=gx,yx,zx,with initial conditions as follows: (4)y0=C1,z0=C2,where D0+α is the Caputo fractional derivatives of order α. The solutions for the above two problems are written as cosine basis functions, whose parameters can be adjusted to minimize an appropriate error function. So we need to compute the gradient of the error with respect to the network parameters. By adjusting the parameters repeatedly, we obtain the numerical solutions when the error values are less than the required accuracy or the training times reach maximum.


## 2. Definitions and Lemma

Definition 1 (see [22]).
The Riemann-Liouville fractional integral of orderα∈R, α>0 of a function f(x)∈Cμ, μ≥-1 is defined as (5)I0+αftx≔1Γα∫0xftdtx-t1-α,x>0.Definition 2 (see [22]).
The Riemann-Liouville and Caputo fractional derivatives of orderα∈R, α>0 are given by (6)DRL0+αfx≔ddxnI0+n-αfx=1Γn-αddxn∫0xftdtx-tα-n+1,D0+αfx≔I0+n-αddxnfx=1Γn-α∫0xd/dtnftdtx-tα-n+1,where (n=[α]+1,x>0).Definition 3 (see [22]).
The classical Mittag-Leffler function is defined by(7)Eαx≔∑k=0∞xkΓαk+1,x∈C,α>0.The generalized Mittag-Leffler function is defined by (8)Eα,βx≔∑k=0∞xkΓαk+β,x,β∈C,α>0.Definition 4.
The functionsSinα,β⁡(x), Cos⁡α,β⁡(x) (x,β∈C,α>0) are defined by (9)Sinα,β⁡x=∑k=1∞-1k+1x2k-1Γα2k-1+β,Cos⁡α,β⁡x=∑k=0∞-1kx2kΓα2k+β.Obviously, Euler’s equations have the following forms: (10)Eα,βix=Cos⁡α,β⁡x+iSinα,β⁡x,Eα,β-ix=Cos⁡α,β⁡x-iSinα,β⁡x.Lemma 5.
IfSinα,β⁡(x) and Cos⁡α,β⁡(x) are defined as in Definition 4, then (11)Da+αx-aβ-1Sinμ,β⁡λx-aμ=x-aβ-α-1Sinμ,β-α⁡λx-aμ,(12)Da+αx-aβ-1Cos⁡μ,β⁡λx-aμ=x-aβ-α-1Cos⁡μ,β-α⁡λx-aμ.Proof.
The beta function was defined byβ~(p,q)≔∫01xp-1(1-x)q-1dx, and we have the following equation: (13)β~p,q=ΓpΓqΓp+q.Then according to the definition of Caputo fractional derivatives, we have (14)Da+αx-aβ-1Sinμ,β⁡λx-aμ=1Γn-α∫a+xd/dtnt-aβ-1Sinμ,β⁡λt-aμx-tα-n+1dt=1Γn-α∫0+x-ad/dξnξβ-1Sinμ,β⁡λξμx-ξ-aα-n+1dξξ=t-a=1Γn-α∫0+1d/dtntβ-1x-aβ-1Sinμ,β⁡λtμx-aμx-anx-aα-n+11-tα-n+1x-adtξ=tx-a=x-aβ-α-1Γn-α∫0+1d/dtntβ-1∑k=1∞-1k+1λtμx-aμ2k-1/Γμ2k-1+β1-tα-n+1dt=x-aβ-α-1Γn-α∑k=1∞-1k+1λx-aμ2k-1Γμ2k-1+β∫0+1d/dtntβ-1+μ2k-11-tα-n+1dt=x-aβ-α-1Γn-α∑k=1∞-1k+1λx-aμ2k-1Γμ2k-1+β∏i=1nβ-i+μ2k-1∫0+1tβ-1+μ2k-1-n1-tα-n+1dt=x-aβ-α-1Γn-α∑k=1∞-1k+1λx-aμ2k-1Γμ2k-1+β-nβ~β+μ2k-1-n,n-α=x-aβ-α-1Γn-α∑k=1∞-1k+1λx-aμ2k-1Γμ2k-1+β-nΓβ+μ2k-1-nΓn-αΓβ+μ2k-1-α=x-aβ-α-1∑k=1∞-1k+1λx-aμ2k-1Γμ2k-1+β-α=x-aβ-α-1Sinμ,β-α⁡λx-aμ.Then (11) holds. Similarly, we obtain (12). In particular, when β=1, μ=1, we have (15)Da+αSin1,1⁡λx-a=Da+αsin⁡λx-a=x-a-αSin1,1-α⁡λx-a,Da+αCos⁡1,1⁡λx-a=Da+αcos⁡⁡λx-a=x-a-αCos⁡1,1-α⁡λx-a.


## 3. Illustration of the Method and Application

### 3.1. The First Neural Network

To describe the method, we consider (1) with initial condition y(0)=C. The jth trial solution satisfying the initial condition is written as (16)yjx=∑i=1Mwi,jcos⁡⁡ix+C-∑i=1Mwi,jcos⁡⁡M+1x,where M represents the number of neurons and wi,j are unknown weights of the network determined in training procedures to reduce the error function: (17)J=12Ej22=12∑k=1Nejk2,Ej=ej1,ej2,…,ejNT,where N represents the number of sample points, ·2 is Euclidean norm, and (18)ejk=fxk,yjxk-D0+αyjxk=fxk,yjxk-xk-α∑i=1Mwi,jCos⁡1,1-α⁡ixk+C-∑i=1Mwi,jCos⁡1,1-α⁡M+1xk,where k=1,2,…,N; then we can adjust the weights wi,j by the following equation: (19)wi,j+1=wi,j+Δwi,j,where(20)Δwi,j=-μ∂J∂wi,j=-μ∑k=1N∂J∂ejk∂ejk∂wi,j=-μ∑k=1Nejkfyxk,yjxk·cos⁡⁡xk-cos⁡⁡M+1xk-xk-αejk·Cos⁡1,1-αxk-Cos⁡1,1-αM+1xk.


### 3.2. Convergence of the Algorithm

Theorem A.
Letμ represent learning rate, let N represent the number of sample points, and let M represent the number of neurons: X=(x1,x2,…,xN),  δ<xi<1, 1≤i≤N. Suppose fy≤L1, Cos⁡1,1-α⁡(x)≤L2 on the interval (δ,1) for 0<δ<1. (From Figure 10, we see that the function Cos⁡1,1-α⁡(x) is bounded when 0≤α≤1.) Then the neural network is convergent on the interval (δ,1) when (21)0<μ<N+14MNL1+δ-αL22.Proof.
LetWj=(w1j,w2j,…,wMj), and then we denote yj(X) by (22)yjX=Wj·G-C-Wj·I1cos⁡⁡M+1X,where G=(cos⁡⁡(X),cos⁡⁡(2X),…,cos⁡⁡(MX))T and I1=(1,1,…,1︷M)T. Then according to (17), we have (23)Ej=ej1,ej2,…,ejN=fX,yjX-D0+αyjX=fX,yjX-XαWjH+C-WjI1Cos⁡1,1-α⁡M+1X,where H=(Cos⁡1,1-α⁡(X),Cos⁡1,1-α⁡(2X),…,Cos⁡1,1-α⁡(MX))T. Then we have (24)∂Ej∂Wj=fyG-I1cos⁡⁡M+1X-XαH-I1Cos⁡1,1-α⁡M+1X.Noting J=(1/2)Ej22, then we get (25)ΔWj=-μ∂J∂Ej∂Ej∂Wj=-μEj∂Ej∂Wj.Define Lyapunov function Vj=(1/2)Ej22; we have (26)ΔVj=12Ej+122-12Ej22.Suppose (27)Ej+1=Ej+ΔEj=Ej+∂Ej∂WjTΔWj,and then in accordance with (25) that yields (28)Ej+1=I-μ∂Ej∂WjT∂Ej∂WjEj,where(29)I=1⋱1N×N.Thus, (30)ΔVj=12I-μ∂Ej∂WjT∂Ej∂WjEj22-12Ej22≤12I-μ∂Ej∂WjT∂Ej∂WjF2-1Ej22,where ·F is Frobenius matrix norm, defined by (aij)n×nF=(∑i=1n∑j=1naij2)1/2. Since Ej22>0, in order to make this neural network converge, we have (31)I-μ∂Ej∂WjT∂Ej∂WjF2<1,which yields (32)I-μ∂Ej∂WjT∂Ej∂WjF<1.Hence, (33)1>I-μ∂Ej∂WjT∂Ej∂WjF≥μ∂Ej∂WjF2-IF=μ∂Ej∂WjF2-N.In accordance with 0<μ<1, we obtain (34)0<μ<N+1∂Ej/∂WjF2.By calculating (25), we get (35)∂Ej∂WjF2≤2L1+δ-αL2⋯2L1+δ-αL2⋮⋮2L1+δ-αL2⋯2L1+δ-αL2M×NF2=4MNL1+δ-αL22.Finally, we have (36)0<μ<N+14MNL1+δ-αL22.


### 3.3. Example

#### 3.3.1. Example  1

We first consider the following linear fractional differential equation:(37)D0+αyx=x2+2Γ3-αx2-α-yx,with condition y(0)=0. The exact solution is y(x)=x2. This equation also can be solved by the following methods: Genetic Algorithm (GA) [21], Grünwald-Letnikov classical numerical technique (GL) [23], and Particle Swarm Optimization (PSO) algorithm [23]. We set the parameters μ=0.001, M=7, and N=10 and train the neural network 4500 times, and the weights of the network for Example 1 are given in Table 1. Figure 1 shows that sample points are on the exact solution curve after training is completed. Then we check whether the other points also match well with the exact solution (see Figure 2). From Figure 3 we see the error values decrease rapidly. Tables 2(a) and 2(b) show the numerical solutions and accuracy for Example 1 by the different methods. In this paper, all numerical experiments are done by using Lenovo T400, Intel Core 2 Duo CPU P8700, 2.53 GHz, and Matlab version R2010b. The neural networks with cosine basis functions have taken about 850 s, but the other algorithms mentioned above need to run about 2,240 s.Table 1
Weights (×10-4) obtained along with the solution of Examples 1,2, and 3.
α
Example1
Example2
Example3
1
0.7
0.5
1
0.7
0.5
1
0.7
0.5
w
1
4857
6310
6665
8226
9941
8880
5415
4070
5101
w
2
-
0506
-
3466
-
4424
4080
0219
2914
-
1589
0680
-
1582
w
3
-
4434
-
2721
-
1362
-
3468
-
0621
-
3593
-
5771
-
5800
-
3972
w
4
-
3170
-
3110
-
4990
1680
1372
3715
-
1309
-
2908
-
4142
w
5
1896
4204
5290
-
1455
-
1468
-
3147
3295
3947
6096
w
6
5534
-
0926
0222
1895
0482
1372
2815
2901
-
0121
w
7
-
6316
0182
-
2151
-
1350
0424
0273
-
4609
-
4203
-
2015Table 2
(a) Comparison of results for the solution of Example1 for α=0.5. (b) Comparison of results for the solution of Example 1 for α=0.75.
(a)
x
y
(
x
)
Numerical solution
Accuracy
GL
PSO
GA
NU
GL
PSO
GA
NU
0.1
0.01
0.0101
1
0
-
4
0.2
0.04
0.0401
0.0404
0.0396
0.0407
1
0
-
4
1
0
-
4
1
0
-
4
1
0
-
4
0.3
0.09
0.0901
0.0907
0.0917
1
0
-
4
1
0
-
4
1
0
-
3
0.4
0.16
0.1601
0.1604
0.1596
0.1621
1
0
-
4
1
0
-
4
1
0
-
4
1
0
-
3
0.5
0.25
0.2501
0.2496
0.2505
1
0
-
4
1
0
-
4
1
0
-
4
0.6
0.36
0.3602
0.3583
0.3573
0.3571
1
0
-
3
1
0
-
3
1
0
-
3
1
0
-
3
0.7
0.49
0.4902
0.4869
0.4853
1
0
-
3
1
0
-
3
1
0
-
3
0.8
0.64
0.6402
0.6362
0.6352
0.6397
1
0
-
3
1
0
-
3
1
0
-
3
1
0
-
4
0.9
0.81
0.8102
0.8069
0.8186
1
0
-
3
1
0
-
3
1
0
-
3
1
1
0.1001
0.1000
0.1004
0.1003
1
0
-
4
1
0
-
5
1
0
-
4
1
0
-
3
(b)
x
y
(
x
)
Numerical solution
Accuracy
GL
PSO
NU
GL
PSO
NU
0.1
0.01
0.0107
0.0103
0.0092
1
0
-
4
1
0
-
4
1
0
-
4
0.2
0.04
0.0413
0.0414
0.0377
1
0
-
3
1
0
-
3
1
0
-
3
0.3
0.09
0.0918
0.0928
0.0875
1
0
-
3
1
0
-
3
1
0
-
3
0.4
0.16
0.1622
0.1636
0.1592
1
0
-
3
1
0
-
3
1
0
-
4
0.5
0.25
0.2527
0.2538
0.2511
1
0
-
3
1
0
-
3
1
0
-
3
0.6
0.36
0.3631
0.3631
0.3609
1
0
-
3
1
0
-
3
1
0
-
4
0.7
0.49
0.4934
0.4918
0.4884
1
0
-
3
1
0
-
3
1
0
-
3
0.8
0.64
0.6438
0.6402
0.6373
1
0
-
3
1
0
-
4
1
0
-
3
0.9
0.81
0.8141
0.8091
0.8106
1
0
-
3
1
0
-
4
1
0
-
4
1
1
1.0044
0.9991
1.0020
1
0
-
3
1
0
-
4
1
0
-
3Figure 1
The learning curve for Example 1.Figure 2
The inspection curve for Example 1.Figure 3
The error curve for Example 1.


#### 3.3.2. Example  2

We secondly consider the following linear fractional differential equation:(38)D0+αyx=cos⁡⁡x+x-αCos⁡1,1-α⁡x-yx,with condition y(0)=1. The exact solution is y(x)=cos⁡(x). We set the parameters μ=0.001, M=7, and N=10 and train the neural network 1000 times, and the weights of the network for Example 2 are given in Table 1. Figures 4, 5, and 6 show that the neural network is still applicable when C≠0. Table 3 shows the exact solution, approximate solution, and accuracy for Example 2.Table 3
Exact solution, approximate solution, and accuracy for Example2.
α
Numerical solution
Accuracy
x
cos
⁡
(
x
)
1
0.7
0.5
1
0.7
0.5
0.1
0.9950
0.9945
0.9967
0.9972
1
0
-
4
1
0
-
3
1
0
-
3
0.2
0.9800
0.9788
0.9852
0.9867
1
0
-
3
1
0
-
3
1
0
-
3
0.3
0.9553
0.9538
0.9620
0.9638
1
0
-
3
1
0
-
3
1
0
-
3
0.4
0.9210
0.9203
0.9249
0.9256
1
0
-
4
1
0
-
3
1
0
-
3
0.5
0.8775
0.8777
0.8758
0.8747
1
0
-
4
1
0
-
3
1
0
-
3
0.6
0.8253
0.8254
0.8196
0.8176
1
0
-
4
1
0
-
3
1
0
-
3
0.7
0.7648
0.7639
0.7607
0.7601
1
0
-
4
1
0
-
3
1
0
-
3
0.8
0.6967
0.6951
0.6990
0.7016
1
0
-
3
1
0
-
3
1
0
-
3
0.9
0.6216
0.6213
0.6286
0.6328
1
0
-
4
1
0
-
3
1
0
-
2
1
0.5403
0.5414
0.5414
0.5405
1
0
-
3
1
0
-
3
1
0
-
4Figure 4
The learning curve for Example 2.Figure 5
The inspection curve for Example 2.Figure 6
The error curve for Example 2.


#### 3.3.3. Example  3

We thirdly consider the following nonlinear fractional differential equation:(39)D0+αyx=x6+Γ3.5Γ3.5-αx2.5-α-xy2x,with condition y(0)=0. The exact solution is y(x)=x5/2. We set the parameters μ=0.001, M=7, and N=10 and train the neural network 1000 times, and the weights of the network for Example 2 are given in Table 1. Table 4 shows the exact solution, approximate solution, and accuracy for Example 3.Table 4
Exact solution, approximate solution, and accuracy for Example3.
α
Numerical solution
Accuracy
x
x
5
/
2
1
0.7
0.5
1
0.7
0.5
0.1
0.0031
0.0022
0.0055
0.0066
1
0
-
4
1
0
-
3
1
0
-
3
0.2
0.0178
0.0133
0.0234
0.0266
1
0
-
3
1
0
-
3
1
0
-
3
0.3
0.0492
0.0426
0.0566
0.0603
1
0
-
3
1
0
-
3
1
0
-
3
0.4
0.1011
0.0972
0.1075
0.1093
1
0
-
3
1
0
-
3
1
0
-
3
0.5
0.1767
0.1773
0.1783
0.1772
1
0
-
4
1
0
-
3
1
0
-
4
0.6
0.2788
0.2797
0.2733
0.2711
1
0
-
4
1
0
-
3
1
0
-
3
0.7
0.4099
0.4055
0.4010
0.4009
1
0
-
3
1
0
-
3
1
0
-
3
0.8
0.5724
0.5643
0.5712
0.5738
1
0
-
3
1
0
-
3
1
0
-
3
0.9
0.7684
0.7670
0.7832
0.7847
1
0
-
3
1
0
-
2
1
0
-
2
1
1
1.0064
1.0105
1.0056
1
0
-
3
1
0
-
2
1
0
-
3


### 3.4. The Second Neural Network

To describe the method, we consider (3) with initial conditions y(0)=C1 and z(0)=C2. The jth trial solutions for the problem are written as (40)yjx=∑i=1Mwi,jcos⁡⁡ix+C1-∑i=1Mwi,jcos⁡⁡M+1x,zjx=∑i=1Mqi,jcos⁡⁡ix+C2-∑i=1Mqi,jcos⁡⁡M+1x,where N represents the number of sample points and wi,j and qi,j are unknown weights of the network determined in training procedures to reduce error function: (41)Jy+z=12Ejy+z22=12∑k=1Nejyk2+12∑k=1Nejzk2,where (42)Ejy+z=ejy1,ejy2,…,ejyN,ejz1,ejz2,…,ejzNT,ejyk=fxk,yjxk,zjxk-D0+αyjxk=fxk,yjxk,zjxk-x-α∑i=1Mwi,jCos⁡1,1-α⁡ixk+C1-∑i=1Mwi,j·Cos⁡1,1-α⁡M+1xk,ejzk=gxk,yjxk,zjxk-D0+αzjxk=gxk,yjxk,zjxk-x-α∑i=1Mqi,jCos⁡1,1-α⁡ixk+C2-∑i=1Mqi,j·Cos⁡1,1-α⁡M+1xk.Then we adjust the weights wi,j and qi,j by the following two equations: (43)wi,j+1=wi,j+Δwi,j,qi,j+1=qi,j+Δqi,j,where(44)Δwi,j=-μ∂Jy+z∂wi,j=-μ∑k=1N∂Jy+z∂ejyk∂ejyk∂wi,j+∑k=1N∂Jy+z∂ejzk∂ejzk∂wi,j=-μ∑k=1Nejyk·fyxk,yjxk,zjxk·cos⁡⁡xk-cos⁡⁡M+1xk-xk-α·Cos⁡1,1-αxk-Cos⁡1,1-αM+1xk-μ∑k=1Nejzkgyxk,yjxk,zjxk·cos⁡⁡xk-cos⁡⁡M+1xk,Δqi,j=-μ∂Jy+z∂qi,j=-μ∑k=1N∂Jy+z∂ejzk∂ejzk∂qi,j+∑k=1N∂Jy+z∂ejyk∂ejyk∂qi,j=-μ∑k=1Nejzk·gzxk,yjxk,zjxk·cos⁡⁡xk-cos⁡⁡M+1xk-xk-α·Cos⁡1,1-αxk-Cos⁡1,1-α⁡M+1xk-μ∑k=1Nejykfzxk,yjxk,zjxkcos⁡⁡xk-cos⁡⁡M+1xk.


### 3.5. Convergence of the Algorithm

Theorem B.
Letμ represent learning rate, let N represent the number of sample points, and let M represent the number of neurons: X=(x1,x2,…,xN), δ<xi<1, 1≤i≤N. Suppose fy≤L1y, fz≤L1z, gy≤L3y, gz≤L3z, Cos⁡1,1-α⁡(x)≤L2 on the interval (δ,1) for 0<δ<1. Then the neural network is convergent on the interval (δ,1) when (45)0<μ<2N+14MNL1y+L3y+δ-αL22+L1z+L3z+δ-αL22.Proof.
LetWjy+z=(w1j,w2j,…,wMj,q1j,q2j,…,qMj), and then we denote yj(X) and zj(X) by (46)yjX=Wjy+zG1,0+C1-Wjy+zI11,0cos⁡⁡M+1X,(47)zjX=Wjy+zG0,1+C2-Wjy+zI10,1cos⁡⁡M+1X,respectively, where (48)Ejy+z=ejy1,ejy2,…,ejyN,ejz1,ejz2,…,ejzN=fX,yjX,zjX-D0+αyjX,gX,yjX,zjX-D0+αzjX=fX,yjX,zjX-XαWjy+zH1,0+C1-Wjy+zI11,0Cos⁡1,1-α⁡M+1X+gX,yjX,zjX-XαWjy+zH0,1+C2-Wjy+zI10,1Cos⁡1,1-α⁡M+1X,H1,0=Cos⁡1,1-α⁡X,Cos⁡1,1-α⁡2X,…,Cos⁡1,1-α⁡MX,0,0,…,0︷MT,H0,1=0,0,…,0︷M,Cos⁡1,1-α⁡X,Cos⁡1,1-α⁡2X,…,Cos⁡1,1-α⁡MXT,G1,0=cos⁡⁡X,cos⁡⁡2X,…,cos⁡⁡MX,0,0,…,0︷MT,G0,1=0,0,…,0︷M,cos⁡⁡X,cos⁡⁡2X,…,cos⁡⁡MXT,I11,0=1,1,…,1︷M,0,0,…,0︷MT,I10,1=0,0,…,0︷M,1,1,…,1︷MT.Then we have (49)∂Ejy+z∂Wjy+z=fyG1,0-I11,0cos⁡⁡M+1X+fzG0,1-I10,1cos⁡⁡M+1X-XαH1,0-I11,0Cos⁡1,1-α⁡M+1X+gzG0,1-I10,1cos⁡⁡M+1X+gyG1,0-I11,0cos⁡⁡M+1X-XαH0,1-I10,1Cos⁡1,1-α⁡M+1X.Define Lyapunov function Vjy+z=(1/2)Ejy+z22; then similarly to the proof of Theorem A we get (50)0<μ<2N+1∂Ejy+z/∂Wjy+zF2.By simply calculating ∂Ejy+z/∂Wjy+z, we have (51)∂Ejy+z∂Wjy+zF2≤2A⋯AB⋯B⋮⋮⋮⋮A⋯AB⋯BM×2NF2=4MNA2+B2,where A=L1y+L3y+δ-αL2 and B=L1z+L3z+δ-αL2, and finally we obtain (52)0<μ<2N+14MNL1y+L3y+δ-αL22+L1z+L3z+δ-αL22.This completes the proof.


### 3.6. Example

#### 3.6.1. Example  4

We first consider the following linear coupled fractional differential equations:(53)D0+αyx=x2+x3+2Γ3-αx2-α-yx-zx,0<x≤1,0<α≤1,D0+αzx=x2+x3+6Γ4-αx3-α-yx-zx,with initial condition as follows: (54)y0=0,z0=0.The exact solution is y(x)=x2 and z(x)=x3. We set the parameters α=0.9, μ=0.001, M=7, and N=10 and train the neural network 2000 times, and the weights of the network for Example 4 are given in Table 5. Figures 7 and 8 show that the sample points and checkpoints are in well agreement with the exact solutions for the problem. Figure 9 shows that the error of the numerical solutions decreases rapidly within the first 50 training times. Table 6 shows the exact solution, approximate solution, and accuracy for Example 4.Table 5
Weights obtained along with the solution of Examples4 and 5.
α
=
0.9
Example4
Example5
w
1
/
q
1
0.5307
/
0.4494
0.8215
/
0.7682
w
2
/
q
2
-
0.0592
/
0.0013
0.3545
/
-
0.7084
w
3
/
q
3
-
0.6414
/
-
0.7451
-
0.1512
/
-
0.2292
w
4
/
q
4
0.0052
/
-
0.0392
-
0.1342
/
-
0.1116
w
5
/
q
5
0.0559
/
0.3705
0.1657
/
0.3582
w
6
/
q
6
0.3998
/
0.2282
-
0.0815
/
0.0781
w
7
/
q
7
-
0.4141
/
-
0.4014
0.0533
/
-
0.2259Table 6
Exact solution, approximate solution, and accuracy for Example4.
α
=
0.9
Numerical solution
Accuracy
x
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
0.1
0.01
0.001
0.0101
0.0020
1
0
-
4
1
0
-
3
0.2
0.04
0.008
0.0408
0.0104
1
0
-
4
1
0
-
3
0.3
0.09
0.027
0.0926
0.0301
1
0
-
3
1
0
-
3
0.4
0.16
0.064
0.1641
0.0664
1
0
-
3
1
0
-
3
0.5
0.25
0.125
0.2529
0.1233
1
0
-
3
1
0
-
3
0.6
0.36
0.216
0.3584
0.2071
1
0
-
3
1
0
-
3
0.7
0.49
0.343
0.4847
0.3290
1
0
-
3
1
0
-
2
0.8
0.64
0.512
0.6389
0.5033
1
0
-
3
1
0
-
3
0.9
0.81
0.729
0.8215
0.7361
1
0
-
2
1
0
-
3
1
1
1
1.0126
1.0076
1
0
-
2
1
0
-
3Figure 7
The learning curve for Example 4.Figure 8
The inspection curve for Example 4.Figure 9
The error curve for Example 4.Figure 10
The graphics of the functionCosα,1-α(x).


#### 3.6.2. Example  5

We second consider the following nonlinear fractional coupled differential equations:(55)D0+αyx=x6+cos⁡⁡x+6Γ4-αx3-α-yx2-zx,0<x≤1,0<α≤1,D0+αzx=x3+cos⁡⁡x+x-αCos⁡1,1-αx-yx-zx,with initial conditions as follows: (56)y0=0,z0=1,The exact solution is y(x)=x3 and z(x)=cos⁡⁡(x). We set the parameters α=0.9, μ=0.001, M=7, and N=10. Numerical solutions in Table 7 show that this network can also be applied to the nonlinear fractional coupled differential equations but we need more time to train the network.Table 7
Exact solution, approximate solution, and accuracy for Example5.
α
=
0.9
Numerical solution
Accuracy
x
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
0.1
0.9950
0.001
0.9926
0.0042
1
0
-
3
1
0
-
3
0.2
0.9800
0.008
0.9834
0.0148
1
0
-
3
1
0
-
3
0.3
0.9553
0.027
0.9625
0.0351
1
0
-
3
1
0
-
3
0.4
0.9210
0.064
0.9267
0.0689
1
0
-
3
1
0
-
3
0.5
0.8775
0.125
0.8773
0.1220
1
0
-
4
1
0
-
3
0.6
0.8253
0.216
0.8198
0.2040
1
0
-
3
1
0
-
3
0.7
0.7648
0.343
0.7604
0.3276
1
0
-
3
1
0
-
2
0.8
0.6967
0.512
0.7004
0.5049
1
0
-
3
1
0
-
3
0.9
0.6216
0.729
0.6336
0.7375
1
0
-
2
1
0
-
3
1
0.5403
1
0.5475
1.0056
1
0
-
3
1
0
-
3


## 3.1. The First Neural Network

To describe the method, we consider (1) with initial condition y(0)=C. The jth trial solution satisfying the initial condition is written as (16)yjx=∑i=1Mwi,jcos⁡⁡ix+C-∑i=1Mwi,jcos⁡⁡M+1x,where M represents the number of neurons and wi,j are unknown weights of the network determined in training procedures to reduce the error function: (17)J=12Ej22=12∑k=1Nejk2,Ej=ej1,ej2,…,ejNT,where N represents the number of sample points, ·2 is Euclidean norm, and (18)ejk=fxk,yjxk-D0+αyjxk=fxk,yjxk-xk-α∑i=1Mwi,jCos⁡1,1-α⁡ixk+C-∑i=1Mwi,jCos⁡1,1-α⁡M+1xk,where k=1,2,…,N; then we can adjust the weights wi,j by the following equation: (19)wi,j+1=wi,j+Δwi,j,where(20)Δwi,j=-μ∂J∂wi,j=-μ∑k=1N∂J∂ejk∂ejk∂wi,j=-μ∑k=1Nejkfyxk,yjxk·cos⁡⁡xk-cos⁡⁡M+1xk-xk-αejk·Cos⁡1,1-αxk-Cos⁡1,1-αM+1xk.


## 3.2. Convergence of the Algorithm

Theorem A.
Letμ represent learning rate, let N represent the number of sample points, and let M represent the number of neurons: X=(x1,x2,…,xN),  δ<xi<1, 1≤i≤N. Suppose fy≤L1, Cos⁡1,1-α⁡(x)≤L2 on the interval (δ,1) for 0<δ<1. (From Figure 10, we see that the function Cos⁡1,1-α⁡(x) is bounded when 0≤α≤1.) Then the neural network is convergent on the interval (δ,1) when (21)0<μ<N+14MNL1+δ-αL22.Proof.
LetWj=(w1j,w2j,…,wMj), and then we denote yj(X) by (22)yjX=Wj·G-C-Wj·I1cos⁡⁡M+1X,where G=(cos⁡⁡(X),cos⁡⁡(2X),…,cos⁡⁡(MX))T and I1=(1,1,…,1︷M)T. Then according to (17), we have (23)Ej=ej1,ej2,…,ejN=fX,yjX-D0+αyjX=fX,yjX-XαWjH+C-WjI1Cos⁡1,1-α⁡M+1X,where H=(Cos⁡1,1-α⁡(X),Cos⁡1,1-α⁡(2X),…,Cos⁡1,1-α⁡(MX))T. Then we have (24)∂Ej∂Wj=fyG-I1cos⁡⁡M+1X-XαH-I1Cos⁡1,1-α⁡M+1X.Noting J=(1/2)Ej22, then we get (25)ΔWj=-μ∂J∂Ej∂Ej∂Wj=-μEj∂Ej∂Wj.Define Lyapunov function Vj=(1/2)Ej22; we have (26)ΔVj=12Ej+122-12Ej22.Suppose (27)Ej+1=Ej+ΔEj=Ej+∂Ej∂WjTΔWj,and then in accordance with (25) that yields (28)Ej+1=I-μ∂Ej∂WjT∂Ej∂WjEj,where(29)I=1⋱1N×N.Thus, (30)ΔVj=12I-μ∂Ej∂WjT∂Ej∂WjEj22-12Ej22≤12I-μ∂Ej∂WjT∂Ej∂WjF2-1Ej22,where ·F is Frobenius matrix norm, defined by (aij)n×nF=(∑i=1n∑j=1naij2)1/2. Since Ej22>0, in order to make this neural network converge, we have (31)I-μ∂Ej∂WjT∂Ej∂WjF2<1,which yields (32)I-μ∂Ej∂WjT∂Ej∂WjF<1.Hence, (33)1>I-μ∂Ej∂WjT∂Ej∂WjF≥μ∂Ej∂WjF2-IF=μ∂Ej∂WjF2-N.In accordance with 0<μ<1, we obtain (34)0<μ<N+1∂Ej/∂WjF2.By calculating (25), we get (35)∂Ej∂WjF2≤2L1+δ-αL2⋯2L1+δ-αL2⋮⋮2L1+δ-αL2⋯2L1+δ-αL2M×NF2=4MNL1+δ-αL22.Finally, we have (36)0<μ<N+14MNL1+δ-αL22.


## 3.3. Example

### 3.3.1. Example  1

We first consider the following linear fractional differential equation:(37)D0+αyx=x2+2Γ3-αx2-α-yx,with condition y(0)=0. The exact solution is y(x)=x2. This equation also can be solved by the following methods: Genetic Algorithm (GA) [21], Grünwald-Letnikov classical numerical technique (GL) [23], and Particle Swarm Optimization (PSO) algorithm [23]. We set the parameters μ=0.001, M=7, and N=10 and train the neural network 4500 times, and the weights of the network for Example 1 are given in Table 1. Figure 1 shows that sample points are on the exact solution curve after training is completed. Then we check whether the other points also match well with the exact solution (see Figure 2). From Figure 3 we see the error values decrease rapidly. Tables 2(a) and 2(b) show the numerical solutions and accuracy for Example 1 by the different methods. In this paper, all numerical experiments are done by using Lenovo T400, Intel Core 2 Duo CPU P8700, 2.53 GHz, and Matlab version R2010b. The neural networks with cosine basis functions have taken about 850 s, but the other algorithms mentioned above need to run about 2,240 s.Table 1
Weights (×10-4) obtained along with the solution of Examples 1,2, and 3.
α
Example1
Example2
Example3
1
0.7
0.5
1
0.7
0.5
1
0.7
0.5
w
1
4857
6310
6665
8226
9941
8880
5415
4070
5101
w
2
-
0506
-
3466
-
4424
4080
0219
2914
-
1589
0680
-
1582
w
3
-
4434
-
2721
-
1362
-
3468
-
0621
-
3593
-
5771
-
5800
-
3972
w
4
-
3170
-
3110
-
4990
1680
1372
3715
-
1309
-
2908
-
4142
w
5
1896
4204
5290
-
1455
-
1468
-
3147
3295
3947
6096
w
6
5534
-
0926
0222
1895
0482
1372
2815
2901
-
0121
w
7
-
6316
0182
-
2151
-
1350
0424
0273
-
4609
-
4203
-
2015Table 2
(a) Comparison of results for the solution of Example1 for α=0.5. (b) Comparison of results for the solution of Example 1 for α=0.75.
(a)
x
y
(
x
)
Numerical solution
Accuracy
GL
PSO
GA
NU
GL
PSO
GA
NU
0.1
0.01
0.0101
1
0
-
4
0.2
0.04
0.0401
0.0404
0.0396
0.0407
1
0
-
4
1
0
-
4
1
0
-
4
1
0
-
4
0.3
0.09
0.0901
0.0907
0.0917
1
0
-
4
1
0
-
4
1
0
-
3
0.4
0.16
0.1601
0.1604
0.1596
0.1621
1
0
-
4
1
0
-
4
1
0
-
4
1
0
-
3
0.5
0.25
0.2501
0.2496
0.2505
1
0
-
4
1
0
-
4
1
0
-
4
0.6
0.36
0.3602
0.3583
0.3573
0.3571
1
0
-
3
1
0
-
3
1
0
-
3
1
0
-
3
0.7
0.49
0.4902
0.4869
0.4853
1
0
-
3
1
0
-
3
1
0
-
3
0.8
0.64
0.6402
0.6362
0.6352
0.6397
1
0
-
3
1
0
-
3
1
0
-
3
1
0
-
4
0.9
0.81
0.8102
0.8069
0.8186
1
0
-
3
1
0
-
3
1
0
-
3
1
1
0.1001
0.1000
0.1004
0.1003
1
0
-
4
1
0
-
5
1
0
-
4
1
0
-
3
(b)
x
y
(
x
)
Numerical solution
Accuracy
GL
PSO
NU
GL
PSO
NU
0.1
0.01
0.0107
0.0103
0.0092
1
0
-
4
1
0
-
4
1
0
-
4
0.2
0.04
0.0413
0.0414
0.0377
1
0
-
3
1
0
-
3
1
0
-
3
0.3
0.09
0.0918
0.0928
0.0875
1
0
-
3
1
0
-
3
1
0
-
3
0.4
0.16
0.1622
0.1636
0.1592
1
0
-
3
1
0
-
3
1
0
-
4
0.5
0.25
0.2527
0.2538
0.2511
1
0
-
3
1
0
-
3
1
0
-
3
0.6
0.36
0.3631
0.3631
0.3609
1
0
-
3
1
0
-
3
1
0
-
4
0.7
0.49
0.4934
0.4918
0.4884
1
0
-
3
1
0
-
3
1
0
-
3
0.8
0.64
0.6438
0.6402
0.6373
1
0
-
3
1
0
-
4
1
0
-
3
0.9
0.81
0.8141
0.8091
0.8106
1
0
-
3
1
0
-
4
1
0
-
4
1
1
1.0044
0.9991
1.0020
1
0
-
3
1
0
-
4
1
0
-
3Figure 1
The learning curve for Example 1.Figure 2
The inspection curve for Example 1.Figure 3
The error curve for Example 1.


### 3.3.2. Example  2

We secondly consider the following linear fractional differential equation:(38)D0+αyx=cos⁡⁡x+x-αCos⁡1,1-α⁡x-yx,with condition y(0)=1. The exact solution is y(x)=cos⁡(x). We set the parameters μ=0.001, M=7, and N=10 and train the neural network 1000 times, and the weights of the network for Example 2 are given in Table 1. Figures 4, 5, and 6 show that the neural network is still applicable when C≠0. Table 3 shows the exact solution, approximate solution, and accuracy for Example 2.Table 3
Exact solution, approximate solution, and accuracy for Example2.
α
Numerical solution
Accuracy
x
cos
⁡
(
x
)
1
0.7
0.5
1
0.7
0.5
0.1
0.9950
0.9945
0.9967
0.9972
1
0
-
4
1
0
-
3
1
0
-
3
0.2
0.9800
0.9788
0.9852
0.9867
1
0
-
3
1
0
-
3
1
0
-
3
0.3
0.9553
0.9538
0.9620
0.9638
1
0
-
3
1
0
-
3
1
0
-
3
0.4
0.9210
0.9203
0.9249
0.9256
1
0
-
4
1
0
-
3
1
0
-
3
0.5
0.8775
0.8777
0.8758
0.8747
1
0
-
4
1
0
-
3
1
0
-
3
0.6
0.8253
0.8254
0.8196
0.8176
1
0
-
4
1
0
-
3
1
0
-
3
0.7
0.7648
0.7639
0.7607
0.7601
1
0
-
4
1
0
-
3
1
0
-
3
0.8
0.6967
0.6951
0.6990
0.7016
1
0
-
3
1
0
-
3
1
0
-
3
0.9
0.6216
0.6213
0.6286
0.6328
1
0
-
4
1
0
-
3
1
0
-
2
1
0.5403
0.5414
0.5414
0.5405
1
0
-
3
1
0
-
3
1
0
-
4Figure 4
The learning curve for Example 2.Figure 5
The inspection curve for Example 2.Figure 6
The error curve for Example 2.


### 3.3.3. Example  3

We thirdly consider the following nonlinear fractional differential equation:(39)D0+αyx=x6+Γ3.5Γ3.5-αx2.5-α-xy2x,with condition y(0)=0. The exact solution is y(x)=x5/2. We set the parameters μ=0.001, M=7, and N=10 and train the neural network 1000 times, and the weights of the network for Example 2 are given in Table 1. Table 4 shows the exact solution, approximate solution, and accuracy for Example 3.Table 4
Exact solution, approximate solution, and accuracy for Example3.
α
Numerical solution
Accuracy
x
x
5
/
2
1
0.7
0.5
1
0.7
0.5
0.1
0.0031
0.0022
0.0055
0.0066
1
0
-
4
1
0
-
3
1
0
-
3
0.2
0.0178
0.0133
0.0234
0.0266
1
0
-
3
1
0
-
3
1
0
-
3
0.3
0.0492
0.0426
0.0566
0.0603
1
0
-
3
1
0
-
3
1
0
-
3
0.4
0.1011
0.0972
0.1075
0.1093
1
0
-
3
1
0
-
3
1
0
-
3
0.5
0.1767
0.1773
0.1783
0.1772
1
0
-
4
1
0
-
3
1
0
-
4
0.6
0.2788
0.2797
0.2733
0.2711
1
0
-
4
1
0
-
3
1
0
-
3
0.7
0.4099
0.4055
0.4010
0.4009
1
0
-
3
1
0
-
3
1
0
-
3
0.8
0.5724
0.5643
0.5712
0.5738
1
0
-
3
1
0
-
3
1
0
-
3
0.9
0.7684
0.7670
0.7832
0.7847
1
0
-
3
1
0
-
2
1
0
-
2
1
1
1.0064
1.0105
1.0056
1
0
-
3
1
0
-
2
1
0
-
3


## 3.3.1. Example  1

We first consider the following linear fractional differential equation:(37)D0+αyx=x2+2Γ3-αx2-α-yx,with condition y(0)=0. The exact solution is y(x)=x2. This equation also can be solved by the following methods: Genetic Algorithm (GA) [21], Grünwald-Letnikov classical numerical technique (GL) [23], and Particle Swarm Optimization (PSO) algorithm [23]. We set the parameters μ=0.001, M=7, and N=10 and train the neural network 4500 times, and the weights of the network for Example 1 are given in Table 1. Figure 1 shows that sample points are on the exact solution curve after training is completed. Then we check whether the other points also match well with the exact solution (see Figure 2). From Figure 3 we see the error values decrease rapidly. Tables 2(a) and 2(b) show the numerical solutions and accuracy for Example 1 by the different methods. In this paper, all numerical experiments are done by using Lenovo T400, Intel Core 2 Duo CPU P8700, 2.53 GHz, and Matlab version R2010b. The neural networks with cosine basis functions have taken about 850 s, but the other algorithms mentioned above need to run about 2,240 s.Table 1
Weights (×10-4) obtained along with the solution of Examples 1,2, and 3.
α
Example1
Example2
Example3
1
0.7
0.5
1
0.7
0.5
1
0.7
0.5
w
1
4857
6310
6665
8226
9941
8880
5415
4070
5101
w
2
-
0506
-
3466
-
4424
4080
0219
2914
-
1589
0680
-
1582
w
3
-
4434
-
2721
-
1362
-
3468
-
0621
-
3593
-
5771
-
5800
-
3972
w
4
-
3170
-
3110
-
4990
1680
1372
3715
-
1309
-
2908
-
4142
w
5
1896
4204
5290
-
1455
-
1468
-
3147
3295
3947
6096
w
6
5534
-
0926
0222
1895
0482
1372
2815
2901
-
0121
w
7
-
6316
0182
-
2151
-
1350
0424
0273
-
4609
-
4203
-
2015Table 2
(a) Comparison of results for the solution of Example1 for α=0.5. (b) Comparison of results for the solution of Example 1 for α=0.75.
(a)
x
y
(
x
)
Numerical solution
Accuracy
GL
PSO
GA
NU
GL
PSO
GA
NU
0.1
0.01
0.0101
1
0
-
4
0.2
0.04
0.0401
0.0404
0.0396
0.0407
1
0
-
4
1
0
-
4
1
0
-
4
1
0
-
4
0.3
0.09
0.0901
0.0907
0.0917
1
0
-
4
1
0
-
4
1
0
-
3
0.4
0.16
0.1601
0.1604
0.1596
0.1621
1
0
-
4
1
0
-
4
1
0
-
4
1
0
-
3
0.5
0.25
0.2501
0.2496
0.2505
1
0
-
4
1
0
-
4
1
0
-
4
0.6
0.36
0.3602
0.3583
0.3573
0.3571
1
0
-
3
1
0
-
3
1
0
-
3
1
0
-
3
0.7
0.49
0.4902
0.4869
0.4853
1
0
-
3
1
0
-
3
1
0
-
3
0.8
0.64
0.6402
0.6362
0.6352
0.6397
1
0
-
3
1
0
-
3
1
0
-
3
1
0
-
4
0.9
0.81
0.8102
0.8069
0.8186
1
0
-
3
1
0
-
3
1
0
-
3
1
1
0.1001
0.1000
0.1004
0.1003
1
0
-
4
1
0
-
5
1
0
-
4
1
0
-
3
(b)
x
y
(
x
)
Numerical solution
Accuracy
GL
PSO
NU
GL
PSO
NU
0.1
0.01
0.0107
0.0103
0.0092
1
0
-
4
1
0
-
4
1
0
-
4
0.2
0.04
0.0413
0.0414
0.0377
1
0
-
3
1
0
-
3
1
0
-
3
0.3
0.09
0.0918
0.0928
0.0875
1
0
-
3
1
0
-
3
1
0
-
3
0.4
0.16
0.1622
0.1636
0.1592
1
0
-
3
1
0
-
3
1
0
-
4
0.5
0.25
0.2527
0.2538
0.2511
1
0
-
3
1
0
-
3
1
0
-
3
0.6
0.36
0.3631
0.3631
0.3609
1
0
-
3
1
0
-
3
1
0
-
4
0.7
0.49
0.4934
0.4918
0.4884
1
0
-
3
1
0
-
3
1
0
-
3
0.8
0.64
0.6438
0.6402
0.6373
1
0
-
3
1
0
-
4
1
0
-
3
0.9
0.81
0.8141
0.8091
0.8106
1
0
-
3
1
0
-
4
1
0
-
4
1
1
1.0044
0.9991
1.0020
1
0
-
3
1
0
-
4
1
0
-
3Figure 1
The learning curve for Example 1.Figure 2
The inspection curve for Example 1.Figure 3
The error curve for Example 1.


## 3.3.2. Example  2

We secondly consider the following linear fractional differential equation:(38)D0+αyx=cos⁡⁡x+x-αCos⁡1,1-α⁡x-yx,with condition y(0)=1. The exact solution is y(x)=cos⁡(x). We set the parameters μ=0.001, M=7, and N=10 and train the neural network 1000 times, and the weights of the network for Example 2 are given in Table 1. Figures 4, 5, and 6 show that the neural network is still applicable when C≠0. Table 3 shows the exact solution, approximate solution, and accuracy for Example 2.Table 3
Exact solution, approximate solution, and accuracy for Example2.
α
Numerical solution
Accuracy
x
cos
⁡
(
x
)
1
0.7
0.5
1
0.7
0.5
0.1
0.9950
0.9945
0.9967
0.9972
1
0
-
4
1
0
-
3
1
0
-
3
0.2
0.9800
0.9788
0.9852
0.9867
1
0
-
3
1
0
-
3
1
0
-
3
0.3
0.9553
0.9538
0.9620
0.9638
1
0
-
3
1
0
-
3
1
0
-
3
0.4
0.9210
0.9203
0.9249
0.9256
1
0
-
4
1
0
-
3
1
0
-
3
0.5
0.8775
0.8777
0.8758
0.8747
1
0
-
4
1
0
-
3
1
0
-
3
0.6
0.8253
0.8254
0.8196
0.8176
1
0
-
4
1
0
-
3
1
0
-
3
0.7
0.7648
0.7639
0.7607
0.7601
1
0
-
4
1
0
-
3
1
0
-
3
0.8
0.6967
0.6951
0.6990
0.7016
1
0
-
3
1
0
-
3
1
0
-
3
0.9
0.6216
0.6213
0.6286
0.6328
1
0
-
4
1
0
-
3
1
0
-
2
1
0.5403
0.5414
0.5414
0.5405
1
0
-
3
1
0
-
3
1
0
-
4Figure 4
The learning curve for Example 2.Figure 5
The inspection curve for Example 2.Figure 6
The error curve for Example 2.


## 3.3.3. Example  3

We thirdly consider the following nonlinear fractional differential equation:(39)D0+αyx=x6+Γ3.5Γ3.5-αx2.5-α-xy2x,with condition y(0)=0. The exact solution is y(x)=x5/2. We set the parameters μ=0.001, M=7, and N=10 and train the neural network 1000 times, and the weights of the network for Example 2 are given in Table 1. Table 4 shows the exact solution, approximate solution, and accuracy for Example 3.Table 4
Exact solution, approximate solution, and accuracy for Example3.
α
Numerical solution
Accuracy
x
x
5
/
2
1
0.7
0.5
1
0.7
0.5
0.1
0.0031
0.0022
0.0055
0.0066
1
0
-
4
1
0
-
3
1
0
-
3
0.2
0.0178
0.0133
0.0234
0.0266
1
0
-
3
1
0
-
3
1
0
-
3
0.3
0.0492
0.0426
0.0566
0.0603
1
0
-
3
1
0
-
3
1
0
-
3
0.4
0.1011
0.0972
0.1075
0.1093
1
0
-
3
1
0
-
3
1
0
-
3
0.5
0.1767
0.1773
0.1783
0.1772
1
0
-
4
1
0
-
3
1
0
-
4
0.6
0.2788
0.2797
0.2733
0.2711
1
0
-
4
1
0
-
3
1
0
-
3
0.7
0.4099
0.4055
0.4010
0.4009
1
0
-
3
1
0
-
3
1
0
-
3
0.8
0.5724
0.5643
0.5712
0.5738
1
0
-
3
1
0
-
3
1
0
-
3
0.9
0.7684
0.7670
0.7832
0.7847
1
0
-
3
1
0
-
2
1
0
-
2
1
1
1.0064
1.0105
1.0056
1
0
-
3
1
0
-
2
1
0
-
3


## 3.4. The Second Neural Network

To describe the method, we consider (3) with initial conditions y(0)=C1 and z(0)=C2. The jth trial solutions for the problem are written as (40)yjx=∑i=1Mwi,jcos⁡⁡ix+C1-∑i=1Mwi,jcos⁡⁡M+1x,zjx=∑i=1Mqi,jcos⁡⁡ix+C2-∑i=1Mqi,jcos⁡⁡M+1x,where N represents the number of sample points and wi,j and qi,j are unknown weights of the network determined in training procedures to reduce error function: (41)Jy+z=12Ejy+z22=12∑k=1Nejyk2+12∑k=1Nejzk2,where (42)Ejy+z=ejy1,ejy2,…,ejyN,ejz1,ejz2,…,ejzNT,ejyk=fxk,yjxk,zjxk-D0+αyjxk=fxk,yjxk,zjxk-x-α∑i=1Mwi,jCos⁡1,1-α⁡ixk+C1-∑i=1Mwi,j·Cos⁡1,1-α⁡M+1xk,ejzk=gxk,yjxk,zjxk-D0+αzjxk=gxk,yjxk,zjxk-x-α∑i=1Mqi,jCos⁡1,1-α⁡ixk+C2-∑i=1Mqi,j·Cos⁡1,1-α⁡M+1xk.Then we adjust the weights wi,j and qi,j by the following two equations: (43)wi,j+1=wi,j+Δwi,j,qi,j+1=qi,j+Δqi,j,where(44)Δwi,j=-μ∂Jy+z∂wi,j=-μ∑k=1N∂Jy+z∂ejyk∂ejyk∂wi,j+∑k=1N∂Jy+z∂ejzk∂ejzk∂wi,j=-μ∑k=1Nejyk·fyxk,yjxk,zjxk·cos⁡⁡xk-cos⁡⁡M+1xk-xk-α·Cos⁡1,1-αxk-Cos⁡1,1-αM+1xk-μ∑k=1Nejzkgyxk,yjxk,zjxk·cos⁡⁡xk-cos⁡⁡M+1xk,Δqi,j=-μ∂Jy+z∂qi,j=-μ∑k=1N∂Jy+z∂ejzk∂ejzk∂qi,j+∑k=1N∂Jy+z∂ejyk∂ejyk∂qi,j=-μ∑k=1Nejzk·gzxk,yjxk,zjxk·cos⁡⁡xk-cos⁡⁡M+1xk-xk-α·Cos⁡1,1-αxk-Cos⁡1,1-α⁡M+1xk-μ∑k=1Nejykfzxk,yjxk,zjxkcos⁡⁡xk-cos⁡⁡M+1xk.


## 3.5. Convergence of the Algorithm

Theorem B.
Letμ represent learning rate, let N represent the number of sample points, and let M represent the number of neurons: X=(x1,x2,…,xN), δ<xi<1, 1≤i≤N. Suppose fy≤L1y, fz≤L1z, gy≤L3y, gz≤L3z, Cos⁡1,1-α⁡(x)≤L2 on the interval (δ,1) for 0<δ<1. Then the neural network is convergent on the interval (δ,1) when (45)0<μ<2N+14MNL1y+L3y+δ-αL22+L1z+L3z+δ-αL22.Proof.
LetWjy+z=(w1j,w2j,…,wMj,q1j,q2j,…,qMj), and then we denote yj(X) and zj(X) by (46)yjX=Wjy+zG1,0+C1-Wjy+zI11,0cos⁡⁡M+1X,(47)zjX=Wjy+zG0,1+C2-Wjy+zI10,1cos⁡⁡M+1X,respectively, where (48)Ejy+z=ejy1,ejy2,…,ejyN,ejz1,ejz2,…,ejzN=fX,yjX,zjX-D0+αyjX,gX,yjX,zjX-D0+αzjX=fX,yjX,zjX-XαWjy+zH1,0+C1-Wjy+zI11,0Cos⁡1,1-α⁡M+1X+gX,yjX,zjX-XαWjy+zH0,1+C2-Wjy+zI10,1Cos⁡1,1-α⁡M+1X,H1,0=Cos⁡1,1-α⁡X,Cos⁡1,1-α⁡2X,…,Cos⁡1,1-α⁡MX,0,0,…,0︷MT,H0,1=0,0,…,0︷M,Cos⁡1,1-α⁡X,Cos⁡1,1-α⁡2X,…,Cos⁡1,1-α⁡MXT,G1,0=cos⁡⁡X,cos⁡⁡2X,…,cos⁡⁡MX,0,0,…,0︷MT,G0,1=0,0,…,0︷M,cos⁡⁡X,cos⁡⁡2X,…,cos⁡⁡MXT,I11,0=1,1,…,1︷M,0,0,…,0︷MT,I10,1=0,0,…,0︷M,1,1,…,1︷MT.Then we have (49)∂Ejy+z∂Wjy+z=fyG1,0-I11,0cos⁡⁡M+1X+fzG0,1-I10,1cos⁡⁡M+1X-XαH1,0-I11,0Cos⁡1,1-α⁡M+1X+gzG0,1-I10,1cos⁡⁡M+1X+gyG1,0-I11,0cos⁡⁡M+1X-XαH0,1-I10,1Cos⁡1,1-α⁡M+1X.Define Lyapunov function Vjy+z=(1/2)Ejy+z22; then similarly to the proof of Theorem A we get (50)0<μ<2N+1∂Ejy+z/∂Wjy+zF2.By simply calculating ∂Ejy+z/∂Wjy+z, we have (51)∂Ejy+z∂Wjy+zF2≤2A⋯AB⋯B⋮⋮⋮⋮A⋯AB⋯BM×2NF2=4MNA2+B2,where A=L1y+L3y+δ-αL2 and B=L1z+L3z+δ-αL2, and finally we obtain (52)0<μ<2N+14MNL1y+L3y+δ-αL22+L1z+L3z+δ-αL22.This completes the proof.


## 3.6. Example

### 3.6.1. Example  4

We first consider the following linear coupled fractional differential equations:(53)D0+αyx=x2+x3+2Γ3-αx2-α-yx-zx,0<x≤1,0<α≤1,D0+αzx=x2+x3+6Γ4-αx3-α-yx-zx,with initial condition as follows: (54)y0=0,z0=0.The exact solution is y(x)=x2 and z(x)=x3. We set the parameters α=0.9, μ=0.001, M=7, and N=10 and train the neural network 2000 times, and the weights of the network for Example 4 are given in Table 5. Figures 7 and 8 show that the sample points and checkpoints are in well agreement with the exact solutions for the problem. Figure 9 shows that the error of the numerical solutions decreases rapidly within the first 50 training times. Table 6 shows the exact solution, approximate solution, and accuracy for Example 4.Table 5
Weights obtained along with the solution of Examples4 and 5.
α
=
0.9
Example4
Example5
w
1
/
q
1
0.5307
/
0.4494
0.8215
/
0.7682
w
2
/
q
2
-
0.0592
/
0.0013
0.3545
/
-
0.7084
w
3
/
q
3
-
0.6414
/
-
0.7451
-
0.1512
/
-
0.2292
w
4
/
q
4
0.0052
/
-
0.0392
-
0.1342
/
-
0.1116
w
5
/
q
5
0.0559
/
0.3705
0.1657
/
0.3582
w
6
/
q
6
0.3998
/
0.2282
-
0.0815
/
0.0781
w
7
/
q
7
-
0.4141
/
-
0.4014
0.0533
/
-
0.2259Table 6
Exact solution, approximate solution, and accuracy for Example4.
α
=
0.9
Numerical solution
Accuracy
x
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
0.1
0.01
0.001
0.0101
0.0020
1
0
-
4
1
0
-
3
0.2
0.04
0.008
0.0408
0.0104
1
0
-
4
1
0
-
3
0.3
0.09
0.027
0.0926
0.0301
1
0
-
3
1
0
-
3
0.4
0.16
0.064
0.1641
0.0664
1
0
-
3
1
0
-
3
0.5
0.25
0.125
0.2529
0.1233
1
0
-
3
1
0
-
3
0.6
0.36
0.216
0.3584
0.2071
1
0
-
3
1
0
-
3
0.7
0.49
0.343
0.4847
0.3290
1
0
-
3
1
0
-
2
0.8
0.64
0.512
0.6389
0.5033
1
0
-
3
1
0
-
3
0.9
0.81
0.729
0.8215
0.7361
1
0
-
2
1
0
-
3
1
1
1
1.0126
1.0076
1
0
-
2
1
0
-
3Figure 7
The learning curve for Example 4.Figure 8
The inspection curve for Example 4.Figure 9
The error curve for Example 4.Figure 10
The graphics of the functionCosα,1-α(x).


### 3.6.2. Example  5

We second consider the following nonlinear fractional coupled differential equations:(55)D0+αyx=x6+cos⁡⁡x+6Γ4-αx3-α-yx2-zx,0<x≤1,0<α≤1,D0+αzx=x3+cos⁡⁡x+x-αCos⁡1,1-αx-yx-zx,with initial conditions as follows: (56)y0=0,z0=1,The exact solution is y(x)=x3 and z(x)=cos⁡⁡(x). We set the parameters α=0.9, μ=0.001, M=7, and N=10. Numerical solutions in Table 7 show that this network can also be applied to the nonlinear fractional coupled differential equations but we need more time to train the network.Table 7
Exact solution, approximate solution, and accuracy for Example5.
α
=
0.9
Numerical solution
Accuracy
x
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
0.1
0.9950
0.001
0.9926
0.0042
1
0
-
3
1
0
-
3
0.2
0.9800
0.008
0.9834
0.0148
1
0
-
3
1
0
-
3
0.3
0.9553
0.027
0.9625
0.0351
1
0
-
3
1
0
-
3
0.4
0.9210
0.064
0.9267
0.0689
1
0
-
3
1
0
-
3
0.5
0.8775
0.125
0.8773
0.1220
1
0
-
4
1
0
-
3
0.6
0.8253
0.216
0.8198
0.2040
1
0
-
3
1
0
-
3
0.7
0.7648
0.343
0.7604
0.3276
1
0
-
3
1
0
-
2
0.8
0.6967
0.512
0.7004
0.5049
1
0
-
3
1
0
-
3
0.9
0.6216
0.729
0.6336
0.7375
1
0
-
2
1
0
-
3
1
0.5403
1
0.5475
1.0056
1
0
-
3
1
0
-
3


## 3.6.1. Example  4

We first consider the following linear coupled fractional differential equations:(53)D0+αyx=x2+x3+2Γ3-αx2-α-yx-zx,0<x≤1,0<α≤1,D0+αzx=x2+x3+6Γ4-αx3-α-yx-zx,with initial condition as follows: (54)y0=0,z0=0.The exact solution is y(x)=x2 and z(x)=x3. We set the parameters α=0.9, μ=0.001, M=7, and N=10 and train the neural network 2000 times, and the weights of the network for Example 4 are given in Table 5. Figures 7 and 8 show that the sample points and checkpoints are in well agreement with the exact solutions for the problem. Figure 9 shows that the error of the numerical solutions decreases rapidly within the first 50 training times. Table 6 shows the exact solution, approximate solution, and accuracy for Example 4.Table 5
Weights obtained along with the solution of Examples4 and 5.
α
=
0.9
Example4
Example5
w
1
/
q
1
0.5307
/
0.4494
0.8215
/
0.7682
w
2
/
q
2
-
0.0592
/
0.0013
0.3545
/
-
0.7084
w
3
/
q
3
-
0.6414
/
-
0.7451
-
0.1512
/
-
0.2292
w
4
/
q
4
0.0052
/
-
0.0392
-
0.1342
/
-
0.1116
w
5
/
q
5
0.0559
/
0.3705
0.1657
/
0.3582
w
6
/
q
6
0.3998
/
0.2282
-
0.0815
/
0.0781
w
7
/
q
7
-
0.4141
/
-
0.4014
0.0533
/
-
0.2259Table 6
Exact solution, approximate solution, and accuracy for Example4.
α
=
0.9
Numerical solution
Accuracy
x
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
0.1
0.01
0.001
0.0101
0.0020
1
0
-
4
1
0
-
3
0.2
0.04
0.008
0.0408
0.0104
1
0
-
4
1
0
-
3
0.3
0.09
0.027
0.0926
0.0301
1
0
-
3
1
0
-
3
0.4
0.16
0.064
0.1641
0.0664
1
0
-
3
1
0
-
3
0.5
0.25
0.125
0.2529
0.1233
1
0
-
3
1
0
-
3
0.6
0.36
0.216
0.3584
0.2071
1
0
-
3
1
0
-
3
0.7
0.49
0.343
0.4847
0.3290
1
0
-
3
1
0
-
2
0.8
0.64
0.512
0.6389
0.5033
1
0
-
3
1
0
-
3
0.9
0.81
0.729
0.8215
0.7361
1
0
-
2
1
0
-
3
1
1
1
1.0126
1.0076
1
0
-
2
1
0
-
3Figure 7
The learning curve for Example 4.Figure 8
The inspection curve for Example 4.Figure 9
The error curve for Example 4.Figure 10
The graphics of the functionCosα,1-α(x).


## 3.6.2. Example  5

We second consider the following nonlinear fractional coupled differential equations:(55)D0+αyx=x6+cos⁡⁡x+6Γ4-αx3-α-yx2-zx,0<x≤1,0<α≤1,D0+αzx=x3+cos⁡⁡x+x-αCos⁡1,1-αx-yx-zx,with initial conditions as follows: (56)y0=0,z0=1,The exact solution is y(x)=x3 and z(x)=cos⁡⁡(x). We set the parameters α=0.9, μ=0.001, M=7, and N=10. Numerical solutions in Table 7 show that this network can also be applied to the nonlinear fractional coupled differential equations but we need more time to train the network.Table 7
Exact solution, approximate solution, and accuracy for Example5.
α
=
0.9
Numerical solution
Accuracy
x
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
y
(
x
)
z
(
x
)
0.1
0.9950
0.001
0.9926
0.0042
1
0
-
3
1
0
-
3
0.2
0.9800
0.008
0.9834
0.0148
1
0
-
3
1
0
-
3
0.3
0.9553
0.027
0.9625
0.0351
1
0
-
3
1
0
-
3
0.4
0.9210
0.064
0.9267
0.0689
1
0
-
3
1
0
-
3
0.5
0.8775
0.125
0.8773
0.1220
1
0
-
4
1
0
-
3
0.6
0.8253
0.216
0.8198
0.2040
1
0
-
3
1
0
-
3
0.7
0.7648
0.343
0.7604
0.3276
1
0
-
3
1
0
-
2
0.8
0.6967
0.512
0.7004
0.5049
1
0
-
3
1
0
-
3
0.9
0.6216
0.729
0.6336
0.7375
1
0
-
2
1
0
-
3
1
0.5403
1
0.5475
1.0056
1
0
-
3
1
0
-
3


## 4. Conclusion

In this paper, by using the neural network, we obtained the numerical solutions for single fractional differential equations and the systems of coupled differential equations of fractional order. The computer graphics demonstrates that numerical results are in well agreement with the exact solutions. In (1), suppose that f(x,y(x))=A(x)+B(x)y+C(x)y2; then the problem transformed into Fractional Riccati Equations (Example  3 in this paper). In (3), suppose that f(x,y(x))=y(x)(r-ay(x)-bz(x)) and g(x,z(x))=z(x)(-d+cy(x)); then the problem transformed into fractional-order Lotka-Volterra predator-prey system. We will consider this problem in another paper. The neural network is a powerful method and is effective for the above two problems, which should be also able to solve fractional partial differential equations.



---

*Source: 439526-2015-10-04.xml*