# Fractional-Order Deep Backpropagation Neural Network

Authors: Chunhui Bao; Yifei Pu; Yi Zhang
Journal: Computational Intelligence and Neuroscience
 (2018)

---

## Abstract

In recent years, the research of artificial neural networks based on fractional calculus has attracted much attention. In this paper, we proposed a fractional-order deep backpropagation (BP) neural network model withL2 regularization. The proposed network was optimized by the fractional gradient descent method with Caputo derivative. We also illustrated the necessary conditions for the convergence of the proposed network. The influence of L2 regularization on the convergence was analyzed with the fractional-order variational method. The experiments have been performed on the MNIST dataset to demonstrate that the proposed network was deterministically convergent and can effectively avoid overfitting.


---

## Body

## 1. Introduction

It is well known that artificial neural networks (ANNs) are the abstraction, simplification, and simulation of the human brains and reflect the basic characteristics of the human brains [1]. In recent years, great progress has been made in the research of deep neural networks. Due to the powerful ability of complex nonlinear mapping, many practical problems have been successfully solved by ANNs in the fields of pattern recognition, intelligent robot, automatic control, prediction, biology, medicine, economics, and other fields [2, 3]. BP neural network is one of the most basic and typical multilayer forward neural networks, which are trained by backpropagation (BP) algorithm. BP, which is an efficient way for optimization of ANNs, was firstly introduced by Werbos in 1974. Then, Rumelhart and McCelland et al. implemented the BP algorithm in detail in 1987 and applied it to the multilayer network version of Minsky [4–6].The fractional calculus has a history as long as the integral order calculus. In the past three hundred years, the theory of fractional calculus has made great progresses [7–11]. Its basics are differentiation and integration of arbitrary fractional order. Nowadays, fractional calculus is widely used in diffusion processes [12–14], viscoelasticity theory [15], automation control [16–18], signal processing [19–21], image processing [22–25], medical imaging [26–28], neural networks [29–37], and many other fields. Due to the long-term memory, nonlocality, and weak singularity characteristics [29–37], fractional calculus has been successfully applied to ANNs. For instance, Boroomand constructed the Hopfield neural networks based on fractional calculus [37]. Kaslik analyzed the stability of Hopfield neural networks [30]. Pu proposed a fractional steepest descent approach and offered a detailed analysis of its learning conditions, stability, and convergence [38]. Wang applied the fractional steepest descent algorithm to train BP neural networks and proved the monotonicity and convergence of a three-layer example [33]. However, there are three limitations in the proposed fractional-order BP neural network models in [33]. First, the neural network in [33] just had 3 layers, which was actually a shadow network and was not proper to demonstrate its potential for deep learning. Second, the fractional order v of this model was restricted to (0,1] without reasonable analysis. Third, the loss function did not contain the regularization term, which is an efficient way to avoid overfitting, especially when the training set has small scalar. Overfitting means that the model has high prediction accuracy on training set but has the low prediction accuracy on testing set. This makes the generalization ability of the model poor, and the application value is greatly reduced.In this paper, we proposed a deep fractional-order BP neural network withL2 regularization term, and the fractional-order v could be any positive real number. With the fractional-order variational method, the influence of L2 regularization on the convergence of the proposed model was exploited. The performance of the proposed model was evaluated on the MINST dataset.The structure of the paper is as follows: in Section2, the definitions and simple properties of fractional calculus are introduced. In Section 3, the proposed fractional-order multilayer BP neural networks are given in detail. In Section 4, the necessary conditions and the influence of L2 regularization for the convergence of the proposed BP algorithm are stated. In Section 5, experimental results are presented to illustrate the effectiveness of our model. Finally, the paper is concluded in Section 6.


## 2. Background Theory for Fractional Calculus

In this section, the basic knowledge of fractional calculus is introduced, including the definitions and several simple properties used in this paper.Different from integer calculus, fractional derivative does not have a unified temporal definition expression up to now. The commonly used definitions of fractional derivative are Grünwald-Letnikov (G-L), Riemann-Liouville (R-L), and Caputo derivatives [7–11].The following is the G-L definition of fractional derivative:(1)DxvaG-Lfx≜limh→0⁡h-v∑k=0x-a/h-vkfx-kh≜limN→∞⁡x-a/N-vΓ-v∑k=0N-1Γk-vΓk+1fx-kx-aNwhere(2)-vk=-v-v+1…-v+k-1k!DxvaG-L denotes the fractional differential operator based on G-L definition, fx denotes a differintegrable function, v is the fractional order, [a,x] is the domain of fx, Γ is the Gamma function, and [·] is the rounding function.The R-L definition of fractional derivative is as follows:(3)DxvaR-Lfx=1Γn-vdndxn∫axfyx-yv-n+1dywhere DxvaR-L denotes the fractional differential operator based on G-L definition; n=[v+1]. Moreover, the G-L fractional derivative can be deduced from the definition of the R-L fractional derivative.The Caputo definition of fractional derivative is as follows:(4)DxvaCfx=1Γn-v∫axx-yn-v-1fnydywhere DxvaC is the fractional differential operator based on Caputo definition, n=[v+1].Fractional calculus is more difficult to compute than integer calculus. Several mathematical properties used in this paper are given here. The fractional differential of a linear combination of differintegral functions is as follows:(5)Dxvλfx+βgx=λDxvfx+βDxvgxwhere fx and gx are differintegral functions and λ and β are constants.The fractional differential of constant functionfx=C, (C is a constant) is different under different definitions:For the G-L definition,(6)DxvaG-Lfx=limN→∞⁡x-a/N-vΓ-v∑k=0N-1Γk-vΓk+1C=Cx-a-vΓ1-vFor the R-L definition,(7)DxvaR-Lfx=Cx-a-vΓ1-v,v>0And for the Caputo definition(8)DxvaCfx=0,v>0According to (6), (7) and (8), we can know that for the G-L and R-L definition, the fractional differential of constant function is not equal to 0. Only with the Caputo definition, the fractional differential of constant function equals to 0, which is consistent to the integer-order calculus. Therefore, the Caputo definition is widely used in solving engineering problems and it was employed to calculate the fractional-order derivative in this paper. The fractional differential of function fx=(x-a)p, p>-1 is as follows:(9)dvx-apdxv=Γp+1x-ap-vΓp-v+1


## 3. Algorithm Description

### 3.1. Fractional-Order Deep BP Neural Networks

In this section, we introduce the fractional-order deep BP neural network withL layers. nl, l=1,2,…,L, is the number of neurons for the l-th layer. Wl=wjilnl+1×nl denotes the weight matrix connecting the l-th layer and the l+1-th layer. fl denotes the corresponding activation function for the l-th layer. Xj and Oj are the input and the corresponding ideal output of the j-th sample and the training sample set is Xj,Ojj=1J. Zl=(z1l,z2l…,znl+1l) denotes the total inputs of l-th layer. If neurons in the l-th layer are not connected to any neurons in previous layer, these neurons are called external outputs of the l-th layer, denoted as A1l. On the contrary, if neurons in the l-th layer are connected to every neuron in previous layer, these neurons are called internal outputs of l-th layer, denoted as A2l. Al=(a1l,a2l…,anll) denotes the total outputs of l-th layer. The forward computing of the fractional-order deep BP neural networks is as follows:(10)A2l=flZl(11)Al=A1lA2l(12)Zl+1=Wl·AlParticularly, external outputs can exist in any layer except the last one. With the square error function, the error corresponding toj-th sample can be denoted as:(13)Ej=12AjL-Oj2=12∑i=1nLajiL-oji2where ajiL denotes the i-th element of AjL, oji denotes the i-th element of Oj.The total error of the neural networks is defined as(14)E=∑j=1JEj=12∑j=1JAjL-Oj2=12∑j=1J∑i=1nLajiL-oji2.In order to minimize the total error of the fractional-order deep BP neural network, the weights are updated by the fractional gradient descent method with Caputo derivative. Leti=1,2,...,nl. The backpropagation of fractional-order deep BP neural networks can be derived with the following steps.Firstly, we define that(15)δil=∂E∂zil.According to (13), we can know that(16)δiL=∂E∂ziL=∑j=1JajiL-ojifL′ziL.Then the relationship betweenδil and δil+1 can be given by(17)δil=∂E∂zil=∑j=1nl+1∂E∂zjl+1∂zjl+1∂zil=∑j=1nl+1δjl+1·wjilfl′zil=fl′zil∑j=1nl+1δjl+1·wjil.Then, according to the chain rule and (17), we have(18)DwjilvE=∂E∂zjl+1·Dwjilvzjl+1=δjl+1·ail·wjil1-vΓ2-v.The updating formula is(19)wjilt+1=wjilt-ηDwjiltvEwhere t∈N denotes the t-th iteration and η>0 is the learning rate.


### 3.2. Fractional Deep BP Neural Networks withL2 Regularization

Fractional-order BP neural network can be overfitted easily when the training set has small scalar.L2 regularization is a useful way to avoid models to be overfitted without modifying the architecture of network. Therefore, by introducing the L2 regularization term into the total error, the modified error function can be presented as(20)EL2=E+λ2W2where W2 denotes the sum of squares of all weights and λ≥0 denotes the regularization parameter.By introducing (18), we have(21)DwjilvEL2=DwjilvE+λwjil2-vΓ3-v.The updating formula is(22)wjilt+1=wjilt-ηDwjiltvEL2where t∈N denotes the t-th iteration and η>0 is the learning rate.


## 3.1. Fractional-Order Deep BP Neural Networks

In this section, we introduce the fractional-order deep BP neural network withL layers. nl, l=1,2,…,L, is the number of neurons for the l-th layer. Wl=wjilnl+1×nl denotes the weight matrix connecting the l-th layer and the l+1-th layer. fl denotes the corresponding activation function for the l-th layer. Xj and Oj are the input and the corresponding ideal output of the j-th sample and the training sample set is Xj,Ojj=1J. Zl=(z1l,z2l…,znl+1l) denotes the total inputs of l-th layer. If neurons in the l-th layer are not connected to any neurons in previous layer, these neurons are called external outputs of the l-th layer, denoted as A1l. On the contrary, if neurons in the l-th layer are connected to every neuron in previous layer, these neurons are called internal outputs of l-th layer, denoted as A2l. Al=(a1l,a2l…,anll) denotes the total outputs of l-th layer. The forward computing of the fractional-order deep BP neural networks is as follows:(10)A2l=flZl(11)Al=A1lA2l(12)Zl+1=Wl·AlParticularly, external outputs can exist in any layer except the last one. With the square error function, the error corresponding toj-th sample can be denoted as:(13)Ej=12AjL-Oj2=12∑i=1nLajiL-oji2where ajiL denotes the i-th element of AjL, oji denotes the i-th element of Oj.The total error of the neural networks is defined as(14)E=∑j=1JEj=12∑j=1JAjL-Oj2=12∑j=1J∑i=1nLajiL-oji2.In order to minimize the total error of the fractional-order deep BP neural network, the weights are updated by the fractional gradient descent method with Caputo derivative. Leti=1,2,...,nl. The backpropagation of fractional-order deep BP neural networks can be derived with the following steps.Firstly, we define that(15)δil=∂E∂zil.According to (13), we can know that(16)δiL=∂E∂ziL=∑j=1JajiL-ojifL′ziL.Then the relationship betweenδil and δil+1 can be given by(17)δil=∂E∂zil=∑j=1nl+1∂E∂zjl+1∂zjl+1∂zil=∑j=1nl+1δjl+1·wjilfl′zil=fl′zil∑j=1nl+1δjl+1·wjil.Then, according to the chain rule and (17), we have(18)DwjilvE=∂E∂zjl+1·Dwjilvzjl+1=δjl+1·ail·wjil1-vΓ2-v.The updating formula is(19)wjilt+1=wjilt-ηDwjiltvEwhere t∈N denotes the t-th iteration and η>0 is the learning rate.


## 3.2. Fractional Deep BP Neural Networks withL2 Regularization

Fractional-order BP neural network can be overfitted easily when the training set has small scalar.L2 regularization is a useful way to avoid models to be overfitted without modifying the architecture of network. Therefore, by introducing the L2 regularization term into the total error, the modified error function can be presented as(20)EL2=E+λ2W2where W2 denotes the sum of squares of all weights and λ≥0 denotes the regularization parameter.By introducing (18), we have(21)DwjilvEL2=DwjilvE+λwjil2-vΓ3-v.The updating formula is(22)wjilt+1=wjilt-ηDwjiltvEL2where t∈N denotes the t-th iteration and η>0 is the learning rate.


## 4. Convergence Analysis

In this section, the convergence of the proposed fractional-order BP neural network is analyzed. According to previous studies [39–42], there are four necessary conditions for the convergence of BP neural networks:(1) The activation functionsfl,(l=1,2,…,L) are bounded and infinitely differentiable on R and all of their corresponding derivatives are also continuous and bounded on R. This condition can be easily satisfied because the most common sigmoid activation functions are uniformly bounded on R and infinitely differentiable.(2) The boundedness of the weight sequencewjilt is valid during training procedure and Ω∈R∑1L-1nl·nl+1 is the domain of all weights with certain boundary.(3) The learning rateη>0 has an upper bound.(4) LetW denote the weights matrix that consists of all weights and ϕ=W∣DWvEL2=0 be the v-order stationary point set of the error function. One necessary condition is that ϕ is a finite set.Then, the influence ofL2 regularization on the convergence is derived by using the fractional-order variational method.According to (20), EL2 is defined as a fractional-order multivariable function. The proposed fractional-order BP algorithm is to minimize EL2. Let U denote the fractional-order extreme point of EL2 and ξ denotes an admissible point. In addition, U is composed of U1,U2,…,UL-1 where Ul(l=1,2,…,L-1) denotes the weights matrix between the l-th and l+1-th layer when EL2 reaches the extreme value. ξ is composed of ξ1,ξ2,…,ξL-1 where ξl corresponds to Ul. The initial weights are random values, so the initial points of weights can be represented as U+α-1ξ, where α is a vector that consists of small parameters α1,α2,…,αL-1, and αl corresponds to Ul and ξl. If α=1, it means αl=1(l=1,2,…,L-1), then U+α-1ξ=U, and EL2 reaches the extreme value. Thus, the process of training the BP neural networks from a random initial weight W to U can be treated as the process of training α with a random initial value to α=1.The fractional-order derivative ofEL2 on U+(α-1)ξ is given as(23)DαvδEL2α=1=Dαvδ1α+δ2αα=1=DαvEU+α-1ξ+Dαvλ2U+α-1ξ2α=1=0where v is the fractional order, which is a positive real number.From (23), we can see that when α=1, if the v-order differential of EU+(α-1)ξ with respect to α is existent, δ1α has a v-order extreme point and we have (24)Dαvδ1αα=1=DαvEU+α-1ξα=1=0.In this case, the output of each layer in the neural networks is still given by (10) and (11) and the input of each layer is turned into the following:(25)Zl+1=Ul+αl-1ξl·Al.Whenαl=1, we have(26)Zl+1=Ul·Al.Without loss of generality, according to (18), for the l-th layer of the networks, the v-order differential of E with respect to αl can be calculated as(27)DαlvEUl+αl-1ξlαl=1=∂E∂Zl+1·DαlvZl+1αl=1=δl+1·AlTξlαl1-vΓ2-vαl=1=ξlδl+1·AlTΓ2-v=0.where δl denotes the column vector DZl1E.Since the value ofξ is stochastic, according to variation principle [43], to allow (24) to be set up, a necessary condition is that for every layer of the networks(28)δl+1·AlTΓ2-v=0.Secondly, without loss of generality, forδ2α we have(29)Dαlvδ2αlαl=1=Dαlvλ2Ul+αl-1ξl2αl=1=∑λUjilξjilΓ2-v+λξjil2Γ3-v-λξjil2Γ2-v=λΓ2-vΓ3-v∑UjilξjilΓ3-v+ξjil2Γ2-v-ξjil2Γ3-v=0To allow (29) to be set up, a necessary condition is(30)λΓ2-vΓ3-v=0.With (28) and (30), the Euler-Lagrange equation of DαvδEL2α=1 can be written as (31)δl+1·AlTΓ2-v+λΓ2-vΓ3-v=0.Equation (31) is the necessary condition for the convergence of the proposed fractional-order BP neural networks with L2 regularization. From (31), we can see that if λ>0, then δl+1·AlT≠0. δl+1·AlT is the first-order derivative of E in terms of U and can be calculated by U and input sample X. It means that the extreme point U of the proposed algorithm is not equal to the extreme point of integer-order BP algorithm or fractional-order BP algorithm. U changes with the different value of λ and v. In addition, it is also clear that the regularization parameter λ is bounded since the values of input samples X and weights W are bounded and v is a constant during the training process.


## 5. Experiments

In this section, the following simulations were carried out to evaluate the performance of the presented algorithm. The simulations have been performed on the MNIST handwritten digital dataset. Each digit in the dataset is a 28 × 28 image. Each image is associated with a label from 0 to 9. We divided each image into four parts, which were top-left, bottom-left, bottom-right, and top-right, and each part was a 14 × 14 matrix. We vectorized each part of the image as a 196 × 1 vector and each label as a 10 × 1 vector.In order to identify the handwritten digits in MNIST dataset, a neural network with 8 layers was proposed. Figure1 shows the topological structure of the neural networks. For the first four layers of the network, each layer has 196 external neurons and 32 internal neurons. The outputs of the external neurons are in turn four parts of an image and the outputs of the internal neurons of the first layer are 1. The last four layers have no external neurons. The fifth layer, sixth layer, and seventh layer have 64 internal nodes and the output layer has ten nodes. The activation functions of all neurons except the first layer are sigmoid functions, which can be given as follows:(32)fx=11+e-x.Figure 1
The topological structure of the neural networks.The MNIST dataset has a total number of 60000 training samples and 10000 testing samples. The simulations demonstrate the performance of the proposed fractional-order BP neural network withL2 regularization, fractional-order BP neural network, traditional BP neural network, and traditional BP neural network with L2 regularization. To evaluate the robustness of our proposed network for a small set of training samples, we set the number of training samples to be (10000, 20000, 30000, 40000, 50000, and 60000). Different fractional v-order derivatives were employed to compute the gradient of error function, where v=1/9, 2/9, 3/9, 4/9, 5/9, 6/9, 7/9, 8/9, 9/9, 10/9, 11/9, 12/9, 13/9, 14/9, 15/9, 16/9, 17/9, 19/9, and 20/9 separately (v=9/9=1 corresponds to standard integer-order derivative for the common BP; v≠2 because if v=2 the change of weights after each iteration is 0, and the weights of the neural networks cannot be updated). The learning rate was set to be 3 and the batch size was set to be 100. The number of epochs n was 300. Two main metrics—training accuracy and testing accuracy—were used to measure the performance of the results from different networks. Each network was trained 5 times and the average values were calculated.In order to explore the relationship between the fractional orders and the neural network performance, the fractional-order neural networks with different orders were trained. Figure2 shows the results of different networks with different sizes of training set. We can find that when the fractional order v exceeds 1.6, both the training and testing accuracies declined rapidly, and when the fractional order v>2, the performances of the fractional BP neural networks were much poorer than that with 0<v<2. The results of v=19/9 and 20/9 were shown in Table 1 as examples. This result is consistent with that for describing physical problems, and usually the limitation 0<v<2 is adopted in the fractional-order models.Table 1
Performances of the algorithms when v>2.
Size of training set v = 19 / 9 v = 20 / 9 Train Accuracy Test Accuracy Train Accuracy Test Accuracy 10000 88.65% 83.52% 76.31% 72.66% 20000 91.04% 89.52% 78.93% 75.97% 30000 93.03% 90.65% 82.51% 80.79% 40000 93.20% 90.53% 82.47% 80.61% 50000 93.02% 91.23% 82.53% 81.60% 60000 93.85% 91.71% 87.32% 86.05%Figure 2
The relationship between the fractional order of gradient descent method and the neural network performance.From Figure2, it can be observed that, with the increase of the size of training set, the performances of the networks were improved visibly. Furthermore, it is also obvious that the training and testing accuracies raised gradually with increasing fractional orders and then reached the peak while v equaled 10/9 or 11/9 order. After that, the training and testing accuracies began to decline rapidly.Table2 shows the optimal orders under training set and testing set separately with different size of training set and it can be noticed that the optimal orders almost concentrated in 10/9 and 11/9. The only exception is that when the number of training samples was 50000, the training accuracy of order 1 was slightly higher than that in 10/9 or 11/9 order case. Generally, for the MNIST dataset the performances of fractional-order BP neural networks are better than integer order.Table 2
Optimal Orders and Highest Accuracies.
Size of training set Optimal order of training set Optimal order of testing set Highest training accuracy Highest testing accuracy 10000 10/9 11/9 98.53% 90.31% 20000 10/9 10/9 98.84% 92.34% 30000 11/9 11/9 99.05% 93.50% 40000 10/9 11/9 99.18% 93.92% 50000 1 10/9 99.20% 94.56% 60000 11/9 11/9 99.20% 95.00%It also can be seen that, in each case, the training accuracy is much bigger than testing accuracy, which means that the BP neural networks have obvious overfitting phenomenon. To avoid overfitting, the integer-order and fractional-order BP neural networks withL2 regularization were trained. With different sizes of training set we chose the regularization parameter λ to be (2×10-5, 1×10-5, 5×10-6, 5×10-6, 5×10-6, and 3×10-6). For the fractional-order neural networks, we chose the fractional order v that had highest testing accuracy in previous simulations. When the numbers of training samples were (10000, 20000, 30000, 40000, 50000, and 60000), we separately set the fractional order v to be (11/9,10/9,11/9,11/9,10/9,11/9).The performance of the proposed fractional-order BP neural networks withL2 regularization and the performance comparison with integer-order BP neural networks (IOBP), integer-order BP neural networks with L2 regularization, and fractional-order BP neural networks (FOBP) in terms of training and testing accuracy are shown in Table 3 and the change of the testing accuracy with the iterations was given in Figure 3Table 3
Performance comparison of different type BP neural networks.
Size of training set Integer-order BP neural networks Fractional-order BP neural networks Integer-order BP neural networks withL2 regularization Fractional-order BP neural networks withL2 regularization Training Accuracy Testing Accuracy Training Accuracy Testing Accuracy Training Accuracy Testing Accuracy Training Accuracy Testing Accuracy Improvement relative to IOBP Improvement relative to FOBP 10000 98.41% 89.87% 98.48% 90.31% 98.45% 93.35% 98.43% 93.95% 4.54% 4.03% 20000 98.81% 92.28% 98.84% 92.34% 98.75% 95.09% 98.79% 95.13% 3.09% 3.02% 30000 98.95% 93.38% 99.05% 93.50% 98.92% 95.15% 98.88% 95.62% 2.40% 2.27% 40000 99.05% 93.83% 99.01% 93.92% 98.96% 95.63% 98.95% 95.83% 2.13% 2.03% 50000 99.20% 94.55% 99.17% 94.56% 99.11% 96.08% 99.15% 96.45% 2.01% 2.00% 60000 99.17% 94.87% 99.20% 95.00% 99.13% 96.51% 99.17% 96.70% 1.93% 1.79% We use the following formula to calculate improvement: improvement of A compared with B = (A-B)÷B.Figure 3
Performance comparison in terms of testing accuracy.In Table3 and Figure 3, it can be seen that, after the addition of L2 regularization to BP neural networks, the training accuracy is slightly decreased but the testing accuracy significantly increased, which indicated that adding L2 regularization can effectively suppress overfitting and improve the generalization of BP neural networks. Furthermore, it can be noticed that after adding L2 regularization the performance of fractional-order BP neural network is better than integer order. One important merit of the L2 regularization is that it gained more benefit while the training set is small. The most possible reason is that the network trained with the smallest number of training samples was affected most by the overfitting. With the increase of the training samples, the model gradually changed from overfitting to underfitting, so the improvement of the regularization method became faint.Then, the stability and convergence of the proposed fractional-order BP neural networks withL2 regularization are demonstrated in Figures 4 and 5. We used the network with optimal order, which means that the size of training set was 60000, fractional-order v was 11/9, and the regularization parameter λ was 3×10-6. Figure 4 shows the change of the total error EL2 during the training process. Without loss of generality, the change of Dw20,205vEL2 was randomly selected and Figure 5 shows the change of it during the training process. It is clear to see that EL2 and DwvEL2 converged fast and stably and were finally close to zero. These observations effectively verify the proposed algorithm is deterministically convergent.Figure 4
Changes of total errorEL2 during the training process.Figure 5
Changes ofDw20,205vEL2 during the training process.


## 6. Conclusion

In this paper, we applied fractional calculus and regularization method to deep BP neural networks. Different from previous studies, the proposed model had no limitations on the number of layers and the fractional-order was extended to arbitrary real number bigger than 0.L2 regularization was also imposed into the errorless function. Meanwhile, we analyzed the profits introduced by the L2 regularization on the convergence of this proposed fractional-order BP network. The numerical results support that the fractional-order BP neural networks with L2 regularization are deterministically convergent and can effectively avoid the overfitting phenomenon. Then, how to apply fractional calculus to other more complex artificial neural networks is an attracted topic in our future work.



---

*Source: 7361628-2018-07-03.xml*