# An Integer-Fractional Gradient Algorithm for Back Propagation Neural Networks
## 2 Method
### Fractional Parameter Update  
We incorporate the fractional derivative parameter update method into the neural network to prevent the neural network from becoming trapped in local optima during the training process [24]. In the existing literature, the Riemann-Liouville fractional derivative, the Caputo fractional derivative, and the Grunwald-Letnikov fractional derivative are the three most commonly used types of fractional-order derivatives. **Definition 1**. : _The Riemann-Liouville fractional-order derivative of \(f(x)\) is defined as_  
\[ \begin{split}\prescript{RL}{a}D_{x}^{\alpha}f(x)=D^{n}(\prescript{ A}{a}D_{x}^{\alpha-n}f(x))=\frac{1}{\Gamma(n-\alpha)}\frac{d^{n}}{dx^{n}}\int_{a}^{x} \frac{f(t)}{(x-t)^{\alpha-n+1}}dt\end{split}  \tag{12}\]  
_where \(\alpha\) is a positive real number and \(n\) is the smallest integer greater than \(\alpha\), and \(\Gamma=\int_{0}^{\infty}t^{z-1}e^{-t}dt\)._  
**Definition 2**. : _The Caputo fractional-order derivative of \(f(x)\) is defined as_  
\[ \begin{split}\prescript{C}{a}D_{x}^{\alpha}f(x)=\frac{1}{\Gamma(n -\alpha)}\int_{a}^{x}\frac{f^{(n)}(t)}{(x-t)^{\alpha-n+1}}dt=\frac{1}{\Gamma( n-\alpha)}\int_{a}^{x}(x-t)^{\alpha-n+1}f(t)dt\end{split}  \tag{13}\]  
_where \(\alpha\) is a positive real number and \(n-1<\alpha<n\)._  
**Definition 3**. : _The Grunwald-Letnikov fractional-order derivative \(f(x)\) is defined as_  
\[ \begin{split}\prescript{GL}{a}D_{x}^{\alpha}=\lim_{h\to 0}\frac{1}{h^{ \alpha}}\sum_{k=0}^{[\frac{x-h}{h^{\alpha}}]}\frac{(-1)^{k}\Gamma(\alpha+1)} {\Gamma(k+1)\Gamma(\alpha-k+1)}f(x-kh)\end{split}  \tag{14}\]  
Figure 1: A three-layer BP neural network architecture. _where \(\alpha\) is a positive real number._  
We employ Caputo fractional-order differentiation in this paper for the following reasons: (1) Caputo fractional-order differentiation has a more unambiguous physical significance. It can be regarded as performing integer-order differentiation on function \(f\) first, followed by integration with a weight. This weight can be viewed as a \"memory effect\", in which the function's value at time \(x\) is influenced by its value at time \(t\), with the degree of influence decreasing with the increase in time interval \(x-t\)[25]. (2) The Caputo fractional order's initial point is the same as that for the integer order. We know that the Caputo fractional-order derivative's initial value problem can be written as \(f(0)\), \(f^{\prime}(0),...f^{(n-1)}(0)\). These are all integer-order derivatives in those forms. Since the initial values are known and the calculation of complicated fractional-order derivatives is not necessary, this greatly simplifies the problem's solution [26]. (3) In comparison to the other two definitions of fractional order, the Caputo fractional-order derivative in the application of neural networks greatly reduces the amount of calculation. Meanwhile, it is closer to the integer order, making it more appropriate for neural networks' error calculation process. In order to simplify the computational complexity of neural networks, this paper chooses the power function and calculates its fractional derivative. For the power function \(f(x)=(x-a)^{p}\), its \(n\)-th derivative is  
\[f^{(n)}(x)=\frac{\Gamma(p+1)}{\Gamma(p-n+1)}(x-a)^{p-n} \tag{15}\]  
Substituting \(f^{(n)}(x)\) into the Caputo derivative definition 2 yields  
\[ \begin{split}\mathop{\mathbb{C}}_{a}D_{x}^{\alpha}=\frac{1}{ \Gamma(n-\alpha)}\int_{a}^{x}\frac{\Gamma(p+1)}{\Gamma(p-n+1)}(t-a)^{p-n}dt \end{split}  \tag{16}\]  
This integral can be further simplified by the properties of the Gamma function. When \(n=p\) (in which case \(p\) is an integer and \(p\geq\alpha\)), the integration becomes relatively simple.