# An Integer-Fractional Gradient Algorithm for Back Propagation Neural Networks
## 2 Method  
### Network Structure  
We employ a three-layer BP neural network structure, as illustrated in Figure 1. In the neural network model, the input layer consists of \(n\) neurons, the hidden layer contains \(h\) neurons, and the output layer has \(m\) neurons. The neural network has both forward propagation and backward propagation. The forward propagation process consists of learning data features of the neural network. For given sample data, let \(\mathbf{x}=(x_{1},\ x_{2},...,\ x_{n})^{T}\) be the variable vector of the input layer, \(\mathbf{y}=(y_{1},\ y_{2},...,\ y_{h})^{T}\) the variable vector of the hidden layer, \(\mathbf{z}=(z_{1},\ z_{2},...,\ z_{m})^{T}\) the variable vector of the output layer, and \(\hat{\mathbf{z}}=(\hat{z}_{1},\hat{z}_{2},...,\hat{z}_{m})^{T}\)the desired output. Moreover, let \(P=\big{(}p_{ji}\big{)}\) be the weight matrix connected with the input layer and the hidden layer, and \(Q=(q_{kj})\) the weight matrix connected to the hidden layer and the output layer, where \(i=1,\ldots,n\), \(j=1,\ldots,h\), and \(k=1,\ldots,m\). The input and output of neurons in the \(j^{\text{th}}\) hidden layer are described by  
\[\text{neth}_{j}=\ \sum_{i=1}^{n}p_{ji}x_{i}\ =P_{j}\mathbf{x} \tag{1}\]  
and  
\[y_{j}=f(\text{neth}_{j}) \tag{2}\]  
where \(P_{j}=\big{(}p_{j1},\ p_{j2},\ldots,\ p_{jn}\big{)}\), and \(f\) is an activation function. The \(k^{\text{th}}\) output layer's input and output are given by  
\[\text{neto}_{k}=\sum_{j=1}^{h}q_{kj}y_{j}=Q_{k}\mathbf{y} \tag{3}\]  
and  
\[z_{k}=f(\text{neto}_{k}) \tag{4}\]  
where \(Q_{k}=(q_{k1},\ q_{k2},\ldots,\ q_{kh})\). Then, in every iteration, we obtain the output result of the neural network. Now, we consider a total of \(t\) sample inputs. For the \(d\)-th sample input (\(d\in\{1,\ldots,t\}\)), the neural network model automatically corrects the weight \(p_{ji}\) and the weight \(q_{kj}\). The total error equation of the neural network's performance evaluation is formulated by  
\[E=\frac{1}{2}\sum_{d=1}^{t}\sum_{k=1}^{m}\Big{(}\hat{z}_{k}-z_{k}^{d}\Big{)}^{ 2}=\sum_{d=1}^{t}g_{d}\Big{(}\text{neto}_{k}^{d}\Big{)}=\sum_{d=1}^{t}g_{d} \Big{(}Q_{k}\mathbf{y}^{d}\Big{)} \tag{5}\]  
where  
\[g_{d}\Big{(}\text{neto}_{k}^{d}\Big{)}=\frac{1}{2}\sum_{k=1}^{m}\Big{(}\hat{ z}_{k}-f\Big{(}\text{neto}_{k}^{d}\Big{)}\Big{)}^{2} \tag{6}\]  
and  
\[\mathbf{y}^{d}=\Big{(}f\Big{(}\text{neth}_{1}^{d}\Big{)},\ f\Big{(}\text{neth }_{2}^{d}\Big{)},...,\ f\Big{(}\text{neth}_{h}^{d}\Big{)}\Big{)}^{T} \tag{7}\]  
The error calculation process above is used as the forward propagation process of the neural network model. Its weight update by gradient descent through the error equation can be regarded as the backpropagation of the neural network. For the \(k\)-th output layer's error \(E_{k}=\frac{1}{2}\sum_{d=1}^{t}\Big{(}\hat{z}_{k}-z_{k}^{d}\Big{)}^{2}\), according to the chain rule, the partial derivative of \(E_{k}\) with respect to \(q_{kj}\) is given by  
\[\frac{\partial E_{k}}{\partial q_{kj}}=\sum_{d=1}^{t}\frac{\partial E_{k}}{ \partial\text{neto}_{k}^{d}}\frac{\partial\text{neto}_{k}^{d}}{\partial q_{kj }}=\sum_{d=1}^{t}g_{d}^{\prime}\Big{(}\text{neto}_{k}^{d}\Big{)}y_{j}^{d} \tag{8}\]  
Similarly, the partial derivative of \(E_{k}\) with respect to \(p_{ji}\) is expressed by  
\[\frac{\partial E_{k}}{\partial p_{ji}}=\sum_{d=1}^{t}g_{d}^{\prime}\Big{(} \text{neto}_{k}^{d}\Big{)}q_{kj}f^{\prime}(\text{neth}_{j}^{d})x_{i}^{d} \tag{9}\]  
Then, the update functions of the weights \(q_{kj}\) and \(p_{ji}\) are calculated by  
\[q_{kj}^{l}=q_{kj}^{l-1}-\eta\frac{\partial E_{k}}{\partial q_{kj}^{l-1}} \tag{10}\]\[p_{ji}^{l}=p_{ji}^{l-1}-\eta\frac{\partial E_{k}}{\partial p_{ji}^{l-1}} \tag{11}\]  
where \(l\) is the iteration number, and \(\eta\) is the learning rate. The neural network's backward propagation process is implemented by using the updated weights, which will be applied to the next iteration.