## 3 B-Spline Neural Network
### The B-Spline Neural Network  
Compared with some previous studies, the proposed B-spline adopts an order 3 B-spline function as activation function rather than a B-spline basis function as in some references [29; 31]. Generally speaking, neural networks always have more than one hidden layer. The structure of our BSNN is shown in Figure 4.  
As shown in Figure 4, the neural network system has multiple inputs and one output. Each input corresponds to several activation functions. The output of BSNN can be mapped from inputs \(x_{i}\) to an output \(L_{d}\) by using B-splines as activation functions. The output of the hidden layer can be written as follows:  
\[h_{l}=\prod_{i=1}^{a}\eta_{i,j}(x_{i}),j=1,2,\cdots,\rho \tag{30}\]  
In which, \(\eta_{i,j}(x_{i})\) means the \(i\)th input corresponding to the \(j\)th activation function; \(\rho\) means the number of inputs; \(a\) means the number of activation functions. Therefore, the output of BSNN is as follows:  
\[L_{d}=\sum_{l=1}^{L}\omega_{l}h_{l}=\sum_{l=1}^{L}\omega_{l}\left(\prod_{i=1}^ {\rho}\eta_{i,j}(x_{i})\right),L=a^{\rho} \tag{31}\]  
The training method is to minimize the error function which is defined as follows:  
\[E(t)=V \tag{32}\]  
The chain rules of parameter update can be written as follows:  
\[\frac{\partial E}{\partial s}\frac{\partial s}{\partial l_{d}}=\frac{p+1}{ \beta}\|\dot{e}\|^{p}\|s\| \tag{33}\]  
\[\Delta\omega_{i}=-\frac{\partial E}{\partial\omega_{i}}=-\frac{\partial E}{ \partial s}\frac{\partial s}{\partial l_{d}}\frac{\partial l_{d}}{\partial \omega_{i}}=\frac{p+1}{\beta}\|\dot{e}\|^{p}\|s\|h_{l} \tag{34}\]  
\[\omega_{i}(t)=\omega_{i}(t-1)+\eta\Delta\omega_{i}+\alpha[\omega_{i}(t-1)- \omega_{i}(t-2)] \tag{35}\]  
Figure 4: The general structure of BSNN. \[\Delta\lambda_{i,j}=-\frac{\partial E}{\partial\lambda_{i,j}}=-\frac{ \partial E}{\partial s}\frac{\partial s}{\partial l_{d}}\frac{\partial l_{d}}{ \partial l_{l}}\frac{\partial h_{l}}{\partial l_{j}}\frac{\partial\eta_{l,j}}{ \partial\lambda_{i,j}} \tag{36}\] \[\lambda_{i,j}(t)=\lambda_{i,j}(t-1)+\eta\Delta\lambda_{i,j}+ \alpha[\lambda_{i,j}(t-1)-\lambda_{i,j}(t-2)]\] (37) \[h_{l}=\prod_{i=1}^{a}\eta_{i,j}(x_{i}),j=1,2,\cdots,\rho \tag{30}\]0 (38) \[h_{l}=\prod_{i=1}^{a}\eta_{i,j}(x_{i}),j=1,2,\cdots,\rho \tag{30}\]1 (39) \[\frac{\partial l_{d}}{\partial h_{l}}\frac{\partial h_{l}}{ \partial\eta_{l,j}}=w_{l}\prod_{ \begin{array}{c}s=1\\ s\
eq i\end{array} }^{\rho-1}\eta_{s,j}(x_{s}),j=1,2,\cdots a\] (40) \[s\
eq i\]  
In these equations, \(\eta\) is the learning rate; \(\alpha\) is the inertial coefficient. \[ \begin{array}{ll}\frac{\partial\eta_{i,j}(x_{j})}{\partial\lambda_{i,j}}&= \frac{\partial\prod\limits_{s=0}^{n}(P_{s}^{ij}+\lambda_{i,j}t_{s}+\kappa_{i,j }b_{s})B_{s,k}(\hat{t})]}{\partial\lambda_{i,j}}\\ &=\frac{\partial\prod\limits_{s=0}^{n}P_{s}^{ij}B_{s,k}(\hat{t})+\sum\limits_ {s=1}^{n}\lambda_{i,j}t_{s}B_{s,k}(\hat{t})+\sum\limits_{s=1}^{n}\kappa_{i,j} b_{s,k}(\hat{t})]}{\partial\lambda_{i,j}}\\ &=\sum\limits_{s=0}^{n}t_{s}B_{s,k}(\hat{t})\end{array}  \tag{41}\]  
The B-spline basis function has an important property which is called \"Partition of Unity\", indicating that the sum of all non-zero degree \(k\) basis functions on span \([t_{i},t_{i+1})\) is 1, i.e., \(\sum_{s=0}^{n}B_{s,k}(t)=1\). According to Assumption 1, the following Equations (42) and (44) can be obtained:  
\[\frac{\partial\eta_{l,j}(x_{j})}{\partial\lambda_{i,j}}=\sum\limits_{s=0}^{n }B_{s,k}(\hat{t})=1 \tag{42}\]  
\[ \begin{array}{ll}\frac{\partial\eta_{l,j}(x_{j})}{\partial\kappa_{i,j}}&= \frac{\partial\prod\limits_{s=0}^{n}(P_{s}^{ij}+\lambda_{i,j}t_{s}+\kappa_{i,j }u_{s})B_{s,k}(\hat{t})]}{\partial\kappa_{i,j}}\\ &=\frac{\partial\prod\limits_{s=0}^{n}P_{s}l_{s,k}(\hat{t})+\lambda_{i,j}t_{s }B_{s,k}(\hat{t})+\kappa_{i,j}u_{s}B_{s,k}(\hat{t})]}{\partial\kappa_{i,j}}\\ &=\sum\limits_{s=0}^{n}w_{s}B_{s,k}(\hat{t})\end{array}  \tag{43}\]  
\[\frac{\partial\eta_{l,j}(x_{j})}{\partial\kappa_{i,j}}=\left\{