## 3 The Extreme-Points-Distribution-Based Neural Network (EDNN)
### Recursive Automatic Differentiation
#### 3.2.1 Derivatives of Hidden Layers  
The derivative of the (\(l+1\))th hidden layer with respect to the input layer can be formulated as a function of the derivatives of the \(l\)th hidden layer with respect to the input layer.  
The \(i\)th node's output of the (\(l+1\))th layer is denoted as \(\mathbf{a}_{i}^{l+1}(\mathbf{x})\), and its \(m\)th-order partial derivative with respect to the input layer's \(n\)th node \(\mathbf{x}_{n}^{l}\) is denoted as \(D^{m}\big{(}d_{j}^{l+1}(\mathbf{x}),\mathbf{x}_{n}^{l}\big{)}\)  
Figure 2: Schematic of the \(EDNN\).  
According to [PERSON]'s formula [39; 40], \(D^{m}\big{(}\mathbf{a}_{i}^{l+1}(\mathbf{x}),\mathbf{x}_{n}^{I}\big{)}\) can be expressed as:  
\[\begin{array}{l}D^{m}\Big{(}\mathbf{a}_{i}^{l+1}(\mathbf{x}),\mathbf{x}_{n}^{I}\Big{)}\\ =\sum_{\begin{subarray}{c}\mathbf{i}_{1}\mathbf{i}_{2}\mathbf{i}_{2}\cdots\mathbf{i}_{m}\end{subarray} \end{subarray}}\Big{(}f_{i}^{l+1}\Big{)}^{(k)}\Big{(}\sum_{j=1}^{S^{l}}\mathbf{w}_{i, j}^{l+1,l}\cdot\mathbf{a}_{j}^{l}(\mathbf{x})+\mathbf{b}_{i}^{l+1}\Big{)}\\ \cdot\Bigg{(}\frac{\sum_{j=1}^{S^{l}}\mathbf{w}_{i,j}^{l+1,l}\cdot D^{1}\big{(}\mathbf{a }_{j}^{l}(\mathbf{x}),\mathbf{x}_{n}^{I}\big{)}}{\mathbf{i}!}\Bigg{)}^{\mathbf{b}_{1}}\\ \cdot\Bigg{(}\frac{\sum_{j=1}^{S^{l}}\mathbf{w}_{i,j}^{l+1,l}\cdot D^{2}\big{(}\mathbf{a }_{j}^{l}(\mathbf{x}),\mathbf{x}_{n}^{I}\big{)}}{2!}\Bigg{)}^{\mathbf{b}_{2}}\cdots\Bigg{(} \frac{\sum_{j=1}^{S^{l}}\mathbf{w}_{i,j}^{l+1,l}\cdot D^{m}\big{(}\mathbf{a}_{j}^{l}( \mathbf{x}),\mathbf{x}_{n}^{I}\big{)}}{m!}\Bigg{)}^{\mathbf{b}_{m}}\end{array} \tag{15}\]  
where the sum is over all different solutions in non-negative integers \(b_{1},b_{2},\cdots,b_{m}\) of \(b_{1}+2b_{2}+\cdots+mb_{m}=m\) and \(k=b_{1}+b_{2}+\cdots+b_{m}\); \(\Big{(}f_{i}^{l+1}\Big{)}^{(k)}\Big{(}\sum_{j=1}^{S^{l}}\mathbf{w}_{i,j}^{l+1,l} \cdot\mathbf{a}_{j}^{l}(\mathbf{x})+\mathbf{b}_{i}^{l+1}\Big{)}\) is the \(k\)th-order derivative of the activation function; \(S^{l}\) is the number of nodes in layer \(l\); \(\mathbf{w}_{i,j}^{l+1,l}\) is the weight of the \(j\)th node in the \(l\)th layer to the \(i\)th node in the \((l+1)\)th layer; \(\mathbf{b}_{i}^{l+1}\) is the bias of the \(i\)th node of the \((l+1)\)th layer; \(l=1,\cdots,(L_{h}-1)\), \(L_{h}\) is the total number of hidden layers.  
When there is no transform in the input layer, the \(m\)th-order partial derivative \(D^{m}\big{(}\mathbf{a}_{i}^{1}(\mathbf{x}),\mathbf{x}_{n}^{I}\big{)}\) of the \(i\)th node's output \(\mathbf{a}_{i}^{1}(\mathbf{x})\) in the first hidden layer with respect to the \(n\)th node of the input layer is shown in expression (16).  
\[D^{m}\Big{(}\mathbf{a}_{i}^{1}(\mathbf{x}),\mathbf{x}_{n}^{I}\Big{)}=\Big{(}f_{i}^{1} \Big{)}^{(m)}\Big{(}\sum_{j=1}^{S^{l}}\mathbf{w}_{i,j}^{1,l}\cdot\mathbf{x}_{j}^{l}+ \mathbf{b}_{i}^{1}\Big{)}\cdot\Big{(}\mathbf{w}_{i,n}^{1,l}\Big{)}^{m} \tag{16}\]  
where \(\big{(}f_{i}^{1}\big{)}^{(m)}\Big{(}\sum_{j=1}^{S^{l}}\mathbf{w}_{i,j}^{1,l}\cdot \mathbf{x}_{j}^{l}+\mathbf{b}_{i}^{1}\Big{)}\) is the \(m\)th-order derivative of the activation function; \(S^{l}\) is the number of nodes of the input layer; \(\mathbf{w}_{i,j}^{1,l}\) is the weight of the \(j\)th node in the input layer to the \(i\)th node in the first hidden layer; \(\mathbf{b}_{i}^{1}\) is the bias of the \(i\)th node of the first hidden layer.