# A CNN-Based Method for Heavy-Metal Ion Detection
## 3 CNN Structure
### Parameter Learning  
Parameter learning is a way to adjust the parameters in a convolutional neural network to ensure the results are closer to reality. Parameter learning is often performed using error backpropagation. That is, the parameters are adjusted inversely by the error generated from the derived results and the real label. The following shows the derivation of a single parameter update. Generally, with layer \(l\) as a convolutional layer, the input features of layer \(l-1\) are mapped as \(\mathrm{X}^{(l-1)}\in\mathrm{R}^{\mathrm{M}\times\mathrm{N}\times\mathrm{D}}\)[33], and the net input of the feature mapping at layer \(l\) is obtained by convolution \(Z^{(l)}\in\mathrm{R}^{\mathrm{M}\times\mathrm{N}\times\mathrm{P}}\). The activation value is a\({}^{l}\), and the \(p(1\leq p\leq P)\) feature of the \(l\) layer maps the net input as shown in Equation (8):  
\[Z^{(l,p)}=\sum_{d=1}^{D}W^{(l,p,d)}\otimes X^{(l-1,d)}+b^{(l,p)}, \tag{8}\]  
where \(W^{(l,p,d)}\) is the convolution kernel, \(b^{(l,p)}\) is the bias [33], and there are \(\mathrm{P}\times\mathrm{D}\) convolution kernels and \(P\) biases in layer \(l\). The chain rule can be used to calculate their gradients separately, and we let the loss function be \(L\), whose partial derivatives with respect to the convolution kernel \(W^{(l,p,d)}\) in layer \(l\) are:  
\[\frac{\partial L}{\partial W^{(l,p,d)}}=\frac{\partial L}{\partial Z^{(l,p)}} \otimes X^{(l-1,d)}. \tag{9}\]  
In HMID-NET, the regression loss is the MSE loss, and the classification loss is the cross-entropy loss, as in Equations (10) and (11):  
\[\mathrm{MSE}=\frac{1}{n}\sum_{i=1}^{m}w_{i}(y_{i}-\hat{y_{i}})^{2} \tag{10}\]  
\[H(p,q)=-\sum_{i=1}^{n}p(x_{i})\log(q(x_{i})). \tag{11}\]  
Similarly, the partial derivative of the bias \(p\) of the layer \(l\)\(b^{(l,p)}\) is shown in Equation (12):  
\[\frac{\partial L}{\partial b^{(l,p)}}=\sum_{i,j}\left[\delta^{(l,p)}\right]_{i,j}. \tag{12}\] \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Hidden Layers**} & **Concentration Error** & **Classification** & **Number of** \\  & **(MRE)** & **Accuracy** & **Parameters** \\ \hline
128 & 6.28\% & 99.99\% & 102,915 \\
80 & 8.02\% & 99.99\% & 64,323 \\
72 & 8.59\% & 99.99\% & 57,891 \\
64 & 8.85\% & 99.99\% & 51,459 \\
56 & 9.82\% & 99.99\% & 45,027 \\
48 & 12.00\% & 99.32\% & 38,595 \\
32 & 16.24\% & 97.13\% & 25,731 \\
16 & 19.20\% & 94.00\% & 12,867 \\ NULL & 102.50\% & 34.00\% & 2403 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: CNN algorithm classification accuracy and concentration prediction accuracy under different hidden layer sizes. The value in the ’number of parameters’ column represents the number of parameters in the fully connected layer. where \(\delta^{(l,p)}=\frac{\partial L}{\partial Z^{(l,p)}}\) is the derivative of the loss function with respect to the feature mapping \(p\) of layer \(l\) via input \(Z^{(l,p)}\). The parameter update of the convolution kernel with bias by the derivatives to obtain a single update is as in Equations (13) and (14):  
\[W^{(l)}=W^{(l)}-\alpha\sum_{p=1}^{p}\delta^{(l,p)}\times rot180\Big{(}a^{(l-1,p )}\Big{)} \tag{13}\]  
\[b^{(l)}=b^{(l)}-\alpha\sum_{p=1}^{p}\sum_{\text{i}j}\Big{[}b^{(l,p)}\Big{]}_{ij}, \tag{14}\]  
where \(\alpha\) is the learning rate. Figure 6 shows a diagram of forward and error backward propagation. For convolutional neural networks, a single time of parameter updating is apparently not enough, and HMID-NET uses stochastic gradient descent (SGD) for parameter learning. The stochastic gradient descent process is as follows:  
1. Randomly initialize parameters \(W\), \(b\);
2. Reorder the samples in the training set;
3. Take the data of batch sizes for parameter learning and updating, and obtain the validation set error rate;
4. Repeat steps 2 and 3 until the validation set error rate no longer decreases;
5. Obtain the global optimum \(W\), \(b\). Figure 7 shows a diagram of SGD. Figure 6: Forward propagation and error backpropagation. Figure 7: Schematic diagram of stochastic gradient descent.