## 5 Experiment and Analysis
### Computation and Results Analysis
#### 5.2.1 The Convolutional Neural Network
(**a**) Accuracy; (**b**) loss. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Learning Rate** & **Train-Loss** & **Train-Acc** & **Test-Loss** & **Test-Acc** \\ \hline
0.0001 & 2.1880 & 0.1114 & 2.1890 & 0.1104 \\
0.0005 & 1.5374 & 0.4343 & 1.5445 & 0.4407 \\
0.001 & 0.5529 & 0.5286 & 0.6247 &  
training and testing accuracies are high, and the time consumption of each iteration is less, meeting the requirements of accuracy and real time. (2) Detailed Results  
(a) Hold-out method  
The hold-out method [40] is used to evaluate the generalization error of the model. First, 4500 samples are randomly disturbed and then a certain proportion of these samples are chosen for training and testing using the hold-out method. Each training lasts for 40 epochs and the evaluation \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Batch-Size** & **Train-Loss** & **Train-Acc** & **Test-Loss** & **Test-Acc** & **Time Cost (s/epoch)** \\ \hline
1 & 0.000021 & 1 & 0.000019 & 1 & 50 \\
3 & 0.00012 & 1 & 0.00014 & 1 & 32 \\
5 & 0.00033 & 1 & 0.00037 & 1 & 23 \\
10 & 0.0084 & 0.9997 & 0.0052 & 1 & 23 \\
20 & 0.0500 & 0.9860 & 0.0299 & 0.9911 & 22 \\
30 & 0.1350 & 0.9711 & 0.1030 & 0.9711 & 22 \\
40 & 0.4711 & 0.8346 & 0.4853 & 0.8119 & 22 \\
50 & 0.7783 & 0.6870 & 0.9526 & 0.5578 & 21 \\
100 & 1.6900 & 0.3394 & 1.8885 & 0.2519 & 18 \\
150 & 2.1296 & 0.1460 & 2.1309 & 0.1615 & 18 \\
200 & 2.1587 & 0.1156 & 2.1601 & 0.1230 & 18 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The loss, accuracy, and time cost under different batch-sizes. Figure 9: The training and testing results under different batch-sizes. (**a**) Accuracy; (**b**) loss; (**c**) time cost. metrics of the test data set are calculated according to Equations (9)-(12). After training and testing five times and calculating the mean value, the results are shown in Table 7.  
According to Table 7, we find that the method of dividing the dataset between training and testing has little effect on the experimental results, and that the four metrics under each division are all 1, with only a small difference in the loss and time consumption. These results demonstrate that the established CNN network model has a good performance. In similar applications of machine learning and CNNs for image classification, approximately 2/3-4/5 samples are generally used for training and the rest are used for testing [44]. Therefore, we use 75% of the data for training and 25% of the data for testing in the next part. (b) Iterative process  
Through all of the above studies, we adopt the CNN structure proposed in Section 3, combined with the Table 1, to determine the following network settings:  
* The dataset is randomly divided using the hold-out method, 75% is divided into the testing set, and 25% is divided into the training set;
* The learning rate is set to 0.01, the batch size is set to 5, and the iteration is set to 40;
* The fully connected layer F1 is processed using \(L_{2}\) regularization;
* The network is trained using an SGD algorithm. * The iterative process of training and testing for 40 epochs is shown in Figure 10.  
From Figure 10, it can be seen that during the 40-epoch iterative process: the training accuracy and testing accuracy increase rapidly and approach 100%, reaching 90% in approximately 10 rounds, and \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Dividing Ways** & & & & & & & **Approximate** \\
**(Train (\%)-Test (\%))** & **Train-Loss** & **Test-Loss** & **Accuracy** & **Precision** & **Recall** & **f1-Score** & **Time Cost** \\  & & & & & & & **(s/epoch)** \\ \hline
60–40 & 0.00049 & 0.00047 & 1 & 1 & 1 & 1 & 21 \\
65–35 & 0.00064 & 0.00065 & 1 & 1 & 1 & 1 & 26 \\
70–30 & 0.00033 & 0.00037 & 1 & 1 & 1 & 1 & 27 \\
75–25 & 0.00037 & 0.00039 & 1 & 1 & 1 & 1 & 32 \\
80–20 & 0.00024 & 0.00026 & 1 & 1 & 1 & 1 & 35 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: The different dividing ways of the hold-out method. Figure 10: The iterative process of training and testing for 40 epochs. reaching 99% in around 17 rounds.