## 4 Results and Discussion
### Performance Evaluation with Various Datasets  
Besides the optimization of the model architecture, constructing an excellent dataset is also a key to improving the model performance. Accordingly, based on the determined detection framework, the model runs on each subset in Table 2 for five times, and the error metrics are analyzed to evaluate the model performance. The quantity and proportion of each pattern in the testing result (one of five runs) are shown in Table 4. We can find that the proportions of arching and normal of the detection results are as high as 49.8% and 50.2% (Subset 2), which is quite close to the ground truth. Compared with Subset 2, the detection results of imbalanced sets are more different from the ground truth, especially for Subset 1. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Subset**} & \multicolumn{2}{c}{**Arching**} & \multicolumn{2}{c}{**Normal**} & \multirow{2}{*}{**Ground Truth**} \\ \cline{2-2} \cline{5-6}  & **Detection Result** & & & & **Ratio (\%)** & \\ \hline
1 & 5434 & 45.7 & 6320 & 54.3 & \\
**2** & **5924** & **49.8** & **5972** & **50.2** & \\
3 & 5757 & 48.4 & 6139 & 51.6 & \\
4 & 5526 & 46.4 & 6370 & 53.6 & \\
5 & 5785 & 48.6 & 6111 & 51.4 & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Quantity and proportion of each pattern in the testing result (one of five runs). \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Case**} & \multirow{2}{*}{**Architecture**} & \multicolumn{2}{c}{**Evaluation Metrics (\%)**} & \multirow{2}{*}{**Time/Epoch**} \\ \cline{3-3} \cline{5-6}  & & **Precision** & & **Recall** & **F\({}_{1}\)-Score** & **(s)** \\ \hline
1 & \(5\times 5\) @ 6, FC 128 & 94.5 & 96.6 & 95.5 & 13.5 \\ \hline
2 & \(5\times 5\) @ 6, \(5\times 5\) @ 12, FC 128 & 96.5 & 97.6 & 97.0 & 13.8 \\ \hline
3 & \(3\times 3\) @ 6, \(3\times 3\) @ 12, \(3\times 3\) @ 24, FC 128 & 97.0 & 97.7 & 97.3 & 7.2 \\ \hline
**4** & \(5\times 5\) @ 6, \(5\times 5\) @ 12, \(5\times 5\) @ 24, FC 128 & **97.8** & **98.2** & **98.0** & **14.0** \\ \hline
5 & \(7\times 7\) @ 6, \(7\times 7\) @ 12, \(7\times 7\) @ 24, FC 128 & 96.7 & 97.0 & 96.8 & 22.6 \\ \hline
6 & \(5\times 5\) @ 6, \(5\times 5\) @ 12, \(5\times 5\) @ 24, FC 128-FC 32 & 96.8 & 98.0 & 97.4 & 14.4 \\ \hline
7 & \(5\times 5\) @ 6, \(5\times 5\) @ 12, \(5\times 5\) @ 24, FC 512-FC 128 & 97.9 & 96.8 & 97.4 & 14.6 \\ \hline
8 & \(5\times 5\) @ 6, \(5\times 5\) @ 12, \(5\times 5\) @ 24, FC 512-FC 128-FC 32 & 95.0 & 97.7 & 96.2 & 14.7 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Arching detection performance of different CNN architectures. To quantify the detection ability of the proposed method, the error metrics of each subset are calculated as depicted in Figure 11. Regardless of the validation set or the testing set, the F\({}_{1}\)-score of each pattern can exceed 98%, showing the high detection accuracy of the proposed model. Specifically, the Precision, Recall, and F\({}_{1}\)-score of \"Arching\" on the testing set (Figure 11b) can reach 98.3%, 98.4%, and 98.4%, respectively. For the detection of the pattern \"Arching\", the cases using Subset 2 outperform the cases using other datasets, as shown in Figure 11a,b. Compared with the metrics of Subset 1, the metrics of Subset 2 are significantly improved, in which the Recall on the testing set increases by 6.2%. It demonstrates that data augmentation operation can effectively enhance the arching feature extraction, thereby improving the model fitting to the pattern \"Arching\". In terms of the data imbalance impact on model performance, Subset 2, Subset 3, Subset 4, and Subset 5 are used for comparative research. At the same training size, with the imbalance degree (i.e., the ratio of \"Normal\" to \"Arching\") increases, the overall detection performance of both the validation set and testing set decreases. As the pattern ratio changes from 1:1 to 6:1, the F\({}_{1}\)-score of the validation set decreases from 98.0% to 92.5%, and the F\({}_{1}\)-score of the testing set decreases from 98.4% to 96.3%.