## 5 Experiments
### Comparative Study  
For the evaluation of deep learning classification performance, the comparison between Faster R-CNN+ResNet101 and some classical neural networks has been shown in reference [4], Table 3. The performance comparison of the network system between the Optimized Faster R-CNN and other more advanced deep learning methods such as R-FCN [45], YOLOv3 [46] is shown in Table 9. In the experiment, the same datasets were used for training, and then the same 120 mixed defect datasets were still used for prediction. Experiments show that the Faster R-CNN can have better performance than the more advanced neural network structure if the design of the framework structure is reasonable and the parameter adjustment is appropriate. Practice has shown that the advanced deep learning system is not the best object detection system for some specific detection objects in the field of industry. Of course, it should not be ruled out that advanced neural networks can be optimized and modified to achieve better performance. YOLOv3 is a more advanced deep learning system than Faster R-CNN. When using standard datasets, YOLOv3 works better, but it is not very good in the defect detection of this paper. The experimental data in some reference [19] also proves at this point, Table 10 shows that there are more defects detected on pit, and the breach, abrasion and scratch images are better, but the accuracy of abrasion detection involving a large number of small points is not high. The R-FCN test results are exactly the opposite. We just analysis some reason between Faster R-CNN and YOLOv3. Reason analysis:(a) YOLOv3 input image is a fixed image of \(416\times 416\) resolution, Faster R-CNN input image size is variable and much smaller than \(416\times 416\), so when extracting features, small targets like pits and small dots can easily lose useful information for inputting large images. Dimmed small images are not easy to lose useful information; (b) YOLOv3 requires a large number of training samples, and Faster R-CNN does not require a large number of training samples, so under the same sample quantity conditions, when Faster R-CNN achieves good results, YOLOv3 is not as good as Faster R-CNN. Certainly, over-fitting must be considered, to prove the model for comparison in the project is the comparable one. The reasons of over-fitting of deep learning include training sample quantity, model and match between the two, etc. Because the training sample quantity is fixed for comparability, so a suitable model has to be selected. To prevent over-fitting, a suitable model is generated by adjusting net architecture, early stopping, regularization and adding noise, etc. But the image and YOLOv3 (YOLOv3 is from another mature trademark recognition project) are fixed, that means, early stopping is a good choice. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Detects** & **Total** & \
\begin{tabular}{c} **Samples** \\ **Detected** \\ \end{tabular} &
\begin{tabular}{c} **Correct Samples** \\ **Detected** \\ \end{tabular} & **Accuracy** & **Precision** & **Recall** \\ \hline Pits & 30 & 10 & 9 & 33\% & 90\% & 30\% \\ Breach & 30 & 30 & 25 & 100\% & 83\% & 83\% \\ Abrasion & 30 & 30 & 23 & 100\% & 77\% & 77\% \\ Scratch & 30 & 30 & 30 & 100\% & 100\% & 100\% \\ Mixed & 120 & 100 & 88 & 83\% & 88\% & 73\% \\ \hline \hline \end{tabular}
\end{table} 
Table 10: YOLOv3 comparison study for four detects base on non-training image. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Methods** & **Total** & \
\begin{tabular}{c} **Samples** \\ **Detected** \\ \end{tabular} &
\begin{tabular}{c} **Correct Samples** \\ **Detected** \\ \end{tabular} & **Accuracy** & **Precision** & **Recall** \\ \hline R-FCN & 120 & 106 & 95 & 88\% & 90\% & 79\% \\ YOLOv3 & 120 & 100 & 88 & 83\% & 88\% & 73\% \\ Ours & 120 & 120 & 119 & 100\% & 99\% & 99\% \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Comparison with other advanced deep learning system. In order to select a suitable model, the early stop strategy is applied in YOLOv3 training.