## 4 Experiments and Results
### Comparison with Faster R-CNN and YOLO-v3  
Table 3 shows the small pit defect detection test results for the RT-SPeeDet, Faster R-CNN, and YOLO-v3 models. Here, SqueezeNet was used as the CNN architecture of RT-SPeeDet, and the bold text shows the best performance for each of the evaluation indexes, including those of Faster R-CNN and YOLO-v3. First, it can be seen that the two models other than RT-SPeeDet were unable to detect any of the defects, and YOLO-v3 in particular was unable to make even a single prediction. As related studies have found, the major causes of the poor detection performance were the loss of object information owing to a convolution in the forward propagation process of the model, as well as the size of the anchor boxes, which were larger than the small pit defects. Although Faster R-CNN predicted boundary boxes containing pit defects, the IoU value was extremely low owing to the large anchor boxes, and all boundary boxes were classified as a misdetection. Based on these results, it can be seen that the two models are unable to properly learn the pit defects in high-resolution images without a separate optimization process.  
To evaluate the pit detection performance of each model in more detail, the positive regions of the film images were divided into 96 images with a pixel resolution of 512 \(\times\) 512 pixels, and training and evaluation were conducted, as in a related study described in Section 2. The results of these experiments are shown in the Faster R-CNN_Cropped and YOLO-v3_Cropped rows of Table 4. Even though the resolution of the input images was reduced, the YOLO-v3 model was unable to find more than 7 of the pits with a recall rate of 10%. Furthermore, because 96 image patches were created for each film image, it took approximately 4 s to search a single image. Faster R-CNN found 42 pits, although it took approximately 14 s to search each image.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Cropping** & **Extracting** & & & & & & \\
**Positive Region** & **Candidate** & **TP** & **TN** & **FP** & **FN** & **Precision** & **Recall** \\
**Speed** & **Region Speed** & & & & & & \\ \hline
0.06 s & 0.03 s & 65 & 0 & 4274 & 3 & 1.5\% & 95.6\% \\ \hline \hline \end{tabular}
\end{table}
Table 2: Candidate region extraction performance of RT-SPeeDet.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Model** & **TP** & **TN** & **FP** & **FN** & **Precision (\%)** & **Recall (\%)** & **Acc (\%)** & **Time (s)** \\ \hline RT-SPeeDet & 54 & 4284 & 1 & 14 & 98.2 & 79.4 & 99.7 & 1.48 \\ Faster R-CNN [6] & 0 & 0 & 68 & 68 & 0 & 0 & 0 & 0.15 \\ YOLO-v3 [9] & 0 & 0 & 0 & 68 & 0 & 0 & 0 & 0.043 \\ Faster & 42 & 0 & 13 & 26 & 76.3 & 61.8 & 51.9 & 14.59 \\ R-CNN_Cropped & 7 & 0 & 4 & 59 & 63.6 & 10.3 & 10 & 4.128 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Performance comparison of various object detectors with small pit defects dataset.  
By contrast, RT-SPEeDet detected pits with 98% precision and approximately 80% recall. Although it achieved a processing time of 1.5 s, the fact that it could process high-resolution images with a size of 8192 \(\times\) 600 pixels without dividing the images shows that it has an excellent processing speed and detection accuracy. Notably, the proposed candidate region image patches were classified into pit defects and background regions with 99.7% accuracy, indicating an excellent pit defect classification performance. However, detection in manufacturing processes, which is the target of this paper, require a processing speed of less than 1 s; therefore, further speed improvements are still needed. To this end, an additional ablation study was conducted to compare the performance based on the use of lightweight architectures. The results of the ablation study are described in the next section.