# A Deep Learning-Based Fragment Detection Approach for the Arena Fragmentation Test
## 3 Experimental Results
### Performance Evaluation  
In order to verify the performance of the proposed deep learning-based warhead fragment detection algorithm, we used the precision and recall, which are evaluation indices. These are primarily used to evaluate object detection algorithms. Precision and recall are defined using True Positive (\(TP\)), False Positive (\(FP\)), and False Negative (\(FN\)). In this study, \(TP\) indicates that real warhead fragments were detected as warhead fragments, \(FP\) indicates that warhead fragments were not actual warhead fragments, and \(FN\) indicates that actual warhead fragments were not detected as warhead fragments. By using this, the precision and recall are calculated as follows:  
\[ \begin{array}{c}precision=\frac{TP}{TP+FP}.\\ recall=\frac{TP}{TP+FN}.\\ \end{array}  \tag{4}\]  
In addition, an evaluation using the \(F\)-measure, which is an evaluation method considering both precision and recall, was conducted. Subscripts of \(F\) represent the parameter \(\beta\) of the \(F\)-measure. The \(F\)-measure was obtained as follows:  
\[F_{\beta}=\left(1+\beta^{2}\right)\frac{precision\times recall}{\left(\beta^{2} \times precision\right)+recall}. \tag{5}\]  
When conventional AFTs detect warhead fragments used physical sensors, the max number of fragments detected in the test was limited only by the number of sensors installed on the steel plate. That is, even if two or more fragments were hit by one sensor, the sensor could detect only the one  
Figure 13: Sampled images collected from four different environments of AFT. (**a**) Case 1. (**b**) Case 2. (**c**) Case 3. (**d**) Case 4.  
piece of fragment data that penetrated for the first time, and the sensor cannot know the exact location of the warhead fragment. In terms of precision and recall, the use of a physical sensor provides high precision but low recall. In addition, the existing system has the disadvantage that the measurement data is contaminated if the fragment hits the sensor-instrument connecting cable before the sensor is hit. The proposed warhead fragment detector was evaluated using three-fold cross-validation as shown in Table 3.  
The performance of the proposed warhead fragment detector for 11 different AFT videos is as follows. The detector showed approximately 93% precision for Cases 1, 2, and 3 however, the recall was approximately 56%. Case 4, on the other hand, demonstrated approximately 77% precision and about 86% recall. In Cases 1, 2, and 3, many types of warhead fragments appeared therefore, the proposed algorithm could not detect approximately 44% of the warhead fragments. On the other hand, in Case 4, as most of the warhead fragments had a stereotype, the proposed algorithm could detect most of the warhead fragments. However, as shown in Figure 14d, the light between the steel plate and holes in the steel plate itself had similar characteristics to the warhead fragments, indicating that the precision dropped to approximately 77%. Overall, it can be seen that it showed a precision of 88% and a high recall of 67%. In other words, 88% of the warhead fragments detected by the proposed warhead fragment detector represented actual warhead fragments, and 67% of the actual warhead fragments were detected. The performance is comparable to that of physical sensors in terms of precision, and was much higher than physical sensors in terms of recall. In particular, the proposed warhead fragment detector had the advantage of finding the exact location of each warhead fragment. In Table 4, we report the performance of the previous image processing-based fragment detection method. As shown in Table 3 and 4, the proposed deep learning-based method outperformed the previous image processing-based method in terms of the precision, \(F_{1}\)-measure, and \(F_{0.5}\)-measure. However, the recall of our method was slightly lower than that of the image processing-based method. In the task of AFT, the precision, \(F_{1}\)-measure, and \(F_{0.5}\)-measure were more important than the recall [22].