## 4 Result and Discussion
### Defect Detection Performance for Metal Transfer Abnormality
This implies that the manner in which the model is trained and the important feature information for classification are unknown. Using only the values of performance measures from the experiments is insufficient for determining the best model and for verifying whether the classification criteria are well trained. In this study, to verify the training validity of the CNN models, the authors applied Grad-CAM, which can visualize the learning of feature information in the images. The results of Grad-CAM are shown in Figure 10.  
As shown in Figure 10, the image is separated into two sections. The original image and trained features from the original image are located on the left section, and the preprocessed image and their trained features on the right section. In each section, the left image represents the true normal case when the CNN model classifies normal data as normal. The right image shows a true abnormal case in which the CNN model classifies abnormal data as abnormal. The image on the top represents the trained data (original and preprocessed images), whereas the images below the first row in Figure 10 show the Grad-CAM images for each CNN model, such as VGG16, VGG16-PRETR, VGG16-PRETR-FINETUNE, and basic CNN for metal transfer. The bright red color in the Grad-CAM image indicates areas with the most prominent effect on model judgment for classification between normal and abnormal cases. By verifying the red area, the feature in the image trained by the CNN model can be visually interpreted. Regarding the training of the original image, the best performance model, VGG16-PRETR, comprises a red area spread across the entire area of the Grad-CAM image. On the contrary, in the true abnormal case, the area emphasized in red is distributed in an abnormal rod shape and at the bottom of the image. The bottom area constitutes the background and is not associated with metal. The learned features shown in red differed between true normal and true abnormal; therefore, the classification performance of VGG16-PRETR is the best. The VGG16 model and basic CNN for metal transfer classification are represented by a wide range of red colors, signifying that these models do not successfully discriminate the targeting features. These models fail to learn the metal edge during the feature training. However, VGG16-PRETR-FINETUNE manages to learn some of the edge parts of the metal rod, but the learned area is extremely narrow as a local area. Considering the model building using the preprocessed images, Grad-CAM shows the edge features of the image from classification. This pattern can be observed in the Grad-CAM images of VGG16, VGG16-PRETR, and VGG16-PRETR-FINETUNE models. In the true normal case, the red area is centered on the metal edges. In the true abnormal case, a red area is formed on the molten circular metal part. The visual description of Grad-CAM provides an understanding regarding an AI model known as an uninterpretable \"black box\". This means that the preprocessed image contributed to the improvement in edge feature recognition by the AI model. Therefore, it is assumed that the preprocessed images appear more suitable for training CNN models in the metal transfer RoI. As shown in Table 2 and Figure 9b, the AUC value of the CNN models trained with the preprocessed image is close to 1. However, the accuracies of the basic CNN for metal transfer, VGG16, and VGG16-PRETR are as low as 0.738, 0.731, and 0.638, respectively. These values are low despite the high AUC value. Meanwhile, the sensitivity is high, and the specificity is low at 0.476, 0.464, and 0.276, respectively, implying that the classification criterion is sensitive to abnormality detection. In this case, the abnormality can be classified perfectly. However, many normal data tend to be classified as abnormal. [PERSON] et al. [63] reported the problem wherein the AUC can be high owing to an inappropriate model (which overestimated or underestimated in all predictions). ROC  
Figure 10: Visual verification using Grad-CAM for original and preprocessed images.