## 4 Result and Discussion
### Defect Detection Performance for Metal Transfer Abnormality  
The performances of the CNN models using the original image (see Figure 3) and the preprocessed image to the contoured image (see Figure 4) are shown in Table 2.  
Figure 9 shows the graphs of the ROC curve for each CNN model. The y-axis represents the probability of a well-predicting abnormality with the true positive rate (TPR) as the sensitivity. The x-axis is the false positive rate (FPR 1-specificity), which refers to the rate at which normal data are incorrectly classified as abnormal. Therefore, the higher \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Data** & **Model** & **AUC** & **Sensitivity** & **Specificity** & **Accuracy** \\ \hline \multirow{4}{*}{Original image} & Basic CNN for metal transfer & 0.674 (\(\pm\)0.215) & 0.799 (\(\pm\)0.400) & 0.298 (\(\pm\)0.380) & 0.639 (\(\pm\)0.167) \\  & VGG16 & 0.495 (\(\pm\)0.010) & 0.200 (\(\pm\)0.400) & 0.400 (\(\pm\)0.495) & 0.392 (\(\pm\)0.145) \\  & VGG16-PRETR & **1*** & **0.999 *** (\(\pm\)0.001)** & **0.965 *** (\(\pm\)0.061)** & **0.988 *** (\(\pm\)0.019)** \\  & VGG16-PRETR-FINETUNE & 0.800 (\(\pm\)0.245) & 0.798 (\(\pm\)0.399) & 0.800 (\(\pm\)0.400) & 0.799 (\(\pm\)0.270) \\ \hline \multirow{4}{*}{Preprocessed image} & Basic CNN for metal transfer & **0.999 *** (\(\pm\)0.001)** & **1*** & 0.476 (\(\pm\)0.243) & 0.738 (\(\pm\)0.121) \\  & VGG16 & 0.939 (\(\pm\)0.044) & 0.999 (\(\pm\)0.001) & 0.464 (\(\pm\)0.164) & 0.731 (\(\pm\)0.082) \\ \cline{1-1}  & VGG16-PRETR & 0.990 (\(\pm\)0.005) & **1*** & 0.276 (\(\pm\)0.271) & 0.638 (\(\pm\)0.135) \\ \cline{1-1}  & VGG16-PRETR-FINETUNE & **0.999 *** (\(\pm\)0.001)** & 0.998 (\(\pm\)0.002) & **0.996 *** (\(\pm\)0.005)** & **0.997 *** (\(\pm\)0.002)** \\ \hline \hline \end{tabular}  
* The best performance for each measure.  
\end{table} 
Table 2: Defect detection performance of CNN models for metal transfer abnormality. Figure 9: **(a) ROC curves for detecting abnormality with original image; (b) ROC curves for detecting abnormality with preprocessed image. **  
the sensitivity and specificity, the larger the AUC covers. The AUC is the reference area for calculating the ROC curve. Models with a high AUC should not be unconditionally adopted. Even with a high AUC, other measures such as accuracy, sensitivity, and specificity should be considered simultaneously. An AUC that is closer to 1 signifies a better model; if it is 0.5, it signifies that a random prediction model with a one-half probability is created. Therefore, a model with an AUC of 0.5 or less should not be used. According to Table 2 and Figure 9, VGG16-PRETR demonstrates the best performance for original image classification. Because the weights of VGG16-PRETR are obtained by pretraining a significant amount of image data from ImageNet, it possesses strong capability in classifying images, and this enables abnormalities to be identified within the original image. Meanwhile, the fine-tuned model (VGG16-PRETR-FINETUNE) trained by the original images deteriorates the classification ability, which suggests that the performance of VGG16-PRETR-FINETUNE cannot be improved with insufficient data. When the classification with preprocessed images is tested, VGG16-PRETR-FINETUNE demonstrates better performance than VGG16-PRETR. Previous studies explaining the effects and mechanisms of fine-tuning [38] have shown that deep CNN models can be more specialized for specific classification tasks when certain convolution blocks are fine-tuned. Fine-tuning yields general feature information using the weights of the initial convolution blocks learned in advance. Subsequently, the weights of the convolution blocks are retrained to obtain abstract feature information that fits the data provided. The experimental results show that the VGG-PRETR-FINETUNE CNN model using fine-tuning can be beneficial for certain classification tasks with simple features in the images. The AI model is known as the \"black box\" model.