## 4 Result and Discussion
### Detection Performance of Weld-Pool/Bead  
The performance of the tested CNN models for the weld-pool/bead RoI is shown in Table 3. Figure 12, below the table, shows the ROC curves for the classification performance of the CNN models. The highest AUC is 0.995, obtained by the basic CNN for the weld-pool/bead. The VGG16 model has the second highest score, 0.982, with the highest accuracy of 0.965. The VGG16-PRETR model, which has the lowest AUC of 0.869, has the lowest accuracy of 0.794. Regarding the weld-pool/bead, each model demonstrates different strengths in each measure. Generally, the basic CNN for the weld-pool/bead and VGG16 demonstrates good performances when the AUC, sensitivity, specificity, and accuracy are considered simultaneously. Both models exhibits good performance values that exceeds 0.9 in each measure. Unlike metal transfer, the weld-pool/bead RoI appears to be sufficient with a simple structure and primitive training using less data. This is because the CNN structure has been proven robust for image classification in other fields. This characteristic of the CNN appears to fit the weld-pool/bead RoI image well. As explained in Section 4.2, Grad-CAM visualizes the feature information that is the basis for classification using the CNN model (see Figure 13). In Figure 13, the top image shows the data used for training and testing. The normal image of the bead shape maintains a uniform height (a straight feature on the upper part). The abnormal bead does not maintain a uniform height and yields a ball-like shape (curve characteristic). The feature information visualization for each model is highlighted in red in Figure 13. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **AUC** & **Sensitivity** & **Specificity** & **Accuracy** \\ \hline Basic CNN for weld-pool/bead & **0.995 * (\(\pm\)0.002)** & 0.933 (\(\pm\)0.033) & **0.985 * (\(\pm\)0.003)** & 0.958 (\(\pm\)0.016) \\ VGG16 & 0.982 (\(\pm\)0.010) & **0.977 * (\(\pm\)0.010)** & 0.953 (\(\pm\)0.033) & **0.965 * (\(\pm\)0.013)** \\ VGG16-PRETR & 0.869 (\(\pm\)0.010) & 0.940 (\(\pm\)0.007) & 0.641 (\(\pm\)0.027) & 0.794 (\(\pm\)0.013) \\ VGG16-PRETR-FINETUNE & 0.910 (\(\pm\)0.051) & 0.961 (\(\pm\)0.038) & 0.803 (\(\pm\)0.099) & 0.884 (\(\pm\)0.039) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance of classification of abnormality in weld-pool/bead. Figure 12: Mean ROC curves of each CNN model for detecting abnormality in weld-pool/bead. Based on Figure 13, for the highest accuracy obtained by VGG16, a red area appears on the top of the bead in the true normal case. In the true abnormal case, the red area appears on the curved edge of the bead. VGG16 appears to have classified the bead shape with the edge feature information. In the basic CNN for the weld-pool/bead with the highest AUC, a large red area appears at the bottom of the bead in the true normal case. The partial red area appears above the bead, as a small area. The bead pattern is observed as the main feature information. In the true abnormal case, a red area appears on the outside of the bead edge. The surrounding background, not the bead, appears to be the main characteristic information used for the classification. For VGG16-PRETR-FINETUNE, some areas appear similar to the basic CNN for the weld-pool/bead and VGG16. However, no consistent feature information is available in the two test images between the normal and abnormal cases. For VGG16-PRETR, the red area is randomly distributed. It appears that feature learning does not perform as intended. In fact, VGG16-PRETR has the lowest AUC, accuracy, and specificity. Based on the results of Table 3, Figures 12 and 13, a high-performance model (VGG16) can be selected for all measurements (AUC, sensitivity, specificity, accuracy, and feature information visualization). In Section 4.2, the effectiveness of fine-tuning in classifying specific images is described. Originally, the weights of VGG16 trained by the ImageNet data are designed to predict 1000 classes.