## 4 Machine Learning Algorithms
### Probability of Detection  
This section shows the probability of detection (POD) of the SVM method. As the results in the previous section show, machine-learning models such as SVM, LG, RF, and KNN show good classification performance in the detection of corrosion and its location. Thus, it is feasible to evaluate the performance of the machine-learning models with a small amount of data and/or small corrosion sizes. Thus, in this section, we only use 20% of the dataset for training the SVM model and with the corrosion volumes in a small range of 15.0 to 25.4 mm\({}^{3}\) (P\({}_{2}\)). Figure 12 shows the classification results of the corrosion using SVM method. The SVM method provides good results on the unseen validation dataset (80% of the total dataset). The average accuracy is 89.48%, with the accuracy of no corrosion, backward, forward, left, and right corrosion being 99.17%, 89.56%, 85.42%, 79.17%, and 91.67%, respectively. The rivets without corrosion are the easiest to detect (99%), and the rivets with corrosion are harder. Among the corrosion, right corrosion is the easiest to detect with 92% accuracy. Figure 11: Validation accuracy of different machine-learning models with: (**a**) a different number of training datasets (ratio with entire dataset) in “increasing” dataset and (**b**) fixed 20% ratio at different corrosion volume ranges in “shifting” dataset. The forward corrosion has low accuracy, but its wrongly classified labels were on the left and right corrosion (10%), and only 6% (just after right corrosion with 4%) corrosion was incorrectly classified as no corrosion. The F1 score is a harmonic factor between precision and recall of the classification system. The F1 score is better for evaluating the performance of the model and its results for each class of corrosion, as shown in Figure 12b and Table 3. The F1 score for backward, forward, left, and right corrosion was 89.9%, 91.53%, 86.36%, and 93.62%, respectively. The F1 scores for right and forward corrosion are the highest values, which means that the right and forward corrosion are easier to detect than the other ones. The area under the curve (AUC) of the forward, backward, left, and right corrosion was 98.45%, 96.52%, 98.58%, and 95.88%, respectively, which are high values. The highest value is for the right corrosion. The AUC value tells us that the right corrosion is the easiest to detect by the model. The comparison of the classification accuracy, F1 score, and AUC of the SVM method with the previous study using \"sum of area\" feature (SoA) [22] is shown in Table 3. The SVM method significantly improves the classification results, which are 26.47%, 3.3%, and 10.64% of the accuracy, F1 score, and AUC, respectively. The ROC of the SVM and the SoA feature method are shown in Figure 13. It is observed that the SVM provides significant results compared to the SoA feature method. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Corrosion**} & \multicolumn{3}{c}{**SoA Method [22]**} & \multicolumn{3}{c}{**SVM Using 20\% Data for Training**} \\ \cline{2-7}  & **Accuracy** & **F1 Score** & **AUC** & **Accuracy** & **F1 Score** & **AUC** \\ \hline
**Backward** & 60.0 & 89.58 & 89.58 & 89.58 & 89.90 & 98.45 \\
**Forward** & 56.0 & 85.42 & 85.42 & 85.42 & 91.53 & 96.52 \\
**Right** & **64.0** & **91.67** & **91.67** & **91.67** & **93.62** & **98.58** \\
**Left** & 61.0 & 80.21 & 80.21 & 80.21 & 86.36 & 95.88 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Evaluation metrics and comparison of the proposed method with the SoA method [22]. Figure 12: Classification results using 20% data for training at a small volume range (P\({}_{2}\):15.0 to 25.4 mm\({}^{3}\)): (**a**) confusion matrix and (**b**) evaluation metrics of accuracy, F1 score, and AUC for each class. Figure 14 shows the POD curves of the SVM method compared to the SoA feature method. The parameters of the POD curves were calculated by mh1823 POD software V4.0.1 developed by [PERSON] [26] based on the MIL-HDBK methodology.