## 4 Experimental Results
### Performance on Disturbance Defect Dataset
#### 4.5.2 In-Depth Analysis the Impact of Various Disturbance Factors  
To provide an outlook of the influence of each disturbance factor, the proposed method's confusion matrix on different interference datasets was provided in this experiment. From the confusion matrix, the first row represented the ground truth of the defect category, and the value in each column indicates the prediction result of the proposed method. First, the inclusion achieved the lowest recall (79.60%) and precision (74.62%) among the six kinds of defects based on Table 7. In the variance 0.3 subset, 5.64% inclusion \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{5}{c}{**Accuracy (\%)**} \\ \cline{2-6}  & **Original** & **Length 5** & **Length 10** & **Length 15** & **Length 20** \\ \hline AlexNet & - & 97.61 & 96.17 & 95.12 & 93.57 \\ VGG16 & - & 98.55 & 95.93 & 95.54 & 94.33 \\ ResNet-18 & - & 98.55 & 96.57 & 95.15 & 94.31 \\ Level 0 * & 99.30 & 68.02 & 53.81 & 45.05 & 40.65 \\ Level 1 * & 99.03 & 94.03 & 74.85 & 63.31 & 56.19 \\ MRFFN * & 99.61 & 82.59 & 67.76 & 58.01 & 51.87 \\ MRFFN + IA * & - & 79.35 & 67.39 & 58.94 & 53.23 \\ Level 0 & - & 98.83 & 97.41 & 96.59 & 95.43 \\ Level 1 & - & 98.64 & 97.51 & 96.78 & 95.50 \\ MRFFN & - & 99.11 & 98.17 & 97.51 & 96.40 \\ MRFFN + IA & - & 99.28 & 98.91 & 98.11 & 97.43 \\ \hline \hline \end{tabular}  
* denotes the model was trained by original dataset.  
\end{table} 
Table 6: The performance of the proposed method on motion blur. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{5}{c}{**Accuracy (\%)**} \\ \cline{2-5}  & **Original** & **Density 0.01** & **Density 0.05** & **Density 0.1** & **Density 0.3** \\ \hline AlexNet & - & 96.80 & 93.99 & 90.69 & 75.48 \\ VGG16 & - & 97.15 & 95.49 & 91.36 & 69.94 \\ ResNet-18 & - & 96.74 & 94.31 & 91.08 & 72.16 \\ Level 0 * & 99.30 & 77.43 & 51.58 & 37.92 & 21.09 \\ Level 1 * & 99.03 & 86.92 & 51.96 & 40.81 & 20.92 \\ MRFFN * & 99.61 & 84.63 & 53.29 & 39.69 & 20.18 \\ MRFFN + IA * & - & 83.25 & 50.65 & 39.53 & 18.73 \\ Level 0 & - & 98.83 & 95.76 & 92.52 & 79.57 \\ Level 1 & - & 97.85 & 96.26 & 94.04 & 86.99 \\ MRFFN & - & 98.87 & 97.01 & 94.66 & 87.09 \\ MRFFN + IA & - & 99.31 & 97.87 & 96.41 & 90.54 \\ \hline \hline \end{tabular}  
* denotes the model was trained by original dataset.  
\end{table} 
Table 5: The performance of the proposed method on salt and pepper noise. samples were misclassified as rolled-in scale and 13.64% rolled-in scale samples were wrongly classified as inclusion. In addition, 10.20% of inclusion samples were misclassified as pitted surface and 5.92% pitted surface samples were wrongly predicted as inclusion. According to Figure 7b,d, it can be discerned that the minority classes of rolled-in scale, inclusion, and pitted surface were mixed with each other. Thus, the similarity between inclusion, rolled-in scale, and pitted surface was exacerbated while applying Gaussian white noise to the raw data. Furthermore, 5.96% of crazing samples were categorized as rolled-in scale as there were \"inter-class\" similarity and \"intra-class\" diversity among the two defects [7]. In Table 8, the overall performance of the proposed method was as similar as the Gaussian white noise dataset. Still, the inclusion achieved the lowest recall (82.00%) and precision (79.74%) results. In addition, the proposed method was influenced by the \"inter-class\" similarity and \"intra-class\" between crazing and rolled-in scale defects in the salt and pepper noise subset. Based on the t-SNE maps in Figure 8b,d, some nodes of inclusion and rolled-in scale, inclusion and pitted surface are very close. Hence, these results lead to misclassifying the MRFFN among the inclusion, pitted surface, and rolled-in scale defects while applying salt and pepper noise. Lastly, the results from Table 9 demonstrate that the models can easily predict the crazing, patches and rolled-in scale defects since these defects achieve high recall.