## 5 Analysis and Result
### Machine Learning Analysis
However, in the case of the bagging method, the boundary is divided by using only alumina damage data itself. Therefore, it is judged that the bagging method establishes the most accurate area in the analysis using the data of this study. Figure 13: Heatmap chart of prediction results using developed machine learning classifiers: (**a**) the heat map result of the SVM classifier; (**b**) the heat map result of the bagging classifier; (**c**) the heat map result of the kNN classifier; (**d**) the heat map result of the DA classifier. In the model development using ML, the accuracy analysis and the F1 score analysis were finally performed for the prediction result of the developed model. In general, the ratio between normal and defect data is similar, and when analyzing two classes, the performance of the model is evaluated using accuracy in the confusion matrix. However, the sample data in this study has significantly less defect data than normal data, and has four classes. Therefore, because it was difficult to accurately evaluate the reliability of the model on the basis of only accuracy, the F1 score was also analyzed. The results of the reliability evaluation and performance of the developed model are presented in Table 2. For each class, the true positive (TP) is the correct prediction for the right class, the false positive (FP) is the incorrect prediction for the right class, and the false negative (FN) is the correct prediction for the wrong class. Precision can be calculated using Equation (5) for TP and FP in each class. Recall can be calculated using Equation (6) for TP and FN in each class. Then, the precision and recall average can be obtained using Equations (7) and (8), and the F1 score can be calculated using Equation (9). The F1 Score is the harmonic mean of recall and precision for evaluating the performance of multiple classification models. \[Precision=TP/(TP+FP) \tag{5}\]  
\[Recall=TP/(TP+FN) \tag{6}\]  
\[Average\ Precision=\{P(A)+P(AD)+P(C)+P(CD)\}/4 \tag{7}\]  
\[Average\ Recall=\{R(A)+R(AD)+R(C)+R(CD)\}/4 \tag{8}\]  
\[\text{F1 Score}=2\times\frac{Average\ Precision\times Average\ Recall}{Average\ Precision+Average\ Recall} \tag{9}\]  
As a result of the analysis of the developed model, the bagging method took about two-three times longer to develop the model than the other three models, and it was determined to be the most reliable model, exhibiting an accuracy of 95.45% and F1 score of 96.88%. The other three models showed similar results in terms of accuracy and F1 score. Because a clear feature was derived at the early stage of feature extraction, a model suitable for classifying normal caps and defects was developed. However, additional analysis was conducted against data overfitting. For SVM, ensemble, kNN, and DA models, the five-fold cross validation and error were analyzed using 'crossval' and 'kfoldLoss' functions in MATLAB. Accuracy of the ensemble and kNN models was reduced by up to 2% compared to the F1 score in Table 2, and the accuracy of the SVM and DA models was reduced by up to 1%. The over-fitting problem between normal cristobalite and alumina data was alleviated and data were closer to the generalized model. Most of the material classification is performed, but there are some overlapping portions at the interface. This may be an error that occurs because the insulator is fabricated by hand. In addition, there is a material classification in damage, but the FRF waveform due to damage is so variable that it may even be at the interface. Moreover, the data distributed at some interfaces and the data distributed at the center of the normal data are reanalyzed in Figure 14. In Figure 14a, the interface data and unusual values are sampled. The PC1 and PC2 values of the selected A1-3 and C1-4 data are shown in Figure 14b. FRF results for normal and interface data and normal and unusual data are plotted in Figure 14c,d.