# A Cost-Sensitive Diagnosis Method Based on the Operation and Maintenance Data of UAV
## 4 Results and Discussion  
The aim of the experiment is to evaluate the effectiveness of the two-step ensemble cost-sensitive classifier, MC-LGB, in a multi-class UAV fault diagnosis with imbalanced data. Adapting to the method of the cost-matrix setting, with the fault criticality given in Section 2.2, the misdiagnosis cost matrix (Table 4) for cost-sensitive classification was computed first by the fault criticality values of the different states of the KPG component. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & & \multicolumn{6}{c}{**Actual State**} \\ \cline{3-8}  & & Normal & \(F_{1}\) & \(F_{2}\) & \(F_{3}\) & \(F_{4}\) & \(F_{5}\) \\ \hline \multirow{6}{*}{**Predicted State**} & Normal & 0 & 3.62 & 3.46 & 16.39 & 24.58 & 1.95 \\  & \(F_{1}\) & 3.46 & 0 & 0.15 & 12.77 & 20.97 & 1.60 \\ \cline{1-1}  & \(F_{2}\) & 3.31 & 0.16 & 0 & 12.93 & 21.12 & 1.45 \\ \cline{1-1}  & \(F_{3}\) & 15.67 & 12.21 & 12.36 & 0 & 8.19 & 13.81 \\ \cline{1-1}  & \(F_{4}\) & 23.51 & 20.05 & 20.20 & 7.84 & 0 & 21.65 \\ \cline{1-1}  & \(F_{5}\) & 1.86 & 1.67 & 1.52 & 14.44 & 22.64 & 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The misdiagnosed cost matrix of the different states of the KPG component. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Fault Mode** & **Number of Cases** & **Criticality** \\ \hline Normal & 1759 & 0 \\ \(F_{1}\) & 480 & 3.46 \\ \(F_{2}\) & 407 & 3.31 \\ \(F_{3}\) & 283 & 15.672 \\ \(F_{4}\) & 113 & 23.508 \\ \(F_{5}\) & 157 & 1.86 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The distribution and criticality of the different states of the KPG component. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Classifier** & **Num. Leaves** & **Max. Depth** & **Subsample** & **Random State** & **Learning Rate** \\ \hline XGBoost & 64 & 7 & 0.9 & 42 & 0.001 \\ LightGBM & 56 & 6 & 0.9 & 42 & 0.001 \\ \hline
**Classifier** & **Optimizer** & **Activation Function 1** & **Activation Function 2** & **Iterations** & **Learning Rate** \\ \hline CNN & Adam & RReLU & Softmax & 15 & 0.001 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The setting of the main hyperparameters of the control group in the experiment. The calculated misdiagnosis cost matrix above was used to construct the proposed two-step cost-sensitive ensemble diagnosis model (MC-LGB). The details of each metric of the proposed method compared with that of the control group of diagnosis models and verified by actual UAV fault data, are shown in Table 5.  
In terms of accuracy and precision (with the exception of the FCNN model's precision, which was below 0.8), all methods achieved a score that was greater than 0.82. Figure 2a-d show the comparison between the traditional classification performance metrics on the KPG dataset. It can be seen that XGBoost is the model with the strongest overall diagnostic ability on the KPG dataset (it has the highest accuracy, recall, and F1-score), with negligible disparities among the classifiers. Surprisingly, LightGBM, with insufficient diagnosis ability on the entire UAV, exhibited an excellent performance on the KPG dataset, especially in terms of precision. On these metrics, the proposed MC-LGB model showed no significant loss, which proves that our effort to enhance the overall diagnosis ability of the MetaCost framework was effective. Figure 2: (**a**) Accuracy of classifiers on the KPG dataset; (**b**) precision of classifiers on the KPG dataset; (**c**) recall of classifiers on the KPG dataset; and (**d**) F1-Score of classifiers on the KPG dataset.