# Acquisition and Processing of UAV Fault Data Based on Time Line Modeling Method
## 5 Experimentation
### Performance on Convolutional Neural Networks  
In order to test the performance of the data set obtained by the TLM method on the neural network, the information of GPS, IMU, RATE, and VIBE were extracted and input into the CNN for training. This is done by converting the data with 16-dimensional features \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Normal** & **GPS** & **Acc** & **Engine** & **RC** & **Class** \\ \hline
847 & 3 & 56 & 35 & 4 & Normal \\
0 & 193 & 0 & 0 & 0 & GPS \\
**112** & 0 & 293 & 17 & 23 & Acc \\
2 & 0 & 0 & 125 & 0 & Engine \\
31 & 0 & 0 & 31 & 201 & RC \\ \hline \hline \end{tabular}
\end{table} 
Table 13: Confusion matrix of random forest algorithm. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & **Correct Classified\%** & **Incorrect Classified\%** & **TP** & **FP** & **Precision** & **Recall** & **F1** \\ \hline Naïve Bayes & 66.95\% & 33.04\% & 0.670 & 0.205 & 0.758 & 0.670 & 0.646 \\ Logistics & 63.76\% & 36.24\% & 0.638 & 0.265 & 0.696 & 0.638 & 0.559 \\ SMO & 62.75\% & 37.25\% & 0.627 & 0.275 & NaN & 0.627 & NaN \\ KNN & 70.65\% & 29.34\% & 0.707 & 0.122 & 0.733 & 0.707 & 0.709 \\ AdaBoost & 57.68\% & 42.32\% & 0.577 & 0.389 & NaN & 0.577 & NaN \\ OneR & 55.04\% & 44.96\% & 0.550 & 0.287 & 0.497 & 0.550 & 0.511 \\ ZeroR & 47.89\% & 52.10\% & 0.479 & 0.479 & NaN & 0.479 & NaN \\ J48 & 77.49\% & 22.50\% & 0.775 & 0.121 & 0.787 & 0.775 & 0.774 \\ RandomForest & **84.08\%** & 15.91\% & 0.841 & 0.081 & 0.851 & 0.841 & 0.840 \\ RandomTree & 75.82\% & 24.17\% & 0.758 & 0.126 & 0.766 & 0.758 & 0.753 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Evaluation of multi-classification algorithms using GPS data. Figure 21: (**a**) Training accuracy and loss change of GPS. (**b**) Training accuracy and loss change of VIBE. From Figure 20a,b, it can be seen that the accuracy of IMU and RATE only converged to 75% during training; from Figure 21a, the accuracy of GPS information converged to about 85% during training. From Figure 21b, the accuracy of VIBE information converged to 63% during training. The test accuracies on the test set are shown in Table 14. It can be seen that the GPS data had the highest classification accuracy of 85.63%. Comparing other machine learning algorithms with CNN, the accuracy of CNN did not have a significant advantage on this task. Through analysis, the gap between normal and abnormal samples was too small; after converting the data to images, the differences between positive and negative samples were further reduced, making the learning of CNN using images difficult. In this paper, two samples from different failures were selected to illustrate the above issues, and their grayscale plots are shown in Figure 22.  
Tables 15-18 show the classification results of the different data obtained from UAVs on CNN, Normal, GPS, Acc, Engine, and RC, referring to the four types of fault classification and normal classification, respectively. In Table 13, which shows the results of testing using the GPS data, the highest classification accuracy of 96.32% was achieved for GPS faults. That is because the differences between the individual features in the GPS information are larger, so it is beneficial to the training of the model. In Table 16, which shows the classification effect using VIBE vibration data, the classification accuracy for Acc fault was 0. By analysis, it was found that in VIBE information, there were many samples duplicated at the Acc fault; there is no change in vibration information at the accelerometer fault, so the features were not learned. The classification effect of the remaining information was between 60% and 80%. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Category** & **Accuracy** & **Recall** \\ \hline GPS data & 85.63\% \\ RATE data & 75.32\% \\ IMU data & 73.03\% \\ VIBE data & 63.15\% \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Test results from CNN.