## 4 Flight Test Results and Discussion
### Random Forest Analysis
#### 4.4.2 Detection Warning Analysis  
The random forest classifier algorithm was used to analyze the detection warning parameters because the algorithm was suitable for classifying discrete parameters. The result of the analysis is plotted in Figure 17, with the top four important parameters being the process time (42.95%), the distance between UAVs (31.28%), the interval time (9.74%), and the relative height between UAVs (3.9%). Meanwhile, the other parameters had a less than 2.5% effect on the detection warning.  
Figure 16: Uncertainty plot of the regression model.  
Figure 17: Important parameters for detection warnings.  
The number of estimators or trees used was 1000 because this number could produce good validation metrics that represented the performance of the model. The results of the validation metrics values were MSE = 0.0226, RMSE = 0.1503, MAE = 0.0226, and R-squared = 0.8905, which indicates that the model had a good fit to the data or high validation. Moreover, the model provided an accuracy score of 0.9774, which means that the correctness of the model prediction was 97.74%.  
Another important performance parameter of the random forest classifier is a confusion matrix. This matrix represents a measurement of how many of a classifier's predictions were correct and incorrect [37]. The resulting confusion matrix for the detection warning is shown in Figure 18.  
The matrix's main diagonal cells represent the correctness of the prediction and other matrix cells represent the incorrectness. Since the matrix's diagonal cells had very high numbers compared with other cells, this indicates that the resulting classifier model had high accuracy in predicting the result.  
For the uncertainty of the prediction model, an error bar plot of the predicted warning values is shown in Figure 19. The length of the error bars indicates the magnitude of the uncertainty.  
Since warning 2 had the shortest error bar, it had the lowest uncertainty confidence level. Meanwhile, warning 0 had the longest error bar, which represents the highest uncertainty. This means that the model's predictions for value 2 were more reliable and consistent, while its predictions for value 0 were less reliable and more variable. However, the warning 1 error bar length was somewhere between warning 2 and warning 0. This means that the model's predictions for warning 1 had a moderate level of uncertainty or variability.  
The presence of an outlier data point at a high instance within the error bar of predicted value 1 suggests that the model encountered a particular instance where its prediction for  
Figure 19: Uncertainty plot of the classifier model.  
Figure 18: Confusion matrix of the detection warning.  
value 1 deviated significantly from the expected trend or pattern. This outlier data point might be related to a few outlier data in warning 1, as shown in Figure 14.  
In comparison with the result from Figure 14 in Section 4.3, the relative distance and relative height were two of the dominant parameters for the resulting warning level. However, the result from the machine learning technique mentioned that the process time was the most significant parameter, and the interval time was in third position.  
This result is not a surprise since the UTM monitoring conducts more calculations when a conflict was detected. Furthermore, when an interval time is large, the possibility of a higher warning level increases. Thus, this result highlights the advantage of machine learning analysis for complex and large datasets compared with classical statistical analysis. For the future development of the UTM monitoring application, the use of a higher computational capability and more efficient coding is highly recommended to improve the process time and interval time.