# A Real-Time Detection of Pilot Workload Using Low-Interference Devices
## 5 Results and Discussion  
### Model Performance  
By employing KNN to identify the physiological feature clusters for appropriate ensemble learning methods, the proposed model demonstrates superior accuracy, real-time performance, and cross-driver generalizability in workload recognition. The model achieves an accuracy of 82.6% on the testing set, with a runtime of only 0.1 s, and maintains a standard deviation of less than 2% across validation sets, indicating high stability.  
From Figure 8, it is shown that the AKE-PLD achieves higher prediction accuracy on new individuals compared to other models. The introduction of the ensemble strategy has improved the model accuracy by about 11% compared to the single-model approach. Moreover, utilizing KNN to identify physiological features and selecting the appropriate ensemble model has further enhanced model accuracy by 21% compared to the original ensemble learner. This demonstrates the accuracy and generalizability of the model, proving the effectiveness of the proposed model's structure.  
The classification performance was evaluated using accuracy, precision, recall, and F1 score, which are defined by the following formulae:  
\[precision=\frac{TP}{TP+FP},\]  
\[recall=\frac{TP}{TP+FN},\]\[F1=\frac{2\times\textit{precision}\times\textit{recall}}{\textit{ precision}+\textit{recall}}.\]  
Among them,  
TP (true positives): the number of correctly predicted positive samples;  
TN (true negatives): the number of correctly predicted negative samples;  
FP (false positives): the number of negative samples incorrectly predicted as positive;  
FN (false negatives): the number of positive samples incorrectly predicted as negative.  
Table 5 details the model performance on three workload levels (Level 1 = low; Level 2 = moderate; and Level 3 = high). Level 1 exhibits excellent performance across all three metrics. Metrics of Level 2 show consistency, but are slightly lower compared to Level 1, suggesting that the model's recognition ability for moderate workload is somewhat inferior. Level 3 has high precision but a lower recall rate, meaning that some of the samples with high workload were not identified by the model.  
Figure 9 displays the model's confusion matrix. Label 1 shows excellent predictive accuracy at 94%. Labels 2 and 3 have lower recall rates of 79% and 75%, each exhibiting some degree of confusion with adjacent labels, suggesting potential areas for improvement.  
Figure 10 shows the ROC curves generated individually for the three workload levels by treating each one as the positive class while considering the remaining classes as negatives. The overall curves are close to the upper left corner, with the areas under the curve (AUC) for the three classes being 0.94, 0.82, and 0.85, respectively. Level 1 still demonstrates excellent discriminative ability, while Levels 2 and 3 are slightly inferior, with Level 3 performing marginally better than Level 2.  
Based on the above results, it can be seen that the predictive performance of the model across the three categories of workload is generally excellent. There is still room for further improvement at higher levels of workload. This may be due to the increased noise in signal collection caused by more intense driving maneuvers under high workload, as well as the greater variability in physiological responses among different individuals at high workload levels.  
Figure 8: The results of ablation experiments.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Workload Level** & **Precision** & **Recall** & **F1-Score** \\ \hline Level 1 & 0.8557 & 0.9431 & 0.8973 \\ Level 2 & 0.7863 & 0.7931 & 0.7897 \\ Level 3 & 0.8493 & 0.7470 & 0.7949 \\ Macro Average & 0.8304 & 0.8278 & 0.8273 \\ Weighted Average & 0.8258 & 0.8258 & 0.8242 \\ \hline \hline \end{tabular}
\end{table}
Table 5: The performance of the three labels in terms of precision, recall, and F1 score.