# Mental Workload Assessment Using Machine Learning Techniques Based on EEG and Eye Tracking Data
## 4 Results and Discussion
### Classification Results  
Different algorithms such as kNN, SVM, ANN, RF, GBM, LightGBM, and XGBoost were utilized for modeling purposes. Table 1 presents the classification performance measures of 111-variable, 4-class classification models performed by using various algorithms. The values provided represent the average of five distinct performance outcomes achieved through five-fold cross validation. Among the models evaluated, the LightGBM algorithm stood out, with the highest accuracy performance of 71.96%. Nevertheless, XGBoost, SVM, and GBM also showcased commendable results. The performance of models using only EEG or only eye variables was investigated using the LightGBM algorithm. The findings revealed that the model incorporating both EEG and eye variables yielded superior results, as anticipated, compared to the models that utilized them separately. Notably, the model solely based on eye variables exhibited better performance (accuracy: 65.67%) than the model solely based on EEG variables (accuracy: 56.15%). Upon examining the classification performances of the five models obtained through cross-validation, it was observed that the model mostly made incorrect predictions for observations belonging to the 1-coded class. Additionally, the misclassification predictions were predominantly observed between classes 0-1 and 2-3. To investigate the impact of reducing the complexity of the problem by reducing the number of classes, the problem was also analyzed as a three-class and two-class scenario. Initially, the observations related to the class coded 1, which had the lowest sensitivity value, were removed from the dataset. The aim was to classify the mental workload as low-medium-high by considering only the observations for the 0-back, 2-back, and 3-back tasks. The accuracy of the three-class model, which included all variables, was determined to be 80.49%. In the two-class model, where only observations for the 0-back and 3-back tasks were considered, an accuracy performance of 89.63% was achieved. Notably, both the three-class and two-class models exhibited strong classifier characteristics, as indicated by Kappa and MCC values exceeding 0.70. We performed a dimension reduction procedure to improve both the classification performance and the comprehensibility of the model. In this study, the recursive feature elimination with cross validation (RFECV) method was employed for dimension reduction due to its superior reliability in feature selection and ability to identify the optimal stopping point. The RFECV method determined the optimal stopping point, leading to the selection of 34 features for the four-class model. Using the identical approach, the three-class \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy (\%)** & **Kappa** &
\begin{tabular}{c} **Matthews Correlation** \\ **Coefficient (MCC)** \\ \end{tabular} & **AUC** \\ \hline KNN & 54.85 & 0.40 & 0.40 & 0.80 \\ SVM & 62.15 & 0.49 & 0.50 & 0.85 \\ ANN & 59.00 & 0.45 & 0.45 & 0.82 \\ RF & 61.44 & 0.49 & 0.49 & 0.85 \\ GBM & 61.93 & 0.49 & 0.49 & 0.85 \\ XGBoost & 70.22 & 0.60 & 0.60 & 0.91 \\ LightGBM & 71.96 & 0.63 & 0.63 & 0.92 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance results of four-class models. Figure 6: Significant frequencies in band power depending on task difficulty. model identified 91 variables, while the two-class model identified 110 variables. Out of the 34 variables chosen for the four-class model, it was observed that 20 of them were derived from EEG data, while the remaining 14 variables were obtained through the eye tracking method. The classification performance exhibited a noticeable shift after implementing dimension reduction and adjusting hyper-parameters, as depicted in Figure 7. It was noted that the accuracy performance experienced a more significant change when a larger number of variables was eliminated, particularly in the four-class model.