# Situational Awareness Classification Based on EEG Signals and Spiking Neural Network
## 4 Experimental Results
### Simulation Results  
Table 1 shows the classification accuracy results when using an open-source classification tool for six different classification models [46]. The performance of the classifiers was evaluated for both the SCTN-based and FFT features. The FFT features were extracted for  
various frame durations (for 5, 10, 15, and 20 ms). The best feature set which resulted in the best classification was selected for comparison. The results show that EEG classification, based on our proposed approach (SCTN), outperformed the FFT-based classification for all five models and demonstrated an accuracy improvement of up to 6.9%. We employed two evaluation methods for situational awareness detection: subject-specific and group-level classification. For the subject-specific classification, the classification network was trained individually for each participant using only their data. For this, 80% of the random samples from the specific subject's EEG data were used for training. The remaining 20% of the subject data which were not used for training were used for testing. Similarly, for the group-level classification, an 80-20 data splitting was also used. For this scenario, the training data set included combined data from all participants randomly selected from 80% of the joint EEG recordings of all participants. The testing was carried out on the remaining 20% mixed dataset, which was not used during the training phase. To further evaluate the SCTN-based classifier, we compared our proposed approach with previous related works applied to EEG classification. Table 2 summarizes the results of four previous studies on detecting mental attention states as described in [24] compared to the results of our proposed SCTN-based classifier. As seen in the table, our method demonstrated better results than all previous studies which used the SVM classifier. Average classification accuracies of 96.8% and 91.7% were achieved for classifying the three mental states while using our method (SCTN) and the SVM method [24], respectively. The proposed architecture was evaluated for individual training for each participant and a common-subject paradigm where a single generic classifier was jointly trained for all participants. The results show that the generic mental state detector performed slightly worse than the subject-specific detectors, demonstrating accuracy degradation of about 4-6% compared with training with individual subjects. The good generalization ability was achieved due to the proposed feature extraction method and the EEG feature mapping, which successfully mitigated the differences between users and highlighted relevant characteristics. The performance evaluation is demonstrated for each subject for both the subject-specific model and the transfer learning and generic model. Table 3 depicts the results of applying a transfer learning (TL) approach (training on one subject and testing on another) for the five participants (**P0-P4**). Additionally, the table also provides the results of the generic model (training with samples from all participants and testing on a specific participant). \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Model** & **Frame Duration (ms)** & **FFT** & **SCTN** \\ \hline Weighted Ensemble & 10 & 90.10\% & **93.86\%** \\ SVM & 20 & 91.51\% & **93.18\%** \\ XGBoost & 10 & 88.82\% & **92.64\%** \\ Random Forest & 5 & 86.81\% & **92.46\%** \\ LightGBM & 15 & 89.42\% & **91.96\%** \\ KNN & 20 & 63.77\% & **68.98\%** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Average classification results. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Reference** & **Mental States** & **Accuracy** \\ \hline [PERSON] and [PERSON] [47] & 3 (fatigue, frustration, attention) & 84.8\% \\ [PERSON] and [PERSON] [48] & 2 (attentive or inattentive) & 92.8\% \\ [PERSON] and [PERSON] [49] & 2 (attentive or inattentive) & 93.33\% \\ [PERSON] et al. [24] & 3 (focused, unfocused, drowsy) & 91.72\% \\
**Our Method** & **3 (focused, unfocused, drowsy)** & **96.8\%** \\ \hline \hline \end{tabular}
\end{table}