# A Fusion Framework for Confusion Analysis in Learning Based on EEG Signals
## 4 Experiments and Discussions
### Analysis of Results  
Tables 3 and 4 present the experimental results of different methods in subject-dependent and subject-independent experiments on the CAL dataset. The binary classification results of the subject-dependent experiments demonstrate that our approach significantly improves accuracy by 33.19%, 23.06%, and 20.77%, respectively, compared to conventional machine learning methods (Naive Bayes, SVM, and Random Forest). Additionally, we can observe that end-to-end deep-learning methods based on CNN, ResNet, and EEGNet perform well (with accuracies of 80.61% and 72.02%, respectively), indicating strong feature extraction capabilities of CNN-based methods. However, due to the limited receptive field of CNN, it struggles to capture global features. In contrast, the Transformer architecture based on the self-attention mechanism excels at capturing global information. Experimental results further confirm the advantages of the Transformer architecture: our method, based on Transformer, achieves an average accuracy increase of 18.47% and 9.88% compared to the CNN-based EEGNet and ResNet. Furthermore, we can observe that the end-to-end LSTM network performs impressively in the binary classification tasks of the subject-dependent experiments, achieving an accuracy of 73.45%. However, when faced with ultra-long time series data such as physiological signals, the end-to-end LSTM still loses information during training, leading to the inability to capture global features. In contrast, our Transformer-based approach can capture global \begin{table}
\begin{tabular}{l c c} \hline \hline
**Methods** & **Binary Classification** & **Four Classification** \\ \hline Naive Bayes w/ PSD & 60.59/0.55 & 40.83/0.27 \\ SVM w/ PSD & 55.71/0.53 & 38.05/0.31 \\ Random Forest w/ PSD & 55.98/0.52 & 35.94/0.25 \\ \hline EEGNet & 64.46/0.61 & 39.14/0.20 \\ LSTM & 61.25/0.59 & 40.53/0.24 \\ ResNet & 57.95/0.55 & 40.05/0.23 \\ \hline Our method w/o PSD & 65.85/0.63 & 40.88/0.39 \\ Our method & **66.08/0.65** & **41.28/0.41** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Subject-independent experiment of different methods on CAL dataset (Acc/F1 score). “w/” for “with” and “w/o” for “without”. The best results are highlighted in **bold**. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Methods** & **Binary Classification** & **Four Classification** \\ \hline Naive Bayes w/ PSD & 57.30/0.52 & 69.72/0.69 \\ SVM w/ PSD & 67.43/0.67 & 48.10/0.33 \\ Random Forest w/ PSD & 69.72/0.69 & 37.72/0.26 \\ \hline EEGNet & 72.02/0.71 & 49.81/0.43 \\ LSTM & 73.45/0.73 & 53.29/0.49 \\ ResNet & 80.61/0.80 & 73.10/0.73 \\ \hline Our method w/o PSD & 85.53/0.85 & 85.99/0.86 \\ Our method & **90.49/0.90** & **87.59/0.87** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Subject-dependent experiment of different methods on CAL dataset (Acc/F1 score). “w/” for “with” and “w/o” for “without”. The best results are highlighted in **bold**. temporal context information, resulting in a 17.04% improvement in accuracy compared to the end-to-end LSTM method. In the subject-dependent experiments for the four-classification task, our method also exhibits strong performance with an accuracy of 87.59%. Compared to the best-performing traditional machine learning method, Naive Bayes, our method achieves an improvement of 17.87% in accuracy. Additionally, compared to the top-performing end-to-end method, ResNet, our method shows an accuracy improvement of 14.49%. Moreover, in subject-independent experiments, our approach achieves the highest accuracy of 68.08% and 41.74% in binary and four-class classifications, respectively. Compared to other methods, our approach achieves an accuracy improvement of 1.62% in binary classification and 0.45% in four-class classification. The F1 score, which is based on precision and recall, is also an important evaluation metric [41]. We present the F1 score of the binary and four-class classification tasks in Figures 3 and 4. Our approach achieved F1 scores of 0.90, 0.87, 0.65, and 0.41 in the experiments.