# Dual Head and Dual Attention in Deep Learning for End-to-End EEG Motor Imagery Classification
## 1 Introduction
However, to the best of our knowledge, the cross-task and cross-subject classification is still challenging. Deep ConvNets [9] and EEGNet [23] can be applied to MI classification [17; 18; 24], P300 detection [25; 26], workload estimation [27; 28; 29], and error- or event-related potential decoding [30], and they become common approaches to learn the selective preprocessed handcraft data. The work in [9] followed a famous method of FBCSP [10] to construct the input data and then trained the data onto CNN with known training features. In particular, Ref. [24] improved performance via a semi-supervised contrastive learning framework with two different networks based on Deep ConvNets and EEGNet. However, the unknown features and/or structures for training need to be explored. Another popular approach is building 3D CNN structures with different sizes of receptive field [17]. Though the multi-branch structure performs better than one branch network, the computational cost grows proportionally with the number of the branches. This is a common problem with complex CNN models which result in more training time and worse real-time performance. In addition, LSTM and RNN also have the same disadvantage since they are slower and take up more memory than other normal activation functions, such as sigmoid, tanh, or rectified linear unit. To handle this problem, Conv1D is selected to learn the temporal features. Thus far, there are many papers based on temporal-spectral features for EEG classification, which have achieved good results, such as [27; 31]. However, the existing CNN-based classification methods depend on a single convolution computation, which limits the classification accuracy. In this work, we desire to exploit valuable intermedia learning signals to enforce the feature value. Two attention mechanisms for temporal and spatial learning are proposed to improve the accuracy of EEG MI classification. In this work, a dual-attention convolution network is proposed to handle subject-independent recognition of MI actions. First, the raw EEG data are filtered by bandpass filter. Then, the time serialized data are divided into segments of equal length. Finally, the dual blocks of deep learning in CNN structure are utilized to learn temporal and spatial-spectral EEG representations. To enhance the learning ability of the MI features and improve the accuracy of EEG MI classification, two attention mechanisms are utilized to enforce the temporal and spatial-spectral characteristics respectively. The block diagram of the proposed DAC-Net framework is shown in Figure 1.  
For a comparable analysis of the role of deep 1D and 2D design choices in EEG-MI decoding, three key questions are addressed:  
* What's the impact on the proposed attention strategy through end-to-end learning? * How to quantify or visualize the interpretability of deep DAC-Net? * How to accelerate the learning manifestation and effectiveness from the MI raw signals via DAC-Net for BMI recognition applications? The remainder of this paper is organized as follows: Section 2 introduces the experiment data. Section 3 describes the preprocessing of EEG signals and the overall architecture of DAC-Net. Section 4 presents experiment results and evaluates the performance of the proposed method. Section 5 describes the interference with data acquisition, as well as providing the visualization of the learned features. Finally, Section 6 concludes this paper.