## 2 Background  
Traditional machine learning methods require handcrafted feature extractors and a classification network. In terms of extracting the spatial features, a common spatial pattern (CSP) [10] and an extended version of CSP, which is a filter bank CSP (FBCSP) [11], are the most common methods. Moreover, short-time Fourier transform (STFT) [12] and continuous wavelet transform (CWT) [13] are used to extract the time-frequency features. Additionally, the feature extractors must be combined with classification, such as support vector machine (SVM) [14], Bayesian classifier [11], and linear discriminant analysis (LDA) [15], to classify the target tasks. However, EEG signals are non-stationary and have a low signal-to-noise ratio, so deep learning methods are researched to improve the weaknesses of traditional machine learning methods.  
Regarding traditional deep learning methods, convolutional neural network (CNN) methods are the most common methods that achieve various state-of-the-art studies. [PERSON] proposed EEGNet [16], which is applied to multiple BCI paradigms, including movement-related cortical potentials (MRCPs), error-related negativity responses (ERNs), sensory-motor rhythms (SMRs), and P300 visual-evoked potentials. ShallowNet, proposed by [PERSON] (2017) [17], measures the performance of different deep levels in convolution networks, from 5 convolution layers to 31 convolution layer residual networks, using ResNet architecture. EEG-TCNet [18] incorporates convolution layers in EEGNet and temporal convolutional networks (TCNs) [19], which can exponentially increase the receptive field size by using dilated causal convolution [20, 21]. Moreover, recurrent neural networks (RNNs), gated recurrent unit (GRU), and long short-term memory (LSTM) are combined with CNN methods to learn long sequence data [22].  
The transformer method is an advanced method that allows the learning of features in parallel and improves the learning of long sequence dependency [23]. Because of the structure of multi-head attention, the transformer method can efficiently capture the relationships in features through attention mechanisms. [PERSON] proposed a Transformer-based one-dimensional convolutional neural network model (CNN-Transformer), which uses 1D-CNN to extract low-level time-frequency features and a transformer module to extract higher-level features [24]. Hybrid CNN-Transformer [25] was proposed as a complex architecture with spatial Transformer and spectral Transformer to encode raw EEG signals to an attention map, and then utilise CNN to explore the local features and temporal Transformer to perceive global temporal features. EEG-Conformer [26], proposed by [PERSON], is a lightweight model with a shallow convolution to extract temporal and spatial features and self-attention mechanisms to learn the relationships in the time domain.  
This study aims to research the contributions of different hyperparameters and different bandpass filter parameters to the CNN method and the Transformer method in MI-BCI. EEG-TCNet and EEG-Conformer were selected for this study. Therefore, there are two sub-projects, as follows:  
* First sub-project: EEG-TCNet and EEG-Conformer are trained and validated with different hyperparameters (i.e., filters, kernel sizes).
* Second sub-project: EEG signals are preprocessed with a bandpass filter, then input in training and testing EEG-TCNet and EEG-Conformer.  
In this research, a new model is introduced, EEG-TCNTransformer, inspired by EEG-TCNet and EEG-Conformer. EEG-TCNTransformer has the advantage of learning low-level temporal features and spatial features by the novel convolution architecture from EEG-TCNet and extracting the global temporal correlation from the self-attention mechanism. Moreover, EEG-TCNTransformer uses raw EEG data, so the bandpass filter is omitted.