# Decoding Visual Motions from EEG Using Attention-Based RNN
## 1 Introduction
When applying the attention mechanism to a time sequence, varied emphases are placed on different parts of it, according to their significance. The attention mechanism is also applicable for EEG analysis, because samples in an EEG sequence are differentially informative. Existing studies utilizing attention-based RNN [16; 19; 20] for EEG analysis apply attention mechanisms to the hidden states of RNN. However, in the RNN architecture, the hidden state is updated recurrently at each time step, so those hidden states near the end are expected to capture more information, which may result in a biased attentive weight towards the later coming samples in RNN [21]. Consequently, the attention weights cannot reveal how relevant a specific part of the raw EEG signal is for the prediction result. This defect is called 'attention bias'. In our framework, we add attention to the input EEG data before RNN computes the feature representation of EEG to avoid this problem. There are two advantages to this: First, attention can directly process the signal to suppress noise and emphasize task-relevant information in it, thereby promoting the neural network to extract more robust features of signals. Second, the weights learned by the attention mechanism can be visualized through a heatmap to increase the interpretability of the deep neural network. Using DL to analyze EEG data is more challenging than to analyze images and audio. This is mainly because of two reasons. First, the number of samples in most EEG datasets is limited [22]. When the model is too complex, such as containing parameters that are much larger than the number of training samples, overfitting may occur, which will weaken its generalization ability. Second, the inter-trial and the inter-subject variabilities also affect the generalization ability of neural networks, because they cause inconsistencies between training data and test data. Generalization performance can be improved through data augmentation [23]. Many data augmentation techniques such as random cropping, noise addition, and affine/rotational distortions and unsupervised generative models (e.g., conditional GANs) have been applied to EEG [24; 25; 26; 27]. The above methods increase the amount of EEG data, but do not consider the differences in SNR between trials. During the EEG acquisition process, the physical and mental state of the subject fluctuates all the time, resulting in varying degrees of artifacts and neural background activities. Therefore, EEG data have various SNRs in different trials, and this variation exists even at different time periods within a trial. Such inter-trial difference undermines models' generalization ability, leading to worse prediction accuracy. In this study, we propose a data augmentation approach that generates artificial signals by randomly averaging EEG data. In the EEG community, averaging is the most common method that changes SNR of EEG, based on the assumption that the task-unrelated brain activity is random noise [28]. Therefore, artificial signals with various SNR can be generated by averaging different numbers of EEG signals. The more SNR patterns included in the training data, the easier it is for the network to keep invariance to different levels of SNR in the test data. Notably, this data augmentation method is implemented during the training stage and, therefore, does not take up additional storage space. The multi-trial EEG classification problem is also considered in this study. When more emphasis is placed on decoding accuracy and less on the real-time requirement, the BCI system performs the averaging for the ensemble of several signals before classification. This is a common practice in the P300 and mVEP BCI paradigms. However, the average potentials are generated in a fixed combination fashion (i.e., each trial is used only once), thus, reducing the number of training examples [29]. This may affect the generalization ability of the DL model and fail to achieve a better classification rate.