# Decoding Visual Motions from EEG Using Attention-Based RNN
## 1 Introduction  
To build a direct communication pathway between the brain and environment, the brain-computer interface (BCI) based on electroencephalography (EEG) has been investigated for decades. It detects, analyzes, and decodes brain activities to translate them into commands for controlling external devices. The visual BCI that relies on external visual stimuli is a popular research direction in this field, owing to its robustness for different individuals and high information transfer rate (ITR) compared with motor imagery and spatial auditory BCI paradigms [1; 2]. The most commonly used stimuli for the visual BCI are the flickering light or flipping of a static image, which evoke EEG signals such as P300 event-related potential (ERP) and steady-state visual evoked potential (SSVEP) [3; 4; 5]. To survive in a dynamic visual world, the human visual system is naturally sensitive to motion. That is, a videostimulus with motion events should be more likely to elicit enhanced brain activities and, therefore, has promise to be used to develop more effective BCI systems. Although the perception of motion has been widely studied by neuroimaging and neurophysiology techniques such as fMRI, PET, and EEG [6; 7; 8], introducing it into the BCI system is a recent development. In some BCI systems for real-time monitoring and control, motion onset and periodic motion are utilized to evoke motion onset visual evoked potential (mVEP) and steady-state motion visual evoked potential (SSMVEP) signals transferring user intentions [9; 10]. Surprisingly, most of the existing mVEP and SSMVEP paradigms only involve one motion mode. Although a few studies used contraction-expansion as stimulus [11; 12; 13], the contraction-expansion is taken as a periodic motion stimulus to evoke SSMVEP rather than two different motions. The visual responses for different motion types are not well utilized to increase the number of control commands for BCI. On the other hand, in some special BCI systems for motion event detection, preference identification and emotion recognition, dynamic videos depicting more motions are chosen as stimuli. However, all these works require long video clips (lasting for more than 5 s), which is not suitable for real-time applications. To overcome the above deficiencies, in this study, short video clips (lasting for 1 s) illustrating four kinds of visual motions (contraction, expansion, rotation, and translation) are selected as stimuli. Previous research [8] demonstrates that the type of motion (translation, contraction, expansion, and rotation) affects the temporal and spatial features of EEG signals. They used grand average potentials obtained in the four motion types and source location analysis to reveal the differences among motions. However, the feasibility of decoding the type of above motions from single-trial EEG has not been investigated. The classification accuracy of EEG signals is a key factor affecting the performance of BCI systems. EEG collected from the scalp is affected by brain background activity and artifacts like eyes blinks and electrooculogram, so its signal-to-noise ratio (SNR) is very low. Therefore, it is still challenging to classify EEG signals, especially when BCI uses stimuli with complex patterns. In the last few years, deep learning (DL) methods have been demonstrated to be effective in extracting motion-related events from the EEG [14; 15; 16]. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are the two most frequently applied DL models. RNNs are capable of capturing long-term dependencies in time sequences, and the context correlation of EEG reflected the type of stimulus [17; 18]. In this study, we introduce attention mechanism into RNN as the decoding model. Attention mechanism has been widely used in natural language processing (NLP), based on the fact that words in a source sentence are not equally important to the target task.