# Brain Activity Recognition Method Based on Attention-Based RNN Mode
## 3 The Proposed Method
### Attention-Based RNN  
The decomposed EEG data are inputted into an RNN model that introduces an attention mechanism to obtain more information about the subject's brain activity. The general encoder-decoder RNN ignores the importance of the feature dimension to the output sequence and defaults to the weight of entire feature dimensions of input sequence. Feature scales of EEG data are matched with the EEG device's sensor nodes. In order to assign different weights to different dimensions of EEG data, the attention mechanism is introduced into an encoder-decoder RNN model, which is composed of three parts: encoder module, attention mechanism module and decoder module. The encoder module should compress input delta-band data into over-encoded C. The attention module enables the encoder to compute a better intermediate code \(C_{a}\) through creating weight W at various dimensions. The decoder module illustrates attention-based code \(C_{a}\) into an EEG recognizable signal. Suppose that the data at the \(i\)-th layer are stated as \(X^{i}=X^{i}_{j^{\prime}}i\in[1,2,\ldots,I],j\in[1,2,\ldots,N^{i}]\) in which is denoted the \(j\)-th dimensional data of \(X^{i}\), I represents the number of neural network layers, and \(N_{i}\) represents the number of dimensions in \(X^{i}\).. At the first layer, in case of =\(\delta\), input sequence contains\(\delta\)-band data. Output sequence as represents the subject's current status, in which K is the number of subject state category. Operation \(f(\bullet)\) is defined as below:  
\[f(X^{i})=X^{i}*W+b \tag{2}\]  
\[f(x^{i-1}_{j},X^{i}_{j-1})=X^{i-1}_{j}*W^{{}^{\prime}}+X^{i-1}_{j}*W^{{}^{ \prime\prime}}+b^{{}^{\prime}} \tag{3}\]  
where \(W,b\), \(W^{\prime}\), \(W^{\prime\prime}\) denote the corresponding weights and deviation parameters. The encoder contains several nonrecursive fully connected neural network layers and one Long Short-Term memory (LSTM) layer. The nonrecursive layer is used to construct and fit a nonlinear function to filter the input \(\delta\)-band data. The data for non-recursive layers can be calculated as (4),  
\[X^{i}+1=f(X^{i}) \tag{4}\]  
The LSTM layer compresses the output of the nonrecurrent layer into a fixed-length sequence as the transfer code C. Assuming that the LSTM model is located in the \(i^{{}^{\prime}}\)-th layer of the encoder, the code C is the output of the LSTM model, \(C=X^{i^{\prime}}_{j}\), and \(X^{i^{\prime}}_{j}\)can be measured by  
\[X^{i^{\prime}}_{j}=T(c^{i^{\prime}}_{j-1},X^{i-1}_{j},X^{i^{\prime}}_{j-1}) \tag{5}\]  
where \(c^{i^{\prime}}_{j-1}\) denotes the hidden state of the \((j-1)\)-th LSTM cell, the operation \(T(\bullet)\) denotes the calculation of LSTM,which can be inferred from the following equations:  
\[X^{i^{\prime}}_{j}=f_{o}\odot tanh(c^{i^{\prime}}_{j}) \tag{6}\]  
\[c^{i^{\prime}}_{j}=f_{f}\odot c^{i^{\prime}}_{j-1}+f_{i}\odot f_{m} \tag{7}\]  
\[f_{o}=sigmoid(f(X^{i^{\prime}-1}_{j},X^{i^{\prime}-1}_{j-1})) \tag{8}\]  
\[f_{f}=sigmoid(f(X^{i^{\prime}-1}_{j},X^{i^{\prime}}_{j-1})) \tag{9}\]  
\[f_{i}=sigmoid(f(X^{i^{\prime}-1}_{j},X^{i^{\prime}}_{j-1})) \tag{10}\]\[f_{m}=tanh(f(X_{j}^{t^{\prime}-1},X_{j-1}^{t^{\prime}})) \tag{11}\]  
where represent the output gate, forget gate, input gate and input modulation gate, respectively, and \(\odot\) denotes the element multiplication. The attention-based module receives the state of the hidden layer as a nonstandardized attention weight, which can be measured by  
\[f(x^{i-1}_{j},X^{i}_{j-1})=X^{i-1}_{j}*W^{{}^{\prime}}+X^{i-1}_{j}*W^{{}^{ \prime\prime}}+b^{{}^{\prime}} \tag{3}\]0  
\(W_{a}^{t}\) is further normalized to obtain  
\[f(x^{i-1}_{j},X^{i}_{j-1})=X^{i-1}_{j}*W^{{}^{\prime}}+X^{i-1}_{j}*W^{{}^{ \prime\prime}}+b^{{}^{\prime}} \tag{3}\]1  
The softmax function normalizes attention weights within [0, 1]. Consequently, attention weight is considered as the probability estimate investigating if there is a correlation between code C and output. In attention mechanism, connection code C will be converted to \(C_{a}\) after being weighted.