# Maximum Marginal Approach on EEG Signal Preprocessing for Emotion Detection
## 4 BiLSTM Network  
BiLSTM is utilized to detect emotions by using the selected features, which constructs a bi-direction recurrent network by using LSTM units [31]. It can capture the past and future information to detect emotions by using non-linear functions.  
BiLSTM consists of forward LSTM and backward LSTM. The main idea of LSTM is to forget the information that is not useful for detecting emotion and pass the valuable information to the future time point.  
As shown in Figure 5, each LSTM unit consists of a forget gate, input gate, and output gate. The information transmission of the LSTM unit is based on the forget gate. It receives the previous hidden state \(h_{t-1}\) and current information \(x_{t}\) to determine whether the information is not useful for detecting emotions. The input gate utilizes previous cell state \(C_{t-1}\), previous hidden state \(h_{t-1}\), and current information \(x_{t}\) to obtain the current cell state \(C\). The output gate is used to give the probability of two emotions by using the current information or previous information. For each LSTM unit, each gate at time \(t\) is calculated as  
\[f_{t}=\delta_{g}(W_{f}\cdot[h_{t-1},x_{t}]+b_{f}) \tag{2}\] \[i_{t}=\delta_{g}(W_{i}\cdot[h_{t-1},x_{t}]+b_{i})\] (3) \[o_{t}=\delta_{g}(W_{o}\cdot[h_{t-1},x_{t}]+b_{o})\] (4) \[C_{t}=f_{t}C_{t-1}+i_{t}\delta_{c}(W_{c}\cdot[h_{t-1},x_{t}]+b_{ c})\] (5) \[h_{t}=o_{t}\delta_{c}(C_{t}) \tag{6}\]  
where \(\delta_{g}\) is sigmoid function, \(\delta_{c}\) is hyperbolic tangent function, \(C\) is the cell states, and \(h\) is the hidden states. The forget gate, output gate, and input gate are formulated as \(f\), \(o\), and \(i\). The weight and basis of each gate are formulated \(W\) and \(b\). The input of each function is calculated by using the current signal value \(x_{t}\) and the previous hidden state \(h_{t-1}\). The input of the first LSTM unit only has the current signal value.  
Figure 4: The dot product of signals at each time point.  
Figure 6 gives an example of EEG-based emotion detection by using BiLSTM, where the input is the selected segments, \(x_{i}\) indicates the value at \(t\), \(h_{t}^{f}\) indicates the hidden state of forward LSTM, and \(h_{t}^{b}\) indicates the hidden state of backward LSTM. The forward LSTM processes the signal from the left to right, and it utilizes hidden layers to pass the information. The current hidden state \(h_{t}^{f}\) is calculated based on the previous hidden state \(h_{t-1}^{f}\) and the current input \(x_{t}\). The backward LSTM passes the future information to the history unit and the previous hidden state \(h_{t-1}^{b}\) is calculated based on the current hidden state \(h_{t}^{b}\) and the input \(x_{t}\).  
We formulate the weights of BiLSTM as \(W=\{W_{f},W_{i},W_{o},W_{c}\}\), the loss function of the BiLSTM is established based on the cross entropy, which is formulated as \(L=\frac{1}{N}\sum_{i=1}^{N}-[y_{i}log(\hat{y}_{i})+(1-y_{i})log(1-y_{i})]\) where \(y\) indicates the real emotion, \(\hat{y}_{i}\) indicates the detected emotion, and \(N\) indicates number of EEG signals. The training of the model is to calculate weights \(W\) and basis \(b\) for minimizing the loss function \(L\). The gradient descent algorithm is used to update the parameters \(W\) and \(b\), which is formulated as \(W=-\eta\frac{\partial L}{\partial W}\) and \(b=b-\eta\frac{\partial L}{\partial b}\) where \(\eta\) indicates the learning rate [32]. The initial  
Figure 5: Architecture of a LSTM unit.  
Figure 6: Example of EEG-based emotion detection by using BiLSTM.  
parameters are given randomly. To update the parameters \(W\) and \(b\) at each layer, the backward propagation algorithm is utilized, which is based on the chain rule to calculate the partial derivatives of compound functions [33].