## 4 The Proposed Method
### Ck-BiLSTM  
In recent years, in order to enable the LSTM network to transmit information to obtain more comprehensive information, more innovative LSTM-based solutions have been proposed to achieve a better understanding of missing data and the subsequent patterns and relationships in the data ([PERSON] et al., 2017). In our lives, we often use bad to describe something, for example, a bad person, which may indicate that \"bad\" is a word with emotional information. It is a modifier of the next word, \"person\". Therefore, when there is less information, it should be deleted at the next time step.  
Based on the questions raised above, we add the emotional information from commonsense knowledge to BiLSTM. We build the extension of emotional semantic knowledge by extending the method proposed by [PERSON] et al. (2017) and others. In order to help filter the information of the previous time step to the next time step, we supplement the original information with commonsense knowledge. We extend BiLSTM and fuse the external commonsense to BiLSTM (shortened for CK-BiLSTM). The CK-BiLSTM architecture is given in Figure 3. Red lines represent commonsense emotional knowledge. At each time step t, we assume that each word can trigger and map the emotional knowledge to andimensional space. We use O to represent the number of emotional knowledge, such as \([\beta_{t,1},\beta_{t,2},\ldots\ldots,\beta_{t,m},\ldots\ldots,\beta_{t,O}]\), where we will use the average vector to embed them.  
\[\beta_{t}=\frac{1}{O}\sum_{m}\beta_{t,m} \tag{8}\]  
The formula of BiLSTM with commonsense knowledge is illustrated below:  
\[f_{t}=\sigma\Big{(}W_{f}[w_{t},h_{t-1},\beta_{t}]+b_{f}\Big{)} \tag{9}\]  
\[i_{t}=\sigma(W_{i}[w_{t},h_{t-1},\beta_{t}]+b_{i}) \tag{10}\]  
\[\widehat{C}_{t}=\tanh(W_{C}[w_{t},h_{t-1}]+b_{C}) \tag{11}\]  
\[C_{t}=f_{t}*C_{t-1}+i_{t}*\widehat{C}_{t} \tag{12}\]  
\[o_{t}=\sigma(W_{o}[w_{t},h_{t-1},\beta_{t}]+b_{o}) \tag{13}\]  
\[o_{t}^{c}=\sigma(W_{o}^{c}[w_{t},h_{t-1},\beta_{t}]+b_{o}^{c}) \tag{14}\]  
\[h_{t}=o_{t}*\tanh(C_{t})+o_{t}^{c}*\tanh(W_{c}\beta_{i}) \tag{15}\]  
where \(f_{t}\) is the forget gate, \(i_{t}\) is the input gate, \(C_{t}\) is the cell state and \(o_{t}\) is the output gate. \(W_{f}\), \(W_{i}\), \(W_{C}\) and \(W_{o}\) are known as weight matrices. \(w_{t}\) and \(h_{t}\) is the input and the hidden state at time \(t\). \(h_{t-1}\) represents the hidden state at time \(t-1\). \(b_{f}\), \(b_{i}\), \(b_{C}\), \(b_{0}\) refer to biased values of different gates. The sigmoid activation function is specified by \(\sigma\) and \(tanh\), and \(*\) is the product of Hadamard. Furthermore, we have an important information supplement activation vector here, that is, \(o_{t}^{c}\), which is a BiLSTM output gate extended with commonsense knowledge. This gate will complement the normal gate \(o_{t}^{c}\) of BiLSTM and send it to the hidden state, as shown in Figure 3, where \(W_{o}^{c}\) is the weight matrices and \(b_{o}^{c}\) is the biased values. The CK-BiLSTM layer outputs a sequence of hidden states \(\{h_{1},\cdots,h_{m}\}\), which are conveyed to an average pooling layer to obtain a tweet representation \(c_{t}\).