## 2 Materials and Methods
### Experimental Model
#### 2.2.3 Semantic Extraction Layer  
The semantic extraction layer of our model utilizes the BiLSTM (bidirectional long short-term memory) model. LSTM, which stands for long short-term memory, was introduced by [PERSON] et al. as a type of recurrent neural network (RNN). It addresses the issues of gradient vanishing and exploding that are present in traditional RNN models during training.  
The structure of the LSTM module is shown in Figure 4. LSTM consists primarily of the cell state \(C_{t}\), temporary cell state \(\widetilde{C}_{t}\), hidden state \(h_{t}\), forget gate \(f_{t}\), memory gate \(i_{t}\), and output gate \(o_{t}\). The operation of LSTM can be summarized as follows: at each time step, the forget gate \(f_{t}\), input gate \(i_{t}\), and output gate \(o_{t}\) control maintenance of the information in the cell state. This facilitates the transmission of useful information for subsequent time steps while discarding irrelevant information. The computation of these gates involves the hidden state \(h_{t-1}\) from the previous time step and the current input \(x_{t}\). The formulas for calculating these gates are expressed in Equations (5)-(10):  
\[f_{t}=\sigma(W_{f}\cdot[h_{t-1},x_{t}]+b_{f}) \tag{5}\]  
\[i_{t}=\sigma(W_{i}\cdot[h_{t-1},x_{t}]+b_{i}) \tag{6}\]  
\[\widetilde{C}_{t}=\tanh(W_{C}\cdot[h_{t-1},x_{t}]+b_{C}) \tag{7}\]  
\[C_{t}=f_{t}*C_{t-1}+i_{t}*\widetilde{C}_{t} \tag{8}\]  
\[o_{t}=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o}) \tag{9}\]  
\[h_{t}=o_{t}*\tanh(C_{t}) \tag{10}\]  
In the equations, \(x_{t}\) represents the input at time step t; \(h_{t-1}\) denotes the hidden state from the previous time step; \(W_{f}\), \(W_{i}\), \(W_{C}\), and \(W_{o}\) represent the weight matrices; \(b_{f}\), \(b_{i}\), \(b_{C}\), and \(b_{o}\) are the bias terms introduced during the training process; and \(\sigma\) represents the sigmoid activation function.  
One issue with unidirectional LSTM is its inability to encode information from later time steps. The issue is addressed by BiLSTM, which combines forward LSTM with backward LSTM. Thus, this enables BiLSTM to consider both the forward and backward directions of textual information simultaneously. This bidirectional nature contributes to improving its performance in sentiment analysis tasks, as sentiment analysis is often influenced by various factors in the text.  
The model selects the feature vector \(E\in R^{n}\), which is the concatenation of the output of the last layer trained using Roberta-wwm-ext and the feature values extracted from  
Figure 4: LSTM module structure.  
the emotion lexicon. This feature vector is then added to weight \(W_{e}\in R^{e\times n}\) to obtain \(A=(a_{1},a_{2},a_{3}\ldots a_{n})\) as the input for the semantic extraction layer. The calculation is expressed in Equation (11):  
\[a_{t}=g_{1}(W_{e}E_{t}+b_{a}) \tag{11}\]  
In the equation, \(1\leq n\leq t\), where n is the dimensionality of the feature vector obtained after the sentence has been trained using Roberta-wwm-ext. \(g_{1}\) is the activation function _Sigmoid_, and \(b_{a}\) is the bias vector.  
The vector input to the hidden layer undergoes forward and backward LSTM operations, resulting in forward and backward outputs, respectively. The final output vector \(v_{i}\) of the BiLSTM is obtained by concatenating the hidden state result \(\overset{\rightarrow}{h}_{t}\) from the last time step of the forward LSTM and the hidden state result \(\overset{\leftarrow}{h}_{t}\) from the last time step of the backward LSTM. The calculation is expressed in Equation (8):  
\[v_{i}=\overset{\rightarrow}{h}_{t}+\overset{\leftarrow}{h}_{t} \tag{12}\]