# Deep Learning-Driven Public Opinion Analysis on the Weibo Topic about AI Art
## 2 Materials and Methods
### LERT_BiSRU++ Sentiment Analysis Model
#### 2.1.2 SRU++  
In specific downstream tasks, the corpus used may exhibit distinctions from the corpora used in the pre-training stage. Additionally, semantic features related to emotional tendencies might be present in long-range dependencies throughout the text, which can be challenging for pre-trained models to capture. To tackle these issues, RNN architectures are used to capture long-range dependencies. RNN utilizes the output from the previous time step combined with the input of the current time step to transmit dependencies, allowing the propagation of long-range information throughout the input sequence. To mitigate issues such as gradient vanishing and exploding, the LSTM model incorporates gate structures to selectively filter out both the output of the previous time step and the input of the current time step, enhancing the information transmission process and reducing the occurrence of gradient vanishing and exploding. However, due to its network structures, the LSTM model depends on having access to all information from the previous time step, exhibiting a high degree of seriality. Consequently, this can result in relatively low computational efficiency during both the training and inference processes when facing large amounts of data. To tackle the above issues, an SRU [19] enhances the calculation process of the gate structures to improve computational efficiency in a parallelized way, which is achieved by introducing light recurrent units and a highway network. The structure of the SRU is depicted in Figure 5.  
In contrast to LSTM, SRU utilizes only the forget gate \(f_{t}\) to compute the current cell status \(c_{t}\), as illustrated in Equations (7) and (8). \[f_{t}=\sigma\Big{(}W_{f}\times x_{t}+v_{f}\odot c_{t-1}+b_{f}\Big{)}, \tag{7}\]  
\[c_{t}=f_{t}\odot c_{t-1}+(1-f_{t})\odot(Wx_{t}). \tag{8}\]  
Figure 5: The structure of SRU. In SRU, the calculation of the forget gate \(f_{t}\) uses a learnable vector \(v_{f}\) to map the cell status of the previous time step, \(c_{t-1}\), into the calculation process, which enables the SRU to calculate \(f_{t}\) and \(c_{t}\) correlated to the known dimension of input data, achieving the parallelization of computation. Moreover, SRU uses the Hadamard product to replace the original matrix product used in LSTM, combined with the conjugate operation, \(1-f_{t}\), to further accelerate the calculation process. Moreover, SRU introduces the highway network to improve the propagation of gradients when facing deeper network structures. SRU utilizes the reset gate \(r_{t}\) to control the transmission of information, combined with a skip connection to optimize, as depicted in Equations (9) and (10). \[r_{t}=\sigma(W_{r}x_{t}+v_{r}\odot c_{t-1}+b_{r}), \tag{9}\]  
\[h_{t}=r_{t}\odot c_{t}+(1-r_{t})\odot x_{t}. \tag{10}\]  
Apart from using the conjugate operation \(1-r_{t}\) to simplify the model parameters, the introduction of \(x_{t}\) used in Equation (10) allows the calculation of the hidden state \(h_{t}\) to use the input of the current time step, which helps with the gradient clipping process during training stages. To further improve the parallelization, SRU uses batch multiplication to enhance the calculation process of matrices \(W\). For a batch input \(X\) with size \(L\) and dimension of \(d\), the batch multiplication can be described as follows:  
\[U^{\top}= \begin{pmatrix}W\\ W_{r}\\ W_{f}\end{pmatrix} X^{\top}, \tag{11}\]  
For the current time step \(t\), \(U_{[t,0]}\), \(U_{[t,1]}\), and \(U_{[t,2]}\) stand for the multiplication results of three matrices used in SRU, which change the equation of SRU as follows:  
\[f_{t}=\sigma\Big{(}U_{[t,2]}+v_{f}\odot c_{t-1}+b\Big{)}, \tag{12}\]  
\[r_{t}=\sigma\Big{(}U_{[t,1]}+v_{r}\odot c_{t-1}+b_{r}\Big{)}, \tag{13}\]  
\[c_{t}=f_{t}\odot c_{t-1}+(1-f_{t})\odot U_{[t,0]}. \tag{14}\]  
In Equation (10), the process uses a linear transformation to calculate \(U\), which can be optimized using self-attention mechanisms.