# Weibo Text Sentiment Analysis Based on BERT and Deep Learning
## 4 Proposed Method
Different words have different effects on judging the sentiment polarity of the entire text. Therefore, the model introduces an attention mechanism to assign different sentiment weights to different words. Word with stronger emotion is given a higher attention, that is, a higher weight is assigned. For the hidden state output \(\mathbf{h}_{t}\) of BiLSTM, the attention weight is allocated as follows:  
\[\mathbf{u}_{t}=\tanh(\mathbf{W}_{w}\mathbf{h}_{t}+\mathbf{b}_{w}) \tag{15}\]  
\[\boldsymbol{\alpha}_{t}=\frac{\exp\big{(}\mathbf{u}_{t}^{T}\mathbf{u}_{w}\big{)} }{\sum_{t}\exp\big{(}\mathbf{u}_{t}^{T}\mathbf{u}_{w}\big{)}} \tag{16}\]  
\[\mathbf{Z}=\sum_{t}\boldsymbol{\alpha}_{t}\mathbf{h}_{t}. \tag{17}\]  
In the above formula, \(\mathbf{u}_{t}\) is the hidden state representation, \(\mathbf{u}_{w}\) is the contextual content vector of the article, \(\mathbf{u}_{w}\) is randomly initialized at the beginning of training and is continuously optimized during the training process. The similarity between \(\mathbf{u}_{t}\) and \(\mathbf{u}_{w}\) is used to judge the importance of the word. We standardize \(\mathbf{u}_{t}\) to get the attention weight, and the calculated attention weight is finally weighted with \(\mathbf{h}_{t}\) and aggregated into \(\mathbf{Z}\). \(\mathbf{Z}\) represents the entire input text vector, which contains the sentiment information of all words in the text. After obtaining the comment vector containing all word information, convolution is performed on the comment vector. Convolution is to extract the most influential local feature information in the input text information through a filter, which can reduce the data dimension and the model can get the position invariant. The convolutional operation is performed according to Formula (1). After the convolutional operation is over, we perform the pooling operation. The role of the pooling layer is to further compress the convolutional feature vector, which can reduce the vector dimension and reduce the computational complexity. Traditional pooling methods include max pooling and average pooling. If max pooling is used, while retaining the most important features, other important features may be omitted. Therefore, this article uses both max pooling and average pooling in parallel. In this way, the two pooled feature vectors are connected together as the final feature vector and sent to the fully connected layer, which can improve the robustness of the feature vector while retaining important features as much as possible. The max pooling and average pooling operations are independently applied to the feature vector after the convolution operation, and then the results of the two are connected to obtain the text vector \(\mathbf{L}_{c}\). After obtaining the pooled \(\mathbf{L}_{c}\) vector, this paper uses batch normalization to speed up network training and reduce overfitting [31]. In order to predict the emotional polarity of the review, the fully connected layer is used to convert the text vector \(\mathbf{L}_{c}\) into the final emotional representation \(\mathbf{X}\), and finally the emotional representation \(\mathbf{X}\) is sent to the output layer to perform the binary classification task using the sigmoid function. In the output layer, the sigmoid function is used to convert the sentiment representation \(\mathbf{X}\) into the approximate probability value \(\mathbf{Y}\) of each emotional polarity category. This function can map the input information to the \([0,1]\) interval. \(\mathbf{Y}\) close to 0 indicates that the sentiment category is close to negative, \(\mathbf{Y}\) close to 1 means that the sentiment category is close to positive, and then the emotional label of the input text is obtained, that is, the positive and negative two-category sentiment analysis task is completed.