# Weibo Text Sentiment Analysis Based on BERT and Deep Learning
## 5 Experiment and Analysis
### Experimental Results
The first part is the traditional deep learning model, including CNN and BiLSTM, the second part is based on transformer models, such as BERT, and the last part is the combined model. As shown in the table, although traditional deep learning models do not need to manually label features, their performance is not satisfactory. It is worth mentioning that the same is a deep learning related model, the accuracy of the BiLSTM network is noticeably higher than CNN. This is because BiLSTM considers the context information of text from two directions and solves the related dependencies of long text. In addition, the attention mechanism can also increase the accuracy by giving different degrees of attention to different words. After introducing the BERT language model, the accuracy of classification has been significantly increased. This is because BERT is a powerful pre-training language model. Unlike traditional language models such as Glove and Word2 vec, BERT can dynamically, based on context, generate word vectors and use them for downstream tasks. The proposed model defeats most competitive models in the accuracy of classification, and reflects its superiority. The advantages are mainly reflected in the following points: First, the model combines the advantages of BiLSTM, the CNN network, and the attention mechanism. Secondly, the model uses the more advanced BERT language model to mine the deep semantics of words and dynamically generate high-quality word vectors. Finally, the \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1** \\ \hline CNN & 73.5\% & 73.7\% & 72.1\% & 72.9\% \\ \hline CNN + Att & 77.6\% & 78.1\% & 76.0\% & 77.0\% \\ \hline BiLSTM & 83.6\% & 83.7\% & 82.9\% & 83.3\% \\ \hline BiLSTM + Att & 86.5\% & 86.4\% & 86.2\% & 86.3\% \\ \hline BERT & 84.4\% & 85.5\% & 82.4\% & 83.9\% \\ \hline BERT + BiLSTM + Att & 89.7\% & 90.9\% & 87.8\% & 89.3\% \\ \hline AC-BiLSTM & 91.6\% & 93.0\% & 89.8\% & 91.4\% \\ \hline ABCDM & 93.1\% & 93.7\% & 92.2\% & 92.9\% \\ \hline OURS & 92.4\% & 92.5\% & 92.4\% & 92.5\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison model experiment results. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Parameter Value** \\ \hline The Dimension of the Word Vector & 768 \\ \hline The Length of the Input text & 12 \\ \hline Dropout & 0.6 \\ \hline Epoch Number & 8 \\ \hline The Number of hidden neurons in BiLSTM Layer & 128 \\ \hline The Number of hidden neurons in CNN Layer & 128 \\ \hline The Size of the Convolutional Kernel & 3 * 3 \\ \hline Learning Rate & 0.01 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Model parameter setting table. model introduces an external sentiment dictionary when constructing the word vector, and strengthens the sentiment intensity of the word vector through the sentiment dictionary. The experimental results show that the accuracy and other performance indicators of the proposed model in this article are lower than those of the ABCDM model. This is because BiLSTM and BiGRU are used in parallel in ABCDM, which give the model the ability to process long text and short text at the same time, so its performance is slightly better than the model proposed in this paper. This has inspired us, and our model is also going to consider parallel mechanisms in the next step. Besides, in order to study the impact of the introduction of sentiment dictionary and CNN on proposed model, we established four controlled experiments to conduct ablation studies based on the model proposed in this article:  
\(-\)CNN \(-\) Lexion: We remove the emotional dictionary and convolutional neural network from our model, so that the remaining model is equivalent to the BERT + BiLSTM + Attention model.