# BERT- and BiLSTM-Based Sentiment Analysis of Online Chinese Buzzwords
## 1 Introduction  
Text sentiment analysis, which can extract the sentiment content of texts, is a common application of natural language processing (NLP) and opinion mining. Moreover, sentiment analysis of online Chinese buzzwords (OCBs) is important in the real world. Specifically, sentiment analysis of OCBs (1) helps microblogs, games, and other online platforms to shield against some improper language, (2) purifies the online environment, and (3) precisely grasps the emotional direction of users. OCBs reflect the real life of society spreading through the Internet. OCBs are also are a kind of living language form [1], which is widely used, produced, and applied to the network. OCBs are emerging and changing, and about 1000 OCBs are generated every year and used by millions of users. OCBs are synonymous with Internet language, which refers to a language generated from the Internet or applied to Internet communication, mainly from online games, chat and comments, and other online social platforms. They usually consist of a mixture of Chinese characters, numbers, English letters, and symbols, such as the harmonic word \"  
in various forms, often have an emotional tendency, and even tend to present vulgarity; and (3) the process of OCBs dissemination often leads to formation of stereotypes in language patterns. It is difficult to form a good social mentality because of the homogenization of people's basic perspectives on social issues or social events and the convergence of their views. For traditional sentiment analysis models, it is difficult to obtain accurate sentiment characteristics of OCBs. Due their irregular structure and semantics, OCBs have no obvious emotional tendency. Besides, the current main text sentiment analysis models are all based on short text [5]. Furthermore, when the amount of existing datasets is large, the convergence time of the model is often too long. To solve the above problems, in this paper, the BERT pre-training model and BiLSTM are introduced to learn deep sentiment features from the context of the irregularity and incomplete semantic structure of OCBs. Furthermore, the BERT pre-training model is fine-tuned to accelerate the convergence rate of the model. Motivation to above problems can be seen in Figure 1. The main contributions are summarized as follows. * The BERT-BiLSTM model is proposed. The model does not require word separation during sentiment analysis and is able to capture deep contextual information from the word order structure. The superiority of BERT-BiLSTM is illustrated in Section 3. * A fine-tuned sentiment analysis of OCBs is proposed to accelerate the model convergence speed. First, the fine-tuning process directly employs the parameters obtained from the pre-training as the initial values of the proposed model. Then, the manually labeled dataset is transferred according to the downstream tasks to balance the relationship between the data and the model. * From the extensive experimental results on the OCBs dataset, we can conclude that BERT-BiLSTM outperforms seven state-of-the-art models in terms of recall and F1-score. In addition, the ablation experiments demonstrate the superiority of the proposed model combining BERT and BiLSTM. The proposed model has significant implications for sentiment analysis of OCBs. Although this model has many contributions and advantages, it also has some limitations:  
* The proposed model is trained on datasets with small data volume (60,000 online Chinese buzzwords texts) in this paper. We will study its application on large-scale datasets in the future. Figure 1: Motivation of sentiment analysis of online Chinese buzzwords. * The proposed model is applied to static data in this paper. However, emotional analysis of dynamic data is also of great significance. For example, DTSCM tracks the change trend of theme emotion in different time segments by capturing the theme of microblog messages in different time segments [6].