# Research on Sentiment Analysis of Online Public Opinion Based on RoBERTa-BiLSTM-Attention Model
## 2 Literature Review
Although such methods can have a better classification performance, they require a large number of well-labelled a priori datasets as the training set to train the model, which not only entails high labour costs but also affects the classification effect of subjective labelling. (3) Sentiment classification based on deep learning. Deep learning-based sentiment classification involves imitating the human brain using multi-layer neuron hierarchical processing for feature extraction to learn sentiment-related features and judge the sentiment tendency in comment text; commonly used deep learning models are CNN, RNN, LSTM, Bi LSTM, GRU, etc. [12]. Many scholars have conducted extensive research on this. [PERSON] proposed that a bidirectional convolutional recurrent neural network architecture with a group enhancement mechanism could outperform state-of-the-art results in sentiment analysis [13]. [PERSON] compared the capabilities of CNN-based and LSTM-based integrated models and conducted empirical studies to provide deep learning researchers with insights into building effective multilayer models for sentiment analysis tasks [14]. [PERSON] and [PERSON] provided a comprehensive overview of various strategies for integrating deep learning and also explained in detail the various features or factors that influence the success of the integration approach [15]. Deep learning is free from complex feature engineering compared to machine learning, but supervised deep learning still requires a large amount of labelled data, while unsupervised learning requires data with high semantic relevance. Some scholars are also actively addressing the limitations of deep learning, which requires labelled data. For example, [PERSON] proposed a rho-hot encoding and flipping model for sentiment labelling but did not consider fine-grained sentiment analysis [16]. [PERSON] proposed SRL-ACO, which improves the accuracy of NLP models by generating additional data without manual data annotation [17]. However, deep learning training still requires a lot of resources and time and it is difficult to explain the basis of its operation. (4) Sentiment classification based on pre-trained models. Pre-trained models are machine learning models that have been trained on a wide range of datasets and gained a wealth of knowledge in tasks such as image recognition, natural language processing, speech recognition, and machine translation. These models significantly improve the performance of sentiment analysis by leveraging existing expertise to save time and resources. As a result, they can be used directly for the intended purpose or be fine-tuned for specific tasks. The current dominant models used for natural language processing are mainly the BERT family (e.g., BERT, RoBERTa, etc.) [18]. [PERSON] and [PERSON] presented a robust baseline for sarcasm detection based on BERT pre-trained language models and found that BERT-based models outperformed many previous models [19]. [PERSON] and other scholars extracted news sentiment through a BERT model adapted to financial markets, identifying the context of each word in a given item [20]. [PERSON] and other scholars used the BERT model to quantify user-user interactionsand then used the Baidu API to identify live pop-up user emotions [21]. In addition to the above scholars using BERT to train models, there are scholars who have used an improved BERT model (RoBERTa) on this basis to achieve better classification accuracy. [PERSON] and other scholars introduced an adaptive pre-training model for sentiment knowledge (ASK-RoBERTa) to capture the dependency between aspects and sentiment words. Experimental results on several public benchmark datasets showed that the model performed well [22]. [PERSON] and other scholars proposed a new hybrid model for sentiment analysis, RoBERTa-GRU, which exploits the advantages of the pre-training method of RoBERTa and the recurrent neural network of GRU to achieve a significant improvement in accuracy on a public benchmark dataset [23].