# Deep Learning-Driven Public Opinion Analysis on the Weibo Topic about AI Art
## 2 Materials and Methods
### LERT_BiSRU++ Sentiment Analysis Model
#### 2.1.1 Lert  
The first step in calculating the semantics expressed by texts is text representation. By transforming the unstructured texts into vectors, text representation facilitates the computation of text, enabling the analysis of text semantics. Based on large corpora, a Masked Language Model (MLM), and a Transformer network, BERT has achieved bidirectional  
Figure 1: The overall structure of the public opinion framework. Figure 2: The LERT_BiSRU++ sentiment analysis model’s structure. contextual text feature extraction and dynamic text semantic representation through the tokenization of the text. In comparison to the fixed-length contextual windows used by Word2 vec, BERT demonstrates superior capabilities in capturing the generic semantic features of the pre-training corpora. However, as the complexity of downstream tasks increases, the generic text representation produced by BERT may encounter challenges in capturing the deep semantic features that exist within the complex corpora used in downstream tasks, which could potentially result in a decline in the performance of deep learning models. The learning process of BERT primarily involves the MLM, which focuses on acquiring knowledge of dependency relationships, combinations, and semantic associations among the input text tokens. Through extensive training on large-scale corpora, BERT can acquire an understanding of certain grammatical principles. This process does not explicitly incorporate grammatical knowledge, which can be useful in enhancing the extraction of deep text features when facing a complex corpus. Based on this concept, the LERT model [17] incorporates three natural language processing tasks--Part-Of-Speech Tagging (POS), Named Entity Recognition (NER), and Dependency Parsing (DEP)--to enrich the pre-training corpora. This procedure is achieved by using the LTP tool [18], as illustrated in Figure 3.  
For the input sentence \"SUES is located in Shencheng Shanghai\", the word segmentation tool first segments it into four words: \"SUES\", \"is located in\", \"Shencheng\", \"Shanghai\". Then, the POS task identifies the part of speech for each word and annotates them accordingly (in Figure 2, the gold letter represents the POS labels for each word). Subsequently, the NER task involves identifying entities within a given text (in Figure 2, the word \"Shanghai\" is recognized as a location). Finally, the DEP task integrates the results of the above two tasks, establishing the syntactic connections between individual words in the sentence (in Figure 2, relationships are presented in blue curves and capitalized red letters). Through this process, LERT augments pre-training corpora by incorporating linguistic features, thereby creating weakly supervised data. To learn the linguistic features in pre-training corpora, LERT enhances the BERT model via multi-task learning, as illustrated in Figure 4. LERT adopts the MLM task from BERT and integrates the linguistic features into the pre-training phase to enable the model to recognize these linguistic labels explicitly. Figure 3: The process of acquiring linguistic labels using the LTP tool. For an input sequence of length \(N\), BERT encodes it into a feature representation of dimension \(d\), with the final hidden state denoted as \(H\in\mathbb{R}^{N\times d}\). The MLM masks \(k\) characters (in Figure 4, the words \"is located in\" and \"Shanghai\" are masked) within the input sequence, and the final hidden state of the masked segment is noted as \(H^{m}\in\mathbb{R}^{k\times d}\). For the pooled output, the hidden state is processed through a fully connected layer (Feed Forward Network, FFN) and regularization to generate a probability vector for multi-class assignments in the vocabulary of the whole corpora, as depicted below. \[\overset{\sim}{H}^{m}=\textbf{LayerNorm}(\textbf{FFN}(H^{m})), \tag{1}\]  
The prediction of the masked word can be viewed as a multi-class classification task within the vocabulary \(E\) of size \(V\) in the corpora.