## 3 Proposed Methodology
### Algorithm Flow of TextCNN + RoBERTa and RNN_Bert_Based
To summarize, each of the four metrics, such as accuracy, precision, recall, and F1 score, has its own set of advantages and disadvantages. In order to comprehensively evaluate the classification performance of the model, the aforementioned four metrics are adopted as the evaluation metrics for this experiment. ### Experiments and Parameter Settings

In order to comprehensively evaluate the effectiveness of the proposed model in extracting both local and global high-level contextual semantic information and to compare its performance against traditional classification models, a series of experimental studies were undertaken. Additionally, this study includes a comparison with several statistically-based traditional machine learning classification models, including XLNet, TextCNN + RoBERTa, Bert, and ALBERT (A Lite Bert), in addition to benchmarking against the original Bert base. 1. Bert was introduced by Google in 2018 and has revolutionized natural language understanding tasks. It is based on the Transformer architecture and trained on two unsupervised tasks: Masked Language Model (MLM) and Next Sentence Prediction (NSP). Bert is capable of understanding context by considering both the left and right context of a word. Nevertheless, it is constrained by the fixed-length input limitation inherent to its token-based approach. 2. The TextCNN + RoBERTa (Convolutional Neural Networks for Efficient Text Classification and Robust Bidirectional Encoder Representations from Transformers) was proposed by [14], which can adapt convolutional neural networks (CNNs). It was originally designed for image data to handle text data. CNNs are effective at capturing local patterns, which makes them well-suited for tasks where n-grams (sequences of words) are of importance. RoBERTa is an enhanced version of Bert that was developed to refine the original Bert model through optimized pre-training. In 2019, FacebookAI introduced RoBERTa, which modifies Bert in several ways to achieve enhanced performance on a range of NLP tasks. The combination of TextCNN and RoBERTa offers several benefits, including the following:

1. Complementary Strengths: While RoBERTa demonstrates proficiency in discerning contextual relationships within textual data, it may exhibit a deficiency in identifying specific local patterns, such as n-grams, that are pivotal for certain tasks. In contrast, TextCNN demonstrates a high proficiency in capturing local patterns, although it is less adept at grasping the intricate contextual nuances that RoBERTa is capable of discerning. 2. Enhanced Feature Representation: The combination of these models allows for the exploitation of RoBERTa's profound, context-aware representations, which are augmented by the capacity of TextCNN to discern salient local features. The combination of these models may result in enhanced classification performance, particularly in tasks where both global context and local patterns are of significance. 3. The TextCNN + RoBERTa hybrid model is an innovative approach that effectively integrates the strengths of convolutional neural networks and transformer-based architectures, resulting in a robust and efficient solution for text classification tasks. This combination enables the model to capture both local and global features in text data, which may result in enhanced performance across a diverse range of NLP applications. 4. XLNet, which was introduced by Google in 2019, represents an improvement upon Bert's bidirectional context understanding. It employs a permutation-based approach, leveraging an autoregressive model similar to the Transformer-XL architecture. This enables the capture of bidirectional context without the need for masked language modeling. The enhanced bidirectional context understanding results in enhanced performance on various downstream tasks. 5. ALBERT (A Lite BERT), which was proposed by Google in 2019, is designed to reduce the computational cost of Bert while maintaining or even improving its performance.