## 3 Proposed Methodology
### Algorithm Flow of TextCNN + RoBERTa and RNN_Bert_Based
This model is being evaluated for its performance in text classification, which means categorizing text data into predefined labels. The learning rate controls how much the model's weights are adjusted with respect to the gradient during training. Next is epoch which is the number of times the model goes through the entire dataset during training. The results demonstrate that with a total batch size of 16 and learning rates of \(2\times 10^{-5}\), accuracies of 91.05, 92.26, 92.27, 92.30, and 92.31 were obtained from episodes 1 to 5, respectively. Figure 8 presents a comparison of the KNN_Bert_based model's accuracy with five epochs. Figure 7: Bert model accuracy comparison by different learning rates with batch size = 16 (a), Bert model accuracy comparison by different learning rates with batch size = 32 (b). #### 4.3.3 Proposed RNN_Bert_Based Method

The integration of KNN with the Bert model leverages the strengths of both techniques. The experiment involved training a Bert model and a Recurrent Neural Network based (RNN_Bert_based) model with the following key parameters for the Bert model part: max_seq_len = 50, epochs = 5, batch_size = 16, learning rate \((lr)\) = \(2\times 10^{-5}\), and patience = 1. For our RNN model, we set the number of hidden units (rnn_hidden_size) to 256, allowing the network to capture complex patterns in the data with a balanced computational cost. The parameter defining the number of classes for the final classification task (num_labels) was set to 2, indicating a binary classification problem. We specified the use of an LSTM (Long Short-Term Memory) layer as the type of RNN. LSTMs are particularly effective for managing long-term dependencies in sequential data due to their gating mechanisms, which regulate information flow and help mitigate issues like vanishing gradients. For more details of parameter values for the experiment, see Table 1.

This configuration ensured our model was well-suited for tasks that require understanding temporal patterns and making binary decisions. Figure 9 presents a comparison of RNN_Bert_based model's accuracy within five epochs. Figure 8: KNN_Bert_based accuracy comparison by different epochs. Figure 9: RNN_Bert_based accuracy comparison by different epochs. #### 4.3.4 Comparison of Proposed RNN_Bert_Based and Other Methods

In this study, we conducted a comparative analysis with several established traditional machine learning classification models, including XLNet, TextCNN + RoBERTa, Bert, and ALBERT (A Lite BERT), comparing to KNN_Bert_based model and our proposed RNN_Bert_based model. The value of parameters utilized in experimentation was set according to the best Bert hyperparameters in our simulation to obtain model higher accuracy. These models represent a spectrum of approaches within the field, ranging from sophisticated transformer architectures like XLNet and ALBERT to more streamlined versions like TextCNN + RoBERTa. The primary objective of this comparison was to assess the performance of the proposed RNN_Bert_based model relative to well-established models across a range of experimental parameters. This comprehensive evaluation allowed us to explore the efficiency of our model in extracting both local and global high-level contextual semantic information in comparison to existing state-of-the-art methods. Following the completion of the experiments and the subsequent analysis of the results, it was observed that the proposed model exhibited competitive performance across multiple evaluation metrics. In particular, it demonstrated superior performance in terms of accuracy, precision, recall, and F1 score in comparison to traditional methods. Furthermore, when benchmarked against the original Bert-based model, the proposed model demonstrated comparable or even improved performance, indicating its effectiveness in capturing nuanced semantic information and achieving robust classification results.