## 3 Proposed Methodology
### Algorithm Flow of TextCNN + RoBERTa and RNN_Bert_Based  
In Algorithm 1, the hybrid model integrates TextCNN and RoBERTa to capitalize on both deep contextual embeddings and convolutional feature extraction [14]. The RoBERTa model is a transformer-based model that provides powerful contextual embeddings by capturing deep semantic information from text. A transformer-based model provides robust contextual embeddings through the capture of profound semantic information from textual data. It is an enhanced iteration of Bert that has been fine-tuned on a substantial corpus for enhanced language comprehension. TextCNN is a convolutional neural network designed for the processing of textual data. Convolutional filters are applied to extract local features from the text embeddings, which can capture important patterns that are useful for text classification tasks. The data is divided into the following categories: The dataset designated as \"X_train_text\" comprises the following elements: The input training texts are represented by the variable y_train. The labels correspond to the aforementioned training data. X_test_text is the input test text that the model will subsequently predict following the completion of the training phase. The input texts from the X_train_text (training) and X_test_text (testing) datasets are provided as input to the RoBERTa model. The RoBERTa model processes each input text and generates a sequence of hidden states, or embeddings, for each token within the text. The most common practice is to take the output from the [CLS] token or apply some form of pooling (such as mean pooling) to obtain a fixed-size vector representing the entire sentence or document. The embeddings from RoBERTa are then passed through the TextCNN component. Subsequently, a max-over-time pooling operation is typically employed to retain the most salient features across the entire text sequence. The outputs from the various convolutional filters are then concatenated to form a single feature vector. This vector represents the most salient features of the text as identified by the convolutional filters. A learning rate (\(lr\)) of 0.00002 is employed, which is a standard value for fine-tuning transformer-based models such as RoBERTa. It determines the extent to which the model's weights are updated during the training phase. The model will undergo five iterations over the entirety of the training dataset, which is equivalent to the number of epochs. A batch size of 16 or 32 signifies the number of samples that the model processes prior to updating its weights. Typically, transformer-based models demonstrate enhanced performance when trained with larger batch sizes (e.g., 32). However, this depends on the availability of computational resources. The binary cross-entropy loss function is employed, which is a  
Figure 6: The integration of Bert and RNN. standard approach for binary classification tasks. It quantifies the discrepancy between the predicted probabilities and the actual binary labels. The provided parameters (learning rate, epoch, batch size, etc.) are set with the objective of optimizing the training process, with the aim of achieving good generalization and performance on the test set.