# RoBERTa-GRU: A Hybrid Deep Learning Model for Enhanced Sentiment Analysis
## 6 Experimental Results and Analysis  
This section presents the experimental results of the BERT, RoBERTa, BERT-GRU, and RoBERTa-GRU on the Twitter US Airline Sentiment dataset without data augmentation, as shown in Table 6. The table shows that the RoBERTa outperforms BERT, achieving an accuracy of 90.67% compared with 89.36% for BERT. The BERT-GRU model achieved an accuracy of 89.56% on the Twitter US Airline Sentiment dataset, while the RoBERTa-GRU achieved the highest accuracy of 91.28%. These results indicate that the addition of a GRU layer further improved the performance of the RoBERTa. Therefore, based on these results, the RoBERTa-GRU was chosen for the proposed models. These results demonstrate the effectiveness of the RoBERTa and its ability to capture more nuanced features in the text data. The effectiveness of the proposed RoBERTa-GRU model is also analyzed and compared against the state-of-the-art approaches. The datasets used in the experiments are divided into three portions: 60% for training, 20% for validation, and 20% for testing. The model is trained for a maximum of 100 epochs with a batch size of 32. To avoid overfitting, an early stopping technique is employed with a patience of 30 epochs based on the validation accuracy. Table 7 compares the accuracy, precision, recall, and F1-score of various machine learning and deep learning models on the IMDb dataset. Among the machine learning models, logistic regression achieves the highest accuracy of 87.12%, whereas the GRU model performs the best among the deep learning methods with an accuracy of 87.88%. However, the proposed RoBERTa-GRU model surpasses all other models with an accuracy of 94.63%. The superiority of the RoBERTa-GRU model can be attributed to several factors. Firstly, the RoBERTa model is a pretrained Transformer-based language model that has been fine-tuned on a large corpus of text data. The pretraining allows the model to learn rich and meaningful representations of the input text, making it more effective in capturing the underlying sentiment of the movie reviews. Moreover, the RoBERTa-GRU model combines the strengths of the RoBERTa and GRU models. The RoBERTa-GRU model uses the RoBERTa's pretrained embeddings as input to the GRU network, which allows the model to learn the context and dependencies of the text more effectively. Additionally, the GRU network's ability to handle sequential data enables the model to capture the long-range relationships between words and phrases, which is essential in sentiment analysis. \begin{table}
\begin{tabular}{c c} \hline
**Model** & **Accuracy (\%)** \\ \hline BERT & 89.36 \\ BERT-GRU & 89.56 \\ RoBERTa & 90.67 \\ RoBERTa-GRU & 91.28 \\ \hline \end{tabular}
\end{table} 
Table 6: The Comparison Results between BERT and RoBERTa. \begin{table}
\begin{tabular}{c c} \hline
**Learning Rate** & **Accuracy (\%)** \\ \hline
0.00001 & 91.52 \\
0.0001 & 89.81 \\
0.001 & 32.69 \\ \hline \end{tabular}
\end{table} 
Table 5: The experimental results of different learning rates (GRU = 256, optimizer = Nadam). Table 8 presents a comparison of various machine learning and deep learning methods on the Sentiment140 dataset. The proposed RoBERTa-GRU model records the highest accuracy of 89.59% among all the methods. This suggests that the RoBERTa-GRU model has a better capability of predicting the sentiment of a given text than other methods. One reason for the superior performance of the RoBERTa-GRU is that the RoBERTa is pretrained using a much larger corpus and a more diverse set of pretraining tasks than other models, such as the LSTM and GRU approaches. This results in better sentence embedding and contextual understanding, which enables the RoBERTa-GRU to capture more nuanced patterns in the text data and make more accurate predictions. Furthermore, the gating mechanism of the GRU selectively updates the hidden state based on the input and the previous hidden state. GRUs have been observed to outperform LSTMs in cases where the data have short-term dependencies.