# RoBERTa-GRU: A Hybrid Deep Learning Model for Enhanced Sentiment Analysis
## 2 Related Works
### Deep Learning Approaches  
In the work by [PERSON] and [PERSON] (2017) [9], a deep learning method, Multilayer Perceptron (MLP), was utilized for sentiment analysis. The authors employed a self-collected dataset of 4000 tweets in Korean and English for their experiments. To preprocess the dataset, several steps were applied, including tokenization, case folding, stemming, and the removal of numbers, stop words, and punctuations. The MLP model consisted of three hidden layers and used Stochastic Gradient Descent (SGD) as the optimizer. The proposed method achieved an accuracy of 75.03%. Similarly, in the work by [PERSON] et al. (2019) [10], an MLP model was used for sentiment analysis on Turkish tweets. The authors used a dataset of 3000 positive and negative Turkish tweets with the hashtag \"15 Temmuz\". The data was preprocessed with the Turkish Deascifier, tokenization, stop-words and punctuation removal, and stemming. To convert the text into embeddings, the authors employed the Word2 vec pretrained model. An MLP model, consisting of six dense layers and three dropout layers, was then used for sentiment classification. The proposed method recorded an accuracy of 81.86% on the dataset. Additionally, [PERSON] et al. (2021) [11] utilized a Multilayer Perceptron (MLP) architecture for sentiment analysis on COVID-19-related tweets. The collected dataset consisted of 101,435 tweets labeled as positive or negative. The preprocessing of the dataset involved the removal of HTML tags and non-letters, tokenization, and stemming. The cleaned texts were then transformed into numerical features using Count Vectorizer and TF-IDF Vectorizer. The resulting features were fed into an MLP model consisting of five hidden layers for sentiment classification. The study found that the MLP model with Count Vectorizer achieved an accuracy of 93.73%. In another work, [PERSON] et al. (2019) [12] proposed a hybrid model that combines a Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (Bi-LSTM) for sentiment analysis. The dataset used in their study included 2003 articles and international news in French, which were labeled as neutral, positive, or negative. The texts were represented as embeddings using the pretrained doc2 vec model. The hybrid model consists of a convolutional layer, a max pooling layer, a Bi-LSTM layer, a dropout layer, and a classification layer. The results showed that the proposed hybrid model achieved an accuracy of 90.66% on the dataset. Similarly, [PERSON] et al. (2020) [13] utilized a hybrid architecture that integrates CNN and Bi-LSTM for sentiment analysis, utilizing the Sentiment140 dataset. This dataset consists of 1.6 million positive and negative tweets, which were preprocessed to remove stop words, numbers, URLs, Twitter users' names, and punctuations, and subjected to case folding and stemming. The hybrid model consisted of an embedding layer using the GloVe pretrained model, one-dimensional CNN layer, Bi-LSTM layer, multiple fully connected layers, dropout layers, and a classification layer. The proposed hybrid model achieved an accuracy of 81.20% on the Sentiment140 dataset. [PERSON] et al. (2020) [14] further improved the hybrid architecture of a CNN and Bi-LSTM by incorporating an attention mechanism. The study used the Internet Movie Database (IMDb) dataset with 50k positive and negative reviews for experiments. The texts were represented using the word2 vec pretrained embedding model. The model was optimized using the Adam optimizer, L2 regularization, and dropout techniques. The proposed model recorded an accuracy of 90.26% on the IMDb dataset. In the study by [PERSON] et al. (2020) [15], a hybrid model combining a CNN and LSTM was proposed for sentiment analysis. The authors used a self-collected dataset containing 100 restaurant reviews from the Foodpanda and Shohoz Food apps, which underwent preprocessing to remove unimportant words and symbols. The texts were then transformed into word embeddings using the word2 vec algorithm.