# Roman Urdu Sentiment Analysis Using Transfer Learning
## 2 Literature Review
### Roman Urdu Sentiment Analysis
[PERSON] et al. [64] examined two deep-learning approaches for sentiment analysis in Roman Urdu to learn more about the language's performance in this area. In practice, the LSTM model and several alternative ML classifiers were explored. In a surprising turn of events, it was determined that the deep learning model outperforms its machine learning counterparts. Some recent studies [64] used a multi-channel hybrid method, proposing a strategy based on BiGRU and word embeddings. Their dataset was also insufficient, totaling only 3241 sentiment evaluations, and it is not yet available to the public. [PERSON] et al. [65] studied the problem of identifying anti-Christian bigotry in Roman Urdu by employing static CNN and other word embedding methods. [PERSON] et al. [62] suggested a Roman Urdu Stemmer-powered SVM. Each user review is stemmed after preprocessing. Bag-of-word transforms input text into a feature vector. SVM classifies and detects user sentiment. [PERSON] [66] proposes a deep recurrent architecture RU-BiLSTM for sentiment analysis in Roman Urdu. RU-ECD and RUSA-19 were used to test our model. Our proposed model beat baseline models in many ways, improving by 6% to 8%. [PERSON], seemab [67] Huggingface's transformer models DistilBERT and XLNet outperform Logistic Regression and Naive Bayes. DistilBERT obtained 100% accuracy on our Roman Urdu dataset with only two epochs of fine-tuning, while XLNet reached 86% accuracy with four epochs. [PERSON] et al. [68] analyzed word embeddings for Roman Urdu and English using CNN-LSTM with typical machine learning classifiers. Long-term dependency preservation (LSTM) and local feature extraction (one-layer CNN). CNN and LSTM feature maps are supplied to machine learning classifiers for final classification. Word embedding models support this. Word2 Vec performed best on RUSA-19 and UCL Glove. ([PERSON] et al., 2022) [69] recently suggested a SA model for natural language using roman Urdu as a case study. They built a new dataset containing 24,000 Roman Urdu sentences and used deep neural network techniques such as CNN, RNN, ID3, and Gradient Boost Trees. Logistic Regression obtained 92.25% accuracy. [PERSON] et al. (2021) [70] employed DL for multimodal Urdu SA utilizing Bi-LSTM and 3D-CNN. [PERSON] et al. (2022) [71] introduced a model to predict sentiment and detect latent topics in texts; they explored topic evolution and sentiment fluctuations. [PERSON] et al. (2022) [72] combined classic vectorization methods with their form of BERT to generate stories. These models help with adversarial training. The intensity of sentiment emotion makes sentiment analysis multiclass other than positive,negative, and neutral. [PERSON] et al. (2022) [PERSON] et al. (2022) suggested an ensemble framework to provide more clarity on sentiment and emotion in text. Table 1 provides a summary of the state-of-the-art literature on Roman Urdu sentiment analysis. After reviewing the existing research and examining the complications encountered, we settled on proposing a CNN that combines the benefits of transfer learning with the attention mechanism via a layer of dedicated attention neurons. In contrast to earlier research, the proposed model uses an attention mechanism following the convolutional layer to extract meaningful words from phrases. To improve performance, the suggested model makes use of transfer learning, which involves applying what is learned in one domain to other unexplored domains.