# Roman Urdu Sentiment Analysis Using Transfer Learning
###### Abstract  
Numerous studies have been conducted to meet the growing need for analytic tools capable of processing increasing amounts of textual data available online, and sentiment analysis has emerged as a frontrunner in this field. Current studies are focused on the English language, while minority languages, such as Roman Urdu, are ignored because of their complex syntax and lexical varieties. In recent years, deep neural networks have become the standard in this field. The entire potential of DL models for text SA has not yet been fully explored, despite their early success. For sentiment analysis, CNN has surpassed in accuracy, although it still has some imperfections. To begin, CNNs need a significant amount of data to train. Second, it presumes that all words have the same impact on the polarity of a statement. To fill these voids, this study proposes a CNN with an attention mechanism and transfer learning to improve SA performance. Compared to state-of-the-art methods, our proposed model appears to have achieved greater classification accuracy in experiments. sentiment analysis (SA); convolutional neural network (CNN); Roman Urdu language; attention mechanism; transfer learning; machine learning (ML); deep learning (DL) +
Footnote †: journal: Applied Sciences  
combining English letters and the Urdu script. Using the English alphabet, the Urdu language can be transformed into Roman Urdu. For instance, the Roman Urdu word for \"Justice\" is \"Insat.\" Internet users in Pakistan overwhelmingly favor Roman Urdu over the traditional Nataliq Urdu (Arabic script), according to a regional study that investigated the significance of Roman Urdu in the retrieval of information across multiple languages [8]. Despite the extensive use of Roman Urdu and Hindi, the problem of conducting SA in either of these languages has not yet been investigated in its entirety. Thus, the main objective of this research will be to explore Roman Urdu SA. ML and DL models are widely used in various fields [9; 10; 11; 12; 13; 14; 15] and are of great success. Recent advancements in ML- and DL-based modes for SA have had a substantial effect on the efficacy of scholarly and scientific endeavors. With its impressive performance in sentiment classification, CNN has earned its place as a pivotal deep-learning model. CNNs are quite effective at sentiment analysis, yet they face several challenges [16; 17]. In the first place, CNNs are limited in that they can only establish the document's polarity; they cannot offer a comprehensive comprehension of the text, for instance, identifying the determinant word for polarity classification [18]. So, even though people have brains, they still cannot highlight the most important points in a document, reducing their efficiency [19; 20]. Second, CNNs are notoriously data-intensive, necessitating a sizable sample size of training data to adequately calibrate the model and a variety of finely tuned hyperparameters [21]. This study's main contribution is the development of a novel deep-learning approach for enhancing the accuracy of Roman Urdu sentiment analysis. In order to accomplish this, we have employed CNN to glean profound contextual semantics. By emulating the attention mechanism seen in human brains and prioritizing more essential words while ignoring less significant ones [22; 23; 24], we may reap the benefits of using both attention mechanisms and transfer learning. Transfer learning has helped us deal with the issue of the abundance of data sets. Training a model on big datasets in the source domain before transferring it to the target domain can increase its performance [25]. Transfer learning is an approach that avoids the enormous amount of training data that CNN requires, and this is the fundamental notion behind it. The study makes the following contributions:  
* We propose a CNN with attention to Roman Urdu SA. * We conducted a series of experiments to investigate the model's sensitivity and find the best settings for the hyper-parameters that would maximize classification accuracy.