# Attention-Based RU-BiLSTM Sentiment Analysis Model for Roman Urdu
## 7 Conclusions  
In E-commerce, a sentiment analysis task on text, which has a colloquial nature, has several challenges. Among these challenges, feature extraction and design of classifiers are the significant ones. Another ability that semantically empowers the deep learning model is pattern recognition. Most of the baseline deep neural networks lack this capability. The classic LSTM has solved this problem to some extent, but it still needs improvement. This research study proposed an attention-based bidirectional LSTM network, RU-BiLSTM, that has shown superior performance on our newly generated Roman Urdu dataset, RUED. The dataset is one of the major contributions of this paper, and it is openly available ([[https://github.com/bilalbaloch1/RU-BiLSTM/blob/main/RUED.csv](https://github.com/bilalbaloch1/RU-BiLSTM/blob/main/RUED.csv)]([https://github.com/bilalbaloch1/RU-BiLSTM/blob/main/RUED.csv](https://github.com/bilalbaloch1/RU-BiLSTM/blob/main/RUED.csv))) (acceed on 15 February 2022) along with all the model code. The code and dataset have been made public so that future researchers may improve upon the accuracy of the dataset. Roman Urdu being a resource-starved language has always needed a benchmark dataset. This study presented the largest ever Roman Urdu dataset containing 26K+ user reviews carefully annotated by three annotators with excellent knowledge of Roman Urdu. The dataset and API for Roman Urdu sentiment analysis is publicly available for future research in this area. We conducted a series of experiments in the presence of three different neural word embedding for Roman Urdu: word2 vec, Glove, and FastText. Among three word embedding schemes, word2 vec embeddings has shown better performance. The neural word embedding and attention layer are used to semantically empower the model and to enhance the pattern extraction capability. The experimental results reveals that the classic LSTM, RNN, RCNN, and CNN have shown an average level performance. On the other hand, our proposed model RU-BiLSTM has shown significant improvement in all four evolutionary metrics i.e., precision, recall, F1-measure, and accuracy. The experimental results shows that our proposed model outperforms the baseline models, with enhanced semantic capability and a significant improvement of 8% in terms of accuracy. Future work regarding Roman Urdu focuses on the creation of a benchmark dataset for fine-grained sentiment analysis with research in transformer learning and on designing a new self-attention mechanism to further improve the performance. Future research is based on the following agendas:The development of a benchmark Roman Urdu dataset to support fine-grained sentiment analysis and question classification. The fine-grained sentiment analysis will include five types of discrete classifications namely, strongly negative, weakly negative, neutral, weakly positive, and strongly positive. Designing an new improved self-attention mechanism to further ameliorate the performance. Employment of sentence transformer learning such as BERT to estimate its performance for Roman Urdu NLP tasks. Conceptualisation, B.A.C., J.B. and M.B. ; methodology, B.A.C., J.B., M.B., S.M.D. and A.S.I. ; validation, B.A.C., A.S.I. and S.M.D. ; formal analysis, B.A.C. and J.B.; investigation, B.A.C., J.B. and S.M.D. ; resources, J.B. and A.S.I. ; data curation, B.A.C. and J.B.; writing--original draft preparation, B.A.C., J.B. and M.B. ; writing--review and editing, A.S.I. and S.M.D. ; visualisation, B.A.C. and M.B. ; supervision, J.B. and A.S.I. ; project administration, J.B. and A.S.I. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable. The authors declare no conflict of interest. The following abbreviations are used in this manuscript: