# Attention-Based RU-BiLSTM Sentiment Analysis Model for Roman Urdu
## 6 Results and Discussion
### Comparison with Baseline Methods  
This section is intended to compare our model with baseline models so as to evaluate the model and to conclude the results. The results of baseline models among with our model are tabulated in Table 6.  
The performance of our proposed method among with SOTA methods of deep learning is reported in Table 6. The efficacy of all models is evaluated using three famous feature representation approaches: word2 vec [18], Glove [59], and FastText [60]. The evaluation metrics used are precision, recall, F1-score, and accuracy. It can be observed that CNN [57] has shown average accuracies of 0.58 and 0.60 under three different feature representation schemes. However, the precision of 0.62 achieved in the case of word2 vec beats many other deep networks. The classic RNN has also shown a performance on par with that of CNN in terms of accuracy; however, the precision score of 0.66 gained under the umbrella of Glove features outperforms all other methods except our proposed model. It is worth mentioning here that classic RNN has shown a poor performance for the FastText feature representation, with an accuracy of 0.51, whereas CNN achieved a better accuracy of 0.60 under the same representation. The RCNN [12] achieved a better score in all three feature representation schemes, with a maximum accuracy of 0.62. On the other hand, GRU [7], which was recently proposed for Roman Urdu sentiment analysis, also showed average accuracies of 0.62 and 0.60. The classic LSTM [56] has also shown a performance equivalent to CNN, RCNN, and GRU, which is against the expectation. Accuracies of 0.62 (word2 vec), 0.59 (FastText), and 0.61 (Glove) were achieved by LSTM. The Bidirectional LSTM (BiLSTM), which was recently proposed for emotion detection [62], has shown a slight improvement and scored an accuracy of 0.64 under the umbrella of word2 vec. Reasonable scores of 0.62 and 0.61 were achieved on FastText and Glove, respectively. In order to mitigate this challenging task, where most of the SOTA methods failed to improve the performance, we propose an attention-based bidirectional network RU-BiLSTM. Our proposed method significantly improved the performance and outperformed the existing models. It can be \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Word Embedding**} & \multicolumn{5}{c}{**Evaluation Metric**} \\ \cline{3-6}  & & **PRE** & **REC** & **F1** & **ACC** \\ \hline SVM [13] & TFIDF & 0.60 & 0.60 & 0.60 & 0.60 \\ \hline \multirow{3}{*}{CNN [57]} & Word2 vec & 0.62 & 0.59 & 0.57 & 0.58 \\  & FastText & 0.60 & 0.60 & 0.60 & 0.60 \\  & Glove & 0.61 & 0.61 & 0.60 & 0.60 \\ \hline \multirow{3}{*}{RNN} & Word2 vec & 0.58 & 0.57 & 0.57 & 0.59 \\  & FastText & 0.53 & 0.51 & 0.50 & 0.51 \\  & Glove & 0.66 & 0.58 & 0.55 & 0.58 \\ \hline \multirow{3}{*}{RCNN [12]} & Word2 vec & 0.63 & 0.62 & 0.62 & 0.62 \\  & FastText & 0.62 & 0.61 & 0.60 & 0.61 \\  & Glove & 0.64 & 0.61 & 0.61 & 0.61 \\ \hline \multirow{3}{*}{GRU [7]} & Word2 vec & 0.63 & 0.62 & 0.62 & 0.62 \\  & FastText & 0.61 & 0.60 & 0.60 & 0.60 \\  & Glove & 0.61 & 0.60 & 0.60 & 0.60 \\ \hline \multirow{3}{*}{LSTM [56]} & Word2 vec & 0.62 & 0.62 & 0.62 & 0.62 \\  & FastText & 0.60 & 0.59 & 0.59 & 0.59 \\  & Glove & 0.61 & 0.61 & 0.61 & 0.61 \\ \hline \multirow{3}{*}{BiLSTM (classic) [62]} & Word2 vec & 0.64 & 0.64 & 0.64 & 0.64 \\  & FastText & 0.63 & 0.62 & 0.062 & 0.62 \\  & Glove & 0.63 & 0.61 & 0.61 & 0.61 \\ \hline \multirow{3}{*}{RU-BiLSTM (Our implementation)} & Word2 vec & 0.68 & 0.67 & 0.67 & 0.67 \\  & FastText & 0.64 & 0.64 & 0.64 & 0.64 \\ \cline{1-1}  & Glove & 0.65 & 0.65 & 0.65 & 0.65 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of different evaluation measures of baseline models with our proposed model for Roman Urdu Sentiment Analysis. observed that our proposed model has shown superior performance in all three embedding schemes and in all four evolutionary measures.