## 4 Results and Analysis
### Original Sentiment of the Arb-AppsReview Dataset  
The experimental results in Table 10 show the performance of ML models that are trained on features extracted by BoW, keeping the sentiments of the original Arb-AppsReview dataset. According to the evaluation metrics, the NB classifier performed better than the other models. It yielded the highest accuracy of 74.25%, a precision of 0.74%, a recall of 0.743, and an F1-score of 0.735. On the contrary, LR acquired the lowest values in terms of accuracy as well as recall, whereas, in terms of precision and F1-score, RF had the worst value. In addition, from the results shown in Figure 5, LR predicted a neutral class with the highest true-positive rate, whereas, in terms of positive class and negative class, NB remains the leading ML model.  
\begin{table}
\begin{tabular}{l l} \hline
**ML Classifier** & **Optimized Hyperparameters’ Settings** \\ \hline RF & Criterion = “entropy”; max\_depth = 1500; min\_samples\_leaf = 7; min\_samples\_split = 5; n\_estimators = 200 \\ Bagging & n\_estimators: 100; max\_samples = 0.5, max\_features = 0.5 \\ SVM & C = 0.1; Gamma = \(1\times 10^{-4}\); Kernel = “Linear” \\ LR & C = \(1\times 10^{-3}\), fit\_intercept = True \\ NB & alpha = \(1\times 10^{-5}\); fit\_prior = True \\ \hline \end{tabular}
\end{table}
Table 9: The optimized hyperparameters’ settings.  
Similarly, with features extracted by TF-IDF, NB produced the highest accuracy of 68.86%, as well as the highest precision, recall, and F1-score of 0.690, 0.689, and 0.671, respectively, as shown in Table 11. Meanwhile, the lowest accuracy was acquired by RF of 58.67% and LR of 58.68%.  
Although the NB model achieved the highest accuracy, it did not deliver optimized results in the prediction of the positive class, as the TP rate achieved by the bagging algorithm was the highest as compared to other ML models, as shown in Figure 6.  
\begin{table}
\begin{tabular}{l l l l l} \hline
**Classifier** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline RF & 68.86\% & 0.674 & 0.689 & 0.669 \\ Bagging & 72.43\% & 0.732 & 0.719 & 0.725 \\ SVM & 71.26\% & 0.706 & 0.713 & 0.696 \\ LR & 65.87\% & 0.720 & 0.659 & 0.675 \\ NB & 74.25\% & 0.740 & 0.743 & 0.735 \\ \hline \end{tabular}
\end{table}
Table 10: Experiments with ML models and BoW on original sentiments.  
Figure 5: True-positive rate (TP Rate) and false-positive rate (FP Rate) of ML models with original sentiments using BoW.  
In the case of the concatenated feature as well as the Word2 Vec representation, as shown in Tables 12 and 13, RF benefited more, and the highest accuracy achieved was when the features were extracted by Word2 Vec. Additionally, the performance of LR improved with the features extracted by joining the BoW and TF-IDF approaches, \(Con^{1}_{feature}\). In addition, using the concatenated features, \(Con^{4}_{feature}\) led to a notable improvement in the performance of SVM. Similarly, the performance of LR increased with combining features using \(Con^{2}_{feature}\).  
In addition, from Figures 7 and 8, it can be observed that most ML classifiers performed well in predicting the positive class. In terms of the TP rate, SVM yielded the highest score for predicting the positive class, whereas, in the case of the negative class, NB outperformed the other models.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Classifier** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline RF & 76.6467\% & 0.809 & 0.766 & 0.726 \\ Bagging & 74.2515\% & 0.693 & 0.743 & 0.697 \\ SVM & 70.6587\% & 0.529 & 0.707 & 0.605 \\ LR & 67.6647\% & 0.725 & 0.677 & 0.695 \\ NB & 68.8623\% & 0.663 & 0.689 & 0.670 \\ \hline \hline \end{tabular}
\end{table}
Table 13: Experiments with ML models and Word2 Vec on original sentiments.  
Figure 6: True-positive rate (TP Rate) and false-positive rate (FP Rate) of ML models with original sentiments using TF-IDF.