# Text Classification of Patient Experience Comments in Saudi Dialect Using Deep Learning Techniques
## 4 Experimental Results
### Results
#### 4.3.3 All Sentiment Data Set (19K with 20 Classes)  
In this experiment, we investigated whether reduction in the number of classes would have an effect on the performance of the models. For this purpose, the classes that had fewer than 300 entries were combined into one class, which we labeled as _other_. The combined classes were Consent, Finance_Billing, Referrals, Confidentiality, Diagnosis, and Patient_Disposition. This resulted in 860 entries in the class _other_. The performance of the models when trained using this data set is detailed in Table 20. As can be seen from  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **Accuracy** & **Macro F1 Score** &
\begin{tabular}{c} **Macro Precision** \\ \end{tabular} & **Macro Recall** \\ \hline BiLSTM + AraVec & 54.00\% & 18.49\% & 19.00\% & 18.00\% \\ Tuned BiLSTM + AraVec & 53.34\% & 19.00\% & 19.00\% & 19.00\% \\ BiGRU + AraVec & 47.57\% & 25.25\% & 28.00\% & 23.00\% \\ Tuned BiGRU + AraVec & 55.44\% & 10.91\% & 12.00\% & 10.00\% \\ BiLSTM\_PX\_Vec & 54.66\% & 14.07\% & 17.00\% & 12.00\% \\ BiLSTM\_PX\_Vec (10 folds) & 33.25\% & 15.77\% & 23.00\% & 12.00\% \\ BiGRU\_PX\_Vec & 47.84\% & 27.27\% & 30.00\% & 25.00\% \\ BiGRU\_PX\_Vec (10 folds) & 34.78\% & 30.00\% & 40.00\% & 24.00\% \\ AraBERTV02 & 60.24\% & 38.83\% & 81.22\% & 25.51\% \\ MarBERTv2 & 56.92\% & 42.41\% & 80.51\% & 28.79\% \\ Qarib & 57.60\% & 41.55\% & 71.77\% & 29.24\% \\ PX\_BERT & 55.14\% & 43.06\% & 63.61\% & 32.54\% \\ \hline \hline \end{tabular}
\end{table}
Table 18: The performance of models using the negative-only (13K) data set.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **Accuracy** & **Macro F1 Score** &
\begin{tabular}{c} **Macro Precision** \\ \end{tabular} & **Macro Recall** \\ \hline BiLSTM\_PX\_Vec & 61.26\% & 26.21\% & 38.00\% & 20.00\% \\ BiLSTM\_PX\_Vec (10 folds) & 53.00\% & 32.69\% & 44.00\% & 26.00\% \\ BiGRU\_PX\_Vec & 62.00\% & 34.64\% & 43.00\% & 29.00\% \\ BiGRU\_PX\_Vec (10 folds) & 54.00\% & 42.88\% & 53.00\% & 36.00\% \\ AraBERTv02 & 57.95\% & 47.10\% & 64.12\% & 37.22\% \\ MarBERTv2 & 57.21\% & 42.68\% & 80.62\% & 29.02\% \\ Qarib & 67.97\% & 47.00\% & 63.65\% & 37.26\% \\ PX\_BERT & 55.84\% & 43.07\% & 67.45\% & 31.63\% \\ \hline \hline \end{tabular}
\end{table}
Table 19: The performance of models using the all-sentiment (19K) data set.  
the results, the accuracy values for all the models ranged between 40.26% and 60.02%, where the fine-tuned BERT-based models appeared to perform better than the DL models in general. Additionally, the BiLSTM and BiGRU with PX_Vec embeddings were better than the cross-validation versions, exceeding them by at least 12%. In terms of precision, the values ranged between 42% and 66%, with the best being AraBERT and the worst being BiLSTM with PX_Vec embeddings. The BERT-based models mainly performed better than the DL models, where the 10-fold cross-validation versions of the DL model performed roughly 10% better than those without cross validation. However, in terms of recall, the values ranged between 26% and 42%, and the best-performing model is the BiGRU with PX_Vec embeddings with 10-fold cross validation, while the worst is the BiLSTM with PX_Vec embeddings. In terms of F1 score, the scores ranged between 32.12% and 48.7%, the best being AraBERT and the worst being BiLSTM with PX_Vec embeddings. Generally speaking, the performance of all models was improved, when compared to their performance for the 19K data set.