# BERT Models for Arabic Text Classification: A Systematic Review
## 3 Results
### Models Evaluation
\begin{tabular}{c c c} \hline \hline
**Model** & **Outperformed** & **Outperformed By** & **Performance Score** \\ \hline \hline \multicolumn{4}{c}{Multilingual BERT, GigaBERT, XLM-RoBERTa, QARiB, MARBERT, ArabicBERT, ArabicBERT, ARBERT, Arabic ALBERT, TF-IDF, hULMonA, FastText, AraELECTRA, AraAGFT2, BOW + TF-IDF, LSTM, Majority Class, SVM, Mazajak, SVM + Mazajak, BiLSTM, AraGPT, Capsule Network, GRU, CNN, CNN-GRU, AraULMFiT, Mazajak + SVM, AraVec + SVM, fasText + SVM, AraVec + skip-gram (SG), AraVec + CBOW, Doc2 vec + MLP, LSA + SVM, Doc2 vec + SVM, TFIDF + SVM, Word2 vec + SVM, Sentence embedding + LR classifier, TF-IDF + LR classifier, BOW + LR classifier, Multilingual BERT + SVM, ARBERT + SVM \\ \hline \multirow{4}{*}{\begin{tabular}{c} AraBERT \\ (Feature \\ extraction) \\ \end{tabular} } & Multilingual BERT +CNN, AraBERT, FastText-SkipGram+ CNN, FastText-SkipGram + BiLSTM, TF-IDF, BiLSTM, LSTM, Doc2 vec\_MLP, LSA\_SVM, Doc2 vec\_SVM, TFIDF\_SVM, Word2 vec\_SVM, Multilingual BERT + SVM \\ \hline \multirow{4}{*}{\begin{tabular}{c} AraBERT \\ (Fine-tuned) \\ \end{tabular} } & Multilingual BERT +CNN, AraBERT, FastText-SkipGram+ CNN, FastText-SkipGram+ BiLSTM, TF-IDF, BiLSTM, LSTM, Doc2 vec\_MLP, LSA\_SVM, Doc2 vec\_SVM, TFIDF\_SVM, Word2 vec\_SVM, Multilingual BERT + SVM \\ \hline \multirow{4}{*}{\begin{tabular}{c} MARBERT \\ (Fine-tuned) \\ \end{tabular} } & AraBERT, ArabicBERT, ARBERT, Arabic \\  & ALBERT, GigaBERT, QARiB, Multilingual \\  & BERT, XLM-RoBERTa, BiLSTM + Mazajak, \\  & Gaussian Naive Bayes, Gated Recurrent Unit \\  & (GRU), AraELECTRA, BiLSTM, AraGPT, \\  & MTL-LSTM, MTL-CNN, CNN-CE, \\  & CNN-AraVec, \\  & Multi-headed-LSTM-CNNGRU, \\  & CNN-LSTM+ Dialect Information, \\  & Multi-headed-LSTM-CNNGRU+ TF-IDF \\ \hline \multirow{4}{*}{\begin{tabular}{c} ArabicBERT \\ (Feature \\ extraction) \\ \end{tabular} } & AraBert, ARBERT, Arabic ALBERT, \\  & GigaBERT, XLM-RoBERTa, Multilingual \\  & BERT, QARiB, BiLSTM, AraGPT2, \\  & AraVec + skip-gram (SG), AraVec + CBOW, \\  & TF-IDF, CNN-Text, Bi-LSTM, SVM + TF-IDF \\ \hline \multirow{4}{*}{\begin{tabular}{c} ArabicBERT \\ (Feature \\ extraction) \\ \end{tabular} } & AraBERT, Multilingual BERT, CNN-Text, \\  & Bi-LSTM, SVM + TF-IDF \\ \hline \hline \end{tabular} \begin{tabular}{c} ArabicBERT \\ (Feature \\ extraction) \\ \end{tabular}  } & ArabicBERT, Multilingual BERT, CNN-Text, \\  & BiLSTM, SVM + TF-IDF \\ \hline \hline \end{tabular} \begin{tabular}{c} ArabicBert, MARBERT, QARiB, ARBERT, \\ AraBERT + Gradient Boosting Trees (GBT), \\ Multi-Task Learning Model + MarBERT, \\  & Average \\ F1 Score: 0.769 \\ \end{tabular} \begin{tabular}{c} ArabicBert, MARBERT, QARiB, ARBERT, \\ F1 Score: 0.99 \\ Lowest \\ F1 Score: 0.791 \\ \end{tabular} \begin{tabular}{c} ArabicBert, MARBERT, QARiB, ARBERT, \\ F1 Score: 0.606 \\ Average \\ F1 Score: 0.791 \\ \end{tabular}  \ \begin{tabular}{c} ArabicBert, \\ F1 Score: 0.79 \\ Average \\ F1 Score: 0.791 \\ \end{tabular} 
\begin{tabular}{c} ArabicBert, \\ F1 Score: 0.934 \\ArabicBERT also achieved good performance both when finetuned and when used as a feature-extraction method. Similar to AraBERT and Multilingual BERT, it achieved its best performance when it was utilized for feature extraction. It outperformed the five models that it was compared against. The finetuned ArabicBERT outperformed 15 models (71%); seven of them were Arabic BERT models. It was outperformed by six models (29%), five of which were Arabic BERT models. The performance of QARiB was also good. It outperformed 11 out of 18 models (61%). The model was able to outperform all the other eight Arabic BERT models, but was also outperformed by six of them. ARBERT and XLM-RoBERTa scored a mixed performance. ARBERT outperformed six BERT and two non-BERT models. It was outperformed by five BERT and two non-BERT models. XLM-RoBERTa outperformed four BERT and five non-BERT models. It was outperformed by five BERT and one non-BERT model. The performance of the remaining two Arabic BERT models was poor.