# A Comprehensive Analysis of Various Tokenizers for Arabic Large Language Models
## 5 Results  
This section presents the models' evaluation results across the target NLP tasks. Table 1 presents the results of the models across all datasets employed for NLI and SD tasks. The table includes training and validation sizes for each dataset, along with the number of labels. For this experiment, we trained the BERT model on five datasets. The results indicate that BERT-WP achieved the highest accuracy scores only on the AraStance and Mawqif datasets, with scores of 84.53% and 77.31%, respectively, while BERT-BBPE achieved the highest F1 score on the AraStance dataset alone. However, BERT-SP demonstrated superiority across the majority of datasets, thus achieving the highest accuracy and F1 scores for most tasks, with overall average scores of 75.92% for accuracy and 72.29% for F1. The results suggest that BERT-SP is a more effective tokenizer for NLI and SD tasks compared to BERT-WP and BERT-BBPE. Moreover, no significant differences were observed between the MSA and DA types. For the SaD task, Table 2 presents the results of BERT models evaluated across all four datasets included. It also provides the details for the training and validation sizes for each dataset. Overall, BERT-SP has outperformed the other two models on the majority of tasks, achieving the highest average accuracy and F1 scores at 79.72% and 69.06%, respectively. BERT-WP achieved the second highest scores for both accuracy and F1, while BERT-BBPE had the lowest overall average scores. For the SA task, Table 3 shows that the performance of all models was quite similar. We fine-tuned the BERT models on six datasets, each with different training and validation sizes. BERT-SP achieved the highest overall average accuracy and F1 scores, at 74.51% and 69.05%, respectively. Meanwhile, BERT-WP achieved the highest F1 score on four \begin{table}
\begin{tabular}{l l l l l l l l l l} \hline \hline  & & \multicolumn{2}{c}{**BERT-WP**} & \multicolumn{2}{c}{**BERT-SP**} & \multicolumn{2}{c}{**BERT-BBPE**} & & \\ \hline
**Dataset** & **Type** & **Acc.** & **F1** & **Acc.** & **F1** & **Acc.** & **F1** & **Train Size** & **Validation Size** & **\#Labels** \\ \hline XNLI & MSA & 74.1 & 74.22 & **74.53** & **74.54** & 73.57 & 73.64 & 393 K & 7.5 K & 3 \\ MultilingualNLI & MSA & 66.45 & 65.81 & **66.86** & **66.13** & 65.46 & 64.74 & 84 K & 21 K & 3 \\ AnsStance & MSA & 71.52 & 65.49 & **78.67** & **74.99** & 77.88 & 73.46 & 3.03 K & 0.75 K & 3 \\ AraStance & MSA & **84.53** & 79.16 & 82.24 & 77.38 & 84.35 & **79.75** & 3.49 K & 0.56 K & 4 \\ Mawqif (stance) & DA & **77.31** & 66.93 & **77.31** & **68.45** & 75.03 & 65.88 & 2.8 K & 0.7 K & 3 \\ \hline Avg & & 74.78 & 70.32 & **75.92** & **72.29** & 75.25 & 71.49 & - & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of BERT models using different tokenizers: WordPiece (BERT-WP), Sentence-Piece (BERT-SP), and Byte-level BPE (BERT-BBPE) for the NLI and SD tasks. \begin{table}
\begin{tabular}{l l l l l l l l l l l} \hline \hline  & & \multicolumn{2}{c}{**BERT-WP**} & \multicolumn{2}{c}{**BERT-SP**} & \multicolumn{2}{c}{**BERT-BBPE**} & & \\ \hline
**Dataset** & **Type** & **Acc.** & **F1** & **Acc.** & **F1** & **Acc.** & **F1** & **Train Size** & **Validation Size** & **\#Labels** \\ \hline Mawqif (sarcasm) & DA & 96.00 & 68.53 & **96.43** & **72.48** & 94.43 & 67.59 & 2.8 K & 0.7 K & 2 \\ Saudilrony & DA & 71.63 & **70.45** & **71.70** & **70.45** & 70.84 & 69.97 & 15.7 K & 3.9 K & 2 \\ iSarcasmEval (sarcasm) & DA & 72.64 & 61.83 & **74.14** & **63.01** & 66.21 & 57.11 & 3.1 K & 1.4 K & 2 \\ ArSarcasmV2 (sarcasm) & DA & **77.26** & **70.84** & 76.63 & 70.32 & 75.96 & 70.78 & 12.54 K & 3 K & 2 \\ \hline Avg & & 79.38 & 67.91 & **79.72** & **69.06** & 76.86 & 66.36 & - & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of BERT models using different tokenizers: WordPiece (BERT-WP), Sentence-Piece (BERT-SP), and Byte-level BPE (BERT-BBPE) for the SaD tasks. individual datasets and the highest accuracy on three.