# A Comprehensive Analysis of Various Tokenizers for Arabic Large Language Models
## 5 Results
\begin{table}
\begin{tabular}{l l l l l l l l l l l} \hline \hline  & & \multicolumn{3}{c}{**BERT-WP**} & \multicolumn{3}{c}{**BERT-SP**} & \multicolumn{3}{c}{**BERT-BBPE**} & & \\ \hline
**Dataset** & **Type** & **Acc.** & **F1** & **Acc.** & **F1** & **Acc.** & **F1** & **Train Size** & **Validation Size** & **\#Labels** \\ \hline Mawqif (sentiment) & DA & 71.46 & **69.82** & **71.61** & 68.96 & 71.32 & 68.49 & 2.8 K & 0.7 K & 3 \\ ArSarcasmV2 (sentiment) & DA & **68.23** & **63.92** & 67.83 & 63.80 & 66.70 & 62.72 & 12.54 K & 3 K & 3 \\ HARD (balanced) & MSA + DA & 89.04 & **86.84** & **89.10** & 86.81 & 89.04 & **86.84** & 84.55 K & 21.14 K & 2 \\ HARD (unbalanced) & MSA + DA & **90.44** & 85.55 & 90.42 & **85.65** & 90.30 & 85.52 & 327.64 K & 81.91 K & 3 \\ ASTD & MSA + DA & 74.27 & 55.68 & **75.42** & **57.40** & 75.22 & 56.43 & 8 K & 2 K & 4 \\ BRAD & MSA & **52.90** & **51.79** & 52.73 & 51.71 & 52.55 & 51.62 & 408.47 K & 102.12 K & 5 \\ \hline Avg & & & 74.39 & 68.93 & **74.51** & **69.05** & 74.18 & 68.60 & - & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of BERT models using different tokenizers: WordPiece (BERT-WP), SentencePiece (BERT-SP), and Bytee-level BPE (BERT-BBPE) for the SA task. For the QA task, Table 6 shows that all three models achieved comparable results across all datasets, which is due to the high complexity of the QA NLP tasks. On average, BERT-SP achieved a slightly better F1 score of 48.03%, while BERT-BBPE recorded the highest EM score at 28.99%. For the NER task, Table 7 presents the F1 scores for each model on the datasets used. The results indicate that BERT-SP significantly outperformed the other models on five out of six tasks, thus achieving the highest overall average score of 84.04%. This suggests that the SentencePiece tokenizer is the most effective approach for NER tasks, regardless of the size of the dataset or the number of labels. In our comparative analysis of BERT models using different tokenizers--WordPiece (BERT-WP), Byte-level BPE (BERT-BBPE), and SentencePiece (BERT-SP)--we measured several performance metrics, including the pretraining time, tokenizer training time, and minimum training loss. Table 8 shows that BERT-BBPE was the most efficient in terms of the training and tokenization time, thus achieving the shortest model pretraining time at 133 GPU hours, tokenizer training time at 6:02 min, and corpus tokenization time at 1:08:41. These results indicate that BERT-BBPE is highly effective in terms of speed and processing efficiency. Additionally, the BBPE tokenizer achieved the best compression rate, thus requiring fewer tokens to encode input texts. However, BERT-BBPE recorded the highest training loss during model pretraining, thus leading it to underperform compared to the other two models in the majority of evaluation tasks, except for the topic classification task, where its efficient compression rate was beneficial. On the other hand, despite its longer tokenization and training time, BERT-SP achieved the lowest training loss at 2.1534 over the same number of training epochs as the other models, thus demonstrating its effectiveness in learning from the training data. Figure 2 illustrates the training loss over the number of steps for each model. In contrast, BERT-WP, while not excelling in any specific category, showed consistent performance across all metrics. This analysis highlights the trade-offs between the tokenizer choice and aspects of