# A Comprehensive Analysis of Various Tokenizers for Arabic Large Language Models
## 6 Conclusions  
Tokenization serves as a crucial foundation for many NLP tasks involving the breakdown of text streams into basic meaningful units called tokens. These tokens are essential inputs for subsequent NLP tasks such as semantic parsing and language modeling. Despite its importance, there is a notable lack of research on how different tokenization approaches affect the performance of Arabic language models. This gap obstructs our understanding of the impact of various tokenization methods on NLP effectiveness and performance. This study addresses this gap by evaluating the effectiveness of three different tokenizers--SentencePiece, WordPiece, and BBPE--by pretraining three Arabic BERT language models (BERT-SP, BERT-WP, and BERT-BBPE). Each model was pretrained using the same corpus and configurations, but they employed different tokenizers. We assessed the impact of each tokenizer by analyzing the performance of each model across seven NLP task groups using 29 Arabic datasets. Our results show that BERT-SP significantly outperformed the other models in most tasks, thus achieving the highest overall average F1 scores: 84.04% in NER, 48.03% in QA, 75.34% in text classification, 69.05% in SA, 69.06% in SaD, and 72.29% in NLI. On the other hand, the BBPE tokenizer demonstrated superior efficiency, thus recording the shortest pretraining time (133 GPU hours), tokenizer  
Figure 2: Pretraining loss over steps for the three BERT models using SentencePiece (SP), Byte-level BPE (BBPE), and WordPiece (WP) tokenizers.  
\begin{table}
\begin{tabular}{l c c c} \hline \hline  & **BERT-WP** & **BERT-BBPE** & **BERT-SP** \\ \hline Model Pretraining Time (GPU hours) & 140 & **133** & 147 \\ Tokenizer Training Time (minutes) & 7:59 & **6:02** & 7:21 \\ Corpus Tokenization Time & 1:22:38 & **1:08:41** & 1:35:36 \\ Avg. No. Tokens per Sequence & 35.216 & **35.128** & 37.902 \\ Min. Training Loss & 2.3755 & 2.4632 & **2.1534** \\ \hline \hline \end{tabular}
\end{table}
Table 8: Comparative performance of BERT models utilizing different tokenization methods: WordPiece, Byte-level BPE, and SentencePiece. Best results in bold.  
training time (6:02 min), and corpus tokenization time (1:08:41). It also achieved the best compression rate, thus requiring fewer tokens per input sequence and enhancing its speed and processing efficiency. This capability allows it to efficiently handle large input texts. Meanwhile, the WordPiece tokenizer showed consistent performance across all evaluated metrics without excelling in any specific area. These findings contribute to the development of more efficient models and better tokenizer selection, thus enhancing feature engineering and overall advancements in NLP applications. Future work should include a more in-depth analysis to understand how various Arabic dialects affect and influence tokenizer performance, as well as evaluating the performance of various tokenizers with generative language models.  
Conceptualization, F.Q. and T.A; Methodology, F.Q.; Software, F.Q.; Validation, F.Q. and T.A.; Formal analysis, F.Q. and T.A.; Investigation, T.A.; Data curation, F.Q.; Writing--original draft, T.A. and F.Q.; Writing--review & editing, T.A. and F.Q. All authors have read and agreed to the published version of the manuscript. This research received no external funding.  
Not applicable.  
The authors declare no conflicts of interest.