###### Abstract  
The research presented in the following paper focuses on the effectiveness of a modern standard Arabic corpus, AraFast, in training transformer models for natural language processing tasks, particularly in Arabic. In the study described herein, four experiments were conducted to evaluate the use of AraFast across different configurations: segmented, unsegmented, and mini versions. The main outcomes of the present study are as follows: Transformer models trained with larger and cleaner versions of AraFast, especially in question-answering, indicate the impact of corpus quality and size on model efficacy. Secondly, a dramatic reduction in training loss was observed with the mini version of AraFast, underscoring the importance of optimizing corpus size for effective training. Moreover, the segmented text format led to a decrease in training loss, highlighting segmentation as a beneficial strategy in Arabic NLP. In addition, using the study findings, challenges in managing noisy data derived from web sources are identified, which were found to significantly hinder model performance. These findings collectively demonstrate the critical role of well-prepared, segmented, and clean corpora in advancing Arabic NLP capabilities. The insights from AraFast's application can guide the development of more efficient NLP models and suggest directions for future research in enhancing Arabic language processing tools. Arable corpora; Arabic corpus; Arabic NLP; Arabic dataset; transformer model; question answering +
Footnote †: journal:  
demonstrated remarkable independent learning abilities in discovering relevant language representations without supervision. Consistent access to large English corpora gives English models an advantage over models for other languages [8]. A comprehensive Arabic corpus that consists of authentic Arabic texts is essential for in-depth linguistic analysis. The findings presented in the following article emphasize the significance of collecting an Arabic corpus and its implications for linguistic research. To this end, roughly 48 corpora were gathered from various sources, including dataset repositories mentioned in scientific papers (32 corpora) and repositories without accompanying publications (16 corpora). The majority of the collected corpora focus on modern standard Arabic (MSA) and classical Arabic (CA), excluding Arabic dialects. The following research questions were addressed in this present study:  
RQ1: What freely available MSA corpora can be found online? RQ2: Does pretraining a transformer model on a large Arabic corpus that includes web-scraped data lead to worse performance than training without web-scraped data and noise? RQ3: Does pretraining a transformer model on a segmented Arabic corpus lead to better performance than training the model on an unsegmented Arabic corpus? RQ4: How does pretraining a transformer model on a large Arabic corpus impact its accuracy in NLP tasks, specifically Arabic QA tasks? The contributions of the present study are as follows:  
1. A detailed exploration and critical evaluation of freely accessible modern standard Arabic (MSA) corpora from various online sources is performed. 2. ArafFast, a comprehensive and large-scale Arabic corpus, is introduced to support advanced NLP tasks. 3. An extensive assessment of the performance impacts of using different versions of the Arabic corpus, including segmented, unsegmented, mini, and noisy variants, in training NLP models is conducted. The remainder of this paper is organized as follows: Section 2 highlights the motivations and main objectives of the present study. Section 3 presents a comprehensive review of the literature on the collection of Arabic corpora. Section 4 describes the research methodology employed in this study in detail. Section 5 details four experiments focused on training an Arabic transformer model using the ArafFast corpus. Section 6 presents the use of the corpus in developing two transformer models and a discussion of the results.