A Large and Balanced Corpus for Fine-grained A rabic Readability Assessment (2025)

Abstract: This paper introduces the Balanced Arabic Readability Evaluation Corpus (BAREC), a large-scale, fine-grained dataset for Arabic readability assessment. BAREC consists of 69,441 sentences spanning 1+ million words, carefully curated to cover 19 readability levels, from kindergarten to postgraduate comprehension. The corpus balances genre diversity, topical coverage, and target audiences, offering a comprehensive resource for evaluating Arabic text complexity. The corpus was fully manually annotated by a large team of annotators. The average pairwise inter-annotator agreement, measured by Quadratic Weighted Kappa, is 81.8%, reflecting a high level of substantial agreement.Beyond presenting the corpus, we benchmark automatic readability assessment across different granularity levels, comparing a range of techniques. Our results highlight the challenges and opportunities in Arabic readability modeling, demonstrating competitive performance across various methods.To support research and education, we make BAREC openly available, along with detailed annotation guidelines and benchmark results: http://barec.camel-lab.com.

## introduction text readability impacts understanding, retention, reading speed, and engagement @xcite . texts above a student's readability level can lead to disengagement @xcite . @xcite highlighted that readability and legibility depend on both external features (e.g., production, fonts) and content. text leveling in classrooms helps match books to students' reading levels, promoting independent reading and comprehension @xcite . developing readability models is crucial for improving literacy, language learning, and academic performance. readability levels have long been a key component of literacy teaching and learning. one of the most widely used systems in english literacy is fountas and pinnell @xcite , which employs qualitative measures to classify texts into 27 levels (a to z+), spanning from kindergarten to adult proficiency. similarly, taha-thomure (2017)'s system for arabic has 19 levels from arabic letters a to q. these fine-grained levels are designed for pedagogical effectiveness, ensuring young readers experience gradual, measurable progress, particularly in early education (k-6) @xcite . a key advantage is that they can be easily mapped to coarser levels with fewer categories, which may be more efficient for broader applications in readability research and automated assessments. in this paper we present the balanced arabic readability evaluation corpus (barec) -a largescale fine-grained readability assessment corpus across a broad space of genres and readability levels. inspired by the taha/arabi21 readability reference @xcite , which has been instrumental in tagging over 9,000 children's books, barec seeks to establish a standardized framework for evaluating sentence-level foot_0 arabic text readability across 19 distinct levels, ranging from kindergarten to postgraduate comprehension. our contributions are: (a) a large-scale curated corpus with 69k+ sentences (1m+ words) spanning diverse genres; and (b) benchmarking of automatic readability assessment models across multiple granularities, including both fine-grained (19 levels) and collapsed tiered systems (e.g., fivelevel and three-level scales) to support various research and application needs, aligning with previous arabic readability frameworks @xcite . ## barec corpus annotation in this section, we summarize the guidelines and annotation process. for more details, see @xcite . in the next section, we discuss corpus selection and statistics. ## barec corpus corpus selection in the process of corpus selection, we aimed to cover a wide educational span as well as different domains and topics. we collected the corpus from 1,922 documents, which we manually categorized into three domains: arts & humanities, social sciences, and stem (details in appendix c.2) and three readership groups: foundational, advanced, and specialized (details in appendix c.3). ## conclusions and future work this paper presented the balanced arabic readability evaluation corpus (barec), a largescale, finely annotated dataset for assessing arabic text readability across 19 levels. with over 69k sentences and 1 million words, it is the largest arabic corpus for readability assessment, covering diverse genres, topics, and audiences, to our knowledge. high inter-annotator agreement ensures reliable an-notations. through benchmarking various readability assessment techniques, we highlighted both the challenges and opportunities in arabic readability modeling, demonstrating promising performance across different methods. looking ahead, we plan to expand the corpus, enhancing its size and diversity to cover additional genres and topics. we also aim to add annotations related to vocabulary leveling and syntactic treebanks to study less-explored genres in syntax. future work will include analyzing readability differences across genres and topics. additionally, the tools we have developed will be integrated into a system to help children's story writers target specific reading levels. the barec dataset, its annotation guidelines, and benchmark results, will be made publicly available to support future research and educational applications in arabic readability assessment. alc for their continued support. we would also like to thank the zayed university zai arabic language research center team, in particular hamda al-hadhrami, maha fatha, and metha talhak, for their valuable contributions to typing materials for the project. we also acknowledge ali gomaa and his team for their additional support in this area. finally, we thank our colleagues at the new york university abu dhabi computational approaches to modeling language (camel) lab, muhammed abu odeh, bashar alhafni, ossama obeid, and mostafa saeed, as well as nour rabih (mohamed bin zayed university of artificial intelligence) for their helpful conversations and feedback.