Strategies for A rabic Readability Modeling (2024)

Abstract: Automatic readability assessment is relevant to building NLP applications for education, content analysis, and accessibility. However, Arabic readability assessment is a challenging task due to Arabic’s morphological richness and limited readability resources. In this paper, we present a set of experimental results on Arabic readability assessment using a diverse range of approaches, from rule-based methods to Arabic pretrained language models. We report our results on a newly created corpus at different textual granularity levels (words and sentence fragments). Our results show that combining different techniques yields the best results, achieving an overall macro F1 score of 86.7 at the word level and 87.9 at the fragment level on a blind test set. We make our code, data, and pretrained models publicly available.

## introduction the task of automatic readability assessment aims at modeling the reading and comprehension difficulty of a given piece of text for a particular target audience. this is relevant to building and enhancing pedagogical natural language processing (nlp) applications, which aid students in language learning @xcite , help teachers with designing curricula and writing assessments @xcite , and enable the personalization of nlp systems' output to target users with different readability levels @xcite . research on english automatic readability assessment have garnered substantial interest in terms of dataset creation @xcite @xcite and modeling advancements @xcite @xcite . in contrast, other languages such as arabic have not received as much attention. we discuss related work in §2, provide an overview of our dataset in §3, describe our models for arabic readability assessment in §4, and discuss results in §5. ## approach we present below the set of models we use for word-level and fragment-level readability labeling. ## results we present and discuss the results of our evaluation below. the complete set of results for word-level and fragment-level labeling across all experimental setups is available in appendix a. ## conclusions and future work we explored the problem of arabic readability assessment using a diverse set of approaches relying on frequency and rule-based models as well as arabic pretrained language models (plms). we reported results using a newly manually created corpus at both the word and fragment levels. we further highlighted the strengths and weaknesses of each approach and underscored the importance of employing different strategies to address arabic readability assessment effectively. our findings demonstrate that combining different modeling techniques yields the best results, achieving an overall macro f 1 score of 86.7 at the word level and 87.9 at the fragment level. in future work, we plan to explore the effect of various linguistic features in enhancing machine learning models for arabic readability assessment. we plan to continue to improve basic enabling technologies such as morphological disambiguation and lemmatization and study their effect on readability models. we further plan to employ our best results in the development of online tools to support pedagogical nlp applications.