Towards Sentence-level Text Readability Assessment for F rench (2023)

Abstract: In this paper, we report on some experiments aimed at exploring the relation between document-level and sentence-level readability assessment for French. These were run on an open-source tailored corpus, which was automatically created by aggregating various sources from children’s literature. On top of providing the research community with a freely available corpus, we report on sentence readability scores obtained when applying both classical approaches (aka readability formulas) and state-of-the-art deep learning techniques (e.g. fine-tuning of large language models). Results show a relatively strong correlation between document-level and sentence-level readability, suggesting ways to reduce the cost of building annotated sentence-level readability datasets.

## introduction text readability assessment can be defined as the ability to automatically estimate the difficulty for someone to understand a given text. while it was primarily designed for selecting materials for textbooks @xcite and based on statistical formulas modelling lexical and syntactic complexity, it has proved useful in many other contexts, such as evaluation of text simplification systems @xcite , and has been extended to modern neural architectures @xcite . depending on the context, estimating readability can take different forms (and rely on different scales) 1 . when aiming at assigning a textbook to pupils, a common scale corresponds to pupils' age. when aiming at assigning learning materials to second-language learners, a common scale corresponds to the common european framework of reference (cefr) for languages (council of @xcite . in some contexts, assessing readability amounts to classifying a given text as simple or complex (e.g. when learning to assess readability on binary corpora such as the geo-geolino corpus for german @xcite ). linear regression models developed for english (such as that of @xcite ) were capable of capturing some degree of lexical and syntactic complexity. some of these were later adapted to other languages such as french @xcite . still, some studies (e.g. @xcite ) showed that they do not always correlate with field data. various attempts at using machine learning techniques for (mainly english) text readability assessment have been carried out since the seminal work of @xcite , who combined statistical language models with surface linguistic features extracted from large datasets. one may cite in particular the work of @xcite in deep machine learning, where authors developed specific word embeddings and neural architecture. readability assessment for french was revisited by @xcite , who explored various statistical algorithms and experimented with several linguistic features. recent advances in this domain include work by @xcite who considered psycholinguistic features (e.g. emotional impact of texts), and by @xcite who fine-tuned a pretrained bert model for cefr classification for french @xcite . as pointed out by @xcite , a common bottleneck in machine learning-based french text readability assessment lies in the scarcity of useful (e.g. labelled) resources. we build upon their work to provide researchers with a tailored and open-source corpus while studying the relation between sentence-level and documentlevel readability assessment. the main contribution of this work is thus the compilation of an average size (1,228 documents) freely available corpus for french as a first language readability assessment, which has been preprocessed to remove noisy data and used to perform sentence-level automatic readability assessment with state-of-the-art bert architectures, giving results in line with those obtained at the document level @xcite . 2 ## existing models and datasets in this section, we briefly discuss the existing readability models and datasets related to our work. these models belong to two main categories: readability formulas and (deep and non-deep) machine learning-based approaches. ## experimental framework in this work, we are studying sentence-level readability assessment for french. the readability scale we are using comes from the context of this work, namely the implementation of a computer assisted language learning environment for french l1 learners. we consider the five following levels (classes): 0 emergent readers 1 short and easy texts 2 long and easy texts 3 lower-intermediate texts 4 upper-intermediate texts these levels are loosely related to the french primary school curriculum, and match the categories available in the online resource used to create our corpus (namely storyweaver, see below). ## conclusion in this paper, we presented a freely available corpus for french sentence-level text readability, which was automatically extracted from online resources and evaluated against state-of-the art deep-learning techniques. results show some correlation between document-level and sentence-level readability assessment, which suggests that extending training corpora could be done by considering labelled documents, thus saving annotation costs.