Hybrid Models for Sentence Readability Assessment (2023)

Abstract: Automatic readability assessment (ARA) predicts how difficult it is for the reader to understand a text. While ARA has traditionally been performed at the passage level, there has been increasing interest in ARA at the sentence level, given its applications in downstream tasks such as text simplification and language exercise generation. Recent research has suggested the effectiveness of hybrid approaches for ARA, but they have yet to be applied on the sentence level. We present the first study that compares neural and hybrid models for sentence-level ARA. We conducted experiments on graded sentences from the Wall Street Journal (WSJ) and a dataset derived from the OneStopEnglish corpus. Experimental results show that both neural and hybrid models outperform traditional classifiers trained on linguistic features. Hybrid models obtained the best accuracy on both datasets, surpassing the previous best result reported on the WSJ dataset by almost 13% absolute.

## introduction text readability is defined as the cognitive load of a reader to comprehend a text @xcite . research on automatic readability assessment (ara) has traditionally aimed at passages (azpiazu and pera, 2019), e.g., labeling a passage with its difficulty level. there has been growing interest in assessing the difficulty of individual sentences @xcite @xcite , given its application in various downstream tasks in natural language processing (nlp). it is essential to generation tasks that are sensitive to language difficulty, such as pedagogical material and exercises @xcite . it also facilitates explainable text simplification @xcite by identifying which sentences require simplification. sentence-level ara is a task in its own right since a substantial drop in performance has been observed when passage-level ara models are applied on individual sentences @xcite . similar to many other nlp tasks, passage-level ara has benefited from the advent of neural approaches @xcite @xcite . recent research has also applied 'hybrid' models, which leverage both linguistically motivated features and neural models @xcite @xcite . for sentence-level ara, although neural models have been evaluated @xcite , there has not been any attempt to integrate linguistic features. this paper applies neural models and hybrid models on sentence-level ara and compares their performance with a non-neural classifier trained on linguistic features. to our knowledge, this is the first study on hybrid models for sentencelevel ara. experimental results show that a hybrid model offers the best performance, and surpasses the previous best result reported on the wall street journal dataset @xcite . foot_0 2 previous work ## data we used the following two datasets in our experiments. detailed statistics are shown in ## conclusion we have presented the first study on hybrid models on automatic readability assessment (ara) at the sentence level. our contribution is two-fold. first, we demonstrated that hybrid models outperform neural models, suggesting that linguistic features can capture salient properties that indicate sentence difficulty. second, we compared three types of hybrid model, and showed that using the neural model's predictions as features in a traditional classifier yielded the best result, surpassing the previous best published result on the wsj dataset by almost 13% absolute. these experimental results are expected to help inform future research on sentence-level ara.