Automating Easy Read Text Segmentation (2024)

Abstract: Easy Read text is one of the main forms of access to information for people with reading difficulties. One of the key characteristics of this type of text is the requirement to split sentences into smaller grammatical segments, to facilitate reading. Automated segmentation methods could foster the creation of Easy Read content, but their viability has yet to be addressed. In this work, we study novel methods for the task, leveraging masked and generative language models, along with constituent parsing. We conduct comprehensive automatic and human evaluations in three languages, analysing the strengths and weaknesses of the proposed alternatives, under scarce resource limitations. Our results highlight the viability of automated Easy Read segmentation and remaining deficiencies compared to expert-driven human segmentation.

## introduction being able to access easy read (er) text, also known as easy-to-read, is critical for large segments of the population, including people with cognitive disabilities or suffering from learning difficulties, among others. 1 this type of text has two main characteristics. first, its content needs to consist of short sentences, using simple vocabulary and grammar as in text simplification, while also including an explanation of complex concepts in simpler terms as needed. additionally, sentences need to be split into separate lines, following natural linguistic boundaries, to facilitate reading. 2 although several aspects of er text automation have been studied @xcite @xcite , to our knowledge automatic segmentation to meet er readability requirements has not yet been explored in detail. the closest related domain where readability constraints require text segmentation along natural linguistic boundaries is subtitling. despite this similarity, subtitling features specific constraints, such as the number of characters per line or characters per second, which are tied to the audiovisual environment. for er text, these constraints do not apply and there is a clear gap in our knowledge regarding optimal methods for text segmentation in this context. in this work, we explore several methods for automated segmentation and evaluate them on expertcrafted er content in three languages, namely basque, english and spanish. considering the scarcity of er resources, we focus our study on methods able to generate segmentation hypotheses with either minimal or no training data. we design a scoring-based approach compatible with both pretrained constituency parsers and pretrained masked language models (mlm). additionally, we explore the use of pretrained generative large language models (llm), querying the models in zero-shot or few-shot fashion, as well as fine-tuning models on the limited available data. we test the selected methods on datasets extracted from easy-to-read material collected from trustworthy sources, performing both automated and manual evaluations of the results. our main results show that mlm-based scoring is a stable alternative across metrics, which may serve as a reasonable basis for er text segmentation, although it still lags behind expert-crafted er text. our main contributions can be summarised as follows: aspects of easy read text generation are related to work on text simplification, as it involves standard lexical and grammatical simplification processes @xcite @xcite . our work is more specifically related to easy read content generation constraints, including sentence segmentation, and automated text segmentation approaches. we describe each aspect in turn below. ## methodology the segmentation problem for er text can be formulated as follows. given a source sentence x, compute an ordered sequence of segments [s 1 , s 2 , ..., s n ] such that the concatenation of all segments is identical to the source x, and all segments s i follow natural linguistic boundaries, ending at a position in the sentence where a reader would naturally pause. although intuitively clear, defining what natural linguistic boundaries or natural pauses actually mean can be a challenge. to avoid the need for a more specific definition, we will consider expert-crafted text segmentation as our reference standard and perform a manual evaluation of these reference datasets in section 5.2. we describe in turn below the different approaches we selected for er segmentation. further details regarding the specific models for each language and scenario are presented in section 4.2. ## experimental setup in this section we describe the corpora we prepared for the task, the specific models selected for each approach and language, and our evaluation metrics. ## error analysis segmentation window. to assess the impact of modelling the guidelines recommendations as a fixed window, we performed a grid search by varying the min and max parameters of the scoringbased models. overall, the selected 5-15 window (f1: 40.80) was optimal in english, closely followed by the 1-10 window at 40.14. for spanish, 1-10 achieved 45.22 and 5-10 yielded 44.09, both outperforming our default (42.16). for basque, 5-15 at 41.71 was outperformed by 5-10 at 43.89. the complete results can be found in appendix c. although improvements could have been achieved with different windows, these would have been somewhat marginal. nonetheless, segmentation references outside our selected window would result in systematic errors for the scoring-based approach. there were 0.29%, 0.19% and 0.03% cases of segments over 15 words, for en, es and eu, respectively, of minimal impact. there were however 26.34%, 14.49% and 24.86% cases of segments under 5 words, for en, es and eu, respectively. these cases contributed significantly to the errors of the scoring-based variants. considering the relatively lax minimum number of words per segment in the reference sets, a possible solution would be to use a predefined segmentation window as a soft constraint rather than a hard limit on the segmentation candidates. ungrammatical segmentation. this type of error is the most damaging for the task, as it directly impacts text readability. although it would be difficult to uncover all failure patterns, we manually examined all cases in english where human evaluators assigned the worst score over both grammaticality and reading pause, in the human evaluation, which amounted to 36 examples. wrongly segmented complex named entities accounted for 25% of these errors, with cases such as: now, thibeau works at @xmath0. another pattern which also accounted for 25% of the cases were breaks after words that can act as a function word or as an independent word, e.g.: one person working in the institution told her @xmath1.. one additional pattern emerged, though only in 7% of the cases: split nominal coordination, influenced by the fact that a segment before a coordination can typically act in isolation, as in: policies can be a set of @xmath2. ner preprocessing, heuristics, or fine-tuning the pretrained model on segmentation data could be alternatives worth exploring for these types of errors, at the cost of a more complex pipeline. ## conclusion we described a comprehensive evaluation of automated segmentation for easy read text generation, a critical component of accessible content for people with reading difficulties. we explored a wide range of methods, including scoring-based segmentation with masked language models or constituency parsing, and generative language models via zero-shot or few-shot prompting, and fine-tuning. although limited er resources are available overall, we collected data that supported minimal training and relatively large test sets in basque, english and spanish. in terms of metrics, the unsupervised mlm scoring approach proved optimal overall, although generative models could achieved higher sigma scores. the main deficiency of the latter was the generation of input altering text and low f1 scores for all variants. fine-tuned llms generated the least amount of imperfect text, among generative variants, and might be a worthy alternative for the task. our human evaluations targeted both mlmbased and human segmentation, with the former lagging behind the latter in most cases, although the results were respectable overall across languages. we also identified segmentation error patterns from the mlm approach, which may provide research paths for future improvements.