A Dataset of Word-Complexity Judgements from Deaf and Hard-of-Hearing Adults for Text Simplification (2022)

Abstract: Research has explored the use of automatic text simplification (ATS), which consists of techniques to make text simpler to read, to provide reading assistance to Deaf and Hard-of-hearing (DHH) adults with various literacy levels. Prior work in this area has identified interest in and benefits from ATS-based reading assistance tools. However, no prior work on ATS has gathered judgements from DHH adults as to what constitutes complex text. Thus, following approaches in prior NLP work, this paper contributes new word-complexity judgements from 11 DHH adults on a dataset of 15,000 English words that had been previously annotated by L2 speakers, which we also augmented to include automatic annotations of linguistic characteristics of the words. Additionally, we conduct a supplementary analysis of the interaction effect between the linguistic characteristics of the words and the groups of annotators. This analysis highlights the importance of collecting judgements from DHH adults for training ATS systems, as it revealed statistically significant interaction effects for nearly all of the linguistic characteristics of the words.

## the dataset our dataset was originally gathered by researchers in @xcite by selecting the 15,000 most frequent words in google's it ngram corpus. word-complexity judgements on all 15,000 words were also obtained from 11 l2 english speakers using a 6-point scale, going from "very simple" (1) to "very complex" (6), and using 6 points to avoid a neutral choice @xcite . in this new work, we expand this dataset by obtaining word-complexity judgements from 11 dhh annotators following that prior approach, and we also compute several linguistic characteristics of the words. the selection of these linguistic characteristics was based on prior work, which had identifed linguistic characteristics (e.g., word length or number of syllables) that had affected text complexity for various reader groups @xcite . ## discussion and conclusion our results provide further evidence for the importance of gathering judgments from intended audiences to train systems using datasets based on those judgements. prior work had suggested that the linguistic characteristics that affect text complexity for different user groups may vary (e.g. word length may affect word complexity for people with dyslexia but less so for l2 speakers) @xcite . through our analysis of interaction effects, we observed that the way several linguistic characteristics affect word-complexity judgements depends indeed on the group of annotators that provide those judgements. thus, it is important to gather judgements from target audiences and build models based on those judgements, which may better capture the nuanced relations between how these different features impact word complexity for target audiences.