The Fewer Splits are Better: Deconstructing Readability in Sentence Splitting (2022)

Abstract: In this work, we focus on sentence splitting, a subfield of text simplification, primarily motivated by an unproven idea that if you divide a sentence into pieces, it should become easier to understand. Our primary goal in this paper is to determine whether this is true. In particular, we ask, does it matter whether we break a sentence into two or three? We report on our findings based on Amazon Mechanical Turk. More specifically, we introduce a Bayesian modeling framework to further investigate to what degree a particular way of splitting the complex sentence affects readability, along with a number of other parameters adopted from diverse perspectives, including clinical linguistics, and cognitive linguistics. The Bayesian modeling experiment provides clear evidence that bisecting the sentence leads to enhanced readability to a degree greater than when we create simplification by trisection.

## introduction in text simplification, one question people often fail to ask is, whether the technology they are driving truly helps people better understand texts. this curious indifference may reflect the tacit recognition of the partiality of datasets covered by the studies @xcite or some murkiness that surrounds the goal of text simplification. as a way to address the situation, we examine a role of simplification in text readability, with a particular focus on sentence splitting. the goal of sentence splitting is to break a sentence into small pieces in a way that they collectively preserve the original meaning. a primary question we ask in this paper is, does a splitting of text affect readability? in the face of a large effort spent in the past on sentence splitting, it comes as a surprise that none of the studies put this question directly to people; in most cases, they ended up asking whether generated texts 'looked simpler' than the original unmodified versions @xcite , which of course does not say much about their readability. we are not even sure whether there was any agreement among people on what constituted simplification. another related question is, how many pieces should we break a sentence into? two, three, or more? in the paper, we focus on a particular setting where we ask whether there is any difference in readability between two-and threesentence splits. we also report on how good or bad sentence splits are that are generated by a finetuned language model, compared to humans'. a general strategy we follow in the paper is to elicit judgments from people on whether simplification made a text anyway readable for them (section 4), and do a bayesian analysis of their responses to identify factors that may have influenced their decisions (section 5). 1 2 related work historically, there have been extensive efforts in esl (english as a second language) to explore the use of simplification as a way to improve reading performance of l2 (second language) students. @xcite presented an array of evidence showing that simplifying text did lead to an improved text comprehension by l2 learners as measured by reading time and and accuracy of their responses to associated questions. they also noticed that simple texts had less lexical diversity, greater word overlap, greater semantic similarity among sentences than more complicated texts. @xcite argued for the importance of cohesiveness as a factor to influence the readability. meanwhile, an elaborative modification of text was found to play a role in enhancing readability, which involves adding information to make the language less ambiguous and rhetorically more explicit. @xcite reported that despite the fact that it made a text longer, the elaborative manipulation of a text produced positive results, with l2 students scoring higher in comprehension questions on modified texts than on the original unmodified versions. while there have been concerted efforts in the past in the nlp community to develop metrics and corpora purported to serve studies in simplification @xcite @xcite @xcite @xcite , they fell far short of addressing how their work contributes to improving the text comprehensibility by readers. part of our goal is to break away from a prevailing view that relegates the readability to a sideline. ## method the data come from two sources, the split and rephrase benchmark (v1.0) (srb, henceforth) @xcite and wikisplit @xcite . srb consists of complex sentences aligned with a set of multi-sentence simplifications varying in size from two to four. wikisplit follows a similar format except that each complex sentence is accompanied only by a two-sentence simiplification. 2 we asked amazon mechanical turk workers (turkers, henceforth) to score simplifications on linguistic qualities as well as to indicate whether they have any preference between twosentence and three-sentence versions in terms of readability. we randomly sampled a portion of srb, creating test data (call it h), which consisted of triplets of the form: bart hum a (two-sentence split) 113 108 b (three-sentence split) -221 separately, we extracted from wikisplit and srb, another dataset b consisting of complex sentences as a source and two-sentence simplifications as a target ( figure 1 gives a screen capture of an initial section of the task. shown under source is a complex sentence or s i for some i . text a and text b correspond to a i and b i , which were displayed in a random order. in total, there were 221 hits ( ## preliminary analysis ## a bayesian perspective a question we are curious about at this point is what are the factors that led turkers to decisions that they made. we answer the question by way of building a bayesian model based on predictors assembled from the past literature on readability and in related fields. ## conclusions in this work, we asked two questions: does cutting up a sentence help the reader better understand the text? and if so, does it matter how many pieces we break it into? we found that splitting does allow the reader to better interact with the text (