Divide-and-Conquer Text Simplification by Scalable Data Enhancement (2022)

Abstract: Text simplification is a task to reduce the complexity of a text while retain its original meaning. It can facilitate people with low-literacy skills or language impairments, such as children and individuals with dyslexia and aphasia, to read and understand complicated materials. Normally, substitution, deletion, reordering, and splitting are considered as four core operations for performing text simplification. Thus an ideal model should be capable of executing these operations appropriately to simplify a text. However, by examining the degree that each operation is exerted in different datasets, we observe that there is a salient discrepancy between the human annotation and existing training data that is widely used for training simplification models. To alleviate this discrepancy, we propose an unsupervised data construction method that distills each simplifying operation into data via different automatic data enhancement measures. The empirical results demonstrate that the resulting dataset SimSim can support models to achieve better performance by performing all operations properly.

## introduction & related work text simplification is a task to reduce the complexity of a text while retain its original meaning. it can facilitate people with low-literacy skills or language impairments, such as children and individuals with dyslexia @xcite and aphasia @xcite , to read and understand complicated materials @xcite . normally, substitution, deletion, reordering, and splitting are considered as four core operations for performing text simplification @xcite . thus an ideal model should be capable of executing these operations appropriately to simplify a text. however, by examining the degree that each operation is exerted in different datasets, we observe that there is an salient discrepancy between the human annotation and existing training data that is widely used for training simplification models. to alleviate this discrepancy, we propose an unsupervised data construction method that distills each simplifying operation into data via different automatic 2 inspecting simplification datasets at its essence, sentence simplification paraphrases a sentence for better readability. it often involves a subset of four rewriting operations/transformations: splitting, dropping, reordering, and substitution @xcite . a high-quality sentence simplfication training dataset, which contains many complex-simple sentence pairs, should be well-aligned to provide a wide coverage of different operations, so that the trained models can have good generalizability. most neural simplification models rely on training with large datasets such as newela @xcite and wikilarge @xcite , which are automatically collated with paired documents written in different readability levels. the quality of auto-collated data has been questioned in prior work @xcite , however, it remains unanswered on how well they represent the real simplification distribution and on which aspects they fall short. this motivated us to propose the following five metrics to quantitatively examine common training datasets: ## simsim: data enhancement by simulating simplification we think that a well-generalizable simplification model needs to be trained on high quality training pairs and simplification knowledge. we propose a method to automatically refine/construct existing training pairs and inject knowledge of simplification by simulating various rewriting transformations. unlike previous datasets (i.e. wikilarge and newela) that heavily rely on paired documents of different readabilities and can hardly scale up, our method is capable of exploiting any text data on the internet as seed thus avoid those limitations. we name the resulting dataset simsim. overview our method starts with a set of seed sentences, then a series of enhancement steps are performed on each seed sentence to generate a new sentence so that the original seed sentence and the new resulting sentence form a complex-simple pair. note that this method not only can enhance original training pairs, it can also construct new pairs solely using complex sentences. this method works on other corpora too, but we leave it for future work. to build seed sentences, we apply bertscore on each sentence pair in wikilarge and newela to check their semantic alignment between the complex and simple sentences. for those badly aligned pairs, we remove their simple sentences to eliminate the noise led by the misalignment. we take the rest sentences as seeds for enhancement. ## conclusion in this study, we observe a significant discrepancy exists between the human simplified sentences and common training data and propose an unsupervised data enhancement method, simsim, to explicitly teach the model appropriate operations by distilling the knowledge into training data. the empirical results show that the resulting dataset simsim can support models to achieve better performance by performing all operations properly.