Controllable Text Simplification with Explicit Paraphrasing (2021)

Abstract: Text Simplification improves the readability of sentences through several rewriting transformations, such as lexical paraphrasing, deletion, and splitting. Current simplification systems are predominantly sequence-to-sequence models that are trained end-to-end to perform all these operations simultaneously. However, such systems limit themselves to mostly deleting words and cannot easily adapt to the requirements of different target audiences. In this paper, we propose a novel hybrid approach that leverages linguistically-motivated rules for splitting and deletion, and couples them with a neural paraphrasing model to produce varied rewriting styles. We introduce a new data augmentation method to improve the paraphrasing capability of our model. Through automatic and manual evaluations, we show that our proposed model establishes a new state-of-the-art for the task, paraphrasing more often than the existing systems, and can control the degree of each simplification operation applied to the input texts.

## introduction text simplification aims to improve the readability of texts with simpler grammar and word choices while preserving meaning @xcite . it provides reading assistance to children @xcite , non-native speakers @xcite @xcite , and people with reading disabilities @xcite . it also helps with downstream natural language processing tasks, such as parsing @xcite , semantic role labelling @xcite , information extraction @xcite , and machine translation (mt, @xcite . we compare our method to several state-of-theart systems in both automatic and human evaluations. our model achieves overall better performance measured by sari @xcite and other metrics, showing that the generated outputs are more similar to those written by human editors. we also demonstrate that our model can control the extent of each simplification operation by: (1) imposing a soft constraint on the percentage of words to be copied from the input in the seq2seq model, thus limiting lexical paraphrasing; and (2) selecting candidates that underwent a desired amount of splitting and/or deletion. finally, we create a new test dataset with multiple human references for newsela @xcite , the widely used text simplification corpus, to specifically evaluate lexical paraphrasing. ## our approach figure 1 shows an overview of our hybrid approach. we combine linguistic rules with data-driven neural models to improve the controllability and diversity of the outputs. given an input complex sentence x, we first generate a set of intermediate simplifications @xmath0, v 2 , . . . , v n } that have undergone splitting and deletion ( §2.1). these intermediate sentences are then used for two purposes: (1) selected by a pairwise neural ranking model ( §2.2) based on the simplification quality and then rewritten by the paraphrasing component; (2) used for data augmentation to improve the diversity of the paraphrasing model ( §2.3). ## experiments in this section, we compare our approach to various sentence simplification models using both automatic and manual evaluations. we show that our model achieves a new state-of-the-art and can adapt easily to different simplification styles, such as paraphrasing and splitting without deletion. ## model analysis in this section, we analyze the contribution of each model component and examine the system errors. ## related work before the advent of neural networks, text simplification approaches performed each operation separately in a pipeline manner using either handcrafted rules @xcite @xcite or data-driven methods based on parallel corpora @xcite @xcite . following neural machine translation, the trend changed to performing all the operations together end-toend @xcite @xcite @xcite ## conclusion we proposed a novel hybrid approach for sentence simplification that performs better and produces more diverse outputs than the existing systems. we designed a new data augmentation method to encourage the model to paraphrase. we created a new dataset, newsela-turk, to evaluate paraphrasing-focused simplifications. we showed that our model can control various attributes of the simplified text, such as number of sentence splits, length, and number of words copied from the input.