Controllable Sentence Simplification via Operation Classification (2022)

Abstract: Different types of transformations have been used to model sentence simplification ranging from mainly local operations such as phrasal or lexical rewriting, deletion and re-ordering to the more global affecting the whole input sentence such as sentence rephrasing, copying and splitting. In this paper, we propose a novel approach to sentence simplification which encompasses four global operations: whether to rephrase or copy and whether to split based on syntactic or discourse structure. We create a novel dataset that can be used to train highly accurate classification systems for these four operations. We propose a controllable-simplification model that tailors simplifications to these operations and show that it outperforms both end-to-end, non-controllable approaches and previous controllable approaches.

## introduction sentence simplification is a text generation task where a sentence is transformed into a simpler version of itself while preserving its core meaning. transformations can involve several different rewrite operations such as word substitutions (lexical paraphrasing), structural modifications (e.g. sentence splitting or syntactic paraphrasing), and deletion. sentence simplification has been shown to aid reader comprehension @xcite @xcite and be a useful preprocessing step for downstream nlp tasks such as relation extraction @xcite and machine translation @xcite @xcite @xcite . modern systems are data-driven, learning to perform transformations from parallel corpora of complex-simple c, s pairs. although many different approaches have been attempted in the past, including statistical machine translation (smt)based methods, nearly all systems proposed in recent years follow a neural sequence-to-sequence approach. as these systems are trained in an endto-end manner they are able to perform lexical and syntactic operations in combination and produce outputs with very high fluency. however, given the black-box nature of these end-to-end systems, they are forced to rely on imperfect training corpora to implicitly learn rewrite operations, many of which occur infrequently @xcite . as a result, neural end-to-end systems have been found to be overly conservative, often making no changes to the original text or being limited to the paraphrasing of short word sequences @xcite . in addition, these systems provide limited capacity for controllability and are unable to express alternative variants of the simplified text @xcite . in response, attempts have been made to produce controllable simplification systems that can constrain either the shape (length, amount of paraphrasing, lexical and syntactic complexity) of the output @xcite or the type of transformation to be applied (e.g., copy, split, merge, rewrite, etc.) @xcite @xcite @xcite . in this work we propose a novel approach to sentence simplification which encompasses four global operations: whether to copy the input sentence (no simplification needed), rephrase it, split it based on syntax, or split it based on discourse structure. we create a novel dataset that can be used to train highly accurate classification systems for these four operations and propose a controllablesimplification model that tailors simplification to them. we compare our model with various alternatives and previous work, using both quantitative metrics and human evaluation, and show that our model outperforms them. we also provide a qualitative analysis of the differences between the best models. ## operation classification we consider 4 operation types: identity, rephrase, syntax-split, and discourse-split. the identity and rephrase classes are equivalent to identical and elaboration from @xcite . in contrast, we split the split class into two distinct groups to capture further nuances of sentence splitting, as was explored in @xcite . syntax-split indicates that a split should be performed based on a syntactic construct, whereas discourse-split indicates that a split should be performed based on a discourse relation. examples of these can be found in appendix a. as we focus on single sentence simplification, we exclude the merge class used in @xcite . ## results and discussion automatic evaluation results are shown in the same examples. however, it shows the diversity of irsd s does not reduce newsela-specific performance. on newsela test data, using the 3-class classifier (ctrl 3, * ) yields higher sari and r split than the 4-class case. this is likely because identity is never predicted thereby encouraging less conservative simplification on a test set where most examples are simplified @xcite excludes all examples with high or low similarity between the input and the reference from the test set). end-to-end vs controllable controllable systems outperform their end-to-end counterpart on all metrics and datasets. in particular, they show a large increase in r split , suggesting that explictly triggering splits via control tokens greatly improves a model's ability to correctly administer splits where needed. using silver operation labels in ctrl oracle shows universally higher scores than classifier-based pipelines, indicating that there is still room for improvement in terms of classification performance. existing systems comparative results with existing systems are summarised in ctrl 3,4 achieves the highest scores on newsela-m, except for being slightly beaten by madexp on sari. it is expected to be better than ctrl 4,4 here as newsela-m does not include any identical c, s pairs and therefore penalises copying inputs. ctrl oracle universally outperforms other systems. on asset, bart n actually achieves the highest sari, closely followed by madexp. we hypothesise that models trained on irsd s might achieve lower sari as the training data includes examples from splitting datasets that do not exhibit any further simplification, leading to more conservative lexical changes when splitting is performed. human evaluation human evaluation results are shown in all of our systems are rated highly accross all criteria and receive better average scores than even the reference. this clearly highlights the capability of pretrained generative models like bart to produce highly fluent texts. further, we see that using a classifier trained on our data to predict operation-specific control-tokens further enhances output (1) complex an acidic goo from the stomach dissolves the snail's flesh so the worm can re-swallow both stomach and prey. ## conclusion in this work we present a new dataset for simplification operation classification and show that it can be used to produce classifiers of much higher accuracy than what has been proposed in existing studies. we show that a controllable system using such a classifier to predict control tokens outperfoms end-to-end baselines and existing systems on a range of datasets and receives extremely high ratings in fluency, adequacy and simplicity from human evaluators. however, this system does result in slightly lower simplicity ratings compared to reference texts and a newsela baseline, suggesting that further improvements can be made to the system or dataset in order to achieve the best possible results across all criteria.