Lexically Constrained Decoding with Edit Operation Prediction for Controllable Text Simplification (2022)

Abstract: Controllable text simplification assists language learners by automatically rewriting complex sentences into simpler forms of a target level. However, existing methods tend to perform conservative edits that keep complex words intact. To address this problem, we employ lexically constrained decoding to encourage rewriting. Specifically, the proposed method predicts edit operations conditioned to a target level and creates positive/negative constraints for words that should/should not appear in an output sentence. The experimental results confirm that our method significantly outperforms previous methods and demonstrates a new state-of-the-art performance.

## introduction text simplification @xcite paraphrases complex sentences into simpler forms. controllable text simplification @xcite @xcite is a task in text simplification that aims to rewrite a sentence for an audience of a specific level. it is a crucial technique in assisting children and nonnative speakers with language learning @xcite . text simplification can be performed based on three approaches: (1) translation-based, (2) editbased, and (3) hybrid approaches. the translationbased approach, e.g., @xcite @xcite @xcite , formalizes text simplification as monolingual machine translation from complex to simple sentences. this approach can rewrite a sentence flexibly; however, it implicitly learns simplification operations through translation. the infrequent nature of simplification operations hinders a model from learning necessary operations, which makes the model conservative to maintain complex words intact @xcite . in contrast, the edit-based approach @xcite @xcite @xcite rewrites an input by applying edit operations of add or replace, keep, and delete to words. this approach can address the conservativeness problem owing to explicit word-by-word edits. however, it lacks the flexibility to rewrite an entire sentence to drastically change its syntactic structure. finally, the hybrid approach takes advantages of the above two by applying lexical constraints to translation-based models. @xcite added weights to a loss function to bias a sequence-to-sequence (seq2seq) model to output certain words. @xcite biased a nonautoregressive simplification model by setting an initial state of decoding, considering the lexical complexity of a source sentence. the constraints in these studies were soft; in contrast, @xcite and @xcite applied a hard constraint using lexically constrained decoding to avoid outputting complex words. in spite of their success, these two methods lack flexibility in their constraints. they only use negative constraints to avoid outputting specified words. however, positive constraints, which encourage the output of specified words, are also valuable for text simplification. in this study, we propose a hybrid method for controllable text simplification with flexible combinations of positive and negative constraints using neurologic decoding @xcite . the proposed method predicts edit operations conditioned on a target level to generate positive and negative lexical constraints sensible to a target level. experiments on newsela @xcite and newsela-auto @xcite reveal that the proposed method outperforms previous methods and achieves a new state-of-the-art performance. the codes and outputs of the proposed method will be released at https://github.com/ t-zetsu/constrainedts . 2 proposed method figure 1 illustrates an overview of the proposed method, in which generated constraints are applied to a seq2seq model via neurologic decoding. ## conclusion we proposed a hybrid method for controllable text simplification that takes both advantages of translation-and edit-based methods using the flexible lexically constrained decoding. the experimental results showed that the proposed method conducts high-quality controllable text simplification on newsela and newsela-auto. we expect that the proposed method also works for text simplification in general, i.e., the binary transformation from complex to simple sentences. this investigation is left for our future work. we will also explore complex combinations of constraints allowed by neurologic decoding in the future.