Keep It Simple: Unsupervised Simplification of Multi-Paragraph Text (2021)

Abstract: This work presents Keep it Simple (KiS), a new approach to unsupervised text simplification which learns to balance a reward across three properties: fluency, salience and simplicity. We train the model with a novel algorithm to optimize the reward (k-SCST), in which the model proposes several candidate simplifications, computes each candidate’s reward, and encourages candidates that outperform the mean reward. Finally, we propose a realistic text comprehension task as an evaluation method for text simplification. When tested on the English news domain, the KiS model outperforms strong supervised baselines by more than 4 SARI points, and can help people complete a comprehension task an average of 18% faster while retaining accuracy, when compared to the original text.

## introduction the main objective of text simplification is to make a complex text accessible to a wide audience by increasing its readability. in contrast with text summarization -in which key content is selected to remain in the summary and other content is elided -in text simplification, ideally all relevant content is preserved. we propose that text simplification algorithms need to balance three properties: (1) fluency: the simplified text should use well-formed english sentences, (2) salience: the simplified text should relay the same information as the original, and (3) simplicity: the simplified text should be syntactically and lexically simpler than the original. figure 1 provides intuition for the necessity of each of the three properties. it shows the original text and the output of the full proposed model compared to three reduced versions: figure 1 : motivating example for the kis method, based on a cbs article @xcite . we optimize a three-component reward: fluency, salience and simplicity. we show model outputs when trained with all three components, and with a missing component. without fluency, the generator has no incentive to generate full sentences, and learns it can boost the simplicity score by generating short phrases with excessive punctuation. without salience, the generator does not gain by covering facts in the original text, and can improve the simplicity score by learning to remove facts (e.g., not mentioning planet mars by name). without simplicity, the generator is not guided to favor syntactically and lexically simpler rewrites. in figure 1 , model no simplicity is in fact more complex than the original according to readability measures. as we show in the related work section (section 2), there are no high-quality, large datasets publicly released for text simplification. in this work, we build on recent progress of reinforcement learning (rl)-based training of text generators: we formulate a reference-free reward for text simplification and directly optimize it, circumventing the need for aligned data. our main contribution is the keep it simple (kis) procedure, a novel unsupervised method for text simplification. applied to the english news domain, kis outperforms several supervised models on common simplification metrics such as sari @xcite and the flesch-kincaid grade level @xcite . a second contribution is a new algorithm for rlbased training of text generators, k-scst, which is an extension of self-critical sequence training @xcite . for each input, we generate k sampled outputs (vs. 2 in scst), and use the mean population reward as a baseline. we show in section 4 that in our domain, k-scst outperforms models trained with scst. a third contribution is a novel evaluation method for text simplification. based on the assumption that simplified text should enable faster reading with better understanding, we propose a realistic text comprehension task. we show that people reading texts simplified by kis are able to complete comprehension tasks faster than comparison texts. another departure from previous work is that we work with paragraphs as units of text. most work in text simplification is done at the sentence level, despite work such as @xcite showing that common simplification phenomena occur at the level of the paragraph, (e.g., the deletion, insertion or re-ordering of full sentences). specifically, we train our models to simplify full paragraphs, and evaluate our models in a human evaluation on short documents (i.e., 3-4 paragraphs). through rigorous empirical evaluation, we demonstrate the strong performance of our approach; automated results show that this unsupervised approach is able to outperform strong supervised models by 4 sari points or more. we publicly released the code and model checkpoints foot_0 . ## kis components in kis, we approach unsupervised simplification as a (non-differentiable) reward maximization problem. as shown in figure 2 , there are four components to the reward: simplicity, fluency, salience and guardrails which are jointly optimized. this is essential to avoid trivial solutions that only consider subsets. we therefore use the product of all components as the total reward, because the product is sensitive to the sharp decrease of a single component. for example, the triggering of a single guardrail leads to the zeroing of the total reward. each component is normalized to the [0, 1] range. def s_score(original,simple): @xmath0,0,1) def target_delta(fstart): # line-fitted from analysis if @xmath1.0: return 0.1 if @xmath2: return 0.5 * fstart-1.9 return 0.8 * fstart-5.6 figure 3: s score algorithm. fkgl computes the flesch-kincaid grade level. ## kis training rennie et al. (2017) introduced self-critical sequence training (scst) as an effective algorithm for reward-based training of text generators, successfully applying it to image captioning. the efficacy of scst was later confirmed on other text generation tasks such as question generation @xcite , and summarization @xcite . in scst, a probabilistic model is used to generate two distinct candidates: c s , a candidate constructed by sampling the word distribution at each step, and ĉ, by taking the argmax of the word distribution at each step. each candidate is scored, obtaining rewards of r s and r, respectively, and the loss is: one limitation in scst occurs when the two sequences achieve comparable rewards (r s r): the loss nears zero, and the model has little to learn, wasting a training sample. in our experiments with scst, this can occur with 30% of samples. we propose an extension of scst, which we call k-scst. we generate k sampled candidates (@xmath3, compute the rewards of each candidate r s1 , ..., r sk , as well as the mean reward achieved by this sampled population: @xmath4... + r sk )/k, which we use as the baseline, instead of r. the loss l becomes: we use a gpt2-medium for the generator, initialized with the released pre-trained checkpoint. experimental details such as data and optimizer used are provided in appendix a.1. in figure 5 , we show results of a direct comparison of scst (@xmath5, 6, 8}, while keeping other components of the training fixed. because of the variance involved in rl training, we recorded six independent training runs for each setting (for a total of 24 runs), and plot the average reward across runs of a setting, as well as the standard error of the mean (sem). we observe that increasing k leads to higher average reward, and less variation in the reward. in our setting, k-scst boosts performance and stabilizes training. we use @xmath6, as increasing k further is impractical due to gpu memory limitations. we believe k-scst's advantage stems from two factors: first, obtaining a better estimate of the distribution of rewards by sampling more outputs, second, by using the mean reward as the baseline, saving on computation of a separate baseline generation. we believe k-scst can also improve learning in other text generation applications and plan to pursue this in future work. ## experiments we present results experimentally validating the kis procedure for text simplification. we give results based on automatic metrics, on a novel human comprehension task, and from an ablation study. ## conclusion we have shown that text simplification can be approached in an unsupervised manner via kis. by optimizing a reward comprised of simplicity, fluency and salience components, kis is able to outperform strong supervised models on automatic metrics (+0.04 in sari). we propose a human comprehension task to evaluate the usefulness of simplification and show that simplifications tend to lead to a measurable speed-up in task completion, with kis texts producing the best speed-up of 18% on average. these are first steps for unsupervised text simplification, and we suggest that future work should focus on adapting the methodology to new domains (i.e., legal), non-english languages, and refining optimized rewards to take factuality into account. discriminator model is initialized with a roberta-base, and retrained every time the training buffer reaches 2,000 samples. the discriminator is reset to the original roberta-base each time the training buffer is full. we use a standard cross-entropy loss, the adam optimizer with a learning rate of 10 -5 and a batch size of 8. each time we retrain, we run for 5 epochs, and checkpoint one model after each epoch. the checkpoint that achieves the highest performance on a validation set becomes the new discriminator for the next round.