Improving Human Text Simplification with Sentence Fusion (2021)

Abstract: The quality of fully automated text simplification systems is not good enough for use in real-world settings; instead, human simplifications are used. In this paper, we examine how to improve the cost and quality of human simplifications by leveraging crowdsourcing. We introduce a graph-based sentence fusion approach to augment human simplifications and a reranking approach to both select high quality simplifications and to allow for targeting simplifications with varying levels of simplicity. Using the Newsela dataset (Xu et al., 2015) we show consistent improvements over experts at varying simplification levels and find that the additional sentence fusion simplifications allow for simpler output than the human simplifications alone.

## introduction research on text simplification has largely focused on fully automated systems, including lexical systems that change words or phrases and sentencelevel systems that make broader changes @xcite @xcite . while the performance of such systems is steadily improving, for most real-world applications, the quality of these systems is still not good enough, particularly in domains where correctness is critical such as health and medical @xcite . in such domains, human experts are still the main creators of simplified text @xcite . the challenge is that these experts are costly to employ and the number of people equipped with the appropriate training and skills is limited. in this paper, we examine a crowdsourcing approach to produce simplifications more efficiently and of higher quality using non-experts. crowdsourcing has been suggested previously as a possible source of text simplifications @xcite , however, no work has addressed quality control or how to deal with varying simplicity targets. the top part of we make three main contributions. first, we describe a new sentence fusion technique for generating additional alternative simplifications based on the original input and the non-expert human simplifications. this allows for many additional simplifications to be generated by combining different portions of the original human simplifications. second, we provide a supervised approach for ranking candidate simplifications, both human generated and sentence fusion generated. this allows the system to pick high quality simplifications from the candidates generated. similar approaches have been used in translation for ranking and selecting both human and system translations @xcite . third, we parameterize the ranking approach to optimize for different levels of simplicity allowing for different simplifications to be chosen depending on the simplicity target. this is particularly useful when combined with the sentence fusion technique which allows for a much broader range of possible candidates than just the human simplifications. we evaluate the proposed system against human expert simplifications and show consistently better results at varying simplicity levels for both simplicity and adequacy. ## improving human simplification crowdsourcing platforms allow for data to be generated quickly with reasonable quality for a modest price @xcite . for text simplification, given a sentence to be simplified, we can solicit human simplifications from the crowdsourcing platform. however, the quality of the resulting simplifications is often of widely varying quality @xcite ; the workers are not experts and it can be difficult to give the workers the appropriate context, e.g., the target audience, etc. we leverage these initial human simplifications to create higher quality simplifications. specifically, given the original sentence, x, and non-expert human simplifications, s 1 , s 2 , ..., s n , the goal is to produce a high-quality simplification of x. previous work in translation @xcite has shown that reasonable results can be obtained by automatically selecting the highest quality non-expert translation from those solicited, however, you are limited to those options available and additional iterations of human improvements were needed to get reasonable results. to address these limitations, we extend the candidate simplifications by generating additional alternative candidate simplifications, s 1 , s 2 , ..., s m , using a graph-based fusion of s 1 , ..., s n . we then rank all of the candidate simplifications, i.e., [s 1 , ..., s n , s 1 , ..., s m , x], which includes the human simplifications, the simplifications generated by sentence fusions, and the original unsimplified sentence (to allow for no simplification), and pick the top ranked option as the final simplification. to rank the sentences, we learn a model that optimizes a scoring function that combines simplicity and adequacy, though any scoring function could be used. we give details on each of these steps below. ## experiments to evaluate our approach, we collected training and testing sets consisting of an original sentence and four human simplifications. to help better train the ranker, we also collected additional training data by scoring some simplifications generated by the sentence fusion approach. to understand the effect of alpha on the output, we trained rankers over 80 values of α, chosen to be densest near α = 1, resulting in 80 different rankers that prioritize different levels of simplicity. we tested each of these models on the test set and compared them to four levels of expert human evaluations based on adequacy, simplicity, and fluency. ## discussion we introduced a new approach for leveraging crowdsourced human simplification that generates additional candidate simplifications using a sentence fusion technique and a reranking approach to pick high-quality simplifications. our proposed approach is capable of producing simplifications that outperform expert human simplifications and the sentence fusion technique is particularly good at generating simpler variants. we also introduced the new task of generating a high-quality text simplification based on crowdsourced simplifications. our sentence fusion algorithm followed by reranking provides one possible approach, but there are a number of areas where it could be improved. we used a graph-based fusion approach, but recent neural approaches that have been applied in abstractive summarization may be adapted @xcite . many aspects of the reranker still need to be further explored. while the reranker did a reasonable job of selecting good candidates across different simplicity levels the oracle study (figure 4 ) suggests that there is still room for improvement and additional features and alternative reranking algorithms should be investigated. the question of how well our trained reranker ports to different domains is also yet to be investigated. future research on the relationships between α and simplicity is needed to establish a standard for choosing appropriate values of α as well. we hope this paper and the associated data provides a good starting point for future research in this area.