On Operations in Automatic Text Simplification (2023)

Abstract: This paper explores the literature of automatic text simplification (ATS) centered on the notion of operations. Operations are the processed of applying certain modifications to a given text in order to transform it. In ATS, the intent of the transformation is to simplify the text. This paper overviews and structures the domain by showing how operations are defined and how they are exploited. We extensively discuss the most recent works on this notion and perform preliminary experiments to automatize operations recognition with large language models (LLMs). Through our overview of the literature and the preliminary experiment with LLMs, this paper provides insights on the topic that can help lead to new directions in ATS research.

## introduction automatic text simplification (ats) is a natural language processing (nlp) task that consists in modifying a text in order to make it more readable or understandable. generally, ats systems work at the sentence leval. they take a sentence as an input and produce a modified version of it, with the objective of making it simpler for a given audience. to characterize the modifications that are performed or aimed at, a lot of different works established various sets of operations. for a broad definition, an operation is a change performed on a textual unit, for example the deletion of a clause or the reformulation of a complex expression with simpler terms. simplifying sentences or documents typically involves more than one operation. in this work, we investigate the ats literature to gather what it says about operations. indeed, while it is always present at every level of works on ats, since the task appeared, operation as a concept has received little attention. our first intention is to provide the community with a structured review of the literature centered on operations, in particular how and why they are used. we also hope to bring a new perspective to feed the current reflection on evaluation in ats and ultimately on the definition of the task. we intend this paper to benefit both newcomers to the field -as we summarize elements from a large number of works of the domain -and active members of the community -as our observations enable new insights. the contributions of this paper are the following: in order to develop these contributions, the paper is organized as follows. we first report the definition of the different types of operations in the literature and how they are exploited (section 2). then we look closely at three recent papers that focus on typologies (section 3). after that we address the question of automatic operation identification -why and how it is performed -and propose a preliminary experiment for the task with large language models (section 4). we finally discuss our insights in section 5 and the limitations of our work in section 6, to finally conclude in section 7. ## categorizing operations in ats this section aims at giving a clear and detailed categorization of what is called "simplification operations" in the ats literature (section 2.1), and how they have been operationalized (section 2.2). ## automation of operations annotation one interest of simplification typologies is to help understand and annotate the operations used to transform a complex sentence into one or more simpler sentences. in case of large corpora, it may be difficult to ask experts to annotate the operations involved in each transformation in the corpus. in such a case, it may be useful to automatically annotate the operations involved in all simplifications in the corpus. this section proposes an overview of the current automation possibilities for the annotation of operations. section 4.1 starts by presenting the currently used methods. section 4.2 shows how large language models (llms) currently perform in this automation task. ## limitations of this study our study comes with a set of limitations that are mostly focused on our preliminary experiments using llms. first of all, while we experimented with 5 llms (bloom, bloomchat, gpt-2, gpt-3.5 and bard), many other exist in the literature. for instance, every month, new llms appear in the top of the hugging face leaderboard foot_2 .determining if the task is completely solvable using llms therefore requires a thorough investigation of many of the existing llms. second, the lack of stability mentioned in section 4.2 stresses the importance of prompt engineering to solve the task. while we tested many different prompts in several different configurations, one can never be sure that another untested prompt would not solve the task at hand. finally, the lack of access to the annotated corpora of the studied typologies made it difficult to evaluate the llm on many examples and to provide quantitative results. a corpus containing ground truth annotations for all typologies on the same examples would allow to quantitatively evaluate the performance of llms. ## conclusion this paper structured the ats literature around the question of operations. indeed, this overlooked angle led us to analyze recent typologies that have been proposed and to highlight their particular features, as well as the differences between them. we described what operations are found in the literature, how they are used and identified (manually and automatically) and provided insights that we hope can help spur new directions for research. in addition to a structured approach of the literature, we also proposed a preliminary experiment investigating the potential of large language models (llms) in the automatic annotation of operations. we show that albeit the new opportunities offered by llms, linguistic operations identification does not seem to be a trivial task. we believe that this task may be an important one to address so as to have a better definition of the task, which would facilitate the implementation in real-world settings.