T ext S implifier: A Modular, Extensible, and Context Sensitive Simplification Framework for Improved Natural Language Understanding (2023)

Abstract: Natural language understanding is fundamental to knowledge acquisition in today’s information society. However, natural language is often ambiguous with frequent occurrences of complex terms, acronyms, and abbreviations that require substitution and disambiguation, for example, by “translation” from complex to simpler text for better understanding. These tasks are usually difficult for people with limited reading skills, second language learners, and non-native speakers. Hence, the development of text simplification systems that are capable of simplifying complex text is of paramount importance. Thus, we conducted a user study to identify which components are essential in a text simplification system. Based on our findings, we proposed an improved text simplification framework, covering a broader range of aspects related to lexical simplification — from complexity identification to lexical substitution and disambiguation — while supplementing the simplified outputs with additional information for better understandability. Based on the improved framework, we developed TextSimplifier, a modularised, context-sensitive, end-to-end simplification framework, and engineered its web implementation. This system targets lexical simplification that identifies complex terms and acronyms followed by their simplification through substitution and disambiguation for better understanding of complex language.

## introduction limited reading or comprehension skills can hinder managing and maintaining a comfortable lifestyle in today's information society. regardless of acquiring skills related to reading and comprehension over many years, sometimes understanding text can be challenging for, for example, people with limited reading skills, cognitive conditions like aphasia or dyslexia @xcite , limited knowledge of technical domains, non-native speakers, and children @xcite . therefore, different methods have been introduced to assist with reading and comprehension of language, ranging from i) manual efforts of "translating" text to more understandable formats to ii) automated simplification methods (see section 2). text simplification aims to modify the content and structure of complex text to output simpler text while preserving meaning. commonly, the two main concepts associated with simplification are identified as readability and understandability. even though these two concepts seem highly coupled, they address two different aspects of simplification @xcite : readability focuses on how complex text can be converted to simple text to make it easier to read. in contrast, understandability is related to how much information a user can grasp from the text. depending on the context and audience for which the text simplification is intended, the focus on improving the readability or understandability may differ. consequently, being sensitive to the different intents, researchers have introduced various methods for simplification (see section 2): if the aim of the simplification is readability improvement, different methods focusing on the simplification of the syntactical structure have been proposed. these methods achieve simplification primarily by deleting, reordering, and splitting sentences to convert them to syntactically simpler formats so that the text is easier to read @xcite . on the other hand, for understandability improvement, most methods focus on generating alternative substitutes for target complex words in text, focusing on the lexical simplicity of the text @xcite . improving the understandability of text benefits many audiences. for example, these understandability-focused simplification methods are helpful for non-native speakers and second-language learners to learn about new languages. moreover, these methods can be helpful for students learning about new technical content or anyone who is not an expert in a specific technical domain. for example, domains like medical and scientific domains contain technical content, which is quite difficult for lay people to understand. hence, extensive research has been done on the improved understandability of complex text (see section 2). text simplification systems focusing on improved understandability of text explore different aspects related to simplification. for example, some methods investigate the complexities in text , @xcite , whereas others investigate the generation of alternatives for complex words @xcite . recent methods of text simplification rely on machine translationbased sequence-to-sequence (seq2seq) models for text simplification @xcite @xcite , which tackle both lexical and syntactic simplification of text. one of the limitations of seq2seq models is that they achieve simplification mainly by reducing the length of the sentences through the deletion of tokens which results in improved readability, however, at the cost of understandability @xcite . hence, when focusing on the understandability aspect of the text, modular approaches which tackle one subtask at a time may yield better outputs. generally, most practical simplification methods targetting lexical simplicity or understandability follow a modular approach with a pipeline proposed by @xcite . this pipeline comprises complex word identification, substitution generation, selection, and ranking methods for improved understandability. however, even though this pipeline has been adopted for many functional simplification systems, they only focus on complex words or phrases and simplification of them. for better understandability identifying other aspects that contribute to the complexity is essential. for example, in technical domains like medical or scientific, technical shorthand (i.e., acronyms or abbreviations for technical terms) are often used for ease of use and to avoid repetitions @xcite . hence, in such instances, shorthand identification and disambiguation of them is crucial for better understandability. moreover, considering the complexities in text, in some instances, gen-erating an alternative word or phrase may not be enough for accurate comprehension, thus requiring additional information. the existing practical lexical simplification systems typically focus on one aspect of simplification, like addressing the complexity by acronyms @xcite or the ambiguity by the polysemic words @xcite . in contrast, some systems rely on the pipeline by @xcite and incorporate several components together @xcite . nevertheless, there are systems, which focus on both lexical and syntactic simplifications @xcite . however, practical systems for lexical simplification at present have a limited coverage of components, thus requiring more comprehensive systems for practical lexical simplification. in this paper, we present an improved text simplification framework targeting lexical simplification, extending the pipeline proposed by @xcite . it consists of the following four components: complex word identification, substitution generation, selection, and ranking. we report on a preliminary user study that we conducted to identify additional components required to enhance the simplification output for better understandability. based on the user study, we have investigated and incorporated different components into the pipeline: i) an acronym identification module to address the complexities of shorthand, specifically acronyms, ii) an acronym disambiguation module to tackle the existence of multiple expansions for an acronym, and iii) an information module to supplement the outputs with more information for better understandability, together with iv) the conventional components. we have combined them as a pipeline to form both an improved framework and its implementation as a web-based system for lexical simplification, focusing on understandability. the proposed simplification system mainly addresses general-language and specialised (scientific/medical) text, due to the availability of resources and models. ## related work the earliest attempt to develop a text simplification system for practical use was made by @xcite , who introduced happi -helping aphasic people process information, a web-based system to assist people with aphasia in reading online information. the system achieved this by providing alternative words for complex words obtained through a database. the database consisted of psycholinguistic information about words like frequency and the familiarity of words used in the simplification process. text simplification systems for improved comprehension targeting lexical simplicity advanced in mid-2010s. @xcite introduced a text simplification system targeting second-language learners of the english language, with an interactive web interface for the users. the simplification was achieved by providing synonyms for complex words. @xcite proposed a resource-light, unsupervised lexical simplification system called light-ls. it relied on large regular text corpus for lexical simplification. a similar web interface to @xcite was introduced by @xcite . the tool was called anita: an intelligent text adaptation tool and it relied on the lexenstein framework by @xcite . anita followed four steps in the simplification process where first candidate substitutes were produced based on a word embedding model followed by selection, ranking and replacement of the complex word. additional information like synonyms and definitions were also provided in the system if a user requested it. their method created user profiles intending to obtain users' feedback to improve the results. @xcite introduced a text simplification tool called lexi which also addressed obtaining users' feedback. the proposed system relied on the pipeline introduced by @xcite and included complex word identification, substitution generation, selection, and ranking components. in addition, lexi used users' feedback to personalise the experience to the target users. pioneering frameworks and systems for both lexical and syntactic simplification processes were also introduced over the years. @xcite presented the simplext project that effectively managed both lexical and syntactic simplification processes for spanish. for lexical simplification, simplext relied on a synonym-based and a rule-based simplification component, while for syntactic simplification handwritten computational grammars were used. similarly, yats by @xcite consisted of lexical and syntactic components to improve text readability and understandability for english. its lexical simplification relied on a vector space model and word frequency simplicity measures to rank synonyms while its syntactic simplification used rule-based syntactic analysis and generation techniques based on part-of-speech tags and syntactic dependency information. following a similar approach to yats, a lexical simplification architecture for spanish, portuguese, catalan, and galician was introduced by @xcite . focusing on the improved lexical simplicity of text, orlando et al. (2021) even though there have been several systems targeting lexical simplification in the recent past, most of these systems used the traditional lexical simplification pipeline by @xcite for simplification, failing to consider the other essential components for improved understandability. thus, it is important to explore beyond the traditional simplification steps when translating the research outputs into useful applications. nevertheless, there has been extensive research in the domain of lexical simplification over the years that predominantly rely on transformer-based language models to improve the understandability of text @xcite . ## user study the development of a functional text simplification system for practical use requires identifying what contributes to the complexity of the text, the aspects that should be considered, and the components that should be included. thus, we conducted a user study to obtain user input on essential components for text simplification. ethical approval (protocol 2021/708) for the user study was obtained from the anu human research ethics committee. the user responses were collected in a survey format. ## proposed framework we proposed a modular text simplification framework for improved lexical simplicity based on feedback from the user study. the proposed framework extends beyond the conventional text simplification systems and pipelines and incorporates components targeting a much broader area of aspects related to lexical simplification. our work is founded on the pipeline by @xcite with components for complex word identification, substitution generation, selection, and ranking. this can be converted into a pipeline with two components at a more abstract level forming it as a pipeline with complex word identification and lexical substitution, with the latter three components of the traditional pipeline falling under lexical substitution. the feedback from the user study indicated that acronyms also contribute to the complexity of text, and hence, we have incorporated an additional component for the acronym identification task. following the acronym identification module, we have integrated a disambiguation module focusing on identifying the correct expansion of an acronym. moreover, because the participants of the user study expressed the importance of a module to provide supplementary information for improved understandability, we have incorporated an information module into the pipeline. our proposed improved framework targets the understandability of natural language (figure 3 ). it consists of 5 main modules: complex word identification, lexical substitution, acronym identification, acronym disambiguation, and information module. ## system design we developed a modular, context-sensitive text simplification system based on the proposed framework focused on improved understandability (figure 4 ) that we have made available at http://130. 56.247.69:8501/ . each major component in the framework is a separate subfield in lexcal simplification. hence, when translating the framework to the development stage, we have proposed new methods and relied on previous works for each component. the datasets used for experiments come complex word identification acronym identification acronym disambiguation information module (to provide definitions, explanations, and links to resources) input text simplified text lexical substitution substitution generation substitution selection substitution ranking word sense disambiguation model prediction score proximity similarity score + xlnet sentence similarity score rule-based lexical substitution complex word identification rule-based acronym identification complex words acronyms wordnet wikipedia information module simplified text with additional information complex text convolution + attention-based triplet networkbased rule-based acronym disambiguation wordnet-based bert-based from general and specialised (scientific/medical) text. its design and development consider dependencies of each of these components. ## discussion in this paper, we have proposed a text simplification framework targeting improved lexical simplicity/language understandability using the feedback obtained through a user study on text complexities. based on the feedback, we have extended the conventional lexical simplification pipeline to incorporate additional components essential for natural language understanding. as a result, we have derived a framework of complex word identification, lexical substitution, acronym identification, and acronym disambiguation components followed by an information module to supplement the simplified output. even though the typical lexical simplification systems focus only on the complexities of complex words and phrases, the evaluation results of the user study indicated that the acronyms contribute to the complexity of the text. one reason might be that acronyms are heavily used in technical domains like scientific and medical domains we come across daily. moreover, the exponential growth in information has increased the use of acronyms. hence, it is essential to identify and disambiguate them to determine the correct expansion corresponding to the meaning of the context and incorporate the relevant components in simplification pipelines. the results from the user study also indicated the importance of providing additional information related to the complexities in the text to improve understandability, thereby helping the readers grasp the knowledge effectively. therefore, it is crucial to incorporate components that supplement the simplified versions of complex text. in our proposed text simplification framework, we have integrated multiple components that all relate to lexical simplification. we have validated and assessed each component separately to ensure their effectiveness. nevertheless, because each task was trained using datasets from different sources, this could potentially impact the final output. therefore, exploring the compatibility of these separate models within a unified system is crucial. moreover, the end-to-end pipeline as a whole was not evaluated. thus, as future work, we expect to create datasets that provide annotations for each important task in a consistent manner, which could further enhance the effectiveness of text simplification methods. given these challenges, the output generated by the complete pipeline has not been evaluated using a simplicity metric in this study. the proposed simplification framework incorporates additional components required for improved language understandability compared to existing simplification systems. it also follows a modular or task-based approach in tackling different aspects related to simplification, which is much more explainable compared to models that rely on one black-box architecture for the simplification task. moreover, its modular architecture eases the integration of new modules addressing other aspects of simplification and new components for each module in the