CASM - Context and Something More in Lexical Simplification (2023)

Abstract: Lexical Simplification is a challenging task that aims to improve the readability of text for nonnative people, people with dyslexia, and any linguistic impairments. It consists of 3 components: 1) Complex Word Identification 2) Substitute Generation 3) Substitute Ranking. Current methods use contextual information as a primary source in all three stages of the simplification pipeline. We argue that while context is an important measure, it alone is not sufficient in the process. In the complex word identification step, contextual information is inadequate, moreover, heavy feature engineering is required to use additional linguistic features. This paper presents a novel architecture for complex word identification that uses a pre-trained transformer model’s information flow through its hidden layers as a feature representation that implicitly encodes all the features required for identification. We portray how database methods and masked language modeling can be complementary to one another in substitute generation and ranking process that is built on the foundational pillars of Simplicity, Grammatical and Semantic correctness, and context preservation. We show that our proposed model generalizes well and outperforms the current state-of-the-art on wellknown datasets.

## introduction the lexical simplification task is an important task for the common people who are having difficulties understanding the language. ls systems can be used to increase the accessibility of information rodrigo @xcite and can help various groups such as non-native speakers, people with linguistic impairments, children, etc. the readability of the text is greatly hindered by the presence of complex words. the words in a text document need to be replaced by identical words without changing their meaning. the lexical simplification pipeline generally consists of three major tasks: • complex word identification ## methodology in this section, we outline each step of our lexical simplification pipeline, namely cwi(complex word identification), sg(substitute generation), and sr(substitute ranking). ## results in the sections below we showcase the evaluation of our approach to various benchmark datasets and metrics. ## conclusion in this paper, we have proposed a novel lexical simplification algorithm. we leverage the use of information flow in transformers for complex word identification, which to the best of our knowledge is the first method that directly represents the hidden layers as sequential feature representation. the results show that our system outperforms the current state-of-the-art model on cwi 2018 and cefr-ls datasets. the importance of supporting features in the simplification process is explored and analyzed. we showcase a procedure in which the contextual information is infused with other simplicity, semantic and syntactic measures in substitute generation and substitute ranking. in the future, we plan to expand this novel idea of capturing information flow which can be easily applied to other nlp tasks such as part-of-speech tagging and named entity recognition. we also plan to investigate our methodology for text simplification.