L e SS : A Computationally-Light Lexical Simplifier for S panish (2023)

Abstract: Due to having knowledge of only basic vocabulary, many people cannot understand up-to-date written information and thus make informed decisions and fully participate in the society. We propose LeSS, a modular lexical simplification architecture that outperforms state-of-the-art lexical simplification systems for Spanish. In addition to its state-of-the-art performance, LeSS is computationally light, using much less disk space, CPU and GPU, and having faster loading and execution time than the transformer-based lexical simplification models which are predominant in the field.

## introduction even in the highly-developed countries, many people (16.7% on average) only have a knowledge of a basic vocabulary thus encountering difficulties in understanding written information on a daily basis @xcite . this limits their active participation in the society and can negatively influence their life choices. according to the adult literacy report from 2013 @xcite , this problem is particularly prominent in spain, where 28.3% of people are in this situation @xcite . lexical simplification (ls) is the process of substituting complex words or phrases with their simpler variants. it is an important factor in making texts more accessible for people with aphasia @xcite @xcite , dyslexia @xcite , autism spectrum disorders @xcite , cognitive impairments @xcite , low literacy levels @xcite , deaf and hard-ofhearing people @xcite , children @xcite , and nonnative speakers @xcite . due to its evident potential for great social impact, lexical simplification has been attracting a growing attention from the natural language processing (nlp) community @xcite . semeval-2021 task 1 on lexical complexity prediction attracted 198 teams, out of which 91 submitted their systems for one of the two sub-tasks, single-word or multi-word lexical complexity prediction @xcite . the recent tsar-2022 shared task on multilingual lexical simplification received 33 system submissions for english, 17 for spanish, and 16 for (brazilian) portuguese @xcite . while it is generally considered that the more frequent words are easier to understand for everyone, which words should be considered as complex or simple can vary from one target population to another @xcite , and is subjective even within one target group @xcite . manual lexical simplification requires an extensive knowledge of the language and the particular simplification needs of the target user, thus being expensive and time-consuming. automatic text simplification systems, in contrast, could offer a possibility for an easier customisation and on-demand personalized simplification. to enable that, it is important to build modular systems which offer possibility of using customized resources and substitute ranking modules. we propose less, a new state-of-the-art lexical simplifier for spanish, that uses less computational power than the previous state of the art and a modular architecture that enables easy customization. foot_0 as it will be shown in section 5, less outperforms the transformer-based state-of-the-art lexical simplification systems for spanish, while being computationally much more efficient. ## architecture of less less is a computationally light lexical simplifier with modular architecture (figure 1 ). it comprises seven modules: document analyzer (da), wordlevel semantic similarity (wss), context-level semantic similarity (css), word frequency (wf), bigram frequency (bf), candidate ranking (cr), and morphological generator (mg). document analyzer (da) performs sentence splitting, tokenization, part-of-speech (pos) tagging, lemmatization, and named entity recognition. word-level semantic similarity (wss) module retrieves as substitution candidates those words whose word embedding vectors have the highest cosine similarity with the word embedding vectors of the target word. it initially selects 30 candidates. from those 30, it filters out those that share the lemma with the target word, and those that contain more than 95% of non-alphabetic characters. when multiple candidates share the lemma among themselves (but not with the target word), only the candidate whose word embedding vector has the highest cosine similarity to the word embedding vector of the target word is retained. context-level semantic similarity (css) module computes cosine similarities between the word embedding vector of the substitution candidate and the context of the target word. this module is envisioned as word sense disambiguation tool. it has been proposed by @xcite with the idea that the simplification candidates which are synonyms of the correct sense of the target word should be more semantically similar to the context of the target word. the context-level semantic similarity (csim) between the target word t and the replacement candidate r is obtained by averaging the cosine similarity between the word embedding vector of the replacement candidate (v r ) and word embedding vectors of each content word (v w ) in the context of the target word (c t ), using the following formula: word frequency (wf) module retrieves the frequency of the target word and the candidate replacements in large corpora. based on the intuition that frequent words are usually simpler to understand, word frequency is often used for the ranking of the substitution candidates in ls systems @xcite . bigram frequency (bf) module returns the average value (arithmetic mean) of the google books bigram frequencies for the bigrams w -1 r and rw +1 , where r is the replacement candidate, w -1 is the word preceding the target word, and w +1 is the word after the target word in the given sentence. if the target word is the first word in the sentence, the module returns the frequency of the bigram rw +1 . if the target word is the last word in the sentence, the module returns the frequency of the bigram w -1 r. the idea behind this module is that the frequency of the word itself is not always a straightforward measure of its simplicity. the bigram frequency is envisioned to capture the influence of the surrounding words on the word simplicity, which is particularly important in the case of phrasal verbs or multi word expressions. how well the replacement candidate fits in a larger context should be captured by word-level and context-level semantic similarity modules. candidate ranking (cr) module computes the final ranking of all replacement candidates, including the target word itself. for each word, it first sums the ranks obtained in separate modules (wss, bf, wf, and css). then, it ranks the candidates based on those sums. when the value for a candidate replacement cannot be calculated in a certain module, e.g. the word does not appear in pretrained word embedding model or frequency library, that candidate will receive the rank '10000' in that module, as a penalty for being infrequent, and as such, probably complex. morphological generator (mg) module returns the inflected form of the replacement candidate (with the same pos tag, gender, and number as the target word) given the lemma of the replacement candidate and the pos tag of the target word. here is important to note that less does not explicitly perform complex word identification. it, instead, follows the idea proposed by @xcite to treat all content words as potentially complex. the complex word identification is, in that case, performed implicitly, by the target word itself being considered as a substitution candidate (together with the 'real' substitution candidates) in the candidate ranking module. ## tools and resources used in less for document analysis (da module), we use freeling @xcite v4.0. foot_1 . for all operations with word embedding vectors (modules wss and css), we use fasttext 2m 300dimensional cased word embeddings. 6 word frequencies (module wf) are obtained using the freely available python wordfreq library, foot_3 which contains word frequencies calculated on the exquisite corpus. 8 the spanish part of this corpus comprises encyclopedic texts (wikipedia), subtitles (opus open subtitles and subtlex), news (newscrawl 2014 and globalvoices), books (google books ngrams 2012), web texts (oscar), short-form social media texts (twitter), and longerform internet comments (reddit). freeling lexicon j48 training data #lemmas #forms corpus #tokens 70,150 669,216 conll09 427,442 for calculating bigram frequences (needed for bf module), we use google books ngrams for spanish. 9 we store pre-calculated bigram frequences in a look-up table and use it for retrieving particular bigram frequencies at the execution time. the range of years used to create the table was @xcite , where all bigrams containing numbers were removed. as mg module, we use the morphological generator proposed by @xcite which combines lexicon-based generation with predictions from decision trees. the lexical categories supported are: verbs, nouns, adjectives, adverbs, pronouns, determiners, and numerals. the lexicon used is the freeling v4.0 morphological dictionary for spanish. when the lexicon has no inflection for a pair (lemma, pos tag), the module uses the j48 model (weka foot_6 implementation of c4.5 decision tree) to predict the sequence of edit operations that can transform an unseen pair (lemma, pos tag) to the correct inflected form. ## evaluation to compare the performance of our system with the state of the art, we evaluate it on the spanish portion of the tsar-2022 shared task dataset, and the alexsis dataset. ## error analysis we performed error analysis on the full alex-sis dataset (381 instances). in 29 instances (7.6%), none of the substitutes generated by less is present in the gold data. in four of those 29 cases, less did not generate any substitutes, as the word embeddings used did not contain those four target words: pitorreo (eng. messing around) -used only in colloquial jargon in spain; expiaciones (eng. atonement) -used in biblical sense; pedanía (eng. district) -used only for special types of districts in spain; and larvas (eng. larva) -used in quotes in a metaphorical sense. in 18 of those 29 cases, less suggested the target word itself as the bestranked replacement candidate. in a real-world scenario, those 22 cases would limit the simplification power of the system but would not be dangerous as they would leave the target word unchanged. in two of the 29 cases where the candidates suggested by less were not found in the gold data, target word itself was suggested by less as the second-best. in another two, less suggested a correct word but with missing reflexive pronoun: preparando instead of preparandose (eng. getting ready), and forjar instead of forjarse (eng. to forge oneself (figuratively)). in one case, less suggested the words peligroso (eng. dangerous) and destructivo (eng. destructive), which were not found in the gold standard but fit the context perfectly, as a simpler substitute for mortífero (eng. lethal). in an-the previous state of the art. our detailed manual error analysis indicated that less often suggests several simpler synonyms. this indicates that less could be used in realworld applications as a writing aid to human editors for faster simplification and customization to different users. in real-world applications, the use of lexical simplification module as a writing aid is preferred over fully automatic lexical simplification, as it allows for customization @xcite and prevents unintended harms to vulnerable populations @xcite . in future, we would like to investigate if this architecture can yield state-of-the-art results in other languages, especially those with limited resources. the currently predominant approaches in the field, based on lsbert and masked language models, have noticeably better performances in english than other languages, even in the case of comparable evaluation datasets (see the results of tsar-2022 shared task @xcite ).