Complex words identification using word-level features for S em E val-2020 Task 1 (2021)

Abstract: This article describes a system to predict the complexity of words for the Lexical Complexity Prediction (LCP) shared task hosted at SemEval 2021 (Task 1) with a new annotated English dataset with a Likert scale. Located in the Lexical Semantics track, the task consisted of predicting the complexity value of the words in context. A machine learning approach was carried out based on the frequency of the words and several characteristics added at word level. Over these features, a supervised random forest regression algorithm was trained. Several runs were performed with different values to observe the performance of the algorithm. For the evaluation, our best results reported a M.A.E score of 0.07347, M.S.E. of 0.00938, and R.M.S.E. of 0.096871. Our experiments showed that, with a greater number of characteristics, the precision of the classification increases.

## introduction the identification of complex words (cwi) is the task of detecting in the content of documents the words that are difficult or complex to understand by the people of a certain group (rico-sulayes, 2020). the cwi and the substitution of words identified as complex may significantly improve readability and understandability of a given text @xcite . cwi has become an area of great interest in recent years for the computational linguistics community in making proposals that allow researchers to develop computational semantic analysis systems, as demonstrated by the shared tasks of cwi in semeval 2016 @xcite , y naacl-htl 2018 @xcite , and the cwi task of the alexs 2020 competition, hosted at iberlef 2020 @xcite . this article introduces a system that has participated in the lexical complexity prediction (lcp) shared task hosted at semeval 2021 (task 1) @xcite . the task releases a new annotated english dataset with a likert scale. located in the lexical semantics track, the task consisted of predicting the complexity value of the words in context. we have explored different features for representing words and multi-words and their context. some preprocessing steps have been evaluated along with the effect of feature selection. predicting which words a given target population has difficulty to understand is a critical step for many nlp applications, such as in text simplification, which has traditionally focused its attention on second language learners, native speakers with low levels of literacy, and people with language disabilities reading @xcite . this task is also known as complex word identification (cwi). the prediction of the lexical complexity carried out with precision can allow to adapt texts according to the needs of the readers @xcite . actually, in an early study in the 1920s, a very simple way to predict the level of difficulty of a text was discovered by educators, who used vocabulary difficulty and sentence length as main indicators @xcite . ## dataset the training data set provided to the participants consisted of an augmented version of complex @xcite . it uses data from three different sources: the bible, europarl, and biomedic texts (see ## the system this section describes the details of the system applied to the task, as our approach to complex word identification. a machine learning approach was followed based on the frequency of the words and further characteristics added at word level. over these features, a supervised random forest regression algorithm was trained. in this section, first, the features considered in the supervised learning approach are introduced. then, the method to determine whether a candidate word is complex or not is detailed. ## conclusion in this article, the results of our participation in task 1: lexical complexity prediction in the lexical semantics track hosted at the semeval 2021 international workshop have been presented. both the training corpus and the evaluation corpus were provided by the sponsoring organization of this competition. we applied machine learning and built the model using the random forest regression algorithm, relying on well-known word based and contextual features. as future work, we plan to perform error analysis on the predictions, to identify the weaknesses of the proposed approach based on a characterization of the instances where the system performs poorly. also, a better analysis of multi-word scenario is foreseen.