Introducing MultiLS-IT: A Dataset for Lexical Simplification in Italian
Authors: Laura Occhipinti (Università di Bologna, 2024)
Topic: Text Simplification

Abstract
Lexical simplification is a fundamental task in Natural Language Processing, aiming to replace complex words with simpler synonyms while preserving the original meaning of the text. This task is crucial for improving the accessibility of texts, particularly for users with reading difficulties, second language learners, and individuals with lower literacy levels. In this paper, we present MultiLS-IT, the first dataset specifically designed for automatic lexical simplification in Italian, as part of the larger multilingual Multi-LS dataset. We provide a detailed account of the data collection and annotation process, including complexity scores and synonym suggestions, along with a comprehensive statistical analysis of the dataset. With MultiLS-IT, we fill a significant gap in the field of Italian lexical simplification, offering a valuable resource for developing and evaluating automatic simplification models. Our analysis highlights the diversity of complexity levels in the dataset and discusses the moderate agreement among annotators, underscoring the subjective nature of lexical complexity assessment.

Introduction
Lexical simplification is a highly complex task within Natural Language Processing, encompassing broader automatic text simplification efforts [1]. It is defined as the task of replacing complex words with simpler synonyms that are more accessible to speakers, while preserving the original text’s meaning [2]. A complex word is one that is difficult for some readers to decode due to various characteristics that hinder comprehension [3, 4]. This area of research is of significant interest both socially and in computational applications. Socially, automatic simplification can enhance text comprehension for individuals with reading difficulties [5, 6], second language learners [7], those with cognitive disabilities [8], or individuals with lower literacy levels [9]. In general, making texts accessible to everyone is a democratic act, as it ensures that information and knowledge are available to all members of society, regardless of their reading ability or educational background [10]. From a computational perspective, it proves valuable for complex tasks such as machine translation [11], information retrieval [12], and summarisation [13] in addition to being an integral part of generic text simplification [1]. The ability to simplify text effectively can improve the performance of these applications by making the input data more uniform and easier to process [2]. Lexical simplification encompasses various subtasks [14]. The two most important ones are: 1. the prediction of word complexity, which involves identifying the words that need to be simplified [15]; 2. the replacement of complex words with simple synonyms [16]. Lexical complexity prediction (1) normally involves assigning a complexity value to a lexical item in context, ranging from 0 to 1, where 0 represents maximum simplicity and 1 denotes complexity [4]. This approach is a more advanced evolution of the traditional binary Complex Word Identification (CWI) [3], which classified words simply as complex or not complex. By moving towards a gradualism approach, lexical complexity prediction provides a finer-grained, continuous assessment of word difficulty, allowing for more tailored simplification efforts. The replacement of complex words with simpler synonyms (2) comprises three subtasks: the generation of substitutes, the ranking based on complexity, and the selection of the most appropriate substitute [14]. This multi-step process ensures that the chosen synonym not only reduces complexity but also fits seamlessly into the original context. One of the major challenges for such a user-dependent and therefore complex task is the lack of extensive annotated linguistic resources needed to train and evaluate automatic simplification models [2, 4]. Annotated datasets are crucial for developing and testing algorithms that can perform these tasks accurately. In this context, we present MultiLS-IT, which is, to the best of our knowledge, the first dataset specifically designed for automatic lexical simplification in the Italian language. This resource is part of a larger multilingual dataset, Multi-LS (Multilingual Lexical Simplification) [17], created for a shared task at the BEA workshop [18]1. The main contributions of this work are: • A detailed description of the data collection and annotation process of the Italian sub-dataset; • A descriptive analysis including statistics and visualizations providing an overview of the dataset’s characteristics; • The establishment of a reference point for future research in lexical simplification for Italian. With this work, we aim to fill a significant gap in lexical simplification research for Italian and provide a solid foundation for future studies and more effective lexical simplification technologies.

Conclusion
In this study, we present MultiLS-IT, the first dataset specifically designed for automatic lexical simplification in Italian. As part of the larger Multi-LS dataset, it addresses a significant gap in resources for lexical simplification in Italian. Despite its limited size, we believe that MultiLS-IT offers a valuable starting point for the development and evaluation of automatic simplification models. Our detailed description of the data collection and annotation process, including complexity ratings and synonym suggestions, provides a protocol that we hope will be followed and extended to increase the resources available for the Italian language. Our analysis revealed that the average complexity score of all target words is 0.276, with a standard deviation of 0.168, highlighting the range of complexity levels within the dataset. Including more diverse and complex contexts would provide a richer resource for training and evaluating simplification models. The inter-annotator agreement value of 0.4230 reflects a moderate level of consistency among annotators, emphasizing the inherent subjectivity in assessing lexical complexity. This relatively low value highlights the need to increase the sample size of both the dataset and the number of annotators to obtain more robust results. Future work should focus on expanding the dataset to include a greater variety of texts and more annotators to improve the reliability and generalizability of the results. Our goal is to create broader resources that enable the development of robust and effective lexical simplification technologies that can improve text accessibility and comprehension for a wide range of readers. In conclusion, while MultiLS-IT represents a significant step forward in the field of lexical simplification for Italian, there is still considerable potential for growth. Expanding the dataset to include a broader range of texts, increasing the number of annotators, and refining the annotation guidelines are all crucial steps toward improving the dataset’s quality. Additionally, the application of more advanced computational models and the exploration of real-world use cases will further contribute to the development of sophisticated tools for lexical simplification. We hope that this dataset will serve as a foundation for future research and development in automatic simplification, ultimately making information more accessible and comprehensible to all.