Simplifying Administrative Texts for Italian L2 Readers with Controllable Transformers Models: A Data-driven Approach
Authors: Martina Miliani, Fernando Alva-Manchego, Alessandro Lenci (Università di Pisa, Cardiff University, 2023)
Topic: Text Simplification

Abstract
This paper presents a data-driven study focused on the automatic simplification of in-domain texts for specific target readers, which is “controlled” through data collected from behavioral analysis. We used these data to create Admin-It-L2, a parallel corpus of original-simplified sentences in the Italian administrative language, in which simplifications are aimed at Italian L2 speakers. Then, we used this corpus to test controllable models for text simplification based on Transformers. Although we obtained a high SARI score of 39.24, we show that this datum alone is not fully reliable in evaluating text simplification.

Introduction
Reading a text in a language that is easy to understand becomes necessary if the information it conveys is crucial to people’s daily lives. This is the case for acts and communications of Public Administrations (PA). In Italy, despite institutions repeatedly encouraging clear writing, PA texts have not abandoned the stylistic figures of “bureaucratese.” Suffering the most from the effects of these linguistic choices are speakers with language disparity. In particular, Italian L2 speakers deal with the Italian bureaucracy, for example, to obtain a visa for which A2-level language certification is required. Yet, comprehension of administrative texts is attested only with the C2 level, according to the CEFR. This paper presents a data-driven approach to simplification, which is “controlled” through data collected from behavioral analysis. We use these data to create Admin-It-L2, a parallel corpus of original-simplified sentences in the administrative language, in which simplifications are aimed at Italian L2 speakers. Then, we employ this corpus to test computational models. The adopted approach exploits the potential of Transformer-based multilingual models, which permit the use of training data also in languages other than Italian. Leveraging corpora in different languages allows us to overcome the limited availability of parallel corpora in Italian and the administrative domain in particular. To sum up the main contributions of this paper: We release Admin-It-L2, a parallel corpus in the Italian administrative language with simplifications aimed at Italian L2 speakers (Sec. 3). To the best of our knowledge, this is the first attempt to create a controlled simplification model specific to the Italian administrative language (Sec. 4). In particular, we train controllable models (Sec. 5) with multilingual data, to simplify texts aimed at Italian L2 speakers, thanks to data collected with behavioral analysis. We use Admin-It-L2 to test such models and we evaluate them with the available metrics for Automatic Text Simplification (ATS). We also manually analyze the produced simplification to assess the validity of such metrics for the Italian bureaucratic language (Sec. 6). Finally, we also show the results of a preliminary and exploratory experiment with ChatGPT. Based on GPT-3.5, this Large Language Model (LLM) gained popularity also thanks to the impressive performance it reaches in several NLP tasks. Thus, we decided to give a brief overview of its potential and limitations when applied to the simplification of Italian administrative texts for Italian L2 speakers (Sec. 7).

Conclusion
In this paper, we presented the first benchmark for the automatic simplification of Italian administrative texts aimed at Italian L2 speakers. We employed data collected from behavioral analysis to build Admin-It-L2, a parallel corpus of original-simplified sentences. Then, we used this corpus to test controllable models for text simplification based on Transformers. Although we obtained high results for SARI, we observed that SARI and BLEU alone cannot be used to evaluate text simplification and that a fully reliable automatic metric for the Italian language is still missing. Finally, we asked ChatGPT to simplify one of the sentences in Admin-It-2, obtaining a higher quality simplification than mT5, although the model presents some limitations, especially in reproducibility, and in some cases, in answer accuracy. For future work, we aim at validating Admin-It-L2 involving human annotation, and, more specifically, domain experts and Italian L2 speakers, to properly evaluate the employed models and the quality of their generations. As for the models, we aim to conduct an ablation study to assess the contribution of each control token. We also would like to adopt a control token to encode the language used in the sentences and employ the corpora in their original versions. Moreover, we aim to create a specific automatic metric to evaluate Italian text simplification. Finally, we would like to test open-source LLMs on administrative text simplification focused on specific target readers, with different prompts, in zero and few-shot settings.