Lupus Alberto: A Transformer-Based Approach for SLE Information Extraction from Italian Clinical Reports
Authors: Livia Lilli, Laura Antenucci, Augusta Ortolan, Silvia Laura Bosello, Maria Antonietta D'Agostino, Stefano Patarnello, Carlotta Masciocchi, Jacopo Lenkowicz (Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Univeristà Cattolica del Sacro Cuore, 2024)
Topic: In-domain IR and IE

Abstract
Natural Language Processing (NLP) is widely used across several fields, such as in medicine, where information often originates from unstructured data sources. This creates the need for automated systems, in order to classify text and extract information from Electronic Health Records (EHRs). However, a significant challenge lies in the limited availability of pre-trained models for less common languages, such as Italian, and for specific medical domains. Our study aims to develop an NLP approach to extract Systemic Lupus Erythematosus (SLE) information from Italian EHRs at Gemelli Hospital in Rome. We then introduce Lupus Alberto, a fine-tuned version of AlBERTo, trained for classifying categories derived from three distinct domains: Diagnosis, Therapy and Symptom. We evaluated Lupus Alberto's performance by comparing it with other baseline approaches, selecting from available BERT-based models for the Italian language and fine-tuning them for the same tasks. Evaluation results show that Lupus Alberto achieves overall F-Scores equal to 79%, 87%, and 76% for the Diagnosis, Therapy, and Symptom domains, respectively. Furthermore, our approach outperformed other baseline models in the Diagnosis and Symptom domains, demonstrating superior performance in identifying and categorizing relevant SLE information, thereby improving clinical decision-making and patient management.

Introduction
Natural Language Processing (NLP) is used in many applications, such as in the medical domain, where the huge amount of unstructured data sources coming from Electronic Health Records (EHRs) generates the need to develop automated systems for text classification and information extraction. However, employing such methods is challenging due to the scarcity of pre-trained models in less common languages like Italian, and for specific medical domains. In this study, we explored the Systemic Lupus Erythematosus (SLE), a complex pathology which involves different organ domains and can occur in patients at several levels of severity. For this reason, information about diagnoses, symptoms and therapies are used by physicians to characterize Lupus patients and to make better informed decisions about therapy changes or time for the next contact visit. However, these Lupic features are not always available in a structured format, then there is the need for NLP approaches in order to interpret clinical reports and extract the desired data. Based on the literature, large language models (LLMs) and transformer-based architectures represent the state-of-the-art for EHR classification tasks AUTHOR. This work aims to develop a transformer-based approach to identify SLE information from unstructured EHRs at the Italian Gemelli Hospital of Rome. We then propose Lupus Alberto, a fine-tuned version of Alberto AUTHOR, the available BERT-based model for the Italian language trained on Italian tweets. In order to assess the Lupus Alberto performance, we compare it with other baseline approaches, choosing among the BERT-based models available for the Italian language, always fine-tuned on the same tasks.

Conclusion
This study aims to deliver a transformer-based approach to extract SLE information from real-world data of the Gemelli Hospital of Rome. The scarcity of available models for the Italian language, specialized in Lupus, prompted us to develop a solution to automate the extraction process of SLE information from Italian EHRs. We especially focused on identifying features in the domains of Diagnosis, Therapy and Symptom, reported as of interest for SLE. Our work shows that Lupus Alberto presents competitive performance if compared to other baseline methods, outperforming especially in the classification of information in the Diagnosis and Symptom domains, achieving F-Scores of 79% and 76%, respectively.