Measuring bias in Instruction-Following models with for the Italian Language
Authors: Dario Onorati, Davide Venditti, Elena Sofia Ruzzetti, Federico Ranaldi, Leonardo Ranaldi, Fabio Massimo Zanzotto (Sapienza Università di Roma, Università di Roma Tor Vergata, Idiap Research Institute, 2024)
Topic: Language Models

Abstract
Instruction-Following Language Models (IFLMs) are the state-of-the-art for solving many downstream tasks. Given their widespread use, there is an urgent need to measure whether the sentences they generate contain toxic information or social biases. In this paper, we propose Prompt Association Test for the Italian language itaPat: a new resource for testing the presence of social bias in different domains in IFLMs. This work also aims to understand whether it is possible to make the responses of these models more fair by using context learning, using ``one-shot anti-stereotypical prompts''.

Introduction
Large Language Models (LLMs) and Instruction-Following Language Models (IFLMs) have achieved human performances in several NLP applications AUTHOR. Their ability to generate text or respond to prompts is increasingly performing and adaptive to different tasks. However, these models learn from data that frequently contains prejudices and stereotypical associations, as data inherently possesses and reflects the social biases generated by humans. Social bias refers to prejudices, stereotypes, or unfair assumptions individuals or groups hold about others based on factors like race, gender, ethnicity, socioeconomic status, or other social characteristics. The LLMs could embed stereotypical associations among social groups during training phase AUTHOR because they learn from huge amounts of data, which may reflect existing social prejudices. The presence of social bias in LLMs can lead to harmful consequences, such as generating biased or discriminatory outputs, perpetuating stereotypes, or unfairly marginalizing certain groups. According with the definition of AUTHOR, we consider a model bias if it systematically prefers the stereotyped association over an anti-stereotyped one. The social bias is the Achille's heel for many Natural Language Processing (NLP) applications AUTHOR. The presence of bias in the NLP models has been detected by means different strategies. AUTHOR proposed the Word Embedding Association Tests (WEAT) to detect the stereotypical associations regarding gender and races in the word embedding vectors, while AUTHOR extended it (SEAT) for the Pre-trained Language Models like BERTAUTHOR and ELMO AUTHOR. The stereotypical domains can be also detected by these sentence encoders using benchmarks AUTHOR. The increased use of LLMs AUTHOR and IFLMs AUTHOR, driven by their ease of use, leads to a series of social problems, including those related to the social bias. In fact, despite the increased capabilities on several tasks of these models, they often reproduce biases that can be learned from training data AUTHOR and generate toxic or offensive content AUTHOR. AUTHOR and AUTHOR extended WEAT and SEAT to detect the stereotypical associations respectively in LLMs and IFLMs. Previous works quantify the amount of associations among social groups generated by English-language models, and it is necessary to develop similar approaches for models, both multilingual and Italian, for the Italian language. In this paper, we propose the Italian Prompt Association Test itaPat: a new resource for testing the presence of social biases in Instruction-Following Language Models (IFLMs) for the Italian language. To quantify the presence of social bias, we created a dataset consisting of the adaptation of prompts present in pat. To enhance the Italian-centric nature of this dataset, the adaptations have been carefully designed according to ISTAT (Italian National Institute of Statistics) data. This involves the identification and selection of the most common Italian first names and nationalities that Italians statistically perceive most negatively based on social trends and prejudices. Then, we test these Italian prompts on both multilingual and Italian IFLMs, and observe whether their answers reflect stereotypical associations. If the model responses align with a stereotype, it indicates that it has internalized and reproduced the ``Italian stereotype" embedded in the data. Finally, we also explore the use of ``one-shot anti-stereotypical prompts'' as a strategy to guide models toward generating fairer and less biased responses. This approach is particularly advantageous because it circumvents the need for computationally intensive fine-tuning or retraining of the models, which would otherwise require substantial resources. Furthermore, our method successfully yields more fairer responses from Italian-focused language models across different social domains.

Conclusion
In this paper, we propose a Prompt Association Test for Italian language itaPat, a resource to quantify the social bias in multilingual and Italian Instruction-Following Language Models (IFLMs) in multiple domains, such as gender, race and age. itaPat is an adaptation of pat AUTHOR on the Italian language. Our experiments with different models show that multilingual model are better at responding to prompts than the Italian models, however they have a greater presence of bias. Consequently, this highlights a significant challenge in the development of AI language models: the need to balance performance improvements with ethical considerations, ensuring that advancements in model capabilities do not compromise the fairness and inclusivity of the outputs generated. Italian models often provide incorrect or repetitive responses, whether stereotypical or anti-stereotypical, which undermines the reliability of the Bias score. Among the Italian models evaluated, LLaMAntino demonstrates the best ability to generate accurate responses; however, it still exhibits a disproportionately high Bias score. Moreover, our proposed methods for enhancing the fairness of model responses lack consistency, as each model exhibits varying levels of responsiveness depending on the specific domain in question. This variability highlights the need for a more tailored approach to bias mitigation that considers the unique characteristics of each model and the contexts in which they operate. We expect itaPat to be an important tool for quantifying the presence of social bias in different dimensions and, therefore, for encouraging the creation of fairer in the multilingual and Italian IFLMs for the Italian language.