Investigating Gender Bias in Large Language Models for the Italian Language
Authors: Elena Sofia Ruzzetti, Dario Onorati, Leonardo Ranaldi, Davide Venditti, Fabio Massimo Zanzotto (Università di Roma Tor Vergata, Sapienza Università di Roma, Idiap Research Institute, 2023)
Topic: Gender and Inclusive Language Studies

Abstract
Large Language Models (LLMs) are becoming increasingly flexible and reliable: the large pre-training phase enables them to capture a large number of real-world linguistic phenomena. However, pre-training on large amounts of data can also cause the representation of harmful biases. In this paper, we propose a method for identifying the presence of gender bias using a list of occupations characterized by a large imbalance between the number of male and female employees.

Introduction
Large language models (LLMs) have achieved superhuman performances in several NLP applications [1, 2]. They demonstrate a clear upward performance trend along with the increasing model size and pretraining data, namely scaling law [3]. However, by overhumanizing learning abilities, it is possible that these LLMs inherit stereotypical associations between social groups and professions [4, 5]. Bias or, better, prejudice [6] is the sword of Damocles of fairness in many data-driven applications, such as facial recognition [7] or recommendation systems [8]. Even in modern NLP, a clear presence of bias in different models has been observed. Bolukbasi et al. [4] detected the presence of stereotypical biases in word embedding vectors measuring association between gender and certain professions, while Caliskan et al. [9] proposed the Word Embedding Association Tests (WEAT) to assess the strength of stereotypical associations regarding gender and races. Similar biases were later observed in Pretrained Language Models. Several benchmarks like SEAT [10], StereoSet [11], and CrowS-Pairs [12] enable testing of Pre-trained Language Models like BERT [13] and ELMo [14]. The advent of LLMs [1, 15, 16, 17] has yet to alleviate this phenomenon. In fact, despite the increasing capabilities of this family of models, the underlying LLMs generate toxic or offensive content [18, 19], and reproduce biases that exist in the training data [20, 21, 22]. While some models can be used in beneficial applications, like in identifying biased texts [23], biases hidden within these models could hinder their abilities [6]. For these reasons, while some previous work quantifies the ability of the models to cause no harm by interviewing human evaluators [24], it is necessary to develop automated approaches to easily test models before they are available to a large group of users. In this paper, we analyze how existing LLMs capture some known stereotyped associations between gender and profession for the Italian language. To quantify the presence of social bias, we created a test dataset (Section 2.1) that allows us to monitor the relation between gender and 171 different occupations. We selected professions that, according to ISTAT data, have a significant imbalance in the number of male employees compared to the number of female employees in Italy. Then, we propose a method to measure the strength of the association between gender and profession (Section 2.2) on different LLMs. Stemming from the stereotype score definition that can be found in Nadeem et al. [11], we define a model biased as it systematically prefers the stereotyped association over an anti-stereotyped one. Finally, we test several LLMs trained on the Italian language and attest that a large number of LLMs available for the Italian language have strong gender biases (Section 3).

Conclusion
In this work, we propose to investigate gender bias for the Italian language in pre-trained LLMs, identifying if and to what extent these models capture real-world imbalances while training on text data. We present a list of professions inspired by official ISTAT data and propose a simple and effective method to quantify the presence of gender bias in occupations. We assess the presence of strong biases across different model families such as BLOOM, LLaMA, and XGLM.