## 2 Related Work  
Despite their success, bias is a fundamental concern in LLMs, which often inherit it from the vast and largely unmoderated training datasets with which they are fed. These models learn, perpetuate, and amplify harmful social biases because they are trained with vast multimodal datasets that include text, images, audio, and video, and often inherit social biases embedded in human-generated content. As society evolves, the biases present in these archived data sources persist and inadvertently permeate LLMs. Bias has broad implications, including compromising efficiency in business decision making, exacerbate health care inequities, lead to misclassification of human-generated text (especially from non-native speakers), perpetuate stereotypes in specific business applications (such as educational recommendations, employment, financial management, and health consultations), affect the quality of interactions in multi-agent systems (especially in creative and emotionally sensitive contexts), and influence the reliability of models in numerical interpretation and reasoning (where a bias in the generation of smaller digits has been identified, contributing to numerical hallucinations) [1; 8; 14]. Social bias is broadly defined as the differing treatment or outcomes between social groups arising from historical and structural power asymmetries. Two main types of harms have been taxonomized [1]. On the one hand, representational harms imply denigrating and subordinating attitudes towards a social group. They include derogatory language (insults or phrases that denigrate a social group), disparate system performance (demeaning comprehension or generation among social groups or linguistic variations), erasure (omission or invisibility of the language and experiences of a social group), exclusionary norms (reinforcement of the normativity of the dominant social group and implicit exclusion or devaluation of other groups), and toxicity (offensive language that attacks, threatens, or incites hatred or violence against a social group). On the other hand, allocational harms involve a disparate distribution of resources or opportunities among social groups. They include direct discrimination (disparate treatment explicitly due to membership in a social group) and indirect discrimination (disparate treatment despite superficially neutral consideration toward social groups, due to proxies or other implicit factors). Specifically naming these harms, understanding their origins in social relations and histories, and recognizing the assumptions made in their conceptualization is crucial to effectively defining the role of NLP technologies in the reproduction of inequity and injustice. Gender bias is one of the most studied types of bias [5; 15; 16]. LLMs have been observed to exhibit gender stereotypes, for example, associating \"she\" with nurse and \"he\" with doctor, or preferring male pronouns in contexts of leadership and professional success. Previous research has also identified gender stereotypes in multilingual training corpora, such as associating women with being \"beautiful, empathetic, and neat\" and men with being \"tough, professional, leaders, and strong\". The potential to perpetuate racial bias in different domains has also been assessed in different studies, with the need for rigorous equity assessments in key application domains such as healthcare [17]. Several studies also highlight the importance of analyzing biases on multiple demographic axes and their interactions, such as gender, race, ethnicity, sexual orientation, religion, marital status, and number of children [15; 18; 19]. For example, a dataset has been developed to assess intersectional bias in South Asian LLM, considering religion, gender, marital status, and the number of children [20]. Entropy bias is defined as the variation in the information content of the generated text under different values of sensitive attributes in a prompt, e.g., showing a gender bias based on the amount of information provided in the responses [21].