I ndi B ias: A Benchmark Dataset to Measure Social Biases in Language Models for I ndian Context (2024)

Abstract: The pervasive influence of social biases in language data has sparked the need for benchmark datasets that capture and evaluate these biases in Large Language Models (LLMs). Existing efforts predominantly focus on English language and the Western context, leaving a void for a reliable dataset that encapsulates India’s unique socio-cultural nuances. To bridge this gap, we introduce IndiBias, a comprehensive benchmarking dataset designed specifically for evaluating social biases in the Indian context. We filter and translate the existing CrowS-Pairs dataset to create a benchmark dataset suited to the Indian context in Hindi language. Additionally, we leverage LLMs including ChatGPT and InstructGPT to augment our dataset with diverse societal biases and stereotypes prevalent in India. The included bias dimensions encompass gender, religion, caste, age, region, physical appearance, and occupation. We also build a resource to address intersectional biases along three intersectional dimensions. Our dataset contains 800 sentence pairs and 300 tuples for bias measurement across different demographics. The dataset is available in English and Hindi, providing a size comparable to existing benchmark datasets. Furthermore, using IndiBias we compare ten different language models on multiple bias measurement metrics. We observed that the language models exhibit more bias across a majority of the intersectional groups. All the scripts utilized and datasets created in this study are publicly available.

## introduction language models (lms) are trained on vast amounts of text data and excel in various natural language processing (nlp) tasks. however, many recent studies have shown evidence of undesirable biases and stereotypes in nlp datasets and models @xcite @xcite . these models stand a risk of reproducing the learned harmful biases in various downstream nlp applications @xcite @xcite which in turn can be significantly detrimental to certain demographic groups. this necessitates the need for high-quality benchmark datasets to measure models' preference for stereotypical associations in diverse social contexts. motivation: india is a country with many different languages, religions, castes, and regional identities. ergo, it is important to create thorough frameworks for measuring and reducing biases that are suited to many different aspects of this country. furthermore, the impact of biases in lms is particularly pronounced in india due to its diverse user base. even though a lot of research has been done to identify the sources of bias in lms, the benchmark datasets such as @xcite , @xcite mostly focus on english language and western culture. this creates a significant gap in understanding and mitigating biases in lms tailored to the indian context. moreover, we found some major logical inconsistencies and fundamental errors in these datasets, which make them unreliable to measure the extent to which nlp systems reproduce stereotypes. @xcite also dissect and highlight some major pitfalls in existing benchmark datasets. additionally, the detection of intersectional bias is missing in the indian context but is crucially needed because of the complex and interconnected nature of social identities present in india. we aim to fill these gaps by proposing indibias, a high-quality comprehensive dataset to measure and quantify lm's biases and stereotypes in the indian context. among the various axes of social disparities in india, we have addressed seven major categories namely gender, religion, caste, age, region, physical appearance, and occupation/socioeconomic status along with three intersectional axes such as gender-religion, gender-caste, and gender-age. our dataset is in hindi and english languages. ## related work bias in lms refers to the presence of unfair or discriminatory behavior exhibited by these models towards certain demographic groups or sensitive topics @xcite . numerous studies demonstrate that lms tend to reflect and amplify societal biases present in the pre-training data @xcite @xcite . while most efforts to detect and mitigate bias in lms focus on the english language and western society, recent works address biases in data and language representations from diverse cultures and languages like arabic @xcite , french (kurpicz-briki, 2020), italian @xcite , etc. there are also few studies addressing this problem in the indian context @xcite . efforts to understand and quantify biases in lms have led to the development of metrics @xcite @xcite and bias benchmark datasets. common benchmark creation approaches include using predefined word sets, template-based sentences @xcite ), or crowd-sourced sentences (crowspairs, @xcite ) to assess bias by examining output generation for certain demographics and measuring model behavior on sensitive attributes. notably, however, there exists a gap in such studies concerning non-western contexts. to address this, @xcite releases an extension of crows-pairs for french with some modifications to the original dataset. recently, researchers have started focusing on such issues in the indian context. based on interviews with 36 indian society and technology experts, @xcite proposed a research agenda for ai fairness in india and have suggested six distinct axes of fairness in india. @xcite have released a fairness evaluation corpus covering stereotypes pertaining to region and religion axes relevant to the indian context in english language. @xcite cover stereotypes across 179 demographic groups from 178 countries through their benchmark dataset. however, to the best of our knowledge, there has been no prior work to create benchmark datasets in hindi language. furthermore, previous works have primarily focused on biases along a single dimension with limited exploration of intersectional biases across multiple relevant characteristics. @xcite propose methods to examine intersectional identities specifically along the race-gender axis using bert and gpt-2 models, with a focus on african american and european american racial groups. in contrast, our work aims to evaluate the extent of intersectional biases in lms trained on a multilingual corpus that includes various indian languages across three intersectional axes, i.e, gender-religion, gender-caste, gender-age. in the following section, we delve into the characterization of social biases in the indian context. ## characterization of social biases in indian context axes of disparities like caste, religion, and region exhibit a rich array of social biases specific to the indian context. caste-based prejudices have a long-standing prevalence in india, despite the efforts for their eradication by the indian society @xcite . the indian entertainment media, too, has highlighted the plight of sections of society that are at the receiving end of caste and construction of sentence pairs and issues mentioned in the concern column are elaborated in sections 4.2.1, 4.3. for more examples, refer to the bhatt et al. (2022) conform with de souza's work by demonstrating similar associations in datasets-wikipedia and indiccorp-en corpus and lms-muril and mbert. works like sahoo et al. (2023), rajadesingan et al. (2019), haokip (2021), sabharwal and sonalkar (2015), and mcduie-ra 2012 also make significant contributions towards spotlighting the specific biases and stereotypes faced by groups of individuals in the indian society. moreover, social biases and stereotypes have a multi-fold nature, possessing global and geocultural context-specific elements. some global axes of social disparities are gender, age, and physical appearance. however, these global axes too exhibit a variation across different demographics. for instance, consider gender, an axis of disparity that sees various commonly experienced biases and stereotypes by women. however, there are also geo-cultural context-specific biases against women which may exhibit a vast amount of variation across the globe. to illustrate this more clearly, consider the following example sentences: s1: women can't do math. s3: women wearing traditional attire in west bengal are seen as cultural ambassadors. sentence s1 represents a stereotype commonly held by the world. in comparison, sentences s2 and s3 demonstrate a complete reversal of stereotypes across different states in india. additionally, these stereotypes may or may not be valid across the globe. the fast adoption of nlp applications in india's legal, medical, education, and media sectors necessitates ensuring lm's fairness for the indian context. hence, it is imperative that the research community builds diverse, reliable, high-quality benchmark datasets designed to measure model bias in a context-specific fashion. ## indibias dataset we take a multifaceted approach to create indibias. it is a composition of modified sentence pairs from crows-pairs (an existing benchmarking dataset) adapted to the indian context, sentences generated using indibias tuples, and template-based sentences created by leveraging the power of llms. the following subsections provide a detailed description of the dataset creation process. ## intersectional biases intersectional bias refers to the discrimination or prejudice that individuals who belong to multiple marginalized groups or have intersecting social identities experience @xcite . it acknowledges that individuals are not subject to biases based solely on a single identity dimension but rather experience a complex interplay of biases that originate from the intersections of their various social categories. we investigate intersectional bias across three dimensions, i.e., gender-religion, gender-caste, gender-age. we use sentence embedding association tests @xcite to measure the degree of biasness of different models using bleached templates. ## conclusion and future work through indibias, we aim to facilitate advancements in the understanding of social biases in llms, with a specific focus on indian languages and cultural contexts. we have released an extensive set of identity-attribute tuples encompassing seven different demographics such as gender, religion, caste, age, region, physical appearance, and occupation, to capture positive and negative stereotypes prevalent in indian society. we follow a translate-filter-modify approach to create an indian version of the crows-pairs dataset in english and hindi languages. we then augment this dataset using manually annotated sentence pairs using the tuple dataset. we conducted a comprehensive bias analysis of different lms using this dataset. in addition, our analysis using seat revealed the existence of intersectional biases in the indian context. this finding highlights the significance of considering the compounded effects of multiple dimensions in lm biases. additionally, we aim to augment the dataset by incorporating sexual orientation instances into the indian crows-pairs. also, we intend to expand such dataset to multiple indian languages.