Re-contextualizing Fairness in NLP : The Case of I ndia (2022)

Abstract: Recent research has revealed undesirable biases in NLP data and models. However, these efforts focus of social disparities in West, and are not directly portable to other geo-cultural contexts. In this paper, we focus on NLP fairness in the context of India. We start with a brief account of the prominent axes of social disparities in India. We build resources for fairness evaluation in the Indian context and use them to demonstrate prediction biases along some of the axes. We then delve deeper into social stereotypes for Region and Religion, demonstrating its prevalence in corpora and models. Finally, we outline a holistic research agenda to re-contextualize NLP fairness research for the Indian context, accounting for Indian societal context, bridging technological gaps in NLP capabilities and resources, and adapting to Indian cultural values. While we focus on India, this framework can be generalized to other geo-cultural contexts.

## introduction while natural language processing (nlp) has seen impressive advancements recently @xcite @xcite , it has also been demonstrated that language technologies may capture, propagate, and amplify societal biases @xcite . although nlp is adopted globally, most studies on assessing and mitigating biases are in the western context, foot_0 focusing on axes of disparities in the west, relying on western data and justice norms, and are not directly portable to non-western contexts @xcite . this is especially troubling for india, a pluralistic nation of 1.4 billion people, with fast-growing investments in nlp from the government and the private sector. 2 there is commendable recent work on fairness in nlp models for indian languages such as @xcite @xcite . but, for a nation with many religions, ethnicities, and cultures, re-contextualizing nlp fairness needs to account for the various axes of social disparities in the indian society, their proxies in language data, the disparate nlp capabilities in indian languages, and the (lack of) resources for bias evaluation. @xcite proposed a research agenda for ai fairness for india based on interviews of 36 experts on indian society and technology. in this paper, we build on their work with a focus on nlp. we start with a brief discussion on the major axes of social disparities in india ( §3). we then discuss the proxies of some of these axes in language and empirically demonstrate prediction biases around these proxies in nlp models ( §4). we then delve deeper into stereotypes along the axes of region and religion, demonstrating their prevalence in data and models ( §5). finally, we build on these empirical demonstrations to propose an overarching research agenda along the societal, technological, and value alignment aspects important to formulating fairness research for the indian context ( §6). while we focus on india in this paper, our framework can be adapted to re-contextualize fairness research for other geo-cultural contexts. to summarize, our main contributions are: (1) an overarching research agenda for nlp fairness in the indian context accounting for societal, technological, and value aspects; (2) resources (curated and created) for enabling fairness evaluations in the indian context available; 3 and (3) empirical demon-strations of prediction biases and over-prevalence of social stereotypes in data and models. ## related work research on undesirable biases has been a growing priority in nlp @xcite @xcite . social biases are shown to be baked into pretrained language models @xcite and models for downstream tasks such as sentiment analysis @xcite and toxicity detection @xcite . while the majority of nlp fairness research focuses on gender @xcite @xcite and racial biases @xcite @xcite , other axes of disparities such as ability @xcite , age @xcite , and sexual orientation @xcite have also gotten some attention. however, the majority of this research is framed in and for the western context, relying on data and values reflecting the west @xcite . recently, fairness research in nlp has also been expanded to non-english languages such as arabic @xcite , japanese @xcite @xcite . evidence of cultural biases for different countries have also been recorded @xcite in lms. our work adds to this line of research. building on @xcite , we take a more holistic approach towards nlp fairness in the specific geo-cultural context of india. more specifically, we re-frame the agenda they proposed along "re-contextualising data and model fairness; empowering communities by participatory action; and enabling an ecosystem for meaningful fairness" with an nlp-centric lens. ## axes of disparities identifying prominent axes of disparities is the first step in laying out a holistic nlp fairness research agenda for the indian context. we follow @xcite who identify the major axes of potential ml (un)fairness ( further, amplified social biases may be faced by those with overlapping categories of marginalized groups. we do not focus on this intersectionality here and leave discussion about it to section 6. ## proxies of axes and predictive disparities bias evaluation in nlp relies on proxies of subgroups in language, such as identity terms and personal names, to reveal the undesirable associations present in models and data @xcite . in the indian context, we identify three major kinds of proxies: identity terms, personal names, and dialectal features. using such proxies however poses unique challenges in the indian context. for example, there are thousands of caste identities and hundreds of ethnolinguistic regional identities that are not codified in any authoritative sources. similarly, there do not exist any large resources that provide subgroup associations for personal names, such as the us census data (for race) or ssa data (for gender) in the west. building exhaustive resources to capture such fine-grained social groups is outside the scope of this paper. however, in this section we curate identity terms and personal names with prototypical identity associations. we adopt a black-box evaluation strategy to demonstrate predictive biases in standard nlp pipelines/models and also demonstrate the utility of india-specific resources. finally we note that these resources and studies are meant to be demonstrative, not exhaustive. ## stereotypes in indian context we now turn our attention to the prevalence of social stereotypes from the indian society in nlp data and models. there is limited literature and resources on social stereotypes in the indian context, as outlined in section 2. notably, de @xcite reported stereotypes around region and religion subgroups in india. they report the top 5 and bottom 5 traits that participants associate with 11 regional and 4 religious identities. but, the study is narrowly scoped to limited adjectives and is from decades ago thus may not reflect the current indian society. recent research within nlp has built large stereotype datasets such as stereoset @xcite and crows-p @xcite to evaluate models, but they may not capture the stereotypes relevant to india. we build a set of stereotypical associations based on prior work but employing indian annotators. like (de souza, 1977), we focus on the region and religion. this choice is motivated by the availability of resources and the challenges in studying the other axes (outlined in section 6). we then use the stereotypes reported by de @xcite and our created dataset to analyse nlp corpora and models for the prevalence of these stereotypes. ## re-contextualizing fairness given the empirical demonstration of biases in the indian context in data and models, we now return to the broader agenda for re-contextualizing nlp fairness. we re-frame the agenda of sambasivan et al. (2019) along three aspects: accounting for social disparities, bridging technological gaps, and adapting to values & norms. ## conclusion in this paper, we holistically re-contextualize fairness research for the indian context taking an nlpcentric lens to @xcite . we lay out a research agenda advocating to account for the societal context in india, bridge technological gaps in capability and resources, and align with local values and norms (section 6). our focus here is on india, but the broader framework of this work can be used to recontextualize fairness for any geocultural context. we outline the prominent axes of disparities in india (section 3), and demonstrate biases around them in nlp models and corpora. to summarize: first, our perturbation analysis reveals that sentiment model predictions are significantly sensitive to regional, religious, and caste identities (section 4.1), and dialectal features (section 4.3). second, our disco analysis shows the necessity of india-specific resources for revealing biases in the indian context (section 4.2). third, we build a stereotype dataset for the indian context and demonstrate preferential encoding of stereotypical associations in both nlp data and models (section 5). while there is more work to be done, we believe this is an essential first step towards a meaningful nlp fairness research agenda for india.