Mitigating Gender Stereotypes in H indi and M arathi (2022)

Abstract: As the use of natural language processing increases in our day-to-day life, the need to address gender bias inherent in these systems also amplifies. This is because the inherent bias interferes with the semantic structure of the output of these systems while performing tasks in natural language processing. While research is being done in English to quantify and mitigate bias, debiasing methods in Indic Languages are either relatively nascent or absent for some Indic languages altogether. Most Indic languages are gendered, i.e., each noun is assigned a gender according to each language’s rules of grammar. As a consequence, evaluation differs from what is done in English. This paper evaluates the gender stereotypes in Hindi and Marathi languages. The methodologies will differ from the ones in the English language because there are masculine and feminine counterparts in the case of some words. We create a dataset of neutral and gendered occupation words, emotion words and measure bias with the help of Embedding Coherence Test (ECT) and Relative Norm Distance (RND). We also attempt to mitigate this bias from the embeddings. Experiments show that our proposed debiasing techniques reduce gender bias in these languages.

## introduction word embeddings are used in most natural language processing tools. apart from capturing semantic information, word embeddings are also known to capture bias in society @xcite . while most research has been focused on languages like english, less research has been done on low-resource languages and languages that have a grammatical gender @xcite . a language with grammatical has a gender associated with every noun irrespective of whether the noun is animate or inanimate, e.g., a river in hindi has * first author feminine gender. in contrast, words like writer have masculine and feminine counterparts. this gender association affects the pronouns, adjectives, and verb forms used during sentence construction. grammatical genders in hindi are masculine and feminine. in marathi, there additionally exists a third neutral gender as well. spoken by more than 600 million people, hindi is the 3rd most spoken language in the world. marathi is the 14th most spoken language with approximately 120 million speakers foot_0 . given the expanse and the amount of people speaking these languages, it is essential to address the bias introduced by the computational applications of these languages. we create a dataset of occupations and emotions. the occupation dataset consists of gendered and neutral occupation titles. the emotion dataset has words of different emotions like anger, fear, joy, and sadness. first, we identify existing gender bias by defining a subspace that captures the gender information. there are several ways to find this information. we use principal component analysis (pca) and relational inner product association (ripa) @xcite . we use the existing metrics for evaluation: embedding coherence test @xcite , relative norm distance @xcite . we modify these formulas so that they are correctly applicable to these gendered languages. we perform our experiments on the fasttext word embeddings. next, we mitigate the gender bias found by the aforementioned using two approaches: projection and partial projection. in summary, the key contributions of this paper are: previous work to quantify and measure bias was done by @xcite . they tried to find out a gender subspace by using gender-definition pairs. they proposed a hard de-biasing method that identifies the gender subspace and tries to remove its components from the embeddings. the majority amount of research on gender bias is being done in english, which is not gendered @xcite . languages like spanish or hindi have a grammatical gender, i.e., every noun is assigned a gender. @xcite was one of the first papers to examine bias in languages with grammatical gender like french and spanish. they used a modified version of the word embedding association test (weat) @xcite to quantify the bias. @xcite suggested mitigation techniques to remove gender bias like data augmentation, gender-swapping, and hard de-biasing according to the downstream task in nlproc. being low-resource languages, there is less research done in languages like hindi and marathi. previous work in indic languages was done by @xcite where they built an svm classifier to identify the bias and classify it. the problem with this method is that it needs a labeled gender dataset beforehand to train the classifier. recent work by @xcite tries to find out bias in english-hindi machine translation. they implement a modified version of the tgbi metric based on grammatical considerations for hindi. tgbi metric is used to detect and evaluate gender bias in machine translation systems. @xcite measure hindi specific societal biases like religion bias and caste bias along with gender bias. ## data in @xcite , the authors have compiled a list of professions in english and tried to find bias in them. similarly, we compile a list of 166 professions, each in hindi and marathi languages. we split the professions into two parts, first is gender-neutral p neu and the other is gendered p gen . similarly, we create a list of words of different emotions similar to the one in @xcite in hindi and marathi languages. the emotions are broadly classified into four types: anger, fear, joy, and sadness. we have verified this data with the help of 2 independent native speakers of these languages. we also create pair of feminine and masculine seed word pairs in both the languages to identify the gender subspace. for example: queen, king. we call them target words t . target words for hindi language is shown in figure 1 . the dataset is available here foot_1 we test the bias on our data using fasttext embeddings. fasttext is a word embedding method that extends the word2vec model. instead of learning vectors for words directly, fasttext represents each word as an n-gram of characters. this helps capture the meaning of shorter words and allows the embeddings to understand suffixes and prefixes. a skip-gram model is trained to learn the embeddings once the word has been represented using character n-grams @xcite . morphology is the field of linguistics that studies the internal structure of words. morphologically rich languages refer to languages that contain a substantial amount of grammatical information @xcite . indic languages are morphologically rich because of the existence of a large number of different word forms. fasttext embeddings are the best choice for indian languages as they are capable of capturing and integrating sub-word information using character n-gram embeddings during training @xcite . ## conclusion and future work in this paper, we attempted to find gender stereotypes on occupations and emotions and tried to debias them. embedding coherence test and relative norm distance were used as a bias metric in the gender subspace. the debiasing methods used were projection and partial projection. but we see that partial projection as a debiasing method works the best in most cases. future work could include trying out these techniques on downstream tasks and checking the performance before and after debiasing. the main problem with experimenting on downstream tasks is the availability of datasets in these languages. we would also like to experiment with debiasing contextual embeddings and large language models. apart from that we would also like to address other types of bias like religion, social and cultural, which are particularly inherent in hindi and