F rench C row S -Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than E nglish (2022)

Abstract: Warning: This paper contains explicit statements of offensive stereotypes which may be upsetting. Much work on biases in natural language processing has addressed biases linked to the social and cultural experience of English speaking individuals in the United States. We seek to widen the scope of bias studies by creating material to measure social bias in language models (LMs) against specific demographic groups in France. We build on the US-centered CrowS-pairs dataset to create a multilingual stereotypes dataset that allows for comparability across languages while also characterizing biases that are specific to each country and language. We introduce 1,679 sentence pairs in French that cover stereotypes in ten types of bias like gender and age. 1,467 sentence pairs are translated from CrowS-pairs and 212 are newly crowdsourced. The sentence pairs contrast stereotypes concerning underadvantaged groups with the same sentence concerning advantaged groups. We find that four widely used language models (three French, one multilingual) favor sentences that express stereotypes in most bias categories. We report on the translation process from English into French, which led to a characterization of stereotypes in CrowS-pairs including the identification of US-centric cultural traits. We offer guidelines to further extend the dataset to other languages and cultural environments.

## introduction human language technologies can have a direct impact on people's everyday life. the natural language processing community who contributes to the development of these technologies has a responsibility to understand the social impact of its research and to address the ethical implications @xcite . the increasing use of large language models has raised many ethical concerns, including the risk of bias and bias amplification @xcite . biases in nlp have received a lot of attention in recent years @xcite . however, the bulk of the work has addressed biases linked to the social and cultural experience of english speaking individuals in the united states. in this work, we seek to widen the scope of bias studies by creating material to measure social bias in multiple languages and social contexts. as a case study, we chose to address biases against specific demographic groups in france. the crows-pairs dataset @xcite was recently developed to address nine types of bias. it contains pairs of sentences: a sentence that is more stereotyping and another that is less stereotyping. the goal is to present masked language models with these sentences to assess how the models rank them. if stereotyped sentences are consistently ranked higher than less stereotyped sentences, it characterizes the existence of bias in the model. while crows-pairs was designed to measure social bias against protected demographic groups in the us, many of the biases, such as gender or age, can also apply to other geographic locations. however, other biases are very specific to the united states, such as those pertaining to african-americans. this study provides a contribution to assessing the prevalence of us-centric contexts in crows-pairs. a recent study focusing on gender bias in english and german has shown that methods to evidence and mitigate bias in english do not necessarily carry well to other languages @xcite . this highlights the importance of addressing bias in language models in multiple languages. we chose to use the crows-pairs dataset as a starting point for our study with the hypothesis that the availability of a multilingual version of the dataset would allow for cross-language comparison of some types of bias. furthermore, we also hypothesized that the process of enriching the dataset with sentence pairs in french would create an opportunity to characterize biases that are specific to each country and language. this work's main contributions are as follows: ## corpus development this work builds on the crows-pairs dataset, that we extend with content in french and english. bias types. we use the nine categories of bias included in the crows-pairs dataset: ethnicity/color, gender/gender identity or expression, socioeconomic status/occupation, nationality, religion, age, sexual orientation, physical appearance, and disability. we did not find a specific definition of disadvantaged groups in europe, although legislation in several european countries target categories that are consistent with the above for programs aiming to increase diversity and inclusion. we decided the nine categories defined in crows-pairs were likely to be relevant in the french context. translation. we randomly divided the 1,508 sentence pairs contained in the crows-pairs dataset in 16 random samples of 90 sentence pairs (plus one of 68 sentence pairs). in each set, we selected one sentence per language pair. the sentence was then translated into french by one of the authors (translator) and the resulting translation was then revised and validated by another author (editor). all authors are native speakers of french who are fluent in english. two have received professional training in translation (one holds a msc in translation); three have received training in linguistics. in the translation process, translators and editors noted the translation process used (for example, if names or other elements were adapted to convey meaning of the stereotype in french) and also marked the sentences perceived as untranslatable due to cultural dissimilarities. ## related work few studies have addressed bias in language models in french. @xcite have investigated semantic bias induced by domain in the context of domain adaptation for machine translation. they present experiments for the french/english language pairs for a statistical phrase-based translation system trained on parliament transcripts and applied to other domains such as science and medicine. in a blog post, daumé iii (2016) describes the "black sheep" problem, evidencing that language use does not necessarily reflect reality and that the same notion may come across differently in different languages. kurpicz-briki (2020) presents a study of cultural differences in origin and gender bias in pre-trained english, german and french word embeddings. the author adapts the weat method @xcite that contains material for measuring bias in english language word embeddings to (swiss) french and german and shows that the bias identified differ between the three languages studied. this is probably the effort that is closest to the present study. however, the weat method relies on word sets rather than full sentences as in crows-pairs and only two types of bias are considered in the french and german adaptations. more importantly, @xcite show that the weat metrics, which was created to measure the biases in the embeddings themselves, does not correlate with results obtained using extrinsic evaluation of biases, using downstream applications. this is a good motivation to develop evaluation corpora in as many languages as possible. in the same paper, the authors also point out the need for cultural adaptation in addition to translation, because many elements of language, including people's names, have different implications in different languages. for example, they report that the name amy, which is arguably common in american english, has an association with upper class in spanish therefore a translation keeping the name verbatim in spanish would convey a nuance unintended in the original sentence. we agree with this analysis and one of our goals was to address it in the translation of the crows-pairs dataset as illustrated in some of the examples in blodgett et al. (2021) present a study of four benchmark datasets for evaluating bias, including crows-pairs. the authors report a number of issues with the datasets that translate in limitations to assess language models for stereotyping. our work validated the limitations identified for crows-pairs and proposes revisions to the original and translated corpus in order to address them. ## conclusion we introduce a revised and extended version for the crows-pairs challenge dataset, which will be made available as a complement to the original resource. the corpus uses the minimal pair paradigm to cover ten categories of bias. our experiments show that widely used language models in english and french exhibit significant bias. the process of extending crows-pairs from english to french highlighted that there are cultural specificities to bias, so that (1) multilingual challenge datasets benefit from bias examples natively sourced from each of the languages and ( 2 ) bias examples would benefit from a formal description such as social frames for a better cross-culture characterization. these are avenues for future work on the dataset.