Analyzing Gender Representation in Multilingual Models (2022)

Abstract: Multilingual language models were shown to allow for nontrivial transfer across scripts and languages. In this work, we study the structure of the internal representations that enable this transfer. We focus on the representations of gender distinctions as a practical case study, and examine the extent to which the gender concept is encoded in shared subspaces across different languages. Our analysis shows that gender representations consist of several prominent components that are shared across languages, alongside language-specific components. The existence of language-independent and language-specific components provides an explanation for an intriguing empirical observation we make”:” while gender classification transfers well across languages, interventions for gender removal trained on a single language do not transfer easily to others.

## introduction pretrained models of contextualized representations @xcite @xcite are known in their ability to capture both explicit and implicit information during training. a special case of these models are multilingual models @xcite , which are pretrained with texts in multiple languages. these models were shown to induce the emergence of similar representations in different languages, a phenomenon that was put to use for transfer between languages in end-tasks @xcite @xcite . however, the underlying mechanism is still not clear, and we do not know yet the full extent to which the representations of these models share information across languages. the rise of pretrained models has been accompanied with growing concern regarding sensitive information they might encode, e.g. gender or ethnic distinctions. pretrained language models were shown to be sensitive to gender information, both when it is explicitly stated in texts, as well as when it can be inferred from implicit information @xcite . we still lack a complete understanding of what the model captures, and the ways to control and change the information in this context as well. in this work, we aim to shed light on the way gender, a popular use case of a human-interpretable concept, is represented in multilingual models, and whether it is encoded in a language-dependant way. in a series of experiments, we uncover a surprising finding: gender-identification ability is highly transferable across languages (section 4.1) but neutralizing gender identification is not (section 4.2). while these two findings may seem contradictory at first glance, this is explained by several levels of gender marking: both cross-lingual and languagespecific (section 5). we start our analysis by training gender classifiers and examining their ability to transfer across languages. we then proceed to identifying "gender subspaces" -subspaces that encode gender -in each language, with the goal of understanding which information is language-specific, and which is shared across languages. following recent work on linear interventions @xcite @xcite , we take an "amnesic" approach: we study the extent to which neutralizing the gender subspace in one language interferes with gender prediction in another language. finally, we analyze the similarity in the gender-encoding components across languages. we find that while linear probes for gender transfer well between languages -that is, a gender classifier that is trained on one language predicts gender well in another language, the method we employ for neutralizing gender fails to transfer across languages. a deeper analysis reveals a finegrained organization of the gender-encoding subspaces across languages: they are spanned by a few main directions, which are largely similar across languages; but in addition to these directions, there are other directions that are language-specific. the existence of several similar directions explains the high degree of transferability of linear gender classifiers across languages, while the existence of a large amount of language-specific information explains the inability to efficiently remove gender information in one language based on another language's representation. we summarize our findings and contributions as follows: (a) we show that gender-identification is highly transferable across languages (section 4.1); (b) we find that neutralizing gender identification does not transfer well across languages (section 4.2); (c) we demonstrate that gender subspaces are spanned by a few directions that are largely similar across languages; and also by other directions that are language-specific (section 5.1); (d) we find that the directions that are shared across languages are the most dominant ones (section 5.2). the code for our experiments is available at https://github.com/gonenhila/ multilingual_gender . ## datasets and multilingual representations for our experiments we use the biosbias dataset @xcite , the multilingual bios-bias dataset @xcite and the multilingual bert model (mbert, @xcite ) as detailed below. ## conclusion towards better understanding of the underlying mechanism of multilingual modeling, in this work we focus on the way gender is represented across languages. we analyze and quantify the extent to which gender information is shared in multilingual representations in english, french and spanish. we find that on the one hand, gender prediction transfers very well across languages: training a linear classifier on english data yields a high quality classifier for french and spanish as well (true for all three languages in both directions). on the other hand, our attempt to transfer gender removal in cross-lingual manner was unsuccessful. we show that these two results are compatible, and together they shed light on the structure of the representation space: we provide experimental evidence that the most salient directions are shared between languages (enabling good transfer of the classifiers), while others are unique per language (interfering with gender removal across languages). the key observation is that a single "good" direction of the gender subspace in one language is enough for cross-lingual gender prediction transfer, while transfer of gender neutralization requires all directions to be shared, otherwise, the remaining ones can be used to recover gender information after the removal of the shared ones.