Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation (2024)

Abstract: Most works on gender bias focus on intrinsic bias — removing traces of information about a protected group from the model’s internal representation. However, these works are often disconnected from the impact of such debiasing on downstream applications, which is the main motivation for debiasing in the first place. In this work, we systematically test how methods for intrinsic debiasing affect neural machine translation models, by measuring the extrinsic bias of such systems under different design choices. We highlight three challenges and mismatches between the debiasing techniques and their end-goal usage, including the choice of embeddings to debias, the mismatch between words and sub-word tokens debiasing, and the effect on different target languages. We find that these considerations have a significant impact on downstream performance and the success of debiasing.

## introduction natural language processing models were shown to over-rely and over-represent gender stereotypes. 2 these can typically be found in their internal representation or predictions. for example, consider the following sentence: in this work, we identify a gap in the literature between intrinsic bias mitigation and its influence on downstream tasks. namely, while extrinsic bias may affect human users in a variety of applications, debiasing techniques often focus only on intrinsic measures, aiming to obfuscate gender from pretrained embeddings @xcite @xcite . these approaches leave many unanswered questions when deploying them within a complex downstream model for specific tasks. as shown in figure 1 , we systemically explore three fundamental challenges when integrating intrinsic debiasing techniques within complex opensource neural mt architectures. we find that different design choices lead to a wide difference in extrinsic bias as well as task performance. first, we explore different approaches to cope with discrepancies between different tokenization strategies. while intrinsic debiasing is largely performed over complete words from a fixed dictionary, modern mt requires mapping those onto subword elements determined via a data-dependant tokenizer. we find that debiasing only complete words outperforms a more naive debiasing of all sub-word tokens. second, several word embedding tables could be debiased within an mt system. therefore, a preliminary architectural question is which of them to debias. we explore various combinations, finding the optimal configuration depends on the intrinsic debiasing technique. third, we explore the effects of debiasing a translation model over three target languages (hebrew, german, and russian). while all three encode morphological noun gender, they differ in script, typology, and morphology. we find that an important factor for debiasing efficiency is the number of words represented as single tokens, a property determined both by the language's morphological properties as well as its sampled distribution in the tokenizer training data. taken together, our results suggest that extrinsic debiasing involves many interdependent challenges which cannot be inferred from an intrinsic outlook. we hope our work will promote more research on combining intrinsic debiasing methods to downstream tasks to produce extrinsically fairer mt models. ## background there is an abundance of debiasing methods in the field @xcite @xcite @xcite @xcite @xcite . most of them focus on intrinsic debiasing. we focus on three prominent methods, outlined below. importantly, all of these methods learn a transformation that can be applied to arbitrary vectors, once the model has finished training, and all were tested mostly intrinsically. ## integrating intrinsic debiasing in mt we examine debiasing methods within the popular encoder-decoder approach to mt, as shown in figure 1 . next, we describe the different research questions addressed in our setup. which embedding to debias? an encoderdecoder model has multiple embedding tables that can be intrinsically debiased: (1) the input matrix of the encoder; (2) the input matrix of the decoder; and (3) the output of the decoder, usually before the softmax layer. 4 we employ different intrinsic debiasing techniques to each of these tables and evaluate their effect on downstream performance. which words to debias? tokenization poses a challenge for extrinsic debiasing as it may introduce discrepancies between the intrinsically debiased elements (complete words) and the mt input model (sub-word tokens) @xcite . we experiment with three different configurations: (1) all-tokens: debiases embeddings of all tokens in the model's vocabulary; how does debiasing affect different languages? we experiment with three target languages that encode morphological gender for nouns, representing different typological features: (1) hebrew, a semitic language with abjad script, (2) russian, a slavic language with a cyrillic script, and (3) german, a germanic language with latin alphabet. ## conclusions and future work we systematically explore different challenges and design choices when integrating intrinsic debiasing methods within complex machine translation sys-tems. we find that it is better to debias only words representative of gender and correspond to single tokens. furthermore, it is important to couple the debiasing method with the specific embedding table (e.g., encoder versus decoder), and that different target languages lead to vastly different results. our findings do not suggest the "correct" approach to addressing bias in machine translation. instead, they highlight the complexity of mitigating gender bias in large model architectures. @xcite studied the effect of debiasing different transformer layers in bert's next sentence prediction task. they found that interventions at the model's inner layers, including attention mechanisms, led to improvements in intrinsic bias mitigation. in future work, it would be interesting to test how debiasing the attention and inner layers of an mt model affects bias. finally, @xcite evaluate the impact of different gender pairs defining the gender direction in various debiasing methods (e.g., hard-debiasing). they noticed that bias measurements can be affected by biases in the pairs themselves like cultural and cognitive biases. future work can explore gender pair selection for the debiasing methods both in source and target languages, and examine how different pairs influence extrinsic bias.