Extending Challenge Sets to Uncover Gender Bias in Machine Translation: Impact of Stereotypical Verbs and Adjectives (2021)

Abstract: Human gender bias is reflected in language and text production. Because state-of-the-art machine translation (MT) systems are trained on large corpora of text, mostly generated by humans, gender bias can also be found in MT. For instance when occupations are translated from a language like English, which mostly uses gender neutral words, to a language like German, which mostly uses a feminine and a masculine version for an occupation, a decision must be made by the MT System. Recent research showed that MT systems are biased towards stereotypical translation of occupations. In 2019 the first, and so far only, challenge set, explicitly designed to measure the extent of gender bias in MT systems has been published. In this set measurement of gender bias is solely based on the translation of occupations. With our paper we present an extension of this challenge set, called WiBeMT, which adds gender-biased adjectives and sentences with gender-biased verbs. The resulting challenge set consists of over 70, 000 sentences and has been translated with three commercial MT systems: DeepL Translator, Microsoft Translator, and Google Translate. Results show a gender bias for all three MT systems. This gender bias is to a great extent significantly influenced by adjectives and to a lesser extent by verbs.

## introduction the problem of unfair and biased models has been recognized as an important problem for many applications of machine learning @xcite . the source of unfairness typically is based on biases in the training data. in many domains, unfairness is caused by sampling biases. in machine translation (mt), however, the main source of unfairness is due to historical or social biases when there is a misalignment between the world as it is and the values or objectives to be encoded and propagated in a model @xcite . one source of imbalance in natural language is the association of specific occupations with gender. typically, occupations in more technical domains as well as occupations with high social status are associated with male gender @xcite . in natural language processing (nlp), gender bias has been investigated for word embeddings @xcite . analogy puzzles such as "man is to king as woman is to x" generated with word2vec foot_1 yield @xmath0, the output is @xmath1. only few publications exist that directly address gender bias and mt. although gender bias is particularly relevant for translations into gender-inflected languages, for instance from english to german, where biased models can result in translation errors @xcite . for the english sentence "the doctor told the nurse that she had been busy.", a human translator would resolve the co-reference of 'she' to the doctor, correctly translating it to 'die ärztin'. however, a neural machine translation model (nmt) trained on a biased dataset, in which most doctors are male might incorrectly default to the masculine form, 'der arzt'. @xcite investigated the prediction of age and gender of a text's author before and after translation. they found that bing translator, deepl translator, and google translate all skew the predictions to be older and more masculine, which shows that nmt systems and the expression of gender interact. according to @xcite , the first systematic analysis of gender bias in mt is from @xcite . they introduce winomt which is the first challenge set explicitly designed to quantify gender bias in mt sys-tems. furthermore, they introduce an automatic evaluation method for eight different languages. evaluating six mt systems, they found a strong preference for masculine translations in all eight gender-inflected languages. the above example sentence, where 'doctor' has been errouneousely translated into its male german form (arzt) is one of the 3, 888 sentences used in winomt. winomt focuses solely on the translation of occupations to evaluate gender bias. in this paper, we present the extended data set wibemt to uncover gender bias in neural machine translation systems. gender stereotypical occupations are augmented by gender stereotypical adjectives and verbs and we investigate how congruent and incongruent combinations impact translation accuracy of the nmt systems deepl, microsoft translator and google translate. in the next section, the original winomt data set is introduced together with our extensions. afterwards, results for the three nmt systems based on 70,686 sentences are presented, followed by a discussion and an outlook. ## constructing the extended challenge set wibemt to construct a more diverse challenge set, we extend winomt, respectively its core data base wino-bias. therefore, we identify verbs and adjectives of high stereotypicality with respect to gender. a gender score is determined by the cosine similarity of these words to a list of gender specific words. to calculate the similarity different pretrained word embeddings are used, where each of these words is represented by a vector. with the resulting most feminine and masculine adjectives the original sentences of winobias are extended. furthermore, new sentences are created combining occupations with gender stereotypical verbs. ## results after the creation of the wibemt challenge set and the translation of all 70, 686 sentences with each nmt system, the translations were categorized as feminine, masculine, neutral, or inconclusive / wrong. the latter will be referred to as "wrong". due to these discrete categories the chi 2 test of independence will be used for all statistical tests. as the extended winobias sentences include a pronoun that defines the gender, the results considering these data can be divided into true and false, and feminine and masculine translations. on the other hand, the verb sentences do not include any cue for a correct gender in the translation. therefore, they are just analyzed for feminine and masculine translations. the calculated feminine-ratio (%t f g) results from all feminine-translations divided by the sum of feminine-and masculine-translations. the calculated correct-gender-ratio (%t cg) results from all translations with correct gender divided by the sum of all translations with correct and incorrect gender. all chi 2 -tests were, if necessary, bonferroni corrected. first, the results for the verb-sentences; second, the results for the extended winobias sentences; and third, further results are presented. ## conclusions and further work the three neural machine translation systems evaluated with respect of their gender bias are black boxes, in so far as the architecture is not publicly available and -even more important -it is not transparent on what data these systems are trained. it is most likely that the gender bias in all three systems is inherited from the data used for training and their use of word embeddings. to give a closer look on gender-bias of the nmt systems deepl, microsoft, and google translate, an extension of the winomt challenge set -the first challenge set designed to measure gender bias in nmt systems -has been presented. while winomt relies solely on the gender of the translation of occupations, our extended set wibemt includes gender-adjectives and genderverbs. thereby, a more detailed assessment of gender bias has been possible. the number of sentences in our challenge set is, with over 70, 000 sentences, nearly 20 times as large as the original winomt challenge set. this makes it less prone to overfitting when used to evaluate or reduce gender bias in nmt systems. we could show that adjectives do significantly influence the gender of translated occupations. against the hypotheses, feminine as well as masculine adjectives skew the translations of nmt systems to more translations with the feminine gender. nevertheless, feminine adjectives still produce significantly more translations with the feminine gender than masculine adjectives do. all three nmt systems prefer translations to the generic masculine when no pronoun defines a correct gender for the translation. despite this preference, 5.7% to 9.3% of all verb sentences are translated to their feminine gender by the mt systems. this can mostly be attributed to a gender bias in translating occupations, as our results regarding verb sentences and occupation categories show. the effect of gender-verbs on the gender of translations only became significant in deepl and microsoft translator and was smaller than the effect of the occupations gender-categories. it can be assumed that google translate only performed better on the verb sentences task, as it generally has the strongest tendency to translate sentences to their masculine gender. surprisingly, gender-adjectives drastically improve the overall accuracy of all three nmt systems when it comes to the correct gender in translations. while the accuracy slightly drops for sentences with masculine pronouns, it drastically improves for sentences with feminine pronouns. therefore, the discrepancy in accuracy between masculine and feminine pronoun sentences de-creases, resulting in lower discrimination. one could even argue that gender-adjectives reduce gender bias in the output of nmt systems, but at the same time, it can be discriminating in itself that, as soon as you add an adjective to describe a person, the instance of this person is more likely to be translated to its feminine gender. further research is certainly needed to find reasons for this effect and to assess the potentials of discrimination. the sentences in our wibemt challenge set areas the original winomt challenge set -constructed in a systematic way. while this allows for a controlled experiment environment, it might also introduce some artificial biases @xcite . a solution could be to collect real-world examples of sentences, which are suitable for gender bias detection. furthermore, the limitation on one source language (english) and one target language (german) does not allow for a generalization of the results. another limitation of our study -as most other studies -is that it does not take into account that gender should not be seen as a binary, but rather a continuous variable. @xcite , for example, outline why "trans exclusionary" co-reference resolution systems can cause harm, which is probably also valid for mt systems. a further extension of the challenge set could help to shine a light on the shortcomings of the inclusion of transgender persons of nmt systems.