Sparks of Fairness: Preliminary Evidence of Commercial Machine Translation as E nglish-to- G erman Gender-Fair Dictionaries (2024)

Abstract: Bilingual dictionaries are bedrock components for several language tasks, including translation. However, dictionaries are traditionally fixed in time, thus excluding those neologisms and neo-morphemes that challenge the language’s nominal morphology. The need for a more dynamic, mutable alternative makes machine translation (MT) systems become an extremely valuable avenue. This paper investigates whether commercial MT can be used as bilingual dictionaries for gender-neutral translation. We focus on the English-to-German pair, where notional gender in the source requires gender inflection in the target. We translated 115 person-referring terms using Google Translate, Microsoft Bing, and DeepL and discovered that while each system is heavily biased towards the masculine gender, DeepL often provides gender-fair alternatives to users, especially with plurals.

## related work bilingual dictionaries are bedrocks of various linguistic applications, including language learning @xcite and translation. motivated by such an important role, several efforts have studied how to extract them at scale @xcite or integrate them in neural machine translation systems @xcite . in this paper, we study modern commercial translation systems' as en-de bilingual dictionaries with a focused eye on gender-fair forms. our work is part of a broader discourse on fairness and inclusivity in machine translation. neural and commercial systems are known to encode stereotypical views on genders @xcite , leading to brittle gender inflection capabilities in grammatical gender languages @xcite , covert biases in genderless languages @xcite , and inadequate handling of neo-morphemes @xcite and named entities @xcite . further, systems are nearly incapable of gender-neutral translation for human entity nouns in en-it @xcite and de-en @xcite . to ground automatic translation with human practices, recent studies have reported on neutralization and genderinclusive strategies used by professional translators @xcite and mt post-editors @xcite . these findings and research efforts underscore the need for our research on commercial mt dictionary capabilities. as sinclair poses it, static, bilingual dictionaries suffer from gaps in coverage, e.g., failing to include neologisms @xcite . this work studies whether modern commercial systems have fixed on traditional gender forms or are indeed "meeting the needs of the multifarious user." ## background as a basis for this work, we first introduce the relationship between gender and language ( §3.1), followed by the definition of gender-fair language (gfl) ( §3.2), and possible strategies in german ( §3.3). ## results first, we summarise the results of the exploratory study ( §5.1), then we provide an overview of the results obtained with deepl and focus on the overt gender of the machine-translated outputs in german ( §5.2). we subsequently analyze which gender-fair, i.e., inclusive and neutral, strategies are found in the singular ( §5.2.1) and plural ( §5.2.2). ## discussion in the present contribution, we were interested in using three commercial mt models as english-to-german gender-fair dictionaries. unsurprisingly, the results suggest that commercial mt models are still systematically biased towards masculine forms when translating from a notional gender, english, into a grammatical gender language, german. in our exploratory study, we find that google translate usually generates gender-specific translations, but only in the masculine and feminine genders. this feature is not available in microsoft bing and deepl. both google translate and microsoft bing provide alternative translations, usually synonyms, in their dictionary interface. these alternatives are, however, generally masculine. conversely, deepl offers numerous alternatives that are also gender-fair. the main study confirms that deepl is heavily biased towards the masculine with about 67% of outputs having this overt gender both in the singular and plural. a great difference emerges between singular and plural: the number of feminine translations significantly decreases from 18% to 2%. conversely, the number of gender-inclusive and neutral translations increases from 1% to 11% and 4% to 15% respectively. there are at least two main reasons for this phenomenon. first, some nouns are gender-specific in the singular form, but not in the plural. for instance, the term "traveller" has different declensions. without articles, the masculine form is "reisender" whilst the feminine is "reisende". in the plural, there is one form only, i.e., "reisende". second, germanspeaking countries have a relatively strong feminist tradition and gender-fair language policies to avoid masculine generics were introduced several years ago @xcite . gfl is now quite common in, e.g., administrative texts where different gender-fair strategies, such as participial forms and gender star (*), are increasingly used for the declension of plural terms @xcite . in the translations generated by deepl, several gender-neutral and inclusive forms were found. gender-neutral forms included compounds with neutral terms such as person (e.g. "der spekulative mensch", the speculating person), abstract nouns (e.g., "die staatsanwaltschaft", the prosecutor's office), and participial forms (e.g. "die vorsitzenden", the presidents). gender-inclusive forms were also found, including binneni (e.g. "die koordinatorinnen", the coordinators), slash (e.g. "die mitbürger/innen", the fellow citizens), and double forms (e.g. "die betreuerinnen und betreuer", the counsellors). though deepl seems to be receptive of genderfair forms that, probably quite rarely, occur in the training data, gender-inclusive strategies found in the outputs are usually outdated mostly because they are considered inclusive of binary genders only. for instance, binneni was once commonly used and studies about its effect on mental representations date back more than twenty years ago @xcite . this strategy has nowadays been replaced by the use of genderinclusive characters such as gender star (*) @xcite . the findings of this study show how current commercial mt systems cannot keep up with linguistic change. the field of gender-fair language is constantly evolving and there is yet no onesize-fits-all solution to issues of gender representation @xcite . in fact, the selection of a gender-fair language strategy is highly context-dependent @xcite @xcite . for this reason, future research endeavours on mt debiasing should be the result of interdisciplinary efforts, involving computational linguistics, sociolinguistics, and translation studies amongst others. to conclude, we discuss three major limitations of the present study. the first concerns the nonreplicability of the results. unfortunately, we don't have insights into the system used by google, deepl, and microsoft bing. updates and/or retraining may lead to changes in the outputs over time. the second involves the analysis of genderneutral and inclusive strategies. as already explained, there is no standard for gfl and creativity is often required. the soundness of genderfair solutions might hence be judged differently among experts and, more importantly, depends on the broader context, i.e. at least the text, in which such solutions are used. finally, we considered one language pair only because of the high amount of manual work involved in the translation of our dataset and its analysis. ## conclusion in this study on gender bias in mt, we investigated the use of three commercial systems as englishto-german gender-fair dictionaries. drawing on a community-created gender-fair dictionary, we developed a dataset including 115 gender-specific terms for which gender-fair alternatives in german were proposed. we then provided the terms with a translation in english both in the singular and plural form. subsequently, we conducted a brief exploratory study with deepl, google translate, and microsoft bing by back-translating into german the first 20 seed nouns contained in our dataset. the results from this exploratory study show that all systems default to male forms. moreover, google translate usually provides gender-specific translations in the masculine and feminine, and microsoft bing offers synonymous translations in the masculine form only. for these reasons, we further conduct our study with deepl which usually generates three to four translations per seed word. in a nutshell, our findings seem to suggest that gfl is starting to appear in deepl outputs, probably due to the relatively widespread gfl use in german-speaking countries. nevertheless, deepl still generates gender-fair forms inconsistently and far more often in the plural. finally, the genderinclusive forms found in the machine translations are generally outdated and exclusive of genders beyond the binary -an issue still under-researched within translation studies and computational linguistics with few exceptions @xcite @xcite .