How adaptive is adaptive machine translation, really? A gender-neutral language use case (2023)

Abstract: This study examines the effectiveness of adaptive machine translation (AMT) for gender-neutral language (GNL) use in English-German translation using the ModernMT engine. It investigates gender bias in initial output and adaptability to two distinct GNL strategies, as well as the influence of translation memory (TM) use on adaptivity. Findings indicate that despite inherent gender bias, machine translation (MT) systems show potential for adapting to GNL with appropriate exposure and training, highlighting the importance of customisation, exposure to diverse examples, and better representation of different forms for enhancing gender-fair translation strategies.

## introduction with increasing adoption of gnl, its reflection in natural language processing (nlp) applications like mt becomes vital. ignoring gnl trends can perpetuate biased representations and inequalities @xcite , and also much of prior work in the field of gender bias is built on techniques which assume gender is binary. amt, which learns from users and adjusts to personal linguistic preferences @xcite , may offer promise for ensuring gnl use in mt, especially given the fact that it reduces post-editing efforts and has shown certain potential for empowering the human in the loop @xcite . this study evaluates the effectiveness of amt for gnl use, focusing on non-binary language in english-german translation using the modernmt 1 engine. our research, while somewhat aligned with the works of @xcite , @xcite and @xcite in their intent to address gender neutrality in mt, offers a distinct approach. while @xcite for post-processing steps to rewrite gendered sentences 1 https://www.modernmt.com/ into gender-neutral ones using either rule-based or neural approaches, and saunders et al. fine-tune a base model with synthetic datasets, our research investigates the real-time adaptability of amt systems to naturally integrate gnl into its translations. their approaches, although effective, often require access to and control over the internal mechanisms of the mt system or extensive task-specific datasets, which is not always feasible with black-box systems like modernmt. our study, on the other hand, explores how these black-box systems can dynamically adapt to gnl in their translation process without the need for post-processing steps or specialised fine-tuning. in this way, we assess the system's intrinsic ability to adjust to gnl, offering a more dynamic and contextual view of how these systems may cope with evolving language trends over the long run. we examine two distinct gnl strategies, such as gender asterisk and de-e-system. the gender asterisk form, which is increasingly utilised in german to inclusively represent all genders, is constructed by appending an asterisk before the gender-specific suffix (e.g., ärzt*innen). conversely, the de-e-system, which was developed by the association for gender neutral german (verein für geschlechtsneutrales deutsch foot_0 ) employs the concept of "inklusivum", a proposed fourth grammatical gender. this system introduces new declination rules and modifications for forming gender-neutral articles, nouns, and pronouns: for example, in this system, nouns typically end with either -e or -re, accompanied by the gender-neutral article de: de schülere (student), de autore (author). two versions of modernmt were deployed in this study: a system without the use of tm, as well as its customised variant, which was exposed to a tm containing gender-neutral forms. the research investigates whether amt exhibits gender bias in initial output, analyses its adaptability when working with two conceptually different gnl strategies, the role of tm in improving results, as well as factors influencing engine adaptation success. four scenarios were studied: using gender asterisk and de-e-system, each with and without tm, using texts based on the 2006 court procedures rules for the australian capital territory, which have been selected due to the fact that they inherently follow gnl principles in english. however, they also contain numerous elements -specifically, terms denoting professional roles or titles (as administrator, real estate agent, director of public prosecutions) -which do not hold gendered connotations in english but could potentially introduce gender bias when translated into german. the study includes 15 sentences for each of the 15 selected role-related nouns that could require gender-neutral adaptation during translation. for instance, consider the sentence: the real estate agent might be appointed to market the land and conduct the sale. in german, this could be translated using the gender asterisk as die*der immobilienmakler*in könnte beauftragt werden, das grundstück zu vermarkten und den verkauf durchzuführen, or with the de-e-system as de immobilienmaklere könnte beauftragt werden, das grundstück zu vermarkten und den verkauf durchzuführen. the evaluation of the engine's adaptive performance was conducted through a blend of quantitative and qualitative methods. for a quantitative perspective, we utilized two automatic metrics, character-based translation edit rate (character) and keystrokes ratio (ksr), and we computed the total number of adapted segments. while these metrics do not directly measure gender bias or adaptivity rate, they provide essential insights into the editing effort required to correct the engine's output, thus serving as proxies for its adaptive performance, as lower values for character and ksr would suggest that the engine is adapting well to the gnl. these metrics also enable a comparison of results across different experiments, such as those employing various gnl strategies or comparing performance with and without tm utilisation. for a more nuanced and targeted understanding of gender bias in the translation process, we complemented these metrics with a qualitative content analysis of specific instances of adaptation and a close examination of gender bias in the initial output. ## conclusion and future research these findings suggest that amt systems, despite being prone to gender bias, have the potential to adapt to gnl forms (especially if they are more widespread) with appropriate exposure, although further refinement and optimisation are necessary to improve their adaptability to gnl. tms facilitated adaptation by exposing the system to diverse gnl examples, thus enhancing its recognition and adaptation of genderneutral variants. solutions for better customisation @xcite ensuring better representation of different forms for various strategies will be crucial in advancing gender-fair translation strategies. it is important to acknowledge that the limited number of sentences may not fully capture all the nuances of translating gnl in broader contexts. this study, therefore, should be considered a preliminary exploration of this complex linguistic area. future research is necessary to validate these results in more extensive and diverse corpora, which would provide a more comprehensive understanding of amt systems' performance when dealing with gnl.