Gender-Neutral Machine Translation Strategies in Practice (2025)

Abstract: Gender-inclusive machine translation (MT) should preserve gender ambiguity in the source to avoid misgendering and representational harms. While gender ambiguity often occurs naturally in notional gender languages such as English, maintaining that gender neutrality in grammatical gender languages is a challenge. Here we assess the sensitivity of 21 MT systems to the need for gender neutrality in response to gender ambiguity in three translation directions of varying difficulty. The specific gender-neutral strategies that are observed in practice are categorized and discussed. Additionally, we examine the effect of binary gender stereotypes on the use of gender-neutral translation. In general, we report a disappointing absence of gender-neutral translations in response to gender ambiguity. However, we observe a small handful of MT systems that switch to gender neutral translation using specific strategies, depending on the target language.

## introduction gender bias in language technologies has long been a concern for the community (see @xcite for an overview). however, machine translation (mt) stands out as a particularly important area of study due to its widespread use, its potential to foster global connections and uplift underserved communities, coupled with the unique technical challenges posed by grammatical gender differences across languages. research on gender bias in machine translation often examines misgendering errors resulting from stereotypes or the default use of masculine forms @xcite . these errors contribute to both representational harms -such as the underrepresentation of female and non-binary individuals, or the limited depiction of women in certain roles @xcite this article is licensed under a creative commons 4.0 licence, no derivative works, attribution, cc-by-nd. @xcite @xcite , inter alia) -and allocative harms, for example the increased burden on female users to manually correct machine-translated text @xcite . based on a human user study, @xcite found that such misgendering errors are particularly harmful for non-binary individuals. here we are concerned with misgendering harms that arise, not due to stereotypes, but rather due to ambiguous or underspecified gender in the source text, a situation that occurs frequently in english. one obvious solution is to use gender-inclusive language (i.e., language that encompasses all gender identities when the gender of a particular referent is unknown, or when gender is unimportant). however, the ease of this strategy varies by language. in notional gender languages, such as english, gender has minimal effects, primarily manifesting through gendered pronouns (e.g., he, himself) and relatively few gender-specific nouns (e.g., brother/sister, husband/wife, waiter/waitress, etc.). english also features some androcentric language (e.g., mankind) and, depending on the text domain and social norms, lends itself to the use of masculine generics when gender is irrelevant (e.g., "any driver in violation will lose his license"). it is easy to see then why gender inclusive language in english is relatively straightforward; the widely established use of "they/themselves" as a singular personal pronoun and the existence of in-vocabulary gender-neutral nouns (e.g., sister/@xmath0, husband/@xmath1, waiter/@xmath2, @xmath3. however, in grammatical gender languages @xcite , multiple parts of speech typically need to agree with the gender of the referent (including nouns, verbs, and adjectives), and furthermore, grammatical gender may be restricted to binary (masculine/feminine) cases. therefore, gender neutrality can require more contrived rewrites, potentially using new or out-of-vocabulary words, and coherence may be sacrificed @xcite . in the context of machine translation, a task designed to preserve meaning and coherence, a difficulty naturally arises in translation from lower to higher grammatical gender agreement contexts. gender neutral translation is the task of preserving gender neutrality in response to gender ambiguity in the source. here, we study machine translation from english into grammatical gender languages, and assess mt systems' sensitivity to the need for gender neutrality. we ask, if appropriate, low-barrier gender-neutral translation options exist in the target language, will modern mt systems use those options in response to gender ambiguity? we find that a deliberate form of gender ambiguity in english (using "they" as a singular personal pronoun) triggers a gender-neutral response in a small handful of surveyed mt systems, but overall, we observe a disappointing insensitivity to gender ambiguity. the results indicate much room for improvement, and we release the scoring code for the test set to enable future work at https://github.com/hillary-dawkins/wmt24gender-dialogue . ## the task of gender neutral translation based on the assertion that gender ambiguity should be preserved through translation to mitigate harm, gender-neutral translation (gnt) is an emergent challenge task to benchmark mt systems for this behaviour @xcite . through a careful analysis of existing genderinclusive language guidelines for both english and italian, and a survey of human participants, piergentili et al. (2023a) define the desiderata of gender neutral translation: neutrality should be maintained in the target if gender is ambiguous given the source, but conversely, gendered language should be used if it is known given the source. this makes gender neutrality a dynamic constraint, meaning mt systems should be sensitive to the need of it. in this study, we define a test set and metric to measure this sensitivity, termed the gender-neutral response, in a controlled way. our test instances are templated to create pairs of inputs that differ only in whether the referent's gender is known or unknown, and adjective translations in target gen-dered languages are used to measure the sensitivity. prior studies in gender-neutral translation have been somewhat limited in scope; there has been a focus on translating gender-neutral pronouns @xcite , including neo-pronouns @xcite , and genderneutral nouns that are prone to masculine generic translation @xcite . these studies are focused on a first-order effect (i.e., the gender of the referent word itself -a pronoun or noun), whereas we are interested in second-order gender effects -the gender agreement of an adjective with its resolved referent. this is representative of the challenge that occurs when translating from english to grammatical gender languages. @xcite was the first, to our knowledge, to create a more robust, natural gnt benchmark (gente), involving real in-the-wild inputs with reference translations (@xmath4. the test set contains both gender-ambiguous and gender-determined inputs paired with both gendered and gender-neutral reference translations. using the gente benchmark, @xcite found that both neural mt systems and large language models (llms) struggle with gender-neutral translation, but we suggest this may be due to the difficulty of the test set: multiple gender agreements are needed -or should be avoided -in each output, and the target language, italian, has only binary gender grammatical cases in its formal form. furthermore, because the inputs are derived from real, naturally occurring language, it is hard to control for the existence (or difficulty) of a coherent gender-neutral translation. here, we take a step back and provide a somewhat easier and more controlled test set for gnt to compliment the gente. uniquely, the construction of the paired inputs, differing only in determined/ambiguous gender, allows us to measure a sensitivity to the need for gender neutrality -a prerequisite for gnt. in our setup, all gendered translations are measured via adjectives, and this provides several advantages. firstly, the choice of gender declension after translation can be determined via dictionary lookup, rather than a learned classifier as in @xcite and @xcite . the dictionary method allows for a more fine-grained analysis of specifically which gender-neutral strategies are being used in translation (e.g., a gender-neutral adjective vs. a noun phrase). secondly, adjectives in our target languages typically have gender-neutral synonyms available, and therefore the barrier to obtaining a gender-neutral translation is low (i.e., a human translator is not expected to struggle with this task). we include three target languages with different grammatical gender cases applied to adjectives: although using templates to study gendered adjectives in isolation does not represent the breadth of real-world complexities, it does provide a starting point grounded in gender resolution that is typical of grammatical gendered languages. ## test suite and participating mt systems we make use of a test suite released as part of a shared challenge task at wmt24 2 : gender resolution in speaker-listener dialogue roles @xcite . all instances in the test suite involve spoken dialogue and meta-context surrounding the dialogue. most instances involve two characters, and both characters may take on the speaker or listener role at times. adjectives within spoken dialogue either refer to the speaker (self-referential) or the listener, and meta-context either resolves the gender of the adjectives' referents, or leaves the gender ambiguous. as a minimal example: ⟨"i'm/you're stubborn", i said to him.⟩ either resolves the gender (you're) or not (i'm). examples of test suite instances supporting the current study are shown in figure 1 , and a full description of the test suite is provided in the appendix. the wmt24 shared task @xcite ) garnered 40 unique mt systems operating in 11 translation directions, and the gender resolution test suite obtained translations from 21 unique mt systems in 3 language directions (@xmath5, cs, es). participating mt systems include commercial online systems, dedicated neural mt systems, and large language models (llms) prompted for mt. throughout, results use the system names as provided by the shared task organizers (i.e., commercial online systems are anonymized). one key 1 though dynamic social norms may permit this to varying degrees. 2 https://www2.statmt.org/wmt24/index.html challenge in gender-neutral translation, as highlighted by @xcite , is the need for paragraph-level inference to accurately determine gender. this consideration makes the wmt24 submissions particularly relevant and timely to study. both the general mt shared task (formerly known as the news translation task) and several challenge tasks placed an emphasis on paragraph-level translation. among the 28 participating systems that specified their translation strategies, 17 reported using a paragraph-level or hybrid approach. ## observed gender-neutral strategies over the entire test suite of 17,966 source adjectives, each translated into three target languages involving 21 unique mt systems, we observed 5 categories of gender-neutral translation strategies in practice: the pretty nurse smiled. "i think i'm awkward", s/he said to the tough engineer. the brawny doctor smiled. "i think i'm jealous", s/he said to the gentle teacher. the pretty nurse smiled. "i think i'm awkward", they said to the tough engineer. the brawny doctor smiled. "i think i'm jealous", they said to the gentle teacher. ## definitions and metrics we define the gender neutrality of an mt system on a specified subset of source adjective inputs as the proportion of gender-neutral translations observed in the output. gender-determined inputs refer to source sentences that provide unambiguous gender information, such that the gender of the adjective referent is known. in contrast, gender-ambiguous inputs do not contain enough information to resolve the gender. for english source inputs, gender ambiguity naturally arises due to gender-neutral firstand second-person pronouns (i, you), and can also be induced through the use of "they" as a singular third-person pronoun. we refer to the former as gender ambiguity by omission, and the latter as active gender ambiguity since it involves a more deliberate choice to avoid "he/she" (and especially to avoid a default masculine form if gender is unknown or unimportant). here we are interested in how mt systems respond to a need to preserve gender ambiguity from source to target, and therefore we define metrics to measure the change in gender neutrality when source inputs switch from determined to ambiguous gender. firstly, baseline gender neutrality (n det ) measures the proportion of gender neutral translations in gender-determined cases, and provides a sense of how various mt systems make use of the gender-neutral strategies by default. baseline gender neutrality is naturally expected to vary by target language due to variable availability, coherence, and ease of the gender-neutral strategies. when gender is made ambiguous, all else being equal, the default masculine response measures the increase in the proportion of masculine-form translations (∆@xmath6, and the gender-neutral response measures the increase in the proportion of gender-neutral translations (∆@xmath7. in figure 1 illustrating the move from determined to ambiguous gender in the source sentences. note that, depending on the context, either high baseline gender neutrality or high gender-neutral response (or some combination) could be defined as the ideal translation behaviour. foot_1 6 observations ## discussion overall, the lack of alternative morphemes for unknown gender is disappointing; we observe alternative morphology (n 5 ) in only 4/21 systems (dubformer, claude-3.5, gpt4, and commandr+), and usually at very low rates (less than 2% of genderambiguous inputs), with the exception of claude-3.5 in the @xmath8. while these llm-based foot_3 investigating further, we observe that popular solutions for casual mt users have a tendency to push binary gender solutions -if gender ambiguity is acknowledged at all. referring to the examples shown in figure 3 , google translate tends to offer default masculine translations for lower resource languages (icelandic), and masculine/feminine variants for higher resource languages (spanish), but never a gender-neutral variant of the gendered adjectives. both claude-3.7 and chatgpt provide masculine default translations given a simple translate instruction. however, chain-of-thought prompting can successfully be used to force the llms to consider the gender of the adjective's referent. binary gender alternatives are produced when llms are instructed to consider gender, and gender-neutral strategies are used only when explicitly reminded that gender is not binary. when gender neutrality is requested, the specific strategies can vary. the examples show claude-3.7 using the neuter case, and chatgpt using an alternative part of speech (e.g., switching "muscular" for "having muscles"). it does appear that prompting can be used to steer the llms towards a preferred gender-neutral strategy (e.g., asking chatgpt to suggest gender-neutral synonyms for "muscular" produces "fuerte (strong)"). large language models in combination with prompting strategies may be a promising direction for gender-neutral machine translation, but some limitations remain. although llms have the power to provide these nuanced explanations with multiple translation options, they often fail to give usable translation options given the context. that is, providing an explanation with feminine and masculine options may be a partially correct response, but it is neither complete (lacking nonbinary options), nor useful given the information in the source (lacking gender neutrality given gender ambiguity). additionally, the explanation provided by the llm must be distilled to a singular output translation if used at scale. ultimately, llm-based methods should be assessed based on a single output translation based on the (limited) information provided by the source input. given that machine translation systems can be steered toward a particular strategy, it becomes important to consider what the ideal gender-neutral translation should be. the availability of gender neutral adjectives, or other parts of speech, naturally varies by target language. forcing gender neutrality can decrease translation quality, both in terms of coherence in the target language and faithfulness to the source input. should gender neutrality in translation only be enforced in response to gender ambiguity (high gender neutral response), or should it be enforced for all inputs (high baseline gender neutrality)? a high gender neutral response indicates less overall intervention, but may introduce a disparity in translation quality for nonbinary genders. finally, it should be noted that the proposed metrics capture a certain aspect of gender-inclusive machine translation, but do not cover all aspects. for example, gender stereotypes can negatively affect translations while maintaining gender neutrality, and this effect is not captured by either the baseline gender neutrality or by the gender neutral response. we observe mismatched adjective translations depending on binary gender stereotypes (e.g. "thick" has divergent translation as either "dumb" (m) or "fat" (f), likewise "modest"→ "humble"(m)/"covered"(f), etc.). if these divergent translations are each gender-neutral in the target language, the stereotype effect will be obscured. therefore, gender neutrality should be considered as just one branch of ideal gender-inclusive machine translation. ## conclusion gender-neutral translation is a path toward genderinclusive machine translation that helps prevent misgendering and representational biases. however, languages with grammatical gender pose significant technical challenges. a prerequisite for mt systems is the ability to recognize when gender neutrality should be applied. in a large-scale analysis of 21 mt systems, we evaluated their sensitivity to gender ambiguity in the source text. while a masculine default response is often observed, the gender-neutral response is lacking. that is, mt systems struggle to switch to gender-neutral strategies, despite the availability of such strategies in the target languages (e.g., grammatical gender neuter cases, abundant gender-neutral in-vocabulary al-ternatives, and an observed capacity to use alternative morphology). our findings highlight the need for further research in this challenging subtask of gender-inclusive translation. perhaps encouragingly, a small handful of systems did demonstrate a non-zero gender-neutral response, though triggered only by active gender ambiguity. future work may examine the properties of these systems that enabled such a response, in order to strengthen the effect in other contexts. additionally, future work might examine the utility of the templated inputs used here, differing only in gender ambiguity, as a training set to improve gender-neutral translation in more complex scenarios, such as those found in the gente benchmark.