Generating Gender Alternatives in Machine Translation (2024)

Abstract: Machine translation (MT) systems often translate terms with ambiguous gender (e.g., English term “the nurse”) into the gendered form that is most prevalent in the systems’ training data (e.g., “enfermera”, the Spanish term for a female nurse). This often reflects and perpetuates harmful stereotypes present in society. With MT user interfaces in mind that allow for resolving gender ambiguity in a frictionless manner, we study the problem of generating all grammatically correct gendered translation alternatives. We open source train and test datasets for five language pairs and establish benchmarks for this task. Our key technical contribution is a novel semi-supervised solution for generating alternatives that integrates seamlessly with standard MT models and maintains high performance without requiring additional components or increasing inference overhead.

## introduction and related work gender 1 biases present in train data are known to bleed into natural language processing (nlp) systems, resulting in dissemination and potential amplification of those biases @xcite . such biases are often also the root cause of errors. a machine translation (mt) system might, for example, translate doctor to the spanish term médico (masculine) instead of médica (feminine), given the input "the doctor asked the nurse to help her in the procedure" @xcite . to avoid prescribing wrong gender assignment, mt systems need to disambiguate gender through context. when the correct gender cannot be determined through context, providing multiple translation alternatives that cover all valid gender choices is a reasonable approach. 1 "gender" in this work refers to binary grammatical gender, and not social gender (male, female, nonbinary). please refer to §limitations for a detailed discussion. numerous prior works have focused on producing correctly gendered translations given contextual gender "hints", such as "to help her" in the example above @xcite @xcite @xcite @xcite . in contrast, the problem of generating all valid and grammatically correct gendered translations has seen far less attention @xcite @xcite . consider the example: "the secretary was angry with the boss." the gender of both secretary and boss remain ambiguous in the absence of additional context: both entities can take either gender. however, and to the best of our knowledge, all existing approaches @xcite @xcite for producing different gendered translations operate on "sentence-level", instead of on "entity-level": they only allow two sentence-level alternatives to surface, in which both secretary and boss are either masculine or feminine: in this work, we introduce a novel approach that operates on entity-level, i.e., it generates four alternatives corresponding to all grammatically valid combinations of gender choices for both entities: when integrated with a proper user interface, our approach provides users with the freedom to choose gender for each entity. we posit that any such system should meet the following practical quality criteria, making the problem challenging: this paper presents several key contributions towards studying the task of generating entity-level alternatives, meeting the above quality criteria: ## entity-level gender alternatives our key insight for efficiently generating entitylevel gender alternatives is to reduce the problem to generating a single translation with embedded gender structures and their gender alignments. consider our previous example: "the secretary was angry with the boss." we want to generate the following entity-level alternatives: since we constraint the alternatives to only differ in gender inflections, we can instead produce a single translation with gender-sensitive phrases grouped together as gender structures, shown in : all alternatives can be derived from this single translation by choosing either the masculine or feminine form in each gender structure. however, doing this naively can give us invalid alternatives that break gender agreement, for example: el secretario estaba enojada con el jefe el secretario la secretaria and enojado enojada correspond to the same entity secretary and cannot have different gender choices. by having gender alignments between each gender structure in the translation and its corresponding gender-ambiguous entity in the source, we can deduce which gender structures are linked together and need to be consistent with each other. let @xmath0. . . x n be the source sentence containing n tokens and let g a ⊆ {1 . . . n} represent the set of indices of gender-ambiguous entities in x. we aim to produce a translation y s : ## datasets to build and evaluate systems producing alternatives, we prepare train and test sets containing gender structures and gender alignment annotations. ## training mt models to generate gender structures and alignments we first present how to train mt models that produce gender structures and alignments, assuming parallel data enriched with gender structures and alignments (for example, g-trans) is available. we then describe a novel data augmentation pipeline that can enrich any regular parallel corpora with gender structures and alignments. given a source sentence @xmath1. . . x n }, translation y s containing gender structures, and gender alignments a, we want to train the mt model to generate y s , a|x. let's assume that y s contains k gender structures and @xmath2. . . a k } where a i represents the source token aligned to the i th gender structure. we serialize each gender structure in y s into a sequence of tokens as follows: m @xmath3, mid, and end are special tokens. the model is then trained to produce gender structures in the form of this sequence. @xcite introduced a technique to train mt models to jointly generate translations and word-alignments. we use their approach to learn generation of gender alignments. let m 1 . . . m k denote the positions of the mid tokens of the gender structures. a specific cross-attention head is chosen and supervised to learn gender alignments. let n and m denote the lengths of the source and the serialized target respectively and let p m×n denote the attention probability distribution computed by the selected head. we train the model with regular cross entropy and an additional alignment loss: this model can generate gender structures and alignments without any additional inference overhead. then, using the procedure described in section 2, all entity-level alternatives can be easily derived from the model outputs. ## data augmentation pipeline g-trans dataset provides supervised data to train mt models in the above manner. however, this dataset is small (50k examples per language pair) and has a restrictive domain, limiting the quality of the trained models. we propose a data augmentation pipeline that can take any regular parallel corpora (containing high quality but potentially biased translations) and augment the translations with gender structures and alignments whenever there are ambiguities in the source. to its corresponding ambiguous entity in ga a ← computegenderalignments(x, ys) output: x, ys, a algorithm 1 gives an overview of the main components of the pipeline, which we describe in detail in the following subsections. it consists of first detecting gender-ambiguous entities in the source sentence ( §5.1), followed by transforming the reference translation into all-masculine/all-feminine translations ( §5.2, §5.3), condensing those into single translation with gender structures, and finally aligning the gender structures ( §5.4). ## evaluation metrics we evaluate our systems' performance using the following metrics: higher δ-bleu indicates more bias. mathematical definitions of alternatives and structure metrics can be found in appendix k. ## experiments and results we will first describe the experimental details and results of our data augmentation pipeline in 7.1 and 7.2. we then present the training details of the mt model generating alternatives end-to-end and how it benefits from data augmentation in 7.3 and 7.4. ## conclusion and future work in this work, we study the task of generating entitylevel alternatives when translating a sentence with gender ambiguities into a language with grammatical gender. we open source first train datasets, encouraging future research towards this task, and develop a data augmentation pipeline that leverages pre-trained mt models and llms to generate even larger train sets. finally, we demonstrate that this data can be used effectively to train deploymentfriendly mt models that generate alternatives without any additional inference cost or model components. our models and pipeline can enable new translation uis that support fine-grained gender control and can also find applications in aiding human translators to automatically point out ambiguities and recommend alternative translations. future work includes exploring other genderless source languages apart from english (e.g., chinese, korean, and japanese) and associated challenges, as well as extending the approach to non-binary and gender-neutral forms @xcite @xcite .