Automatic detection of (potential) factors in the source text leading to gender bias in machine translation (2024)

Abstract: This research project aims to develop a comprehensive methodology to help make machine translation (MT) systems more gender-inclusive for society. The goal is the creation of a detection system, a machine learning (ML) model trained on manual annotations, that can automatically analyse source data and detect and highlight words and phrases that influence the gender bias inflection in target translations.The main research outputs will be (1) a manually annotated dataset, (2) a taxonomy, and (3) a fine-tuned model.

## introduction with an increasing use of and interest in the developments of machine translation (mt) and a growing demand for gender inclusiveness in society, research on gender bias in mt is increasing @xcite ). an mt system is considered to be gender-biased if it "systematically and unfairly discriminates against certain individuals or groups in favor of others" @xcite , perpetuating "inaccurate and potentially discriminatory stereotypes" in society @xcite . word-level studies show that word embeddings used in mt training are highly gender-inflected, where word embeddings of different parts-ofspeech (pos) strongly cluster based on their gender in varying domains (e.g., in sports, kitchen, big tech, sexual profanities) @xcite . however, research on word embeddings is limited to word-level analysis and has not yet been extended to the full context of natural language source sentences and resulting systematic gender associations in machine translations. ## project description in this research project, we apply a novel approach to analyse aligned linguistic and morphological features with a focus on gender in source and target texts and apply machine learning methodologies. english is chosen as the source language, where role names are generally not marked with a gender (e.g. teacher) and the target translations are analysed in german and spanish, grammatical gender languages @xcite , where gender is clearly marked (e.g. lehrer/lehrerin). the aim is to automatically identify and classify "trigger words" in a source context that influence the grammatical gender inflection in the target translation (i.e. whether an mt translates a person as female or male, or instead opts to neutralise or rephrase the word). the project consists of the following main deliverables that differ from previous research: (1) a manually annotated dataset of human gender associations in sentence contexts, (2) a taxonomy based on these annotations, (3) a comparative analysis of human gender associations vs. the mt gender inflections, and (4) a fine-tuned large language model (llm) that highlights trigger words for gender in a source text. ## aligned projects in line with this research project, the phd fellow, first author of this paper, is a co-founder and member of debiasbyus foot_0 and a co-organiser of the two international workshops on gender-inclusive translation technologies. ## conclusion our benchmark could make mt users aware of gender inflections of source texts that are machine translated, technologically support translators in post-editing mt output, direct developers of mt systems to persisting issues of gender bias, and help content creators identify potential triggers in text that may lead to gender-biased translations.