How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translation
Authors: Marco Gaido, Dennis Fucci, Matteo Negri, Luisa Bentivogli (Fondazione Bruno Kessler, Università di Trento, 2023)
Topic: Gender and Inclusive Language Studies

Abstract
When translating from notional gender languages (e.g., English) into grammatical gender languages (e.g., Italian), the generated translation requires explicit gender assignments for various words, including those referring to the speaker. When the source sentence does not convey the speaker’s gender, speech translation (ST) models either rely on the possibly-misleading vocal traits of the speaker or default to the masculine gender, the most frequent in existing training corpora. To avoid such biased and not inclusive behaviors, the gender assignment of speaker-related expressions should be guided by externally-provided metadata about the speaker’s gender. While previous work has shown that the most effective solution is represented by separate, dedicated gender-specific models, the goal of this paper is to achieve the same results by integrating the speaker’s gender metadata into a single “multi-gender” neural ST model, easier to maintain. Our experiments demonstrate that a single multi-gender model outperforms gender-specialized ones when trained from scratch (with gender accuracy gains up to 12.9 for feminine forms), while fine-tuning from existing ST models does not lead to competitive results.

Introduction
Spurred by growing concerns about fairness in language technologies, research on understanding and mitigating gender bias in automatic translation is gaining traction. The bias of automatic systems is extremely evident when it comes to ambiguous sentences or expressions, where there are no explicit cues in the source content about the correct gender assignment of a referent (e.g., en: The doctor arrived – it: Il/La dottore/essa è arrivato/a). In this setting, the state-of-the-art neural models often choose the masculine forms or perpetuate stereotypical assignments, as they reflect the condition statistically more likely based on their (biased) training data. This situation frequently occurs when the source language is genderless or employs notional gender, expressing gender in a limited set of parts of speech, and the target language follows a grammatical gender system, embedding gender distinctions throughout a broad inventory of parts of speech. Focusing on the case in which the source language is English, a notional gender language, and the target language is Italian, a grammatical gender language, a frequent instance of this condition is represented by first-person references, i.e. by the words and expressions referred to the speaker (e.g., en: I am a young researcher – it: Sono una/un giovane ricercatrice/tore). In this case, text-to-text machine translation (MT) models mostly output masculine forms, while direct (or end-to-end) speech-to-text translation (ST) systems partly rely on the biological cue of the speaker’s vocal traits to assign gender. However, direct ST models are still largely biased toward producing masculine forms, and, most importantly, biological aspects are related to the sex rather than to the gender of an individual. Hence, their exploitation is not inclusive of all people, harming several groups such as transgenders. As a solution, proposed to leverage external metadata about the speaker’s gender to control the gender assignment of words referred to the speaker. Specifically, they investigated two approaches: i) the development of two separate gender-specialized models, fine-tuned on gender-specific data as also proposed later in MT, and ii) a single multi-gender model, where the speaker gender is a tag fed to a single model as in multilingual systems. While the second solution would be preferable (as the specialized solution involves the higher cost of maintaining two separate models), the experiments demonstrate that specialized models outperform the multi-gender approach by a large margin in terms of gender accuracy. In light of the above, in this paper we address the following research questions: i) why do specialized models outperform multi-gender ones? ii) Can we build competitive multi-gender systems? Through experiments on English-Italian translation of TED talks, we show that the low accuracy of multi-gender models comes from the initialization with the weights of a gender-unaware ST system and the inability to override the behavior of the base ST model (i.e., the reliance on vocal cues) during the fine-tuning stage. We also try to address this problem with two solutions: i) a contrastive loss that penalizes the extraction of gender cues from speech input, and ii) altering vocal properties of training data to misalign gender cues with gender tags and gender translations. Despite the slight improvements brought by these solutions in gender accuracy and overall translation quality, none of them effectively close the performance gap with the specialized solution. However, training multi-gender models from scratch yields competitive results, outperforming the specialized approach with gender accuracy gains of up to 12.9 points for feminine translations. Therefore, we recommend building multi-gender models from scratch, while building them on top of existing systems remains an open research question.

Conclusion
In this paper, we studied the effect of different training strategies to build multi-gender ST models, i.e., models informed of the gender of the speaker by an explicit gender tag. Focusing on English-Italian translations, we demonstrated that the low accuracy of multi-gender models shown by previous work stems from their initialization with gender-unaware ST system weights and the inability to effectively override the reliance on vocal cues during fine-tuning. On the other hand, training multi-gender models from scratch proved to be an effective solution, outperforming the approach based on the creation of two gender-specialized models. As training from scratch is not always feasible, we also experimented with two methods to enhance the reliance on the gender tag in fine-tuned multi-gender models: penalizing the extraction of gender cues from speech input and altering the vocal properties of the speakers in the training data to avoid the alignment between biological cues and gender tags and translations. While these solutions partially improved gender accuracy and overall translation quality in fine-tuned multi-gender models, they did not close the gap with specialized models. Therefore, further research is needed in this direction.