Fine-Tuning vs Prompting Techniques for Gender-Fair Rewriting of Machine Translations (2025)

Abstract: Increasing attention is being dedicated by the NLP community to gender-fair practices, including emerging forms of non-binary language. Given the shift to the prompting paradigm for multiple tasks, direct comparisons between prompted and fine-tuned models in this context are lacking. We aim to fill this gap by comparing prompt engineering and fine-tuning techniques for gender-fair rewriting in Italian. We do so by framing a rewriting task where Italian gender-marked translations from English gender-ambiguous sentences are adapted into a gender-neutral alternative using direct non-binary language. We augment existing datasets with gender-neutral translations and conduct experiments to determine the best architecture and approach to complete such task, by fine-tuning and prompting seq2seq encoder-decoder and autoregressive decoder-only models. We show that smaller seq2seq models can reach good performance when fine-tuned, even with relatively little data; when it comes to prompts, including task demonstrations is crucial, and we find that chat-tuned models reach the best results in a few-shot setting. We achieve promising results, especially in contexts of limited data and resources.

## introduction current practices in many languages involve the use of the masculine gender as a generic form @xcite , a norm -which we refer to as masculine generics (mg)-that may result in the erasure of other gender identities, including both women and non-binary (nb) people foot_0 . nlp models, based on dominant linguistic practices, reproduce this behavior @xcite . the reliance on mg implies the under-representation of people who do not identify as men @xcite , as well as an increased effort and a reduced quality of service for them @xcite ; examples of representational and allocational harms @xcite , respectively. in this paper, we acknowledge the limited availability of nlp resources for nb language and, especially, the lack of a shared nb "grammar" for italian, so we produce original, detailed guidelines for the use of an italian nb language paradigm. they are included in appendix a and are meant to serve as a basis for future works and for further discussions around the topic, with the ultimate aim of fostering the recognition of nb identities in italian language technologies. our guidelines were written by one of the authors of this paper, and they were validated by experienced italian linguists. we also define a rewriting task based on replacing masculine and feminine gender marks with nb endings, inspired by the fair reformulation task described by @xcite . we focus on existing translations of gender-ambiguous english sentences: while the reference translations we collected use masculine or feminine gender marks, our goal is to obtain new translations that preserve the gender neutrality of the source sentences. the spans we aim to replace include examples of gendered language used in an overextended or generic way, as defined by @xcite . to do so, we manually rewrite existing italian translations that use gendered language so that they maintain the gender ambiguity of the corresponding english source sentences. we then use the original translations as inputs and our rewritten translations as labels to expand on recent related works on gender-fair nlp by comparing transfer and in-context learning on both encoder-decoder and decoder-only architectures. while our approach is essentially monolingual, it is meant to be applied not only to italian texts which use gendered language and mg, but also to gender-marked translations provided by machine translation (mt) or human translators (in this case, it can be defined as a post-editing task). sentences obtained this way could potentially be used to make texts inclusive of all gender identities, as well as to train future nlp models on more diverse datasets. we release the data we used in our experiments, the outputs of our models, and the main scripts used to carry out this study. 2 the rest of the paper is distributed as follows. section 2 provides background on gender and language. we discuss our conceptualization of gender bias in section 3. we present related work in section 4. section 5 describes our approach and section 6 discusses our results. section 7 draws conclusions and discusses future work. ## background the relationship between gender and language is especially relevant in the context of translation, due to the need to resolve discrepancies between different gender systems @xcite . we focus on english and italian. english is a representative of notional gender languages, where only a few classes -mostly pronouns -are gender-marked, while nouns, verbs, and adjectives are usually genderambiguous; i.e., they can refer to people of any gender identity. italian is a grammatical gender language, where most words are gender-marked and, usually, all components of a noun or verb phrase have to be inflected according to the same gender. refer to @xcite for an overview of the grammatical gender systems of various languages. mg are also used in the context of translation; for example, referents whose gender is ambiguous in english are often translated as masculine in italian. from the 1970s, feminist movements initiated the debate around the social dynamics underlying the use of mg and gendered language (see @xcite . more recently, the need for alternative solutions has been reiterated by works on cognitive biases resulting from the extended use of masculine words @xcite . interest in nb language increased starting in the 2010s @xcite . nb language includes various sets of linguistic practices aimed at representing nb and gendernon-conforming identities. this is especially challenging in highly inflected languages with a binary 2 https://github.com/tinffoil/dnl_rewriter specific corpus id ## bias statement in the context of this paper, we consider as biased behavior the use of both masculine and feminine gender marks whenever referring to specific individuals whose gender identity is unknown or nb, to groups of people that may include individuals of various gender identities, or to generic referents that do not identify a specific individual. we define all these cases collectively as binary generics, as they all imply taking the gender binary as the general norm. a translation is biased according to our definition if it contains one or more masculine or feminine gender marks when the corresponding source text does not, as defined by @xcite in their desiderata for gender-neutral translation. in such cases, the translation should be rewritten, and that is our goal in this study. we consider binary generics to be harmful as they erase the existence of people whose gender identity does not adhere to the gender binary, and as they imply the risk of misgendering individuals who do not recognize themselves in a binary gender @xcite . earlier approaches to gender-fair nlp include reducing the association of certain words to the masculine or feminine gender in an embedding space @xcite , or making training data more balanced through counterfactual data augmentation (cda; @xcite , mainly by converting gender marks from masculine to feminine and vice versa. while the former was proven to be a superficial solution @xcite , data balanced through some form of cda has been extensively used to create evaluation benchmarks @xcite @xcite or to fine-tune models on downstream tasks or specific datasets @xcite ; costa-jussà and de jorge, 2020). gender-fair post-editing has been proposed as a solution @xcite and it requires less data, since it relies on robust models that can provide high-quality translations, although biased. crucially, such a task can be automated; e.g., @xcite . similarly to @xcite , @xcite obtain training data through a rule-based algorithm, then train a model on a rewriting task; bartl and leavy (2024) use their rewriting system to create a gender-fair dataset and fine-tune large language models (llms) on it. this task can generally be referred to as gender-fair rewriting or reformulation @xcite . recently, the popularization of conversational llms has brought attention on prompting techniques for obtaining gender-fair texts. the novelty of many gender-fair communication strategies and the limited availability of task-specific datasets make prompting a very promising approach in this area. @xcite al. (2024), vanmassenhove (2024), savoldi et al. (2024b), and piergentili et al. (2024) all compare different prompting strategies for gender-fair mt: specifically, the former two aim at obtaining all possible combinations of (binary) gender marks in the translations of genderambiguous source sentences, while the latter two focus on inl and dnl, respectively. finally, @xcite use a similar approach to reduce gender stereotyping in generative models. we adopt a nb perspective and focus on automatic gender-fair post-editing or rewriting in italian. we use this approach to directly compare task-specific fine-tuning and zero-and few-shot prompting for gender-fair nlp. to the best of our knowledge, the only existing work partially comparable to our own is the one by @xcite , who test their (prompted) models on the same test set and using the same metrics. however, their study is fundamentally different in that it focuses on translation rather than on rewriting. ## method we carry out experiments with various models, with the ultimate goal of obtaining a gender-neutral alternative for each translation in our dataset, as exemplified in our guidelines are also similar to @xcite use this paradigm in their dataset. in this section, we describe the data we used (5.1), the models we included (5.2), and the experiments we carried out (5.4). ## results and discussion as mentioned in section 4, the closest work to our own is @xcite . we thus use similar metrics to evaluate our models and will also refer to their results in this section; however, mt metrics are not comparable across the two studies, since they focus on translation rather than on rewriting. although we could not carry out a systematic qualitative evaluation of the models' outputs, we randomly extracted 10 sentences from the predictions of each prompted/fine-tuned model in the final experiments, and we analyzed them to verify where the most common mistakes are concentrated. we find that, apart from hallucinations, our models struggle the most with long dependencies, for example rewriting some noun phrases only partially, and with overgeneration, for example extending the schwa on nouns that do not refer to human beings. ## conclusions and future work in this paper, we discussed the lack of recognition of non-binary identities in language and the implications of this on language technologies, with a focus on italian and on the western european and north american context. to address this problem from a technical point of view, we designed a rewriting task and evaluated models representing different architectures and nlp paradigms, while comparing the results obtained through prompting and fine-tuning methods. as training data, we used previously released evaluation benchmarks where gender-ambiguous english sentences are paired with gender-marked italian translations; we manually added alternative translations using direct non-binary language according to our original guidelines, and we trained our models to rewrite each original translation into our reformulation. we evaluated two seq2seq encoder-decoder models and three causal decoder-only models. we included different versions of these models with varying dimensions based on memory requirements for each experiment, and we conducted a preliminary evaluation and an ablation study to investigate the impact of a variety of parameters on performance. we achieved promising results and suggest some possible directions for future developments. on one hand, fine-tuning benefits all models, and we demonstrate that it can guarantee better results even with smaller models. on the other hand, given the innovative nature of our task, prompting seems to only be effective when examples of such task are included in the prompt, and when the model is able to effectively learn from them and generalize. in our case, chat-tuned models were the only ones to yield satisfactory results in this setting. an important aspect to consider is that we used a rather small dataset, and fine-tuning results would likely improve with more data. despite the notoriously scarce availability of data in this domain, collecting more than this seems feasible, especially as prompting techniques to obtain annotated data from llms likely improve in the future. for example, future works could involve prompting strong models to obtain a basis from which to create more annotated data, and then fine-tuning cheaper models using the resulting, bigger dataset to obtain the final dnl sentences. an example of a similar approach is @xcite , who fine-tune an nmt model to follow instructions using translations generated by causal llms. moreover, annotations could be added to the input data, so as to explicitly identify specific spans holding gender information, both for prompting and for fine-tuning. our results could also be improved by implementing more refined prompt design. for example, breaking the task into simpler, consecutive steps would likely prove beneficial to the rewriter: this could be achieved both with chainof-thought prompts @xcite or thanks to the the improved "reasoning" capabilities of models such as @xcite . another direction that could be investigated in the future is the multilingual generalization of our approach, for example by fully leveraging mt0's multilingual capabilities through crosslingual training. finally, we plan to validate our approach by carrying out a more thorough manual investigation of the models' outputs and implementing human evaluation metrics.