What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study (2024)

Abstract: Gender bias in machine translation (MT) is recognized as an issue that can harm people and society. And yet, advancements in the field rarely involve people, the final MT users, or inform how they might be impacted by biased technologies. Current evaluations are often restricted to automatic methods, which offer an opaque estimate of what the downstream impact of gender disparities might be. We conduct an extensive human-centered study to examine if and to what extent bias in MT brings harms with tangible costs, such as quality of service gaps across women and men. To this aim, we collect behavioral data from ~90 participants, who post-edited MT outputs to ensure correct gender translation. Across multiple datasets, languages, and types of users, our study shows that feminine post-editing demands significantly more technical and temporal effort, also corresponding to higher financial costs. Existing bias measurements, however, fail to reflect the found disparities. Our findings advocate for human-centered approaches that can inform the societal impact of bias.

## introduction natural language processing (nlp) has evolved from an academic specialty to countless commercial applications that can both benefit and negatively affect people's lives. with the widespread use of these technologies, researching the ethical and social impact of nlp has become increasingly crucial @xcite , with gender fairness being a major concern @xcite . in machine translation (mt) gender bias has received significant attention, also in the public domain @xcite . numerous studies have shown that mt perpetuates harmful stereotypes @xcite and is skewed towards masculine forms that under-represent women @xcite . figure 1 : harms as assessed in our study design. we task participants with the post-editing of an mt output into both feminine and masculine gender. we collect behavioural data (i.e. time and technical effort) and assess higher workload and economic costs associated with feminine translations. as emphasized by @xcite -if we regard mt as a resource in its own right -such representational disparities might directly imply allocative harms, i.e. differential access to material benefits that make a social group or individual worse-off @xcite . for instance, a woman using an mt system to translate her biography (i.e. the first sentence in english in figure 1 ) into italian would need more effort (i.e. represented by insertions -in green, and substitutions -in red and green -in figure 1 ) to revise incorrect masculine references, thus experiencing a disparity in the quality of the service. despite acknowledging the potential harm to individuals, research on gender bias in mt primarily focuses on in-lab automatic evaluations. such assessments, however, are only assumed to reflect a real-world downstream effect, without verifying if and to what extent biased models might concretely impact users interacting with a system. to address this gap, we examine the effect of gender bias in mt with a human-centered perspec-tive. specifically, we ask: does gender bias in mt imply tangible service disparities across men and women? and if so, can we meaningfully quantify them via more human-centered measures? to take stock of the current research landscape, we review the involvement of human subjects in prior literature on gender and mt. motivated by the outcome, we conduct extensive experiments across multiple datasets, languages, and users. in a controlled setup, 88 participants post-edited mt outputs to ensure either feminine or masculine gender translation. 1 in the process, we track behavioral data -i.e. time to edit and number of edits -to compare effort across genders. based on this, we estimate the associated cost for post-editing into each gender if the work were assigned to a third-party translator. our main findings are: to sum up, our study marks a step towards understanding the implications of gender bias in mt. while harms have so far been implied, or inferred from automatic scores as a proxy for downstream impact, here we empirically show that gender bias can bring unfair service disparities. what's more, we quantify bias with measures that are more meaningful for potentially impacted individuals: workload and economic costs. behvaioural data and post-edits are made available at https://huggingface.co/datasets/fb k-mt/gender-bias-pe . acl anthology search for a systematic review of prior work, we query the acl anthology. 4 as keywords, we specify our application of interest -e.g. "mt" and "translation" -combined with "gender" or "bias". for a more channelled query focusing on people involved in bias assessment, we also add other human-engagement dedicated keywords (e.g. "user", "survey"). as of april 2024, our search returned 251 articles, 96 of which also matched the human-engagement keywords. upon manual inspection, we retained 105 in-scope manuscripts, 5 and discarded unrelated papers focusing on other definitions of the keywords (e.g. "inductive bias"). the in-scope papers were finally reviewed by focusing on the presence, or lack thereof, of human involvement. for more details on our search and selection, see appendix a. review we report the results of our review in figure 2 . as the image shows, we attest to a steady increase in publications related to gender in mt, in particular from 2020 onwards. in line with our expectations and the general trend in nlp, however, 2016 2017 2018 2019 2020 2021 2022 2023 we attest a severe lack of human engagement. in fact, only 24 works rely on humans to measure bias, though in a different capacity, which we distinguish into three conceptual categories. in 18 papers, we find that people -often expert linguists (e.g. vanmassenhove et al. (2021a); soler uguet et al. (2023)) -are involved in manual eval-uation. this serves to either ensure correlation with bias metrics (e.g. kocmi et al. ( 2020 )) or to gain qualitative insights that defy automatic approaches @xcite . while indeed valuable, such analyses are a support for structured, often annotation-based model-centric evaluations -i.e. that inform and quantify models' behaviour. differently, the 5 papers in the survey category focus on the feedback and experiences of potentially impacted groups of users (e.g. @xcite ; daems and hackenbuchner ( 2022 )). for instance, to grasp user preference in how models should handle the translation of novel, non-binary pronouns from english -e.g. ze, xe (lauscher et al., 2023) 6 , or to understand the potential tradeoff between overall quality and inclusivity goals @xcite . finally, the study by @xcite recounts a participatory action research, where a community-led approach with different stakeholders informs the state and potential direction for gender fair mt. overall, despite this recent trend towards surveys or participatory methods, humans are rarely involved to estimate gender bias in mt. moreover, if involved, people mostly serve in the capacity of evaluators, supporting model-centric assessments rather than being considered as potentially impacted users. our finding stands in contrast with a qualitative survey by @xcite , which found mt as an application with a high risk for downstream harms in the context of gender bias. further motivated by such evidence, we carry out a quantitative, empirical study -to the best of our knowledge, the very first of its kind -focusing on human-centered assessments. in particular, we examine whether gender bias in machine translation leads to disparities in the quality of service offered to women and men, by considering different datasets, languages, and users ( §3.1). ## experimental setup we simulate the conditions in which an end user needs the translation of a text referring to themas described in §1 and exemplified in figure 1 . to strike a balance between controlled conditions for reliable findings while keeping a realistic scenario, the study is realized as a post-editing task (pe), where participants are asked to also ensure that human references are rendered as either feminine or masculine. the same output sentences are edited twice (once per gender), thus allowing to isolate any difference in effort as a gender-related factor. note that our experiments are based on sentences that always require to translate gender and enable focused analyses. as we further discuss in 7, we thus mimic scenarios that often require to manage gender mentions to human referents, as in the case of biographies, cvs, and administrative texts. ## discussion we found strong evidence for the human-centered impact of bias in mt, with a quality of service disparity that can disproportionately affect women. such allocative harm is evident in the extra time and energy required for feminine gender translation. note that our results are likely conservative, involving experienced users with high language proficiency. indeed, in less controlled conditions, or among individuals with lower proficiency in either target or source language, such a negative impact would likely be even greater. misgendered references may go unnoticed, propagating errors in texts and communications, or necessitating the use of external resources such as dictionaries to be fixed. due to experimental constraints ( §3.1), such a scenario remains open to future analyses. to better frame the implications of our findings, we conclude with two critical reflections. first, individuals might rely on third-party language services to translate their text, thus raising the question: can gender bias imply a differential in economic cost? second, while informative assessments that center users are crucial to guide the field forward, are current automatic evaluations able to capture such human-centered disparities? someone has to pay for the cost of gender bias. we explore the economic costs of f and m postediting considering two stakeholders: i) a final user, who buys the pe text from ii) a third-party translator. as a case study, we analyze the three en-it datasets edited by professionals ( §4.1) -using averaged hter and source words shown in hter rate: with this method, prices are adjusted based on the actual technical effort required to post-edit, with lower edit rates leading to lower costs, and vice versa. following existing price schemes (localization, 2022), foot_13 hter below 10 is paid at 35% of the word rate (i.e. c0.0315 per word), whereas hter between 10-20 is paid at 40% (i.e. c0.036 per word). hence, and as shown in word rate: this pricing is based on source text length, where the cost per word is decided a priori. for pe tasks, the word-rates vary depending on the content or the language @xcite . 24 for en-it data from a general domain such as ours, a 35% word rate could be paid. given that -to the best of our knowledge -this type of pricing does not consider gendered content, the same word-rate would be indiscriminately applied to both feminine and masculine pe. thus, as shown in to sum up, this analysis shows that gender bias has an economic cost which can unfairly fall onto either one of the two pe stakeholders. besides financial implications, unfair compensation could also invite less edits than necessary, thus compromising the quality of feminine pe. analysing such potential quality-oriented implications is a crucial aspect for future research. automatic bias measurements do not reliably correlate with human-centered measures. methods to quantify bias are key to much research that seeks to monitor the creation of equitable technologies @xcite . in this context, growing evidence underscored how intrinsic metricsfocusing on models' representations-might not be a reliable bias indicator in downstream, real-world tasks, as assessed with extrinsic ones-focusing on models' output @xcite @xcite . arguably, however, even extrinsic measures are model-centric ( §2), and only assumed to reflect more reliably the downstream harms to individuals. we verify this assumption by comparing our human-centered measures of differential effort with the automatic evaluations associated with mt-geneval and must-she ( §3.1). as in the original papers, we use the set of contrastive f/m target references foot_16 to compute gender-related performance differences with bleu foot_17 @xcite , i.e. bleu f -bleu m . scatter plots of the automatic (i.e., bleu score) and human-centric metrics (i.e., hter and te) differences, in absolute values, are reported in figure 5 . we provide aggregate results for all languages, datasets and users. 28 looking at our results, we notice a pearson-r of -0.19 for ∆ abs hter and ∆ abs bleu (figure 5a ), and -0.18 for ∆ abs secs_per_word and ∆ abs bleu (figure 5b ). the negative correlation is expected since, while for bleu the higher the better, the opposite is true for both hter and te. still, the results clearly indicate that both temporal and technical efforts are in weak correlation @xcite with automatic scores. on the one hand, it is known that time measures may not always have a linear relationship with textual differences measured by automatic metrics @xcite , e.g. even small edits can require a high cognitive load and more time. on the other hand, given that both bleu and hter capture surface modifications and quantity of edits, their weak correlations are particularly noteworthy. 29 a moderate correlation (person-r 0.54) is found only between the human-centered measures hter and te. as observed in figure 5c , the higher the number of edits, the more time required. overall, our results suggest that existing modelcentric measures of gender bias in mt might not reliably reflect the downstream harms to users. while the contrastive evaluation approaches explored here have been used to reveal gender gaps (bentivogli et al., 2020; currey et al., 2022), they do not correlate with or accurately reflect the magnitude of disparities found through more concrete, humancentered measures. 30 to ensure that advancements in the field prioritize impacted individuals, future research should explore both the metrics and the data used to compute them @xcite . this includes investigating how automatic metrics relate to human-centered measures and how they can be translated into more transparent, userrelevant evaluations @xcite . ## conclusion from cars' safety measures more effective for men, ( @xcite , to virtual reality headsets that are too big to wear @xcite , evidence of social and technological advances being less functional for women, or even harmful, abounds (criado-perez, 2019). while it is increasingly acknowledged that also language technologies can contribute to broader patterns of gender bias, still little is known about their tangible impact on people. our study represents a novel effort to empirically examine the implications of gender bias in mt with a human-centered perspective. previous research has often inferred the downstream impact of bias based on automatic, model-centric scores. in contrast, we provide concrete empirical evidence showing that gender bias in mt leads to tangible service disparities, which can disproportionately affect women. also, we quantify these disparities using measures that are more meaningful to impacted individuals, such as workload and economic costs.our study advocates for an understanding of bias and its impact that centers on the actual users of this technology to guide the field. to this aim, we make our collected data and metadata publicly available for future studies on the topic.