Toxicity in Multilingual Machine Translation at Scale (2023)

Abstract: Machine Translation systems can produce different types of errors, some of which are characterized as critical or catastrophic due to the specific negative impact that they can have on users. In this paper we focus on one type of critical error: added toxicity. We evaluate and analyze added toxicity when translating a large evaluation dataset (HOLISTICBIAS, over 472k sentences, covering 13 demographic axes) from English into 164 languages. An automatic toxicity evaluation shows that added toxicity across languages varies from 0% to 5%. The output languages with the most added toxicity tend to be low-resource ones, and the demographic axes with the most added toxicity include sexual orientation, gender and sex, and ability. We also perform human evaluation on a subset of 8 translation directions, confirming the prevalence of true added toxicity. We use a measurement of the amount of source contribution to the translation, where a low source contribution implies hallucination, to interpret what causes toxicity. Making use of the input attributions allows us to explain toxicity, because the source contributions significantly correlate with toxicity for 84% of languages studied. Given our findings, our recommendations to reduce added toxicity are to curate training data to avoid mistranslations, mitigate hallucination and check unstable translations.

## introduction machine translation (mt) systems are typically evaluated in terms of translation quality either by automatic or human measures. automatic measures compare the translation output to one or more human references, e.g., @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite , or pretrained embeddings, e.g., @xcite ; @xcite . human measures use annotators to rank translation outputs, e.g., @xcite ; @xcite . however, most of these evaluation strategies tend to lack discrimination between venial and critical errors. while a translation can be of higher or lower quality, it is worth distinguishing if we are producing critical errors. the critical error detection task aims at predicting sentence-level binary scores indicating whether or not a translation contains a critical error (not limited to toxicity) @xcite , and sharou and @xcite provide a taxonomy to classify critical errors. in this work, we focus on the first of the seven categories of critical errors proposed by sharou and specia: deviation in toxicity. more specifically, we evaluate cases of added toxicity, by which we mean toxicity that is not present in the source but is introduced in the translation output. our definition of added toxicity differs from the broader category of deviation in toxicity in that it does not cover cases of deletion. nllb @xcite evaluates potential added toxicity on machine translations of the flores-200 benchmark dataset using wordlist-based detectors. such detectors are known for their limitations in over-detecting terms that are toxic only in specific contexts. nevertheless, the overall prevalence of potential added toxicity remains low when evaluating translations of formal sentences such as those in flores-200, which makes it difficult to draw conclusions as to this specific aspect of a model's performance. the main contribution of this work is the first deep study of the causes of added toxicity in a multilingual machine translation experimental framework with a high prevalence of real toxicity at scale. for this purpose, we combine the previously defined toxicity detection methodology @xcite , the controlled evaluation dataset holisticbias @xcite , and the alti+ interpretability method @xcite . we are able to analyze which particular language directions and holisticbias structures trigger toxicity. moreover, we perform a human evaluation of the toxicity detection methodology for a subset of eight out-of-english translation directions, and we find that the false positive rates are below 1% in five translation directions. false negatives are below 3% in all translation directions. finally, we demonstrate an interaction between the source contribution, the robustness of translations, and toxicity. we use alti+ to observe that 45.6% of the toxic translations have a high source contribution, which hints that much of these toxic translations may be caused by mistranslations, and that the rest may be correlated with hallucination @xcite . this suggests that hallucination may add toxicity. we use gini impurity @xcite , a common splitting criterion in decision trees, to measure the relative amount of diversity (i.e. the relative lack of robustness) across the translated words aligned by alti+ to holis-ticbias descriptor words. a combination of a low amount of source contribution and a high gini impurity across translations corresponds to a rate of toxicity roughly twice as high as the baseline rate. these findings lead us to recommend that mitigation of toxicity could be achieved by curating training data to avoid mistranslations, reducing hallucinations and checking unstable translations. ## definitions and background definitions sharou and specia (2022) define deviation in toxicity as "instances where the translation may incite hate, violence, profanity or abuse against an individual or a group (a religion, race, gender, etc.) due to incorrect translations". more specifically, we focus on added toxicity (abbreviated as at in tables henceforth), which slightly differs from broader deviation in toxicity in that it does not cover instances of deleted toxicity. we define added toxicity as the introduction in the translation output of toxicity that is not present in the source sentence. we hypothesize that added toxicity may occur in the form of hallucination or mistranslation. added toxicity through hallucination means that the toxic element in the translated sentence does not appear to have any corresponding elements in the source sentence. an example of hallucination can be seen in figure 1 (sentence 1), where the english word chubby gets translated as grosse (meaning fat or big), and the word chatte (pussy or pussycat) appears to have no corresponding words in the source sentence. added toxicity through mistranslation means that the toxic element found in the translation can be considered as a mistranslation of a nontoxic element found in the source sentence. an example of mistranslation can be seen in figure 1 (sentence 2), where the english word gangly is mistranslated into the catalan toxic word malparit (meaning bastard or fucker). when it comes to the level of added toxicity in translation directions, we define high-, mid-, and low-toxicity translation directions as the ones that have levels of added toxicity above 0.5%, between 0.1% and 0.5%, and below 0.1%, respectively. these percentages are computed following the approach in section 4. we differentiate between high-and low-resource languages following nllb @xcite . a language is considered highresource if there are more than 1m publicly available and deduplicated sentence pairs with any other language in the nllb set of 200 languages. toxicity detection methodology nllb team et al. @xcite propose a toxicity detection method based on wordlists for 200 languages. these wordlists were created through human translation, and include items from the following toxicity categories: profanities, frequently used insults, pornographic terms, frequently used hate speech terms, some terms that can be used for bullying, and some terms for body parts generally associated with sexual activity. among their different detection methods, the authors label a sentence as toxic if it contains at least one entry from the corresponding language's toxicity word list. an entry is considered to be present in a sentence if it is either surrounded by spaces, separators (such as punctuation marks), or sentence boundaries, and thus this method would not detect words such as bass or assistant when looking for the toxic entry ass. one advantage of this type of classifier is transparency, which diminishes the possibility of covering up biases @xcite . alternate methods, such as classifiers, foot_0 are available for english and a few other languages but cannot be used in massively multilingual environments. holisticbias holisticbias consists of over 472k english sentences (e.g., "i am a disabled parent.") used in the context of a twoperson conversation. sentences are typically cre-hallucination sentence 1 __eng_latn__ i love being chubby. </@xmath0. ## proposed experimental methodology we combine the toxicity detection methodology, holisticbias, and the alti+ method to study added toxicity in multilingual machine translation at scale. 2 we demonstrate that holisticbias is a challenging demographic dataset that triggers added toxicity in machine translation (section 4). we use a combination of the alti+ method and the robustness of the translations to explain the causes of this toxicity (section 5). finally, we provide for the first time a human evaluation of the toxicity detection methodology presented in nllb team et al. (2022) (section 6). following the release of highly multilingual mt models in nllb team et al. (2022), we are using the 3.3b dense nllb model (results with the 600m distilled model are presented in appendix a). 3 we translated the holisticbias dataset, which contains 472,991 english sentences, into 164 of these 200 languages ( in this section, we provide a coarse and fine-grained analysis of added toxicity in the experimental setting defined in previous section. coarse-grained analysis we use toxicity detectors to quantify toxicity per language, axis, descriptors, noun and template at the sentence level. ## phenomena causing toxicity we explore the information from measuring the source contribution to translations, as well as the robustness in translations, in relation to toxicity. ## human evaluation of the toxicity detection methodology toxicity lists help detect strings that are always toxic regardless of context (e.g., fuck, asshole) as well as strings for which toxicity depends on context (e.g., tits, prick). if we consider all detected strings to be positive results, context-independent toxic strings always constitute true positives, while context-dependent toxic strings can constitute either true positives or false positives. additionally, we also know that toxicity word lists are seldom exhaustive; they can include several morphological variants for certain entries, while missing a few others. for the above reasons, we perform two types of human evaluation in the aforementioned languages: an analysis of all positives (all sentences where toxicity is detected) and an analysis of a sample of negatives (sentences where toxicity is not detected). language shown is in appendix f. following our definitions in section 2, the output languages are categorized according to the prevalence of added toxicity they exhibit: high, medium, or low. we perform a manual evaluation for several languages in each category. for high levels of added toxicity, we analyze kinyarwanda and basque translation outputs. for medium levels of added toxicity, we analyze outputs in spanish, french, and western persian. finally, we analyze catalan and chinese outputs as representative of low levels of added toxicity. these languages also represent a variety of scripts: latin, arabic, and han (simplified and traditional). ## conclusions this paper provides added toxicity detection and analysis in a highly multilingual environment (164 languages). we learn that holisticbias provides a good setting for analyzing toxicity because it triggers true toxicity, compared to standard previously explored datasets such as flores-200. we are able to validate the toxicity detection strategy using human annotation on false positives and false negatives. additionally, we find insightful conclusions regarding the relationship between toxicity and demographic represented in holisticbias, such as that the demographic axes represented in holisticbias with the most added toxicity include sexual orientation, gender/sex, and ability. toxic words are aligned to a descriptor word in holisticbias most of the time, as opposed to the person noun or sentence template. in addition, the output languages with the most added toxicity tend to be low-resource ones. in the future, we want to explore if the amount of toxicity in the training data may appreciably correlate with added toxicity. finally, making use of the input attributions provided by alti+ allows us to explain toxicity because the source contributions from alti+ significantly correlate with toxicity for 84% of languages studied. we observe that 45.6% of added toxicity has a high source contribution. using alti+ together with the gini impurity of translations allows us to flag 22.3% of toxic translations. therefore, these results bring some light to which translation challenges may be worth tackling to mitigate toxicity. the first recommendation is curating training data to avoid mistranslations that add toxicity. 9 this could potentially mitigate the toxicity created with high source contribution. the second recommendation is mit-igating hallucinations, which may reduce toxicity in cases where we have a low source contribution. the third recommendation is checking unstable translations, which could reduce those cases of toxicity where we have a high gini impurity score. code and data are on github. 10 .