Toxicity in Multilingual Machine Translation at Scale
Authors: Marta R. Costa-jussà
Eric Michael Smith
Christophe Ropers
Daniel Edward Licht
Jean Maillard
Javier Ferrando
Carlos Escolano
Venue: EMNLP 2023 Findings (2023)
Keywords: Toxicity
Multilingual Machine Translation
Scale
TL;DR: We evaluate and analyze added toxicity when translating a large evaluation dataset (HOLISTICBIAS, over 472k sentences, covering 13 demographic axes) from English into 164 languages.

Abstract: Machine Translation systems can produce different types of errors, some of which are characterized as critical or catastrophic due to the specific negative impact that they can have on users. In this paper we focus on one type of critical error: added toxicity. We evaluate and analyze added toxicity when translating a large evaluation dataset (HOLISTICBIAS, over 472k sentences, covering 13 demographic axes) from English into 164 languages. An automatic toxicity evaluation shows that added toxicity across languages varies from 0% to 5%. The output languages with the most added toxicity tend to be low-resource ones, and the demographic axes with the most added toxicity include sexual orientation, gender and sex, and ability. We
also perform human evaluation on a subset of 8 translation directions, confirming the prevalence
of true added toxicity. We use a measurement of the amount of source contribution to the translation, where a low source contribution implies hallucination, to interpret what causes toxicity. Making use of the input attributions allows us to explain toxicity, because the source contributions significantly correlate with toxicity for 84% of languages studied. Given our findings, our recommendations to reduce added toxicity are to curate training data to avoid mistranslations, mitigate hallucination and check unstable translations.

Toxicity in Multilingual Machine Translation at Scale
Marta R. Costa-jussat, Eric Smith Christophe Roperst, Daniel Lichti, Jean Maillard i
Javier Ferrando" & Carlos Escolano"
+FAIR, Meta
* Universitatt Politècnica de Catalunya
(costajussa, ems, chrisropers, , dlicht, jeerichhttjnnn.ee... jeean)Jemeta com
{javier. wwwrrrfro.rrroneeee f carlos.cc..oo ewc.laao)@upc.ccccooooo edu
Abstract
et al. (2019). Human measures use annotators rank translation outputs, e.g., Licht et (2022); Akhbardeh et al. (2021). However, most of these evaluation strategies tend to lack discrimination between venial and critical errors. While a transla- tion can be of higher or lower quality, it is worth distinguishing if we are producing critical errors. The eritical error detection task aims predicting sentence- binary scores indicating whether or not translation contains a critical error (not lim- ited to toxicity) (Specia et al., 2021), and Sharou and Specia (2022) provide a taxonomy to classify critical errors. In this work, we focus on the first of the seven categories of critical errors proposed by Sharou and Specia: deviation in toxicity. More specifically,
Machine Translation systems can produce dif- ferent types of errors, some of which are char- acterized as critical or catastrophic due to the specific negative impact that they can have users. In this paper we focus on one type of critical error: added toxicity. We evaluate and analyze added toxicity when translating a large evaluation dataset (HOLISTICBIAS, over 472k sentences, covering demographic axes) from English into 164 languages. An automatic toxi- city evaluation shows that added toxicity across languages varies from 0% 5%. The output languages with the most added toxicity tend be low-resource ones, and the demographic axes with the most added toxicity include sex- ual orientation, gender and sex, and ability. We also perform human evaluation on subset of 8 translation directions, confirming the preva- lence of true added toxicity. We use measure- of the amount of source contribution to the translation, where a low source contribution implics hallucination, to interpret what causes toxicity. Making of the input attributions al- lows us to explain toxicity, because the source contributions significantly correlate with toxi- city for 84% of languages studied. Given our findings, our recommendations to reduce added toxicity are curate training data avoid mis- translations, mitigate hallucination and check unstable translations.
evaluate cases of added toxicity, by which we mean toxicity that is present in the source but is introduced in the translation out- put. Our definition of added toxicity differs from the broader category of deviation in toxicity that it does not cover cases deletion. NLLB Team et al. (2022) evaluates potential added toxi- city on machine translations of the LORES-2000 benchmark dataset using wordlist-based detectors. Such detectors known for their limitations in over-detecting terms that toxic only in specific contexts. Nevertheless, the overall prevalence of potential added toxicity remains low when evaluat- ing translations formal sentences such as those in FLORES-2000 which makes it difficult to draw conclusions as to this specific aspect of a model's ss performance.
this paper contains examples of tox- icity that may be offensive or upsetting in nature.
WARNING:
1
Introduction
The main contribution of this work is the first deep study of the causes of added toxicity in mul- tilingual machine translation experimental frame- work with high prevalence of real toxicity at scale. For this purpose, we combine the previously defined toxicity detection methodology (NLLB Team al., 2022), the controlled evaluation dataset HOlisticBIAs (Smith et al., 2022), and the ALTI+ interpretability method (Ferrando et al., 2022a).
Machine Translation (MT) systems are typically evaluated in terms of translation quality either by automatic or human measures. Automatic mea- sures compare the translation output one or more human references, e.g., Papineni al. (2002);
Popović (2015a); Lo (2019); Rei et al. (2020); Sellam et al. (2020); Freitag et al. (2021), or pre- trained embeddings, e.g., Lo (2019); Yankovskaya

---

We are able to analyze which particular language directions and HOLISTICBLAS structures trigger toxicity. Moreover, we perform a human evalu- ation of the toxicity detection methodology for a subset of eight out-of-English translation direc- tions, and we find that the false positive rates are below 1% in five translation directions. False neg- atives are below 3% in all translation directions. Finally, we demonstrate interaction between the source contribution, the robustness of transla- tions, and toxicity. We use ALTT+ to observe that 45.6% of the toxic translations have a high source contribution, which hints that much of these toxic translations may be caused by mistranslations, and that the rest may be correlated with hallucination (Ferrando et al., 2022a). This suggests that hallu- cination may add toxicity. We use Gini impurity (Breiman, 1996), a common splitting criterion in decision trees, to measure the relative amount of diversity (i.e. the relative lack of robustness) across the translated words aligned by ALTI+ to HOLIS- TICBIAS descriptor words. combination of a low amount of source contribution and high Gini impurity across translations corresponds to a rate of toxicity roughly twice as high as the baseline rate. These findings lead to recommend that mitigation of toxicity could be achieved by curat- ing training data to avoid mistranslations, reducing hallucinations and checking unstable translations.
big), and the word chatte (pussy or pussycat) ap- pears have no corresponding words in the souree sentence. Added toxicity through mistranslation means that the toxic element found in the trans- lation can be considered as a mistranslation of a nontoxic element found in the source sentence. An example of mistranslation can be seen in Figure 1 (Sentence 2), where the English word gangly is mistranslated into the Catalan toxic word malparit (meaning bastard or fucker).
When it comes to the level of added toxicity in translation directions, we define high-, mid-, and low-toxicity translation directions as the ones that have levels of added toxicity above 0.5%, be- tween 0.1 % and 0.5%, and below 0.1 respec- tively. These percentages are computed following the approach in section 4. We differentiate between high- and low-resource languages following NLLB Team et al. (2022). A language is considered high- resource if there are more than 1M publicly avail- able and deduplicated sentence pairs with any other language in the NLLB set of 200 languages.
Toxicity detection methodology NLLB Team et (2022) propose toxicity detection method based on wordlists for 200 languages. These wordlists were created through human translation. and include items from the following toxicity cate- gories: profanities, frequently used insults, porno- graphic terms, frequently used hate speech terms, some terms that can be used for bullying, and some terms for body parts generally associated with sex- ual activity. Among their different detection meth- ods, the authors label a sentence as toxic if it con- tains at least one entry from the corresponding lan- guage's 's toxicity word list. An entry is considered to be present in sentence if it is either surrounded by spaces, separators (such punctuation marks), sentence boundaries, and thus this method would not detect words such as bass or assistant when looking for the toxic entry ass. One advantage of this type of classifier is transparency, which dimin- ishes the possibility of covering up biases (Xu et al., 2021). I). Alternte methods, such as classifiers, 1 are available for English and few other languages but cannot be used in massively multilingual environ- ments.
2 Definitions and Background
Definitions Sharou and Specia (2022) define de- viation in toxicity as "instances where the transla- tion may incite hate, violence, profanity or abuse against an individual or a group (a religion, race, gender, etc.) due to incorrect translations". More specifically, we focus on added toxicity (abbrevi- ated as AT in tables henceforth), which slightly differs from broader deviation in toxicity in that it does not cover instances of deleted toxicity. We de- fine added toxicity as the introduction in the trans- lation output of toxicity that is not present in the source sentence.
We hypothesize that added toxicity may occur in the form of hallucination or mistranslation. Added toxicity through hallucination means that the toxic element in the translated sentence does not appear have any corresponding elements in the source sentence. An example of hallucination can be seen in Figure 1 (Sentence I), where the English word chubby gets translated as grosse (meaning far or
HOLISTICBIAS HOLISTICBIAS consists of over 472k English sentences (c.g.. "I am a dis- abled parent.)) used in the context of a two- person conversation. Sentences are typically cre-
For instance, wttp://wperspccivepp.ccoo

---

Hallucination
Sentence
-ng Lain Leig_LLin_____ love being chubby,
</>>
</>>
Fra Latn
J J'adore être une grosse chatte.
Mistranslation
Sentence 2
eng Lain I'm gangly child.




cal Lain
Hola, sóc un nen malparit.
Correct translation
Sentence 3
eng_ Latn Hi! I'm 50-ycar-old woman.
</>
</>>
cat Latn
Hola, sóc una dona de 50 anys.
Figure I: Examples translations in English-to- Frenchh English-to-Spanin or English-to-Catall Sentences show input attributions for bold words in the cases of hallucination (sentence 1); mistranslation (sentence 2); and correct translation (sentence 3). We observe that the hallucination example focuses more in the target context than in the source sentence compared to the other two examples.
ated from combining a sentence template (e.g.. "/ am a (NOUN PHRASEJ."), a noun (e.g., parent), and a descriptor (e.g., disabled) from a list of nearly 600 descriptors across 13 demographic axes such as ability, race/ethnicity, or gender/sex. The descrip- tors can come before the noun ("I am a disabled parent."), "), after the noun ("I am a parent who is hard of hearing.") "), or in place of a separate noun ("I am disabled.") The noun can imply certain gender (girl, boy) or avoid gender references (child, kid). Sentence templates allow for both singu- lar and plural forms of the descriptor/noun phrase ("What do you think about disabled parents?") Other datasets consisting of slotting terms into templates were introduced by Kurita et al. (2019); May et al. (2019); Sheng et al. (2019); Brown et al. (2020); Webster et al. (2020). The advantage of templates is that terms can be swapped in and out to measure different forms of social biases, such stereotypical associations (Tan and Celis, 2019). Other strategies for creating bias datasets include careful handerafting of grammars (Renduchintala al., 2021), collecting prompts from the begin- nings of existing text sentences (Dhamala et al., 2021), and swapping demographic terms in exist- ing text, either heuristically (Ma et al., 2021; Wang et al., 2021; Zhao et al., 2019; Papakipos and Bit- ton, 2022) or using trained neural language models (Qian al., 2022).
examples of these input attributions in Figure 1. In Neural MT, attention weights in the cross-attention module have been used to extract source-target proxy for input attribution scores (Kobayashi et al., 2020; Zenkel al., 2019; Chen al., 2020), even though they are limited to provid- ing layer-wise explanations. Gradient- based meth- ods (Ding et al., 2019) have also been proposed: in this case the gradient of the prediction with respect to the token embeddings is computed, reflecting how sensitive a certain class is to small changes in the input. These methods have been tradition- ally used to obtain input attribution scores of the source sentence, ignoring the influence of the target prefix, which fed into the decoder at each gen- crating step. ALTI+ is the extension of ALTI (Fer- rando et al., 2022b) to the encoder-decoder setting in NMT. ALTI (Aggregation of Layer-wise Token- to-token Interactions) is an interpretability method for encoder-based Transformers. For each layer, it measures the contribution of each token represen- tation to the output of the layer. Then, combines the layer-wise contributions to track the influence of the input tokens to the final layer output. ALTI+ applies the same principles to account for the influ- ence of the target prefix as well. For each decoding time step t, ALTI+ provides a vector of input attri- butions E IR (8) [T], where S and T are the input tokens of the encoder and decoder respectively. We refer the source contribution the prediction as the sum of the attributions of the encoder in- put tokens to the decoding step t, _ ri,e- The source-prediction alignment is computed by taking
alignments
ALTI+ method Input attributions are type of local explanation that assigns score to each of the input tokens, indicating how much each of the tokens contributes to the model prediction. See

---

Following the release of highly multilingual MT models in NLLB Team et al. (2022), we are using the 3.3B dense NLLBB model (results with the 600M distilled model are presented in Appendix A).3 We translated the HOLISTICBIAS dataset, which con- tains 472,991 English sentences, into 164 of these 200 languages (Table 2) in order to evaluate the toxicity of the translations. discarded for one of three reasons. First, for 27 languages, 4 tokenization on non-word characters is not sufficient distinguish words from each an- other. Even using SPM tokenization (Kudo and Richardson, 2018a) on both the sentences and the toxic words list cannot provide a solution to this problem. Second, for seven languages, 5 issues such as UNKs or untranslated English text prevent easy alignment of word splittings with the results of the the input token of the encoder with highest attribu- tion, arg max(( : 1, ...... [S|)]. We exploit ...
ALTI+ method. Third, for two languages, the tox- icity lists are too inaccurate in that they include many entries whose toxicity sensitive to context.
both source contributions and word alignments for
a aie-ggained analysis of toxicity as well as an approach to flag temptative toxic translations. We consider source contribution to be low when it is smaller than threshold of 40%, in which case we consider the target word is much more likely to be the result of model hallucination: this threshold corresponds to a region of particularly high toxicity (section 5).
4 Quantification of added toxicity
In this section, we provide coarse and fine- fine-grained analysis of added toxicity in the experimental set- ting defined in previous section.
Coarse-grained analysis We use toxicity detec- tors to quantify toxicity per language, axis, descrip- tors, noun and template at the sentence level.
3 Proposed Experimental Methodology
By language. Figure 2 shows large variation in toxicity as a function of language and dataset. The HOLISTICBIAS dataset shows generally higher rates of added toxicity than FL.ORS-- removed any language with >5% toxicity because it is the threshold above which we found malformed wordlists. Then, toxicity varies from 0% to 5%. 6 languages have 22% toxicity, all with a Latin script: Luo, Tswana, Yoruba, Southwestern Dinka, Indonesian, and Tok Pisin. According to the defini- tion of high and low resource languages in section 2, all of these languages are low-resource except for Indonesian and Tswana. All but 13 languages have less than 1 1% toxicity. The variation in these percentages may be an effect of the quality of the translation model, or it may reflect issues with rel- ative sensitivity across the toxicity lists in each language. By comparison, no sentences in the orig- inal English HOLISTICBIAS dataset are found contain toxicity. There is no discernible correlation between the rate of added toxicity per language and the fidelity of the translations: its Pearson's 's r with the chrF score (Popović, 2015h) is -0.06 (95% con- fidence interval via bootstrapping: -0.23 to +0.12), Note that since HOLISTICBIAS is only available in English, we compute the quality of translations based FLORES- (NLLB Team et al., 2022).
We combine the toxicity detection methodology. HOLISTICBIAS, and the ALTI+ method to study added toxicity in multilingual machine translation at scale.? 2 We demonstrate that HOLISTICBIAS is challenging demographic dataset that triggers added toxicity in machine translation (section 4). We use a combination of the ALTI+ method and the robustness of the translations to explain the causes of this toxicity (section 5). Finally, we provide for the first time a human evaluation of the toxicity detection methodology presented in NLLB Team et al. (2022) (section 6).
We have
36 languages were
By axis Figure 2 shows the distribution of toxic translations per category and how they vary per language. Differently, when looking into the cate- gories that have a higher concentration of toxicity among the 13 axes of HOLISTICBIAS, the high- est rates of toxicity are found in translations of terms in the nonce (non-sense) axis (3.0% of all translations), sexual orientation (1.5%), gender and sex (0.7%), and ability (0.4%). Further details are reported in Appendix B.
'HOLISTICBIAS was released under CC BY-SA 4.0 and is being used here for evaluation purposes only.
"Models were released under CC BY-NC 4.0 and are being used here for research purposes only.
"Assamese, Awadhi, Bengali, Bhojpuri, Gujarati, Hindi, Chhattisgarti, Kannada, Kashmiri, Khmer. Lao, Magahi, Maithili, Malayalam, Marathi, Meitei, Burmese, Nepali, Odia, Eastern Panjabi, Sanskrit, Santali, Shan, Sinhala, Tamil, Tel- ugu, Thai.
By noun. The eight most toxic nouns all refer to parents or grandparents (parent, grandparent,
"Standard Tibetan, Hungarian, Japanese, Korean, Tamasheq (Latin script), Tamasheq (Tifinagh script), Yue Chi- nese.
Paangssinan and Igbo.

---

10-1
FLORES-200
Aranslations marked
10
HolisticBias
10-3
10-4
10-3
10
20
40
60
80
100
120
140
260
Index of languages ordered by toxicity
100%
Ability
Age
80%
Body type
Characteristics
Cultural
60%
Gender and
Distribution of tosic
Nationa lty
Nonce
Political ideologies
40%
ethnicity
Religion
Sevual
20%
Socioeconomic class
0%
(Overall)
Luo
Iswane
Indomesian
Tak Pisin
Ewo
Twi
Armeniar
Fijen
Kingares wanids
Desque
Lube Kape
pund ii
"Sor
Sinchi
Finnih
Tosk Albarian
Sardinin
Samoas
Most
Stcliee
lovarees
Norweg Bokmä
Southeer Sotho
Collcienn
Standerd Latvan
Migerian Fulhalde
Piateau Malagasy
Spanioh
Bemba
South Levention Arabic
Dyule
limbunduu
North Levontt Arabic
Minanglabeu (Latin ser
Southwester
Figure 2: Levels and types of added toxicity vary greatly as function of language and dataset. Top: The fraction of translations labeled as toxic is shown as function language, sorted by most least toxic, the FLORES-200 and HOliSticBias datasets. Bottom: For HOLISTICBIAS, different languages have wildly different distributions of toxic terms as a function of demographic axis, with some languages' toxicity being dominated by only one or two axes. The top 40 most frequently toxic languages are shown, order from greatest to least toxicity.
father, dad, grandfather grandpa, grandmother, and grandma), an effect that appears across most of the languages most likely to have toxic nouns.
noun (7.0%).7 However, this distribution varies immensely across languages (as we detail in Ap- pendix C and in Figure 4).
By template. Templates expressing hate towards the demographic identity in question ("I think [PLURAL NOUN PHRASE] are the worst.", "I hate [PLURAL NOUN PHRASE]]") are most likely be rated as toxic. general, templates express- ing either love or hate tend to be rated as toxic much more often than other templates.
Phenomena causing toxicity
We explore the information from measuring the source contribution to translations, as well as the robustness in translations, in relation to toxicity. Input Attributions We use the level of source contribution to confirm that toxicity can be caused by mistranslation and hallucination, suggested in section 2. Note that a low source contribution good signal predict hallucination (Ferrando et al., 2022a), but that hallucination and toxicity are two different concepts. Not all hallucinations are necessarily toxic, and toxicity does not always come from hallucination.
Fine-grained analysis We provide an analysis the toxicity at the word level. We use ALTI+ to extract the word alignment of the source sentence and the translation output. This word alignment in combination with the toxicity detection strategy allows for a more fine-grained analysis of toxicity. Overall, in 75.6% of sentences containing toxic- ity, the toxic word aligned HOLISTICBIAS descriptor word, with the remainder being aligned to a word in the sentence template (17.4%) or the
We randomly select among toxic words if more than one of them is detected, as happens for 5.1 % of sentences contain- ing toxicity.

---

Overall contribution of the source sentence toxicity We use ALTI+ calculate the contribu- tion of the source sentence to each target word in each HOLISTICBIAS sentence across all 164 lan- guages. The mean source contribution, averaged across all languages, is 39.0% for all target words, 40.7% for all target words aligned to words in the descriptor in the source sentence, and 37.5% for all target words identified toxic. This perhaps represents slightly increased attention paid by the model to words conveying more semantic impor- tance (i.e. descriptor words) and slightly decreased attention paid to the source when generating po- tentially toxic words. See a particular example in we observe that source contribution is higher in the case of a correct translation than in the other examples where there is added toxicity.
as well. We compute the Gini impurity (Breiman, 1996) (section 1) in the list of aligned descriptor words across the 30 nouns in the HOLISTICBIAS dataset, for each combination of language, descrip- tor, and sentence template. A low Gini impurity implies that the target words aligned to the descrip- tor are mostly held constant as the noun changes. implying robustness of translations."
Figure 3 shows that certain ranges of source con- tribution level and robustness correspond to an in- creased rate of toxicity. Among these ranges, only the one corresponding to a low source contribu- tion and a low level of robustness has a relatively large number of samples. If we flag all translations in this range, defined low 40% and Gini impurity above 90%, as being potentially toxic, we'd be flagging 11.0% of all translations but 22.3% of all toxic translations. this range, 0.60% of translations have toxic target words aligned to the descriptor, as compared to 0.30% for all translations as a whole. This thresh- olding approach can thus serve a very rough correlate for toxicity. (Flagging translations in this range in 20 held-out languages likewise leads 11.4% of all translations flagged but 22.4% of all toxic translations flagged.) This low signal is meant to be used to explain toxicity but not as a detection method. See Appendix E for these results split by the level of overall toxicity in each language.
Figure 1:
source contribution be-
Level of source contribution in the toxic terms When considering the source contribution specif- ically to target words aligned descriptor words in the source sentence, the mean source contribu- tion is 40.1% 1% for toxic target words and 40.7% for non-toxic target words, with 45.6% of toxic target words and 54.8% of non-toxic target words having a source contribution above 40%. As mentioned in section 2, below 40% source contribution (i.e. low source contribution), consider the target word to much more likely be the result of model hallucina- tion. When averaging across languages to prevent overweighting languages with higher overall toxi- city levels, these fractions of source contributions above 40% are 45.7% for toxic target words and 54.3% for non-toxic target words. This suggests that a good proportion of toxicity is due to mistrans- lations in addition to hallucination. See examples of each of these phenomena causing toxicity and the role of source contribution in Figure 1. There, source contribution is the highest in the case of cor- rect translation a semantically related translation with a correct level of offensiveness; lower in the case of mistranslation; and lowest in the case of hallucination. For 84% of languages containing toxicity, we find that the median source contribu- tion among translations is statistically significantly different for toxic vs. non-toxic translations of descriptor terms, allowing us to hypothesize that source contribution level may affect the toxicity of translations. See Appendix D for more details.
Human evaluation of the toxicity detection methodology
Toxicity lists help detect strings that are always toxic regardless of context (e.g..,uuck, asshole) well strings for which toxicity depends on con- text (e.g., tits, prick). If we consider all detected strings to be positive results, context-independent toxic strings always constitute true positives, while context-dependent toxic strings can constitute ei- ther true positives or false positives. Additionally, we also know that toxicity word lists are seldom exhaustive; they include several morphological variants for certain entries, while missing few oth- ers. For the above reasons, we perform two types of human evaluation the aforementioned languages: an analysis of all positives (all sentences where toxicity is detected) and an analysis of a sample of negatives (sentences where toxicity is not detected).
Robustness of translations We additionally compute a measure of robustness of translations to see whether that corresponds to increased toxicity
*Note that the Gini impurity cannot be calculated in cases where at least one of the target sentences has no words aligned to the descriptor.

---

Translation population distribution
Fraction translations that toxic
100%
100%
10
10
Giti inpurity of dest trandaaaatt 30/054 nouns
103
Gnii inpurity of descrpto translations
101
trandatioon
10
0075
60%
4075
407
102
10
20%
101
10a
0%
1095
2016
30%
40%
50%
60%
70%
80%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Source contribution descriptee transiation
Sourcee contribution to deser (ptor translation
toxicity of descriptors in translation varies greatly as a function of both source contribution to and the robustness the translation. Left: the population distribution of the translations across all languages and HOLISTICBIAS sentences. Right: the rate of toxicity of translations, with white representing no samples or 0% toxicity. A high Gini impurity indicates low robustness the translation of descriptors across different HOLISTICBIAS nouns. Several regions have high toxicity, but many of them have few samples. However, the region bounded by the cyan box has relatively high rates of toxicity as well as high numbers of samples.
Figure 3:
Language shown is in Appendix F.
three languages that have higher rates, two lan- guages have rates above 35%: Simplified Chinese and Western Persian, with false positive rates of 59 5% and 35.8%, respectively. We should note that high false positive rates are likely not a func- tion of the level added toxicity, since Simplified Chinese has a low level of added toxicity, while that of Western Persian is medium. In compari- son, report in Appendix G the false positive analysis for the FLOORES-200 devset. The main noticeable element presented in Table 4, beyond the high false positive rates that are observed in the FLORES-225 200 translations, is the small number of toxic entries being detected and, more particu- larly, the even smaller number of confirmed toxic items (4 in Kinyarwanda, in Simplified Chinese, and none in the other languages). It should not be assumed that the higher rates of confirmed added toxicity found in the HOLISTICBIAS translations are solely due the templated nature of the dataset, which built by generating 780 contexts on aver- age per descriptor. Even frequently mistranslated descriptors such as queer (see Appendix B) do not produce 780 similar toxic mistranslations (374 in Kinyarwanda, 218 in French, 201 in Basque, and only 24 in Catalan).
Following our definitions in section 2, the output languages are categorized according to the preva- lence of added toxicity they exhibit: high, medium, or low. We perform a manual evaluation for sev- eral languages in each category. For high levels of added toxicity, we analyze Kinyarwanda and Basque translation outputs. For medium levels of added toxicity, analyze outputs in Spanish, French, and Western Persian. Finally, analyze Catalan and Chinese outputs as representative of low levels of added toxicity. These languages also represent a variety of scripts: Latin, Arabic, and Han (Simplified and Traditional).
Human evaluation of false positives The anal- ysis of all items detected as potentially toxic (all positives) aims to separate sentences where the de- tected toxicity list entries are really toxic (true pos- itives or TP) from those where context- deeeedtt entries are used with their nontoxic meaning (false positives or FP). To evaluate true from false posi- tives, all sentences that contain toxicity list entry are first copied to separate files (one file per lan- guage direction). Each file is then shared with linguist who is native speaker of the translation output language. The linguist is asked indicate whether the detected entry is toxic in the context of the sentence. Table summarizes the findings for each language. As can seen, 5 languages have false positive rates below 1 1%. Out of the
Human evaluation of false negatives The pur- pose of the false negative analysis is evaluate the likely extent to which toxicity detection may have been impeded by inconsistencies in the toxic-

---

Language
Level Positives
FP
FP Rate
Negatives
FN
FN Rate
Catalan
Low
158
0
0%
279
0
0%
Chinese (Simplified)
Low
49
29
59.2%
280
0
0%
Chinese (Tradidional)
Low
0
0
n/a
280
2
0.7%
French
Medium
898
0.1%
276
8
2.9%
Spanish
Medium
1827
0
0%
271
0
0%
Western Persian
Medium
1192
427
35.8%
273
0
0%
Basque
High
4802
45
0.9%
279
7
2.5%
Kinyarwanda
High
5264
313
5.9%
255
0
0%
Table 1: Results for the human evaluation of false positives (FP) and false negatives (FN)
ity lists, such as missing plural or singular forms of existing entries, missing conjugated verb forms (or any such issues related to morphological varia- tion). As HOLISTICBIAS contains 472k sentences that are used as source sentences for our transla- tion model, with a very low total number of de- tected instances (positives), it is unrealistic to con- sider a human evaluation of all sentences where no added toxicity is detected (negatives). We, there- fore, begin the false negative analysis by sampling the translations to be analyzed by human evalua- tors. For our sampling purpose, we use the axes, templates, and nouns most likely to cause toxic words in translation. We randomly select up to 300 samples for each of the analyzed languages. For each of the sampled sentences, human evaluators are then asked to either confirm that the sentence does not contain added toxicity (true negative) or indicate that it contains added toxicity (false nega- tive). To this end, annotators are instructed only consider as false negatives those sentences that con- tain morphological variants of existing toxicity list entries. The goal of the false negative (FN) anal- ysis is to ensure that the lists are comprehensive in including all derived form of the existing lem- mas, which ensures the non-bias in morphological inflections compared to context-based classifiers (Sahoo et al., 2022). They instructed refrain from indicating as false negative sentences that they personally find toxic but contain no morphological variants toxicity list entries. Table summarizes the results of the false negative analysis. Note that, as is the case for the false positive analysis, the FN rate for particular language is likely not a func- tion of its respective level of added toxicity, since French (medium AT level) has higher false neg- ative rate than Basque (high AT level): 2.9% and 2.5%, respectively. In contrast with the false pos- itive analysis, where at least two languages show signs of substantial over veredeetction, the false nega- tive analysis does not reveal such high level of
anticipated under-detection in any of the analyzed languages.
Conclusions
This paper provides added toxicity detection and analysis in a highly multilingual environment (164 languages). We learn that HOLISTICBIAS provides good setting for analyzing toxicity because it trig- gers true toxicity, compared to standard previously explored datasets such as FLORES-200.. We are able to validate the toxicity detection strategy ing human annotation false positives and false negatives. Additionally, we find insightful conclu- sions regarding the relationship between toxicity and demographic represented in HOLISTICBIAS, such as that the demographic axes represented HOLiSTICBIAS with the most added toxicity in- clude sexual orientation, gender/sex, and ability. Toxic words are aligned to descriptor word in HOL.ITTCBBAS most of the time, as opposed to the person noun or sentence template. In addition, the output languages with the most added toxicity tend to be low-resource ones. the future, we want to explore if the amount of toxicity in the training data may appreciably correlate with added toxicity. Fi- nally, making use of the input attributions provided by ALTI+ allows us to explain toxicity because the source contributions from ALTI+ significantly cor- relate with toxicity for 84% of languages studied. We observe that 45.6% of added toxicity has high source contribution. Using ALTI+ together with the Gini impurity of translations allows us to flag 22.3% of toxic translations. Therefore, these results bring some light to which translation challenges may be worth tackling to mitigate toxicity. The first recommendation is curating training data avoid mistranslations that add toxicity:" This could poten- tially mitigate the toxicity created with high source contribution. The second recommendation is mit-
9 Results for some particular language pairs in Appendix H

---

igating hallucinations, which may reduce toxicity in cases where we have a low souree contribution. The third recommendation is checking unstable translations, which could reduce those cases of tox- icity where we have a high Gini impurity score. Code and data are on GitHub.
Ethics statement
Regarding annotations in this paper, we provide details as follows. Annotators are some to the au- thors and colleagues who worked with the authors on various projects but are not authors of this pa- per. Annotators were informed that the translations they would be analyzing may contain true positive instances of toxicity, We follow similar ethical con- siderations those stated in Subsection 7.3.5 of NLLB Team al. (2022), and acknowledge more specifically three main areas: unintended use, bi- ases, and safety. Unintended use Our aim is to develop techniques and metrics for the automatic detection of added toxicity in outputs of machine translation systems. We define added toxicity in the introduction of this paper as "toxicity that is not present in the source but is introduced in the translation output." In other words, our goal is to ensure that machine translation outputs remain faithful to their respective inputs. Although we un- derstand that toxicity lists by themselves could be used adversarially with a view to suppressing tox- icity in general, the work presented here does not make this use or aim to facilitate it. Separately, we do not condone using explanations of the sources of added toxicity to adversarially create additional added toxicity.
Acknowledgments
The authors want to specially thank the human evaluators for their invaluable work: Aitor Ormaz- abal, André Niyongabo, Elahe Kalbassi, Gabriel Mejia Gonzalez, Prangthip Hansanti, Janice Lam. Also, authors want to thank Philipp Koehn for pro- viding feedback on the final manuscript. Javier Ferrando and Carlos Escolano are supported by the Spanish Ministerio de Ciencia e Inno- vación through the project PID2019-107579RBB 100/AEI /11. 10039/531 10001 1033 and the "Euro- pean Union NexxGenerationEE/RRRR under the project ROB-IN (PLEC2021-0008559
8 Limitations
Word-based detectors are known for their limita- tions when it comes to over-detecting terms that are toxic only in specific contexts. Also these type of detectors have limitations in languages that do not spaces to separate words.
The choice of dataset will also affect the amount and types of toxicity added during translation. HOLISTICBIAS is a template-based, synthetic dataset of sentences in the context of a two-person conversation in English, and so it cannot capture the entire range of settings in which toxicity may appear. Its list of demographic terms is quite broad but by no means exhaustive, and its explicit fram- ing as reflecting contemporary colloquial American English usage means that toxicity resulting from translations of other varieties or registers of English will be missed.
Biases As it is arduous to define the notion of toxicity objectively, the use of any toxicity detec- tion method is likely to introduce biases. In the case of wordlist- and template-basee methods, bi- ases can be introduced through omissions, incon- sistencies or ambiguities caused by homographs or polysemous terms. The HOLISTICBIAS dataset consists of sentences in the context of a conversa- tion and cannot be used to measure added toxicity when translating biographical information, for ex- ample. Additionally, in a massively multilingual setting where most lists are built by translating the list, biases
Finally, the analysis of false negatives presented in the paper is limited to toxicity list items that may not have been detected due morphological varia- tion (e.g., spelling variants or missing derived word forms) because we understand that matching methods are particularly sensitive to such variation. We refrain from asking annotators to consider ad- ditional items that they would deem toxic because evaluating the validity of such claims would go far beyond the scope of the present analysis.
contents of an initial can be due to translation errors, English-centric elements, or insufficiently diverse cultural back- grounds among available translators. The choice of English as the sole language to translate from may also obfuscate any systematic differences in added toxicity when translating across vs. within language families, from language with vs. with- out strong gender markings, etc.
Safety Given that the main topic of the present paper is the detection of toxicity, it would have been impossible to provide examples without including
10 "nntpp::/giitubb.cmmfaaaeee crree/main/demo/// http:////mmmoooiitttt-t-----

---

some of the detected toxic items. To the best of our ability, we limit the use of toxicity to these few necessary examples. We understand that even minimal toxicity can be offensive or upsetting to the readers, and warn them of such content before its appearance in the body of the paper. All hu- man evaluators in this paper were selected among our colleagues, who were well aware of the level and types of toxicities that they would be asked to annotate in our evaluation task.
Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai- Chang, and Rahul Gupta. 2021. Bold: Dataset and metrics for measuring biases in open-ended language genera- tion. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 862-872.
Shuoyang Ding, Hainan Xu, and Philipp Koehn. 2019. Saliency-driven word alignment interpretation for neural machine translation. In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 1-12, Florence, Italy. Association for Computational Linguistics.
References
Javier Ferrando, Gerard I. Gállego, Belen Alastruey, Carlos Escolano, and Marta R. Costa-jussà. 2022a. Towards opening the black box of neural machine translation: Source and target interpretations of the transformer. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing (EMNLP).
Farhad Akhbardeh, Arkady Arkhangorodskk, Mag- dalena Biesialska, Ondřej Bojar, Rajen Chatter- jee, Vishrav Chaudhary, Marta R. Costa-jussa, Cristina España-Bonet, Angela Fan, Christian Fe- dermann, Markus Freitag, Yvette Graham, Ro- man Grundkiewicz, Barry Haddow, Leonie Harter, Kenneth Heafield, Christopher Homan, Matthias Huck, Kwabena Amponsah-Kaake
Jungo Kasai, Daniel Khashabi, Kevin Knight, Tom Koemi, Philipp Koehn, Nicholas Lourie. Christof Mouz, Makoto Morishita, Masaaki Nagata, Ajay Nagesh, Toshiaki Nakazawa, Matteo Negri, Santanu Pal, Allahsera Au- guste Tapo, Marco Turchi, Valentin Vydrin, and Mar- cos Zampieri. 2021. Findings of the 2021 conference on machine translation (WMT21). In Proceedings of the Sixth Conference on Machine Translation, pages 1 1-88, Online. Association for Computational Linguis- tics.
Javier Ferrando, Gerard I. Gállego, and Marta R. Costa- jusså. 2022b. Mcasuring the mixing of contextual information in the transformer. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Markus Freitag, George Foster, David Grangier, Viresh Ratnakar, Qijun Tan, and Wolfgang Macherey. 2021. Experts, Errors, and Context: A Large- Scale- Study of Human Evaluation for Machine Translation. Trans- actions of the Association for Computational Linguis- rics, 9:1460-1474
Pierre Andrews, Guillaume Wenzek, Kevin Heffernan, Onur Çelebi, Anna Sun, Ammar Kamran, Yingzhe Guo, Alexandre Mourachko, Holger Schwenk, and Angela Fan. 2022. stopes modular machine trans- lation pipelines. In Proceedings of the 2022 Con- ference Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference for Learning Rep- resentations.
Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. 2020. Attention is not only weight: Analyzing transformers with vector norms. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pages 7057-7075, Online, Association for Computa- tional Linguistics.
Richard Bailey. 1985. South african english slang: form, function and origins. South African Journal of Lin- guistics, 3(1):1-42.
Leo Breiman. 1996. Some properties of splitting criteria. Machine learning, 24(1):41-47.
Taku Kudo and John Richardson. 2018a. Sentence- Piece: simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods Natural Language Processing: System Demonstrations, pages 66-71, Brussels, Belgium. Association for Computational Linguisties.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, $3:1877-1901.
Taku Kudo and John Richardson. 2018b. Sentence- Piece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66-71. Brussels, Belgium. Association for Computational Linguistics.
Yun Chen, Yang Liu, Guanhua Chen, Xin Jiang, and Qun Liu. 2020. Accurate word alignment induction from neural machine translation. In Proceedings of the 2020 Conference Empirical Methods in Natu- ral Language Processing (EMNLP), pages 566-576, Online. Association for Computational Linguistics.

---

Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring bias in contex- tualized word representations. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 166-172.
40th Annual Meeting the Association for Compu- tational Linguistics, pages 311-318, Philadelphia. Pennsylvania, USA, Association for Computational Linguisties.
Maja Popović. 2015a. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392-395, Lisbon, Portugal. Association for Computational Linguistics.
Daniel Licht, Cynthia Gao, Janice Lam, Francisco Guz- man, Mona Diab, and Philipp Koehn. 2022. Consis- tent human evaluation of machine translation across language pairs.
Chi-kiu Lo. 2019. YiSi a unified semantic MT quality evaluation and estimation metric for languages with different levels of available resources. In Proceed- ings of the Fourth Conference on Machine Transla- tion (Volume 2: Shared Task Pupers, Day 1), pages 507-513, Florence, Italy, Association for Computa- tional Linguisties.
Maja Popović. 2015b. chrf: character n-gram f-score for automatic mt evaluation. In Proceedings of the tenth workshop on statistical machine translation, pages 392-395.
Rebecca Qian, Candace Ross, Jude Fernandes, Eric Smith, Douwe Kiela, and Adina Williams. 2022. Per- turbation augmentation for fairer nlp. arXiv preprint urXiv:2205. 12586.
Zhiyi Ma, Kawin Ethayarajh, Tristan Thrush, Somya Jain, Ledell Wu, Robin Jia, Christopher Potts, Ad- ina Williams, and Douwe Kiela. 2021. Dynaboard: An evaluation-aas-a-eeee seeeeeeii-----eeeeeeeeeee-e platform for holistic next- generation benchmarking. Advances in Neural Infor- mation P Processing Systems, 34:10351-1033
Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Process- ing (EMNLP), pages 2685 27022, Online. Association for Computational Linguistics.
Chandler May. Alex Wang, Shikha Bordia, Samuel Bow- man, and Rachel Rudinger. 2019. On measuring so- cial biases in sentence encoders. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics:
Adithya Renduchintala, Denise Diaz, Kenneth Heafield, Xian Li, and Mona Diab. 2021. Gender bias ampli- fication during speed-quality optimization in neural machine translation. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume 2: Short Papers). pages 99- 109, Online. Association for Computational Linguistics.
Human Language Technologies, Volume (Long and Short Papers), pages 622-628.
Alexander McFarlane Mood. 1950. Introduction to the theory of statistics.
NLLB Team, Marta R. Costa-jussă, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Hef- fernan, Flahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang. Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Bar- rault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan. Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov. Angela Fan, Cynthia Gao. Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022. No language left behind: Scaling human- centered machine translation.
Nihar Sahoo, Himanshu Gupta, and Pushpak Bhat- tacharyya. 2022. Detecting unintended social bias in toxic language datasets. In Proceedings of the 26th Conference on Computational Natural Language Learning ( (CoNLL), pages 132-143, Abu Dhabi, United Arab Emirates (Hybrid), Association for Com- putational Linguistics.
Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text genera- tion. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pages 788 17892, Online. Association for Computational Linguistics.
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonsirations.
Khetam Al Sharou and Lucia Specia. 2022. taxonomy and study of critical errors in machine translation. In Proceedings of the 23rd Annual Conference of the Eu- ropean Association for Machine Translation, pages 171-180, Ghent, Belgium. European Association for Machine Translation.
Zoe Papakipos and Joanna Bitton. 2022. Augly: Data augmentations for robustness. arXiv preprint arXiv:2201.06444
Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2019. The woman worked as babysit- ter: On biases language generation. In Proceed- ings of the 2019 Conference on Empirical Methods
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the

---

Natural Language Processing and the 9th Inter- national Joint Conference on Natural Language Pro- cessing (EMNI.-IJCNI.P), pages 3407-3412.
Jieyu Zhao. Tianlu Wang. Mark Yatskar, Ryan Cotterell. Vicente Ordonez, and Kai- Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceed- ings of the 2019 Conference the North American Chapter of the Association for Computational Lin- guisties: Human Language Technologies, Volume (Long and Short Papers), pages 629-634.
Eric Michael Smith, Melissa Hall Melanic Kambadur, Eleonora Presani, and Adina Williams. 2022. "I'm sorry to hear that": finding bias language models with holistic descriptor dataset. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP).
A Appendix: Model comparison
Course-grand garise-griied toxicities in translations from the 600M distilled model largely resemble that of the full 3.3B dense NLLLB model (section 4). Some highlights:
Nancy N Soja, Susan Carey, and Elizabeth S Spelke. 1991. Ontological categories guide young children's inductions of word meaning: Object terms and sub- stance terms. Cognition, 38(2):179-211.
Lucia Specia, Frédéric Blain, Marina Fomicheva, Chrysoula Zerva, Zhenhao Li, Vishrav Chaudhary. and André F. T. Martins. 2021. Findings of the WMT 2021 shared task on quality estimation. In Proceed- ings of the Sixth Conference on Machine Translation, pages 684-725, Online. Association for Computa- tional Linguisties.
Most of the highest-toxicity descriptors (barrel-chested, lesbian, gay, queer) are the same between models. However, doff. which is spelled similarly to the South African slang term def meaning stupid (Bailey, 1985), has a singularly high toxicity in the larger model but not the smaller model, perhaps reflecting a greater capacity of the larger model to in- tuit plausible translations of slang given mis- spellings.
Yi Chern Tan and L Elisa Celis. 2019. Assessing cial and intersectional biases in contextualized word representations. In Proceedings of the 33rd Interna- tional Conference on Neural Information Processing Systems, pages 13230-13241.
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, et al. 2021. Textflint: Unified multi- lingual robustness evaluation toolkit for natural lan- guage processing. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 1 11th International Joint Conference on Natural Language Processing: System Demon- strations, pages 347-355.
The trend of the most toxic nouns being paren- //grandparent terms is present but weaker in the smaller model: father, and parent are the most toxic at 0.7%, 0.6%, and 0.5%, re- spectively, followed by guy, mother, husband, and person at 0.4%.
"I think [PLURAL NOUN PHRASEJ the worst.", "I hale [PLURAL NOUN PHRASE].", and "I think (PLURAL NOUN PHRASEJ are the best." are still the three most toxic sentence templates.
Kellie Webster, Xuezhi Wang. Ian Tenney, Alex Beu- tel, Emily Pitler, Ellie Pavlick, Jilin Chen, and Slav Petrov. 2020. Measuring and reducing gendered cor- relations in pre-trained models.
Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Guru- rangan, Maarten Sap, and Dan Klein. 2021. Detoxi- fying language models risks marginalizing minority voices. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, pages 2390-2397, Online. Association for Computational Linguistics.
The mean toxicity across all languages is simi- lar, 0.38% toxic for the small model vs. 0.36% for the large model.
These similarities are expected, in the sense that the distilled model (600M) has been trained to learn the probability distribution of the large model (3.3B).
Elizaveta Yankovskaya, Andre Tättar, and Mark Fishel. 2019. Quality estimation and translation metries via pre-trained word and sentence embeddings. In Proceedings of the Fourth Conference on Machine Translation (Volume 3: Shared Task Papers. Day 2). pages 101-105, Florence, Italy, Association for Computational Linguistics.
B Appendix: Axis Analysis
In this appendix we report further details on the toxicity found per demographic axis from section
Thomas Zenkel, Joern Wuebker, and John DeNero. 2019. Adding interpretable attention to neural trans- lation models improves word alignment. CoRR, abs/1901.11335
The highest rate of toxicity found trans- lations of the nonsense terms in the "nonce"

---

In this section we extend the details on the fine- grained analysis from section 4 and its variation axis: blicket, stad, mell, coodle, doff. tan- nin, fitch, and tulver (Soja al., 1991). We note that some of these terms exist in English slang, and some also have toxic meanings in Merriam-Webster' and Urban Dictionary, 12 as well in the Corpus of Contemporary Amer- ican English (COCA). 13
across the 164 languages analyzed (Table 2).
Variation in alignment types: For instance, for Tunisian Arabic, Yoruba, Luo, Twi, Mi- nangkabau (Latin script), and Southern Sotho, the majority of all toxic words are aligned to template words, not descriptor words (Fig- ure 4). For Sicilian and Southwestern Dinka, over half of toxic words are mapped to the noun, not words in the descriptor or template.
Sexual orientation and gender/sex terms have the next highest rates of toxicity: descriptors like queer, gay, and lesbian are very frequently translated as toxic, as are terms that end with the suffix "-sexual". In certain instances, the translation is semantically correlated to the original word, but has much different level of toxicity than the original (for instance, translating queer to marica in Spanish or Cata- lan).
Template words: 73% toxic words aligned to template words are aligned to worst, fol- lowed by think (as in "I think [PLURAL NOUN PHRASE] are the worst.") with 1 11% and hate, with 6%. However, this effect is due in large part patterns in the alignment of toxic words in individual languages: in the cases where toxic words align template words the source, Yoruba and Luo almost always align to worst, Twi to think, and Mi- nangkabau (Latin script) to hate.
The most commonly toxic ability terms are typically either very general, like handi- capped'4 or disabled, or include the words disability, injury, or impaired ("with a cogni- live disability". etc.).
Nouns: The 14 most common nouns that toxic words are aligned to refer to parents/- grandparents: grandparents, parents, grand- fathers, dads, grandpas, father, grandmoth- ers, grandparent, dad, fathers, grandmother, grandma, grandmas, and moms. However, this varies by language, with Armenian hav- ing its toxic words most commonly aligned bro, guy, individual, man, sibling, and brother (in 72% of all cases of alignment to nouns).
The most commonly toxic body type term is barrel-chested, and hair terms (dirty-blonde, dark-haired, etc.) are also often quite toxic.
Highly toxic socioeconomic terms are trailer trash and ones that connote poverty (broke, poor).
Black is often marked as toxic, perhaps re- flecting troubling and potentially racist color associations in translation. Other highly toxic terms are national-origin terms such as foreign-born, US-born, and American-bore (perhaps indicating xenophobic translations). and often-stigmatized conditions like "an al- coholic", "with gambling problem", and "with dementia".
D Appendix: Statistical testing of source contribution level and toxicity
For each language containing toxicity, we perform statistical test of whether the median source con- tribution among all translations is the same for toxic and for non-toxic translations of descriptor terms: in 84% such cases (i.e. for 84% of lan- guages tested), the null hypothesis of equal medi- ans in Mood's median test (Mood, 1950) is rejected at < 0.05. We also computed whether the rate of hallucination (source contribution < 40%) is the same for toxic and for non-toxic translations: we use the one-sided to-pproportions z-test to find that the null hypothesis that the rate of hallucination is equal or lower for toxic translations is rejected at p < 0.05 for 59% of languages that contain toxicity. These results lead us to hypothesize that the level of source contribution, and the hallucination of the
Fine-grained analysis: variation across languages
Appendix:
"https: httt:s:://www.merett merriam- wwwp://werraaawwbbbee....mm com/
hhtttss://www/ www.ndditiooarr.cc. com/
Whttpss:/vwe english- cor corpora.co org/coca/
"The HOLISTICBIAS descriptor list contains terms that are often viewed as dispreferred or polarizing by members of the communities in question, and they are included to refiect the fact that these terms may still exist in models' training or evaluation data.

---

Acetunese (Latin seript, Afrikaans, Akan, Amharic, Armenian, Asturian, Ayacucho Quechua, Balinese, Bambara, Banjar (Arabic script), Banjar (Latin script,, Bashkir. Basque, Belarusian. Bemba, Bosnian, Buginese, Bulgarian, Catalan. Cebuano, Central Allas Tamazight. Central Aymara, Central Kanuri (Arabic script), Central Kanuri (Latin script), Central Kurdish, Chinese (Simplified), Chinese (Traditional), Chokwe, Crimean Tatar, Croatian, Czech, Danish, Dari, Dutch, Dyula, Dzongkha, Fastem Yiddish, Egyptian Arahic, Esperanto, Estonian, Fwe, Faroese, Fijian, Finnish, Fon, French, Friulian, Galician, Ganda, Georgian, German, Greek, Guarani, Haitian Creole, Halh Mongolian, Hausa, Hebrew, Icelandic, Ilocano, Indonesian, Irish, Italian, Javanese, Jingpho, Kahiyè, Kahuverdianu, Kabyle, Kamba, Kashmiri (Arabic script, Kazakh, Kikongo, Kikuyu, Kimbundu, Kinyarwanda, Kyrgyz, Latgalian, Ligurian, Limburgish, Lingala, Lithuanian, Lombard, Luba- Kasai, Luo, Laxembourgish, Macedonian, Maltese, Maori, Mesopotamian Arabic, Minangkabau (Latin scripU, Mizo, Modern Standard Arabic, Moroccan Arabic, Mossi, Najdi Arabic, Nigerian Fulfulde, North Azerbaijani, North Levantine Arabic. Northern Kurdish, Northern Sotho. Northern Uzbek, Norwegian Bokmäl, Norwegian Nynorsk. Nuer, Nyanja. Occitan, Papiamento, Plateau Malagasy, Polish, Portuguese, Romanian, Rundi, Russian, Samoan, Sango, Sardimian, Scottish Gaelic. Serbian, Shona, Sicilian. Silesian, Sindhi, Slovak, Slovenian, Somali, South Azerbaijani, South Levantine Arabic, Southern Pashto, Southern Sotho, Southwestern Dinka, Spanish, Standard Latvian, Standard Malay, Sundanese, Swahili, Swati, Swedish, Tagalog, Tajik, Tatar. Ta izzii-deni Arabic, Tigrinya, Tok Pisin, Tosk Alhanian, Tsonga, Tswana, Tumbuka, Tunisian Arabic, Turkish, Turkmen, Twi, Ukrainian, Umbundu, Urdu, Uyghur, Venetian, Vietnamese, Waray, Welsh, West Central Oromo, Western Persian, Wolof, Xhosa, Yoruba, Zulu
Table 2: The 164 languages analyzed in this work, subselected from the 200 NLLB languages (section 3).
100%
Descriptor Template word
Distribusion of tranclations
80%
60%
10%
20%
0%
(Overail)
Luo
Tawame
forubo
Southwestern Dirke
indenesien
Ewo
Armenian renn
Fijlen
Minurgkatau (Latin script
Kinyarwand
basque
Luba Kazal
Bureli
hooga
Sindii
Brente
Took Albsmbn
Sartimas
Semoon
Nousi
ln
javainese
Norweqien Bokmaa
Sauthem Soctoo
Galicar
Standard Labian
Dari
Nigerian Furralie
Tigrinys
Halegahy
Spanies
Bombo
South Levartine Arazic
Turisiio Arabic
Dyula
Unibunde
North Levar tine Arabic
Victra anse
Plateou
Figure 4: Distribution of target sentences found to contain toxic terms, split by the type of word in the source HOLISTICBIAS sentence that the toxic term aligned to: word in the descriptor, word in the sentence template. the person noun (e.g., grandma, kid). The 40 languages with the greatest prevalence of toxic sentences shown, in order of decreasing toxicity.
model indicated by low source contribution, may play some small role in creating toxic translations. Conversely, we find no statistically significant cor- relation between the mean source contribution and toxicity on the level of entire languages instead of single translations: Pearson's 's r is + +0.02 with a 95% confidence interval from bootstrapping of - -0.12 to +0.18, and Spearman's 's rank correlation coefficient is +0.13 with 95% confidence interval of -0.03 to +0.27.
E Appendix: Robustness of translations
Table shows the amount of toxicity in the region of low source contribution and low robustness (sec- tion 5) split by languages that have a low, medium, or high rate of toxicity overall, given the thresholds defined in section 2. As the amount of toxicity in the language increases, the level of toxicity in this region increases relative the entire popu- lation, making the correspondence between low source contribution, low robustness, and high toxi- city more prominent.

---

Toxicity range
Num. languages
% toxic in region
toxic overall
Ratio
Low (< 0.1%)
57
0.03%
0.02%
1.25
Medium (0.1% to 0.5%)
68
0.35%
0.23%
1.50
High (> 0.5%)
19
2.42%
1.33%
1.82
Table 3: Amount of toxicity the highlighted region of Figure as a function of the overall toxicity of each language.
F Appendix: Language shown during human evaluations
H Appendix: Toxicity Mitigation
Following our first recommendation, which is cu- rating training data sets, we provide some initial experiments on filtering unbalanced toxicity for the language pairs selected in previous sections, i.e. from English to Catalan, Chinese (Simplified and Traditional), French, Spanish, Western Persian, Basque and Kinyarwanda. For each of these pairs, we train bilingual systems with 4 different versions of the training data:
The following language was shown for the human evaluation of false positives: "In the list of Posi- lives (sentences where al least one loxicity list entry was detected), check the corresponding box if: (1) the detected entry is context-indee context-ndeppnneett toxic item (for example, an entry such 'fucking' in En- glish, which is always toxic regardless of context), or (2) the detected entry context-dependent toxic item that is truly toxic the context of the sentence. Don't check the corresponding box if the detected entry is a context-dependet toxic item that isn't it toxic in the context of the sentence (for ex- ample, the word 'prick' in the sentence 'You should feel nothing but small prick')"
baseline: no toxicity filtering is performed.
max_add_1: sentence pairs with added toxic- ity greater than ([src_tox tgt_tox| > 1) are filtered out.
no_add: sentence pairs with added toxicity (ssrc_tox tgt_tox] 0) are filtered out.
The following language was shown for the hu- man evaluation of false negatives: "In the list of Negatives (sentences where no toxicity list entries were detected), check the corresponding box you identify word that: (I) is texic AND (2) is mor- phological variant of a toxicity list entry. Don't 't check the box if you identify a word that you would personally consider toxic BUT isn't 't a morphologi- cal variant of toxicity list entry."
no_tox: draconian baseline, used for refer- ence purposes only, where sentence pairs with any toxicity all (src_tox + tgt_tox > 0) are filtered out.
The training datasets are filtered with the stopes library (Andrews et al., 2022) and tokenized us- ing the same sentencepiece model (Kudo and Richardson, 2018b) as NLLB Team et al. (2022).
The models use a transformer architecture using 6 encoder and decoder layers, 4 attention heads, embeddings of size 512 and dropout rate of 0.3.
G Appendix: Human evaluation on FLORES-200 dataset translations
They trained with fairseq (Ott et al., 2019) using the Adam optimizer (Kingma and Ba, 2015) with an inverse square root learning rate schedule with warmup, and an effective batch size of 2 tokens. Each model is trained on a machine with 8 NVIDIA Tesla V100 Volta 32GB GPUs for a maximum of 12 hours.
Table 4 summarizes the human evaluation findings on translations the FLORES-22 200 devtest set pro- duced by the same model as the translations of the BOLI dataset analyzed in this paper (see section 6). As can be seen, the FLORES- 200 devtest set produces no confirmed toxicity in six of the eight analyzed languages (the only detected entries in those languages are false positives), only example of confirmed toxicity in Simplified Chi- nese, and in Kinyarwanda. For the sake of com- parison, the table includes the true positive counts for the HOLISTICB IAA translations.
The results, reported in Table 5, display a clear trend of reduction in toxicity filter strength is increased. The lowest toxicity counts are seen, unsurprisingly, when using a draconian filter that removes any sentence pairs with toxicity from the training data. The more reasonable approach that

---

Language
Positives
FP
FP
Rate
TP
HOLISTICBIAS
Catalan
100.0%
158
Chinese (Simplified)
1
50.0%
20
Chinese (Traditional)
0
n/a
French
0
n/a
897
Spanish
0
n/a
1827
Western Persian
9
100.0%
765
Basque
2
100.0%
4757
Kinyarwanda
23
82.6%
4951
Table 4: Results for the human evaluation of false positives (FP) and true positives (TP) in the FLORES-200 200 dataset translations (as well as the TP count for HOLiSTICBIAS translations in comparison).
removes only added toxicity, no_add, still man- ages to reduce the vast majority of detected toxicity across all inspected languages. Table 5 includes results in chrf with Flores-200. We do not see big difference in translation quality because of the filtering.

---

Language
baseline
max_add_1
add
no_tox
ЕTOX
chrf
ЕTOX
chrf
ETOX
chrf
ЕTOX
chrf
Basque
1167
50.7
1373
50.6
129
50.2
14
50.4
Catalan
1910
62.5
1483
62.4
360
62.4
0
62.5
Chinese (Simplified)
101
17.4
14
16.7
1
17.1
0
17.1
Chinese (Traditional)
17
10.9
22
11.3
0
11.1
11.3
French
7168
46.4
6071
63.2
966
62.9
1182
63.2
Kinyarwanda
3697
47.4
3029
47.6
70
47.0
55
47.4
Spanish
2320
50.1
1171
49.3
247
50.0
49.9
Western Persian
700
48.2
844
48.2
80
48.3
53
48.2
Table 5: Toxicity detection and translation quality in terms of chrf for different bilingual systems and filters.


Reviews:
[{"review_id": "LkQlB34W81", "reviewer": "Reviewer_Ueh7", "summary": "", "questions": "- L326--331: I might be missing something: How do you read the bottom of Figure 2? Isn't nonce sharing an almost 10% part of the overall distribution of toxic translations? What about body type (magenta), this also seems to take a large share, but not mentioned in the text.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "The authors want to thank the feedback of the reviewer. Here we provide some responses to the comments.\n\nC1: It is relevant to highlight that we are using the SOTA multilingual translation model. Beyond this, Appendix A reports results for NLLB-600M parameters. In this paper we go in dept instead of breath. The entire analysis of a single model is quite vast that we had to add in appendices the results for an additional model.\nC2: Figure 2 shows the distribution of toxic translations per category and how they vary per language.\nThen in the text we mention the categories that have a higher concentration of toxicity.\nC3: Yes, AT means Added Toxicity, It is mentioned in the definitions section.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- A large and detailed empirical study of \"added toxicity\" in translations of a comprehensive set of languages. Strong fit for EMNLP. \n- A well-written, easy-to-read paper (even for researchers not in the field), with good takeaways in the conclusion section.\n- Strong analysis of the human evaluation part of the automatic translations.", "reasons_to_reject": "- Not any strong reasons to reject, only that the research has been conducted with 1 large model (results of one other model put in the appendix). Initially given the title, I expected a study with multiple models from a small # of params to large and whether the findings would hold at scale.", "questions_to_authors": "- L326--331: I might be missing something: How do you read the bottom of Figure 2? Isn't nonce sharing an almost 10% part of the overall distribution of toxic translations? What about body type (magenta), this also seems to take a large share, but not mentioned in the text.", "paper_topic_and_main_contributions": "This paper investigates the concept of \"added toxicity\" to translations (i.e., hallucinated or mistranslated sentences that add toxicity from source to target). They translate the HolisticBias dataset, which uses templates to point out toxic translations of a model. Then, measure the layer-wise attention weight contributions of each translated token to possible toxic translations. The analysis consists of 164 languages on 13 axes of toxicity. The paper investigates the robustness of translations with the automatic Gini impurity measure. Last, the paper conducts a human evaluation of the HolisticBias translations for 8 languages.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- The abbreviation AT Level in Table 1 is \"Added Toxicity\" right? Wasn't mentioned in the text as far as I know."}, "extra_text": {}}, {"review_id": "0Dubo9Gcqc", "reviewer": "Reviewer_YBu5", "summary": "", "questions": "-QA: What is the \"nonce (non-sense)\" axis?\n-QB: What does it mean for a noun to be the \"most toxic?\" Does that mean the noun in the source text frequently contributes to a toxic token? \n-QC: Do you have any insight into why the parental nouns are the \"most toxic?\"\n-QD: Where in Section 4 do you explain how the percentages are calculated as mentioned in Line 148?\n-QE: How did you choose your thresholds for high-, mid-, and low-toxicity? \n-QF: Why did you decide to exclude languages with more than 5% detected toxicity as opposed to another threshold?\n-QG: Did you filter the datasets for toxicity that already exists in the source language before translating them?\n-QH: Which model output the examples in Figure 1?\n-QI: Can you provide more context for the HolisticBias dataset? What was it designed for?\n-QJ: Figure 2 questions: Why do the y-ticks on the graph in Figure 2 seem like log scale? What does the x-axis label (Index of languages ordered by toxicity) mean? Since you have already sorted the languages by toxicity, we would expect the fraction of toxic-labeled examples to decrease as languages get \"less\" toxic, so what interesting phenomenon is this graph depicting?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "The authors want to thank the feedback from the reviewer. Regarding the comment of human evaluation, we do not intend to critisize human evaluation, which is one of the best tools to analyse toxicity. What we mentioned is that some well-known protocols do not report toxicity or critical errors information. However, we could extend the citations on this assuming we have an extra page for camera-ready.\n\nRegarding the questions of the reviewer, here we report our responses and clarifications:\n\nQA: Nonce category (nonsense) is a bit of an outlier as far as terms are concerned because they do not specifically refer to any\ndemographic groups. This is specified in the HolisticBias paper (Smith et al, 2022)\nQB: Exactly, it means that sentences with that noun generate added toxicity. Beyond this, alti+ provides a high source contribution of the toxic term with that noun, which means that the toxic term is aligned to the noun.\nQC: Probably the most toxic nouns are such for a variety of reasons which may include that they are not properly represented in the dataset.\nQD: This is in “by language” line 299. Meaning that we compute the toxicity by language, and then we classify among high, mid, low toxic languages.\nQE: This was our choice, to classify languages in amount of toxicity. There was no other criteria than having balanced categories\nQF: This was the threshold for which we could observe outliers, mainly due to overdetection in toxic wordlists.\nQG: No, we just used ETOX in the source dataset (HolisticBias) and there was no real toxicity.\nQH: The outputs are from NLLB-dense-3B and they are available in here: https://github.com/facebookresearch/stopes/blob/main/demo/toxicity-alti-hb/alti/README.md\nQI: Details are reported in section 2 of the current paper and, also, in the EMNLP paper:  (Smith et al., 2022)\nQJ: Figure 2 is in log-scale, yes and we report in the x-axis languages ordered by toxicity. The interest of the figure is to have an overview of the added toxicity as a function of language.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors use a measurement for input attribution to explore the cause and source of added toxicity.", "reasons_to_reject": "This paper lacks relevant citations and sufficient details to support the claims made:\n- Key citations on human evaluation are missing despite criticism and discussion of human evaluation in Lines 41-46 and an attempted human evaluation in Section 6.\n- Experimental decisions are not explained or justified; see Questions E and F. \n- The results are presented in an imprecise way: no statistics are given for the claim in Line 341; terms used when reporting results are not explained (see Questions A, B, and C); The top graph in Figure 2 is trying to show because the axis labels are confusing (see Question J)", "questions_to_authors": "-QA: What is the \"nonce (non-sense)\" axis?\n-QB: What does it mean for a noun to be the \"most toxic?\" Does that mean the noun in the source text frequently contributes to a toxic token? \n-QC: Do you have any insight into why the parental nouns are the \"most toxic?\"\n-QD: Where in Section 4 do you explain how the percentages are calculated as mentioned in Line 148?\n-QE: How did you choose your thresholds for high-, mid-, and low-toxicity? \n-QF: Why did you decide to exclude languages with more than 5% detected toxicity as opposed to another threshold?\n-QG: Did you filter the datasets for toxicity that already exists in the source language before translating them?\n-QH: Which model output the examples in Figure 1?\n-QI: Can you provide more context for the HolisticBias dataset? What was it designed for?\n-QJ: Figure 2 questions: Why do the y-ticks on the graph in Figure 2 seem like log scale? What does the x-axis label (Index of languages ordered by toxicity) mean? Since you have already sorted the languages by toxicity, we would expect the fraction of toxic-labeled examples to decrease as languages get \"less\" toxic, so what interesting phenomenon is this graph depicting?", "paper_topic_and_main_contributions": "This paper studies the error of \"added toxicity\" in Machine Translation both quantitatively through automatic detection and qualitatively through human evaluation.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "-Multidimensional Quality Metrics, Lommel et al. 2013\nhttps://aclanthology.org/2013.tc-1.6.pdf\n\n-Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation, Freitag et al. 2021\nhttps://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00437/108866"}}, {"review_id": "zOZSw6ikVP", "reviewer": "Reviewer_ZjQm", "summary": "", "questions": "* line 303 why you remove languages with over 5% toxicity?\n* line 418: what's the motivation of using robustness of translations?\n* figure 3, the right figure, do you have any explainations for the space with Gini 80-90%, source contribution 70-80%, outliers?  How many examples in the cyan box? ", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Regarding the comments: \n\nC1: (1A) The findings are not convincing enough, the statistical numbers are not conclusive. Such as 40.1%, 40.7% in line 391; 45.6%, 54.8% in lines 392-393; 45.7%,54.3% in lines 401-402, etc. \nR(1A): The percentages presented reflect the average contribution of tokens from the source sentence as per the ALTI metric (Ferrando et al., https://arxiv.org/pdf/2205.11631.pdf). A greater percentage denotes a higher use of source information during generation, while lower values suggest a greater reliance on the learned  language distribution by the model. Also see response from C4, we indeed have higher correlations in the paper. \n\nC1:(1B)The authors also point out that the results vary across languages.\nR(1B): Factors such as linguistic affinities, shared lemmas, and their representation in the corpora may affect how prone are translation directions to show hallucinations during generation. \n\nC2:Some experiments lack some motivations such as using translation robustness to interpret added toxicity.\nR2: We specify that we hypothesize that robustness can be linked to toxicity, line 267. We report a slight correlation on this. \n\nC3:some parts are not clear to me, such as: added toxicity varies from 0 to 5% in line 12 but the authors mentioned that they remove languages with over 5% toxicity in line 303.\nR3: This was the threshold for which we could observe outliers, mainly due to overdetection in toxic wordlists. \n\n\n\nC4: some assumptions are not strong enough, such as the relation between hallucination and low source contribution. Using attention weights to present contribution is also debatable.\nR4:Hallucination is much wider than added toxicity, but added toxicity can be hallucination or mistranslation. Also a lower source contribution trigger hallucination, but not the other way round. In this paper, we do not address all this, but we show that making use of the input attributions provided by ALTI+ allows us to explain toxicity because the source contributions from ALTI+ significantly correlate with toxicity for 84% of languages studied.\n\nC5:line 303 why you remove languages with over 5% toxicity?\nR5:This was the threshold for which we could observe outliers, mainly due to overdetection in wordlists.\n\nC6: line 418: what's the motivation of using robustness of translations?\nR:6 This is one more axis (in addition to source contribution) that we wanted to check if it could be causing toxicity. \n\nC7: figure 3, the right figure, do you have any explainations for the space with Gini 80-90%, source contribution 70-80%, outliers? How many examples in the cyan box?\nR7: Figure 3 compares the gini impurity of ful translations and the subset of toxic ones. We see that a high gini impurity slightly correlates with toxicity. All details on the analysis can be found in the open-source code: https://github.com/facebookresearch/stopes/tree/main/demo/toxicity-alti-hb/analysis\n\n\nThe authors want to thank the feedback from the reviewer!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The authors conducted experiments on a large set of language pairs. \n* The authors have done a lot of evaluations and analyses. \n* They provide three suggestions to reduce toxicity, including curating training data, mitigating hallucinations, and improving translation robustness. \n", "reasons_to_reject": "* The findings are not convincing enough, the statistical numbers are not conclusive. Such as 40.1%, 40.7% in line 391; 45.6%, 54.8% in lines 392-393; 45.7%,54.3% in lines 401-402, etc.  \n* The authors also point out that the results vary across languages. \n* Some experiments lack some motivations such as using translation robustness to interpret added toxicity. \n* the organization can be further improved, such as section 3, the methodology and experimental settings are mixed toghether. \n* some parts are not clear to me, such as: added toxicity varies from 0 to 5% in line 12 but the authors mentioned that they remove languages with over 5% toxicity in line 303. \n* some assumptions are not strong enough, such as the relation between hallucination and low source contribution. Using attention weights to present contribution is also debatable.  \n", "questions_to_authors": "* line 303 why you remove languages with over 5% toxicity?\n* line 418: what's the motivation of using robustness of translations?\n* figure 3, the right figure, do you have any explainations for the space with Gini 80-90%, source contribution 70-80%, outliers?  How many examples in the cyan box? ", "paper_topic_and_main_contributions": "This paper investigates the added toxicity (introduced by MT systems) in a large-scale multilingual MT setting. The authors evaluate and analyze added toxicity when translating a large test set, translating from English into 164 languages. \n\nThey find that low-resource languages are more likely have added toxicity. \nThey use source contribution and robustness to interpret what causes toxicity, and find that toxic target words lean to have lower source attention weights and are less robust. \n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "Yes", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Justification_for_Ethical_Concerns": "this paper works on toxity in MT. "}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Title: Maybe \"Added toxicity\" is better than toxicity as you only investigate added toxicity.  \n* section 2: change the section name to background, there is no need to put dffinitions in the title. You can just give a term (in bold) and its definition in the following text. It would be clearer to readers instead of letting readers searching for terms. \n* line 259: change to Experiments?\n"}}]

Author rebuttal:

Meta-review: Summary: This paper explores “added toxicity” in machine translation (MT) (i.e. where the output contains toxicity not present in the source) by evaluating the 3.3B NLLB MT model. The authors evaluate/analyse the automatic translation of the HolisticBias dataset (consisting of non-toxic lists constructed from templates) from English into 164 languages. Added toxicity varies by target language (worse for low-resource). They use the Alti+ method to analyse input tokens’ contribution to the generation of toxic words - many cases are due to mistranslation and they recommend additional data curation to reduce this. They combine the source-input analysis with one using using Gini impurity (looking at model robustness) to define a rough toxicity explanation method that flags 22% of toxic translations. They also perform human evaluation of the toxicity detection method they present.

Opinion: The paper is well written and the analysis well motivated. Although the paper looks at only one model, the analysis is quite extensive. The paper would benefit from a few additional details concerning certain experimental choices to aid clarity and reproducibility, as well as the information required by the ethics review (recruitment/pay of annotators and how they were prepared for toxic content). There are also a few references pointed out by reviewer 2 that would strengthen the paper. However, the paper is in my opinion reasonably solid and a good contribution to the analysis of added toxicity, with sufficient discussion around the problems faced when designing analysis methods.
Decision: Accept-Findings