Toxicity in Multilingual Machine Translation at Scale
Authors: Marta R. Costa-jussà
Christophe Ropers
Eric Michael Smith
Daniel Licht
Carlos Escolano
Javier Ferrando
Venue: Submitted to ICLR 2023 (2023)
Keywords: Toxicity
Holistic Bias
Input Attributions
Multilingual Machine Translation
Scale

Abstract: Machine Translation systems can produce different types of errors, some of which get characterized as critical or catastrophic due to the specific negative impact they can have on users. Automatic or human evaluation metrics do not necessarily differentiate between such critical errors and more innocuous ones. In this paper we focus on one type of critical error: added toxicity. We evaluate and analyze added toxicity when translating a large evaluation dataset (HOLISTICBIAS, over 472k sentences, covering 13 demographic axes) from English into 164 languages. The toxicity automatic evaluation shows that added toxicity across languages varies from 0% to 5%. The output languages with the most added toxicity tend to be low-resource ones, and the demographic axes with the most added toxicity include sexual orientation, gender and sex, and ability. We also perform human evaluation on a subset of 8 directions, confirming the prevalence of true
added toxicity.

We use a measurement of the amount of source contribution to the translation, where a low source contribution implies hallucination, to interpret what causes toxicity. We observe that the source contribution is somewhat correlated with toxicity but that 45.6% of added toxic words have a high source contribution, suggesting that much of the added toxicity may be due to mistranslations. Combining the signal of source contribution level with a measurement of translation robustness allows us to flag 22.3% of added toxicity, suggesting that added toxicity may be related to both hallucination and the stability of translations in different contexts. Given these findings, our recommendations to reduce added toxicity are to curate training data to avoid mistranslations, mitigate hallucination and check unstable translations.

Under review as a conference paper at ICLR 2023
TOXICITY IN MULTILINGUAL MACHINE TRANSLATION AT SCALE
Anonymous authors Paper under double-blind review
ABSTRACT
Machine Translation systems produce different types of errors, some of which get characterized as critical or catastrophic due to the specific negative impact they can have on users. Automatic human evaluation metries do neces- sarily differentiate between such critical errors and more innocuous ones. In this paper we focus on one type of critical error: added toxicity. We evaluate and an- alyze added toxicity when translating a large evaluation dataset (HoLISTICBIL BIAS, over 472k sentences, covering 13 demographic axes) from English into 164 lan- guages. The toxicity automatic evaluation shows that added toxicity across lan- guages varies from 0% 5%. The output languages with the most added toxicity tend to be low-resource ones, and the demographic axes with the most added tox- icity include sexual orientation, gender and sex, and ability. We also perform hu- subset of translation directions, confirming the prevalence
man evaluation of true added toxicity.
We use a measurement of the amount of source contribution to the translation, where low source contribution implies hallucination, interpret what causes toxicity. We observe that the source contribution is somewhat correlated with toxicity but that 45.6% of added toxic words have a high source contribution, sug- gesting that much of the added toxicity may be due to mistranslations Combining the signal of source contribution level with a measurement of translation robust- ness allows us to flag 22.3% of added toxicity, suggesting that added toxicity may related to both hallucination and the stability of translations in different con- texts. Given these findings, our recommendations to reduce added toxicity are to curate training data to avoid mistranslations, mitigate hallucination and check unstable translations.
WARNING: this paper contains examples of
toxicity that may be offensive upsetting in nature.
INTRODUCTION
Machine Translation (MT) systems are typically evaluated in terms of translation quality either by automatic or human measures. Automatic measures compare the translation output more human references, e.g. [Papineni et al.] (2022); Popović (2015); ; Rei et al. (2020); ?. Human measures use annotators rank translation outputs, e.g. Licht al. (2022) Akhbardch al. (202I). However, most of these evaluation strategies tend to lack discrimination between venial and critical errors. While translation can be of higher or lower quality, it worth distinguishing if are producing critical errors. Vilar et al. (2006) is an example of taxonomy for translation errors in general. More recently, there the critical error detection task which aims at predicting sentence- level binary scores indicating whether or not translation contains critical error (not limited to toxicity) SSpecia et al.] (202T) and [Sharou & Special (2022) provide taxonomy to classify critical errors. In this work, we focus the first of the Sharou and Specia: deviation in toxicity. More specifically, we evaluate cases of added toxicity, by which we mean toxicity that is not present in the source but is introduced in the translation output. Our definition of added toxicity differs from the broader category of deviation in toxicity in that does not cover cases of deletion.
categories of critical errors proposed by
The study of added toxicity is made both difficult and necessary by the fact that such critical errors rather infrequent, especially in informative discourse (e.g., Wikipedia, news), but have signif-

---

Under review as a conference paper at ICLR 2023
icant impact on translation safety and user trust. Previous work by the NLLB Team al. (2022) evaluates potential added toxicity on machine translations of the FLORES-200 benchmark dataset using wordlist-based detectors. Such detectors over-detecting terms that are toxic only in specific contexts. Nevertheless, the overall prevalence of potential added toxicity remains low when evaluating translations of formal sentences such those in FLORES-200, which makes it difficult to draw conclusions as to this specific aspect of a model's performance.
known for their limitations when it comes to
To circumvent the problem posed by the low prevalence of toxicity in our test sets, which may not reflect the prevalence of toxicity in our models, we use the recently proposed bias evaluation dataset HOLISTICBIAS (Smith et al. (022). This English-only (American English) dataset has been used to evaluate variety of demographic biases in language modeling (Qian et al., 2022 Smith et al. 2022). The dataset contains over 472k sentences (100 time larger than typical evaluation sets) and designed to trigger biased behaviors in language models. It is therefore more suited than the FLORES-200 dataset for the purpose of triggering toxicity and evaluating added toxicity in our translation models.
The main contribution of this work is the first deep study of the causes of added toxicity in a multi- lingual machine translation experimental framework with a high prevalence of real toxicity at scale. For this purposes, we combine the previously defined toxicity detection methodology (NLLB Team et al. 2022). the controlled HOLISTICBIAS evaluation dataset (Smith et al.] 2022). and the A interpretability method (Ferrando et al. 2022a). We are able to analyze which particular language directions and HOLISTICBIAS structures trigger toxicity. Moreover, we perform human evalua- tion of the toxicity detection methodology for subset of eight out-of-English translation directions, and find that the false positive rates are below 1% in five translation directions. False negatives are below 3% in all translation directions. Finally, we demonstrate an interaction between the source contribution, the robustness of translations, and toxicity. We use ALTI+ to observe that 45.6% of the toxic translations have a high source contribution, which hints that much of these toxic translations may be caused by mistranslations, and that the rest may be correlated with hallucination (Ferrando ct al. 2022a). This suggests that hallucination may add toxicity. We use Gini impurity (Breiman 1996.. common splitting criterion in decision trees, to measure the relative amount of diversity (i.e. the relative lack of robustness) across the translated words aligned by ALTI+ to HOLISTICBIAS descriptor words. A combination of low amount of source contribution and high Gini impurity across translations corresponds to a rate of toxicity roughly twice as high as the baseline rate. These findings lead us recommend that mitigation of toxicity could be achieved by curating training data to avoid mistranslations, reducing hallucinations and checking unstable translations.
DEFINITIONS AND BACKGROUND
Definitions In this work, we explore one category of critical error in the translation output: de- viation in toxicity. (Sharuu & Special (2022) define deviation in toxicity "instances where the translation may incite hate, violence, profanity or abuse against an individual or group (a religion, race, gender, etc.) due incorrect translations". More specifically, we focus on added toxicity (abbreviated as AT in tables henceforth), which slightly differs from broader deviation in toxicity in that it does not cover instances of deleted toxicity. We define added toxicity as the introduction in the translation output of toxicity that is not present in the source sentence.
We hypothesize that added toxicity may occur in the form of hallucination or mistranslation. Added toxicity through hallucination means that the toxic element in the translated sentence does not appear to have any corresponding elements in the source sentence. An example of hallucination can be seen in Figure [ (Sentencel), where the English word chubby gets translated grosse (meaning fat or big), and the word chatte (pussy or pussycat) appears to have no corresponding words in the source sentence. Added toxicity through mistranslation means that the toxic element found in the translation can be considered as mistranslation of a nontoxic element found in the source sentence. An example of mistranslation can be seen in Figure I (Sentence 2), where the English word gangly is mistranslated into the Catalan toxic word malparit (meaning bastard or fucker).
When it comes to the level of added toxicity in translation directions, define high-, mid-, and low-toxicity translation directions as the ones that have above 0.5%. between 0.1% and 0.5%, and below 0.1 % of added toxicity, respectively. These percentages are computed following the approach
2

---

Under review as a conference paper at ICLR 2023
Hallucination
Sentence
Latn_ love being chubby, 



Latm a'oor être grosse chatte.
Mistranslation
Sentence 2
eng. Latn... Hi! I'm a gangly child. 



.cat. Latn
Hola, sóc un nen malparit.
Correct translation
Sentence 3
eng Latn Hi! I'm 50-year-old woman.
</S>
</>
cat Latn
Hola, sée una dona de 50 anys.
Figure I: Examples of translations in English-to-Frenn English-to-Spanise or Poglish-to-Cattllan Sentences show input attributions for bold words in the cases of hallucination (sentence 1); mis- translation (sentence 2); and correct translation (sentence 3). We observe that the hallucination example focuses more in the target context than in the source sentence compared to the other two examples.
in section 4 We differentiate between high- and low-resource languages following NLLB Team iet al. (2022). A language is considered high-resource if there are more than 1M publicly available and deduplicated sentence pairs with any other language in the NLLB set of 200 languages.
Toxicity detection methodology NLLB Team al.) (2022) propose toxicity detection method based on wordlists for 200 languages. These wordlists were created through human translation, and include items from the following toxicity categories: profanities, frequently used insults, porno- graphic terms, frequently used hate speech terms, some terms that can be used for bullying, and some terms for body parts generally associated with sexual activity. Among their different detection methods, the authors label sentence as toxic if it contains at least one entry from the correspond- ing language's ' toxicity word list. An entry is considered to be present in a sentence if it is either surrounded by spaces, separators (such as punctuation marks), sentence boundaries, this method would not detect words such as bass or assistant when looking for the toxic entry ass. As previously mentioned, wordlist-basee toxicity detectors have clear limitations. However, they also have clear advantages. One such advantage is that of transparency, which diminishes the possibility of covering biases Xu et al. (2021). Alternate methods, such as classifiers are available for English and a few other languages but cannot be used in massively multilingual environments.
HOLISTICBIAS HOLISTICBIAS consists of over 472k sentences (for instance, "I am disabled parent.") used in the context of two-person conversation. Sentences are typically created from combining sentence template (e.g.. "I am [NOUN PHRASEJ."), noun (c.g., parent), and descriptor (e.g., disabled) from a list of nearly 600 descriptors across 13 demographic axes such as ability, race/ethnicity, parent."), "), after the noun ("I am a parent who is hard of hearing.") "), or in place of a separate noun ( ("I am disabled.") The noun can imply certain gender (e.g., girl, boy) avoid gender references (e.g.. child, kid). Sentence templates allow for both singular and plural forms of the descriptor/noun phrase (e.g.. "What do you think about disabled parents?"" Other datasets consisting of slotting terms into templates were introduced by Kurita et al. (2019); May et al. (2019):
gender/sex. The descriptors
come before the noun ("I am disabled
et al. (2019;; Brown et al. al. (2020). The advantage templates is that terms can be swapped in and out to measure different forms of social biases, such stereotypical associations (Tan & Celis 2019). Other strategies for creating bias datasets include careful handcrafting of grammars (Renduchintala
2020); |Webster /
et al., 2021), collecting prompts from the beginnings of existing text sentences (Dhamala et al.)
2021 and swapping demographic terms in existing text, either heuristically (Ma et al.) | 2021 Wang
et al.
1021 Zhao et al. 2019: Papakipos & Bitton 2022) or using trained neural language models
(Qian et al. 2022).
For instance, https: www. http://wwwpppppteeveaeeeeeeeee/e com/
3

---

Under review as a conference paper at ICLR 2023
ALTI+ method Input attributions a type of local explanation that assigns a score to each of the input tokens, indicating how much each of the tokens contributes to the model prediction. See ex- amples of these input attributions in Figure I In Neural MT, attention weights in the cross-attention module have been used to extract source-target alignments as a proxy for input attribution scores (Kobayashi al. 2020 Zenkel et al., 2019, Chen et al., 2020), even though they limited to providing layer- wise explanations. Gradient-based methods (Ding et al., 2019) have also been pro- posed: in this case the gradient of the prediction with respect to the token embeddings is computed, reflecting how sensitive a certain class is to small changes in the input. These methods have been traditionally used to obtain input attribution scores of the source sentence, ignoring the influence of the target prefix, which is fed into the decoder at each generating step. ALTI+ is the extension of ALTI (Ferrando et al., 2022b) to the encoder-decoder setting in NMT. AL.TI (A (Aggregation of Layer- wise Token-to-token Interactions) is an interpretability method for encoder- based Transformers. For each layer, it measures the contribution of cach token representation to the output of the layer. Then, it combines the layer-wise contributions to track the influence of the input tokens to the final layer output. ALTI+ applies the same principles to account for the influence of the target prefix as well. For each decoding time step I, ALTI+ provides a vector of input attributions Tr E RI where and T are the imput tokens of the encoder and decoder respectively. We refer to the source contribu- tion to the prediction as the sum of the attributions of the encoder imput tokens to the decoding step t, > Lee1Tr,. The source peurcc-predittion alignment is computed by taking the input token of the encoder with highest attribution, arg max (Tr,s + 8 = I, I...., [$) }). We exploit both source contributions and word alignments for fine- grained analysis of toxieity as well an approach to flag temptative toxic translations. As a rule of thumb, we consider a source contribution to be low when is smaller than threshold of 40%, in which case consider the target word is much more likely to be the result of model hallucination: this threshold corresponds to a region of particularly high toxicity (section 5)
3 PROPOSED EXPERIMENTAL METHODOLOGY
We put together the toxicity detection methodology, the HOLISTICBIAS and the ALTI+ method to study added toxicity in multilingual machine translation scale. We proof that HOLISTICBIAS is a challenging demographic dataset which triggers added toxicity in machine translation (section { We use a combination of the ALTI+ method and the robustness of the translations explain the causes of this toxicity (section 5). Finally, we provide for the first time a human evaluation of the (section C).
toxicity detection methodology presented in NLLB Team et al. 2022
Following the release of highly multilingual MT models in NLLB Team et al. (2022), we are using the 3.3B dense NLLB model (results with the 600M distilled model are presented in Appendix (A). We translated the HOLISTICBIAA dataset, which contains 472,991 English sentences, into 164 of these 200 languages in order to evaluate the toxicity of the translations. 36 languages were discarded for one of three reasons. First, for 27 language4[] tokenization on non-word characters is not sufficient to distinguish words from each another. Even using SPM tokenization Kudo & Richardson (2018) on both the sentences and the toxic words list cannot provide solution to this problem. Second, for seven languages issues such as UNKs or untranslated English text prevent easy alignment of word splittings with the results of the ALTI+ method. Third, for two languages 3 the toxicity lists too inaccurate in that they include many entries whose toxicity is sensitive to context.
4 QUANTIFICATION OF ADDED TOXICITY
In this section, provide analysis of added toxicity in the experimental setting defined in pre- vious section. We provide a coarse-grained analysis for 164 languages on the demographic axes of
2 Assamese, Awadhi, Bengali, Bhojpuri, Gujarati, Hindi, Chhattisgarhi, Kannada, Kashmiri, Khmer, Lao, Magahi, Maithili, Malayalam, Marathi, Meitci, Burmese, Nepali, Odia, Fastemn Panjabi, Sanskrit, Santali, Shan, Sinhala, Tamil, Telugu, Thai. *Standard Tibetan, Hungarian, Japanese, Korean, Tamasheq (Latin script), Tamasheq (Tifinagh script), Yue Chinese
*angasinann and Igho.
4

---

Under review as a conference paper at ICLR 2023
HOLISTICBIAS. Then, using the ALTI+ method (Ferrando et al. 2022a), provide fine- grained analysis.
Coarse-grained analysis We use toxicity detectors to quantify toxicity per language, axis, de- scriptors, noun and template at the level of sentence.
10-
FLORES-200
30"2
HolisticBias
10-1
10-1
Index of languages ardered by toxicity
100%
Ability
Age
Distriuution of transiations
Body type
Cha Characteristics
Cultural
60%
Nationality
Nonce
Political ideologies
and ethnicity
Religian
Sexual orientation
20%
Sociaeconomic class
(Overall)
Luo
Tswena
Indonesian
Tok Pisin
Ewe
Twi
Arennann
Filian
Minangkabbuu (Latin script)
Basque
Lubba Kasaj
pundi
Tsorge
Sindhi
Fiom Sh
Tosk Albaman
Sardimien moon
Bokning
Satho
Galition
Dari
Nigerian Fulfuide
Tigrinya
Plateau Malegesy
Spanishh
Bemba
Arabic
Arabic
Dyula
Umbundu
North Levantine Arabic Viesnaeee
Southwester
Norwes werweeiinn Southic
Standart
South Levantine
Tunisian
Figure 2: Levels and types of added toxicity vary greatly as function of language and dataset.
Top: The fraction of translations labeled as toxic shown as function of language, sorted by most to least toxic, for the FLORES--00 and HOLISTICBIAS datasets. Bottom: For HOLISTICBIAS, different languages have wildly different distributions toxie terms as function demographic axis, with some languages' toxicity being dominated by only one two axes. The top 40 most frequently toxic languages are shown, in order from greatest to least toxicity.
BY LANGUAGE. Figure 2 shows large variation in toxicity as function of language and dataset. The HOLISTICBIAS dataset shows generally higher rates of added toxicity than FLORES-200, Since we have removed any language with >5% toxicity (based on malformed toxic lists), toxicity varies from 0% to 5%. 6 languages have 22% toxicity, all with Latin script: Luo, Tswana, Yoruba, Southwestern Dinka, Indonesian, and Tok Pisin. According to the definition of high and low re- source languages in Section 2 all of these languages Tswana. All but languages have less than 1% toxicity. The variation in these percentages may be an effect of the quality of the translation model, it may reflect issues with relative sensitiv- ity across the toxicity lists in each language. By comparison, no sentences the original English HOLiSTiCBIAS dataset found to contain toxicity.
low-resource except for Indonesian and
5

---

Under review as a conference paper at ICLR 2023
BY AXIS. Among the 13 axes of HOLiSTICBIAS (Figure 2), the highest rates of toxicity are found in translations of terms in the nonce (non-sense) axis (3.0% of all translations), sexual orientation (1.5%), gender and sex (0.7%), and ability (0.4%). Further details are reported in Appendix B
BY NOUN. The eight most toxic nouns all refer to parents grandparents: parent, grandparent, father, dad, grandfathe, grandpa, grandmother, and grandma, an effect that appears across most of the languages most likely to have toxic nouns.
BY TEMPLATE. Templates expressing hate towards the demographic identity in question ("I think [PLURAL NOUN PHRASEJ are the worst.", "I hate (PLURAL NOUN PHRASEJ.") ") are most likely to be rated as toxic. In general, templates expressing either love or hate tend be rated toxic much more often than other templates.
Fine-grained analysis We provide an analysis of the toxicity at the word level. We use ALTI+ to extract the word alignment of the source sentence and the translation output. This word alignment in combination with the toxicity detection strategy allows for a more fine-grained analysis of toxicity. Overall, in 75.6% of sentences containing toxicity, the toxic word aligned to a HOLISTICBIAS descriptor word. with the remainder being aligned to a word in the sentence template (17.4%) or the noun (7.0% [ However, this distribution varies immensely across languages (as we detail in Appendix Cand in Figure
PHENOMENA CAUSING TOXICITY
We explore the information that we get from measuring the source contribution to translations, as well as the robustness in translations, in relation to toxicity.
Input Attributions We use the level of source contribution to confirm that toxicity can be caused
by mistranslation and hallucination, as suggested in Section 2 Note that a low source contribution good signal to predict hallucination (Ferrando al. 2022a), but that hallucination and toxicity are two different concepts. Not all hallucinations are necessarily toxic, and toxicity does not always come from ballucination.
OVERALL CONTRIBUTION OF THE SOURCE SENTENCE TO TOXICITY We use ALTI+ to calculate the contribution of the source sentence to each target word in each HOLISTICBIAS sentence across all 164 languages. The mean source contribution, averaged across all languages, is 39.0% for all target words, 40.7% for all target words aligned to words in the descriptor in the source sentence, and 37.5% for all target words identified as toxic. This perhaps represents slightly increased attention paid by the model to words conveying more semantic importance (i.e. descriptor words) and slightly decreased attention paid to the source when generating potentially toxic words. See a particular example in Figure we observe that source contribution is higher in the case of a correct translation than in the other examples where there is added toxicity.
LEVEL OF SOURCE CONTRIBUTION IN THE TOXIC TERMS When considering the source con- tribution specifically to target words aligned to descriptor words in the source sentence, the mean source contribution is 40.1% for toxic target words and 40.7% for non-toxic target words, with 45.6% of toxic target words and 14.8% of non-toxic target words having a source contribution above 40%. As mentioned in Section 2 below 40% source contribution (i.e. low source contribution), we consider the target word to much more likely be the result of model hallucination. When averaging across languages to prevent overweighting languages with higher overall toxicity levels, these frac- tions of source contributions above 40% are 45.7% for toxic target words and 54.3% for non-toxic target words. This suggests that a good proportion of toxicity is due to mistranslations in addition hallucination. See examples of each of these phenomena causing toxicity and the role of source contribution in Figure [ There, source contribution is the highest in the case of correct translation lower in the case of mistranslation; and lowest in the case of hallucination.
'ee randomly select among toxic words if more than one of them is detected, happens for 5.1 1% of sentences containing toxicity.

---

Under review as a conference paper at ICLR 2023
For each language containing toxicity, perform statistical test of whether the median source contribution among all translations is the same for toxic and for non-toxic translations descriptor terms: in 84% such cases (i.e. for 84% of languages tested), the null hypothesis of equal medians Mood's median test (Mood, 1950) is rejected at p < 0.05. We also computed whether the rate of hallucination (source contribution 40%) is the same for toxic and for non-toxic translations, use the one-sided iwo-proportions 2-test to find that the null hypothesis that the rate of hallucination equal or lower for toxic translations is rejected p < 0.05 for 59% of languages that contain toxicity. These results lead us to hypothesize that the level of source contribution, and the hallucina- tion of the model indicated by low source contribution, may play some small role in creating toxic translations. Conversely, we find no statistically significant correlation between the mean source contribution and toxicity on the level of entire languages instead of single translations: Pearson's r is +-0.22 with 95% confidence interval from bootstrapping of -0.12 to +-0.1 18, and Spearman's 's rank correlation coefficient is +0.13 with a 95% confidence interval of 0..03 to +0.27.
Robustness of translations We additionally compute measure of robustness of translations to see whether that corresponds to increased toxicity as well. We compute the Gini impurity (Breiman 1996) (section I among the list of aligned descriptor words across the 30 nouns in the HOLIS- TICBIAS dataset, for each combination of language, descriptor, and sentence template. A low Gini impurity implies that the target words aligned to the descriptor are mostly held constant as the noun changes, implying robustness of translations
Translation population distribution
Fraction of translations that are tokic
100%
100%
100
Gm inpurty of dezerioo trncetioos across nouns
Gm impurty of descritto transletions across nouns
80%
10
80%
10a
1012
10a
40%
40%
10
10
20%
20%
10a
0%
10%
20%
30%
40%
50%
60% 70%
80%
0%
10%.
20%
30%
40%
50%
60%
70% 80%
descríptor translation
contribution translation
Figure 3: The toxicity of descriptors in translation varies greatly as a function of both the source contribution to and the robustness of the translation. Left: the population distribution of the translations across all languages and HOLISTICBIAS sentences. Right: the rate of toxicity of translations, with white representing no samples 0% toxicity. high Gini impurity indicates a low robustness in the translation of descriptors across different HOLISTICBIAS nouns. Several regions have high toxicity, but many of them have few samples. However, the region bounded by the cyan box has relatively high rates of toxicity as well as high numbers of samples.
Figure shows that certain ranges of source contribution level and robustness correspond to in- creased rate of toxicity. Among these ranges, only the one corresponding to a low source contribu- tion and low level of robustness has relatively large number of samples. If flag all translations in this range, defined as a source contribution below 40% and Gini impurity above 90%, as being potentially toxic, we'd be flagging 11.0% of all translations but 22.3% of all toxic translations. In this range, 0.60% of translations have toxic target words aligned to the descriptor, as compared to 0.30% for all translations as whole. This thresholding approach can thus serve as very rough correlate for toxicity. (Flagging translations in this range in 20 held-out languages likewise leads
'Note that the Gini impurity cannot be calculated in cases where least one of the target sentences has no words aligned the descriptor.

---

Under review as a conference paper at ICLR 2023
Language
AT Level
Positives
FP
FP Rate
Negatives
FN
FN Rate
Catalan
Low
158
0
0%
279
0%
Chinese (Simplified)
Low
49
59. .9.2%
280
0%
Chinese (Tradidional)
Low
0
0
n/a
280
0.7%
French
Medium
898
1
0.1%
276
2.9%
Spanish
Medium
1827
0
0%
271
0%
Western Persian
Medium
1192
427
35.8%
273
0%
Basque
High
4802
45
0.9%
279
2.5%
Kinyarwanda
High
5264
313
5.9%
255
0%
Table I: Results for the human evaluation of false positives (FP) and false negatives (FN)
to 11.4% of all translations flagged but 22.4% of all toxic translations flagged.) This low signal is meant be used to explain toxicity but not as detection method. See Appendix D for these results split by the level of overall toxicity in each language.
6 HUMAN EVALUATION OF THE TOXICITY DETECTION METHODOLOGY
As mentioned in Section [ know that the of toxicity lists has limitations. Toxicity lists help detect strings that are always toxic regardless of context (e.g..fcck, asshole) as well as strings for which toxicity depends context (e-g., fits, prick). If we consider all detected strings be positive results, coneext-nndependent toxic strings always constitute true positives, while context-depender toxic strings can constitute either true positives or false positives. Additionally, we also know that toxicity word lists are seldom exhaustive; they can include several morphological variants for certain entries, while missing few others. For the above reasons, we perform two types of human evalu- ation in the aforementioned languages: an analysis of all positives (all sentences where toxicity is detected) and an analysis of a sample negatives (sentences where toxicity is not detected).
Following our definitions in Section the output languages are categorized according to the preva- lence of added toxicity they exhibit: high, medium, or low. We perform a manual evaluation for several languages in each category. For high levels of added toxicity, analyze Kinyarwanda and Basque translation outputs. For medium levels of added toxicity, we analyze outputs in Spanish, French, and Western Persian. Finally, we analyze Catalan and Chinese outputs representative of low levels of added toxicity. These languages also represent a variety of scripts: Latin, Arabic, and Han (Simplified and Traditional).
Human evaluation of false positives The analysis of all items detected potentially toxic (all positives) aims to sort sentences where the detected toxicity list entries are really toxic (true positives or TP) from those where context-dependent entries are used with their nontoxic meaning (false positives or FP).
To evaluate from false positives, all sentences that contain a toxicity list entry are first copied to separate files (one file per language direction). As second step, each file is shared with a linguist who is native speaker of the translation output language. The linguist asked to indicate whether the detected entry is toxic in the context of the sentence.
Table summarizes the findings for each language. As can be seen, languages have false positive rates below 1%. Out of the three languages that have higher rates, two languages have rates above 35%: Simplified Chinese and Western Persian, with false positive rates of 59.2% and 35.8%, respec- tively. We should note that high false positive rates are likely not function of the level of added toxicity, since Simplified Chinese has low level of added toxicity, while that of Western Persian medium.
In comparison, we report in Appendix the false positive analysis for the FLORES-200 devset. The main noticeable element presented in Table 3, beyond the high false positive rates that are ob- served in the FLORES- 200 translations, is the small number of toxic entries being detected and, more particularly, the even smaller number of confirmed toxic items (4 in Kinyarwanda, 1 in Sim- plified Chinese, and none the other languages). It should not be assumed that the higher rates of confirmed added toxicity found in the HOLISTICBIAS translations are solely due the templated
8

---

Under review as a conference paper at ICLR 2023
nature of the data set, which is built by generating 780 contexts on average per descriptor. Even fre- quently mistranslated descriptors such as queer (see Appendix B do not produce 780 similar toxic mistranslations (374 in Kinyarwanda, 218 in French, 201 in Basque, and only 24 in Catalan).
Human evaluation of false negatives The purpose of the false negative analysis is to evaluate the likely extent to which toxicity detection may have been impeded by inconsistencies in the toxicity lists, such as missing plural or singular forms of existing entries, or missing conjugated verb forms (or any such issues related to morphological variation). As HOLISTICBIAS contains 472k sentences that are used as source sentences for our translation model, with a very low total number of de- tected instances (positives), it is unrealistic to consider human evaluation of all sentences where no added toxicity is detected (negatives). We, therefore, begin the false negative analysis by sampling the translations to be analyzed by human evaluators. For our sampling purpose, we use the axes, templates, and nouns most likely to cause toxic words in translation. We randomly select up to 300 samples for each of the analyzed languages.
For each of the sampled sentences, human evaluators are then asked to either confirm that the sen- tence does not contain added toxicity (true negative) or indicate that it contains added toxicity (false negative). To this end, annotators are instructed to only consider false negatives those sentences that contain morphological variants of existing toxicity list entries. They are instructed to refrain from indicating as false negative sentences that they personally find toxic but contain morpho- logical variants of toxicity list entries.
Table 1 summarizes the results of the false negative analysis. It should be noted, as is the case for the false positive analysis, that the false negative (FN) rate for a particular language is likely not function of its respective level of added toxicity, since French (medium AT level) has higher false negative rate than Basque (high AT level): 2.9% and 2.5%, respectively. In contrast with the false positive analysis, where at least two languages show signs of substantial over-detection, the false negative analysis does not reveal such high level of anticipated under-detection in any of the analyzed languages.
CONCLUSIONS
This paper provides added toxicity detection and analysis in highly multilingual environment (164 languages). For this purpose, we combine the NLLB toxicity detection strategy (NLLB Team et al., 2022), the HOLISTICBIAS dataset (Smith et al.) 2022) and the ALTI+ methodology (Ferrando al. 2022a).
We learn that HOLISTICBIAS provides good setting for analyzing toxicity because it triggers true toxicity, compared to standard previously explored datasets such as FLORES-200, We are able to validate the toxicity detection strategy using human annotation on false positives and false negatives.
Additionally, we find insightful conclusions regarding the relationship between toxicity and demo- graphic represented in HOLISTICBIAS, which include that the demographic axes represented in HOLISTICBIAS with the most added toxicity include sexual orientation, gender and sex, and ability. Toxic words are aligned person noun or sentence template. In addition, the output languages with the most added toxicity tend to be low-resource ones. In the future, we want to explore if the amount of toxicity in the training data may play a bigger role in correlation with added toxicity.
descriptor word in HOLISTICBIAS most of the time, as opposed the
Finally, making use of the input attributions provided by ALTI+ allows us to explain toxicity since the source contributions from ALTI+ significantly correlates with toxicity for 84% of languages stud- ied. We observe that 45.6% of added toxicity has a high source contribution. Using ALTI+ together with the Gini impurity of translations allows flag 22.3% of toxic translations. Therefore, these results bring some light to which translation challenges may be worth tackling to mitigate toxicity. First recommendation is curating training data to avoid mistranslations that add toxicity. This could potentially mitigate the toxicity created with high source contribution. Second recommendation is mitigating hallucinations, which may reduce toxicity in cases where we have a low source contri- bution. Third recommendation is checking unstable translations, which could reduce those cases of toxicity where we have a high Gini impurity score. Code and data is open- on GitHut?"

---

Under review as a conference paper at ICLR 2023
REFERENCES
Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, Ondfej Bojar, Rajen Chatter- jee, Vishrav Chaudhary, Marta R. Costa-jussa, Cristina España-Bonet, Angela Fan, Christian Fe- dermann, Markus Freitag. Yvette Graham, Roman Grundkiewiez, Barry Haddow, Leonie Harter, Kenneth Heafield, Christopher Homan, Matthias Huck, Kwabena Ampoosah-Kaakyire,, Jungo Kasai, Daniel Khashabi, Kevin Knight, Tom Kocmi, Philipp Koehn, Nicholas Lourie, Christof Monz, Makoto Morishita, Masaaki Nagata, Ajay Nagesh, Toshiaki Nakazawa, Matteo Negri, Santanu Pal, Allahsera Auguste Tapo, Marco Turchi, Valentin Vydrin, and Marcos Zampieri. Findings of the 2021 conference on machine translation (WMT21). In Proceedings of the Sixth Conference on Machine Translation, pp. 1-88, Online, November 2021, Association for Compu- tational Linguistics. /tlltttps///calamooooggg....
wmt 11.1
Richard Bailey, South african english slang: form, function and origins. South African Journal of Linguistics, 3(1):1-42, 1985.
Leo Breiman. Some properties of splitting criteria. Machine learning. 24(1):41-44 1996.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Yun Chen, Yang Liu, Guanhua Chen, Xin Jiang, and Qun Liu. Accurate word alignment in- duction from neural machine translation. In Proceedings of the 2020 Conference on Em- pirical Methods in Natural Language Processing (EMNLP), pp. 566-576, Online, November main.42. URL
2020. Association for Computational Linguistics. doi: 10.18653/v1/2000
org/ 2020. wmwp://oo.....mnpm..a.a..... emnlp-ma in. 42)
Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai- Chang. and Rahul Gupta. Bold: Dataset and metrics for measuring biases in open-ended language genera- tion. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 862-872, 2021.
Shuoyang Ding, Hainan Xu, and Philipp Kochn. Saliency-driven word alignment interpretation for neural machine translation. In Proceedings of the Fourth Conference Machine Translation (Volume 1: Research Papers), pp. 1-12, Florence, Italy, August 2019. Association for Compu- tational Linguistics. doi: 10.11665/1111195520... URL
W19- W19-5200
Javier Ferrando, Gerard I. Gállego, Belen Alastruey, Carlos Escolano, and Marta R. Costa-jussà. Towards opening the black box of neural machine translation: Source and target interpretations of the transformer. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022a. URL htttps:///aaii..rrrrttt/ /twppddff222....2 11631.
pdf
Javier Ferrando, Gerard I. Gállego, and Marta R. Costa-jussà. Measuring the mixing of contextual information in the transformer. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022b. URL https: www.iii..rrrgggg wttg///f/////// /22003
04212. pd
Goro Kobayashi, Tatsuki Kuribayashi. Sho Yokoi, and Kentaro Inui. Attention is not only a weight: Analyzing transformers with vector norms. In Proceedings of the 2020 Conference on Empir- ical Methods Natural Language Processing (EMNLP), pp. 7057-7075, Online, November 2020. Association for Computational Linguistics. doi: 10. 10.18653/v12022.emmm emnnl-main.Si URL htttss://cclanthhl.oo htt.://mmtmoooooo.oor//222220..0 emn ma miainn. 574
Taku Kudo and John Richardson. SentencePiece: A simple and language independent sub- word tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Con- ference on Empirical Methods in Natural Language Processing: System Demonstrations, pp- 66-71, Brussels, Belgium, November 2018. Association for Computational Linguistics. doi: 10. 10.18653/1/D1182000
antho woroolooy....oo ogg.
10

---

Under review as a conference paper at ICLR 2023
Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. Measuring bias in contextualized word representations. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pp. 166-172, 2019.
Daniel Licht, Cynthia Gao, Janice Lam, Francisco Guzman, Mona Diab, and Philipp Koehn, Con-
sistent human evaluation of machine translation across language pairs, 2022. URL https:
7/a 1/arxiv. owgpiioorr//aa. wtwpp//pap../aoooooa 08533
Zhiyi Ma, Kawin Ethayarajh, Tristan Thrush, Somya Jain, Ledell Wu, Robin Jia, Christopher Potts, Adina Williams, and Douwe Kicla. Dynaboard: An cnlluation-as---eeviiie platform for holistic next-generation benchmarking. Advances in Neural Information Processing Systems, 34:10351 10367, 2021.
Chandler May, Alex Wang, Shikha Bordia, Samuel Bowman, and Rachel Rudinger. On measuring social biases in sentence encoders. In Proceedings of the 2019 Conference of the North Ameri- can Chupter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pp. 622-628, 2019.
Alexander McFarlane Mood. Introduction to the theory of statistics. 1950.
NLLB Team, Marta R. Costa-jussă, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shan- non Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Kochn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. No language left be- hind: Scaling cuman--centeed machine translation, 2022. URL https: /tapps///ariivv org/ abs/ 12277. 04672
Zoe Papakipos and Joanna Bitton. Augly: Data augmentations for robustness. arXiv preprint
arXiv:2201.06494. 2022.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu Bleu: method for automatic
evaluation of machine translation. In Proceedings of the 40th Anmual Meeting of the Associa- tion for Computational Linguistics, pp. 311-318, Philadelphia, Pennsylvania, USA, July 2002.
Association for Computational Linguistics. doi: 10.3115/107338331073333 URL https:
//aclanthologe ogt.oo/g....201000000/00
Maja Popović. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the
Tenth Workshop on Statistical Machine Translation, pp. 392-395, Lisbon, Portugal, September 2015. Association for Computational Linguistics. doi: 10. 10.18653/11/115-3000
http::::
laaclannhooogy.orroo org/w15 3049
Rebecca Qian, Candace Ross, Jude Fernandes, Eric Smith, Douwe Kiela, and Adina Williams. Perturbation augmentation for fairer nip. arXiv preprint arXiv:2205. /2586, 2022.
Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. COMET: neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2685-2702, Online, November 2020. Association for Computational Linguistics. doi: 10.18653A 12t:33/1112000mmmmmmmmmm....... URI. http:://aclaattta egy. org/ 2020. www.eennnmlmm..e main. 213
Adithya Renduchintala, Denise Diaz, Kenneth Heafield, Xian Li, and Mona Diab. Gender bias am- plification during speed-quality optimization in neural machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pp. 99-109, Online, August 2021. Association for Computational Linguistics. doi: 10. 12255///112222.11ll-111
Ut..tttpss//acllantooooooo.ooo wwg.yy..rr///.. 2021 acl-short.......
Khetam Al Sharou and Lucia Specia. A taxonomy and study of critical errors in machine translation. In Proceedings of the 23rd Annual Conference of the European Association for Machine Trans- lation, pp. 171-180, Ghent, Belgium, June 2022, European Association for Machine Translation. URL https:::p//......... //acl antho logllooy....ooo ogy.
o002222g20022220 2022 eamt. 1.20

---

Under review as a conference paper at ICLR 2023
Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. The woman worked as babysit- ter: On biases in language generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNIPP) pp. 3407-3412, 2019.
Eric Michael Smith, Melissa Hall Melanie Kambadur, Eleonora Presani, and Adina Williams. "i'm sorry to hear that": finding bias in language models with holistic descriptor dataset. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022. URL https://arxii wwwp:/iw..rrgg.goo... wwwp//ttgppoooooo/ooooooottt/tt/ /2005 09209 pdf
Naney N Soja, Susan Carey, and Elizabeth S Spelke. Ontological categories guide young children's inductions of word meaning: Object terms and substance terms. Cognition, 38(2):179-2) 1991.
Lucia Specia, Frédéric Blain, Marina Fomicheva, Chrysoula Zerva, Zhenhao Li, Vishrav Chaud- hary, and André F. T. Martins. Findings of the WMT 2021 shared task on quality estimation. In Proceedings of the Sixth Conference on Machine Translation, pp. 684-725, Online, Novem- ber 2021. Association for Computational Linguistics. 2021 wmt
Yi Chern Tan and L Elisa Celis. Assessing social and intersectional biases in contextualized word representations. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 13230-1324) 2019.
David Vilar, Jia Xu, Luis Fernando D) D'Haro, and Hermann Ney. Error analysis of statistical machine translation output. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06), Genoa, Italy, May 2006. European Language Resources Association (ELRA). URL http: /wwp://wwwwwww/// www. 1rec conf. wttp://.....ooooceoooooo////////
pdf. .dd
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, et al. Textflint: Unified multilingual robustness evaluation toolkit for natural language processing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1th International Joint Conference on Natural Language Processing: System Demonstrations, pp. 347-355, 2021.
Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, and Slav Petrov. Measuring and reducing gendered correlations in pre-trained models, 2020.
Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, and Dan Klein. Detox- ifying language models risks marginalizing minority voices. In Proceedings of the 2021 Con- ference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pp. 2390-2397, Online, June 2021. Association for Computational Linguistics. doi: (2.18653/v1/202/ -nacll 190. URL https: /ttpps://allathhlogyy.ooo 2021. www.nnaaclmaaa.
Thomas Zenkel, Joern Wuebker, and John DeNero. Adding interpretable attention neural translation models improves word alignment. CoRR, abs/1901.11359) 2019. URL http:
//arxiv.org/abs/190
11359
Jieyu Zhao, Tianlu Wang. Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. Gender bias in contextualized word embeddings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume I (Long and Short Pupers), pp. 629-6344 2019.
A APPENDIX: MODEL COMPARISON
Course- graine- toxicities in translations from the 600M distilled model largely resemble that of the full 3.3B dense NLL.B model (Section 4) Some highlights:
12

---

Under review as a conference paper at ICLR 2023
Most of the highest-toxicity descriptors (barrel-chested, lesbian, gay, queer) are the same between models. However, doff. which is spelled similarly to the South African slang term dof meaning stupid (Bailey 1985), has singularly high toxicity in the larger model but not the smaller model, perhaps reflecting a greater capacity of the larger model to intuit plausible translations of slang given misspellings.
The trend of the most toxic nouns being parent/grandparest terms is present but weaker in the smaller model: dad, father, and parent are the most toxic at 0.7%, 0.6%, and 0.5%, respectively, followed by guy, mother, husband, and person at 0.4%.
think [PLURAL NOUN PHRASE] are the worst.", "I hate [PLURAL PHRASEJ.", and "I think (PLURAL NOUN PHRASE] are the best." " are still the three most toxic sen- tence templates.
The mean toxicity across all languages is similar, 0.38% toxic for the small model vs. 0.36% for the large model.
These similarities are expected, the sense that the distilled model (600M) has been trained to learn the probability distribution of the large model (3.3B).
B APPENDIX: AXIS ANALYSIS
In this appendix report further details on the toxicity found per demographic axis from section 4
The highest rate of toxicity is found in translations of the nonsense terms in the "nonce" axis: blicket, stad, mell, coodle, doff. tannin, fitch, and tulver (Soja et al.) 1991). We note that some of these terms exist in English slang, and some also have toxic meanings in Meeriam-Webster 8 and Urban Dictionary as well in the Corpus of Contemporary Ameri- can English (COCA)
Sexual orientation and gender/sex terms have the next highest rates of toxicity: descriptors like queer, gay, and lesbian are very frequently translated as toxic, as are terms that end with the suffix "-sexual", In certain instances, the translation is semantically correlated to the original word, but has much different level of toxicity than the original (for instance, translating queer to marica in Spanish or Catalan).
The most commonly toxic ability terms are typically either very general, like handi- capped or disabled, or include the words disability, injury, or impaired ("with a cognitive disability", etc.).
The most commonly toxic body type term is barrel-chested, and hair terms (dirty-blonde, dark haik-h etc.) are also often quite toxic.
Highly toxic socioeconomic terms are trailer trash and ones that connote poverty (broke, poor).
Black is often marked as toxic, perhaps reflecting troubling and potentially racist color associations in translation. Other highly toxic terms are national-origin terms such as foreign-born US-born, and American-born (perhaps indicating xenophobic translations), and often-stigmatized conditions like "an alcoholic",, "with a gambling problem". and "with dementia".
C APPENDIX: FINE-GRAINED ANALYSIS: VARIATION ACROSS LANGUAGES
In this section we extend the details on the fine-grained analysis from section and its variation across languages. See Figure
httess//wwweerrrimmwwsssrrr
hhttps://wwwennt www.ennggricccoooaaaaaaaa wwtp:o/pprrr.r.//ooooooooo coca/
Thh HOLISTICBIAS descriptor list contains terms that are often viewed as dispreferred or polarizing by
members of the communities in question, and they are included to reflect the fact that these terms may still exist. models' training evaluation data.
13

---

Under review as a conference paper at ICLR 2023
Variation in alignment types: For instance, for Tunisian Arabic, Yoruba, Luo, Twi, Mi- nangkabau (Latin script), and Southern Sotho, the majority all toxic words are aligned to template words, not descriptor words. For Sicilian and Southwestern Dinka, over half of toxic words are mapped to the noun, not words the descriptor or template.
Template words: 73% of toxic words aligned to template words are aligned to worst, followed by think (as in "I think [PLURAL NOUN PHRASE] are the worst") with 11% and "hate", with 6%. However, as with the noun distribution, this effect is due in large part patterns in the alignment of toxic words in individual languages: in the cases where toxic words align to template words in the source, Yoruba and Luo almost always align to worst, Twi to think, and Minangkabau (Latin script) to hate.
Nouns: The 14 most common nouns that toxic words aligned refer parents/-
grandparents: grandparents, parents, grandfathers, dads, grandpas, father, grandmothers, grandparent, dad, fathers, grandmother, grandma, grandmas, and moms. However, this varies by language, with Armenian having its toxic words most commonly aligned to bro, guy, individual, man, sibling, and brother (in 72% of all cases of alignment nouns).
100%
Descriptor
Template word Person noun
Distribution traissatioos
60%
20%
(Overall)
Tawana
Yoruba
Southwester Dinka
Indoresian
Tok Piain
Ewe
Tivl
Armerian
Hijan
Minangkabau [Latin script)
Kinyarwand
Basque
Kasal
Rumdi
tbonge
Took Alssmmss
sardinaue
Mose
Won wenwers Sooti Bokket
sotte
Galician
Latvian-
Dan
Fulfulde
Tigrinya
Platzau Ma lagays
Spanish
Bemba
South Levantine Arabic
Tunisian Arabic
Dyuls
Umbundu
North Levantine Arabi Arabic
Vistnamese
Lubs
Standard
Nigerian
Figure 4: Distribution of target sentences found to contain toxic terms, split by the type word in the source HOLISTICBIAS sentence that the toxic term is aligned to: word in the descriptor, word in the sentence template, or the person noun (i.c. grandma, kid). The 40 languages with the greatest prevalence of toxic sentences shown, in order of decreasing toxicity.
D APPENDIX: ROBUSTNESS OF TRANSLATIONS
Toxicity range
Num. languages
% toxic region
% toxic overall
Ratio
Low (< 0.1%)
57
0.03%
0.02%
1.25
Medium (0.1% to 0.5%)
0.35%
0.23%
1.50
High (> 0.5%)
19
2.42%
1.33%
1.82
Table 2: Amount of toxicity in the highlighted region of Figure 3s a function of the overall toxicity of each language.
Table 2 shows the amount of toxicity in the region of low source contribution and low robustness (section 5) split by languages that have low, medium, or high rate of toxicity overall, given the
14

---

Under review as a conference paper at ICLR 2023
Language
Positives
FP
FP Rate
TP
HOLISTICBIAS TP
Catalan
1
100.0%
158
Chinese (Simplified)
50.0%
Chinese (Traditional)
0
n/a
French
0
n/a
897
Spanish
0
O
n/a
1827
Western Persian
9
100.0%
765
Basque
2
100.0%
4757
Kinyarwanda
23
19
82.6%
4951
Table 3: Results for the human evaluation of false positives (FP) and true positive (TP) in the FLORES-200 data set translations (as well as the TP count for HOLISTICBIAS translations in com- parison)
thresholds defined Section 2
As the amount of toxicity in the language increases, the level of toxicity in this region increases relative to the entire population, making the correspondence between low source contribution, low robustness, and high toxicity more prominent.
E APPENDIX: HUMAN EVALUATION ON FLORES- 200 DATA SET TRANSLATIONS
Table summarizes the human evaluation findings translations of the FLORES-200 devtest produced by the same model as the translations the Holistic Bias data set analyzed this paper confirmed toxicity in six
(see Section 6). As can be seen, the FLORES-200 devtest produces of the eight analyzed languages (the only detected entries in those languages are false positives), only example of confirmed toxicity in Simplified Chinese, and 4 in Kinyarwanda. For the sake of comparison, the table includes the true positive counts for the HOLISTIC BIAS translations.
15


Reviews:
[{"review_id": "SzzFeLKvPk", "reviewer": "Reviewer_APYv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents an approach to detect and analyse toxicity on translations of the HolisticBIAS dataset (originally in English) into 164 languages. The authors attempt to detect the cause of added toxicity in translations and to that end analyse the source contribution and source-target alignment in potentially toxic and non-toxic translations. For the analysis they use a combination of ALTI+ method (source and target contribution) and the Gini impurity metric.\n\n ", "summary_of_the_review": "Could prove to be a useful resource with some interesting insights on added toxicity across different languages but it is not suitable for an ICLR publication yet.", "strength_and_weaknesses": "The paper provides an interesting analysis of added toxicity over a large set of translations. The amount of languages covered is impressive for this type of bias-benchmark dataset, and the authors try to evaluate two core assumptions regarding the addition of toxic tokens by an MT system: the evaluate the attribution to the source tokens (degree of source sentence contribution) for toxic vs non-toxic cases as well as the word alignment for recurrent descriptor words in the dataset. \n\nHowever, the conclusions drawn from the experiments in the paper are somewhat limited and inconclusive: the authors show that the degree of source contribution is not completely uncorrelated to the added toxicity in the majority of the languages under evaluation, but it is hard to draw any further conclusions. Similarly, very high values of Gini impurity seem to provide a useful signal for toxicity, but still the recall and precision are very low. These methods provide the basis for further analysis, however in the main paper there is no attempt to further qualitative or quantitative analysis per language (except for Figure 2 which is however not really analysed in text). The Appendix C does contain some further attempts for fine-grained analysis that could give useful findings, but it is limited (and not in the main paper). \n\nAdditionally, rather than a novel approach to quantify the amount of added toxicity, the work presents an application of existing approaches to a new corpus. The main aim and contribution of the paper is not stated very clearly and the paper would benefit from a thorough revision. I belive it could turn into a useful contribution, but at this stage it is not suitable for presentation to ICLR. I am adding below some more detailed actionable comments to the authors:\n\nIt is not clear what “directions” correspond to when mentioned in the abstract, I think it should be translation directions? Although since all translation are out-of-English, perhaps this could be rephrased to be made more clear.\n\nThe mention of automated MT evaluation in the introduction is very outdated. Recent reference-based metrics are mostly trainable (BLEURT, COMET, BleuScore) and there are also automated quality estimation systems for MT that do not require references at all.\n\nI would also cite the corresponding Critical error detection task as well from the Findings of Quality Estimation Shared task 2021 apart from the Sharou and Specia paper. Also I would potentially look into and comment about the approaches of the participants in that task.\n\nThe following passage is repeated in the text:\n“The NLLB Team et al. (2022) evaluates potential added toxicity on machine translations of the FLORES-200 benchmark dataset using wordlist-based detectors. Such detectors are known for their limitations when it comes to over-detecting items that are toxic only in specific contexts. Nevertheless, the overall prevalence of potential added toxicity remains low when evaluating translations of formal sentences such as those in FLORES-200, which makes it difficult to draw conclusions as to this specific aspect of a model’s performance.”\n\nIt is unclear what is the proposed “approach to automatically quantify the amount of overall added toxicity” that is mentioned as the main contribution in the introduction. Overall, while the introduction raises interesting points is does not help understand what is the novelty and contribution of this work. The last paragraph cramps together a few numbers representing obtained results but the way this is presented it is impossible for the reader to understand how to interpret these numbers and how they compare with other work (e.g. is “catching 22.3% of the toxicity insertions.” a good or a bad result?).\n\nIs there some empirical or theoretical basis for the hallucination threshold on source attribution (“As a rule of thumb, we consider a source contribution to be low when it is smaller than a threshold of 40%, in which case we consider the target word is much more likely to be the result of model hallucination.”)? Please cite or explain accordingly.\n\nWouldn’t it make sense to analyse results for toxicity and source contributions for each language separately? Especially since as mentioned by the authors, different languages display different levels of toxicity, while the translation quality and characteristics per language is also known to vary.\n\nIn the statistical significance test you mention: “If source contribution and toxicity were completely uncorrelated, we would expect to find a result at least this significant for only roughly 5% of languages.” —> I don’t understand this statement, since the test is performed for each language separately? I suspect this is a typo and would like to invite the authors to rephrase/correct.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "71TXbl6nBH", "reviewer": "Reviewer_HKPu", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper describes a method for characterizing the amount of added toxicity in a multilingual machine translation system’s output. The toxicity detection method is a list-based approach that is unchanged from an earlier publication (NLLB Team et al 2022, cited prominently and repeatedly). The main contributions here are:\n\n(1) Moving from FLORES as the test set to translations from English of the recently proposed Holistic Bias corpus. The main insight seems to be that for toxicity analysis, one doesn’t really need references so much as source sentences that cover topics that are likely to elicit the introduction of toxicity.\n\n(2) They provide an analysis of properties of sentences that tend to elicit added toxicity in Section 3.1 Outside of the bottom half of figure 2, this is mostly descriptive. We get the flavor of what matters with little quantification (“most of the languages”, “much more often”).\n\n(3) They use explainability methods plus robustness characterizations to try to explain added toxicity, and to propose a very weak toxicity detector.\n\n(4) They provide a human verification of the toxicity detector’s output, characterizing the False Positive rate by checking each toxic addition, and using insights from (2) to aim annotators toward examples likely to have False Negatives.", "summary_of_the_review": "This is not a perfect paper, but it’s solid work on an important topic. I suspect that the solution landed on here (word lists + a template-based corpus) will be good enough for most industrial labs; i.e.: this has a shot at being the definitive citation for the problem of detecting and characterizing added toxicity in MT. I would have liked to have seen more input-related numbers and visualizations over broad text-based descriptions, and I wasn’t crazy about the explainability portion, but I don’t think that should hold the paper back from the public.", "strength_and_weaknesses": "Strengths:\n\nThis is an important problem, and it’s nice to have a paper dedicated to it. \n\nThe paper is mostly clear and easy to follow.\n\nThe proposed list-based method with synthetic but targeted source sentences seems to strike a good balance of interpretability and recall of actual problems. The included human check goes a long way toward validating this approach. It is likely to be adopted by others.\n\nThe characterization of sentences that have added toxicity (plus the details in the appendix) are nice sources of insight and inspiration.\n\nWeaknesses:\n\nI think the paper’s primary contribution (the adoption of the Holistic Bias corpus as a set of source sentences) would have been stronger if they had compared the discovered toxicity rates to those of FLORES-200. It feels like you have to read both papers in order to follow the first and most important sentence of their conclusion.\n\nI didn’t get much out of the Phenomena Causing Toxicity section or the Robustness of Translations section. This felt somewhat out of place compared with the rest of the paper. It was hard to follow, with statistics and methods being introduced and tested in rapid fire, and I’m not sure what was accomplished. I don’t think the goal was to build a glass-box toxicity detector. It would be hard to beat the black-box detector that uses the same word-list based methodology as the gold standard. Instead, the goal appears to have been to explain where toxicity came from in the model. To that end, we are left with the intuition that it is some mix of hallucination and mistranslation of the source. I think this section would have hit a lot harder if the authors had acted on one of their three suggestions (clean mistranslated data, avoid hallucination, be weary of instability) as a way of validating this approach to explanation.\n\nIt’s also strange that there is a large section on explanation / detection, and another section on characterizing based on input attributes, but the authors never contrast the two. Which is more predictive of added toxicity? Input characteristics or glass-box features?\n\nI would have liked to have seen more discussion of the False Positive human evaluation. I couldn’t believe it when that section ended! What caused all those False Positives? Could they be fixed by removing one or two problematic list entries, or was it more complicated than that? IE: do these languages inherently resist the list-based approach?\n\nI would have also liked to have seen more discussion (and numbers) regarding the types of sentences leading to added toxicity. As suggested in my summary, it feels like we have the take the authors’ descriptions at face value, with few backing numbers.\n\nThere are some editing mistakes that detract from an otherwise clean paper:\n\nThe last few sentences of the second paragraph of the introduction are repeated.\n\nThe first sentence of the third paragraph of 4.1 talks about “toxicity rates” when I think it means to discuss “false positive rates”.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "5gEbFiG9PK", "reviewer": "Reviewer_PvYr", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper discusses added toxicity detection. The authors conducted substantial analysis in a larger scale multilingual data set containing 164 languages in total. In the work, they combined the NLLB toxicity detection strategy, the HOLISTICBIAS dataset, and the ALTI+ methodology. This type of analysis or data set are useful to assess the MT systems outputs from the ethical perspectives, especially the MT systems suffer from generating offensive outputs regardless of source context or inputs and such errors are critical to human users.  ", "summary_of_the_review": "The paper provides extensive results and analysis on the added toxicity found in MT outputs. Such offensive tokens are known to be a critical error in a practical word, and from the ethical viewpoints, the research topic is worthwhile studying. However, the paper is limited to reporting the analyses and numbers, and it might lack of discussion on how to modify the translation outputs after detection, and how to fix the model itself by removing the toxic data. In this context, I would suggest the authors to try out the proposed methodology as a data filtering tool and by training the model on the filtered-out data, \nI would like to learn how much they can successfully suppress the toxicity in the MT outputs at the end.\n\n+++++++\n\nthank you for considering my comments. Since they are not addressed clearly, I will have my score unchanged.", "strength_and_weaknesses": "Strengths\n- Extensive results and analyses are conducted across languages. \n\nWeakness\n- The paper summarizes the toxicity detection methodology, but do you have any thoughts on how to modify the detected sentences and how to give the feedbacks to have the model less toxic", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2KXJspyXETg", "reviewer": "Reviewer_mzzx", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new paradigm to detect toxicity from multilingual machine translation systems. Basically, the proposed paradigm combines a wordlist method, a specific dataset, and an interpretability method. By using this paradigm, it examines the translation outputs from a multilingual machine translation system. Finally, it reveals some valuable findings about the translation errors from the MNMT system which are very helpful to understand the target MNMT system. \n\n", "summary_of_the_review": "See details from weaknesses and strengths. ", "strength_and_weaknesses": "Strengths:\n1. This paper studies an important topic for multilingual machine translation.\n2. This paper reveals some valuable findings according to human evaluation.\n\nWeaknesses:\n1. The paradigm part of this paper is not clear to me. In section 2, almost all of spaces are used to present something like preliminaries such as definition, an existing dataset and existing interpretability method. However, it does not explain the paradigm to detect toxicity, i.e., how to combine all the three for the new paradigm. Therefore, it is not clear for me what is the technical contribution and novelty of the proposed paradigm on toxicity detection.\n\n2. It seems that the proposed toxicity detection paradigm is dependent on the word alignment and interpretability method. However, this paper does not study its effects on different interpretability methods. In addition, the interpretability method used by this method is presented by an arxiv paper commented as work in progress. In fact, there are many word alignment toolkits which may be much better than the alignment model used in this paper. Moreover, the findings are from the particular dataset, which are very short and not general enough. It is unclear these findings hold on other datasets.\n\nMinor issues:\nThe sentence appears twice in the paragraph two section 1: \"Nevertheless, the overall prevalence of potential added toxicity remains low when evaluating translations of formal sentences such as those in FLORES-200, which makes it difficult to draw conclusions as to this specific aspect of a model’s performance. \".\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject