From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models (2024)

Abstract: To date, toxicity mitigation in language models has almost entirely been focused on single-language settings. As language models embrace multilingual capabilities, it’s crucial our safety measures keep pace. Recognizing this research gap, our approach expands the scope of conventional toxicity mitigation to address the complexities presented by multiple languages. In the absence of sufficient annotated datasets across languages, we employ translated data to evaluate and enhance our mitigation techniques. We also compare finetuning mitigation approaches against retrieval-augmented techniques under both static and continual toxicity mitigation scenarios. This allows us to examine the effects of translation quality and the cross-lingual transfer on toxicity mitigation. We also explore how model size and data quantity affect the success of these mitigation efforts. Covering nine languages, our study represents a broad array of linguistic families and levels of resource availability, ranging from high to mid-resource languages. Through comprehensive experiments, we provide insights into the complexities of multilingual toxicity mitigation, offering valuable insights and paving the way for future research in this increasingly important field.

## introduction breakthroughs with large language models (llms) have led to remarkable gains on openended tasks @xcite @xcite , resulting in widespread adoption by users all over the world. however, this widespread adoption has introduced a range of -possibly unknown -harms. one of these which is known and well-documented (sheng in our work, we investigate the axes of modeling framework choices, evaluation of toxicity, and dataset characteristics. @xcite @xcite is the generation of toxic text which hinders the safe deployment of llms @xcite and limits the practical utility to users @xcite . despite the adoption of llms across the world and increasingly in languages outside of english @xcite , toxicity mitigation techniques have, to our knowledge, only focused on english @xcite @xcite @xcite @xcite . given that the web is highly multilingual, multi-cultural, and typographically diverse, monolingual approaches are not sufficient in the real world, as they naturally exclude a great portion of users and cultures. additionally, previous studies have demonstrated that concentrating safety mitigation efforts solely on english makes models prone to safety violations in a multilingual setting @xcite @xcite . this implies that while model completions in the english language may be safe, the same prompts in other languages could yield unsafe outputs. therefore, it is essential to extend safety mitigation efforts to encompass a wide range of languages. in this work, we present the first large-scale study of multilingual toxicity mitigation. in total, we assess the toxicity of 9 languages that span 5 different scripts, with models ranging from 1.3b to 13b parameters. to conduct this extensive evaluation, we first create the datasets for both training and evaluation. we extend existing english datasets commonly employed for toxicity mitigation and evaluation studies, specifically civilcomments @xcite and holisticbias @xcite by incorporating translations of these datasets into other 8 languages. these expanded datasets are used for training and evaluation of multilingual toxicity mitigation, while also establishing a foundational benchmark for future research in this field. to the best of our knowledge, we are the first to examine toxicity mitigation techniques for multilingual text generation. given the absence of a previously established framework for evaluating toxicity mitigation, we explore various axes of mitigation to inform future research focused on multilingual harm reduction. this includes the effectiveness of adopting translated text, the impact of translation quality, and the performance of mitigation techniques in both a static and continual learning setting. moreover, we address the challenge of consistently evaluating toxicity across diverse languages in a single multilingual model and provide insights into the pros and cons of using retrieval augmentation versus finetuning-based techniques. we employ two distinct approaches that have demonstrated competitive performance in the realtoxici-typrompts benchmark for english @xcite , namely dexperts @xcite and goodtriever @xcite . the experimental axes we cover are seen in figure 1 and our core contributions can be enumerated as follows: ## experimental setup our investigation into multilingual toxicity mitigation focuses on three pivotal areas: (1) the usage of translated data to tackle toxicity mitigation; ## multilingual mitigation results this section explores the core experimental investigation of our study: determining whether toxicity mitigation techniques evaluated solely in monolingual settings can be effectively extended to multilingual contexts. a significant obstacle in expanding these techniques to multiple languages is the challenge of comparing toxicity across languages. toxicity scores differ across languages. it is expected that perspectiveapi's toxicity scores for different languages are not entirely comparable, as the website acknowledges how models have different performances across languages. 5 prior work also showed variations in perspectiveapi's toxicity scores across languages of both in-language (nogara et al., 2023), and translated data (kobellarz and silva, 2022). we understand other factors might be entangled in these results, such as content difference and translation quality. in appendix d, we show how scores still differ significantly in a more controlled setting of simple toxic sentences that are easy to translate and would most likely be toxic in every language. ## leveraging translated data for toxicity mitigation a key challenge for extending mitigation techniques to additional data is the lack of domainspecific training data. put simply, lower-resourced languages lack large, well-curated toxicity datasets. this section explores the viability of using translated data to extend toxicity mitigation efforts to lower-resourced languages, despite its challenges such as inaccuracies inherent in translation tools, especially for languages with fewer resources. ## ablation studies we explore additional factors that might affect multilingual toxicity mitigation: 1) varying the training language order, 2) comparing mitigation given translation of the same instances (translating parallel text) versus promoting augmentation diversity by translating a different subset for each language (translation non-parallel), 3) varying the size of the ## discussion and conclusions in this study, we conducted the first comprehensive exploration of multilingual toxicity mitigation for text generation, specifically targeting settings that go beyond the traditionally english-centric ap-proach. we explored several key areas, including mitigation strategies, data characteristics, the use and impact of translation data, and the scalability of these techniques. we also showed some of the intricacies of evaluating toxicity comparably across multiple languages. identifying comparable toxic behaviors for fair cross-language comparisons presented a significant issue. to address this, we adopted a templatebased dataset for our evaluation set, designed to standardize comparisons across languages and simplify the addition of more languages in the future. this method was chosen to guarantee the consistency of content across languages within our evaluation set. however, this approach is not without its drawbacks, such as potential cultural insensitivity or the impractical nature of some prompts due to the randomness in their generation. moreover, building on observations that perspec-tiveapi scores vary across languages and previous research indicating that translation can change sentiment, our study further demonstrates that translation tends to decrease sentence toxicity. despite a heavy reliance on translation, our findings suggest that translated datasets can outperform larger, inlanguage datasets. this observation is supported by two hypotheses: (1) translated data is more indistribution to perspectiveapi as most multilingual data used in training were originally translated from english @xcite , and (2) there are inherent differences in toxicity levels between the translated and in-language datasets we used. as we conclude, it's clear that our research marks just the beginning of a critical and expanding field of study. as systems increasingly become multilingual and multicultural, the development of safety measures and benchmarks must evolve to ensure inclusiveness. this study lays the groundwork for future endeavors in the essential task of multilingual toxicity mitigation for text generation, urging a shift towards more globally aware and culturally sensitive language technologies.