M u T ox: Universal MU ltilingual Audio-based TOX icity Dataset and Zero-shot Detector (2024)

Abstract: Research in toxicity detection in natural language processing for the speech modality (audio-based) is quite limited, particularly for languages other than English. To address these limitations and lay the groundwork for truly multilingual audio-based toxicity detection, we introduce MuTox, the first highly multilingual audio-based dataset with toxicity labels which covers 14 different linguistic families. The dataset comprises 20,000 audio utterances for English and Spanish, and 4,000 for the other 28 languages. To demonstrate the quality of this dataset, we trained the MuTox audio-based toxicity classifier, which enables zero-shot toxicity detection across a wide range of languages. This classifier performs on par with existing text-based trainable classifiers, while expanding the language coverage more than tenfold. When compared to a wordlist-based classifier that covers a similar number of languages, MuTox improves F1-Score by an average of 100%. This significant improvement underscores the potential of MuTox in advancing the field of audio-based toxicity detection.

## annotation guidelines this section reports the detailed guidelines that we provide to annotators to detect toxic content in audio speech at the level of single utterances. this toxicity could be due to aspects of lexical semantics or of perlocutionary effects. for this we provide annotators with the following definitions: what qualifies as toxicity for this task? the notion of toxicity can be elusive because of its inherent subjectivity; i.e. it greatly depends on the lived experience of the person who is on the receiving end of an utterance. for the task at hand, we arbitrarily considered the following cases as toxic: what does not qualify as toxicity for this task? annotators were instructed to refrain from labeling as toxic the following types of language: to complete the task, annotators were asked to answer the below questions: to help annotators reach a faster determination as to which words or phrases could qualify as toxic in this exercise, we also pointed them to the publicly available toxicity-200 foot_3 word list repository. ## mutox dataset description mutox is composed of 30 languages. two languages, english and spanish, contain a larger amount of annotated data (20k utterances/sentences each language), while the rest of languages (see section 4.2), contain a smaller amount of annotated data (4k utterances/sentences each language). ## mutox audio-based toxicity classifier methodology: mutox classifier. we feed our toxicity classifier, mutox, with both audio-based and text-based toxicity-labeled data. the audiobased toxicity classifier depicted on figure 3 follows a simple architecture consisting of an encoder, turning input text or audio speech into a fixed-size representation vector, and a binary classifier composed of three simple feed-forward layers. speech data sonar speech encoder text data sonar text encoder multilingual & multimodal embeddings toxicity binary classifier mutox implementation. we use multimodal and multilingual sonar encoders from @xcite . this choice is motivated by the broad language coverage at the time of this study (english, spanish, and hp languages) and the zero-shot capabilities of sonar: once trained on a set of languages, the classifier head can be used on top of any compatible sonar encoder handling another language. for the classifier, we use variable input sizes for the 3 feedforward layers (1024, 512, and 128). we train with a binary cross entropy loss with logits and adam optimizer with an initial learning rate of 0.001. in order to compare zero-shot (zs) vs supervised performance, we train a first classifier with english and spanish training data only and test on hp languages. we then train a second classifier on all training data available and report results on the same hp languages. ## conclusions in this paper, we introduced mutox, a highly multilingual dataset and a massive multilingual toxicity detector that lays the groundwork for the largely unexplored task of multilingual audio-based toxicity detection. our mutox dataset enables benchmarking of multilingual audio-based toxicity detection 6 except for swh and cat where precision is of max(asr-etox, 0.1) across 30 languages. our mutox classifier, compared to cascade tools (speech recognition followed by text-based toxicity classifiers) that have similar coverage (more than hundreds of languages), shows superior performance for all evaluated tasks. data foot_5 and code foot_6 with their corresponding data and model cards are freely available for further research and development. in future work, we primarily aim to explore more complex architectures for training mutox detector. additionally, we want to evaluate mutox's performance on the task of added toxicity. we also intend to use it to analyze and improve etox wordlists, as detailed in the appendix c. this ongoing work will continue to advance the field of multilingual audio-based toxicity detection.