Improving the Detection of Multilingual Online Attacks with Rich Social Media Data from S ingapore (2023)

Abstract: Toxic content is a global problem, but most resources for detecting toxic content are in English. When datasets are created in other languages, they often focus exclusively on one language or dialect. In many cultural and geographical settings, however, it is common to code-mix languages, combining and interchanging them throughout conversations. To shine a light on this practice, and enable more research into code-mixed toxic content, we introduce SOA, a new multilingual dataset of online attacks. Using the multilingual city-state of Singapore as a starting point, we collect a large corpus of Reddit comments in Indonesian, Malay, Singlish, and other languages, and provide fine-grained hierarchical labels for online attacks. We publish the corpus with rich metadata, as well as additional unlabelled data for domain adaptation. We share comprehensive baseline results, show how the metadata can be used for granular error analysis, and demonstrate the benefits of domain adaptation for detecting multilingual online attacks.

## introduction toxic content, such as hate speech and abuse, is a global problem, but most resources for detecting toxic content are in english @xcite @xcite . this makes it difficult to develop effective models for detecting toxic content in other languages, and as a consequence, non-english speakers across the world are less protected against toxic content. new datasets and models for non-english languages often focus exclusively on one language or dialect. in many cultural and geographical settings, however, languages and dialects are often code-mixed, i.e., combined or used interchangeably within a conversation or even a single utter-* work completed at rewire. ance @xcite . so far, this practice has received very limited attention in toxic content research, with most work on code-mixed content focusing on hinglish, which is a mix of hindi and english (e.g. @xcite @xcite . in this article, we take a step towards addressing this issue by introducing soa, a new multilingual dataset of singapore-centered online attacks. singapore is a multilingual city-state in southeast asia, with five million inhabitants from a wide range of ethnic, religious, and cultural backgrounds. singapore's official languages are english, malay, singaporean mandarin and tamil, but many other languages are widely spoken, including the codemixed singlish language and indonesian, which is closely related to malay. using the r/singapore subreddit as a starting point, we collect a large corpus of reddit comments in indonesian, malay, singlish and other languages. we select 15,000 comments for annotation with a diverse set of sampling methods. we provide fine-grained hierarchical labels for online attacks, as well as language identification, from trained, native-speaking annotators. we also publish rich metadata, such as timestamps, anonymised user ids and source subreddit for all comments, and make available the complete unlabelled pool of 3,196,400 comments that the labelled data was sampled from. for the new dataset, we share comprehensive baseline results for a suite of mono-and multilingual models, finding that indonesian models adapted to twitter data perform best out-of-the-box. we show how the rich metadata we provide can be leveraged for more granular error analysis, finding that the advantages of the indonesian models over multilingual models stem from the language distribution in our data. finally, we demonstrate how the unlabelled pool of comments we provide can be used for adapting models to the domain of our data, finding that this domain adaptation creates clear performance benefits, especially for models not pre-trained on any social media data. overall, we make two main research contributions. 1) we publish a new dataset for multilingual online attacks in under-resourced languages with fine-grained hierarchical labels, rich metadata and additional resources. 2) we provide comprehensive baseline results, and demonstrate how metadata and additional resources can be used to evaluate and improve classification models. together, we hope that these contributions will enable more research into code-mixed toxic content for under-resourced languages, and thus serve to improve how non-english speakers across the world are protected online. foot_0 ## taxonomy of online attacks research into toxic online content and its automated detection is marred by definitional challenges, with much disagreement about the exact characteristics of core concepts @xcite @xcite . following @xcite , we use toxicity as an umbrella term for various kinds of disruptive online content, with online attacks being a particular type of toxic content. other types of toxic content include spam, sexually explicit language or the use of profanity. we define online attacks as content that directs anger, aggression or maliciousness at an identifiable target. this includes insults, threats, and inciting harm and violence, and being overtly abusive. a range of entities, individuals and groups can be targeted by an attack. related concepts include abuse, which is a subset of online attacks directed at just individuals or groups, and hate speech, which is commonly defined as a subset of online attacks directed at groups with protected characteristics, such as race, gender or sexual orientation @xcite . the taxonomy of online attacks, which we introduce in this article and use for data annotation, is hierarchical and comprises two levels. the first level is binary, indicating whether content is an online attack based on our definition. if an online attack is present, the second level lists the potential targets of the attack, split into 1) individuals, 2) social groups, 3) the media, 4) institutions and government, and 5) other. as it relates to other taxonomies of toxic content, our hierarchical setup takes inspiration from how @xcite classify offensive language. @xcite and @xcite , like us, also differentiate between attacks targeting persons and attacks targeting groups. vidgen et al. (2019) also separate out attacks against institutions. our taxonomy is also more general, compared to work that focuses on specific targets of attacks, such as women @xcite , muslims @xcite , or trans people (lu and jurgens, 2022). ## discussion the results for our baseline models suggest clear benefits from model scale for detecting online attacks in our dataset. xlm-r is much like mbert, but it has more model parameters and was pretrained on a larger corpus. accordingly, it performs much better than mbert. our results also show the benefits of adapting models to social media data, even if adaptation data and task data come from different social media platforms. xlm-r and xlm-t, like indobert and indobertweet, are the same, except for additional pre-training on twitter data. in our baseline results ( multilingual models do not appear to have an advantage over indonesian monolingual models for our dataset. this can likely be explained by indonesian content making up most of the dataset ( §3.3), and other languages in the dataset, like malay and singlish, sharing a lot of similarity with indonesian. as we found in our error analysis, the monolingual indonesian indobertweet model outperforms xlm-t, the strongest multilingual model, on indonesian content ( overall, the dataset appears to be moderately challenging for models, with performance differences between baselines that align with general intuition and other research. however, there are also some limitations to our dataset and experiments, which we discuss in a separate section following the conclusion below.