DeTox: A Comprehensive Dataset for G erman Offensive Language and Conversation Analysis (2022)

Abstract: In this work, we present a new publicly available offensive language dataset of 10.278 German social media comments collected in the first half of 2021 that were annotated by in total six annotators. With twelve different annotation categories, it is far more comprehensive than other datasets, and goes beyond just hate speech detection. The labels aim in particular also at toxicity, criminal relevance and discrimination types of comments. Furthermore, about half of the comments are from coherent parts of conversations, which opens the possibility to consider the comments’ contexts and do conversation analyses in order to research the contagion of offensive language in conversations.

## introduction with the increasing popularity of social networks in the last decade, people started to communicate more and more online, organising themselves in groups and social networks in general. it became easier than ever before to interact with foreign people because geographical distance played no role any more. while this is a great opportunity for our society, it does not come without risks regarding toxic and offensive language. whereas many research groups focus mainly on binary hate speech classification, offensive language contains several other aspects. these include insult, threat, and discrimination based on charac-teristics such as age, gender, ethnicity, religion, and sexual orientation. the two main tasks to limit the amount of toxic language are detection and classification, e.g., to identify threats at an early stage or to effectively support criminal investigators in their work. the basis to train algorithms that can support such assessments are labelled datasets of high quality as well as quantity. such datasets exist only for a few languages, e.g. @xcite provided the contextual abuse dataset with fine grained labels for english language. for many languages, including german, this is a limiting research factor. therefore, motivated by the concrete application to assist a fine granular classification of offensive comments in a reporting centre for hate comments foot_0 of the german state government, we present a new german dataset that aims among others at hate speech, toxicity, sentiment, target, but also at criminal relevance (regarding german law) and threat. it contributes in three main aspects: (1) with 10,278 annotated comments it provides a new valuable resource for the german hate speech community. (2) having twelve different labels per comment opens broad research and application options beyond basic hate speech detection and (3) the inclusion of whole conversation threads being partly annotated allows to make use of comments contexts as well as other supervised and unsupervised conversation analyses. dataset source # comments tasks bretschneider and peters (2017) facebook 5,600 binary hate speech and intensity (moderate or clearly) ross et al. (2017) twitter 470 binary hate speech and intensity (scale 1-6) germeval 2018 and 2019 @xcite twitter 15,567 coarse: offense, other fine: abuse, insult, profanity twitter 2,888 implicit, explicit hasoc 2019 @xcite twitter, facebook 4,669 coarse: binary offense fine: hate, offensive, profane germeval 2021 @xcite facebook 4,188 toxic, non-toxic engaging comments fact-claiming comments ## data collection and description for this dataset we used twitter as the data source, as it grants free access to most of its tweets for research purposes, and it is possible to (automatically) extract tweets by multiple criteria via the twitter-api. the use of twitter text data guarantees a high interpretability (see sec. 2.2) and thus allows algorithms to be developed using this data. in contrast to previous related work, we aimed not only at collecting single comments but also at collecting whole conversations or parts thereof, which means tweets or comments and their reply trees. both require different data collection strategies, which will be explained in the following sections. all collected comments and conversations are in german language and posted in the first half of 2021. the most present topics in the media during the time we crawled the data were the corona pandemic with all its aspects, as well as politics related to the elections of the german bundestag in september 2021. ## data annotation the annotation scheme was established to best support the specific task of building models for fine grained classification in the mentioned reporting office for hate comments but also with a view to future research. this resulted in a comprehensive annotation schema with twelve different categories at two levels. furthermore, various metadata such as annotation time and duration were logged to leave the possibility not only to use the dataset to train models but also for future analyses like the inter-rater-agreement-learning described by @xcite that uses annotation metadata to compute the reliability of annotators. ## results and discussion in this section, we will first outline general specifications like the frequencies of the annotated labels of the dataset. in the second part the annotation quality containing measures for the iaa and the annotator consistency are presented and finally a closer look at the conversations is taken. as the dataset is very comprehensive, we can only show selected, most important statistics and results. ## baseline models the categories hate speech, toxicity and sentiment were selected to train simple baseline models on. we did not make use of comments contexts so far, this will be done in later work. even though toxicity and sentiment are regression tasks, we used classification models for them as this heavily improved the performance for the underrepresented classes (high toxicity and positive sentiment). in most categories the svm outperforms the mlp and the transformer models slightly outperform the svm. in particular, the macro recall scores of the svm, which are relevant for detection of offensive language, are higher compared to the mlp and on a same level as the transformer models. overall the mlp tended to have a higher performance on the majority class but a much lower performance on the minority class. in the other models this gap was mostly smaller. gbert produced better results for hate speech detection while xlm-r had better macro average scores for toxicity. ## limitations of the dataset even though the data collection and annotations were done as properly as possible, the dataset has some limitations. twitter as a data source has some disadvantages: first, it is just one of many social networks. every network brings its own properties and influences therefore the people, their writing style and communication standards. second, comments on twitter are moderated and therefore offensive language might have already been removed before our data collection. a more general problem, which is partly but not exclusively caused by the method of keyword search, is the presence of selected topics limiting generalizability. regarding the annotations, even with three annotations per comment the number is relatively small resulting to a high influence of possible biased annotations or annotators. the comprehensive annotation schema is complex to annotate, and the definitions of hate speech and toxicity naturally leave a lot of room for personal interpretations. further, the annotations for the legal paragraphs should be treated carefully, as no annotator (and also no one of the authors) has a legal background. finally, the annotations for legal paragraphs are specific to german legislation. ## conclusion modern machine learning methods require sufficient amounts of annotated data that are of high quality and at the same time, the annotations must be granular enough that the learned models can be used effectively in real applications. for this dataset the data was carefully selected and biases were avoided as much as possible. the annotation schema was developed together with first-hand users from a reporting office for offensive comments in germany. during the annotation process, the quality was systematically monitored and adjusted. parts of the data are available together with their conversation contexts (i.e. its parent comments and replies). we have conducted initial statistical data analyses with the annotated data, which we will continue in the future and trained baseline models on selected categories. the dataset gives the possibility to train models on high quality annotated data that go beyond binary classification tasks. moreover, it can be used to build more complex algorithms which may take the comments' context into account and even conversation analyses and analyses regarding the spread of offensive language are possible. nology. additionally, this contribution has been funded by the project "detox" (cybersecurity research funding of the hessian ministry of the interior and sports) which is a collaboration with the hessian cybercompetencecenter (hessen3c).