A ustro T ox: A Dataset for Target-Based A ustrian G erman Offensive Language Detection (2024)

Abstract: Model interpretability in toxicity detection greatly profits from token-level annotations. However, currently, such annotations are only available in English. We introduce a dataset annotated for offensive language detection sourced from a news forum, notable for its incorporation of the Austrian German dialect, comprising 4,562 user comments. In addition to binary offensiveness classification, we identify spans within each comment constituting vulgar language or representing targets of offensive statements. We evaluate fine-tuned Transformer models as well as large language models in a zero- and few-shot fashion. The results indicate that while fine-tuned models excel in detecting linguistic peculiarities such as vulgar dialect, large language models demonstrate superior performance in detecting offensiveness in AustroTox.

## introduction in recent years, research in the domain of content moderation has transitioned from a unidimensional to a multidimensional perspective @xcite . a one-size-fits-all approach is unable to accommodate the diverse needs of global users @xcite whose perceptions of what constitutes harmful content is contingent upon individual, contextual, and geographical factors @xcite @xcite . scholars call for less centralized and more personalized mechanisms of moderation to account for such multifaceted differences @xcite , particularly when it comes to countryspecific and subsequently linguistic nuances @xcite . intolerant user figure 1 : the posts show the importance of the target of an offensive statement in determining its severity and demonstrate how non-offensive remarks can be vulgar. for determining the harmfulness of an offensive statement, its target is decisive @xcite . perceptions of targets of offensive comments are subject to change based on individual, contextual, cultural and intersectional factors @xcite @xcite making it, therefore, crucial to effectively identify emerging targets of such statements. figure 1 depicts examples highlighting the importance of the target of an offensive statement in determining its severity. in order to study the detection capabilities of language models in an austrian cultural and linguistic context, we create a corpus of austrian german comments. our main contributions are: ## related work research focused on identifying spans within offensive statements is primarily focused on english user comments. examples of annotated spans in english comments are the targets of offensive statements @xcite , the spans contributing to the offensiveness label @xcite , and the spans comprising a violation of a moderation policy @xcite . we list all public german datasets covering tasks related to offensiveness detection in ## dataset creation data source we source austrotox from the austrian newspaper derstandard 3 , a viennese daily publication with a left-liberal stance covering domestic and international news and topics such as economy, panorama, web, sport, culture, lifestyle, science, health, education, and family. the der-standard forum is one of the largest discourse platforms in the german-speaking world. despite the left-liberal stance of the newspaper, this perspec-tive is not reflective of the forum's community, as derstandard is actively working on being a lowthreshold discussion platform open to everybody. as we focus on the austrian dialect, this austrian news media outlet's comment sections are a suitable sample to draw from. we argue that the forum's expansive community and the diverse range of articles and forums offered on the derstandard website help towards minimizing bias in austro-tox. professional moderators ensure the exclusion of hate speech which is illegal in austria (government, 2023) in the forum, this results in hardly any hate speech and a focus on offensive speech in the austrotox dataset. pre-filtering comments in order to pre-filter potentially toxic comments and comments which are not considered as toxic by existing moderation technologies, we apply stratified sampling based on the toxicity score provided by the perspective api (jigsaw). the toxicity score is between 0 (not toxic) and 1 (severely toxic). we compute the toxicity score for 123,108 posts. out of these posts, 873 exhibit a toxicity score between 0.9 and 1. we add these comments to the data to be annotated. furthermore, we create the following strata defined by the toxicity score: 0-0.3, 0.3-0.5, 0.5-0.7, 0.7-0.9. then, we randomly sample comments from each stratum. we use the following proportions for the counts of comments from the different strata: 9 : 9 : 9 : 11. austrotox encompasses responses to 532 articles or discussion forums on any topics covered by derstandard. the comments were posted between november 4, 2021, and november 10, 2021. the articles and forums where the comments appear stem from a broader time period. ## experiments fine-tuned language models we fine-tune and evaluate german bert and electra models @xcite ( prompted llms we additionally evaluate the class and span detection capabilities of not finetuned llms. we use the following large language models for our experiments: gpt 3.5 foot_3 (gpt-3.5turbo-1106) @xcite , gpt 4 foot_4 (gpt-4-1106-preview) @xcite , leolm 7b chat foot_5 , and mistral 7 (jiang et al., 2023). we avaluate them in a zero-shot and five-shot scenario ( for the llm evaluation, we distinguish between multitask prediction (predicting offensiveness, vulgarities and targets) and offensiveness-only classification. we create prompts that contain an offensiveness definition, article title, and the post to be classified. in the five-shot scenario, we additionally provide five titles and posts with labels that are randomly sampled from the training set for each prediction. we require the llm to respond in json 8 . preliminary experiments showed that only gpt-3.5 and gpt-4 were able to produce consistently valid json responses. we thus only evaluate these two models in the multi-task setup. for le-olm and mistral, we adjust the prompt, requiring them to respond with only 0 or 1, and define the token with the higher logit as the model's prediction. offensive vulgar target post-level token-level token-level params binary macro binary macro micro macro bert de 110m .69 ± .02 .74 ± .02 .68 ± .03 .84 ± .01 .22 ± .04 .56 ± .03 bert-dbm .71 ± .02 .75 ± .02 .69 ± .02 .84 ± .01 .23 ± .02 .57 ± .03 gbert base 110m .72 ± .02 .76 ± .01 .69 ± .04 .84 ± .02 .23 ± .02 .57 ± .02 gelectra .50 ± .25 .57 ± .12 .69 ± .03 .85 ± .01 .24 ± .02 .58 ± .02 gbert large 337m .73 ± .02 .75 ± .03 .71 ± .03 .85 ± .01 .21 ± .12 .53 ± .16 leolm 0-shot 7b .61 ± 02 .54 ± 03 ----5-shot .52 ± 02 .57 ± 02 ----mistral 0-shot 7.24b .30 ± 02 .53 ± 01 ----5-shot .55 ± 03 .64 ± 02 ----gpt 3.5 0-shot -.68 ± 01 .64 ± 02 .40 ± 02 .69 ± 01 .17 ± 01 .52 ± 01 5-shot .72 ± 01 .72 ± 02 .43 ± 02 .70 ± 01 .20 ± 01 .55 ± 01 gpt 4 0-shot -.70 ± 02 .77 ± 02 .36 ± 04 .67 ± 02 .20 ± 02 .55 ± 03 5-shot .76 ± 03 .81 ± 02 .41 ± 02 .70 ± 01 .22 ± 03 .58 ± 03 to ensure comparability for the token-level classification tasks, we tokenize the spans generated by the gpt-models with the gbert tokenizer. evaluation outcomes the micro f 1 on the targets for the four-class target classification is generally low due to a high prevalence of the negative class. the fine-tuned models slightly outperform the llms at detecting the targets of offensive statements. ## conclusion we presented austrotox, a dataset comprising user comments in austrian german, annotated for offensiveness. we annotated spans within the comments comprising targeted individuals, groups, or other entities through offensive statements and spans comprising vulgarities. an evaluation on our dataset indicates that the smaller language models we fine-tuned and tested excel in detecting vulgar dialect, whereas the llms we tested demonstrate superior performance in identifying offensiveness within austrotox.