TUW - I nf at G erm E val2021: Rule-based and Hybrid Methods for Detecting Toxic, Engaging, and Fact-Claiming Comments (2021)

Abstract: This paper describes our methods submitted for the GermEval 2021 shared task on identifying toxic, engaging and fact-claiming comments in social media texts (Risch et al., 2021). We explore simple strategies for semi-automatic generation of rule-based systems with high precision and low recall, and use them to achieve slight overall improvements over a standard BERT-based classifier.

## introduction we present our systems submitted to the ger-meval 2021 shared task on identifying toxic, engaging and fact-claiming comments in social media texts @xcite . we focus on strategies for building simple rule-based systems that are both explainable and customizable for end users. we also train a simple bert-based classifier for comparison, and to evaluate its performance when combined with our high-precision rule-based systems. after a short overview in section 2 of the task and the datasets used we describe our methods for creating rule-based systems in section 4 and the bert-based baseline system in section 3. section 5 describes how these systems were combined into simple ensemble models, section 6 presents quantitative results on the 2021 test set, and a manual qualitative analysis on a sample of our output for subtask 1 is provided in section 7. all systems described in this paper are publicly available under an mit license from the repository tuw-inf-germeval2021 1 , along with instructions for reproducing our results. ## task and datasets the dataset of the 2021 shared task contains 3,244 comments from the facebook page of a german news broadcast, from discussions between february and july 2019, manually annotated for three categories corresponding to the three subtasks: whether a comment is toxic, engaging, and/or fact-claiming. definitions for each category and a detailed description of the annotation process are given in the task overview paper @xcite . for developing our rule-based system for toxicity detection we also used a corpus of annotated tweets from germeval challenges of previous years @xcite , the 2018 and 2019 datasets contain nearly 11,000 german tweets. comments in the 2021 dataset were parts of discussion threads related to individual news items. the dataset does not contain such threads, only individual comments, and this is also how they were presented to annotators. however, some fragments of the initial posts ('teaser texts') were made available to annotators as context, but were not included in the dataset because of privacy concerns @xcite . this means that in some cases our models may not have had access to the full information that led annotators to their decisions. some possible examples will be presented in the manual analysis in section 7. when experimenting with our methods, we split the 3,244 comments of the training dataset into two parts, training our ml models and developing our rules using only 2,434 comments (75 %) and validating our approach on the remaining 811 (25 %). some entities in the dataset have been anonymized by the organizers, introducing placeholders such as @user, @medium, and @moderator. in addition we also masked urls, currency symbols, and numbers. for our bert-based experiments we also replaced emoticons with their german textual representations, using the emoji library foot_1 . a german dictionary has been added to this library only days before the submission deadline. in our submissions we use our own dictionary foot_2 , created from the english resource using the google translate api via the translate python library foot_3 . ## bert-based classification language models based on the transformer architecture @xcite such as bert @xcite provide the basis of strong baseline systems across a wide range of tasks in natural language processing, and some of the top-performing systems in the 2019 ger-meval challenge also use bert @xcite . for our experiments we used the model bert-basegerman-cased foot_4 a publicly available bert model trained only on german data. for each subtask we trained a neural network with a single linear classification layer on top of bert. metaparameters were set based on performance on the validation set. we used adam optimizer with a weight decay value of 10 -5 and initial learning rate of 10 -5 . we set batch size to 8 and trained each model for 10 epochs and determined the optimal number of iterations based on either f-score or precision on the validation set (see section 5 for details). for the final submissions we trained on the full training set (including the validation set). ## rule-based methods we explore simple strategies for both manual and semi-automatic generation of lists of words and phrases that can be used in rule-based systems that consider a comment toxic if and only if it contains any of the words or phrases in a list. our goal is to facilitate the rapid creation of such simple rule-based systems because they are often preferred in real-world applications due to their fully transparent and explainable nature. any decision made by such a system, whether true or false, positive or negative, can be directly attributed to one or more terms in the input or to the fact that no terms in the input are present in the list of key terms. this offers straightforward ways for users to update the rules in a way that changes a particular decision, by removing keyphrases causing false positives or adding them to fix false negatives. whether or not this process is actually beneficial for the overall accuracy of a model, it is in line with common business needs, most typically with the common experience that once users have reported an error, they expect it to be corrected. the experiments in this section and the qualitative analysis in section 7 were performed only for the toxicity detection task. for the toxicity detection task we experimented with simple strategies for automatic bootstrapping of keyword lists, which are then reviewed and corrected manually. the method involves extracting simple patterns from comments in the training data and ranking them according to their potential as rules, i.e. looking for patterns that in themselves have a very high precision as predictors of toxicity. we searched for patterns characteristic of comments labeled as toxic in the form of a few simple feature types, including unigrams and bigrams of words or lemmas, with or without part-of-speech tag for potential disambiguation purposes. we also tried limiting the space of unigram features to nouns only, or nouns and adjectives only. for tokenization, lemmatization, and part-of-speech tagging we use the stanza library @xcite using the gsd german model with resources_version 1.2.0. we achieved best results when limiting our search to word unigrams only and ranking them separately for nouns (including proper nouns) and for all other parts-of-speech. to extract patterns with a high potential as rules we experimented with simple scoring schemes for ranking all features based on the number of true and false positives they would contribute if used as strict rules, i.e. the number of positive and negative examples containing them as patterns. to assess the efficiency of these strategies, i.e. whether the patterns they extract are generally good rule candidates that can be edited into curated lists, we observed their behaviour in the portion of the dataset used for training the ml models. the validation set was only used for infrequent overall quantitative evaluations and not for observing patterns, since for the purposes of manual rule creation this would have meant using the validation data for training. the strategy we found most effective was to consider patterns with at least 5 occurrences in the training dataset and rank them with the scoring scheme t p -100•f p , where t p and f p are the number of true and false positives detected by that pattern. lists created this way require manual editing so as not to introduce artefacts. for example, the top words in the training dataset for this year's germeval task contain words like hamburg and fleissig 'hard-working' because these words happened to occur in several comments labelled as toxic but none of the nontoxic ones, thereby getting ranked just as high as dummheit 'stupidity' and bullshit, terms that we actually want to keep for the edited list. the majority of good patterns comes from the larger toxicity dataset available to us, the 2018 and 2019 germeval training datasets @xcite . while in the smaller 2021 dataset the top-ranked patterns occurred in no more than 3 or 4 positive examples of toxicity, the combined training datasets of previous years allowed us to find patterns with 15-25 positive examples, a much stronger indicator that a word might be a good keyphrase for domain-independent detection of offensive speech. indeed, the list of the 10 highest-ranked nouns barely need post-editing, they are vasall 'vassal', invasoren 'invaders', abschaum 'scum', heuchler 'howler', dumm 'dumb', kreatur 'creature', ficker 'fucker', titten 'boobs', scheisse 'shit', volksverräterin 'traitor of the people'. we note that emoji characters are also handled by stanza as individual words and some of them also appear in the final keyword lists, such as these characters: in an independent effort we also used the 2021 training dataset for all three subtasks to observe simple patterns that can be used as high-precision predictors of each category. two of the patterns we identified were introduced in the final rule-based system: for the toxicity detection task we categorize a comment as toxic if it contains at least two words with at least four characters each written in all-caps. this rule on its own achieved 91% precision and 4% recall on our validation set. for subtask 3, if a comment contains an url and this url is not the only content of the comment, we categorize it as fact-claiming. this rule achieved 93% precision and 10% recall on the validation set. ## ensemble our three submissions for the shared task are combinations of the systems described in sections 3 and 4. our first submission contains the decisions of the bert-based classifiers for each subtask, using the number of iterations determined as yielding the highest f-score (2nd for subtask 1, 1st for subtask 2, and 5th for subtask 3). submission 2 is the union of submission 1 and our rule-based systems, i.e. for each subtask we label comments as toxic/factclaiming if either the bert-based model or our rule-based system would classify it as such (we did not use any rules for subtask 2). finally, submission 3 is our attempt at a system with higher precision at the cost of recall, here we use our rules together with bert models from the iterations yielding the highest precision (8th for subtask 1, 1st for subtask 2, and 1st for subtask 3). we note that this is different from training a machine learning model for high precision, which could be achieved by e.g. a weighted loss function. in case of subtask 2, both precision and f-score were optimal after the same number of training epochs. since we did not use any rules for detecting engaging comments, our output for this subtask was identical in all three submissions. ## quantitative results quantitative evaluation of our methods is performed based on the test set provided by the organizers. we follow the official evaluation methodology and calculate precision, recall, and f-score for both classes in each subtask and also the macro-average across classes for each figure. results on toxicity detection (subtask 1) are presented in for the task of detecting engaging comments (subtask 2) we did not develop any rule-based system and the same bert model was determined to be optimal for both precision and f-score, therefore we used the output from the same bert model in all our submissions. here we omit results for this subtask due to lack of space. results on detecting fact-claiming comments (subtask 3) are presented in ## qualitative analysis the main focus of our rule-based experiments was the toxicity detection. we performed a detailed qualitative analysis on a sample of the test dataset on this subtask. based on the labels assigned by the bert model of submission 1 and the ground truth labels we extracted a sample of 40 comments, 10 from each of the four categories true positive (tp), false positive (fp), true negative (tn), and false negative (fn). our goal with this setup is to go beyond error analysis, which would only focus on false positives and false negatives. given the subjectivity of the task and of the possible discrepancies between the information available to annotators and to our models (see section 2), we wished to inspect a sample that is balanced across both predicted and ground truth labels. since the bert model of submission 3 performed poorly, probably due to overfitting, for the purposes of this analysis we shall focus on the output of the bert model of submission 1 (trained for 2 epochs, for maximum f-score) and our rule-based system. the number of comments in each category for both of these systems and their combination is presented in the sample of true positives, as expected, contains many comments that are clearly identifiable as such based on surface patterns such as the word dumm 'stupid' (tp1, tp10) or the emojis and (tp2). false negatives (fn) would be expected to exhibit the opposite pattern, these are comments that humans agreed are toxic but models failed to detect them as such. indeed this group contains several examples where a deep understanding of the comment is necessary to account for its toxicity, demonstrating the complexity of the task. for example, to understand that comment fn9 der deutsche war schon immer naiv… 'the germans have always been naive' is in some way uncivil, one must know that some types of statements about some types of groups are not acceptable and at the same time be able to identify germans and naive as concepts belonging to these 'types'. indeed, if a human expert were to build a complex rule-based model for the toxicity detection task, it may very well contain patterns such as protected_group + negative_predicate and lexica for what words and phrases are to be considered as belonging to each of these categories. other examples require knowledge of idioms, such as the comment fn1 geht's noch? which as a phrase can be translated as 'are you crazy?' foot_5 . perhaps the most puzzling examples of false negatives are those that were probably interpreted as sarcastic by annotators, such as fn2 good luck mr. president trump make america great again. ich würde ihn wählen. tp4 schon wieder ein klugscheiss!! tp5 danke carla super weiter so tp6 trump biden tp7 was war jetzt so schlimm in den letzten vier jahren? unter trump bleibt die welt friedlich. gewinnt biden, wirds krieg geben. guten tipp an zamperoni, verlass deine frau! tp8 tp9 covid karls paniktour tp10 @user seit ihr so naiv, oder tut ihr nur so dumm?????? fp1* @user es gibt halt leider in jeder altersgruppe asoziale mitmenschen. fp2* @user gott sei dank gibt es eine meinungsfreiheit und verschiedene auffassungen. und ganz nebenbei gefragt, was hat diese dame denn für vorschläge gemacht? fp3* frau "blaerbock" in ihrem element. fehlt nur noch eine horde "kobolde"! fp4 @user stimmt schon, gegen den würde reagen wie bernie sanders wirken. fp5 pandemie? 97% der deutschen sind weder infiziert, noch krank! fp6 schnell viel blödsinn reden… fp7 sitzt da die schwester von didi hamann(augenzucken)? fp8 …keine ahnung…woll?!? fp9 dünnes eis frau prof. fp10 tja, mit ideologie wird es kalt und dunkel hier! tn1* @user woher wissen sie das? glauben heißt nicht wissen tn2* sie wissen schon dass dies in der konsequenz vielen menschen die sich schutzlos infizieren das leben kosten wird ? tn3* @user paint it red tn4* @user weil er es kann. tn5* @user ne tn6* @user das ist leider auch wahr tn7* wer's glaubt tn8* @user sehr guter kommentar tn9* trump tn10* weiterhin gute besserung! fn1* geht's noch? fn2* good luck mr. president trump make america great again. ich würde ihn wählen. fn3* tourett? fn4* politiker mit verstand ?…und das in zusammenhang mit dem namen trump ? fn5* und wenn ich mir die frage stelle, ob ich einen arbeitsplatz habe und die miete bezahlen kann, dann wähle ich biden? fn6* trump wird gewinnen und das ist gut so. auf die gesichter der hetzmedien bin ich gespannt fn7* @user immer politisch korrekt verstehst du fn8* @user viel meinung für wenig ahnung! fn9* @user der deutsche war schon immer naiv, und sie sind es auch. fn10* mimimimi turning to comments that were not labeled as toxic by annotators, in the sample of true negatives (tn) we did not find any controversial examples. the false positives (fp), on the other hand, once again provide examples of the inherent difficulty and subjectivity of the task. consider e.g. fp6 schnell viel blödsinn reden… 'quickly talk a lot of nonsense…' and fp10 tja, mit ideologie wird es kalt und dunkel hier! 'well, with ideology it gets cold and dark here!'. we believe that it would be necessary to also consider the post fragments that annotators had access to but were not included in the dataset (see section 2) to determine how such comments could have been unanimously labeled as non-toxic. several fp examples, however, are clearly not toxic, and in case of bert one can only speculate as to why they were falsely classified as such. the three comments that were also false positives of the rule-based system (fp1-3) were misclassified because of the presence of the words asozial 'asocial', meinungsfreiheit 'freedom of opinion', and horde 'horde', illustrating the limitations of purely keyword-based methods. the analysis in this section was intended to provide examples of the types of challenges a model of toxicity must concern itself with. while it is limited to a small sample from a single dataset, we believe it illustrates a range of problems that are typical for this task. in particular, false negative predictions are responsible for more than 70% of errors made by both of our top-performing systems, and our analysis suggests that identifying most of these would require more complex rules for modeling specific types of toxicity and the ability to detect sarcasm. ## conclusion we described simple methods for the semiautomatic construction of rule-based systems for detecting toxicity in social media, and used them to improve the performance of a bertbased classifier on the dataset of the 2021 ger-meval shared task. a manual error analysis was provided to illustrate the most challenging aspects of the task.