Countering hate on social media: Large scale classification of hate and counter speech (2020)

Abstract: Hateful rhetoric is plaguing online discourse, fostering extreme societal movements and possibly giving rise to real-world violence. A potential solution to this growing global problem is citizen-generated counter speech where citizens actively engage with hate speech to restore civil non-polarized discourse. However, its actual effectiveness in curbing the spread of hatred is unknown and hard to quantify. One major obstacle to researching this question is a lack of large labeled data sets for training automated classifiers to identify counter speech. Here we use a unique situation in Germany where self-labeling groups engaged in organized online hate and counter speech. We use an ensemble learning algorithm which pairs a variety of paragraph embeddings with regularized logistic regression functions to classify both hate and counter speech in a corpus of millions of relevant tweets from these two groups. Our pipeline achieves macro F1 scores on out of sample balanced test sets ranging from 0.76 to 0.97—accuracy in line and even exceeding the state of the art. We then use the classifier to discover hate and counter speech in more than 135,000 fully-resolved Twitter conversations occurring from 2013 to 2018 and study their frequency and interaction. Altogether, our results highlight the potential of automated methods to evaluate the impact of coordinated counter speech in stabilizing conversations on social media.

## introduction hate speech is a growing problem in many countries @xcite , it can have serious psychological consequences @xcite , and is related to, and perhaps even contributing to, real-world violence @xcite . while censorship can help curb hate speech @xcite , it can also impinge on civil liberties and might merely disperse rather than reduce hate @xcite . a promising alternative approach to reduce toxic discourse without recourse to censorship is so-called counter speech, which broadly refers to citizens' response to hateful speech in order to stop it, reduce its consequences, and discourage it @xcite . it is unknown, however, whether counter speech is actually effective due to the lack of systematic large-scale studies on its impact @xcite . a major reason has been the difficulty of designing automated algorithms for discovering counter speech in large online corpora, stemming mostly from the lack of labeled training sets including both hate and counter speech. past studies that provided insightful analyses of the effectiveness of counter speech mostly used hand-coded examples and were thus limited to small samples of discourse @xcite @xcite @xcite . we perform the first large-scale classification study of hate and counter speech, using a unique situation in germany, where self-labeling hate and counter speech groups engaged in discussions around current societal topics such as immigration and elections. one is "reconquista germanica" (rg), a highly-organized hate group which aimed to disrupt political discussions and promote the right-wing populist, nationalist party alternative für deutschland (afd). at their peak time, rg had between 1,500 and 3,000 active members. the counter group "reconquista internet" (ri) formed in late april 2018 with the aim of countering rg's hateful messaging through counter speech and to re-balance the public discourse. within the first week, approximately 45,000 users joined the discord server where ri was being organized. at their peak, ri had an estimated 62,000 registered and verified members, of which over 4,000 were active on their discord server for the first few months. however, ri quickly lost a significant amount of active members, splintering into independent though cooperating smaller groups. we collected millions of tweets from members of these two groups and built labeled training set orders of magnitude larger than past studies of counter speech. by building an ensemble learning system with this large corpus we trained highly accurate classifiers which matched human judgment. we also used this system to study more than 135,000 conversations on german twitter to understand the interactions between counter and hate groups online-an important first step in studying the impacts of counter speech on a large scale. ## results classification results all combinations of feature extraction models m d2v and classification functions h ✓ produced a total of @xmath0. we found that the top 10 highest performing parameter sets across all five balanced training sets were the same for all m d2v . in particular, a max-imum distance between the current and predicted word within a sentence of 5, ignoring all words that occurred less than 10 times, and an initial learning rate of 0.025 with a minimum learning rate of 0.00025, resulted in the highest accuracy. each of these top performing experts were trained for 20 epochs, with a distributed bag-of-words framework. the optimal preprocessing parameters were also the same across the top 10. each of these used light stop word removal. the optimal document labeling was also the same across these 10 experts, namely the author-group labeling. recall that this labeling scheme tagged each tweet with the authors' unique identifier as well as the known group identification (rg vs ri). while the top models had the same training parameters aside from , the models were trained across 5 different training sets, which led to experts that could differ significantly. these top 10 experts had individual f1 scores of 0.755 ± 0.0012 on their individual test sets, when forced to make a classification (@xmath1. taking into account that each t out,i was balanced, containing 50,000 hate tweets and 50,000 counter tweets, these f1 scores do not suffer from accuracy inflation that would occur with an unbalanced test set @xcite . this result compares well to previous studies that used smaller unbalanced data sets and achieved f1 scores ranging from 0.49 to 0.77 @xcite @xcite . as mentioned in section 3, we did not only use experts in isolation, but also in an ensemble learning approach where the experts could vote on the class label for each tweet in a given test set. due to variations in the training sets and parameters, each expert had a slightly different view of the language, suggesting that combining their knowledge might be beneficial. using the top 10 experts as a panel, instead of individually as just discussed, we obtained an improved average f1 score across all 5 out-of-sample test sets of 0.7616 ± 0.00083. increasing the size of the panel to include the top 25 experts resulted in an average f1 score across the 5 test sets of 0.7618 ± 0.0007, see we also obtained improved results when we varied confidence threshold and allowed the experts to withhold their vote on contentious tweets. increasing the confidence threshold naturally decreased the number of tweets classified as hate or counter speech. as expected, we found that this led to an increased overall precision, recall, and f1 score, since the labeled tweets were those for which the panel was more certain (see the scores @xmath2. the thresholding procedure causes many examples-correctly and incorrectly labeled-to be ignored from these calculations, which obviously may bias these scores in unpredictable ways. even so, these scores provide a rough approximation of how accurate we can expect the classifier to be when applied to the reply tree dataset at a given confidence threshold. as discussed earlier, while this makes these scores challenging to interpret, the threshold is necessary to avoid mislabeling neutral speech in these conversations. comparison to human judgment our crowdsourcing results, shown in figure 2 , suggest that our automated classifier aligns well with human judgment. overall correlation between classifier scores and human judgments was @xmath3.94. the correlation was somewhat lower for tweets classified as counter speech (@xmath4.75) than for those classified as hate (@xmath5.96). this could indicate that to humans counter speech looks more like 'neutral' discourse than hate speech does, or this could be a reflection of the slightly weaker counter speech labeling scheme described in section 3.1 and suggest that counter speech is more challenging for the classifier to identify, or a combination of both. as expected, classifier scores around 0.5 received intermediate hate scores from human judges as well. the labels assigned by human judges were not used during the classification training or annotation process and were only used as a sanity check on the alignment of the classifier with human judgment. ## conclusion online hate speech is a problem shared by every social media platform, and yet there are still no clear solutions to this growing problem. a potential solution aimed at returning online discourse to civility is citizen-generated counter speech. until now, studying counter speech and its effectiveness has been limited to small-scale hand-labeled studies. in this paper, we leveraged a unique situation in germany to perform the first large-scale automated classification of counter speech. our methods provided f1 scores on a balanced set of 100,000 out-of-sample tweets ranging from 0.76 to 0.97 depending on the confidence threshold being used. beyond accuracy measures, we used crowdsourcing to verify that the conclusions reached by our classifier were in-line with human judgment. we were able to use this classification algorithm to identify hate and counter speech in over 135,000 fully resolved twitter conversations from 2013-2018. our results suggest that counter speech might have contributed to depolarization of discussions and that organized counter speech by ri might have stimulated further counter speech and attracted less hateful responses. while causality cannot be established due to the many other ongoing societal processes at the time, our results suggest that organized counter speech may be a powerful solution to combating the spread of hate online. we hope that the framework developed in this paper will be a starting point to understand the dynamics between hate and counter speech and help develop actionable strategies.