A Weakly Supervised Classifier and Dataset of White Supremacist Language (2023)

Abstract: We present a dataset and classifier for detecting the language of white supremacist extremism, a growing issue in online hate speech. Our weakly supervised classifier is trained on large datasets of text from explicitly white supremacist domains paired with neutral and anti-racist data from similar domains. We demonstrate that this approach improves generalization performance to new domains. Incorporating anti-racist texts as counterexamples to white supremacist language mitigates bias.

## introduction the spread of white supremacist extremism online has motivated offline violence, including recent mass shootings in christchurch, el paso, pittsburgh, and buffalo. though some research in natural language processing has focused on types of hate speech, such as anti-black racism @xcite and misogyny @xcite , little work has focused on detecting specific hateful ideologies. practitioners have called for such systems, particularly for white supremacism @xcite . to detect white supremacist language, we build text classifiers trained on data from a large, diverse set of explicitly white supremacist online spaces, filtered to ideological topics. 1 in a weakly supervised set-up, we train discriminative classifiers to distinguish texts in white supremacist domains from texts in similar online spaces that are not known for white supremacism. these classifiers outperform prior work in white supremacist classification on three annotated datasets, and we find that the best-performing models use a combination of weakly and manually annotated data. hate speech classifiers often have difficulty generalizing beyond data they were trained on @xcite . we evaluate our classifiers on unseen datasets annotated for white supremacism from a variety of domains and find strong generalization performance for models that incorporate weakly annotated data. hate speech classifiers often learn to associate any mention of marginalized identities with hate, regardless of context @xcite . to address this potential issue with white supremacist classification, we incorporate anti-racist texts, which often mention marginalized identities in positive contexts, as counter-examples to white supremacist texts. evaluating on a synthetic test set with mentions of marginalized identities in a variety of contexts @xcite , we find that including anti-racist texts helps mitigate this bias. ## the language of white supremacist extremism this work focuses on white supremacist extremism, social movements advocating for the superiority of white people and domination or separation from other races @xcite . this fringe movement both exploits the bigotries widely held in societies with structural white supremacism and makes them explicit @xcite @xcite . key beliefs of white supremacist extremism are that race and gender hierarchies are fixed, that white people's "natural" power is threatened, and that action is needed to protect the white race @xcite @xcite . many qualitative studies have examined the language of white supremacism @xcite @xcite ). computational models have been developed to identify affect @xcite , hate speech (de @xcite , and violent intent @xcite within white supremacist forums. two other studies have built models to detect white supremacist ideology in text. @xcite test word2vec/bilstm models, pre-trained on a corpus of unlabeled white supremacist forum data, as well as bert models. to estimate the prevalence of white supremacism on twitter after the 2016 us election, @xcite build a dictionary-based classifier and validate their findings with unlabeled alt-right reddit data. in contrast, we use a large, domain-general white supremacist corpus with carefully selected negative training examples to build a weakly supervised discriminative classifier for white supremacism. ## weakly annotated data it is difficult for annotators to determine whether the short texts commonly used in nlp and computational social science, such as tweets, express white supremacism or other far-right ideologies. @xcite struggle to reach adequate white supremacy hate speech figure 1 : conceptualization of the relationship between hate speech and white supremacism used in this paper. much of white supremacist language includes text that would be considered hate speech, i.e. attacks against those with marginalized identities. however, we also aim to capture text that expresses white supremacist ideology without direct hate speech, such as the glorification of nazism. finally, some hate speech would not fit as expressing a white supremacist ideology, such as antisemitism within a black nationalist context. instead of focusing on individual posts, we turn to weak supervision, approaches to quickly and cheaply label large amounts of training data based on rules, knowledge bases or other domain knowledge @xcite . weakly supervised learning has been used in nlp for tasks such as cyberbullying detection @xcite , sentiment analysis @xcite , dialogue systems @xcite and others @xcite . for training the discriminative white supremacist classifier, we draw on three sources of text data with "natural" (weak) labels: white supremacist domains and organizations, neutral data with similar topics, and anti-racist blogs and organizations. ## classification due to the success of bert-based hate speech models @xcite , we select the parameter-efficient distil-bert model @xcite to compare data configurations 5 . we use a learning rate of 2×10 -5 , batch size of 16, and select the epoch with the highest roc auc on a 10% development set, up to 5 epochs. training each model took approximately 8 hours on an nvidia rtx a6000 gpu. we train models on binary white supremacist classification. all posts in the white supremacist corpus, after sampling and filtering, are labeled 'white supremacist'. posts in neutral and anti-racist corpora are labeled 'not white supremacist'. we also test combining weakly labeled data with manually annotated data from existing datasets (see below) and our own annotation of white supremacist posts in lda topics. since there is relatively little manually annotated data, we duplicate it 5 times in these cases, to a size of 57,645 posts. ## results ## conclusion ideologies such as white supremacy are difficult to annotate and detect from short texts. we use weakly supervised data from domains known for white supremacist ideology to develop classifiers that outperform and generalize more effectively than prior work. incorporating texts from an antiracist perspective mitigates lexical bias. to apply a white supremacist language classifier to varied domains, our results show the benefit of using such weakly supervised data, especially in combination with a small amount of annotated data. other methods for combining these data could be explored in future work, such as approaches that use reinforcement learning to select unlabeled data for training @xcite . incorporating social science insights and looking for specific tenets of white supremacist extremism could also lead to better classification. this classifier could be applied to measure the prevalence or spread of white supremacist ideology through online social networks.