Human-in-the-Loop Hate Speech Classification in a Multilingual Context (2022)

Abstract: The shift of public debate to the digital sphere has been accompanied by a rise in online hate speech. While many promising approaches for hate speech classification have been proposed, studies often focus only on a single language, usually English, and do not address three key concerns: post-deployment performance, classifier maintenance and infrastructural limitations. In this paper, we introduce a new human-in-the-loop BERT-based hate speech classification pipeline and trace its development from initial data collection and annotation all the way to post-deployment. Our classifier, trained using data from our original corpus of over 422k examples, is specifically developed for the inherently multilingual setting of Switzerland and outperforms with its F1 score of 80.5 the currently best-performing BERT-based multilingual classifier by 5.8 F1 points in German and 3.6 F1 points in French. Our systematic evaluations over a 12-month period further highlight the vital importance of continuous, human-in-the-loop classifier maintenance to ensure robust hate speech classification post-deployment.

## introduction hate speech, often taken to designate insults and attacks against individuals or groups based on their inherent traits, is an expression which is widely used but on whose definition there is no general consensus. given the subjective and contextual nature of hate speech, it is left open to media platforms @xcite @xcite , research groups @xcite @xcite @xcite @xcite and individuals to decide what to include under this notion and whether to adjust its definition @xcite . particularly difficult is drawing the line between hate speech, toxic speech and humour expressed through e.g. irony, sarcasm or euphemisms. the challenges described in this paper show that configuring a stable and robust hate speech classifier is not a trivial task. while this applies to big tech companies who have the data and infrastructure to create, train and maintain their own classification systems, it is often prohibitive for smaller, mostly regional, companies and (online) media outlets. for them, no tailored off-the-shelf solution exists, they often do not have the in-house capacity to develop and maintain their own classification system and are bound by data protection rules tightly regulating which data can be shared with commercial services. the swiss context presents a particularly challenging case given that many instances of hate speech are deeply ingrained in the multilingual, sociocultural and political contexts of switzerland (cf. e.g. @xcite @xcite @xcite and are not picked up by currently available toxic speech classifiers. this situation is exacerbated by the fact that, up to the point of completing this study, no annotated hate speech data was available on which a solid baseline classifier for the swiss context could have been trained. the classification framework we developed addresses an important research gap regarding longterm, stable hate speech classification and is directly motivated by how such a classifier would be deployed in practice. a core feature of this system is that the human analysts remain both in control and at the centre of the decision making process, which we believe is vital to ensuring good postdeployment performance and enabling classifier maintenance. in building our human-in-the-loop hate speech classification pipeline, we closely collaborated with several swiss media outlets and a swiss ngo working on countering online hate. in what follows, we will describe in more detail the steps involved in constructing this pipeline from the data collection stage all the way to postdeployment testing and the challenges encountered in the process. due to non-disclosure agreements signed with our partners, we are not able to disseminate our dataset or code, but we can share the technical details of our framework and approach. we also include an extensive appendix describing the nature of our dataset and details relating to the annotation process. ## related work research on online hate speech and its detection has risen exponentially over the past few years, yielding a wide variety of approaches to tackling the phenomenon. given the large body of work, numerous surveys and overviews outlining the main directions of research have been published on the subject @xcite @xcite @xcite @xcite @xcite @xcite . the research they review has mostly focused on optimising classifier performance by focusing on different types of input data, preprocessing steps, languages, classifier types, model architectures and feature engineering options, and by performing classification on various types of problematic language and target groups (cf. additionally, a series of shared tasks on offensive, abusive and hate speech classification have featured in nlp and linguistics competitions (cf. the results of these challenges highlight that both the model selection and the nature of the dataset have a direct influence on classification performance whereby transformer models @xcite , particularly those based on the bert architecture @xcite , significantly outperform other models. multiclass classification approaches perform consistently worse than do binary ones. when more than one language is present in the dataset, classification is usually not carried out in a multilingual setting but for each language individually. classifiers for english often perform much better due to the sheer number of data points and pretrained models available, reaching f1 scores of 83.0 for binary and 66.6 for multiclass classification @xcite . in contrast, the best-performing german language classifiers reach f1 scores of 77.0 for binary and 54.9 for multiclass classification, which can partly be attributed to the complex structure of the german language @xcite . outside of these challenges, the thus-far best-performing bert-based multilingual classifier reaches an f1 score of 74.7 for german and 76.9 for french @xcite . despite this proliferation of research on hate speech, the topic has, to date, received very limited scholarly attention in the swiss context, a particularly complex linguistic, sociocultural and political landscape. a notable exception is the work by @xcite , who explore gendered hate speech in swiss whatsapp messages. research on post-deployment performance, classifier maintenance and infrastructural limitations is similarly scarce. some attempts at discussing post-deployment performance in terms of crossdomain and concept drifts have been made @xcite @xcite @xcite @xcite @xcite and show that performances of deployed models are often lower than those obtained in vitro, sometimes dropping as much as 15 f1 points @xcite . while these insights are valuable, all of these studies only artificially post-test models with publicly available datasets: none of them presents long-term pre-and post-deployment results. similarly, infrastructural challenges encountered in the process of training, evaluating and testing remain understudied. @xcite provide the currently most complete examination of challenges related to automatic hate speech detection. of the 19 action points salminen et al. describe, the setup of our classification pipeline allows us to contribute to addressing the following five: ap02 (developing a multilingual hate speech classifier), ap03 (acquiring new samples periodically to refresh the training sets used for model development), ap13 (collecting and testing datasets from multiple social media platforms), ap18 (measuring and displaying overall error rates of online hate classifiers to end-users such as moderators) and ap19 (providing probabilities and explanations of labels provided by the hate classifier for individual samples). ## results and discussion to train and evaluate our classifier, we performed a series of experiments which align with the three stages of our development process: (1) incremental training which took place during the annotation process and where the classifier was retrained on a weekly basis; (2) pre-deployment experiments to cross-examine the final model results; and (3) post-deployment experiments where we tested the best-performing model in situ. with the exception of one pre-deployment experiment, all predeployment training was carried out on data where occurrences of toxic speech were treated as positive examples. all experiments are reported using the weighted f1 score to account for the fact that postdeployment evaluation took place on a random subeven though such a saturation point may eventually be reached in vitro, it is unlikely that this will ever happen in situ, given the shifting nature of language usage, topics of discussion and similar factors. post-deployment retraining is therefore never-ending and is evaluated in the same way as during the annotation process. as such, f1 remains vital for post-deployment classifier maintenance. however, since the model is, upon deployment, fine-tuned for the (swiss) german and french contexts, retraining would require fewer new data points -just enough to introduce new features -and would not have to be carried out as frequently as we needed to perform it, but certainly in moments of 'dramatic' language shifts. as such, f1 can also be used for adjusting the retraining strategy and thus lowering carbon footprints, costs and not least human effort. ## conclusion this paper presents a multilingual human-inthe-loop bert-based hate speech classification pipeline adapted to the swiss context. by systematically tracing and reporting classifier performances from the start of the annotation process to postdeployment, we highlight important practical considerations for fine-tuning a state-of-the-art bertbased classification model. the most important is that testing classifiers post-deployment is vital for configuring robust systems, as various types of drift cause performances to drop. next to focusing on ameliorating systems pre-deployment, future research should thus also look into possible mitigation techniques to stabilise classifier performance post-deployment. in addition to recommending thresholds, a first step in that direction is to discard the idea of a static, one-time-only-trained model in favour of an adaptive learning concept @xcite . in this paradigm, the classifier functions like a living organism and evolves together with the ever-changing uses of language, and alterations in mindsets and topics. this can only be achieved by actively maintaining the classifier post-deployment, i.e. by dynamically adjusting the definition of hate speech and the annotation schemes, by periodically retraining the model and by allowing it progressively to accumulate and adapt its knowledge of hate speech.