Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech
Year: 2023

Abstract:
Hate speech detection faces two significant challenges: 1) the limited availability of labeled data and 2) the high variability of hate speech across different contexts and languages. Prompting brings a ray of hope to these challenges. It allows injecting a model with taskspecific knowledge without relying on labeled data. This paper explores zero-shot learning with prompting for hate speech detection. We investigate how well zero-shot learning can detect hate speech in 3 languages with limited labeled data. We experiment with various large language models and verbalizers on 8 benchmark datasets. Our findings highlight the impact of prompt selection on the results. They also suggest that prompting, specifically with recent large language models, can achieve performance comparable to and surpass fine-tuned models, making it a promising alternative for under-resourced languages. Our findings highlight the potential of prompting for hate speech detection and show how both the prompt and the model have a significant impact on achieving more accurate predictions in this task.

text: The 7th Workshop on Online Abuse and Harms (WOAH), pages 60–68 July 13, 2023 c©2023 Association for Computational Linguistics

heading: 1 Introduction
text: The rising prevalence of online hate speech and its harmful effects have made hate speech detection a central task in natural language processing (NLP). Despite progress, the prevalent supervised learning approaches encounter significant challenges: many languages or contexts have little or no labeled data (Poletto et al., 2021). Hate speech is also subjective and context-dependent, as it is influenced by factors such as demographics, social norms, and cultural backgrounds (Talat and Hovy, 2016).
To overcome these challenges, approaches like zero-shot learning (ZSL) and prompting of large language models (LLMs) have emerged.1 Both
1Note that ZSL could be used with various models, whereas prompting is specific to LLMs. Here, we use ZSL to prompt LLMs without additional labeled examples in the prompt (few-shot learning), but only the target sentence.
use a template to process the original text and the class labels as verbalizers. This approach leverages the LLM’s knowledge to predict the likelihood of the (class) verbalizers in the template. These verbalizers guide the model’s understanding of a specific task. For binary hate speech detection, the template might be “<text>. This text is <verbalizer>”, where <verbalizer> can be “hateful” or “non-hateful”. For the input, “I hate you. This text is”, the LLM should associate a higher likelihood with the verbalizer completion “hateful”. By picking the more likely completion, this approach requires no training data. It has shown promising results in various NLP applications (Zhao et al., 2023; Su et al., 2022; Wei et al., 2022; Brown et al., 2020).However, to date, its effectiveness for hate speech detection remains largely unexplored.
We comprehensively evaluate ZSL with prompting for hate speech detection to better understand its capabilities. The choice of appropriate verbalizers is a key factor in the effectiveness of prompting (Plaza-del-Arco et al., 2022; Liu et al., 2023). To this end, we systematically compare various verbalizers across multiple models. We evaluate the performance of conventional transformer models and more recent instruction fine-tuned LLMs on 8 benchmark datasets to assess their robustness. Furthermore, we test our approach on two languages with limited labeled data (Italian and Spanish). Our results show that ZSL with prompting matches or surpasses the performance of fine-tuned models, particularly in instruction fine-tuned models.
Contributions 1) We investigate the effectiveness of ZSL with prompting for hate speech detection 2) We conduct a systematic exploration and comparison of various verbalizers across 5 models 3) We extend our investigation to two languages with limited labeled data. Our code is publicly available at https://github.com/MilaNLProc/ prompting_hate_speech.
60

heading: 2 Datasets
text: We compare our results on 8 benchmark datasets using binary classification. See Table 1 for details. They differ in terms of size, corpus source, and labels. More details are in Appendix A.

heading: 3 Prompting for Zero-Shot Hate Speech
text: Classification
We use ZSL with prompting to evaluate the models’ ability to detect hate speech. First, we test various encoder models to select the best verbalizers. We then test those verbalizers on recent instruction fine-tuned LLMs and compare to encoder models.
Encoder-based Language Models For our experiments, we use the following prompt template: “<text> This text is <verbalizers>”. We then check the LLM likelihood of hateful and nonhateful verbalizers and select the most probable completion as final prediction. We test all 25 possible pairs from the following lists. For hate: harmful, abusive, offensive, hateful, toxic, and for non-hate respectful, kind, polite, neutral, positive.
We compare three different language models: BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), and DeBERTa (He et al., 2020). We use OpenPrompt (Ding et al., 2022), a standard framework for prompt learning over pre-trained language models.
Instruction Fine-tuned Language Models We experiment with recent instruction fine-tuned language models. They are fine-tuned on a large set of varied instructions that use an intuitive description of the downstream task to answer natural language prompts. In this approach, we formulate the prompt template as “Classify this text as <verbnon-hate> or <verbhate>. <text>. Answer:”, for the verbalizers (verbnon-hate, verbhate) we consider the best pair obtained with the encoder models, and for the prompt models, we use the Fine-tuned Language
Net (FLAN-T5) model (Chung et al., 2022) and mT0 (Muennighoff et al., 2022). Note that FLANT5 has been trained for toxic language detection.
Baseline We used (1) a RoBERTa model finetuned with supervised training on each hate speech dataset and (2) the commercial Perspective API.2

heading: 4 Results

heading: 4.1 Encoder models
text: Table 2 shows the results of several encoder models on multiple hate speech detection benchmark datasets. Overall, the best-performing encoder model across different datasets is RoBERTaLARGE obtained the best macro-F1 score in 5 out of 8 datasets. Regarding the verbalizers, the pair positive and polite yield the best results in identifying non-hateful speech, while hateful and toxic prove best for detecting hate speech. This highlights the need for careful selection of verbalizers to achieve optimal performance in this task.
Identifying Best Verbalizers To select the best pair of verbalizers that work well across models and datasets for hate speech detection, we averaged the different performance metrics by model and dataset across all folds. As shown in Table 3, the bestperforming verbalizer pair is respectful-toxic, which achieves the highest macro-F1 score of 42.74. The verbalizers most commonly associated with the non-hate speech class are respectful and polite, while toxic and hateful are more commonly associated with hate speech. We select the best verbalizer pair (respectful-toxic) to conduct additional experiments.

heading: 4.2 Encoder vs. Instruction Fine-tuned LLMs
text: In this section, we compare the results obtained by prompting the encoder-based models and the instruction fine-tuned models. The results are shown in Table 4. These models are prompted using the best pair of verbalizers we found in the encoderbased models, which is respectful-toxic. In general, the recent models mT0 and FLAN-T5 outperform the encoder-based models by a large margin showing an average improvement of 39.75% and 65.33% over the encoder models, respectively. In particular, FLAN-T5 exhibits remarkable performance in detecting hate speech across various datasets, which can be attributed to its prior finetuning for toxic detection. This suggests that the
2https://www.perspectiveapi.com/
knowledge learned from detecting toxic language is transferable and can be leveraged to improve hate speech detection in other datasets. In addition, we conduct a comparison between the supervised learning upper bound, a fine-tuned RoBERTaBASE model, and the instruction fine-tuned models in our ZSL experiments. Our findings show that the instruction fine-tuned models achieve comparable performance, and FLAN-T5 even surpasses the RoBERTaBASE fine-tuned model in some datasets, such as GHC, HATEXPLAIN, and MLMA. Overall, the DAVIDSON dataset achieves the highest performance among all the datasets, with a macro-F1 score of 83.30. In contrast, the MLMA dataset obtains the lowest macro-F1 score of 54.35, which is expected given its complexity arising from the low inter-annotator agreement. Notably, the performance on the HATEVAL dataset (65.38) exhibits an improvement over the participant results’ mean (44.84) in the competition (Basile et al., 2019). On the DYNABENCH dataset, the FLAN-T5 model’s result (58.08) is similar to that of fine-tuning the RoBERTaBASE fine-tuned model (61.76), despite the dataset’s complexity with a large number of challenging perturbations that make it harder for models to detect hate speech accurately. Finally, we compared our approach with Perspective API, the most popular commercial tool for toxicity detection. FLAN-T5 is outperforming it in 6 cases out of 8, demonstrating prompting to be a more accurate
solution. While the varying degrees of difficulty across datasets in hate speech detection is demonstrated in these results, the potential of instruction fine-tuned models to achieve state-of-the-art performance on various benchmarks without requiring fine-tuning on a specific dataset is highlighted. This insight is especially valuable for subjective tasks like hate speech, where the complex nature of labeling this phenomenon can make it challenging to find labeled datasets.

heading: 5 Results on Multi-Lingual Datasets
text: We also investigated the effectiveness of ZSL with prompting in a multilingual context, which is often more challenging due to the scarcity or unavailability of data. We present the outcomes achieved by multilingual models: multilingual XLM-R (Conneau et al., 2020) as encoder model and mT0 and FLAN-T5 as instruction fined-tuned models. The prompt has been written in English following the same templates presented in Section 3 and using the best-performing verbalizer pair respectful-toxic. We use the experimental settings adopted in Nozza (2021), comparing our method with their fine-tuned XLM-R model. Thus, the dataset comprises English (EN), Spanish (ES), and Italian (IT). The HatEval (Basile et al., 2019) shared task dataset on hate speech against immigrants and women on Twitter is adopted for English and Spanish. For Italian, two different corpora proposed for Evalita shared tasks (Caselli et al., 2018) are considered: the automatic misogyny identification challenge (AMI) (Fersini et al., 2018) for hate speech towards women, and the hate speech detection shared task on Facebook and Twitter (HaSpeeDe) (Bosco et al., 2018) for hate speech towards immigrants.
The results are shown in Table 5. Regarding the ZSL approaches, the instruction fine-tuned models outperform XLM-R, with FLAN-T5 achieving the highest macro-F1 score on all languages. The
ZSL models, as expected, did not outperform the fine-tuned XLM-R. However, the results obtained from the ZSL models are still considered adequate. Spanish, in particular, achieves comparable results with FLAN-T5 to the fine-tuned XLM-R. FLANT5 achieves better results in English because it is not affected by overfitting issues that arise during training (Nozza, 2021). These findings suggest that prompting with instruction fine-tuned LLMs is a promising method for hate speech detection in both mono and multilingual settings, without language-specific fine-tuning.

heading: 6 Related Work
text: Hate speech classification received increased attention in recent years. Supervised learning methods are the most common (Poletto et al., 2021; Fortuna et al., 2022). Among these methods, fine-tuning transformer-based LLMs emerged as the dominant paradigm (Plaza-del-Arco et al., 2020; Sarkar et al., 2021; Singh and Li, 2021; Caselli et al., 2021; Kirk et al., 2022, inter alia). However, they face significant challenges, like the limited availability of labeled data, especially in languages other than English (Schmidt and Wiegand, 2017; Fortuna and Nunes, 2018), and the subjective nature of hate speech, which varies based on cultural background, personal experiences, and individual beliefs.
LLMs have led to innovative techniques like prompting (Liu et al., 2023) that use zero-shot and
few-shot learning paradigms without needing labeled data. Recent works have explored these new techniques for hate speech detection. Chiu et al. (2021) use the prompts “Is the following text sexist? Answer yes or no” and “Classify the following texts into racist, sexist, or neither” to detect hate speech, with GPT-3 showing that LLMs have a role to play in hate speech detection. Schick et al. (2021) explore toxicity in LLMs using comparable prompts to self-diagnose toxicity during the decoding. They use the RealToxicityPrompts dataset (Gehman et al., 2020). (Goldzycher and Schneider, 2022) develop NLI-based zero-shot hate speech detection approaches using prompts as a hypothesis as proposed by Yin et al. (2019). Their results outperform fine-tuned models. Our work ZSL for hate speech classification differs from previous approaches as follows. (1) We provide a comprehensive evaluation of ZSL with prompting on multiple benchmark datasets, offering new insights into the effectiveness of this technique. (2) We explore the impact of the selection of verbalizers and models for the task, and (3) we compare the performance of encoder models with the recent LLMs based on instruction fine-tuning.

heading: 7 Conclusion
text: This paper presents a comprehensive evaluation of ZSL with prompting for hate speech classification. We have compared both encoder and instruction fine-tuned LLMs. Our experiments across different benchmark data sets showed that ZSL with prompting is a promising option to address the challenges presented in supervised learning systems. However, it also highlights the importance of carefully selecting the model and appropriate verbalizers, as they can significantly affect performance. Our results also show that recent LLMs based on instruction
fine-tuning play an essential role in hate speech detection. Further exploration of prompt formulation could lead to their continued growth in this area. Additionally, our multilingual experiments show that our proposed methods can be applied to other languages with comparable results.
Future research could investigate the bias presence (Dixon et al., 2018; Attanasio et al., 2022) and robustness (Röttger et al., 2021, 2022) of ZSL prompting for hate speech detection models, also in multilingual settings.
Limitations While promising, our work presents limitations that need to be acknowledged. Firstly, we did not explore the best verbalizers for instruction finetuned language models, which could have further enhanced the performance of the models explored in this study, due to computational cost and the specific goals of the research. Secondly, we selected benchmark datasets based on their popularity and diversity, which might not be representative of all possible datasets in hate speech detection. We also acknowledge that, in addition to the languages examined in this paper, there are a number of other languages that may present unique challenges and characteristics for detecting hate speech. Our decision as to which languages to include in the multilingual experiment was based on a direct comparison with state-of-the-art research. Finally, we utilized the latest open-source language models for our experiments, but we did not explore other recent language models, such as the GPT family, primarily because they are not open and reasonably reproducible3, and therefore the community may encounter challenges in replicating our results. These limitations provide directions for future research to improve and expand upon our work.
Ethics Statement To ensure data privacy and protection, we use publicly available benchmark datasets for hate speech detection and do not collect any personal or sensitive information. Additionally, we acknowledge that the detection of hate speech can be a sensitive topic; therefore, we report the results of our experiments in a responsible and appropriate manner. Lastly, we acknowledge that language models trained on large datasets have the potential to perpetuate bias and discrimination, and we strive to
3https://hackingsemantics.xyz/2023/ closed-baselines/
mitigate these risks by carefully selecting and evaluating our models and verbalizers to ensure fairness and impartiality.

heading: Acknowledgements
text: This project has in part received funding from Fondazione Cariplo (grant No. 2020-4288, MONICA). The authors are members of the MilaNLP group and the Data and Marketing Insights Unit of the Bocconi Institute for Data Science and Analysis.

heading: A Dataset Details
text: Vidgen et al. (2021) (DYNABENCH) introduced a novel framework for dynamically creating benchmark corpora. The task assigned to the annotators involved identifying adversarial examples, which are instances that would be classified incorrectly by the target model and are particularly challenging to detect. The dataset contains a significant proportion of hateful entries, accounting for 54% of the dataset.
Kennedy et al. (2020b) (MHS) gathered a large collection of comments from diverse social media platforms (YouTube, Twitter, and Reddit). To label the comments, they used a crowdsourcing platform where four different ratings were given to each comment. To ensure a comprehensive assessment, the authors made certain that every annotator evaluated comments that spanned the entire hate speech scale. Since the dataset is annotated with a continuous hate score, we used a threshold set to binarise the problem: if value < -1 → 0 and if value > 0.5 → 1.
Kennedy et al. (2022) (GHC) presented the Gab Hate Corpus, a multi-label English dataset of posts sourced from gab.com, a social networking platform. To label the comments, at least three annotators labeled them under one of the following categories: Call for Violence, Assault on Human Dignity, or Not Hateful. Following Kennedy et al. (2020a), we aggregate the first two for obtaining the hateful class.
Basile et al. (2019) (HATEVAL) created the HatEval corpus for the HatEval campaign in SemEval. The dataset consists of tweets that were manually
annotated via crowdsourcing for hate speech. To collect the tweets, they follow three different strategies: (1) monitoring potential victims of hate accounts, (2) downloading the history of identified haters, and (3) filtering Twitter streams with keywords, i.e., words, hashtags, and stems. The corpus contains a total of 24,802 tweets.
Talat and Hovy (2016) (HSHP) provided a dataset consisting of 16,914 tweets that were collected using Twitter’s streaming API and filtered using a set of hate speech-related keywords related to religious, sexual, gender, and ethnic minorities. The tweets were then manually annotated by two annotators for the presence of hate speech.
Davidson et al. (2017) (DAVIDSON) created a dataset of 24,802 tweets annotated for the presence of hate speech and offensive language. The tweets were crawled using keywords related to a hate speech lexicon. Each tweet was labeled by three or more people into one of three categories: hate speech, offensive language, or neither. We aggregate the first two for obtaining the hateful class.
Mathew et al. (2021) (HATEXPLAIN) collected English posts from Twitter and Gab social media platforms. Afterward, a crowdsourcing platform was employed to categorize each post into three categories: hate speech, offensive speech, or normal speech. In addition to this, the annotators were tasked with identifying the target communities mentioned in the posts, as well as the specific portions of the post which formed the basis of their labeling decision. Finally, the majority voting decision was used to determine the final label. By combining the hate and offensive targets, the hateful class was formed. We combine the hate and offensive posts to obtain the hateful class.
Ousidhoum et al. (2019) (MLMA) presented a multilingual multi-aspect hate speech dataset comprising English, French, and Arabic tweets that encompass various targets and hostility types. Each tweet is labeled by 5 annotators, and then the majority vote is used to decide the final label. The average Krippendorff scores for inter-annotator agreement (IAA) are 0.153, 0.244, and 0.202 for English, French, and Arabic, respectively.
B Implementation Details We implement the fine-tuned version of RoBERTaBASE with the following hyperparameter configuration for training: epochs are set to 3, batch size to 8, and the number of epochs
to 3. For the ZSL models, we used the default hyperparameters presented in Hugging Face. We fine-tune RoBERTaBASE for three epochs. We perform 5-fold partitions and report the results on the test set.
Hugging Face model cars BERTBASE-uncased4, RoBERTaBASE5, RoBERTaLARGE6, DeBERTaV3-BASE7, DeBERTaV3-LARGE8, XLMRoBERTaLARGE9, mT010, and FLAN-T511.
Computing Infrastructure We run the experiments on one machine with the following characteristics: it is equipped with three NVIDIA RTX A6000 and has 48GB of RAM.
4https://huggingface.co/bert-base-uncased 5https://huggingface.co/roberta-base 6https://huggingface.co/roberta-large 7https://huggingface.co/microsoft/
deberta-v3-base 8https://huggingface.co/microsoft/ deberta-v3-large 9https://huggingface.co/xlm-roberta-large
10https://huggingface.co/bigscience/mT0-xxl 11https://huggingface.co/google/flan-t5-xl

heading: Future Work
text: Future research could investigate the bias presence (Dixon et al. , 2018; Attanasio et al. , 2022) and robustness (Röttger et al. , 2021, 2022) of ZSL prompting for hate speech detection models, also in multilingual settings. Limitations While promising, our work presents limitations that need to be acknowledged. Firstly, we did not explore the best verbalizers for instruction finetuned language models, which could have further enhanced the performance of the models explored in this study, due to computational cost and the specific goals of the research. Secondly, we selected benchmark datasets based on their popularity and diversity, which might not be representative of all possible datasets in hate speech detection. We also acknowledge that, in addition to the languages examined in this paper, there are a number of other languages that may present unique challenges and characteristics for detecting hate speech. Our decision as to which languages to include in the multilingual experiment was based on a direct comparison with state-of-the-art research. Finally, we utilized the latest open-source language models for our experiments, but we did not explore other recent language models, such as the GPT family, primarily because they are not open and reasonably reproducible3, and therefore the community may encounter challenges in replicating our results. These limitations provide directions for future research to improve and expand upon our work