H ate COT : An Explanation-Enhanced Dataset for Generalizable Offensive Speech Detection via Large Language Models (2024)

Abstract: The widespread use of social media necessitates reliable and efficient detection of offensive content to mitigate harmful effects. Although sophisticated models perform well on individual datasets, they often fail to generalize due to varying definitions and labeling of “offensive content.” In this paper, we introduce HateCOT, an English dataset with over 52,000 samples from diverse sources, featuring explanations generated by GPT-3.5Turbo and curated by humans. We demonstrate that pretraining on HateCOT significantly enhances the performance of open-source Large Language Models on three benchmark datasets for offensive content detection in both zero-shot and few-shot settings, despite differences in domain and task. Additionally, HateCOT facilitates effective K-shot fine-tuning of LLMs with limited data and improves the quality of their explanations, as confirmed by our human evaluation.

## introduction as social media has become indispensable to modern discourse, this channel of communication has amplified the propagation of offensive content. speech that promotes hateful sentiments thrives on such platforms, leading to real and significant harm on their audience @xcite . however, "offensive content" is still a contested construct, as what is and is not allowed varies by platform. in research, different approaches analyze semantically similar but still distinct concepts: cyber-bullying, toxicity, sexist, racist, hate etc. @xcite @xcite , further highlighting this contestedness. compounding the challenge, reliable detection of offensive content typically requires significant amounts of data. sophisticated models tend to be data-hungry, and the process of curating a dataset tailored to a specific use case can be costly, timeconsuming, and emotionally challenging for annotators @xcite . the typical pipeline consists of collecting samples based on topic-relevant key words, then recruiting either crowdworkers or experts to annotate data before developing classification models @xcite . each step incurs investment and may inject subtle idiosyncrasies proportionate to the size of the downstream dataset, further limiting transferable usefulness to related tasks @xcite . the size of a dataset also does not necessarily guarantee cross-domain transferrability @xcite . in practical settings, users may desire transparency from social media platforms. therefore, the ability to provide human-understandable justification based on platform-specific policy becomes an attractive feature for content moderation. nevertheless, current techniques often still fail to offer intuitive explanatory signals @xcite @xcite . in this work, we attempt to simultaneously reduce the cost of data curation, enhance crossdataset generalization, and address the necessity of explainable decisions for offensive content detection. specifically, our main contributions are: ## building hatecot we first describe the process to identify the candidate datasets from literature, and the procedure to obtain annotation-guided explanations from these samples (section 3.1). we then perform a set of validation experiments to optimize the data's parameters for downstream tasks before augmenting our corpus to its eventual size (section 3.2). ## quality of explanations in addition to the enhanced classification prowess, we investigate whether pretraining on hatecot also improves the quality of explanation llms. to this end, the following 2 human quality assurance (qa) experiments are conducted. in qa 1, we assess if human annotators prefer the explanations generated by the base or hatecot-pretrained llms (cot-t5-xl, llama 7b, llama 13b). in qa 2, we perform in-domain k-shot finetuning on the aforementioned models and examine how annotators evaluate their generated explanations. an additional assessment of target identification is presented in appendix a.6. ## conclusion we show empirically that our hatecot dataset considerably enhances offensive speech detection even with limited training data while producing highquality explanations. we invite future research to explore other benefits of llm-augmented data and extend them to other related low-resource areas.