Data-Efficient Methods For Improving Hate Speech Detection (2023)

Abstract: Scarcity of large-scale datasets, especially for resource-impoverished languages motivates exploration of data-efficient methods for hate speech detection. Hateful intents are expressed explicitly (use of cuss, swear, abusive words) and implicitly (indirect and contextual). In this work, we progress implicit and explicit hate speech detection using an input-level data augmentation technique, task reformulation using entailment and cross-learning across five languages. Our proposed data augmentation technique EasyMix, improves the performance across all english datasets by ~1% and across multilingual datasets by ~1-9%. We also observe substantial gains of ~2-8% by reformulating hate speech detection as entail problem. We further probe the contextual models and observe that higher layers encode implicit hate while lower layers focus on explicit hate, highlighting the importance of token-level understanding for explicit and context-level for implicit hate speech detection. Code and Dataset splits - https://anonymous.4open.science/r/data_efficient_hatedetect/

## introduction deep learning based methods @xcite @xcite have shown impressive results in detecting hate speech. transformer based models @xcite @xcite @xcite have further pushed the state-of-the-art by leveraging large amount of unlabeled data in a self-supervised manner. various hate speech detection datasets have been contributed in textual @xcite @xcite , audio @xcite and visual @xcite domains. however, these algorithms are data-hungry and motivate development of algorithms which are data-efficient. to tackle this, we introduce an input-level data augmentation technique easymixup and improve hate speech detection in monolingual and multilingual settings. easymixup is inspired by mixup based augmentation techniques which are broadly categorized into input-level mixup @xcite @xcite and hidden-level mixup @xcite . easymixup follows the input-level paradigm and leverages a simple observation that the label of a hateful instance is preserved on concatenation with a hateful or nonhateful instance. similarly, label of a non-hateful instance does not change on concatenation with another non-hateful instance. we also study the efficacy of reformulating hate speech detection as entailment-style problem. we extend the work by @xcite and perform detailed experiments under implicit, explicit and multilingual settings. we observe that monolingual entailment performs better than english based entailment. this observation is intuitive because the models are pretrained using pair of sentences from same language and monolingual entailment reflects the same settings. majority of the existing textual datasets focus on explicit hate speech where swear, cuss, abusive words are used to express the hateful intent. in contrast, implicit hate speech employs subtle, indirect and contextual ways for expressing hate speech making it extremely harmful and difficult, as shown in @xcite . acknowledging this difference of expression, we explore the relationship between explicit and implicit hate speech using cross-learning and observe strong correlations. we also perform probing experiments and observe that lower layers focus on explicit hate speech while higher layers are responsible for encoding implicit hate speech. this alludes to the hypothesis that implicit hate speech is more contextual in nature and requires more understanding, while explicit hate speech can be detected by leveraging lower-level information. in summary, our main contributions are: figure 1: (top) mixing hateful (red) samples with either hate (red) or non-hate (green) samples doesn't change the final label. similarly mixing two non-hate (green) samples preserve the final label. (bottom) posing hatespeech classification as entailment task. [best viewed in color] 2 methodology 2.1 easymixup easymixup is an input-level data augmentation technique where we leverage the observation that ground truth label of a hateful sample does not change on concatenation with another hateful or non-hateful samples. similarly, concatenation of a non-hate sample with another non-hate sample results in a novel non-hate sample as shown in figure 1(top). more formally, let's say (s i , y i ) is the sentence and it's corresponding label @xmath0, non-hate} in a minibatch s and d is the entire dataset, @xmath1, y 0 ), (s 1 , y 1 ), ..., (s n , y n )|(s i , y i ) ∈ d} for every sample in the batch, s @xmath2, we randomly select (s i , y i ) ∈ d with s i ̸ = s i and augmentation probability p aug to create new augmented sample: s @xmath3, s i ), y @xmath4: ϕ(s i , s i ) = concat(s i ; s i ) @xmath5; s i ) , otherwise ## conclusion in this work, we introduced a novel inputlevel data-augmentation technique, easymixup which shows performance gains over monolingual and multilingual settings. we also explored reformulation of hate speech classification as entailment-style problem and achieved substantial performance gains using monolingual entailment. we also performed layer probing to find that higher layers encode implicit hate information, while lower layers are more focused on explicit hate speech highlighting the contextual nature of implicit and token-level dependence of explicit hate speech. in future work, we would like to explore how easymixup and entailment-style perform when ensembled together in both mono, multi-lingual settings.