Probing Critical Learning Dynamics of PLM s for Hate Speech Detection (2024)

Abstract: Despite the widespread adoption, there is a lack of research into how various critical aspects of pretrained language models (PLMs) affect their performance in hate speech detection. Through five research questions, our findings and recommendations lay the groundwork for empirically investigating different aspects of PLMs’ use in hate speech detection. We deep dive into comparing different pretrained models, evaluating their seed robustness, finetuning settings, and the impact of pretraining data collection time. Our analysis reveals early peaks for downstream tasks during pretraining, the limited benefit of employing a more recent pretraining corpus, and the significance of specific layers during finetuning. We further call into question the use of domain-specific models and highlight the need for dynamic datasets for benchmarking hate speech detection.

## introduction the transformer-based language models (lms) @xcite @xcite have been a game-changer in nlp. consequently, researchers have adopted pretrained language models (plms) to detect hate speech. however, the choice of the plm employed for hate detection is often arbitrary and relies on default hyperparameters @xcite . despite plms being prone to variability in performance @xcite , there is limited research comparing training settings for subjective tasks like hate speech detection. note, this study follows the definition of hate speech provided by @xcite -"a language targeted at a group or individual intended to derogatory, humiliate, or insult." research questions. figure 1 provides an overview of our research questions (rq). we broadly study two critical elements of plms by analyzing (i) the impact of different pretraining * equal contribution figure 1 : research overview: the study comprises five research questions @xcite to empirically analyze the pretraining and finetuning strategies for plm variants employed for hate detection. a typical plm-inspired pipeline involves working with one or more checkpoints, i.e., plm model weights obtained after pretraining. the checkpoint is then finetuned for downstream tasks by keeping one or more layers of plm trainable along with a trainable classification head (ch). finally, the plm + ch generates predictions on incoming test samples. dataset source labels platform of origin time of collection dataset size train dev test waseem waseem and hovy (2016) h, nh twitter prior to jun '16 6077 2026 2701 davidson davidson et al. (2017) h, nh twitter prior to mar '17 13940 4647 6196 founta founta et al. (2018) h, nh twitter march '17 -april '17 33293 11098 14798 olid* zampieri et al. (2019) off, not twitter prior to jun '19 9930 3310 860 hatexplain mathew et al. (2021) h, nh twitter & gab jan '19 -june '20 11303 3768 5024 dynabench vidgen et al. (2021) h, nh synthetic (human-generated) sept '20 -jan '21 23143 7715 10286 toxigen hartvigsen et al. (2022) h, nh synthetic (llm generated) prior to jul '22 141159 47054 62738 contributions. while previous studies on hate speech modeling perform hyperparameter tuning, they examine either a single architecture @xcite , a single plm @xcite , or a single dataset @xcite . one of our work's core contributions is to examine different plms, seeds, and datasets under one study. consequently, we observe that the dynamics of plms for hate detection differ significantly from the other use cases @xcite . there are interesting trends in pretraining learning dynamics, with peaks at early checkpoints. we find pretraining over newer data unhelpful. consequently, on the pretraining end, we observe that general-purpose plms with a complex classification head can be as efficient as domain-specific plms @xcite . unlike bert @xcite , for mbert finetuning, the last layer is not the most effective for hate detection. to the best of our knowledge, we are the first to evaluate plms' learning dynamics for hate speech detection foot_0 . overall, the study examines seven datasets under diverse settings. the aim is not to derive a consistent pattern but rather to examine whether any pattern exists among the datasets w.r.t. different settings discussed in the rqs. ## related work early attempts at hate speech detection employed linguistic features @xcite and recurrent architectures @xcite . however, with the arrival of the transformer architecture @xcite , hate speech tasks also gained a significant boost @xcite @xcite . however, most studies adopted the default setting to finetune plms. meanwhile, deep learning models are criticized to be black boxes. while heuristics such as lime @xcite and shap (lundberg and lee, 2017), among others, attempt to make these models interpretable, they are limited to perturbations in the input space rather than the latent space. more recently, work on mechanistic interpretability @xcite attempts to understand how transformers build their predictions across layers. control over high-level properties of the generated text, such as toxicity, can be obtained by tweaking and promoting certain concepts in the vocabulary space @xcite . interpretability @xcite , finding best practices @xcite and sufficiency @xcite in hate speech have always been open research areas. while toxicity and biases encoded by pretrained plms @xcite is an essential area of research, our work focuses on the downstream finetuning of plms for hate detection. note on dataset characteristics. during our preliminary analysis, we performed data drift experiments to see how distinguishable the hs datasets are from each other @xcite . from ## analysis of the pretrained backbones variability in pretraining strategies should lead to variability in the performance of downstream tasks. to explore this for hate speech detection, we start with analyzing pretraining weight initialization on the final checkpoint and then move to investigate intermediate checkpoints and pretraining corpus. ## analysis of the finetuning schemes during finetuning, the plm layers closer to the classification head capture the maximum taskspecific information @xcite . hence, setting the lower layers parameters untrainable is a standard finetuning practice. while layer-wise analyses have been explored in various nlp tasks @xcite , a comprehensive examination across models, datasets and finetuning scenarios has been notably absent in the hate speech domain. experiments in this section are run on four bert variants -bert (devlin et al., 2019), bertweet (nguyen et al., 2020), hatebert @xcite , and multilingual-bert (mbert) @xcite . ## takeaways and recommendations this section summarises the major takeaways that would allow practitioners to make effective choices when modeling plms for hate speech detection. bert bertweet hatebert mbert combined model 0.0 0.2 0.4 0.6 0.8 1.0 % distribution across settings r1 f r1 nf r2 f r2 nf r3 f r3 nf r4 f r4 nf (a) best region distribution. bert bertweet hatebert mbert combined model 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 % distribution across settings r1 f r1 nf r2 f r2 nf r3 f r3 nf r4 f r4 nf (b) worst region distribution. combined captures the overall trend across all bert-variants and datasets. region r 1 includes layers l 1 to l 3 , r 2 from l 4 to l 6 , r 3 from l 7 to l 9 and r 4 from l 10 to l 12 . suffix f implies that the region was frozen while other regions were trainable, and the n f suffix implies all other regions were frozen while only that region was trainable. in each row, two columns with the same dataset name as the one in the row correspond to in-domain evaluation, the others correspond to out-of-domain evaluation. as the use of generative ai tools for crowdsourcing is on the rise @xcite , it is imperative to equip hate speech researchers to deal with a broader ai-assisted system than just finetuning plms. moreover, using computational methods at every step of the hate detection pipeline should always be humanaided. 4. in rq4, we reinstated the status quo of finetuning the last few layers to obtain the best performance to largely hold for hate detection. yet, in the case of mbert, we observed that the middle and lower layers are much more critical. we recommend that tasks employing multilingual or non-english hate speech detection using mbert should start with keeping the middle layers unfrozen for finetuning. by comparing four bert variants on seven datasets and three seeds, it appears that the region-wise performance of plms is a characteristic of the underlying plm and the task domain at hand and is less impacted by variation in datasets. such intuitions can help narrow the experiments one has to run to obtain better classification configurations. further, based on the best seed, layer, and plm combinations obtained in rq4 ( ## conclusion due to the subjective nature of hate speech, no standard benchmarking exists. we take this opportunity to explore the patterns in finetuning plms for hate detection through a series of experiments over five research questions. we hope each experiment in this study lays the ground for future work to improve our understanding of how plms model hatefulness and their deployment to detect hate. in the future, we would like to extend our analysis against adversarial settings, bias mitigation, a broader language set, and auto-regressive llms.