SlowGuess/NAACL_2022_OCR huggingface dataset:
Our final contextual regularization loss function is thus a linear combination of the losses mentioned in Equation 20 and 22 as described below:
$$
\mathcal {L} _ {\text {C o n t e x t u a l}} ^ {\text {r e g r e s s i o n}} = \lambda \mathcal {L} _ {\mathrm {M S E}} + (1 - \lambda) \mathcal {L} _ {\mathrm {C M S E}} \tag {23}
$$
We experiment with the downstream task of hate speech detection. We establish the effectiveness of our proposed approaches by demonstrating that they are: (i) mutually independent of each other, (ii) independent of the base language model used, and (iii) applicable across various social network datasets. We validate these claims by comparing our approaches with the following baselines:
(i) To establish mutual independence among the training phases, we train a base language model (BERT from Devlin et al. (2019)) with each of the training phases separately. An improvement over the performance of the base model in each phase would indicate the advantage of our approach over naive training.
(ii) To establish independence from the base language model, we train multiple baseline language models during each of the above-mentioned separate training phases (HateBERT during Phase
I; BERT and HateXplain during the remaining phases). Consistent performance improvement irrespective of the language model would again indicate the advantage of our approach.
(iii) We use three datasets from two different social networks and two types of tasks (classification and regression). This establishes the capability of our methods to solve a range of heterogeneous tasks across contrasting datasets.
We next describe the datasets and the implementation details of our approaches.
Social networks: In particular we experiment with two popular social networks (a) gab.ai (GAB), and (b) reddit.com (Reddit). Our choice of the dataset was guided by the availability of the context graph and additional data collection time. Baumgartne (2018) had scraped GAB and made the network freely available for academic use. On the other hand, Reddit has an  $\mathsf{API}^4$  available to get the public domain data. Therefore, these websites were favorable for our experiments. Since Reddit involved additional data collection (a time consuming process), we chose a popular dataset that contains less than 10,000 datapoints.
Annotated hate speech data: We use the following english hate speech datasets for our experiments (See Table 1 for more information on dataset statistics) – (i) HateXplain-GAB dataset (Mathew et al., 2021) (contains data from GAB), (ii) LTI-GAB dataset (Qian et al., 2019) (contains data from GAB) and, (iii) Reddit (Hada et al., 2021) (contains data from Reddit).
We use only the GAB subset of the annotated data from both the datasets (i) and (ii), because the social network context graph for GAB is publicly available. The GAB subset of dataset (i) has around 10K annotated data samples, which are already divided into 80-10-10 train-val-test split. Dataset (i) has three class annotations (hate speech, offensive, and normal). Dataset (ii) contains intervention sen
<sup>4</sup>https://www.reddit.com/dev/api/
Phase	Model	HX-GAB5(3 class)	LTI-GAB (2 class)	Ruddit (regression)
Acc	Macro-F1	Acc	F1 (toxic)	MSE	MAE
Existing approaches as baselines	TF-IDF	0.6337	0.5604	0.8993	0.8824	0.1128	0.2651
BERT	0.6763	0.6376	0.9141	0.9019	0.1041	0.2521
HateXplain†5	0.6905	0.6511	0.9177	0.9062	0.1036	0.2520
Continual pre-train-ing (CP phase)	HateBERT	0.6843	0.6458	0.9166	0.9040	0.1029	0.2516
HateLM	0.6882	0.6493	0.9174	0.9058	0.1018	0.2474
User Anchored self-supervision (UA phase)	BERT + UA	0.6950	0.6632	0.9192	0.9089	0.0994	0.2367
HateXplain + UA	0.7058	0.6680	0.9211	0.9105	0.0989	0.2329
HateLM + UA	0.7087	0.6711	0.9215	0.9117	0.0958	0.2229
Contextual Regu-larization (CR phase)	BERT + CR	0.6819	0.6452	0.9176	0.9051	0.1019	0.2438
HateXplain + CR	0.6935	0.6555	0.9199	0.9088	0.1011	0.2410
HateLM + CR	0.6924	0.6558	0.9200	0.9096	0.0995	0.2342
UA and CR phase together	BERT + UA + CR	0.7017	0.6673	0.9211	0.9104	0.0968	0.2314
HateXplain + UA + CR	0.7099	0.6702	0.9236	0.9122	0.0955	0.2291
CRUSH*	0.7133	0.6749	0.9234	0.9149	0.0921	0.2188
Table 2: Experimental outcomes of our approaches. Our model CRUSH ≡ HateLM + UA + CR. For reporting results (except CR and UA+CR phases) the encoder models were attached with a classifier (refer section 4.2) and fine-tuned (using the loss mentioned in Equation 13). Across the columns: we show the results with three datasets. In HateXplain (HX)'s and Learning to Intervene (LTI)'s results, higher metrics are better. In RDDit results, lower metrics are better. Across the rows: we show the various models grouped by the phase of training along with corresponding baselines. In each group, we have indicated the best-performing models' results corresponding to the evaluation metric in bold. The overall best performing models' results have additionally been italicized. The models denoted by  $\dagger$  (the best competing baseline) and  $*$  (CRUSH) are significantly different (M-W U test with  $p < 0.05$ ) across all datasets.
Figure 3: [Best viewed in color] F1-scores of correctly classified posts grouped by three user contexts and three post contexts. The improvement along the user contexts and post contexts demonstrate the utility of the CR phases (user & post context based).
tences along with which sentences to intervene and a binary label - hate and non-hate. We use a 90-10 train-test split with the random seed 2021, and among the training set we use  $10\%$  randomly sampled data for validation. Dataset (iii) was collected from Reddit and is labeled for regression task. The dataset contains ratings corresponding to the measure of hate within a sentence ranging between  $-1$  and  $1$ , with higher numbers corresponding to a higher extent of hate. There are about 6K examples
in this dataset. We use a train test split similar to that we used in the dataset (ii).
Our pre-processing procedure for all the textual data, both labeled and unlabelled is exactly the same as that of HateXplain. (Mathew et al., 2021). Unlabelled data for self-supervision: We get the threads corresponding to the annotated Reddit datapoints using the Reddit API and use that as our unlabelled corpora for self-supervision in case of the regression task. For GAB, we use the whole network datapadump available on Pushshift (Baumgartne, 2018) as mentioned in Section 3.2.
For our encoder, we use the pre-trained bert-base-uncased that is available on huggingface  $^{6}$ . By continual pre-training on this model, we obtain our HateLM using the text sequences from GAB and RAL-E as described in section 3.1. Afterward, we obtain a 768 dimensional pooled output from our encoder which is then be passed onto the classifier/regressor head. For both our classifier and regressor model, we select a neural
<sup>5</sup>HateXplain refers to both a dataset and the corresponding model provided in the paper. However, since we only use the GAB subset of the HateXplain dataset, we require to fine-tune the HateXplain model on this subset, for it to be comparable to other models, and therefore it is considered a baseline.
Figure 4: [Best viewed in color] Comparison of model error in low data regime across the three datasets. Our model (black solid line) consistently outperforms existing approaches that do not use the social network context. For plots showing comparisons with non-contextualized LMs (e.g., TF-IDF) as well, refer to Figure 5 in Appendix.
Word	BERT	HateLM
black	everywhere, free, not	racist, evil, stupid
muslim	accepted, optional, allowed	terrorist, evil, shit
jews	persecuted, excluded, present	idiots, bad, stupid
men	equal, free, citizens	evil, people, bad
women	excluded, only, bias	dumb, evil, stupid
Table 3: Top-3 next word predictions by each LM when the corresponding word is given as prompt. The hateful words as the output of the next word prediction demonstrate the effectiveness of the hate-infusion into LM.
network consisting of a 2 Fully-connected layers. The hidden layer for the same is chosen to have 128 dimensions along with a ReLU activation. The input dimension is dependent on the output of the encoder, and in our case is set to 768. The output dimension is 1, 2 or 3 corresponding to the regressor, binary and ternary classifier. We used Adam optimizer with batch size 48, max seq length 128 and learning rates  $3e - 6$  and  $2e - 5$  for our encoder and classifier/regressor respectively. Our approaches do not require additional context during inference (see Figure 2(d)).
We frame the ablation studies to answer a series of interesting questions. These are:
Q1: How much does self-supervised learning objective during pre-training help in the downstream task (if at all)?
Q2: Similarly, how much does contextual regularization during fine-tuning help?
Q3: Do these two approaches work only in con-
junction with each other?
In addition to the above, we perform the following ablation experiments to validate the usefulness of various components of our model:
Q4: How much of the performance can be attributed to the detection of isolated hateful words without any contextual information in place? To answer this question, we use the TF-IDF (Sammut and Webb, 2010) vector embedding of each text sequence. since TF-IDF computes representations solely based on the frequency of words without any contextual information.
Q5: Have the hateful words been contextually incorporated into the hate-infused language model? Here, we prompt our hate-infused language model and study the next word it predicts.
Q6: How do these approaches perform in a few-shot setting?
The Table 2 compares the performance of our methods with related models and baselines. We report the performance metrics according to the test sets of the corresponding datasets.
[1] It is evident from Table 2 that both CP and UA phases (self-supervised phases) lead to improvements over the baselines. This answers Q1. To answer how useful regularization was we can look at the CR phase results which are again better than the baselines thus answering Q2.
Further, we can also see that UA and CR phase results individually beat the baseline. Furthermore, combined UA+CR phase outperforms all models. Hence, the conjunction hypothesis is valid thus answering Q3.
Post (along with ground-truth label)	BERT	HateLM	CRUSH
if your humor is based on racism homophobia sexism and rape you're not f**king funny go home (non-hate)	X	✓	✓
<user> ah man i f**king hate you so much (hateful)	X	✓	✓
the french government position is that france is made stronger by the immigrants it is bribing to leave porter (non-hate)	X	X	✓
i am presuming he means the standard left wing idiots lots of hypocritical women amongst them think maria ladenburger obvi-ously you are against these muhammedans (hateful)	X	X	✓
a covington catholic a native american indian and a black hebrew israelite walk into a bar (non-hate)	X	X	X
if money was grown on trees women would be dating monkeys oh wait never mind (hateful)	X	X	X
Table 4: Qualitative results using text sequences from HateXplain dataset.  $\checkmark$  indicates sentences were classified properly by the corresponding models while  $\times$  indicates incorrect classification. MLM based training of BERT and HateLM does not seem to capture the complexities of hate speech properly, thus CRUSH outperforms them.
[2] Q4 is answered by the considerable improvement over the TF-IDF baseline as can be seen in Table 2. In addition, Figure 3 shows the advantages gained by using CRUSH over HateLM. It demonstrates both the user-level and the post-level contextual information from the network incorporated into our CRUSH model are individually helpful over models that do not have social network context information.
[3] Table 3 answers Q5 by showing some of the prompt completion results for BERT vs our Hate-infused LM. It can be noticed that although HateLM definitely has been incorporated with racial and ethnic hate, it surprisingly does not discriminate between genders.
[4] Figure 4 shows the few-shot training results with small percentages of data sampled from our training datasets (2% to 20% data points) uniformly at random. Intuitively our model is already able to outperform all other baselines consistently even without context because of the self-supervised training procedures we propose during the CP and UA phases which captures the hate & users' style+bias. This is evident from the results of CP & UA phases in Table 3. So later adding context in few-shot setting simply further enhances our model thus answering Q6 (see Appendix for full results).
Discussion of some qualitative examples: Table 4 presents a few text sequences from the HateXplain dataset and their corresponding results (classification/misclassification) for the models - BERT, HateLM, and CRUSH. It can be noticed that the first two sentences are properly classified by HateLM and CRUSH but not by vanilla BERT.
That leads us to believe that BERT predictions are heavily dependent on the token occurrences rather than the context in the sentence where the tokens have occurred. As for the next two sentences, it is evident that the sentence representations learnt by CRUSH (due to UA + CR phases of training, see Section 3.2, 3.3) are superior to those learnt by the HateLM, hence it classifies these text sequences better than simple MLM based training. Finally, the last two sentences indicate that CRUSH (along with the other approaches) still lacks the ability to identify humor, sarcasm, and implicit hate speech, which are known to be difficult problems.
In this paper, we provide approaches to infuse social network context using the self-supervised user-attribute pre-training objective combined with the contextual regularization objective on top of traditional MLM for hate speech tasks. We empirically demonstrate the advantage of our methods by improving over existing competitive baselines in hate speech detection and scoring tasks across three different datasets. We also show that our method performs superior in the low data regime as well when compared to existing approaches. We also do ablations to understand the benefits of each objective separately. Future work include exploiting the relations among users, using different base models capable of incorporating longer contexts, and trying to address hard problems like sarcasm and implicit hate speech detection in social networks.
All the datasets that we use are publicly available. We report only aggregated results in the paper. For context mining, we have used data either available in the public domain or available through official APIs of the corresponding social media. Neither have we, nor do we intend to share any personally identifiable information with this paper. We also make our codebase publicly available here - https: //github.com/parag1604/CRUSH.
Our model CRUSH helps advance the state-of-the-art in hate speech detection by incorporating continual pre-training, capturing user writing biases and leveraging both user & post context (which is publicly available as well). This in turn should be able limit the amount of hateful threads in social networks/media by better detection of hate speech, thus promoting a more friendly and welcoming environment for people from all race, religion, ethnicity, gender etc.
Unfortunately, these models are not completely free from all potential negative impact. One such example being that our models have hate knowledge infused within them during the CP phase (refer Section 3.1) of our training pipeline. As shown in Table 3, these language models could be used potentially used for generating hateful words/sentences given an initial prompt.
However, the hateful words generated by the HateLM can be also identified by the platform if the platform uses a hate speech detection algorithm like ours or uses a set of hate lexicons to directly filter out such keywords generated (Gitari et al., 2015). Moreover, our final model - CRUSH - is not exactly suitable for toxic language generation as it is further pre-trained on the user style discrimination task and fine-tuned on the hate speech classification task making its classification capabilities (potential use by the social media platforms to detect hate speech) stronger than the hate speech generation capabilities (the potential of the model being abused with malicious intent). So, the platforms using our better-informed model will easily detect any hate speech generated by the model.
Moreover, large-scale technologies for better generation of hate speech using GPT-2 and other generative models (Wullach et al., 2021; Hartvigsen et al., 2022) (with the intention to train hate speech classifiers better) already exist and as a model trained only for classification, our model is likely to be far weaker than these models for gen
eration of harmful content. While these language models can be used both to benefit or disrupt our lifestyle just like any other technology (Douglas, 2014), we urge the researchers to exercise ultimate caution while using them, including ours. For the same reason, we do not make the trained model parameters publicly available (except for the code to promote reproducibility).
Also, one of the factors increasing bias in the classification model is the data it is trained upon. Hence, any potential bias in the datasets that are manually annotated by human beings (who are not free from bias) can result the model being biased for/against some specific target groups.
To incorporate better inductive bias into the model, we have trained the model to initially map text sequences posted by the same user to vectors that are close in the embedding space. This helps the model to identify the dialects of the various users and help the model perform better. However, this inductive bias may cluster the linguistic characteristics of some groups which in turn might potentially increase the chances of that particular language style being classified as hate speech if the annotated data contains only hateful instances from that particular dialect. This may make the model biased if those particular dialects are used predominantly by some protected category (Blacks, Jews, Women, Mexican etc.).
The most simple and effective solution here would be to annotate data for each dialect/vernacular group, potentially stratifying the dataset into clusters using the pre-trained language model. If there are balanced examples from each of these dialect/vernacular or at least each protected category (Blacks, Jews, Women, Mexican etc.) for both the hate and non-hate categories, such biases can be well mitigated.
Further, our model like any other hate speech model is not suitable for in-the-wild deployment without explicit human scrutiny. Language use is often very specific to each platform. So, the distribution of words and data-points may not match the training data distribution of the model. Thus, it is extremely important to first test the model on the platform, check for potential biases against the protected categories and individual linguistic groups/dialects and deploy it as a preliminary filter assisting the human experts detecting hate speech.
We thank Facebook (Meta Platforms, Inc.) for sponsoring the stay of PD at IIT Kharagpur through the Ethics in AI Research grant and Tata Consultancy Services (TCS) for funding SC with the TCS PhD fellowship.
Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017. Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion, WWW '17 Companion, page 759-760, Republic and Canton of Geneva, CHE. International World Wide Web Conferences Steering Committee.
Jason Baumgartne. 2018. gab.ai corpora on pushshift.io. https://files.pushshift.io/gab/. Added: 2018-11-30, Announcement: https://twitter.com/jasonbaumgartne/status/1068610394992926720.
Tommaso Caselli, Valerio Basile, Jelena Mitrovic, and Michael Granitzer. 2021. HateBERT: Retraining BERT for abusive language detection in English. In Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), pages 17-25, Online. Association for Computational Linguistics.
Marco Del Tredici, Diego Marcheggiani, Sabine Schulte im Walde, and Raquel Fernandez. 2019. You shall know a user by the company it keeps: Dynamic representations for social media users in NLP. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4707-4717, Hong Kong, China. Association for Computational Linguistics.
Fabio Del Vigna, Andrea Cimino, Felice Dell'Orletta, Marinella Petrocchi, and Maurizio Tesconi. 2017. Hate me, hate me not: Hate speech detection on facebook. In Proceedings of the First Italian Conference on Cybersecurity, ITASEC 2017.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Thomas Douglas. 2014. The dual-use problem, scientific isolationism and the division of moral labour. Monash bioethics review, 32:86-105.
Njagi Dennis Gitari, Zhang Zuping, Zuping Zhang, Hanyurwimfura Damien, and Jun Long. 2015. A lexicon-based approach for hate speech detection. In International Journal of Multimedia and Ubiquitous Engineering, volume 10, pages 215-230.
Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don't stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342-8360, Online. Association for Computational Linguistics.
Rishav Hada, Sohi Sudhir, Pushkar Mishra, Helen Yannakoudakis, Saif M. Mohammad, and Ekaterina Shutova. 2021. Reddit: Norms of offensiveness for English Reddit comments. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2700-2717, Online. Association for Computational Linguistics.
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. arXiv preprint arXiv:2203.09509.
James M. Hughes, Nicholas J. Foti, David C. Krakauer, and Daniel N. Rockmore. 2012. Quantitative patterns of stylistic influence in the evolution of literature. Proceedings of the National Academy of Sciences, 109(20):7682-7686.
Andrew Jakubowicz. 2017. Alt_right white lite: Trolling, hate speech and cyber racism on social media. Cosmopolitan Civil Societies: An Interdisciplinary Journal, 9:41.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. In Advances in Neural Information Processing Systems, volume 33, pages 18661-18673. Curran Associates, Inc.
Rohan Kshirsagar, Tyrus Cukuvac, Kathy McKeown, and Susan McGregor. 2018. Predictive embeddings for hate speech detection on twitter. In *Abusive Language Online Workshop*, EMNLP 2018, pages 26-32.
Shervin Malmasi and Marcos Zampieri. 2017. Detecting hate speech in social media. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 467-472, Varna, Bulgaria. INCOMA Ltd.
Binny Mathew, Anurag Illendula, Punyajoy Saha, Soumya Sarkar, Pawan Goyal, and Animesh Mukherjee. 2020. Hate begets hate: A temporal study of hate speech. Proc. ACM Hum.-Comput. Interact., 4(CSCW2).