SlowGuess/NAACL_2022_OCR huggingface dataset:
Blending Gold and Silver We adopt the method by Shnarch et al. (2018) to determine whether Silver annotations are beneficial. There are two phases in the training process:  $m$  blending epochs using all Gold and a fraction of Silver, and then  $n$  epochs using all Gold. In each blending epoch, Silver instances are fed in a random order to the network.
The fraction of Silver is determined by a blending factor  $\alpha \in [0..1]$ . The first blending epoch is trained with all Gold and all Silver, and the amount of Silver to blend is reduced by  $\alpha$  in each epoch.
Pretraining with Related Tasks We also experiment with several corpora to investigate whether pretraining with related tasks is beneficial. Specifically, we pretrain our models with existing corpora annotating: (1) hateful comments: hateful or not hateful (Qian et al., 2019), and hate speech, offensive, or neither (Davidson et al., 2017); (2) sentiment: negative, neutral, or positive (Rosenthal et al., 2017); (3) sarcasm: sarcasm or not sarcasm (Ghosh et al., 2020); and (4) stance: agree, neutral, or attack (Pougue-Biyong et al., 2021).
We present results with the test split in Table 6. The majority baseline always predicts Neutral. The remaining rows present the results with the different training settings: training with the Target or both the Parent and Target; training with only Gold or blending Silver annotations; and pretraining with related tasks. We provide here results pretraining with the most beneficial task, stance detection, and present additional results in the Appendices. Blending Gold and Silver annotations requires tuning  $\alpha$ . We did so empirically using the training and validations splits, like other hyperparameters. We found the optimal value to be 0.3 when blending Silver (+ Silver rows) and 1.0 when blending Silver and pretraining with a related task (+Silver + Related task rows).
As shown in Table 6, blending Gold and Sil
Error Type	%	Example	Parent_Target	Target
Lack of information	48	Parent: Women can hover..?Target: No, they can't, but for some reason they keep trying and it gets sh*t everywhere.	Hate	Neutral
Negation	27	Parent: It's a joke you pu**y.Target: I don't see sexism as a joke, especially on a site dedicated to calling out sexism.	Counter-hate	Neutral
Sarcasm or irony	19	Parent: You must have been a real baller banging out those eighth graders as a High School senior.Target: Glad to see you have no rational argument left except childish jokes. We're done here pal.	Counter-hate	Hate
Hate without swear words	8	Parent: Name a dildo 'misogyny' so you can *literally* internalize it.Target: lol. Misogyny can already turn me on so that's a good idea.	Hate	Neutral
Table 7: Most common error types made by the Target only network (Target) that are fixed by the context-aware neural network (Parent_Target).
ver annotations obtains better results (F1 weighted average) than using only Gold (Target: 0.61 vs. 0.58; Parent_Target: 0.63 vs. 0.61). We also find that models pretrained for stance detection obtain better results than pretrained with other tasks (see detailed results in the Appendices). Pretraining with stance detection data benefits models trained without context (Target: 0.61 vs. 0.58) and models with context (Parent_Target: 0.63 vs. 0.61). These results indicate that stance information between Parent and Target is useful to determine whether the Target is Hate, Counter-Hate or Neutral.
We make two observations about the results obtained using neither strategy. First, using the Target alone obtains much better results than the majority baseline (0.58 vs. 0.34). In other words, modeling the Target allows the network to identify some instances of Hate and Counter-hate despite the ground truth requires the Parent. Second, incorporating the Parent comment is beneficial: the F1 score for all classes are higher (Hate: 0.59 vs. 0.56, Counter-hate 0.44 vs. 0.38, Neutral 0.70 vs. 0.69), and so is the weighted average (0.61 vs. 0.58). The findings are consistent (weighted F1) using either strategy (+Silver: 0.63 vs 0.61, +Related task: 0.63 vs 0.61) or both (0.64 vs. 0.61). The F1 scores of the three classes with Parent_Target models are equal to or better than those by Target only models.
Finally, the network (a) blending Gold and Silver annotations and (b) pretraining with stance detection achieves the best performance (Parent_Target + Silver + Related task: 0.64). This result is statistically significant  $(p < 0.01)$  compared to Target only model without blending Silver or pretraining with related tasks.
When is adding the context beneficial? When does our best model make mistakes? To investigate these questions, we manually analyze the following:
- The errors made by the Target only network that are fixed by the context-aware network (Trained with Parent_Target, Table 7).
- The errors made by the context-aware network pretrained on related task (stance) and blending Silver annotations (Parent_Target+Silver+Related task, Table 8).
When does the context complement Target? We analyze the errors made by the network using only the Target that are fixed by the context-aware network (Trained with Parent_Target). Table 7 exemplifies the most common error types.
The most frequent type of error fixed by the context-aware model is when there is Lack of information in the Target (48%). In this case, the Parent comment is crucial to determine the label of the Target. In the example, knowing what the author of Target refers to (i.e., a rhetorical question, Women can hover?) is crucial to determine that the Target is humiliating women as a group.
The second most frequent error type is Negation (27%). In the example in Table 7, taking into account the Parent allows the context-aware network to identify that the author of the Target is scolding the author of Parent and thus countering hate.
Nobata et al. (2016) and Qian et al. (2019) have pointed out that sarcasm and irony make detecting abusive and hateful content difficult. We find evidence supporting this claim. We also discover that by incorporating the Parent comment, a substantial amount of these errors are fixed. Indeed,  $19\%$  of
Error Type	%	Example	Ground Truth	Predicted
Negation	28	Parent: Those damn f**king white males, ruining it for every-one else. I'm going to a corner to process my guilt.Target: Don't forget male isn't a gender, it's a disease.	Hate	Counter-hate
Rhetorical question	27	Parent: Men are the ones that made inequality.Target: Do you get paid to be a dumba** in the internet?	Hate	Counter-hate
Hate without swear words	8	Parent: Circumcision is good for men.Target: Cut off the clitoris of women and cut of their breasts because of breast cancer then.	Hate	Neutral
Non-hate with swear words	8	Parent: <I wonder if feminists ever consider that? No. They are b**ches incapable of empathy.Target: This is the sh*t that gets screen capped and spread around to give this sub a bad name.	Counter-hate	Hate
Intricate text	7	Parent: Ah it's this again, f**king her and her cronies.Target: I have lost all respect for her.	Neutral	Hate
Table 8: Most common errors made by the best context-aware network (predictions by Parent_Target+Silver+Related task) compared to the ground truth.
errors fixed by the context-aware network include sarcasm or irony in the Target comment.
Finally, the context-aware network taking into account the Parent fixes many errors (8%) in which the Target comment is Hate despite it does not contain swear words. In the example, the Target is introducing additional hateful content, which can be identified by the context-aware model when the Parent information is used.
When does the best model make errors? In order to find out the most common error types made by the best model (context-aware, Parent_Target+Silver+Related task), we manually analyze 200 random samples in which the output of the network differs from the ground truth. Table 8 shows the results of the analysis.
Despite  $27\%$  of errors fixed by the context-aware network (i.e., taking into account the Parent) include negation in the Target, negation is the most common type of errors made by our best network  $(28\%)$ . The example in Table 8 is especially challenging as it includes a double negation.
We observe that Rhetorical questions are almost as common (27%). This finding is consistent with the findings by Schmidt and Wiegand (2017). In the example, the best model fails to realize that the Target is hateful, as it disdains the author of Parent.
Swear words are present in a substantial number of errors. Wrongly predicting a Target without swear words as Counter-hate or Neutral accounts for  $8\%$  of errors, and wrongly predicting a Target with swear words as Hate accounts for another  $8\%$  of errors. As pointed out by Davidson et al. (2017),
hate speech may not contain hate or swear words. And vice versa, comments containing swear words may not be hateful (Zhang and Luo, 2019).
Finally, we observe Intricate text in  $7\%$  errors. Our best model identifies the Target ("I have lost all respect for her.") as Hate probably because by identifying the agreeing stance on the Parent. Indeed, the author of Target expresses his/her attitude without vilifying others. Hence, the ground truth label is Neutral.
Conversational context does matter in Hate and Counter-hate detection. We have demonstrated so by (a) analyzing whether humans perceive user-generated content as Hate or Counter-hate depending on whether we show them the Parent comment and (b) investigating whether neural networks benefit from incorporating the Parent. We find that  $38.3\%$  of human judgments change when we show the Parent to annotators. Experimental results demonstrate that networks incorporating the Parent yield better results. Additionally, we show that noisy instances (Silver data) and pretraining with relevant datasets improves model performance. We have created and released a corpus of 6,846 (Parent, Target) pairs of Reddit comments with the Target annotated as Hate, Neutral or Counter-hate.
Our work have several limitations. First, we only consider context as the parent comment. While considering additional context might be sometimes beneficial, doing so would require careful design to not bias annotations (Dutta et al., 2020). Our research agenda includes exploring reliably strate
gies to consider more context and identify which parts are most important. Second, people may have different opinions about what constitutes hate and counter speech due to different tolerances in online aggression. We obtained the ground truth according to annotators' reliability (MACE scores), which may lead to controversial samples falling in the Silver set and thus being considered only for training (not for testing). Finally, the keywords sampling used to create our corpus may introduce biases. Despite we partially mitigate the issue by considering hateful comments in both the Parent and Target, community-based sampling (Vidgen et al., 2021) could be applied in our future work.
We use the PushShift API to collect data from Reddit. Our collection process is consistent with Reddit's Terms of Service. The data are accessed through the data dumps on Google's BigQuery using Python.
Reddit can be considered a public space for discussion which differs from a private messaging service (Vidgen et al., 2021). Users consent to have their data made available to third parties including academics when they sign up to Reddit. Existing ethical guidelines state that in this situation explicit consent is not required from each user (Procter et al., 2019). We obfuscate user names as User_A or User_B to reduce the possibility of identifying users. In compliance with Reddit's policy, we would like to make sure that our dataset will be reused for non-commercial research only.
The Reddit comments in this dataset were annotated by annotators using Amazon Mechanical Turk. We have followed all requirements introduced by the platform for tasks containing adult content. A warning was added in the task title. Annotators need to pass the Adult Content Qualification Test before working on our tasks. Annotators were compensated on average with $8 per hour. We paid them regardless of whether we accepted their work. Annotators' IDs are not included in the dataset.
This work was supported by Research Seed Grant of the UNT College of Information. We would like to thank the anonymous reviewers for their insightful comments and suggestions.
Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Comput. Linguist., 34(4):555-596.
Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the Fourteenth International AAAI Conference on Web and Social Media, ICWSM 2020, Held Virtually, Original Venue: Atlanta, Georgia, USA, June 8-11, 2020, pages 830-839. AAAI Press.
Manuela Caiani, Benedetta Carlotti, and Enrico Padoan. 2021. Online hate speech and the radical right in times of pandemic: The italian and english cases. Javnost - The Public, 28(2):202-218.
Yi-Ling Chung, Elizaveta Kuzmenko, Serra Sinem Tekiroglu, and Marco Guerini. 2019. CONAN - COunter NArratives through nichesourcing: a multilingual dataset of responses to fight online hate speech. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2819-2829, Florence, Italy. Association for Computational Linguistics.
Scott A Crossley, Kristopher Kyle, and Danielle S McNamara. 2017. Sentiment analysis and social cognition engine (seance): An automatic tool for sentiment, social cognition, and social-order analysis. Behavior research methods, 49(3):803-821.
Thomas Davidson, Dana Warmsley, Michael W. Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the Eleventh International Conference on Web and Social Media, ICWSM 2017, Montréal, Québec, Canada, May 15-18, 2017, pages 512-515. AAAI Press.
Subhabrata Dutta, Dipankar Das, and Tanmoy Chakraborty. 2020. Changing views: Persuasion modeling and argument extraction from online discussions. Information Processing & Management, 57(2):102085.
Mai ElSherief, Vivek Kulkarni, Dana Nguyen, William Yang Wang, and Elizabeth M. Belding. 2018. Hate lingo: A target-based linguistic analysis of hate speech in social media. In Proceedings of the Twelfth International Conference on Web and Social Media, ICWSM 2018, Stanford, California, USA, June 25-28, 2018, pages 42-51. AAAI Press.
$^{5}$ https://pushshift.io/api-parameters/  
 $^{6}$ https://pushshift.io/  
using-bigquery-with-edgit-data/  
 $^{7}$ https://www.reddit.com/wiki/  
api-terms/
Margherita Fanton, Helena Bonaldi, Serra Sinem Tekiroglu, and Marco Guerini. 2021. Human-in-the-loop for data collection: a multi-target counter narrative dataset to fight online hate speech. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3226-3240, Online. Association for Computational Linguistics.
Tracie Farrell, Miriam Fernandez, Jakub Novotny, and Harith Alani. 2019. Exploring misogyny across the manosphere in reddit. In Proceedings of the 10th ACM Conference on Web Science, WebSci '19, page 87-96, New York, NY, USA. Association for Computing Machinery.
Paula Fortuna and Sérgio Nunes. 2018. A survey on automatic detection of hate speech in text. ACM Comput. Surv., 51(4).
Lei Gao and Ruihong Huang. 2017. Detecting online hate speech using context aware models. In Proceedings of the International Conference on Recent Advances in Natural Language Processing, RANLP 2017, pages 260-266, Varna, Bulgaria. INCOMA Ltd.
Joshua Garland, Keyan Ghazi-Zahedi, Jean-Gabriel Young, Laurent Hébert-Dufresne, and Mirta Galesic. 2020. Countering hate on social media: Large scale classification of hate and counter speech. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pages 102-112, Online. Association for Computational Linguistics.
Debanjan Ghosh, Avijit Vajpayee, and Smaranda Muresan. 2020. A report on the 2020 sarcasm detection shared task. In Proceedings of the Second Workshop on Figurative Language Processing, pages 1-11, Online. Association for Computational Linguistics.
Bing He, Caleb Ziems, Sandeep Soni, Naren Ramakrishnan, Diyi Yang, and Srijan Kumar. 2021. Racism is a virus: Anti-asian hate and counterspeech in social media during the Covid-19 crisis. In Proceedings of the 2021 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, page 90–94, New York, NY, USA. Association for Computing Machinery.
Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy. 2013. Learning whom to trust with MACE. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1120-1130, Atlanta, Georgia. Association for Computational Linguistics.
Nicola F Johnson, R Leahy, N Johnson Restrepo, Nicolas Velasquez, Ming Zheng, P Manrique, P Devkota, and Stefan Wuchty. 2019. Hidden resilience and adaptive dynamics of the global online hate ecology. Nature, 573(7773):261-265.
Klaus Krippendorff. 2011. Computing krippendorff's alpha-reliability. https://repository.upenn.edu/asc_papers/43/. Accessed: 2021-02-08.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
Binny Mathew, Anurag Illendula, Punyajoy Saha, Soumya Sarkar, Pawan Goyal, and Animesh Mukherjee. 2020. Hate begets hate: A temporal study of hate speech. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW2):1-24.
Binny Mathew, Punyajoy Saha, Hardik Tharad, Subham Rajgaria, Prajwal Singhania, Suman Kalyan Maity, Pawan Goyal, and Animesh Mukherjee. 2019. Thou shalt not hate: Countering online hate speech. In Proceedings of the Thirteenth International Conference on Web and Social Media, ICWSM 2019, Munich, Germany, June 11-14, 2019, pages 369-380. AAAI Press.
Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. Hatexplain: A benchmark dataset for explainable hate speech detection. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 14867-14875. AAAI Press.
Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153-157.
Stefano Menini, Alessio Palmero Aprosio, and Sara Tonelli. 2021. Abuse is contextual, what about nlp? the role of context in abusive language annotation and detection. CoRR, abs/2103.14916.
Chikashi Nobata, Joel R. Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive language detection in online user content. In Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11 - 15, 2016, pages 145-153. ACM.
Alexandra Olteanu, Carlos Castillo, Jeremy Boy, and Kush R. Varshney. 2018. The effect of extremist violence on hateful speech online. In Proceedings of the Twelfth International Conference on Web and Social Media, ICWSM 2018, Stanford, California, USA, June 25-28, 2018, pages 221-230. AAAI Press.
John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain, and Ion Androutsopoulos. 2020. Toxicity detection: Does context really matter? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4296-4305, Online. Association for Computational Linguistics.
John Pougue-Biyong, Valentina Semenova, Alexandre Matton, Rachel Han, Aerin Kim, Renaud Lambiotte, and Doyne Farmer. 2021. DEBAGREEMENT: A comment-reply dataset for (dis)agreement detection in online debates. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
Rob Procter, Helena Webb, Pete Burnap, William Housley, Adam Edwards, Matthew L. Williams, and Marina Jirotka. 2019. A study of cyber hate on twitter with implications for social media governance strategies. In Proceedings of the 2019 Truth and Trust Online Conference (TTO 2019), London, UK, October 4-5, 2019.
Yada Pruksachatkun, Phil Yeres, Haokun Liu, Jason Phang, Phu Mon Htut, Alex Wang, Ian Tenney, and Samuel R. Bowman. 2020. jiant: A software toolkit for research on general-purpose text understanding models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 109-117, Online. Association for Computational Linguistics.
Jing Qian, Anna Bethke, Yinyin Liu, Elizabeth Belding, and William Yang Wang. 2019. A benchmark dataset for learning to intervene in online hate speech. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4755-4764, Hong Kong, China. Association for Computational Linguistics.
Yafeng Ren, Yue Zhang, Meishan Zhang, and Donghong Ji. 2016. Context-sensitive twitter sentiment classification using neural network. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI'16, page 215-221. AAAI Press.
Mohammadreza Rezvan, Saeedeh Shekarpour, Lakshika Balasuriya, Krishnaprasad Thirunarayan, Valerie L. Shalin, and Amit Sheth. 2018. A quality type-aware annotated corpus and lexicon for harassment research. In Proceedings of the 10th ACM Conference on Web Science, WebSci '18, page 33-36, New York, NY, USA. Association for Computing Machinery.
Robert D Richards and Clay Calvert. 2000. Counterspeech 2000: A new look at the old remedy for bad speech. *BYUL Rev.*, page 553.
Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. SemEval-2017 task 4: Sentiment analysis in Twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 502-518, Vancouver, Canada. Association for Computational Linguistics.
Marta Sabou, Kalina Bontcheva, Leon Derczynski, and Arno Scharl. 2014. Corpus annotation through crowdsourcing: Towards best practice guidelines. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14),