SlowGuess/ACL_2020_OCR huggingface dataset:
Joseph Hoover, Mohammad Atari, Aida Mostafazadeh Davani, Brendan Kennedy, Gwenyth Portillo-Wightman, Leigh Yeh, Drew Kogon, and Morteza Dehghani. 2019. Bound in hatred: The role of group-based morality in acts of hate. *PsyArxiv* Preprint 10.31234/osf.io/359me.
Xisen Jin, Zhongyu Wei, Junyi Du, Xiangyang Xue, and Xiang Ren. 2020. Towards hierarchical importance attribution: Explaining compositional semantics for neural sequence models. In International Conference on Learning Representations.
Brendan Kennedy, Mohammad Atari, Aida M Davani, Leigh Yeh, Ali Omrani, Yehsong Kim, Kris Coombs Jr., Shreya Havaldar, Gwenyth Portillo-Wightman, Elaine Gonzalez, Joe Hoover, Aida Azatian, Gabriel Cardenas, Alyzeh Hussain, Austin Lara, Adam Omary, Christina Park, Xin Wang, Clarisa Wijaya, Yong Zhang, Beth Meyerowitz, and Morteza Dehghani. 2020. The gab hate corpus: A collection of 27k posts annotated for hate speech.
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations.
Lucas Lima, Julio CS Reis, Philippe Melo, Fabricio Murai, Leandro Araujo, Pantelis Vikatos, and Fabricio Benevenuto. 2018. Inside the right-leaning echo chambers: Characterizing gab, an unmoderated social system. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 515-522. IEEE.
Sean MacAvaney, Hao-Ren Yao, Eugene Yang, Katina Russell, Nazli Goharian, and Ophir Frieder. 2019. Hate speech detection: Challenges and solutions. *PloS one*, 14(8).
Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019. Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages. In Proceedings of the 11th Forum for Information Retrieval Evaluation, pages 14-17.
Priscilla Marie Meddaugh and Jack Kay. 2009. Hate speech or "reasonable racism?" the other in stormfront. Journal of Mass Media Ethics, 24(4):251-268.
Shruthi Mohan, Apala Guha, Michael Harris, Fred Popowich, Ashley Schuster, and Chris Priebe. 2017. The impact of toxic language on the health of reddit communities. In *Canadian Conference on Artificial Intelligence*, pages 51-56. Springer.
Mainack Mondal, Leandro Araujo Silva, and Fabricio Benevenuto. 2017. A measurement study of hate speech in social media. In Proceedings of the 28th ACM Conference on Hypertext and Social Media, pages 85-94. ACM.
Aida Mostafazadeh Davani, Mohammad Atari, Brendan Kennedy, Shreya Havaldar, and Morteza Dehghani. 2020. Hatred is in the eye of the annotator: Hate speech classifiers learn human-like social stereotypes (in press). In 31st Annual Conference of the Cognitive Science Society (CogSci).
W. James Murdoch, Peter J. Liu, and Bin Yu. 2018. Beyond word importance: Contextual decomposition to extract interactions from LSTMs. In International Conference on Learning Representations.
Timothy Niven and Hung-Yu Kao. 2019. Probing neural network comprehension of natural language arguments. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4658-4664.
Alexandra Olteanu, Carlos Castillo, Jeremy Boy, and Kush R Varshney. 2018. The effect of extremist violence on hateful speech online. In Twelfth International AAAI Conference on Web and Social Media.
Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing gender bias in abusive language detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2799-2804.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. Open AI Blog.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144. ACM.
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1668-1678.
Leandro Silva, Mainack Mondal, Denzil Correa, Fabricio Benevenuto, and Ingmar Weber. 2016. Analyzing the targets of hate in online social media. In Tenth International AAAI Conference on Web and Social Media.
Chandan Singh, W. James Murdoch, and Bin Yu. 2019. Hierarchical interpretations for neural network predictions. In International Conference on Learning Representations.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 3319-3328. JMLR.org.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008.
Jeremy Waldron. 2012. The harm in hate speech. Harvard University Press.
William Warner and Julia Hirschberg. 2012. Detecting hate speech on the world wide web. In Proceedings of the second workshop on language in social media, pages 19-26. Association for Computational Linguistics.
Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding abuse: A typology of abusive language detection subtasks. In Proceedings of the First Workshop on Abusive Language Online, pages 78-84.
Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the NAACL student research workshop, pages 88-93.
Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of abusive language: the problem of biased datasets. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 602-608.
Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web, pages 1391-1399. International World Wide Web Conferences Steering Committee.
Savvas Zannettou, Barry Bradlyn, Emiliano De Cristofaro, Haewoon Kwak, Michael Sirivianos, Gianluca Stringini, and Jeremy Blackburn. 2018. What is gab: A bastion of free speech or an alt-right echo chamber. In *Companion Proceedings of the The Web Conference* 2018, pages 1007-1014. International World Wide Web Conferences Steering Committee.
A.1 Full List of Curated Group Identifiers
muslim jew jews white islam blacks muslims women whites gay black democrat islamic allah jewish lesbian transgender race brown woman mexican religion homosexual homosexuality africans
Table 3: 25 group identifiers selected from top weighted words in the TF-IDF BOW linear classifier on the GHC.
jew jews mexican blacks jewish brown black muslim homosexual islam
A.2 Visualizations of Effect of Regularization
Figure 4: Hierarchical explanations on a test instance from the NYT dataset where false positive predictions are corrected.
Training Details. We fine-tune over the BERT-base model using the public code†, where the batch size is set to 32 and the learning rate of the Adam (Kingma and Ba, 2015) optimizer is set to  $2 \times 10^{-5}$ . The validation is performed every 200 iterations and the learning rate is halved when the validation F1 decreases. The training stops when the learning rate is halved for 5 times. To handle the data imbalance issue, we reweight the training loss so that positive examples are weighted 10
Figure 5: Hierarchical explanations on a test instance from the Gab dataset where both models make correct positive predictions. However, the explanations reveal that only the regularized model is making correct predictions for correct reasons.
times as negative examples on the Gab dataset and 8 times on the Stormfront dataset.
Explanation Algorithm Details. For the SOC algorithm, we set the number of samples and the size of the context window as 20 and 20 respectively for explanation analysis, and set two parameters as 5 and 5 respectively for explanation regularization.
A.4 Cross-Domain Performance
In addition to evaluating each model within-domain (i.e., training on  $\mathrm{GHC}_{train}$  and evaluating on  $\mathrm{GHC}_{test}$ ) we evaluated each model across domains. The results of these experiments, conducted in the same way as before, are presented in Table 5.
Table 4: 10 group identifiers selected for the Stormfront dataset.
Method / Dataset	Gab → Stf. F1	Stf. → Gab F1
BoW	32.39	46.71
BERT	42.84 ± 1.2	53.80 ± 5.5
BoW + WR	27.45	44.81
BERT + WR	39.10 ± 1.3	55.31 ± 4.0
BERT + OC (α=0.1)	40.60 ± 1.6	56.90 ± 1.8
BERT + SOC (α=0.1)	41.88 ± 1.0	55.75 ± 2.1
BERT + SOC (α=1.0)	39.20 ± 2.7	56.82 ± 3.9
Table 5: Cross domain F1 on Gab, Stormfront (Stf.) datasets. We report mean and standard deviation of the performance within 10 runs for BERT, BERT + WR (word removal), BERT + OC, and BERT + SOC.
$\dagger$  https://github.com/huggingface/ transformers