SlowGuess/NAACL_2022_OCR huggingface dataset:
Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond accuracy: Behavioral testing of NLP models with CheckList. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902-4912, Online. Association for Computational Linguistics.
Alexis Ross, Ana Marasovic, and Matthew Peters. 2021. Explaining NLP models via minimal contrastive editing (MiCE). In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3840-3852, Online. Association for Computational Linguistics.
Alexis Ross, Ana Marasovic, and Matthew E Peters. 2020. Explaining NLP models via minimal contrastive editing (MiCE). arXiv preprint arXiv:2012.13985.
Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. 2021. HateCheck: Functional tests for hate speech detection models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 41-58, Online. Association for Computational Linguistics.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the International Conference on Machine Learning, pages 3319-3328. PMLR.
Jennifer Wortman Vaughan and Hanna Wallach. 2021. A human-centered agenda for intelligible machine learning. In Marcello Pelillo and Teresa Scantamburlo, editors, *Machines We Trust: Perspectives on Dependable AI*. The MIT Press.
Bertie Vidgen, Dong Nguyen, Helen Margetts, Patricia Rossini, and Rebekah Tromble. 2021. Introducing CAD: the contextual abuse dataset. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2289-2303.
David Watson, Limor Gultchin, Ankur Taly, and Luciano Floridi. 2021. Local explanations via necessity and sufficiency: Unifying theory and practice. In Proceedings of Machine Learning Research, volume 161, page 1382-1392.
Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld. 2021. Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6707-6723, Online. Association for Computational Linguistics.
Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web, pages 1391-1399.
Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using "annotator rationales" to improve machine learning for text categorization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, pages 260-267.
Yuan Zhang, Jason Baldridge, and Luheng He. 2019. Paws: Paraphrase adversaries from word scrambling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1298-1308.
A Data, Training and Generation Details for the Infilling Language Model
To fine-tune the ILM model, we use the following four datasets: Wikipedia Toxicity $^{6}$  (Wulczyn et al., 2017), Founta2018 $^{7}$  (Founta et al., 2018), Civil Comments $^{8}$  (Borkan et al., 2019), and Vidgen2021 $^{9}$  (Vidgen et al., 2021). The datasets contain English-language utterances, and cover different domains (Twitter post, Reddit posts, Wikipedia comments, and comments from news websites). The datasets have been created to study abusive language, and are commonly used to train and evaluate classification models that detect various sub-categories of online abuse, such as hate speech, toxicity, personal attacks, etc. All datasets except Founta2018 are in the public domain and licensed for research purposes. Founta2018 dataset is being used with the permission of the first author.
The details on each dataset are provided in Table A.1. For the Wikipedia Toxicity dataset, a large portion of the data is from conversations about Wikipedia-specific topics. To not skew our generation model, we filter these instances following the unsupervised method presented by Nejadgholi and Kritchenko (2020)<sup>10</sup>. Because the Civil Comments dataset is significantly larger than the rest, we randomly sample 30K neutral instances and discard the rest. After filtering, the compound dataset of neutral instances consists of 130,430 instances in total. As preprocessing, we replace URLs, mentions and emojis with special tokens.
To train the ILM, we fine-tune GPT-2 (1.5B parameters) for 4 epochs with the default hyperparameters provided by Donahue et al. (2020). The training takes approximately 2.5 hours on a Tesla V100-SXM2 GPU. Although the original ILM is trained by infilling words,  $n$ -grams, sentences and paragraphs, we modify the objective to only infill words and  $n$ -grams.
We generate perturbations once for the 120 Hate-Check cases, and evaluate all models on the same set of perturbations. The number of perturbations are chosen so that to have approximately 100 per
Dataset	Source	Class	Size
Wikipedia Toxicity (Wulczyn et al., 2017)	Wikipedia comments	Normal	36,121
Founta2018 (Founta et al., 2018)	Twitter posts	Normal	53,236
Civil Comments (Borkan et al., 2019)	Comments on news sites	Normal	30,000
Vidgen2021 (Vidgen et al., 2021)	Reddit posts	Non-Abusive	11,073
Total	130,430
Table A.1: Description of the training data used to fine-tune the ILM model.
turbed instances for each token for the necessity calculation, and 100 instances for the sufficiency calculation. This results in a total of 66,120 perturbed instances, and takes approximately 6 hours to generate on a  $2.3\mathrm{GHz}$  Quad-Core Intel Core i7 CPU.
B Data and Training Details for Hate Speech Classifiers
We fine-tune six BERT (Devlin et al., 2019) classifiers on three different datasets and with two different labelling schemes (hate speech vs. abusive language) for each. The datasets include: Davidson2017 $^{11}$  (Davidson et al., 2017), Founta2018 (Founta et al., 2018), and Vidgen2021 (Vidgen et al., 2021). The datasets contain English-language posts from two online platforms, Twitter and Reddit. The details on each dataset are provided in Table B.1.
We train two models on the dataset of Founta et al. (2018). For Founta2018-hate, we binarize the labels to map hate annotations as positive, and the rest as the negative class. For Founta2018-abuse, we label both hate and abuse annotations as positive, and the rest as negative. To illustrate that our method can provide explanations for models trained on data that is not explicitly modelled by our perturbation generator, we also train models on two versions of the dataset of Davidson et al. (2017): Davidson2017-abuse and Davidson2017-hate, which are binarized in the same manner.
The dataset of Vidgen et al. (2021) provides a hierarchical labelling scheme, the top distinction being abusive vs. non-abusive. We binarize Vidgen2021-abuse based on these labels. For Vidgen2021-hate, we take the positive class to be those instances that are labelled identity-directed abuse, and label the rest as the negative class.
11https://github.com/t-davidson/hate-speech-and-offensive-language/tree/master/data
$^{6}$ https://figshare.com/articles/  
dataset/Wikipedia_Talk_Labels_Toxicity/4563973  
 $^{7}$ https://github.com/ENCASEH2020/hatespeech-twitter  
 $^{8}$ https://bit.ly/3Kfaveb  
 $^{9}$ https://zenodo.org/record/4881008#.YeBBQ2jMKUk  
 $^{10}$ https://github.com/IsarNejad/cross_dataset_toxicity
Classifier	Dataset	Positive Class	Negative Class	Size
Train	Dev	Test
Founta2018-hate	Founta2018(Founta et al., 2018)	Hateful	NormalAbusive	62,445	7,806	7,806
Founta2018-abuse	Founta2018(Founta et al., 2018)	HatefulAbusive	Normal	62,445	7,806	7,806
Davidson2017-hate	Davidson2017(Davidson et al., 2017)	Hate	NeitherOffensive	19,826	2,478	2,479
Davidson2017-abuse	Davidson2017(Davidson et al., 2017)	HateOffensive	Neither	19,826	2,478	2,479
Vidgen2021-hate	Vidgen2021(Vidgen et al., 2021)	Identitydirected abuse	Non-abusivePerson-directed abuseAffiliation-directed abuse	13,585	4,527	5,308
Vidgen2021-abuse	Vidgen2021(Vidgen et al., 2021)	Abusive	Non-abusive	13,585	4,527	5,308
Table B.1: Description of the datasets used to fine-tune hate speech and abuse detection classifiers.
Micro F1	Macro F1	Training time (mins)
Founta2018-hate	0.94	0.67	28
Founta2018-abuse	0.94	0.93	28
Davidson2017-hate	0.94	0.70	7
Davidson2017-abuse	0.96	0.93	7
Vidgen2021-hate	0.91	0.71	22
Vidgen2021-abuse	0.85	0.72	22
We employ the same pre-processing steps as in the experiments by Röttger et al. (2021), and replace URLs, mentions and emojis with special tokens. We fine-tune a BERT model from the Hugging Face library $^{12}$  on each of these datasets on a single Tesla V100-SXM2 GPU. Each model has 110M trainable parameters. We follow the implementation of Röttger et al. (2021) and use their hyper-parameters of 3 epochs, batch size of 16, learning rate of 5e-5 and weight decay of 0.01. We also employ weighted cross-entropy loss that corrects for the class imbalance in data. For the training/development/test splits, we use the standard split for Vidgen2021 provided by the creators of the dataset, and use a stratified 80/10/10 split for the other datasets, making sure that the splits are the same for the hate and abuse versions of each, and correspond to the training set for ILM when applicable. The classification performance of these models on the held-out test sets is shown in Table B.2, together with the training times for each. We can observe that the reported scores are within a few percentage points of the previously published
Table B.2: Micro- and macro-averaged F1-scores and training times for each BERT model trained and evaluated on the given datasets.
women	Muslims
necc	suff	necc	suff
Founta2018-hate	0.53	0.30	0.72	0.81
Founta2018-abuse	0.19	0.34	0.36	0.82
Davidson2017-hate	0.44	0.21	0.88	0.74
Davidson2017-abuse	0.55	0.44	0.52	0.41
Vidgen2021-hate	0.87	0.71	0.93	0.88
Vidgen2021-abuse	0.62	0.64	0.64	0.88
Table B.3: Average necessity and sufficiency scores calculated by masking rather than perturbing selected tokens, for the identity terms in explicitly hateful cases of HateCheck (F1, F2, and F3) targeting women or Muslims for the three classifiers trained on hate, and three classifiers trained on abuse labels.
results (Röttger et al., 2021). All reported results are from a single run.
C Calculating Necessity and Sufficiency with Masking
In Section 1 we have argued that using the mask token from the pre-training objective in feature attribution methods has several drawbacks. Nevertheless, in Table B.3 we report the results of a modified version of our experiment presented in Section 5 where we keep the number and the location of the perturbations the same as the original experiments, but instead of perturbing the chosen tokens using an LM, we replace them with the mask token. The results show that although the values are different than their counterparts in the main experiment, the overall trends remain the same, and support the hypotheses presented in Section 5. Evaluating the classifier with the masked input is faster than explicitly generating perturbations, but the method ceases to be model agnostic and looses transparency. The results still suggest that
12https://huggingface.co/ bert-base-uncased
evaluating necessity and sufficiency with masked rather than perturbed inputs might be preferable in contexts where latency is more important than transparency, or as a pre-processing step to choose which inputs and tokens to focus on for in-depth analysis with explicit perturbations. We leave further explorations of this avenue for future work.