SlowGuess/NAACL_2022_OCR huggingface dataset:
Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 67-73.
Amir Feder, Katherine A Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, Justin Grimmer, Roi Reichart, Margaret E Roberts, et al. 2021. Causal inference in natural language processing: Estimation, prediction, interpretation and beyond.
Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, and Ben Zhou. 2020. Evaluating Models' Local Decision Boundaries via Contrast Sets. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1307-1323, Online. Association for Computational Linguistics.
Tarleton Gillespie. 2018. Custodians of the Internet. Yale University Press.
Tarleton Gillespie. 2020. Content moderation, ai, and the question of scale. *Big Data & Society*, 7(2):2053951720943234.
Robert Gorwa. 2019. The platform governance triangle: Conceptualising the informal regulation of online content. *Internet Policy Review*, 8(2):1-22.
Robert Gorwa, Reuben Binns, and Christian Katzenbach. 2020. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. *Big Data & Society*, 7(1):2053951719897945.
Kishonna L Gray and Krysten Stein. 2021. We 'said her name'and got zucked': Black Women Callingout the Carceral Logics of Digital Platforms. Gender & Society, 35(4):538-545.
Oliver L Haimson, Daniel Delmonaco, Peipei Nie, and Andrea Wegner. 2021. Disproportionate removals and differing content moderation experiences for conservative, transgender, and black social media users: Marginalization and moderation gray areas. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2):1-35.
Zhijing Jin, Julius von Kugelgen, Jingwei Ni, Tejas Vaidhya, Ayush Kaushal, Mrinmaya Sachan, and Bernhard Schoelkopf. 2021. Causal direction of data collection matters: Implications of causal and anticausal learning for NLP. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9499-9513, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
David Jurgens, Libby Hemphill, and Eshwar Chandrasekharan. 2019. A just and comprehensive strategy for using NLP to address online abuse. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3658-3666, Florence, Italy. Association for Computational Linguistics.
Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. 2019. Learning the difference that makes a difference with counterfactually-augmented data. In International Conference on Learning Representations.
Katherine Keith, Douglas Rice, and Brendan O'Connor. 2021. Text as causal mediators: Research design for causal estimates of differential treatment of social groups via language aspects. In Proceedings of the First Workshop on Causal Inference and NLP, pages 21-32, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Davani, Morteza Dehghani, and Xiang Ren. 2020. Contextualizing hate speech classifiers with post-hoc explanation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5435-5442.
Fereshte Khani and Percy Liang. 2021. Removing spurious features can hurt accuracy and affect groups disproportionately. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 196-205.
Preslav Nakov, Vibha Nayak, Kyle Dent, Ameya Bhatawdekar, Sheikh Muhammad Sarwar, Momchil Hardalov, Yoan Dinkov, Dimitrina Zlatkova, Guillaume Bouchard, and Isabelle Augenstein. 2021. Detecting abusive language on online platforms: A critical analysis. arXiv preprint arXiv:2103.00153.
Debora Nozza, Claudia Volpetti, and Elisabetta Fersini. 2019. Unintended Bias in Misogyny Detection. In IEEE/WIC/ACM International Conference on Web Intelligence, WI '19, page 149-155, New York, NY, USA. Association for Computing Machinery.
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825-2830.
Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond Accuracy: Behavioral Testing of NLP Models with CheckList. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902-4912, Online. Association for Computational Linguistics.
Sarah T Roberts. 2019. Behind the screen. Yale University Press.
Francisco Rodriguez-Sanchez, Jorge Carrillo de Albornoz, Laura Plaza, Julio Gonzalo, Paolo Rosso, Miriam Comet, and Trinidad Donoso. 2021. Overview of exist 2021: sexism identification in social networks. Procesamento del Lenguaje Natural, 67(0).
Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. 2021. HateCheck: Functional tests for hate speech detection models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 41-58, Online. Association for Computational Linguistics.
Mattia Samory, Indira Sen, Julian Kohne, Fabian Flock, and Claudia Wagner. 2021. Call me sexist, but...: Revisiting sexism detection using psychological scales and adversarial samples.
Indira Sen, Mattia Samory, Fabian Flock, Claudia Wagner, and Isabelle Augenstein. 2021. How does counterfactually augmented data impact models for social computing constructs? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 325-344, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Bertie Vidgen, Alex Harris, Dong Nguyen, Rebekah Tromble, Scott Hale, and Helen Margetts. 2019. Challenges and frontiers in abusive content detection. In Proceedings of the Third Workshop on Abusive Language Online, pages 80-93, Florence, Italy. Association for Computational Linguistics.
Bertie Vidgen, Tristan Thrush, Zeerak Waseem, and Douwe Kiela. 2021. Learning from the worst: Dynamically generated datasets to improve online hate detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1667-1682, Online. Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld. 2021. Polyjuice: Generating counterfactuals for explaining, evaluating, and improving
models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6707-6723, Online. Association for Computational Linguistics.
This is the appendix for the paper, "Counterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection". The appendix contains results of the logistic regression models trained on different types of CAD (7), performance on in-domain data (8), and details for facilitating reproducibility (9).
7 Performance of LogReg Models trained on different types of CAD
In Figures 3 and 4, we present th results for logistic regression models trained on different types of CAD. We note that, similar to results for the CF and nCF models, logistic regression models have lower performance than BERT models. Furthermore, we also note that, like the BERT models, the logistic regression models trained on construct-driven CAD (CF_const) have high false positive rates compared to models trained on other types of CAD.
8 In-domain Performance
In Table 4, we report the performance of the CF and nCF models on the in-domain datasets. To ensure fair comparison with the results in 2, instead of computing results on the entire test set, we subset it in a manner similar to ISG; i.e., we retain only those instances which have identity words for hate speech and gender word for sexism. The results are in line with what Sen et al. reported — nCF models perform better in the in-domain datasets. We note that even though CF models have lower F1 score, they have a higher FPR even in-domain.
We report the results of models trained on different types of CAD in Table 5. Notably, the CF_const models have the highest FPR similar to results on ISG and HC, but also have the lowest F1 score in the in-domain subset.
9 Reproducibility
9.1 Compute Infrastructure
For the logistic regression models we used the scikit learn package (Pedregosa et al., 2011) and for finetuning BERT, we used the Transformers library from HuggingFace (Wolf et al., 2020). All models were trained or finetuned on a 40 core Intel(R) Xeon(R) CPU E5-2690 (without GPU).
Figure 3: FPR and F1 for different types of CF Logistic Regression models, with the nCF model as a baseline on ISG. For hate speech, CF_const has the highest FPR and the highest F1, while the models perform equally well for sexism.
Figure 4: F1 and FPR for different types of CF logistic regression models for HC, with the nCF model as a baseline. Models trained on construct-driven CAD (CF_const) incur the high FPR, especially for sexism.
9.2 Model Training Details: Hyperparameters and Time Taken
We preprocess all the data by removing social media features such as hashtags and mentions. The hyperparameter bounds for LR models are:
1. stopwords: English, none, English without negation words
2. norm: ('11', '12')
3. C: (0.01, 0.1, 1)
4. penalty: ('12', '11')
while for BERT we use:
1. epochs:[4, 5]
2. learning rate: 2e-5, 3e-5, 5e-5
For LR, we have 36 combinations over 5 fold cross-validation, leading to 180 fits, while for BERT, we have 6 combinations also over 5 fold CV, leading to 30 fits.
We use gridsearch for determining hyperparameter, where the metric for selection was macro F1. Run times and hyperparameter configurations for the best performance for all CF (with randomly sampled  $50\%$  data) and nCF models (RQ1) are included in Table 6. The hyperparameters and run times for the CF models trained on different types of CAD (RQ2) are in Table 7.
#In-domain Subset	pos class prop	construct	model	mode	macro F1	FPR	FNR
105	65	hate speech	bert	CF	0.92	0.10	0.06
nCF	0.99	0.00	0.02
logreg	CF	0.77	0.20	0.23
nCF	0.90	0.10	0.09
886	501	sexism	bert	CF	0.76	0.41	0.09
nCF	0.81	0.21	0.17
logreg	CF	0.65	0.44	0.26
nCF	0.75	0.24	0.26
Table 4: Performance of CF and nCF models on the subset of the in-domain dataset containing identity terms. As reported in (Sen et al., 2021) and in contrast to out-of-domain performance, nCF models have higher F1 scores in the in-domain dataset, while CF models still have higher FPR as they do in ISG.
construct	model	mode	macro F1	FPR	FNR
hate speech	bert	CF_mix	0.98	0.02	0.02
CF_const	0.96	0.08	0.02
CF_agn	0.99	0.00	0.02
logreg	CF_mix	0.90	0.12	0.08
CF_const	0.88	0.15	0.09
CF_agn	0.95	0.02	0.06
sexism	bert	CF_mix	0.80	0.25	0.16
CF_const	0.79	0.29	0.14
CF_agn	0.80	0.27	0.13
logreg	CF_mix	0.74	0.25	0.27
CF_const	0.68	0.35	0.28
CF_agn	0.74	0.23	0.28
Table 5: Performance of different types of CF models on the subset of the in-domain dataset containing identity terms. Models trained on construct-driven CAD (CF_const) have the lowest F1 score while having the highest FPR.
9.3 Metrics
The evaluation metrics used in this paper are macro average F1, False Positive Rate (FPR) and False Negative Rate (FNR). We used the sklearn implementation of the macro F1 score: https://scikit-learn.org/stable/modules/ generated/sklearn.metrics.accuracy_ recall_fscore_support.html. The code for computing FPR and FNR is included in our code (uploaded with the submission)
9.4 Model Parameters
Model parameters are included in Table 8.
construct	model	best model hyperparameters	time to train (one run)
sexism	CF LR	english, 12, 0.01, 12	5.42s
CF BERT	epochs: 5, learning rate: 2e-5	3h42m20s
nCF LR	none, 12, 0.01, 12	4.87s
nCF BERT	epochs: 5, learning rate: 2e-5	3h38m57s
hate speech	CF LR	english without negation, 12, 0.01, 12	26.27s
CF BERT	epochs: 4, learning rate: 5e-5	17h54m03s
nCF LR	english without negation, 12, 0.01, 12	26.67s
nCF BERT	epochs: 5, learning rate: 5e-5	17h39m29s
Table 6: Hyperparameters for CF (trained on  ${50}\%$  CAD) and nCF models.
construct	model	best model hyperparams	time to train (one run)
sexism	CF_c LR	english, 11, 1, 11	5.91s
CF_a LR	english without negation, 11, 1, 11	6.15s
CF_r LR	english, 12, 0.1, 12	5.27s
CF_c BERT	epochs: 5, learning rate: 5e-5	3h42m20s
CF_a BERT	epochs: 5, learning rate: 3e-5	3h34m36s
CF_r BERT	epochs: 5, learning rate: 2e-5	3h50m18s
hate speech	CF_c LR	english without negation, 11, 1, 11	33.35s
CF_a LR	english without negation, 11, 0.1, 11	30.08s
CF_r LR	none, 11, 0.1, 11	32.67s
CF_c BERT	epochs: 5, learning rate: 3e-5	18h09m11s
CF_a BERT	epochs: 5, learning rate: 3e-5	17h58m33s
CF_r BERT	epochs: 5, learning rate: 2e-5	17h49m46s
Table 7: CF models trained on different types of CAD.
construct	model	#params
Sexism	CF LR	4750
nCF LR	5505
CF BERT	110M
nCF BERT
Hate speech	CF LR	13763
nCF LR	14800
CF BERT	110M
nCF BERT
Table 8: Number of model parameters for the CF and nCF models.