SlowGuess/ACL_2020_OCR huggingface dataset:
for semantic compositionality over a sentiment treebank. In EMNLP.
Chen Sun, Austin Oliver Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid. 2019a. Videobert: A joint model for video and language representation learning. CoRR, abs/1904.01766.
Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019b. Mitigating gender bias in natural language processing: Literature review. In ACL.
Nathaniel Swinger, Maria De-Arteaga, Neil Thomas Heffernan IV, Mark DM Leiserson, and Adam Tauman Kalai. 2019. What are the biases in my word embedding? In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES '19, page 305-311, New York, NY, USA. Association for Computing Machinery.
Jack Urbanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktaschel, Douwe Kiela, Arthur Szlam, and Jason Weston. 2019. Learning to speak and act in a fantasy text adventure game. CoRR, abs/1903.03094.
Sumithra Velupillai, Hanna Suominen, Maria Liakata, Angus Roberts, Anoop D. Shah, Katherine Morley, David Osborn, Joseph Hayes, Robert Stewart, Johnny Downs, Wendy Chapman, and Rina Dutta. 2018. Using clinical natural language processing for health outcomes research: Overview and actionable suggestions for future advances. Journal of Biomedical Informatics, 88:11 - 19.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Black-boxNLP.
Chenguang Wang, Mu Li, and Alexander J. Smola. 2019. Language models with transformers. CoRR, abs/1904.09408.
Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. 2018. Neural network acceptability judgments. arXiv preprint arXiv:1805.12471.
Shijie Wu and Mark Dredze. 2019. Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 833-844, Hong Kong, China. Association for Computational Linguistics.
Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. From recognition to cognition: Visual commonsense reasoning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
5512
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In AIES.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Coterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 629-634, Minneapolis, Minnesota. Association for Computational Linguistics.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979-2989, Copenhagen, Denmark. Association for Computational Linguistics.
Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018. Learning gender-neutral word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847–4853, Brussels, Belgium. Association for Computational Linguistics.
Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, and Kai-Wei Chang. 2019. Examining gender bias in languages with grammatical gender. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5276-5284, Hong Kong, China. Association for Computational Linguistics.
A Debiasing Details
We provide some details on estimating the bias subspaces and debiasing steps.
Bias Attribute Words: Table 6 shows the bias attribute words we used to estimate the bias subspaces for binary gender bias and multiclass religious biases.
Datasets: We provide some details on dataset downloading below:
1. WikiText-2 was downloaded from https://github.com/pytorch/examples/tree/master/word_language_model. We took the first  $10\%$  of WikiText-2 sentences as naturally occurring templates representative of highly formal text.
2. Hacker News and Reddit Subreddit data collected from news and discussion forums related to topics ranging from politics to electronics was downloaded from https://github.com/minimaxir/textgenrnn/.
B.1 BERT
All three variants of BERT (BERT, BERT post SST, BERT post CoLA) are uncased base model with hyper-parameters described in Table 7.
For all three models, the second output "pooled_output" of BERT is treated as the sentence embedding. The variant BERT is the pretrained model with weights downloaded from https://s3.amazon.com/models.huggingface.
co/bert/bert-base-uncased.tar.gz. The variant BERT post SST is BERT after being finetuned on the Stanford Sentiment Treebank(SST-2) task, a binary single-sentence classification task (Socher et al., 2013). During fine-tuning, we first normalize the sentence embedding and then feed it into a linear layer for classification. The variant BERT post CoLA is BERT fine-tuned on the Corpus of Linguistic Acceptability (CoLA) task, a binary single-sentence classification task. Normalization and classification are done exactly the same as BERT post SST. All BERT models are fine-tuned for 3 epochs which is the default hyper-parameter in the huggingface transformers repository. Debiasing for BERT models that are fine-tuned is done just before the classification layer.
Binary Gender
man, woman
boy, girl	Multiclass Religion
he, she	jewish, christian, muslim
father, mother	jews, christians, muslims
son, daughter	torah, bible, quran
guy, gal	synagogue, church, mosque
male, female	rabbi, priest, imam
his, her	judaism, christianity, islam
himself, herself
John, Mary
Table 6: Word pairs to estimate the binary gender bias subspace (left) and the 3-class religion bias subspace (right).
Hyper-parameter	Value
attention-probs_dropoutprob	0.1
hidden_ACT	gelu
hidden_dropoutprob	0.1
hidden_size	768
initializer_range	0.02
intermediate_size	3072
max_position_embeddings	512
num attentio_heads	12
num_hidden_layers	12
type_vocab_size	2
vocab_size	30522
Table 7: Configuration of BERT models, including BERT, BERT  $\rightarrow$  SST, and BERT  $\rightarrow$  CoLA.
B.2 ELMo
We use the ElmoEmbedder from allenlp COMMANDS.elmo. We perform summation over the aggregated layer outputs. The resulting sentence representation is a time sequence vector with data dimension 1024. When computing gender direction, we perform mean pooling over the time dimension to obtain a 1024-dimensional vector for each definitional sentence. In debiasing, we remove the gender direction from each time step of each sentence representation. We then feed the debiased representation into an LSTM with hidden size 512. Finally, the last hidden state of the LSTM goes through a fully connected layer to make predictions.
We also studied the effect of templates on BERT fine-tuned on CoLA as well. Steps taken are exactly the same as described in Effect of Templates: Same Domain, More Quantity and Effect of Templates: Same Quantity, More Domains. Results are plotted in Figure 4. It shows
Figure 4: Evaluation of Bias Removal on BERT fine-tuned on CoLA with varying percentage of data from a single domain (left) and varying number of domains with fixed total size (right).
that debiasing performance improves and stabilizes with the number of sentence templates as well as the number of domains.