Abstract: We examine whether neural natural language processing (NLP) systems reflect historical biases in training data. We define a general benchmark to quantify gender bias in a variety of neural NLP tasks. Our empirical evaluation with state-of-the-art neural coreference resolution and textbook RNN-based language models trained on benchmark datasets finds significant gender bias in how models view occupations. We then mitigate bias with CDA: a generic methodology for corpus augmentation via causal interventions that breaks associations between gendered and gender-neutral words. We empirically show that CDA effectively decreases gender bias while preserving accuracy. We also explore the space of mitigation strategies with CDA, a prior approach to word embedding debiasing (WED), and their compositions. We show that CDA outperforms WED, drastically so when word embeddings are trained. For pre-trained embeddings, the two methods can be effectively composed. We also find that as training proceeds on the original data set with gradient descent the gender bias grows as the loss reduces, indicating that the optimization encourages bias; CDA mitigates this behavior.

Introduction: Natural language processing (NLP) with neural networks has grown in importance over the last few years. They provide state-of-the-art models for tasks like coreference resolution, language modeling, and machine translation BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 . However, since these models are trained on human language texts, a natural question is whether they exhibit bias based on gender or other characteristics, and, if so, how should this bias be mitigated. This is the question that we address in this paper. Prior work provides evidence of bias in autocomplete suggestions BIBREF5 and differences in accuracy of speech recognition based on gender and dialect BIBREF6 on popular online platforms. Word embeddings, initial pre-processors in many NLP tasks, embed words of a natural language into a vector space of limited dimension to use as their semantic representation. BIBREF7 and BIBREF8 observed that popular word embeddings including word2vec BIBREF9 exhibit gender bias mirroring stereotypical gender associations such as the eponymous BIBREF7 "Man is to computer programmer as Woman is to homemaker". Yet the question of how to measure bias in a general way for neural NLP tasks has not been studied. Our first contribution is a general benchmark to quantify gender bias in a variety of neural NLP tasks. Our definition of bias loosely follows the idea of causal testing: matched pairs of individuals (instances) that differ in only a targeted concept (like gender) are evaluated by a model and the difference in outcomes (or scores) is interpreted as the causal influence of the concept in the scrutinized model. The definition is parametric in the scoring function and the target concept. Natural scoring functions exist for a number of neural natural language processing tasks. We instantiate the definition for two important tasks—coreference resolution and language modeling. Coreference resolution is the task of finding words and expressions referring to the same entity in a natural language text. The goal of language modeling is to model the distribution of word sequences. For neural coreference resolution models, we measure the gender coreference score disparity between gender-neutral words and gendered words like the disparity between “doctor” and “he” relative to “doctor” and “she” pictured as edge weights in Figure FIGREF2 . For language models, we measure the disparities of emission log-likelihood of gender-neutral words conditioned on gendered sentence prefixes as is shown in Figure FIGREF2 . Our empirical evaluation with state-of-the-art neural coreference resolution and textbook RNN-based language models BIBREF2 , BIBREF1 , BIBREF10 trained on benchmark datasets finds gender bias in these models . Next we turn our attention to mitigating the bias. BIBREF7 introduced a technique for debiasing word embeddings which has been shown to mitigate unwanted associations in analogy tasks while preserving the embedding's semantic properties. Given their widespread use, a natural question is whether this technique is sufficient to eliminate bias from downstream tasks like coreference resolution and language modeling. As our second contribution, we explore this question empirically. We find that while the technique does reduce bias, the residual bias is considerable. We further discover that debiasing models that make use of embeddings that are co-trained with their other parameters BIBREF1 , BIBREF10 exhibit a significant drop in accuracy. Our third contribution is counterfactual data augmentation (CDA): a generic methodology to mitigate bias in neural NLP tasks. For each training instance, the method adds a copy with an intervention on its targeted words, replacing each with its partner, while maintaining the same, non-intervened, ground truth. The method results in a dataset of matched pairs with ground truth independent of the target distinction (see Figure FIGREF2 and Figure FIGREF2 for examples). This encourages learning algorithms to not pick up on the distinction. Our empirical evaluation shows that CDA effectively decreases gender bias while preserving accuracy. We also explore the space of mitigation strategies with CDA, a prior approach to word embedding debiasing (WED), and their compositions. We show that CDA outperforms WED, drastically so when word embeddings are co-trained. For pre-trained embeddings, the two methods can be effectively composed. We also find that as training proceeds on the original data set with gradient descent the gender bias grows as the loss reduces, indicating that the optimization encourages bias; CDA mitigates this behavior. In the body of this paper we present necessary background (Section SECREF2 ), our methods (Sections SECREF3 and SECREF4 ), their evaluation (Section SECREF5 ), and speculate on future research (Section SECREF6 ).

Question: How much improvement do they get?
Answer: Their GTRS approach got an improvement of 3.89% compared to SVM and 27.91% compared to Pawlak.