Adversarial Scrubbing of Demographic Information for Text Classification (2021)

Abstract: Contextual representations learned by language models can often encode undesirable attributes, like demographic associations of the users, while being trained for an unrelated target task. We aim to scrub such undesirable attributes and learn fair representations while maintaining performance on the target task. In this paper, we present an adversarial learning framework “Adversarial Scrubber” (AdS), to debias contextual representations. We perform theoretical analysis to show that our framework converges without leaking demographic information under certain conditions. We extend previous evaluation techniques by evaluating debiasing performance using Minimum Description Length (MDL) probing. Experimental evaluations on 8 datasets show that AdS generates representations with minimal information about demographic attributes while being maximally informative about the target task.

## introduction automated systems are increasingly being used for real-world applications like filtering college applications @xcite , determining credit eligibility @xcite , making hiring decisions @xcite , etc. for such tasks, predictive models are trained on data coming from human decisions, which are often biased against certain demographic groups @xcite @xcite . biased decisions based on demographic attributes can have lasting economic, social and cultural consequences. natural language text is highly indicative of demographic attributes of the author @xcite @xcite @xcite @xcite . language models can often encode such demographic associations even without having direct access to them. prior works have shown that intermediate representations in a deep learning model encode demographic associations of the author or person being spoken about @xcite @xcite . therefore, it is important to ensure that decision functions do not make predictions based on such representations. in this work, we focus on removing demographic attributes encoded in data representations during training text classification systems. to this end, we present "adversarial scrubber" (ads) to remove information pertaining to protected attributes (like gender or race) from intermediate representations during training for a target task (like hate speech detection). removal of such features ensures that any prediction model built on top of those representations will be agnostic to demographic information during decision-making. ads can be used as a plug-and-play module during training any text classification model to learn fair intermediate representations. the framework consists of 4 modules: encoder, scrubber, bias discriminator and target classifier. the encoder generates contextual representation of an input text. taking these encoded contextual representations as input, the scrubber tries to produce fair representations for the target task. the bias discriminator and target classifier predict the protected attribute and target label respectively from the scrubber's output. the framework is trained end-to-end in an adversarial manner @xcite . we provide theoretical analysis to show that under certain conditions encoder and scrubber converge without leaking information about the protected attribute. we evaluate our framework on 5 dialogue datasets, 2 twitter-based datasets and a biographies dataset with different target task and protected attribute settings. we extend previous evaluation methodology for debiasing by measuring minimum description length (mdl) @xcite of labels given representations, instead of probing accuracy. mdl provides a finergrained evaluation benchmark for measuring debiasing performance. we compute mdl using offthe-shelf classifiers 1 making it easier to reproduce. upon training using ads framework, we observe a significant gain in mdl for protected attribute prediction as compared to fine-tuning for the target task. our contributions are: ## related work contextual representations learned during training for a target task can be indicative of features unrelated to the task. such representations can often encode undesirable demographic attributes, as observed in unsupervised word embeddings @xcite and sentence embeddings @xcite . prior work has analysed bias in different nlp systems like machine translation @xcite ; font and costa-jussa, 2019; saunders and byrne, 2020), nli @xcite ), text classification @xcite @xcite , language generation @xcite among others. debiasing sensitive attributes for fair classification was introduced as an optimization problem by @xcite . since then, adversarial training @xcite frameworks have been explored for protecting sensitive attributes for nlp tasks @xcite @xcite ). 1 we use mlpclassifier modules from scikit-learn. our work is most similar to @xcite , which achieves fairness by blindness by learning intermediate representations which are oblivious to a protected attribute. we compare the performance of ads with @xcite in our experiments. ## adversarial scrubber ads takes text documents {x 1 , x 2 , . . . , x n } as input from a dataset d with corresponding target labels {y 1 , y 2 , . . . , y n }. every input x i is also associated with a protected attribute z @xmath0, 2, ...k}. our goal is to construct a model f (x) such that it doesn't rely on z i while making the prediction y @xmath1. the framework consists of 4 modules: (i) encoder h(•) with weights θ h , (ii) scrubber s(•) with weights θ s , (iii) bias discriminator d(•) with weights θ d and (iv) target classifier c(•) with weights θ c as shown in figure 1 . the encoder receives a text input x i , and produces an embedding e @xmath2, which is forwarded to the scrubber. the goal of the scrubber is to produce representation u @xmath3, such that y i can be easily inferred from u i by the target classifier, c, but u i does not have the information required to predict the protected attribute z i by the bias discriminator d. our setup also includes a probing network q, which helps in evaluating the fairness of the learned representations. bias discriminator d is updated using the gradients: update the encoder h, scrubber s, and task classifier c using the gradients: in the rest of this section, we describe ads assuming a single bias discriminator. however, ads can easily be extended to incorporate multiple discriminators for removing several protected attributes (discussed in section 6.1). scrubber: the scrubber receives the input representation h(x i ) from encoder and generates representation u @xmath4. the goal of the scrubber is to produce representations such that the bias discriminator finds it difficult to predict the protected attribute z i . to this end, we consider two loss functions: entropy loss: in the entropy loss, the encoder and scrubber parameters are jointly optimized to increase the entropy of the prediction probability distribution, h(d(u i )). target classifier: the target classifier predicts the target label y i from u i by optimizing the cross entropy loss: l c (c(u i ), y i ). the scrubber, target classifier, and encoder parameters are updated simultaneously to minimize the following loss: bias discriminator: the bias discriminator, which predicts the protected attribute z i , is trained to reduce the cross-entropy loss for predicting training: the bias discriminator and scrubber (along with target classifier and encoder) are trained in an iterative manner as shown in algorithm 1. first, the bias discriminator is updated using gradients from the loss in equation 1. then, the encoder, scrubber and target classifier are updated simultaneously using the gradients shown in equation 2 . probing network: @xcite showed that in an adversarial setup even when the discriminator achieves random performance for predicting z, it is still possible to retrieve z using a separately trained classifier. therefore, to evaluate the amount of information related to y and z present in representations u, we use a probing network q. after ads is trained, we train q on representations h(x) and s(h(x)), to predict y and z (q is trained to predict y and z separately). we consider an information leak from a representation, if z can be predicted from it with above random performance. if the prediction performance of q for z is significantly above the random baseline, it means that there is information leakage of the protected attribute and it is not successfully guarded. ## theoretical analysis proposition 1. minimizing l s is equivalent to increasing bias discriminator loss l d . proof: entropy and δ-loss components of l s tries to increase the bias discriminator loss. the discriminator cross-entropy loss l d can be written as: the same holds true for the δ-loss component. δ(o i ) reduces the probability assigned to the true output class which increases the cross entropy loss l d (detailed proof provided in appendix a.2 due to space constraint). minimizing the entropy and δloss components of the scrubber loss l s increases l d for a fixed bias discriminator. therefore, assuming our framework converges to (θ * s , θ * h , θ * d ) using gradient updates from l s we have: proposition 2. let the discriminator loss l d be convex in θ d , and continuous differentiable for all θ d . let us assume the following: (a) θ proof: as the bias discriminator converges to θ * d , we have: we can show that: proof: an optimal bias discriminator d(•) minimizes the prediction entropy, thereby increasing the entropy and δ-loss. given (θ as l d does not decrease, and d(•) is optimal it shows that no additional information about z is revealed which the bias discriminator can leverage to reduce l d . this shows that starting from (θ ## experiments in this section, we describe our experimental setup and evaluate ads on several benchmark datasets. ## results the evaluation results for all datasets are reported in f1-score significantly above the random baseline, (iii) "ads -h(x)" setup achieves similar f1 score for predicting y, but still has a f1-score for z significantly above the random baseline. (iv) "ads s(h(x))" performs the best in terms of guarding the protected attribute z (lowest prediction f1-score and highest mdl) by achieving near random f1score across all datasets. it is also able to maintain performance on the target task, as we observe only a slight drop compared to the fine-tuning performance ("w/o adversary h(x)" for predicting y). dial & pan16: next, we focus on the twitterbased datasets dial & pan16, where the target task is sentiment classification/mention detection and the protected attribute is one of the demographic associations (race/gender/age) of the author. the evaluation results are reported in comparison with prior work: we report two metrics following elazar and goldberg (2018): (i) ∆ z -which denotes the performance above the random baseline for z (50% for both pan16 and dial) (ii) acc y -is the probing accuracy on the setup pan16 age (z 1 ) gender (z 2 ) mention (y) f1↓ mdl↑ f1↓ mdl↑ f1↑ mdl↓ random 50.0 -50.0 -50.0 w/o adversary h(x) 66.5 196.4 69.3 192.0 88.6 6.8 ads s(h(x)) -(age) 61.5 224.2 62.6 218.7 88.7 14.3 ads s(h(x)) -(gender) 60.6 222.6 64.2 216.8 88.6 12.9 ads s(h(x)) -(both) ## conclusion in this work, we proposed adversarial scrubber (ads) to remove demographic information from contextual representations. theoretical analysis showed that under certain conditions, our framework converges without leaking information about protected attributes. we extend previous evaluation metrics to evaluate fairness of representations by using mdl. experimental evaluations on 8 datasets show that ads is better at protecting demographic attributes than baselines. we show that our approach is scalable and can be used to remove multiple protected attributes simultaneously. future work can explore leveraging ads towards learning fair representations in other nlp tasks.