Controlling Bias Exposure for Fair Interpretable Predictions (2022)

Abstract: Recent work on reducing bias in NLP models usually focuses on protecting or isolating information related to a sensitive attribute (like gender or race). However, when sensitive information is semantically entangled with the task information of the input, e.g., gender information is predictive for a profession, a fair trade-off between task performance and bias mitigation is difficult to achieve. Existing approaches perform this trade-off by eliminating bias information from the latent space, lacking control over how much bias is necessarily required to be removed. We argue that a favorable debiasing method should use sensitive information ‘fairly’, rather than blindly eliminating it (Caliskan et al., 2017; Sun et al., 2019; Bogen et al., 2020). In this work, we provide a novel debiasing algorithm by adjustingthe predictive model’s belief to (1) ignore the sensitive information if it is not useful for the task; (2) use sensitive information minimally as necessary for the prediction (while also incurring a penalty). Experimental results on two text classification tasks (influenced by gender) and an open-ended generation task (influenced by race) indicate that our model achieves a desirable trade-off between debiasing and task performance along with producing debiased rationales as evidence.

## introduction human-written language contains implicit or explicit biases and stereotypes, which make their way into deep natural language processing (nlp) systems through the learning procedure. emerging works show that biases may have worrisome influence and even lead to unfair outcomes in various nlp tasks like text classification @xcite @xcite , coreference resolution @xcite , toxicity detection @xcite @xcite , language modeling @xcite @xcite , etc. recently, several works have attempted to address bias issues in nlp tasks. one stream of approaches is sensitive attribute protection @xcite @xcite @xcite , which mitigates bias by isolating or protecting certain sensitive attributes like race or gender from decision making. however, real-world human-written language is complicated and there are often cases where sensitive information is entangled tightly with the semantics of the sentence @xcite . in this situation, protecting the attribute will unavoidably affect the model's performance. for example, isolating all the underlined words in example 1. he is a congressman and he is good at singing. might misguide a 'profession' classifier to get a result of a singer (instead of a congressman). the balance between bias mitigation and other desired goals is challenging in current debiasing scenarios @xcite . conceptually, debias methods that protect sensitive attributes in some latent space may achieve such a delicate equilibrium if bias is reduced to some precise degree. however, controlling the degree of debiasing in a transparent fashion is challenging @xcite as these methods @xcite @xcite operate in a black-box style, providing no evidence for bias mitigation or task performance. hence, it remains hard for human users to understand and trust the underlying debiasing mechanism. inspired by @xcite , we believe a favorable debiasing method should aim to teach a model to behave fairly instead of blinding its perspective from certain sensitive information @xcite . to this end, we propose a novel debiasing algorithm that produces evidence behind a task prediction while constrainwe regulate the contribution (energy) of each token responsible for 'profession' classification according to their predictability of 'gender'. task energy of a biased token is decreased and re-allocated to its replacement. we design our algorithm based on following principles: it is fair to (1) ignore a sensitive information if it is not useful for the task prediction; recent works @xcite have shown that rationales are an effective way to justify the reasoning behind a prediction from a neural model. therefore, we work with rationales for task prediction and measure their importance based on energy for both task prediction and being biased. we eventually optimize the task rationale in such a way that all tokens of the task rationales will have low bias energy without sacrificing the task performance by blindly removing all bias information. we evaluate our method on two classification tasks that are influenced by gender and an opedended generation task that is influenced by race as a sensitive attribute. comprehensive experiments reveal that our method achieves best trade-off between task performance and bias mitigation, simultaneously producing concise and faithful rationales. we indeed observe that extreme debiasing in baselines hurt task performance whereas performance-aware removal of sensitive information does not affect model performance, rather improves interpretability. to the best of our knowledge, our work is the first to investigate debiasing using interpretable models and we hope that this work will provide a new perspective of controllable debiasing for fair interpretable models. our codes are released in https://github.com/ zexuehe/interpretable_debiasing . ## related work debiasing on data is a debiasing method that focuses on augmenting or cleaning the existing datasets. counterfactual data augmentation (cda) @xcite replaces the bias component of each example in a dataset with a counterfactual one. several works followed cda to propose specific augmentation functions for coreference resolution @xcite , machine translation (saunders and byrne, 2020; costa-jussà and de jorge, 2020), language modeling @xcite . despite being effective, cda's augmenting functions are heuristic and require human intervention. data cleaning for debiasing aims to generate a neutral version of biased input with paraphrasing techniques such as back-translation @xcite and rewriting @xcite , however it is often challenging to maintain the same semantic meaning before and after paraphrasing. debiasing on representation methods usually operate on the embedding space of inputs @xcite or tokens @xcite @xcite . the sensitive information is removed by optimizing the encoder with reversed gradients from a bias discriminator @xcite , or projecting the latent space to an orthogonal subspace @xcite . some works also design the regularization techniques for equalizing bias-specific tokens @xcite . however, these methods are typically black-box, and controlling the degree of debiasing is often difficult bias rationale extractor 𝒙 𝒛 𝒃 𝒙 ⊙ encoder prediction error sparsity constraint task rationale extractor pretrained bias rationale extractor 𝒛 𝒕 𝒆 𝒕 𝒙 ⊙ encoder debiasing constraint sparsity constraint prediction error 𝒙 𝒆 𝒃 step2: debias and predict on task step1: extract bias rationale we first pretrain a bias rationale extraction framework and obtain bias energy for each input token. then we train a fair task prediction model where the task rationales are regulated by a debiasing constraint based on bias energy. a token with high bias energy will be penalized for being in task rationale with a decrease in its original task importance. our method aims to understand bias in predictive models and mitigate it while maintaining task performance in a controllable and interpretable fashion. in general, our method does not contradict previous works in terms of debiasing, and can be flexibly combined with other debiasing methods (e.g., cda first, then ours). ## approach in this section, we introduce our interpretable debiasing algorithm that uses a 'fair' amount of sensitive information in the important parts of input (a.k.a. rationale). we aim to perform a predictive task (e.g., predicting a profession based on a biography) while minimizing the impact of sensitive information (e.g., gender) with minimally affecting the performance of the original task. given an input, there are tokens that are predictive of the task output (we call them task rationales) and there are tokens that carry the sensitive information (we call them bias rationales). with energy functions, we measure how important a token is for the task output or how sensitive it is. by constraining the use of biased input tokens, we control the task energy so that the model is allowed to be exposed to a minimum of bias that is necessary to the task. ## conclusion we proposed a fair and interpretable debiasing method that can control bias exposure by balancing bias mitigation and task performance. while previous methods often debias too strongly or with lesser control and transparency, we show, on three different tasks, that our method achieves the best trade-off between task performance and bias mitigation, while producing the most faithful rationales for the debiased task prediction. we also indicate cases where it is even necessary to keep sensitive information that is useful for task output. our model provides fair control on bias exposure, especially in such cases, instead of blindly debiasing the input with minimal interpretation.