Attributing Fair Decisions with Attention Interventions (2022)

Abstract: The widespread use of Artificial Intelligence (AI) in consequential domains, such as health-care and parole decision-making systems, has drawn intense scrutiny on the fairness of these methods. However, ensuring fairness is often insufficient as the rationale for a contentious decision needs to be audited, understood, and defended. We propose that the attention mechanism can be used to ensure fair outcomes while simultaneously providing feature attributions to account for how a decision was made. Toward this goal, we design an attention-based model that can be leveraged as an attribution framework. It can identify features responsible for both performance and fairness of the model through attention interventions and attention weight manipulation. Using this attribution framework, we then design a post-processing bias mitigation strategy and compare it with a suite of baselines. We demonstrate the versatility of our approach by conducting experiments on two distinct data types, tabular and textual.

## introduction machine learning algorithms that optimize for performance (e.g., accuracy) often result in unfair outcomes @xcite . these algorithms capture biases present in the training datasets causing discrimination toward different groups. as machine learning continues to be adopted into fields where discriminatory treatments can lead to legal penalties, fairness and interpretability have become a necessity and a legal incentive in addition to an ethical responsibility @xcite . existing methods for fair machine learning include applying complex transformations to the data so that resulting representations are fair @xcite @xcite @xcite , adding regularizers to incorporate fairness @xcite @xcite , or modifying the outcomes of unfair machine learning algorithms to ensure fairness @xcite , among others. here we present an alternative approach, which works by identifying the significance of different features in causing unfairness and reducing their effect on the outcomes using an attention-based mechanism. with the advancement of transformer models and the attention mechanism @xcite , recent research in natural language processing (nlp) has tried to analyze the effects and the interpretability of the attention weights on the decision making process @xcite @xcite . taking inspiration from these works, we propose to use an attention-based mechanism to study the fairness of a model. the attention mechanism provides an intuitive way to capture the effect of each attribute on the outcomes. thus, by introducing the attention mechanism, we can analyze the effect of specific input features on the model's fairness. we form visualizations that explain model outcomes and help us decide which attributes contribute to accuracy vs. fairness. we also show and confirm the observed effect of indirect discrimination in previous work @xcite @xcite in which even with the absence of the sensitive attribute, we can still have an unfair model due to the existence of proxy attributes. furthermore, we show that in certain scenarios those proxy attributes contribute more to the model unfairness than the sensitive attribute itself. based on the above observations, we propose a post-processing bias mitigation technique by diminishing the weights of features most responsible for causing unfairness. we perform studies on datasets with different modalities and show the flexibility of our framework on both tabu-lar and large-scale text data, which is an advantage over existing interpretable non-neural and non-attention-based models. furthermore, our approach provides a competitive and interpretable baseline compared to several recent fair learning techniques. ## approach in this section, we describe our classification model that incorporates the attention mechanism. it can be applied to both text and tabular data and is inspired by works in attention-based models in text-classification @xcite . we incorporate attention over the input features. next, we describe how this attention over features can attribute the model's unfairness to certain features. finally, using this attribution framework, we propose a post-processing approach for mitigating unfairness. in this work, we focus on binary classification tasks. we assume access to a dataset of triplets @xmath0, y i , a i } n @xmath1, where x i , y i , a i are i.i.d. samples from data distribution p(x, y, a). @xmath2, . . . a l } is a discrete variable with l possible values and denotes the sensitive or protected attributes with respect to which we want to be fair, @xmath3, 1} is the true label, @xmath4. we use ŷo to denote the binary outcome of the original model, and ŷk z will represent the binary outcome of a model in which the attention weights corresponding to k th feature are zeroed out. our framework is flexible and general that it can be used to find attribution for any fairness notion. more particularly, we work with the group fairness measures like statistical parity @xcite , equalized odds @xcite , and equality of opportunity @xcite , which are defined as: foot_0 statistical parity difference (spd): ## experimental setup we perform a suite of experiments on synthetic and real-world datasets to evaluate our attention based interpretable fairness framework. the experiments on synthetic data are intended to elucidate interpretability in controlled settings, where we can manipulate the relations between input and output feature. the experiments on real-world data aim to validate the effectiveness of the proposed approach on both tabular and non-tabular (textual) data. ## discussion in this work, we analyzed how attention weights contribute to fairness and accuracy of a predictive model. we proposed an attribution method that leverages the attention mechanism and showed the effectiveness of this approach on both tabular and text data. using this interpretable attribution framework we then introduced a post-processing bias mitigation strategy based on attention weight manipulation. we validated the proposed framework by conducting experiments with different baselines, fairness metrics, and data modalities.