Debiasing with Sufficient Projection: A General Theoretical Framework for Vector Representations (2024)

Abstract: Pre-trained vector representations in natural language processing often inadvertently encode undesirable social biases. Identifying and removing unwanted biased information from vector representation is an evolving and significant challenge. Our study uniquely addresses this issue from the perspective of statistical independence, proposing a framework for reducing bias by transforming vector representations to an unbiased subspace using sufficient projection. The key to our framework lies in its generality: it adeptly mitigates bias across both debiasing and fairness tasks, and across various vector representation types, including word embeddings and output representations of transformer models. Importantly, we establish the connection between debiasing and fairness, offering theoretical guarantees and elucidating our algorithm’s efficacy. Through extensive evaluation of intrinsic and extrinsic metrics, our method achieves superior performance in bias reduction while maintaining high task performance, and offers superior computational efficiency.

## introduction natural language processing (nlp) models have made significant strides in recent years, with much of their success attributed to representation learning -the process of creating effective vector representations for textual data. various research has been conducted in this area, including static word embedding @xcite , contextualized embedding @xcite @xcite , sentence embedding @xcite in addition to other representation methods. however, as vector representations have been applied in a wide range of real-life scenarios, researchers have discovered that stereotypical biases and spurious correlations can be transferred from * corresponding author human-generated corpora to vector representations and models @xcite @xcite . this has the potential to produce biased and unfair outcomes in various downstream tasks @xcite and can even lead to serious social problems. for instance, in the word analogy task presented in @xcite , it was found that the vector representation for -→ she was closer to ---→ nurse than the representation for -→ he was to ---→ doctor. @xcite found a performance gap between different genders in text classification tasks. the bias and fairness issues in nlp models are primarily caused by the unbalanced and stereotypical nature of the training corpora. @xcite described this as unbalanced model behavior in relation to certain socially sensitive topics such as gender, race, and religion. to quantify biases in nlp, two types of bias evaluation metrics(intrinsic and extrinsic) have been proposed. however, recent research has shown that in most cases, there is a weak correlation between them @xcite @xcite . there remains a significant research gap in understanding how to bridge these two kinds of tasks. in our research, we employ statistical independence to establish a theoretical linkage between these tasks, offering insights into the interplay between intrinsic and extrinsic biases. various methods have been proposed for reducing bias in nlp, but it remains a challenge to effectively mitigate bias while maintaining high model performance. furthermore, it is particularly difficult for debiasing methods to efficiently address both intrinsic and extrinsic biases at the same time, as discussed in the related works section. in this paper, we propose a general debiasing method that can effectively mitigate bias across both debiasing and fairness tasks. our key contributions include the following: the structure of this paper is as follows. we begin with a comprehensive review of existing research on bias evaluation and debiasing techniques in nlp. we then introduce our methodology, including our proposed algorithm. we present experimental results on a range of gender bias evaluation tasks, showcasing the effectiveness of our approach. finally, we provide a theoretical bridge and guarantee, and a discussion of our method. our proposed debiasing method is based on the controlled removal of specific information from vector representations, which is closely related to the task of disentangling representations @xcite . previous research in this area includes methods for removing bias from static embeddings, such as projecting the word embedding into the orthogonal space of the gender direction ( -→ he --→ she) @xcite , re-training the entire embedding using some loss functions @xcite , and utilizing the ideas in causal inference @xcite . there are several similar projection-based methods like iterative nullspace linear projection (inlp; @xcite ), rlace @xcite . we discuss the detailed comparison and advantages of our method in appendix a.2. evaluating bias in nlp the measurement methods for evaluating bias in pre-trained word embeddings and language models can be broadly divided into two categories: intrinsic and extrinsic evaluations. intrinsic bias evaluations probe the bias within pre-trained word embeddings and language models. common methods include measuring the geometry in embedding space, such as the word embedding association test (weat; @xcite ) and sentence encoder association test (seat; @xcite ). additionally, @xcite @xcite propose metrics using the likelihood score. furthermore, research suggests that some debiasing methods may only hide bias, and thus additional measurement approaches are needed @xcite . the extrinsic bias is specific to certain downstream tasks. in the text classification task, de-arteaga et al. (2019); blodgett et al. (2016) proposed two benchmark datasets and used the equal opportunity measure from fairness literature. zhao et al. (2018a) proposed the winobias benchmark for coreference resolution. as well as other benchmarks, such as bias-nli @xcite and in machine translation @xcite . however, recent research has indicated that intrin-sic bias in embeddings or models typically does not have a strong correlation with bias in downstream tasks @xcite . @xcite found out that the debiased models re-learn the bias from the finetuning datasets, showing that only debiasing upstream models may not be enough to eliminate bias in downstream tasks. in our work, we conduct comprehensive evaluation experiments in both intrinsic and extrinsic tasks. additionally, our approach avoids the issue of re-learning bias by directly addressing the vector representation in downstream tasks. ## sufficient universal projection (sup) 4.1 subspace estimation in this subsection, we will explain the process of estimating the subspace s 2 = s z|x in different scenarios. we will begin with the simplest case where p 3 = 1, meaning @xmath0. sliced inverse regression (sir) is a classical dimension reduction method proposed by @xcite for univariate response z. we provide the detailed scheme for sir applied on the data set {(x i , z i )} n @xmath1.3. the main procedure of sir is: (1) divide the support of z into h intervals and calculate the covariance matrix of x for each interval, (2) calculate the weighted covariance matrix based on h intervals, (3) obtain the directions from the weighted covariance matrix. the space spanned by the directions @xmath2, . . . , β q ) provided by sir is a consistent estimation of s z|x . for multivariate sensitive attributes, a direct analogy of the slicing strategy in sir no longer works, as the number of partitions of the support of @xmath3, . . . , z p 3 ) ∈ r p 3 becomes h p 3 and thus suffers the curse of dimensionality. to address the limitation of the original sir, the pooled marginal slicing (pms) estimator proposed in @xcite combines the subspaces s z i |x estimated by univariate response sir to get the directions for the multivariate response, which is motivated by the following proposition. proof. for any @xmath4, we have @xmath5, thus b xz ⊂ b xz j . recall that s z|x is the intersection of all elements in b xz . therefore, we have s z j |x ⊆ s z|x for @xmath6, . . . , p 3 . proposition 4.1 indicates that s z j |x can be used to recover s z|x , which guarantees the theoretical properties of pms estimator. it also naturally lifts the curse of dimensionality. let z ij denote the j-th coordinate of i-th sample. we apply sir to data set {(x i , z ij )} n @xmath7, . . . , p 3 . then we define the weighted sum of estimators as m @xmath8, where w i can be chosen as either equal weights or proportional to the leading eigenvalues of m i . then the leading q eigenvectors ψ 1 , . . . , ψ q of m pms can be used to recover s z|x . the detailed implementation of obtaining m pms is summarized in algorithm 2 in the appendix. ## bridge between debiasing and fairness in this section, we provide a theoretical analysis of how our proposed method can incorporate debiasing and fairness tasks into a unified framework and handle them simultaneously. as previously discussed, both tasks aim to achieve conditional independence with respect to certain variables. here, we will demonstrate how minimal subspace s 2 bridges these tasks. we provide the following theorem to prove the effectiveness of our framework in dealing with both debiasing and fairness tasks. please refer to appendix a.7 for detailed proof. theorem 6.1. with the settings defined in section 3 and linearity assumption, suppose theorem 6.1 states that the projected representation @xmath9, which achieves the goal of debiasing task. moreover, if the subspace spanned by sensitive attribute z (s z|x ) is included in the subspace spanned by target attribute y (s y |x ), we can achieve the goal of fairness task by projecting the original representation on (i -q). the theoretical property is consistent with the experimental results shown above. for the debiasing task, the matrix @xmath10, then i -q forms a sufficient projection defined in theorem 6.1, which shows great improvement upon existing state-of-the-art methods. for fairness task, the eigenvectors {ψ j } q @xmath11.1. if we have span{q y } ⊆ span{q}, then we can set @xmath12. however, in real data, this condition is usually violated, which means span{q y } ⊈ span{q}. therefore, the sup may not achieve the optimal fair representation in downstream tasks. ## conclusion and future work in this paper, we propose a theoretically grounded framework for reducing bias by projecting vector representations to an unbiased subspace. it can reduce biased information effectively in both intrinsic and extrinsic tasks, as well as different kinds of representations. in addition, we provide a theoretical guarantee about the effectiveness of our method in reducing biased information. finally, our method not only surpasses existing state-of-theart approaches in bias mitigation while maintaining robust task performance but also achieves superior computational efficiency. while this work has demonstrated its effectiveness in various tasks, it has the potential to be applied to other applications that rely on vector representation. we are also interested in combining our method with the different other notions of fairness. we aim to explore these directions in future work. the scheme of the canada-uk artificial intelligence initiative. the project title is bias: responsible ai for gender and ethnic labour market equality. bei jiang and linglong kong were partially supported by grants from the canada cifar ai chairs program, the alberta machine intelligence institute (amii), and natural sciences and engineering council of canada (nserc), and linglong kong was also partially supported by grants from the canada research chair program from nserc. we also thank all the constructive suggestions and comments from the reviewers. therefore, in model (3), both s 2 and s ′ 2 are sufficient dimension reduction subspaces. the central subspace s z|x is equal to s 2 = {β 1 , β 2 }, which is the minimal subspace.