Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection (2022)

Abstract: Bias elimination and recent probing studies attempt to remove specific information from embedding spaces. Here it is important to remove as much of the target information as possible, while preserving any other information present. INLP is a popular recent method which removes specific information through iterative nullspace projections.Multiple iterations, however, increase the risk that information other than the target is negatively affected.We introduce two methods that find a single targeted projection: Mean Projection (MP, more efficient) and Tukey Median Projection (TMP, with theoretical guarantees). Our comparison between MP and INLP shows that (1) one MP projection removes linear separability based on the target and (2) MP has less impact on the overall space.Further analysis shows that applying random projections after MP leads to the same overall effects on the embedding space as the multiple projections of INLP. Applying one targeted (MP) projection hence is methodologically cleaner than applying multiple (INLP) projections that introduce random effects.

## introduction word embedding spaces can contain rich information that is valuable for a wide range of nlp tasks. high quality word embeddings should capture the semantic attributes associated with a word's meaning. though we can establish that points representing words with similar semantic attributes tend to be close to each other, it is challenging to reason which attributes are captured to what extent and, in particular, which dimensions capture these attributes @xcite . but even though we do not (yet) know exactly which attributes are represented in which way, we nevertheless may want to ensure that one particular attribute is no longer present in the embedding. for example, because this attribute is a protected attribute such as gender which can lead to harmful bias, or because we want to test how the attribute was affecting a system in a probing setup. there are various approaches to remove a particular attribute from an embedding. clearly it is important to remove as much of the target attribute as possible while preserving other information present. arguably linear projections are one of the least intrusive methods to transform an embedding space. iterative nullspace projection @xcite is a recent popular method which follows this paradigm and uses multiple linear projections. inlp achieves linear guarding of the protected attribute, that is, a linear classifier can no longer separate instances that have the attribute from instances that do not. however, it remains unclear to what extent inlp affects the remainder of the embedding space. in general it requires 10-15 iterations (projections) to achieve linear guarding; it is hence likely that other attributes are negatively affected as well. this is particularly problematic in settings where inlp is used to determine what impact a specific attribute has on overall systems. in this paper, we introduce two methods that find a single targeted projection which achieves linear guarding at once. specifically, our mean projection [mp] method uses the mean of the points in each class to determine the projection direction. the mean is more robust to unbalanced class distributions than inlp. the mean can, however, be overly sensitive to outliers. we prove that our second method, the tukey median projection @xcite , finds nearly worst-case optimal projections for any input. unfortunately, computing the tukey median @xcite is computationally expensive in high dimensions. in our experiments we hence compared only mp to inlp. specifically, we carried out the gender debiasing experiments of @xcite with both methods. they show that 1. mp only needs one projection for linearly guarding gender where inlp needs 10-15. inlp projections improve simlex scores and reduce weat scores more than mp. a priori it is unclear why their many projections should have this effect. we investigated and show that the same improvements appear after applying random projections (either after mp or after the first inlp projections). applying one mp projection to linearly guard an attribute is hence methodologically cleaner than applying multiple (inlp) projections. ## related work multiple methods for removing bias from embeddings have been suggested. bias can be addressed at the level of the training data @xcite , the training process @xcite , and the resulting model itself @xcite @xcite . debiasing an existing model has the clear advantage that it can be done with relatively little data and that the model does not need to be retrained. in this paper, we focus on removing bias through transforming the embedding space. transformations of embedding spaces can be performed by means of linear projections @xcite , multiple linear projections @xcite , or via non-linear kernels @xcite . @xcite introduce a method called iterative nullspace projection (inlp) where they attempt to remove the bias by projecting points iteratively. our methods achieve linear guarding after a single linear projection. as such we directly improve upon the inlp method, which we describe in detail in section 3. in the remainder of this section, we focus on more recent post-training approaches and applications of inlp. similar to @xcite , @xcite aim to find a linear subspace of the data such that removing that subspace using projection removes bias in the data optimally. doing so necessarily requires a metric to measure how well bias has been removed by a specific projection. @xcite use the classification loss of various classifiers as metrics. that is, a higher classification loss after projection corresponds to better bias removal. this directly translates into a minimax optimization problem, which the authors refer to as a minimax game. the authors present different strategies to solve this optimization problem for their chosen classifiers. a similar strategy was previously described by @xcite . here the corresponding minimax optimization problem is referred to as "maximizing inseparability". @xcite show that this problem can be solved efficiently under certain convexity assumptions on the metric. @xcite independently developed a projection method which is very similar to our mp. their method also uses class means to find the projection direction, but proceeds via a projection to the origin, and hence needs one additional projection. the goal of the paper, however, is quite different, namely the study of human brain reactions to syntactic and semantic features of words. as such, it does not include a systematic comparison to inlp or a comprehensive analysis of the impact on the embedding space. @xcite in recent and unpublished work compute linear projection vectors by minimizing the covariance between biased word vectors and the protected attribute. they generally need two projections to achieve comparable or better performance to inlp. the paper does not report on exactly the same experiments as we do, but for all experiments reported in both papers, our method mp performs similarly or better. @xcite use a different kind of linear transformation on embedding spaces, namely rotation. their goal is to disentangle two particular attributes, such as gender and occupation. to do so, they rotate the embedding space in such as way that the subspaces corresponding to the two attributes become orthogonal. on the positive side, their approach does not remove a dimension and hence arguably retains more information in the embedding. however, it will not remove all bias with respect to gender, but only gender bias with respect to a single specified attribute, e.g., occupation. more generally, it will remove the interaction between the two specified attributes and not actually completely remove an attribute from the embedding. inlp has recently gained importance for probing studies. probing has been criticized because (1) results are difficult to interpret, and (2) it is usually not possible to test whether the target attribute is also relevant for downstream tasks. @xcite propose amnesic probing, which employs inlp to remove the target attribute and then tests change in performance of the adapted embeddings on downstream tasks. inlp has since been used for this purpose in several other studies @xcite @xcite @xcite . when using a projection-based removal method such as inlp or our mp in a probing setup, it is particularly important to ensure that all other information is preserved. if this is not the case, the probing study may lead to misleading conclusions. we hence recommend future such studies to consider using mp instead of inlp. ## projection methods in this section we introduce our two new projection methods mp and tmp. furthermore, we theoretically analyze their relative performance when (linearly) debiasing word embeddings, also in comparison to the existing method inlp. here we focus on binary classification for ease of explanation. both methods can however also be used for multi-class classification (see section 4.1) and then require n -1 projections for n classes. we start with a formal description of the problem we study. our input is a set of word embedding vectors in r d . we interpret these vectors as a set of points @xmath0, . . . , p n } in r d and often use the term "point" when referring to an embedding vector. every point (word) has an associated set of discrete attributes (@xmath1, . . . , a n }). let a * be the attribute we aim to remove. for simplicity, we assume that a * is a binary attribute with values {-1, +1}. we denote the two resulting classes of points as p -= {p @xmath2. we want to find a transformation p ′ of our point set p such that any linear classifier c trained on p ′ to classify a * cannot significantly outperform a classifier that labels by the majority class. that is, p ′ is linearly guarded with respect to a * . in the following we consider only transformations which consist of one or more projections. let p be a point in p . we define its projection p w along a unit vector w as p @xmath3. the projection along the vector w maps points to the hyperplane h w which is orthogonal to w (h w ⊥w) and contains the origin. to evaluate how well inlp, mp, and tmp do in terms of linear guarding, we consider the number of misclassifications with respect to a * by the best possible linear classifier after a single projection; the higher the number of misclassifications, the better the method performs. if a single projection is sufficient to achieve linear guarding, then arguably the semantic encoding of other attributes in our word embeddings are preserved as well as possible. from a theoretical perspective, tmp is the most effective method; it increases the number of misclassifications the most. however, it is costly to compute exactly for data in 300+ dimensions. mp is not as effective in theory as tmp, but it is more effective than inlp. furthermore, mp is very easy to compute and appears to be very effective in practice. hence, we evaluate the efficacy of mp extensively in section 4 and recommend to use mp for linear guarding in practice. ## experiments in this section, we present our main experimental results. 3 we first describe the outcome of the original inlp experiments and how our results compare (section 4.1). we then outline the additional experiments we carried out to gain more insight into surprising results of the original inlp experiments as well as those of our own method (section 4.2). ## conclusion this paper started from the idea that one targeted projection results in similar linear guarding and more stability of the remaining space compared to iterative nullspace projections. we proposed two methods to find such projections: mean projection (mp) and tukey median projection (tmp). we compared performance of mean projection, which is still susceptible to outliers but more efficient to compute than tmp, to the gender based experiments using 35 iterative nullspace projections (inlp) reported in @xcite . our results show that mp obtains comparable linear guarding in one projection where inlp requires multiple iterations. they also confirm that the rest of the space remains more stable through fewer changes in nearest neighbors of randomly chosen words and similarity scores that are closer to the original scores compared to inlp. two results led to further questions: similarity scores improved after applying inlp and inlp resulted in better weat scores than mp. we conducted additional experiments to test whether the extra debiasing iterations performed in inlp end up removing more subtle representations of gender bias that are missed by the single mp iteration. the results show that it is unlikely that inlp can indeed remove more subtle representations of bias for two reasons: (1) we observed that most effects occur once linear guarding has already been achieved. (2) more importantly, similar performance is obtained by running random projections after either mp or the first 8 inlp iterations. we therefore conclude that these additional effects are not related to removing representations of the targeted attribute, but rather a side effect of reducing the dimensionality of the space. this leads us to the overall conclusion that a single targeted projection is indeed preferable over iterative nullspace projections when aiming to remove an attribute. this finding is particular important for the recent line of research that uses inlp for interpretability: these studies test the effect of removing a target attribute and their conclusions should not be confounded by other changes to the space. our theoretical findings in section 3 suggest that mp might produce poor results for skewed data. hence we plan to analyze mp more rigorously on synthetic data, specifically, data where the points are not centered around the mean. in this way we hope to identify scenarios where mp produces poor projections; subsequently we can test if tmp produces better projections for the same scenarios.