OSC a R : Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings (2021)

Abstract: Language representations are known to carry stereotypical biases and, as a result, lead to biased predictions in downstream tasks. While existing methods are effective at mitigating biases by linear projection, such methods are too aggressive: they not only remove bias, but also erase valuable information from word embeddings. We develop new measures for evaluating specific information retention that demonstrate the tradeoff between bias removal and information retention. To address this challenge, we propose OSCaR (Orthogonal Subspace Correction and Rectification), a bias-mitigating method that focuses on disentangling biased associations between concepts instead of removing concepts wholesale. Our experiments on gender biases show that OSCaR is a well-balanced approach that ensures that semantic information is retained in the embeddings and bias is also effectively mitigated.

## introduction word embeddings are used extensively across natural language processing (nlp) and succinctly capture not only the syntactic and semantic structure of language, but also word meaning in context. as such, word embeddings are essential building blocks for today's state-of-the-art in nlp. but they are also known to capture a significant amount of stereotypical associations (e.g., @xcite @xcite related to gender, race, nationality, or religion, which can manifest in unwanted and/or potentially harmful ways in downstream tasks @xcite @xcite . such potentially problematic associations, when embedded in word representations, can lead to incorrect and unfair decisions about large groups of people. while the term "bias" has many meanings, in this paper we use it to refer to these unwanted stereotypical associations. existing methods to mitigate these effects either require expensive retraining of vectors @xcite which can be inefficient, or projecting out information contained along an entire subspace representing a protected concept (such as gender or race) in the embedding space (e.g., @xcite @xcite . projective approaches are difficult to control as they are either insufficient: removing a subspace can still leave residual bias @xcite , or too aggressive: in the case of gender, also unnecessarily altering the association between the word pregnant and words like female and mother. in tasks such as coreference resolution, removing such associations could hinder reference resolution. to quantify how much valid information is retained, we look at the output space of models. following @xcite , we use natural language inference (nli) as an effective quantitative probe. here, we construct the hypothesis by making minimal edits to a premise, and observe model prediction conditioned on such changes. for example, premise: a matriarch sold a watch. hypothesis: a woman sold a watch. here, the objective is to determine if the hypothesis is entailed by the premise, contradicted by it, or neither (neutral to it). a glove-based nli @xcite model, without any explicit form of bias mitigation, predicts label entail with a high probability of 97%; the notion of a matriarch being a woman is correctly identified by the model. however, after projective debiasing, the model classifies the pair as neutral with a probability 62% while the probability of the label entail drops to much lower at 16%. that is, aggressive mitigation of gender representations erases valid gender associations. ideally, we should correct problematic associations without erasing valid ones. to this end, we propose oscar (othogonal subspace correction and rectification) which orthogonalizes and rectifies identified subspaces of concepts that are incorrectly associated in an embedding space. embeddings outside the subspace are stretched in a graded manner or untouched. our contributions are: our contributions, focusing specifically on representations rather than classifiers, are important given the preponderance of distributed representations of text across nlp. predictions from systems that use such representations, if unchecked, could lead to real-world decisions (involving, e.g. hiring) that "systematically and unfairly discriminate against certain individuals or groups of individuals in favor of others" @xcite . in an effort to prevent such transition of representational harms into allocational harms @xcite @xcite , we look at mitigating stereotyping biases at the source, i.e. the embedding space. ## orthogonal subspace correction and rectification (oscar) we describe a new geometric operation, an alternative to the linear projection-based ones in the literature. this operator applies a graded rotation on the embedding space; it rectifies two identified directions (e.g., male-vs-female and occupations) which should ideally be independent of each other, so they become orthogonal, and remaining parts of their span are rotated at a varying rate so the operation is sub-differentiable. given d-dimensional embeddings, oscar requires us to first identify two subspaces, denoted v 1 and v 2 , for which we desire the embedding space to not have interdependence. we will use subspaces representing male-vs-female (v 1 ) and occupation (v 2 ) as running examples, following previous work @xcite @xcite . specifically we identify 1-dimensional vectors v 1 ∈ v 1 and v 2 ∈ v 2 best capturing these subspaces from word lists; those used for male-vs-female and occupation are listed in the supplement. we ensure v 1 = v 2 , and normalize so v 1 = v 2 = 1. we restrict the representation adjustment to the subspace @xmath0, v 2 ) that is defined by, and spans the male-vs-female and occupation directions. in particular, we can identify an orthogonal basis for s using vectors v 1 and the v 1 and v 2 vectors are orthogonal). we then restrict any word vector @xmath1.g., job) to s as two coordinates π s (x) = ( v 1 , x , v 2 , x ). we will adjust only these coordinates, and leave all d-2 other orthogonal components fixed. we now restrict our attention to within the subspace s. in detail, we do this by defining a d × d rotation matrix u . the first two rows are v 1 and v 2 . the next d-2 rows are any set u 3 , . . . , u d which complete the orthogonal basis. we then rotate all data vectors x by u (as u x). then we manipulate the first 2 coordinates (x 1 , x 2 ) to f (x 1 , x 2 ), described below, and then reattach the last d-2 coordinates, and rotate the space back by u t . next we devise the function f which is applied to each word vector @xmath2. see illustration in figure 1 . given male-vs-female v 1 and occupation v 2 subspaces let θ = arccos( v 1 , v 2 ) be the angle which captures their correlation, and θ = π 2 -θ . now define a 2 × 2 rotation matrix which would rotate v 2 to v 2 (orthogonal to v 1 ). @xmath3. however, the function f should be the identity male-vs-female o c c u p a t io n occupation -corrected the occupation x male-vs-female subspace e.g., "job" and for any other word vector x, it should provide a smooth partial application of this rotation so f is continuous. in particular, for each data point @xmath4, we will determine an angle θ x and apply a rotation matrix towards defining θ x (how much to rotate a word vector x on this plane), we calculate two other measurements φ = arccos v 1 , x x and d 2 = v 2 , x this effectively orthogonalizes the components of all points exactly along the male-vs-female v 1 and occupation v 2 directions. so, points lying in subspaces especially near v 2 get moved the most, those near v 1 are moved little, and the rest of the embedding space faces a graded rotation. the information contained outside the occupation×malevs-female subspace s remains the same, thus preserving most of the inherent structure and content of the original embedding representation. we have an almost everywhere differentiable operation applied onto all vectors in the space, enabling us to extend this method to contextualized embeddings. it can now be part of the model specification and integrated with the gradient-based finetuning step. further, it is a post processing step applied onto an embedding space, thus its computational cost is relatively low, and it is easily malleable for a given task for which specific subspaces may be desired as independent. ## experimental methodology debiasing methods. gender association with other concepts and its reduction has been observed on glove embeddings @xcite @xcite @xcite . however, they are mostly projection-based, and begin by identifying a subspace represented as a vector v. for fair comparison across all these methods, we determine v with embeddings of he and she. some methods require an auxillary set of definitionally male/female words g are treated separately (supplement d.3 and d.4). linear projection (lp): we consider this as the simplest operation. for every word vector w, we project it onto an identified direction v to remove the respective component: afterwards, the w vector (of d-dim) lies in a (d-1)dim subspace, but still retains d coordinates; the subspace v is removed. @xcite showed such simple projection reduces the most bias and has the least residual bias. hard debiasing (hd): the original debiasing method, by @xcite , begins with the same projection operation as above, but uses g to only apply it to a subset of all words. first, it trains a linear svm to separate g from random words. all words labeled as part of g, called g , are also assumed definitionally male-vs-female, and not adjusted by projection. the exception is another subset g ⊂ g (see supplement d.3) which come in pairs (e.g., man-woman). these words are projected along v, but then are put through another operation called equalize. this ensures that after projection, each pair is the same distance apart as they were before projection, but entirely not within the subspace defined by v. as we will observe (similar to @xcite ), equalization and the set g retains certain male-vs-female information in the embedding (compared to projection), but has trouble generalizing when used on words that may carry stereotypical male-vs-female connotations outside of g (such as proper names). ## experimental results intrinsic measurement of bias. the first column uses sets @xmath5, the same words used to define the mitigation direction. this is the only experiment where the test/train split is not maintained for oscar (and other methods). the second column uses two sets of 59 definitionally male and female words as a and b, and the third uses 700 statistically male and female names for each set a and b (both in supplement d.7). the baseline row (row 1, unmodified glove) shows these methods provide similar weat * scores. the lp method retains the least information for tests based on he-she or other male-vs-female words. these scores are followed closely behind by the scores for inlp in these tasks. the single projection of lp retains more info when compared against statistically male and female names, whereas inlp's multiple projections appear to remove more of this information, since only some extrinsic metric of gendered information preserved. for the new sirt task we observe the performance using glove and roberta in ## discussion & conclusion the propagation of undesirable and stereotypical associations learned from data into decisions made by language models maintains a vicious cycle of biases. combating biases before deploying representations is thus extremely vital. but this poses its own challenges. word embeddings capture a lot of information implicitly in relatively few dimensions. these implicit associations are what makes them state-of-the-art at tackling different language modeling tasks. breaking down these associations for bias rectification, thus, has to be done carefully so as to not destroy the structure of the embeddings. oscar's rectification of associations helps integrate both these aspects, allowing it to be more suitable at making word embeddings more usable. moreover, being computationally lightweight and sub-differentiable, it is simple to apply adaptively without extensive retraining. oscar dissociates concepts which may be otherwise lost or overlapped in distributed representations by re-orthogonalizing them. we envision that this method can be used for many types of unwanted associations (age-ability, religion-virtue perception, etc). since it only decouples specific associations, informative components of these features will remain unchanged. all the experiments in this paper are based on english embeddings, models and tests; extending our analysis to other languages is important future work. further, we be-lieve oscar can extend in a straightforward way beyond nlp to other distributed, vectorized representations (e.g., of images, graphs, or spatial data) which can also exhibit stereotypical associations.