Gold Doesn’t Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Information (2023)

Abstract: We describe a simple and effective method (Spectral Attribute removaL; SAL) to remove private or guarded information from neural representations. Our method uses matrix decomposition to project the input representations into directions with reduced covariance with the guarded information rather than maximal covariance as factorization methods normally use. We begin with linear information removal and proceed to generalize our algorithm to the case of nonlinear information removal using kernels. Our experiments demonstrate that our algorithm retains better main task performance after removing the guarded information compared to previous work. In addition, our experiments demonstrate that we need a relatively small amount of guarded attribute data to remove information about these attributes, which lowers the exposure to sensitive data and is more suitable for low-resource scenarios.

## introduction natural language processing (nlp) models currently play a critical role in decision-supporting systems. their predictions are often affected by undesirable biases encoded in real-world data they are trained on. making sensitive predictions based on irrelevant input attributes such as gender, race, or religion (protected or guarded attributes) impacts user trust and the practical broad utility of nlp methods. in recent years, representation learning approaches have become the mainstay of input encoding in nlp. while representation learning has yielded state-of-the-art results in many nlp tasks, controlling or inspecting the information encoded in these representations is hard. thus, using rulebased methods to remove unwanted information from such representations is often not feasible. in the context of protected attributes, @xcite showed that word embeddings trained on the google news corpus encode gender stereotypes. later, @xcite expanded this work and showed that word embeddings trained on the reddit l2 corpus @xcite encode race and religion biases. we propose a simple yet effective technique to remove protected attribute information from neural representations. our method, dubbed sal for spectral attribute removal, applies singular value decomposition (svd) on a covariance matrix between the input representation and the protected attributes and prunes highly co-varying directions. figure 1 demonstrates how professional biography text representations from labeled gender clusters (each biography is marked with the gender of its subject; @xcite for different professions expand after the use of sal, and become closer, implying a higher spread of each profession representations after sal ( §5.2.2). in addition, we overcome the linear removal limitations of sal and previous work by using eigenvalue decomposition of kernel matrices to obtain projections into directions with reduced covariance in the kernel feature space. we refer to this method as ksal (for kernel sal). sal outperforms the recent method of @xcite aimed at solving the same problem and is able to remove guarded information much faster while retaining better performance for the main task. further experiments demonstrate that our method performs well even when the available data for the protected attributes is limited. ## problem formulation and notation for an integer n we denote by [n] the set {1, . . . , n}. for a vector v, we denote by ||v|| 2 its ℓ 2 norm. matrices and vectors are in boldface font (with uppercase or lowercase letters, respectively). random variable vectors are also denoted by boldface uppercase letters. for a matrix a, we denote by a j its jth column (or by a i:j the matrix with columns a k for @xmath0, . . . , j). vectors are assumed to be column vectors. in our problem formulation, we assume three random variables: @xmath1, @xmath2. samples of x are the inputs for a classifier to predict corresponding samples of y. the random vector z represents the guarded attributes. we want to maximize the ability to predict y from x, while minimizing the ability to predict z from x. without loss of generality, we assume that the mean values of x, y and z are 0, and that d ′ ≤ d. 2 we assume n samples of (x, y, z), denoted by (x (i) , y (i) , z (i) ) for @xmath3. these samples are used to train the classifier to predict the target values (y) from the inputs (x). these samples are also used to remove the information from the inputs based on the guarded attributes (z). ## erasing principal directions we describe sal in this section. we explain the use of svd on cross-covariance matrices ( §3.1) and describe the core algorithm in §3.2. ## kernel extension to sal to enrich the type of information that is detected as co-varying, it is possible to use two feature functions, ϕ : r @xmath4: r d ′ → r m ′ , and apply the procedure in §3 on e[ϕ(x)(ψ(z)) ⊤ ]. in that case, we can erase the information from ϕ(x) and treat it as the input for further classification. if the classifier is already learned, it would have to take input vectors of the form ϕ(x), otherwise, it can be re-trained with the erased inputs. ## experiments in our experiments, our main comparison algorithm is the iterative null space projection (inlp) algorithm of @xcite , which aims at solving an equivalent problem to ours. for the word embedding debiasing and fair classification (both setups), we follow the experimental settings of ravfogel et al. (2020). 5 sal provides linear guarding, similarly to inlp, while ksal also captures nonlinear regularities with respect to z (one-hot vector). we can provide such guarding for representations of state-of-the-art encoders (such as bert), provided the representations are eventually fed into a classifier for prediction. the protected attributes we experiment with are gender and race. datasets for debiasing word embeddings ( §5.1), we use 7,500 male and female associated words, 15k words overall. the dataset train/validation/test split sizes are (49%/21%/30%). all the splits are balanced, i.e., containing an equal amount of male and female associated words. for the fair sentiment classification task ( §5.2), we use 10k training examples across all authors' ethnicity ratios (0.5, 0.6, 0.7, and 0.8). all training sets have an equal amount of positive and negative sentiment examples. the test set is balanced for both sentiment and authors' ethnicity labels. for the profession classification task ( §5.2.2), the data train/validation/test split sizes are (65%/10%/25%), and all the splits combined contain 115k samples. ## related work in their influential work, @xcite revealed that word embeddings for many genderneutral terms show a gender bias. @xcite presented a customized training scheme for word embeddings, which minimizes the negative distances between words in the two groups, e.g., male and female related words, for gender debiasing. @xcite demonstrated that bias remains deeply intertwined in word embeddings even after using the above methods. for example, they showed several methods that can accurately predict the gender associated with gender-neutral words, even after applying the methods mentioned above. similar to @xcite , they concluded that removing a small number of intuitively selected gender directions cannot guarantee the elimination of bias. motivated by this conclusion, @xcite presented iterative null space projection (inlp). this debiasing algorithm iteratively projects features into a space where a linear classifier cannot predict the guarded attribute. the debiased representations are linearly guarded, i.e., they cannot guarantee bias removal beyond the linear level. indeed, they show a simple nonlinear classifier can achieve high accuracy when predicting the guarded attribute. their approach is also related to that of @xcite . previous work uses adversarial methods @xcite for information removal @xcite @xcite @xcite with the one by @xcite being related to ours through the use of the mini-max theorem with the squared-error loss on the reconstruction of a matrix similar to our covariance matrix. in addition, methods based on similarity measures between neural representations @xcite were developed. to support the increasing interest in fair classification, @xcite presented an open-source framework for standardizing the evaluation of debiasing methods. finally, most relevant to this paper is an extension of sal to the unaligned case, where protected attributes are not paired with input examples @xcite . ## conclusions we presented a method for removing information from learned representations. we extended our method by using kernels, showing we can provide an effective nonlinear guarding. we also experimented with real-world low-resource situations, in which only a small guarded attribute dataset is provided for information removal.