Venue: NAACL 2024

# Debiasing with Sufficient Projection: A General Theoretical Framework for Vector Representations

Enze Shi, Lei Ding, Linglong Kong , Bei Jiang

Corresponding author

###### Abstract

Pre-trained vector representations in natural language processing often inadvertently encode undesirable social biases. Identifying and removing unwanted biased information from vector representation is an evolving and significant challenge. Our study uniquely addresses this issue from the perspective of statistical independence, proposing a framework for reducing bias by transforming vector representations to an unbiased subspace using sufficient projection. The key to our framework lies in its generality: it adeptly mitigates bias across both debiasing and fairness tasks, and across various vector representation types, including word embeddings and output representations of transformer models. Importantly, we establish the connection between debiasing and fairness, offering theoretical guarantees and elucidating our algorithm's efficacy. Through extensive evaluation of intrinsic and extrinsic metrics, our method achieves superior performance in bias reduction while maintaining high task performance, and offers superior computational efficiency.

Department of Mathematical and Statistical Sciences

University of Alberta

{eshi, lding1, lkong, bei1}@ualberta.ca

+
Footnote †: _Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 5960–5975

June 16-21, 2024 02024 Association for Computational Linguistics

## 1 Introduction

Natural Language Processing (NLP) models have made significant strides in recent years, with much of their success attributed to representation learning - the process of creating effective vector representations for textual data. Various research has been conducted in this area, including static word embedding Mikolov et al. (2013); Pennington et al. (2014), contextualized embedding Peters et al. (2018); Devlin et al. (2018); Radford et al. (2019), sentence embedding Reimers and Gurevych (2019) in addition to other representation methods.

However, as vector representations have been applied in a wide range of real-life scenarios, researchers have discovered that stereotypical biases and spurious correlations can be transferred from human-generated corpora to vector representations and models Bolukbasi et al. (2016); Caliskan et al. (2017); Vig et al. (2020). This has the potential to produce biased and unfair outcomes in various downstream tasks Kurita et al. (2019) and can even lead to serious social problems. For instance, in the word analogy task presented in Bolukbasi et al. (2016), it was found that the vector representation for \(\overrightarrow{she}\) was closer to \(\overrightarrow{nurse}\) than the representation for \(\overrightarrow{he}\) was to \(\overrightarrow{doctor}\)De-Arteaga et al. (2019) found a performance gap between different genders in text classification tasks.

The bias and fairness issues in NLP models are primarily caused by the unbalanced and stereotypical nature of the training corpora. Liang et al. (2020) described this as unbalanced model behavior in relation to certain socially sensitive topics such as gender, race, and religion. To quantify biases in NLP, two types of bias evaluation metrics(intrinsic and extrinsic) have been proposed. However, recent research has shown that in most cases, there is a weak correlation between them Goldfarb-Tarrant et al. (2021); Kaneko et al. (2022); Cao et al. (2022). There remains a significant research gap in understanding how to bridge these two kinds of tasks. In our research, we employ statistical independence to establish a theoretical linkage between these tasks, offering insights into the interplay between intrinsic and extrinsic biases.

Various methods have been proposed for reducing bias in NLP, but it remains a challenge to effectively mitigate bias while maintaining high model performance. Furthermore, it is particularly difficult for debiasing methods to efficiently address both intrinsic and extrinsic biases at the same time, as discussed in the related works section.

In this paper, we propose a general debiasing method that can effectively mitigate bias across both debiasing and fairness tasks. Our key contributions include the following:

* We are the first to scope the debiasing and fairness tasks unitedly through statistical independence, providing a detailed theoretical analysis aimed at identifying the minimal subspace that contains bias information. This analysis enables us to determine the optimal projection matrix for these tasks and bridge the connection between them.
* Our algorithm showcases its effectiveness in both intrinsic (bias embedding evaluation) and extrinsic (fairness text classification) evaluation metrics. It is versatile, adapting to different embedding methods and sensitive variable types, making it universally applicable.
* Our method improves upon existing state-of-the-art methods while still maintaining good task performance compared to the original model and stands out due to its superior computational efficiency.

The structure of this paper is as follows. We begin with a comprehensive review of existing research on bias evaluation and debiasing techniques in NLP. We then introduce our methodology, including our proposed algorithm. We present experimental results on a range of gender bias evaluation tasks, showcasing the effectiveness of our approach. Finally, we provide a theoretical bridge and guarantee, and a discussion of our method.

## 2 Related Works

Debiasing Methods in NLPER Researchers have been focusing on reducing bias from each component of NLP models. The most intuitive idea of debiasing is through counterfactual data augmentations Zmigrod et al. (2019); Dinan et al. (2020); Barikeri et al. (2021), which involves re-balancing a corpus by swapping bias attribute words (e.g., he/she) in a dataset. The re-balanced corpus is then used for further training to debias a model. While this approach is simple and can be applied to all tasks, it does not perform well in terms of debiasing and requires additional computational resources for model re-training. Another direction is fine-tuning pre-trained transformer-based language models using methods such as projection Kaneko and Bollegala (2021), adversarial Han et al. (2021), contrastive Cheng et al. (2020); Shen et al. (2021); He et al. (2022), dropout Webster et al. (2020), and prompting Schick et al. (2021); Guo et al. (2022). These methods show effectiveness in reducing bias in various intrinsic evaluation tasks. However, when deploying these debiased models to downstream tasks, especially fine-tuning on task-specific datasets, the debiased language model can still re-learn social bias, making these debiasing methods less effective.

Our proposed debiasing method is based on the controlled removal of specific information from vector representations, which is closely related to the task of disentangling representations Bengio et al. (2013). Previous research in this area includes methods for removing bias from static embeddings, such as projecting the word embedding into the orthogonal space of the gender direction (\(\overrightarrow{he}\) - \(\overrightarrow{she}\)Bolukbasi et al. (2016), Re-training the entire embedding using some loss functions Kaneko and Bollegala (2019), and utilizing the ideas in causal inference Yang and Feng (2020); Ding et al. (2022). There are several similar projection-based methods like Iterative Nullspace Linear Projection (INLP; Ravfogel et al. (2020)), RLACE Ravfogel et al. (2022). We discuss the detailed comparison and advantages of our method in Appendix A.2.

Evaluating Bias in NLPThe measurement methods for evaluating bias in pre-trained word embeddings and language models can be broadly divided into two categories: Intrinsic and Extrinsic evaluations. Intrinsic bias evaluations probe the bias within pre-trained word embeddings and language models. Common methods include measuring the geometry in embedding space, such as the Word Embedding Association Test (WEAT; Caliskan et al. (2017)) and Sentence Encoder Association Test (SEAT; May et al. (2019)). Additionally, Kurita et al. (2019); Nangia et al. (2020); Nadeem et al. (2021) propose metrics using the likelihood score. Furthermore, research suggests that some debiasing methods may only hide bias, and thus additional measurement approaches are needed Gonen and Goldberg (2019).

The extrinsic bias is specific to certain downstream tasks. In the text classification task, DeArteaga et al. (2019); Blodgett et al. (2016) proposed two benchmark datasets and used the equal opportunity measure from fairness literature. Zhao et al. (2018) proposed the WinoBias benchmark for Coreference resolution. As well as other benchmarks, such as Bias-NLI Dev et al. (2020) and in machine translation Stanovsky et al. (2019). However, recent research has indicated that intrin 

sic bias in embeddings or models typically does not have a strong correlation with bias in downstream tasksGoldfarb-Tarrant et al. (2021); Cao et al. (2022). Kaneko et al. (2022) found out that the debiased models re-learn the bias from the fine-tuning datasets, showing that only debiasing upstream models may not be enough to eliminate bias in downstream tasks.

In our work, we conduct comprehensive evaluation experiments in both intrinsic and extrinsic tasks. Additionally, our approach avoids the issue of re-learning bias by directly addressing the vector representation in downstream tasks.

## 3 Methodology

### Problem Setup

We consider the problem of removing sensitive information inside the vector representation. Given the representation vector \({\mathbf{X}}\in\mathbb{R}^{p_{1}}\) accompanied with the target attribute \({\mathbf{Y}}\in\mathbb{R}^{p_{2}}\) and the sensitive attribute \({\mathbf{Z}}\in\mathbb{R}^{p_{3}}\), our goal is to find a map \(g:\mathbb{R}^{p_{1}}\mapsto\mathbb{R}^{p_{1}}\) such that:

* \(g({\mathbf{X}})\) is uncorrelated with \({\mathbf{Z}}\);
* \(g({\mathbf{X}})\) maintains ability to predict \({\mathbf{Y}}\).

In other words, the new representation \({\mathbf{X}}=g({\mathbf{X}})\) removes the sensitive information \({\mathbf{Z}}\) contained in the original representation while preserving other useful information in \({\mathbf{X}}\). The notations given above incorporate debiasing and fairness tasks into the same framework, which are formulated in the following definitions: Let \(\widetilde{{\mathbf{X}}}=g({\mathbf{X}})\) and \(f_{1}\) be the mechanism with input \(\widetilde{{\mathbf{X}}}\). Then \(\widetilde{{\mathbf{X}}}\) is said to be a debiased representation if \(f_{1}(\widetilde{{\mathbf{X}}})\perp\!\!\!\perp{\mathbf{Z}}\). Let \(\widetilde{{\mathbf{X}}}=g({\mathbf{X}})\) and \(f_{2}\) be the mechanism trained by \((\widetilde{{\mathbf{X}}},{\mathbf{Y}})\) to predict \({\mathbf{Y}}\). Then \(\widetilde{{\mathbf{X}}}\) is said to be a fair representation if \(f_{2}(\widetilde{{\mathbf{X}}})\perp\!\!\!\perp{\mathbf{Z}}\mid{\mathbf{Y}}\).

Our formulated definition directly aligns with the objectives of each task. In the debiasing task, given the input \({\mathbf{X}}\), the mechanism \(f_{1}\) is biased if the output \(f_{1}({\mathbf{X}})\) relies heavily on \({\mathbf{Z}}\). Therefore, the goal is to make \({\mathbf{Z}}\) and the output of the mechanism \(f_{1}({\mathbf{X}})\) to be **independent** given the representation \({\mathbf{X}}\). Similarly, for fairness tasks, the aim is to develop a fair mechanism \(f_{2}\), ensuring its prediction is independent of sensitive information \({\mathbf{Z}}\) conditioned on \(Y\) (**Conditional Independence**) Hardt et al. (2016). Note that \(f_{1}\) or \(f_{2}\) represent a specific mechanism with input \(\tilde{X}\), which differs in different tasks. For instance, in the WEAT and SEAT tasks, \(f_{1}\) denotes the function \(s(\widetilde{X},A,B)\), which is a measure of association strength. In the Fairness Text Classification, \(f_{2}\) is specifically used to denote the classifier for target label \(Y\).

### Motivation

As previously discussed, our goal is to identify a mapping \(g\) such that \(\widetilde{{\mathbf{X}}}=g({\mathbf{X}})\) possesses the desired properties. A direct approach involves constructing a \(p_{1}\times p_{1}\) projection matrix \(P\) and applying the linear transformation \(g({\mathbf{X}})=P{\mathbf{X}}\). When we restrict the transformation \(g\) into linear projections, the original vector \({\mathbf{X}}\) can be decomposed into \({\mathbf{X}}=P{\mathbf{X}}+(I-P){\mathbf{X}}\), where \(I\) is the \(p_{1}\times p_{1}\) identity matrix. Letting \(\mathcal{S}_{1}\) and \(\mathcal{S}_{2}\) represent the spaces spanned by \(P\) and \(I-P\) respectively, the representation space is then decomposed as \(\mathbb{R}^{p_{1}}=\mathcal{S}_{1}\bigoplus\mathcal{S}_{2}\). For a debiased representation \(\widetilde{{\mathbf{X}}}=P{\mathbf{X}}\), \(\mathcal{S}_{1}\) should be structured to minimize the information regarding \({\mathbf{Z}}\), while \(\mathcal{S}_{2}\) should encapsulate as much of the information regarding \({\mathbf{Z}}\) as possible.

Consequently, the debiasing goal transforms into the identification of the subspaces \(\mathcal{S}_{1}\) and \(\mathcal{S}_{2}\). It's crucial to note that the majority of information regarding \({\mathbf{Z}}\) resides within \(\mathcal{S}_{2}\). Therefore, the expression \((I-P){\mathbf{X}}\) emerges as a potential predictor for \({\mathbf{Z}}\). Throughout this paper, we adhere to the following linearity assumption, Weak Linearity Suppose \(Q=I-P\) has rank \(q\) and orthogonal basis \((\beta_{1},\ldots,\beta_{q})\), with each \(\beta_{j}\) belonging to \(\mathbb{R}^{p_{1}}\). Then we assume \(Z\) can be modeled by projecting \(X\) onto \(q\) distinct directions:

\[Z=f(\beta_{1}^{\top}X,\dots,\beta_{q}^{\top}X,\varepsilon), \tag{1}\]

where \(f\) is an unknown function, which can be linear or nonlinear, and \(\varepsilon\) denotes the random effect. Contrary to the strong linearity assumption in other projection-based methods such as Ravfogel et al. (2020), which postulates that \({\bf Z}\) can be expressed as \({\bf Z}=W{\bf X}\) for \(W\in\mathbb{R}^{p_{3}\times p_{1}}\), our approach adopts a weak linearity assumption. This assumption introduces an unknown function \(f\), which can be either linear or nonlinear, thereby including the traditional strong linearity assumption. A toy example regrading weak and strong linearity can be found in Appendix A.1.

Based on the discussion above, the primary objective of this study is articulated as follows: 

initially, to identify the direction matrix \(B=(\beta_{1},\ldots,\beta_{q})\in\mathbb{R}^{p_{1}\times q}\) ensuring that \(B^{\top}\mathbf{X}\) captures most of the information about \(\mathbf{Z}\). Then given this direction matrix, we can obtain a debiased representation of \(\mathbf{X}\) by projecting it onto the subspace orthogonal to \(\mathrm{Span}\{\mathrm{B}\}\). Note that \(\mathrm{Span}\{\mathrm{B}\}\) is equivalent to \(\mathrm{Span}\{\mathrm{Q}\}=\mathcal{S}_{2}\), and hence, its corresponding orthogonal subspace \(\mathcal{S}_{1}\) is spanned by \(P\) as preliminarily defined. This approach underpins the theoretical foundation of our proposed algorithm, details of which will be explored in the forthcoming sections.

### Minimal Subspace

It is crucial that we want the subspace \(\mathcal{S}_{2}\) with the desired property as **small** as possible so that we can retain the utility of \(\mathbf{X}\) after projecting it on \(\mathcal{S}_{1}\). Essentially, we want to find the matrix \(Q\) with minimum rank \(q\). Consider the random variables \(\mathbf{X}\in\mathbb{R}^{p_{1}}\) and \(\mathbf{Z}\in\mathbb{R}^{p_{3}}\). If there exists a full rank matrix \(B\in\mathbb{R}^{p_{1}\times q}\), such that \(\mathbf{Z}\mbox{$\perp\!\!\!\perp$}\mathbf{X}~{}|~{}B^{\top}\mathbf{X}~{}(\mathbf{X}\) is independent of \(\mathbf{Z}\) conditioned on its projections on \(B\)), then the column space of the matrix \(B\), denoted as \(\mathrm{Span}\{B\}\), is called a sufficient dimension reduction subspace of \(\mathbf{Z}\) with respect to \(\mathbf{X}\). The intersection of all the dimension reduction subspaces is called the central subspace and denoted as \(\mathcal{S}_{\mathbf{Z}|\mathbf{X}}\). That is \(\mathcal{S}_{\mathbf{Z}|\mathbf{X}}=\bigcap_{B\in\mathcal{B}_{XZ}}\mathrm{Span}\{B\}\), where

The dimension of the central subspace is denoted as \(\dim(\mathcal{S}_{\mathbf{Z}|\mathbf{X}})\). When \(\mathrm{Span}\{B\}\) is the central subspace, we have \(\dim(\mathcal{S}_{\mathbf{Z}|\mathbf{X}})=\dim(\mathrm{Span}\{B\})=q\). See Cook and Li (2002) for more details. A toy example of minimal subspace can be found in Appendix A.1.

If \(\mathbf{Z}\mbox{$\perp\!\!\!\perp$}\mathbf{X}~{}|~{}B^{\top}\mathbf{X}\), then \(B^{\top}\mathbf{X}\) are most useful to predict \(\mathbf{Z}\) based on \(\mathbf{X}\), which is exactly the case in Model (1). Therefore, the central subspace \(\mathcal{S}_{\mathbf{Z}|\mathbf{X}}\) is the minimal subspace processing the desired property and serves as a promising candidate for the expected subspace \(\mathcal{S}_{2}\). We will illustrate the estimation procedure in the next section, which is robust to any kind of mapping \(f\) in Model (1) and only relies on the data set \(\{(X_{i},Z_{i})\}_{i=1}^{n}\).

Note that Model (1) is an ideal case based on the linearity assumption. In real-world applications, using only \(q\) directions might not be sufficient to cover all the information of \(\mathbf{Z}\) because there might be nonlinear correlations between \(\mathbf{X}\) and \(\mathbf{Z}\). However, since the \(q\) directions can cover most of the information, it is still safe to use Model (1) in practice. Specifically, in NLP tasks, we may assume that the sensitive attribute \(\mathbf{Z}\) can be predicted by the projections of the representation \(\mathbf{X}\) onto \(q\) directions with some unknown mapping \(f\).

## 4 Sufficient Universal Projection (SUP)

### Subspace Estimation

In this subsection, we will explain the process of estimating the subspace \(\mathcal{S}_{2}=\mathcal{S}_{\mathbf{Z}|\mathbf{X}}\) in different scenarios. We will begin with the simplest case where \(p_{3}=1\), meaning \(\mathbf{Z}\in\mathbb{R}\) is a scalar. Sliced Inverse Regression (SIR) is a classical dimension reduction method proposed by Li (1991) for univariate response \(\mathbf{Z}\). We provide the detailed scheme for SIR applied on the data set \(\{(X_{i},Z_{i})\}_{i=1}^{n}\) in the Appendix A.3. The main procedure of SIR is: (1) divide the support of \(\mathbf{Z}\) into \(H\) intervals and calculate the covariance matrix of \(\mathbf{X}\) for each interval, (2) calculate the weighted covariance matrix based on \(H\) intervals, (3) obtain the directions from the weighted covariance matrix. The space spanned by the directions \(B=(\beta_{1},\ldots,\beta_{q})\) provided by SIR is a consistent estimation of \(\mathcal{S}_{\mathbf{Z}|\mathbf{X}}\).

For multivariate sensitive attributes, a direct analogy of the slicing strategy in SIR no longer works, as the number of partitions of the support of \(\mathbf{Z}=(\mathbf{Z}_{1},\ldots,\mathbf{Z}_{p_{3}})\in\mathbb{R}^{p_{3}}\) becomes \(H^{p_{3}}\) and thus suffers the curse of dimensionality. To address the limitation of the original SIR, the Pooled Marginal Slicing (PMS) estimator proposed in Aragon (1997) combines the subspaces \(\mathcal{S}_{\mathbf{Z}_{i}|\mathbf{X}}\) estimated by univariate response SIR to get the directions for the multivariate response, which is motivated by the following proposition.

**Proposition 4.1**.: _Note that \(\mathbf{Z}\mbox{$\perp\!\!\!\perp$}\mathbf{X}~{}|~{}B^{\top}\mathbf{X}\) implies \(\mathbf{Z}_{j}\mbox{$\perp\!\!\!\perp$}\mathbf{X}~{}|~{}B^{\top}\mathbf{X}\). Therefore, for \(j=1,\dots,p_{3}\), \(\mathcal{S}_{\mathbf{Z}_{j}|\mathbf{X}}\subseteq\mathcal{S}_{\mathbf{Z}|\mathbf{X}}\)._

Proof.: For any \(B\in\mathcal{B}_{XZ}\), we have \(B\in\mathcal{B}_{XZ_{j}}\), thus \(\mathcal{B}_{XZ}\subset\mathcal{B}_{XZ_{j}}\). Recall that \(\mathcal{S}_{\mathbf{Z}|\mathbf{X}}\) is the intersection of all elements in \(\mathcal{B}_{XZ}\). Therefore, we have \(\mathcal{S}_{\mathbf{Z}_{j}|\mathbf{X}}\subseteq\mathcal{S}_{\mathbf{Z}|\mathbf{X}}\) for \(j=1,\dots,p_{3}\). 

Proposition 4.1 indicates that \(\mathcal{S}_{\mathbf{Z}_{j}|\mathbf{X}}\) can be used to recover \(\mathcal{S}_{\mathbf{Z}|\mathbf{X}}\), which guarantees the theoretical properties of PMS estimator. It also naturally lifts the curse of dimensionality. Let \(Z_{ij}\) denote the \(j\)-th coordinate of \(i\)-th sample. We apply SIR to data set \(\{(X_{i},Z_{ij})\}_{i=1}^{n}\) and obtain the estimators \(M_{i}^{\text{SIR}}\) for \(j=1,\dots,p_{3}\). Then we define the weighted sum of estimators as 

\(w_{i}\) can be chosen as either equal weights or proportional to the leading eigenvalues of \(M_{i}\). Then the leading \(q\) eigenvectors \(\psi_{1},\ldots,\psi_{q}\) of \(M^{\mathrm{PMS}}\) can be used to recover \(\mathcal{S}_{\mathbf{Z}|\mathbf{X}}\). The detailed implementation of obtaining \(M^{\mathrm{PMS}}\) is summarized in Algorithm 2 in the Appendix.

### Algorithm Implementation

To obtain the debiased representation, we first collect the original vector representation \(X_{i}\) and the associated sensitive attribute \(Z_{i}\). Note that \(Z_{i}\) can be labeled by humans or learned from the training data. Specifically, we place no restrictions on the structure of \(Z_{i}\) - it can be either discrete labels \(Z_{i}\in\{1,2,\dots,k\}\) representing gender or race, or a continuous variable. When \(Z_{i}\) is a continuous variable, we directly set \(\widetilde{Z}_{i}=Z_{i}\) and handle \(\{(X_{i},\widetilde{Z}_{i})\}\) as discussed above. If \(Z_{i}\) is a categorical variable with choices \(\{1,2,\dots,k\}\), we first train a classifier \(f_{\rm cls}\) based on the data set \(\{(X_{i},Z_{i})\}_{i=1}^{n}\), whose output is the probability of \(X_{i}\) belonging to each category, then denote

\[\widetilde{Z}_{i}=f_{\rm cls}(X_{i})=(\widetilde{Z}_{i1},\dots,\widetilde{Z}_ {ik})\in\mathbb{R}^{k},\]

where \(\sum_{j=1}^{k}\widetilde{Z}_{ij}=1\). In both scenarios, the attribute \(Z_{i}\) is converted to the vector variable with continuous support. Then we can obtain the PMS estimator \(M^{\mathrm{PMS}}\) based on \(\{(X_{i},\widetilde{Z}_{i})\}_{i=1}^{n}\) with its leading \(q\) eigenvectors \(\psi_{1},\ldots,\psi_{q}\). The projection matrix is defined as \(P=I-\sum_{i=1}^{q}\psi_{i}\psi_{i}^{\top}\). Intuitively, \(Q=\sum_{i=1}^{q}\psi_{i}\psi_{i}^{\top}\) is the estimated central mean space regarding \(Z\), and the space spanned by this matrix contains most of the information we want to eliminate. The procedure is outlined in Algorithm 1 and in Figure 1.

```
0: Data \(\{(X_{i},Z_{i})\}_{i=1}^{n}\), partition \(H\) and number of dimension \(q\);
0: Sufficient projection \(P\);
1:if\(Z_{i}\) is continuous then
2: Set \(\widetilde{Z}_{i}=Z_{i}\);
3:else if\(Z_{i}\) is discretethen
4: Train a classifier \(f_{\text{cls}}\) by \(\{(X_{i},Z_{i})\}_{i=1}^{n}\);
5: Set \(\widetilde{Z}_{i}=f_{\text{cls}}(X_{i})\);
6:endif
7: Obtain \(M^{\mathrm{PMS}}\) using \(\{(X_{i},\widetilde{Z}_{i})\}_{i=1}^{n}\) by Algorithm 2;
8: Calculate the leading eigenvectors \(\{\psi_{j}\}_{j=1}^{q}\) of \(M^{\mathrm{PMS}}\);
9: Obtain \(P=I_{p_{1}}-\sum_{j=1}^{q}\psi_{j}\psi_{j}^{\top}\);
10:Return:\(P\).
```

**Algorithm 1** Sufficient Universal Projection (SUP)

It is worth emphasizing that SUP is a general framework for bias elimination, and we have no assumption on the type of representation. Therefore, our proposed method is universally robust to both static and contextualized embeddings with different dimensions and can be applied to any downstream tasks.

### Connection with Existing Method

Assumption 3.4 guarantees that our statistical model encompasses a wider family of models, indicating that our method relies on relatively less stringent conditions. In this subsection, we provide a brief comparison of our method to other projection-based methods.

**INLP**(Rawfogel et al., 2020): Both our method SUP and INLP employ linear projections to minimize the influence of the sensitive attribute \(\mathbf{Z}\) in the representations. INLP seeks to identify the null space of the weight matrix \(W\) of linear classifier \(\mathbf{Z}=f(W\mathbf{X})\). This framework can be viewed as a specific instance of the Model 1

**RLACE**(Rawfogel et al., 2022a): Both SUP and RLACE operate under weak linearity assumption as expressed in Assumption 3.4. In RLACE, the function \(f\) in Model (1) is interpreted as the inverse of a link function in the generalized linear model. In contrast, our approach imposes no specific constraints on the form of \(f\).

A detailed analysis and comparison with existing methods are available in Appendix A.2.

## 5 Experiment and Settings

In all experiments conducted, the number of partitions was consistently set to 100. The optimal number of dimensions to be removed was determined through cross-validation. Detailed insights into the impact of the parameter \(q\) on performance

Figure 1: The flowchart of Sufficient Universal Projection algorithm.

 

can be found in Appendix A.8. Codes are available at the GitHub.1

Footnote 1: [https://github.com/EnzeShi/Debiasing-with-Sufficient-Projection-A-General-Theoretical-Framework-for-Vector-Representations](https://github.com/EnzeShi/Debiasing-with-Sufficient-Projection-A-General-Theoretical-Framework-for-Vector-Representations)

### Static Word Embedding Evaluation Tasks

We begin by demonstrating our method in the context of debiaising static word embeddings using 300-dimensional GloVe embeddings Pennington et al. (2014) pre-trained on English Wikipedia data. We first calculate the cosine similarity between each word embedding with the gender direction \(\overrightarrow{he}\) - \(\overrightarrow{she}\). From these scores, we select words in two categories: those with the top \(9000\) highest scores, representing male-associated words, and those with the top \(9000\) lowest scores, representing female-associated words. The label of each class is the sensitive attribute \(Z\) and the projection matrix \(P\) is calculated through Algorithm 1. For a fair comparison, the following evaluations are based on the methodology outlined in Gonen and Goldberg (2019); Ding et al. (2022). We compare our results with the following baseline methods: harddebiasing method Hard Bolukbasi et al. (2016), gender-preserving depiasing method GP Kaneko and Bollegala (2019), word vector learning method GN Zhao et al. (2018), half-sibling regression HSR Yang and Feng (2020), INLP Ravfogel et al. (2020) and DeSIP Ding et al. (2022).

Clustering Gender Biased Words.Biased words tend to cluster together, and debiased embeddings may not escape this phenomenon. We use K-means clustering (K=2) to split the top 500 male-biased and top 500 female-biased words. A visualization graph is presented in Appendix 5.2. In Table 1 column one, we report the accuracy in splitting the 1,000 words into male and female clusters. Our method brings about a 50% reduction compared with the original GloVe and about 20% compared with the second-best method.

CorrelationUsing the top 50,000 most frequent words as targets, we calculate the Pearson correlation coefficient between the bias-by-projection and bias-by-neighbor results. The latter is calculated using the neighborhood metric, which counts the percentage of male and female-biased words within the 100 nearest neighbors of each target word. The result is presented in the second column of Table 1, and we achieve the lowest correlation coefficient.

Profession WordsIn this task, we determine the correlation between the original bias and the number of male neighbors among the 100 nearest neighbors of profession words, as listed by Bolukbasi et al. (2016); Zhao et al. (2018). The correlation coefficient is shown in Table 1. Our method reduces the coefficient by 20% compared with the original GloVe and achieves the best result.

ClassifcationWe selected the top 2,500 biased words for each gender and trained a support vector machine (SVM) model using 1,000 randomly sampled words for each baseline model. We then applied the trained classifier to the remaining 4,000 words to predict gender bias direction. The prediction accuracy is shown in Table 1. Lower accuracy implies that the original embedding does not contain enough gender-related information. Our method has the least accuracy among all debiasing methods, indicating that it preserves the least gender bias.

Word Embedding Association TestThe WEAT Caliskan et al. (2017) is a permutation-based test that measures bias in word embeddings. Refer to Appendix A.5 for the details of WEAT. The results are reported in terms of absolute effect sizes(\(d\)) and \(p\)-values (\(p\)). The effect size is a normalized measure of how separated two distributions are. A high effect size indicates a larger bias between the target and attribute words, and the \(p\)-value denotes whether the bias is statistically significant or not. We conducted two experiments employing the Pleasant & Unpleasant (Task 1) and Career & Family (Task 2) word sets, with male and female names serving as the attribute sets. As illustrated in Table 1, the outcomes from both tasks reveal that the \(p\)-values are not statistically significant, suggesting an absence of significant bias. Furthermore, we observed the smallest effect sizes in both tasks, highlighting our methodology's effectiveness in minimizing bias within word embeddings.

### t-SNE Visualization

To visually demonstrate the effectiveness of our proposed method in reducing gender bias, we selected the top 500 male- and female-biased embeddings. Using t-SNE projection, we compare the original GloVe and our debiased embeddings. Figure 2 shows the separation of male- and female-biased embeddings in two different colors. It can be observed that our method has mixed the male- and female-biased embeddings effectively.

 



### Word Similarity Tasks

While reducing bias is our primary goal, it is crucial not to destroy other semantic information encoded in word embeddings. We evaluate our algorithm by the following word similarity tests: RG65 (Rubenstein and Goodenough, 1965), WordSim-353 (Finkelstein et al., 2001), Rarewords (Luong et al., 2013), MEN (Bruni et al., 2014), MTurk-287 (Radinsky et al., 2011), and MTurk-771 (Halawi et al., 2012), SimLex-999 (Hill et al., 2015), and SimVerb-3500 (Gerz et al., 2016). These datasets associated with each task contain word pairs and a corresponding human-annotated similarity score. We calculate Spearman's rank correlation coefficient between the two ranks. The results of our method and the original GloVe are shown in Table 2. We observe an overall non-decreasing performance in most of the tasks, showing that the semantic information is protected.

### Sentence Embedding Association Test (SEAT)

In addition to testing on static word embeddings, we also test on contextualized word embeddings. SEAT (May et al., 2019), extends the WEAT test by leveraging simple templates such as 'This is a <word>' to obtain the individual word's contextualized embedding. We use the implementation results from (Meade et al., 2022). The baseline includes BERT base uncased, CDA and Dropout (Webster et al., 2020), SentDebias Liang et al. (2020), and INLP.

To train projections for the topics of gender, race, and religion, we used the vocabulary from the GloVe model. All words were divided into groups according to their cosine similarities with pre-determined hint words: [he, she] for gender, [black people, white people] for race, and [Christianity, Jewish, Islam] for religion. Using BERT representations, we selected the top 75k words for gender, 75k for race, and 30k for religion from each group and associated them with their group labels as the input dataset for Algorithm 1.

For a detailed list of the SEAT tests used to measure each type of bias in our work, the complete results, we refer readers to Appendix A.6. In Table 2, we display the average effect size for each SEAT task category evaluated. Our findings reveal superior performance in two out of the three tasks while delivering comparable results to the INLP method in the Gender task. Notably, our SUP method exhibits enhanced performance across a variety of bias-influenced topics.

### Extrinsic: Fairness Text Classification

For the extrinsic task, we consider the fairness text classification problem. We conduct experiments over three different tasks - sentiment analysis (MOJI), biography classification (BIOS), and toxic comment classification (Toxic). The detail of

\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & Clustering & Correlation & Profession & Classify \\ \hline GloVe & 1.0000 & 0.7727 & 0.8200 & 0.9980 \\ Hard & 0.8050 & 0.6884 & 0.7161 & 0.9068 \\ GP & 1.0000 & 0.7700 & 0.8102 & 0.9978 \\ GN & 0.8560 & 0.7336 & 0.7p

the datasets is described as follows:

**MOJI** is a sentiment classification dataset collected by (Blodgett et al., 2016) that contains tweets from either African-American English or Standard American English. Each of the text data is labeled with a binary 'race' label based on the kind of English they use. The binary sentiment score is annotated by the emoji contained in the tweets. We compose the training data set as follows: AAE-happy = 40%, SAE- happy = 10%, AAE-sad = 10%, and SAE-sad = 40%. We used the train, dev, and test splits of 100k/8k/8k instances, respectively.

**BIOS** dataset (De-Arteaga et al., 2019) is a personal biography classification dataset annotated by gender and 28 classes of occupation. We follow the same split setup for the BIOS data as in (DerAretega et al., 2019), and the ratio of train:dev:test is \(65\%:10\%:25\%\).

**Toxic** dataset features text sourced from the Talk Pages of Wikipedia, where each comment has been categorized by human assessors as either toxic or non-toxic. Our research employs the same division of data as specified by (Dixon et al., 2018), enabling us to test the efficacy of our method in reducing discrimination against minority groups.

The fairness criterion is defined by _Equality of Opportunity_ (EO), i.e. a classifier is considered fair if its prediction is independent of the sensitive attribute given the true label. For BIOS and MOJI data, it is measured by considering the gap in the True Positive Rate (TPR) between different sensitive attribute groups:

\[TPR_{z,y}=P[\hat{Y}=y|Z=z,Y=y],\] \[GAP^{TPR}_{y}=TPR_{z,y}-TPR_{x^{\prime},y}.\]

The root-mean-square (RMS) gap over all groups is \(GAP^{TPR}_{RMS}=\sqrt{\frac{1}{|C|}\sum_{y\in C}(GAP^{TPR}_{y})^{2}}\).

We follow the original implementation of MOJI and BIOS that use race and gender labels as sensitive attributes \(Z\). The results are shown in Table 3. We report the Accuracy, the \(GAP_{RMS}\), and the Time in seconds for BIOS and MOJI. The baselines are from (Rayfogel et al., 2020), (Rawfogel et al., 2022a), (Chowdhury and Chaturvedi, 2022), and (Rawfogel et al., 2022b).

In Table 3, we implement each task using BERT and establish it as the baseline - this represents the results without any fairness considerations. Our findings reveal that in the BIOS task, while the INLP and FaRM achieve low RMS, it is accompanied by a compromise in accuracy. In contrast, our SUP method demonstrates balanced performance on both fronts. For MOJI, our algorithm stands out, yielding the smallest discrepancy gap among all methods, all the while maintaining uncompromised accuracy. Moreover, our algorithm benefits from having an explicit solution, eliminating the need for iterative calculations, and running significantly faster than many existing baselines.

In addition, we also conduct experiments on a more challenging dataset: the Toxic Comment Classification (Dixon et al., 2018). Within this dataset, each sample may belong to **multiple sensitive attribute groups**, embodying intersectionality in biases. For instance, a single comment might simultaneously belong to 'black' and 'gay' sensitive groups. We adhere to the definitions and gap measurements outlined by Dixon et al. (2018), \(GAP_{\mathrm{toxic}}=\sum_{z\in\mathcal{Z}}{\left|TPR_{z,0}-\right.}\)\(\mathrm{mean}_{z\in\mathcal{Z}}{\left(TPR_{z,0}\right)}\rvert\), where \(\mathrm{mean}_{z\in\mathcal{Z}}{\left(TPR_{z,0}\right)}\) is the average of TPR gaps of all sensitive attributes. where \(\mathrm{mean}_{z\in\mathcal{Z}}{\left(TPR_{z,0}\right)}\) represents the average of True Positive Rate (TPR) gaps across all sensitive attributes. The sensitive attribute in this scenario is depicted as a 50-dimensional vector, illustrating the relative frequency of sensitive words within sentences.

For the original BERT model, the Area Under the Curve (AUC) was 95.5, and the \(GAP_{\mathrm{toxic}}\) was 7.34. By employing our method, we man

\begin{table}
\begin{tabular}{l r r r r} \hline \hline  & RG65 & WS & RW & MEN \\ \hline GloVe & 0.7540 & 0.6199 & 0.3722 & 0.7216 \\
**SUP** & **0.7913** & **0.6617** & **0.3986** & **0.7423** \\ \hline  & MT-287 & MT-771 & SimLex & SimVerb \\ \hline GloVe & 0.6480 & 0.6486 & 0.3474 & 0.2038 \\
**SUP** & 0.6349 & **0.6792** & **0.3949** & **0.2493** \\ \hline \hline \end{tabular}
\end{table}
Table 2: **Left: Word similarity results. A higher value indicates a better semantic correlation; Right: SEAT average effect sizes for debiased BERT. A lower number in each column indicates better debiasing performance. The best results are boldfaced. Baseline results are from (Meade et al., 2022).** 

aged to maintain the AUC at 95.0 while reducing the \(GAP_{\mathrm{toxic}}\) to 5.95, showcasing the efficacy of our approach in mitigating biases while preserving model performance. It is crucial to highlight that our methodology effectively manages the intricacies of intersectional biases in toxic comment classification, a complexity not adequately addressed by other baseline algorithms. For a more detailed discussion, please refer to Appendix A.2.

## 6 Bridge between Debiasing and Fairness

In this section, we provide a theoretical analysis of how our proposed method can incorporate debiasing and fairness tasks into a unified framework and handle them simultaneously. As previously discussed, both tasks aim to achieve conditional independence with respect to certain variables. Here, we will demonstrate how minimal subspace \(\mathcal{S}_{2}\) bridges these tasks. We provide the following theorem to prove the effectiveness of our framework in dealing with both debiasing and fairness tasks. Please refer to Appendix A.7 for detailed proof.

**Theorem 6.1**.: _With the settings defined in Section 3 and linearity assumption, suppose \(\mathbf{X}\perp\!\!\!\perp\mathbf{Z}\mid Q\mathbf{X}\), then \(\widetilde{\mathbf{X}}=(I-Q)\mathbf{X}\) is a debiased representation. Further, suppose \(\mathbf{X}\perp\!\!\!\perp\mathbf{Y}\mid Q_{y}\mathbf{X}\), if \(\operatorname*{Span}\{Q_{y}\}\subseteq\operatorname*{Span}\{Q\}\), then \(\widetilde{\mathbf{X}}=(I-Q)\mathbf{X}\) is a fair representation._

Theorem 6.1 states that the projected representation \(\widetilde{\mathbf{X}}=(I-Q)\mathbf{X}\) has no correlation with the sensitive attributes \(\mathbf{Z}\), which achieves the goal of debiasing task. Moreover, if the subspace spanned by sensitive attribute \(\mathbf{Z}\) (\(\mathcal{S}_{\mathbf{Z}|\mathbf{X}}\)) is included in the subspace spanned by target attribute \(\mathbf{Y}\) (\(\mathcal{S}_{\mathbf{Y}|\mathbf{X}}\)), we can achieve the goal of fairness task by projecting the original representation on \((I-Q)\).

The theoretical property is consistent with the experimental results shown above. For the debiasing task, the matrix \(Q=\sum_{i=1}^{q}\psi_{i}\psi_{i}^{\top}\) in Algorithm 1 is the estimated central mean space regarding \(Z\), then \(I-Q\) forms a sufficient projection defined in Theorem 6.1, which shows great improvement upon existing state-of-the-art methods. For fairness task, the eigenvectors \(\left\{\psi_{j}\right\}_{j=1}^{q}\) calculated in Algorithm 1 recovers the matrix \(Q\) stated in Theorem 6.1. If we have \(\operatorname*{Span}\{Q_{y}\}\subseteq\operatorname*{Span}\{Q\}\), then we can set \(\widetilde{\mathbf{X}}=(I-Q)\mathbf{X}\) to get the fair representation. However, in real data, this condition is usually violated, which means \(\operatorname*{Span}\{Q_{y}\}\nsubseteq\operatorname*{Span}\{Q\}\). Therefore, the SUP may not achieve the optimal fair representation in downstream tasks.

## 7 Conclusion and Future Work

In this paper, we propose a theoretically grounded framework for reducing bias by projecting vector representations to an unbiased subspace. It can reduce biased information effectively in both intrinsic and extrinsic tasks, as well as different kinds of representations. In addition, we provide a theoretical guarantee about the effectiveness of our method in reducing biased information. Finally, our method not only surpasses existing state-of-the-art approaches in bias mitigation while maintaining robust task performance but also achieves superior computational efficiency.

While this work has demonstrated its effectiveness in various tasks, it has the potential to be applied to other applications that rely on vector representation. We are also interested in combining our method with the different other notions of fairness. We aim to explore these directions in future work.

## Acknowledgements

This work was supported by the Economic and Social Research Council (ESRC ES/T012382/1) and the Social Sciences and Humanities Research Council (SSHRC 2003-2019-0003) under

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{4}{c}{BIOS} & \multicolumn{4}{c}{MOJI} \\ \cline{2-7}  & Acc.(\(\uparrow\)) & GAP(\(\downarrow\)) & Time(\(\downarrow\)) & Acc.(\(\uparrow\)) & GAP(\(\downarrow\)) & Time(\(\downarrow\)) \\ \hline BERT & 79.1 & 14.5 & - & 71.6 & 31.0 & - \\ +INLP & 71.9 & 9.9 & 271 & 62.2 & 15.8 & 1003 \\ +RLACE & 76.9 & 13.2 & 4312 & 72.2 & 15.4 & 2456 \\ +FaRM(unconstrained) & 55 & **7.9** & 6723 & 63.5 & 14.0 & 4162 \\ +Kernel(Poly) & **79.9** & 16.8 & 3914 & 72.9 & 17.3 & 8861 \\ +Kernel(RBF) & 60.7 & 18.0 & 3487 & **74.1** & 13.3 & 5496 \\ \hline
**+SUP** & 76.4 & 12.7 & **6.76** & 69.1 & **10.5** & **33.04** \\ \hline \hline \end{tabular}
\end{table}
Table 3: **Left: Result of BIOS text classification. Predict using [CLS] token. Right: Result of MOJI text classification. The best result is boldfaced.** 

the scheme of the Canada-UK Artificial Intelligence Initiative. The project title is BIAS: Responsible AI for Gender and Ethnic Labour Market Equality. Bei Jiang and Linglong Kong were partially supported by grants from the Canada CIFAR AI Chairs program, the Alberta Machine Intelligence Institute (AMII), and Natural Sciences and Engineering Council of Canada (NSERC), and Linglong Kong was also partially supported by grants from the Canada Research Chair program from NSERC. We also thank all the constructive suggestions and comments from the reviewers.

## References

* Aragon [1997] Yves Aragon. 1997. A gauss implementation of multivariate sliced inverse regression. _Computational Statistics_, 12(3):355-372.
* Barikeri et al. [2021] Soumya Barikeri, Anne Lauscher, Ivan Vulic, and Goran Glavas. 2021. RedditBias: A real-world resource for bias evaluation and debiasing of conversational language models. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 1941-1955, Online. Association for Computational Linguistics.
* Bengio et al. [2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. _IEEE transactions on pattern analysis and machine intelligence_, 35(8):1798-1828.
* Blodgett et al. [2016] Su Lin Blodgett, Lisa Green, and Brendan O'Connor. 2016. Demographic dialectal variation in social media: A case study of african-american english. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 1119-1130.
* Bolukbasi et al. [2016] Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. _Advances in neural information processing systems_, 29.
* Bruni et al. [2014] Elia Bruni, Nam-Khanh Tran, and Marco Baroni. 2014. Multimodal distributional semantics. _Journal of artificial intelligence research_, 49:1-47.
* Caliskan et al. [2017] Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. _Science_, 356(6334):183-186.
* Cao et al. [2022] Yang Cao, Yada Pruksachatkun, Kai-Wei Chang, Rahul Gupta, Varun Kumar, Jwala Dhamala, and Aram Galstyan. 2022. On the intrinsic and extrinsic fairness evaluation metrics for contextualized language representations. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 561-570.
* Cheng et al. [2020] Pengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si, and Lawrence Carin. 2020. Fairfill: Contrastive neural debiasing method for pretrained text encoders. In _International Conference on Learning Representations_.
* Chowdhury and Chaturvedi [2022] Somnath Basu Roy Chowdhury and Sniglha Chaturvedi. 2022. Learning fair representations via rate-distortion maximization. _Transactions of the Association for Computational Linguistics_, 10:1159-1174.
* Cook and Li [2002] R Dennis Cook and Bing Li. 2002. Dimension reduction for conditional mean in regression. _The Annals of Statistics_, 30(2):455-474.
* De-Arteaga et al. [2020] Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Gejvik, Krishnarn Kenthapadi, and Adam Tauman Kalai. 2020. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In _proceedings of the Conference on Fairness, Accountability, and Transparency_, pages 120-128.
* Dev et al. [2020] Sunipa Dev, Tao Li, Jeff M Phillips, and Vivek Srikumar. 2020. On measuring and mitigating biased inferences of word embeddings. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 34, pages 7659-7666.
* Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. _arXiv preprint arXiv:1810.04805_.
* Dinan et al. [2020] Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, and Jason Weston. 2020. Queens are powerful too: Mitigating gender bias in dialogue generation. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 8173-8188, Online. Association for Computational Linguistics.
* Ding et al. [2022] Lei Ding, Dengdeng Yu, Jinhan Xie, Wenxing Guo, Shenggang Hu, Meichen Liu, Linglong Kong, Hongsheng Dai, Yanchun Bao, and Bei Jiang. 2022. Word embeddings via causal inference: Gender bias reducing and semantic information preserving. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 36, pages 11864-11872.
* Dixon et al. [2018] Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and mitigating unintended bias in text classification. In _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society_, pages 67-73.
* Finkelstein et al. [2001] Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2001. Placing search in context: The concept revisited. In _Proceedings of the 10th international conference on World Wide Web_, pages 406-414.
* Gerz et al. [2016] Daniela Gerz, Ivan Vulic, Felix Hill, Roi Reichart, and Anna Korhonen. 2016. Simverb-3500: A large-scale evaluation set of verb similarity. _arXiv preprint arXiv:1608.00869_ 

Seraphina Goldfarb-Tarrant, Rebecca Marchant, Ricardo Munoz Sanchez, Mugdha Pandya, and Adam Lopez. 2021. Intrinsic bias metrics do not correlate with application bias. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 1926-1940.
* Gonen and Goldberg [2019] Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. _NAACL-HLT_.
* Guo et al. [2022] Yue Guo, Yi Yang, and Ahmed Abbasi. 2022. Autodebias: Debiasing masked language models with automated biased prompts. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1012-1023.
* Halawi et al. [2012] Guy Halawi, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. 2012. Large-scale learning of word relatedness with constraints. In _Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining_, pages 1406-1414.
* Han et al. [2021] Xudong Han, Timothy Baldwin, and Trevor Cohn. 2021. Diverse adversaries for mitigating bias in training. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_, pages 2760-2765.
* Hardt et al. [2016] Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. _Advances in neural information processing systems_, 29.
* He et al. [2022] Jacqueline He, Mengzhou Xia, Christiane Fellbaum, and Danqi Chen. 2022. Mabel: Attenuating gender bias using textual entailment data. _arXiv preprint arXiv:2210.14975_.
* Hill et al. [2015] Felix Hill, Roi Reichart, and Anna Korhonen. 2015. Simlex-999: Evaluating semantic models with (genuine) similarity estimation. _Computational Linguistics_, 41(4):665-695.
* Kaneko and Bollegala [2019] Masahiro Kaneko and Danushka Bollegala. 2019. Gender-preserving debiasing for pre-trained word embeddings. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 1641-1650.
* Kaneko and Bollegala [2021] Masahiro Kaneko and Danushka Bollegala. 2021. Debiasing pre-trained contextualised embeddings. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_, pages 1256-1266.
* Kaneko et al. [2022] Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki. 2022. Debiasing isn't enough!-on the effectiveness of debiasing mlms and their social biases in downstream tasks. In _Proceedings of the 29th International Conference on Computational Linguistics_, pages 1299-1310.
* Kurita et al. [2019] Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring bias in contextualized word representations. In _Proceedings of the First Workshop on Gender Bias in Natural Language Processing_, pages 166-172.
* Li [1991] Ker-Chavi Li. 1991. Sliced inverse regression for dimension reduction. _Journal of the American Statistical Association_, 86(414):316-327.
* Liang et al. [2020] Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2020. Towards debiasing sentence representations. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
* Luong et al. [2013] Minh-Thang Luong, Richard Socher, and Christopher D Manning. 2013. Better word representations with recursive neural networks for morphology. In _Proceedings of the seventeenth conference on computational natural language learning_, pages 104-113.
* May et al. [2019] Chandler May, Alex Wang, Shikha Bordia, Samuel Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 622-628.
* Meade et al. [2022] Nicholas Meade, Elinor Poole-Dayan, and Siva Reddy. 2022. An empirical survey of the effectiveness of debiasing techniques for pre-trained language models. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1878-1898.
* Mikolov et al. [2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. _Advances in neural information processing systems_, 26.
* Nadeem et al. [2021] Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. Stereoset: Measuring stereotypical bias in pretrained language models. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 5356-5371.
* Nangia et al. [2020] Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel Bowman. 2020. Crows-pairs: A challenge dataset for measuring social biases in masked language models. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 1953-1967.
* Pennington et al. [2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_, pages 1532-1543 

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics.
* Radford et al. [2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. _OpenAI blog_, 1(8):9.
* Radinsky et al. [2011] Kira Radinsky, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. 2011. A word at a time: computing word relatedness using temporal semantic analysis. In _Proceedings of the 20th international conference on World wide web_, pages 337-346.
* Ravfogel et al. [2020]Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. Null it out: Guarding protected attributes by iterative nullspace projection. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 7237-7256.
* Ravorgel et al. [2022a]Shauli Ravorgel, Michael Twiton, Yoav Goldberg, and Ryan D Cotterell. 2022a. Linear adversarial concept erasure. In _International Conference on Machine Learning_, pages 18400-18421. PMLR.
* Ravfogel et al. [2022b]Shauli Ravfogel, Francisco Vargas, Yoav Goldberg, and Ryan Cotterell. 2022b. Adversarial concept erasure in kernel space. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_, pages 6034-6055.
* Reimers and Gurevych [2019] Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3982-3992.
* Richards et al. [2016]Richards et al. 2016. Non-binary or genderqueer genders. _INTERNATIONAL REVIEW OF PSYCHIATRY_, 28(1):95-102.
* Rubenstein and Goodenough [1965] Herbert Rubenstein and John B Goodenough. 1965. Contextual correlates of synonymy. _Communications of the ACM_, 8(10):627-633.
* Schick et al. [2021] Timo Schick, Sahana Udupa, and Hinrich Schutze. 2021. Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in nlp. _Transactions of the Association for Computational Linguistics_, 9:1408-1424.
* Shen et al. [2021]Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. 2021. Contrastive learning for fair representations. _arXiv preprint arXiv:2109.10645_.
* Stanovsky et al. [2019] Gabriel Stanovsky, Noah A Smith, and Luke Zettlemoyer. 2019. Evaluating gender bias in machine translation. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 1679-1684.
* Vig et al. [2020] Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. _Advances in Neural Information Processing Systems_, 33:12388-12401.
* Webster et al. [2020] Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beitel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020. Measuring and reducing gendered correlations in pre-trained models. _arXiv preprint arXiv:2010.06032_.
* Yang and Feng [2020]Zekun Yang and Juan Feng. 2020. A causal inference method for reducing gender bias in word embedding relations. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 34, pages 9434-9441.
* Zhao et al. [2018a]Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender bias in coreference resolution: Evaluation and debiasing methods. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_, pages 15-20.
* Zhao et al. [2018b]Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning gender-neutral word embeddings. _arXiv preprint arXiv:1809.01496_.
* Zmigrod et al. [2019] Ran Zmigrod, Sabrina J Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 1651-1661.

## Appendix A Appendix

### Toy example for weak linearity and minimal subspace

Suppose \({\mathbf{X}}\in\mathbb{R}^{4}\) and \({\mathbf{Z}}\in\mathbb{R}\), with two orthogonal directions \(\beta_{1}=(\frac{\sqrt{2}}{2},\frac{\sqrt{2}}{2},0,0)\) and \(\beta_{2}=(\frac{\sqrt{2}}{2},-\frac{\sqrt{2}}{2},0,0)\). Consider the following two models:

\[{\mathbf{Z}} =\beta_{1}^{\top}{\mathbf{X}}+\beta_{2}^{\top}{\mathbf{X}}+\varepsilon, \tag{2}\] \[{\mathbf{Z}} =\sin{(\beta_{1}^{\top}{\mathbf{X}})}+\exp(-\beta_{2}^{\top}{\mathbf{X}})+\varepsilon. \tag{3}\]

Then (2) satisfies the strong linearity assumption and (3) satisfies the weak linearity assumption.

For model (3), denote \(\beta_{3}=(0,0,1,0)\) and \(\beta_{4}=(0,0,0,1)\), then \(\mathrm{Span}\{\beta_{1},\beta_{2},\beta_{3},\beta_{4}\}=\mathbb{R}^{4}\), the entire representation space. Let \(\mathcal{S}_{2}=\mathrm{Span}\{\beta_{1},\beta_{2}\}\) and \(\mathcal{S}_{2}^{\prime}=\mathrm{Span}\{\beta_{1},\beta_{2},\beta_{3}\}\), with 

their associated orthogonal subspaces \({\mathcal{S}_{1}}=\mathrm{Span}\{\beta_{3},\beta_{4}\}\) and \({\mathcal{S}^{\prime}_{1}}=\mathrm{Span}\{\beta_{4}\}\). Note that both \({\mathcal{S}_{2}}\) and \({\mathcal{S}^{\prime}_{2}}\) contain all the directions related to \(\mathbf{Z}\). But \({\mathcal{S}_{2}}\) has smaller dimension than \({\mathcal{S}^{\prime}_{2}}\), thus we will lose less information when we project \(\mathbf{X}\) on \({\mathcal{S}_{1}}\).

Therefore, in model (3), both \({\mathcal{S}_{2}}\) and \({\mathcal{S}^{\prime}_{2}}\) are sufficient dimension reduction subspaces. The central subspace \({\mathcal{S}_{\mathbf{Z}|\mathbf{X}}}\) is equal to \({\mathcal{S}_{2}}=\{\beta_{1},\beta_{2}\}\), which is the minimal subspace.

### Comparison with other projection methods

We conduct a comparative analysis between our method and other projection-based methods.

**INLP:** Both our method SUP and INLP employ linear projections to minimize the influence of the sensitive attribute \(\mathbf{Z}\) in the representations. The underlying principle of INLP revolves around identifying the null space of the weight matrix, denoted as \(W\in\mathbb{R}^{k\times p_{1}}\), which corresponds to the parameters of linear classifier \(\mathbf{Z}=f(W\mathbf{X})\), where \(f\) represents the classifier function and \(k\) is the number of classes. This framework can be viewed as a specific instance of the Model 1, where the subspace spanned by \(\beta_{1},\ldots,\beta_{q}\) exactly corresponds to the union of row spaces of \(W_{i}\) during iterations. Specifically, INLP captures \(k\) directions (rows of weight matrix \(W_{j}\)) at each iteration, while SUP finds \(q\) directions in a single run, which is more flexible and computationally efficient.

**RLACE:** Both SUP and RLACE operate under linearity assumption as expressed in Assumption 3.4. In RLACE, the function \(f\) in Model (1) is interpreted as the inverse of a link function in the generalized linear model. In contrast, our approach imposes no specific constraints on the form of \(f\). While RLACE achieves debiasing by solving a minimax problem to identify the projection \(P\) that safeguards the sensitive attribute, our method directly estimates the directions with a closed form, offering superior computational efficiency.

**Advantage of SUP:** The main distinction between our proposed methodology and existing projection-based debiasing methods pertains to the range of tasks they can address. For instance, INLP is principally designed for handling categorical sensitive attributes. In the context of the toxic data task, the sensitive attribute is no longer a categorical variable, thereby undermining the effectiveness of INLP. However, it is important to note that our SUP algorithm does not violate the structure of Model (1) under the linearity assumption. As a result, our approach remains capable of estimating the directions \(\beta_{1},\ldots,\beta_{q}\) and mitigating bias through the Algorithm 1. This highlights the versatility of our SUP algorithm, showcasing its capability to adeptly manage a spectrum of uni-/multi-variate and discrete/continuous sensitive datasets. The capability of managing a sensitive attribute as a continuous variable also aligns more closely with contemporary sociological understandings. For instance, consider the interpretation of gender as a spectrumRichards et al. (2016) rather than a binary categorization. As such, models that can accommodate continuous variables for sensitive attributes are better equipped to reflect these more nuanced perspectives, thereby promoting fairness and inclusivity in their outcomes.

### Scheme for SIR Estimator

Suppose the data set \(\{(X_{i},Z_{i})\}_{i=1}^{n}\) is given, then the steps for SIR are summarized as :

1. Standardizing \(\mathbf{X}\) by the transformation \(\tilde{\mathbf{X}}_{i}=C_{X}^{-1/2}(\mathbf{X}_{i}-\mu_{X})\), where \(\mu_{X}\) and \(C_{X}\) are the mean vector and covariance matrix of \(X\).
2. Slice the range of \(Z\) into \(H\) intervals \(\{J_{h}\}_{h=1}^{H}\). Estimate the weight \(p_{h}=(1/n)\sum_{i=1}^{n}I(Z_{i}\in J_{h})\) and compute the sample mean \(m_{h}=(1/np_{h})\sum_{Z_{i}\in J_{h}}\tilde{\mathbf{X}}_{i}\) on each sliced interval.
3. Form \(M^{\rm SIR}=\sum_{h=1}^{H}p_{h}m_{h}m_{h}^{\top}\) and let \(\phi_{k}\) be its eigenvectors. The directions are estimated by \(\beta_{k}=C_{X}^{-1/2}\phi_{k}\) for \(k=1,\dots,q\).

### PMS Estimator Implementation

For multivariate variable \(Z\in\mathbb{R}^{p_{3}}\), let \(Z_{ij}\) denote the \(j\)-th coordinate of \(i\)-th sample, the PMS estimator can be achieved through the following Algorithm 2.

The weights \(w_{j}\) can be chosen as either equal weights or proportional to the leading eigenvalues of \(M_{j}\). Then the leading \(q\) eigenvectors \(\psi_{1},\ldots,\psi_{q}\) of \(M^{\rm PMS}\) can be used to recover \({\mathcal{S}_{\mathbf{Z}|\mathbf{X}}}\).

### Detail of WEAT

Let \(X\) and \(Y\) be two sets of target words of equal size \(n\) with their embedding \(\{x_{i}\}_{i=1}^{n}\) and \(\{y_{i}\}_{i=1}^{n}\), and \(A\), \(B\) the two sets of attribute words with their embedding \(\{a_{i}\}_{i=1}^{|A|}\) and \(\{b_{i}\}_{i=1}^{|B|}\). The WEAT uses the difference of averaged distance to measure the similarity of a vector \(w\) to two sets \(A\) and \(B\). The 

test statistic is

\[s(X,Y,A,B)=\sum_{x\in X}s(x,A,B)-\sum_{y\in Y}s(y,A,B)\]

where

\[s(w,A,B)=\frac{1}{|A|}\sum_{a\in A}\cos(w,a)-\frac{1}{|B|}\sum_{b\in B}\cos(w ,b)\]

In other words, \(s(w,A,B)\) measures the association of the word \(w\) with the attribute, and \(s(X,Y,A,B)\) measures the differential association of the two sets of target words with the attribute.

Let \(\left\{(X_{i},Y_{i})\right\}_{i}\) denote all the partitions of \(X\cup Y\) into two sets of equal size. The one-sided \(p\)-value of the permutation test is

\[\Pr_{i}\left[s\left(X_{i},Y_{i},A,B\right)>s(X,Y,A,B)\right]\]

The effect size is

\[\frac{\mathrm{mean}_{x\in X}s(x,A,B)-\mathrm{mean}_{y\in Y}s(y,A,B)}{\text{ std-dev}_{w\in X\cup Y}s(w,A,B)}\]

It is a normalized measure of how separated the two distributions (of associations between the target and attribute) are.

All word lists are from (Caliskan et al., 2017). Because GloVe embeddings are uncased, we use lowercase words.

### Detail of SEAT

In this section, we provide a complete set of results for all SEAT tests. All of the baseline results are from (Meade et al., 2022). Also, for detailed attribute word sets and the target word sets, please refer to their GitHub repo. Table 4 are tasks for Gender debias. Table 5 are tasks for Race debias. Table 6 are tasks for Religion debias.

### Proof of Theorem 6.1

Proof.: According to the definition of conditional independence, for any measurable function \(f\), we have \(f(X)\perp\!\!\!\perp Z\mid QX\) because the randomness of \(f(X)\) only comes from the random variable \(X\).

For the debias task, notice that \(X\perp\!\!\!\perp Z\mid QX\), thus \(X\perp\!\!\!\perp Z\mid QX\). It implies that \(Z\) only depends on \(QX\). Therefore, if we eliminate those correlated part and denote \(\widetilde{X}=(I-Q)X\), we have \(\widetilde{X}\perp\!\!\!\perp Z\). It achieves the goal of the debias task defined above.

For the fairness task, if we assume \(X\perp\!\!\!\perp Y\mid Q_{y}X\), which implies \(Y=f_{0}(Q_{y}X)\) for some measurable function \(f_{0}\). Notice that \(\mathrm{Span}\{Q_{y}\}\subset\mathrm{Span}\{Q\}\), then \(\mathrm{Span}\{Q_{y}\}\) is orthogonal to \(\mathrm{Span}\{I-Q\}\), which implies \((I-Q)X\perp\!\!\!\perp Q_{y}X\). Therefore, \((I-Q)X\perp\!\!\!\perp Z\mid Y\) since the randomness of \(Y\) comes from \(Q_{y}X\). It achieves the goal of the fairness task defined above if we let \(\widetilde{X}=(I-Q)X\). 

**Remark A.1**.: _We should emphasize that in the above theorem, the random vectors \(X\), \(Y\), and \(Z\) are defined on the Euclidean space \(\mathbb{R}^{p_{1}}\), \(\mathbb{R}^{p_{2}}\) and \(\mathbb{R}^{p_{3}}\) respectively. For each random variable, taking \(X\) as an example, the sample space is defined as \(\Omega=\mathcal{B}(\mathbb{R}^{p_{1}})\), which is Borel set generated by all open set on \(\mathbb{R}^{p_{1}}\), and the \(\sigma\)-algebra \(\Sigma\) is generated by \(\Omega\), i.e. \(\Sigma=\sigma(\Omega)\). In this way, for any measurable function \(f\) satisfying the sample space of \(f(X)\) is included in the sample space of \(X\), we have \(\sigma(f(X))\subset\sigma(X)\), and thus the desired properties of conditional independence hold in the proof_

### Effect of \(q\)

In this section, we use MOJI data to illustrate the impact of the dimension removed \(q\) on performance of debiasing and task accuracy, see Figure 3 below.

It is important to note that the debiasing procedure may distort the relevant concepts or key information, denoted by \(Y\). Generally, as \(q\) increases, both the sensitive information \(Z\) and part of the target inform 

debiased representation. This occurs due to the intersection of the subspaces spanned by \(Z\) and \(Y\). Consequently, a rise in \(q\) leads to a simultaneous reduction in accuracy and the gap, illustrating a delicate equilibrium and trade-off between targeting and debiasing performance.

### Limitations

All our result is based on the English dataset, as there is a lack of benchmark of fairness in other languages. Also, we only consider the transformation under a linear framework, where we aim to find the projection matrix \(P\). However, the estimation procedure for the central subspace \({\mathcal{S}_{\mathbf{Z}|\mathbf{X}}}\) has been well developed and can find nonlinear transformation \(g\), which we leave for future exploration. Also, for the SEAT evaluation, there are some researchers point out that SEAT sometimes not able to detect the

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & \multicolumn{5}{c}{SEAT Religion Tasks} \\ \hline Model & Religion-1 & Religion-1b & Religion-2 & Religion-2b & Avg. Effect Size (\(\downarrow\)) \\ \hline BERT & 0.744 & -0.067 & 1.009 & -0.147 & 0.492 \\ CDA & 0.355 & -0.104 & 0.424 & -0.474 & 0.339 \\ Dropout & 0.535 & 0.109 & 0.436 & -0.428 & 0.377 \\ INLP & 0.473 & -0.301 & 0.787 & -0.280 & 0.460 \\ SentDebias & 0.728 & 0.003 & 0.985 & 0.038 & 0.439 \\ \hline SUP & 0.392 & -0.066 & 0.492 & 0.092 & **0.261** \\ \hline \hline \end{tabular}
\end{table}
Table 6: SEAT effect sizes for religion debiased BERT. Effect sizes closer to 0 are indicative of less biased model representations.

Figure 3: Trends of accuracy and GAP for MOJI data with number of dimension \(q\) removed.

\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline  & \multicolumn{5}{c}{SEAT Race Tasks} \\ \hline Model & ABW-1 & ABW-2 & SEAT-3 & SEAT-3b & SEAT-4 & SEAT-5 & SEAT-5b & Avg. Effect Size (\(\downarrow\)) \\ \hline BERT & -0.079 & 0.690 & 0.778 & 0.469 & 0.901 & 0.887 & 0.539 & 0.620 \\ CDA & 0.231 & 0.619 & 0.824 & 0.510 & 0.896 & 0.418 & 0.486 & 0.569 \\ Dropout & 0.415 & 0.690 & 0.698 & 0.476 & 0.683 & 0.417 & 0.495 & 0.554 \\ INLP & 0.295 & 0.565 & 0.799 & 0.370 & 0.976 & 1.039 & 0.432 & 0.639 \\ SentDebias & -0.067 & 0.684 & 0.776 & 0.451 & 0.902 & 0.891 & 0.513 & 0.612 \\ \hline SUP & 0.019 & 0.428 & 0.542 & 0.193 & 0.611 & 0.716 & 0.514 & **0.432** \\ \hline \hline \end{tabular}
\end{table}
Table 4: SEAT effect sizes for gender debiased BERT. Effect sizes closer to 0 are indicative of less biased model representations.

 

bias inside the language model. But compared with other debiasing studies that only report on SEAT, we test our method on much more comprehensive experiments.

### Ethics Statement

Our research is fundamentally methodological in nature, focusing on the development of strategies to mitigate biases in NLP. We have taken careful measures to ensure that our work adheres to recognized ethical guidelines. For all evaluations related to bias and fairness, we have strictly followed established protocols, utilizing well-known tasks to evaluate biases related to gender, religion, and race. It is important to clarify that our use of these tasks is for analytical purposes only, with the sole intention of understanding and minimizing the biases present in AI systems. Our goal is to promote fairness and inclusivity in AI, and we firmly advocate for the respectful and unbiased treatment of all individuals, irrespective of their gender, religion, or race.

#### Reproducibility

Hyperparameter tuning: For our method, the main hyperparameter is the \(q\): the number of directions we want to project. We use regular grid search to find the best hyperparameter. For classifiers mentioned in Algorithm 1, we use the logistic classifier in sklearn.

Computational detail: We conduct all our experiments on an Ubuntu Server with CPU AMD Ryzen Threadripper 3990X 64-Core Processor and 256G RAM. Since our experiments do not need many computational resources (no retraining or fine-tuning), no GPU is needed.

Baseline results: Most of the baseline results are from recently published papers of well-known conferences. In static embedding evaluation, the INLP results are calculated by our code using the embedding they provided, which has a slightly better result than they reported in their paper