Preserving Task-Relevant Information Under Linear Concept Removal
Authors: Floris Holstege
Shauli Ravfogel
Bram Wouters
Venue: NeurIPS 2025 poster ()
Keywords: ersaure; linear erasure
Primary area: deep_learning
TL;DR: We derive a closed-form solution for a linear erasure projection that preserves the covariance with the main task labels.

Abstract: Modern neural networks often encode unwanted concepts alongside task-relevant information, leading to fairness and interpretability concerns. Existing post-hoc approaches can remove undesired concepts but often degrade useful signals. We introduce SPLINCE—Simultaneous Projection for LINear concept removal and Covariance prEservation—which eliminates sensitive concepts from representations while exactly preserving their covariance with a target label. SPLINCE achieves this via an oblique projection that ``splices out'' the unwanted direction yet protects important label correlations. Theoretically, it is the unique solution that removes linear concept predictability and maintains target covariance with minimal embedding distortion. Empirically, SPLINCE outperforms baselines on benchmarks such as Bias in Bios and Winobias, removing protected attributes while minimally damaging main-task information.

Preserving Task-Relevant Information Under Linear Concept Removal
Floris Holstege *tt Shauli Ravfogel* Bram Wouterst
University of Amsterdam, Department of Quantitative Economics New York University, Center for Data Science Tinbergen Institute t
Abstract
Modern neural networks often encode unwanted concepts alongside task-relevant information, leading to fairness and interpretability concerns. Existing post-hoc approaches can remove undesired concepts but often degrade useful signals. We introduce SPLINCE SPLINCF--Smmllaneeus Projection for L.Near concept removal and Covariance prEsevvation-wia wwhich eliminates sensitive concepts from representa- tions while exactly preserving their covariance with target label. SPLINCE achieves this via an oblique projection that "splices out" the unwanted direction yet protects important label correlations. Theoretically. it is the unique solution that removes linear concept predictability and maintains target covariance with minimal embedding distortion. Empirically, SPLINCE outperforms baselines benchmarks such as Bias in Bios and Winobias, removing protected attributes while minimally damaging main-task information.
Introduction
Deep neural networks (DNNs), including Language Models (LMs), have achieved great success in natural language processing (NLP) P) by learning rich representations of text, often referred to as embeddings [Cao 2024 Wang et al. 2024a]. These embeddings were shown also encode undesired information, such as markers of gender, leading to biased predictions [Bolukbasi et al.) 2016). In response, variety of concept-removal methods has been developed to remove undesired information from embeddings. Examples of such methods are iterative nullspace projection (INLP, |Ravfogel ict al. (2200), Linear adversarial concept erasure (RLACE, Ravfogel et al. (2022), D) Spectral Attribute Removal (SAL, Shao et al. [2023). ), and Least-squares Concept Erasure (LEACE, [Belrose al.
22033). ). The shared objective of these methods is to make a concept-such as gender sender--udeetctabble
by any linear classifier, while preserving the original embeddings much as possible.
Previous work has noted that a drawback post-hoe concept-removal methods that addition to removing a particular concept, they tend to also eliminate other concepts and information from embeddings [Feder al. 2021 Belinkov 2022 Kumar al., 2022 Guerner et al. 2025 Ravfogel al.] 2025). Consider, for instance, scenario where we wish to remove the effect of gender markers on classifier that screens CVs for job applications. Naively applying concept-erasure techniques to removes gender markers from the input representations may inadvertently harm the model's performance on the primary task profession prediction, since in real-world data, certain professions are strongly associated with gender. As a result, the erasure may distort relevant information, undermining both interpretability and utility.
In this paper, we seek to address key drawback of post-hoc concept-removal methods. Our contribution introduce SPLINCE, projection that (similar LEACE SAL) prevents any
"Equal contribution. Correspondence f. f.g.g.h.o www.hssggeuuaa.................. ml.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

linear classifier from predicting a concept, while also preserving the covariance with a task of interest. Mathematically, we construct an oblique projection that places the covariance between the representations and a protected attribute in its kernel, while maintaining the covariance between the representations and the main-task label in its range.
We prove that if a linear classifier is re-fitted after projection without regularization, any two pro- jections that share the same kernel (i.e., that linearly erase the same subspace) will induce identical loss. In that sense, SPLINCE and previous methods such as LEACE [Belrose et al. 2023 (as well as more naive versions that do not explicitly aim maintain the minimality of the projection) are all equivalent. For causal model interventions aiming to interpret its behavior-a situation where the underlying model is necessarily frozen- we argue that SPLINCE may perform a more surgical intervention, akin to minimizing the side effects of crasure on related concepts (c.g., removing gender bias while preserving grammatical gender). Empirically, we show that in realistic classification setting, SPLINCE improves fairness in highly challenging, imbalanced scenario, and removes stereotypes while maintaining correlated factual information.
2 Related work
Concept-removal: in response to growing concerns about DNNs relying on problematic or harmful concepts, a range of adversarial methods were developed to remove concepts from the embeddings of neural networks [Xie et al. 2017, Zhang et al., 2018]. However, these methods were later deemed unsuccessful at removing concepts [Elazar and Goldberg 2018). Subsequently, many works (including this) focused on preventing a linear classifier from predicting concept as more tractable alternative. This line of work is supported by the linear subspace hypothesis (Bolukbasi et al. 2016), which argues that concepts are represented in linear subspaces of embeddings (for more elaborate discussion, see Park et al. (024)).
Existing linear concept-removal methods: iterative nullspace projection (INLP, Ravfogel et al. 20200) trains linear classifier to predict a concept, and projects embeddings the nullspace of the parameters of the linear classifier. This is repeated until the concept can no longer be predicted by the linear classifier. Relaxed Linear Adversarial Concept Erasure (RLACE, Ravfogel et al. [2022) trains an orthogonal projection matrix such that a concept cannot be predicted by a linear classifier. These works were followed up by Least-squares Concept Erasure (LEACE, [elrrose et al. [2023). which ensures that no linear classifier can predict the concept (hereafter referred to as linear guarledness) while minimally altering the embeddings. Spectral attribute removal (SAL., [Shao et al. [2023]) projects the embeddings orthogonal to the first & eigenvectors of the covariance matrix Cov(æ, x).
Mean Projection (MP, Hagghghatkhah et al. 2022 projects embeddings to the nullspace of the mean difference between embeddings with and without concepts. It is equivalent to SAL when the concept is binary. SAL and MP also guarantee linear guardedness (similar to LEACE), whereas other methods may or may not satisfy this criterion.
Linear concept-removal while preserving task-relevant information: previous work suggests that linear concept-removal methods remove task-relevant information in addition to the concept they seek to remove [Belinkov 2022 Kumar et al., 2022 Guerner et al.) 2025). response, several alternatives have been proposed to address this issue, all removing a different linear subspace (Dev cct al. |021 Holstege et al., 2024 Barceva et al.] 2024 Shi ct al. 2024). However, each of these approaches sacrifices linear guardedness in order to retain more task relevant information. An alternative approach is to explicitly optimize for fairness while maintaining task performance (Shen cet al. 10 However, this approach is more resource-intensive, it cannot be applied to the frozen representations of a pretrained model. Moreover, because it modifies the original representations, it is unsuitable for scenarios where the intervention is intended to simulate causal experiments on the behavior of a pretrained LM, as discussed in section 4.2
In this paper, we study how to retain task-relevant information while maintaining linear guardedness. Recent work has also focused on applying projections to parameters of DNNs instead of embeddings
Limisiewicz et al. 2024 2025 Arditi et al. 2024]. This is outside of the scope of this paper.

---

3 Theory
We consider random vectors æ E IR and 2. Here, a can be any vector of features, but should generally be thought of as embeddings of deep neural network. In most cases we consider, they are the last-layer embeddings. The vector z represents the concept to be removed. can be a binary or one-hot-encoded label, or continuous in the case of a regression setting.
The general idea of linear concept removal is to apply an affine transformation r(æ) Pa + b, where P Raxd and b Rd that prevents classifiers from recovering the concept represented by from the features æ. A special case of this objective aims to achieve linear guardedness [Ravfogel et al. 2023 Belloos et al. 2023). J. the inability of linear classifiers to predict the concept. Concretely, they show that linear guardedness is equivalent to zero covariance between the transformed features and the concept to be removed, i.e., Cov(r(x), z) PE = O, where Σx, Cov( (x: 2) is the cross-covariance matrix of æ and 2, and the symbol 0 can refer both to a zero vector and zero matrix. This condition, which we will refer as the kernel constraint, only requires the kernel of P to contain the column space colsp(E... (2..2) C IR4. The intuition is that P removes directions in the feature space that are linearly correlated with z, making it impossible for linear classifiers to use the transformed features to predict x. Importantly, this means that the requirement of linear guardedness does not uniquely determine the affine transformation. Belrose et al. [002] use this freedom to minimize the impact of the transformation on the distance between the original and projected representations, driven by the intuition that the minimal-norm projection would minimally damage other semantic
information encoded therein.
This problem turns out to have a closed-form solution: Belrose et al. 2023 show that for centered data, i.e., E|ær] O, the constrained optimization problem
arg min
Px - xI|M]]
PΣ = 0
PeR4xd
has solution PLEACE W UU' W, where W = -E is whitening matrix and U is matrix whose orthonormal columns span the orthogonal complement of colsp WEs,s), which is the column space of the covariance matrix between æ and z after whitening. Here, we denote by Ex, E Rdxd the variance-covariance matrix of x, by A the Moore-Penross, pseudoinverse of matrix A, and by A 1/2 the p.s.d. square root of p.s.d. matrix A. The resulting transformation is an oblique projection with kernel colsp(E..z) and range determined by the whitening matrix W. The intuition is that this is the smallest possible kernel that satisfies the kernel constraint in equation [ while the chosen range minimizes the distortion caused by a projection with this kernel.
3.1 SPLINCE: Ensuring linear guardedness while preserving task-relevant covariance
The LEACE projection ensures linear guardedness while minimizing the distortion of the features, but oblivious of the main task of the model. A small expected norm squared may not optimally preserve information that is actually useful for the task at hand. Suppose now there is a random vector y y that represents the task-relevant information. Similar to the concept vector z, can be binary. one-hot encoded or continuous. We conjecture that task-relevant information in the features æ is located the directions that linearly covariate with y. In other words, is located in the column space of the covariance matrix Σn.y = Cov (æ, y) Indeed, removing this particular subspace completely prevents lincar classification [Belrose et al. 2023].
In order to preserve this task-relevant information, we require the affine transformation r(x) = Px+ not only to produce features that are linearly guarded for z, but also to leave the covariance between x and y invariant. For this approach we cast the name SPLINCE (Simultaneous Projection for LINear concept removal and Covariance prEservation), which can be seen as an extension of LEACE. It eliminates sensitive concepts from representations while exactly preserving their covariance with a target label. The SPLINCE optimization problem formulated in Theorem and its solution is given by equation 4 which is the main theoretical contribution of this paper.
Theorem 1. Let x and z, y be random vectors with finite second moments, non-zero covariances 1/2 be whitening between and 2, and between E and y, and E[x] 0. Let W (E
mairix. Define linear subspaces u+ = colsp( Em.z) and V = colsp WE + UU=, where
u = n (colsp( WExxx) + (assp)) Assume u n (Wll)) {0). Then the
3

---

optimization problem
arg min B PERaX
| 27 =1|M
(2)
subject to the two constraints
PΣ = O, and PE = Eg
(3)
to be referred to as the kernel and range constraint, respectively, has the solution
P'PLINCE
-W'v
(4)
where U and V are matrices whose orthonormal columns span U and V, respectively.
The proof of Theorem[ias given in Appendix A.
Compared to the LEACE optimization problem, the only difference is the additional condition preserving task-relevant information, PÊ Σx g which we refer to as the range constraint. Similar to LEACE, SPLINCE is an oblique transformation with kernel colsp(2 =. The difference lies in the range, which now contains colsp( Ex.y) order to fulfill the range constraint. Intuitively, the freedom that the LEACE optimization problem gives to the choice of the range partially used preserve the task-relevant information, i.e., the covariance between æ and y. The remainder of the freedom is used to minimize the distortion caused by the affine transformation, leading to whitening and unwhitening, similar LEACE.
The main assumption of Theorem is that 24 n colsp( WE = (0). which satisfied as long as the subspaces spanned by (2v(2,2) =) and Cov(x, y) do not perfectly overlap. For the case where and are binary variables, this assumption is equivalent to requiring that the covariance vectors Cov(æ, 2) and Cov(æ, y) are linearly independent (i.c., not proportional).
We note that, perhaps counter-intuitiveb, SPLINCE is not necessarily equivalent to LEACE colsp(E... ) and colsp( E orthogonal subspaces. Only if those subspaces orthogonal after whitening, SPLINCE and LEACE are equivalent. In that case the orthogonal projection of LEACE then already contains the task-relevant directions colsp( W WW my.y)) in its range. We also note that, similar to LEACE [Belrose et al. 2023). in the case of non-centered data, i.e., E[xx] j O, the optimal affine transformation requires the addition of constant b PPPINCE
E[ar) P P [a.r). Finally, we give a visual illustration of the steps the projection matrix suggested by Theorem
in Figure
Wr
P SPLINCE
P LEACE
yslizel
Plssce
WI:
y=0.=]
y=0l=00
-1
-2
-2
(b)
(c)
(d)
Figure 1: Illustration the different steps for the projection suggested by Theorem I on two- dimensional data. The data (a) is whitened (b). Then, use V(U"V)- to project parallel to WE onto WEz,y- and subsequently unwhiten (c). With LEACE, the En,y altered (d).
3.2 Last-layer linear concept removal with re-training
The first use case SPLINCE we consider linear concept removal applied to the embeddings of a DNN, after which a linear classifier is fitted on the transformed embeddings. This can be useful if it demanded that a predictive model does not make use sensitive concepts, like gender or race.
If we compare SPLINCE with other concept removal methods that guarantee linear guardedness, namely LEACE and SAL (Shao al. 2023), we observe that they are all projections with the same kernel colsp ). They differ in the choice of the range. Interestingly, we find that the predictions
of a linear classifier that trained without regularization on the transformed embeddings are not affected by this choice of the range. In other words, all concept removal methods that ensure linear

---

guardedness will lead to the same predictions after re- re-training a linear classifier without regularization. We verify this empirically in Appendix B.1
This result is formalized in Theorem 2 in which we consider training a model f(x; 0) that only depends on the embeddings æ: and parameters @ through their inner product. Examples of such models are linear and logistic regression, the latter typically being used as the linear classifier re-trained on the embeddings. Before fitting, a projection is applied to the embeddings. We consider two projections that have the same kernel, but different ranges. Theorem 2 shows that, in the case of strictly convex loss function without regularization, both fitted models lead to the same predictions.
Theorem 2 (Equivalent predictions after oblique re-training). Consider observations (xx y) E R x and model f( 0) that only depends on the inputs æ and parameters 0 through their inner product, L.e, [( 0) = f'(x'' f). Suppose we have data {(rx: ys {x))k== which is organized in a design matrix X E RRxxd and Y E y Before fatiing the model, we apply an oblique transformation to the features T. We consider two projections that have the same kernel U E R. but different ranges A, B E Rd We denote the corresponding transformation matrices as PA, PB. and we define A = PAT x E A and æs Pεœ E B. Let L(X0, Y) be a loss function with unique minimizer. Then the following two minimizers
0 = -arg min A(X Y); 0 = arg min C(X80, Y) BEB
(5)
BEA
lead to the exact same predictions. In other words, x;) 0A = x 0 for any I E Idd.
The proof of Theorem is given in Appendix A.2 The intuition behind this result is that there exists an invertible linear transformation between the data after P. and the data after Pg- In other words, the choice of the range does not determine how much (linear) information about the target variable is lost in the oblique projection. This is solely determined by the choice of the kernel.
We point out that the assumption of unique minimizers corresponds to strictly convex loss functions, which is a common assumption for linear and logistic regression [Albert and Anderson 1984). In addition, note that the constraints in equationSjor the parameters to A, B is without loss of generality. Because of the inner product, components perpendicular to the subspaces do not affect predictions.
3.3 When does changing the range matter?
Theorem 2 provides case where SPLINCE will lead the same predictions as other concept removal methods that ensure linear guardedness (e.g., SAL and LEACE). Here, we identify two practical cases where applying projections with the same kernel and different ranges will typically lead to different predictions.
1. When re-training the last layer with regularization: if we include regularization term (such as ||0||2 or [0]l1) in our loss function L. then it no longer exclusively depends on the parameters via the inner product Xe. This will generally lead to x 0 * 0 for any two projections with the same kernel and different ranges.
2. When not re- re-training the last layer: applying the same parameters to projected embeddings that lie in two different subspaces will typically not lead to the same predictions. we consider projections P. and Pg with ranges A and B, then the predictions can only be the same the parameters 0" lie in the orthogonal complement of both PA and PB, i.c.,
'P'00 = x: T
xT TTPPA P8) T 0.
(6)
Since the projections considered in this paper are constructed without knowledge of the parameters, these will typically lie outside the orthogonal complements. Note that this use case is relevant for language modeling, where re-training the parameters of the last layer is typically not feasible in terms of computational resources and/or data availability.
For these cases we expect SPLINCE to outperform other methods that ensure linear guardedness, is designed to preserve task-relevant information. We empirically investigate this in the next section
"Se this link code for the experiments, well an implementation of SPLINCE.
5

---

4 Experiments
This section is structured as follows. We start by investigating classification tasks when the last layer of the model is re-fitted with regularization. Then, in Section 442 we investigate language modeling, when the last layer of the language model (LM) is not re-trained post-projection. Finally, in order to qualitatively assess the effect of the different projections, we apply SPLINCE to black and white image data in Section 4.3 Across experiments, we compare SPLINCE to two other projections: LEACE and SAL (see Section 2 We focus on these projections since, similar to SPLINCE, they guarantee linear guardedness with regards to concept, and only differ in choice of range, Furthermore, LEACE and SAL have been shown to outperform other existing concept-removal
methods such as INLP and RLACE, which may or may not satisfy linear guardedness.
4.1 Classification where the last layer is re-trained with regularization
We focus on two classification problems. First, we use the Bias in Bios dataset professions and biographies from Ar-Araa et al. (2019). We focus on the set of biographies which carry the 'professor' label, Mprof E {0, and scek to remove the concept of whether the subject male or not, Egender E {0, 1}. Second, inspired by Huang et al. [2024), we use the Multilingual Text Detoxification dataset from Dementieva et al. [2024]. We focus on three languages (English, German and French), making the concept-labed 2lang E (1,233 3) non-binary. This dataset consists of texts from users that classified toxic or non-toxic, ytox
10.
Set-up of the experiment: we seek investigate the impact of each projection as the correlation between the task of interest (Uprot, Utox) and the concept to remove (zgender, #lang) becomes stronger. We expect that as the relationship becomes stronger, the difference between SPLINCE and other projections becomes greater. The reason is that stronger correlated labels have covariances with the embeddings that are typically more aligned. Removing the concept is then more likely to also remove information about the task of interest.
To alter the relationship between the task of interest and concept, we create smaller versions of the original datasets, where we vary the extent to which Vproř; Ftox co-occur with respectively Egender: Blang. For the Bias Bios dataset, we vary p(yprof a zgender = a) with a E {0, 1}. iee. the conditional probability that the biography is of a professor and male or not professor and female. For the Multilingual Text Detoxification dataset we vary p(yrox = 1 zlang = 1), e.g., the conditional probability that toxic comment appears in the English language. We balance with respect to respectively Sprof, Yhax- In order to measure how much of the task-relevant information is retained after the projection, we create test set where there is correlation between Vprof yhox and the respective concepts Zgender, Zlang. Additional details on the datasets are given in Appendix C.I
Models and training procedure: for the Bias in Bios dataset, we finetune a BERT model (Devlin al. 2019) to classify the profession. For the Multilingual Text Detoxification dataset we finetune mululingual E5 (ME5) embeddings Wang et al. [2024b) to classify the sentiment. For the BERT model, we add linear layer on top the embeddings of the [CLS] tokens for classification. For the MES embeddings, we add a linear layer on top of the average over all tokens. We apply projections to the last-layer embeddings- the [CLS] token for the BERT model, and the average over all tokens for the ME5 model. Afterwards we re-fit logistic regression with l2 regularization. We tune the strength of the 12 regularization based on a validation set. This entire procedure (finetuning, projection, re-fitting, le penalty selection) is repeated per seed. Additional details are given in Appendix C.2
Results: the results of the experiments for both datasets are given in Figure 2 We focus on overall accuracy as well as worst-group accuracy. Worst-group accuracy is defined as the lowest accuracy for all combinations of the task and concept. A low vorst-group accuracy reflects that a model relies on the correlation between the task and concept in the training data (Sagawa et al. 2020]. For both datasets, as the relationship between task and concept becomes stronger, SPLINCE outperforms the other projections in both accuracy and worst-group accuracy. As the correlation between the task and concept becomes stronger, SAL and LEACE remove a significant part of En,y, contrary to SPLINCE. This is illustrated for the Bias in Bios dataset in Figure 7 in Appendix B.2

---

Bias in Bios dataset
Worst-group accurncy
90%
91%
Accuracy
85%
89%
80%
75%
87%
0.8
plao Egender =Q)
pOpmff Egender
Multilingual Text Detoxification dataset
95%
Worst-gas Worst-group -or--rrup accuracy
80%
Accuracy
60%
85%
SAL
BRM
40%
80%
LEACE
SPLINCE
0.5
0.6
0.7
0.8
0.9
0.5
0.6
0.7
0.8
0.9
Piyoxic
Flang = 1)
p Pood
Zlan
Figure 2: Performance of different projections on the Bias in Bios and Multilingual Text Detoxification dataset. We re-train the last-layer after applying each projection. Points are based on the average over 3 seeds, 5 seeds respectively for the two datasets. The error bars reflect the 95% confidence interval.
4.2 Language modeling
We focus on two language modelling tasks using the Llama series of models [Tuvrron et al. 2023
Grattafiori et al. 2024). First, we use a dataset from Limisiewiez et al. [2024) which we refer to as the profession dataset. Inspired by [Bolukbasi et al. [016]. this dataset contains prompt templates containing professions, which need to finished by the LM (e.g., 'the plumber wanted that'). Each profession has stereotype score Zstereo- This indicates how strongly profession is connected with the male gender through stereotypical cues (c.g., plumber has a high stereotype score, while nurse has low one). Each profession also has factual score ylact, which indicates how strong profession is connected to the male gender through factual information (c.g., waiter has a high factual score, but waitress has low one).
Second, we use the Winobias dataset from Zhao al. (2018. Each prompt contains two professions and pronouns. Prompts marked pro-stereotypical or anti-stereotypical. denoted 2pro-stereo {0, 13. In pro-stereotypical prompts, the coreference links to a profession with the stereotypical gender matching the gender of the pronoun. An example is 'The mechanic gave the clerk present because he won the lottery. He refers to'. In anti-stereotypical cases, the profession's 's stereotypically assumed gender is different from the gender of the pronouns. The task is to finish the prompt with one of the 40 professions, with the correct profession denoted Yprofession E 21, 40). For additional details on both datasets, see Appendix C.I
Set-up the experiments: for the profession dataset, our goal to create an LM that does not rely on stereotypical cues, but on factual information. We estimate the extent to which the LM M relies on stereotypical cues or factual information as follows. Let the/tshe the tokens for "he" "he"/"she" Let t; denote tokens for prompt i, and pm (tnelt.) the probability assigned by a model of the "he" token conditional on the prompt b, We measure the log-odds ratio between the probability of the next token being 'he" or 'she" as
P. P.M (/ine (t:)) PM (fstue (t:)
oddShe/shhe; log
and estimate the linear regression
oddshe/she.. = Estereo, Fstereo,(Bsteree + Stact + â.
(8)
Intuitively, the coefficients indicate what extent the difference in the probability of assigning "he" or "she" can be explained by stereotypical cues factual information Limisiewicz et al. 2024
7

---

For the Winobias dataset, we seek to create an LM that is able to provide the correct profession, regardless whether or not the coreference link is pro-stereotypical. We seek to remove 2pro-storn stereo while preserving the covariance between the embeddings and !profession-
Results: for the experiment the profession dataset, the results shown in Table We report the exponent of the coefficients in Equation 7 as this tells us how more likely the 'he" token becomes relative to the "she' token after one-unit increase in either the stereotypical or factual score. After applying any of the three projections, the extent to which the model relies on stereotypical information is greatly reduced, per the reduction in exp(Pxteroo). The extent to which the model relies on factual information after a projection greatly reduced when applying SAL or LEACE, whereas it increased or preserved after applying SPLINCE.
Table I: Results of applying different projections to the last layer of various Llama models for the profession dataset.
Model
Projection
exp(Batermo)
exp((Brrrt)
Original
3,59
15,71
+SAL
0,80
5,90
Llama 2 7B
+LEACE
0,85
12,14
SPLINCE
0.79
24.27*
Original
3,84
20,2
+SAL
0,84
4,81
Llama 13B
+LEACE
0,88
16,32
SPLINCE
0,81
33,24
Original
3,98
19,02
+SAL
0,87
3,50
Llama 3 8B
+LEACE
0,88
7,68
SPLINCE
0,82
13,43*
Note: the indicates that difference between the factual coefficient of our projection and the factual coefficient of LEACE is statistically significant at the 1 % level according one-tailed t-test. The exponent of the coofficients estimates how the odds ratio changes with one-unit change Rsseroo and Vfact, respectively.
For the experiment on the Winobias dataset, the results are shown in Figure 3 three Llama models, SPLINCE improves coreference accuracy more than the other projections. In particular, it strongly increases the accuracy for anti-stereotypical prompts. In Appendix B.6 results for additional LM's outside of the Llama series.
For two out of
Original Medel
1SAL
LEECE
ISPLINCE
65.9%
CLON
Anti Stcrotppicall..ccooooooooooo Accurnzrr (No
17.2%
13.4%
Anaraay (N)
MuA siu
yy
40%
Moris
Ja
LJme 1B
Liara 13B
Elama 5B
Figure 3: Results of applying different projections to the last layer of various Llama models for the Winobias dataset. The left plot shows the accuracy and half anti-stereotypica prompts. The right plot shows the accuracy on the anti-stereotypical prompts in this test set.
consisting of half pro-stereotypica
4.3 Application to image data
conduct an experiment for the CelebA dataset [Liu al. 2015 that is similar to one from Ravfogel et al. 2022 Kleindessner et al. 2023 Holstege et al. [024]. The goal of the experiment to qualitatively show what features are removed by each projection. The concept to remove whether not someone smiling, denoted Semiling E {0, 1). and seek to preserve whether not

---

someone wears glasses, Mglasses E {0,1). We subsample 10,000 images from the original CelebA dataset such that PYlglasses
Esmillnn
a) 0.9.
In Figure we illustrate the effect each projection on the raw pixels, for several images. SPLINCE accentuates parts of the image that useful for distinguishing images with and without glasses. For instance, it tends to make the areas around the eyes lighter when someone does not wear glasses, and darken when they do. This shows that SPLINCE mitigates the damage the "glasses" features exposed to a linear classifier, despite of the high correlation. We illustrate that this holds on average across the whole dataset Appendix B.3
In Appendix B.5 we include additional results CelebA dataset as introduced here, as well as the Waterbirds dataset Sagawa al. (2020]. SPLINCE performs relatively worse for these vision classification tasks than the NLP classification tasks in 4.1 This gap in performance between the vision and NLP classification tasks is interesting direction for future research.
Original
LEACE
SPLINCE
SAL (Dift)
LEACE (Ditt)
SPLINCE (Diff)
0.1
0.10
0.03
-0.10
0,20
Figure 4: Application of different projections to raw pixel data of CelebA. The first columns shows the original image. The next four columns show the image, after the respective projection. The final three columns indicate the difference between the original image and the image after the projection.
Discussion and Limitations
Theoretically, we show that the range of the ceaectionnpatticlaarly.y whether or not includes Σx.y matter only if one does not re-fit the last linear layer, refit it with regularization term. However, we lack a theoretical understanding that would explain under which conditions does matter, and when it is expected outperform LEACE. In Appendix A.3 provide initial investigation into this question, showing SPLINCE is guaranteed to outperform LEACE when we whitened. Future work should study whether preserving
freeze the last layer and the embeddings main-task covariance is optimal under more general settings. In this paper, we focus on preserving that quantity is intuitively related to main-task performance; however, we note that the SPLINCE objective can be modified to preserve any direction that is not identical to the covariance with the protected attribute.
In addition, a limitation of the SPLINCE objective is that prioritizes preserving Ex.y over min- imizing Px x [ This can potentially lead to distortive changes to the embeddings. We investigate this limitation in Appendix B.4 As the SPLINCE objective sometimes fails when inter- vening in middle layers, we aim to study the adaption of the SPLINCE objective for preserving the directions in the representation space that being used by LM in some middle layer.
While we did not investigate a multi-modal setting (e.g. CLIP, Radford et al.) (202T) ), one potential limitation of SPLINCE is that covariance subspaces might not be aligned across modalities.

---

Finally, see Appendix for a discussion on ethical considerations when applying SPLINCE.
Conclusion
We introduce SPLINCE, a method that generalizes previous concept-erasure methods by provably removing the ability to linearly predict sensitive information, while maintaining the covariance between the representations and another main-task label. Our analysis pins the problem down to pair of geometric constraints-placinne colsp(> (y. to lie its range-and proves that the oblique projector of Theorem minimum-distortion solution under these constraints. Experimentally. SPLINCE tends to better preserve average and worst-group accuracy on the Bias in Bios and Multilingual Text Detoxification able to
colsp(2..2 in the kernel of the projection while forcing
the unique
tasks when the task-concept correlation is high. In influence stereotypical bias while preserving factual gender information; and in most models, is better in preserving LM's ability to perform co-reference after debiasing in the Winobias dataset. Future work should formalize whether maintaining the covariance with the main-task translates into main-task loss guarantees and develop variants over the SPLINCE objective that impose weaker distortion when applied to earlier hidden layer, or performs better for vision datasets.
modelling setting,
References
A. Albert and J. Anderson. On the Existence of Maximum Likelihood Estimates in Logistic Regression Models. Biometrika, 71(1):1-10, 1984. ISSN 0006-3444. doi: 10.2307/2336390.
Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models mediated by single direction, 2024. URL https
//arxiv. abs/2406 11717
Dilyara Bareeva, Maximilian Dreyer, Frederik Pahde, Wojciech Samek, and Sebastian Lapuschkin. Reactive model correction: Mitigating harm to task-relevant features via conditional bias sup- pression. 2024 IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops (CVPRW), pages 3532-3541, 2024. 269148614
URL httth:tt/////t.. /attps://api www.stteamcncooolaaoo.ooooococ .ot. Irr/CorpusID:
Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguis- fics, 48(1):207 March 2022. doi: 10. 10.1cl//coli 00422. URL httpss://cclanthhllsle
org/2022.co c1-1.7
Nora Belrose, David Schneider-loseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. Leace: Perfect linear concept erasure in closed form. In A. Oh, T. Nau- mann, Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances Neu- ral Information Processing Systems, volume 36, pages 66044-66063. Curran Associates, Inc., 2023. URL httpp: httops://prooceedingg. (0062216661sda///66557//22333333
wtwpeeeeeednnnn.nee.eee..eeem neurins. wwtppppeppappppppeppppae.ee.eeee
Tolga Bolukbasi, Kai-Wei Chang. James Zou, Venkatesh Saligrama, and Adam Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS' 16, page 4356-4364, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 9781510838819.
Hongliu Cao. Recent advances in text embedding: A comprehensive review of top-performing 01607
methods on the mteb benchmark, 2024. URL
Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in bios:
case study of semantic representation bias high-stakes setting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* '19, page 120-128, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450 1781450361155 doi: 10. 145/ 3287560.3287572. URL https://doi. http:///oooooog//111 111.1...11111111111111 3287572
10

---

Daryna Dementieva, Daniil Moskovskiy, Nikolay Babakov, Abinew Ali Ayele, Naquee Rizwan, Frolian Schneider, Xintog Wang. Seid Muhie Yimam, Dmitry Ustalov, Elisei Stakovskii, Alisa Smirnova, Ashraf Elnagar, Animesh Mukherjee, and Alexander Panchenko. Overview of the multilingual text detoxification task at pan 2024. In Guglielmo Faggioli, Nicola Ferro, Petra Galuščáková, and Alba García Seco de Herrera, editors, Working Notes of CLEF 2024- and Labs of the Evaluation Forum. CEUR-WS.org 2024.
Conference
Sunipa Dev, Tao Li, Jeff M Phillips, and Vivek Srikumar. OSCaR: Orthogonal subspace correction and rectification of biases in word embeddings. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau tan-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5034-5050, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.118333v1/222/ http:/22222mmmmmmmmmm/mmm emnlp-main. 411. URL https: httttss:/aclaathoolltt/
2021 emnnp-mminn 4117
Jacob Devlin, Ming-Wet Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 12.18855/1//N1-114 1423/
org/
Yanai Elazar and Yoav Goldberg. Adversarial removal of demographic attributes from text data. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ichi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 11-21, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10.18653/v1/DI8-1DD 1002. URL
1002/
Amir Feder, Nadav Oved, Uri Shalit, and Roi Reichart. Causalm: Causal model explanation through counterfactual language models. Computational Linguistics, 47(2):333-386, 2021.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang. Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archic Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino. Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lec, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary Laurens der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh. Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathicu Rita, Maya Pavlova, Melanic Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng. Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Scohyun Sonia Kim, Sergey Edunoy, Shaoliang Nie,
11

---

Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan. Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vitor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao. Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence. Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang. Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li. David Adkins. David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang. Due Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpem. Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Tufanov, Ilias Leontiadis, trina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam. Japhet Asher, Jean-Baptise Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong. Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko. Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark. Mike Macey, Mike Wang. Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White. Navyata Bawa, Nayan Singhal. Nick Egebo. Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng. Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang. Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto. Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang. Sinong Wang, Sncha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanic Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad lonescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang. Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniy Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi,
12

---

Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary Vito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The Ilama 3 herd of models, 2024. wttppttss//axxii.rrrrrr.. abo/ 2407 21783
Clément Guerner, Tianyu Liu, Anej Svete, Alexander Warstadt, and Ryan Cotterell. A geometric notion of causal probing, 2025. URL http::///axwvvrro//aaae/ 2307 15054
Pantea Haghighatkhah, Antske Fokkens, Pia Sommerauer, Bettina Speckmann, and Kevin Verbeek. Better hit the nail on the head than beat around the bush: Removing protected attributes with single projection. In Yoav Goldberg, Zoritsaa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8395-84 8416, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 00.18653/v1/222/ wew.lp-emlpppp.eee 55m URL 575/
org/ 2022. emnlp-main
Floris Holstege, Bram Wouters, Noud Van Giersbergen, and Cees Diks. Removing spurious concepts from neural network representations via joint subspace estimation. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 4 International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 18568-18610, PMLR, 21-27 Jul 2024. URL httts://prooceedings mlr. arress//2235holsteeee html
Zhiqi Huang, Puxuan Yu, Shauli Ravfogel, and James Allan. Language concept erasure for language- invariant dense retrieval. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. pages 15261-13273, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. 18653//1/0024..mmlmmmmoo 736. URL http::::/aaaaaatooooooooooooooee 2024 emmlp-main. 736/
Mojan Javaheripi, Sébastien Bubeck, Marah Abdin, Jyoti Aneja, Sebastien Bubeck, Caio César Teodoro Mendes, Weizhu Chen, Allie Del Giorno, Ronen Eldan, Sivakanth Gopi, et al. Phi-2: The surprising power of small language models. Microsoft Research Blog, 2023.
Albert Q. Jiang. Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Ana Lachaux. Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL BtLhttts:://tt
abs 2bs0/ 06825
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. arXiv preprint arXiv:2204.02937 2022.
Matthäus Kleindessner, Michele Donini, Chris Russell, and Muhammad Bilal Zafar. Efficient fair pea for fair representation learning. In Francisco Ruiz, Jennifer Dy, and Jan Willem van de Meent, editors, Proceedings The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pages 5250-5270. PMLR, 25-27 Apr 2023. URL http:ttts:/p/rcpeddhiiiie
mlr. press/ pwwps//w0///////pppp 0e2200/kleiindssseee222222 hhmml
Abhinav Kumar, Chenhao Tan, and Amit Sharma. Probing classifiers are unreliable for concept removal and detection. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 17994-18008 Cur- ran Associates, Inc., 2022. URL 002/5iiee22ff000000cooaaaas/cle3 cww.:eeeeeeepppeeeeoooooorroeeo Conference pdf
nww.ips..p.... htc/paper_ffleespppprr/
Tomasz Limisiewicz. David Mareček, and Tomás Musil. Debiasing algorithm through model adaptation, 2024. URL htttptt////xxxi wtt.v//rrg//bbb/1..119999 2310.
Tomasz Limisiewicz, David Mareček, and Tomáx Musil. Dual debiasing: Remove stereotypes and keep factual gender for fair language modeling and translation, 2025. UtLpttts:://axxi org
abs/2501 10150
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In Proceedings of International Conference Computer Vision (ICCV), December 2015.
13

---

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL https 1aazzv..rg/aa///111.05555
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the ge- ometry of large language models. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp. editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 39643-39666, PMLR, 21-27 Jul 2024. URL https
//proceedings.com mlr http://inn..m....es////eeeae 2ttps///222//pak22.cc2m html
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages URL https::/proceeeiiggg .mlr. prwss/ http://v1//rraadd///e/ html.
PMLR, 18-24 Jul 2021.
Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. Null it out:
Guarding protected attributes by iterative nullspace projection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7237-7256, Online, July 2020. Association for Computational Linguistics. doi: 10. 10.18653/w1/202..acc maw.://..000..mm.......441... URL https
//aclanthology. org/ 20/g/2020.o0 aa20.acc--am
acl - main. 647
Shauli Ravfogel, Michael Twiton, Yoav Goldberg, and Ryan Cotterell. Linear adversarial concept erasure. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Muchine Learning Reseurch, pages 18400-18421. PMLR, 17-23 Jul 2022. URL httttttt//pocceediiiiee
mnr. pttps///11///aaaaeeee22222 html
Shauli Ravfogel, Yoav Goldberg, and Ryan Cotterell. Log-linear Guardedness and its Implications. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 9413-9431, Toronto, Canada, 2023. Association for Computational Linguistics. doi: 10. 1/ 2023.acI-ll long..0...
Shauli Ravfogel, Anej Svete, Vésteinn Snæbbarnnason, and Ryan Cotterell. Gumbel counterfactual generation from language models, 2025. URL https:://txti::i //apiiv///xxvv. org/aoa///////eeooooo/ooooeaeeet abaaa/aa/1111 0t///2111..1 2411.07180
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In International Conference on Learning Representations (ICLR), 2020.
Shun Shao, Yftah Ziser, and Shay B. Cohen. Gold doesn't 't always glitter: Spectral removal of linear and nonlinear guarded attribute information. Andreas Vlachos and Isabelle Augenstein, editors, Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 161 1611-1622, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. doi: 118653/112022.eeaclmmm leacl macl-maain.118
URL htttps://aclanthologg.. wrg.lllooyyoog////// org/2023
Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. Contrastive learning for fair representations. arXiv preprint arXiv:2 109. 10645, 2021.
Enze Shi, Lei Ding, Linglong Kong, and Bei Jiang. Debiasing with sufficient projection: A general theoretical framework for vector representations. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume I: Long Pupers), pages 5960-5975, Mexico City, Mexico, June 2024. Association for Computational Linguisties. doi: 10. htwp://22222..alloooog...3 URL hwtp:httt///eet httttp::/aclaanhoooogy www.etthhoooooorooooeooo org/ /024 naacl-long.c 332/
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cris- tian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,
14

---

Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, lliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023. URL htt.hhtp::// arxiv. org/ abs 09288
Liang Wang, Nan Yang. Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei, Improving text embeddings with large language models, 2024a. URL https://arrxvvoor//bbpll 00368
Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multi- lingual e5 text embeddings: A technical report, 2024b. URL https:///aaii hargg/bb/244000 05672.
Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pictro Perona. Caltech-uesd birds 200. Technical Report CNS-TR-201, Caltech, 2010. URL
https://www.viii
caltech,
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. Huggingface's 's transformers: State-of-the-art natural language processing. CoRR, abs/1910.03771. 2019. URL
httt::/arxiv.. org/
Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig. Controllable invariance through adversarial feature learning. Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, page 585-596, Red Hook, NY, USA, 2017.
Curran Associates Inc. ISBN 9781510860964.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES '18, page 335-340, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781450360126 doi: 10.1145/3288721.32788888 3278779
hRLtts/s:///at org/oogg//// 10. 1145/3278721
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai- Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Short Pupers), pages 15-20, New Orleans, Louisiana. June 2018, Association for Computational Linguistics. doi: 10.18653/v1ml 1/N10/11/N1-0000 URL htt.ittt///alaaatoolloggg org/ 2118-2003/
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Antonio Torralba, and Aude Oliva. Places: An image database for deep scene understanding. Arxiv Computing Research Repository (CoRR). abs/1610.020553 2016. URL http: org/ abs/aass111 02055
15

---

A Proofs of theorems
A.1 Proof of theorem
We will prove Theorem I by making of basis tailored to the problem. In the end will transform the resulting formula back to the basis-independent formulation of equation
Let m = rank rrank( with 1 & m I d. Let W = colsp(W) be the subspace in R in which has non-zero variance. Without loss of generality, we will define basis for where first m coordinates of a lie in W. The last 1 = d - coordinates lie in its orthogonal complement W- Our a can now be written as
I IR E
We will use d; E IR denote elements from the first coordinates, and Fs E R denote elements from the final coordinates.
also assume that this basis orthonormal with respect to inner product M, such that: Mr = Eie 0g22 for fixed o1.-., Ozd > 0. Creating such a basis is always possible by standard orthogonalization procedures, now restricted the coordinates of the respective subspaces W and W- As a consequence of this, the optimization problem defined in Theorem can be decomposed in independent optimization problems, one for each term in the sum that corresponds the norm (squared), i.e., one for each coordinate of æ. To be more concrete, the optimization problem becomes for E {1.....d)
arg min E ((Px); x:)?] subject to Cov((Px); z) = O, PeRaxa
subject to Cov((Par;; y) Cov v),
(10)
where 2; E R denotes the th component of x and (Px), E the ith component of Px. The weights a1,,.. Og are left out, they become irrelevant if manage find the minimum for each 2,.
Lemma 1. Let
Om. Of,1
P
(11)
0r.m
where E mm and where 0m,i an (m x -)-matrix of zeros and the other zero-matrices are defined similarly, A solution to the optimization problem. for (1,..,,),
arg min E ((P)) 2,)² subject to Cov 2) =0, Pepaxa
subject to Cov y) Cov (z:,y),
(12)
corresponds then via equation 77 a solution of the original optimization problem of Theorem 7
Proof. We start by dividing P in four block matrices,
P B C
P
(13)
D
where P. E mm B mm
C gllmm D gxxl We now proceed to determine these matrices, starting with a solution for and D. We do this by solving the optimization problem defined in equation 10 for the final rows. For E (l..., ..., d), we write
(Px); = Costy +
Σ Dpstis j=m+1
(14)
1=1
where p = l 1. We use the indexation via p because we use the rows p E {1 {1,.... l] of the matrices C and D respectively to represent (Px), The objective in equation 10 for E {l.... d}
16

---

AW, where and corresponding p can be written as
27
m
((Pz): - 2,)²
E
Copttt +
E
Dostty -b
(15)
j=m+1
= E
(16)
j=1
= (CCoov(ē,))) 2)(²)
(17)
where in the second equality we used that ã; = 0 almost surely and in the final equality we used We note that the values for Dp. do not matter for the objective. Furthere, because Cov (ær, @b) is p.s.d., we can achieve the minimum of equation 17 by setting C = 0. For simplicity, we also set D O. This then also trivially satisfies the kernel and range constraints for the components i E {l....,} d).
that E. [[:
For i E {1, m). we can write
(Px); =
Σ ==1
Agjo +
Σ f=++1
Bay
(18)
can set 0 for the same reason as we chose = The objective and constraints for P are then as in equation] 12 This concludes the proof of Lemma
In order to simplify the remaining objective for P in Lemma 1 we write P W Σe.a Rmx is full-rank, symmetric and p.s.d., and thus invertible. Because of this, optimizing for P is equivalent optimizing for A. Note that in this notation,
Σg. 0m,1 0g
Eg.
(19)
0r,1
an we can write the whitening matrix W, and its Moore-Penrose inverse
W
0 Ot.
W
Omn!! Or.1
W
=
W
0
Om.!
O
(20)
Using that can write ã;
m Ww); W i.j
(Wt),)
(21)
j=1
17

---

the remaining objective becomes, for ioris E m...,....
E ((Px); -., = E (Pā)) x
E
Ais -
2
Var
Σ (Ass W())
+ E
Σ
(Ais W (W
Var Σ (Ai. (W))
m
Σ (ĀLa WT.)(A - Wor))o (Wa).)
h=lk=1
= (A (A- W w- (A- : (A-w-1) --
(22)
where we used that B[(W)),] = 0 and Cov( Wä, Wæ) = Im by definition of the whitening matrix.
The constraints on P in the optimization problem in equation 12 translate into constraints on and the optimization problem can be recast
À arg min AcC,nea
Σ A
W
(23a)
i.
where
C1 = M ME R
WM 0m
(23b)
C2 MERTX WMWE WEan
(23c)
With more than one objective and two constraints, this a constrained multiple optimization problem. We have seen before that it can be decomposed in separate constrained optimization problems. Each of these problems has a convex objective function and linear constraints, making the optimum À * uniquely defined.
The constraints C and C2 can be interpreted as follows: À must be such that the columns of WE,,. are in the kernel of WÀ and that the columns of WEa.y are eigenvectors of WA. This can be achieved by means of oblique projection. If we define the following linear subspaces,
ú colsp WE
(24)
û ün (oolsp WE + colsp (W
(25)
D = colsp (W + +ū
(26)
and define Ü and V the matrices whose columns an orthonormal basis of ú and V respectively, then
P ( (ÚTV) ÜT
(27)
the transformation matrix of an oblique projection whose kernel formed by the columns of
W WE,, and whose range include the columns of W Da.y. The latter means that the columns of
W WEgy are eigenvectors of P Pobl-
We claim that any À E C, nC2 can written as BP obl where B obeys the second constraint, i.e.,
B E C2. This identification is not unique, as multiple B lead the same A. We formalize this claim in the following lemma.
18

---

i=1 Lemma 2. Let us define
Bpani =
BBPsl obl
B C
(28)
Then Bpa Cc
Proof. It is obvious that Bp S C1 n C2; so we focus on proving that C₁ n C2 S Bpas. For this, take an arbitrary M E Ci n C2. Let
colspl WE colsp Waa.). W1, W2, wik
be basis of I. where the wj E IRm are mutually orthonormal and orthogonal to WEa, and WE WEs.y We then define matrix B E mmxm in terms of its action this basis, i.e.,
BWE EF = 0, BWE, = WWIWEa. Bw, = Mwj,
j = 1,2....
This implies that M BP and that B C2. This concludes the proof of the lemma.
L
Lemma Denables us to reformulate the optimization problem of equation as follows. Define the set
[ w-) BB W w- T m
8'
arg min
(29)
BEC₁
This is set of solutions to a different constrained optimization problem than equation 23 All solutions equivalent in the sense that for any Bi, Bi B* have that B obi Seen optimization for BPabi, the objective convex and the constraints linear, making the optimum unique. Hence, À Pol for any B E 1
B obl-
Now, we claim that W-1 E B" The argument is that W-1 is solution to the unconstrained equivalent of the optimization problem equation 29 and, conveniently, also obeys the constraint C2. To see this, define
C. = (BPasi w- (BP w-1) T
(30)
for E {l,...,m) m), and take the derivative of the loss function with respect to elements of B,
ac:
(B-Ww1) Pobl) i.
1-
(31)
where used that Pobl is idempotent. One solution these first order conditions is B W-1. It also obvious that W e C2. Tracing the proof backwards, conclude that
p
W-LP
obl
W
O
(32)
O
0g,1
solves the original constrained optimization problem Thhoree)
!
This expression is specific for the basis we chose at the beginning of the proof. If we let be the matrix whose columns the (orthonormal) vectors of the basis and define
V - S V
U = S (!)
(33)
then we can write this result basis- insiiiindeeendent way as
n" w
Pobl
0m.1
W
w W W.
with Poni (UUT))
(34)
Oz.m
Oz
This corresponds to P
P'PlINCE in equation 4 and concludes the proof of Theorem |
19

---

A.2 Proof of theorem 2
In order to prove theorem 2 we first prove the following Lemma.
Lemma 3. Let PA.Ps Raxd be (not necessarily orthogonal) projection matrices P PA
and P = Pg with the same kemel Ker(P.) Ker(PB) = U. Set A == Range PA), = Range(Ps).
(ange(PA))
Define
F:A-+ B,
F(P Pgæ, Yx Rd.
Then F is linear isomorphism: it is well defined, linear, bijective, hence invertible.
Proof. We prove the Lemma by showing well defined, linear and bijective.
Well defined. If P PAY. then P PA(# y) O, x y l Ker(P and therefore Ps(æ y) = 0, i.. ((P.x) F(P
Linearity. For 21 PAT11 22 PA22 and R:
F(z1 + 22) = Ps(01 + x2) Pιœi + PBæ2 = F(z1) + F(22),
F(azı) (s(az1) aP g (z1).
Injectivity. If F(21) F222), then Pεæ Pι and (æ) - x2) E Eu = Ker(Ke) 21 -2.
Surjectivity. Both A and B have dimension d - dimul; linear, injective map between finite dimensional spaces of equal dimension is automatically surjective.
Thus F is linear isomorphism.
Proof of Theorem 2 map
Because Ker(PAl Ker(PP) U1 Lemma provides an invertible linear
F:A
B,
F(P.a))
PBæ.
Step 1 - Transferring parameters.
For any A E A define
Og F T
where F denotes the transpose of the inverse map F-1 Then, for every E Rd,
0
x 0
(F 1 TO
so the two parameter vectors yield identical predictions.
Step 2 Empirical minimizers. Let 0 == arg minge (( Y) be the (unique) minimizer
over A, and set 0% := F-TT', TO Because Xg FX the fitted predictions match:
X F X
Hence both parameter choices achieve the same empirical loss value.
Step 3 Uniqueness over B. Since L has unique minimizer B and 0% attains the minimal loss, it must coincide with the optimizer in equation Therefore, for every input Rd..
x T
which proves the theorem.
A.3 Excess risk of SPLINCE vs. LEACE without re-training the last layer
In this section, we provide two theorems that show conditions under which SPLINCE guaranteed to outperform LEACE when the last layer is frozen (e.g. not re-trained, contrasting the set-up of Theorem 2h. In Theorem we prove that for linear regression with whitened data from any distribution, SPLINCE does not degrade the performance of frozen last layer. In contrast, applying
20

---

LEACE may lead to an increase in the loss. In Theorem we prove similar statement, but for the case where the embeddings follow standard multivariate Gaussian distribution.
We emphasize that both theorems are limited in their applicability.. we generally would not expect last-layer embeddings to be whitened or follow standard multivariate Gaussian distribution. Further work is required understand more general conditions when SPLINCE might outperform LEACE.
Theorem 3 (Excess-risk of LEACE and SPLINCE in regression setting). Let X E IR be whitened data with Σz. = La and E|a] = Og. Assume the response model Y X,3 + e, where E[E] On. Var(E) = 0² and Ele X] = 02.-
Let Q = Ls z. where P= projects onto the subspace spanned by concept variable z; let QUPIINEE = P be projection satisfying Σx.y E (Ranee().. Then:
1. The excess risk of LEACE is greater than or equal to zero, A A 2
2. The excess risk of SPLINCE is zero, (SPULNCC)) = O.
Proof. For any square matrix define the risk
R(Q) = E((Y XQ 33]
and the excess risk
AR(Q) R(Q) R(Ia).
Using Y = XB + we have
Y Y- 3= x3 - -
= QB
(1)
Hence
R(Q) E (c X(I. -Q (3)) (c X(I. - Q ]0]]
= Ele E) +2 EE [[eTT([, Q }} +B B (Ia Q)X X(I. - x(Ia-QT)a)
02 3T (I
I Q 1
0² [I Q 3
(2)
because Σx,x Id-
Subtracting R(Ia) = x² from (2) yields the expression
AR(Q) = ||(T Q 01181
Take Q QLEACE = Ia Ps. Since Ps is projector, (Ia (I)= Ia - Pz; thus
AR(QLEACE) = |P23113
Unless P.,3= = O₂ (the degenerate case where (3 lies fully outside the concept subspace), this quantity strictly positive.
Let Q QSPLINCE -PT By assumption Ex.y Rango(P), and under whitening Exy. Hence
P3 and (Ia-P 3 (Ia 3 Og. Applying the boxed identity, AR AR(Q
Theorem 4 (Excess-risk bound of L.EACE and SPLINCE
rngistic-regresssion setting). X Baxd contain i..dd. rows x ~ N(O,,, Em..)). Assume the conditional model
P(y= x) xx B),
g(s) = 1+e-
with true parameter 3 E Rdd
21

---

Define Q = Id - Ps. where P= projects onto the subspace spanned by concept variable 2; define QULIINEE = P, where P is an orthogonal projector satisfying Σz.y E Range(P).
Let A Q) = Eie(x' Q" (.9) E(x B,y]] denote the (population) excess logistic risk, with (8, log(1 + ys.
Then:
1. (LEACE)
1 P= Ex, PzB.
0 < A(QUEACE)
In the whitened case Ex, La this reduces to (U(UEE)))) 2 P
2. (SPLINCE) Dx A(QMIINCE). 0.
Proof. Define
: x B,
8: x: Q
8(x) = 8-= === =Q - -aaB3 =x (La = (Ia-Q))(-P Q-)(33).
Three facts about the logistic loss.
Fact 1. (s, (ss) y,
Fact 2. (s) = (1 o 1
Fact 3. By the value theorem, for some 0 = (xr E (0, 1),
82 (s 08, v)
((s.)) {( +) +
Taking expectations and using Fact 3,
A (Q) = E (E(( 3) - [ss
(35)
(s + [ 65, v)
(36)
Using Ely æ] = a(s) (by model assumption),
pl E' (s. = F. (a() B[((()) y 2r] = 0.
(37)
Fact says 8" < 1. hence
1
B(820()) s P
(38)
Because x ~ NNOO,, Ew,x),
E [ B (I. Q) Ex me (La DB.
(39)
Thus
0 < (Q) < 3 - (4e - -
(40)
For Gaussian x and differentiable g, Stein's lemma states E[g(x) x] = Σz. E[Vg(=)). Taking
g(x) = oxx" () (a scalar function) gives
En.y = E(zy) = Exx [s)) = Σn. (o(())
(41)
Exa 3 B[((s)(1 o(s))]
(42)
22

---

Because C' > O, we can solve for the true parameter:
1 C
3
E Em
(43)
Substituting yields for any Q
(Q)))= S
T D a.y u.a (Ia Q) ) Dx.y
(44)
8C
Take Q QLEACE = Ia Ps. Since (Ia - Q) = P= a projector,
(QQuACE)) I
P P.EEN,
8C
Unless P- P=Em.y = Og (degenerate), the RHS is strictly positive, proving the first half of the theorem.
Let Q QPPLINCE P where PE.,y Ez.y- Then (Ia - - (Is P)E = 0, so the right-hand side of equation 44 is zero. Because Ar QQ) by definition, (Q(UPIINC)) = 0.
B Additional results
In this section, we report several results addition to the experiments described in Section
B.1 Comparing the projections for different levels of regularization
In this subsection, investigate how the difference in performance between SPLINCE, LEACE and SAL changes as a function of regularization. We use the Bias in Bios dataset and conduct the experiment outlined in Section 4.1 but instead of selecting the la regularization based validation set we fix the level or regularization. The results of this procedure are shown in Figure 5 and [6] As we lower the level of le regularization (e.g. 0.0001) the difference between the methods becomes indistinghuishable from zero.
2-1
4.0%
2-0.1
2=0.01
2.0%
2=00001
Worst- group accumey
5.0%
Ascuracy
0.0%
I
0.0%
-2.0%
-5.0%
4.0%
-10.0%
0.6
0.5
0.8
P Faunder
PO Feesda =0)
Figure 5: The difference between the SPLINCE and LEACE projections on the Bias in Bios dataset for different levels of la regularization. We show the difference (worst-group) accuracy of SPLINCE minus the (worst-group) accuracy of LEACE. We re-train the last-layer after applying each projection. Points are based on the average over 3 seeds. The error bars reflect the 95% confidence interval.
23

---

8.0%
2=1
15.0%
2-0.1
6.0%
2-0.01
10.0%
2=0.001
4.0%
Worst group accuracy
2-0.0001
5.0%
Accuracy
2.0%
0.0%
0.0%
-2.0%
-5.0%
-400%
-10.0%
-6.0%
-15.0%
0.6
0.9
0.5
0.7
0.8
Pr Faender
P0 Fgendo
Figure 6: The difference between the SPLINCE and SAL projections on the Bias in Bios dataset for different levels l2 regularization. We show the difference (worst-group) accuracy of SPLINCE minus the (worst-group) accuracy of SAL. We re-train the last-layer after applying each projection. Points are based on the average over seeds. The error bars reflect the 95% confidence interval.
B.2 Removal of covariance for the Bias in Bios dataset for different projections
In this subsection, we briefly investigate the effect of different projections on the extent to which Ex N..e is preserved for the Bias in Bios dataset.
To quantify the extent which Ex..oort is preserved, we measure the ratio of the squared l2 norm of the transformed covariance PE after the projection P and original covariance Mn, Mpeo" Figure
7 shows the effect of changing the conditional probability p(Uprot For SPLINCE, by design, Ea.yport is preserved regardless of P(Pprof
Agender a) on this ratio.
Σgender a). and the
ratio remains 1. As plyprof a &gender a) increases, SAL and LEACE lead to removal of
Ee Mpror For instance, at p(prrof = Ægeuder = a) 0.9, after LEACE and SAL the ratio between
the transformed and original covariances become respectively 0.07 and 0.001.
LEACE
SAL SPLINCE
0.5
0.9 0.6
0.7
0.N
poyat @1 Rgeoder
PE m. Mprof
Figure The ratio
as a function of the relationship between yproff Zgender
Mprof
B.3
The mean difference between the original and transformed images for the CelebA dataset
To verify that the dynamics illustrated in Figure 4 hold across images, we measure the average difference between the original image before and after a projection. This is shown in Figure & for all combinations of zamiling and Vglasess. For individuals with glasses & not smiling, SPLINCE heavily accentuates the glasses by (on average) making them darker. For individuals without glasses smiling, SPLINCE makes the area around the eyes lighter.

---

SAL
LEACE
SPLINCE
Gioseos, Smillnn
Giooves, net nnttsmiiie smiilingg
No glasses, not smiling No glasses, Smilling
-04
Figure The mean difference between the original image and after a projection for every combination of Asmiling: Vglasses
B.4 Applying the projection to multiple layers
Previous work suggests to transform embeddings in multiple, earlier layers in order amplify the effect of a projection (see, for instance [elrose et al. [2023), or Limisiewwcz et al.] [2024] for an example where parameters adapted via projection). In this subsection, repeat several of the experiments in Section 4 for different layers.
Bias in Bios: we apply the projections (SAL,, LEACE, SPLINCE) to one of the 5 last layers of a BERT model. this case, we do not re-train the subsequent layers. The accuracy after this procedure, per layer, is provided in Figure 9 and for worst-group accuracy in Figure I0 For later layers, similar to the results reported in Section 4.1 SPLINCE outperforms the other projections when the conditional probability p(Yprot
Zgender a) increases.
Per layer, we report the Σ La llz and Eg ll2 in respectively Table 2 and In the earlier layers (7-10), both " lll and Ee ll2 are relatively low, indicating relatively little covariance between the embeddings and Zgender, Uprof- As Ln, ll and Dm lla increase in later layers (1 (11-12), the difference between the projections also becomes clearer.
Table 2: The [x Agender 1/2 for the biography dataset per layer
Table 3: The EE Upror lla for the biography dataset per layer
Zgender a)
pfyprof a Zgender
Q.)
Layer
0,5
0,6
0,7
0,8
0,9
Layer
0,5
0,6
0,7
0,8
0,9
7
0,30
0,33
0,33
0,33
0,32
0,12
0,14
0,18
0,23
0,28
8
0,42
0,44
0,44
0,43
0,43
0,11
0.14
0,21
0,28
0,36
9
0,33
0,34
0,35
0,35
0,36
0,09
0,12
0,18
0,24
0,31
10
0,34
0,37
0,36
0,33
0,34
10
0,19
0.25
0,27
0.26
0,31
11
1,07
1,14
1,10
1,09
1,02
0,45
0.60
0,71
0.89
0,94
12
1,37
1,45
1,47
1,52
1,81
12
0.57
0,72
1,02
1,36
1,85
25

---

Layer
92%
9256
91%
915-
Accuracy
90%
90%
8956
8956
SAL
68%
LEACE
SPLINCE
875
0.5
0.6
0.7
0.31
10.5
0.5
0.6
0.7
0.81
0.9
PO Egender
PO Egender
Layer
Layer 10
90%-
Ascuracy
8056
PUpel
Rgender =
P0 Egender ()
Layer
Layer
0.56
9256
91%
Acquricy
90%
Accuracy
90%
3956
89%
855-
87%
8856
0.5
0.6
0.7
0.8
0.9
0.5
0.8 P'per Eyenter ())
0.6
0.7
0.9
POpoot Egender
Figure 9: Average accuracy for different projections the Bias in Bios dataset, for each of the layers preceding the last layer. We do not re-train the subsequent layers after applying the projection. The points the the average 3 seeds. The error bars reflect with the 95% % confidence interval.
26

---

Layer
90%6
85%
88%
Worst-gne gost-grruppP Accuricy
Wonst-grup Accurccy
8056
8656-
8256
X256-
80%
SAL
75%
7856-
LEACE
70%
SPLINCE
76%
0.7
0.3
0055
0.5
0.6
0.7
0.8
0.9
PO Egender
Egender
Layer
Layer 10
90%
90%-
88%
Wonst-ghot group Accuracy
Moost-grnuppp.. Accuracy
855-
81%
82%
80%-
80%
785
75%
P0pel
Reender =
P0 Egender ()
Layer
Layer
90%-
8856
Warst groug Accuracy
grougg Accuracy
88%
8656
84%
R25
80%
80%
78%
78%
0.5
0.6
0.7
0.8
0.9
0.5
0.6
0.7
0.8
0.9
P0poo Egender
POper Eyenter ())
Figure 10: Worst-group accuracy for different projections on the Biography dataset, for the 5 layers preceding the last layer. We do not re-train the last-layer after applying the projection. The points are the the average over 3 seeds. The error bars reflect with the 95% confidence interval.
27

---

Profession dataser: When applying the projections, we start by the first layer in the sequence of layers where we alter the embeddings, After determining the projection for the embeddings this layer, we subsequently determine it for the next, taking into account the projection of the previous layer. Table 4 shows the result of this procedure on Llama 2 7B. It remains the case that after SPLINCE applying SPLINCE to multiple layers, the model relies to greater extent factual information than when using SAL or LEACE.
Table 4: Results of applying different projections to different layers for the profession dataset on Llama 2 7B.
Model
Layers
Method
exp( (astero)))
exp(Branc
Original
3,59
15,71
SAL
1,14
4,07
Last 3
LEACE
1,04
14,30
SPLINCE
1,00
37,94
Original
3,59
15,71
SAL
0,86
5,29
Llama 2 7B
Last 5
LEACE
0,63
14,35
SPLINCE
0,64
15,09*
Original
3,39
15,71
SAL
1,18
6,48
Last 9
LEACE
0,90
26.12
SPLINCE
0,87
78.17
Note: indicates that the difference between factual coefficient of our projection coefficient of LEACE is statistically significant at the level according to one-tailed t-test. The exponent of the coefficients estimates how the odds ratio changes with a onc-unit change in Esterco and Vfact, respectively.
factual
Winobias dataser: Similar to the Profession dataset, we apply the projections to embeddings of subsequent layers, taking into account the projection the previous layer. As illustrated per Figure II we observe that the performance of SPLINCE decreases as we apply it to more layers. Potentially, IR This result highlights
this is because of the (large) dimensionality of Ex, Mpprarnnttttn
potential limitation of our projection.
Original Model
-SAL
+LEACE
-SPLINCE
65%
60%
$7.2%
3tt
$P.M6
Slate
55%
5085
35%
Las Layer
Layes
Las Layevx
Layes
Figure 11: Results of applying different projections to multiple layers on the overall accuracy for the Winobias dataset for the Llama 2 7B model.
B.5 Additional results for vision datasets
In this section, we briefly investigate the performance of SPLINCE and other projections on two vision datasets: Waterbirds (Sagawa et al. (2020) and the CelebA dataset described in Section 4.3
Similar to the experiments in Section 4.1 we alter the extent to which the main-task co-occurs with the concept. For the Waterbirds, we seek to predict whether or not a land or waterbird present (Ybird E {0, 13), while removing the concept of the land or water background, denoted Pback E (0, 1). For the CelebA dataset, we alter the extent to which an image with glasses Yglasess co-occurs with Semiling. Details the datasets and training procedure can be found in Appendix C
28

---

We also compare SPLINCE to two benchmark out-of out-off-dssie distribution (OOD) generalization methods. First, deep feature reweighting (DFR, Kirichenko al.] [2022]) where a model is trained on subsampled dataset, where each combination of the main-task and concept occurs with equal probability. We implement the version of DFR where the subsampled dataset comes from the training data, referred to DFRTR in Kirichenko et al. [022). Second, we apply group distributional robust optimization (GDRO, SSaawaa et al. (2220) ) to the last layer. Details on the implementation both methods can be found in Appendix C.2 We compare SPLINCE to these two methods for the vision datasets, as well as the NLP classification tasks outlined 4.1
Figure 12 compares each projection for the Waterbirds and CelebA dataset. For the Waterbirds dataset, the performance of each projection ( SAL, LEACE, SPLINCE) strongly deteriotates as the correlation between the main-task and the concept increases. For the CelebA dataset, ERM gives superior performance compared to the projections. These results indicate that concept-removal methods such as SPLINCE, well as SAL and LEACE, perform relatively worse on the last-layer embeddings of vision datasets rather than those generated for NLP tasks. This is in line with previous work [Holstege et al. 2024] and an interesting direction for future research.
CelebA (black white images) dataset
95%
Worst-group accuracy
90%
Accuracy
80%
88%
ERM SAL
80%
LEACE
SPLINCE
75%
p'ylasoes a Remlling a)
Piyiuee =0 2eailingg = 0)
Waterbirds dataset 90%
90%
Worst-group accuracy
80%
Accuracy
85%
70%
80%
SAL
LEACE
75%
SPLINCE
50%
0.50
0,60
0.50
0.90
0.95
0.50
0.70
0,90
0.95
P(riid
2oock
0)
POyid
Drack R)
Figure 12: Performance of different projections on the Waterbirds and CelebA dataset. We re-train the last-layer after applying each projection. Points based the average over seeds for each of the two datasets. The error bards reflect the 95% confidence interval.
In Figure 13 we present the comparison of SPLINCE to DFRTR and GDRO for the Waterbirds and CelebA dataset. Both methods strongly outperform SPLINCE. It is worth emphasizing that SPLINCE is not explicitly designed to achieve strong outto--distribution (OOD) generalization rather to achieve certain fairness guarantees (e.g. linear guardedness) while maintaining maik-task performance. In Figure [3 we present the comparison of SPLINCE to DFRTR and GDRO for the Bias in Bios and Multilingual Text Detoxification dataset. SPLINCE performs similar both methods for the Bias in Bios dataset, and outperforms both (at high correlation) for the Multilingual Text Detoxification dataset. This result further empirical evidence that SPLINCE performs better for NLP tasks than vision datasets as it is able to perform similar or better than methods designed for OOD generalization (DFRTR and GDRO) for these two datasets. One potential reason that SPLINCE strongly outperforms DFRTR and GDRO for the Multilingual Text Detoxification dataset that there a greater number of possible combinations of the main-task and concept, as well as smaller sample size, causing DFRTR and GDRO to overfit on the training data.
29

---

CelebA (black and white images) dataset
Worst- accuracy
90%
Accuracy
90%
80%
ERM SPLINCE
70%
DFR GDRO
80%
60%
planr a Farnder a)
Popoo a a)
Waterbirds dataset 90%
Worst-go Worst-group wrsstggroup accuracy
90%
85%
70%
ERM
80%
SPLINCE DFRIR
60%
75%
30%
P(riid a Dback
Plyid
Dback
Figure 13: Performance of SPLINCE compared to deep feature reweighting (DFRTR) and Group Distributional Robust Optimization (GDRO) the Waterbirds and CelebA dataset. Each method is applied to the last layer embeddings. Points are based on the average over seeds for each of the two datasets. The error bards reflect the 95% confidence interval.
Bias in Bios dataset
Worst-group accuracy
90%
91%
85%
90%
89%
ERM
80%
SPLINCE
88%
DERRT
75%
GDRO
0.5
0.6
0.7
0.8
0.9
0.5
0.6
0.7
0.8
0.9
Poypdd Epender #)
P0pmf
Egender =
Multilingual Text Detoxification dataset
95%
Worst-group accurncy
80%
Accuracy
90%
60%-
ERM
40%
85%
SPLINCE
DFR
20%
GDRO
80%
0.5
0.6
0.7
0.8
0.9
0.5
0.6
0.7
0.8
0.9
Phoossc a Klang
Phoooi a1 Zlang
Figure 14: Performance of SPLINCE compared to deep feature reweighting (DFRTR) and Group Distributional Robust Optimization (GDRO) the Bias in Bios and Multilingual Text Detoxification dataset. Each method is applied to the last layer embeddings. Points are based on the average over seeds for each of the two datasets. The error bards reflect the 95% confidence interval.
B.6
Additional results for Large Language Models
this section, we investigate SPLINCE and other projections on same language tasks as outlined
but for two additional LLMs: the Mistral v0.3 7B model [Jiang et al. 2023), and the Phi-2 model (Javaheripi et al. 2023) from Microsoft. both cases, we use the base models as available on Huggingface, and we apply each projection to the last layer embeddings.
in 4.2
30

---

The results for the profession dataset are presented in Table and for the Winobias dataset in Figure [3] For the profession dataset, the results are in line with the results for the Llama models as presented in Table [ After applying each projection, the extent to which the models rely on factual information greatly reduced, but this reduction is smallest for SPLINCE. For the Winobias dataset, we observe little to no change for the Phi-2 model after any of the projections. Most likely this is related to the overall poor performance of the Phi-2 model on this task, potentially due to its relatively smaller size compared to the other models (2.7B parameters). For the Mistral 7B v0.3 model, we observe an increase in the accuracy on anti-stereotypical prompts after applying SPLINCE.
Table 5: Results of applying different projections to the last layer the Mistral 7B v0.3 and Phi-2 models for the profession dataset.
Model
Projection
exp( (Bstereo)
exp (Brnet)
Original
3,70
24,62
+SAL
0,95
4,86
Mistral 7B v.03
+LEACE
1,34
10,0
+SPLINCE
1.37
14,76*
Original
1,77
15,72
+SAL
0,77
5,44
Phi-2 (2.7B)
LLACE
0,77
7,58
+ SPLINCE
0,74
11,23"
Note: indicates that difference between factual coefficient our projection and the factual coefficient of LEACE is statistically significant at the 1 % level according to a onc-tailed t-test. The exponent of the coefficients estimates how the odds ratio changes with onc-unit change in Zetereo and Vfact, respectively.
Orighal Modd
ISAL
ILTAGE
ISPE INCE.
50%
Origial Madel
ISAL
-LTACE
ISPLINCE
60%
40%
Stuti
Accurey (P)
50%
40%
20%-
20%
99
18.0%
7:99
10%
10%
Misral
Phi
Mistal 7B
2h
Figure 15: Results of applying different projections to the last layer of the Mistral 7B v0.3 and Phi-2 models for the Winobias dataset. The left plot shows The left plot shows the accuracy test consisting of half pro-stereotypical and half anti-stereotypical prompts. The right plot shows the accuracy the anti-stereotypical prompts in this test set.
B.7 Applying the projections to the full Bias in Bios dataset
For completeness, we also show results for applying each projection the complete Bias in Bios dataset, with all 28 professions, in Table b Here, we observe that when not re-training the last layer, both SPLINCE and LEACE outperform SAL. This is presumably because both SPLINCE and LEACE are better able to preserve the original embeddings compared to SAL. When re- re-training, all methods become statistically indistinghuishable in terms of performance, despite being trained with regularization (in contrast to the set-up discussed in Theorem) 2). Potentially this is because the relationship between the 28 professions and gender is not as strong as in the setting considered in Section 4.1
31

---

Table 6: Results for the complete Bias in Bios dataset
Last layer re-trained
Re-trained
Method
Acc.
Acc. per class
TPR Gap
Acc.
Acc. per class
TPR Gap
Original
81.31 (0.13)
65.52 (0.17)
14.20 (0.09)
81.55 (1.15)
72.94 (0.78)
14.24 (0.08)
SAL
78.35 (0.19)
62.17 (0.20)
13.26 (0.10)
81,47 (1.12)
72.30 (0.85)
12.90 (0.17)
LEACE
81.07 (0.13)
65.09 (0.17)
12.12 (0.05)
81,62 (1.20)
72.74 (1.26)
13.13 (0.26)
SPLINCE
81.11 (0.13)
65.22 (0.16)
12.36 (0.07)
81.64 (1.07)
73.08 (1.14)
13.27 (0.01)
Note: the average is based three random sceds. The standard error is reported between brackets, The 'Acc. per class' refers the average accuracy over all 28 professions. The "TPR Gap' refers the difference in the True Positive Rate for biographics of males and females.
C
Additional information on experiments
C.1
Datasets
Below, we provide additional details each dataset used in Section 4
Bias bios datasel: the original dataset consists of 28 professions, with 255,710 samples in the training set and 98,344 samples in the test set. We subsample 75,000 observations for the training set, 10,000 for the validation set, and 25,000 for the test set. For all three sets, we subsample such that p(Mprot 1) = 0.5. For the test set, break the relationship between the professor profession and gender by setting p(yprof = Zgender = a) = 0.5 for a E {0, 1]. For the training and validation Egender = to study how different projections perform function of the relationship between the profession and gender.
set, we vary p(yprof
Multilingual dataser: we use the dataset from Dementieva et al. (2024). as hosted on Huggingface here After filtering the dataset for our selected languages (English, French, German), we subsample 3334 observations from the training set, 446 for the validation set, and 800 for the test set. For the test set, we break the relationship between the English language and sentiment by setting p(Vtox = 1| Plang = 1) 0.5. Below, we give an example of the division of samples of the dataset when p(ytox
2lang = 1) = 0.9.
Table 7: Example of the division of samples in the Multilingual Text Detoxification dataset for
P(yeox Plang 1) = 0.9
English
German
French
y (toxic)
1500
83
83 750
y 0 (not toxic)
166
750
Profession dataset: consists of a total of 320 professions. We randomly select 219 professions for the training set, and use the remaining 101 for the test set. This is different from Limistewicz et al. (2024), since we select a random set of professions for training, and use the remaining professions for testing. Each profession has an associated 2atereo and Viact score, based on Bolukbasi et al. [2016]. Per profession, we create prompts, based on the prompt templates Table [ This leads to training set that consists of 2409 samples. For the test set, we subsample 950 prompts (from total of 1111) such that the correlation between the Estereo and Dlact score This is in order to disentangle the relationship between each score and our variable of interest oddshe/she, defined in equation 7
32

---

Table 8: The 11 prompt templates used for the Profession dataset
Prompt
The <profession>wantedd that
The <profession>laughed because
The <profession>went home because
The <profession>dessredd that
The <profession>wished that
The <profession><ried because
The <profession>ate because The <profession>said that
The <profession>>an because
The <profession>stayed up because The <profession<whisprred that
Winobias dataser: the original dataset from Zhao et al. (2018) consists of sentences that were created follow two prototypical templates. We focus on the first prototypical format, which
[entity1] [interacts with] [entity2] [conjunction] [pronoun] [circumstances]
We 792 sentences for the training set, and 792 sentences for the test Both the training and test set contain 396 sentences that are aatti-ttreooyypcal,, and 396 that are 'pro-stereotypical" Both the training and test contain 40 professions that either filled in to [entity]] or [entity2] in the template above.
CelebA dataser:
We downscale the images to 50 by 50 grey-scale images, flatten them to 2,500- dimensional vectors, and apply each projection to the raw pixels. We then subsample 10,000 images, and fit each projection method on this training set.
Waterbirds dataser: introduced by Sagawa et al. [2020]: [Zhou et al. 2016) and the CUB dataset [Welinder et al. 2010). A 'water background" is set by selecting an image from lake and ocean categories in the places dataset, and the 'land background is set based on the broadleaf and bamboo forest categories. A waterbird/land is then pasted in front of the background. When creating new versions the dataset, we change the p(ybird = a Zback = a) for E 0, 1 and keep the size of the training set at 4.775/1,199 for the training and validation set respectively. For the test set, we select 5,796 samples where p(ybied = a Tback = a) 0.5.
combination of the Places dataset
C.2 Details on models and training procedure
BERT: we pre-trained BERT model implemented in the transformer. package (Woff et al.) 2019]:
arom_pretraineee ("bert- "aaee-uccssdd" When finetuning on the Bias in bios dataset, we use the same hyper-parametee as Belrooe al.
aartForrequenceCasssiiicc
2023), training with a batch size of 16, learning rate of 10-5 and weight decay of 10 using an
SGD optimizer, for 2 epochs.
Multilingual E5: we use the multilingual ES model as implemented in the transformett package [Wolf et al., 2019]: AutoModel. from wwwtoomppetraineeee
When fine-
tuning on the Muliilinuual text detoxiftcation dataset, we use a batch size of 16, a learning rate of 5 (10-5), and weight decay of 10 10-2, using the AdamW optimizer [oshchilov and Hutter 2019 for 5 epochs.
Llama models: on Huggingface. We determine each projection using the embeddings of the last token a prompt. During test time, apply the projection to each token, after the embeddings are normalized via the RMSNorm operation. When applying the projection to multiple layers, we start at the earliest layer, and calculate the projection. Then, when calculating the projection for the next layer, apply the projection from the earlier layer, and so forth.
the base model of Llama 2 7B, Llama 2 13B and Llama 3.1 8B available
Last layer re-training: When re-training the last layer. In this case, we run gradient descent (GD) using the standard implementation SGDCIassi from scikit-learn. We select the strength of the la from {1, 0.1, 0.01, 0.00011 and select the best value based on the worst-group
33

---

accuracy on the validation set. We use the original parameters of the last layer as a starting value. We fit the SGDClassifier using a tolerance 0.0001 and run for maximum 1000 epochs.
When implementing DFRTR, we subsampled set from the training dataset where each group has an equal size. Groups are defined as possible combinations of the main-task and the concept (e.g. in the Waterbirds dataset, there are four groups, Ybird {0, 1} and Tback {0, 1)). For GDRO, we use a learning rate = 0.1 to update the weights per group after each gradient descent step, similar to Sagawa et 2020
Vision datasets: For the Waterbirds dataset, we use the ResNet50 architecture implemented in the torchvision package: torchvisiion models, ResNet50 (pretrained-Truqe model using the parameters of Kirichenko et al. [2022]: a learning rate of 10-3, a weight decay of 10-3 batch size of 32, and for 100 epochs without early stopping. We stochastic gradient descent (SGD) with a momentum parameter of 0.9. For CelebA, we simply run a logistic regression, akin to last-layer retraining, the downscaled grey-scale images.
We finetune the
D Ethical considerations
As with any technique that aims to ensure that the predictions of machine learning (ML) model are fair, practitioners should exercise caution when deploying SPLINCE in real-woris settings where decisions can affect people's 's lives. Naturally, our work considers specific technical notions of fairesss and is evaluated on limited number datasets, that not reflect all considerations one should take into account in deployment. "Fairness" is multifaceted construct, and our approach addresses only certain dimensions. Consequently, practitioners must evaluate the performance of SPLINCE within their specific context, align it with the fairness notion(s) they prioritize, and remain alert to potential unintended consequences. Importantly. SPLINCE targets a very specific definition of bias, quantified by the ability of a linear model to predict protected attribute. The method is not necessarily expected to work for non-lines models, or for other definitions of fairness.
34

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims substantiated by theoretical arguments (Section 3 and empirical evidence (Section[4) 4).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: See Section 5 and references therein.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
35

---

The assumptions are stated in the theorems. The proofs can be found in Appendix A
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: See Section
4 and references therein.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
36

---

Answer: [Yes]
Justification; We provide a repository (https://github. http:/twww.coooo/oooote/////////
which contains an implementation of SPLINCE, well as code for reproducing our results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//nips. //nips.cc/pil wwcppbblc/guuiicuceccccccoeooopp wet.ggudddee/eeebbiniinn
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Section
4 and references therein.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
See Section
4 and references therein.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
37

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [No]
Justification: The computer resources used for this paper were very modest compared to nowadays standards and therefore not mentioned explicitly.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttge//hharrppp..ccc/bb////hhht
Answer: [Yes]
Justification: paper.
There were no pressing ethical issues related the research conducted for this
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper mentions potential positive societal impacts: fairness, interpretability. See for for a discussion of broader ethical considerations when using our method.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
38

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper does not include/use data models with high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: No existing assets used.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
39

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
40

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: See Section 4
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https: /ttpaarrpp..cc//ooofeeeeeee////s for what should or should not be described.
41


Reviews:
[{"review_id": "17VOEWz1af", "reviewer": "Reviewer_z2Ty", "summary": "Linear concept removal aims to follow the linear representation hypothesis, i.e. concepts in a feature learner such as a neural network, exist as linear combinations of each other. By assuming this hypothesis to be true, techniques are primarily aimed at projecting out biased directions in the representation space. \n\nThe proposed method, Splice, follows related work in this research area, and as such, is a relevant method for concept debiasing. It aims to solve a constrained optimization problem derived from [1], but with an important distinction - task relevant information, i.e. features that correspond to the actual label at hand, must be preserved when learning the projection matrix (the operator P on the covariance of the features and label must be an identity transformation). Further, the authors show that the operator effect on the prediction is determined only by the choice of kernel, and independent of the range. Experiments demonstrate the usefulness of Splice on shortcut removal in language and image data. \n\n[1] Belrose, Nora, et al. \"Leace: Perfect linear concept erasure in closed form.\" Advances in Neural Information Processing Systems 36 (2023): 66044-66063.", "questions": "1. Minor comment first: This method is a namesake of a CLIP-based explainability method called Splice [1], which caused a great deal of confusion for me initially, and might cause future confusion as well. Please consider a different name. \n2. The U and V matrices in Theorem 1 have orthonormal columns. I wonder if there are some connections to SVD and other matrix factorization techniques (U and V being the left and right singular vectors in this case). Your paper would then open up possibilities of spectral properties of such spurious features. \n3. Since the authors adopt the term \"concepts\" explicitly, I can't help but mention a strong line of recent work in concept activation vectors that probe similar ideas, albeit not in the domain of shortcut learning. Please see [2]. Is there any particular reason the authors use the term \"concepts\"?\n4. What also interests me is if the feature matrix represents some sort of alignment between modalities, e.g. CLIP features. How would P behave in this multi-modal setup? There is a risk that the covariance between the vision features and labels are preserved but not the text, and vice versa. \n\n[1] Bhalla, Usha, et al. \"Interpreting clip with sparse linear concept embeddings (splice).\" Advances in Neural Information Processing Systems 37 (2024): 84298-84328.\n\n[2] Fel, Thomas, et al. \"A holistic approach to unifying automatic concept extraction and concept importance estimation.\" Advances in Neural Information Processing Systems 36 (2023): 54805-54818.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. This paper is extremely well written and easy to follow. Every result proceeds intuitively from a previous result, and all design choices are well motivated. \n2. Both theorems are relevant results, and while Theorem 1 essentially builds entirely on [1], the added constraint is a significant difference in my view and counts as novel. \n3. Intuitive experimental results on both language and image shortcuts. Figure 2 demonstrates Splice is more robust to WGA as opposed to other projectors. Figure 3 demonstrates that the harmful effect of the bias variable is mitigated better by Splice (although I would argue just based on the illustration provided that LEACE is also pretty satisfactory). \n\nWeaknesses. \n\n1. The most obvious weaknesses perhaps, are the reliance on binary labels to demonstrate usefulness (this is not entirely the paper's limitation, but rather the limitation of the entire field of shortcut learning), and the absence of multi-shortcut results, such as UrbanCars. See [2]. \n2. There are also no results on Waterbirds, which is the de-facto dataset for evaluating worst-group accuracy. Further, there seems to be no consistency in the experiments chosen. While WGA seems to be a metric to observe for language data, it is not re-used in the image data. Please consider having WGA for both modalities, and also adding Waterbirds. \n3. An out-of-distribution evaluation set can be a set where the shortcut of interest is explicitly removed. Have the authors considered doing an experiment like this? For example, an OOD case of Waterbirds is when the eval set has landbirds exclusively with water backgrounds, and waterbirds exclusively with land backgrounds. What would be the interpretation of P in this case? \n4. I am not convinced that this method works when a single shortcut of interest is unknown. At the very least, there should be a discussion on *where* exactly the spurious features lie. Based on how the optimisation is currently set up, I am curious whether the span of the learned P belongs to the span of the orthogonal complement of the spurious features. My hunch is it is, but a result explicitly investigating this connection would be extremely interesting. For example, see Theorem 2 in [3].\n\n[2] Li, Zhiheng, et al. \"A whac-a-mole dilemma: Shortcuts come in multiples where mitigating one amplifies others.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[3] Ruiz Luyten, Max, and Mihaela van der Schaar. \"A theoretical design of concept sets: improving the predictability of concept bottleneck models.\" Advances in Neural Information Processing Systems 37 (2024): 100160-100195.", "was_revised": true, "final_justification": "I have kept my final score unchanged and recommend acceptance, as I think the added constraint on the original LEACE loss is a novelty, and leads to some interesting consequences. I did not increase my score because of limited coverage and success of the proposed method on the group robustness paradigm. Second, I am not sure whether the authors should claim guarantees about \"fairness\", since none of the traditional fairness metrics have been evaluated on the paper. I have taken into account other reviewers' critique of the paper as well in my final decision.", "rebuttal": "We are grateful to reviewer z2Ty for carefully examining the paper and their feedback.\nWe also thank them for their remarks that the paper is 'extremely well-written', and that both theorems in the paper are 'relevant results'. Below, we address the feedback point-by-point.\n\n>The most obvious ... such as UrbanCars. See [1].\n\nWe agree with reviewer z2Ty that this is a limitation of our approach. It requires access to both a concept label $z$ and a task of interest $y$. We also agree that this is a limitation of the current literature on concept-removal - virtually all existing approaches (see for instance [4, 5, 6, 7]) require a concept label $z$. Extending these approaches (including SPLICE) to a scenario where no (or limited) labels for $z$ are available is an interesting area of future research.\n\nWhile our paper does not include multi-shortcuts as suggested in [1], our suggested projection can be easily extended to this case. The $z$ will then include both the labels of the first spurious attribute (e.g., background) as well as the second spurious attribute (e.g., another co-occurring object).\n\n>There are also ... adding Waterbirds.\n\nWe thank the reviewer for this suggestion. We emphasize that the goal of our proposed projection (or concept-removal methods in general) is not to have a state-of-the-art performance in terms of worst-group accuracy. Instead, it is to achieve certain fairness guarantees (via linear guardedness) and give the user control over features used by the deep neural network (DNN).\n\nPer suggestion of the reviewer, we have evaluated SPLICE, as well as the baselines in the paper (SAL, LEACE) on the Waterbirds dataset. We have also evaluated each projection in a similar manner on the black-white version of the CelebA dataset in Section 4.3. In the original version of the manuscript we did not consider this evaluation for this dataset, since it was only used to illustrate which features are removed by each projection. \n\nFor Waterbirds, SPLICE performs similar to LEACE and SAL, although it is worth noting that all concept-removal methods perform poorly on this dataset (in comparison to standard group robustness methods - see our reply to reviewer 5q88). For CelebA, SPLICE outperforms LEACE and SAL.\n\n(Note that we only report here WGA and not average accuracy, whose results are in line with WGA, due to length constraints of the rebuttal. We will gladly provide the average accuracy tables during the discussion phase, if the reviewer wishes us to.)\n\n=== Waterbirds Dataset ===\nWorst-Group Accuracy (%)\n| Method | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | 0.95 |\n|:-------|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n| ERM | 87.5 [86.5-88.6] | 86.7 [85.5-87.9] | 81.8 [80.3-83.3] | 76.3 [75.0-77.5] | 69.3 [66.7-71.8] | 55.6 [51.6-59.6] |\n| SPLICE | 88.0 [87.3-88.6] | 85.3 [83.8-86.7] | 79.2 [78.9-79.5] | 75.2 [73.1-77.3] | 63.2 [62.1-64.3] | 55.8 [55.2-56.4] |\n| SAL | 88.3 [87.9-88.8] | 84.1 [83.0-85.3] | 82.4 [80.7-84.0] | 73.7 [70.8-76.5] | 61.5 [58.8-64.1] | 53.1 [49.1-57.1] |\n| LEACE | 88.0 [87.4-88.6] | 85.1 [83.6-86.7] | 80.6 [79.9-81.3] | 78.4 [77.8-78.9] | 65.3 [64.1-66.5] | 55.9 [53.9-57.9] |\n\n=== CelebA Dataset ===\nWorst-Group Accuracy (%)\n| Method | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |\n|:-------|:-----:|:-----:|:-----:|:-----:|:-----:|\n| ERM | 91.9 [91.0-92.7] | 91.7 [90.4-93.0] | 90.5 [88.2-92.7] | 86.5 [85.1-87.9] | 87.1 [85.0-89.2] |\n| SPLICE | 91.6 [90.6-92.6] | 87.9 [86.9-89.0] | 80.2 [79.4-81.0] | 70.0 [68.9-71.0] | 60.9 [60.0-61.8] |\n| SAL | 92.6 [92.0-93.1] | 86.5 [85.5-87.6] | 79.1 [76.6-81.5] | 67.4 [66.1-68.7] | 56.1 [54.4-57.9] |\n| LEACE | 91.7 [90.9-92.6] | 87.5 [86.0-89.0] | 80.4 [78.9-81.8] | 67.7 [66.9-68.5] | 57.4 [56.4-58.3] |\n\nBased on these results, it appears that SPLICE (as well as other concept-removal methods) performs much better for classification tasks in the language domain (Bias in Bios, Multilingual detoxification dataset) than in vision (Waterbirds, CelebA). We believe this limitation is an interesting direction for future research. We will include this limitation and the results for Waterbirds and CelebA in an updated version of the manuscript.\n\n> An out-of-distribution evaluation ... interpretation of $\\mathbf{P}$ in this case?\n\nAs said in our previous reply, the primary goal of SPLICE is not OOD generalization. Instead, its aim is linearly guarded concept removal while maintaining main-task performance. The experimental setup of OOD generalization is used to test to what extend SPLICE (and other existing methods) achieves this goal.\n\nThat being said, the default setup of OOD generalization experiments is a training set with a spurious correlation and a test set without this spurious correlation. In the case of Waterbirds, this amounts to a test set with four equal-size subgroups. The evaluation metrics are typically the average accuracy and the worst-group accuracy. The reviewer suggests to work with a test set consisting of only two of the four subgroups: landbirds on water and waterbirds on land.\n\nIt does not seem directly clear what this would add to the current experiment. The average accuracy now becomes less informative, as it neglects performance on the majority subgroups. Moreover, it is unlikely that this change will affect the worst-group accuracy, as the models typically perform the worst on one of the minority subgroups. Finally, the projection $\\mathbf{P}$ is defined independently from the choice of the test set and its interpretation therefore is unaltered: it removes (linear) information about the spurious feature from the embeddings, while maintaining the covariance with the main-task label.\n\n> I am not convinced ... example, see Theorem 2 in [2].\n\nIn reply to the first concern of the reviewer, it is true that SPLICE assumes knowledge of the labels associated with the spurious correlation (as acknowledged earlier in our reply). And this, of course, also assumes that the shortcut itself is known. We point out that this also holds for competing concept-removal methods like LEACE and SAL.\n\nRegarding the location of the spurious features, it is typically not true that the span/range of the learned $\\mathbf{P}$ belongs to the span of the orthogonal complement of the spurious features. The kernel of $\\mathbf{P}$ is spanned by the spurious features, but $\\mathbf{P}$ is typically an oblique projection and not an orthogonal projection. This was already the case for LEACE, due to the whitening transformation which is shared with SPLICE. Following the framework of Thm. 2 in [2], this suggests a bias in the predictive model after projection of the last-layer embeddings. We agree with the reviewer that it would be very interesting to investigate this connection, possibly in the context of a bias-variance trade-off. We thank the reviewer for highlighting the connection with [2] and will include this in the final version of the paper.\n\n> Minor comment first: This ... a different name.\n\nWe thank the reviewer for highlighting this namesake method and will consider to update the name from SPLICE to SPLINCE (Simultaneous Projection for LINear concept removal and Covariance prEservation) for the final version of the paper to avoid confusion.\n\n> The U and V matrices ... such spurious features.\n\nThe fact that the matrices U and V in Theorem 1 have orthonormal columns is just for technical convenience. This choice of bases for U and V, respectively, is always possible and keeps Equation 4 simple. Furthermore, the matrices U and V act on the whitened features (see W in Eq. 4), whose covariance matrix has a trivial spectral decomposition. We therefore do not think that SVD (or other factorization techniques) applied to the matrices U and V would lead to new insights.\n\n> Since the authors adopt ... the term \"concepts\"?\n\nWe adopt the term to be line with the literature on concept-removal methods. In this literature, a concept refers to a representation in the data of a human-defined object or phenomenon (in line with the original paper introducing concept activation vectors [8]). Concepts are usually represented via binary, one-hot encoded, or continuous variables, and often align with protected attributes (e.g., gender).\n\n> What also interests me ... and vice versa.\n\nWe thank the reviewer for raising the question as to how our projection would function with CLIP. Suppose that for a text-image pair, we have the text features $\\boldsymbol{t}$, and the image features $\\boldsymbol{v}$. If for the text-image pair the concept labels $z$ and $y$ are available, one can simply apply our projection to both  $\\boldsymbol{t}$ and $\\boldsymbol{v}$. This approach appears to address the risk raised by the reviewer.\n\nThe reviewer also raises an interesting point as to the interpretation of $\\mathbf{P}$ if it is applied to a vector that represents alignment between modalities. In this case, SPLICE would work in a similar fashion, only now operating on the features where the text features $\\boldsymbol{t}$ and vision features $\\boldsymbol{v}$ align.\n\n[1] Li, Zhiheng, et al. \"A whac-a-mole dilemma: Shortcuts come in multiples where mitigating one amplifies others.\" 2023\n\n[2] Ruiz Luyten, Max, and Mihaela van der Schaar. \"A theoretical design of concept sets: improving the predictability of concept bottleneck models.\" 2024\n\n[3] Fel, Thomas, et al. \"A holistic approach to unifying automatic concept extraction and concept importance estimation.\" 2023\n\n[4] Ravfogel, S., et al. Linear adversarial concept erasure. 2022\n\n[5] Belrose, N., et al. Perfect linear concept erasure in closed form. 2023\n\n[6] Ravfogel, et al. Null it out: Guarding protected attributes by iterative nullspace projection. 2020\n\n[7] Shun Shao, et al. Gold doesn’t always glitter: Spectral removal of linear and nonlinear guarded attribute information. 2023\n\n[8] Kim, B., et al. Interpretability beyond feature attribution: Quantitative testing with concept activation\nvectors (TCAV). 2018"}, {"review_id": "wpHYhCyW4F", "reviewer": "Reviewer_5q88", "summary": "This paper introduces SPLICE, a linear projection method designed to remove sensitive concepts from neural network embeddings while preserving task-relevant information. SPLICE extends previous methods LEACE by adding an additional constraint to preserve covariance between features and task labels. The authors validate their approach using classification datasets such as Bias in Bios and Multilingual Text Detoxification, as well as language modeling benchmarks, demonstrating improved retention of task-relevant information.", "questions": "- How exactly are the embeddings computed for language modeling tasks without retraining LLM parameters?\n\n- Can you clarify the closed-form solutions and practical computation of SPLICE in language model experiments?\n\n- Why were existing robust methods such as DFR, AFR, and SELF not compared in your experiments?\n\n- What does the comma in Table 1 and following tables mean?", "limitations": "- Incremental contribution with limited theoretical novelty.\n\n- Insufficient clarity regarding implementation and computational details.\n\n- Limited dataset size and lack of standard group robustness metrics for CelebA.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- The idea of preserving task-relevant covariance while removing sensitive attributes is conceptually appealing.\n\n- SPLICE provides theoretical justification for its methodology, including closed-form solutions.\n\n- Experiments demonstrate potential benefits in preserving important information in debiasing tasks.\n\nWeaknesses:\n\n- The contribution is incremental, primarily extending LEACE by introducing an additional constraint $P \\Sigma_{x,y} =  \\Sigma_{x,y}$.\n\n- The implementation details regarding last-layer retraining and language modeling are unclear, especially the calculation of features for language modeling.\n\n- Experimental validation is limited, with no worst-group accuracy reported on the widely-used CelebA dataset.\n\n- Lacks comparison with established group robustness methods like DFR, AFR, and SELF.", "was_revised": true, "final_justification": "SPLICE introduce only a minor modification to LEACE based on the original LEACE paper, which limits the theoretical contribution. Empirically, the use of Task-Relevant Information is reasonable and potentially useful to real-world applications. Given this paper is a theoretical paper, I have updated my score accordingly while still maintain a borderline opinion.", "rebuttal": "We are grateful to reviewer 5q88 for the careful and considerate review of our paper, as well as for the suggestions for improvement, which we find extremely valuable. We also thank them for commenting that our approach is 'conceptually appealing'. Below, we address the feedback point-by-point.\n\n> The contribution is incremental, primarily extending LEACE by introducing an additional constraint \n\nWe agree with the reviewer that the method we introduce extends LEACE by introducing an additional constraint. However, by doing so, we are (to the best of our knowledge) the first to consider preservation of main-task information in the context of linearly guarded concept removal. This is a well-known problem in the general field of post-hoc (linear) concept removal methods [see, e.g., references in line 26 of the manuscript], but our paper makes it clear that it is possible to address this problem in conjunction with the demand of linear guardedness.\n\nFurthermore, as a by-product our paper also advances the more general understanding of linear concept erasure methods. Theorem 2 of the manuscript shows that the range of the projection does not impact the predictions of linear models (which includes a logistic regression on last-layer embeddings) that are re-trained after the projection without regularization. Focusing on linearly guarded projections, this means that in this setting, methods like SAL, LEACE and SPLICE only improve on a basic orthogonal projection because of regularization. We consider these theoretical advances of our paper as valuable contributions to the existing literature about (linear) concept erasure.\n\n>   The implementation details regarding last-layer retraining and language modeling are unclear, especially the calculation of features for language modeling\n\n> Can you clarify the closed-form solutions and practical computation of SPLICE in language model experiments?\n\nWe thank the reviewer for allowing us to clarify how the projections (SAL, LEACE, SPLICE) are applied to an LLM. We discuss the two cases from the paper.\n\n**When applied to the last layer**:  we apply the projections to the embeddings $\\boldsymbol{x} \\in \\mathbb{R}^d$ before they are mapped to the logits of the LLM over the vocabulary. Let $\\mathcal{V}$ be the vocabulary of the LLM, and $\\mathbf{A} \\in \\mathbb{R}^{d \\times |\\mathcal{V}|}$  the weights of the last layer of the LLM. The logits at the last layer are now given by $h(\\boldsymbol{x}) = \\boldsymbol{x} \\mathbf{P} \\mathbf{A}$, where $\\mathbf{P} \\in \\mathbb{R}^{d\\times d}$ is the respective projection matrix (SAL, LEACE, or SPLICE).\n\n**When applied to the middle layers**: suppose we apply the projection to the last $k$ layers out of a total of $L$. Let $\\boldsymbol{x}^{(L - k)} \\in \\mathbb{R}^{d_{L-k}}$ denote the output of  layer $L - k$. We then apply the projections to these embeddings as $ h(\\boldsymbol{x}^{(L - k)}) = \\boldsymbol{x}^{(L - k)}\\mathbf{P}$. These transformed embeddings  $h(\\boldsymbol{x}^{(L - k)})$ are then processed by layer $L - k + 1$, after which we again gather the embeddings and fit a projection. This is repeated until we get to the last layer.\n\nNote that the description above applies to the embeddings for a single token. As noted in the appendix, when determining the projection on our training data, we use the embeddings of the last token of a prompt. During test time, we apply the projection to each token. If the embeddings are normalized via the RMSNorm operation (as is the case for the Llama models considered), we apply the projection after the normalization.\n\nWe will clarify our approach in an updated version of the manuscript.\n\n> How exactly are the embeddings computed for language modeling tasks without retraining LLM parameters?\n\nThere is no need to re-train the LLM to calculate the embeddings. We can simply gather the embeddings during a forward pass for a prompt.\n\n> Experimental validation is limited, with no worst-group accuracy reported on the widely-used CelebA dataset.\n\nWe thank the reviewer for raising this issue. In the original manuscript, we conducted the experiment on the black \\& white images from the CelebA dataset in section 4.3 in order to visually illustrate the differences between the different projections. Per the suggestion of the reviewer, we have conducted an experiment where we alter the correlation between smiling and whether or not someone is wearing glasses in these images. See our reply to reviewer z2Ty for the results of this experiment, which we will add to the final version of the paper.\n\n> Lacks comparison with established group robustness methods like DFR, AFR, and SELF \n> Why were existing robust methods such as DFR, AFR, and SELF not compared in your experiments?\n\nWe thank the reviewer for this suggestion. As stated in response to reviewer z2Ty, we emphasize that the goal of our suggested projection (or concept-removal methods in general) is not to have a state-of-the-art performance in terms of worst-group accuracy. It is to achieve fairness guarantees (via linear guardedness) and give  control over features used by DNNs.\n\nWe have added a comparison with two popular group robustness methods. First, we use deep feature reweighting (DFR, see [1]), using the same training data available to SPLICE. This amounts to the $\\mathrm{DFR}_{\\mathrm{tr}}$ method introduced in [1]. Second, we apply group-distributional robust optimization (GDRO) to the last layer of the trained DNN.\n\n(Note that we only report here worst-group accuracy and not average accuracy, whose results are in line with worst-group accuracy, due to length constraints of the rebuttal. We will gladly provide the average accuracy tables during the discussion phase, if the reviewer wishes us to.)\n\n=== Waterbirds dataset ===\nWorst-Group Accuracy (%)\n| Method  | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | 0.95 |\n|:-------|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n| ERM | 87.51 [86.46-88.56] | 86.66 [85.45-87.86] | 81.81 [80.28-83.34] | 76.26 [75.04-77.49] | 69.25 [66.66-71.84] | 55.6 [51.57-59.63] |\n| SPLICE | 87.95 [87.35-88.56] | 85.28 [83.84-86.71] | 79.16 [78.86-79.45] | 75.23 [73.14-77.32] | 63.18 [62.1-64.26] | 55.81 [55.2-56.42] |\n| $\\mathrm{DFR}_{\\mathrm{TR}}$ | 88.03 [87.16-88.9] | 85.0 [84.27-85.73] | 85.37 [84.05-86.68] | 84.19 [83.22-85.15] | 84.84 [83.09-86.59] | 82.56 [80.8-84.32] |\n| GDRO | 85.47 [82.78-88.16] | 85.3 [83.2-87.4] | 87.06 [86.1-88.03] | 84.02 [82.88-85.16] | 81.73 [79.09-84.37] | 75.45 [72.5-78.4] |\n\n=== CelebA dataset ===\nWorst-Group Accuracy (%)\n| Method  | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |\n|:-------|:-----:|:-----:|:-----:|:-----:|:-----:|\n| ERM | 91.85 [90.96-92.74] | 91.71 [90.41-93.01] | 90.47 [88.2-92.74] | 86.47 [85.06-87.89] | 87.05 [84.96-89.15] |\n| SPLICE | 91.56 [90.57-92.56] | 87.93 [86.86-88.99] | 80.22 [79.39-81.05] | 69.96 [68.94-70.99] | 60.87 [59.95-61.79] |\n| $\\mathrm{DFR}_{\\mathrm{TR}}$ | 92.36 [91.65-93.08] | 90.98 [89.94-92.02] | 89.67 [89.31-90.04] | 88.22 [87.68-88.75] | 86.11 [85.12-87.1] |\n| GDRO | 87.78 [86.35-89.22] | 85.53 [83.2-87.85] | 85.38 [84.08-86.68] | 84.22 [82.25-86.19] | 84.58 [83.61-85.55] |\n\n=== Bias in Bios dataset ===\nWorst-Group Accuracy (%)\n| Method  | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |\n|:-------|:-----:|:-----:|:-----:|:-----:|:-----:|\n| ERM | 90.96 [90.91-91.01] | 88.45 [88.01-88.89] | 87.29 [87.27-87.3] | 82.19 [81.54-82.83] | 71.99 [71.79-72.19] |\n| SPLICE | 91.28 [91.25-91.31] | 90.41 [90.39-90.43] | 90.22 [90.2-90.24] | 89.74 [89.54-89.95] | 88.39 [88.35-88.43] |\n| $\\mathrm{DFR}_{\\mathrm{TR}}$ | 91.31 [90.99-91.62] | 90.57 [90.26-90.87] | 90.39 [89.67-91.11] | 89.75 [88.42-91.08] | 89.47 [88.44-90.49] |\n| GDRO | 91.31 [91.18-91.44] | 90.23 [89.61-90.85] | 88.59 [88.54-88.63] | 88.84 [88.5-89.17] | 85.87 [84.21-87.54] |\n\n=== Multilingual dataset ===\nWorst-Group Accuracy (%)\n| Method  | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |\n|:-------|:-----:|:-----:|:-----:|:-----:|:-----:|\n| ERM | 84.32 [82.94-85.69] | 81.45 [79.91-82.98] | 78.46 [76.18-80.74] | 70.77 [66.7-74.84] | 34.23 [26.13-42.33] |\n| SPLICE | 85.56 [84.5-86.63] | 82.57 [80.51-84.62] | 77.88 [75.43-80.34] | 74.42 [70.44-78.4] | 73.12 [69.5-76.75] |\n| $\\mathrm{DFR}_{\\mathrm{TR}}$ | 83.93 [81.76-86.1] | 82.17 [81.22-83.12] | 80.58 [78.02-83.13] | 72.31 [68.92-75.7] | 29.62 [10.04-49.2] |\n| GDRO | 83.74 [81.21-86.26] | 81.79 [79.53-84.05] | 76.54 [73.24-79.84] | 68.85 [64.61-73.09] | 20.19 [8.87-31.52] |\n\nThe results show that SPLICE is strongly outperformed by DFR and GDRO for the Waterbirds and CelebA datasets. We emphasize that this is also true for the other concept-removal methods in our baseline, namely LEACE and SAL (see our reply to reviewer z2Ty for experimental verification). For the Bias in Bios dataset, SPLICE performs similar to both DFR and GDRO. For the Multilingual dataset, it outperforms both methods when there is a strong relationship between language and toxicity (0.9). As noted in our reply to reviewer z2Ty, we believe this discrepancy in performance between vision and language classification tasks is an interesting area of future research. \n\n[1] Kirichenko, P., Izmailov, P., and Wilson, A. G. Last layer re-training is sufficient for robustness to spurious correlations. In The Eleventh International Conference on Learning Representations, ICLR 2023. \n\n> What does the comma in Table 1 and following tables mean?\n\nThe comma's are used to denote decimal points. We apologize for any confusion created by this convention."}, {"review_id": "azeScwbtGe", "reviewer": "Reviewer_ynPj", "summary": "This paper introduces a method for concept erasure that aims to maximally preserve task-relevant information. For example, one might want a neural network to be insensitive to the concept of gender to mitigate gender bias. While existing approaches can remove the targeted concept, they often also degrade the performance on the main task by inadvertently discarding task-relevant information. The authors propose a method called SPLICE, which applies an affine transformation to the feature representations. This transformation ensures that the resulting features are *linearly guarded* against the target concept—i.e., they exhibit zero covariance with the concept to be removed—while preserving the covariance between the features and the main task. The paper includes a theoretical result showing that training a classifier on such transformed features, using any linearly guarded method and without regularization, yields the same classifier as training on the original features. The method's effectiveness is demonstrated in three settings: (1) bias removal from textual embeddings, where a linear layer is trained with regularization for classification; (2) language modeling, where the final layer is trained after projection; and (3) concept erasure from image data.", "questions": "- Please discuss which applications call for preserving main task performance (as in SPLICE) versus internal feature representations (as in LEACE).\n\n- For each main assumption of SPLICE (e.g., $U^{\\perp} \\cap \\text{colsp}(W\\Sigma_{x,y}) = \\{ 0 \\}$), please provide theoretical or empirical justification for its realism.\n\n- Please define “task-relevant information” and clarify how to choose a suitable proxy for it in practice.", "limitations": "The authors adequately addressed the limitations.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n\n* The paper is well-written and clearly presented, with helpful visualizations that aid in understanding the SPLICE method.\n* It makes a valuable theoretical contribution by illustrating how the choice of concept erasure method, among those satisfying linear guardedness, should depend on the specific objective, such as preserving overall feature information or maintaining main task performance.\n* The paper presents theoretical results characterizing when linear guardedness uniquely determines the resulting model, even if different projection methods are used.\n\n**Weaknesses**\n* The concept of \"task-relevant information\" is not clearly defined in the paper and appears to be conflated with the classification target variable.\n* There is no discussion of potential bias in the task-relevant information, \\$\\mathbf{y}\\$, which could itself be a result of biased data-generating processes. For instance, in the case of debiasing a classifier screening job applications (lines 27–32), the dataset may inherently reflect gender bias. Preserving covariance with such a task variable might inadvertently retain the very bias the method aims to remove, especially when applying SPLICE in earlier layers.\n* The paper lacks a discussion of which applications might benefit more from preserving main task performance (as done by SPLICE) versus preserving internal representations (as in methods like LEACE).\n* The assumption $U^{\\perp} \\cap \\text{colsp}(W\\Sigma_{x,y}) =\\{0\\}$ (line 133) is quite strong and may not hold in general.\n* The theoretical justification and empirical performance of the method seem to apply primarily to the later or final layers of a model (line 323, Appendix B.4).\n\n**Minor Weaknesses**\n\n* The methods discussed in lines 73–74 may still satisfy linear guardedness, even if they do not explicitly mention it, unless a theoretical proof is provided to the contrary.\n* Typo: “an projection” → “a projection” (line 167)", "was_revised": true, "final_justification": "This paper makes a valuable contribution to the concept erasure literature, both by deriving an interesting theoretical result regarding the family of linearly guarded methods and by presenting a new approach that demonstrates strong empirical performance. The authors’ rebuttal further reinforced my recommendation to accept.", "rebuttal": "We thank reviewer ynPj for their delicate consideration of the paper and feedback. We also thank them for stating that the paper is 'well-written and clearly presented', and 'makes a valuable theoretical contribution'. Below, we address their feedback point-by-point.\n\n> The concept of \"task-relevant information\" is not clearly defined in the paper and appears to be conflated with the classification target variable\n\n> Please define “task-relevant information” and clarify how to choose a suitable proxy for it in practice.\n\nWe thank the reviewer for pointing this out and allowing us to clarify the notion of \"task-relevant information\". Ideally, we would want to preserve the features that are relevant for recovering the performance of the main task. In the LEACE paper, it is proven that *erasing* the covariance with the sensitive information variable prevents a linear model from recovering this information. Extending this link between covariance and information, here we want to make sure that the covariance with the classification target variable, $\\mathrm{Cov}(\\boldsymbol{x}, \\boldsymbol{y})$, is *preserved*. We agree that alternative notions of ``task-relevant information'' are possible, but we believe that instantiation of this information by the covariance with the target variable is a natural extension of LEACE. Moreover, this notion admits a closed-form solution and is empirically useful, as we show in the paper. Note that under isotropic features, it is easy to prove that preserving $\\mathrm{Cov}(\\boldsymbol{x}, \\boldsymbol{y})$ is sufficient to preserve performance for classifying $\\boldsymbol{y}$. We will include a proof for this in an updated version of the appendix.\n\n> There is no discussion of potential bias in the task-relevant information, which could itself be a result of biased data-generating processes. For instance, in the case of debiasing a classifier screening job applications (lines 27–32), the dataset may inherently reflect gender bias. Preserving covariance with such a task variable might inadvertently retain the very bias the method aims to remove, especially when applying SPLICE in earlier layers.\n\nAfter the SPLICE (or LEACE, or SAL) projection the embeddings are linearly guarded for the spurious concept label $\\boldsymbol{z}.$ This means that, restricted to linear models, the projected embeddings cannot be meaningfully used to predict the spurious label. This is also the case if the spurious label is predicted indirectly, via a linear model that predicts the main-task label $\\boldsymbol{y}.$ The logic is that the projection also inevitably affects any linear prediction model for $\\boldsymbol{y},$ in such a way that the correlation between the prediction of $\\boldsymbol{y}$ and the spurious label $\\boldsymbol{z}$ is broken.\n\nWe emphasize that information about the spurious concept might still be present in the projected embeddings, but only in a non-linear fashion. This information can indeed be uncovered by a DNN if SPLICE is applied only to some earlier layers of the network, as the reviewer suggests. Therefore, if SPLICE is applied to earlier layers of the network, it should also be applied to all subsequent layers. This is done in the LEACE paper, where it is called scrubbing [1].\n\nIn a revised version of the manuscript, we intend to communicate these points more clearly.\n\n> The paper lacks a discussion of which applications might benefit more from preserving main task performance (as done by SPLICE) versus preserving internal representations (as in methods like LEACE)\n\n> Please discuss which applications call for preserving main task performance (as in SPLICE) versus internal feature representations (as in LEACE).\n\nWe thank the reviewer for allowing us to make the comparison between SPLICE and LEACE more explicit. The key difference between the two methods is that SPLICE fulfills the constraint of preserving $\\mathrm{Cov}(\\boldsymbol{x}, \\boldsymbol{y})$, at the cost of minimizing $\\mathbb{E}[||\\boldsymbol{x} - \\mathbf{P}\\boldsymbol{x}||_2^2]$. This can be particularly useful if there is some sort of correlation between $\\boldsymbol{y}$ and $\\boldsymbol{z},$ which there typically is, leading LEACE to remove $\\mathrm{Cov}(\\boldsymbol{x}, \\boldsymbol{y})$. One typically expects a bigger advantage of SPLICE compared to LEACE when this correlation is stronger. This is illustrated empirically in Appendix B.2. The additional constraint of SPLICE comes at a cost - it minimizes $\\mathbb{E}[||\\boldsymbol{x} - \\mathbf{P}\\boldsymbol{x}||_2^2]$ to a lesser extent. Our results in Appendix B.4 indicate that this can potentially pose a problem when applying SPLICE at middle layers.\n\n> The assumption  (line 133) is quite strong and may not hold in general\n\n> For each main assumption of SPLICE (e.g., ), please provide theoretical or empirical justification for its realism.\n\nWe believe that the assumption $U^\\perp \\cap \\mathrm{colsp}(W \\Sigma_{\\boldsymbol{x},\\boldsymbol{y}}) = \\mathbf{0}$ is not very strong.  It only says that the linear subspaces of the covariance directions $\\mathrm{Cov}(\\boldsymbol{x}, \\boldsymbol{y})$ and $\\mathrm{Cov}(\\boldsymbol{x}, \\boldsymbol{z})$ do not perfectly overlap. For the case of binary labels, this means that the covariance vectors are not the very same vector (up to a scaling factor). To be more concrete, for the Waterbirds dataset this would mean there are only landbirds on a land background and waterbirds on a water background. This is typically not the case in realistic datasets and would make any attempt to distinguish main-task and spurious features based on the labels futile. The assumption still allows for vectors with a high cosine similarity, i.e., we do not assume no overlap between the underlying features that encode the two concepts. We intend to clarify these subtleties around the assumption in the final version of the paper.\n\n> The theoretical justification and empirical performance of the method seem to apply primarily to the later or final layers of a model (line 323, Appendix B.4).\n\nWe broadly agree with this limitation as raised by the reviewer. There is indeed a risk that SPLICE performs worse for middle layers, as it deprioritizes minimizing $\\mathbb{E}[||\\boldsymbol{x} - \\mathbf{P}\\boldsymbol{x}||_2^2]$ in favor of fulfilling an additional constraint. However, the results in Appendix B.4 also indicate that there are cases where SPLICE applied intermediate layers can be beneficial (such as in the Bias in Bios dataset, or the experiment with the Profession dataset).\n\n> The methods discussed in lines 73–74 may still satisfy linear guardedness, even if they do not explicitly mention it, unless a theoretical proof is provided to the contrary.\n\nWe thank the reviewer for pointing this out. We will adapt the writing in our related work section, as well as our writing in line 214-217 to reflect this point.\n\n> Typo: “an projection” → “a projection” (line 167)\n\nWe thank the reviewer for raising this typo and will adapt it in an updated version of the manuscript.\n\n[1] Belrose, N., Schneider-Joseph, D., Ravfogel, S., Cotterell, R., Raff, E., and Biderman, S. Leace: Perfect linear concept erasure in closed form. In\nOh, A., Naumann, T., Globerson, A., Saenko,\nK., Hardt, M., and Levine, S. (eds.), Advances\nin Neural Information Processing Systems, volume 36, pp. 66044–66063."}, {"review_id": "g1kb6pY053", "reviewer": "Reviewer_hbbQ", "summary": "This paper introduces a post-hoc method that provably removes sensitive concepts from neural representations while exactly preserving their covariance with task-relevant labels, achieving effective concept removal with minimal distortion to task-relevant information.", "questions": "Q1: SPLICE demonstrates superior performance compared to baseline methods in two out of three tested Llama models in Fig. 3. Could the authors provide insight into the potential causes of this performance variation across different Llama model versions? To strengthen the generalizability of these findings, would the authors consider expanding the evaluation to include additional LLM architectures?\n\nQ2: In the CelebA experiment, the difference maps for SPLICE show notable changes around the glasses regions. Intuitively, since the smile concept is being removed, changes should be concentrated primarily in the smile regions rather than the glasses regions. Could the authors clarify this observation?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths \n-  The paper is well-written and easy to follow.\n- It introduces SPLICE, a theoretically grounded oblique projection method that uniquely enforces both the removal of sensitive concepts and the preservation of task-relevant information.\n- Extensive experimental results are provided across language and vision tasks.\n\n### Weaknesses\n- The paper addresses the linear concept erasure problem and assumes that task-relevant information can be represented through covariance.   However, these assumptions may not hold in complex representations within large models, where concepts are often encoded in a highly entangled manner. Experimental results further reveal that the proposed method performs less effectively when applied to intermediate layers.\n- Some relevant works are missing in the comparison and discussion. In particular, methods that also aim to preserve task-relevant information during concept erasure, such as [1] and [2] ([2] is potentially considered concurrent work under the NeurIPS policy).\n\n[1] Robust Concept Erasure via Kernelized Rate-Distortion Maximization, NeurIPS 2023\n\n[2] Fundamental Limits of Perfect Concept Erasure, AISTATS 2025", "was_revised": true, "final_justification": "The rebuttal clarifies that the paper fits within the scope of linear representation and will add experiments on other open-source LLMs. Therefore, I keep the current positive rating.  \n\nThe paper will also include a dedicated ethics statement, so I update the ** Ethical Concerns** to *Minor*.", "rebuttal": "We thank reviewer hbbQ for their thorough examination of the paper and the suggested points of improvement. We also thank them for pointing out that the paper contains 'Extensive experimental results' and our approach is 'theoretically grounded'. Below, we address their feedback point-by-point.\n\n> The paper addresses the linear concept erasure problem and assumes that task-relevant information can be represented through covariance. However, these assumptions may not hold in complex representations within large models, where concepts are often encoded in a highly entangled manner. Experimental results further reveal that the proposed method performs less effectively when applied to intermediate layers.\n\nWe thank the reviewer for highlighting this point and giving us the opportunity to clarify what we mean by \"task‑relevant information\". Our work fits within a broad line of research on linear *representation engineering*—that is, using linear algebra to manipulate neural network representations and control the information they encode. This area now includes dozens of studies, mostly on large language models but also on other neural architectures, and it has proved useful for applications ranging from bias mitigation to causal analysis. In particular, the \"linear representation hypothesis\" — the claim that many concepts are, to a large extent, linearly encoded — has become central to mechanistic interpretability [1, 2, 3]. Within this context, our contribution can be viewed as a natural extension of erasure methods: we show that it is possible not only to *remove* unwanted information, but also to *preserve* information that we wish to keep.\n\nAt the same time, we acknowledge that many features are *also* non‑linearly encoded; hence, the linear representation hypothesis holds only approximately. As the reviewer points out, this limitation may explain the weaker performance we observe in middle layers. Nonetheless, we believe that providing a closed‑form solution for the linear case is valuable (the experimental section lists several concrete use cases). In future work we plan to extend our objective to the non‑linear setting via, for example, kernelization — an approach that has already proven effective in adapting other linear techniques [4].\n\n[1] Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V., and Kalai, A. T. Man is to computer programmer as woman is to homemaker? debiasing word embeddings.\nIn Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 29. \n\n[2] Vargas, F. and Cotterell, R. Exploring the linear subspace hypothesis in gender bias mitigation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2902–2913, Online, November 2020.\n\n[3] Kiho Park, Yo Joong Choe, and Victor Veitch. 2024. The linear representation hypothesis and the geometry of large language models. In Proceedings of the 41st International Conference on Machine Learning (ICML'24), Vol. 235. JMLR.org, Article 1605, 39643–39666.\n\n[4]  Ravfogel, Shauli, et al. \"Adversarial concept erasure in kernel space.\" Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. \n\n\n> Some relevant works are missing in the comparison and discussion. In particular, methods that also aim to preserve task-relevant information during concept erasure, such as [1] and [2] ([2] is potentially considered concurrent work under the NeurIPS policy).\n\nWe thank the reviewer for raising these relevant works. We will include both works in the related work section of an updated version of the manuscript. \n\nOur reasoning for not including methods such as [1] and [2] in our manuscript is that we focus on a) linear transformations that b) guarantee linear guardedness. We believe this focus is appropriate as methods that fulfill these criteria are similar in their fairness guarantees and computational resources. If we expand the comparison to methods beyond these criteria, we would have to include a whole range of concept-removal methods in addition to [1, 2] - see for instance several methods mentioned in the related work section - and significantly increase the scope of the paper.\n\n[1] Robust Concept Erasure via Kernelized Rate-Distortion Maximization, NeurIPS 2023\n\n[2] Fundamental Limits of Perfect Concept Erasure, AISTATS 2025\n\n> Q1: SPLICE demonstrates superior performance compared to baseline methods in two out of three tested Llama models in Fig. 3. Could the authors provide insight into the potential causes of this performance variation across different Llama model versions? To strengthen the generalizability of these findings, would the authors consider expanding the evaluation to include additional LLM architectures?\n\nIt is true that there is no formal guarantee that SPLICE always significantly outperforms baseline methods, although the paper tries to make clear heuristically and empirically that it often does. The fact that SPLICE demonstrates superior performance compared to baseline methods in two out of three tested Llama models, should be seen in this light. We do not have a detailed explanation as to why there is variation in performance between the Llama 2 and 3.\n\nThat being said, we agree with the suggestion of the reviewer to strengthen the generalizability of our findings. We are currently working on repeating the experiments in section 4.2 for other open-source LLMs, namely Mistral 7B and phi-2. We will include these experiments in an updated version of the manuscript.  \n\n\n> Q2: In the CelebA experiment, the difference maps for SPLICE show notable changes around the glasses regions. Intuitively, since the smile concept is being removed, changes should be concentrated primarily in the smile regions rather than the glasses regions. Could the authors clarify this observation?\n\nWe thank the reviewer for allowing us to clarify the results in section 4.3 of the paper. Figure 4 and Figure 8 (the latter is in Appendix B.3) show that SPLICE leads to a transformation of the image around both the eyes and the mouth. This is as expected. The change around the eyes is to preserve $\\mathrm{Cov}(\\boldsymbol{x}, y_{\\mathrm{glasses}})$. The alterations around the mouth are in order to ensure linear guardedness with respect to the concept $z_{\\mathrm{glasses}}.$"}]

Author rebuttal:

Meta-review: This work focuses on the problem of removing unwanted concepts from model embeddings, while preserving task-relevant information and otherwise modifying the embedding to a minimal degree. Specifically, the aim is to make it so that the unwanted concept can not be linearly identified. The proposed approach,  SPLICE, addresses the issue that other post-hoc concept removal methods can inadvertently erase other information or concepts that are useful for the task at hand, which could otherwise lead to degraded task performance.  SPLICE addresses this by preserving covariance with the target task. This is achieved by constructing a projection that places the covariance between the representations and a protected attribute in its kernel, while maintaining the covariance between the representations and the -task label in its range. Further, the authors provide theoretical analysis that shows that training a classifier on the edited embeddings, using any linearly guarded method and without regularization, will result in learning the same classifier as training on the original embedding.

This paper is well written and clear, provides strong theoretical contributions, proposes a simple yet effective approach, and generally performs well on the empirical evaluations. 

Several weaknesses were identified during the review process. Primarily, the requirement for concept and task labels,  a lack of comparison on Waterbirds (which has become a very common, "de facto" dataset for evaluating group accuracy) and worst group accuracy for CelebA, the approach being a straightforward extension of LEACE, some implementation details being unclear, lack of comparison to established group robustness methods ( DFR, AFR, and SELF), and concerns regarding the case where the task label and target concept are correlated. 

The authors addressed most of these concerns during the rebuttal phase, particularly by adding new results on waterbirds and WGA for CelebaA, adding in a comparison to DFR, and clarifying the assumptions of their approach and behavior when task and concept are correlated. 

After the rebuttal, some concerns remained. Novelty remained an issue for some reviewers, and while the new results on waterbirds are appreciated they show that the method may not method perform as well in the group robustness paradigm. However, regarding novelty, I agree with other reviewers that the primary strength and focus of this work is its new theoretical insights and argument that preserving main-task performance rather than minimizing change to the model's activations (as is done by other approaches) can often offer better utility. And regarding the WGA performance, I agree with reviewers that this is an interesting observation and should be noted by the authors in their limitation section, but does not negate the contributions of the work. 

I agree with reviewers that this is valuable work that should be disseminated to the broader research community, and thus recommend acceptance. I urge the authors to follow through on their promise to the incorporate the recommendations of the ethics reviewers into a dedicated ethics statement in the final manuscript.
Decision: Accept (poster)