Rethinking Sparse Autoencoders: Select-and-Project for Fairness and Control from Encoder Features Alone

## Abstract

Abstract Sparse Autoencoders (SAEs) have proven valuable due to their ability to provide interpretable and steerable representations. Current debiasing methods based on SAEs manipulate these sparse activations presuming that feature representations are housed within decoder weights. We challenge this fundamental assumption and introduce an encoder-focused alternative for representation debiasing, contributing three key findings: (i) we highlight an unconventional SAE feature selection strategy, (ii) we propose a novel SAE debiasing methodology that orthogonalizes input embeddings against encoder weights, and (iii) we establish a performance-preserving mechanism during debiasing through encoder weight interpolation. Our S election and P rojection framework, termed S&P TopK , surpasses conventional SAE usage in fairness metrics by a factor of up to 3.2 3.2 and advances state-of-the-art test-time VLM debiasing results by a factor of up to 1.8 1.8 while maintaining downstream performance.

## 1 Introduction

Sparse Autoencoders (SAEs) have become pivotal in mechanistic interpretability through their ability to factorize neural network representations into interpretable components DBLP:conf/iclr/GaoTTGTRSL025; DBLP:conf/iclr/HubenCRES24; SAEBench. These sparse decompositions are commonly employed for model steering anthropic2024sae; SAeUron, especially in debiasing contexts where researchers conventionally zero out specific activations associated with unwanted features. Given that this masking operation yields a weighted combination of decoder weights, the prevailing assumption posits that SAE’s semantic features are stored within the decoder.

Departing from existing literature, we introduce a SAE-based, encoder-centric debiasing framework.
Our methodology, illustrated in Figure [1](https://arxiv.org/html/2509.10809v1#S1.F1) and elaborated in Section [3](https://arxiv.org/html/2509.10809v1#S3), follows a three-stage process: after computing SAE preactivations, we (i) employ a selection mechanism to identify relevant features, (ii) calculate a weighted sum of the encoder weights corresponding to the selected features to derive a unified bias axis, and (iii) compute a projection that orthogonalizes input vectors relative to this identified axis. We term this feature Selection and Projection methodology S&P TopK, reflecting its use of the top-k features that encode a desired protected attribute. The resulting projection can be applied to debias any input with respect to the specified attribute for any given downstream application. The main contributions embedded within our approach can be summarized as follows:

Figure: Figure 1: Illustration of the proposed S&P TopK protocol. The main steps of our approach are highlighted in green. We first employ a selection mechanism to identify relevant SAE features. We further propose a debiasing procedure based on orthogonalizing input embeddings with respect to encoder weights. To this end, we compute in the second step a weighted sum of the encoder weights corresponding to the selected features to derive a unified bias axis. Finally, we compute a projection that orthogonalizes input vectors relative to this identified axis.
Refer to caption: x1.png

1. Challenging the conventional use of SAEs. In lieu of masking protected SAE attributes, under the assumption that feature representations are stored in decoder weights, we propose orthogonalizing input embeddings with respect to encoder weights.

2. Highlighting an atypical approach to feature selection. Our investigation showed that natural approaches to identifying relevant SAE features, such as relying on CLIP scores or training linear probes, are not always optimal, and we highlight Stylist stylist as a robust alternative.

3. Proposing a mechanism designed to maintain downstream performance during debiasing. While latent feature masking and orthogonal projections based on SAE features affect the downstream performance, we show that linearly interpolating encoder weights before computing the orthogonal projection fully preserves downstream performance.

4. Empirical validation showcasing state-of-the-art test-time debiasing performance. We conduct a test-time VLM debiasing study on the CelebA and FairFace datasets, showcasing that our approach surpasses conventional SAE usage in terms of inducing fairness, as measured by conventional KL Divergence metrics, improving results by a factor of $3.2$ and advances state-of-the-art results by a factor of $1.8$ while maintaining downstream performance.

## 2 Problem Setting

We consider the problem of debiasing embeddings obtained with a large pretrained model (e.g. VLM) at test-time in retrieval and classification setups. Test-time VLM debiasing constitutes a research domain focused on eliminating a protected attribute $\mathbf{a}$ (e.g. , gender) from VLM (e.g. , CLIPradford2021learning) representations, while preserving other attribute information. Under this paradigm, the VLM operates as a black-box system with modifications applied exclusively to final output representations. This setup features a reference dataset $\mathcal{R}$ containing images labeled with a desired protected attribute $\mathbf{a}$ (e.g. , male and female annotations) in order to pinpoint it for removal. We designate $\mathcal{R}_{a_{i}}$ as the reference subset where attribute $\mathbf{a}$ assumes value $a_{i}$.

Debiasing performance is assesed on retrieval and classification benchmarks
and quantified via KL divergence and MaxSkew metrics (gerych2024bendvlm,) that compare the distribution of $\mathbf{a}$ in the dataset against its distribution within retrieved results. Furthermore, downstream performance is measured for classification tasks by means of the worst group ROC-AUC (wgROC-AUC) metric gerych2024bendvlm, where groups are formed by means of combining attributes and labels, e.g. (female, blonde hair) for CelebA.

## 3 Method

Our approach utilizes a JumpReLU SAE rajamanoharan2024jumping trained on VLM image embeddings $x\in\mathbb{R}^{n}$. The SAE framework incorporates linear encoder and decoder layers with projection matrices $E\in\mathbb{R}^{n,m},D\in\mathbb{R}^{m,n}$
and biases $b_{E}\in\mathbb{R}^{m},b_{D}\in\mathbb{R}^{n}$.
Preactivations within the SAE are expressed as $z=(x-b_{D})E+b_{E}$,
leading to activations $\hat{z}=\mbox{JumpReLU}(z)$ and subsequent reconstructions $\hat{x}=\hat{z}D+b_{D}$.
For selective weight operations, we establish the notation $W_{:,i}=[W_{1,i}W_{2,i}\dots W_{n,i}]^{\top}$ to represent individual weight columns, and $W_{:,I}=[W_{i_{1}}W_{i_{2}}\dots W_{i_{q}}]$ for index collections $I=(i_{1},i_{2},\dots i_{q})$, thereby enabling precise weight subset selection. Similarly, we use $W_{I,:}$ to denote row selection operations.

**Table 1: We present results on CelebA evaluating various combinations of feature selection and removal protocols, while simultaneously demonstrating the effectiveness of our proposed axis interpolation technique. Our findings reveal that interpolation preserves downstream accuracy, Stylist outperforms linear probing as a selection mechanism, and projection against encoder weights substantially exceeds masked reconstruction in terms of debiasing performance.**
| Selection | Removal | Interpolation | KL $\downarrow$ | MaxSkew $\downarrow$ | wgROC-AUC $\uparrow$ |
| --- | --- | --- | --- | --- | --- |
| None | None | - | 0.113880 | 0.293723 | 0.754743 |
| CLIP Score | $\perp$ TopK Encoder Weights | - | 0.164876 | 0.308559 | 0.744376 |
| LP | $\perp$ TopK Encoder Weights | - | 0.055613 | 0.250359 | 0.631793 |
| Stylist | $\perp$ TopK Encoder Weights | - | 0.035051 | 0.235039 | 0.629358 |
| Stylist | Masked Reconstruction | N/A | 0.061290 | 0.263063 | 0.527940 |
| Stylist | $\perp$ TopK Decoder Weights | - | 0.067286 | 0.299477 | 0.651578 |
| Stylist | $\perp$ TopK Encoder Weights | ✓ | 0.079235 | 0.260566 | 0.752426 |

Feature selection. We forward three approaches aimed at identifying the SAE features that correspond to the protected attribute $\mathbf{a}$:
(i) using a linear probe to weight SAE features (ii) using CLIP score to identify which features are correlated with the protected concept and (iii) using Stylist to see which SAE features vary the most across the different reference subsets $R_{a_{i}}$.

For the initial methodology, we train a Linear Probe (LP) on the reference dataset $\mathcal{R}$ to predict attribute $\mathbf{a}$ using SAE preactivations as input features. Subsequently, we rank SAE features according to the absolute magnitude of their corresponding classifier weights.

For the second methodology, we generate text prompts following the template "a photo of a $a_{i}$" for each instantiation $a_{i}$ (e.g. , male, female) of the protected attribute $\mathbf{a}$. We subsequently compute CLIP scores between these prompts and image embeddings $x\in\mathcal{R}$ from the reference dataset. SAE features are then ranked according to the correlation between their preactivation values across samples in $\mathcal{R}$ and the corresponding CLIP scores for each sample.

Finally, we highlight the applicability of Stylist stylist within the current context, a technique originally proposed for novelty detection. Stylist ranks features by computing the average Wasserstein distance between the distributions of preactivation values $z_{t}$ of each feature $t$ across different reference subsets $\mathcal{R}_{a_{i}}$ and $\mathcal{R}_{a_{j}}$ where $i\neq j$. Since attribute $\mathbf{a}$ represents the primary source of variation among reference subsets, features exhibiting the largest distributional distances should correspond to encodings of $\mathbf{a}$.

After applying the chosen selection method, we proceed with the top-k SAE features, and denote the selected subset as $S$.

Synthesizing the protected attribute axis. We train a Logistic Regression classifier with weights $w\in\mathbb{R}^{k}$ to predict attribute $\mathbf{a}$ using preactivations from the chosen feature subset $S$. The variation axis $v\in\mathbb{R}^{n}$ for attribute $\mathbf{a}$ is subsequently constructed through a weighted aggregation of encoder weights, where the weights correspond to the learned classifier parameters: $v=E_{:,S}w^{\top}$.

Orthogonal projection. We orthogonalize the image embeddings $x$ with respect to the identified axis $v$ through projection using the orthogonal projection matrix $V=\mathcal{I}_{n}-v(v^{\top}v)^{-1}v^{\top}$ chuang2023debiasing, where $\mathcal{I}_{n}$ denotes the $n$-dimensional identity matrix.

## 4 Experimental setup

Datasets. We use CelebA celeba, which contains over 200,000 images annotated with facial attributes, to analyze gender bias in hair color classification. We evaluate both accuracy and fairness. We use FairFace fairface, with over $100{,}000$ demographically balanced images, for fairness evaluation of the stereotype-based retrieval tasks (e.g. violent person, burglar), that reflect gender bias.

Models. We employ CLIP ViT-B/16 as the target VLM for debiasing. We train the JumpReLU SAE rajamanoharan2024jumping with $16{,}384$ features, following the methodology outlined in (anthropic2025jumpsae,) on approximately $37$M images from CC12M changpinyo2021conceptual, ImageNet-21k (ridnik2021imagenet,), ImageNet-1k (deng2009imagenet,), ImageNet-A (hendrycks2021natural,), ImageNet-R (hendrycks2021many,), ImageNet-Sketch (wang2019learning,) and a small subset of LAION-2B-en (schuhmann2022laion,).

## 5 Results

We provide extended details about the experimental setup in Appx. [E](https://arxiv.org/html/2509.10809v1#A5) and additional results in Appx. [F](https://arxiv.org/html/2509.10809v1#A6). We summarize the main takeaways from our experiments as follows:

Maintaining downstream performance.
As shown in Table [1](https://arxiv.org/html/2509.10809v1#S3.T1) and Table [4](https://arxiv.org/html/2509.10809v1#A6.T4) from Appx. [F](https://arxiv.org/html/2509.10809v1#A6), Regular SAE usage via Masked Reconstruction leads to a noticeable drop in wgROC-AUC.
In contrast, our projection-based debiasing consistently preserves more performance than masked reconstruction. Finally, and most importantly, we highlight that our proposed interpolation strategy fully preserves downstream task accuracy across all combinations of selection and removal methods.

Unconventional feature selection. As shown in Tables [1](https://arxiv.org/html/2509.10809v1#S3.T1), [5](https://arxiv.org/html/2509.10809v1#A6.T5) and [4](https://arxiv.org/html/2509.10809v1#A6.T4) the selection based on CLIP score does not manage to pinpoint relevant features. Furthermore, the selection based on linear probing is not always optimal: on CelebA, when projecting with respect to encoder weights, the selection provided by Stylist yields KL Divergence results which are better by a factor of $1.5$.

Encoder-based debiasing.. As shown in Tables [1](https://arxiv.org/html/2509.10809v1#S3.T1), [5](https://arxiv.org/html/2509.10809v1#A6.T5) and [4](https://arxiv.org/html/2509.10809v1#A6.T4) computing the projection matrix based on encoder weights rather than decoder weights yields a $1.9$x increase in performance, measured in terms of KL Divergence, on CelebA and a $2.5$x improvement on FairFace. We further highlight that our proposed debiasing mechanism based on orthogonalizing with respect to encoder weights, outperforms the standard procedure of masked reconstruction, yielding a $1.3$x increase in perfromance on FairFace and a $1.7$x increase in performance on CelebA.

State-of-the-art test-time debiasing results.
As shown in Table [2](https://arxiv.org/html/2509.10809v1#S5.T2), our method significantly outperforms Regular SAE debiasing with selection based on Linear Probing (LP) and removal via Masked Reconstruction (MR), yielding a $3.2$x improvement in KL Divergence. Furthermore, when combined with prompt debiasing techniques, it manages to improve upon the state-of-the-art results, yielding a $1.8$x improvement. We observe a similar outcome on the FairFace dataset, as shown in Table [6](https://arxiv.org/html/2509.10809v1#A6.T6). Furthermore, unlike BendVLM and OrthoProj, our method does not make use of CLIP’s contrastive properties, making it applicable to unimodal and generative models as well.

**Table 2: CelebA evaluation encompassing multiple state-of-the-art methods, where asterisk-marked (*) results are sourced from gerych2024bendvlm. Findings reveal that our approach significantly surpasses the standard SAE debiasing procedure utilizing linear probe-based selection and masked reconstruction removal. Notably, our method helps establish new state-of-the-art results for KL Divergence and MaxSkew when combined with BendVLM.**
|  | Debiases | Debiases | Downstream |  |  |
| --- | --- | --- | --- | --- | --- |
| Method | Input | Prompt | Knowledge | KL $\downarrow$ | MaxSkew $\downarrow$ |
| Vanilla | - | - | - | $.1138\pm.0059$ | $.2937\pm.0077$ |
| Regular SAE (LP & MR) | ✓ | - | - | $.2604\pm.1540$ | $.5735\pm.1790$ |
| BendVLM P0 gerych2024bendvlm | - | ✓ | - | $.1485\pm.0052$ | $.2915\pm.0178$ |
| S&P TopK | ✓ | - | - | $.0792\pm.0067$ | $.2605\pm.0148$ |
| OrthoProj* chuang2023debiasing | - | ✓ | - | $.0710\pm.0030$ | $.2520\pm.0060$ |
| OrthoCali* chuang2023debiasing | - | ✓ | ✓ | $.0590\pm.0010$ | $.2600\pm.0040$ |
| BendVLM gerych2024bendvlm | - | ✓ | ✓ | $.0186\pm.0062$ | $.1803\pm.0316$ |
| S&P TopK + BendVLM | ✓ | ✓ | ✓ | $\textbf{.0101}\pm.0044$ | $\textbf{.1153}\pm.0266$ |

## 6 Conclusion

We reexamined conventional SAE-based representation debiasing. By exploiting encoder weights through our selection and projection architecture, complemented by interpolation, our S&P TopK approach realizes substantial fairness enhancements without compromising task utility. Results on CelebA and FairFace establish new state-of-the-art performance in test-time VLM debiasing.

## References

- [1]
Anthropic.
Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet.
[https://transformer-circuits.pub/2024/scaling-monosemanticity/](https://transformer-circuits.pub/2024/scaling-monosemanticity/), 2024.
Accessed: 2025-08-18.
- [2]
Anthropic.
Dictionary Learning Optimization Techniques.
[https://transformer-circuits.pub/2025/january-update/index.html](https://transformer-circuits.pub/2025/january-update/index.html), 2025.
Accessed: 2025-08-18.
- [3]
Hugo Berg, Siobhan Hall, Yash Bhalgat, Hannah Kirk, Aleksandar Shtedritski, and Max Bain.
A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning.
In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 806–822. Association for Computational Linguistics, 2022.
- [4]
Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut.
Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3558–3568, 2021.
- [5]
Ching-Yao Chuang, Varun Jampani, Yuanzhen Li, Antonio Torralba, and Stefanie Jegelka.
Debiasing vision-language models via biased prompts.
arXiv preprint arXiv:2302.00070, 2023.
- [6]
Bartosz Cywinski and Kamil Deja.
SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders.
ICML, 2025.
- [7]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.
ImageNet: A large-scale hierarchical image database.
In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
- [8]
Joshua Engels, Logan Riggs Smith, and Max Tegmark.
Decomposing The Dark Matter of Sparse Autoencoders.
Trans. Mach. Learn. Res., 2025, 2025.
- [9]
Eoin Farrell, Yeu-Tong Lau, and Arthur Conmy.
Applying sparse autoencoders to unlearn knowledge in language models.
arXiv preprint arXiv:2410.19278, 2024.
- [10]
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu.
Scaling and evaluating sparse autoencoders.
In ICLR, 2025.
- [11]
Walter Gerych, Haoran Zhang, Kimia Hamidieh, Eileen Pan, Maanas K Sharma, Tom Hartvigsen, and Marzyeh Ghassemi.
Bendvlm: Test-time debiasing of vision-language embeddings.
Advances in Neural Information Processing Systems, 37:62480–62502, 2024.
- [12]
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al.
The many faces of robustness: A critical analysis of out-of-distribution generalization.
In Proceedings of the IEEE/CVF international conference on computer vision, pages 8340–8349, 2021.
- [13]
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song.
Natural adversarial examples.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15262–15271, 2021.
- [14]
Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey.
Sparse autoencoders find highly interpretable features in language models.
In ICLR, 2024.
- [15]
Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan, Max Tegmark, and Neel Nanda.
Are sparse autoencoders useful? A case study in sparse probing.
ICML, 2025.
- [16]
Kimmo Karkkainen and Jungseock Joo.
FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation.
In WACV, 2021.
- [17]
Adam Karvonen, Can Rager, Johnny Lin, Curt Tigges, Joseph Isaac Bloom, David Chanin, Yeu-Tong Lau, Eoin Farrell, Callum McDougall, Kola Ayonrinde, Matthew Wearden, Arthur Conmy, Samuel Marks, and Neel Nanda.
Saebench: A comprehensive benchmark for sparse autoencoders in language model interpretability.
ICML, 2025.
- [18]
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al.
Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV).
In International conference on machine learning, pages 2668–2677. PMLR, 2018.
- [19]
Zhiheng Li, Ivan Evtimov, Albert Gordo, Caner Hazirbas, Tal Hassner, Cristian Canton Ferrer, Chenliang Xu, and Mark Ibrahim.
A whac-a-mole dilemma: Shortcuts come in multiples where mitigating one amplifies others.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20071–20082, 2023.
- [20]
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.
Deep Learning Face Attributes in the Wild.
In ICCV, 2015.
- [21]
Harry Mayne, Yushi Yang, and Adam Mahdi.
Can sparse autoencoders be used to decompose and interpret steering vectors?
NeurIPS, 2024.
- [22]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.
Learning transferable visual models from natural language supervision.
In International conference on machine learning, pages 8748–8763. PmLR, 2021.
- [23]
Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda.
Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders.
arXiv preprint arXiv:2407.14435, 2024.
- [24]
Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zelnik-Manor.
Imagenet-21k pretraining for the masses.
arXiv preprint arXiv:2104.10972, 2021.
- [25]
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al.
Laion-5B: An open large-scale dataset for training next generation image-text models.
Advances in neural information processing systems, 35:25278–25294, 2022.
- [26]
Stefan Smeu, Elena Burceanu, Emanuela Haller, and Andrei Liviu Nicolicioiu.
Robust novelty detection through style-conscious feature ranking.
WACV, 2025.
- [27]
Lewis Smith, Senthooran Rajamanoharan, Arthur Conmy, Callum McDougall, Janos Kramar, Tom Lieberum, Rohin Shah, and Neel Nanda.
Negative Results for Sparse Autoencoders On Downstream Tasks and Deprioritising SAE Research.
[https://deepmindsafetyresearch.medium.com/negative-results-for-sparse-autoencoders-on-downstream-tasks-and-deprioritising-sae-research-6cadcfc125b9](https://deepmindsafetyresearch.medium.com/negative-results-for-sparse-autoencoders-on-downstream-tasks-and-deprioritising-sae-research-6cadcfc125b9), 2025.
Accessed: 2025-08-18.
- [28]
Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing.
Learning Robust Global Representations by Penalizing Local Predictive Power.
In Advances in Neural Information Processing Systems, pages 10506–10518, 2019.

## Appendix

## Appendix A Limitations

Our methodology requires a SAE trained on image embeddings of the VLM to be debiased. This implies that a new SAE must be trained for each new VLM that one wants to debias, leading to increased costs.
The current formulation of the interpolation step only targets binary attributes, but other protected attributes (e.g. , race or religion) do not fall in this category.
Lastly, the current work also does not address the whac-a-mole dilemma [19], a known phenomena whereby mitigating one bias leads to an amplification of a different bias

## Appendix B Software

The code that reproduces the main experiment can be accessed at the following [link](https://anonymous.4open.science/r/snp_topk-3552/README.md).

## Appendix C Related work

### C.1 Sparse Autoencoders

While Sparse Autoencoders present themselves are a remarkable and useful approach to model steering and interpretability, there has been a recent wave of pessimism in the literature. A recent systematic evaluation [27] shows that SAEs perform worse than linear probes on an out-of-distribution harmful-intent detection task. Similar negative results have appeared for interpretability, unlearning, steering, robustness [9, 15, 21]. Kantamneni et al. [15] found that SAE probes fail to offer a consistent overall advantage when added to a simulated practitioner’s toolkit. Mayne et al. [21] analyzed the use of SAEs for interpreting steering vectors finding that (i) steering vectors fall outside the input distribution for which SAEs are designed, and (ii) steering vectors can have meaningful negative projections in SAE feature directions, which SAEs are not designed to accommodate. Farell et al.[9] found that "zero ablating features is ineffective" and that simultaneous interventions across multiple SAE features, while capable of unlearning various topics, produce comparable or greater unwanted side effects than existing techniques. These findings suggest that substantial improvements in either SAE quality or intervention methodologies are necessary. Through our work we aim to forward a new perspective upon SAE usage which may alleviate some of the existing pessimism.

### C.2 Test-time Debiasing

Berg et al. [3] propose a VLM debiasing method that adds a trainable soft prefix to textual prompts in order to suppress the protected attribute.
The soft prefix is trained such that it only suppresses the attribute in prompts that do not explicitly feature said attribute, maintaining the image-text alignment in such situations.
This is achieved through a mixture of the original CLIP [22] loss and an adversarial loss that prevents an MLP from predicting the protected attribute of an image based on its CLIP scores with respect to prompts that do not feature the attribute.

Chuang et al. [5] introduce two debiasing methods, dubbed OrthProj and OrthCali. In OrthProj they make the query embeddings orthogonal to text embeddings of prompts featuring only instances of the protected attribute. OrthCali starts from the projection matrix of OrthProj and calibrates it such that it also minimizes the post-projection distance between embeddings of prompt-pairs that feature the attribute of interest but differ only in the value of $\mathbf{a}$, the protected one (e.g. , ’a photo of a male doctor’ and ’a photo of a female doctor’).

BendVLM [11] is a state-of-the-art two-stage debiasing method that uses additional information from the downstream task. For a given retrieval prompt (e.g. , "a photo of a doctor") it estimates a local protected attribute axis from embeddings of prompts featuring both the protected (gender) and target (doctor) attributes. It then optimizes the text embedding to be equidistant from a set of reference image embeddings that feature the target attribute but differ in value of the protected attribute.

## Appendix D Intuition

Our work is motivated by the conceptual similarity between Concept Activation Vectors (CAVs) [18] and SAE encoder weights. CAVs represent directional vectors pointing toward samples containing the concept of interest and away from those lacking it. We observe analogous behavior in SAE encoder weights, which function as attribute detectors. For feature activation to occur, the corresponding encoder weight must exhibit positive cosine similarity with samples containing the target attribute and negative similarity with samples lacking it (since preactivation $z_{i}$ can be expressed as $\cos(x,E_{:,i})|x||E_{:,i}|$, representing the cosine similarity scaled by vector norms).

In our application, we seek features corresponding to concepts like ’male’ or ’female’. However, SAE features do not encode pure ’male’ or ’female’ attributes, but rather composite representations such as ’human + male’ and ’human + female’. These features consequently capture human characteristics (e.g. , hair, eyes) alongside gender information. Direct projection onto existing encoder features removes not only gender concepts but also essential human traits like hair-related features, explaining performance degradation on CelebA. Our interpolation approach using linear classifier weights effectively computes the difference between ’human + male’ and ’human + female’ features by assigning positive weights to one gender’s features and negative weights to the other, thereby eliminating the shared ’human’ component and yielding a ’male - female’ variation axis that preserves task-relevant information during projection.

The interpolated SAE encoder axis outperforms regular CAVs trained on image embeddings due to several key factors. Since interpolation weights $w$ are trained on SAE preactivations, the operation $(xE_{:,S})w$ can be regrouped as $x(E_{:,S}w)$. With $u=E_{:,S}w$ representing a vector in $\mathbb{R}^{n}$, we effectively learn a vector using the same data as the CAV baseline in Table [3](https://arxiv.org/html/2509.10809v1#A4.T3). The crucial difference is that $u$ is constrained to a lower-dimensional subspace defined by the span of columns in $E_{:,S}$. This constraint prevents $u$ from exploiting spurious correlations for classification, as it can only utilize concepts encoded by the selected features. Consequently, with proper feature set $S$ selection, a CAV for the target attribute can be learned even from noisy data containing spurious correlations, and with fewer examples due to the reduced parameter count compared to regular CAVs ($k\ll n$).

**Table 3: Comparison of our method and the CAV baseline on the CelebA dataset.**
| Method | KL $\downarrow$ | MaxSkew $\downarrow$ | wgROC-AUC $\uparrow$ |
| --- | --- | --- | --- |
| Vanilla | 0.113880 | 0.293723 | 0.754743 |
| CAV | 0.145891 | 0.288400 | 0.754424 |
| S&P TopK | 0.079235 | 0.260566 | 0.752426 |

## Appendix E Experimental Setup Details

We engage two lines of experimentation in the context of the current work.

Debiasing with Sparse Autoencoders. Our first experimental line investigates optimal sparse autoencoder utilization for test-time debiasing. Results are presented in Tables [1](https://arxiv.org/html/2509.10809v1#S3.T1), [5](https://arxiv.org/html/2509.10809v1#A6.T5) and [4](https://arxiv.org/html/2509.10809v1#A6.T4). We evaluate three SAE feature selection methods: CLIP score correlation (CLIP Score), linear probing (LP), and Stylist. Additionally, we assess different debiasing approaches: Masked Reconstruction, orthogonal projection against encoder weights ("$\perp$ TopK Encoder Weights"), and orthogonal projection against decoder weights ("$\perp$ TopK Decoder Weights"). We also validate our proposed weight interpolation technique for preserving downstream performance.

In Tables [1](https://arxiv.org/html/2509.10809v1#S3.T1) and [5](https://arxiv.org/html/2509.10809v1#A6.T5), we initially assess selection methods using a fixed removal approach without weight interpolation. After identifying the optimal selection method, we present results across different removal techniques. Table [4](https://arxiv.org/html/2509.10809v1#A6.T4) provides comprehensive results covering all combinations of selection, removal, and interpolation choices.

Masked Reconstruction baseline. To account for the inherent reconstruction error of SAEs [8] we follow standard procedure and subtract from the original input the reconstruction of the selected features. That is, the final debiased input is $x-\hat{z}_{:,S}D_{S,:}$.

Comparison with state-of-the-art results. Our second experimental line compares our method against existing state-of-the-art approaches in Tables [2](https://arxiv.org/html/2509.10809v1#S5.T2) and [6](https://arxiv.org/html/2509.10809v1#A6.T6). We evaluate against OrthoProj and OrthoCali [5], both the "P0" projection component and complete BendVLM method [11], and a standard SAE debiasing protocol using Linear Probing (LP) for selection and Mask Reconstruction (MR) for removal.

We distinguish between debiasing approaches based on their intervention targets. Our method operates on input images, while others like BendVLM debias CLIP prompts used for retrieval. During calibration, both OrthoCali and BendVLM leverage downstream task prompts for more informed debiasing.

We also evaluate our method combined with BendVLM, as they complement each other by debiasing input embeddings and retrieval prompts respectively.

Tasks. On both datasets we debias the image embeddings with respect to the ’gender’ attribute. We note that the FairFace [16] dataset is only annotated for ’race’, ’gender’ and ’age’, which constitute protected attributes. As such there is no annotated downstream task on which the wgROC-AUC performance metric can be reported, as opposed to the CelebA [20] dataset where the ’hair-color’ attribute is used.

Implementation details. We consistently set $k=16$ throughout our experimental evaluation. Linear probes are implemented as Logistic Regressors featuring an L2 penalty, no bias, and class balancing weights. For all experiments we followed the setup proposed by BendVLM [11], which implies a 5-fold validation using 50% of the samples as a reference dataset. The KL divergence and MaxSkew metrics are computed using the top 500 retrieved samples. Consequently, in Tables [1](https://arxiv.org/html/2509.10809v1#S3.T1) and [5](https://arxiv.org/html/2509.10809v1#A6.T5) we report the mean and the confidence intervals for all methods.

## Appendix F Additional Results

**Table 4: We present results on CelebA evaluating all combinations of feature selection and removal protocols, while simultaneously demonstrating the effectiveness of our proposed axis interpolation technique.**
| Selection | Removal | Interpolation | KL $\downarrow$ | MaxSkew $\downarrow$ | wgROC-AUC $\uparrow$ |
| --- | --- | --- | --- | --- | --- |
| None | None | - | 0.113880 | 0.293723 | 0.754743 |
| CLIP Score | Masked Reconstruction | N/A | 0.101393 | 0.237183 | 0.747717 |
| CLIP Score | $\perp$ TopK Decoder Weights | - | 0.096210 | 0.305892 | 0.750047 |
| CLIP Score | $\perp$ TopK Encoder Weights | - | 0.164876 | 0.308559 | 0.744376 |
| CLIP Score | $\perp$ TopK Decoder Weights | ✓ | 0.130474 | 0.332680 | 0.755762 |
| CLIP Score | $\perp$ TopK Encoder Weights | ✓ | 0.122708 | 0.317317 | 0.753262 |
| LP | Masked Reconstruction | N/A | 0.260455 | 0.573577 | 0.521133 |
| LP | $\perp$ TopK Decoder Weights | - | 0.083154 | 0.319729 | 0.654926 |
| LP | $\perp$ TopK Encoder Weights | - | 0.055613 | 0.250359 | 0.631793 |
| LP | $\perp$ TopK Decoder Weights | ✓ | 0.103445 | 0.275211 | 0.753322 |
| LP | $\perp$ TopK Encoder Weights | ✓ | 0.104126 | 0.288260 | 0.751229 |
| Stylist | Masked Reconstruction | N/A | 0.061290 | 0.263063 | 0.527940 |
| Stylist | $\perp$ TopK Decoder Weights | - | 0.067286 | 0.299477 | 0.651578 |
| Stylist | $\perp$ TopK Encoder Weights | - | 0.035051 | 0.235039 | 0.629358 |
| Stylist | $\perp$ TopK Decoder Weights | ✓ | 0.098754 | 0.317625 | 0.751755 |
| Stylist | $\perp$ TopK Encoder Weights | ✓ | 0.079235 | 0.260566 | 0.752426 |

We present additional results on the FairFace dataset in Tables [5](https://arxiv.org/html/2509.10809v1#A6.T5) and [6](https://arxiv.org/html/2509.10809v1#A6.T6), along with comprehensive results covering all selection, removal, and interpolation combinations in Table [4](https://arxiv.org/html/2509.10809v1#A6.T4). These findings reinforce the conclusions outlined in Section [5](https://arxiv.org/html/2509.10809v1#S5). Notably, linear probing yields optimal SAE feature selection for FairFace, demonstrating that no universal best method exists for feature identification. However, Stylist achieves comparable performance and exhibits greater overall robustness across datasets.

**Table 5: We present results on FairFace evaluating various combinations of feature selection and removal protocols, while simultaneously demonstrating the effectiveness of our proposed axis interpolation technique. Our findings reveal that linear probing outperforms Stylist as a selection mechanism on this dataset, and that projection against encoder weights still exceeds masked reconstruction in terms of debiasing performance.**
| Selection | Removal | KL $\downarrow$ | MaxSkew $\downarrow$ |
| --- | --- | --- | --- |
| None | None | 0.129757 | 0.334185 |
| CLIP Score | $\perp$ TopK Encoder Weights | 0.346062 | 0.560762 |
| LP | $\perp$ TopK Encoder Weights | 0.041860 | 0.195931 |
| Stylist | $\perp$ TopK Encoder Weights | 0.047666 | 0.204429 |
| LP | Masked Reconstruction | 0.057230 | 0.224937 |
| LP | $\perp$ TopK Decoder Weights | 0.105178 | 0.325300 |

**Table 6: FairFace evaluation encompassing multiple state-of-the-art methods, where asterisk-marked (*) results are sourced from [11]. Findings reveal that our approach surpasses the standard SAE debiasing procedure utilizing linear probe-based selection and masked reconstruction removal. Notably, our method helps establish new state-of-the-art results for KL Divergence and MaxSkew when combined with BendVLM.**
|  | Debiases | Debiases | Downstream |  |  |
| --- | --- | --- | --- | --- | --- |
| Method | Input | Prompt | Knowledge | KL $\downarrow$ | MaxSkew $\downarrow$ |
| Vanilla | - | - | - | $.1297\pm.0025$ | $.3341\pm.0056$ |
| Regular SAE | ✓ | - | - | $.0572\pm.0147$ | $.2249\pm.0296$ |
| BendVLM P0 | - | ✓ | - | $.3283\pm.0038$ | $.5147\pm.0060$ |
| S&P TopK | ✓ | - | - | $.0476\pm.0062$ | $.2044\pm.0157$ |
| OrthoProj* | - | ✓ | - | $.3400\pm.0030$ | $.5200\pm.0010$ |
| OrthoCali* | - | ✓ | ✓ | $.4260\pm.0020$ | $.6060\pm.0010$ |
| BendVLM | - | ✓ | ✓ | $.0100\pm.0016$ | $.1166\pm.0101$ |
| S&P TopK + BendVLM | ✓ | ✓ | ✓ | $\textbf{.0080}\pm.0029$ | $\textbf{.1001}\pm.0241$ |

## Peer Reviews

confidence: 2
contribution: 2
initial_rating: 2
presentation: 2
questions: Provide theoretical explanations. Why does LP bring stability while Mean/Sign fails;

Can we visualize the features before and after modification to prove that certain regions have biased features?

The model results may be sensitive to the value of K and the random selection process. It is recommended to provide performance curves for different K values.

The article mentions “For each targeted behavior (aggressiveness and sycophancy), we train a logistic regression classifier on the SAE activations to distinguish between the behavior and its corresponding non-behavior counterpart (i.e., aggressive versus non-aggressive, and sycophantic versus non-sycophantic responses).” does it need to train a feature selector for each requirement
soundness: 2
strengths: The paper's primary originality lies in its proposal to rethink the standard application of Sparse Autoencoders for model steering. It shifts the paradigm from a "decoder-centric" approach to an "encoder-centric" one. This work identifies and challenges the implicit assumption that SAE semantics are primarily stored in the decoder. 

A key finding is that while traditional masked reconstruction significantly harms downstream task utility (wgAUC), the proposed S&P Top-K method (with interpolation) successfully preserves it .In LLM experiments, the method demonstrates substantial gains over the baseline (e.g., 3.6x improvement in sycophancy reduction) 

By identifying a key weakness in the conventional "masked reconstruction" approach—namely, poor utility preservation and ineffective control —the paper offers a practical and more effective alternative. The proposed S&P Top-K method is computationally lightweight, training-free, and demonstrates superior performance. These findings provide a valuable contribution that could inform future "best practices" for applying SAEs to model steering and debiasing.
summary: This paper revisits sparse autoencoders and proposes an encoder-centric framework for model steering and debiasing without relying on the decoder. Traditional SAE-based steering methods modify sparse latent activations and decode them back into the model’s embedding space, which can introduce reconstruction errors and high computational cost.

The proposed method instead directly operates on encoder representations: it selects the top-K encoder features correlated with a target attribute or behavior, optionally aggregates them into a single control axis, and applies an orthogonal projection in the embedding space to suppress or enhance specific semantic directions.

The framework is training-free and lightweight, demonstrated on vision–language models (CelebA, FairFace) for fairness debiasing and large language models (Llama-3-8B) for behavior control (reducing aggressiveness and sycophancy), showing up to 3.2× and 3.6× improvements over decoder-based masked reconstruction baselines.
weaknesses: Limited methodological novelty: The core contribution mainly reuses existing components of sparse autoencoders and linear subspace projection. The idea of encoder-only steering through orthogonal projection has conceptual overlap with prior works. To enhance originality, the authors could integrate a theoretical analysis of how sparsity contributes to the interpretability or robustness of the learned control directions, rather than merely showing empirical improvements.

Insufficient interpretability and visualization: While the paper argues that sparsity leads to interpretable feature directions, it does not provide qualitative visualizations (e.g., activation heatmaps or t-SNE plots of projected embeddings). Without showing how selected Top-K features correspond to semantic concepts, it is difficult to validate interpretability claims. Providing such visualization would strongly reinforce the motivation of sparsity-driven controllability.

Algorithms are limited: Its success depends on a complex and carefully tuned process, where the interpolation method must be LP weighted and feature selectors must be trained for the data. The high degree of process dependency and extreme sensitivity to specific component selection greatly increase the difficulty of method tuning and weaken its usability

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: see weakness.
soundness: 3
strengths: - New viewpoints: this paper proposes a different view that shift the focus from decoder-centric to encoder-centric SAE interventions.
- Computationally efficient and plug-and-play: no model retraining and only a single projection operation at inference.
- Good performance: experiments on both VLM and LLM shows effectiveness of the method.
summary: This paper introduces S&P Top-K, an encoder-centric method for model steering (e.g. controlling model behavior). It selects the Top-K SAE encoder features (instead of a sum of the decoder weights) that are most related to a target attribute and can combine them into a single control direction. At test time, the method removes the target information by projecting the model’s embedding away from that direction.
It requires no retraining and adds very little computation. The approach is evaluated on vision-language fairness tasks and LLM behavior steering, and they report consistent improvements over traditional methods.
weaknesses: - Motivation & problem framing are weak. The paper does not convincingly justify why encoder-centric intervention is necessary beyond empirical gains. It does not clearly articulate why an encoder-centric intervention is necessary or what specific failure modes of existing decoder-based SAE approaches motivate the new design. 
Even if the paper aims to “rethink” the intervention mechanism, it lacks a modeling or analytical treatment of existing methods, which further limits the clarity and strength of its claimed novelty.

- Lack of related works
The related-work section does not adequately cover prior studies, e.g.
[1] SEMANTICS-ADAPTIVE ACTIVATION INTERVENTION FOR LLMS VIA DYNAMIC STEERING VECTORS, ICLR'2025
[2] INTERPRETABLE STEERING OF LARGE LANGUAGE MODELS WITH FEATURE GUIDED ACTIVATION ADDITIONS, ICLR'2025 workshop
[3] IMPROVING INSTRUCTION-FOLLOWING IN LANGUAGE MODELS THROUGH ACTIVATION STEERING, ICLR'2025
[4] Analyzing the Generalization and Reliability of Steering Vectors, NIPS'2024
[5] Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models, NIPS'2025
[6] Evaluating feature steering: A case study in mitigating social biases, Anthropic
...

- Lack of theoretical grounding.
The method is described procedurally, but the theory behind the encoder-centric is not presented. It also lacks analysis of identifiability, feature entanglement, or projection optimality, leaving the approach without a solid conceptual foundation.

- Limited or unfair Evaluation
Since the paper discusses a general methodological paradigm, choosing only fairness scenarios for comparison is not comprehensive enough. Other aspects, such as trustworthiness, knowledge entities, social bias, etc., should also be considered. Also, comparison with related methods is necessary and required to be discussed.

- Experiments concerns. 
1. Key sensitivities (SAE training data/quality, choice of k, α, and whac-a-mole effects) are not thoroughly explored;
2. The time cost and the efficiency should be discussed;
3. Behavioral evaluation on LLMs relies heavily on an internal “LLM-as-a-judge” protocol (potentially subjective and model-dependent) with limited human evaluation and limited reporting of statistical significance/variance.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: 1. What does '(re)training-free' mean in the context of this work? It is used only twice: in the abstract and in contribution 2.
2. More details are needed regarding the "S&P Top-K + BendVLM" method in the paper, beyond what's written in L273.
3. How can results for the Orth-Proj and Orth-Cal methods be sourced from (Gerych et al., 2024)? Is the experimental setup and implementation identical?

**Other feedback**:
- The term 'Masked Reconstruction' is confusing. It cannot be found in the papers referenced next to it in L166 (Cywinski & Deja, 2025; Anthropic, 2024), and it is not conventionally used in the context of SAEs. This appears to be a new term, which makes the Abstract and Introduction unclear, particularly Figure 1.
- Figure 1 conveys little information. It's a single example where method A leads to a better prediction than method B. On another example, it might be the opposite. Why did it happen? How do methods A and B differ? How general is this example?
- Figure 2 is not understandable without actually reading the paper. What does interpolation do? What is the single bias axis? It overlooks the fact that the goal of the method is to remove information about the concept (orthogonalized against). Additionally, placing two figures before the Introduction is excessive. 
- The Stylist approach proposed by Smeu et al. (2025) is a major factor contributing to the method's performance, so it should be briefly described in the main text or the Appendix.
- L145: typo in $z$ should be `\bm{z}`
- L158: what is $\mathbf{Y}$?
- L236: typo in the citation "?"
- L315: "Worst Group AUC ROC" should be described in the paper
- Many typos in how quotation marks are written.
soundness: 2
strengths: 1. This work proposes a significant paradigm shift in controlling large models with SAEs, which has the potential to spark impactful future work in this new direction.
2. Applying the method to two domains (vision, language) demonstrates its versatility and simplicity.
3. The paper is generally well-written and reports results appropriately.
summary: This work proposes improving the inference of sparse autoencoders (SAEs) by projecting the input onto the selected encoder directions, rather than reconstructing it with the decoder. The decoder is only used for training the SAE. Experiments with the CLIP ViT-B/16 and Llama 3 8B Instruct models demonstrate the validity of the approach for steering, specifically debiasing, at inference time.
weaknesses: 1. The method lacks theoretical grounding. It seems that some of the steps outlined in Section 3 are trial-and-error approaches without a clear intuition for why approach A would be more appropriate than B, or whether a particular heuristic is even necessary.
2. Experiments are weak: 
    - a) The experimental setup is limited to a single SAE architecture and expansion rate $k$ per model.
    - b) The baseline SAE / Masked Reconstruction can be arbitrarily weak without calibrating the steering strength - see [a] and references given there.
    - c) Some experimental results seem inconclusive, e.g. the methods' rankings differ between Table 2 (CelebA) and Table 3 (FairFace).
    - d) The comparison with CAV-like methods (Table 5) should be given in the main text and extended to include its more recent extensions like [b].
3. In [c], SAEs are also used to debias CLIP on CelebA (Figure 6).
4. The paper should include code to reproduce the results.

[a] To steer or not to steer? Mechanistic error reduction with abstention for language models. ICML 2025

[b] Navigating neural space: Revisiting concept activation vectors to overcome directional divergence. ICLR 2025

[c] Interpreting CLIP with hierarchical sparse autoencoders. ICML 2025

---

confidence: 4
contribution: 3
initial_rating: 4
presentation: 2
questions: How would the S&P Top-K method perform if applied to decoder features, and what underlying factors intuitively differentiate encoder features from decoder features in this context?
soundness: 2
strengths: This paper has several strengths:

1. **Cross-model validation**: The method is evaluated on both language and multimodal models, demonstrating strong generality and robustness.
2. **Encoder-centric paradigm**: It shifts focus from decoder reconstruction to encoder feature control, offering a novel and interpretable perspective.
3. **Low integration cost**: Once SAEs are trained, the method requires no additional fine-tuning, making it efficient and practical for downstream use.
summary: This paper introduces the **S&P Top-K method**, which selects salient features from Sparse Autoencoders and applies orthogonal projection in the embedding space to achieve effective debiasing and behavioral control. The approach emphasizes encoder-centric representation steering and contrasts with decoder-based reconstruction methods. The authors conduct extensive experiments across language and multimodal models, demonstrating strong performance in mitigating bias and controlling behaviors such as sycophancy, outperforming established baselines like masked reconstruction.
weaknesses: 1. **Methodological clarity**

I find several aspects of the proposed method are insufficiently explained. For example, Section 3 lacks sufficient explanation of key steps, such as how the learned weight vector functions as a feature-importance signal, by cosine similarity or other criterion? Also, C mentions “distributional variations across sensitive attribute values” without clearly defining which distribution is being analyzed.

2. **Related work comparison**

The paper is not well-situated in the context of existing studies on SAE-based feature manipulation. Recent works such as 

[1] Do I know this Entity? Knowledge Awareness and Hallucinations in Language Models, 2024 

[2] Scaling and Evaluating Sparse Autoencoders, 2024

also use SAE-derived features to control model behavior by subtracting decoder-weight features from the representation space. The lack of direct comparison with these methods weakens the empirical positioning. 

Moreover, the orthogonal projection technique in S&P Top-K appears conceptually similar to the projection approach used in [1], which should be acknowledged and contrasted more explicitly.

3. **Formatting and grammatical issues**

The paper contains several grammatical errors (e.g., line 113), formatting inconsistencies (line 236), and incomplete references (line 236), which detract from its overall polish. 

4. **Limited scope of evaluation**

The experimental coverage could be expanded to include additional model architectures such as Gemma-based SAEs and a broader set of topics related to model trustworthiness, which would strengthen the paper’s generality and impact.