Title: Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: sparse autoencoders

interpretability

steering

vision and language models
Abstract: Sparse Autoencoders (SAEs) have recently gained attention as a means to improve the interpretability and steerability of Large Language Models (LLMs), both of which are essential for AI safety. In this work, we extend the application of SAEs to Vision-Language Models (VLMs), such as CLIP, and introduce a comprehensive framework for evaluating monosemanticity at the neuron-level in visual representations. To ensure that our evaluation aligns with human perception, we propose a benchmark derived from a large-scale user study. Our experimental results reveal that SAEs trained on VLMs significantly enhance the monosemanticity of individual neurons, with sparsity and wide latents being the most influential factors. Further, we demonstrate that applying SAE interventions on CLIP's vision encoder directly steers multimodal LLM outputs (e.g., LLaVA), without any modifications to the underlying language model. These findings emphasize the practicality and efficacy of SAEs as an unsupervised tool for enhancing both interpretability and control of VLMs.
Code and benchmark data are available at https://github.com/ExplainableML/sae-for-vlm.

Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models

Abstract

Sparse Autoencoders (SAEs) have recently gained attention as a means to improve the interpretability and steerability of Large Language Models (LLMs), both of which are essential for AI safety. In this work, we extend the application of SAEs to Vision-Language Models (VLMs), such as CLIP, and introduce a comprehensive framework for evaluating monosemanticity at the neuron-level in visual representations. To ensure that our evaluation aligns with human perception, we propose a benchmark derived from a large-scale user study. Our experimental results reveal that SAEs trained on VLMs significantly enhance the monosemanticity of individual neurons, with sparsity and wide latents being the most influential factors. Further, we demonstrate that applying SAE interventions on CLIP's vision encoder directly steers multimodal LLM outputs (e.g., LLaVA), without any modifications to the underlying language model. These findings emphasize the practicality and efficacy of SAEs as an unsupervised tool for enhancing both interpretability and control of VLMs. Code and benchmark data are available at https://github.com/ExplainableML/sae-for-vlm .

1 Introduction

In recent years, Vision-Language Models (VLMs) like CLIP [47] and SigLIP [61] have gained widespread adoption, owing to their capacity for simultaneous reasoning over visual and textual modalities. They have found a surge of applications in various modalities, such as in audio [14, 57] and medicine [62], transferring to new tasks with minimal supervision. Yet our current understanding of VLM internals remains limited, necessitating methods that can systematically probe their representations. Sparse AutoEncoders (SAEs) [37] are an effective approach to probing the internal representations of such models. They efficiently discover concepts (abstract features shared between data points) through their simple architecture learned as a post-hoc reconstruction task. Although analysis with SAEs is popular for Large Language Models (LLMs) [7, 23, 48], for VLMs it has been limited to interpretable classification [35, 49], or the discovery of concepts shared across models [53].

Intuitively, SAEs reconstruct activations via a higher-dimensional space to disentangle distinct concepts from their overlapping representations in neural activations [7]. Neurons at different layers within deep neural networks are known to be naturally polysemantic [41], meaning that they can be strongly activated for multiple unrelated concepts such as cellphones and rulers. One common explanation for this behavior is the superposition hypothesis [3, 17], stating that concepts are encoded as linear combination of neurons. SAEs explicitly attempt to solve this issue by separating the entangled concepts into distinct representations. Despite their widespread use in research, the absence of a metric to evaluate SAEs at the neuron-level still hinders their practicality as an interpretation tool.

MS =

0.85

MS

=

0.01

…

SAE

…

Activating images:

Activating images:

VLM

VLM

Imieoi

0O

紫

紫

Discovering neurons encoding human-interpretable concepts requires analyzing them individually, which is even more tedious as layers of both the model and SAE are wide.

VLM In this work, we quantitatively evaluate SAEs for VLMs through monosemanticity , defined as the similarity between inputs that strongly activate a neuron. We propose the MonoSemanticity score (MS) for vision tasks, that measures the pairwise similarity of images weighted by the activations for a given neuron. Unlike natural language where individual words require surrounding context (such as complete sentences) to clearly identify their meanings, individual images can directly activate neurons without additional context. We define highly activating images as images that strongly fire a particular neuron. Greater similarity among these images suggests that the neuron is more narrowly focused on a single concept, reflecting higher neuron monosemanticity. Using our MS score, we observe and validate that neurons in SAE are significantly more monosemantic (see Figure 1, right) than the original neurons (see Figure 1, left). The neuron of the original VLM typically has a low MS score, it fires for a wide range of objects, from cellphones to rulers . On the other hand, neurons within the SAE are more focused on a single concept, e.g. parrots , obtaining a higher score. This holds even for SAE with the same width as the original layer, implying that the sparse reconstruction objective inherently improves the separability of concepts. We further conduct a large-scale study to quantitatively assess alignment of our proposed MS score with human interpretation of monosemanticity. The results confirm that the difference between scores of two neurons strongly correlates with humans perceiving the higher-scoring neuron as more focused on a single concept.

Finally, we illustrate applicability of the monosemanticity of vision SAEs by transferring a CLIPbased SAE onto Multimodal LLMs (MLLMs), e.g. LLaVA [36]. Intervening on a single monosemantic SAE neuron in the vision encoder while keeping the LLM untouched allows steering the overall MLLM generated output to either insert or suppress the concept encoded in the selected SAE neuron. We summarize our contributions as follows:

We propose the MonoSemanticity score (MS) for SAEs in vision tasks, that computes neuron-wise activation-weighted pairwise similarity of image embeddings. To validate our MS score against human judgment, we conduct a large-scale user study, the results of which can also serve as a benchmark for future research.

We quantitatively compare MS between SAEs, and across their neurons. We find that Matryoshka SAE [9, 39] achieves overall superior MS, and that wider and sparser latents lead to better scores.

We leverage the well-separability of concepts in SAE layers to intervene on neuron activations and steer outputs of MLLMs to insert or suppress any discovered concept.

2 Related Work

Sparse Autoencoders. Recent studies have repurposed traditional dictionary learning to enhance LLM and VLM interpretability [6, 46]. Specifically, there has been success in interpreting and steering LLMs with features learned by SAEs [16, 52]. Several enhancements to SAE mechanisms have been introduced, including new activation functions such as Batch TopK [8] or JumpReLU [48],

…

…

…

…

…

…

𝜙

1

…

…

…

…

{X

1

, X

2

, X

3

,..., X

N

}

…

…

…

…

…

SAE

…

M 1

…

(a) Extracting embeddings and activations

(b) Pairwise similarity matrix

(c) Activation-weighted average

E

Vision

Encoder

f

E(x1)

E(x2)

E(x3)

E(xN)

SN1

aN

E(x1)E(x2)E(x3)

E(XN)

N1

S

Rk

nm

Snm

and insights from Matryoshka representation learning [34] for SAEs [9, 39]. We analyze and evaluate neuron-level monosemanticity of SAEs in VLMs and their downstream uses.

Vision-Language Models. Since Contrastive Language-Image Pretraining (CLIP) [38, 47, 61], many models have emerged that align images and text in a shared embedding space [32, 56] or generate text conditioned on image inputs [12, 19, 36]. They have achieved strong results on benchmarks [59] and found many use-cases [5, 24, 26, 28]. As trust in these models has become a concern [4, 27, 60], understanding decision-making and ensuring safety, for example through steering, is increasingly important [2, 29]. Consequently, prior work has examined their internal representations [21, 22, 40, 45], uncovering interpretable neurons [15]. We demonstrate that SAEs enable more effective interpretation and control than directly operating on raw features.

SAEs for VLMs. Building on the success of SAEs in interpreting LLMs, researchers have tried applying them to vision and vision-language models, typically on CLIP [20, 35, 49, 50] and other vision encoders (e.g. DINOv2 [44]). There has also been interest in interpreting the denoising diffusion models [25] using SAEs [11, 30, 31, 51], discovering common concepts across different vision encoders [53], and applying them on multimodal LLMs [63]. Concurrently, monosemanticity of multimodal models has been investigated [58], although focusing on inter-modality differences. Our MS score provides a grounded, quantitative measure of the monosemanticity of individual neurons which is empirically aligned with human perception. Overall, we create a rigorous evaluation framework for SAEs in VLMs, as well as demonstrating their use for steering multimodal LLMs.

3 Sparse Autoencoders for VLMs

3.1 Background and Formulation of SAEs

SAEs implement a form of sparse dictionary learning, where the goal is to learn a sparse decomposition of a signal into an overcomplete dictionary of atoms [42]. More specifically, an SAE consists of linear layers W enc ∈ R d × ω and W dec ∈ R ω × d as encoder and decoder , with a non-linear activation function σ : R ω → R ω . Both layers share a bias term b ∈ R d subtracted from the encoder's input and later added to the decoder's output. The width ω of the latent SAE layer is chosen as a factor of the original dimension, such that ω := d × ε , where ε is called the expansion factor .

In general, SAEs are applied on embeddings v ∈ R d of a given layer l of the model to explain f : X → Y , such that f l : X → R d represents the composition of the first l layers and X ⊂ R d i is the space of input images. A given input image x ∈ X is first transformed into the corresponding embedding vector v := f l ( x ) , before being decomposed by the SAE into a vector of activations ϕ ( v ) ∈ R ω , and its reconstruction vector ˆ v ∈ R d is obtained by:

The linear layers W enc and W dec composing the SAE are learned through a reconstruction objective R and sparsity regularization S , to minimize the following loss:

where λ is a hyperparameter governing the overall sparsity of the decomposition. The most simple instantiation [7, 49] uses a ReLU activation, an L 2 reconstruction objective and an L 1 sparsity penalty, such that

The (Batch) TopK SAEs [8, 23, 37] use a TopK activation function governing the sparsity directly through K . Finally, Matryoshka SAEs [9, 39] group neuron activations ϕ i ( v ) into different levels of sizes M , to obtain a nested dictionary trained with multiple reconstruction objectives:

where ϕ 1: m corresponds to keeping only the first m neuron activations, and setting the others to zero. It is important to note that Matryoshka SAEs can be combined with any SAE variant, e.g. with BatchTopK [9] or ReLU [39], as only the reconstruction objective is modified.

3.2 Monosemanticity Score

A neuron's interpretability increases as its representation becomes disentangled into a single, clear concept. Therefore, quantifying the monosemanticity of individual neurons helps identify the most interpretable ones, while aggregating these scores across an entire layer allows assessing the overall semantic clarity and quality of the representations learned by the SAE. We propose measuring monosemanticity by computing pairwise similarities between images that strongly activate a given neuron, where high similarity indicates these images likely represent the same concept. These similarities can be efficiently approximated using deep embeddings from a pretrained image encoder E . Since selecting a fixed number of top-activating images is challenging due to varying levels of specialization across neurons, we instead evaluate monosemanticity over a large, diverse set of unseen images, weighting each image by its activation strength for the neuron.

We formally describe our proposed MonoSemanticity score (MS) below, with an illustration given in Figure 2. This metric can be computed for each of the ω neurons extracted from the SAE. Given a diverse set of images I = { x n ∈ X } N n =1 , and a pretrained image encoder E , we first extract embeddings to obtain a pairwise similarity matrix S = [ s nm ] n,m ∈ [ -1 , 1] N × N , which captures semantic similarity between each pair of images. The similarity s nm of the pair ( x n , x m ) is computed as the cosine similarity between the corresponding pair of embedding vectors:

We then collect activation vectors { a k = [ a k n ] n ∈ R N } ω k =1 across all ω neurons, for all images in the dataset I . Specifically, for each image x n , the activation of the k -th neuron:

where l represents the layer at which the SAE is applied, f l is the composition of the first l layers of the explained model, and ϕ k is the k -th neuron of ϕ ( v n ) (or of v n when evaluating neurons of the original layer l of f ). To ensure a consistent activation scale, we apply min-max normalization to each a k , yielding ˜ a k := [˜ a k n ] n ∈ [0 , 1] N , where

Using these normalized activations, we compute a relevance matrix R k = [ r k nm ] n,m ∈ [0 , 1] N × N for each one of the ω neurons, which quantifies the shared neuron activation of each image pair:

Finally, our proposed score MS k ∈ [ -1 , 1] for the k -th neuron is computed as the average pairwise similarity weighted by the relevance, without considering same image pairs ( x n , x n ) :

N#5164

N#8345

N#1054

N#1918

N#1639

N#1527

N#9599

N#5718

N#173

0.9

0.0

MS

元

MINLTA

KIAORA

OAAD

3.3 Steering MLLMs with Vision SAEs

N#5164 N#5156 N#289 N#8148 N#5718 Finding monosemantic neurons is not only useful for interpretability. The SAEs let us induce controllable semantic biases in the response of MLLMs without modifying the underlying model parameters, and without touching any textual part. In other words, we can steer the model into seeing or not targeted concepts in the image. Our MS score becomes a strong tool to help select the most monosemantic neurons for a precise and efficient steering.

N#10024 N#320 N#3181 N#6664 N#7626 We first describe LLaVA [36] as an example MLLM architecture. The LLaVA model g : X × T → T expects a pair of image and text ( x , t ) and outputs a text answer o , where T ⊂ R d t is the word embedding space. Internally, it converts the image x into t x ∈ N token embeddings { v i } t x i =1 obtained from vision encoder f l : X → R d × t X composed of the first l layers of CLIP [47]. These embeddings are then projected into visual tokens H x ∈ R d t × t x in the word embedding space, and are finally fed along with tokenized text H t ∈ R d t × t t into the pretrained LLM (e.g. LLaMA [54] or Vicuna [10]) to obtain the output text o .

We modify this architecture by injecting a pretrained SAE ( ϕ, ψ ) of width ω at the token-level after the vision encoder f l . For all token embeddings v i ∈ R d , i ∈ { 1 , . . . , t x } , we first extract the SAE decomposition into activation a i := ϕ ( v i ) ∈ R ω across all neurons. After identifying the neuron k ∈ { 1 , . . . , ω } representing the targeted concept, to steer the overall model g towards this concept, we manipulate the SAE activations of all token embeddings for the neuron k to obtain { ˆ a i ∈ R ω } t X i =1 :

̸

where α ∈ R is the intervention value we want to apply to the activation of neuron k . Finally, we decode the manipulated activation vectors for each token ˆ a i back into a manipulated token embedding ˆ v i = ψ ( ˆ a i ) ∈ R d with the SAE decoder. Token embeddings are then processed as usual to generate the steered LLaVA's response. We include an illustration of the overall process in the Appendix.

4 Experiments

4.1 Experimental Settings

We apply SAEs to explain fixed and pretrained CLIP ViT-L/14-336px [47], SigLIP SoViT-400m/14384px [61], AIMv2 L/14-224px [19], and WebSSL MAE-300m/14-224px [18]. The SAEs are trained on activation vectors pre-extracted from the model's responses to ImageNet [13] images. For CLIP, activation vectors are extracted from the classification ( CLS ) tokens in the residual stream after layers l ∈ { 11 , 17 , 22 , 23 } , or from the output of the final projection layer. For steering experiments, however, the SAEs are trained on activation vectors corresponding to two random token embeddings

per image, taken from layer l = 22 . For other encoders, we similarly use the CLS tokens from the final layers, or two random token embeddings if a CLS token is not available.

In the following sections, we are interested in both BatchTopK [8] and Matryoshka BatchTopK SAEs [9] variants. If not stated otherwise, we set the groups of Matryoshka SAEs as M = { 0 . 0625 ω, 0 . 1875 ω, 0 . 4375 ω, ω } , which roughly corresponds to doubling the size of the number of neurons added with each level down. For the BatchTopK activation, we fix the maximum number of non-zero latent neurons to K = 20 . Both SAE types are compared across a wide range of expansion factors ε ∈ { 1 , 2 , 4 , 8 , 16 , 64 } . All SAEs are optimized for 10 5 steps with minibatches of size 4096 using Adam optimizer [33], with the learning rate initialized at 16 125 √ ω following previous work [23]. To measure SAE performance, we use R 2 for reconstruction quality and the L 0 norm for the activation sparsity of ϕ ( v ) . Throughout the paper, we quantify MS of neurons using DINOv2 ViT-B [44] as image encoder E , and present more analysis with different encoders in Appendix. Experiments are run on a single NVIDIA A100 GPU.

4.2 Evaluating Interpretability of VLM Neurons

4.2.1 Alignment of MS with human perception

We first illustrate the correlation between MS score and the underlying monosemanticity of neurons, with examples in Figure 3 of the 16 highest activating images for neurons with decreasing MS from left to right. We observe that the highest scoring neurons (with MS = 0 . 9 , on the far left) are firing for images representing the same object, i.e. close-up pictures of a koala (on the top) and a hockey (on the bottom). As the score decreases, the corresponding neurons fire for less similar or even completely different objects or scenes. To verify this observation in a more quantitative way, we conducted a large scale user study on the Mechanical Turk platform. This

0.0-0.1

0.1-0.2

0.2-0.3

0.3-0.4

0.4-0.5

0.5-0.6

0.6-0.7

0.7-0.8

0.8-0.9

0.5

0.6

0.7

0.8

0.9

1.0

study resulted in a total of 1000 questions across 71 unique users, with 3 answers per question aggregated through majority voting. Results of this study are presented in Figure 4, and we provide more details on the setup in Appendix. When asked to select the set of more monosemantic images from a pair of sets ( a, b ) , the users answered in accordance to the MS in 82 . 8% of the cases, assuming δ = | MS ( a ) -MS ( b ) | ∼ U (0 , 0 . 9) . This alignment rate monotonically raises from 56 . 6% for δ ∈ (0 . 0 , 0 . 1) to 100 . 0% for δ ∈ (0 . 8 , 0 . 9) , highlighting that users especially agree with MS as the difference in monosemanticity between the two sets becomes more pronounced. This demonstrates that MS can be used as a reliable measure aligning well with human perception of similarity. Detailed results of the user study are released for future benchmark of image similarity and monosemanticity.

4.2.2 Monosemanticity of SAEs

In Table 1a, we report MS of the highest scoring neurons of two SAE types (BatchTopK [8] and Matryoshka BatchTopK [9]) trained at different layers with various expansion factors ε . We also include results for original neurons of the corresponding layer decomposed by SAEs ('No SAE'). We observe that SAEs' neurons consistently have significantly higher MS for their best neuron when compared to original ones, implying that SAEs are better separating and disentangling concepts between their neurons. Interestingly, while the highest MS score is increasing with higher expansion factor ε , i.e. with increased width ω of the SAE layer, this holds true already for expansion factor ε = 1 , meaning that the disentanglement of concepts is also linked to the sparse dictionary learning and not only to the increased dimensionality. Finally, comparing SAE variants, we observe that while the Matryoshka reconstruction objective improves the concept separation at same expansion factor, it also achieves about 2 or 3 points lower R 2 for the same expansion factors (more details in Appendix).

(a) Highest MS scores of neurons in various CLIP ViT-Large [47] layers.

SAE

type

Layer

No

SAE

Expansion factor

×

1

×

2

×

4

×

8

×

16

×

64

BatchTopK

[8]

11

0.01

0.61

0.73

0.71

0.87

0.90

1.00

17

0.01

0.65

0.79

0.86

0.86

0.93

1.00

22

0.01

0.66

0.79

0.80

0.88

0.92

1.00

23

0.01

0.73

0.72

0.83

0.89

0.93

1.00

last

0.01

0.57

0.78

0.78

0.81

0.85

1.00

Matryoshka

[9, 39]

11

0.01

0.84

0.90

0.95

1.00

0.89

1.00

17

0.01

0.86

0.84

0.93

0.94

0.96

1.00

22

0.01

0.83

0.83

0.87

0.94

1.00

1.00

23

0.01

0.82

0.84

0.89

0.93

0.96

1.00

last

0.01

0.82

0.91

0.89

0.93

0.91

1.00

(b) Top MS scores of neurons from last layers of different vision en- coders. Improvements in MS score from applying Matryoshka SAEs are consistent across all the models.

Vision

Encoder

No

SAE

Exp. factor

×

1

×

4

WebSSL [18]

0.01

0.79

0.92

CLIP [47]

0.01

0.82

0.89

SigLIP [61]

0.01

0.83

0.88

AIMv2 [19]

0.01

0.59

0.85

0.0

0.2

0.4

0.6

0.8

1.0

Normalized Index

0.00

0.25

0.50

0.75

Monosemanticity Score

Expansion factor

No SAE

(a) Impact of expansion factor

ε

0.0

0.2

0.4

0.6

0.8

1.0

Normalized Index

0.00

0.25

0.50

0.75

Monosemanticity Score

K

No SAE

1

10

20

50

(b) Impact of sparsity factor

K

Table 1b presents the highest observed MS scores among neurons in the last layers of various image encoders, both before ('No SAE') and after attaching Matryoshka SAEs. We find that the SAE latent neurons outperform the original neurons in every case. As before, increasing the expansion factor ε helps discover more monosemantic units. This suggests the universality of the SAE approach across vision representations derived from different training objectives.

To analyze monosemanticity across all neurons, we plot in Figure 5a the scores of both the original neurons and the Matryoshka SAE neurons from the last layer of model f . We ordered neurons by decreasing scores and normalized neuron indices to the [0 , 1] interval to better compare SAEs with different widths. These results confirm our analysis above, and demonstrate that the vast majority of neurons within SAEs have improved MS compared to the original neurons. Even when comparing with ε = 1 , i.e. with same width between the SAE and original layers, we can see that about 90% of the neurons within the SAE have better scores than the original neurons, proving once again that the sparse decomposition objective inherently induces a better separation of concepts between neurons. Furthermore, MS scores increase overall with the expansion factor until a certain point ( ε = 4 ), after which they decrease overall, reaching even lower values than ε = 1 . Although the relative fraction of neurons at different values of MS is decreasing for very wide latents, the absolute number is still increasing. We refer the reader to Appendix for MS with raw (unnormalized) neuron indices.

The relationship between the sparsity level K used when training Matryoshka SAEs and the scores of the learned neurons is illustrated in Figure 5b. We observe that a stricter sparsity constraint decomposes the representation into more monosemantic features overall. However, this does not

s desire, e's sweet words,

aper, me,

a river, re and free.

ite,

,

hite, pointy.

(a) Concept insertion

(b) Concept suppression

In the vast expanse of white,

A love so pure and bright,

Shines through the darkness,

With a warmth that's so inviting.

A

pencil

, yellow and white,

Sits on a piece of wood,

A pink eraser, soft and white,

A red

pencil

, sharp and pointy.

'Write me a short

love poem'

+

A

pencil

on a piece of paper,

A love poem for you and me,

With words that flow like a river,

And feelings that are pure and free.

A

pencil

, a

pencil

,

A

pencil

, a

pencil

,

A

pencil

, a

pencil

,

A is all I need to feel.

Insertion of

pencil

(Neuron #39):

'Write me a short

love poem'

+

'Write me a short

love poem'

+

'Write me a short

love poem'

+

Text prompt + Input image

Generated response

Pencil Neuron

Shines through the darkness,

In the vast expanse of white, A love so pure and bright, Write me a short love poem + Intervention on N#39 0 = imply that the highest sparsity ( K = 1 ) is always the best choice, as improvements in MS come at the cost of reduced reconstruction quality. In the same setup, the R 2 varies from 31 . 3% at the lowest K = 1 to 74 . 9% at the highest K = 50 . To balance interpretability and reconstruction quality, we set K = 20 for which the R 2 remains at a reasonable 66 . 8% . Detailed results are in the Appendix.

4.3 Steering Multimodal LLMs

With a warmth that's so inviting.

A

pencil on a piece of paper,

A love poem for you and me, With words that flow like a river, And feelings that are pure and free. Write me a short love poem + Intervention on N#39 Pencil Neuron 30 = We train the Matryoshka BatchTopK SAE [9] with expansion factor ε = 64 on random token embeddings from layer l = 22 of the CLIP vision encoder obtained for the ImageNet training data. The trained SAE is plugged after the vision encoder of LLaVA-1.5-7b [36] (uses Vicuna [10] LLM).

A

pencil

, yellow and white,

Sits on a piece of wood, A pink eraser, soft and white, A red pencil , sharp and pointy. A pencil , a pencil , A pencil , a pencil , A pencil , a pencil , A is all I need to feel. Write me a short love poem + Write me a short love poem + Intervention on N#39 Pencil Neuron 40 Intervention on N#39 Pencil Neuron 50 = = Quantitative Results. We first compare the performance of SAE-based steering for VLMs against Difference-in-Means (DiffMean) [2], a popular approach based on activation steering. For each of 100 SAE neurons in LLaVA, we identify its top-activating image grid. To perform concept insertion, we boost the neuron's activation and prompt LLaVA with 10 diverse text queries, such as 'Propose a math word problem,' or 'Invent a new holiday,' then evaluate if the output contains the concept and still responds to the prompt. For concept supression, we apply a negative intervention, ask LLaV A to describe the images, and check whether the concept is removed and unrelated images are still described correctly. The evaluation is done with a LLM-as-a-judge setup using GPT-4.1-mini [43]. We provide the full details of the prompts and the evaluation procedure in Appendix B. The results in Table 2 demonstrate that SAE-based directions outperform DiffMean in both concept insertion and suppression. For insertion, SAE effectively introduces the intended concept ( 48 . 7% vs. 53 . 1% ) while maintaining much stronger adherence to the base prompt ( 85 . 5% vs. 66 . 2% ). For suppression, it performs on par with DiffMean in removing the target concept ( 64 . 4% vs. 64 . 0% ), yet far surpasses it in preserving unrelated content ( 81 . 4% vs. 38 . 7% ).

+

+

Original

-5

-20

Generated response

'What is shown

on the image ?'

+

Suppression

Text prompt + Input image

Generated response

The image shows a wooden cutting board with a bowl of strawberries and

blueberries on it. There are also

two wooden utensils

, possibly serving as tongs

or spoons, placed near the bowl.

The image shows a cutting board with a

knife

and a bowl of strawberries on it.

The image shows a wooden cutting board with a bowl of strawberries and

blueberries on it. The bowl is filled with strawberries and blueberries, and there

are also a couple of strawberries placed on the cutting board.

The image shows a

laptop computer

sitting on a desk, with a pencil holder

containing several pencils placed next to it.

The image shows a

computer monitor

with a keyboard and a mouse on a desk.

The image shows a pile of pencils and a basket of pencils, with a sign hanging

above them. The pencils are displayed in a creative manner, possibly as a

decoration or a unique way to store them.

neuron

strength

'What is shown

on the image ?'

+

original

Q=-5

Q=-20

original

Q=-5

Q=-20

'What is shown on the image ?' + To further confirm the steering capabilities of SAE neurons, we also perform a sanity check on the steering capabilities of the SAE neurons. We directly compare CLIP similarity scores between the top16 images activating a given neuron and the corresponding text outputs, both before and after steering. We prompt the model with 'What is shown on the image? Use exactly one word.' and compare its original answer to the one generated after fixing the activation of a specific SAE neuron to α = 100 , varying one neuron at a time. In the first setup, a white image is used while intervening on the first 1000 neurons to isolate the neuron manipulation's effect. In the second, 1000

'What is shown on the image ?' + Generated response s a wooden cutting board with a bowl of strawberries and . There are also two wooden utensils , possibly serving as tongs s a cutting board with a knife and a bowl of strawberries on it. random ImageNet images are used while steering only 10 neurons to test effects on natural inputs. The results in Table 3 clearly illustrates that steering increases image-text similarity scores. For context, we compute reference similarities in CLIP space: the average similarity between each ImageNet class image and its class name sets an upper bound ( 0 . 283 ± 0 . 034 ), while random image-class pairs set a lower bound ( 0 . 185 ± 0 . 028 ). Neuron steering yields a relative gain of 22% within this range, highlighting the significance of the results.

d near the bowl.

s a wooden cutting board with a bowl of strawberries and . The bowl is filled with strawberries and blueberries, and there of strawberries placed on the cutting board. s a laptop computer sitting on a desk, with a pencil holder al pencils placed next to it. s a computer monitor with a keyboard and a mouse on a desk. Qualitative Examples. Figure 6 illustrates the effectiveness of concept insertion by manipulating a single neuron of the SAE. We prompt the model with the instruction 'Write me a short love poem', along with a white image. By intervening on an SAE neuron associated to the pencil concept and increasing the corresponding activation value, we observe the impact on the generated output text. While the initial output mentions the 'white' color and focuses on the textual instruction, i.e. 'love' poem, the output becomes more and more focused on pencil attributes as we manually increase the intervention value α (most highly activating images for the selected neuron is in Appendix) until it only mentions pencils . We provide more examples with a different input prompt ('Generate a scientific article title') in Appendix, for which steered LLaV A exhibits a similar behavior.

s a pile of pencils and a basket of pencils, with a sign hanging pencils are displayed in a creative manner, possibly as a nique way to store them. In Figure 7, we show that, by clamping a specific neuron to negative value, we can suppress a concept. LLaVA is asked to answer what is shown on the images given a photo of a cutting board with knives and strawberries and a photo of laptop and pencil holder full of pencils. By default the model generates correct descriptions containing all the objects. However, when we intervene by decreasing the activation value α of the neurons associated with knife and laptop , the resulting descriptions progressively omit these concepts. This provides a promising strategy for filtering out harmful or undesired content at an early stage, before it even reaches the language model.

Suppression neuron

strength

Original

Original

-5

-2

-5

-2

5 Conclusion

Weintroduced the MonoSemanticity score (MS), a quantitative metric for evaluating monosemanticity at the neuron level in SAEs trained on VLMs. Our analysis revealed that SAEs primarily increased monosemanticity through sparsity and wider latents and highlighted the superior performance of Matryoshka SAEs. We further verified the alignment of MS with human perception through a large-scale human study. Leveraging the clear separation of concepts encoded in SAEs, we explored their effectiveness for unsupervised, concept-based steering of multimodal LLMs, highlighting a promising direction for future research. Potential extensions of this work include adapting our metric to text representations and investigating the interplay between specialized (low-level) and broad (high-level) concepts within learned representations.

Limitations. We focused our evaluation on various SAE architectures, the most common dictionary learning implementations that scale effectively to large VLMs. However, our MS metric is modelagnostic and could be applied to study other comparable approaches as well. High MS score neurons do not always produce precise effects when used to steer MLLM outputs. For example, a golden retriever neuron from an SAE trained on ImageNet can trigger any dog-related output. This could happen because, while SAEs can disentangle detailed classes in the dataset, MLLMs may have limited fine-grained understanding and may not be perfectly aligned with the vision encoder. Moreover, a fraction of the SAE neurons that act as feature detectors do not exhibit any clear steering effect [1].

Acknowledgments and Disclosure of Funding

This work was partially funded by the ERC (853489 - DEXIM) and the Alfried Krupp von Bohlen und Halbach Foundation, which we thank for their generous support. We are also grateful for partial support from the Pioneer Centre for AI, DNRF grant number P1. Shyamgopal Karthik thanks the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for support. Mateusz Pach would like to thank the European Laboratory for Learning and Intelligent Systems (ELLIS) PhD program for support.

References

Dana Arad, Aaron Mueller, and Yonatan Belinkov. Saes are good for steering-if you select the right features. arXiv preprint arXiv:2505.20063 , 2025.

Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. NeurIPS , 2024.

Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic structure of word senses, with applications to polysemy. Transactions of the Association for Computational Linguistics , 6:483-495, 2018.

Jessica Bader, Leander Girrbach, Stephan Alaniz, and Zeynep Akata. Sub: Benchmarking cbm generalization via synthetic attribute substitutions. ICCV , 2025.

Jessica Bader, Mateusz Pach, Maria A Bravo, Serge Belongie, and Zeynep Akata. Stitch: Training-free position control in multimodal diffusion transformers. arXiv preprint arXiv:2509.26644 , 2025.

Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P Calmon, and Himabindu Lakkaraju. Interpreting clip with sparse linear concept embeddings (splice). arXiv preprint arXiv:2402.10376 , 2024.

Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread , 2, 2023.

Bart Bussmann, Patrick Leask, and Neel Nanda. Batchtopk sparse autoencoders. arXiv preprint arXiv:2412.06410 , 2024.

Bart Bussmann, Patrick Leask, and Neel Nanda. Learning multi-level features with matryoshka saes. AI Alignment Forum , 2024.

Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023.

Bartosz Cywi´ nski and Kamil Deja. Saeuron: Interpretable concept unlearning in diffusion models with sparse autoencoders. arXiv preprint arXiv:2501.18052 , 2025.

Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. InstructBLIP: Towards general-purpose vision-language models with instruction tuning. In Thirty-seventh Conference on Neural Information Processing Systems , 2023.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

Satvik Dixit, Laurie M. Heller, and Chris Donahue. Vision language models are few-shot audio spectrogram classifiers, 2024.

Amil Dravid, Yossi Gandelsman, Alexei A. Efros, and Assaf Shocher. Rosetta neurons: Mining the common units in a model zoo. In ICCV , October 2023.

Esin Durmus, Alex Tamkin, Jack Clark, Jerry Wei, Jonathan Marcus, Joshua Batson, Kunal Handa, Liane Lovitt, Meg Tong, Miles McCain, Oliver Rausch, Saffron Huang, Sam Bowman, Stuart Ritchie, Tom Henighan, and Deep Ganguli. Evaluating feature steering: A case study in mitigating social biases, 2024.

Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition. arXiv preprint arXiv:2209.10652 , 2022.

David Fan, Shengbang Tong, Jiachen Zhu, Koustuv Sinha, Zhuang Liu, Xinlei Chen, Michael Rabbat, Nicolas Ballas, Yann LeCun, Amir Bar, and Saining Xie. Scaling language-free visual representation learning. ICCV , 2025.

Enrico Fini, Mustafa Shukor, Xiujun Li, Philipp Dufter, Michal Klein, David Haldimann, Sai Aitharaju, Victor Guilherme Turrisi da Costa, Louis Béthune, Zhe Gan, Alexander T Toshev, Marcin Eichner, Moin Nabi, Yinfei Yang, Joshua M. Susskind, and Alaaeldin El-Nouby. Multimodal autoregressive pre-training of large vision encoders. CVPR , 2025.

Hugo Fry. Towards multimodal interpretability: Learning sparse interpretable features in vision transformers, April 2024.

Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip's image representation via text-based decomposition. In ICLR , 2024.

Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting the second-order effects of neurons in clip. In ICLR , 2025.

Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. In The Thirteenth International Conference on Learning Representations , 2025.

Iryna Hartsock and Ghulam Rasool. Vision-language models for medical report generation and visual question answering: A review. Frontiers in artificial intelligence , 7:1430984, 2024.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS , 2020.

Yuan Hu, Jianlong Yuan, Congcong Wen, Xiaonan Lu, Yu Liu, and Xiang Li. Rsgpt: A remote sensing vision language model and benchmark. ISPRS Journal of Photogrammetry and Remote Sensing , 224:272286, 2025.

Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems , 43(2):155, 2025.

Shyamgopal Karthik, Karsten Roth, Massimiliano Mancini, and Zeynep Akata. Vision-by-language for training-free compositional image retrieval. In ICLR , 2024.

Pegah Khayatan, Mustafa Shukor, Jayneel Parekh, Arnaud Dapogny, and Matthieu Cord. Analyzing finetuning representation shift for multimodal llms steering. ICCV , 2025.

Dahye Kim and Deepti Ghadiyaram. Concept steerers: Leveraging k-sparse autoencoders for controllable generations. arXiv preprint arXiv:2501.19066 , 2025.

Dahye Kim, Xavier Thomas, and Deepti Ghadiyaram. Revelio: Interpreting and leveraging semantic information in diffusion models. arXiv preprint arXiv:2411.16725 , 2024.

Sanghwan Kim, Rui Xiao, Mariana-Iuliana Georgescu, Stephan Alaniz, and Zeynep Akata. Cosmos: Cross-modality self-distillation for vision language pre-training. arXiv preprint arXiv:2412.01814 , 2024.

Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.

Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, et al. Matryoshka representation learning. Advances in Neural Information Processing Systems , 35:30233-30249, 2022.

Hyesu Lim, Jinho Choi, Jaegul Choo, and Steffen Schneider. Sparse autoencoders reveal selective remapping of visual concepts during adaptation. arXiv preprint arXiv:2412.05276 , 2024.

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems , 36:34892-34916, 2023.

Alireza Makhzani and Brendan Frey. K-sparse autoencoders. arXiv preprint arXiv:1312.5663 , 2013.

Norman Mu, Alexander Kirillov, David Wagner, and Saining Xie. Slip: Self-supervision meets languageimage pre-training. In ECCV , 2022.

Noa Nabeshima. Matryoshka sparse autoencoders. AI Alignment Forum , 2024.

Tuomas Oikarinen and Tsui-Wei Weng. CLIP-dissect: Automatic description of neuron representations in deep vision networks. In ICLR , 2023.

Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill , 2(11):e7, 2017.

Bruno A Olshausen and David J Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision research , 37(23):3311-3325, 1997.

OpenAI. Introducing gpt-4.1 in the api, 2025.

Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 , 2023.

Konstantinos Panousis and Sotirios Chatzis. Discover: making vision networks interpretable via competition and dissection. Advances in Neural Information Processing Systems , 36:27063-27078, 2023.

Jayneel Parekh, Pegah Khayatan, Mustafa Shukor, Alasdair Newson, and Matthieu Cord. A concept-based explainability framework for large multimodal models. NeurIPS , 2024.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PmLR, 2021.

Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. arXiv preprint arXiv:2407.14435 , 2024.

Sukrut Rao, Sweta Mahajan, Moritz Böhle, and Bernt Schiele. Discover-then-name: Task-agnostic concept bottlenecks via automated concept discovery. In European Conference on Computer Vision , pages 444-461. Springer, 2024.

Samuel Stevens, Wei-Lun Chao, Tanya Berger-Wolf, and Yu Su. Sparse autoencoders for scientifically rigorous interpretation of vision models. arXiv preprint arXiv:2502.06755 , 2025.

Viacheslav Surkov, Chris Wendler, Mikhail Terekhov, Justin Deschenaux, Robert West, and Caglar Gulcehre. Unpacking sdxl turbo: Interpreting text-to-image models with sparse autoencoders. arXiv preprint arXiv:2410.22366 , 2024.

Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. Transformer Circuits Thread , 2024.

Harrish Thasarathan, Julian Forsyth, Thomas Fel, Matthew Kowal, and Konstantinos Derpanis. Universal sparse autoencoders: Interpretable cross-model concept alignment. arXiv preprint arXiv:2502.03714 , 2025.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 , 2023.

Grant Van Horn, Elijah Cole, Sara Beery, Kimberly Wilber, Serge Belongie, and Oisin Mac Aodha. Benchmarking representation learning for natural world image collections. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12884-12893, 2021.

Rui Xiao, Sanghwan Kim, Mariana-Iuliana Georgescu, Zeynep Akata, and Stephan Alaniz. Flair: Vlm with fine-grained language-informed image representations. arXiv preprint arXiv:2412.03561 , 2024.

Zhifeng Xie, Shengye Yu, Qile He, and Mengtian Li. Sonicvisionlm: Playing sound with vision language models, 2024.

Hanqi Yan, Xiangxiang Cui, Lu Yin, Paul Pu Liang, Yulan He, and Yifei Wang. The multi-faceted monosemanticity in multimodal representations. arXiv preprint arXiv:2502.14888 , 2025.

Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9556-9567, 2024.

Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bags-of-words, and what to do about it? In The Eleventh International Conference on Learning Representations .

Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision , pages 11975-11986, 2023.

Kai Zhang, Rong Zhou, Eashan Adhikarla, Zhiling Yan, Yixin Liu, Jun Yu, Zhengliang Liu, Xun Chen, Brian D. Davison, Hui Ren, Jing Huang, Chen Chen, Yuyin Zhou, Sunyang Fu, Wei Liu, Tianming Liu, Xiang Li, Yong Chen, Lifang He, James Zou, Quanzheng Li, Hongfang Liu, and Lichao Sun. Biomedgpt: A generalist vision-language foundation model for diverse biomedical tasks. arXiv preprint arXiv:2305.17100 , 2023.

Kaichen Zhang, Yifei Shen, Bo Li, and Ziwei Liu. Large multi-modal models can interpret features in large multi-modal models. arXiv preprint arXiv:2411.14982 , 2024.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We present our Monosemanticity score in Section 3.2, the user study validating alignment with human perception in Section 4.2.1, detailed analysis of MS in Section 4.2.2, and results of steering MLLMs in Section 4.3.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss limitations in Section 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We describe all experimental settings in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Our code and the data associated to the use study will be available upon acceptance.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We describe all training settings of the SAEs in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We include standard deviations of MS across neurons in Appendix. We also provide the statistics of the results of the user study in the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We mention computation ressources in Section 4.1. All experiments have been conducted on a single NVIDIA A100 GPU with either 40 or 80 GB memory.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: All authors have reviewed and conform to the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We include a discussion on broader impacts in the Appendix.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We do not plan to release any model and did not scrap data from the Internet.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We provide citations for the data and models used in our experiments.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We will include documentation for our code and on the data gathering process when we will release them upon acceptance.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [Yes]

Justification: We include the full description of the experiment, with instructions given and compensation in Appendix.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [Yes]

Justification: Our institution does not require IRB approval for these tasks.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [Yes]

Justification: We evaluate steering on MLLMs, which integrate LLMs as a component.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Contents

A Broader Impact

Our work contributes to the field of interpretability and alignment, which are essential components for building safe AI systems. Our MonoSemanticity score provides a new way to evaluate the effectiveness of recently popular dictionary learning methods, such as sparse autoencoders (SAEs), by incorporating human judgment into the evaluation process. This makes it easier to assess and build trust in systems that use SAEs.

In addition, we show that SAEs can be highly effective in steering applications. They can be used to encourage or discourage specific behaviors in models, or to help models recognize or ignore certain concepts, including potentially dangerous ones. This is especially useful for ensuring that models produce desired outputs and remain aligned with human values and goals.

B More details on steering

We illustrate in Figure A1 how we steer LLaVA-like models. We separately train SAEs on top of the pretrained CLIP vision encoder to reconstruct the token embeddings v i , and then attach it back after the vision encoder during inference. Intervening on a neuron within the SAE layer steers the reconstructed tokens ˆ v i towards the activated concept, which then steers the LLM's generated output. We present in Figure A2 additional examples of LLaVA prompted to generate scientific titles, and the outputs before and after intervening on SAE neurons. Increasing the activation of specific neurons will modify the outputs to include elements from images highly activating the corresponding neuron.

'It is a panda.' 0 0

…

Vision

Encoder

…

…

…

LLM

…

…

Projection

'What is it?'

Intervention on

Panda Neuron

紫

HOOO

V1

V2V3

紫

Q = 50

X

medium

high

The Effects of White Backgrounds on Visual Perception and Attention.

a scientific

article title'

+

Intervention

on neuron            strength

Text prompt + Input image

Generated response

a scientific

article title'

+

a scientific

article title'

+

The Effects of White

on the Perception of

Black and White Patterns.

The Effects of

on the Density of

Black and White

.

The Effects of White Backgrounds on Visual Perception and Attention.

The Evolution of a Ruin: A Chronological Analysis of

the Deterioration of a White Building.

The Evolution of

Shipwreck

Preservation:

A Case Study of the RMS Titanic.

The Effects of White Backgrounds on Visual Perception and Attention.

The Effects of Color and Light on Mood and Cognitive Functioning:

A Study of Color-Tinted Lighting in a Neuropsychological Laboratory.

The

Rainbow

Connection: Exploring the Spectrum of Colors

in a Multicolored World.

medium

high

medium

high

The steering capabilities discussed in Section 4.3 are evaluated using an LLM-as-judge setup with the following prompts:

'Write me a short love poem,'

'Generate a scientific article title,'

'Give me a four-item to-do list,'

'Write me a two-verse rap song,'

'Propose a math word problem,'

'Write a paragraph from a Wikipedia page,'

'Invent a new holiday,'

'Write a dialogue,'

'Write a newspaper headline and first paragraph,'

'Give a conversation starter for a party.'

C User study

To validate the alignment of our MonoSemanticity score (MS) with human judgment, we conducted a user study. Example questions are shown in Figures A3, A4, and A5. Each question shows two grids of images:

where each grid contains the 16 images with the highest activations for two neurons k x and k y , respectively. Formally, for each i = 1 , . . . , 16 ,

where a k n is the activation of neuron k on image x n , and rank n ( a k n ) is the rank of image x n when sorting all images by their activation values in descending order. Images I come from training set of the ImageNet.

For each neuron pair ( k x , k y ) , we asked three human annotators the question: 'Which set of images looks more similar and focused on the same thing?' Each annotator gave an answer r j ∈ { k x , k y } for j = 1 , 2 , 3 . The final human choice was decided by majority vote:

At the same time, we answered the question using Monosemanticity Score:

We say the MS and users are aligned if their decision is the same:

The overall alignment score is the fraction of all neuron pairs where the MS and humans are aligned:

where Q is the set of all neuron pairs evaluated.

In total, we collected 1,000 user pair rankings with the help of 71 annotators on the Mechanical Turk platform. The number of answers per annotator ranged from 1 to 205, with a median of 24. Annotators were compensated at a rate of $0 . 02 per answer.

The neurons used in the study were randomly selected from the last layer of CLIP ViT-L, BatchTopK SAE ( ε = 4 , K = 20 ) trained on the last layer of CLIP ViT-L, Matryoshka SAE ( ε = 4 , K = 20 ) trained on the last layer of CLIP ViT-L, and BatchTopK SAE ( ε = 4 , K = 20 ) trained on the last layer of SigLIP SoViT-400m.

In addition to the plot presenting the user study results in the main paper, we also provide Table A1, which reports the exact values obtained, along with the sizes of each group categorized by MS distances between neuron pairs. When designing the questions, we balanced the number of pairs within each distance interval. Our goal is to evaluate MS computed using embeddings from two different image encoders E , namely DINOv2 ViT-B and CLIP ViT-B. As a result, the group sizes are not perfectly equal due to necessary trade-offs. Nevertheless, all groups are sufficiently large and of comparable size.

se suus js youm apiap o s! ys an saeu! ai jo pu e se pauasaidal yoee 'sias aw! y id e ym pauasad aq lm no

Which set of images looks more alike and focused on the same thing?

buyt awes au uo pasnooh pue ewis alow yool

[8] es

Click to show/hide instructions

Example:

[]as

[0

[a] as

[1] 1s

[] 1#s O [a] 1es O

Submit

se suus js youm apiap o s! ys an saeu! ai jo pu e se pauasaidal yoee 'sias aw! y id e ym pauasad aq lm no

Which set of images looks more alike and focused on the same thing?

uu awes au uo pesnong pue ewis aw yool

[8] es

Explanationbehind answerSet[A]:

Click to show/hide instructions

Example:

[]as

[]s O

zbuyn wes au uo pasnoo, pue aye aow syool sabew! yo jes ym

m

[a] as

[1] 1s

[] #s O [a] 1es O

Submit

p sahewi suuos jas youm apiap ot s! yse an saew! ai jo pub e se pauasaidal yoea 'sias aew! yo jed e um pauasaid aq lm no

Which set of images looks more alike and focused on the same thing?

un awes au uo pasnoot pue ewis aow yool

[a] es

Click to show/hide instructions

Example:

[]as

[]s O

zbuyn awes au uo pasnoo, pue ayle aow syoo sabew! jo jes ym

[a] as

tradet

orubba

西

[1] 1s

[] #s O [a] 1es O

wqns

D Benchmark

While MS shows very good results in our user study, we anticipate the development of improved alternatives in the future. To facilitate such advancements, we will release our collected data as a benchmark for evaluating neuron monosemanticity.

The benchmark will include the following files:

pairs.csv - Contains 1000 pairs of neurons ( r x , r y ) , along with user preferences R user ( k x ,k y ) and MS values computed using two different image encoders: DINOv2 ViT-B and CLIP ViTB. Each row includes the following columns: k_x , k_y , R_user , MS_x_dino , MS_y_dino , MS_x_clip , MS_y_clip .

top16_images.csv - Lists the 16 most activating images from the ImageNet training set for each neuron used in the study. Columns: k , x_1 , . . . , x_16 .

activations.csv - Provides activation values of all 50,000 ImageNet validation images for each neuron. Columns: k , a_1 , . . . , a_50000 .

With this data and by following our evaluation procedure, researchers will be able to compare their methods directly to MS under same conditions. They will have access to the same underlying information, specifically the complete set of neuron activations on the ImageNet validation set.

E Additional results on monosemanticity

E.1 Unnormalized plots

Monosemanticity scores across all neurons, without normalized index, are shown in Figure A6.We observe that neurons cover a wider range of scores as we increase the width of the SAE layer. Furthermore, for a given threshold of monosemanticity, the number of neurons having a score higher than this threshold is also increasing with the width.

0

10000

20000

30000

40000

Index

0.00

0.25

0.50

0.75

Monosemanticity Score

Expansion factor

No SAE

x1

x2

x4

x16

x64

E.2 Detailed statistics and more models

We report in Tables A2 and A5, the average ( ± std), best and worst monosemanticity scores across neurons for the two SAE variants, attached at different layers and for increasing expansion factors. Although average scores remain similar when increasing expansion factor, we observe a high increase between the original layer and an SAE with expansion factor ε = 1 . The best scores get consistentely better as expansion factor gets increased.

Until now, our analysis has focused on SAEs trained on CLIP ViT-L activations, evaluated using the MS score computed from embeddings produced by the DINOv2 image encoder E . To broaden this investigation, we now consider SAEs trained on activations from SigLIP SoViT-400m. As an alternative image encoder E , we adopt CLIP ViT-B for evaluation.

Tables A3 and A6 show average, best and worst MS computed using CLIP ViT-B as the vision encoder E . Even though less distinctively than in original setup, the neurons from SAEs still score better compared to the ones originally found in the model.

In Tables A4 and A7, we report MS statistics for SAEs trained for SigLIP SoViT-400m model computed using CLIP ViT-B as the vision encoder E . The results highly resemble the ones for CLIP ViT-L model.

In Figure A7 we plot MS across single neurons. We consider setups in which (a) neurons of CLIP ViT-L are evaluated with DINOv2 as the image encoder E , (b) neurons of CLIP ViT-L are evaluated with CLIP ViT-B as E , and (c) neurons of SigLIP SoViT-400m are evaluated with CLIP ViT-B as E . In all three cases SAE neurons are more monosemantic compared to the original neurons of the models. It shows that MS results are consistent across different architectures being both explained and used as E .

0.0

0.2

0.4

0.6

0.8

1.0

Normalized Index

0.0

0.2

0.4

0.6

0.8

(a) Neurons of CLIP ViT-L evaluated with DINOv2 ViT-B as the image encoder E

0.0

0.2

0.4

0.6

0.8

1.0

0.4

0.5

0.6

0.7

0.8

Monosemanticity Score

(b) Neurons of CLIP ViT-L

evaluated with CLIP ViT-B as the

image encoder

E

0.0

0.2

0.4

0.6

0.8

1.0

0.4

0.6

0.8

No SAE

SAE

(c) Neurons of SigLIP SoViT 400m evaluated with CLIP ViT-B as the image encoder E

In Figures A8 and A9, we plot again MS scores across neurons for SAEs trained with different expansion factors and sparsity levels, but using CLIP ViT-B as the image encoder E . We observe very similar patterns when compared to the MS computed using DINOv2 ViT-B. Both higher expansion factor and lower sparsity helps find more of the monosemantic units.

0.0

0.2

0.4

0.6

0.8

1.0

Normalized Index

0.2

0.4

0.6

0.8

1.0

Monosemanticity Score

Expansion factor

No SAE

(a) Impact of expension factor ε

0.0

0.2

0.4

0.6

0.8

1.0

Normalized Index

0.4

0.6

0.8

1.0

Monosemanticity Score

K

No SAE

1

10

20

50

(b) Impact of sparsity factor

K

0

2000

4000

6000

8000 1000012000

Index

0.2

0.4

0.6

0.8

1.0

Monosemanticity Score

Expansion factor

No SAE

E.3 Matryoshka hierarchies

We train and evaluate the SAE on embeddings extracted from iNaturalist [55] dataset using an expansion factor ε = 2 and groups of size M = { 3 , 16 , 69 , 359 , 1536 } . These group sizes correspond to the numbers of nodes of the first 5 levels of the species taxonomy tree of the dataset, i.e. the respective number of 'kingdoms' , 'phylums' , 'classes' , 'orders' , and 'families' .

To measure the granularity of the concepts, we map each neuron to the most fitting depth in the iNaturalist taxonomy tree to compare the hierarchy of concepts within the Matryoshka SAE with human-defined ones. To obtain this neuron-to-depth mapping, we select the top16 activating images per neuron, and compute the average depth of the Lowest Common Ancestors (LCA) in the taxonomy tree for each pair of images. For instance, given a neuron with an average LCA depth of 2 , we can assume that images activating this neuron are associated to species from multiple 'classes' of the same 'phylum' . We report the average assigned LCA depth of neurons across the Matryoshka group level in Table A8. We notice that average LCA depths are correlated with the level, suggesting that the Matryoshka hierarchy can be aligned with human-defined hierarchy. We additionally aggregate statistics of MS of neurons for each level. Average and maximum MS also correlates with the level, confirming that the most specialized neurons are found in the lowest levels.

F Reconstruction of SAEs

In Table A9 and Table A10, we report respectively R 2 and sparsity ( L 0 ), for the two SAE variants we compare in Section 4.2. As BatchTopK activation enforces sparsity on batch-level , during test-time it is replaced with ReLU ( x -γ ) , with x is the input and γ is a vector of thresholds estimated for each neuron, as the average of the minimum positive activation values across a number of batches. For this reason the test-time sparsity may slightly differ from K fixed at the value of 20 in our case.

We report in Table A11 the detailed metrics ( R 2 , L 0 and statistics of MS) obtained for SAEs trained with different K values considered in Section 4.2.

G Uniqueness of concepts

The sparse reconstruction objective regularizes the SAE activations to focus on different concepts. To confirm it in practice, we collect top-16 highest activating images for each neuron of SAE and compute Jaccard Index J between every pair of neurons. The images come from training set. We exclude 10 out of 12288 neurons for which we found less than 16 activating images and use Matryoshka SAE trained on the last layer with expansion factor of 16 . We find that J > 0 for 16000 out of 75368503 pairs ( > 0 . 03% ) and J > 0 . 5 for only 20 pairs, which shows very high uniqueness of learned concepts.

H Additional qualitative results

We illustrate in Figure A10 the highly activating images for the 'Pencil' neuron, which we used for steering in Figure 6. In Figures A11 and A12 we provide more randomly selected examples of neurons for which we computed MS using two different image encoders. In both cases we see a clear correlation between score and similarity of images in a grid.

0.9 > MS > 0.8

0.8 > MS > 0.7

0.7 > MS > 0.4

0.4 > MS > 0.3

0.3 > MS > 0.2

N#2669

N#2693

N#7344

N#5468

N#5207

N#2523

N#2741

N#8109

N#5697

N#9447

N#6677

N#3371

N#6421

N#8105

N#8370

N#5450

N#4147

N#1223

N#8501

0.9

0.0

MS (using DINOv2 ViT-B as the image encoder

E

)

N

VERRiOR

E-Mail

N#6030

N#7319

N#5034

N#1293

N#8905

0.9 > MS > 0.8

0.8 > MS > 0.7

0.7 > MS > 0.4

0.4 > MS > 0.3

0.3 > MS > 0.2

N#7430

N#9924

N#7327

N#9356

N#10017

N#810

N#3620

N#10365

N#797

N#7363

N#4982

N#2395

N#1793

N#2673

N#9867

N#9917

N#7304

N#5504

N#9007

N#6568

N#6029

N#3508

N#3711

N#7119

N#7322

0.9

0.0

MS (using CLIP ViT-B as the image encoder

E

)

SENIOR

APEX

CAT

HBUSeRCOM

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel metric for evaluating sparse autoencoders (SAEs) applied to image-text encoders such as CLIP. The metric appears to be correlated with human evaluations, offering a promising direction for interpreting multimodal representations. Additionally, the authors explore how SAE neurons can be used to steer multimodal large language models (VLMs), providing qualitative evidence of controllability. The paper echoes insights from prior work on LLMs, such as the increase in monosemanticity with wider SAEs.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n* The proposal of a metric for evaluating SAEs is needed and important. The metric is intuitive and seems to be correlation with human judgment.\n\n* The qualitative results show that interventions on SAE neurons in vision encoders can meaningfully steer the output of VLMs.\n\n* The paper is well written, clearly structured, and easy to follow.\n\n\n**Weakneses:**\n\n* Limited scope of encoders: The experiments are limited to CLIP-style encoders (CLIP and SigLIP). It is unclear whether the approach generalizes to other types of vision encoders. Including results on more recent state-of-the-art models (e.g., AIMv2 [1]) would strengthen the contribution.\n\n* The paper does not discuss or compare with other recent approaches to VLM steering, such as activation-based steering methods [2]. Activation steering is a common baseline in LLMs [3] and VLMs. A comparison and discussion with other steering method would properly position this paper.\n\n* The method focuses exclusively on steering the vision encoder. How does this compare with steering interventions applied within the LLM, such as on vision tokens after the connector in multimodal LLMs? A discussion of the trade-offs or benefits would be valuable.\n\n* Lack of steering quantitative evaluation: The paper only presents qualitative results for steering, which makes it difficult to assess practical effectiveness. Quantitative metrics or user studies would provide stronger evidence of the method’s utility.\n\n* The proposed metric includes a relevance score. It would be helpful to show how much this component contributes to overall performance -- e.g., through an ablation comparing versions with and without it.\n\n* The paper could better articulate how the proposed method helps reveal differences between vision encoders. Are there insights into architectural or representational differences?\n\n[1] \"Analyzing Fine-tuning Representation Shift for Multimodal LLMs Steering alignment.\" ICCV (2025).\n\n[2]  \"Multimodal autoregressive pre-training of large vision encoders.\" CVPR (2025).\n\n[3] \"Refusal in language models is mediated by a single direction.\"  NeurIPS (2024).\n\nQUESTIONS:\nPlease check weakneses.\n\nLIMITATIONS:\nVery short limitation paragraph at the end of the paper. I expect longer discussion on limitations as meantioned in the weakneses section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces a novel approach for evaluating the monosemanticity of Sparse Autoencoders (SAEs). The key contribution is the introduction of the MonoSemanticity (MS) score, which essentially quantifies the semantic clarity of each neuron based on its activation with respect to semantically similar inputs. The authors validate the proposed approach using a large-scale user study, an empirical evaluation of various SAE architectures and with a practical application, examining the potential of steering MLLM using SAE neurons.\n\nSTRENGTHS AND WEAKNESSES:\nThe introduction of a quantitative, activation-weighted similarity metric (MS score) for neuron-level monosemanticity is both timely and useful for the interpretability community. Indeed, there is a plethora of recent works that use SAEs for interpretability, without being able to validate what the SAEs have learned and how \"pure\" the representations are. \n\nDemonstrations of concept-based intervention in MLLMs (e.g., inserting or suppressing concepts in generated responses) showcase real-world value.\n\nThe authors also present a rigorous study on the effect of the expansion and sparsity of the considered SAE to the final MS score. \n\nI find that this work draws a lot of inspiration from network dissection approaches such as [1],[2], there is hardly any discussion about the similarities of the proposed method. The authors only mention \"Therefore, there have been investigations into the representations of these models which have found interpretable neurons\", without making any connections to these existing works. \n\nThe steering experiments are highly illustrative. Still, it would be useful to see failure cases or limitations, e.g., when MS is high but steering doesn’t meaningfully influence the LLM output and others. \n\nThe authors mention in the main text that \"we quantify MS of neurons using DINOv2 ViT-B\" (p.6 l.184). Did the authors investigate the sensitivity of the MS quantification with respect to different embedding spaces in the same setting? There are some tables in the appendix, but I would like some insight, if some encoders seemed to be more \"appropriate\" for this analysis. \n\n[1] Oikarinen et al., CLIP-dissect: Automatic description of neuron representations in deep vision networks, ICLR 2023\n\n[2] Panousis et al, DISCOVER: Making Vision Networks Interpretable via Competition and Dissection\n\nQUESTIONS:\nAn intriguing direction is whether this metric and setup can be used as a training signal to induce monosemantic neurons from scratch (potentially with some modifications)? This could advance the idea of learning networks that are inherently interpretable by design, rather than through post hoc analysis or post hoc SAE training.\n\nLIMITATIONS:\nThe authors have a limitation section with some comments. \nI don't see any other significant limitation that should be discussed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates the use of Sparse Autoencoders (SAEs) to enhance the interpretability of Vision-Language Models (VLMs), such as CLIP, by decomposing their representations into more monosemantic features. It introduces a novel quantitative metric, the MonoSemanticity Score (MS), to evaluate the extent to which individual neurons encode single concepts. The authors demonstrate that SAEs, particularly Matryoshka SAEs, produce neurons with higher monosemanticity compared to raw VLM neurons, as evidenced by greater similarity among highly activating images. The study further explores the practical implications of this increased monosemanticity, showing that SAE neurons enable unsupervised, concept-based steering of multimodal large language models (LLMs). This steering capability is demonstrated through experiments that manipulate neuron activations to emphasize or suppress specific concepts in model outputs, such as generating poems or descriptions biased toward or away from certain objects (e.g., pencils or knives). The paper provides experimental results on the impact of sparsity factors and validates the MS metric through human studies, highlighting its alignment with human perception. The work concludes by suggesting future research directions, such as extending the MS metric to text representations and exploring other dictionary learning methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The introduction of the MonoSemanticity Score (MS) is a novel contribution, providing a quantitative framework to assess neuron-level interpretability in VLMs. This metric addresses a gap in the literature by offering a systematic way to measure monosemanticity, which is critical for understanding and controlling complex models. The application of SAEs to VLMs, particularly the focus on Matryoshka SAEs, represents an innovative extension of prior work in dictionary learning.\n\n2. The ability to steer multimodal LLMs by manipulating SAE neuron activations opens new avenues for unsupervised control and interpretability, with potential applications in safe AI deployment and content filtering. The findings have broad implications for improving the transparency and controllability of VLMs, which are increasingly critical in real-world applications.\n\n3. The experimental setup is robust, with detailed descriptions of settings (Section 4.1) ensuring reproducibility. The use of CLIP ViT-Large and the exploration of sparsity factors (K) provide a thorough evaluation of SAE performance. The human study validating the MS metric strengthens the paper’s claims about its perceptual relevance.\n\n4. The paper is generally well-structured, with clear explanations of the MS metric computation (Figure 2) and illustrative examples (Figures 3, 6, 7). The figures effectively communicate the qualitative differences in monosemanticity and the impact of steering interventions.\n\nWeaknesses\n\n1. While the experimental results are compelling, the paper lacks a comprehensive comparison with alternative interpretability methods beyond SAEs (e.g., other dictionary learning approaches or feature visualization techniques). This limits the ability to assess whether SAEs are the optimal tool for achieving monosemanticity. Additionally, the trade-off between reconstruction quality and monosemanticity (e.g., R² dropping to 31.3% at K=1) is acknowledged but not thoroughly analyzed, leaving questions about practical deployment.\n\n2. The paper’s discussion of Matryoshka SAEs is somewhat underdeveloped. While referenced as a superior approach, the specific architectural or training differences that contribute to their performance are not clearly articulated, which may confuse readers unfamiliar with the referenced work. The extensive truncation of content (e.g., 11,944 characters in Section 4) also suggests that critical experimental details may be missing or relegated to an appendix, reducing accessibility.\n\n3. The steering experiments, while promising, are limited to specific use cases (e.g., pencils, knives) and lack generalization to broader or more complex concepts. The societal impact of steering, particularly for filtering harmful content, is mentioned but not deeply explored, missing an opportunity to connect the work to pressing ethical challenges in AI.\n\n4. While the MS metric is novel, the core idea of using SAEs for interpretability builds heavily on prior work (e.g., [3], [4], [5]). The paper could better distinguish its contributions from those of its predecessors to clarify its incremental novelty.\n\nQUESTIONS:\n1. Comparison with Alternative Methods: The paper focuses on SAEs but only briefly mentions other dictionary learning methods as a limitation (Section 5). Could the authors compare SAEs with alternative interpretability techniques, such as feature visualization [29] or other sparse coding approaches [30], to demonstrate their relative effectiveness? A clear comparison could strengthen the case for SAEs and potentially increase the paper’s evaluation score by addressing the Quality dimension more comprehensively.\n\n2. Trade-off Analysis: The trade-off between monosemanticity and reconstruction quality is noted (e.g., R² of 31.3% at K=1 vs. 66.8% at K=20). Could the authors provide a more detailed analysis of how this trade-off impacts downstream tasks, such as classification or generation accuracy, to guide practical deployment? Quantifying the impact on performance metrics could clarify the practical utility of SAEs and address concerns about Quality.\n\n3. Generalization of Steering: The steering experiments focus on specific concepts (e.g., pencils, knives). Can the authors demonstrate the applicability of their approach to more abstract or complex concepts, such as emotions or scenes? Providing such evidence could enhance the Significance of the work by showing broader applicability, potentially elevating the evaluation score.\n\n4. Matryoshka SAE Details: The paper highlights Matryoshka SAEs as superior but lacks detailed explanations of their implementation differences compared to standard SAEs. Could the authors clarify these differences and their impact on monosemanticity? Improved Clarity in this area could address reader confusion and strengthen the paper’s technical contribution.\n\n5. Ethical Implications: The potential for steering to filter harmful content is mentioned but not deeply explored. Could the authors discuss specific failure modes or risks (e.g., unintended biases in concept suppression) and propose mitigation strategies? Addressing this could improve the evaluation of ethical considerations, potentially increasing the score by demonstrating responsible research practices.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}