VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set
Authors: Shufan Shen
Junshu Sun
Qingming Huang
Shuhui Wang
Venue: NeurIPS 2025 poster ()
Keywords: Vision-Language Models
Representation Interpretability
Vision-Language Alignment
Sparse Autoencoder
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We propose a sparse autoencoder that maps the semantics of vision and language representations into a unified concept set.

Abstract: The alignment of vision-language representations endows current Vision-Language Models (VLMs) with strong multi-modal reasoning capabilities. However, the interpretability of the alignment component remains uninvestigated due to the difficulty in mapping the semantics of multi-modal representations into a unified concept set. To address this problem, we propose VL-SAE, a sparse autoencoder that encodes vision-language representations into its hidden activations. Each neuron in the hidden layer correlates to a concept represented by semantically similar images and texts, thereby interpreting these representations with a unified concept set. To establish the neuron-concept correlation, we encourage semantically similar representations to exhibit consistent neuron activations during self-supervised training. First, to measure the semantic similarity of multi-modal representations, we perform their alignment in an explicit form based on cosine similarity. Second, we construct the VL-SAE with a distance-based encoder and two modality-specific decoders to ensure the activation consistency of semantically similar representations. Experiments across multiple VLMs (e.g., CLIP, LLaVA) demonstrate the superior capability of VL-SAE in interpreting and enhancing the vision-language alignment. For interpretation, the alignment between vision and language representations can be understood by comparing their semantics with concepts. For enhancement, the alignment can be strengthened by aligning vision-language representations at the concept level, contributing to performance improvements in downstream tasks, including zero-shot image classification and hallucination elimination. Codes are provided in the supplementary and will be released to GitHub.

VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set
Shufan Shen 1.2
Junshu Sun
Qingming Huang Shuhui Wang
State Key Lab. of Al Safety, Institute of Computing Technology, Chinese Academy Sciences
University of Chinese Academy of Sciences
{shenshufan22z, sunjunshu21s, wangshuhui)@iict. ww..cta.......oo gmhuangQucas.ac. cn
Abstract
The alignment of vision-language representations endows current Vision-Language Models (VLMs) with strong multi-modal reasoning capabilities. However, the interpretability of the alignment component remains uninvestigated due to the difficulty in mapping the semantics of multi-moda representations into unified concept set. To address this problem, we propose VL-SAE, sparse autoencoder that encodes vision- aaiion-anguuge representations into its hidden activations. Each neu- ron in its hidden layer correlates to concept represented by semantically similar images and texts, thereby interpreting these representations with unified concept set. To establish the neuron neuron-concee concept correlation, we encourage semantically similar representations to exhibit consistent neuron activations during self-supervised train- ing. First, to measure the semantic similarity of multi-modal representations, we perform their alignment an explicit form based on cosine similarity. Second, we construct the VL-SAE with distance-base encoder and two modality-speciif de- coders to ensure the activation consistency semantically similar representations. Experiments across multiple VLMs (e.g., CLIP, LLaVA) demonstrate the superior capability VL-SAE in interpreting and enhancing the vision-language alignment. For interpretation, the alignment between vision and language representations be understood by comparing their semantics with concepts. For enhancement, the alignment can be strengthened by aligning vision-languaan representations at the concept level, contributing to performance improvements downstream tasks, including zero-shot image classification and hallucination elimination. Codes are available at htpps//gituh.coo/ssggueer//
Introduction
Vision- Vision-Language Models (VLMs) have demonstrated remarkable capabilities in multi-modal under- standing and reasoning, largely attributed the various training objectives [22, 41] and architec- tures [57, 27, 66, 65] that effectively align the semantics of vision and language representations. This alignment mechanism serves as a core component of VLMs, enabling them to make predictions by integrating information from both modalities [41, 34, 60, 27, 55, 17]. Nevertheless, current compre- hension of the alignment mechanism remains insufficient [13, 49, 38, 2]. This limited comprehension hinders our ability to analyze and address the misalignment cases, such as hallucinations [33, 26, 21]. To tackle this challenge, existing methods interpret representations VLMs by mapping their seman- ties to concepts. However, these methods either solely focus on the vision [32, 2] or the language [38] representations, as shown in Figure (a)) It remains an open problem to interpret vision-language alignment, which requires not only understanding the representation semantics but also comparing the semantics of both modalities in an interpretable manner [49].
Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Vision Representation
Language Representation
Concept Activation
Interpret
difleent consuppl person works at his desk in office
the view of the lighthouse
sky
(a)
Figure I: (a) Current interpretation methods are designed for single-modal representations. (b) Using current methods for each modality leads to mismatch in the concept sets, hindering the interpretation of the lision-languaa alignment. (c) We propose the VL-SAE to interpret the alignment mechanism by mapping the representation semantics of both modalities into unified concept set.
straightforward approach to address this problem to apply existing methods [46, 8] to map the representation semantics of each modality to concepts separately. With these concepts, we can compare the semantics of vision and language representations for interpreting their alignment mechanism. However, current methods struggle to conduct this comparison with concepts, they cannot effectively map the semantics of vision and language representations into a unified concept set. These methods acquire the concept set in pre-defined [24, 46, 2] or learnable [35, 8, 32] manner. Pre-defined methods rely on manually constructed concept sets, which often fail to capture the full spectrum of representation semantics [46, 63] and suffer from limited scalability due to the need to collect labeled samples for each concept [24]. Learnable methods employ the hidden activations of pre-trained sparse autoencoder (SAE) acquire the concept set, where each hidden neuron correlates to a concept and is defined by the samples that most strongly activate it. The neuron-concept correlation is established via end-to-end learning under self-supervision, eliminating the need for concept-specific annotations [8, 32j. Nevertheless, the end-to-end learning introduces uncontrollability into the concept set. Due to the uncontrollability. applying two separate SAEs vi- sion and language representations respectively causes the concept mismatch, i.e. neurons at the same location of different SAEs being correlated with different concepts, illustrated in Figure 1(b). As result, semantically similar vision and language representations can exhibit inconsistent activations under concept mismatch, hindering the comparisons of representation semanties with SAE.
As the concept mismatch stems from applying separate SAEs to different modalities, using a shared SAE for both modalities seems to be a promising solution. By encouraging semantically similar representations to exhibit consistent activations during self-supervised training, the hidden layer neurons can be co-activated by semantically similar images and texts as shown in Figure Icc), thereby mitigating the concept mismatch. Unfortunately, current SAEs designed for single-modal scenarios fail to achieve this objective for two primary reasons. (i) In contrast single-modal representations that use the inner product for measuring semantic similarity. multi-moda representations of different VLMs exhibit varying alignment strategies [34, 27, 43], which complicates the measurement of their semantic similarity. (ii) Multi-modal representations exhibit modality-speciffi distributions [50], making it difficult to ensure the activation consistency of semantically similar representations.
In this paper, we propose the VL-SAE, an SAE architecture equipped with an auxiliary autoencoder, mitigate the concept mismatch. We perform the representation alignment in explicit form with the auxiliary autoencoder to measure their semantic similarity and then employ VL-SAE to constrain the activation consistency among semantically similar representations. Specifically, (i) for the alignment strategy, we propose an auxiliary autoencoder that maps the original representations its intermediate representations. The autoencoder optimized via contrastive loss [43] to ensure that vision- -aaiunnl representations with higher semantic similarity exhibit greater cosine similarity between their intermediate representations. (ii) For the architecture of VL-SAE, we propose an encoder that activates neurons based on the Euclidean distance between normalized representations and neuron weights. This metric satisfies the triangle inequality and is related the cosine similarity, which correlates with semantic similarity rather than distributional information. These properties ensure that only semantically similar representations can close to the same weights and activate corresponding neurons. Additionally, we propose separate decoders to capture the distributional information of each modality for representation reconstruction, thereby preventing the encoder from embedding modality-speciif information into neuron activations that leads to concept mismatch.

---

In experiments, we construct VL-SAE on multiple VLMs, including Contrastive VLMs (CVLMs) [43] that achieve alignment via retrieval tasks and Large VLMs (LVLMs) [34, 1] that rely on question answering tasks. Exaluations of the proposed VL-SAE and current architectures [8, 15] demonstrate that VL-SAE possesses superior ability for mapping the semantics of vision and language represen- tations into a unified concept set. Subsequently, we utilize VL-SAE to interpret and enhance the vision-language alignment mechanism of VLMs. For interpretation. we explain the model predic- tion by visualizing the activated concepts of vision and language representations during inference. For enhancement, we propose two strategies to improve the performance of CVLM in zero-shot image classification [25, 59, 18] and LVLM in hallucination elimination [30] by enhancing the vision- lasounlanguuae alignment at the concept level. The contributions of our work are as follows:
We propose the VL-SAE, model to interpret the alignment mechanism of VLMs by mapping the semantics of vision-language representations into a unified concept set.
We apply VL-SAE on multiple widely-used VLMs and demonstrate the superior quality of the concept set learned by VL-SAE in experiments.
We show that VI-SAE can be employed to interpret the model prediction by visualizing the activated concepts, enhancing the performance of CVLMs on zero-shot image classification, and eliminating the hallucinations of LVLMs.
2 Related Work
Vision-Language Models. To handle the vision-language reasoning tasks, researchers have devel- oped multiple VLMs [41, 34, 1, 43, 60, 55, 17] by effectively aligning their vision and language representations. According to the pre-training tasks, these models can be categorized into Contrastive VLMs (CVLMs) [11, 43] that utilize retrieval tasks and Large VLMs (LVLMs) [34, 1, 60, 55, 17] that incorporate Large Language Models (LLMs) [56] for pre-training with question answering tasks. For CVLMs, current models include a vision encoder and text encoder, applied with contrastive training strategies that encourage semantically similar representations to achieve high inner product values [11]. For LVLMs, existing methods compose the model with vision encoder, an LLM, and a light-weight connector that maps the output of the vision encoder to input tokens of the LLM. Vision-language representations are implicitly aligned through pre-training with question answering tasks [57, 1]. Despite the various architectures and pre-training paradigms of VLMs, our proposed VL-SAE can interpret and enhance the alignment mechanism of their vision-language representations.
Representation Interpretation. Revealing the semantics of representations stands as one of the primary challenges tackled by interpretable machine learning [40]. Previous methods [24, 62, 64, 46) pre-define a concept set and then collect corresponding samples to derive concept vectors in the representation space. The representations are projected onto these vectors for interpretation. These methods are costly due to the requirements for constructing concept sets and collecting relevant samples [24]. Moreover, these methods encounter challenges in modifying model behavior by adjusting corresponding interpretations, as it is difficult to map these interpretations back to the original representations [46]. Recently, Sparse Autoencoder (SAE) [8. 32, 15, 28] has been recognized as an effective method to interpret representations and modify model behaviors through self--upprrvseed learning. Despite achieving advanced performance, current SAES remain unsuitable for vision-language representations due to the inconsistency between the learned concept sets of both modalities. To address this problem, we propose the VL-SAE with distance-basse -ased encoder and two modality-specific decoders to interpret vision-language representations with a unified concept set.
Understanding the Internal Mechanism of VLMs. There has been an increasing interest in investigating the internal mechanisms of VLMs through the lens of multi-modality [9]. Neuron-bases methods [16, 44, 37] reveal the existence of multi-modal neurons that translate vision information to corresponding information in text modality. These neurons are located through important scores like gradient [44] and metrics leveraging architecture information [37]. Representation-basee based methods attempt to interpret the semantics of vision-language representations with concepts. For CVLMs, SpLiCE [2] and TEXTSPAN [14] disentangle the vision representations of CLIP [41] to texts. For LVLMss, Parekh et al. [38] decompose the token representations to vision-language concepts. SAE- [35] leverages sparse autoencoder to token representations for efficient data sampling. Our method falls within the representation-bast based category. In contrast to existing methods that solely focus on
3

---

the vision or language representations, our VL-SAE investigates the alignment of vision-languue language representations by separately mapping their semantics into unified concept set.
3 Methodology
We first introduce the architecture of current VLMs [41, 34, 1] and SAE [8, 32] (Section 3.1). Then, we perform the VLM alignment in an explicit form (Section 3.2). Finally, we propose the VL-SAE to map the semantics of vision-language representations into a unified concept set (Section 3.3).
3.1 Preliminaries
Contrastive Vision Language Models (CVLMs). Existing CVLMs [41, 43] typically consist of vision encoder and a language encoder. Given an input image and text, these encoders independently compute their respective representations. The semantic similarity between the image and text is then estimated through the cosine similarity between their corresponding representations. By collecting a large number of image-text pairs, CVLMs achieve the Dision-language alignment by maximizing the representation similarity between semantically similar image-text pairs.
Large Vision Language Models (LVLMss. A general LVLM architecture [34, 27, 1] includes a vision encoder, an L.MM, and a connector. An input image is first encoded through the vision encoder and then transformed into image tokens via the connector. Subsequently, these image tokens are concatenated with text tokens and provided as input to the LLM for text generation. With the image-text pairs, the vision-language representations of LVLMs are aligned by pre-training the model to generate textual answers for the text question related to the image content.
To conduct a uniform analysis of both types of models, we utilize the outputs of the vision and language encoders in CVLM for analysis. For LVLM, representations of the image and text tokens in the LLM's hidden layer are averaged across the token axis for analysis. For convenience, we utilize X,, X2 E Rd to uniformly represent the vision-language representations of CVLM and LVLM.
Sparse Autoencoder (SAE).SA SAE is an autoencoder with sparsity constraints on its hidden activations. Given an input Rd, SAE first transforms it hidden activations h Rh using encoder E: Rd + Rh,, and then maps it back to the representation space via decoder D ; R² + Rd,
& = D(h) = (Exx))))).
where a R² + R denotes the sparsification function. SAE is trained with the reconstruction loss || -x|2 in a sulf-suue manner. For sparsity constraints, previous methods usually select ReLU as the sparsification function a and integrate the / norm of h into the loss function [8]. Recently, researchers have found that directly adopting the top-k operation as sparsification function not only eliminates the need for an additional loss term but also demonstrates superior scalability [15, 46).
3.2 Explicit Representation Alignment
To ensure the activation consistency among semantically similar vision-language representations for alleviating the concept mismatch, a prerequisite is measuring the semantic similarity of representations from both modalities. For semantic similarity measurement, more semantically similar vision- language representations have higher cosine similarity in CVLMs [41, 43), which is explicitly constrained via the pre-training objective. However, it difficult measure the semantic similarity of LVLM representations [34, 1] due to their implicit alignment mechanism [50]. To address this problem, we propose employing an auxiliary autoencoder to convert the implicit alignment mechanism into an explicit alignment mechanism. Specifically, given an image-text pair, we input them into the VLM extract representations (X,, Xt). The autoencoder transforms these representations into intermediate representations x x Rd via encoders {Eu, Et} and then maps them back into the original representations &,, &1 Rdd via decoders (D.., Di). (x(x)) (Ei()))
x D, (x() D,, (x,,(),,,, &
With the contrastive loss InfoNCE [41] and the reconstruction loss, the intermediate representations of image-text pairs can be aligned in cosine similarity while preserving the information of the original representations (details of InfoNCE are provided in Appendix A),
L(x,, xi) Inf (NfE xf, x x( ||&.. xx.11 + |||
xx 11.

---

where X . denote the intermediate representations of negative samples with dissimilar seman- ties from the given image-text pair. With this autoencoder, can transform implicitly aligned representations into explicitly aligned intermediate representations, enabling the measurement of semantic similarity between vision and language representations. Note that this component is only utilized for LVLMs with the implicit alignment mechanism. For CVLMs with the explicit alignment mechanism, we directly use their original representations x X Xj = X; for interpretation.
3.3 Interpreting Vision-Language Representations with a Unified Concept Set
Based on explicitly aligned represen- tations, we propose the VL-SAE with distance-based encoder and two mopality-specific decoders to ensure the activation consistency of semanti- cally similar representations, thereby alleviating the concept mismatch and interpreting multi-modal representa- tions with a unified concept set.
Representation
Top-K Activation Arca
Concept Activation
Learnable Weight Vector
IR
X
h
D
X
Encoder. As the vision-language rep- resentations are aligned through co- sine similarity, propose an encoder that activates neurons based on cosine similarity between the corre- sponding learnable weight vectors and the imput representations. However, since cosine similarity does not satisfy the triangle inequality, two representa- tions with high cosine similarity may not exhibit consistent similarity with the same weight vector, which hinders their activation consistency. To address this problem, design distance g R Rd - [0, 2] that utilizes the Euclidean distance between normalized inputs. Given the representation x" E: IR and weight vector w, E Rd corresponding to the i-th hidden neuron, their distance is calculated by
X
h
D
X
Distance-Based
Mlodaiit-Speccife Decoder
Encoder
Figure 2: VL-SAE consists of a distance- based encoder that maps semantically similar vision-language representations to similar activations, and two modality specific decoders that reconstruct original representations from these activations.
g( w.)
x
W;
= /2 2xos(x", w:),
(4)
x
wwlle
where cos(x", w;) denotes the cosine similarity x" and w,. Since the distance g a variant of the Euclidean distance, it naturally satisfies the triangle inequality. This indicates that the difference in distance values between the weight vector w; and vasion-language representations x xf cannot exceed the distance between the two representations,
(5) The proposed metric exhibits negative correlation with cosine similarity, suggesting that as the cosine similarity of vision-language representation increases, the upper bound of the difference in their activations decreases accordingly. We utilize the negative value of y as the activation value, thereby ensuring the positive correlation between the activation value and cosine similarity.
[g(x w.) g(x], wi) g(x,,xf)
Ex (xx
[1 = g(x", w.) = v2 2cos(x w.),
(6)
where Ex Ex(x") [i]] denotes the i-th value of (xxx e R² We introduce constant 2 to ensure the non non-negativity of the activation values. For the sparsification function a, we employ the top- k function (i.e., retaining top-k largest values while setting others to zero) considering its strong scalability [15, 47, 48],
= ==Tox(E((xx)),
The number of activated neurons & serves hyper-parameter. This distance-based encoder bounds the activation discrepancy of semantically similar representations through the triangle inequality, promoting each neuron to be co-activated by semantically similar images and texts.
5

---

Decoder. Despite bounding the activation discrepancy of lasion-language representations through the distance-basee based encoder, remains insufficient for interpreting representations from distinct distribu- tions. This is because the decoder reconstructs representations solely based on hidden activations. Reconstructing vision-language representations with the same decoder leads to the incorporation of distributional information into the hidden activations during training, thereby reducing the activation consistency of lasiuala representations that have similar semantics.
To map the hidden activations into representations of distinct distributions, we propose employing separate decoders for each modality. Given the activations hg, h, R of vision and language representations x xf E R. two moddlltt-sppecifi decoders D Dj are utilized to transform these activations back to their original representations, respectively,
, = (h,)), xi = (hi)-
(8)
VL-SAE trained through the reconstruction loss of visual-language representations,
C(x) xf) = || x 1 - || x1122-
(9)
With the separate decoders for storing distributional information, VL-SAE can encode semantically similar vision-language representations to consistent concept activations as interpretations, and then map these activations back to representations of distinct distributions.
Experiments
In experiments, we first provide the implementation details for constructing VL-SAE based on VLM representations in Section 4.1. Next, we evaluate the concepts Iearned by VL-SAE in Section 4.2. Finally, we integrate VL-SAE into the inference process of pre-trained VLMs interpret and enhance their vasion-language alignment mechanisms in Section 4.3.
4.1 Implementation Details for VI-SAE Construction.
VLM Selection. To demonstrate the broad applicability of the proposed VL-SAE, we build it upon multiple representative VLMs, including CVLMSs (OpenCLIP [43] with ViT-B/32, ViT-B/16, ViT-L/14, ViT-H/14) pre-trained with retrieval tasks, and LVLMs (LLaVA115 [34], Qwen-VI. (1]) pre-trained with question answering tasks. For CVLMs, VL-SAE is constructed using the output from both their vision and language encoders. For LVLMs. VL-SAE is constructed using the output features of the LLM hidden layers (the 29-th layer of LLa and the 26-th layer of Qwen-VL).
Datasets. We fed the CC3M dataset [45] containing 3 million image-text pairs into the VLM to extract the corresponding vision- language representations. These representations are randomly divided into training and test sets at ratio of 4:1.
Training Strategies. For LVLMS [34, 11, first train the auxiliary autoencoder for 50 epochs to perform the explicit representation alignment. The training process is configured with a batch size of 2048, weight decay of 0.01, and learning rate of 5e-5. Then, the VL-SAE is trained for 10 epochs based on the intermediate representations of the autoencoder with a batch size of 512 and a learning rate of le-4. For CVLMs [43], we adopt the same strategy as the LVLMs but omit the training of the auxiliary autoencoder because CVLM representations are naturally aligned through cosine similarity.
4.2 Evaluating the Quality of VL-SAE Concept Set
Quantitative Evaluation. For the concept set, we evaluate (i) whether each neuron is activated by semantically similar images and texts, and (ii) whether diverse semantic concepts are learned across different neurons. These two aspects are quantitatively assessed using the CLIP score [41]. Specifically, we first gather the maximally activating images and texts for each neuron of the VL- SAE's hidden layer to describe its semanties. Then, the CLIP score is employed to quantify the semantic similarity between images and texts within the same neuron (Intra-Sime -imilarity)) and across different neurons -imelr-Simllartt).. The Intra-Similarity evaluates the semantic consistency of images and texts within each concept, while the Inter- iitrr-Simllarity assesses the semantic diversity across the concept Detailed descriptions of both metrics are presented in Appendix A.
For comparison, we consider two baselines using current SAE architectures (i) SAE-D, which employs distinct SAEs for vision and language representations, and (ii) SAE-S, which uses single

---

OpenCTIP-VViE -VT--1114
OpenCLIP- -
LLaVAL
Qwen-VL
Similrrty
Intra Somilarry
Intro Similaarcy
Intra- Simihrty
0.0
6.28
321
0.15
0.2
3022
2012
255
2014
600
312
8.14
2010
25
312
18-4
201
Top-k
Top-K
Top-K
Top-K
OpenCLIP- VIT-VV1--11
OpenCLIP- VIT--V1-1
5
Qwen-VL
laser Simitatt
Ioser Simitrity
Iaser Simiterty
Iaser Simbtaryy
0.10
312
2044
1424
Top-K
Top-K
Top-X
Top-K
SAB-D Intra (1)
SAB-S Encee (1]
VLSAB Ieten [f]
SAE-D Lncer (1)
SAB-S Eater (1)
VISAD Inter (.)
Figure 3: Quantitative evaluation of the learned concept set. We compare the VL-SAE with other methods on the consistency between vision and language semanties within the same neuron (Intra-
Similarity) and the semantic diversity across different neurons
on multiple VLMs.
SAE-D
SAE-S
VL-SAE
Soccer player, head coach of the national team. Made in symbol with heart colors
chaos and wires. A summary is when the main are restated.
Hundreds devotoes are celebrating the festival. Students play with colors on the eve of festival.
landmark red and white lighthouse.
tricolor.
A view of the lighthouse.
Figure 4: Qualitative comparisons among concepts of SAE-D, SAE-S and the proposed VL-SAE. All SAES are trained with the vision-language representations of LLaVA 1.5.
SAE shared by both modalities. As shown in Figure 3, VL-SAF exhibits higher intra-similarity and lower inter-similarity compared to these methods, consistently across various VLMs and different numbers of activated concepts. This suggests that the concepts learned by VL-SAE possess more consistent vision-language semantics and richer semantic diversity than current methods.
Qualitative Evaluation. We provide qualitative comparisons among SAE-D, SAE-S, and our VL- SAE in Figure 4. SAE-D, which employs separate SAEs for each modality, can extract concepts with consistent semantics within the same modality but encounters the concept mismatch issue as illustrated in Figure 1(b). SAE-S, which utilizes shared SAE for both modalities, demonstrates less significant concept mismatch compared to SAE-D. However, semantic inconsistencies still exist, such the conflation of the semantics of "wire" and "summary tutorials" under a single concept. In contrast, the concepts in VL-SAE consist of samples with consistently aligned semantics.
4.3 Interpreting and Enhancing the Vision-Language Alignment
With the VL-SAE that correlates the semantics of vision and language representations with a unified concept set, we can interpret and enhance the vision-languag alignment mechanism in CVLMs and LVLMs by comparing the semantics of both modalities using the concept set.
Interpreting Vision-Language Alignment of CVLMSs Given an image-text pair, we separately map the corresponding vision and language representations into the unified concept through VL-SAE. Figure depicts several examples by visualizing the concepts activated by representations for each modality and the aligned concepts that are co-activated by both modalities. We observe that (i) single- modal representations can activate concepts with irrelevant semantics (e.g.. the representations of motorcycle image activate the concept of a car). This phenomenon reveals the mismatch between representation similarity and semantic similarity in single-modal samples, which arises due to contrastive learning's inability to effectively model the relationships among representations within the same modality. (ii) The aligned concepts exhibit strong semantic similarity to both the given image and text, suggesting that irrelevant concepts activated by a single-modal representation are not simultancously activated by representations of the other modality.
7

---

Input Image
Concepts activated by image embeddings
Concepts activated by text embeddings
Concepts co-activated
image & text embeddings
Motorcycle and scooter next to each other.
Motorcycle and scopter each other.
Motoreycle and scooter next to each other.
Bicycles motorcycle center.
black black top.
Motorcycle parked by dirt road.
Input Text
A black Honda
motorcycle parked in front of garage. Vintage classic mini cooper showing front grill. Motorcycla parked by road. Figure 5: Interpreting the vision-language alignment of Open an image-text pair, we visualize the concepts activated by corresponding vision and language repre- sentations separately, as well as the aligned concepts co-activated by both modalities. Concepts only related to single-mode representations
Bicycles motorcycle
in a town center.
with VL-SAE. Given
highlighted in purple.
Table 1: Performance of OpenCLIP with varying model sizes in zero-shot image classification tasks.
Caltech
Country211
Mean Acc.
Cifar10
Cifar100
DTD
Eurosat
Flowers
Food
GTSRB
MNIST
SST2
STLI0
Syna. 1999
Pets
ViT-B/32
86.5
93.6
75.5
16.7
55.9
47.2
71.6
82.7
49.3
69.8
90.7
57.5
96.6
68.7
68.7
+VL-SAE
86.7
93.8
75.7
16.7
56.9
51.0
71.8
82.8
50.3
71.8
90.7
57.9
96.9
69.4
69.5
ViT-B/16
86.7
94.9
76.9
20.3
56.5
52.6
71.4
86.6
46.1
66.0
90.3
59.7
97.9
70.8
69.8
+VL-SAE
87.2
95.1
77.0
20.4
56.7
53.2
71.6
86.8
49.1
67.8
90.9
60.4
97.9
71.4
70.4
VIT-L/14
87.6
95.8
78.3
24.4
61.5
57.8
74.4
88.8
51.6
64.5
92.9
60.3
98.5
74.0
72.2
+VL-SAE
87.9
95.9
78.6
24.5
62,8
60.3
74.5
88.8
51.9
69.1
93.0
60.6
98.6
74.3
72.9
ViT-H/14
88.2
97.5
84.7
29.8
67.9
72.7
92.7
58.3
72.9
94.5
64.1
75.2
76.9
+VL-SAE
88.5
97.7
85.0
30.0
68.4
76.7
80.1
92.9
59.5
76.1
94.7
65.2
98.6
75.3
77.8
Enhancing Vision-Language Alignment of CVLMs. According to the above interpretations, mapping vision-language representations unified concept enables filtering the irrelevant concepts activated by single-modal representations. Inspired by this, we enhance the rision-language alignment CVLMs by aligning their mulli-modia representations in concepts. Specifically, for an input image and text æz, their semantic similarity depends not only on the cosine similarities of their representations x,, Xt, but also on the cosine similarities of their concept activations h h,
y cos((o X X1) + a,.cos(h, hi),
(10)
where a₂ is task-specific hyperparameter that controls the proportion of concept-based predic- tions. We enhance the alignment mechanism of multiple CVLMs and conduct evaluations across various zero-shot image classification datasets [59, 25, 41, 18, 36, 3, 19, 10, 39, 51, 7, 61]. As Table shows, mapping multi-modaa representations to a unified concept set for predictions achieves consistent performance improvement across different models and datasets.
Interpreting Vision-Language Alignment of LVLMs. For LVLMs, we interpret their alignment mechanism by visualizing the concepts activated by the vision and language representations during the text generation process. As Figure shows, given the input image of a kitchen including ovens and stoves, the generated text exhibits object hallucinations of the microwave and refrigerator. By analyzing the activated concepts, we observe that the semantics both vision and language representations are correlated with the kitchen scene but emphasize different objects. Specifically, the vision representation focuses on ovens and stoves, whereas the language representation associated with microwaves and refrigerators, which correspond to the hallucinated objects. This phenomenon indicates that the object information in the vision representation is inadequately transferred into the language representation, resulting in hallucinations in the generated text.
Enhancing Vision-Lannn Language Alignment of LYLMs. As the hallucinations L.V.MM are correlated to the misalignment between vision and language representations, we leverage VL-SAE to mitigate

---

Input Image
Concepts activated by image embeddings
Generated Text
Concepts activated by text embeddings =
The image features Kitchen with
stainless steel oven and stove top
positioned
Thss 1s a gos stove with ovinss
addition to the oven and stove top.
metal refrigerator
kitchen also contains sink, microwave, and refrigerator: There
are several bottles placed around
White kitchen with on electric stove and an aven.
kitchen has new refrigerafor, microwave and utensils
kitchen, with some on
countertops and others on the floor.
cup can be seen on the countertop
Cooking stove In the kitchen
well. The kitchen appears
Worker holds pizza JusT taken from the oven
well-equipped and ready for cooking.
Figure 6: Interpreting the vision-language alignment of LLaVA1.5 with VI-SAE. Given an input image and the text generated by LVL.Ms, we visualize the concepts activated by corresponding vision and language representations. The hallucination concept activated by language representations facilitates deeper understanding of object hallucinations in the text.
Table 2: Experimental results on POPE for ballucination elimination.
LLaVA1.5
Qwen-VI.
Setting
Decoding
Accuracy
Precision
Recall
Score
Accuracy
Precision
Recall
Fl Score
Regular
82.93
92.01
72.13
80.87
85.20
95.99
73.47
83.23
Random
VCD
85,53
93.68
76.20
84.04
86.33
96.11
75.73
84.71
VL-SAE
87.07
97.28
76.27
85.50
86.47
97.15
75.13
84.74
Regular
81.13
87.96
72.13
79.27
83.80
94.47
71.80
81.59
Popular
VCD
83.63
89.51
76.20
82.31
85.73
94.30
76.10
84.21
VL-SAE
85.87
94.39
76.27
84.37
85.97
96.15
74.93
84.22
Regular
78.67
83,03
72.07
77.16
82.33
89.88
72.87
80.49
Adversarial
VCD
81.10
84.47
76.20
80.13
83.77
90.03
75.93
82.39
VL-SAE
83.60
89.44
76.20
82.29
86.10
97.29
74.26
84.23
hallucinations by enhancing the vision-language alignment at the concept level. Specifically, with the vision and language representations X Xt, we obtain refined language representation & by aligning its concept activation h; with vision concept activation h,, through VL-SAE,
(1 - (1)x) + DDD (hr + (h,)))
(11)
The modified language representation & is utilized to mitigate hallucinations through contrastive decoding [26]. More implementation details are provided Appendix A. Table 2 shows the perfor- mance comparisons on the POPE [30] benchmark. As method for representation interpretation, VL-SAE surpasses the VCD [26] that specializes in hallucination elimination. The effectiveness of
VL-SAE further highlights the potential of representation interpretation methods to improve model performance on downstream tasks by enhancing the vision-language alignment.
4.4 Ablation Studies Visualizations
Effects of the VL-SAE Architecture. The ablation studies of the proposed components are provided in Table First, replacing the standard encoder with the cosine based encoder (+Cosine-bul based Encoder), which activates hidden neurons based on cosine similarity, can improve the quality of learned concepts, as vision-language representations are explicitly aligned through cosine similarity.
Moreover, replacing the co- sine similarity with the dis- tance proposed in Equation 4 can improve the concept quality (+Distance- based En- coder). This is because the distance satisfies the triangle inequality, enabling each hid- den neuron to be activated by more semantically similar visual- visual-languse language representations.
Table 3: Ablation studies of the proposed architecture.
OpenCLIP OpenCLIP-Vii--H14
LLaVAI.5
Intra-Sim, (?) Inter-Sim. (ntra-ta (f) Inter-Sim (L)
Standard SAE
0.1890
0.1688
0.2086
0.1902
Auxiliary Autoencoder +Cosine-hased Encoder
0.2092
0.1908
0.1891
0.1518
0.2103
0.1902
+ +Distance- based Encoder
0.2016
0.1357
0,2216
0.1842
+Modality Decoder
0.2134
0.1149
0.2257
0.1828
-Auxiliary Autoencoder
0.2084
0.2034
Additionally, the improvement achieved by adopting modality-

---

Table 4: Concept quality of VL-SAE pre-trained with
Table 5: Ablation studies of the sparsifi- cation method in VI-SAF.
different proportions of the CC3M dataset.
Data Percentage
10%
30%
50%
70%
100%
Method Intra-Similarity
Inter-Similarity
Intra-Simi Siriiariillartty (1)
0.2029
0.2129
0.2196
0,2222
0,2299
0.2142
0.1809
Inter-Similarity (0)
0.1597
0.1306
0.126
0.1256
0.1220
Top-K 0.2442
0,1373
Concept 2153
Concept 5478
Concept 6169
Concept 7301
16
201 HEV
Classic and romantic photograph of couple. Bouquet by person at this beautiful wedding.
Summer vocation young Baseball player hits a-run home Baseball player hits run scoring single.
A banner or poster for happy new year. background for happy new year.
woman lying on beach.
Sunny summer at the beach with palm trees and glasses.
Figure 7: Qualitative evaluation of the learned concept set. We provide the maximally activating images and texts for several concepts VL-SAE trained with the representations LLaVA 1.5.
specific decoders stems from mitigating the adverse influence of distributional discrepancies in multi-modal representations on the encoder. Furthermore, directly training the VL-SAE with orig- inal representations of LVLMs (-Auxiliary Autoencoder) causes the failure of other components, highlighting the necessity of transforming the alignment mechanism an explicit form.
Effects of the Dataset Volume. In Table 4, we train VL-SAE models with different proportions of the CC3M dataset and report their aitra-Smmllartty and 00229~0.22999 2092299 and Simi--iiilartty (0. 1597--0.111 1220) of the concepts learned by VL-SAE improve accordingly. This suggests that VL-SAE can learn higher-quality concepts by leveraging larger amounts of data in a self-supervised manner.
As the volume of data increases,
the Autr-Simillartt (0.2029->0.
Effects of the Sparsification Method. In Table 5, we compare the performance of utilizing Top- [47, 46] and LI loss [8] for sparsification. The coefficient of LI loss is set to le-4. We find that employing Top-K sparsification yields superior concept quality compared to L1 loss, as the latter lacks precise control over the number of activated hidden neurons. This limitation hinders its ability to effectively balance sparsity in neuron activation with the self-supervised representation reconstruction loss. Our results are consistent with previous studies [15], which also indicate that Top-K sparsification is a more effective and scalable approach.
Visualizations of Concepts Learned by VL-SAE. In Figure 7, we visualize the learned concepts of VL-SAE, which are represented by their maximally activating images and texts. The images and texts of the same concept exhibit highly similar semantics, demonstrating that VL-SAE can effectively alleviate the concept mismatch. Additionally, VL-SAE obtains concepts with abstract semantics rather than focusing exclusively on semantics related to visual appearance. For example, concept 5478 is style-agnostic and activated by both natural and cartoon images, which depict humans lying on a beach. Concept 7301 is color-agnostic and activated by images of happy new year posters.
5 Conclusion
In this work, map the semantics of vision-language representations into a unified concept set with VL-SAE, which is trained by encouraging the consistency of its hidden activations among semantically similar representations under self-supervision. To measure the semantic similarity of multi-modal representations, we perform their alignment in an explicit form that correlates the semantic similarity with cosine similarity. To ensure the activation consistency among semantically similar representations, we propose the VL-SAE architecture, including a distance- two sppecfff decoders. Experimental results demonstrate the superior ability of VL-SAE to interpret and enhance the alignment mechanism of VLMs. For future work, we will apply the proposed method more VLMs and investigate capabilities in additional tasks such as model unlearning and continuous learning. For limitations discussion, please refer to Appendix C.4.
encoder and
10

---

Acknowledgement and Disclosure of Funding
This work was supported in part by the National Key R&D Program of China under Grant 2023YFC2505004, in part by the National Natural Science Foundation of China 62236008, and in part by the Fundamental Research Funds for the Central Universities. The authors would like to thank Siqi Zhang, Yue Wu, and the anonymous reviewers for their constructive comments and suggestions that improved this manuscript.
References
[1] Jinze Bai, Shuai Bai, Shusheng Yang. Shijie Wang. Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, 2023.
[2] Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flávio P. Calmon, and Himabindu Lakkaraju. Interpreting CLIP with sparse linear concept embeddings (splice). In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 38: Annual Conference Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10- 15, 2024, 2024.
[3] Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. mood-i00-mining discriminative components with random forests. In Computer vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 6- 12, 2014, proceedings, part VI 13, pages 446 461. Springer, 2014.
[4] Hila Chefer, Shir Gur, and Lior Wolf. Generic attention model explainability for interpreting bi-modal and encoder-decoder transformers. In Proceedings of the IEEF/CVF international conference on computer vision, pages 397-406, 2021.
[5] Junxi Chen, Liang Li, Yunbin Tu, Li Su, Zhe Xue, and Qingming Huang. Generalizing single- frame supervision to event-level understanding for video anomaly detection. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.
[6] Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In 2014 IEEE Conference on Computer Vision and Putlern Recognition, CVPR 2014, Columbus, OH, USA, June 23-28, 2014, pages 3606-3613. IEEE Computer Society, 2014.
[7] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsuper- vised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215-223. JMLR Workshop and Conference Proceedings. 2011.
[8] Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models. ArXiv preprint, abs//330908800,, 2023.
[9] Yunkai Dang, Kaichen Huang, Jiahao Huo, Yibo Yan, Sirui Huang, Dongrui Liu, Mengxi Gao, Jie Zhang, Chen Qian, Kun Wang, et al. Explainable and interpretable multimodal large language models: A comprehensive survey. ArXiv preprint, abs/2412.021044 2024.
[10] Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web). IEEE signal processing magazine, 29(6):141114 2012.
[11] Xin Dong, Shangyu Chen, and Sinno Jialin Pan. Learning to prune deep neural networks via layer-wise optimal brain surgeon. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA. USA, pages 4857-4867, 2017.
11

---

[12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[13] Jiali Duan, Liqun Chen, Son Tran, Jinyu Yang, Yi Xu, Belinda Zeng, and Trishul Chilimbi. Multi- modal alignment using representation codebook. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 15630-15639. IEFE, 2022.
[14] Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip's image representa- tion via text-based decomposition. ArXiv preprint, abs/2310.05916, 2023.
[15] Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. ArXiv preprint, abs/2406.0400 00093, 2024.
[16] Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrow, Ludwig Schubert, Alec Radford, and Chris Olah. Multimodal neurons in artificial neural networks. Distill, 6(3)=330, 2021.
[17] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. ArXiv preprint, abs/2407.21783. 2024.
[18] Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12()):2117-2226, 2019.
[19] Sebastian Houben, Johannes Stallkamp, Jan Salmen, Mare Schlipsing, and Christian Igel. Detection of traffic signs in real-world images: The german traffic sign detection benchmark. In The 2013 international joint conference on neural networks (IJCNN), pages 1-8. leee, 2013.
[20] Edward J. Hu, Yelong Shen, Phillip Wallis. Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
[21] Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospectionnalllocatoo. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13418-13427, 2024.
[22] Chao Jia, Yinfci Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hicu Pham, Quoc V. Le, Yun- Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 1904-4911 PMLR, 2021.
[23] Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, et al. Exploring concept depth: How large language models acquire knowledge and concept at different layers? ArXiv preprint, abs/2404.07066, 2024.
[24] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Viégas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmissun, Stockholm, Sweden, July 10-15 15, 2018 volume 80 of Proceedings of Machine Learning Research, pages 2673-2682 PMLR, 2018
12

---

[25] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[26] Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large languaee models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 13872-13882, 2024.
[27] Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. BLIP-2: bootstrapping language- image pre-training with frozen image encoders and large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Searlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honoluiu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 19730-19742. PMLR, 2023.
[28] Qing Li, Jiahui Geng, Derui Zhu, Fengyu Cai, Chenyang Lyu, and Fakhri Karray. Sauce: Selective concept unlearning in vision-language models with sparse autoencoders. ArXiv preprint, abs/2503. 14530, 2025.
[29] Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. In Anna Rogers, Jordan Boad-Graber, and Naoaki Okazaki, editors, Proceed- ings of the s/ Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 12286122122 Toronto, Canada, 2023. Association for Computational Linguistics.
[30] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 292-305, Singapore, 2023. Association for Computational Linguistics.
[31] Dongze Lian, Daquan Zhou, Jiashi Feng, and Xinchao Wang. Scaling shifting your features: A new baseline for efficient model tuning. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, Cho, and A. Oh, editors, Advances Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
[32] Hyesu Lim, Jinho Choi, Jaegul Choo, and Steffen Schneider. Sparse autoencoders reveal selective remapping of visual concepts during adaptation. ArXiv preprint, abs/2412.05276, 2024.
[33] Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. A survey on hallucination in large vision-language models. ArXiv preprint, abs/2402.002 00253, 2024.
[34] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10- 16, 2023, 2023.
[35] Hantao Lou, Changye Li, Jiaming Ji, and Yaodong Yang. Sae-v: Interpreting multimodal models for enhanced alignment. ArXiv preprini, 14./2502..15144 2025.
[36] Maria-Elena Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Sixth Indian Conference on Computer Vision, Graphics & Image Pro- cessing, ICVGIP 2008, Bhubaneswar. India, 16-19 December 2008, pages 722-729, IEEE Computer Society, 2008
[37] Haowen Pan, Yixin Cao, Xiaozhi Wang, Xun Yang, and Meng Wang. Finding and editing multi-modal neurons in pre-trained transformers. ArXiv preprint, abs/2311.07470. 2023.
13

---

[38] Jayneel Parekh, Pegah Khayatan, Mustafa Shukor, Alasdair Newson, and Matthieu Cord. A concept-based explainability framework for large multimodal models. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors, Advances Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NewrIPS 2024, Vancouver, BC, Canada, December 10- 15, 2024, 2024.
[39] Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V. Jawahar, Cats and dogs. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, June 16-21, 2012, pages 3498-3505. IEEE Computer Society, 2012.
[40] Eleonora Poeta, Gabriele Ciravegna, Eliana Pastor, Tania Cerquitelli, and Elena Baralis. Concept- based explainable artificial intelligence: A survey. ArXiv preprint, 1tss/332.112366. 2023.
[41] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 8748-8763, PMLR, 2021.
[42] Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Ob- ject hallucination in image captioning. In Ellen Riloff, David Chiang, Julia Hockenmaier, and iic' Tsujii, editors, Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pages 4035-4045, Brussels, Belgium, 2018. Association for Computational Linguisties.
[43] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarezyk, and Jenia Jitsev. LAION-5B: an open large-scale dataset for training next generation image-text models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
[44] Sarah Schwettmann, Neil Chowdhury, Samuel Klein, David Bau, and Antonio Torralba. Multi- modal neurons in pretrained text-only transformers. In IEEE/CVF International Conference on Computer Vision, ICCV 2023 Workshops, Paris, France, October 2-6, 2023, pages 2854-2859. IEEE, 2023.
[45] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2556-2565, Melbourne, Australia, 2018. Association for Computational Linguistics.
[46] Shufan Shen, Zhaobo Qi. Junshu Sun, Qingming Huang, Qi Tian, and Shuhui Wang. Enhancing pre-trained representation classifiability can boost its interpretability. In The Thirteenth Inter- national Conference Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025.
[47] Shufan Shen, Junshu Sun, Xiangyang Ji, Qingming Huang, and Shuhui Wang. Expanding sparse tuning for low memory usage. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 38: Annual Conference Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024.
[48] Shufan Shen, Junshu Sun, Shuhui Wang, and Qingming Huang. Kernelized sparse fine-tuning with bi-level parameter competition for vision models. arXiv preprint arXiv:2510.2403: 2025.
14

---

[49] Dong Shu, Haiyan Zhao, Jingyu Hu, Weiru Liu, Ali Payani, Lu Cheng, and Mengnan Du. Large vision-language model alignment and misalignment: A survey through the lens of explainability. ArXiv preprint, abs/2501 1.1111,,,,,,, 2025.
[50] Mustafa Shukor and Matthieu Cord. Implicit multimodal alignment: On the generalization of frozen Iims to multimodal inputs. ArXiv preprint, abs/2405. 16700, 2024.
[51] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In David Yarowsky, Timothy Baldwin, Anna Korhonen, Karen Livescu, and Steven Bethard, editors, Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1642, Seattle, Washington, USA, 2013. Association for Computational Linguistics.
[52] Junshu Sun, Wanxing Chang, Chenxue Yang. Qingming Huang, and Shuhui Wang. Relieving the Over-Aggregating Effect in Graph Transformers, 2025.
[53] Junshu Sun, Shuhui Wang, Xinzhe Han, Zhe Xue, and Qingming Huang. All in a row: Com- pressed convolution networks for graphs. In Proceedings of the 40th International Conference on Machine Learning. volume 202 of Proceedings of Machine Learning Research, pages 33061-33076. PMLR, 23-29 Jul 2023.
[54] Junshu Sun, Chenxue Yang. Xiangyang Ji, Qingming Huang, and Shuhui Wang. Towards dynamic message passing on graphs. In Advances in Neural Information Processing Systems, volume 37, pages 80936-80964. Curran Associates, Inc., 2024.
[55] Gemini Team, Rohan Anil, Sehastian Borgeaud, Jean- Saan-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, al. Gemini: family of highly capable multimodal models. ArXiv preprint, (lbs/2212.1111 1805, 2023.
[56] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. ArXiv preprint, abs/2307 .09288, 2023.
[57] Cheng-Hao Tu, Zheda Mai, and Wei-Lun Chao. Visual query tuning: Towards effective usage of intermediate representations for parameter and memory efficient transfer learning. In IEEE/CVF Conference on Computer Vision and Putlern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 7725-7735. IEEE, 2023.
[58] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 911)). 2008
[59] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
[60] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. ArXiv preprint, abs/2409. 2024.
[61] Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. SUN database: Large-scale scene recognition from abbey to zoo. In The Twenty-Third IEEE Con- ference Computer Vision and Puttern Recognition, CVPR 2010, San Francisco, CA, USA, 13-18 June 2010, pages 3485-3492 IEEE Computer Society, 2010.
[62] Xinyue Xu, Yi Qin, Lu Mi, Hao Wang, and Xiaomeng Li. Energy based concept bottleneck models: Unifying prediction, concept intervention, and probabilistic interpretations. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
[63] Mert Yüksekgönül, Maggie Wang, and James Zou. Post-hoe concept bottleneck models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
15

---

[64] Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Zohreh Shams, Frédéric Precioso, Stefano Melacci, Adrian Weller, Pietro Lió, and Mateja Jamnik. Concept embedding models: Beyond the accuracy- explainability trade-off. In Sanmi Koyejo, S. Mohamed, Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022.
[65] Siqi Zhang, Yanyuan Qiao, Qunbo Wang, Longteng Guo, Zhihua Wei, and Jing Liu. Flexvin: Flexible adaptation for diverse vision-and-language navigation tasks. arXiv preprint arXiv:2503.13366 2025.
[66] Siqi Zhang, Yanyuan Qiao, Qunbo Wang, Zike Yan, Qi Wu, Zhihua Wei, and Jing Liu. Cosmo: Combination of selective memorization for low-cost ussio-and-langee navigation. arXiv preprint arXiv:2503. 2025.
[67] Chenyang Zhao, Kun Wang, Xingyu Zeng, Rui Zhao, and Antoni B Chan. Sradient-based visual explanation for wansformer-basee clip. In International Conference on Machine Learning, pages 61072-61091. PMLR, 2024.
16

---

Vision Modality
Language Modality
Activation Count
Activation Count
Sorted Concept Index
Sorted Concept. Index
Figure A8: Activation count of each concept in VL-SAE constructed for LLaVA1.5 [34].
Table A6: The a value of OpenCLIP with varying model sizes in zero-shot image classification tasks.
Caltech111
Country211
Cifar
Cifar100
DTD
Eurosat
Flowers
Food
GTSRB
MNIST
Pets
SST2
STLI0
6im39
ViT-B/32
0.6
0.6
0.3
0.1
0.1
0.2
0.1
0.5
0.4
1.0
0.1
0.3
0.1
0.2
VIT-B/16
0.6
0.5
0.3
0.1
0.1
0.1
0.1
0.5
0.5
1.0
0.1
0.2
0.1
0.2
ViT-L/14
0.6
0.4
0.3
0.3
0.1
0.3
0.2
0.1
0.2
1.0
0.1
0.1
0.9
0.1
ViT-H/14
0.4
0.8
0.6
0.1
0.3
0.7
0.9
0.5
1.0
0.5
0.1
A More Implementation Details
A.1 Details of the VL-SAE Construction and Evaluation
For the InfoNCE loss utilized in Section 3.2, given an image-text pair (xf; xf) and other N with inter- mediate representations { ( (x xf,r which denote the (x+-, xi xf] ) in Equation 3,
the InfoNCE loss can be represented by
cool
cos cos
exp(
exp(
InfoNCE xf, x(
log
-log
N
cool0xg.
cus OOxe
i=
exp(
Zim1
exp(
(A12)
where 7 denotes the temperature hyperparameterr, which is set to 0.07. The InfoNCE loss is computed for image-text pairs within the same batch and aggregated to train the auxiliary autoencoder.
For the construction of the VL-SAE and auxiliary autoencoder, the modules except for the distance- based encoder are implemented with a linear layer. We set the hidden ratio to in both CVLMs and LVLMs, indicating that the number of neurons in the hidden layer is 8 times the representation dimension. Unless otherwise specified, the number of activated neurons for each representation is set 256 for all CVLMs and LVLMs. For the quantitative evaluation of the concept set, we randomly select several concepts from the autoencoder to compute their intra-similarity and inter-similarity. We calculate these metrics through five random trials, and the average value adopted be presented in Figure 3.
The impact of the number of concepts used for evaluation is shown in Table A7, The metrics showed no significant differences when more than 100 concepts are used. Since the evaluation needs to be repeated multiple times, we select 100 concepts in each iteration for efficient evaluation.
For the evaluation metrics Figure for the i-th neuron, we first obtain its maximally activating images and texts. Then, these images and texts are transformed into representations with a pre-trained OpenCLIP-ViT-H/44 [43]. The averaged image and representations x x are utilized compute the Inter-Similarity and Inter-Similaric metrics. The inter-Similarity is computed as the average cosine similarity between the image and text representations within the same neuron.
Simintro
cos(Oxc.x))
(A13)
7
17

---

Table A7: Impact of the number of concepts on the evaluation metrics based on OpenCLIP OpenCLIP-ViT-B-II ViT-B-16.
Concept Num.
100
200
300
400
500
Intra Similarity (SAE-S)
0.2200
0.2257
0.2281
0.2245
0.2247
Intra Similarity (VL-SAE)
0.2445
0.2508
0.2489
0.2476
0.2469
Inter Similarity (SAE-S)
0.1347
0.1293
0.1300
0.1306
0.1308
Inter Similarity (VL-SAE)
0.1230
0.1214
0.1182
0.1181
0.1195
Table A8: Number of concepts within different SAEs.
Method
Hidden Neuron
Dead Neuron
Concept Num.
SAE-D
32768 (4096x8)
54
32714
SAE-S 32768 (4096x8)
46
32722
VL-SAE
32768 (4096x8)
15
32753
For the AnterSiimilarity it is computed as the average cosine similarity between the image and text representations within the different neurons.
Siminter
cos(O
x xf)
(A14)
h((
i=
For the resource requirements, the VL-SAE models for different VLMs [43, 34, 1] are all trained using a single NVIDIA GeForce RTX 4090 GPU. For CVLM [43], full-precision training is employed to ensure optimal performance and stability. For LVLM [34, 1], half-precision (FP16) training is utilized to effectively reduce resource requirements while maintaining model efficiency.
A.2 Number of Concepts in VL-SAE
In Table A8, we provide the number of concepts in different SAE architectures based on LLaVA 1.5 with 4096 feature dimensions. The number of concepts is upper bounded by the number of hidden neurons in SAEs. Moreover, the number of concepts is influenced by the training process and SAE architectures. This stems from the inevitable existence of dead neurons in SAEs [15], which are never activated. The results show that VI-SAE learns more concepts compared to existing methods.
A.3 Details of Interpreting Representations with VL-SAE.
We observe that a subset of concepts VL-SAE are activated at high frequencies, as Figure A8 shown, i.e,, almost all samples activate these concepts. This frequent activation makes it challenging for these concepts to be associated with specific semantics, as shown in Tale A9, Consequently, in the interpretation process, divide the target representation's neuron activation by the average activation value of each neuron, thereby mitigating the impact of these high-frequency concepts on the interpretation.
Table A9: The concepts of VL-SAE relevant to a kitchen image with and without re- re-weighting. All concepts are shown in text form. The concept corresponding to the nigh-frequency neuron denoted by (*), which exhibits irrelevant semantics with the input kitchen image.
Before re-weighting
After re-weighting
toast popping out a toaster hard rock artist performs (*) burning on a gas stove in the kitchen cans in refrigerator in a restaurant this is gas stove with electric ovens
toast popping out a toaster burning on a gas stove in the kitchen cans in refrigerator in a restaurant this is gas stove with electric ovens gas fucled rings on domestic cooker
18

---

Concept 1588
Concept 6556
Concept 10392
Concept 11204
being trucked the rainforest on the coast. Region is largest natural rainforest in the world.
Aerial city skyline.
Portrait of young laughing.
player plays during the first day of the opening test match against cricket team
You can see structure on clear days.
Man looks the camera expressing happiness.
Concept 17489
Concept 18372
Concept 24787
Concept 26203
A man works at laptop computer an office. Worker working the office during working time.
Person attends premiere
A landmark red and white
Map of the state on white
during festival.
lighthouse.
background. gray background.
Actor attends the premiere during festival.
view of the lighthouse.
Map
Figure A9: More concepts learned by VL-SAE. We provide the maximally activating images and texts for several concepts of VL-SAE trained for LLaVA1.5.
A.4 Details of the Zero-shot Image Classification Task
Considering the varying degrees of alignment between the concept set of VL-SAE and the target categories of the downstream task, experiment with multiple values of {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9) and select the one that performs best. The selected val- presented in Table A6. We observed that models of varying sizes tended to select different a values, whereas for certain datasets like MNIST and Pets, the optimal alpha values remained consistent across different models. Moreover, observe that the optimal alpha values for some datasets, such as CIFAR-10 and Caltech101 are significantly higher than those for other datasets. This discrepancy stems from the strong correlation between their target categories and the dataset utilized to train the VL-SAE model.
A.5 Details of the Hallucination Elimination Task
With the modified representation & in Equation 11, we adopt the contrastive decoding strategy [26] mitigate the hallucination in LVL.Ms. Specifically,, with the representations of vision and language tokens x,t E R. xd Xt. E xd the model generates two distinct output distributions: (i) one conditioned the original representations; and (ii) the other conditioned the original vision representations and the language representations with their mean value across the token dimension replaced by &).
X
xi R1.
(A15)
where Xtr[i] E Rd denotes the representation of the i-th text token, x; E Ed denotes the language representations averaged across the token dimension. Then, new contrastive probability distribution is computed by exploiting the differences between the two initially obtained distributions. The new contrastive distribution is formulated as
Pea(f) Xot, ku) softmax[(1 logit,y X,r, x + ared logit, Xvt. &n)]. (A16)
Larger values indicate stronger amplification of differences between the two distributions. Moreover, we also utilize the adaptive plausibility constraints [29] that conduct contrastive decoding contingent upon the confidence level 3dd associated with the output distribution with original repre- sentations. practice, we set aket and 3ad 0.6 and 0.8, respectively. Additionally, the values of and B in Equation are determined to be 0.7 and 0.9, respectively.
19

---

Regular:
The image features smiley face drawn the inside shiny surface of black part, either including metal plastic part. There is gop, spotlight, between the metal or plastic part and it appears to be sitting a motor. The face looks out of place in such an environment, adding playful touch to the scene.
With VL-SAE:
The image features close-up view of black object, possibly a wheel or piece of machinery, with a sticker on It. The sticker has picture of smiling face and the words "Fender Blender" written below The face appears to cartoon-like character, adding playful touch object.
Regular:
image shows gray bird perched on top of a metal the main focus of the scene, and appears to looking carefully at something the distance. Near the bird, there another bird sitting farther down the fence. The entire scene conveys close-up shot of the birds the fence.
With VL-SAE:
The image shows small bird perched on top of metal Fence. The bird is standing on the fence, looking the camera. The fence is made metal bars, bird positioned composed of several metal bars, creating stricter and The captures the bird's curiosity and its interaction with the fence.
middle
fence. The fence barrier.
Figure A10: Illustration of generated captions by different decoding methods with LLaVA 1.5 as the
backbone model. Hallucinated content is highlighted in purple.
Table A10: Human evaluation of concepts learned by different SAEs.
Method
OpenCLIP- ViT/H LLaVA 1.5
SAE-D
0.6
0.1
33.5 SAE-S
35.8
VL-SAE
65.9
64.1
B More Experimental Results
B.1 Human Evaluation
We randomly select 100 concepts from SAE-D, SAE-S, and VL-SAE, respectively, and invite 10 participants evaluate these concepts. For each iteration, participants are required to evaluate the quality of three given concepts and select the one with the highest quality. The method corresponding the selected concept is awarded one point. The concept quality of these methods is measured by the average score assigned by all participants. As Table A10 shows, concepts learned by VL-SAE possess higher quality than those learned by other methods. This comparison further highlights the effectiveness of our proposed VI-SAF..
B.2 More Results for Hallucination Elimination
Beyond the "Yes-or-No" discriminative evaluations on the POPE datasets, we evaluate the VL-SAE on open-ended caption generation using the CHAIR benchmark [42]. We randomly select 500 samples from CHAIR for evaluation and calculate the average of the results obtained from five independent trials. The results in Table All show the superior performance of VL-SAE over the compared methods. Specifically, VL-SAE reduces object hallucinations in generated captions, as evidenced by lower CHAIRs and CHAIR, scores. addition, VL-SAE enhances the detail of the generated captions, indicated by higher Recall scores.
20

---

Table A11: Results on the subset of CHAIR.
Model
Decoding
CHAIRs ())
CHAIR, (4)
Recall (1)
Avg. Len
Regular
53.4
17.6
72.3
103.2
LLaVA1.5
VCD
55.0
16.3
76.0
102.5
VL-SAE
47.8
13.3
76.3
100.7
Regular
44.6
16.1
60.7
97.2
Qwen-VL
VCD
42.6
13.9
60.0
94.1
VL-SAE
39.6
10.7
63.3
94.0
Table A12: Training cost of VL-SAE based on representations OpenCLIP.
Base Model
ViT-B/16
ViT-L/14
ViT-H/14
FLOPs
0.03G 132s
0.06G
0.10G
Training Time
228s
446s
B.3 Resource Requirements of VL-SAE
Training Costs. Table A12 presents the training cost of VL-SAE. First, the model size increases, the dimensionality of its learned representations grows, leading to higher computational demands for training VL-SAE. Second, despite this scaling trend, the overall training cost of VL-SAE remains remarkably low. For instance, when compared to LoRA [20], a widely adopted method known for its computational efficiency, training VL-SAE significantly fewer FLOPs (0.03G v.s. 1.91G). This minimal computational overhead stems from the small parameter volume in VL-SAE, which is approximately equivalent that of two linear layers.
OpenCLIP-VV O representations incurs
Inference Cost. Table A13 shows the parameters and computation cost of VL-SAE, For inference speed, present the influences of VL-SAE in Table A14. The results indicate that VL-SAE has a negligible impact. This is because VL-SAE comprises only two linear layers with significantly smaller parameters than pre- pre-trained VLMs.
B.4 Additional Ablation Studies
Selection of Hyperparameter ag. We first provide accuracy comparisons under three settings: baseline (ag- 0), task-agnostic (a, 0.1), and task-specific (a, determined by tasks). The results are shown in Table A15, First, the task-agnosti v.s. 72.2% on ViT-I, and 77.2% v.s. 76.9% on ViT-H). This indicates that a does not need to be task-specific for performance improvement and can generalize across tasks. However, these tasks originate from different domains and exhibit distinct preferences for hyperparameters [31], as evidenced by the best performance of the task-specific scheme. If the objective is to achieve optimal performance, the a values of each task should be determined individually, To determine the a value of each task, we conduct experiments with multiple values E [0, 1] intervals of 0.1 and select the one that performs the best. Note that the cost of conducting multiple experiments is very low. For instance, when using the Food- 101 dataset, each experiment single NVIDIA 4090 GPU takes only approximately 90 seconds.
scheme performs better than the baseline (72.4%
Selection of Hyperparameter
For the hyperparameter oy in Equation 11, We conduct experiments by selecting values at intervals of 0. within the range of [0.5 0.9), and the one with the best performance is selected. We report the performance under different values in Table A16. The highest performance across different settings is achieved at 0; 0.7.
Table A13: Additional parameter and computation cost of VL-SAE.
Model
Total Param.
VL-SAE Param.
Total FLOPs
VL-SAE FLOPs
1LaVA-1.5
7529M
467M 25M
14.03G 3.15G
0.875G
OpenCI.IP-Vi-H
1011M
0.034G
21

---

Table A14: Throughput (sample/s) of the original models and those equipped with VL-SAE.
Method OpenCLIP-ViT-B OpenCLIP-ViTHL OpenCLIP-ViT-H LLaVA 1.5
Original
935.32 935.13
213.62
99.45
11.34
+VL-SAE
213.42
99.29
10.69
Table A15: Ablation of hyperparameter Or in Equation 10.
Method
OpenCLIP-VIL ViT-L OpenCLIP- ViT-H
baseline (ac 0)
72.2
76,9
task-agnostic (a, 0.1)
72.4
77.2
task-specific (a,, determined by tasks)
72.9
77.8
CC3M training. In the training process of VL-SAE, the CC3M dataset is employed to integrate VL-SAE as a general plug-in in for VLMs, rather than to augment the intrinsic capabilities these VLMs. In training strategies, the VLM parameters remain frozen without any adjustments. To further evaluate the impact of CC3M dataset, we provide comparisons between the base model (CLIP- ViT-B/16), the base model fine-tuned by CC3M, and the base model equipped with VL-SAE in Table A17. The results show that fine-tuning does not lead to improved performance on downstream tasks. This is because CC3M serves subset of the VL.M pre-training dataset. does not provide the base model with any additional task-relevant information. The superior performance of VL-SAE compared the fine-tuning model indicates that the improvement stems from the mapping from representations to concepts, rather than the incorporation of CC3M.
B.5
Additional Visualizations
B.5.1
Concepts Learned by VI-SAE
We provide more visualization of the concepts learned by VL-SAE in Figure A9. These results highlight the superior capability of VL-SAE in acquiring vision-language concept set with rich semantics, such as natural landscapes, urban environments, human expressions, and maps.
B.5.2 Cases for Hallucination Elimination
We present qualitative examples of different decoding methods in Figure A10. Regular decoding [34] often produces object hallucinations, especially in the latter part of the outputs. In contrast, utilizing VL.-SAE effectively mitigates the object hallucinations by enhancing the vision-language alignment the concept level during the decoding process.
B.5.3 Visualization of the Representations and Learnable Weights
Figure A1l illustrates the t-SNE [58] visualization for the vision-language representations of VLMs and the learnable weights in the encoder of the VL-SAE. We observe that the majority of learnable weights are positioned midway between vision and language representations, rather than being skewed toward either modality. This position enables the corresponding neurons to exhibit consistent activation values across different modalities, which helps map the vision-language representations into unified concept set.
Table A16: Ablation studies on the hyperparameter a; of Equation 11. All experiments are conducted on POPE benchmark with LLaVA 1.5.
Setting
0.5
0.6
0.7
0.8
0.9
Random
85.29
85.32
85.50
85.41
85.31
Popular
84.09
84.15
84.37
84.23
84.12
Adversarial
81.92
82.10
82.29
81.95
81.93
22

---

Table A17: Ablation studies on the impact of the CC3M dataset.
Method
Base Model
Base Model+ -unnng
Base Model+VL-SAM
Mean Accuracy
69.8
69.7
70.4
OLenCLIP (ViT-I-14)
CLI (ViT-H-114
Vieio Repescattaiaa
Vaka Eeproestaticn
Longuge Bejesconation
Langiage Repersentoion
Leazsale Weights
Leanaile Welgee
Figure Al1: Visualizations of the vision-languag, representations in the VLMs and the learnable weights in the VL-SAE encoder. Experiments are conducted with 1-SNE [58] for dimension reduction.
C
Discussion
C.1 Criterion of Representation Selection in LVLMs
We extract the representations from the outputs of the 29-th layer in LLaVA 1.5 and the 26-th layer in Qwen-VLL VL. These layers are chosen because the representations in the top layers tend to be more interpretable in terms of concepts [23]. Furthermore. at these layers, both vision and language representations preserve their respective information effectively. As shown in Figure A12, starting from the 27-th layer of Qwen-Vi VL,, the distributions of the two types of representations begin to converge, leading to potential loss of distinct visual information. This convergence hinders the process of leveraging vision representations to constrain language representations for reducing hallucinations.
LLaVA 15 27
LLVV 13 lyye
LLaVA 15 lover 29
Quen-V 16
Quen-NL, 1owe--77
Quen-V laym-22
Figure A12: Visualizations of the lanionn representations in different layers of VLMs. Experiments are conducted with t-SNE [58] for dimension reduction.
23

---

C.2 Metries for Evaluating the Concept Set
For the quantitative evaluation of concept collections, we utilized CLIP similarity for automated assessment in Figure and Table 3. However, the numerical differences in CLIP similarity might not entirely capture the improvements in concept quality achieved by VL-SAE relative other methods. This is because the numerical gap in CLIP similarity between semantically similar and dissimilar image-text pairs can sometimes minimal. Therefore, is important establish more comprehensive evaluation mechanism for SAEs based on multi-modal representations in the future.
C.3 Discussions with attribution-based methods
We discuss and compare VL-SAE, a concept- approach, with attribution- based methods [67, 4] for VLMs, enabling users to select the most uppropriate method based on specific application scenar- ios. Attribution-baat
methods generate heatmap for each input sample to indicate the importance of individual components of the input data (e.g.. image pixels), whereas VL-SAE constructs a concept set across varying samples interpret the semanties of their hidden representations.
Based on their paradigms, we compare them in the perspective of intuitiveness, expressivity and ability improve model performance. (i) Intuitiveness. Compared with VL-SAE, attribution-based methods offer a more intuitive interpretation, as they visualize attribution scores directly on input images. (ii) Expressivity. Compared with attribution-based methods, VL-SAE exhibits stronger expressivity as it reveals how the model understands the semantics of input data with concepts, rather than simply providing numerical information to identify the important image regions. (iii) Ability to improve model performance. In contrast to attribution-base for model prediction, VL-SAE can act as an intervener to improve model performance by modifying hidden representations during inference.
methods that only serve as an observer
C.4 Limitation Discussion
Despite successfully correlating the semantics of vision-language representations with unified con- cept set, the VL-SAE encounters the same challenges as the SAE architectures for other models (e.g.. LLMs [56] and vision models [12, 5]., such as dead neurons [15]. limited relational modeling, and the impact of high-frequency neurons on interpretations as shown in Figure A8. Additionally, the strategy of enhancing the alignment mechanism with VL-SAE simple and fails fully unlock the potential of the unified concept set. To address these limitations, we anticipate future work to design SAES that encourage the neurons to be uniformly activated, incorporate relational modeling [53, 54, 52] among concepts, and enable more effective integration into the prediction processes of VLMs.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation in Appendix C.4
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
25

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please refer Section 4.1, Appendix A and the codes in the supplementary.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
26

---

Answer: [Yes]
Justification: Please refer to the codes in the supplementary.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /nips wwtppblic/guiiiiiccccceeecooopee httpe/guuueeeeeeuubeeeeeeeee /at////ooomisssseeeeee ssisssolicyyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: Please refer to Section 4.1.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: This paper does not report error bars following the practice of previous studies [8, 32].
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
27

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer to Appendix A.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https //te:///nnuu..peeeee httppbliic/thiiccuuiieee
Answer: [Yes]
Justification: This paper conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed. Because it just focuses on the internel mechanism of pre-trained models.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
28

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Please refer Section 4.1 and Appendix A.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
29

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Please refer the codes in the supplementary.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
30

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLM is used only for writing, editing, formatting purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https (https://netll /ttp://eeeuu.pp/eeee /ttp:/pp//./..eeeeeeee////////// for what should or should not be described.
31


Reviews:
[{"review_id": "LJcZUXzrju", "reviewer": "Reviewer_LRaD", "summary": "The authors propose Explicit Representation Alignment and VL-SAE to enhance interpretation of VLLM and address concept mismatch in Large Vision-Language Models. Each neuron in the hidden layer correlates to a concept represented by semantically similar images and texts, thereby interpreting these representations with a unified concept set. To establish the neuron-concept correlation, they encourage semantically similar representations to exhibit consistent neuron activations during self-supervised training. Experiments across multiple VLMs (e.g., CLIP, LLaVA) demonstrate the superior capability of VL-SAE in interpreting and enhancing the vision-language alignment.", "questions": "1. I cannot tell the difference in figure 5.\n2. Please label the better model for individual class in table 2\n3. The workflow of how to get figure 5 and 6 is not clear. Which is input and which is output and why do you put all of these materials here?\n4. What is the computation and data scale for training an extra autoencode.\n5. Will this work for larger VLMs? Will the data and computation, especially data required scaled accordingly. If so, will the cost be affordable?\n6. Some error analysis may be more helpful in the appendix to point future research direction.", "limitations": "1. The presentation is hard for me to understand.\n2. The author should give a contact workflow of how to utilize this method and align the workflow with experiments like figure 5 and 6.\n3. More training details of reproducing this work should be mentioned.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. It proposes a method to explain the correlation between text and vision and avoid concept mismatch. \n2. It introduces a distance-based encoder that ensures activation consistency for semantically similar representations via cosine similarity and Euclidean distance constraints. It also uses modality-specific decoders to handle distributional differences between vision and language representations, mitigating concept mismatch.\n3. It builds on strong theoretical model.\n4. Figure 3 is clear and shows the advantage.\n\nWeakness:\n1. The presentation of the paper is not so clear, especially the workflow of the experiments.\n2. I cannot get the information that wants to expressed in figure 5 and 6.\n3. There is no human evaluation which will reflect real preference in usage.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your valuable feedback on our manuscript. Your comments have been instrumental in guiding our future work.\n\n## W1 \\& L1: Explanations of the experiment workflow.\nThank you for your sincere comments! However, we may not fully understand your specific concerns. To thoroughly address them, we will describe the overall framework of our experiments. \n\n1. First, we train the VL-SAE for pre-trained VLMs in a self-supervised manner based on their representations (in Section 4.2). \n2. Then, we integrate VL-SAE into the inference process to interpret the semantics of model representations (in L295-305 for CVLMs, L317-326 for LVLMs). \n3. Based on the interpretation results, we design methods to intervene in the model predictions via VL-SAE, thereby achieving performance improvement (in L306-316 for CVLMs, L327-337 for LVLMs).\n\n**Revision**. We will emphasize the overall framework of the experiments at the beginning of Section 4 in the revision. Please let us know if your concerns are from other points. We are standing by to address them.\n\n## W2 \\& Q1 \\& Q3 \\& L2: Explanations of Figure 5 and 6.\nThank you for your detailed feedback! We will re-introduce Figures 5 and 6 to you from four aspects: the objective, the workflow (Q3, L2), the differences of concepts from different columns (Q1), and the insights we can derive from these figures (W2). \n\n- **Objective**. We present these figures to show the ability of VL-SAE to interpret the alignment of vision-language representations during the prediction process.\n\n- **The workflow in Figures 5 and 6 (Q3, L2)**. \n    1. Given a sample, we feed it into VLMs for inference.\n    2. During the inference process, we extract the vision-language representations from the hidden layers. \n    3. These representations are mapped to human-comprehensible concepts by VL-SAE. \n    4. We display several concepts most relevant to the representation semantics in Figures 5 and 6. Each concept is represented by several images and texts.\n    5. With these concepts correlated to vision and language representations, we can analyze the internal mechanism of the model's prediction for this sample.\n\n- **Differences of concepts from different columns in Figure 5 (Q1)**. The first and second columns display the concepts relevant to vision and text representations, respectively, while the third column shows concepts highly relevant to both modalities. \n\n- **Insights of Figures 5 and 6 (W2)**. In Figures 5 and 6, we `observe several concepts where the two modalities are not aligned`. For example, the \"car\" concept is irrelevant to the input text in Figure 5, and the \"microwave\" concept correlates with hallucinated text rather than the input image in Figure 6. These misalignment concepts may affect the multi-modal reasoning during inference. To address this issue, we introduce the methods detailed in L306–316 and L327–337, which effectively mitigate misalignment and lead to improved model performance.\n\n**Revision**. We will describe the workflow of Figures 5 and 6 more clearly, and highlight the misalignment concepts in the figures with a special color to avoid confusion.\n\n## W3: Human Evaluation Results.\nWe appreciate your valuable suggestions that motivate us to provide more comprehensive evaluations. \n\n- **Experiments**. Following your suggestions, we randomly select 100 concepts from SAE-D, SAE-S, and VL-SAE, respectively, and invite 10 participants to evaluate these concepts. For each iteration, participants are required to evaluate the quality of three given concepts and select the one with the highest quality. The method corresponding to the selected concept is awarded one point. The concept quality of these methods is measured by the average score assigned by all participants.\n\n    | Method | OpenCLIP-ViT/H | LLaVA 1.5 |\n    | ------ | -------------- | --------- |\n    | SAE-D  | 0.6            | 0.1       |\n    | SAE-S  | 33.5           | 35.8      |\n    | VL-SAE | **65.9**       | **64.1**  |\n\n- **Analysis**. `The concepts learned by VL-SAE possess higher quality than those learned by other methods`. This comparison further highlights the effectiveness of our proposed VL-SAE.\n\n**Revision**. The scale of current human evaluation is constrained by the limited rebuttal period. Based on the above experiments, we plan to expand the evaluation scale and incorporate it into the revision.\n\n## Q2: Refining Table 2.\nThank you for your advice! Following your suggestion, we will label the better model for the individual task in the revision.\n\n## Q4: Computation and dataset requirements.\n**Computation**: To demonstrate the training efficiency of VL-SAE, we compare it with LoRA, a widely recognized computationally efficient method. The FLOPs of training VL-SAE with OpenCLIP-ViT-B/16 representations are significantly lower than those of LoRA fine-tuning `(0.03G v.s. 1.91G)`. The minimal training cost stems from the small number of parameters in VL-SAE, which is equivalent to approximately two linear layers.\n\n**Dataset Scale**: As illustrated in Lines 247-249, all autoencoders are trained on the CC3M dataset, which comprises `3 million image-text pairs`.\n\n**Usability**: In practice, with the representations of the pre-trained model, we can complete the training of VL-SAE in `less than 10 minutes using a single NVIDIA RTX 4090`, which is efficient and easy to implement.\n\n**Revision**. We will incorporate the comparisons of training FLOPs in the revision.\n\n## Q5: Scalability and training cost of VL-SAE.\nThank you for your insightful questions that help us further clarify the low cost and scalability of VL-SAE.\n\n- **VL-SAE works for larger VLMs**. We provide the performance gains achieved by applying VL-SAE to CVLMs of different sizes with the same dataset for pre-training (CC3M). The mean accuracy across various zero-shot image classification tasks in Table 2 is reported as follows.\n\n    | Base Model        | Params. | Original Acc. | +VL-SAE | $\\Delta$ Acc. |\n    | ----------------- | ------- | ------------- | ------- | ---------- |\n    | OpenCLIP-ViT-B/32 | 152M    | 68.7          | 69.5    | 0.8        |\n    | OpenCLIP-ViT-B/16 | 177M    | 69.8          | 70.4    | 0.6        |\n    | OpenCLIP-ViT-L/14 | 428M    | 72.2          | 72.9    | 0.7        |\n    | OpenCLIP-ViT-H/14 | 1367M   | 76.9          | 77.8    | 0.9        |\n\n    With the same amount of data, VL-SAE can provide consistent performance gains for models of different sizes. Therefore, the `data scale does not need to be proportional to the model size`.\n\n- **The training cost of VL-SAE is affordable**. We provide the training FLOPs of VL-SAE with models of different sizes, as well as the total training time required on a single NVIDIA RTX 4090.\n\n    | Base  Model   | OpenCLIP-ViT-B/16 | OpenCLIP-ViT-L/14 | OpenCLIP-ViT-H/14 |\n    | ------------- | ----------------- | ----------------- | ----------------- |\n    | FLOPs         | 0.03G             | 0.06G             | 0.10G             |\n    | Training Time | 132s              | 228s              | 446s              |\n\n    The training computations and time of VL-SAE increase with the model size. Nevertheless, benefiting from the small number of parameters, the `pre-training of VL-SAE can be completed in less than 450 seconds using a single NVIDIA RTX 4090`, demonstrating its high computational efficiency.\n\n**Revision**. We will incorporate the above experiments and analysis into the revision.\n## Q6: Error Analysis.\n\nThank you for your comments, which have helped us gain perspectives on future research directions through the analysis of failure cases. \n\n- **Current limitations**. As discussed in the limitation section, `although this is not the central focus of our study, VL-SAE encounters the issue of frequently activated neurons, which is prevalent among existing SAEs`. These neurons can be activated by semantically irrelevant samples, reducing the quality of interpretations for these samples. \n\n- **Our Solution**. We have mitigated this issue within the inference process by re-weighting the concept relevancy with the method in Section A.2. The concepts relevant to a kitchen image with and without re-weighting are presented as follows. Due to the inability to upload images, they can only be shown in text form here. The concept corresponding to the high-frequency neuron is denoted by (*), which exhibits irrelevant semantics with the input kitchen image.\n\n    | Before re-weighting                        | After re-weighting                                |\n    | --------------------------------------- | ---------------------------------------------- |\n    | toast popping out of a toaster          | toast popping out of a toaster                 |\n    | hard rock artist performs (*)           | burning on a gas stove in the kitchen          |\n    | burning on a gas stove in the kitchen   | cans in a refrigerator in a restaurant         |\n    | cans in a refrigerator in a restaurant  | this is a gas stove with electric ovens        |\n    | this is a gas stove with electric ovens | gas fueled rings on a domestic cooker |\n    \n- **Future Directions**. Although we have mitigated the issues caused by frequently activated neurons at the inference stage, these neurons essentially stem from the training process of SAE. We will address this issue more comprehensively at the training stage in our future work.\n\n**Revision**. We will incorporate the above analysis of failure cases in the appendix.\n\n## L3: More training details.\nThank you for your feedback. The details of training VL-SAE are presented in Section 4.1 and A.1. The details regarding the application of VL-SAE to interpret and enhance vision-language alignment are provided in Section A.2~A.4. \n\nPlease let us know if the information provided in these sections does not address your concerns. We are standing by to address them."}, {"review_id": "BA9AvqkyZV", "reviewer": "Reviewer_5uDR", "summary": "Vision-Language models (VLMs) achieve remarkable performance on various downstream tasks, which can be attributed to the inductive biases which facilitate vision-language alignment. However, it remains a substantial challenge to interpret vision-language alignment. This work proposes VL-SAE to map the semantics of vision-language representations to a unified concept set and thus interpret the alignment mechanism of VLMs. Further experiments apply VL-SAE to a set of VLMs to show the high quality of concept sets from VL-SAE. Last, this work applies VL-SAE to interpret Contrastive VLMs’ predictions by visualizing the activated concepts, and also to further improve model’s performance on zero-shot image classification and hallucination benchmarks.", "questions": "1. L209, what is $\\theta$?\n2. L254-255, the authors mention that the training of autoencoders is skipped. Does it mean the SAEs are trained directly on the fly? Also, does training of the autoencoder hurt CVLMs performance?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The idea of having a unified concept set to explain vision-language alignment is interesting and important.\n2. Experimental results show the effectiveness of VL-SAE on learning unified and diverse concept sets. \n3. Experimental results show that VL-SAE can further improve VLM’s performance on zero-shot image classification and hallucination benchmark.\n4. Qualitative results further demonstrate the unified concepts’ quality.\n\n**Weaknesses**\n1. L209, the paper mentions triangle inequality for several times, and claims that the distance g satisfies the triangle inequality. However, there is no further clarification or proof to support this. \n2. L289-290, what is the difference between “-Auxiliary autoencoder” and the baseline (Table 1)? If I understand correctly, “-Auxiliary autoencoder” means no training of autoencoders, which reflects the setting in L254-255 where CVLMs skip the training of autoencoders as well. Should we consider adding this training phase in?\n3. For Figure 5, it is great to show qualitative results of the activated concepts. However, it lacks human studies to quantitatively measure the benefits of using VL-SAEs to interpret vision-language alignment.\n4. L312, does $\\alpha$ have to be task-specific? Is that possible to generalize this across tasks? How is this value determined for each task?\n5. For Table 2, it is unclear whether it is the CC3M training that brings the performance improvement. For the non VL-SAE rows, do these models also further finetuned on CC3M dataset? Ideally, there should be 3 rows per bracket: original model, original model + CC3M finetuning, original model + CC3M finetuning with VL-SAE.\n6. For Equation 11, $\\alpha$ overlaps with that in Equation 10. How is this $\\alpha$ value determined? How will it change model’s performance?\n7. L335 (Table 3), similar to the above, is VCD also further finetuned on CC3M dataset?", "was_revised": true, "final_justification": "The rebuttal resolves majority of my concerns, especially the clarifications of the auxiliary autoencoders and the additional experiments on CC3M and human study. I hope the authors can incorporate my comments into the final version.", "rebuttal": "We would like to thank you for the constructive comments! We will diligently follow your suggestions to further improve our manuscripts.\n\n## W1: Proof of L209.\nThank you for your careful reading! As illustrated in line 207, `the function g is defined as the Euclidean distance between normalized vectors, which inherently satisfies the triangle inequality [R1]`. To address your concerns, we provide the proof as follows.\n - **Proof**: Given $\\mathbf{a},\\mathbf{b},\\mathbf{c}\\in\\mathbb{R}^d$, Let's denote their normalized vectors as $\\bar{\\mathbf{a}}=\\frac{\\mathbf{a}}{\\Vert\\mathbf{a}\\Vert_2}, \\bar{\\mathbf{b}}=\\frac{\\mathbf{b}}{\\Vert\\mathbf{b}\\Vert_2}, \\bar{\\mathbf{c}}=\\frac{\\mathbf{c}}{\\Vert\\mathbf{c}\\Vert_2}$. The function $g(\\mathbf{a},\\mathbf{b})$ can be represented as the L2-norm of $\\mathbf{x}=\\bar{\\mathbf{a}}-\\bar{\\mathbf{b}}$,\n $$g(\\mathbf{a},\\mathbf{b})=\\Vert\\frac{\\mathbf{a}}{\\Vert\\mathbf{a}\\Vert_2}-\\frac{\\mathbf{b}}{\\Vert\\mathbf{b}\\Vert_2}\\Vert_2 =\\Vert\\bar{\\mathbf{a}}-\\bar{\\mathbf{b}}\\Vert_2=\\Vert\\mathbf{x}\\Vert_2$$\n Similarly, we can represent $g(\\mathbf{b},\\mathbf{c})$ with $\\mathbf{y}=\\bar{\\mathbf{c}}-\\bar{\\mathbf{b}}$,\n $$g(\\mathbf{b},\\mathbf{c})=\\Vert\\frac{\\mathbf{c}}{\\Vert\\mathbf{c}\\Vert_2}-\\frac{\\mathbf{b}}{\\Vert\\mathbf{b}\\Vert_2}\\Vert_2 =\\Vert\\bar{\\mathbf{c}}-\\bar{\\mathbf{b}}\\Vert_2=\\Vert\\mathbf{y}\\Vert_2$$\n $g(\\mathbf{b},\\mathbf{c})$ with $\\mathbf{y}=\\bar{\\mathbf{c}}-\\bar{\\mathbf{b}}$,\n $g(\\mathbf{a},\\mathbf{c})$ can be represented by the Euclidean distance between $\\mathbf{x}$ and $\\mathbf{y}$,\n $$g(\\mathbf{a},\\mathbf{c}) =\\Vert\\bar{\\mathbf{a}}-\\bar{\\mathbf{c}}\\Vert_2=\\Vert(\\bar{\\mathbf{a}}-\\bar{\\mathbf{b}})-(\\bar{\\mathbf{c}}-\\bar{\\mathbf{b}})\\Vert_2=\\Vert\\mathbf{x}-\\mathbf{y}\\Vert_2$$\n According to the Cauchy-Schwarz inequality, we get:\n $$\\vert\\Vert\\mathbf{x}\\Vert_2-\\Vert\\mathbf{y}\\Vert_2\\vert\\le \\Vert\\mathbf{x}-\\mathbf{y}\\Vert_2$$\n This inequality can derive Equation (5) in our paper.\n $$\\vert g(\\mathbf{a},\\mathbf{b})-g(\\mathbf{b},\\mathbf{c})\\vert\\le g(\\mathbf{a},\\mathbf{c})$$\n\n**Revision**. We will integrate the aforementioned proof into the Appendix.\n\n## W2 \\& Q2: Clarification of the auxiliary autoencoder.\nThank you for your comments that help us better clarify our implementation details! To avoid misunderstanding, we first introduce the components of our framework and then clarify the content of L289–290 and L254–255.\n\n1. **Components of our framework**. There are `two autoencoders` in our framework. One is the auxiliary autoencoder, designed for LVLMs to transform their implicitly aligned multi-modal representations into explicitly aligned representations that correlate semantic similarity with cosine similarity. The other is the VL-SAE with distance-based encoder and modality-specific decoder, designed to interpret explicitly aligned representations.\n2. **Clarification of L289-290**. The \"-Auxiliary autoencoder\" means directly training the VL-SAE with implicitly aligned representations of LVLMs rather than explicitly aligned representations of the auxiliary autoencoder. `Compared with the baseline, the \"-Auxiliary autoencoder\" version replaces the standard SAE with the VL-SAE`. \n3. **Clarification of L254-255**. We inadvertently omitted writing the \"auxiliary\" in L254-255.\nIn CVLM, training an auxiliary autoencoder is unnecessary because the CVLM representations are already explicitly aligned. `The VL-SAE for CVLMs still requires training`.\n\n**Revision**. We will revise \"training of the autoencoder\" to \"training of the auxiliary autoencoder\" in L254–255, and clarify the role of the auxiliary autoencoder in Section 3.2.\n\n## W3: Human evaluations.\nWe appreciate your suggestions that help us improve the comprehensiveness of our evaluations. \n\n- **Experiments**. We randomly select 100 concepts from SAE-D, SAE-S, and VL-SAE, respectively, and invite 10 participants to evaluate these concepts. For each iteration, participants are required to assess the quality of three given concepts and select the one with the highest quality. The method corresponding to the selected concept is awarded one point. The concept quality of these methods is measured by the average score assigned by all participants.\n\n    | Method | OpenCLIP-ViT/H | LLaVA 1.5 |\n    | ------ | -------------- | --------- |\n    | SAE-D  | 0.6            | 0.1       |\n    | SAE-S  | 33.5           | 35.8      |\n    | VL-SAE | **65.9**       | **64.1**  |\n\n- **Analysis**. `The concepts learned by VL-SAE possess higher quality than those learned by other methods`. This comparison further highlights the effectiveness of VL-SAE.\n\n**Revision**. The scale of current human evaluation is constrained by the limited rebuttal period. Based on the above experiments, we plan to expand the evaluation scale and incorporate it into the revision.\n\n## W4: Selection of hyper parameters $\\alpha$.\nThis question is valuable and prompts us to enhance the ability of VL-SAE in improving performance. \n\nWe first provide accuracy comparisons under three settings: baseline ($\\alpha=0$), task-agnostic ($\\alpha=0.1$), and task-specific ($\\alpha$ determined by tasks). Then, we address your concerns by analyzing the results.\n\n| Base Model    | OpenCLIP-ViT-L | OpenCLIP-ViT-H |\n| ------------- | -------------- | -------------- |\n| baseline      | 72.2           | 76.9           |\n| task-agnostic | 72.4           | 77.2           |\n| task-specific | **72.9**       | **77.8**       |\n\n1. The task-agnostic scheme performs better than the baseline (72.4\\% v.s. 72.2\\% on ViT-L, and 77.2\\% v.s. 76.9\\% on ViT-H). This indicates that `$\\alpha$ does not need to be task-specific for performance improvement and can generalize across tasks`.\n1. However, these tasks originate from different domains and exhibit distinct preferences for hyperparameters [R2], as evidenced by the best performance of the task-specific scheme. `If the objective is to achieve optimal performance, the $\\alpha$ values of each task should be determined individually`, \n3. To determine the $\\alpha$ value of each task, we conduct experiments with multiple values $\\alpha\\in[0,1]$ at intervals of 0.1 and select the one that performs the best. Note that `the cost of conducting multiple experiments is very low`. For instance, when using the Food-101 dataset, each experiment on a single NVIDIA 4090 GPU takes only approximately 90 seconds.\n\n**Revision**. We will incorporate the above experiments into the revision and improve the performance of the task-agnostic scheme in future work.\n\n## W5: Effects of CC3M training.\nThank you for your detailed feedback! `The CC3M dataset is employed to integrate VL-SAE as a general plug-in for VLMs, rather than to augment the intrinsic capabilities of these VLMs`. In our training strategies, the VLM parameters remain frozen without any adjustments.\n\nTo address your concern, we provide comparisons between the base model (CLIP-ViT-B/16), the base model fine-tuned by CC3M, and the base model equipped with VL-SAE. \n\n| Method        | Base Model | Base Model+Fine-tuning | Base Model+VL-SAE |\n| ------------- | ---------- | ---------------------- | ----------------- |\n| Mean Accuracy | 69.8       | 69.7                   | **70.4**          |\n\n1. `Fine-tuning does not lead to improved performance on downstream tasks`. This is because CC3M serves as a subset of the VLM pre-training dataset. It does not provide the base model with any additional task-relevant information. \n2. The superior performance of VL-SAE compared to the fine-tuning model indicates that `the improvement stems from the mapping from representations to concepts`, rather than the incorporation of CC3M.\n\n**Revision**. We will incorporate the above experiments and analysis into the revision.\n\n## W6: Explanations of the hyperparameters in Equation 11.\nThank you for pointing out the confused notations.\n\n- **Notation Overlap**. We will re-denote the $\\alpha$ in Equation 11 as $\\alpha_l$, and denote the hyperparameter in Equation 10 as $\\alpha_c$ to eliminate the notation overlap.\n- **Value Determination**. We conduct experiments by selecting values at intervals of 0.1 within the range of $[0.5-0.9]$, and the one with the best performance is selected.\n- **Performances of varying $\\alpha_l$ values**. We report the performance under different $\\alpha_l$ values. The highest performance across different settings is achieved at $\\alpha_l=0.7$.\n\n    | Setting     | 0.5   | 0.6   | 0.7   | 0.8   | 0.9   |\n    | ----------- | ----- | ----- | ----- | ----- | ----- |\n    | Random      | 85.29 | 85.32 | **85.50** | 85.41 | 85.31 |\n    | Popular     | 84.09 | 84.15 | **84.37** | 84.23 | 84.12 |\n    | Adversarial | 81.92 | 82.10 | **82.29** | 81.95 | 81.93 |\n    \n**Revision**. We will employ distinct notations to represent the hyperparameters of Equations 10 and 11, and integrate the above experiments into the revision.\n\n## W7: Clarification of Table 3.\nThank you for your comments! As a training-free method without learnable parameters, VCD cannot be fine-tuned on CC3M. \n\nHowever, as a subset of the pre-training data for VLMs, CC3M is used to endow VL-SAE with the ability to process VLM representations, which is irrelevant to downstream tasks. Since `neither VL-SAE nor VCD has been exposed to information relevant to downstream tasks`, the comparison in Table 3 is fair. \n\n**Revision**. We will further clarify the usage of the CC3M dataset in the revision.\n\n## Q1: Explanations of L209.\nThank you for your careful reading! $\\theta_{\\mathbf{x},\\mathbf{y}}$ denotes the angle between two vectors $\\mathbf{x}$ and $\\mathbf{y}$, and we use $cos(\\theta_{\\mathbf{x},\\mathbf{y}})$ to represent the cosine similarity between these two vectors.\n\n**Revision**. We will clarify the meaning of this notation in Section 3.3 of the revision.\n\n[R1] The Cauchy-Schwarz Inequality: A Geometric Proof. The American Mathematical Monthly, 1989.\n\n[R2] Scaling \\& shifting your features: A new baseline for efficient model tuning, NeurIPS'22."}, {"review_id": "Fy0gk5578m", "reviewer": "Reviewer_i9F9", "summary": "This paper focuses on interpreting the alignment mechanism between vision and language representations of pre-trained VLMs. The authors propose VL-SAE, a novel sparse autoencoder architecture for multi-modal representations, which maps vision-language semantics into a unified concept set. In experiments, the proposed method is evaluated across multiple pre-trained VLMs. Furthermore, two strategies are proposed to enhance model performance on downstream tasks using VL-SAE, including the zero-shot classification for CVLMs and hallucination elimination LVLMs.", "questions": "According to the above weaknesses, I recommend that the authors provide further ablation studies focusing on sparsity constraints and the architecture of the auxiliary autoencoder, as well as report the additional costs introduced by VL-SAE.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1.\tThe authors propose a novel SAE architecture for multi-modal representations, enabling the interpretation of their alignment mechanism.\n2.\tThe proposed method can be applied to models with different architectures, such as CVLMs and LVLMs, demonstrating strong scalability.\n3.\tBeyond interpreting representations, the authors propose two strategies to enhance the alignment of pre-trained VLMs, which lead to improved performance on downstream tasks.\n4.\tThis paper is well-written and easy to follow.\n5.\tThe authors conduct extensive experiments to demonstrate the effectiveness of VL-SAE.\nWeaknesses:\n1.\tAs all SAEs are pre-trained on the same dataset, it would be interesting to explore how the performance of VL-SAE varies with different volumes of pre-training data.\n2.\tWhile the proposed approach can achieve performance improvements, it is important to consider the additional cost introduced by integrating VL-SAE into the model inference pipeline.\n3.\tWhy does the auxiliary autoencoder described in Section 3.2 also require the use of modality-specific encoders and decoders?\n4.\tIt would be better to present the qualitative comparison between SAE-S, SAE-D and VL-SAE of Figure A8 in the main text.\n5.\tI am interested in exploring the potential effects of utilizing L1 loss as the sparsity constraint, as opposed to employing Top-k selection.", "was_revised": true, "final_justification": "The authors propose a novel SAE architecture for multi-modal representations, enabling the interpretation of their alignment mechanism.   Extensive experiments have demonstrated the effectiveness of VL-SAE. The rebuttal has addressed all my concerns about this work. So I recommend it for acceptance.", "rebuttal": "We would like to thank you for the detailed comments! We will diligently follow your guidance to further improve our work and manuscripts.\n\n## W1: Experiments with different volumes of pre-training data.\nFollowing your suggestion, we train VL-SAE with different volumes of the dataset. \n\n- **Experiments**: We train VL-SAE models with different proportions of the CC3M dataset and report their Intra-Similarity and Inter-Similarity.\n\n    | Data Percentage                 | 10\\%    | 30\\%    | 50\\%    | 70\\%    | 100\\%       |\n    | ------------------------------- | ------ | ------ | ------ | ------ | ---------- |\n    | Intra-Similarity ($\\uparrow$)   | 0.2029 | 0.2129 | 0.2196 | 0.2222 | **0.2299** |\n    | Inter-Similarity ($\\downarrow$) | 0.1597 | 0.1306 | 0.126  | 0.1256 | **0.1220** |\n\n\n- **Analysis**: We observe that VL-SAE demonstrates strong scalability. As the volume of data increases, the Intra-Similarity (0.2029 $\\rightarrow$ 0.2299) and Inter-Similarity (0.1597 $\\rightarrow$ 0.1220) of the concepts learned by VL-SAE improve accordingly.\n\n**Revision**. We will incorporate the above experiments and analysis into the main text.\n\n## W2: Additional cost of incorporating VL-SAE. \nThank you for your insightful comments! Following your suggestion, we provide the additional cost of incorporating VL-SAE in terms of memory usage, computations, and inference speed.\n\n- **Memory usage**:\n\n    | Model          | Total Param. | VL-SAE Param. | Proportion |\n    | -------------- | ------------ | ------------- | ---------- |\n    | LLaVA-1.5      | 7529M        | 467M          | 0.06       |\n    | OpenCLIP-ViT-H | 1011M        | 25M           | 0.02       |\n\n- **Computation**:\n\n    | Model          | Total FLOPs | VL-SAE FLOPs | Proportion |\n    | -------------- | ----------- | ------------ | ---------- |\n    | LLaVA-1.5      | 14.03G      | 0.875G       | 0.06       |\n    | OpenCLIP-ViT-H | 3.15G       | 0.034G       | 0.01      |\n\n- **Inference Speed**: The experiments are provided in Table A6 in the appendix.\n\n- **Analysis**: According to the above experimental results, the additional cost of incorporating VL-SAE is nearly negligible. It is cost-effective to allocate less than 6\\% of the parameter and computational resources to achieve interpretability in the model inference process.\n\n**Revision**. We will incorporate the above experiments and analysis into the revision.\n\n## W3: Clarifications of the auxiliary autoencoder.\n\nThank you for your valuable suggestions! We will explain the usage of modality-specific encoders and decoders in terms of motivation and performance.\n\n- **Motivation**: First, the auxiliary autoencoder should possess strong expressive capabilities, as it is required to directly reconstruct the high-dimensional vision-language representations of LVLMs. Therefore, we design separate encoders and decoders for each modality, rather than relying on a shared model, to enhance the model's performance on this reconstruction task. Second, inspired by the successful training protocol of CLIP, our approach aims to establish a correlation between the semantic similarity and the representation cosine similarity. Therefore, we adopt a modality-specific architecture similar to that of CLIP.\n\n- **Performance**: To address your concern, we provide comparisons between the shared and modality-specific strategies. We observe that employing a modality-specific strategy for the auxiliary autoencoder results in lower reconstruction loss and improved concept quality of the corresponding VL-SAE.\n\n    | Method            | Recon. Loss | Intra-Similarity ($\\uparrow$) | Inter-Similarity ($\\downarrow$) |\n    | ----------------- | ----------- | ---------------- | ---------------- |\n    | share             | 0.0047      | 0.2157           | 0.2021           |\n    | modality-specific | **0.0034**      | **0.2283**           | **0.1762**           |\n\n**Revision**. We will incorporate the above analysis into the revision.\n\n## W4: Moving Figure A8.\nThank you for your suggestion! We will present Figure A8 in Section 4.2 in the revised paper.\n\n## W5: Utilizing L1 loss for sparsification.\n\n- **Experiments**: Following your suggestion, we compare the performance of utilizing Top-k and L1 loss for sparsification. We set the coefficient of L1 loss to 1e-4. \n\n    | Method | Intra-Similarity ($\\uparrow$) | Inter-Similarity ($\\downarrow$) |\n    | ------ | ---------------- | ---------------- |\n    | L1     | 0.2142           | 0.1809           |\n    | Top-K  | **0.2442**           | **0.1373**           |\n\n- **Analysis**: We find that utilizing Top-K for sparsification achieves better performance than L1 loss. This is because the L1 loss cannot directly control the representation sparsity. Our results are consistent with previous studies [R1], which also indicate that Top-K sparsification is a more effective approach.\n\n**Revision**. We will incorporate the above experiments and analysis into the revision.\n\n[R1] Scaling and evaluating sparse autoencoders, ICLR'25."}, {"review_id": "jJNM4E50oH", "reviewer": "Reviewer_obSK", "summary": "This paper aims to solve a problem that current methods cannot map the semantics of vision and language representations into a unified concept set, when interpreting representations of VLMs by mapping their semantics to concepts. The paper proposed to apply a shared SAE (sparse autoencoder) for both modalities. Besides the interpretation method evaluation, the author also provides two applications for enhancing the alignments of both the CVLMs and LVLMs.", "questions": "1. In the introduction, it said, “This limited comprehension hinders our ability to analyze and address the misalignment cases, such as hallucinations. To tackle this challenge, existing methods interpret representations of VLMs by mapping their semantics to concepts.”, There are other interpretation methods that explain the matching of image-text pair by attention heat maps, such as Grad-ECLIP[1], and GAE[2]. Should discuss the Pros and Cons of explanation via concepts and attention maps.\n\n[1] Zhao C, Wang K, Zeng X, et al. Gradient-based visual explanation for transformer-based clip[C]//International Conference on Machine Learning. PMLR, 2024: 61072-61091.\n\n[2] Chefer H, Gur S, Wolf L. Generic attention-model explainability for interpreting bi-modal and encoder-decoder transformers[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 397-406.\n\n2. My main concern about the interpretation methods using SAE is that this kind of method seems to build a way to find similar samples for the given image-text pair in the dataset. We may find some regular patterns from the found set, but this kind of method still does not provide which concepts/objects in the specific image are matched with which words in the text description, which, in my opinion, is a more meaningful interpretation way.\n3. Will the number of concepts be limited by the number of hidden neuron activations? Based on A.1, 100 concepts are randomly selected to conduct the evaluation. Then, how many concepts will be learned in total?\n4. In Figures 5 and 6, it’s unclear about the relationship between the image examples and the text under the image. For example, in Fig. 5, in the concept of image, there are three images, with the following text: “Motorcycle and scooter next to each other.” Are the three images and the text selected from the same node? So why is there only one text while three images? Moreover, there seems to be no scooter in these images, why? \n5. In the enhancing vision-language alignment of CVLMs part, I didn’t find the training details, like the training datasets, and not sure whether the training set will influence the zero-shot performance.\n\nI will consider increasing the rating after the questions are solved.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper is well-written and easy to follow.\n- The methodology is displayed clearly and thoroughly. \n- Comprehensive experiments are conducted.\n\nWeeknesses:\nsee the questions.", "was_revised": true, "final_justification": "Thank you for the rebuttal. This resolves majority of my concerns. I will keep my score.", "rebuttal": "We would like to thank you for the insightful comments! We will diligently follow your guidance to enhance the clarity and comprehensibility of our discussions.\n\n## Q1: Discussions with attribution-based methods.\n\nThank you for sharing these valuable papers, which significantly enrich our discussion and enable a more comprehensive analysis! \n\nFollowing your suggestions, we first provide the essential paradigm of attribution-based and concept-based methods. Then we compare them in terms of intuitiveness, expressivity, and ability to improve model performance.\n\n1. **Paradigm**. \n\n    - Attribution-based methods generate a heat map for each sample to represent the importance of input data (e.g., image pixels).\n    \n    - Concept-based methods construct a concept set for varying samples to interpret the semantics of their hidden representations.\n\n2. **Comparisons**. \n\n    - **Intuitiveness**. Compared with concept-based methods, `attribution-based methods offer a more intuitive interpretation`, as they visualize attribution scores directly on input images. \n\n    - **Expressivity**. Compared with attribution-based methods, `concept-based methods exhibit stronger expressivity` as they reveal how the model understands the semantics of input data with concepts, rather than simply providing numerical information to identify the important image regions. \n\n    - **Ability to improve model performance**. In contrast to attribution-based methods that only serve as an observer for model prediction, `concept-based methods can act as an intervener to improve model performance` by modifying hidden representations during inference (e.g., our experiments in Sections 4.3). \n\n**Revision**. We will incorporate the above discussion and previous studies (i.e., Grad-ECLIP and GAE) into the revision. \n\n## Q2: Discussions of the interpretation form.\n\nWe appreciate your insightful comments that inspire us to incorporate sample-specific concepts into SAEs for more intuitive interpretations.\n\nTo address your concern, we first analyze the Pros and Cons of using sample-specific concepts. Then, we provide potential improvements to integrate their advantages into SAEs.\n\n1. **Pros**: `Extracting concepts from input samples (i.e., image regions, words) can offer intuitive interpretations` similar to those you described. \n2. **Cons**: However, the sample-specific property `makes it challenging to analyze and intervene in the predictions across different samples`, which is important in the field of mechanism interpretability [R1].\n3. **Potential improvements**: Despite SAEs offering the analysis and intervention across samples, we agree with your suggestions that they should be improved to provide more intuitive interpretations. We present this potential improvement as part of future work:\n\n    - Given an input sample, we plan to build sample-specific concepts and correlate them with multi-modal concepts of VL-SAE. With the correlation, the VL-SAE concepts serve as a bridge, enabling the match of image regions and words.\n\n**Revision**: We will incorporate the above discussions in the revision, and attempt to improve the interpretation form of VL-SAE in our future work.\n\n## Q3: Number of concepts in VL-SAE\n\nThank you for your careful reading of our paper! \n\n1. The number of concepts is upper bounded by the number of hidden neurons (denoted by $h$ in L216). \n2. The number of concepts is influenced by the training process and SAE architectures. This stems from the inevitable existence of dead neurons in SAEs [R2], which are never activated. We provide the number of concepts within different SAEs based on LLaVA 1.5 with 4096 feature dimensions. `VL-SAE learns more concepts compared to existing methods`.\n\n    | Method | Hidden Neuron         | Dead Neuron | Concept Number |\n    | ------ | --------------------- | ----------- | -------------- |\n    | SAE-D  | 32768 (4096$\\times$8) | 54          | 32714          |\n    | SAE-S  | 32768 (4096$\\times$8) | 46          | 32722          |\n    | VL-SAE | 32768 (4096$\\times$8) | **15**      | **32753**      |\n\n3. For evaluation, we observe that the metrics `showed no significant differences when more than 100 concepts are used`. Since the evaluation needs to be repeated multiple times, we select 100 concepts in each iteration for efficient evaluation.\n\n    | Concept Num.              | 100    | 200    | 300    | 400    | 500    |\n    | ------------------------- | ------ | ------ | ------ | ------ | ------ |\n    | Intra Similarity (SAE-S)  | 0.2200 | 0.2257 | 0.2281 | 0.2245 | 0.2247 |\n    | Intra Similarity (VL-SAE) | **0.2445** | **0.2508** | **0.2489** | **0.2476** | **0.2469** |\n    | Inter Similarity (SAE-S)  | 0.1347 | 0.1293 | 0.1300 | 0.1306 | 0.1308 |\n    | Inter Similarity (VL-SAE) | **0.1230** | **0.1214** | **0.1182** | **0.1181** | **0.1195** |\n    \n**Revision**. We will incorporate the above analysis into Section 4 of the revision.\n\n## Q4: Clarifications of Figure 5 and Figure 6.\nThank you for your comments that help us refine the illustration of Figures 5 and 6!\n\n1. `The three images and the text are maximum activation samples of the same hidden neuron`. We represent each concept with samples that maximally activate its corresponding neurons in SAE, following [R3]. Given that VL-SAE can process multi-modal inputs, we choose both images and texts to represent each concept. \n2. `The number of images and texts is not strictly fixed`. For example, we use two images and two sentences to represent concepts in Figure 4. We present three images and one sentence per concept in Figures 5 and 6 due to page limitations.\n3. `The concept semantics in SAE are represented by the summarization of its maximum activation samples`. Using a single sentence may introduce bias, like \"scooter\" as you mentioned. We provide more texts of this concept as follows: \n    - \"Motorcycle at meeting for bikers an annual event held\"\n    - \"Motorcycle parked by the side\"\n    - \"Motorcycle stood on its rest parked on the pavement\"\n    - \"Motorcycle at a rally or meeting\"\n    \n    We can observe that this concept can be summarized as a semantic related to motorcycles.\n\n**Revision**: In contrast to displaying maximum activation texts, we will employ LLMs to summarize these texts following [R4] to provide a clearer visualization in Figures 5 and 6.\n\n## Q5: Implementation details of Table 2.\nThank you for your careful reading. We would like to clarify that `VL-SAE can enhance VLMs without fine-tuning on downstream tasks`. The VL-SAE is only pre-trained with the CC3M dataset self-supervisedly.\n\nTo address your concern, we investigate the influence of CC3M on downstream tasks by comparing the base model (CLIP-ViT-B/16), the base model fine-tuned by CC3M, and the base model equipped with VL-SAE. \n\n| Method        | Base Model | Base Model+Fine-tuning | Base Model+VL-SAE |\n| ------------- | ---------- | ---------------------- | ----------------- |\n| Mean Accuracy | 69.8       | 69.7                   | **70.4**          |\n\n1. `Fine-tuning the base model with CC3M cannot improve performance`. This is because CC3M is a subset of the VLM pre-training dataset. It does not provide the VLM with any additional task-relevant information. \n\n2. The superior performance of VL-SAE indicates that `the improvement stems from the mapping from representations to concepts`, rather than the information of CC3M.\n\n**Revision**. We will incorporate the above experiments and implementation details into Section 4.3 of the revision.\n\n[R1] Open Problems in Mechanistic Interpretability, Arxiv'25.\n\n[R2] Scaling and evaluating sparse autoencoders, ICLR'25.\n\n[R3] Sparse autoencoders reveal selective remapping of visual concepts during adaptation, ICLR'25.\n\n[R4] Sparse autoencoders find highly interpretable features in language models, ICLR'24"}]

Author rebuttal:

Meta-review: This submissions proposes to map the semantics of vision-language representations to a unified concept set and thus interpret the alignment mechanism of VLMs, spanning a bridge between contrastive VLMs like CLIP, which learn explicit alignment, and LVLMs, which do this alignment implicitely. During the review process the paper received mixed reviews with ratings from 3/BR to 5/A. The reviewers appreciated the core idea and the interpretability aspects.

Several weaknesses were raised:
- Presentation and writing, in particular the experimental section which lacks details
- Qualitative results are difficult to evaluate.
- Questions on the origins of gains
- Sensitiviy on the amount of training data
- Comparisons to competing work 

Post author/reviewer discussion also raised the problem of the introduction of a new distance function without any ablation or justification of this choice. 

The authors provided a rebuttal and attempted answers to some of the concerns, in particular some limited experiments with human evaluation (10 people), sensitivity experiments, which most reviewers found convincing. While the paper still has flaws, the AC judges that the advantages outweight the issues and recommends acceptance.

The authors are requested to provide the additional ablatation in the camera ready version.
Decision: Accept (poster)