Interpreting the linear structure of vision-language model embedding spaces
Authors: Isabel Papadimitriou
Huangyuan Su
Thomas Fel
Sham M. Kakade
Stephanie Gil
Venue: COLM 2025 (2025)
Keywords: vision-language models
interpretability
cross-modality meaning
TL;DR: We train and release sparse autoencoders on the joint vision-language spaces of four models, and examine what the linear concepts reveal about the joint organization of meaning and modality

Abstract: Vision-language models encode images and text in a joint space, minimizing the distance between corresponding image and text pairs. How are language and images organized in this joint space, and how do the models encode meaning and modality? To investigate this, we train and release sparse autoencoders (SAEs) on the embedding spaces of four vision-language models (CLIP, SigLIP, SigLIP2, and AIMv2). SAEs approximate model embeddings as sparse linear combinations of learned directions, or ``concepts''. We find that, compared to other methods of linear feature learning, SAEs are better at reconstructing the real embeddings, while also able to retain the most sparsity. Retraining SAEs with different seeds or different data diet leads to two findings: the rare, specific concepts captured by the SAEs are liable to change drastically, but we also show that commonly-activating concepts are remarkably stable across runs. Interestingly, while most concepts activate primarily for one modality, we find they are not merely encoding modality per se. Many are almost orthogonal to the subspace that defines modality, and the concept directions do not function as good modality classifiers, suggesting that they encode cross-modal semantics. To quantify this bridging behavior, we introduce the Bridge Score, a metric that identifies concept pairs which are both co-activated across aligned image-text inputs and geometrically aligned in the shared space. This reveals that even single-modality concepts can collaborate to support cross-modal integration. We release interactive demos of the SAEs for all models, allowing researchers to explore the organization of the concept spaces. Overall, our findings uncover a sparse linear structure within VLM embedding spaces that is shaped by modality, yet stitched together through latent bridges—offering new insight into how multimodal meaning is constructed.

Published as a conference paper at COLM 2025
Interpreting the Linear Structure of Vision-language Model Embedding Spaces
Isabel Papadimitriou Huangyuan Su Thomas Fel* Sham Kakade..b Stephanie Gil
"Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University è "Department of Computer Science, Harvard University
{ [isabelpapadimi triou, csu, tfel, sham, sgi 1 }eg. sarvard.edu
Abstract
Vision-language models encode images and text in a joint space, mini- mizing the distance between corresponding image and text pairs. How are language and images organized in this joint space, and how do the models encode meaning and modality? To investigate this, train and release sparse autoencoders (SAEs) on the embedding spaces of four vision- language models (CLIP, SigLIP, SigLIP2, and AIMv2). SAEs approximate model embeddings as sparse linear combinations of learned directions, "concepts". We find that, compared to other methods of linear feature learning, SAEs are better at reconstructing the real embeddings, while also able to retain the most sparsity. Retraining SAEs with different seeds or different data diet leads to two findings: the rare, specific concepts cap- tured by the SAEs are liable to change drastically, but we also show that remarkably stable across runs. Interest- ingly, while most concepts activate primarily for one modality, we find they not merely encoding modality per se. Many are almost orthogonal to the subspace that defines modality, and the concept directions do not func- tion as good modality classifiers, suggesting that they encode cross-modal semantics. To quantify this bridging behavior, we introduce the Bridge Score, a metric that identifies concept pairs which are both co-activated across aligned image-text inputs and geometrically aligned in the shared space. This reveals that even single-modality concepts can collaborate to support cross- cross-modal integration. We release interactive demos of the SAES for all models, allowing researchers to explore the organization of the concept spaces. Overall, our findings uncover sparse linear structure within VLM embedding spaces that is shaped by modality, yet stitched together through latent bridges bridges-offering new insight into how multimodal meaning is constructed.
commonly-activatiin concepts
1 Introduction
How do vision-language models (VLMs) organize their internal space in order to relate text and image inputs? Multimodal models encode images and text in a joint space, enabling many impressive downstream multimodal applications. In this paper, we explore how multimodal models encode meaning and modality in order to try and understand how cross-modal meaning is expressed in the embedding space of vision-language models.
We approach this question through dictionary learning: the class of methods that consists of finding linear directions (or, "concepts") in the latent space of the model, that can break down each embedding into a linear combination of more interpretable directions. We train multiple dictionary learning methods on top of the four VLMs under consideration (CLIP (Radford et al., 2021), SigLIP (Zhai et al., 2023) SigLIP2 (Ischannen et al., 2025)
* These authors contributed equally to this work.

---

Published as a conference paper at COLM 2025
and AIMv2 (El-Nouby et al., 2024; Fini et al., 2024)), and settle BatchTopK Sparse Autoencoders (SAEs) as the method that dominates the frontier of the tradeoff between remaining faithful to the original embedding space while being maximally sparse. Using the concepts extracted by BatchTopk SAEs, we run a series of investigations to understand how vision-language embedding spaces work. Our analysis is grounded in ambitious empirical effort: we train and compare SAEs across four distinct vision-language models, using hundreds of thousands of activations, and carry out a thorough suite of evaluations that dissect the geometric, statistical, and modality-related structure of their embedding spaces.
We show that SAEs trained with different seeds and with different training data mixtures are stable and robust when we consider the concepts that are commonly used (high- energy concepts), but very unstable on the concepts that are used very rarely (low-energy concepts) (Section 4).
We show that almost all concepts are single-modality concepts: they activate almost exclusively either for text or for image concepts, and this holds across all models that we try (Section 5.1)
However, we also show that this does not mean that the concepts lie within the separate cones of the image and text activations: across models, a large proportion of concepts are almost orthogonal (but not totally orthogonal) to the subspace that encodes modality, and therefore encode largely cross-modal dimensions of meaning (Section 5.2).
We develop VLM-Explore (https: htt////lmomooeppeee cancept-visuall aoualization com/), an interactive visualization tool that lets researchers explore the linear concepts in a model, and how they connect modalities (Section 6).
Overall, our findings show how the linear structure of vision-language embedding space can be used to understand the mechanics of joint vision-languaage space, and we provide the SAEs and the visualization tools for researchers to further explore these connections. Our exploration into the workings of VLM spaces reveals an interesting two-sided conclusion: while the space is organized primarily in terms of modality, single-modality can still be related through high cosine similarities on the subspace that is orthogonal to modality, creating cross- moosl bridges of meaning.
2 Background and related work
Linear Representations in Neural Networks The representation spaces of neural networks have been often shown to encode many important features (like syntactic or object-category information) largely linearly (Alain & Bengio, 2016; Hewitt & Manning, 2019; Belinkov, 2022; Li et al., 2016; Conneau et al., 2018; Fel et al., 2024a), since, after all, the operations on these spaces are largely linear In vision-language models, strong performance on zero-shot tasks suggests that their embeddings are structured along meaningful directions. Empirical evidence from probing and representation analyses further supports the idea that semantic content is linearly organized Muttenthaler et al. (2023; 2024); Nanda et al. (2021); Tamkin et al. (2023); Bricken et al. (2023).
Sparse Coding and Dictionary Learning Dictionary Learning (Tošić & Frossard, 2011; Rubinstein et al., 2010; Elad, 2010; Mairal et al., 2014; Dumitrescu & Irofti, 2018) has emerged as a foundational framework in signal processing and machine learning for uncovering latent structure in high-dimensiionl data. It builds upon early theories of Sparse Coding, developed in computational neuroscience Olshausen & Field (1996; 1997); Foldiak & Endres (2008), where sensory inputs were modeled as sparse superpositions of overcomplete basis functions. The core objective is to recover representations tions where each data point is approximated as a linear combination of small number of dictionary atoms Hurley & Rickard (2009); Eamaz et al. (2022). This sparsity constraint encourages interpretability by associating individual basis vectors with meaningful latent components. In neural network interpretability, sparse decompositions have gained traction as tools to extract interesting features from learned representations (Elhage et al., 2022; Cunningham et al., 2023; Fel et al., 2023; 2024b; Rajamanoharan et al., 2024; Gorton, 2024; Surkov et al., 2024).
2

---

Published as a conference paper at COLM 2025
Siglip2
Siglip
Clip
Aimv2
1.00
0.90
0.90
0.9
0.95
0.85
0.8
0.90
0.80
0.80
0.85
0.75
0.75
0.80
0.70
0.70
0.75
100
50
20
10
100
50
20
10
100
50
20
10
100
50
20
10
fc
fo
fc
fo
Semi-NMF
Vanilla SAE
Top-K SAE
JumpRaLUSAE SAE
BatchTopk SAE
Figure 1: Selecting sparse dictionary learning method: the Expressivity-Spariit trade- off. Pareto fronts for five dictionary learning methods applied to four vision-language models (CLIP, SigLIP, SigLIP 2, AIMv2). Each curve shows the trade-off between reconstruc- tion quality (R2 score) and sparsity level (6g norm of Z). The three SAEs (TopK, JumpReLU, and BatchTopK) consistently achieve the best balance, with Batch To slightly dominating other sparse autoencoder variants.
Sparse autoencoders (SAEs) (Makhzani & Frey, 2014) scalable instantiation of Sparse Dictionary Learning implemented by training a small neural network. Recent work has shown that SAEs extract semantically meaningful concepts (Bricken et al., 2023; Bussmann et al., 2024; Thasarathan et al., 2025; Paulo & Belrose, 2025; Gao et al., 2025; Bhalla et al., 2024; Fel et al., 2025), often yielding more interpretable and compositional decompositions than PCA or ICA (Bussmann et al., 2024; Braun et al., 2024; Chen et al., 2024; Fel et al., 2024b; Makelov et al., 2023).
Modality and Interpretability in Multimodal Models A growing body of work has examined how multimodal models encode meaning and modality in their internal repre- sentations. Despite being trained to align modalities, vision-language models exhibit persistent modality gap, where text and image embeddings occupy distinct conical regions in the joint space (Liang et al., 2022; Shukor Cord, 2024). However, examining the neurons (Goh et al., 2021; Shaham et al., 2024), representations (Bhalla et al., 2024; Parekh et al., 2024; Wu al., 2024) shows that multimodal models showcase aspects of cross-modal internal processing.
3 Methods: Sparse Autoencoders on Vision-Language Models
3.1 Background and Notation.
We denote vectors by lowercase bold letters (e.g., x) and matrices by uppercase bold letters (e-g., X). We write [n] to denote the index set [1,...,n] The unit b2-ball in I4 is defined as {x E Rd ||x||| < 1). We consider a general multimodal representation learning setting in which vision-language model maps image or text inputs x E X' into a shared representation space A C RR. A collection of n such embeddings is stored in matrix A E Rexd Dictionary Learning seeks to approximate A linear combination of concept vectors from a dictionary E IRexd, using a sparse code matrix Rixc.
D*) arg min |
ZD
(1)
Z
where ||
|| denotes the Frobenius norm, and dictionary atoms are constrained to lie on B.
In the case of Sparse Autoencoders (SAEs), an encoder network 4), single-layer MLP, maps embeddings A to sparse codes Z via linear transformation and projection (Hindupur
3

---

Published as a conference paper at COLM 2025
et al., 2025) operator II (:):
Z 4 = ((W bb}
(2)
where W E Raxe and b E R' are learnable encoder parameters. In particular, we consider several SAEs that can be described by their projection operators such as:
ReLU((x) = ma
(x) =
JumpReLU(x) max 0, x - 0) + H(x- 0)
(3)
Txp arg min 1=-xpp x11 s.t. ||=||o k, = 0
Where H is the Heaviside step function. We will also explore a simple extension, BatchTopK, which flexibly shares applies the TopK operation across the entire batch.
3.2 Selecting a sparse dictionary method: Expressivity vs sparsity
An ideal dictionary learning method should produce representations that are both expres- sive and sparse. In practice, this means that model embeddings should activate only a small number of meaningful concepts, indicating that the learned dictionary is both specific and semantically aligned with the data. In our first set of preliminary experiments, we compare five methods this trade-off, and present our results in Figure
Dictionary learning methods: We test five different dictionary learning methods: Semi- Nonnegative Matrix Factorization (Semi-NMF) (Parekh et al., 2025; Lee & Seung, 1999), Sparse Autoencoders (SAEs), Top-K SAEs (Gao et al., 2025), JumpReLU SAEs (Rajamanoha- ran et al., 2024), and BatchTopK SAEs (Bussmann et al., 2024), and train using Overcomplete.
Models: We run experiments on four models: CLIP (Radford et al., 2021), SigLIP (Zhai et al., 2023), and SigLIP2 (Tschannen al., 2025), which are encoder models, and AIMv2 (El-Nouby et al., 2024; Fini et al., 2024), which is an autoregressive vision-language model
Data: We train all dictionary learning methods on reconstructing the activation matrix A, consisting of 600,000 normalized embeddings from passing the COCO dataset (Lin et al., 2014) through the models, with each A; B, the unit bz-ball
Results: We evaluate the expressivity and sparsity of all methods, and present our results in Figure 1. We measure sparsity as the number of non-zero entries in the code vector (i.e., the Eo of Z). For different levels of sparsity, we plot the reconstruction quality (expressivity): the R² score between the original activations A and their reconstructions ZD. The trade-off captures how well the learned codes retain information while encouraging interpretability through sparsity.
Our results indicate that all four SAE variants outperform Semi-NMF at fixed sparsity levels, with TopK achieving slightly better performance overall. In practice, we also find that training TopK and BatchTopK SAEs is more stable and requires less hyperparameter tuning. For these reasons, we use the BatchTopK SAEs for our subsequent analysis experiments.
3.3 Metrics: The geometry and statistics of learned concepts
To assess the interpretability and quality of the learned representations, we introduce four complementary metrics that go beyond the standard sparsityrreconstructin trade-off: energy, stability, modality score, and bridge score. Each of these measures captures different axis of representational information:
Energy. Energy is measure of how often each concept is used: its average activation strength across a dataset. Formally, for a concept i, we define its energy as: Energy, = (2,).. Energy provides statistical view over the dataset, revealing where attention should be focused for interpreting the learned representation.
https://github http.///eeeeeeeentitttttee///eee

---

Published as a conference paper at COLM 2025
1.0
1.0
1.0
over 5 random seeds
0.8
512 concepts, 99% energy
0.8
0.8
99% top concepts,
Energy
Stability
0.92 stability
0.6
0.4
0.4
0.2
0.16 overall stability
0.2
0.2
0.0
0.0
2000
4000
2000
4000
0.0
0.5
1.0
Number of concepts
Number of concepts
Energy
(ranked by energy)
Figure 2: The concepts that use most of the energy are stable Here we compare the relationship between energy and stability: (Left) Energy concentrated with 512 concepts getting 99% of the energy, and the other 3,500 concepts only appearing in 1% of the total coefficient weight. (Center) When training with different random seeds, the stability when we consider all 4,096 concepts is low 0.16 (Right) When we weight concepts by energy, we can see that the concepts that are used often reconstruction are actually stable, with stability of 0.92 if we take the top 512 energetic concepts from each run. The instability comes from concepts that are rarely if ever actually used.
few concepts,
Stability. Stabilityy quantifies how consistent the learned dictionaries between two runs. If two independently trained SAEs on the same model produce significantly different dictionaries, suggests that the learned concepts are not robust, which weakens their interpretability. Following Fel et al. (2025) and Spielman et al. (2012), we compute the stability between two dictionaries D, D' E Rexd by optimally aligning their rows using the Hungarian algorithm, which solves for a permutation matrix P P(c) that maximizes the total similarity between matched concept vectors:
Stability(D,DD)) = max
Tr(D TDD).
PEP(c)
higher stability score indicates that concept representations are reproducible across training runs, reinforcing their semantic reliability.
Modality Score. In the context of VLMs, a central question whether a concept is modality-specific shared across modalities. To probe this, introduce the modal- ity score, which quantifies how much a concept contributes to reconstructing image versus text inputs. Let and T be the empirical distributions of sparse codes z from image (1) and text (T) inputs, respectively. We define the modality score of a concept i as the fraction of expected activation energy assigned to image-based inputs:
(z))(2))
ModalityScore;
Ez-1 (z,) + (E2~T(Z))
A modality score near indicates that the concept fires only image inputs, while score near suggests text dominance. A value around & indicates a multi-modal concept.
Bridge Score. While the modality score offers a per-concept perspective, many vi- sion-language tasks rely coordinated interactions between concepts across modalities. To capture these interactions, we introduce the bridge matrix B E Rexe, which quantifies how pairs of concepts contribute to the alignment of modalities. Let (z,, ~ 7 pair of sparse codes obtained from matching image-text input pair, drawn from joint empirical distribution over aligned data points. Let D RRexd denote the shared dictionary. The bridge matrix is defined as:
B E (a
(z,
Zr
④ DD
(6)
co-activation
alignment

---

Published as a conference paper at COLM 2025
5:1
1:2
Image concept
Text
concept
Figure The geometry of high-energy concepts is stable across data mixtures. Top: each concept has equal size, Bottom: size is dependent on energy. UMAP visualization of the SAE concept spaces under different image-text data mixtures. Color indicates the modality score of each concept. While the dominant modality in the training data strongly influences how many concepts are recovered per modality (top row), the most energetic concepts (botfom row) remain relatively stable across mixtures. The set of high-energy text and image concepts remains relatively consistent regardless of the input distribution. Additional results for the other three models are provided in Appendix A, Figure 9.
with @ being the Hadamard product. The first term captures statistical co-activation between concepts: how frequently concept in the image code and concept j in the code are simultaneously active for semantically aligned inputs. The second term reflects the geometric alignment between the corresponding dictionary atoms, computed as the cosine similarity between dictionary atoms The elementwise product combines both aspects-activation and alig single interpretable structure.
The resulting matrix B reveals how concepts jointly operate across modalities. In this sense, the model succeeds at cross-modal alignment. It highlights not just which concepts are shared, but how they collaborate structurally and statistically to support the model's semantic integration of vision and language.
the bridge matrix provides insight into why
4 Analysis 1: Are We Finding Consistent Model Features?
Before analyzing the geometry and semantic content of the concepts extracted by our SAEs (Section 5), we first examine whether these concepts are consistent and stable: do we get similar concepts across different conditions? We study stability across: (i) random seeds-to to test sensitivity to training stochasticity, and (ii) data modality mixtures test dependence on the distribution of input activations.
4.1 The concepts that most of the energy stable
We train five SAEs with different random seeds on the same SigLIP2 activations, each using 4096 concepts, and compute the average pairwise stability (Equation 4) between them. As shown in Figure (center), at first glance, overall stability is low: the mean similarity between dictionaries is just 0.16, suggesting high sensitivity to random initialization. How- ever, this apparent instability is misleading. On the left, see that reconstruction energy is heavily concentrated in a small subset of concepts: the top 512 concepts account for approximately 99% of the total activation mass. To isolate the functionally relevant features, we recompute stability using only the top-k most energetic concepts from each run. We find that high-energy concepts remarkably stable, with near-perfect alignment across

---

Published as a conference paper at COLM 2025
M Modality score histogram
CLIP
SigLiP
gigLiP2
AlMv2
imae
600
400
200
0.00
0.25
0.75
0.00
0.50
1.00 0.25
0.50
0.75
1.00
0.25
0.50
0.75
Modality score
texT
Modality score histogram, weighted by energy
CLIP
LigLP
SigLIP2
AIMv2
image
15-
Density
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75 1.00
0.00
0.25
0.50
0.75
1.00
Modality score
Figure 4: Most concepts are single-modality Histograms of the modality scores (top) and the modality scores weighted by energy (bottom) of every concept in the SAEs. On every model, the modes are the two extremes: concepts either activated only by text or only by image. Weighting by energy makes this much more prominent, showcasing that almost all of the reconstruction coefficients go to concepts that are single-modality.
seeds (Figure 2, right). The instability arises almost entirely from the low-energy tail of the dictionary connonrry-coceptss that contribute little to the reconstruction objective. We conclude that SAEs trained with different seeds reliably recover a core set of functional, high-usage concepts.
4.2
The geometry of high-energy concepts is stable across data mixtures
Though we have seen that the high-use concepts retrieved by the SAE are stable across random initializations, there still the question of how much the recovered concepts are influenced by the data used to train the SAEs. Do SAE dictionaries reflect the structure in the model or simply the statistics of the dataset used to train the SAE? To test this, we train SAEs varying the modality mixture of the training data: how many of the activations we use to train the SAE come from image inputs vs. text inputs. We train SAEs on seven data compositions, ranging from heavily image-dominant (5:1 image:text) to heavily text- dominant (1:5). Results are shown in Figure 3, where we contrast the UMAP geometry of the learned concepts concepts (top) with the geometry of the concepts when the size of each point is weighted by the concept's energy. As expected, changing the data mixture shifts the overall modality distribution of the learned concepts: the text-heavy SAEs learn more text concepts (blue points) and the image-heavy SAEs learn more image concepts. However, when weighting by energy (bottom row), the geometry of the most energetic concepts remain stable across all mixtures: the visualizations look remarkably consistent across data mixtures. High-energy directions consistently appear regardless of the training distribution, and exhibit similar activation patterns.
Discussion: Why low-energy concepts? Our evaluations show that different SAEs dis- cover widely different low-energy features. Does Does this mean that low-energy concepts are useless? Not necessarily. Leveraging VL.M-Explore (Section 6) we observe that many low-energy concepts correspond to coherent but rare patterns, such as "yaks" or "valen- tines day hearts." These concepts may be semantically meaningful, but their contribution to reconstruction is small, and thus SAEs do not consistently recover them. By contrast, high-energy concepts typically capture more concepts like "red" "two women" and are consistently recovered because they support wide range of reconstructions, We provide some illustrative examples of concepts in Appendix B.
7

---

Published as a conference paper at COLM 2025
Cononess of using Clp.
Cononess Astivations using Sglip
Coneenes Activatoons using siglio2
Coneness of Activations using Alhhy2
bod look
tonbian:
Image- saggg-
image imagee
imagermmae text image
book moogg
lubb maag
tolangge
Denstt
Coiino Sinlunity
Codene Semiurky
Gavrt Semvlatiyy
Crrune Senilarity
Figure 5: Image and text activations lie in separate cones. We plot the distribution of cosine similarities between model activations. Image-image and text-text pairs (blue/ green) are close, while image-text pairs (orange) are much farther, indicating that embeddings occupy separate modality-specifie cones.
These analyses show that while SAE dictionaries may appear unstable, this instability is concentrated in low-energy concepts. The high-energy subset-reeponsible for nearly all reconstruction- both robust to random initialization and is also not simply a reflection of the data. These findings guide the rest of this paper. They suggest that SAEs can be used as reliable tools for interpreting VLM embeddings- only when conditioned on energy. Concepts with high energy form stable and informative basis; concepts with low energy may still be meaningful, but should be treated with caution due to their instability.
Analysis 2: Are We Finding Features With Cross-Modal Meaning?
Having established that SAEs recover stable and consistent high-energy features, we now ask whether these features capture structure that spans both modalities. Specifically, we analyze whether the learned concepts reveal aspects of the geometry of the joint vision- language embedding space, and whether they support cross-modal alignment.
5.1 Most concepts are single-modality
Our first observation is that the majority of concepts activate predominantly a single modality-either text or image. As defined in Equation 5, the modality score of a concept measures the fraction of its total activation energy that comes from image inputs. A score near 1 indicates image-specific usage; a score near 0 indicates text-specific usage. In Figure 4, we plot the modality score distribution across all concepts for SAES trained on each model. The top row shows raw counts: while some concepts cluster around 0.5, the distributions are clearly bimodal, with dominant modes at and 1. This pattern becomes more pronounced when we weight each concept by its energy (bottom): almost all of the energy across all models is concentrated on concepts that are nearly exclusive to one modality.
We conclude that the most influential directions in the SAE dictionary those responsible for the bulk of reconstruction are overwhelmingly single-modality, despite the shared embedding space.
5.2
Image and text activations lie in separate cones but many concepts are orthogonal to this difference
In all models that we analyze, we find that the activations of image and text inputs lie in separate, narrow cones. To measure this, we compare the cosine distance between activations of different modalities to the distance between activations of the same modality. If the average cosine similarity between a set of points significantly different from zero, this means that the points lie in narrow cone (with respect to the origin), rather than being distributed around the unit ball evenly. Following findings by Tyshchuk al. (2023) (as well as Mimno & Thompson (2017) and Ethayarajh (2019) in the language-only context), we find that image and text activations lie on separate, narrow cones. We present our results in Figure 5: embeddings of the same modality (green, blue) are significantly closer than those of different modalities (orange), which are on average almost orthogonal. This suggests that image and text activations reside in distinct conical regions of the embedding space.
8

---

Published as a conference paper at COLM 2025
Can we use a concept to classify images text?
CLIP
SigLIP
SigLIP2
AIMv2
8-
Density
6-
0.5
0.6
0.7
0.8
0.9
1.0
0.5
0.6
0.7
0.8
0.9
1.0
0.5
0.6
0.7
0.8
0.9
1.0
0.5
0.6
0.7
0.8
0.9
1.0
Classifier accuracy
Weighted by energy - Can we use a concept to classify images vs text?
CLIP
gLIP
SigLIP2
A1Kv2
Densky
10-
0
1.0 0.8 0.9
1.0
0.6 0.7
0.8
0.9
0.5
0.6
0.7
0.8
0.9
Classifier accuracy
Figure 6: Many concepts are not aligned with the modality directions. Histogram of the accuracies of concepts used as classifiers (on the bottom each concept is weighted by its energy). An accuracy of 0.5 means that the concept is orthogonal to every linear direction that defines modality (and therefore likely encodes some aspect of meaning independent from modality), and an accuracy direction. We see that, especially in CLIP and AIMv2, there significant proportion of concepts that are almost orthogonal to the modality subspace. When we weight by energy, that accuracy-1 concepts high-energy, but that concepts less aligned with modality still receive sigmificant amount of energy.
means that the concept is aligned with modality
Do the concept vectors align with the modality structure? If they follow the geometry of the modality cones that separate image from text, they likely encode little cross-modal content. But if many lie near directions orthogonal to the modality subspace, they may instead reflect shared meaning. We test this by testing how well each concept performs as linear modality classifier. To do this, we project input activations onto the direction defined by the concept, and measure how well it separates image from text. High accu- racy implies alignment with modality; near-chance performance suggests the concept is modality- and may encode cross-modal structure. Figure 6 presents the distribution of classification accuracies for all concepts (left) and weighted by energy (right). While many concepts particularly high-energy ones are highly predictive of modality, substantial number achieve near-random accuracy. These findings suggest that many concepts geometri- cally close to modality-agnostic subspace, even though they activate for specific modality This points to the existence of sparse linear directions that support semantic alignment across modalities while remaining sensitive to modality itself.
5.3 Proposal: Cross-modal alignment in VLM linear concepts
Our results lead us to a proposal of how the linear directions found by SAEs in multimodal space function, of which we provide schematic in Figure
1. The image and text activations in the VLMs we study lie in separate, narrow cones (Figure 5), and SAE concepts tend activate primarily for one modality (Figure 4).
2. However, SAE concepts largely do not lie along the linear subspace that separates the image and text cones (Figure 6), and in practice they can express semantics bridge between modalities. The Bridge Score (Equation 6) shows how there are

---

Published as a conference paper at COLM 2025
many-to-many relationships between related concepts in text and image space (see Appendix B, Figure 16 for an illustrative example)
3. How can we resolve this tension? The resolution lies in considering the SAE projection effect (Figure 8): due to the sparsity constraints (like TopK), SAE concepts only activate for a tiny number of the input activations that they are well-aligned with. If an SAE concept almost orthogonal to the modality but leans slightly towards text, it is very unlikely to ever be in the top-K concepts for image input. Though the concept will seem like text-only concept, it is in fact aligned with many image outputs.
subspace,
Multimodal semantic directions
Text
Images
-0.6
-0.4
-0.2
0.0
On average, almost orthogonal cones
Figure 7: A schematic of our proposal toy 3D space. Though images and text can lie in separate orthogonal cones with respect to some subspace (in this case the x-y plane), cross-modal directions can emerge (in this case, the z direction coming towards us represents the cross-modal semantics of " "grrfffe"). In large latent spaces, many dimen- sions can form a complex multimodal space that the z-axis is standing in for here.
Figure 8: The SAE projection ef- fect: We plot the dot product of a set of text (blue) and image (or- ange) activations with an SAE con- cept. Even though the concept di- rection does not separate modality, the TopK thresholding step (dotted line) may consistently select only one modality
VLM-Explore: An interactive, functional concept explorer
Lastly, we present VLM-Explore (https://vlmm httts:://vmmcconcptttvvv visullliii ization. com), an interactive visualization tool designed facilitate the exploration and analysis of our SAE concept representations. VLM-Explore offers an intuitive visualization that combines into one interactive figure four important aspects that help us interpret and debug VLMs. These are: 1) the UMAP structure shows how the linear directions of the latent space relate to each other 2) the maximally activating examples panel shows what each linear feature actually represents 3) Each feature is colored according to its Modality Score (Equation 5) showing how the two modalities share the space, and 4) the Bridge Score connections show what what semantics connect the two modalities.
7 Discussion and Conclusion
We study the linear structure of vision-language embedding spaces using SAEs trained on four VLMs.. While concept dictionaries vary across seeds and data mixtures, the high-energy subset consistently recovered and accounts for nearly all reconstruction. Most concepts are single-modality in usage but often lie near the modality-agnosti subspace, suggesting shared structure and a modality score shaped by SAE thresholding rather than direction. introduce the Bridge Score, identifying concept
To analyze cross-modal alignment, pairs that are both geometrically aligned and co-activated. We release VLMM-Exllre,, an interactive tool to visualize these structures across models.
These insights lay a foundation for building more interpretable and controllable multimodal models, and open the door to future research into structured alignment mechanisms.
10

---

Published as a conference paper at COLM 2025
Acknowledgments
This work has been made possible in part by a gift from the Chan Zuckerberg Initiative Foundation to establish the Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University.
References
Guillaume Alain and Yoshua Bengio. classifier probes. ArXiv e-print, 2016.
Understanding intermediate layers using linear
Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 2022.
Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P Calmon, and Himabindu Lakkaraju. Interpreting clip with sparse linear concept embeddings (splice). ArXiv e-print, 2024.
Dan Braun, Jordan Taylor, Nicholas Goldowsky-Di Dill,,,, and Lee Sharkey. Identifying func- tionally important features with end-to-end sparse dictionary learning. ArXiv e-print, 2024.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-oodss, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023.
Bart Bussmann, Patrick Leask, and Neel Nanda. Batchtopk sparse autoencoders. ArXiv e-print, 2024.
Yiting Chen, Zhanpeng Zhou, and Junchi Yan. Going beyond neural network feature similarity: The network feature complexity and its interpretation using category theory. Proceedings of the International Conference Learning Representations (ICLR), 2024.
Alexis Conneau, German Kruszewski, Guillaume Lample, Loic Barrault, and Marco Baroni. What you can cram into a single vector: Probing sentence embeddings for linguistic properties. Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2018.
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. ArXiv e-print, 2023.
Bogdan Dumitrescu and Paul Irofti. Dictionary learning algorithms and applications. 2018.
Arian Eamaz, Farhang Yeganegi, and Mojtaba Soltanalian. On the building blocks of sparsity measures. JEFF Signal Processing Letters, 2022.
Alaaeldin El-Nouby, Michal Klein, Shuangfei Zhai, Miguel Angel Bautista, Alexander Toshev, Vaishaal Šhankar, Joshua M Susskind, and Armand Joulin. Scalable pre-training of large autoregressive image models. ArXiv e-print, 2024.
Michael Elad. Sparse and redundant representations: from theory to applications in signal and image processing. 2010.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodde Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition. Transformer Circuits Thread, 2022.
Kawin Ethayarajh. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNIP--JCCNIP), pp. 55-65, 2019.
11

---

Published as a conference paper at COLM 2025
Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadène, and Thomas Serre. Craft: Concept recursive activation factorization for explainability, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Thomas Fel, Louis Bethune, Andrew Kyle Lampinen, Thomas Serre, and Katherine Her- mann. Understanding visual feature reliance through the lens of complexity. Advances in Neural Information Processing Systems (NeurIPS), 2024a.
Thomas Fel, Victor Boutin, Louis Béthune, Rémi Cadène, Mazda Moayeri, Léo Andéol, Mathieu Chalvidal, and Thomas Serre. A holistic approach to unifying automatic concept extraction and concept importance estimation. Advances in Neural Information Processing Systems, 36, 2024b.
Thomas Fel, Ekdeep Singh Lubana, Jacob S. Prince, Matthew Kowal, Victor Boutin, Isabel Papadimitriou, Binxu Wang, Martin Wattenberg, Demba Ba, and Talia Konkle. Archetypal sae: Adaptive and stable dictionary learning for concept extraction in large vision models. ArXiv e-print, 2025.
Enrico Fini, Mustafa Shukor, Xiujun Li, Philipp Dufter, Michal Klein, David Haldimann, Sai Aitharaju, Victor Guilherme Turrisi da Costa, Louis Béthune, Zhe Gan, et al. Multimodal autoregressive pre-training of large vision encoders. ArXiv e-print, 2024.
Peter Foldiak and Dominík Maria Endres. Sparse coding. ArXiv e-print, 2008
Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. Pro- ceedings of the International Conference on Learning Representations (ICLR), 2025.
Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah. Multimodal neurons in artificial neural networks. Distill, 2021.
Liv Gorton. The missing curve detectors of inceptionvl: Applying sparse autoencoders to inceptionvl early vision. ArXiv e-print, 2024.
John Hewitt and Christopher D Manning, A structural probe for finding syntax in word representations. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2019.
Sai Sumedh R. Hindupur, Ekdeep Singh Lubana, Thomas Fel, and Demba Ba. Projecting assumptions: The duality between sparse autoencoders and concept geometry. ArXiv e-print, 2025.
Niall Hurley and Scott Rickard. Comparing measures of sparsity. IEEE Transactions on Information Theory, 2009.
Daniel D Lee and H Sebastian Seung. Learning the parts of objects by non-negative matrix factorization. nature, 401(67555:778-755 -91, 1999.
Jiwei Li, Will Monroe, and Dan Jurafsky. Understanding neural networks through represen- tation erasure. ArXiv e-print, 2016.
Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. Advances in Neural Information Processing Systems (NeurIPS), 2022.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. Proceedings of the IEFF. European Conference on Computer Vision (ECCV), 2014.
Julien Mairal, Francis Bach, and Jean Ponce. Sparse modeling for image and vision process- ing. Foundations and Trends in Compuler Graphics and Vision, 2014.
12

---

Published as a conference paper at COLM 2025
Aleksandar Makelov, Georg Lange, and Neel Nanda. Is this the subspace you are looking for? an interpretability illusion for subspace activation patching. ArXio e-print, 2023.
Alireza Makhzani and Brendan Frey. K-sparse autoencoders. Proceedings of the International Conference on Learning Representations (ICLR), 2014.
David Mimno and Laure Thompson. The strange geometry of skip-gram with negative sampling. In Empirical Methods in Natural Language Processing, 2017.
Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A Vandermeulen, and Simon Kornblith. Human alignment of neural network representations. Proceedings of the International Conference on Learning Representations (ICLR), 2023.
Lukas Muttenthaler, Lorenz Linhardt, Jonas Dippel, Robert A Vandermeulen, Katherine Hermann, Andrew Lampinen, and Simon Kornblith. Improving neural network repre- sentations using human similarity judgments. Advances in Neural Information Processing Systems (NeurlPS), 2024.
Vedant Nanda, Ayan Majumdar, Camila Kolling, John P Dickerson, Krishna P Gummadi, Bradley C Love, and Adrian Weller. Exploring alignment of representations with human perception. Proceedings of the AAAT Conference Artificial Intelligence (AAAI), 2021.
Bruno A Olshausen and David Field. Emergence of simple-cell receptive field properties by learning sparse code for natural images. Nature, 1996.
Bruno Olshausen and David J Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision Research, 1997.
Jayneel Parekh, Pegah Khayatan, Mustafa Shukor, Alasdair Newson, and Matthieu Cord. A concept-based explainability framework for large multimodal models. ArXiv e-print, 2024.
Jayneel Parekh, Pegah Khayatan, Mustafa Shukor, Alasdair Newson, and Matthieu Cord. A concept-based explainability framework for large multimodal models. Advances in Neural Information Processing Systems, 37:135783-135818, 2025.
Gonçalo Paulo and Nora Belrose. Sparse autoencoders trained on the same data learn different features. ArXiv e-print, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 9748-8763. PmLR, 2021.
Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, Janos Kramar, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. ArXiv e-print, 2024.
Ron Rubinstein, Alfred M Bruckstein, and Michael Elad. Dictionaries for sparse representa- tion modeling. Proceedings of the IEEE, 2010.
Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram, Evan Hernan- dez, Jacob Andreas, and Antonio Torralba. A multimodal automated interpretability agent. Proceedings of the International Conference on Machine Learning (ICML), 2024.
Mustafa Shukor and Matthieu Cord. Implicit multimodal alignment: On the generalization of frozen 1lms multimodal inputs. ArXiv e-print, 2024.
Daniel A Spielman, Huan Wang, and John Wright. Exact recovery of sparsely-used dictio- naries. The Journal of Machine Learning Research (JMLR), 2012.
Viacheslav Surkov, Chris Wendler, Mikhail Terekhov, Justin Deschenaux, Robert West, and Caglar Gulcehre. Unpacking sdxl turbo: Interpreting text-to-image models with sparse autoencoders. ArXiv e-print, 2024.
13

---

Published as a conference paper at COLM 2025
Alex Tamkin, Mohammad Taufeeque, and Noah D Goodman. Codebook features: Sparse and discrete interpretability for neural networks. ArXio e-print, 2023.
Harrish Thasarathan, Julian Forsyth, Thomas Fel, Matthew Kowal, and Konstantinos Derpa- nis. Universal sparse autoencoders: Interpretable cross-model concept alignment. ArXiv e-print, 2025.
Ivana Tošić and Pascal Frossard. Dictionary learning. IEEE Signal Processing Magazine, 2011.
Michael Ischannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hénaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. Siglip 2: Multilin- gual vision-language encoders with improved semantic understanding, localization, and dense features. Ar Xiv e-print, 2025.
Kirill Tyshchuk, Polina Karpikova, Andrew Spiridonov, Anastasiia Prutianova, Anton Raz- zhigaev, and Alexander l'anchenko. On isotropy of multimodal embeddings, Information, 14(7):392, 2023.
Zhaofeng Wu, Xinyan Velocity Yu, Dani Yogatama, Jiasen Lu, and Yoon Kim. The semantic hub hypothesis: Language models share semantic representations across languages and modalities. arviv preprint ar Xrio:24111.04986 2024.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2023.
A Additional Figures and Expanded Discussion
A.1 UMAP Visualizations Across Data Mixtures for all models
Figure 9 shows UMAP projections of the learned SAE concept dictionaries across different training data mixtures. Each subfigure corresponds to a different VLM (SigLIP2, SigLIP, CLIP), with UMAP applied to the learned dictionary atoms. These visualizations comple- ment the main Figure 3, now extended to multiple models.
The top rows present concepts with uniform dot size, providing an unweighted view of the dictionary's s structure. Here, modality is encoded by color reflecting the modality score (as defined in Eq. of the main paper). The bottom rows rescale each concept's dot size according to its energy score, revealing the relative contribution of each concept to the reconstruction task.
Our key observations include:
Across all models, when data mixtures are skewed (e-g., more image than text inputs), the distribution of concepts by modality shifts accordingly. This confirms the sensitivity low-energy, rarely-used concepts to training data statistics.
However, the bottom rows show that energy-dominant (i.e., frequently activated) con- cepts remain consistently placed in similar areas of the space, regardless of training mixture. These concepts represent stable semantic core that is robust to dataset compo- sition.
This supports our conclusion from Section 4 that the SAE models recover stable, high- energy subspace, even under strong shifts in input distributions.
14

---

Published as a conference paper at COLM 2025
Siglip2
2:1
1:1
1:5
Image concept
1.0
0.8
0.4
0.2
0.0
Text concept
Siglip
5:1
2:1
1:1
1:2
1:5
Image concept
1.0
0.8
0.6
0.4
0.2
Text concept
Clip
1:5
Image concept 1.0
0.8
0.6
0.2
0.0
Text concept
Figure 9: UMAP projections of SAE concept dictionaries across various image-text training mixtures for SigLIP2, SigLIP, and CLIP. Top row: equal dot size. Bottom row: dot size scaled by energy. Related to Figure 3.
15

---

Published as a conference paper at COLM 2025
B Illustrative Examples from the Interactive Demo
B.1 Examples of concepts
We present some qualitative examples of what kinds of concepts the SAEs that we train in the paper fire for, showing concepts from all 4 models. We showcase concepts of different abstractions, starting from more surface-level concepts and going to more abstract concepts. These concepts all be accessed by putting their number into the Specific ID field in https://// http::///mmmmme/ down.
hetps:://ll--conceptt- concept www.szatiioin..... com/ after selecting the correct VLM from the drop-
SigLIP2 concept #2261
Pasta with broccoli
Figure 10: Coherent concept, with similar visual profile. Concept #2261 in SigLIP2 fires for pictures of pasta and broccoli, which is interpretable concept, where all of the images look pretty similar in terms of color, texture, and shape composition
AIMv2 concept #51 14 Haircut
Coherent concept, with different visual profiles. Concept #5114 in AIMv2 fires for pictures of people getting haircuts. Though these pictures have fewer surface-level elements that are consistent between them, they all depict the same higher-level interpretable event.
Figure 11:
16

---

Published as a conference paper at COLM 2025
CLIP concept #1033 Old film photos with rounded edges
Figure 12: Similar visual profiles, different semantics. Concept #1033 in CLIP fires for pictures that have vintage film coloring, rounded edges, and are outdoors. Though the surface-level visual profiles of the top activating examples for this concepts are related, there aren't discernible higher-level semantics that connect them.
SigLIP concept #5812 Many things
Highest BridgeScore concepts:
#4080:
#3231: line of brief cases sit next to group of white and black vases sitting next to each other. Several grinder sandwiches with lettuce and tomato and cheese.
each other
line of elephants that welking in line
Figure 13: Higher-level abstract semantic concept. Concept #5812 in SigLIP fires for pictures of many disorganized things. The concepts which are most related to it by BridgeScore are text concepts detailing the idea of many things, rather than any other more surface-level semantics of the images (like, "toys"), hinting that the model has learned cross-modal representation of the abstract notion of "several"
17

---

Published as a conference paper at COLM 2025
SigLIP concept #2826 Blue
woman wearing blue outfit and blue veil.
A blue park bench painted blue sits blue sidewalk.
A man and women wearing some odd blue things.
TWo blue chairs sitting next to blue wooden table.
blue chrome motorcycle with dark blue seat.
Figure 14: Text concept that is surface-level and visually-directed. Concept #2826 in SigLIP fires for text that describes blue objects. This is a surface-level text feature, describable by the identity of one token. It is also feature of the text that relates directly to the visual modality.
SigLIP concept #1926 Hypotheticals, opinions, modals
Maybe the picked that dog because they have same color hair.
She didn't 't expect that there would be this many birds feed.
She's 's probably not going to get that frisbee in time.
Only idiot would put mustard pair sugar frosted donuts.
Perhaps they shouldn t be playing outside such smoggy day.
Figure 15: Abstract linguistic text concept. Concept #1926 in Sig fires for text that contains uncertainty and/or modal verbs like "would" "should". This concept is picking on abstract linguistic features about hypothetical. There is no clear relationship in the visual modality between these captions (unlike the "blue" text), and instead there language feature that connects them.
18

---

Published as a conference paper at COLM 2025
B.2 Illustration of the BridgeScore
fruit sits on table in
busy scene in open narket.
box filled with lots of oranges nets.
Man sitting on surfboard in water mearing wet suit
nan holding surf board
Figure 16: Bridge score identifies semantically aligned concept pairs across modalities. Even though most concepts are unimodal in activation, many form meaningful connections across modalities via the bridge score. Edges connect concept pairs that are both geometri- cally aligned and co-activated relationships between clusters of related concepts, enabling gross-moda alignment despite sparse unimodal structure.
paired examples. These links often reflect many-to-many
C Training Details for SAEs
We report the full training configuration used for the sparse autoencoders (SAEs) in this work. All SAEs were trained frozen activations from the COCO dataset, consisting of 600,000 image and text embeddings per model. Each SAE was trained for 30 epochs, with batch size of 1024, totaling approximately 18 million training examples. The encoder consists of a single linear projection without hidden layers, mapping input activations to 4096-dimensional concept space. Unless specified, used sparsity k 5, enforcing exactly five non-zero, non-negative coefficients per input.
Optimization was performed using the AdamW optimizer with cosine learning rate schedule. The learning rate warmed up from 1 10-6 peak of 5 10-1, then decayed back 10 6, We used weight decay of 1 x 1 and applied gradient clipping with maximum global norm of 1.0. All model embeddings were (z-normalized prior to encoding. For image inputs, the original images were resized to 256 pixels on the shorter side and center-cropped to 224 x 224 before encoding by the VLMs. Text inputs were processed using the default tokenizer for each model.
We observed consistent convergence of the reconstruction loss around epoch 10, but trained for 30 epochs to ensure full stability of the learned dictionaries. We used the Overcomplete toolbox for training.
htttss//ttttuu.ccc/////eerrntttt
19


Reviews:
[{"review_id": "XzQ7KCEZE1", "reviewer": "Reviewer_6WMN", "summary": "This paper investigates the internal organization of joint image-text embedding spaces in vision-language models (VLMs) using sparse autoencoders (SAEs). By decomposing embeddings into sparse linear combinations of learned \"concept\" directions, the authors analyze how semantic and modality-specific structures are encoded across multiple VLMs (e.g., CLIP). They introduce four metrics, that is, energy, stability, modality score, and bridge score, to quantify the alignment, robustness, and modality balance of these concepts. The study finds that (1) high-energy concepts are consistently recovered across training runs; (2) and often correspond to interpretable, modality-biased components; (3) while low-energy concepts are unstable, many modality-specific concepts lie near-orthogonal, suggesting potential for shared semantic grounding. The authors also release VLM-Explore, an interactive tool for visualizing and probing these concept spaces.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The idea of using SAEs to interpret VLM embedding spaces is well-motivated. The introduced metrics are clearly defined and together provide a structured framework for probing semantic and modality-aligned directions within the joint embedding space. \n2. The findings are shown to be consistent across multiple VLMs (CLIP, SigLIP, SigLIP2, AIMv2) and robust across training runs, especially for high-energy concepts.\n3. Some findings are particularly insightful. For example, the observation that SAE-derived concepts must be interpreted in conjunction with their energy levels, since only high-energy concepts are stable and meaningful across different runs. \n4. The paper introduces VLM-Explore, an interactive visualization tool that enables intuitive exploration of extracted concepts and their associated metrics.", "reasons_to_reject": "1. While the paper introduces four evaluation metrics, all evaluations are internal to the proposed SAE framework. This makes the results feel somewhat self-contained. There is no external validation or downstream task demonstrating that the learned concepts or metrics correspond to useful behavior or interpretability outcomes.\n2. The paper lacks interpretation or case studies of the extracted concepts. The findings are presented almost entirely through internal metrics, without concrete examples or illustrations of what the learned concepts mean (e.g., color, shape, object category). This limits the interpretability impact of the work and leaves the notion of the learned \"concept\" abstract.\n3. The paper lacks a discussion of practical applications for the extracted concepts and metrics. Without connecting the results to real-world interpretability use cases or downstream tasks, the broader utility of the work remains unclear.\n4. The structure could be improved for clarity. Some dataset descriptions and evaluation procedures are embedded within the methods section, making it difficult to follow the overall experimental flow. The description of VLM-Explore could benefit from being presented in a dedicated section later in the paper.", "questions_to_authors": "1. How do you envision the learned concepts and metrics (e.g., modality score, bridge score) being used in practice? Could they support downstream tasks such as model debugging, interpretability, or control?\n2. Have you observed any qualitatively interpretable concepts extracted by the SAEs? If so, it would be helpful to include examples or visualizations to clarify the nature of these concepts.\n3. The VLM-Explore tool is a good idea. Could you elaborate on its intended use cases?\n\nLine 78 - Missing period\n\nLine 133 - Misspelling\n\nLine 365 - Empty ethics statement", "extra_scores": {}}, {"review_id": "Ir4OIjt80X", "reviewer": "Reviewer_WVqz", "summary": "This paper investigates the linear structure of the joint embedding space in four Vision-Language Models (CLIP, SigLIP, SigLIP2, AIMv2) using Sparse Autoencoders (SAEs). The authors train SAEs to decompose embeddings into sparse linear \"concepts,\" finding SAEs offer a good sparsity-reconstruction trade-off. Key findings include: high-energy concepts are stable across training runs, while low-energy ones are not ; most concepts are highly modality-specific (activating primarily for text or image); however, many concepts, including energetic ones, are geometrically near-orthogonal to the modality-distinguishing subspace, suggesting they encode cross-modal meaning despite unimodal activation patterns. The authors introduce metrics like Modality Score and Bridge Score, and release an interactive visualization tool (VLM-Explore).", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This study applies and compares multiple dictionary learning methods (SAEs vs. Semi-NMF) across four different VLMs.\n- Some interesting findings are presented, which reveal nuanced structure, particularly the separation of concepts' modal-specific activation patterns and their near-orthogonal geometric alignment relative to modality.\n- It provides trained SAEs and an interactive visualization tool (VLM-Explore) for community use.", "reasons_to_reject": "- A large family of representation models are missing from the study: A notable omission in this study is the family of VLM-based embedding models that rely on a shared Transformer backbone and pooling module for generating both textual and visual representations. These models are capable of producing high-quality embeddings and might exhibit a different representational structure compared to the examined CLIP-style and autoregressive VLMs. We strongly recommend including these architectures in future revisions, as doing so would broaden the study's coverage of mainstream VLM representations and potentially yield complementary findings.\n- The \"SAE Projection Effect\": The explanation for how concepts can be near-orthogonal yet unimodal relies heavily on the non-linear projection step of the SAE. This raises questions about whether the findings reflect the intrinsic linear structure of the VLM space or an artifact introduced by the SAE methodology itself.\n- Scalability/Generalization of Findings: Analysis is based on embeddings from one dataset (COCO). It's unclear if the observed structure (e.g., modality scores, orthogonality) holds across different data distributions or more complex VLM tasks.\n\n[1] Jiang, Ting, et al. \"E5-v: Universal embeddings with multimodal large language models.\" arXiv preprint arXiv:2407.12580 (2024).\n[2] Jiang, Ziyan, et al. \"Vlm2vec: Training vision-language models for massive multimodal embedding tasks.\" arXiv preprint arXiv:2410.05160 (2024).\n[3] Lin, Sheng-Chieh, et al. \"Mm-embed: Universal multimodal retrieval with multimodal llms.\" arXiv preprint arXiv:2411.02571 (2024).", "questions_to_authors": "- Hyperparameter Tuning: What specific hyperparameters were tuned for the Sparse Autoencoders (SAEs), and what procedure was employed?\n- Training Convergence: How was training convergence of the SAEs determined (e.g., specific thresholds or observation of plateaus)?\n- Equation 4 (Stability): How does the number of concepts (c) impact the Stability score in Equation 4?\n- How might the observed linear structures (concept modality, orthogonality) change when analyzing embeddings derived from more diverse datasets or task-specific fine-tuned VLMs?", "extra_scores": {}}, {"review_id": "ATwvLpaeae", "reviewer": "Reviewer_VzYu", "summary": "The authors explore the linear structure of the embedding spaces of vision-language models (VLMs) using sparse auto-encoders (SAEs) trained on CLIP, AIMv2. While the specific rare concepts found by these models can change, the frequently used concepts, which are more important, are stable across different training conditions. Even though the vision-language embedding space is of joint modalities, the concepts usually activate for a single modality only. Many concepts are also orthogonal and have cross-modal meaning due to the sparsity projection. They also introduces a score (Bridge Score) to identify concept pairs which are crucial for cross-modal alignment and an interactive visualization tool for it called VLM-Explore.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The SAE-based approach for interpreting embedding spaces provides meaningful and interesting insights into concept organization and modality bias.\n- It provides robust empirical findings across multiple VLMs w.r.t. stability of high-energy concepts and geometric properties of modality despite unimodal activation.\n- Bridge Score and VLM-Explore are useful resources for future research in this space.", "reasons_to_reject": "- The reliance on SAEs for interpreting VLM structure could be potentially limiting and providing an incomplete picture since it is missing non-linear dynamics. \n- The interpretation of the VLM space is inherently limed by the potential limitations of SAEs rather than solely reflecting the VLM's intrinsic structure.", "questions_to_authors": "How would the findings of the paper generalize to other VLM architectures (ie not contrastive) or to embeddings from different layers within the models?", "extra_scores": {}}, {"review_id": "GJQOPv8jRq", "reviewer": "Reviewer_Hk33", "summary": "The paper presents the training and release of sparse autoencoders (SAEs) applied to the encoding spaces of four vision-language models: CLIP, SigLIP, SigLIP2, and AIMv2. It demonstrates how the linear structure within the vision-language embedding space can be leveraged to better understand the joint vision-language representation. Additionally, the authors provide the trained SAEs, visualization tools, and introduce new evaluation metrics including stability, modality score, and bridge score to assess the models.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Dictionary learning is employed as a method to identify linear directions, or \"concepts,\" within the model's latent space, enabling each embedding to be represented as a linear combination of more interpretable components.\n\n2. Sparse autoencoders (SAEs) are trained and compared across four different vision-language models using hundreds of thousands of activations, followed by comprehensive evaluations that analyze the geometric, statistical, and modality-related structures of their embedding spaces.\n\n3. The study introduces new metrics-stability, energy, modality, and bridge scores-that extend beyond the traditional sparsity–reconstruction trade-off to better assess the quality and characteristics of the learned representations.", "reasons_to_reject": "1. Although the paper centers on analyzing embedding spaces of vision-language models, it does not clearly specify which datasets were used to generate the quantitative examples presented in Figure 12.\n\n2. The term \"sparse auto-encoders (SAE)\" may be misleading, as the method does not create autoencoders in the traditional sense but rather represents embeddings as sparse, weighted sums of linear “concepts.”\n\n3. The study focuses solely on encoder-based vision-language models such as CLIP and SigLIP2, overlooking other notable models like LLaVA and its variants.", "questions_to_authors": "1. The rationale behind selecting only encoder-based vision-language models for interpretation is not clearly explained.\n\n2. While the paper states that training TopK and BatchTopK sparse autoencoders (SAEs) offers greater stability and requires less hyperparameter tuning, detailed training procedures are not provided.\n\n3. Providing qualitative analysis to illustrate what constitutes image and text concepts would enhance understanding.\n\n\nMinor Comments\n\n1. Equation-3 needs to define the parameters used. \n2. https://vlm-concept-visualization.com/ only shows Energy as check-box, do not provide any other options? Is it supposed to have only this option?", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper investigates the joint space of image and text representations learned by vision-language models (VLMs) in terms of the concepts encoded and how the modalities are separated yet bridged. The authors first extract representations of the images and captions from the COCO dataset using CLIP, SigLIP, SigLIP2, AIMv2. The authors train sparse-autoencoders (SAEs) using these representations to obtain a linear combination of concepts per representation. 4 metrics are proposed to quantify the stability, energy, modality-specificity and bridging functions of concepts. The authors find that a subset of concepts are highly activated and stable over training runs. The findings also yield interesting structural insights regarding the connections between modalities, where most concepts activate more for a single modality; yet, the majority of concepts are geometrically near-orthogonal to modality. The authors also release a tool where the representation space can be inspected visually.

**Pros**

Clarity: The paper is well written and clear. 

Quality: The reviewers appreciated the robust findings and the meaningful insights offered in the paper.

Originality: Although there exist various works investigating the shared space of  vision-and-language embeddings, this paper offers unique insights with the help of learned dictionaries of linear concepts and novel metrics. 

Significance:  This is an interesting paper that uncovers significant features of shared multimodal representational spaces, which would be informative for improving and understanding VLMs.

**Cons**

The authors provided very insightful responses to the reviewers. In fact, 2 reviewers upgraded their scores. Below, I mention the issues that the authors convincingly addressed and should incorporate in the paper.

- Potential limitations of using SAEs (missing non-linear dynamics, how the interpretations are influenced by the SAE methodology itself and to what extent they reveal the intrinsic features of VLM representations)
- Downstream tasks and practical applications that would benefit from the findings 
- Generalization to other models and a stronger justification for the choice of the models
- Concrete examples of the interpretable concepts 
- Details of the hyperparameters and training procedure

I also suggest explaining Semi-NMF and BatchTopK a bit more (please check line 127 for clarity), and providing more details regarding the axes of Figure 7 and the SAE projection effect.
Decision: Accept